A team of computing science students from the University of Glasgow are the winners of a major international competition to develop next-generation conversational assistants.
The GRILL team (grilllab.ai) from the School of Computing Science took first place in Amazon’s Alexa Prize TaskBot Challenge, beating out nine leading research teams worldwide. The team wins a cash prize of $500,000 (£414,000).
PhD students Sophie Fischer, Carlos Gemmell, Iain Mackie, Paul Owoicho and Federico Rossetto made up the Alexa GRILL team.
The competition, launched in June 2021, challenged university students to develop conversational agents that assist people in completing cooking and do-it-yourself home improvement tasks that required multiple steps and complex decisions.
In May last year, the team was selected as one of ten entrants to participate in the challenge, receiving a research grant of $250,000 (£176,500) to support 12 months of work. They also received four Amazon Alexa devices, free access to Amazon Web Services, and support from members of the Amazon Alexa team.
In April, they were picked as one of five teams to compete in the finals.
The TaskBot Challenge is the first conversational AI challenge to incorporate multimodal customer experiences; in addition to receiving verbal instructions, customers are presented with images or videos to guide them through the task.
The team’s goal: build taskbots that assist customers in multi-step tasks, such as baking a birthday cake or fixing a scratch on a car — and adapt those instructions based on the resources and tools available to the customer. If, for example, a customer ran out of an ingredient halfway through a recipe or didn’t have a specific tool for a DIY project, the taskbot had to adjust the plan and suggest possible solutions.
Customers interacted with the taskbots by saying, ‘Alexa, let’s work together’. That prompt initiated an interaction with one of the taskbots. After the interaction ended, customers rated — on a scale from 1 to 5 — how helpful that taskbot was with the task.
Success required the teams to address many complex AI challenges, from knowledge representation and inference, and commonsense and causal reasoning, to language understanding and generation, requiring the fusion of multiple AI techniques.
Team member Federico Rossetto said: “The experience of taking the ambitious Alexa Prize TaskBot goals from ideas and research to prototype and launching on a tight timeline was thrilling and humbling.”
Ian Mackie said the team realised it was essential to provide a wide variety of images and videos that help illustrate the techniques involved in each task. “We learned that users learn and enjoy a task more when there are rich image and video elements,” he said. To do that, the students relied on image and video augmentations, applying multimedia elements that could be reusable across different tasks.
Carlos Gemmell, GRILLBot’s project leader, added: “Ultimately, the goal of taskbots is one shared by the research vision from our research lab: to enable people to understand the world and make people’s lives better.”
Dr. Jeff Dalton, a lecturer in the School of Computing Science, is the head of the Glasgow Representation and Information Learning (GRILL) Lab, which lends its name to the team’s project, and helped guide the team’s development of GRILLBot.
Dr. Dalton said: “I’m pleased and proud that the GRILL team placed first in the Alexa Prize TaskBot Challenge. The last 12 months were a rollercoaster and challenged the team to overcome tough challenges and develop a conversational assistant capable of truly adapting to users’ needs.”
The team said: “We find it exciting because realising the vision requires fundamental advances in core research areas of machine intelligence. Ultimately, the goal of taskbots is one shared by the research vision from our research lab, to enable people to understand the world and make people’s lives better.”