A new project from a University of Glasgow computing scientist is setting out to close the gap between virtual and physical spaces to create immersive new forms of social interaction online.
Dr Julie Williamson, of the School of Computing Science, has received a €2m (£1.75m) grant from the European Research Council to support the project, called ‘Future Social Interaction in XR’, or FUSION.
Over the next five years, Dr Williamson will work to find new ways of capturing and modelling complex social signals in virtual reality (VR) and extended reality (XR) spaces.
The project’s ultimate aim is to create online experiences which use VR and XR technologies to seamlessly mix virtual avatars alongside their flesh-and-blood counterparts, creating experiences which are difficult to distinguish from real-life interactions.
Dr Williamson, a senior lecturer in human-computer interaction at the University of Glasgow, said: “Many of us became very familiar with virtual meeting software like Zoom and Skype to help us maintain contact with friends, family and co-workers during covid lockdowns.
“While those tools can be very useful, they can also be frustrating experiences. People talk over each other or don’t make consistent eye contact with their cameras, for example, and it’s impossible to see non-verbal cues like body language if you’re restricted to only seeing people’s faces.
“More advanced technologies like virtual reality headsets can allow users to feel more present together, but they’re still very crude approximations compared to face-to-face interactions.
“Social signals like gestures, eye contact and personal space are currently very difficult to recreate in virtual spaces, which often prevents interactions with other people from feeling realistic.
“What we’re aiming to do with FUSION is dissolve the barriers between virtual and physical realities to create social experiences that accurately capture the nuances of human behaviour.”
The FUSION project, which builds on Dr Williamson’s expertise in human-computer interaction, will be based at the University of Glasgow’s Mazumdar-Shaw Advanced Research Centre, which is home to a state-of-the art space dedicated to XR research.
In the ARC-XR lab, Dr Williamson will use cameras and sensors to observe volunteers’ voices, gestures and positions as they interact with each other in person and online using headsets, helping to build up models of social signals between people and across realities.
Analysis of those social signals will help develop a new database of behaviour patterns that persist across virtual and in-person spaces. The outputs of the analysis will help create new software that can smooth out the rougher edges of mixed-use communications and stabilise interactions to create more compelling immersive experiences.
Those stabilisation techniques could include subtle repositioning of users to create more useful group formations, or small adjustments of eyelines to better replicate in-person eye contact. When conversations overlap, the software could manipulate audio to direct the group’s attention to a single speaker.
Dr Williamson added: “Hybrid spaces which overlap virtual spaces and the real world have huge potential for allowing us to connect and collaborate online for work and play, but the technology just isn’t there yet for it to feel seamless and stable.
“By the end of five years of research on FUSION, we’ll have developed a powerful suite of advanced tools for creating much more realistic hybrid interactions. That could help form the foundations of a future where immersive social interactions in digital spaces become meaningful social experiences.
“I’m grateful to the European Research Council for their support, and I’m looking forward to getting started.”
Dr Williamson’s new funding comes from the European Research Council’s Consolidator Grant programme. She is one of 308 researchers from across the continent who have received support in the latest round of Consolidator Grant funding, which totals €627m (£548m).
Consolidator Grants support excellent scientists and scholars at the career stage where they may still be consolidating their own independent research teams to pursue their most promising scientific ideas.
Iliana Ivanova, European Commissioner for Innovation, Research, Culture, Education and Youth, said: “I extend my heartfelt congratulations to all the brilliant researchers who have been selected for ERC Consolidator Grants. I’m especially thrilled to note the significant increase in the representation of women among the winners for the third consecutive year in this prestigious grant competition. This positive trend not only reflects the outstanding contributions of women researchers but also highlights the strides we are making towards a more inclusive and diverse scientific community.”
President of the European Research Council Prof. Maria Leptin said: “The new Consolidator Grant winners represent some of the best of European research. It is disappointing that we cannot support every deserving project simply due to budget constraints; around 100 proposals identified as excellent in our rigorous evaluation will be left unfunded. Can Europe afford to let such talent go unrealised? We need to collectively advocate for increased investment in research and innovation. Our shared goal must be to ensure that no brilliant idea goes unfunded in Europe, and no promising career is left unfulfilled.”