Grounding Human-Centred Ai on Embodied Multimodal Interaction

Our community, Grounding Human-Centred AI on Embodied Multimodal Interaction, has the objective to investigate the impact and effects of the rapid expansion of AI and language technology in everyday life, education, and scientific research. We want to study the extent of such impacts by building on our experience with linguistic and psycholinguistic research methodologies in the domain of human interaction and multimodality.

 

Connected Community Activities

Planned Education Activities

• Student exchanges
• Academic staff exchanges
• Admin staff exchanges
• Workshops to discuss and consult on harmonisation to enable us to plan joint programmes and teaching
• Promote each other’s online courses Plan collaborative applications under Erasmus
• Plan collaborative applications under Erasmus

Planned Research Activities

• Student exchanges
• Academic staff exchanges
• Workshops, meetings, and online discussions and consultations on each group’s methodologies and research activities
• Meetings on how to overcome obstacles in utilising each other resources and facilities
• Invite partners and their collaborators to research activities online like reading groups or special lectures
• Plan collaborative applications under Erasmus and other funding agencies

Activities 

Beyond Words
Workshop

Workshop 
3-4 October 2024

The workshop was adressed to students and researchers in the fields of linguistics, psychology, cognitive science, computer science, natural language processing, language technology, computer vision, cognitive computing, AI, and related areas interested in questions concerning how language is used in different contexts, with support of different modalities, and how such behaviour can be interpreted and generated by computational models.

For more information visit the dedicated webpage: Eutopia-beyond-words

Upcoming activities

Stay tuned for more information

Community Members

Lead: Eleni Gregoromichelaki

Email address: eleni.gregoromichelaki@gu.se

Eleni Gregoromichelaki is a Professor of Linguistics within the Linguistics, Logic, and Theory of Science unit of FLoV at the University of Gothenburg. She has a background in both in Linguistics and Computational Linguistics. She has an MSc in Computational Linguistics and PhD in Linguistics from King’s College London where she investigated formal and computational models of psycholinguistically realistic grammars.  Currently, she is interested in the modelling of dialogue by researching how human conversation fits into more general processes of interaction in the natural world. Interaction from this perspective involves not only the verbal exchange of signals but also multimodal perception/action and, even more generally, theories of information in technology, biology, and physics. Recently, she has also been investigating the significance of the development of Large Language Models (LLMs) for theories of language and cognition.

Lead: Simon Dobnik

Email address: simon.dobnik@gu.se

Simon Dobnik is a Professor of Computational Linguistics at University of Gothenburg where he leads the Cognitive Systems research group at the Centre for Linguistic Theory and Studies in Probability (CLASP) and teaches within the Masters in Language Technology programme (MLT). He has experience with both textual (linguistic) and non-textual (images, robots) domains of natural language processing. His main research interests are computational representation of meaning (semantics), computational models of language and perception, human-robot interaction, and scenarios with low-resource data. In his research, he focuses on building, evaluating, and improving language models in a way that is informed by the properties of machine learning algorithms, representation of knowledge in the training data, and research in linguistics and psychology on human interaction. Examples of his work include learning language with robots, learning grounded language models, generation of image captions, (visual) question answering, natural language inference, learning language models from small and variable data, and evaluation of bias.

Partner: Martina Wiltschko

Email address: martina.wiltschko@upf.edu

Martina Wiltschko is an ICREA research Professor at the Universitat Pompeu Fabra in Barcelona. She is a theoretical linguist, focussing on syntax and its interfaces. She obtained her PhD in 1995 (focussing on German syntax), spent much of her career at the University of British Columbia until 2019 when she assumed her current position. During her tenure at UBC, Wiltschko focussed mainly on language variation and fieldwork (culminating in her 2014 Cambridge University monograph on the universal structure of categories). She then developed an interest in the nature of language in interaction which led to her 2021 Cambridge University monograph on the grammar of interactional language. She has recently started a project (currently funded by the Spanish government) on the nature of human machine interaction, focussing on the role of interactional language.

Partner: Peter Sutton

Email address: peterroger.sutton@upf.edu

Peter Sutton works on topics including countability (the count-mass distinction), polysemy and vagueness. More broadly, his interests lie in the interactions between lexical and compositional semantics, including the roles of context and world knowledge. His educational background is in philosophy, primarily the philosophy of language, but he has been working in linguistics departments since 2014. His research combines a number of different approaches to the analysis of language including using probabilistic and richly typed semantic theories, as well as incorporating corpora analysis and computational simulations. Peter currently has his own postdoc project on Polysemy and Countability at UPF Barcelona.

Partner: Robin Cooper

Email address: robin.cooper@ling.gu.se

Robin Cooper is a Senior Researcher and Professor emeritus at the Department of Philosophy, Linguistics and Theory of Science at the University of Gothenburg.  His main research interests are semantics and dialogue.

Partner: Špela Arhar Holdt

Email address: Spela.ArharHoldt@ff.uni-lj.si
 

Špela Arhar Holdt is a Research Associate at the Centre for Language Resources and Technologies at the University of Ljubljana. Her background is in corpus linguistics and Slovene studies. The main purpose of her work is to develop openly available language data for Slovene, such as language corpora and training sets, lexical databases, and methodology for data annotation and extraction. Her research interests are also user involvement and language didactics.

Partner: Kaja Dobrovoljc

Email address: kaja.dobrovoljc@ff.uni-lj.si
 

Kaja Dobrovoljc is a research associate at the Faculty of Arts, University of Ljubljana, and the Department for Artificial Intelligence at the Jozef Stefan Institute. Her main research interests lie in the design, creation and evaluation of digital language resources and their application in the field of corpus linguistics. After completing her PhD on discourse-structuring devices in spoken and online communication, she has participated in various several national and international projects aimed at developing foundational language resources and technologies for the Slovene language. At presents, she leads a national project 'A Treebank-Driven Approach to the Study of Spoken Slovene' and co-leads the Corpus Annotation working group as part of the COST Action 'UniDive: Universality, Diversity and Idiosyncrasy in Language Technology'

Partner: Slavko Žitnik

Email address: slavko.zitnik@fri.uni-lj.si

Slavko Žitnik works as an assistant professor at the Faculty of computer and information science, University of Ljubljana. He is also a vice-dean for education, chairman of the Slovenian Language Technologies Society, chairman of the Slovenian Society INFORMATIKA and advisory council member of the National and university library. He is teaching courses related to data science, databases, semantics and natural language processing. His main research work combines various text processing fields with application areas such as information retrieval, semantic Web, information extraction and data spaces.

Partner: Darja Fišer

Email address: darja.fiser@ff.uni-lj.si, darja.fiser@inz.si
 

Darja Fišer is Executive Director of CLARIN. In her role, she ensures CLARIN’s leading role in the landscape of research infrastructures for social sciences and humanities, as well as its organisational and financial sustainability within and beyond Europe, both in academia and industry.

Darja Fišer has previously held different roles at CLARIN ERIC maximising the integration of CLARIN’s resources and tools in both the established as well as emerging multidisciplinary research agendas, methodological frameworks, and communities of practice.

Darja Fišer has a background in corpus linguistics and language resource creation. She has been Associate Professor at the Faculty of Arts, University of Ljubljana, since 2019, Senior Research Fellow at the Institute of Contemporary History since 2021, and is leading the new national research programme for Digital Humanities in Slovenia. She is also serving as a member of the Scientific Advisory Board of the Austrian Centre for Digital Humanities at the Austrian Academy of Sciences, the National Interdisciplinary Research E-Infrastructure for Bulgarian Language and Cultural Heritage Resources and Technologies, and the Czech National Corpus research infrastructure of the Institute of the Czech National Corpus at Charles University.

Partner: Staffan Larsson

Email address: staffan.larsson@ling.gu.se

Staffan Larsson is Professor of Computational Linguistics at the Department of Philosophy, Linguistics and Theory of Science at the University of Gothenburg. He is a member of the Centre for Studies in Language and Probability (CLASP), as well as Chief Science Officier and co-founder of Talkamatic AB. He is a member of the editorial noards of Dialogue and Discourse and the Journal of Logic and Computation. His areas of interest include dialogue, dialogue systems, language and cognition, pragmatics, formal semantics, semantic coordination, in-vehicle dialogue systems, philosophy of language.

Partner: Nikolai Ilinykh

Email address: nikolai.ilinykh@gu.se

Nikolai Ilinykh is a PhD student at the University of Gothenburg and a computational linguist and NLP researcher since 2015. Before that, he studied Cognitive Science, AI, and Linguistics. He analyses, designs, and constructs computational models that explore the relationship between language and the world. His current research focuses on the nature of language-and-vision models for tasks such as image captioning, image paragraph generation, visual question answering, embodied question answering, and visual dialogue. He also has a general interest in multi-modality, generation, AI ethics, robotics, and embodied systems.

How to get involved? 

Please contact:
Prof Eleni Gregoromichelaki,
Linguistics, Logic, and Theory of Science, FLoV
University of Gothenburg
eleni.gregoromichelaki@gu.se