The Multimodal Learning Analytics Handbook

Michail (Michalis) Giannakos is a professor of interaction design and learning technologies at the Department of Computer Science of the Norwegian University of Science and Technology (NTNU). He is the head of the Learner-Computer Interaction lab and his research focuses on the design and study of emerging technologies in online and hybrid education settings, and on developing new ways for humans to interact with interactive learning systems. Giannakos has co-authored more than 200 manuscripts published in prestigious peer-reviewed journals and conferences (including Computers & Education, Computers in Human Behavior, IEEE Pervasive Computing, IEEE TLT, BIT, BJET, ACM TOCE, ACM IDC, ICLS/CSCL, Interact). Giannakos, together with Mike Horn, is the Editor-in-Chief of the International Journal of Child-Computer Interaction (Elsevier). He is also on the Editorial Board of IEEE Transactions in Learning Technology, IEEE Transactions on Education, Behaviour & Information Technology, and the International Journal of Information Management, and has served as a guest editor on highly recognized journals such as BJET, Computers in Human Behavior, ACM TOCE and IEEE Multimedia. He has served as an evaluator for the European Commission (EC) and the US-NSF, and he recently authored a book on Experimental Studies in Learning Technology and Child-Computer Interaction (Springer) and co-authored the Educational Data Literacy textbook (Springer). Giannakos has worked at several research projects funded by diverse sources like the EC, Microsoft Research, The Research Council of Norway (RCN), US-NSF, the German agency for international academic cooperation and Cheng Endowment. Giannakos is one of the experts in the Norwegian task force (formed by the ministry of education and research) for introducing learning analytics to Norwegian K-12 schools and universities. He is also a recipient of a Marie Curie/ERCIM fellowship, the Norwegian Young Research Talent award and he is one of the outstanding academic fellows of NTNU (2017-2022). Giannakos usually attends the ACM Interaction Design and Children (IDC) conference (he was a Chair for IDC 2018, and PC-chair for IDC 2022), the ACM conference on Innovation and Technology in Computer Science Education (ITiCSE) (he was a Chair for ITiCSE 2020), and the Learning Analytics and Knowledge (LAK) conference. Daniel Spikol is Associate Professor of Computational Thinking in the Computer Science and Science Education departments at the University of Copenhagen (UCPH). He is part of the university's Center for Digital Education, which investigates and helps support digital education transformation. Daniel holds a PhD in Computer Science for Linnaeus University, Sweden (2010), an MSc in Visual Studies from Massachusetts Institute of Technology (1992), and a BFA from Rhode Island School of Design (1987). The central theme of his research investigates how to integrate computational thinking and tinkering into education. His work combines physical computing and robotics with STEM and STEAM education to provide an authentic context to engage people in science and computing. Additionally, his research investigates how people collaborate with multimodal learning analytics (inspired by social signal processing ambient computing). Daniel combines science, design, digital art, and computer science to investigate how people learn and play. He has 70 peer-reviewed scientific publications. He has worked for different companies and research institutes and founded several companies. Daniel has been active in developing SOLAR's SIG for Multimodal Learning Analytics, participated in the program committees for different conferences such as LAK, EC-TEL, ICALT, AIED, IDC, and ICQE. He has also been a principal investigator for European and national research projects. Previous to UCPH, he was the scientific manager of the Internet of Things and People Research Lab at Malmoe University and part of the Computer Science and Media Technology department. Daniele Di Mitri is a research group leader at the Education Technology group at DIPF Leibniz Institute for Research and Information in Education and a lecturer at the Goethe University of Frankfurt. Daniele received his Ph.D. title at the Open University of The Netherlands (2020) in Learning Analytics. His Ph.D. project, the Multimodal Tutor, investigated the potentials of collecting and analysing multimodal data during physical interactions for automatic feedback and human behaviour analysis. Daniele's current research focuses on designing responsible Artificial Intelligence applications for education and human support. He is a "Johanna Quandt Young Academy" fellow. He was elected "AI Newcomer 2021" at the KI Camp by the German informatics society and received the "Martin Wolpers Award" in technology-enhanced learning 2018. He is a member of the editorial board of Frontiers in Artificial Intelligence journal, a member of the CrossMMLA, a special interest group of the Society of Learning Analytics Research, and he is the chair of the Learning Analytics Hackathon (LAKathon) series. Kshitij Sharma is is associate professor of learning technologies at the Department of Computer Science of the Norwegian University of Science and Technology (NTNU). He is member of the Learner-Computer Interaction lab and his research focuses on the using artificial intelligence and machine learning to study students' behaviour and design emerging technologies in online and hybrid education settings for them to learn in more effective and efficient manner. Sharma holds a Ph.D. in computer science from EPFL, Lausanne, Switzerland. Sharma has co-authored more than 100 manuscripts published in prestigious peer-reviewed journals and conferences (including BJET, JCAL, Computers in Human Behavior, IEEE Pervasive Computing, IEEE TLT, BIT, BJET, IMWUT, ACM CHI, ACM IDC, ICLS/CSCL). His research is focused on studying relations between students' profile (individual or collaborating) and interaction patterns with technology. Sharma's research interests center on making sense of students' experiences and actions to optimize and redesign the systems. The goal is to understand why and how diverse learner categories (based on their motivation, strategies, expertise) use technologies (online and standalone) in the ways that they do and then to adapt to their needs (knowledge management and enterprise-adapted e-learning). Xavier Ochoa is Assistant Professor of Learning Analytics in the Department of Administration, Leadership, and Technology of the Steinhardt School of Culture, Education, and Human Development. He is also core-faculty of the Learning Analytics Research Network (NYU - LEARN). Xavier holds a Ph.D. in Engineering (Computer Sciences) from the University of Leuven - Belgium (2008), an M.Sc. in Applied Computer Sciences from the Vrije Universiteit Brussels - Belgium (2002), and a B.S. in Computer Science from Escuela Superior Politecnica del Litoral (ESPOL) - Ecuador (2000). Since the start of his research career, Xavier has been working at the convergence point of Education and Technology. His research work focuses on how to use cutting-edge technology to augment human capabilities in education to innovate pedagogical practices that, in turn, lead to better learning processes. He started his career working on quantitative analysis and modeling of the production, finding, consumption, and reuse of digital learning materials. This work conducted his research into the nascent field of Learning Analytics where he was invited as a keynote of the first Learning Analytics and Knowledge conference (LAK). Currently, Xavier works on Multimodal Learning Analytics, a sub-field that involve the use of advanced quantitative techniques from the field of learning analytics, together with recent advances in machine learning, computer vision, speech and natural language processing, and smart sensors to build and study the educational effect of tools that augment the awareness, self-reflection, sense- and decision-making of students and instructors in both physical and online learning environments. He is the lead investigator of the Augment-Ed research group. Xavier Ochoa has served as Vice-president of the Society for Learning Analytics Research (SoLAR) and is currently the co-Editor-in-Chief of the Journal of Learning Analytics. He is also a founding member and long-time coordinator of the Latin American Community on Learning Technologies (LACLO). He has also received the Best Researcher Award (2018 and 2012) and Best Professor Award (2013) at ESPOL and the 5-year best Researcher in Computer Science by the Institute of Electrical and Electronics Engineers (IEEE) in Ecuador (2014). He has also received several Best Paper Awards at international conferences, including LAK. Rawad Hammad is a Programme Leader for MSc Computing, Programme Leader for MSc Digital Education, Technology Enhanced Learning Research Group Leader, and a Senior Lecturer in Computer Science and Digital Technologies at the University of East London. Rawad has an extensive experience on Software Engineering, Technology Enhanced Learning (TEL), Artificial Intelligence in Education, and Smart Health research and practice. Rawad contributed to and led various international projects, published research articles and has been involved in various conference programme committees including EC-TEL, LAK, AIED and BUSTECH. Moreover, Rawad is supervising postgraduate students, currently 5 PhD students in addition to varied number of MSc students. Rawad is an executive committee member of the International Society of the Artificial Intelligence in Education (AIED) and committee member of different research bodies, conferences including: EC-TEL, LAK, AI for Post-COVID Education, Networks in Education. In 2018, he received his PhD in Software Engineering from the University of the West of England (UWE). In 2010, he received his MSc in Cognitive Computing from Goldsmiths University of London. He led numerous international projects such as: TRANSFER and SmartTech which include partnerships with different universities, research centres, governmental institutes from different countries including Japan, Germany, Finland and Middle East. Prior to coming to University of East London, Rawad was a Senior Education Solutions Researcher/Analyst at King's College London; Researcher at Centre for Complex Cooperative Systems at the University of the West of England.

Introduction to Multimodal Learning Analytics.- Multimodal Learning Analytics and the Design of Learning Spaces.- Multimodal Systems for Automated Oral Presentation Feedback: A Comparative Analysis.- Modeling the Complex Interplay Between Monitoring Events for Regulated Learning with Psychological Networks.- The role of metacognition and self-regulation on clinical reasoning: Leveraging multimodal learning analytics to transform medical education.- Intermodality in Multimodal Learning Analytics for Cognitive Theory Development: A Case from Embodied Design for Mathematics Learning.- Bridging the Gap between Informal Learning Pedagogy and Multimodal Learning Analytics.- Multimodal Learning Experience for Deliberate Practice.- CDM4MMLA: Contextualized Data Model for MultiModal Learning Analytics.- A Physiology-Aware Learning Analytics Framework.- Once more with Feeling - Emotions in Multimodal Learning Analytics.- The evidence of impact and ethical considerations of Multimodal Learning Analytics: A Systematic Literature Review.- Sensor-based analytics in education: Lessons learned from research in Multimodal Learning Analytics.- Framing the future of Multimodal Learning Analytics.