Intelligent Support for Computer Science Education

Barbara Di Eugenio is Professor in the Department of Computer Science at the University of Illinois at Chicago (UIC), Chicago, IL, USA. There she leads the NLP laboratory (https://nlp.lab.uic.edu). Dr. Di Eugenio holds a Ph.D. in Computer Science from the University of Pennsylvania (1993); she joined UIC in 1999. Her interests focus on the theory and practice of Natural Language Processing, with applications to educational technology, health care, human robot interaction, and social media. Dr. Di Eugenio is an NSF CAREER awardee (2002), and a UIC University Scholar (2018-21). Her research has been supported by the National Science Foundation, the National Insti- tute of Health, the Office of Naval Research, Motorola, Yahoo!, Politecnico di Torino, and the Qatar Research Foundation. She has graduated 12 PhD students and 30 Master's students, and published more than one hundred refereed publications. Davide Fossati is currently a Senior Lecturer in Computer Science at Emory University in Atlanta, GA, USA. Prior to joining the faculty at Emory in 2016, Dr. Fossati held positions at the Georgia Institute of Technology (2009-2010) and Carnegie Mellon University (2010-2015). He received his Ph.D. in Computer Science from the University of Illinois at Chicago in 2009. He also holds an M.Sc. degree in Computer Engineering from the Po- litecnico di Milano, Italy (2004), and an M.Sc. in Computer Science from the University of Illinois at Chicago (2003). Dr. Fossati's primary scholarly focus is Technology Enhanced Learning, with particular interest in the development of Artificial Intelligence systems to support Computer Science education. Nick Green is a technology professional with 20 years of research and development experience in academia and industry. Dr. Green received his Ph.D. in Computer Science from the University of Illinois at Chicago in 2017, where he focused on educational technology, natural language processing, and software engineering. Outside of academia, he has worked for companies such as Sony Interactive Entertainment and Facebook. He has a passion for the startup scene where he is also a serial entrepreneur having founded companies in fields such as security and precision agriculture.

About the Authors

Contributors

Preface

Acknowledgments

Section I Four Scientific Pillars

Chapter 1 Introduction

1.1 AN INTERDISCIPLINARY PERSPECTIVE

1.2 THE STRUCTURE OF THE BOOK

Chapter 2 Related Work With Stellan Ohlsson

2.1 COGNITION AND MULTIPLE MODES OF LEARNING

2.1.1 Background

2.1.2 Nine Modes of Learning

2.1.2.1 Discussion

2.2 PRAGMATICS AND DIALOGUE PROCESSING

2.3 INTRODUCTORY COMPUTER SCIENCE EDUCATION

2.3.1 Elementary and Secondary Education

2.3.2 From high school to college

2.3.3 Post-Secondary Education for CS Majors

2.4 INTELLIGENT TUTORING SYSTEMS (ITSS)

2.4.1 Natural Language Processing (NLP) for ITSs

2.4.2 Modes of learning and ITSs

2.4.2.1 Positive and negative feedback

2.4.2.2 Worked-Out Examples

2.4.2.3 Analogy

2.5 ITSS FOR COMPUTER SCIENCE AND NLP

2.5.1 ITSs for CS

2.5.1.1 NLP in ITSs for CS

Section II From Human Tutoring to ChiQat-Tutor

Chapter 3 Human tutoring dialogues and their analysis

With Stellan Ohlsson, Mehrdad Alizadeh, Lin Chen, and Rachel Harsley

3.1 DATA COLLECTION

3.1.1 Learning outcomes in human tutoring

3.1.2 Measuring learning gains

3.1.3 Learning effects

3.2 TRANSCRIPTION AND ANNOTATION

3.2.1 Annotation

3.2.1.1 Validating the corpus annotation

3.3 DISTRIBUTIONAL ANALYSIS

3.3.1 Elementary Dialogue Acts

3.3.2 Student Initiative

3.3.3 Episodic Strategies

3.4 INSIGHTS FROM THE CORPUS: PEDAGOGICAL MOVES AND LEARNING

3.4.1 Individual Dialog Acts (Type 1 Models)

3.4.2 Sequences of Dialogue Acts (Type 2 Models)

3.4.2.1 Bigram Models

3.4.2.2 Trigram Models

3.4.3 Episodic strategies (Type 3 Models)

3.4.3.1 Worked-out examples

3.4.3.2 Analogies

3.5 SUMMARY: INSIGHTS FROM HUMAN TUTORING ANAL YSIS

Chapter 4 ChiQat Tutor and its architecture

With Omar AlZoubi and Christopher Brown

4.1 THE DOMAIN MODEL

4.1.1 Problem definitions

4.1.2 Solution definitions

4.1.3 Worked-Out Examples

4.1.4 The Procedural Knowledge Model

4.2 USER INTERFACE

4.3 A BIRD’S EYE VIEW OF CHIQAT TUTOR IN ACTION

4.3.1 Solution evaluator

4.4 TUTOR MODULE

4.4.1 Code feedback: syntax and executability

4.4.2 Reactive & Proactive feedback

4.4.2.1 Reactive procedural feedback

4.4.2.2 Proactive procedural feedback

4.5 TRAINING THE PKM GRAPHS

Chapter 5 Evaluation in the classroom

With Rachel Harsley and Stellan Ohlsson

5.1 EVALUATION METRICS

5.2 LEARNING WITH PROACTIVE AND REACTIVE FEEDBACK

5.2.1 Insights on learning from student behavior and perceptions of ChiQat-Tutor-v1

5.2.1.1 Student behavior

5.2.1.2 Student satisfaction

5.2.2 Chiqat-Tutor, Version 1: Summary of findings

5.3 LEARNING WITH WORKED OUT EXAMPLES AND ANAL OGY

5.3.1 WOE and Analogy Conditions

5.3.1.1 Standard WOEs

5.3.1.2 Length and usage of WOEs

5.3.1.3 Analogical content in WOEs

5.3.2 Learning Linked Lists among non-majors

5.3.3 Learning Linked Lists among majors

5.3.4 Learning and Initial Student Knowledge

5.3.4.1 Mining the logs: Predicting Knowledge

5.3.5 Chiqat-Tutor, Version 2: Summary of findings

Section III Extending ChiQat-Tutor

Chapter 6 Beyond Linked Lists: Binary Search Trees and Recursion

With Mehrdad Alizadeh and Omar AlZoubi

6.1 BINARY SEARCH TREES

6.1.1 Pilot evaluation

6.2 RECURSION

6.2.1 Models for Teaching Recursion

6.2.1.1 Conceptual Models

6.2.1.2 Program Visualization

6.2.2 A Hybrid Model for Teaching Recursion ChiQat-Tutor

6.2.3 Evaluation of the recursion module

6.2.3.1 Experimental Protocol

6.2.3.2 Experiments at CMU Qatar

6.2.3.3 Experiments at UIC

6.2.4 Analysis of students’ interactions with the system 160

6.3 SUMMARY

Chapter 7 A practical guide to extending ChiQat Tutor

7.1 AN IMPLEMENTATION ARCHITECTURE

7.2 CASE STUDY: THE STACK TUTOR PLUGIN

7.2.1 Stack Plugin Design

7.2.2 Class structure

7.2.3 Setting up the stage

7.2.4 Graphical interface

7.2.5 Stack problem logic and feedback

Chapter 8 Conclusions

8.1 WHERE WE ARE, AND LESSONS LEARNED

8.2 FUTURE WORK

8.2.1 Extending the curriculum

8.2.2 Enhancing communication with the student

8.2.3 Mining the user logs, and deep learning

Appendix A A primer on data structures

Appendix B Pre /post tests

Appendix C Annotation Manuals

Appendix D Linked List Problem Set

Appendix E Stack Plugin Full Code

Bibliography