Adoption of Data Analytics in Higher Education Learning and Teaching (eBook)

Preface 6
Contents 10
About the Editors 20
no 20
About the Contributors 22
Part I: Focussing the Organisation in the Adoption Process 40
Chapter 1: Adoption of Learning Analytics 41
1.1 Introduction 41
1.2 Innovation Diffusion 42
1.2.1 Six Characteristics of an Innovation 42
1.2.2 Communication Channels 44
1.2.3 Encompassing Social Systems 44
1.2.4 Summary of Innovation Diffusion 45
1.3 Improving Higher Education with the Adoption of Learning Analytics 45
1.3.1 Acquiring Students 46
1.3.2 Promoting Learning 46
1.3.3 Offering Timely Relevant Content 47
1.3.4 Delivery Methods 48
1.3.5 Supporting Alumni Networks 48
1.3.6 Cases 49
1.3.6.1 Analytics Teams in Business Units of a University 49
1.3.6.2 Adoption of Learning Analytics in Challenge-Based Learning Design 50
1.4 Discussion and Outlook 52
References 54
Chapter 2: The Politics of Learning Analytics 59
2.1 Unfolding Scenarios 59
2.2 The Promises and Challenges of Big Data and Learning Analytics for Higher Education 60
2.3 Ethical and Legal Frameworks of Big Data and Learning Analytics 62
2.4 Knowledge Production: Algorithmic and Datafied Education and Its Consequences 63
2.4.1 On Algorithms and Data 64
2.4.2 On Interpretation and Contextualisation 66
2.4.3 On Communicating and Adopting a Learning Analytics Culture 67
2.4.4 On Data Reduction and Knowledge Production 68
2.5 Ecologies of Learning: Towards a Measured Learning Analytics 69
2.6 Implications for Universities Adopting Learning Analytics 69
References 73
Chapter 3: A Framework to Support Interdisciplinary Engagement with Learning Analytics 77
3.1 Introduction 77
3.1.1 What We Mean by Interdisciplinary 78
3.1.2 Big Data and Learning Analytics 78
3.2 Learning Analytics in Higher Education 80
3.2.1 Organizational Drivers 80
3.2.2 Classroom-Level Use of Learning Analytics 81
3.3 An Interdisciplinary Approach 82
3.3.1 Awareness – What Is Being Collected and Why 83
3.3.2 Access – Who Can Get to the Data 84
3.3.3 Resources – Where Is the Data Stored 85
3.4 Future Directions 87
3.5 Conclusion 88
References 88
Chapter 4: The Framework of Learning Analytics for Prevention, Intervention, and Postvention in E-Learning Environments 91
4.1 Introduction 91
4.1.1 Prevention 93
4.1.2 Intervention 94
4.1.3 Postvention 95
4.1.4 Differences Between Prevention, Intervention, and Postvention 96
4.2 Proposed Framework 97
4.2.1 Dropout 98
4.2.2 Avoidance of Learning Activities 100
4.2.3 Failing Learning Performance 100
4.2.4 Locus of Control 101
4.2.5 Academic Procrastination 101
4.3 Conclusion and Discussion 102
References 104
Chapter 5: The LAVA Model: Learning Analytics Meets Visual Analytics 108
5.1 Introduction 108
5.2 Human-Centered Learning Analytics 109
5.3 Visual Analytics 110
5.4 The LAVA Model 111
5.5 The LAVA Model in Action 113
5.5.1 Learning Activities 115
5.5.2 Data Collection 115
5.5.3 Data Storage and Pre-processing 115
5.5.4 Analysis 116
5.5.5 Visualization 116
5.5.6 Perception and Knowledge 116
5.5.7 Exploration 117
5.5.7.1 Goal 117
5.5.7.2 Question 118
5.5.7.3 Indicator 118
5.5.8 Action 124
5.6 Evaluation 124
5.6.1 Method 125
5.6.1.1 Setting 125
5.6.1.2 Participants 125
5.6.2 Usefulness 126
5.6.3 Usability 127
5.7 Conclusion 128
References 128
Chapter 6: See You at the Intersection: Bringing Together Different Approaches to Uncover Deeper Analytics Insights 131
6.1 Introduction 131
6.2 The Story So Far 132
6.2.1 Centralized Support 134
6.2.2 System Generated Reports 135
6.3 Research Sprints 136
6.3.1 The First Year Chemistry Curriculum 138
6.3.2 The French Language Curriculum 139
6.3.3 The Analysis of Student Course Progress 141
6.4 Conclusion 142
6.4.1 Future Directions 143
References 145
Chapter 7: “Trust the Process!”: Implementing Learning Analytics in Higher Education Institutions 148
7.1 Introduction 148
7.2 Adoption of Learning Analytics 149
7.2.1 Issues and Challenges of LA Adoption 150
7.2.2 Leadership of LA Adoption 152
7.2.3 Models of LA Adoption 153
7.3 Adapted Roma Model for Bottom-Up Adoption 155
7.4 Adoption of Learning Analytics at Aalen UAS 156
7.4.1 A Small Project as Starting Point 156
7.4.2 Closing the Gap Between Teachers and Learners 158
7.4.3 Extension to Higher Levels 160
7.4.4 Summary of the Adoption Process 161
7.5 Outlook and Conclusion 162
References 166
Part II: Focussing the Learner and Teacher in the Adoption Process 170
Chapter 8: Students’ Adoption of Learner Analytics 171
8.1 Introduction 171
8.2 Methodology 174
8.3 Results 176
8.3.1 Implementation of a Learner Analytics Platform 177
8.3.2 Adoption of Connect Analytics in the Live Pilot 178
8.3.2.1 Somewhat Active but Irregular Users (N = 12) 180
8.3.2.2 Active and Regular Users (N = 6) 182
8.3.2.3 Somewhat Active and Regular Users (N = 6) 183
8.3.2.4 Active but Irregular Users (N = 5) 183
8.3.2.5 Sporadics and Users with No Logging Activity (N = 49) 183
8.3.3 Students’ Feedback on Connect Analytics After the Live Pilot 184
8.4 Discussion: Understanding Students’ Adoption of Learner Analytics 186
8.5 Conclusions 189
References 190
Chapter 9: Learning Analytics and the Measurement of Learning Engagement 193
9.1 Introduction 193
9.2 This Study 194
9.2.1 Context 195
9.2.2 Instrument and Procedure 196
9.2.3 Data Analysis 198
9.3 Results 198
9.3.1 Descriptive Statistics of Survey-Based Measures 198
9.3.2 Cluster-Based Learning Profiles 199
9.3.3 Learning Profiles and Course Performance 201
9.3.4 Bivariate Relationships Between Engagement Indicators and Course Performance 202
9.3.5 Multivariate Relationships Between Engagement Indicators and Course Performance 203
9.3.6 Bivariate Relationships Between Survey-Based Engagement Scores and Log-Based Engagement Indicator 204
9.4 Findings and Discussion 205
9.5 Conclusion 209
References 209
Chapter 10: Stakeholder Perspectives (Staff and Students) on Institution-Wide Use of Learning Analytics to Improve Learning and Teaching Outcomes 211
10.1 Introduction and Context 211
10.2 Approach 213
10.3 Staff Perspectives on LA 214
10.4 Students’ Perspectives on LA 219
10.5 Comparing Responses from Staff and Students – the ‘Standout’ Messages 225
10.5.1 Awareness of Learning Analytics and Data Collection 225
10.5.2 How LA Might Be Used to Support Learning 227
10.5.3 Concerns 228
10.5.4 Practical Actions for More Effective Use of LA 230
10.6 Conclusion 231
References 232
Chapter 11: How and Why Faculty Adopt Learning Analytics 235
11.1 Introduction 235
11.2 Background 236
11.2.1 Learning Analytics Implementation and Adoption: Institutions 236
11.2.2 Learning Analytics Implementation and Adoption: Teachers 237
11.2.3 Theoretical Framework – Diffusion of Innovations 239
11.3 Methods 240
11.3.1 Research Questions 240
11.3.2 The SRES as a LA Platform 241
11.3.3 Data Collection 244
11.4 Findings 245
11.4.1 Perceived Attributes of the Innovation 245
11.4.2 Relative Advantage 246
11.4.2.1 Compatibility 248
11.4.2.2 Complexity 248
11.4.2.3 Trialability 249
11.4.2.4 Observability 249
11.4.3 Communication Channels 250
11.5 Discussion & Conclusions
References 253
Chapter 12: Supporting Faculty Adoption of Learning Analytics within the Complex World of Higher Education 255
12.1 Introduction 255
12.1.1 Background 256
12.1.2 The Bay View Alliance 258
12.1.3 The Learning Analytics Research Collaborative 258
12.2 The Cycle of Progress for Sustainable Change 259
12.2.1 Awareness 260
12.2.2 Understanding 261
12.2.3 Action 261
12.2.4 Reflection 261
12.3 Methodolgy 262
12.4 Results 263
12.4.1 Vignettes 263
12.4.2 Cultural Change Indicators 265
12.4.3 Program Support 266
12.5 Discussion 267
12.5.1 Commonalities and Contrasts 267
12.5.2 Theoretical Framework 268
12.5.3 Implications and Limitations 269
12.5.4 Future Directions 270
12.6 Conclusion 271
References 272
Chapter 13: It’s All About the Intervention: Reflections on Building Staff Capacity for Using Learning Analytics to Support Student Success 274
13.1 Introduction 274
13.2 Learning Analytics 275
13.3 How Tutors Support Students 276
13.4 Enhancing the Tutoring/Advising Process Using Learning Analytics 278
13.4.1 Methodology 279
13.4.2 Case Study: Using Learning Analytics to Support Students at Nottingham Trent University 280
13.4.2.1 Implementation of Learning Analytics 280
13.4.2.2 Building Staff Capacity to Support Students Using Learning Analytics 281
13.5 Trigger/Prompt 281
13.6 Communication 282
13.7 Intervention 283
13.7.1 Summary for Building Staff Capacity to Support Students Using Learning Analytics 285
13.8 Institutional Recommendations 285
13.9 Conclusions 286
References 287
Chapter 14: Experiences in Scaling Up Learning Analytics in Blended Learning Scenarios 290
14.1 Introduction 290
14.2 Methodology 292
14.2.1 Collecting Learning Analytics Requirements 293
14.2.2 Evaluation Strategies 294
14.3 Scaling Up Learning Analytics 295
14.3.1 Building the Requirements 295
14.3.1.1 Outcome-Driven Innovation and Exploratory Data Analysis: Results 295
14.3.1.2 Building the Requirements: Literature Reviews 296
14.3.1.3 Building the Requirements: Outcomes 298
14.3.2 Institutional Regulation Preparation 299
14.3.3 Learning Analytics Services Implementation 300
14.3.4 Data Management 301
14.3.5 Analytics Engine 303
14.3.6 Results Visualization 304
14.4 Evaluation Strategies for LA 305
14.4.1 Study Setting 305
14.4.2 Evaluation Findings 308
14.5 Lessons Learned and Conclusions 310
References 312
Part III: Cases of Learning Analytics Adoption 316
Chapter 15: Building Confidence in Learning Analytics Solutions: Two Complementary Pilot Studies 317
15.1 Introduction 317
15.2 Related Works 318
15.3 1st Pilot Study: Mining Academic Data 320
15.3.1 Context and Goals 321
15.3.2 Graduating Versus Dropping out 322
15.3.3 Typical Completing Behaviors 324
15.3.4 Discussion 326
15.4 2nd Pilot Study 327
15.4.1 Context and Goals 327
15.4.2 Design of a Student-Centered Dashboard 328
15.4.3 Usability of the Dashboard 331
15.4.4 Discussion 332
15.5 Conclusion 333
References 334
Chapter 16: Leadership and Maturity: How Do They Affect Learning Analytics Adoption in Latin America? 336
16.1 Introduction 336
16.2 Methods 338
16.2.1 Research Design 338
16.2.2 Research Context 339
16.2.3 Data Collection 340
16.2.4 Data Analysis 341
16.3 Case Descriptions 342
16.3.1 Adoption of NoteMyProgress in PUC-Chile 342
16.3.2 Adoption of TrAC in UACh 343
16.3.3 Adoption of the Redesigned Academic Counseling System in ESPOL 346
16.3.4 Adoption of Dashboards in UCuenca 349
16.4 Findings of Cross-Case Analysis 352
16.4.1 Leadership 352
16.4.2 Organizational Maturity 353
16.5 Lessons Learned and Conclusion 354
References 355
Chapter 17: Adoption of Bring-Your-Own-Device Examinations and Data Analytics 358
17.1 Introduction 358
17.2 The Evolution of Digital Examinations 360
17.3 BYOD Examination Implementation Case Study 361
17.3.1 Infrastructure 362
17.3.1.1 Wi-Fi 362
17.3.1.2 Power 362
17.3.2 Human Factors 363
17.3.2.1 Students 364
17.3.2.1.1 Outcomes 364
17.3.2.1.2 Perceptions 364
17.3.2.1.3 Typing Versus Handwriting 365
17.3.2.1.4 Accessibility and Disability 365
17.3.2.1.5 Device Ownership 365
17.3.2.1.6 Promoting Adoption by Students 366
17.3.2.2 Administrators and Professional Staff 366
17.3.2.3 Academics 367
17.3.2.3.1 Exam Creation 368
17.3.2.3.2 Exam Marking 368
17.3.2.3.3 Promoting Adoption by Academic Staff 368
17.4 Bring-Your-Own-Device Examinations Data Analysis Case Study 369
17.4.1 Methodology 370
17.4.2 Results and Discussion 370
17.4.3 Areas for Consideration 374
17.5 Conclusions and the Future of Exam Analytics 376
References 378
Chapter 18: Experiential Learning in Labs and Multimodal Learning Analytics 380
18.1 Introduction 380
18.2 Theoretical Background 381
18.2.1 Lab-Based Learning 382
18.2.2 Experiential Learning in Laboratory-Based Learning Scenarios 383
18.2.3 Multimodal Learning Analytics 385
18.3 Learning Scenario Descriptions and Their Connection to Experiential Learning 387
18.3.1 RFID Measuring Cabinet at the Hochschule für Technik Stuttgart (HFT Stuttgart) 387
18.3.1.1 Overall Goal of the Scenario 387
18.3.1.2 Lab Scenario 387
18.3.1.3 Learning Outcomes 389
18.3.2 RFID Lab at University of Parma: Experimental Construction of RSSI Curves 389
18.3.2.1 Overall Goal of the Scenario 389
18.3.2.2 Lab Scenario 390
18.3.2.3 Learning Outcomes 392
18.3.3 Connecting Experiential Learning to Lab Learning Scenarios 393
18.3.4 Enhancing Lab Learning Activities with Learning Analytics 394
18.3.5 Technical Infrastructure for Lab-Based Learning and MLA 397
18.4 Discussion and Conclusion 400
References 402
Chapter 19: Web Analytics as Extension for a Learning Analytics Dashboard of a Massive Open Online Platform 405
19.1 Introduction 405
19.2 Related Work 407
19.3 Concept of the LA Cockpit 408
19.3.1 Activity Measurement 409
19.3.2 Web Analytics 410
19.3.3 Metrics and Visualization 412
19.4 Implementation 412
19.4.1 Device Statistics 412
19.4.2 Activity Calendar 413
19.4.3 Heatmap 414
19.5 Discussion 414
19.5.1 First Evaluation Results 415
19.5.2 Limitations 416
19.6 Conclusion 417
References 419
Chapter 20: A Dimensionality Reduction Method for Time Series Analysis of Student Behavior to Predict Dropout in Massive Open Online Courses 421
20.1 Introduction 421
20.1.1 Research on Student Attrition Prediction in MOOCS 422
20.1.2 Clickstream Data for Prediction of Student Attrition 424
20.2 Related Works 425
20.3 Experiment 426
20.4 Results 429
20.5 Discussion 430
20.6 Conclusions and Implications 433
References 435
Chapter 21: Evidence-Based Learning Design Through Learning Analytics 437
21.1 Introduction 437
21.1.1 Learning Design and Learning Analytics 438
21.1.2 Course Design Archetypes 439
21.2 Methodology 440
21.3 Findings 442
21.3.1 Distribution of Archetypes at the Local Institution 443
21.3.2 Comparison Between the Analysis of the Original Data and Local Data 443
21.3.3 Consistency Between Archetypes Extracted from Analytics and Instructors’ Predictions 447
21.4 Discussion 450
References 452
Index 455