Statistical Methods for the Evaluation of University Systems (eBook)

Preface 4
Contents 8
Contributors 10
Part I Introduction: Different Perspectives of the Evaluation of the Italian University System 13
1 TES – From Impressionism to Expressionism 14
Lorenzo Bernardi 14
1.1 Foreword: Excusatio Non Petita 14
1.2 Pars Destruens: Accusatio Manifesta 15
1.3 Pars Costruens: Non nova, Sed Nove 17
1.3.1 Guiding Principles 17
1.3.2 The Proposal: A First, Almost Utopian Design 21
1.3.3 A Possible Design 23
2 The Assessment of University Teaching by Students: TheOrganizational Perspective 26
Luigi Enrico Golzio 26
2.1 Assessment in Organisations 26
2.2 Assessment by Students in Italian Universities 28
2.3 Assessment by Students as an Organisational Process 29
2.4 The Content of Assessment by Students 34
2.5 The Case of the University of Sassari 39
References 42
3 University League Tables 43
L. Bernardi, P. Bolzonello, and A. Tuzzi 43
3.1 Introduction 43
3.2 The Censis Ranking System 44
3.3 Indicators for Evaluation and Measurement 45
3.4 The Censis Data 47
3.4.1 Normalization and Aggregation 47
3.4.2 The Simple Indicators Used by Censis 48
3.4.3 Preliminary Analysis 48
3.5 Alternative Ways to Analyse the Data 52
3.6 Results 55
3.7 Conclusions 56
References 60
Part II The Evaluation in the Italian Universities: Student Teaching Evaluation 62
4 Structural Equation Models and Student Evaluation of Teaching: A PLS Path Modeling Study 63
Simona Balzano and Laura Trinchera 63
4.1 Introduction 63
4.2 PLS Approach to Structural Equation Models 64
4.3 Applying PLS-PM to Students Evaluation of Teaching 67
4.3.1 The Data and Model Specification 67
4.3.2 The Results 69
4.4 Concluding Remarks 73
References 73
5 A Study on University Students' Opinions about Teaching Quality: A Model Based Approach for Clustering Ordinal Data 75
Marcella Corduas 75
5.1 Introduction 75
5.2 A Mixture Distribution for Ordinal Data 76
5.3 The Kullback-Liebler Divergence 77
5.4 Clustering 78
5.5 The Analysis of Students' Opinions 79
5.5.1 The Data Set 79
5.5.2 The Results 80
5.6 Final Remarks 84
References 84
6 The Impact of Teaching Evaluation: Factors that Favour Positive Views from Student Representatives 86
Simone Gerzeli 86
6.1 Introduction 86
6.2 Methods 87
6.2.1 Study Design 87
6.2.2 Statistical Analysis 88
6.3 Results 90
6.3.1 Respondents 90
6.3.2 The Availability and Discussion of the Teaching Evaluation Results 91
6.3.3 Changes Induced by the Results of the Teaching Evaluation 93
6.3.4 The Usefulness of the Teaching Evaluation as Perceived by the Student Representatives 94
6.3.5 The Multilevel Regression Model 96
6.4 Concluding Remarks 97
References 99
7 University Teaching and Students' Perception: Models of the Evaluation Process 100
Maria Iannario and Domenico Piccolo 100
7.1 Introduction 100
7.2 Measurement of Students' Perception About Teaching Quality 101
7.3 Perception and Rating as Complex Decisions 102
7.4 Latent Variables and Item Response Theory 103
7.5 An Alternative Model for the Evaluation Process 106
7.5.1 Rationale for CUB Models 107
7.5.2 CUB Models 107
7.6 Empirical Evidences for University Teaching Evaluation 109
7.6.1 CUB Models Without Covariates 109
7.6.2 CUB Models with Covariates 111
7.7 Concluding Remarks 114
References 115
8 Students' Evaluation of Teaching Effectiveness: Satisfaction and Related Factors 120
Michele Lalla, Patrizio Frederic, and Davide Ferrari 120
8.1 Introduction 120
8.2 Literature Review 122
8.3 Questionnaire and Data 124
8.4 Models and Results 127
8.5 Conclusions 134
References 135
Part III The Evaluation in the Italian Universities: Statistical Methods for Careers and Services Evaluation 137
9 Modeling Ordinal Item Responses via Binary GLMMs and Alternative Link Functions: An Application to Measurement of a Perceived Service Quality 138
Vito M.R. Muggeo and Fabio Aiello 138
9.1 Introduction 138
9.2 Data 139
9.3 Methods 141
9.3.1 The GLMM Framework 141
9.3.2 Alternative Link Functions 142
9.4 Results 143
9.5 Conclusions 147
References 148
10 Analyzing Undergraduate Student Graduation Delay: ALongitudinal Perspective 149
Paola Costantini and Maria Prosperina Vitale 149
10.1 Introduction 149
10.2 The Graduation Delay Issue 150
10.3 Measuring and Analyzing Graduation Delay 151
10.4 Defining a Longitudinal Graduation Delay Indicator 153
10.5 Latent Curve Model to Monitor Student Careers 154
10.6 A Case Study: The Delay Patterns of a Cohort of Undergraduate Students 155
10.6.1 A Conditional Linear Latent Curve Model 156
10.7 Some Concluding Remarks 162
References 162
11 Assessing the Quality of the Management of Degree Programsby Latent Class Analysis 164
Isabella Sulis and Mariano Porcu 164
11.1 Introduction 164
11.2 Building up a Composite Indicator 164
11.2.1 A Measure of the Perceived Quality of a University Service 165
11.3 Methodological Issues 166
11.3.1 Sorting Latent Classes 167
11.4 The Application 167
11.4.1 The Data 167
11.4.2 The Analysis 169
11.5 Final Remarks 174
References 174
Part IV Research Design and Data for Evaluation: University Between the High School and the Labour Market 176
12 The Multicriteria Electre III Model Applied to the Evaluation of the Placement of University Graduates 177
Rosalinda Allegro and Ornella Giambalvo 177
12.1 Preliminary Remarks 177
12.2 The Data 178
12.3 The Multicriteria Electre III Model 182
12.4 Groups of Disciplines Ranking 191
12.5 Final Remarks 195
References 196
13 Competences and Professional Options of the Italian Graduates: Results from the Textual Analysis of the Degree Course Information Data 197
S.Balbi, C.Crocetta, M.F.Romano, S.Zaccarin, and E.Zavarrone 197
13.1 Introduction 197
13.2 Textual Analysis of the University Education Offer 198
13.3 Identification of the Competences Offeredand the Job Possibilities 200
13.4 Consistency of the Education Paths with Employment Perspectives 204
13.5 Coherence Between the Three Year Degree Courses and the Specialised Degree Programs 207
13.6 Final Remarks 208
References 209
14 After the PhD: A Study of Career Paths, Job and Training Satisfaction Among PhD Graduates from an Italian University 210
Stefano Campostrini 210
14.1 Introduction 210
14.2 The Survey 211
14.3 How PhD Graduates Evaluate Doctorate Programme Teaching 212
14.4 How PhD Graduates Evaluate Their Doctorate Programme Research Experiences 214
14.5 How PhD Graduates Evaluate Their Doctorate Programme Research Experiences 215
14.6 PhD Holder Levels of General Job Satisfaction 219
14.7 The Impact of Evaluation: A Reflection on the Use of the Evaluation 222
References 223
15 Secondary School Choices in Italy: Ability or Social Background? 224
Dalit Contini and Andrea Scagni 224
15.1 Introduction 224
15.2 The Methodology 226
15.3 The Analysis for Italy 228
15.3.1 Institutional Features 228
15.3.2 The Data 229
15.3.3 Final Marks in Lower Secondary School 229
15.4 Sample Selection 231
15.4.1 The Problem 231
15.4.2 Supporting the Assumptions 233
15.5 The Empirical Analysis 235
15.5.1 Semi-parametric Approach 235
15.5.2 Sample Selection Correction Factors 236
15.5.3 Results 237
15.6 Conclusions 244
References 245
16 Labour Market Outcomes for Ph.D. Graduates 247
Antonella D'Agostino and Giulio Ghellini 247
16.1 Introduction 247
16.2 The Data 248
16.3 Socio-Biographic Background of Ph.D. Graduates 249
16.4 Employment Status and Earnings One-Year After the Ph.D. Thesis 250
16.5 Modelling Earnings of Ph.D. Graduates 252
16.5.1 Methodological Background 252
16.5.2 Sensitivity Analysis 254
16.5.3 Results 255
16.6 Further Discussion and Conclusions 257
References 259
17 Labour Market Performance of University Graduates: Evidence from Italy 261
B. d'Hombres, S. Tarantola, and D. Van Nijlen 261
17.1 Introduction 261
17.2 Review of the Literature on the Determinants of Labor Market Outcomes of Graduates 263
17.3 Labor Market Determinants of Italian Graduate Students 264
17.3.1 Data 264
17.3.2 Empirical Methodology 268
17.3.3 Empirical Results 271
17.4 Faculty-Performance Indicators: The Case of Economics 275
17.4.1 Unadjusted Versus Adjusted Ranking Based on Labor Market Outcomes 275
17.4.2 Uncertainty Associated with Rankings 279
17.5 Conclusion 281
References 282