Multimodal Usability (eBook)

Preface 6
Acknowledgements 8
Contents 9
1 Structure, Usability, Readership 17
1.1 Goals 17
1.2 How to Work on Usability Two Simple Models 18
1.2.1 Model 1: A Well-Stuffed Bag of Usability Information 18
1.2.2 Model 2: Usability in the Software Engineering Life Cycle 19
1.3 Structure and Scope of This Book 20
1.3.1 Concepts, Methods, Data Handling -- Overall Book Structure 20
1.3.2 Theory and Practice -- Hen and Egg 21
1.4 Box 1.1System vs. System Model 20
1.3.3 Intermezzo, Case, Example, Overview, Exercise, Nerdy Stuff 22
1.4 What Is Usability 23
1.4.1 What Is Usability -- The Short Answer 23
1.4.2 Usability in Perspective -- Systems and Other Artefacts 23
1.4.3 The Golden Age of HCI and After 25
1.4.4 Towards Multimodal Usability and Blending with People 26
1.5 Box 1.2Blending with People 23
1.4.5 Decomposing ''Usability'': The Terminology 27
1.4.6 Decomposing Usability: The Substance of Fitting 28
1.4.7 Implications -- Seven Questions 31
1.5 Usability Matters But How Much? 32
1.6 Readers Guide 33
1.7 Key Points 34
References 35
2 Intermezzo 1: Three Multimodal Cases 36
2.1 Contents and Origins 36
2.2 Whats Next? 37
References 38
3 Creating a Model of Use 39
3.1 AMITUDE A Model of System Use 40
3.1.1 Unpacking AMITUDE 40
3.1.2 AMITUDE Unexposed vs. AMITUDE Exposed 42
3.2 Application Type 45
3.2.1 The Notion of Application Type and Its Use 45
3.2.2 The Cases as Application Types 45
3.2.3 Classifications of Application Types 47
3.3 Users and People 47
3.3.1 Roles of Users and People in Development for Usability 48
3.4 Box 3.1Representative User Groups 47
3.3.2 People, A Global Tour 50
3.3.3 Layer 1, the Mind: Cognition, Conation and Emotion, Attention and Awareness 50
3.3.4 Layer 2, Adding the Body 52
3.3.5 Layer 3, History, Culture and Gender 53
3.3.6 User Profiles 54
3.3.7 Case User Profile Analyses 56
3.4 Tasks and Other Activities, Domain 57
3.4.1 Tasks, Other Activities, Domain 57
3.4.2 User Task Analysis: A Spoken Dialogue System Example 58
3.4.3 Generalisation: Task Model Scope, Structure and Sharing 60
Box 3.2 Literal Fitting 62
3.4.4 Other User Task Classifiers 63
3.4.5 Beyond Task Orientation: Domain-Oriented Systems 65
3.4.6 Case Task Analyses 65
3.4.7 Nerdy Stuff: Different Task Notions 67
3.5 Use Environment 67
3.5.1 Case Use Environment Analyses 68
3.5.2 Use Environment Factors Checklist 69
3.5.3 Physical and Social Use Environments 70
3.5.4 Environmental Responsibility 71
3.6 Interaction 72
3.6.1 Generalising Interaction: Information Presentation and Exchange 72
3.6.2 Generalising Two-Way Communication: Awareness, Initiative, Acts and Style 75
3.6.3 Case Interaction 77
3.7 Key Points 77
References 78
4 Modalities and Devices 80
4.1 What Is a Multimodal System? 80
4.1.1 Some Views Lead Nowhere 81
4.1.2 Sensor Systems, Carriers, Media, Thresholds 82
4.1.3 What Is a Modality? Modalities and Usability 83
4.1.4 Input and Output Modalities, Symmetry and Asymmetry 84
4.1.5 Unimodal and Multimodal Systems 85
4.2 Which Modalities Exist? Taxonomy of Unimodal Modalities 85
4.2.1 Taxonomy Structure 88
4.2.2 Nerdy Stuff: Origins of the Modality Taxonomy 89
4.2.3 Sharpening the Imagination 92
4.2.4 A Toolbox of Modalities Walk-Through 94
4.2.5 Information Channels 99
4.3 Practical Use of Modalities 100
4.3.1 Modality Description: The Classical Multimodal GUI 101
4.3.2 Modality Properties 102
4.4 Multimodal Representation 106
4.4.1 How to Choose Modality Candidates, Modality Aptitude 106
4.4.2 Relationships Among Modality Candidates 110
4.4.3 Modality Combinations of Established Usability 113
4.4.4 Combining Modalities -- Science or Alchemy? 114
4.4.5 Modality Analysis of the Cases 115
4.5 Input/Output Devices 119
4.5.1 Issues of Device Analysis 120
4.5.2 Device Analysis of the Cases 121
4.6 Key Points 123
References 123
5 Intermezzo 2: Status on Cases and Next Steps 125
5.1 Case AMITUDE Models of Use 125
5.2 Case Usability Goals, Requirements and Evaluation Criteria 125
5.3 Towards a Broader Perspective on Usability Work 131
6 Common Approaches, Methods, Planning 132
6.1 Common Usability Approaches 132
6.1.1 Thinking 133
6.1.2 Theory 134
6.1.3 Related Systems and Projects 136
6.1.4 Relevant Empirical Data, an ECA Example 136
6.1.5 Descriptive Project Sources 137
6.1.6 Experience 137
6.2 Methods for Usability 138
6.2.1 Usability Methods Are User-Centred 139
6.2.2 Usability Methods Presented in This Book 140
6.2.3 Method Group 1: Question-Answering Question-answering 142
6.2.4 Method Group 2: Meetings with Discussion Meeting with discussion 142
6.2.5 Method Group 3: Observation 144
6.2.6 Method Group 4: Imagination 145
6.2.7 Method Group 5: Interaction with System 145
6.2.8 Which and How Many Methods to Choose 145
6.2.9 Method Presentation Template 148
6.3 Writing a Usability Workplan 149
6.4 Writing a Usability Method Plan 150
Box 6.1 Do a Usability Method Plan Next Time? 151
6.4.1 Method Plan Overview 152
6.4.2 Data Collection Purpose 152
6.4.3 Getting the Right Data 153
6.4.4 Communication with Data Producers: Avoid Priming 153
Box 6.2 Priming Subjects 154
6.4.5 Subject Recruitment, Third Parties 155
6.4.6 Staff Roles and Responsibilities 155
6.4.7 Location, Equipment, Other Materials, Data, Results 155
6.4.8 Method Script 156
6.5 Key Points 157
References 157
7 Intermezzo 3: Case Usability Workplan, Design 159
7.1 Case Usability Workplans 159
7.2 Case Design 162
7.2.1 Sudoku 162
7.2.2 Treasure Hunt 165
7.2.3 Maths 166
References 169
8 Question-Answering 170
8.1 About Interviews 170
8.1.1 Purposes 171
8.1.2 Structured, Unstructured and Semi-Structured Interviews 171
8.1.3 Closed Versus Open Questions, Priming Issues 172
8.1.4 Running an Interview 173
8.1.5 Data 174
8.1.6 Advantages and Drawbacks 175
8.2 About Questionnaires 175
8.2.1 Purposes 175
8.2.2 How to Design a Questionnaire 176
8.2.3 How to Ask a Question 178
8.2.4 Getting Questionnaires Filled 179
8.2.5 Data 181
8.2.6 Advantages and Drawbacks 181
8.3 User Survey 182
8.4 Customer Interviews and Questionnaires 187
8.5 Expert Interviews and Questionnaires 189
8.6 Screening Interviews and Questionnaires 192
8.7 Pre-Test Interviews and Questionnaires 195
8.8 Post-Test Interviews and Questionnaires 199
References 204
9 Meetings with Discussion 205
9.1 Focus Group Meetings 205
9.2 Stakeholder Meetings 208
9.3 Workshops and Other Meetings with User Representatives 212
References 215
10 Observation of Users 217
10.1 Macro-Behavioural Field Methods 217
10.2 Micro-Behavioural Field Observation 222
10.3 Category Sorting 225
10.4 Observation of Users in Real Time 229
10.5 Human Data Collection in the Lab 233
10.6 Box 10.1 How Not to Do Human Data Collection 233
References 238
11 Imagination 240
11.1 Use Cases and Scenarios 240
11.2 Personas 244
11.3 Cognitive Walk-Through 249
11.4 Guideline-Based Usability Development and Evaluation 253
11.5 Usability Standards 262
11.5.1 ISO/IEC Usability Work Methodology 263
11.5.2 Comparison with This Book 267
References 268
12 Interaction with the System 270
12.1 Mock-Up 270
12.2 Wizard of Oz 274
12.3 Implemented Prototype Lab Test 278
12.4 Field Test 283
12.5 Think-Aloud 288
References 292
13 Lab Sessions with Subjects 294
13.1 Lab Test and Development Methods. Micro-Control 294
13.2 Session Preparation Subject Recruitment 295
13.2.1 How to Select Subjects 296
13.2.2 Number of Subjects 297
13.2.3 Where to Get Subjects 298
13.2.4 Information for Initial Subject Contact 298
13.2.5 Will All Subjects Show Up? 299
13.2.6 Remuneration of Subjects Remuneration of subjects 300
13.3 Session Preparation Material and Equipment 300
13.3.1 Checklist for What to Do Before the Session 301
13.3.2 System Model, Software, Equipment 302
13.3.3 Presentation Material, Instructions, Instructions 302
13.3.4 Consent Forms and Other Permissions 303
13.3.5 Scenarios and Priming 304
13.3.6 Guides, Manuals, Other Written Support 308
13.3.7 Subject Debriefing Guide 309
13.4 During the Session 309
13.4.1 Subjects Arrive 309
13.4.2 Introduction and Instructions to Subjects, Subject Training 310
13.4.3 Carrying Out the Session 311
13.4.4 When and How to Help a Test Subject 312
13.4.5 What Not to Say When Subjects Ask Questions 312
13.4.6 When to Deviate from the Method Plan 313
13.4.7 The Experimenter 314
13.5 After the Session 314
13.5.1 Subject Debriefing 314
13.5.2 Data Handling 315
13.5.3 Post-Session Subject Contact Post-session Subject contact 315
13.6 Key Points 315
References 315
14 Intermezzo 4: Case Usability Method Plan 316
14.1 Data Collection Purpose 316
14.2 Getting the Right Data 316
14.3 Communication with the Data Producers 316
14.4 Subject Recruitment, A Representative User Group 318
14.5 Staff Roles and Responsibilities 319
14.6 Location, Equipment, Other Materials, Data, Results 319
14.7 Method Script 320
15 Data Handling 322
15.1 The Data Handling Cycle 322
15.2 The Nature of Data, Corpora, Data Resources 324
15.3 Raw Data Files, Data Book-Keeping, Meta-Data 326
15.3.1 Raw Data Files 326
15.3.2 Data Book-Keeping 327
15.3.3 Meta-Data 328
15.4 Preparing to Use the Data 328
15.4.1 Raw Data Validation 329
Box 15.1 Data Collection Story--Speech for Consumer Devices 329
15.4.2 Raw Data Post-Processing 330
15.5 Raw Data Annotation 331
15.5.1 Bird Watching and Data Coding 333
Box 15.2 Bird Watching Story 333
15.5.2 Purposes of Corpus Annotation 336
15.5.3 Orthographic Transcription: An Example 337
15.5.4 Multimodal Annotation, Blending with People 339
15.5.5 Languages and Coding Schemes for Blending with People 342
15.5.6 Coding Tools 350
15.6 Coding Procedure and Coding Best Practice 353
15.7 Key Points 354
References 354
16 Usability Data Analysis and Evaluation 357
16.1 Data Analysis: Process and Issues 357
16.1.1 Context of Analysis 358
16.1.2 A General Model of Usability Data Analysis 359
16.1.3 Preliminary Analysis and Report 361
16.1.4 Data Analysis 361
16.1.5 Problem Analysis: Identify Implications and Priorities 364
16.2 Usability Evaluation 367
16.2.1 Usability Arguments, Requirements, Criteria 368
16.2.2 Usability Evaluation Criteria: The Evaluation Hierarchy 369
16.3 Types of Evaluation Results and Purposes, Semi-Nerdy Stuff 371
16.3.1 Technical Issues Versus Design Issues 372
16.3.2 Objective and Subjective Evaluation 372
16.3.3 Quantitative and Qualitative Evaluation 373
16.3.4 Five Different Evaluation Purposes 374
16.4 Types of Evaluation Criteria 376
16.4.1 Common Evaluation Criteria from ISO Usability Standards 376
16.4.2 Other Kinds and Sources of Criteria 379
16.5 Usability Evaluation in Practice 380
16.5.1 Sudoku and Other Evaluations 381
16.5.2 Task Success: An Example Usability Evaluation Criterion 382
16.6 Box 16.1 Task Success Rate 380
16.5.3 Sudoku Evaluation: General Points 384
16.6 Reporting the Results of Data Analysis 389
16.7 Key Points 390
References 390
17 Intermezzo 5: Sudoku Usability Evaluation 392
17.1 Data 392
17.1.1 Context of Data Analysis 393
17.1.2 Anomalies 393
17.1.3 Interview Questions: Three Issues 393
17.1.4 User Statistics 394
17.1.5 Game Statistics 395
17.1.6 Data Validation 396
17.2 Technical Issues 396
17.2.1 Robustness Robustness 397
17.2.2 Speech and Pointing 397
17.2.3 Other Issues 400
17.3 Modality Appropriateness 400
17.3.1 Component Modality Appropriateness 401
17.3.2 Gameplay Using Speech, Pointing and Graphics 405
17.3.3 Information Appropriateness 407
17.4 Functional Issues 409
17.5 User Interviews: Closed Questions Overview 415
17.6 Conclusions 416
References 417
18 Multimodal Usability: Conclusions and Future Work 418
18.1 Simple to Grasp? 418
18.2 Nerdy Stuff: Generalisations of HCI Made in This Book 418
18.2.1 Generalised Scope of Contents for Multimodal HCI Multimodal HCI See HCI 419
18.2.2 Generalised Methodology for Multimodal HCI 420
18.2.3 Generalised Framework and Theory of Multimodal HCI 421
18.3 Future Work 423
Abbreviations 425
Index 427