Missing the Mark (eBook)

Dennis Sherwood is a management consultant with experience of solving complex problems. He has a Physics Masters from the University of Cambridge, an MPhil in Molecular Biophysics and Biochemistry from Yale University and a PhD in biology from the University of California in San Diego. After being a consulting partner at Deloitte Haskins + Sells, and Coopers & Lybrand, he became an executive director at Goldman Sachs. He now runs his own business, The Silver Bullet Machine Manufacturing Company Limited, specialising in organisational creativity and innovation. He is author of 14 books. Dr Robin Bevan is Headteacher of Southend High School for Boys. He was National President of the National Education Union in 2020-21.

DEDICATION. To the unknown, but very large, number of young people who have been damaged by the award of wrong exam grades, in the hope that this will not happen in the future

STATISTICS. Over the decade from 2010 to 2019, a total of about 70 million GCSE, AS and A-level grades were awarded following each year's summer exams in England. • Of which around 17.5 million were wrong. • Yes,17.5 million. • That's about 1 wrong grade in every 4.

FOREWORD. Foreword by Dr Robin Bevan Headteacher, Southend High School for Boys NEU Past National President, 2020-21. old standard! Well, maybe not! For many years England's GCSE and A-level qualifications enjoy an international reputation as world-leading. This book forensically analyses grades

1. EXAM GRADES ARE IMPORTANT. A'Level and GCSE grades can affect life chances, yet the regulator Ofqual's own statistics show that 1 in 4 grades can be wrong. This book gives all the evidence, discusses the implications, and – most importantly – offers some solutions

2. EXAMS IN ENGLAND. Deals with GCSE, AS-Level and A-level exams, exam centres and schools, awarding bodies, the regulators Ofsted, DfE, and Ofqual, the House of Commons Education Committee, marking, grade structures, grade boundaries, criterion referencing, cohort referencing, norm referencing

3. ARE EXAM GRADES 99.2% ACCURATE? Edexcel's claim that grades are 99.2% accurate on results day (taken from Pearson-Edexcel's website), assesses comments made about grade reliability by School Standards Minister Nick Gibb, Ofqual's Chief Regulator Glenys Stacey and Ofqual's Marking Consistency Metrics

4. TWO IMPORTANT WORDS: 'ACCURATE' AND 'RELIABLE'. What does 'accurate' mean in the context of exams? Can exam marks ever be accurate? Looking at how marking by different examiners can alter the grade. The reliability of a grade is the probability that an originally-awarded grade is confirmed

5. SUMMER 2016: OFQUAL MAKE IT HARDER TO APPEAL. Until 2015, candidates unhappy with their grades could – for a fee – request a re-mark. But in May 2016, Ofqual changed the rules for challenges and appeals, intentionally denying access to an expert second opinion. Why did the regulator do that?

6. OFQUAL'S FIRST MEASURES OF GRADE RELIABILITY. In 2015, Ofqual carried out an extensive study in which the entire cohorts of GCSE, AS and A-level scripts, in 12 subjects, were marked twice: firstly, as normal, by an ordinary examiner; secondly, by a senior examiner, whose mark was designated 'true'

7. OFQUAL'S REAL MEASURES OF GRADE RELIABILITY. Using Ofqual data, the author calculates the reliability of mathematics grades is 96%, chemistry 92%, physics 88% etc right down to the lower arts subjects such as English literature (58%), history (56%) and combined English language and literature (52%)

8. WHY GRADES ARE UNRELIABLE. Three reasons why marking (marketing error) is not the problem. Instead, there is a more powerful explanation – fuzziness. Ofqual admit: 'There is often no single, correct mark for a question'. So marks may legitimately vary, causing valid but 'fuzzy' results

9. NOVEMBER 2018 TO SUMMER 2019. Newspaper sized on Ofqual's admission about grade unreliability, with reports in the Daily Telegraph and Sunday Times. Ofqual insists that marking error is to blame for any problems but stresses that results and grades are overwhelmingly accurate

10. 2020: CAGS AND RANK ORDERS. Due to the Covid epidemic, the UK government cancels all physical GCSE, AS-Level and A-Level exams and replaces them with an Ofqual algorithm, which is crude. 'The details of the algorithm were both important and missing from Ofqual's Guidance documents.'

11. THE GREAT CAG CAR CRASH. On results day, Thursday 13th August, and over the next few days, progressively more stories surfaced on some 'peculiarities' in the algorithm's results.