Grade Moderation : 2020 Hindsight
There are no easy answers to the 2020 grading problem, that much is clear. The disgruntled murmurings in Scotland may be nothing compared to the cacophony of anguish we’re about to be subjected to on this side of the border over the next couple of weeks. There are going to be some very distressed students, and parents - understandably, justifiably so. I sympathise with those suggesting that we should just let all teacher predicted grades stand without moderation. But on the other hand Nicola Sturgeon and John Swinney do have a point when they say that this would rather undermine the credibility of the system on which we apparently have come to rely. The question we should now be asking is: are we ok with being completely reliant on a system whose ethical foundations are so shaky in the first place?
The main aim of the algorithms used to adjust this year’s scores appears to be to preserve the credibility of our exam grading system - through manually correcting teacher assessments - so as to ensure that the overall national set of results looks as close as possible to how it did in previous years. Thus giving the appearance of consistency. People like consistency. It’s predictable, reassuring, comforting.
So they have been frantically moderating students' grades in order to create an appearance of normality so realistic that the edifice of our grading system doesn't crumble entirely. Like a hapless assassin, putting sunglasses on a corpse and propping them up so as to fool unsuspecting passersby.
Except individual stories are very different from national statistics, as the government is finding out. Stories will emerge, are emerging, of students who have been the victims of compounded misfortune. There will be students who have not only had to endure this unprecedented disruption to their lives, the lack of closure stemming from not being able to complete their final school year, the myriad difficulties of working hard and aiming high in a low-achieving school. Perhaps taking big personal risks by fighting against a dominant social current of low-achievement. I know these students, I’ve taught these students. Students who had the chance, after four, five, six years of hardship, to finally succeed despite the odds stacked against them. That they have been denied this chance to overcome their circumstances is bad enough. But to now find themselves once again penalised because of these very circumstances… While their peers at the more successful school up the road are rewarded once again for the luck they've already enjoyed. It’s nauseating to even think about.
But let’s not kid ourselves into thinking that this unfairness is anything new. The inequalities, the unfairness of it all, it’s always been there. Go to any comprehensive school and you'll find teachers who could tell you all about the bright, deserving students who just under-performed on the day, who never got on with exams, the bright, deserving students whose strengths could never be reflected in one final grade. Students with additional needs, for whom an extra 25% was never quite going to cut it. Students with crippling test anxiety. Bright, deserving students who could have done so much better if they'd just had a 1:1 tutor, or their own room, or at least access to a laptop, or just regular bedtimes, or regular meals. Bright, hard-working students who deserve more than the letters we give them when they leave.
The unfairness has always been there. What this current situation has done is simply to force us to explicitly and ‘manually’ factor this inequity into the calculations ourselves, rather than have it done by some invisible hand. This is not creating new problems, it just brings to the surface all the ones that used to remain quietly hidden away.
Hence why I've started seeing this talked about online in terms of issues facing machine learning and the trolley problem.
“there has to be a small number of high achievers, and a big group of 'average' folk - and someone HAS to fail. We are so in thrall of the idea that intelligence, ability and attainment are inextricably bound together, this is how it must be… if every teacher's estimates had been accepted, no-one would complain, and we'd continue to support a system that is inherently manipulative, because it's not about what a kid can do, but what they can do in comparison with others. We'd carry on our merry way, because giving a kid a single grade at the end of a year is comfortingly familiar."
So instead of just trying to keep things looking ‘normal’ until we can go back to how things were, we find ourselves with a unique opportunity to reflect and re-examine things, which it would be foolish to pass up. I am expecting to see a flood of individual cases shared on social media where students have been unfairly downgraded. I am also expecting to have to console a number of actual, distraught children, and their parents, when I go in on ‘results’ day. But let’s not be fooled into thinking that this is a uniquely unfair year. This current debate is just yet another quirk that 2020 has thrown up, where we are being forced to confront ethical issues that have previously lain hidden within an exams system we have all but forgotten how to question.
 |
| Clouds (1905) by Arkhip Kuindzhi |
The main aim of the algorithms used to adjust this year’s scores appears to be to preserve the credibility of our exam grading system - through manually correcting teacher assessments - so as to ensure that the overall national set of results looks as close as possible to how it did in previous years. Thus giving the appearance of consistency. People like consistency. It’s predictable, reassuring, comforting.
“In designing the statistical model we took a number of decisions that work in students’ favour and mean that results overall will be no worse than previous years." (Spokesperson for Ofqual)
So they have been frantically moderating students' grades in order to create an appearance of normality so realistic that the edifice of our grading system doesn't crumble entirely. Like a hapless assassin, putting sunglasses on a corpse and propping them up so as to fool unsuspecting passersby.
Except individual stories are very different from national statistics, as the government is finding out. Stories will emerge, are emerging, of students who have been the victims of compounded misfortune. There will be students who have not only had to endure this unprecedented disruption to their lives, the lack of closure stemming from not being able to complete their final school year, the myriad difficulties of working hard and aiming high in a low-achieving school. Perhaps taking big personal risks by fighting against a dominant social current of low-achievement. I know these students, I’ve taught these students. Students who had the chance, after four, five, six years of hardship, to finally succeed despite the odds stacked against them. That they have been denied this chance to overcome their circumstances is bad enough. But to now find themselves once again penalised because of these very circumstances… While their peers at the more successful school up the road are rewarded once again for the luck they've already enjoyed. It’s nauseating to even think about.
But let’s not kid ourselves into thinking that this unfairness is anything new. The inequalities, the unfairness of it all, it’s always been there. Go to any comprehensive school and you'll find teachers who could tell you all about the bright, deserving students who just under-performed on the day, who never got on with exams, the bright, deserving students whose strengths could never be reflected in one final grade. Students with additional needs, for whom an extra 25% was never quite going to cut it. Students with crippling test anxiety. Bright, deserving students who could have done so much better if they'd just had a 1:1 tutor, or their own room, or at least access to a laptop, or just regular bedtimes, or regular meals. Bright, hard-working students who deserve more than the letters we give them when they leave.
 |
| Judgement Day by Behzad Tabar |
The unfairness has always been there. What this current situation has done is simply to force us to explicitly and ‘manually’ factor this inequity into the calculations ourselves, rather than have it done by some invisible hand. This is not creating new problems, it just brings to the surface all the ones that used to remain quietly hidden away.
Hence why I've started seeing this talked about online in terms of issues facing machine learning and the trolley problem.
Basically we are happy, up to a point, to accept that some people will get hurt and some people won’t. We say it’s horrible, but inside we think well that’s just life, and as long as we’re not personally affected we move on. But when you give someone a lever, or when you have to a programme a driverless car to make ‘decisions’ about who is going to get hurt and who will be safe, or if you give teachers the power to decide their own students’ exam grades, then you run into ethical problems that never really existed before. Because regardless of how fair or unfair the results are (whatever this means), we are still left more unhappy by the fact that those decisions appear to be taken by something new, someone that we don’t really trust, or who might have motives, biases. Rather than the cold, motiveless, reassuring systems that we have come to accept.
And let's not forget the uniquely demoralising position this puts the children in too, from a perspective of psychological and personal development. Years of awaiting this final empowering moment of agency, only to find it taken away at the last moment. And sure they might have wasted this moment. They might not have done. As someone from the social mobility commission put it, "unlike in normal years, they don’t have the security of knowing there is some kind of relationship with something they’ve actually done." (Emphasis mine)
It’s as if someone’s taken the lid off the exams engine, and found that there is a component that makes it harder for students from poorer backgrounds to do well. Now they are told to build this engine from scratch, they are forced to sit round the table and discuss whether or not to put this component back in. Keep it in, and you’re knowingly opting to preserve social inequalities when you had the chance to reduce them. Leave it out, and you risk seriously compromising the structural credibility of the engine. And remember, it was running just fine while we didn't have to look too closely at how it worked.
And let's not forget the uniquely demoralising position this puts the children in too, from a perspective of psychological and personal development. Years of awaiting this final empowering moment of agency, only to find it taken away at the last moment. And sure they might have wasted this moment. They might not have done. As someone from the social mobility commission put it, "unlike in normal years, they don’t have the security of knowing there is some kind of relationship with something they’ve actually done." (Emphasis mine)
 |
| Oriental Memory by Guang-Yu Zhan |
It’s as if someone’s taken the lid off the exams engine, and found that there is a component that makes it harder for students from poorer backgrounds to do well. Now they are told to build this engine from scratch, they are forced to sit round the table and discuss whether or not to put this component back in. Keep it in, and you’re knowingly opting to preserve social inequalities when you had the chance to reduce them. Leave it out, and you risk seriously compromising the structural credibility of the engine. And remember, it was running just fine while we didn't have to look too closely at how it worked.
It's something that came up a little while ago when Amazon tried to develop an AI tool that scanned the CVs of job applicants automatically, so that their staff didn't have to. Except to build the software they used all the data and patterns from the previous ten years of CV vetting by their human staff. What they ended up unwittingly developing was an algorithm that mimicked (and thus exposed) all the massive gender biases that had been rife in their recruitment systems, but which had previously gone unchallenged. The AI tool was eventually scrapped. The algorithms used in American courts to predict the likelihood of someone re-offending, which turned out to be significantly biased against black prisoners, still seem to be in use.
So perhaps this is an engine that needed to be dismantled anyway. Perhaps we should be concerned about depending so heavily on a single set of norm-referenced tests designed simply to allocate students their positions on a bell curve. In reading about this I came across a thread by Raymond Soltysek, who has frankly articulated these issues better than I ever could:
“there has to be a small number of high achievers, and a big group of 'average' folk - and someone HAS to fail. We are so in thrall of the idea that intelligence, ability and attainment are inextricably bound together, this is how it must be… if every teacher's estimates had been accepted, no-one would complain, and we'd continue to support a system that is inherently manipulative, because it's not about what a kid can do, but what they can do in comparison with others. We'd carry on our merry way, because giving a kid a single grade at the end of a year is comfortingly familiar."
So instead of just trying to keep things looking ‘normal’ until we can go back to how things were, we find ourselves with a unique opportunity to reflect and re-examine things, which it would be foolish to pass up. I am expecting to see a flood of individual cases shared on social media where students have been unfairly downgraded. I am also expecting to have to console a number of actual, distraught children, and their parents, when I go in on ‘results’ day. But let’s not be fooled into thinking that this is a uniquely unfair year. This current debate is just yet another quirk that 2020 has thrown up, where we are being forced to confront ethical issues that have previously lain hidden within an exams system we have all but forgotten how to question.
Comments
Post a Comment