Questioning the role of AI in exam marking

In Jan­u­ary 2020, Ofqual invit­ed schools to sub­mit stu­dent essays for a research project to explore the poten­tial of arti­fi­cial intel­li­gence (AI) in exam mark­ing. In the accom­pa­ny­ing blog, the exam reg­u­la­tor reas­sured teach­ers and pupils this was just a pre­lim­i­nary test and: “We wouldn’t sud­den­ly see AI being used at scale in mark­ing high-pro­file qual­i­fi­ca­tions overnight.”

Just sev­en months lat­er, prime min­is­ter Boris John­son was blam­ing a “mutant algo­rithm” for an exam fias­co that saw more than 40 per cent of A‑level stu­dents in Eng­land get down­grad­ed, includ­ing many high achiev­ers from dis­ad­van­taged back­grounds. That led to the AI mark­ing study being put on hold. 

This despite the A‑level algo­rithm being based on sta­tis­ti­cal meth­ods rather than AI, not to men­tion it was try­ing to achieve the impos­si­ble by gen­er­at­ing exam results with­out there being any exams.

Still, in the public’s mind, it was all part of the same prob­lem. A sud­den and unset­tling ced­ing of pow­er to opaque machine-led oper­at­ing sys­tems with real-world impli­ca­tions for young people’s futures. As Robert Hal­fon, Con­ser­v­a­tive MP and chair­man of the edu­ca­tion select com­mit­tee, says: “What Ofqual needs now is a peri­od of long reflec­tion and inter­nal exam­i­na­tion rather than an AI rev­o­lu­tion.”  

Use of algorithms and AI

Algo­rithms, sta­tis­tics, data sci­ence and AI are already wide­ly used in edu­ca­tion. Ofqual them­selves have been using algo­rithms for years to off­set grade infla­tion and smooth out region­al dis­crep­an­cies with­out any pub­lic fuss or wor­ry.

AI is used in pla­gia­rism detec­tion, exam mark­ing and tutor­ing apps with real-time feed­back, such as On-Task and San­ta for TOEIC (Test of Eng­lish for Inter­na­tion­al Com­mu­ni­ca­tion) in South Korea, which has more than one mil­lion sub­scribers and appears to rapid­ly improve stu­dent test scores in just 24 hours using an intel­li­gent machine learn­ing-based algo­rithm.

In Amer­i­ca, AI-dri­ven plat­form Bak­pax that auto-grades stu­dents’ work, and is free and com­pat­i­ble with Google Class­room, has been prov­ing pop­u­lar with teach­ers dur­ing the pan­dem­ic. Its mar­keters promise teach­ers “more time for your stu­dents or your­self” and to “pro­vide stu­dents with instant feed­back when they’re still most engaged”, along with per­for­mance insights on which top­ics are eas­i­er or more chal­leng­ing for stu­dents.

Dee Kane­jiya, founder and chief exec­u­tive of Cog­nii, an AI-based plat­form that uses nat­ur­al lan­guage con­ver­sion to assess pas­sages of longer text that have tra­di­tion­al­ly been hard­er for AI to grade accu­rate­ly, wants to help cor­rect what he sees as an over-reliance on mul­ti­ple choice ques­tions in US assess­ments. 

He believes these do not help stu­dents in the real world and is a for­mat that favours boys over girls. But mark­ing longer answers is time con­sum­ing for teach­ers and there­fore expen­sive, which is where he hopes Cog­nii can help.

Kane­jiya is excit­ed about the poten­tial of AI to free up teach­ers from repet­i­tive tasks such as mark­ing, though he insists it isn’t about replac­ing them. “You get more time for that inti­mate rela­tion­ship between fac­ul­ty and stu­dents if teach­ers are not grad­ing,” he says. “They can spend more qual­i­ty time with the stu­dents, time for the emo­tion­al side of things, which they’re good at.” 

He also thinks cloud-based AI sys­tems such as Cog­nii could play a cru­cial role in improv­ing the access and afford­abil­i­ty of edu­ca­tion glob­al­ly, espe­cial­ly in coun­tries which suf­fer teacher short­ages.

Being aware of data bias

But the poten­tial labour and cost ben­e­fits to using AI in edu­ca­tion inevitably come with some down­sides. Last year there was a sto­ry about stu­dents gam­ing an AI mark­ing sys­tem by typ­ing in lots of key­words in oth­er­wise inco­her­ent sen­tences and scor­ing full marks. Kane­jiya says they have checks in place to pre­vent that type of abuse with Cog­nii. “We have fac­tored in syn­tax and seman­tics to the sys­tem so that couldn’t hap­pen,” he says.

Algo­rith­mic and AI bias is a real con­cern as well. We expect these mod­els to be neu­tral and impar­tial, but the data we feed them means they are often sub­ject to many of society’s exist­ing bias­es and can dis­crim­i­nate against cer­tain user groups. 

Han­sol Lee and Ren­zhe Yu are post­grad­u­ate stu­dents at the Cor­nell Future of Learn­ing Lab and experts in algo­rith­mic fair­ness. “Machines learn his­tor­i­cal prin­ci­ples and rules, and there­fore learn what to apply to the future,” says Yu. “But that his­tor­i­cal data will con­tain inequal­i­ties, such as stu­dents of colour have had low­er achieve­ment in the past or black stu­dents don’t learn maths. That sim­ple rule could make the sys­tem rec­om­mend those stu­dents don’t learn maths.”

Bias can also occur if an AI sys­tem is trained on a dataset that has less data for a cer­tain stu­dent group. It’s a data rep­re­sen­ta­tion prob­lem that isn’t delib­er­ate, but nonethe­less exists. “There might not be a quick-and-easy fix,” says Lee. “But it’s impor­tant to be aware of the prob­lem so you can find oth­er ways to make the sys­tem less biased.”

In adap­tive learn­ing tests, which are often used in pri­vate school entrance exam­i­na­tions in the UK, stu­dents are exposed to a dif­fer­ent ques­tion path accord­ing to each answer they give, which presents its own con­cerns. 

“One study found the algo­rithm can make a more accu­rate diag­no­sis of the student’s per­for­mance if they’re a quick learn­er on the more advanced path,” says Yu. “So, any­thing it rec­om­mends to the quick learn­er would be more appro­pri­ate, but using the same AI, the slow learn­er will start to suf­fer.”

Learning about learning

Last sum­mer gave the British pub­lic an uncom­fort­able insight into the dan­gers of data-sci­ence mod­el­ling in exam mark­ing. Dr Rose Luckin is pro­fes­sor of learn­er-cen­tred design at Uni­ver­si­ty Col­lege Lon­don and direc­tor of EDUCATE, a hub for edu­ca­tion­al tech­nol­o­gy star­tups. Is she wor­ried the A‑level deba­cle will derail the use of AI in UK edu­ca­tion? “It has set the cause back,” she says, but cau­tions against reject­ing its use entire­ly because of con­cerns regard­ing algo­rith­mic fair­ness.

Luckin adds: “To avoid AI because it’s too risky would be a huge shame, as there is lots of poten­tial for schools and espe­cial­ly for dis­ad­van­taged learn­ers.”

These ben­e­fits include a more tai­lored and adap­tive assess­ment sys­tem cen­tred on the indi­vid­ual learn­er, rather than the cur­rent one-size-fits-all mod­el that favours a cer­tain type of stu­dent who is good at exams. 

“At the moment, we assess what’s quite easy to assess,” says Luckin. “But AI lets us assess a num­ber of things we can’t assess that are things soci­ety needs for the fourth indus­tri­al rev­o­lu­tion, such as col­lab­o­ra­tive prob­lem solv­ing, which PISA [Pro­gramme for Inter­na­tion­al Stu­dent Assess­ment] intro­duced a cou­ple of years ago, metacog­ni­tive aware­ness, self-reg­u­la­tion; incred­i­bly impor­tant things that boost learn­ing.”

She says we could use AI to do con­tin­u­ous for­ma­tive assess­ment, rather than one-off exams. “That could help us real­ly under­stand the learn­ing process, as well as the learn­ing prod­uct, so it becomes a learn­ing activ­i­ty not just an assess­ment activ­i­ty. You can learn about your­self as a learn­er, what your strengths and weak­ness­es are, where you need to focus more atten­tion and what cop­ing strate­gies work for you,” says Luckin.

One thing she doesn’t want is for AI in edu­ca­tion to be sole­ly reduced to the role of auto-mark­ing exams, as she thinks this would be a missed oppor­tu­ni­ty. “Assess­ment will always be the tail that wags the dog in edu­ca­tion,” she says. “It’s so impor­tant to the sys­tem, to the gov­ern­ment, but also to par­ents, so I think there will be a strong focus on using AI in assess­ment. 

“But what I fear is that we’ll invest mon­ey and skill and exper­tise in automat­ing some­thing that per­haps itself is not the right thing, rather than look­ing at how we could do this dif­fer­ent­ly.”

Instead Luckin would love AI to ush­er in a future where the learn­er them­selves demon­strates what they’ve learnt. “My real dream is where the learn­ers them­selves say, ‘I think I should have this grade’ and bring out all the evi­dence built up over years to demon­strate why, show­ing they have under­stood them­selves well enough to pull that togeth­er, which would tell you so much about that indi­vid­ual,” she says.

How could this be scaled for some­thing as big, com­plex and life dic­tat­ing as uni­ver­si­ty admis­sions? “There will be ways of dig­i­tal­is­ing that,” says Luckin. “I imag­ine there would be some sort of dig­i­tal gate or point through which a stu­dent pass­es and they demon­strate their cre­den­tials, and over time you would be able to auto­mate that.

“We’re not miles away from this tech­ni­cal­ly; you’d need broad­band con­nec­tiv­i­ty every­where, but what is much hard­er is the human accep­tance of it. At what point do you feel you can say to par­ents, ‘OK, we’re phas­ing out the exams now’? They’d say, ‘Well how does my child get to the next stage?’ And I com­plete­ly under­stand that.”

The best approach would prob­a­bly involve a hybrid tran­si­tion peri­od until you reached a point where peo­ple felt con­fi­dent in the replace­ment as a “truer assess­ment” of an individual’s learn­ing and strengths, “cel­e­brat­ing human intel­li­gence and the non-cog­ni­tive skills which dif­fer­en­ti­ate us from machines”, Luckin con­cludes.