Using VR and AR to make education engaging

The use of tech­nol­o­gy in teach­ing has become the norm, but some busi­ness­es and uni­ver­si­ties have tak­en it to the next lev­el through vir­tu­al and aug­ment­ed real­i­ty.

VR and AR, once best known for fun appli­ca­tions like the Pokémon Go app and Snapchat fil­ters, are increas­ing­ly being used along­side tra­di­tion­al train­ing. A recent study of 151 adults by Neu­ro-Insight, Mind­share UK and Zap­par found AR had a sig­nif­i­cant­ly pos­i­tive impact on both atten­tion span and infor­ma­tion reten­tion.

There are lim­i­ta­tions though, cost, access and user will­ing­ness to name a few. But as it becomes more main­stream, immer­sive learn­ing has huge poten­tial to boost skills and improve edu­ca­tion­al expe­ri­ences.

Helping surgeons practise operations

VR is being used to train sur­geons before they even step into the oper­at­ing the­atre. Dr Alex Aquili­na, an orthopaedic sur­geon based in Bris­tol Children’s Hos­pi­tal has been using Virti’s VR tech to cre­ate 360-degree sur­gi­cal videos. Cam­eras are set up in the­atres, with patient and staff con­sent, and oper­a­tions filmed from all angles. Patients are anonymised and trainee sur­geons watch on-demand through a head­set or via Zoom train­ing ses­sions.

This process does not sub­sti­tute real-life train­ing but enhances it, says Aquili­na. It also enables junior doc­tors to learn with­out need­ing to come into hos­pi­tal dur­ing the ongo­ing pan­dem­ic, reduc­ing infec­tion risk to them­selves and patients. 

“Your in-surgery learn­ing oppor­tu­ni­ties, when you’re actu­al­ly hold­ing the knife, are real­ly pre­cious,” he says. “The more informed you are before­hand, the more you’ll ben­e­fit.” 

The films teach trainees about the whole process, from set­ting up equip­ment and assem­bling joint implants to brief­ing staff and posi­tion­ing the patient cor­rect­ly. For all these, video is a more effec­tive for­mat than read­ing man­u­als or look­ing at pho­tos, says Aquili­na.

In future, he plans to embed AR ele­ments, such as close-up footage of laparo­scop­ic (key­hole) pro­ce­dures, info­graph­ics and ques­tions, where the user would be led down dif­fer­ent filmed routes based on their answers. 

There are some tech­ni­cal chal­lenges, such as mit­i­gat­ing cam­era glare from the theatre’s bright lights, and eth­i­cal con­cerns around patient and staff con­sent. But Aquili­na believes VR and AR have huge poten­tial to empow­er patients; some­one with dia­betes who has wit­nessed a foot ampu­ta­tion may be more like­ly to man­age their con­di­tion well, for exam­ple.

“It helps demys­ti­fy the oper­at­ing the­atre,” he says. “I’m hop­ing increas­ing acces­si­bil­i­ty to what sur­geons do can enhance recov­ery and help pre­vent future dis­ease.”

Teaching children about nature

AR apps are being used to engage school­child­ren and bring top­ics to life. The Muse­um Alive app, devel­oped by Alche­my Immer­sive in part­ner­ship with Sir David Atten­bor­ough, allows users to place extinct ani­mals and their nat­ur­al habi­tats in the real world by acti­vat­ing the user’s smart­phone cam­era to “project” ani­ma­tions into their liv­ing room.

This is cou­pled with oth­er learn­ing resources, such as voiceovers, sound­scapes, maps, writ­ten infor­ma­tion and pre-record­ed films. “Apps like this give chil­dren agency with­in a sto­ry,” says Elliot Graves, cre­ative and tech­ni­cal lead at Alche­my Immer­sive. “When chil­dren are active­ly par­tic­i­pat­ing, they are far more like­ly to retain infor­ma­tion. Once this tech is worked into the nation­al cur­ricu­lum, it could have a huge impact.” 

Alche­my Immer­sive worked with Durham Uni­ver­si­ty to con­duct school work­shops to test the impact of immer­sive tech. Dr Noam Leshem, asso­ciate pro­fes­sor of geog­ra­phy, set up 11 to 14 year olds with a VR project called Por­traits of No-Man’s Land, where they looked at redefin­ing areas of con­flict in Colom­bia, Cyprus and France. 

Leshem found VR should be used along­side oth­er resources, like dis­cus­sion groups and work­sheets, to be tru­ly effec­tive. “Stu­dents were curi­ous and excit­ed,” he says. “But we don’t know what will hap­pen when the nov­el­ty of VR wears off. Grab­bing their atten­tion is the first step, ensur­ing VR stream­lines into the class­room will be hard­er.”  

The tech­ni­cal skills gap between teach­ers and stu­dents will also need to be addressed before it makes its way into main­stream edu­ca­tion. “VR head­sets still appear intim­i­dat­ing to some teach­ers, even though most stu­dents find them intu­itive,” says Leshem. “Price has dropped but bud­gets are tight; con­vinc­ing schools this is a worth­while invest­ment will remain a hur­dle over the next few years.” 

Training construction workers

The Con­struc­tion Indus­try Train­ing Board is invest­ing £3 mil­lion in sev­er­al immer­sive tech projects that will help improve indus­try train­ing. The biggest is Project Con­vert, which deliv­ers four dif­fer­ent types of train­ing to six UK-wide col­leges: VR build­ing, where stu­dents con­struct from scratch; VR scaf­fold­ing and work­ing at heights; VR drone train­ing, for site sur­veyance; and AR wood­work­ing and paint-spray­ing, which incor­po­rates ele­ments like a vibrat­ing mag­ne­tised table to repli­cate the feel­ing of phys­i­cal pres­sure dur­ing car­pen­try.

Gareth Evans, cen­tre man­ag­er at Con­struc­tion Wales Inno­va­tion Cen­tre (CWIC), which is lead­ing on Project Con­vert, says stu­dents are not nor­mal­ly exposed to these skills before employ­ment due to cost or health and safe­ty con­cerns. “Using VR first means that when they nav­i­gate real drones, they’re not crash­ing them into trees,” he says. “When they move onto phys­i­cal wood­work­ing machines, they’re not risk­ing los­ing fin­gers. It helps them learn from their mis­takes.”

The tech­nol­o­gy is being used by stu­dents at the Uni­ver­si­ty of Wales Trin­i­ty Saint David, where CWIC is based. Along­side help­ing them avoid acci­dents or cost­ly build­ing alter­ations, the range of train­ing opens their eyes to dif­fer­ent jobs, says tutor and land­scape archi­tect Sheila Holmes, and allows her to assess stu­dents more accu­rate­ly. “The soft­ware mea­sures the stu­dents’ deci­sions and knowl­edge. I can view their progress in real time and talk to them about their mate­r­i­al selec­tions and process­es,” she says.

For a minor­i­ty prone to ver­ti­go or claus­tro­pho­bia, for exam­ple, VR expe­ri­ences like work­ing at heights or in con­fined spaces might not be appro­pri­ate, says Evans. But the main hur­dle is in encour­ag­ing less tech-savvy edu­ca­tors to try it. “Some staff can be ner­vous about VR,” he says. “The biggest chal­lenge isn’t so much the tech itself, it’s chang­ing people’s per­cep­tions.”

Exploring ancient civilisations

The Giza Project is a dig­i­tal archae­ol­o­gy ini­tia­tive based at Har­vard Uni­ver­si­ty, which curates records of the Giza Pyra­mids in ancient Egypt. Anthro­pol­o­gy and Egyp­tol­ogy stu­dents can explore the 5,000-year-old civil­i­sa­tion through VR sim­u­la­tions, either via a head­set, a uni­ver­si­ty lab or an online video ver­sion. Dur­ing lock­down, tutors have shared 360-degree videos over Zoom.

The stu­dents can under­take vir­tu­al con­ser­va­tion work such as rebuild­ing archi­tec­ture or repaint­ing tombs. They can build archi­tec­ture from scratch using evi­dence such as stud­ies of par­al­lel build­ings. Avatars can be placed in scenes to give a sense of scale and con­text. The project both helps stu­dents visu­alise how ancient Egypt would have looked and offers them oppor­tu­ni­ties to con­tribute to dig­i­tal research with their own builds. 

It also chal­lenges stu­dents to con­sid­er eth­i­cal issues, such as the accu­ra­cy of por­tray­als of ancient Egypt in mod­ern media like film and gam­ing, encour­ag­ing them to be more ana­lyt­i­cal. “The VR helps them dis­tin­guish between the­o­ry and archae­o­log­i­cal­ly attest­ed designs,” says Peter Der Manuelian, pro­fes­sor of Egyp­tol­ogy at Har­vard. “Ulti­mate­ly, it helps them look at Egypt­ian civil­i­sa­tion through the lens of its most impor­tant site.” 

He hopes the immer­sion ele­ment will spark under­grad­u­ates’ fas­ci­na­tion. “We’re hop­ing for a renewed inter­est in the human past and to encour­age some stu­dents to major in archae­ol­o­gy,” he says. “The aim is to edu­cate stu­dents on sources, doc­u­men­ta­tion and accu­ra­cy, but also to make edu­ca­tion fun.”