How tech can bridge the global digital language divide

It’s easy to think that any­one with an inter­net con­nec­tion can access the infor­ma­tion, net­work­ing plat­forms and con­ve­nience-enhanc­ing tech found online. Unfor­tu­nate­ly, not. Of the world’s 7,151 lan­guages, more than half don’t have any dig­i­tal foot­print. This phe­nom­e­non is described as the dig­i­tal lan­guage divide and unless it is addressed, could accel­er­ate the extinc­tion of thou­sands of lan­guages. 

World lan­guage resource Eth­no­logue report­ed that about 40% of lan­guages are now endan­gered, often with few­er than 1,000 speak­ers remain­ing. Mean­while, just 23 lan­guages account for more than half the world’s pop­u­la­tion. If a lan­guage falls out of use, so does knowl­edge of the rich cul­tures and his­to­ries they describe. 

Tech start­up Deriva­tion aims to address the prob­lem, using lan­guage insight solu­tions and pio­neer­ing AI. Stephen Jones, Deriva­tion, act­ing CEO, explains: “Our solu­tions include mea­sur­ing every liv­ing lan­guage to assess their progress towards increased and sus­tain­able rel­e­vance in the dig­i­tal age. Lin­guis­tics informs us that lan­guages thrive on a ‘use it or lose it’ basis – if no one is writ­ing, read­ing, sign­ing or speak­ing a lan­guage, then over gen­er­a­tions its usage declines and even­tu­al­ly it sim­ply falls out of use. As the world relies more on tech­nol­o­gy, that use-or-lose equa­tion is shift­ing into the dig­i­tal are­na.” 

A lack of dig­i­tal infra­struc­ture (such as key­boards, fonts, oper­at­ing sys­tems) sup­port­ing a lan­guage usu­al­ly indi­cates real-world vul­ner­a­bil­i­ties are affect­ing a lan­guage and its native speak­ers. Among the most vul­ner­a­ble both online and in the real world are non-writ­ten lan­guages – those used in areas that have low lit­er­a­cy rates, insuf­fi­cient inter­net or mobile phone cov­er­age or areas with slow eco­nom­ic growth. Deriva­tion research sug­gests that lan­guages which are used in an offi­cial capac­i­ty (such as in gov­ern­ment, legal, eco­nom­ic, health and edu­ca­tion sys­tems) are far more like­ly to be dig­i­tal­ly sup­port­ed than non-insti­tu­tion­al lan­guages. 

Jones notes: “The dig­i­tal lan­guage divide is felt less in Europe, where there are few­er region­al-spe­cif­ic lan­guages, and seen more in Africa and Asia, where is gen­er­al­ly a high­er con­cen­tra­tion of indige­nous lan­guages. 

“If you speak an ‘offi­cial lan­guage’, par­tic­u­lar­ly one in Europe, you are much more like­ly to find dig­i­tal options avail­able in your moth­er tongue. If, though, your first lan­guage is an indige­nous one, and par­tic­u­lar­ly if it is one that doesn’t have offi­cial recog­ni­tion in Asia and Africa, you are more like­ly to face dig­i­tal lan­guage exclu­sion and forced to oper­ate in a dif­fer­ent lan­guage.”

The exclu­sion of these lan­guages from the dig­i­tal sphere is both a human­i­tar­i­an and an eco­nom­ic issue, as native speak­ers of vul­ner­a­ble lan­guages have lit­tle or no access to edu­ca­tion­al mate­ri­als and social inter­ac­tion, while busi­ness­es lose out on mar­ket­ing oppor­tu­ni­ties and valu­able data from with­in these com­mu­ni­ties.

In his co-authored research paper, Sus­tain­ing Lan­guage Use, lin­guist Dr Gary Simons says the preser­va­tion of indige­nous lan­guage is also a mat­ter of pub­lic health and human rights: “Mem­bers of a com­mu­ni­ty which is in the process of los­ing its lan­guage and cul­ture expe­ri­ence sig­nif­i­cant amounts of dis­rup­tion and stress in all areas of life. A child grow­ing up in a com­mu­ni­ty which is viewed with dis­dain devel­ops a self-image that reflects that expe­ri­ence. 

“As social norms are aban­doned, they fre­quent­ly are not replaced all at once or with ade­quate equiv­a­lents. This can lead to social ten­sions, divi­sions in the com­mu­ni­ty, dis­rup­tive and harm­ful pat­terns of behav­iour, and even vio­lence. There is evi­dence that com­mu­ni­ties expe­ri­enc­ing such a tran­si­tion may have ele­vat­ed lev­els of alco­holism, drug addic­tion, HIV/AIDS, and sui­cide.” He goes on to cite the Unit­ed Nations Dec­la­ra­tion on the Rights of Indige­nous Peo­ples (Arti­cle 13.1), which states: “Indige­nous peo­ples have the right to revi­talise, use, devel­op and trans­mit to future gen­er­a­tions their his­to­ries, lan­guages, oral tra­di­tions, philoso­phies, writ­ing sys­tems and lit­er­a­tures, and to des­ig­nate and retain their own names for com­mu­ni­ties, places and per­sons.”

Aside from the human-cen­tric argu­ments, Simons notes the impor­tance of indige­nous lan­guage preser­va­tion in eco­log­i­cal sus­tain­abil­i­ty, in that “many minor­i­ty lan­guage com­mu­ni­ties pos­sess high­ly devel­oped bod­ies of knowl­edge about their phys­i­cal envi­ron­ment and have elab­o­rat­ed tech­nolo­gies to adapt and make use of their envi­ron­ment that are trans­mit­ted through equal­ly high­ly devel­oped lin­guis­tic forms in their lan­guages. With the loss of those lan­guages and cul­tures entire areas of human knowl­edge are also at risk and at the same time bio­log­i­cal diver­si­ty is also threat­ened.”

For­tu­nate­ly, the emerg­ing field of lan­guage ana­lyt­ics – a com­bi­na­tion of busi­ness intel­li­gence, big data and lin­guis­tics – is used to inform devel­op­ing AI and machine learn­ing tools that can work to bridge the gap between estab­lished lan­guages and those being left behind. 

SIL is a non-prof­it organ­i­sa­tion involved in 1,600 active lan­guage projects in 98 coun­tries. SIL data sci­en­tist Daniel White­nack says that although the rapid inte­gra­tion of AI into exist­ing dig­i­tal sys­tems has “the poten­tial to fur­ther mar­gin­alise already-mar­gin­alised lan­guage com­mu­ni­ties, arti­fi­cial intel­li­gence cre­ates amaz­ing new pos­si­bil­i­ties for much of the world”. Specif­i­cal­ly, the advance­ment of nat­ur­al lan­guage pro­cess­ing (NLP), which “has enabled researchers and engi­neers to deci­pher long-lost lan­guages, trans­late speech in one lan­guage to speech in anoth­er lan­guage direct­ly with­out con­vert­ing to text, gen­er­ate long-form text that adapts to the style and con­tent of human prompts, and trans­late between lan­guage pairs nev­er seen explic­it­ly by com­put­er sys­tems”.

Because they relied heav­i­ly on “man­u­al­ly craft­ed lin­guis­tic resources”, ear­li­er NLP mod­els large­ly ignored vul­ner­a­ble indige­nous lan­guages but now, White­nack says, “large tech com­pa­nies, aca­d­e­m­ic insti­tu­tions, and grass­roots organ­i­sa­tions are begin­ning to realise that NLP meth­ods could and should be extend­ed to minor­i­ty lan­guages. For a tech­nol­o­gy like machine trans­la­tion, this shift is evi­dent from Facebook’s recent request for pro­pos­als on neur­al machine trans­la­tion for low resource lan­guages, Google’s focus on build­ing trans­la­tions sys­tems for “the next thou­sand lan­guages”, and the large, mul­ti-insti­tute Masakhane grass­roots organisation’s research efforts that focus on machine trans­la­tion for African lan­guages.” 

The new atten­tion placed on AI and NLP for indige­nous lan­guages will enable local com­mu­ni­ty mem­bers, organ­i­sa­tions, and non-prof­its to build momen­tum towards new, mul­ti­lin­gual appli­ca­tions that give minor­i­ty lan­guage speak­ers a chance to bring their voic­es into the glob­al con­ver­sa­tion and shift­ing econ­o­my. This will shift the atten­tion, cur­rent­ly direct­ed towards Eng­lish and Man­darin, to a more democ­ra­tised focus and par­tic­i­pa­tion across lan­guage com­mu­ni­ties.