Dr Fazal Ali

Soon our chil­dren at school may speak to a holo­gram of a Carib Chief in the Cloud. To have a vir­tu­al con­ver­sa­tion to­mor­row with any­one about their life with­out them be­ing in the room is quite pos­si­ble. At the Illi­nois Holo­caust Mu­se­um, a 3-D pro­jec­tion of Aaron El­ster an­swers ques­tions at the Illi­nois Holo­caust Mu­se­um. But El­ster is else­where. Vis­i­tors in­ter­act with what ap­pears to be a holo­gram of the Holo­caust sur­vivor cre­at­ed us­ing Pep­per’s Ghost il­lu­sion tech­niques. The in­stal­la­tion is part of an oral his­to­ry project built around sur­vivors of the Jew­ish Holo­caust.

Dig­i­tal Hu­man­i­ties blur the stale de­part­men­tal bound­aries amongst his­to­ry, lan­guage and lin­guis­tics, mu­sic, mu­se­ums, ar­ti­fi­cial in­tel­li­gence, film, gam­ing, an­thro­pol­o­gy, and sound tech­nol­o­gy. It pro­vides nu­mer­ous op­por­tu­ni­ties for stu­dents work­ing with tex­tu­al, vi­su­al and mul­ti­me­dia con­tent as they ex­plore the role of dig­i­tal me­dia in mod­el­ling, cu­rat­ing, analysing and in­ter­pret­ing dig­i­tal rep­re­sen­ta­tions of cul­ture.

Tech­nol­o­gy now makes it pos­si­ble for First Peo­ple de­scen­dants of Carib and Arawak holo­caust sur­vivors to give au­then­tic tes­ti­mo­ny—now and far in­to the fu­ture—about the geno­cide of the Kali­na, Warao, Kalipuna, Nepuyo, Taino, and the Aru­a­ca peo­ples who lived here 7,000 years ago. Fan­ning smoke with a feath­er—the Shaman among the de­scen­dants of sur­vivors like Chief Ri­car­do Her­nan­dez makes it im­pos­si­ble for us to for­get their geno­cide.

Af­ter re­volt­ing in Are­na, many First Peo­ples on the run plunged to their death in the ocean. 84 were cap­tured. 61 shot dead. 21 hanged in St Joseph and their bod­ies dis­mem­bered and dis­played in the streets. The mas­sacre was in St Joseph, not Are­na.

The Holo­caust Mu­se­um project in Illi­nois took near­ly three years to com­plete. Sur­vivors were seat­ed in the mid­dle of a half dome stu­dio filled with 100 high-de­f­i­n­i­tion cam­eras and 6,000 lights to cap­ture them from mul­ti­ple an­gles. The ex­hib­it used voice recog­ni­tion tech­nol­o­gy and ma­chine learn­ing to ask sur­vivors’ ques­tions about their per­son­al or­deals and hear an­swers that be­come more rel­e­vant as the tech­nol­o­gy evolves and the sys­tem learns. Ma­chine learn­ing is the rea­son for the rapid im­prove­ment in the ca­pa­bil­i­ties of voice-ac­ti­vat­ed user in­ter­face sys­tems like Alexa. Da­ta and ma­chine learn­ing is the foun­da­tion of Alexa’s pow­er, and it’s on­ly get­ting stronger as its pop­u­lar­i­ty and the amount of da­ta it gath­ers in­crease. Every time Alexa makes a mis­take in in­ter­pret­ing a re­quest, that da­ta is used to make the sys­tem smarter the next time around.

The In­sti­tute for Cre­ative Tech­nolo­gies at the Uni­ver­si­ty of South­ern Cal­i­for­nia (USC) col­lab­o­rat­ed with the USC Shoah Foun­da­tion found­ed by Steven Spiel­berg on the Holo­caust project. The aim was to pre­serve Holo­caust and oth­er geno­cide sur­vivor tes­ti­monies. Be­fore a vis­i­tor in­ter­acts with the ‘holo­gram-hu­man’ the guest views a short video in which the sur­vivor nar­rates their per­son­al sto­ry and then quite un­ex­pect­ed­ly the im­age leans for­ward and says, ‘But I have so much more to tell you. Now I’d like you to ask me ques­tions.’

Like Ap­ple’s Siri tech­nol­o­gy, the voice recog­ni­tion sys­tem re­sponds by pick­ing up on key words from ques­tions. The holo­gram can an­swer thou­sands of ques­tions and it of­fers an al­most life­like con­ver­sa­tion­al op­por­tu­ni­ty. Heather Maio, the man­ag­ing di­rec­tor of ‘Con­science Dis­play’, a com­pa­ny that cre­ates re­al­is­tic com­bat scenes com­plete with di­a­logue and vi­su­als for train­ing drills for mil­i­tary per­son­nel, en­vi­sioned the in­ter­ac­tive sto­ry­telling ex­hi­bi­tion. To make the holo­gram, El­ster had to sit still in a chair for sev­er­al days un­der lights and cam­eras an­swer­ing more than 2,000 nu­anced ques­tions.

At the De­part­ment of Com­pu­ta­tion­al Lin­guis­tics in Cam­bridge sci­en­tists con­tend that to be ‘in­tel­li­gent’ a com­put­er sys­tem must be able to learn from ex­pe­ri­ence. That is, the ma­chine must be able to learn to process in­for­ma­tion and per­form tasks by con­sid­er­ing ex­am­ples with­out hav­ing been giv­en any task-spe­cif­ic rules. The ‘neu­rons’ that un­der­gird such a ma­chine are math­e­mat­i­cal func­tions con­tain­ing ‘weights’ and ‘bi­as­es’. When da­ta is in­putted the neu­rons per­form com­pu­ta­tions and then ap­ply the weights and the bi­as­es. To­day these sys­tems have many lay­ers and so they are re­ferred to as deep. It is here, in­side this deep black box, where al­go­rithms evolve and self-learn that sci­en­tists do not know how the sys­tem ar­rives at its re­sults.

The next steps hinge on get­ting such ma­chines to flag a mis­take and to learn from the er­ror as it trains it­self to use da­ta from mul­ti­ple da­ta banks and to take ad­van­tage of ad­vances in nat­ur­al lan­guage pro­cess­ing.

It has tak­en decades to un­der­stand nat­ur­al hu­man speech to the point where voice-ac­ti­vat­ed in­ter­faces like Alexa are suf­fi­cient­ly en­abled. While there is some ca­pa­bil­i­ty con­tained in the Echo cylin­der such as speak­ers, a mi­cro­phone and a small com­put­er, its re­al ca­pa­bil­i­ties oc­cur once it sends what­ev­er you have told Alexa to the cloud to be in­ter­pret­ed by ‘Alexa Voice Ser­vices’ which pars­es the record­ing in­to com­mands it un­der­stands. Then, the sys­tem sends the rel­e­vant out­put back to your de­vice.

Ama­zon al­lows and en­cour­ages ap­proved de­vel­op­ers free ac­cess to Alexa Voice Ser­vices so that they can cre­ate new Alexa skills to aug­ment the sys­tem’s skill-set just as Ap­ple did with the app store. As a re­sult of this open­ness, the list of skills that Alexa (cur­rent­ly over 30,000) can help with con­tin­ues to grow rapid­ly. Per­haps by 2030, our chil­dren may speak to the Chief in the Cloud.