How digital technology can reveal people’s untold stories

Professor Tim Hitchcock has pioneered new ways to explore and interrogate historical archives with digital technology.

“The process has contributed to a change in how people think about themselves: Their ‘right’ to be remembered.” 

Tim Hitchcock, Professor of Digital History at the University of Sussex, is reflecting on one of the many positive outcomes of his involvement in digitising historical records to create online resources. 

While there’s an abundance of history books and accessible data about the ruling elite, Hitchcock’s focus has been on the untold lives of the British working classes – “history from the below, in the British Marxist tradition,” he asserts. 

After more than 20 years of exploring the most effective ways of bringing together historical records and digital technology, Hitchcock and his co-researchers at Sheffield, Liverpool, Oxford and the University of Tasmania have created The Digital Panopticon, a searchable database of 400 million records of more than 250,000 UK people who were accused of crime between 1780 and 1925, many of whom were deported to penal colonies in Australia. 

The database has a wide range of search fields – such as year and place of birth, criminal record, and exact physical descriptions - making it possible to compare the impact of transportation and imprisonment on reoffending, desistence, family lives and health. 

Challenging perceptions of the past

“Historians have tended to be historians of administration, and what this does is change that facility to allow you to write histories of people,” says Hitchcock. “Collectively and globally we have created an alternative way of interrogating archives and changing the focus.” 

This rich source of data not only gives an insight into the lives of individuals, it challenges many beliefs and perceptions about the past. 

For example, analysis of these records has shown that less than half those sentenced to death were executed, while only two thirds of those assigned for transportation to Australia left the UK’s shores (an effective punishment, it seems, as they were less likely to re-offend than those who were imprisoned). 

The journey for Hitchcock began in 1998 when he was one of the first to use XML, then a newly created digital publishing standard that made documents readable and, most importantly, searchable via embedded ‘tags’, on a large scale.  

His first project was to digitise the trial transcripts of the Old Bailey criminal courts in London from 1674 to 1913 to create Old Bailey Proceedings Online, a 127-million-word database that would have taken researchers years to analyse using traditional paper-based methods.  

Information gleaned from this, and supplemented by other archives, allowed Hitchcock to create London Lives, an online resource (and a book) presenting a historical picture of London’s paupers and criminal classes in the 18^th century. 

The success of this led to collaborations with other archives to expand the project. Now The Digital Panopticon, which was launched in 2017, includes more than 50 other datasets that help to create a detailed picture of criminals deported to Australia. 

Hitchcock says: “We know how much money they had when they arrived, and what they looked like. We even know about their tattoos. In 1902, when Buffalo Bill’s Wild West show came to the UK, every place they stopped they had a tent with a tattooist. So suddenly you get hundreds of people with Buffalo Bill tattoos on their arms.  

“It’s a kind of cultural change that nobody knew about. The Digital Panopticon has, by happenstance, allowed us to understand the culture of a particular group in a new way.” 

New ways of identifying people

He adds: “From the criminal registers in the 1790s, which were all about physical descriptions, right through to the use of fingerprints in the 1890s and now DNA identification, we’re discovering and managing new ways of identifying people.” 

Being able to piece together individual, untold personal stories with this wealth of data is one of the most rewarding aspects of the project for professional researchers – and those curious about the lives of their ancestors. 

The website features some 120 profiles of people, such as Lydia Lloyd, a habitual recidivist who, as the records show, committed multiple thefts and burglaries from the age of 16 until her death at the age of 50 in 1893.  

Born in poverty, with prostitution as her main source of income, Lloyd’s crimes carried heavier punishments the older she got. At her final court appearance, she was sent down for ten years for stealing a shawl when drunk; an unthinkably harsh sentence by today’s standards. 

And equal ‘right’ to be remembered

With online visitors from more than 210 countries, The Digital Panopticon has been a proven success in terms of reach and interest. It has also spurred Hitchcock into thinking more about the politics of digitisation and how it is privileges the Global North. 

“I’m working on an idea I describe as ‘the right to be remembered’,” he says. “The average working-class person in 1840s Manchester probably has 40 or 50 items in the archive related to them. The average person in 1840s Bengal probably has one item. The same person in central Africa has none. So if everybody has an equal right to be remembered in cultural terms, how do we deal with the fact that the archive itself is fundamentally skewed?” 

He would like to create an algorithm that takes into account the weight of every archive globally. “This would give a measure of the relationship between archival rarity and the individual,” he says. “A single document about sub-Saharan Africa in 1840 has to bear the intellectual and cultural weight of perhaps a million people. The same document in 1840s Manchester has to bear the weight of just one person. 

“That then allows you to ask how the history we are building and writing is shaped not just by the archive that it’s built from, but by the underlying populations that have a right to be represented in it.”

---