Video: George Dyson on Big Data’s Birth and Where It’s Headed


A look at the past suggests that a "crisis of imagination" may be holding back the tide of future tech innovation.

At the London 2012 Strata Conference, author and tech historian George Dyson talked about how humanity arrived at today’s world of high-speed and big data functionality, and where he sees things going. Dyson started out with his own idea of why big data exists. The cost of throwing something away, said Dyson, started to exceed the machine cost of keeping it. And so, big data was born.

In reviewing some of the other steps that led to today’s complex data storage and data handling technologies, Dyson surveyed some primitive computing blueprints and ideas about technology from centuries past, such as writing by German mathematician and philosopher G.W. Leibniz in 1679 on a binary computer, Charles Babbage’s "difference engine," and work done by Alfred Smee in the mid-1800s that pre-considered a type of machine that would work like a contemporary search engine. (Learn more about the earliest work in computer programming in The Pioneers of Computer Programming.)

From these ideas, which were profoundly ahead of their time, Dyson went on to mention Hollerith cards and other technologies that started to actually implement some of the ideas that had been previously been limited by a lack of availability of engineered hardware. With a major focus on Alan Turing’s groundbreaking work on data input and output, Dyson also references John von Neuman and the Manhattan Project as a collaboration that necessitated handling a lot of data in specific ways.

In addition, Dyson outlines some of the principles that other 20th century scientists used to accomplish so much, even with very limited amounts of memory. By contrast, Dyson called the modern computer system "inefficient" in that data is handled in a linear way, with a specific address for each piece of data. Citing Von Neuman’s philosophy of procedural coding and other ideas about technical advancement, Dyson called for a renovation of how developers use data, and a move toward principles like three-dimensional computing, pulse frequency coding, and the consideration of analog technologies that work much like the human brain.

This video provides a rich history of data storage and data handling. In doing so, it also acts as a powerful indictment of a "crisis of imagination" that may be holding back the tide of future tech innovation.


Related Terms

Justin Stoltzfus

Justin Stoltzfus is an independent blogger and business consultant assisting a range of businesses in developing media solutions for new campaigns and ongoing operations. He is a graduate of James Madison University.Stoltzfus spent several years as a staffer at the Intelligencer Journal in Lancaster, Penn., before the merger of the city’s two daily newspapers in 2007. He also reported for the twin weekly newspapers in the area, the Ephrata Review and the Lititz Record.More recently, he has cultivated connections with various companies as an independent consultant, writer and trainer, collecting bylines in print and Web publications, and establishing a reputation…