The Database’s “Moon Landing"

Humans have collected information for thousands of years. Whether it be through cave paintings, parish registers or complex archival systems, we’ve always found ways to gather and store information. The Sumerians in Mesopotamia kept medical records on stone tablets while the ancient Egyptians amassed vast amounts of data on papyrus scrolls.

These were the databases of the past.

But what do Harry Potter, Agatha Christie’s best crime novels and the digital databases of today have in common? Well, all stories feature a key turning point on a train.

To store and retrieve data manually is time-consuming and tedious. The German statistician Herman Hollerith likely thought the same when he began working for the American census bureau in 1890.

The US population had grown substantially, and it was time for a census. Hollerith drew inspiration from how train conductors punched tickets to record details about passengers like gender and age—and he developed a similar system for the census.

Hollerith’s electric tabulation machine functioned in many ways like an early, advanced spreadsheet, where information about citizens could be automatically sorted—saving the census bureau lots of time and millions of dollars.

Many patents and inventions later, Hollerith founded the company that would eventually become “International Business Machines”. Or IBM as we know it today.

We can thank IBM for hard drives, magnetic tape, RAM and many of the components that in the 60s, 70s and 80s made computers into fantastically powerful tools. And we can thank Hollerith and his electric tabulation machine for the art of processing data: A craft that forms the basis for computers, data processing and, in fact, today’s information society.

The tabulation machine is perhaps one of the earliest examples of a database as we know it. For not only did it function as a tool to collect data. There were also mechanisms that made sorting and organisation more efficient. And with the help of some additional equipment, you could filter rows or lines and group certain datasets.

The post-war electronic revolutions led to the development of computers with increasingly stronger processing power. And eventually also processors, memory, networks and storage space.

The first digital databases actually worked largely in the same way as the first tabulation machines. They consisted of “flat” files, that is, sequential, uniform data in lines and rows.

To identify the data, therefore, the machines had to start from the beginning of the files and search sequentially. This was a slow and tedious process, especially with large amounts of data.

Think of it this way: Imagine you had to search through a book without a table of contents, chapters or bookmarks, looking for a specific paragraph about, say, “random number generators." As it happens, the paragraph is found on page 768. It would take quite some time to locate it if you had to start from the beginning.

Quite soon, however, better solutions were found.

The space race, in particular, sped up the rapid development of computers toward the end of the 1960s. When President Kennedy urged American industries to create faster and better digital systems to achieve moon landings, IBM responded with improved databases.

These databases were no longer “books”, but network models linked together with nodes, pointers and hierarchical structures (compare it to identifying information in a family tree, instead of in a book).

And from there, Armstrong took his historic small step for mankind...

Imagine if we could locate data based on relevant tags or defined relationships, rather than sequence or hierarchy?

This was the idea that led to the creation of searchable databases, or what we now know as relational databases. We’ll delve deeper into the specifics of a relational database in the upcoming section.