Information theory and cool museums

Imagine some text, sound, image, or video. Imagine transporting that media to another place or time. Encode the media as a sequence of symbols called a message. Set aside any notion of meaning the message represents. Send the message through a channel such as a telephone cable or hard drive. Recover the media by decoding the message.

How to measure the amount of information in the message? Andrei Nikolaevich Kolmogorov describes three approaches in his paper Three Approaches to the Quantitative Definition of Information. A combinatorial approach counts how many symbols the message contains. A probabilistic approach called Shannon Entropy weights more heavily symbols which are unlikely to occur given what symbols have appeared before. An algorithmic approach called Kolmogorov Complexity measures the length of the shortest computer program that generates the message.

How to make the message as small as possible for efficiency? Most messages contain redundancy or patterns. Compression algorithms remove redundancy or patterns. Lossless compression preserves the message exactly, as one might require for text. Lossy compression shrinks the message more but also distorts it somewhat, as one might tolerate for sound or images.

How to protect the message from corruption due to noise in the channel that may randomly flip some of the bits? Ironically, now add redundancy; but add it in clever ways that allow for the detection and even correction of errors.

A history, a theory, a flood

In addition to the textbook, the Information Theory syllabus listed a less technical book called The Information by James Gleick. It tells the story of humanity's accelerating dance with information with countless anecdotes from ancient history to the present day. Each advance brought profound societal changes, affecting lifestyle and thinking in ways people from before the technology could not understand.

These are some of my favorite stories.

• Two-pitched drums formed an unparalleled communications network between African villages for centuries. Drummers pounded out messages with drumbeats, adding lots of extra words to help listeners better sort through the unavoidable ambiguity in the system.

• Writing emerged from ancient spoken languages and made it possible for information to pass unaltered through space and time.

• Landowners in the American West created a large improvised telephone network through the barbed wire of their fences during the early days of telephony.

• Robert Cawdrey compiled the first English dictionary. Early dictionaries served to document the words people were using, versus opining which words belonged in the English language and which did not. He also invented alphabetical ordering.

• Before people harnessed electricity, Charles Babbage designed a mechanical general purpose computer. His collaborator Ada Lovelace worked out how to solve problems on it. Even though her programs could never run, she is widely considered the very first computer programmer.

• Claude Shannon and other brilliant thinkers formalized information theory. They laid the theoretical foundations for modern electronic communication.

After the class I visited two related museums with my parents. Spending time with historical artifacts from the worlds of printing and telephony brought to life for me a large swath of all that history.

UVM campus at dusk

Black ghostlike statues glide across the courtyard just before sunset, around when I stopped working and started graduate school. In warmer weather, students hang cozy hammocks between these trees.

Museum of Printing

The Museum of Printing is in Haverhill, Massachusetts. It has hands-on exhibits and knowledgeable staff happy to answer questions and offer guided tours. From its About Page,

The Museum of Printing is dedicated to preserving the rich history of the graphic arts, printing and typesetting technology, and printing craftsmanship. In addition to many special collections and small exhibits, the Museum contains hundreds of antique printing, typesetting, and bindery machines, as well as a library of books and printing-related documents.

Someone even walked me through the process of making a cast of my name from a light molten metal in the same way an operator would prepare a page for mass production. After printing enough copies, they could melt the cast down again and reuse the metal for another page, but I got to keep my name!

Ancient chinese printing technology

Symbols were selected from this wheel and used in something like a printing press. This technology predated the European printing press, but it was tightly controlled by the government and never spread widely.

Bible page from 1830

Bibles were an early priority for the printing press during the Protestant revolution and beyond. They played an important role in European and American society, and continue to play an important role for many today.

Colored illustration from a Bible

Many painstakingly colored illustrations from very old Bibles were on display here. I wish I understood the significance of the details in these illustrations better than I do.

Early color printer

This was one of the earlier electronic machines that could print in color at scale. The more modern machines displayed nearby illustrate the miniaturization this technology underwent.

Old computers

These memorable artifacts from my living past mark the beginning of an era of alternatives to printing text. My fascination with them also set the trajectory of my career path.

Ornate printing press

This massive ornate printing press near the entryway had a golden American Eagle at the top. It was among the most visually striking printing presses on display.

Printing my name

In a hands-on demo, we filled this mold with molten metal to create a stamp for printing my name. Whole pages of text were created this way, and the stamps recycled by melting them again.

Typewriters

My parents posing near a wall of typewriters similar to the one I used in childhood to type papers for school. I remember the smell of the ink, the feel of the keys, and fixing errors with the eraser of a pencil.

New Hampshire Telephone Museum

The New Hampshire Telephone Museum is in Warner, New Hampshire. I recommend visiting the museum to anyone interested learning about the history of telegraphs and telephones or seeing firsthand some of the artifacts on which our communications infrastructure is built, past and present. It also has several hands-on demos: telegraphs, manual switchboards, automated switchboards, and early coin operated telephones. There is also a large room dedicated to the life of Alexander Bell.

Telegraph

Before telephones, there were telegraphs. Operators used a key to tap out messages. Visitors to the museum could tap out messages of their own between two such telegraphs.

Older telephone

The museum had many telephones on display, from the oldest originals to the more modern, in a series of progressions.

Manual switchboard

An overwhelmed overworked operator would route and manage calls by plugging cables into this switchboard. The switchboard had indicators to show when lines were no longer in use or the paid time had expired.

Telephone cable cross section

Just imagine the many conversations happening side by side along these wires, and the sheer weight of it all.

Undersea telephone cable

To connect continents, telephone cables had to be routed under the ocean. Many extra layers helped the cables survive and operate in the harsh undersea environment.

Automated switchboard

Automated mechanical switchboards eventually replaced manual ones. I never fully understood how it worked, but this switchboard could route calls between two actual phones in the exhibit!

Newer telephone

This more modern telephone is similar to those I used growing up. Users had to spin the dial for each digit in the number they wanted to call, and when they released it, the phone sent a different number of clicks over the line.

Into the future

Technological progress comes hard. It spurs economic development, forces change, raises questions of justice, and creates barriers of understanding between people who adapt and those who do not. The people who lived before the telegraph could likely understand very little of my life. Barring global catastrophe, technological progress will only continue. I will likely understand very little of the lives of the people who come next.