when did physics become an information science? (the birth of the bit)

image by image editor under cc license

in 2004 stephen hawking paid up on a bet he had with john preskill about whether or not information could escape a black hole. at the time my understanding of physics was very rudimentary (and only slightly less so today), and i did not understand at all about why they were talking about information in the same way that one would talk about particles.

it's clear on why particles and even energy can't escape a black hole since the density of the hole is so great that nothing could have enough energy to escape, but why is it important that information can't escape a black hole?

information is a concept that relates to human organization of the world around us. why, i wondered, is it considered a physics matter if information cannot escape a black hole? when and why did physics start concerning itself with abstract concepts like information instead of keeping focused on the physcial natural world around us?

it's something that's been in the back of my mind ever since. every once in a while, as i learn more about different aspects of physics, i see more reference to information in different areas of physics where it's being treated like particles (like how the clearly faster-than-light "communication" of quantum entanglement does not allow for information to go faster than the speed-of-light, thus the universal speed limit is not violated), but the connection of how or why information should be treated like matter and energy was never made clear.

i knew that einstein showed that energy and particles can be viewed as the same thing (his famous  E=mc² equation), but when did information enter that equation?

i recently got an insight into that question in an online course i'm taking from the sante fe institute on Complexity.

one of the lectures explains how energy is represented as information which then makes information something that should apply to our understanding of physics. 


first, you might want to watch the introduction lecture (10 minutes), then watch the lecture on maxwell's demon (9 minutes).

basically maxwell (circa 1875) conceived of a thought experiment that would demonstrate a violation of the second law of thermodynamics (which is summarized as "in an isolated system, entropy always increases"). maxwell imagined a creature (he used the term "homunculous" which was incorrectly translated as "demon") that could sort slow and fast moving molecules in a box from each other by simply watching the molecules and opening a door to let the fast ones travel to one side while the slow ones collected to the other side. when that demon performed such a task, it would take less energy to sort the molecules than there would be entropy decreased (since the system uses the momentum of the molecules instead of requiring extra work to sort them).

maxwell was convinced that this meant the second law of thermodynamics was not actually a law, but rather a statistical certainty and his thought experiment spawned a fierce debate in physics circles.

but then a few years later (circa 1920), szilard pointed out that the contraption could not work if the entity wasn't aware of the fast and slow moving particles. processing that information in the entities head was using energy, sufficient enough to restore the second law of thermodynamics.

szilard's contribution to the debate introduced the consideration of information being subject to all the laws of physics just like energy and matter.

as a bonus, szilard used the term bit when discussing the discrete units of information that would work in the demon's head which claude shannon then co-opted to name the binary digit to represent the smallest unit of data in his seminal A Mathematical Theory of Communication.

and thus the bit was born!