Entropy and Information Theory

 For the last couple of months, I have spent a long time researching entropy. It would be an understatement to say that there is absolutely no law that governs our existence as the second law of thermodynamics. The second law, however, only hints towards entropy. The third law of thermodynamics defines it. I hope you are aware of what entropy is. Sparingly, this is how we define entropy:

Entropy is the total number of possible arrangements of particles in a system.

As per the second law, the total entropy of an isolated system can never decrease.


There is a striking similarity between entropy and information. If you are a computer scientist, you might know about Maxwell demon's paradox which was solved by information theory in the late 20th century. The solution claimed that the thought experiment for the paradox violated the second law of thermodynamics and hence not applicable. The paradox is still being discussed and debated even today. I am not going to talk about the paradox or its solution. In this write-up, I am going to talk about the implications of it.

We are in the age of information. We have access to almost all the information that we need at the tip of our fingers. We can also create information using pocket-based computing devices. A phone camera can capture a high-quality picture in a raw, uncompressed format that can be easily processed later. The size of each picture can be anywhere between 1 MB to 48 MB depending upon the camera settings. People often click 5-6 pictures of the same viewport (i.e. frame) to make sure that at least one of them is satisfactory. Let's say the average size of a picture is about 4 MB and someone clicked 5 pictures of the same view port, we would have 20 MB of disk space storing the same information - the information here is how the object in the viewport was visible to the human eyes from a particular angle at a particular time. Let's say the images are uploaded to the cloud. The machines in the cloud will create at least 7 geo-replicas of the image. A total of 140 MBs of data will be written to magnetic or solid-state drives. The data represents the information. The data is like a snapshot of something in its historical existence. The disks stores the data in binary format - bits of 0s and 1s.

Let's say the user now wants to delete the images permanently from the cloud. The machines in the cloud will go to all the replications and remove the images. When the images are removed, the bits corresponding to the images will be reset to 0. You may think that the images and hence the information is lost now. Not exactly! Information is never lost. The information is actually converted to a format that is no longer interpretable by humans or computers. Let me elaborate!

The disks use spindles to write information on to them. Spindles interact with tiny molecular structures to modify them. The molecular structures have two possible states representing the binaries. When data is being erased, the molecular structure needs to be changed. At the atomic level, the electrons need to change their quantum level and hence release EM waves (assuming that their quantum level decreases upon reset). EM waves carry energy as heat. A release in heat implies an increase in entropy. If we knew the amount of energy released by each electron, we can restore the state of electrons and hence the molecular structure and hence the entire information as well. But for now, the information is present in the EM waves that were released. The EM waves travel throughout the universe carrying the information. Every piece of information stays in the universe forever in some scrambled form that we can never interpret. This also implies that the total information of the universe can never decrease.  I hope that I have provided enough hints for the conclusion that I am going to draw here:

Entropy is information. Information is entropy.

The more information we create, the more entropy we infer. If we could represent the particles of the universe by an entropy state diagram and build a machine that would take an entropy state as input and build the universe for that state as output, we could travel to any point in time. We can have access to all the information of the world. But such machines violate the second law of thermodynamics and hence they can never exist.

The goal of the universe is to expand and reach the maximum entropic state in which every particle is placed billions of miles away from each other and has the lowest energy state (the perfect equilibrium). In short, the goal of the universe is to get destroyed. All naturally occurring processes are in favor of increasing entropy. The most natural processes are those which can increase the entropy of the universe by the maximum amount. All metabolic processes of living beings are natural because they increase entropy faster. Our emotions are based on chemicals interacting in such a way that maximum possible entropy is released.

In fact, I strongly believe that the birth of computers and the information age was because of the fact that it has the maximum potential to release tons of entropy through the use of computing devices. Data is the raw form of the information today. Most of the organizations around the world are fueled by data today. People around the world are generating trillions of bits of data every day. All these data represents the snapshot of a historical moment of their life. What people don't realize is - Data is NOT the oil to fuel our future. Data is the oil to burn our future faster. The more data we process, the faster the entropy grows, and faster the world will be erased. 

[Thinking out loud, it might be reasonable to say that humans progressed from Monarchy to Democracy so that people can have freedom of expression - creating more information and hence more entropy; the transition being natural because it favored entropy].

Comments

Popular posts from this blog

DDoS Attack on Bitotsav '19 Website

Rendering Performance Evaluation - Android SDK vs React Native

Popular ethical hacking tricks on LAN