The Ising Model and CA, Part I: Thermodynamics in CA

Can thermodynamics be applied to a cellular automaton? Can we use it to create a cellular automaton that reproduces the Ising model phase transition?

This is a multipart series. Read the others here:

Part I: Thermodynamics in CA

Part II: The Ising CA

Introduction

The Ising model is a famous model of phase transition in a ferromagnetic material. It can be simulated using many methods, famously MCMC methods, leading to pretty looking simulations like this one:

A simulation of the Ising model using the Metropolis Algorithm, Starting below T_c and finishing above it

This basically uses a randomized method that ensures that the generated grid is sampled from the probability distribution we would get if we coupled such a system to a heat bath. The Ising model doesn’t really have any inherent dynamics. The only fixed thing is the energy of each state and the Boltzmann distribution. The main problem with directly trying to port such a simulation to a cellular automaton is that we do not have any way to do a probabilistic action as CA’s are completely deterministic. Also, this system does not really conserve energy, and if I really wanted to do physics using CA, I want it to at least try and imitate real life physics.

Thermodynamics? In my cellular automata?

Well, thermodynamics usually makes no sense in a CA context. What is energy? Temperature? The stereotypical example of a CA, Conway’s Game of Life, has pretty much no conserved quantities. It also doesn’t have time reversibility. So it is a pretty bad model of physics and so thermodynamics makes no sense here.

In order to actually have a sensible notion of thermodynamics, we will have to design a CA with these properties:

  • Time reversibility
  • Presence of some conserved quantity
  • Chaotic behavior

Lucky for us there already exists a CA with these properties… behold! the HPP lattice gas!


If you have heard of HPP before, you might be thinking that this makes no sense. HPP is a model of a fluid, and since all particles have the exact same average kinetic energy, you might think that every configuration would have the exact same temperature.

In order to tackle these problems, we will have to properly define and reinterpret things. In the following sections, we will think of each particle of the HPP gas as an ‘energy packet’. Hence, conservation of energy just becomes equivalent to the conservation of particle number. We shall consider one particle to be one unit of energy.

The HPP lattice gas happens to be its own time reversal under a simple state transformation, which proves its reversibility. And visually, it seems chaotic at high enough densities.

Calculating Thermodynamic Variables

Entropy

Now, consider a box with some volume and with some number of energy packets. In this context, now how do we define temperature? To do this, let us introduce another important concept in Statistical mechanics: Entropy.

Entropy, using the most basic definition is this:

S=∑i−piln⁡(pi) S=\sum_{i}{-p_i\ln{(p_i)}}

where pip_i represents the probability of a certain state ii .

Now, I will be taking a huge leap of faith and assuming the following: HPP behaves completely randomly after a small amount of time, i.e. every possible state with the same energy is equally probable. There are a few reasons why this isn’t completely unjustified (like reversibility) but as of now, I have no idea how to prove it in a fully satisfactory way and I am not sure if it is even true. But making this assumption will help us simplify our calculations.

Taking the above statement as a postulate, we get the following expression for entropy of a V=4NMV=4NM (as each square holds at most 4 particles) board with ρV\rho V particles (I will be ignoring edge effects and assuming V≫1V\gg1 )

Npossiblestates=(VρV)=V!(ρV)!(V−ρV)! N_{possible states}={V\choose{\rho V}}=\frac{V!}{(\rho V)!(V-\rho V)!}

pi=1Npossiblestates=(ρV)!(V−ρV)!V! p_i=\frac{1}{N_{possible states}}=\frac{(\rho V)!(V-\rho V)!}{V!}

S=∑i−piln⁡(pi)=ln⁡V!(ρV)!(V−ρV)!≈V(−ρln⁡(ρ)−(1−ρ)ln⁡(1−ρ)) S=\sum_{i}{-p_i\ln{(p_i)}}=\ln{\frac{V!}{(\rho V)!(V-\rho V)!}}\approx V(-\rho\ln(\rho)-(1−\rho)\ln(1−\rho))

The Second Law Of Thermodynamics

Now, we may ask, does this entropy have the same behaviour as in our world? Does total entropy of a system always increase with time?

As a thought experiment, consider connecting two boxes with different densities. Let us take the total entropy to be the sum of the entropies of the two boxes. With time, we should expect the density of both boxes to go to the same equilibrium value.


A straightforward calculation will tell us that the completely uniform state happens to be the maximal entropy state. Hence the second law of thermodynamics seems to hold here! This should not actually be surprising if we understand what entropy is measuring and why it always increases in a system.

Consider there are kk boxes, each with entropy SiS_i . Assume that we do not know anything else about our system, hence the total set of possibilities XX has cardinality

n(X)=eSX=e∑Si n(X)=e^{S_{X}}=e^{\sum{S_i}}

Imagine that after some fixed, but large number of steps, it almost surely results in some state YY with some entropy SYS_Y . Since we assumed that all that the elements of XX result in some element of YY , hence the evolution of XX for TT steps is a subset of YY . Because the dynamics of HPP are bijective, this means that

evolveT(X)⊆Y  ⟹  n(evolveT(X))≤n(Y)  ⟹  n(X)≤n(Y)  ⟹  ∑Si=SX≤SY evolve^T(X) \subseteq Y \implies n(evolve^T(X))\leq n(Y) \implies n(X) \leq n(Y) \implies \sum {S_i}=S_X \leq S_Y

Hence, we have shown that entropy cannot decrease in all the cases. This derivation can be adapted to bound the exact maximum probability of a certain entropy decreasing transition.

Interestingly, due to the reversibility of HPP, it is actually very easy to construct specific magic states that actually violate the second law, for example (The magic happens around t=180t=180 ):


But this does not contradict our derivation, because our results should hold only if we know nothing about the system other than its macroscopic properties. An analogous result holds for the real world, there do exist such weird states in classical mechanics but due to chaos theory it is pretty much impossible to build such a second law violator in real life.

Temperature

Now, we can move on to define temperature. The most fundamental definition of temperature is the amount of energy needed to change the entropy of the system by one unit. To me, this is a very beautiful definition as it immediately makes it clear why energy must always flow from a hot body to a cold one: due to the second law! Let us consider a volume V container with density ρ\rho as before. Imagine one packet of energy is inserted, i.e. ΔQ=1\Delta{Q}=1 . Then,

ΔS=ln⁡(VρV+1)−ln⁡(VρV)=ln⁡V−ρVρV+1≈ln⁡1−ρρ \Delta{S} = \ln{V\choose{\rho V+1}}-\ln{V\choose{\rho V}}=\ln{\frac{V−\rho V}{\rho V+1}}\approx\ln\frac{1-\rho}{\rho}

So the temperature,

T=ΔQΔS=1ln⁡1−ρρ T=\frac{\Delta{Q}}{\Delta{S}} = \frac{1}{\ln\frac{1-\rho}{\rho}}

Which is interesting as it only depends on the density. On reflection, this should make sense: Temperature is an intensive variable, so it shouldn’t depend on the size, only on the density of the chaos.

One funny thing you might notice is that temperature here isn’t actually strictly positive: when the density exceeds 50%, entropy decreases with gain in energy. This concept of negative temperature is actually well known, but is very rare in real life. And these states behave as if they are hotter than infinite temperature. For this reason, in some places, I will prefer to use thermodynamic beta instead, which is much nicer to work with and is just the reciprocal of temperature in natural units.

How is the HPP gas relevant to the Ising model?

In order to set up a CA we need a heat bath. So why not use this as a heat bath? Unlike the Ising model, we have an extremely simple formula to calculate the temperature, so we can try and couple it with the Ising CA and use it to calculate temperature. The key insight is that the laws of thermodynamics must hold regardless of the exact details of the coupling, as long as we respect reversibility and energy conservation.

This is extremely vague for now, I will expand upon more details of how exactly we are going to simulate the Ising model and how the coupling works in my next posts. For now, you can play around with final rule and try to figure out how it works.