Stochastic thermodynamics of computation

Jan Korbel

CSH Workshop "Computation in dynamical systems", Obergurgl

slides available at: www.slides.com/jankorbel

Why should we care about thermodynamics of computation?

• Computers consume 6-10% of total electricity
• A part of the energy is inevitably transferred to a waste heat
• Most research in CS has been focused on the performance of the computation, not taking into account the costs 
• A little is known about whether and how the energetic costs can be eliminated

Fundamental costs of computing

• The fundamental question is what are the inevitable costs of computation and what costs can be mitigated
• The notoriously known is the Landauer's bound $$Q \geq - k T \Delta S$$
• Originally, it was used to lower-bound the dissipated heat of a bit eraser. The eraser changes the initial distribution \(\{1/2,1/2\}\) to the final distribution \(\{1,0\}\), so \(\Delta S = - \ln 2\) and we obtain the famous formula $$Q \geq k T \ln 2$$

General form of Landauer's bound

• More generally, the Landauer's bound is a direct consequence of the second law of thermodynamics
• A central quantity in non-equilbrium thermodynamics is the entropy production $$\sigma = \Delta S + \sum_i \frac{Q_i}{k T_i}$$
• The main property of the entropy production is that it cannot be negative, i.e., \(\sigma \geq 0\)
• From this, we obtain that $$ \sum_i \frac{Q_i}{k T_i} > - \Delta S$$
• Typically the computation is designed to lower the entropy, so we get a strictly positive bound on the dissipated heat

Parallel 2-bit eraser

• What is Landauer's cost of the simultaneous erasure of two bits \(B_{1,2}\) with initial marginal distributions \(\{1/2,1/2\}\)?
• Naively, one can think that it is \( 2 kT \ln 2\) because we erase two bits
• By using the general formula, the cost is the drop in the joint entropy of the two bits. 
• The initial joint entropy can be expressed as $$S(B_1,B_2) = S(B_1) + S(B_2) + I(B_1,B_2)$$ where \(S(B_{1,2}) =- \ln 2\) is the entropy of the marginal initial distribution and \(I(B_1,B_2)\) is the mutual information
• Thus, the landauer cost can be expressed as $$Q \geq kT 2 \ln 2 + k T I(B_1,B_2)$$
• The mutual information is a special case of a mismatch cost

Parallel 2-bit eraser

Logical and thermodynamic reversibility

• In the previous decades, there has been a debate about the relationship of logical reversibility and thermodynamic reversibility
•  Logical reversibility: a computation is logically reversible if and only if, for any output logical state, there is a unique input logical state.
• Thermodynamic reversibility: a process is thermodynamically reversible if and only if the entropy production is equal to zero (quasi-static process)

• Historically, some authors were pointing out a relation between logical and thermodynamic reversibility
• However, several papers have shown that logical and thermodynamic (ir)reversibility are, in fact, completely independent properties of a physical process

Logical and thermodynamic reversibility

Example 1: bit erasure

Example 2: measurement

Relevance of Landauers bound

• While Landauer bound gives us a fundamental  bound of computation, it is well known that the actual computers, both artificial and natural, dissipate much more energy than Landauer's bound predicts
• Even for a bit erasure, the bound can only be achieved by a quasistatic process (that takes infinite time) and with the optimal protocol
• Real computers are designed to compute in finite time and do much more than just a bit erasure
• In general, physical constraints of computers lead to increased heat dissipation

Stochastic thermodynamics

• Since real computations are performed in finite time, using the framework of equilibrium thermodynamics is not sufficient for characterizing the thermal dissipation of computation
• Thus, it is necessary to use another framework that can incorporate the far-from-equilibrium dynamics of computation
• Stochastic thermodynamics provides us with powerful tools that enable to connect the theory of stochastic processes to far-from-equilibrium thermodynamics

Stochastic thermodynamics

• Stochastic thermodynamics is a field that emerged in 90's 
• Its original application was in non-equilibrium thermodynamics of mesoscopic systems, as chemical reaction networks and molecular motors
• Probably the most popular result are the fluctuation theorems extending the validity of the 2nd law of thermodynamics to the case trajectory quantities
• The direct corollaries as Crooks fluctuation theorem and Jarzynski equality relate work done on a system with the free energy difference

Fluctuation theorems

Mismatch cost

• In the previous example, we observed a special case of a mismatch cost
• The mismatch cost is an additional term to the entropy production caused by the fact that the control protocol of a process was designed to optimize a (computation) process  given the particular initial distribution but the actual distribution is different
• Consider a physical process with initial distribution \(q_{t_0}(x)\) that minimizes the entropy production
• The actual initial distribution is \(p_{t_0}(x)\)
• The EP can be then expressed as $$\sigma(p_{t_0}) = \sigma(q_{t_0}) + D_{KL}(p_{t_0}\|q_{t_0}) - D_{KL}(p_{t_f} \| q_{t_f})$$
• \(D_{KL}(p\|q) = \sum_x p(x) \log \frac{p(x)}{q(x)}\) is the Kullback-Leibler divergence

Mismatch cost for a 2-bit eraser

• In the case of a 2-bit eraser, \(q_{t_0}(B_1,B_2) = p_{t_0}(B_1) p_{t_0}(B_2)\) and therefore

$$D_{KL}(p_{t_0}(B_1,B_2)\|p_{t_0}(B_1) p_{t_0}(B_2)) = I(B_1,B_2)$$

• This particular type of mismatch cost is called modularity cost which is the cost for the fact that the subsystems are statistically coupled
• Therefore, each time a system is build from two or more statistically coupled subsystems (which is a typical setup in all computational devices) we pay an extra cost

Speed limit theorems

$$ \sigma (\tau)  \geq\frac{\left(\sum_x |p_0(x) -p_\tau(x)|\right)^2}{2 A_{\text{tot}}(\tau)}$$

• Another aspect of  computation is the time of computation
• One could expect that faster computation leads to more dissipated heat
• The lower bound is provided by the speed limit theorem which can be formulated as

where \(p_0\) is the initial distribution, \(p_{\tau}\) is the final distribution, and \(A_{tot}\) is the total activity  which is the average number of state transitions that occur during the computational process.

• The term in the enumerator is the square of the \(L_1\) distance between the initial and the final state

Thermodynamic uncertainty relation

$$\sigma (\tau)  \geq \frac{2 \langle J (\tau) \rangle^2}{\mathrm{Var}(J (\tau))} $$

• Another contribution to the EP is due to the cost of precision
• Suppose we choose an increment function \(d(x', x)\). Such a function can be any observable, real-valued function of state transitions \(x' \to x\) that is anti-symmetric under the interchange of its two arguments. 
• The current \(J\) associated with that function is the value of the associated observable summed over all state transitions in a trajectory.
• The entropy production can be then lower-bounded by the normalized precision of a current

SLT & TUR

Example: two equivalent circuits

\(\sum_x |p_i(x) - p_f(x)| = 3/8\)

Initial distribution: input states - uniform, internal states - 0

\(\sum_x |p_i(x) - p_f(x)| = 6/8\)

Possible consequences for CS

• In theoretical CS, a computational device is typically an abstract, generic model of any entity that computes (transforms an input into an output)
• We think about computation in an abstract way: we count the number of operations and how they scale
• In applied CS, the programmers are also thinking about other costs, runtime, memory, etc.
• Similarly, we can think about other costs as dissipated energy

Mapping between design features of a computer and its performance through resource costs

Possible consequences for CS

• Depending on the particular task, the amount of dissipated heat can depend not only on theoretical computation but also on the physical representation of the computational device
• It is not only the architecture of the computational devide, but the physical substrate and representation of computational states that can have a large impact

Probabilistic computation

• One possible example of a non-conventional approach to computation is the probabilistic computation
• Similarly to quantum computation, one might generalize the standard binary representation of the information to a probabilistic representation
• This approach might be useful when dealing with stochastic problems (MC simulations, Bayesian networks, Markov models)
• Probabilistic computing has a wide range of applications, including machine learning, robotics, computer vision, natural language processing, and cognitive computing.

Neuromorphic computing

• Another example of a non-conventiional computation approach is neuromorphic computing
• Here, contrary to standard CMOS-based computers, the architecture is inspired by the structure and function of the human brain
• While this approach might be useful in specific computation tasks, it might be also more energetically favorable

Conclusions

• Thermodynamics of Computation  is an important aspect of CS
• It is important to understand which energetic costs are fundamental and which can be optimized by using different approaches (algorithms, architecture, physical representation)
• Non-conventional approaches to computation (that are also the topic of this workshop) can have important consequences to thermodynamics of computation

More resources

• David's review paper: J. Phys. A: Math. Theor. 52 193001
• David's lectures on YouTube
• Perspective paper (in prep)