## Nonequilibrium thermodynamics of uncertain stochastic processes

Jan Korbel     &    David Wolpert

WOST IV

Slides are available at: https://slides.com/jankorbel

Reference: https://arxiv.org/abs/2210.05249

## Idealized experiment

$$T$$

$$X$$

$$P(T)$$

$$X$$

3rd trial

$$T_3$$

$$X_3$$

...

## In reality

Measure a quantity $$X$$ and the assume that the temperature $$T$$ can be measured with infinite precision

Temperature is measured with limited precision, can change between experiments

$$T=?$$

## In many experiments

• We do not know the exact value of
• number of heat baths
• temperatures
• chemical potentials
• energy spectrum
• control protocol
• transition rates
• initial distribution

## Example: 3-state system

• Consider a simple example of a 3-state system coupled to one of three apparatuses.
• We want to measure the distribution $$p_{t_f}(E)$$ at final time $$t_f$$
• There are two possible scenarios:

## Example: 3-state system

Effective scenario: we do not know which apparatus is coupled to our system but we can repeat the experiment many times with the same apparatus

Phenomenological scenario: the apparatus is randomly rechosen each time we run the experiment

## Consequences for the experiment

• Whether the experimenter can adjust the experiment to the actual value of the apparatus has significant consequences for the experiment.

• Let's illustrate the difference on a simple example of a moving optical tweezer with uncertain stiffness.
• We consider a particle in a laser trap potential described by Langevin equation $$\dot{x} = -\frac{\partial V}{\partial x} + \xi$$ with the quadratic potential $$V_k(x,t) = \frac{k}{2} (x - \lambda(t))^2$$ where $$k$$ is the stiffness.
• We want to move the trap from $$\lambda_i=0$$ at $$t_i = 0$$ to $$\lambda_f$$ at time $$t_f$$
• It is possible to calculate the optimal protocol that minimizes the work

## Consequences for the experiment

• Let us suppose that the stiffness can change each time due to imprecisions in the experimental setup. We assume the stiffness is sampled from a distribution $$p(k)$$
• We compare two scenarios:
1. Adapted: The experimenter can measure $$k$$ for each run and can adapt the protocol accordingly
2. Unadapted: The experimenter can only measure average stiffness $$\bar{k}$$ and adopts the protocol for that single stiffness $$\bar{k}$$ for all runs

The average work as a function of standard deviation of the distribution $$p(k)$$

## Thermodynamics of systems coupled to uncertain environment

• Consider a set of apparatuses $$\mathcal{A}$$.
• For each apparatus $$\alpha \in \mathcal{A}$$, we have a system with a precise number of baths, temperatures, chemical potentials, etc. satisfying local detailed balance
• We consider a probability distribution $$P^\alpha$$ over the apparatuses

Effective value over the apparatuses can be defined as

$$\overline{X}:= \int \mathrm{d} P^\alpha X^\alpha$$

Effective distribution $$\bar{p}_x(t)$$ fulfills the equation

$$\dot{\bar{p}}_x(t) = \sum_{x'} \int \mathrm{d} P^\alpha K^\alpha_{xx'} p^\alpha_{x'}(t)$$   which is generally non-Markovian

## Effective ensemble stochastic thermodynamics

• Expected internal energy is $$\bar{U} = \int \mathrm{d} P^\alpha \sum_x p^\alpha_t(x) u^\alpha(x)$$
• Expected first law of thermodynamics

$$\dot{\bar{U}} = \dot{\bar{Q}} + \dot{\bar{W}}$$

• Expected ensemble entropy $$\bar{S} = - \sum_x \int \mathrm{d} P^\alpha p_t^\alpha(x) \ln p_t^\alpha(x)$$
• Expected second law of thermodynamics

$$\dot{\bar{S}} = \dot{\bar{\Sigma}} + \dot{\bar{\mathcal{E}}}$$

• where $$\dot{\bar{\Sigma}} \geq 0$$
• and $$\dot{\bar{\mathcal{E}}} = \overline{\beta \dot{Q}}$$ - no explicit relation between $$\dot{\bar{\mathcal{E}}}$$ and $$\dot{\bar{Q}}$$

## Two types of optimal work

• For each scenario, we can define the minimum dissipated work
1. Adapted: For each apparatus $$\alpha$$, we can choose the optimal protocol $$\lambda_\alpha(t)$$ minimizing the work. The effective optimal work is $$W^{ad}_{\min} = \int \mathrm{d} P^\alpha \min_{\lambda_\alpha(t)}(W^\alpha[\lambda_\alpha(t)])$$
2. Unadapted: Since we cannot adopt the protocol for each apparatus separately, we find a single protocol that minimizes the effective work $$W^{unad}_{\min} = \min_{\lambda(t)} \left( \int \mathrm{d} P^\alpha W^{\alpha}[\lambda(t)] \right)$$
• The effective dissipated work is in both cases the difference between the average work obtained from the given set of protocols (adapted scenario) or a single protocol (unadapted scenario, from the optimal work

## Uncertainty in initial distribution

• We consider that the apparatus is fixed, except for $$p_{t_i}$$. We denote the set of possible initial distributions $$p_{t_i}^\alpha$$ where each distribution has probability $$p(\alpha)$$ to appear.
• We consider a distribution $$q_i$$ that minimizes the expected work.
• For any other distribution, it is possible to express the dissipated work as $$W_{diss}(p_{t_i}^\alpha) = D_{KL}(p_{t_i}^\alpha || q_{t_i}) - D_{KL}(p_{t_f}^\alpha || q_{t_f}) + W_{diss}(q_{t_i})$$
• If $$p_{t_f}^\alpha = q_{t_f}$$ for all $$\alpha$$, i.e., the process has the same ending distribution regardless of the initial distribution, we obtain $$\overline{W}_{diss} =\int \mathrm{d} P^\alpha W_{diss}(p_{t_i}^\alpha) \geq \int \mathrm{d} P^\alpha D_{KL}(p_{t_i}^\alpha || \bar{p}_{t_i}) = D_{JS}(\{p_{t_i}^\alpha\}^\alpha)$$

## Phenomenological EP and FTs

• Let us now focus on the case where the apparatus changes after each stochastic trajectory $$\pmb{x}$$ is generated. We are not able to measure $$\pmb{P}(\pmb{x}|\alpha)$$ but only $$\overline{\pmb{P}}(\pmb{x})$$
• Denote $$\pmb{P}^\alpha(\pmb{x}) \equiv \pmb{P} (\pmb{x}|\alpha)$$ and $$\pmb{P}(\pmb{x},\alpha) = \pmb{P}(\pmb{x}|\alpha) P^\alpha$$. $$^{\dag}$$ denotes time-reversal
• Effective ensemble EP $$\bar{\Sigma}= D_{KL}(\pmb{P}(\pmb{x},\alpha)||\pmb{P}^\dag(\pmb{x}^{\dag},\alpha))$$
• By using the chain rule for KL-divergence $$\underbrace{D_{KL}(\pmb{P}(\pmb{x},\alpha)||\pmb{P}^\dag (\pmb{x}^{\dag},\alpha))}_{\bar{\Sigma}} = \underbrace{D_{KL}(\overline{\pmb{P}}(\pmb{x})||\overline{\pmb{P}}^{\dag}(\pmb{x}^{\dag}))}_{\Phi} + \underbrace{D_{KL}(\pmb{P}(\alpha|\pmb{x})||\tilde{\pmb{P}}(\alpha|\tilde{\pmb{x}}))}_{\Lambda}$$
• Here $$\pmb{P}(\alpha|\pmb{x}) = \frac{\pmb{P}(\pmb{x}|\alpha) P^\alpha}{\pmb{P}(\pmb{x})}$$
• We have three types of EP
1. Effective EP $$\bar{\Sigma}$$
2. Phenomenological EP $$\Phi$$
3. Likelihood EP $$\Lambda$$

## Phenomenological EP

Phenomenological EP $$\Phi = D_{KL}(\overline{\pmb{P}}(\pmb{x})||\overline{\pmb{P}}^{\dag}(\pmb{x}^{\dag})) = \int \mathcal{D} \pmb{x} \pmb{P}(\pmb{x}) \color{pink}{\ln \frac{\pmb{P}(\pmb{x})}{{\pmb{P}^\dag}({\pmb{x}}^\dag)}}$$

Phenomenological trajectory EP is $$\color{pink}{\phi(\pmb{x}) = \ln \frac{\overline{\pmb{P}}(\pmb{x})}{\overline{\pmb{P}}^\dag(\pmb{x}^{\dag})}}$$

It is straightforward to show that $$\phi$$ fullfills detailed fluctuation theorem

$$\frac{P(\phi)}{P^\dag(-\phi)} = e^{\phi}$$

Phenomenological EP describes thermodynamics for the case of expected probability

It is a lower bound for the effective EP:   $$\overline{\Sigma} \geq \Phi$$

## Likelihood EP

Likelihood EP $$\Lambda = D_{KL}(\pmb{P}(\alpha|\pmb{x})||{\pmb{P}^{\dag}}(\alpha|\pmb{x}^\dag)) = \int \mathrm{d} P^\alpha \pmb{P}(\alpha|\pmb{x}) {\color{pink} \ln \frac{\pmb{P}(\alpha|\pmb{x})}{\pmb{P}^\dag(\alpha|\pmb{x}^\dag)}}$$

Likelood trajectory EP as $${\color{pink}\lambda(\alpha|\pmb{x})}:= \sigma(\pmb{x}|\alpha) - \phi(\pmb{x}) = {\color{pink}\ln \frac{\pmb{P}(\alpha|\pmb{x})}{\pmb{P}^\dag(\alpha|\pmb{x}^\dag)}}$$

We can also show that $$\omega$$ fulfills Detailed FT:

$$\frac{P(\lambda_{\pmb{x}})}{\tilde{P}(-\lambda_{\pmb{x}^\dag})} = e^{\lambda_{\pmb{x}}}$$

From Integrated FT, we obtain that $$\Lambda_{\pmb{x}} = \langle \lambda_{\pmb{x}} \rangle_{P(\lambda_{\pmb{x}})} \geq 0$$

By averaging over all trajectories $$\Lambda = \langle \Lambda_{\pmb{x}} \rangle_{\pmb{P}(\pmb{x})} \geq 0$$

Likelihood EP tells us how much more one can learn about the apparatus by observing a forward trajectory versus a backward trajectory

## Other topics

Further topics included in the paper

• Maximal work extractions with uncertain temperatures
• Dynamics of the thermodynamic value of information when there are uncertain thermodynamic parameters

Future steps:

• Systems with uncertain energy spectrums
• Experiments with uncertain control protocols
• Complete analysis of maximal work extraction
• Extension to time-dependent apparatuses
• Extension to Hidden Markov models

Thanks!