H-theorem
Encyclopedia
In Classical Statistical Mechanics
, the H-theorem, introduced by Ludwig Boltzmann
in 1872, describes the increase in the entropy
of an ideal gas
in an irreversible process. H-theorem follows from considerations of Boltzmann's equation. Claude Shannon denoted his measure of information entropy H after the H-theorem.
It appears to predict an irreversible increase in entropy, despite microscopically reversible dynamics. This has led to much discussion.
where summation runs over all possible distinct states of the system, and pi is the probability that the system could be found in the i-th state.
This is closely related to the entropy formula of Gibbs,
and we shall (following e.g., Waldram (1985), p. 39) proceed using S rather than H.
First, differentiating with respect to time gives
(using the fact that ∑ dpi/dt = 0, since ∑ pi = 1).
Now Fermi's golden rule
gives a master equation
for the average rate of quantum jumps from state α to β; and from state β to α. For an isolated system the jumps will make contributions
where the reversibility of the dynamics ensures that the same transition constant ναβ appears in both expressions.
So
But the two brackets will have the same sign, so each contribution to dS/dt cannot be negative.
Therefore
for an isolated system.
The same mathematics is sometimes used to show that relative entropy is a Lyapunov function
of a Markov process
in detailed balance
, and other chemistry contexts.
H is a forerunner of Shannon's information entropy
. The article on Shannon's information entropy
contains a
good explanation of the discrete counterpart of the quantity H, known as the information entropy or information uncertainty (with a minus sign). By extending the discrete information entropy to the continuous information entropy, also called differential entropy
, one obtains the expression in Eq.(1), and thus a better feel for the meaning of H.
The H-theorem's connection between information and entropy plays a central role in a recent controversy called the Black hole information paradox
.
Tolman offers the following equations for the definition of the quantity H in Boltzmann's original H theorem.
Here we sum over the i regions into which µ-space is divided.
This relation can also be written in integral form.
H can also be written in terms of the number of molecules present in the i cells.
Statistical mechanics
Statistical mechanics or statistical thermodynamicsThe terms statistical mechanics and statistical thermodynamics are used interchangeably...
, the H-theorem, introduced by Ludwig Boltzmann
Ludwig Boltzmann
Ludwig Eduard Boltzmann was an Austrian physicist famous for his founding contributions in the fields of statistical mechanics and statistical thermodynamics...
in 1872, describes the increase in the entropy
Entropy
Entropy is a thermodynamic property that can be used to determine the energy available for useful work in a thermodynamic process, such as in energy conversion devices, engines, or machines. Such devices can only be driven by convertible energy, and have a theoretical maximum efficiency when...
of an ideal gas
Ideal gas
An ideal gas is a theoretical gas composed of a set of randomly-moving, non-interacting point particles. The ideal gas concept is useful because it obeys the ideal gas law, a simplified equation of state, and is amenable to analysis under statistical mechanics.At normal conditions such as...
in an irreversible process. H-theorem follows from considerations of Boltzmann's equation. Claude Shannon denoted his measure of information entropy H after the H-theorem.
It appears to predict an irreversible increase in entropy, despite microscopically reversible dynamics. This has led to much discussion.
Quantum mechanical H-theorem
In Quantum Statistical Mechanics (which is the quantum version of Classical Statistical Mechanics), the H-function is the function:
where summation runs over all possible distinct states of the system, and pi is the probability that the system could be found in the i-th state.
This is closely related to the entropy formula of Gibbs,
and we shall (following e.g., Waldram (1985), p. 39) proceed using S rather than H.
First, differentiating with respect to time gives
(using the fact that ∑ dpi/dt = 0, since ∑ pi = 1).
Now Fermi's golden rule
Fermi's golden rule
In quantum physics, Fermi's golden rule is a way to calculate the transition rate from one energy eigenstate of a quantum system into a continuum of energy eigenstates, due to a perturbation....
gives a master equation
Master equation
In physics and chemistry and related fields, master equations are used to describe the time-evolution of a system that can be modelled as being in exactly one of countable number of states at any given time, and where switching between states is treated probabilistically...
for the average rate of quantum jumps from state α to β; and from state β to α. For an isolated system the jumps will make contributions
where the reversibility of the dynamics ensures that the same transition constant ναβ appears in both expressions.
So
But the two brackets will have the same sign, so each contribution to dS/dt cannot be negative.
Therefore
for an isolated system.
The same mathematics is sometimes used to show that relative entropy is a Lyapunov function
Lyapunov function
In the theory of ordinary differential equations , Lyapunov functions are scalar functions that may be used to prove the stability of an equilibrium of an ODE. Named after the Russian mathematician Aleksandr Mikhailovich Lyapunov, Lyapunov functions are important to stability theory and control...
of a Markov process
Markov process
In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time-varying random phenomenon for which a specific property holds...
in detailed balance
Detailed balance
The principle of detailed balance is formulated for kinetic systems which are decomposed into elementary processes : At equilibrium, each elementary process should be equilibrated by its reverse process....
, and other chemistry contexts.
H is a forerunner of Shannon's information entropy
Information entropy
In information theory, entropy is a measure of the uncertainty associated with a random variable. In this context, the term usually refers to the Shannon entropy, which quantifies the expected value of the information contained in a message, usually in units such as bits...
. The article on Shannon's information entropy
Information entropy
In information theory, entropy is a measure of the uncertainty associated with a random variable. In this context, the term usually refers to the Shannon entropy, which quantifies the expected value of the information contained in a message, usually in units such as bits...
contains a
good explanation of the discrete counterpart of the quantity H, known as the information entropy or information uncertainty (with a minus sign). By extending the discrete information entropy to the continuous information entropy, also called differential entropy
Differential entropy
Differential entropy is a concept in information theory that extends the idea of entropy, a measure of average surprisal of a random variable, to continuous probability distributions.-Definition:...
, one obtains the expression in Eq.(1), and thus a better feel for the meaning of H.
The H-theorem's connection between information and entropy plays a central role in a recent controversy called the Black hole information paradox
Black hole information paradox
The black hole information paradox results from the combination of quantum mechanics and general relativity. It suggests that physical information could disappear in a black hole, allowing many physical states to evolve into the same state...
.
Boltzmann's H-theorem
Starting with a function f that defines the number of molecules in small region of µ-space denoted byTolman offers the following equations for the definition of the quantity H in Boltzmann's original H theorem.
Here we sum over the i regions into which µ-space is divided.
This relation can also be written in integral form.
H can also be written in terms of the number of molecules present in the i cells.
-
An additional way to calculate the quantity H is:
Where P is the probability of finding a system chosen at random from the specified microcanonical ensembleMicrocanonical ensembleIn statistical physics, the microcanonical ensemble is a theoretical tool used to describe the thermodynamic properties of an isolated system. In such a system, the possible macrostates of the system all have the same energy and the probability for the system to be in any given microstate is the same...
And can finally be written as:
where G may be spoken of as the number of classical states.
The quantity H can also be defined as the integral over velocity space :
(1)
where P(v) is the probability.
Using the Boltzmann equation one can prove that H can only decrease.
For a system of N statistically independent particles, H is related to the thermodynamic entropy S through:
so, according to the H-theorem, S can only increase.
However, Loschmidt objected that it should not be possible to deduce an irreversible process from time-symmetric dynamics and a time-symmetric formalism: something must be wrong (Loschmidt's paradoxLoschmidt's paradoxLoschmidt's paradox, also known as the reversibility paradox, is the objection that it should not be possible to deduce an irreversible process from time-symmetric dynamics...
). The explanation is that Boltzmann's equation is based on the assumption of "molecular chaosMolecular chaosIn kinetic theory in physics, molecular chaos is the assumption that the velocities of colliding particles are uncorrelated, and independent of position...
", i.e., that it is acceptable for all the particles to be considered independent and uncorrelated. This in fact breaks time reversal symmetry and therefore begs the question.
Analysis
At the heart of the H-theorem is the replacement of 1-state to 1-state deterministic dynamics by many-state to many-state MarkovianMarkov processIn probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time-varying random phenomenon for which a specific property holds...
mixing, with information lost at each Markovian transition.
Gull is correct that, with the powers of Laplace's demonLaplace's demonIn the history of science, Laplace's demon was the first published articulation of causal or scientific determinism by Pierre-Simon Laplace in 1814...
, one could in principle map forward exactly the ensemble of the original possible states of the N-particle system exactly, and lose no information. But this would not be very interesting. Part of the program of statistical mechanics, not least the MaxEnt schoolMaximum entropy thermodynamicsIn physics, maximum entropy thermodynamics views equilibrium thermodynamics and statistical mechanics as inference processes. More specifically, MaxEnt applies inference techniques rooted in Shannon information theory, Bayesian probability, and the principle of maximum entropy...
of which Gull is an enthusiastic proponent, is to see just how much of the detail information in the system one can ignore, and yet still correctly predict experimentally reproducible results.
The H-theorem's program of regularly throwing information away, either by systematically ignoring detailed correlations between particles, or between particular sub-systems, or through systematic regular coarse-graining, leads to predictions such as those from the Boltzmann equationBoltzmann equationThe Boltzmann equation, also often known as the Boltzmann transport equation, devised by Ludwig Boltzmann, describes the statistical distribution of one particle in rarefied gas...
for dilute ideal gases or from the recent entropy-production fluctuation theoremFluctuation theoremThe fluctuation theorem , which originated from statistical mechanics, deals with the relative probability that the entropy of a system which is currently away from thermodynamic equilibrium will increase or decrease over a given amount of time...
, which are useful and reproducibly observable. They also mean that we have learnt something qualitative about the system, and which parts of its information are useful for which purposes, which is additional beyond even the full specification of the microscopic dynamical particle trajectories.
(It may be interesting that having rounded on the H-theorem for not considering the microscopic detail of the microscopic dynamics, Gull then chooses to demonstrate the power of the extended-time MaxEnt/Gibbsian method by applying it to a Brownian motion example - a not so dissimilar replacement of detailed deterministic dynamical information by a simplified stochastic/probabilistic summary!)
However, it is an assumption that the H-theorem's coarse-graining is not getting rid of any 'interesting' information. With such an assumption, one moves firmly into the domain of predictive physics: if the assumption goes wrong, it may produce predictions which are systematically and reproducibly wrong.
See also
- Loschmidt's paradoxLoschmidt's paradoxLoschmidt's paradox, also known as the reversibility paradox, is the objection that it should not be possible to deduce an irreversible process from time-symmetric dynamics...
- Arrow of timeArrow of timeThe arrow of time, or time’s arrow, is a term coined in 1927 by the British astronomer Arthur Eddington to describe the "one-way direction" or "asymmetry" of time...
- Second law of thermodynamicsSecond law of thermodynamicsThe second law of thermodynamics is an expression of the tendency that over time, differences in temperature, pressure, and chemical potential equilibrate in an isolated physical system. From the state of thermodynamic equilibrium, the law deduced the principle of the increase of entropy and...
- Fluctuation theoremFluctuation theoremThe fluctuation theorem , which originated from statistical mechanics, deals with the relative probability that the entropy of a system which is currently away from thermodynamic equilibrium will increase or decrease over a given amount of time...
-
