Not free as in speech or free as in beer, nor as in the sense of perpetual motion machines, zero point energy or pills that turn your water into petroleum.
My question is - how much more credible than these latter examples is the “free energy principle” as a unifying principle for learning systems?
The chief pusher of this wheelbarrow appears to be UCL’s Karl Friston.
He starts his latest Nature Reviews Neuroscience with this statement of the principle:
The free-energy principle says that any self-organizing system that is at equilibrium with its environment must minimize its free energy.
Is that “must” in the sense of obligation, or probability? Self-organising in what sense? What class of equilibrium? What is our chief experimental evidence for this hypothesis? Rather than a no-nonsense unpacking of these, the article goes on to meander through an ocean fashionable other stuff (The Bayesian Brain Hypothesis) which I have not yet trawled for salient details.
Fortunately we do get a definition of free energy itself, with a diagram, which
...shows the dependencies among the quantities that define free energy. These include the internal states of the brain \(\mu(t)\) and quantities describing its exchange with the environment: sensory signals (and their motion) \(\bar{s}(t) = [s,s',s''…]^T\) plus action \(a(t)\). The environment is described by equations of motion, which specify the trajectory of its hidden states. The causes \(\vartheta \supset {\bar{x}, \theta, \gamma }\) of sensory input comprise hidden states \(\bar{x} (t)\), parameters \(\theta\), and precisions \(\gamma\) controlling the amplitude of the random fluctuations \(\bar{z}(t)\) and \(\bar{w}(t)\). Internal brain states and action minimize free energy \(F(\bar{s}, \mu)\), which is a function of sensory input and a probabilistic representation \(q(\vartheta|\mu)\) of its causes. This representation is called the recognition density and is encoded by internal states \(\mu\).
The free energy depends on two probability densities: the recognition density \(q(\vartheta|\mu)\) and one that generates sensory samples and their causes, \(p(\bar{s},\vartheta|m)\). The latter represents a probabilistic generative model (denoted by m), the form of which is entailed by the agent or brain...
\[\begin{split}F = -<\ln p(\bar{s},\vartheta|m)>_q + -<\ln q(\vartheta|\mu)>_q\end{split}\]
All well and good, but so far the language reads all like an old-style prescriptive grammarian - “any right thinking brain, seeking to avoid the vice of sloth and decadent perception after the manner of heathens and foreigners, would do well to seek to maximise its free energy before partaking of a stimulating and refreshing physical recreation such as a game of cricket.”.
Presumably as I drill deeper, i will find traces of the descriptive, the predictions to be made and experiments done, but this is an alarming start.
See also: Exergy, Landauer’s Principle.