The free energy principle made simpler but not too simple. (English) Zbl 07737538
Summary: This paper provides a concise description of the free energy principle, starting from a formulation of random dynamical systems in terms of a Langevin equation and ending with a Bayesian mechanics that can be read as a physics of sentience. It rehearses the key steps using standard results from statistical physics. These steps entail (i) establishing a particular partition of states based upon conditional independencies that inherit from sparsely coupled dynamics, (ii) unpacking the implications of this partition in terms of Bayesian inference and (iii) describing the paths of particular states with a variational principle of least action. Teleologically, the free energy principle offers a normative account of self-organisation in terms of optimal Bayesian design and decision-making, in the sense of maximising marginal likelihood or Bayesian model evidence. In summary, starting from a description of the world in terms of random dynamical systems, we end up with a description of self-organisation as sentient behaviour that can be interpreted as self-evidencing; namely, self-assembly, autopoiesis or active inference.
MSC:
| 82C27 | Dynamic critical phenomena in statistical mechanics |
Software:
PRMLTReferences:
| [1] | Clark, A., Whatever next? Predictive brains, situated agents, and the future of cognitive science, Behav. Brain Sci., 36, 3, 181-204 (2013) |
| [2] | Crauel, H.; Flandoli, F., Attractors for random dynamical systems, Probab. Theory Related Fields, 100, 3, 365-393 (1994) · Zbl 0819.58023 |
| [3] | Arnold, L., (Random Dynamical Systems. Random Dynamical Systems, Springer Monographs in Mathematics (1998), Springer-Verlag: Springer-Verlag Berlin Heidelberg) · Zbl 0906.34001 |
| [4] | Hohwy, J., The self-evidencing brain, Noûs, 50, 2, 259-285 (2016) |
| [5] | Friston, K., A free energy principle for a particular physics (2019), [q-bio]. arXiv:1906.10184 |
| [6] | Noether, E., Invarianten beliebiger Differentialausdrücke, (Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-Physikalische Klasse. Vol. 1918 (1918)), 37-44 · JFM 46.0675.01 |
| [7] | Da Costa, L.; Friston, K.; Heins, C.; Pavliotis, G. A., Bayesian mechanics for stationary processes, Proc. R. Soc. A Math. Phys. Eng. Sci., 477, 2256, Article 20210518 pp. (2021), arXiv:2106.13830 |
| [8] | Friston, K.; Heins, C.; Ueltzhöffer, K.; Da Costa, L.; Parr, T., Stochastic chaos and Markov blankets, Entropy, 23, 9, 1220 (2021) |
| [9] | Pavliotis, G. A., (Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations. Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations, Texts in Applied Mathematics, vol. 60 (2014), Springer: Springer New York) · Zbl 1318.60003 |
| [10] | Seifert, U., Stochastic thermodynamics, fluctuation theorems and molecular machines, Rep. Progr. Phys., 75, 12, Article 126001 pp. (2012) |
| [11] | Crauel, H., Global random attractors are uniquely determined by attracting deterministic compact sets, Ann. Mat. Pura Appl., 176, 1, 57-72 (1999) · Zbl 0954.37027 |
| [12] | Jarzynski, C., Nonequilibrium equality for free energy differences, Phys. Rev. Lett., 78, 14, 2690-2693 (1997) |
| [13] | Carr, J., Applications of Centre Manifold Theory (1982) |
| [14] | Haken, H., (Synergetics: An Introduction Nonequilibrium Phase Transitions and Self-Organization in Physics, Chemistry and Biology. Synergetics: An Introduction Nonequilibrium Phase Transitions and Self-Organization in Physics, Chemistry and Biology, Springer Series in Synergetics (1978), Springer-Verlag: Springer-Verlag Berlin Heidelberg) · Zbl 0396.93001 |
| [15] | Koide, T., Perturbative expansion of irreversible work in Fokker-Planck equation \(\less\) i \(\greater\) à la \(\less /I \greater\) quantum mechanics, J. Phys. A, 50, 32, Article 325001 pp. (2017) · Zbl 1375.82088 |
| [16] | Risken, H.; Frank, T., (The Fokker-Planck Equation: Methods of Solution and Applications. The Fokker-Planck Equation: Methods of Solution and Applications, Springer Series in Synergetics (1996), Springer-Verlag: Springer-Verlag Berlin Heidelberg) · Zbl 0866.60071 |
| [17] | Dürr, D.; Bach, A., The Onsager-Machlup function as Lagrangian for the most probable path of a diffusion process, Comm. Math. Phys., 60, 2, 153-170 (1978) · Zbl 0377.60044 |
| [18] | Arsenović, D.; Burić, N.; Davidović, D. M.; Prvanović, S., Lagrangian form of Schrödinger equation, Found. Phys., 44, 7, 725-735 (2014) · Zbl 1302.81089 |
| [19] | Krasnov, K., A gauge-theoretic approach to gravity, Proc. R. Soc. A Math. Phys. Eng. Sci., 468, 2144, 2129-2173 (2012) · Zbl 1371.81222 |
| [20] | Kleeman, R., A path integral formalism for Non-equilibrium Hamiltonian statistical systems, J. Stat. Phys., 158, 6, 1271-1297 (2015) · Zbl 1319.82018 |
| [21] | Schrodinger, E., What Is Life?: with mind and matter and autobiographical sketches (2012), Cambridge University Press: Cambridge University Press Cambridge, New York · Zbl 1254.01052 |
| [22] | Pearl, J., Graphical models for probabilistic and causal reasoning, (Smets, P., Quantified Representation of Uncertainty and Imprecision. Quantified Representation of Uncertainty and Imprecision, Handbook of Defeasible Reasoning and Uncertainty Management Systems (1998), Springer Netherlands: Springer Netherlands Dordrecht), 367-389 · Zbl 0933.03020 |
| [23] | Pearl, J., Causality (2009), Cambridge University Press: Cambridge University Press Cambridge, U.K., New York · Zbl 1188.68291 |
| [24] | Nicolis, G.; Prigogine, I., Self-Organization in Nonequilibrium Systems: from Dissipative Structures to Order Through Fluctuations (1977), Wiley-Blackwell: Wiley-Blackwell New York · Zbl 0363.93005 |
| [25] | Graham, R., Covariant formulation of non-equilibrium statistical thermodynamics, Z. Phys. B Condense. Matter, 26, 4, 397-405 (1977) |
| [26] | Eyink, G. L.; Lebowitz, J. L.; Spohn, H., Hydrodynamics and fluctuations outside of local equilibrium: Driven diffusive systems, J. Stat. Phys., 83, 3, 385-472 (1996) · Zbl 1081.82595 |
| [27] | Shi, J.; Chen, T.; Yuan, R.; Yuan, B.; Ao, P., Relation of a new interpretation of stochastic differential equations to ito process, J. Stat. Phys., 148, 3, 579-590 (2012) · Zbl 1251.82043 |
| [28] | Ma, Y.-A.; Chen, T.; Fox, E. B., A complete recipe for stochastic gradient MCMC (2015), [math, stat] arXiv:1506.04696 |
| [29] | Barp, A.; Takao, S.; Betancourt, M.; Arnaudon, A.; Girolami, M., A unifying and canonical description of measure-preserving diffusions (2021), [math, stat]. arXiv:2105.02845 |
| [30] | Da Costa, L.; Pavliotis, G. A., The entropy production of stationary diffusions, Journal of Physics A: Mathematical and Theoretical (2023) · Zbl 07735293 |
| [31] | Ao, P., Potential in stochastic differential equations: Novel construction, J. Phys. A: Math. Gen., 37, 3, L25-L30 (2004) · Zbl 1050.60056 |
| [32] | R. Yuan, Y. Ma, B. Yuan, P. Ao, Potential Function in Dynamical Systems and the Relation with Lyapunov Function, in: Proceedings of the 30th Chinese Control Conference, 2011, pp. 6573-6580. |
| [33] | Girolami, M.; Calderhead, B., Riemann manifold Langevin and Hamiltonian Monte Carlo methods, J. R. Stat. Soc. Ser. B Stat. Methodol., 73, 2, 123-214 (2011) · Zbl 1411.62071 |
| [34] | Amari, S.-i., Natural gradient works efficiently in learning, 36 (1998) |
| [35] | Kerr, W.; Graham, A., Generalized phase space version of Langevin equations and associated Fokker-Planck equations, Eur. Phys. J. B Condens. Matter Complex Syst., 15, 2, 305-311 (2000) |
| [36] | Friston, K.; Stephan, K.; Li, B.; Daunizeau, J., Generalised Filtering, Math. Probl. Eng., 2010, 1-34 (2010) · Zbl 1189.94032 |
| [37] | Friston, K.; Da Costa, L.; Sakthivadivel, D. A.R.; Heins, C.; Pavliotis, G. A.; Ramstead, M.; Parr, T., Path integrals, particular kinds, and strange things (2022), arXiv. arXiv:2210.12761. http://dx.doi.org/10.48550/arXiv.2210.12761 |
| [38] | Lee, J. M., Introduction to Topological Manifolds (2011), Springer: Springer New York, NY · Zbl 1209.57001 |
| [39] | Parr, T.; Da Costa, L.; Friston, K., Markov blankets, information geometry and stochastic thermodynamics, Phil. Trans. R. Soc. A, 378, 2164, Article 20190159 pp. (2020) · Zbl 1462.82051 |
| [40] | Ay, N.; Jost, J.; Lê, H. V.; Schwachhöfer, L., Information geometry, Ergebnisse Der Mathematik Und Ihrer Grenzgebiete, Vol. 34 (2017), Springer International Publishing: Springer International Publishing Cham · Zbl 1383.53002 |
| [41] | Amari, S., Information Geometry and Its Applications (2016), Springer · Zbl 1350.94001 |
| [42] | Da Costa, L.; Parr, T.; Sengupta, B.; Friston, K., Neural dynamics under active inference: plausibility and efficiency of information processing, Entropy, 23, 4, 454 (2021) |
| [43] | Amari, S.-i.; Nagaoka, H., (Methods of Information Geometry. Methods of Information Geometry, Translations of Mathematical Monographs, vol. 191 (2007), American Mathematical Society) |
| [44] | Bishop, C. M., (Pattern Recognition and Machine Learning. Pattern Recognition and Machine Learning, Information Science and Statistics (2006), Springer: Springer New York) · Zbl 1107.68072 |
| [45] | Ueltzhöffer, K.; Da Costa, L.; Friston, K. J., Variational free energy, individual fitness, and population dynamics under acute stress: Comment on “Dynamic and thermodynamic models of adaptation” by Alexander N. Gorban et Al, Phys. Life Rev., 37, 111-115 (2021) |
| [46] | Jaynes, E. T., Information theory and statistical mechanics, Phys. Rev., 106, 4, 620-630 (1957) · Zbl 0084.43701 |
| [47] | Lasota, A.; MacKey, M. C., Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics (1994), Springer-Verlag · Zbl 0784.58005 |
| [48] | Beal, M. J., Variational Algorithms for Approximate Bayesian Inference (2003), University of London, (Ph.D. thesis) |
| [49] | Winn, J.; Bishop, C. M., Variational message passing, J. Mach. Learn. Res., 34 (2005) · Zbl 1222.68332 |
| [50] | Lang, C. J.G.; Kneidl, O.; Hielscher-Fastabend, M.; Heckmann, J. G., Voice recognition in aphasic and non-aphasic stroke patients, J. Neurol., 256, 8, 1303-1306 (2009) |
| [51] | Kloeden, P. E.; Platen, E., (Numerical Solution of Stochastic Differential Equations. Numerical Solution of Stochastic Differential Equations, Stochastic Modelling and Applied Probability (1992), Springer-Verlag: Springer-Verlag Berlin Heidelberg) · Zbl 0752.60043 |
| [52] | Schiff, S. J.; Sauer, T., Kalman filter control of a model of spatiotemporal cortical dynamics, J. Neural Eng., 5, 1, 1-8 (2008) |
| [53] | Hunt, B. R.; Ott, E.; Yorke, J. A., Differentiable generalized synchronization of chaos, Phys. Rev. E, 55, 4, 4029-4034 (1997) |
| [54] | Jafri, H. H.; Singh, R. K.B.; Ramaswamy, R., Generalized synchrony of coupled stochastic processes with multiplicative noise, Phys. Rev. E, 94, 5, Article 052216 pp. (2016) |
| [55] | Buendía, V.; Villegas, P.; Burioni, R.; Muñoz, M. A., The broad edge of synchronization: Griffiths effects and collective phenomena in brain networks, Phil. Trans. R. Soc. A, 380, 2227, Article 20200424 pp. (2022) |
| [56] | Friston, K. J.; Frith, C. D., Active inference, communication and hermeneutics, Cortex; a J. Devoted Study Nervous Syst. Behav., 68, 129-143 (2015) |
| [57] | Friston, K., The free-energy principle: A unified brain theory?, Nat. Rev. Neurosci., 11, 2, 127-138 (2010) |
| [58] | Buckley, C. L.; Kim, C. S.; McGregor, S.; Seth, A. K., The free energy principle for action and perception: A mathematical review, J. Math. Psych., 81, 55-79 (2017) · Zbl 1397.91535 |
| [59] | Friston, K. J.; Daunizeau, J.; Kiebel, S. J.; Sporns, O., Reinforcement learning or active inference?, PLoS One, 4, 7, Article e6421 pp. (2009) |
| [60] | Friston, K. J.; Daunizeau, J.; Kilner, J.; Kiebel, S. J., Action and behavior: A free-energy formulation, Biol. Cybernet., 102, 3, 227-260 (2010) |
| [61] | Ueltzhöffer, K., Deep active inference, Biol. Cybernet., 112, 6, 547-573 (2018), arXiv:1709.02341 · Zbl 1402.92028 |
| [62] | Koudahl, M. T.; de Vries, B., A worked example of Fokker-Planck-based active inference, (Verbelen, T.; Lanillos, P.; Buckley, C. L.; De Boom, C., Active Inference. Active Inference, Communications in Computer and Information Science (2020), Springer International Publishing: Springer International Publishing Cham), 28-34 |
| [63] | Barto, A.; Sutton, R., Reinforcement Learning: an Introduction (1992) |
| [64] | Todorov, E.; Jordan, M. I., Optimal feedback control as a theory of motor coordination, Nature Neurosci., 5, 11, 1226-1235 (2002) |
| [65] | Bossaerts, P.; Murawski, C., From behavioural economics to neuroeconomics to decision neuroscience: The ascent of biology in research on human decision making, Curr. Opin. Behav. Sci., 5, 37-42 (2015) |
| [66] | Von Neumann, J.; Morgenstern, O., (Theory of Games and Economic Behavior. Theory of Games and Economic Behavior, Theory of Games and Economic Behavior (1944), Princeton University Press: Princeton University Press Princeton, NJ, US) · Zbl 0063.05930 |
| [67] | Optican, L. M.; Richmond, B. J., Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis, J. Neurophysiol., 57, 1, 162-178 (1987) |
| [68] | Linsker, R., Perceptual neural organization: some approaches based on network models and information theory, Annu. Rev. Neurosci., 13, 1, 257-281 (1990) |
| [69] | Barlow, H. B., Possible Principles Underlying the Transformations of Sensory Messages (1961), The MIT Press |
| [70] | Friston, K.; Kilner, J.; Harrison, L., A free energy principle for the brain, J. Physiol.-Paris, 100, 1-3, 70-87 (2006) |
| [71] | Kauffman, S. A., The Origins of Order: Self-Organization and Selection in Evolution (1993), Oxford University Press |
| [72] | Ashby, W. R., Principles of the self-organizing dynamic system, J. Gen. Psychol., 37, 2, 125-128 (1947) |
| [73] | Conant, R. C.; Ashby, W. R., Every good regulator of a system must be a model of that system, Int. J. Syst. Sci., 1, 2, 89-97 (1970) · Zbl 0206.14602 |
| [74] | Bernard, C., Lectures on the Phenomena of Life Common to Animals and Plants (1974), Thomas |
| [75] | MacKay, D. J.C., Free energy minimisation algorithm for decoding and cryptanalysis, Electron. Lett., 31, 6, 446-447 (1995) |
| [76] | MacKay, D. J.C., Information Theory, Inference and Learning Algorithms (2003), Cambridge University Press: Cambridge University Press Cambridge, UK ; New York · Zbl 1055.94001 |
| [77] | Bogacz, R., A tutorial on the free-energy framework for modelling perception and learning, J. Math. Psych., 76, 198-211 (2017) · Zbl 1396.91638 |
| [78] | von Helmholtz, H.; Southall, J. P.C., Helmholtz’s Treatise on Physiological Optics (1962), Dover Publications: Dover Publications New York |
| [79] | Gregory, R. L., Perceptions as hypotheses, Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci., 290, 1038, 181-197 (1980) |
| [80] | Dayan, P.; Hinton, G. E.; Neal, R. M.; Zemel, R. S., The Helmholtz machine, Neural Comput., 7, 5, 889-904 (1995) |
| [81] | Landauer, R., Irreversibility and heat generation in the computing process, IBM J. Res. Dev., 5, 3, 183-191 (1961) · Zbl 1160.68305 |
| [82] | Wong, E.; Zakai, M., On the relation between ordinary and stochastic differential equations, Internat. J. Engrg. Sci., 3, 2, 213-229 (1965) · Zbl 0131.16401 |
| [83] | Lindquist, A.; Picci, G., Realization theory for multivariate stationary Gaussian processes, SIAM J. Control Optim., 23, 6, 809-857 (1985) · Zbl 0593.93048 |
| [84] | Mitter, S.; Picci, G.; Lindquist, A., Toward a theory of nonlinear stochastic realization, (Feedback and Synthesis of Linear and Nonlinear Systems (1981)) |
| [85] | B. Balaji, K. Friston, Bayesian State Estimation Using Generalized Coordinates, in: I. Kadar (Ed.), SPIE Defense, Security, and Sensing, Orlando, Florida, United States, 2011, p. 80501Y, http://dx.doi.org/10.1117/12.883513. |
| [86] | Biscay, R.; Jimenez, J. C.; Riera, J. J.; Valdes, P. A., Local Linearization method for the numerical solution of stochastic differential equations, Ann. Inst. Statist. Math., 48, 4, 631-644 (1996) · Zbl 1002.60545 |
| [87] | Cox, D. R.; Miller, H. D., The Theory of Stochastic Processes (1977), Chapman and Hall/CRC: Chapman and Hall/CRC London · Zbl 0359.60004 |
| [88] | Parr, T.; Pezzulo, G.; Friston, K. J., Active Inference: The Free Energy Principle in Mind, Brain, and Behavior (2022), MIT Press: MIT Press Cambridge, MA, USA |
| [89] | Friston, K.; Mattout, J.; Trujillo-Barreto, N.; Ashburner, J.; Penny, W., Variational free energy and the Laplace approximation, NeuroImage, 34, 1, 220-234 (2007) |
| [90] | Friston, K. J., Variational filtering, NeuroImage, 41, 3, 747-766 (2008) |
| [91] | Friston, K. J.; Trujillo-Barreto, N.; Daunizeau, J., DEM: A variational treatment of dynamic systems, NeuroImage, 41, 3, 849-885 (2008) |
| [92] | Loeliger, H.-A., Least squares and Kalman filtering on forney graphs, (Blahut, R. E.; Koetter, R., Codes, Graphs, and Systems: a Celebration of the Life and Career of G. David Forney, Jr. on the Occasion of His Sixtieth Birthday. Codes, Graphs, and Systems: a Celebration of the Life and Career of G. David Forney, Jr. on the Occasion of His Sixtieth Birthday, The Kluwer International Series in Engineering and Computer Science (2002), Springer US: Springer US Boston, MA), 113-135 · Zbl 1147.93414 |
| [93] | Kappen, H. J., Path integrals and symmetry breaking for optimal control theory, J. Stat. Mech. Theory Exp., 2005, 11, P11011 (2005) · Zbl 1456.49027 |
| [94] | Todorov, E., General duality between optimal control and estimation, (2008 47th IEEE Conference on Decision and Control (2008)), 4286-4292 |
| [95] | van den Broek, B.; Wiegerinck, W.; Kappen, B., Risk sensitive path integral control, UAI (2010) |
| [96] | Friston, K., Life as we know it, J. R. Soc. Interface, 10, 86, Article 20130475 pp. (2013) |
| [97] | Friston, K.; Mattout, J.; Kilner, J., Action understanding and active inference, Biol. Cybernet., 104, 1-2, 137-160 (2011) · Zbl 1232.92036 |
| [98] | Feldman, A. G., New insights into action-perception coupling, Exp. Brain Res., 194, 1, 39-58 (2009) |
| [99] | Mansell, W., Control of perception should be operationalized as a fundamental property of the nervous system, Top. Cogn. Sci., 3, 2, 257-261 (2011) |
| [100] | Toutounji, H.; Pipa, G., Spatiotemporal computations of an excitable and plastic brain: neuronal plasticity leads to noise-robust and noise-constructive computations, PLoS Comput. Biol., 10, 3, Article e1003512 pp. (2014) |
| [101] | Gallese, V.; Goldman, A., Mirror neurons and the simulation theory of mind-reading, Trends in Cognitive Sciences, 2, 12, 493-501 (1998) |
| [102] | Rizzolatti, G.; Craighero, L., The mirror-neuron system, Annu. Rev. Neurosci., 27, 169-192 (2004) |
| [103] | Kilner, J. M.; Friston, K. J.; Frith, C. D., Predictive coding: An account of the mirror neuron system, Cogn. Process., 8, 3, 159-166 (2007) |
| [104] | Dauwels, J., On variational message passing on factor graphs, (2007 IEEE International Symposium on Information Theory (2007), IEEE: IEEE Nice), 2546-2550 |
| [105] | Feynman, R., Statistical Mechanics: a Set of Lectures (1998), Westview Press: Westview Press Boulder, Colo · Zbl 0997.82500 |
| [106] | Andres, D., On the motion of spikes: turbulent-like neuronal activity in the human Basal Ganglia, Front. Hum. Neurosci., 12 (2018) |
| [107] | Deco, G.; Kringelbach, M. L., Turbulent-like Dynamics in the Human Brain, Cell Rep., 33, 10, Article 108471 pp. (2020) |
| [108] | Lopes da Silva, F., Neural mechanisms underlying brain waves: From neural membranes to networks, Electroencephalogr. Clin. Neurophysiol., 79, 2, 81-93 (1991) |
| [109] | Kopell, N.; Whittington, M. A.; Kramer, M. A., Neuronal assembly dynamics in the Beta1 frequency range permits short-term memory, Proc. Natl. Acad. Sci., 108, 9, 3779-3784 (2011) |
| [110] | Arnal, L. H.; Giraud, A.-L., Cortical oscillations and sensory predictions, Trends in Cognitive Sciences, 16, 7, 390-398 (2012) |
| [111] | Buzsáki, G.; Logothetis, N.; Singer, W., Scaling brain size, keeping timing: Evolutionary preservation of brain rhythms, Neuron, 80, 3, 751-764 (2013) |
| [112] | Ott, E.; Grebogi, C.; Yorke, J. A., Controlling chaos, Phys. Rev. Lett., 64, 11, 1196-1199 (1990) · Zbl 0964.37501 |
| [113] | Hwang, C.-R.; Hwang-Ma, S.-Y.; Sheu, S.-J., Accelerating diffusions, Ann. Appl. Probab., 15, 2, 1433-1444 (2005) · Zbl 1069.60065 |
| [114] | Hwang, C.-R.; Hwang-Ma, S.-Y.; Sheu, S.-J., Accelerating Gaussian diffusions, Ann. Appl. Probab., 3, 3, 897-913 (1993) · Zbl 0780.60074 |
| [115] | Lelièvre, T.; Nier, F.; Pavliotis, G. A., Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion, J. Stat. Phys., 152, 2, 237-274 (2013), arXiv:1212.0876 · Zbl 1276.82042 |
| [116] | Aslimani, N.; Ellaia, R., A new hybrid algorithm combining a new chaos optimization approach with gradient descent for high dimensional optimization problems, Comput. Appl. Math., 37, 3, 2460-2488 (2018) · Zbl 1409.90147 |
| [117] | Gross, J.; Hoogenboom, N.; Thut, G.; Schyns, P.; Panzeri, S.; Belin, P.; Garrod, S., Speech rhythms and multiplexed oscillatory sensory coding in the human brain, PLoS Biol., 11, 12, Article e1001752 pp. (2013) |
| [118] | Buzsáki, G.; Draguhn, A., Neuronal oscillations in cortical networks, Science, 304, 5679, 1926-1929 (2004) |
| [119] | Csikszentmihalyi, M., Flow: The Psychology of Optimal Experience (2008), HarperCollins e-books |
| [120] | H. Attias, Planning by probabilistic inference, in: 9th Int. Workshop on Artificial Intelligence and Statistics, 2003, p. 8. |
| [121] | Botvinick, M.; Toussaint, M., Planning as inference, Trends in Cognitive Sciences, 16, 10, 485-488 (2012) |
| [122] | Kaplan, R.; Friston, K. J., Planning and navigation as active inference, Biol. Cybernet., 112, 4, 323-343 (2018) · Zbl 1400.92607 |
| [123] | Friston, K.; FitzGerald, T.; Rigoli, F.; Schwartenbeck, P.; Pezzulo, G., Active inference: a process theory, Neural Comput., 29, 1, 1-49 (2017) · Zbl 1414.92092 |
| [124] | Da Costa, L.; Parr, T.; Sajid, N.; Veselic, S.; Neacsu, V.; Friston, K., Active inference on discrete state-spaces: A synthesis, J. Math. Psych., 99, Article 102447 pp. (2020) · Zbl 1455.91190 |
| [125] | Barp, A.; Da Costa, L.; França, G.; Friston, K.; Girolami, M.; Jordan, M. I.; Pavliotis, G. A., Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents, Vol. 46, 21-78 (2022), arXiv:2203.10592 · Zbl 1524.62021 |
| [126] | Kahneman, D.; Tversky, A., Prospect theory: an analysis of decision under risk, Econometrica, 47, 2, 263-291 (1979), arXiv:1914185 · Zbl 0411.90012 |
| [127] | Levine, S., Reinforcement learning and control as probabilistic inference: tutorial and review (2018), [cs, stat]. arXiv:1805.00909 |
| [128] | K. Rawlik, M. Toussaint, S. Vijayakumar, On stochastic optimal control and reinforcement learning by approximate inference, in: Twenty-Third International Joint Conference on Artificial Intelligence, 2013. |
| [129] | Toussaint, M., Robot trajectory optimization using approximate inference, (Proceedings of the 26th Annual International Conference on Machine Learning. Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09 (2009), Association for Computing Machinery: Association for Computing Machinery Montreal, Quebec, Canada), 1049-1056 |
| [130] | Kalman, R. E., A new approach to linear filtering and prediction problems, J. Basic Eng., 82, 1, 35-45 (1960) |
| [131] | Kappen, H. J.; Gómez, V.; Opper, M., Optimal control as a graphical model inference problem, Mach. Learn., 87, 2, 159-182 (2012), arXiv:0901.0633 · Zbl 1243.93133 |
| [132] | Ziebart, B., Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy (2010), Carnegie Mellon University: Carnegie Mellon University Pittsburgh, (Ph.D. thesis) |
| [133] | Klyubin, A. S.; Polani, D.; Nehaniv, C. L., Keep your options open: An information-based driving principle for sensorimotor systems, PLoS One, 3, 12, Article e4018 pp. (2008) |
| [134] | Lindley, D. V., On a measure of the information provided by an experiment, Ann. Math. Stat., 27, 4, 986-1005 (1956), arXiv:2237191 · Zbl 0073.14103 |
| [135] | MacKay, D. J.C., Information-based objective functions for active data selection, Neural Comput., 4, 4, 590-604 (1992) |
| [136] | Oudeyer, P.-Y.; Kaplan, F., What is intrinsic motivation? A typology of computational approaches, Front. Neurorobot., 1, 6 (2007) |
| [137] | Schmidhuber, J., Formal theory of creativity, fun, and intrinsic motivation (1990-2010), IEEE Trans. Auton. Ment. Dev., 2, 3, 230-247 (2010) |
| [138] | Barto, A.; Mirolli, M.; Baldassarre, G., Novelty or surprise?, Front. Psychol., 4 (2013) |
| [139] | Sun, Y.; Gomez, F.; Schmidhuber, J., Planning to be surprised: optimal Bayesian exploration in dynamic environments (2011), [cs, stat]. arXiv:1103.5708 |
| [140] | Deci, E.; Ryan, R. M., Intrinsic Motivation and Self-Determination in Human Behavior, Perspectives in Social Psychology (1985), Springer US: Springer US New York |
| [141] | Itti, L.; Baldi, P., Bayesian surprise attracts human attention, Vis. Res., 49, 10, 1295-1306 (2009) |
| [142] | Parr, T.; Sajid, N.; Da Costa, L.; Mirza, M. B.; Friston, K. J., Generative models for active vision, Front. Neurorobot., 15 (2021) |
| [143] | Bellman, R. E., Dynamic Programming (1957), Princeton University Press: Princeton University Press Princeton, NJ, US · Zbl 0077.13605 |
| [144] | Åström, K. J., Optimal control of Markov processes with incomplete state information, J. Math. Anal. Appl., 10, 1, 174-205 (1965) · Zbl 0137.35803 |
| [145] | Berger, J. O., (Statistical Decision Theory and Bayesian Analysis. Statistical Decision Theory and Bayesian Analysis, Springer Series in Statistics (1985), Springer-Verlag: Springer-Verlag New York) · Zbl 0572.62008 |
| [146] | Parr, T.; Friston, K. J., Generalised free energy and active inference, Biol. Cybernet., 113, 5, 495-513 (2019) · Zbl 1425.91384 |
| [147] | Da Costa, L.; Sajid, N.; Parr, T.; Friston, K.; Smith, R., Reward maximization through discrete active inference, Neural Comput., 35, 5, 807-852 (2023) · Zbl 1520.91292 |
| [148] | Friston, K.; Da Costa, L.; Hafner, D.; Hesp, C.; Parr, T., Sophisticated inference, Neural Comput., 33, 3, 713-763 (2021) · Zbl 1469.91023 |
| [149] | Sajid, N.; Da Costa, L.; Parr, T.; Friston, K., Active inference, Bayesian optimal design, and expected utility, (The Drive for Knowledge: the Science of Human Information Seeking (2022)) |
| [150] | Hafner, D.; Ortega, P. A.; Ba, J.; Parr, T.; Friston, K.; Heess, N., Action and perception as divergence minimization (2020), [cs, math, stat]. arXiv:2009.01791 |
| [151] | Millidge, B.; Tschantz, A.; Seth, A. K.; Buckley, C. L., On the relationship between active inference and control as inference, (Verbelen, T.; Lanillos, P.; Buckley, C. L.; De Boom, C., Active Inference. Active Inference, Communications in Computer and Information Science (2020), Springer International Publishing: Springer International Publishing Cham), 3-11 |
| [152] | FitzGerald, T. H.B.; Schwartenbeck, P.; Moutoussis, M.; Dolan, R. J.; Friston, K., Active inference, evidence accumulation, and the urn task, Neural Comput., 27, 2, 306-328 (2015) · Zbl 1414.92091 |
| [153] | Schwartenbeck, P.; FitzGerald, T. H.B.; Mathys, C.; Dolan, R.; Kronbichler, M.; Friston, K., Evidence for surprise minimization over value maximization in choice behavior, Sci. Rep., 5, 16575 (2015) |
| [154] | Sajid, N.; Ball, P. J.; Parr, T.; Friston, K. J., Active inference: demystified and compared, Neural Comput., 33, 3, 674-712 (2021), arXiv:1909.10863 · Zbl 1520.68156 |
| [155] | Friston, K. J.; Rosch, R.; Parr, T.; Price, C.; Bowman, H., Deep temporal models and active inference, Neurosci. Biobehav. Rev., 90, 486-501 (2018) |
| [156] | Mirza, M. B.; Adams, R. A.; Mathys, C. D.; Friston, K. J., Scene construction, visual foraging, and active inference, Front. Comput. Neurosci., 10 (2016) |
| [157] | Bruineberg, J.; Rietveld, E., Self-organization, free energy minimization, and optimal grip on a field of affordances, Front. Hum. Neurosci., 8 (2014) |
| [158] | Friston, K. J.; Parr, T.; de Vries, B., The graphical brain: Belief propagation and active inference, Netw. Neurosci., 1, 4, 381-414 (2017) |
| [159] | Smith, R.; Friston, K. J.; Whyte, C. J., A step-by-step tutorial on active inference and its application to empirical data, J. Math. Psych., 107, Article 102632 pp. (2022) · Zbl 1484.91352 |
| [160] | Yedidia, J.; Freeman, W.; Weiss, Y., Constructing free-energy approximations and generalized belief propagation algorithms, IEEE Trans. Inform. Theory, 51, 7, 2282-2312 (2005) · Zbl 1283.94023 |
| [161] | Parr, T.; Markovic, D.; Kiebel, S. J.; Friston, K. J., Neuronal message passing using Mean-field, Bethe, and Marginal approximations, Sci. Rep., 9, 1, 1889 (2019) |
| [162] | Friston, K.; Adams, R.; Perrinet, L.; Breakspear, M., Perceptions as Hypotheses: Saccades as experiments, Front. Psychol., 3 (2012) |
| [163] | Balietti, S.; Klein, B.; Riedl, C., Optimal design of experiments to identify latent behavioral types, Exp. Econ., 24, 3, 772-799 (2021), arXiv:1807.07024 |
| [164] | Wald, A., An essentially complete class of admissible decision functions, Ann. Math. Stat., 18, 4, 549-555 (1947) · Zbl 0029.30604 |
| [165] | Brown, L. D., A complete class theorem for statistical problems with finite sample spaces, Ann. Statist., 9, 6, 1289-1300 (1981) · Zbl 0476.62006 |
| [166] | Yildiz, I. B.; von Kriegstein, K.; Kiebel, S. J., From birdsong to human speech recognition: Bayesian inference on a hierarchy of nonlinear dynamical systems, PLoS Comput. Biol., 9, 9, Article e1003219 pp. (2013) |
| [167] | Haken, H.; Portugali, J., Information and selforganization: A unifying approach and applications, Entropy, 18, 6, 197 (2016) |
| [168] | Prigogine, I., Time, structure, and fluctuations, Science, 201, 4358, 777-785 (1978) |
| [169] | Sakthivadivel, D. A.R., Entropy-maximising diffusions satisfy a parallel transport law (2023), arXiv. arXiv:2203.08119. http://dx.doi.org/10.48550/arXiv.2203.08119 |
| [170] | Klyubin, A.; Polani, D.; Nehaniv, C., Empowerment: A universal agent-centric measure of control, (2005 IEEE Congress on Evolutionary Computation, Vol. 1 (2005)), 128-135 |
| [171] | Tishby, N.; Pereira, F. C.; Bialek, W., The information bottleneck method (2000), arXiv |
| [172] | Still, S.; Precup, D., An information-theoretic approach to curiosity-driven reinforcement learning, Theory Biosci.=Theorie in Den Biowissenschaften, 131, 3, 139-148 (2012) |
| [173] | Still, S.; Sivak, D. A.; Bell, A. J.; Crooks, G. E., Thermodynamics of prediction, Phys. Rev. Lett., 109, 12, Article 120604 pp. (2012) |
| [174] | Botvinick, M. M.; Niv, Y.; Barto, A. G., Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective, Cognition, 113, 3, 262-280 (2009) |
| [175] | Manicka, S.; Levin, M., Modeling somatic computation with non-neural bioelectric networks, Sci. Rep., 9, 1, 18612 (2019) |
| [176] | Palacios, E. R.; Razi, A.; Parr, T.; Kirchhoff, M.; Friston, K., On Markov blankets and hierarchical self-organisation, J. Theoret. Biol., 486, Article 110089 pp. (2020) · Zbl 1429.92037 |
| [177] | Kelso, J. A.S., (Dynamic Patterns: the Self-Organization of Brain and Behavior. Dynamic Patterns: the Self-Organization of Brain and Behavior, Complex Adaptive Systems (1995), A Bradford Book: A Bradford Book Cambridge, MA, USA) |
| [178] | Deco, G.; Jirsa, V. K.; Robinson, P. A.; Breakspear, M.; Friston, K.; Sporns, O., The dynamic brain: from spiking neurons to neural masses and cortical fields, PLoS Comput. Biol., 4, 8, Article e1000092 pp. (2008) |
| [179] | Kelso, J. A.S., Unifying Large- and small-scale theories of coordination, Entropy, 23, 5, 537 (2021) |
| [180] | Friston, K.; Levin, M.; Sengupta, B.; Pezzulo, G., Knowing one’s place: A free-energy approach to pattern regulation, J. R. Soc. Interface, 12, 105, Article 20141383 pp. (2015) |
| [181] | Ramstead, M. J.D.; Badcock, P. B.; Friston, K. J., Answering Schrödinger’s question: A free-energy formulation, Phys. Life Rev., 24, 1-16 (2018) |
| [182] | Tschantz, A.; Baltieri, M.; Seth, A. K.; Buckley, C. L., Scaling active inference (2019), [cs, eess, math, stat]. arXiv:1911.10601 |
| [183] | Barp, A.; Da Costa, L.; França, G.; Friston, K.; Girolami, M.; Jordan, M. I.; Pavliotis, G. A., Geometric methods for sampling, optimisation, inference and adaptive agents, (Geometry and Statistics, No. 46. Geometry and Statistics, No. 46, Handbook of Statistics (2022), Academic Press), 21-78 · Zbl 1524.62021 |
| [184] | Fountas, Z.; Sajid, N.; Mediano, P. A.M.; Friston, K., Deep active inference agents using Monte-Carlo methods (2020), [cs, q-bio, stat]. arXiv:2006.04176 |
| [185] | Çatal, O.; Verbelen, T.; Van de Maele, T.; Dhoedt, B.; Safron, A., Robot navigation as hierarchical active inference, Neural Netw., 142, 192-204 (2021) · Zbl 1521.68222 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
