{"id":"arxiv:2302.11681","kind":"article","title":"Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding","authors":["Zhen-Ye Huang","Ruyi Zhou","Miao Huang","Hai-Jun Zhou"],"author_count":4,"year":2023,"published":"2023-02-22","venue":"Science China Physics, Mechanics Astronomy 67, 260511 (2024)","cited_by":6,"doi":"10.1007/s11433-024-2341-2","arxiv_id":"2302.11681","nct_id":null,"source_url":"https://arxiv.org/abs/2302.11681","archive_url":"https://space-kitty.com/papers/arxiv:2302.11681","updated":"2024-01-11","pmid":null,"pmcid":null,"openalex_id":"W4396929189","pdf_url":"https://arxiv.org/pdf/2302.11681","abstract":"Lateral predictive coding is a recurrent neural network which creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here we investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding analytically and by numerical minimization of a free energy. We observe several phase transitions in the synaptic weight matrix, especially a continuous transition which breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal-gas law in an extended temperature range and saturates the efficiency upper-bound of energy utilization. These results bring theoretical insights on the emergence and evolution of complex internal models in predictive processing systems.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["q-bio.NC","cond-mat.dis-nn"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"neuroai","name":"NeuroAI & Computational Neuroscience","domain_slug":"neuroscience"}],"cited_in_dossiers":[]}