Complex systems with many degrees of freedom and/or many interacting units are ubiquitous in nature and artificial systems ranging from a flock of birds to living cells to artificial neural networks. These complex systems exhibit fascinating collective behaviors (e.g., flocking of bird, swarming of bacterial cells); carry out essential biological functions (e.g., sensory adaptation, cellular motility, and biochemical oscillations for time keeping); and perform memory and learning tasks at or near human level accuracy (e.g., associative memory and deep learning in artificial neural networks). However, most of these complex systems operate in a regime far out of equilibrium in which equilibrium statistical mechanics fails to describe even their steady state properties. Therefore, understanding behaviors of these complex systems presents both a major challenge and a great opportunity for developing a theoretical framework for studying highly non-equilibrium systems including active matter, living cells, and artificial neural networks. In these lectures, we will present some of our original works in understanding collective behaviors in active matter, information processing in noisy cellular biological systems, and learning dynamics in artificial neural networks. Specific topics include:
1) Flocking theory for collective phenomenon in active matter.
2) Information processing and energy-accuracy tradeoff in biological systems.
3) Learning dynamics and generalization in deep neural networks.

• "Stochastic Processes in Physics and Chemistry", N. G. Van Kampen, first 5 chapters

2) Flocking theory.

• "Long Range Order in a 2-dimensional Dynamical XY Model: How Birds Fly Together", J. Toner and Yuhai Tu, Phys. Rev. Lett. (PRL), 75(23), 4326 (1995).

• "Flocks, herds, and schools: A quantitative theory of flocking", J. Toner, Yuhai Tu, Phys. Rev. E, 58(4), 4828(1998).

• **"Hydrodynamics and phases of flocks", John Toner, Yuhai Tu, Sriram Ramaswamy, Annals of Physics 318(1):170-244 (2005).

3) Information processing in biological systems: mechanisms and cost-accuracy tradeoff

• **"Quantitative Modeling of Bacterial Chemotaxis: Signal Amplification and Accurate Adaptation", Yuhai Tu, Annu. Rev. Biophys. 2013. 42: 337-59.

• **"Information Processing in Bacteria: Memory, Computation, and Statistical Physics: A Key Issues Review", Ganhui Lan, Yuhai Tu, Reports on Progress in Physics, 79(5), pp. 52601-52617, 2016.

• **"Adaptation in Living Systems", Yuhai Tu and Wouter-Jan Rappel, Annu. Rev. Condens. Matter Phys. 2018.9:183-205, 2018.

• "The energy-speed-accuracy trade-off in sensory adaptation", G. Lan, P. Sartori, S. Neumann, V. Sourjik, and Yuhai Tu, Nature Physics 8, 422-€“428, 2012.

• "The free-energy cost of accurate biochemical oscillations", Yuansheng Cao, Hongli Wang, Qi Ouyang, and Yuhai Tu, Nature Physics, 11, 772, 2015.

• "The energy cost and optimal design for synchronization of coupled molecular oscillators", D. Zhang, Y. Cao, Q. Ouyang, Y. Tu, Nature Physics 16, 95-100 (2020).

4) Machine learning

• "The inverse variance-flatness relation in Stochastic-Gradient-Descent is critical for finding flat minima", Y. Feng and Y. Tu, PNAS, 118 (9), 2021.

• "Phases of learning dynamics in artificial neural networks: in the absence and presence of mislabeled data", Y. Feng and Y. Tu, Machine Learning: Science and Technology (MLST), July 19, 2021.

** Review papers