Abstract: In this paper, the nonlinear dynamical behaviors of neural network systems, its relations with the function of the networks, and the applications in science and engineering are summarized. The mechanisms of two kinds of dynamical processes, weight dynamics and activation dynamics in neural networks, the stabilities and robust stabilities, the properties, tensor descriptions, attractivities and complexities of high order correlation neural networks, the bifurcation theory of design and synthesis of neural ne...