Recurrent Neural Networks

Dr. Salem’s current research interests include: Neural Networks and Learning Systems, Blind Signal Deconvolution and Extraction, Dynamical Systems and Chaos, Integrated CMOS Sensing and Processing. He was the Chairman of the IEEE Technical Committee on Real-Time Control Computing and Signal Processing (1994–1996). He was the Chairman of the CAS Technical Committee on Neural Systems and Their Applications (1997–1998). He served on the IEEE Neural Network Council (1999–2000), and was the first Vice President of the IEEE Neural Network Council for Technical Activities (1999–2001). He was the Guest Co-Editor of the IEEE-CAS Special Issue on Bifurcations and Chaos in Circuits and Systems July 1988 (with T. Matsumoto), the Special Issue on Micro-Electronic Hardware Implementation of Soft Computing: Neural and Fuzzy Networks with Learning, Journal of Computers and Electrical Engineering, July 1999 (with T. Yamakawa), and the Special Issue on Digital and Analog Arrays, in the Journal of Circuits, Systems, and Computers, August 1999 (with M. Ahmadi). He was the recipient of the IEEE CAS Golden Jubilee Award (1999), the IEEE Third Millennium Award (2000), and The CAS Darlington Best Paper Award (2001).With a team of students, he also received the U.S. Semiconductor Research Corporation (SRC) Phase II Finalist Award (2000). He was a Distinguished Lecturer of the IEEE CAS Society in 2000–2001. He was an Associate Editor and Guest Editor for numerous IEEE and other transactions including the IEEE Circuits and Systems, IEEE Neural Networks, the Journal of Circuits, Systems, and Computers, and the Journal of Computer and Electrical Engineering. He was the Chairman of the Engineering Foundation Conference on Qualitative Methods for Nonlinear Dynamics. He served in several capacities in several conferences including the General Chair of the IEEE Midwest Symposium on Circuits and Systems in Lansing, MI, in 2000 and also in 2021.   He was a Visiting Professor at UC, Berkeley (1983), the California Institute of Technology, Pasadena (1992), and the University of Minnesota, Twin Cities (1993). He joined MSU in 1985 and has been a Professor since1991. He has worked and consulted for several companies including General Motors, Ford, Smith’s Industries, Intersignal, IC Tech Inc., and Clarity LLC. He has authored more than 250 technical papers, and co-edited the textbook (Dynamical Systems Approaches to Nonlinear Problems in Circuits and Systems, (SIAM, 1988). He is a co-inventor of more than 14 patents on adaptive nonlinear signal processing, neural networks, and sensors.

Introduction.- 1. Network Architectures.- 2. Learning Processes.- 3. Recurrent Neural Networks (RNN).- 4. Gated RNN: The Long Short-Term Memory (LSTM) RNN.- 5. Gated RNN: The Gated Recurrent Unit (GRU) RNN.- 6. Gated RNN: The Minimal Gated Unit (MGU) RNN.