Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- 1Intelligence in Biological Systems
- 2Principles of Artificial Neural Networks
- 3Artificial Synapses
- 4Artificial Neurons
- 5Examples of Applications in Artificial Neural Networks
- 6System Design
- 7Neuromorphic Algorithms
- 8Lifelong Learning with AI Algorithms and Hardware
- 9Practice Problems
- Index
- References
2 - Principles of Artificial Neural Networks
Published online by Cambridge University Press: 01 January 2026
Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- 1Intelligence in Biological Systems
- 2Principles of Artificial Neural Networks
- 3Artificial Synapses
- 4Artificial Neurons
- 5Examples of Applications in Artificial Neural Networks
- 6System Design
- 7Neuromorphic Algorithms
- 8Lifelong Learning with AI Algorithms and Hardware
- 9Practice Problems
- Index
- References
Summary
The chapter introduces fundamental principles of deep learning. We discuss supervised learning of feedforward neural networks by considering a binary classification problem. Gradient descent techniques and backpropagation learning algorithms are introduced as means of training neural networks. The impact of neuron activations and convolutional and residual network architectures on the learning performance are discussed. Finally, regularization techniques such as batch normalization and dropout are introduced for improving the accuracy of trained models. The chapter is essential to connect advances in conventional deep learning algorithms to neuromorphic concepts.
Keywords
Information
- Type
- Chapter
- Information
- Introduction to Neuromorphic Computing , pp. 15 - 26Publisher: Cambridge University PressPrint publication year: 2026
References
Li, Z., Liu, F., Yang, W., Peng, S. and Zhou, J., 2021. A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 33(12), pp. 6999–7019.Google Scholar
Devlin, J., 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.Google Scholar
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S. and Avila, R., 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774.Google Scholar
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. and Polosukhin, I., 2017. Attention is all you need. Advances in Neural Information Processing Systems, 30, pp. 1–11.Google Scholar
Jospin, L. V., Laga, H., Boussaid, F., Buntine, W. and Bennamoun, M., 2022. Hands-on Bayesian neural networks: A tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2), pp. 29–48.10.1109/MCI.2022.3155327CrossRefGoogle Scholar
Coursera Deep Learning Specialization Course, www.coursera.org/specializations/deep-learning.Google Scholar
Amari, S. I., 1993. Backpropagation and stochastic gradient descent method. Neurocomputing, 5(4–5), pp. 185–196.10.1016/0925-2312(93)90006-OCrossRefGoogle Scholar
Nair, V. and Hinton, G. E., 2010. Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10) (pp. 807–814).Google Scholar
Maas, A. L., Hannun, A. Y. and Ng, A. Y., 2013, June. Rectifier nonlinearities improve neural network acoustic models. In Proceedings of the ICML (Vol. 30, No. 1, p. 3).Google Scholar
Hendrycks, D. and Gimpel, K., 2016. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415.Google Scholar
LeCun, Y., Bottou, L., Bengio, Y. and Haffner, P., 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), pp. 2278–2324.10.1109/5.726791CrossRefGoogle Scholar
Krizhevsky, A., Sutskever, I. and Hinton, G. E., 2017. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), pp. 84–90.10.1145/3065386CrossRefGoogle Scholar
Simonyan, K. and Zisserman, A., 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.Google Scholar
Deng, L., 2012. The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6), pp. 141–142.Google Scholar
Krizhevsky, A. and Hinton, G., 2009. Learning multiple layers of features from tiny images. www.cs.utoronto.ca/~kriz/learning-features-2009-TR.pdf.Google Scholar
Deng, J., Dong, W., Socher, R., Li, L. J., Li, K. and Fei-Fei, L., 2009, June. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (pp. 248–255). IEEE.10.1109/CVPR.2009.5206848CrossRefGoogle Scholar
He, K., Zhang, X., Ren, S. and Sun, J., 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).10.1109/CVPR.2016.90CrossRefGoogle Scholar
Bjorck, N., Gomes, C. P., Selman, B. and Weinberger, K. Q., 2018. Understanding batch normalization. Advances in Neural Information Processing Systems, 31, pp. 1–12.Google Scholar
Ioffe, S., 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167.Google Scholar
Accessibility standard: WCAG 2.2 A
Why this information is here
This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.Accessibility Information
The PDF of this chapter complies with version 2.2 of the
Web Content Accessibility Guidelines
(WCAG), offering more comprehensive accessibility measures for a broad range
of users and meets the basic
(A)
level of WCAG compliance, addressing essential accessibility barriers.
Content Navigation
Table of contents navigation
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Reading Order & Textual Equivalents
Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
Short alternative textual descriptions
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.