Introduction

A network learns features the earlier chapters had to specify by hand, and there is no linear-algebra page and no classical computer-vision page in this book. After this chapter the reader can follow a net from the gradient to the architecture and can place vision, audio, tabular, and sequence nets on that map. The chapter goes in that order: first how a net is trained, then the architectures built on that training, then the modalities those architectures are applied to.

Training

Training starts with the gradient, so Calculus comes first, as the starting point for the matrix calculus used when differentiating through neural nets. Game Theory follows, for game theory as it shows up in computer science and AI. With those in hand, Deep Neural Nets Basics covers the building blocks of deep nets (perceptrons, multilayer networks, activations, and loss) and how training runs, from gradient descent, batch size, learning rate, and optimizers to batch normalization, dropout, and reading train/val accuracy. Once a net trains, Deep Network Optimization is about making it smaller or more teachable through pruning, knowledge distillation, and lottery tickets, and about debugging nets that fail to train. Deep Neural Frameworks closes the part with the libraries used in practice, PyTorch, fast.ai, and Keras, and the NVIDIA CUDA stack that runs them on a GPU.

Architectures

The same training loop drives very different shapes of network. Autoencoders are the reconstruction setups used for dimensionality reduction, with variational autoencoders after the basics. Convolutional Nets are the nets for vision-style feature learning, with capsule nets, transfer learning, and visualization of what they learn. Recurrent Nets handle sequences and time series with back propagation, LSTM and GRU cells, and related variants. The Attention notes have moved on to the Transformers and pretrained language model pages, and Transformers collects the illustrated guides and papers on the Transformer architecture and its family of variants. Bayesian Neural Nets add prediction uncertainty to a net. Graph Neural Nets cover graph convolutional networks and node-embedding methods such as DeepWalk and Node2vec. Siamese Nets learn a similarity between inputs for one-shot learning rather than a flat class label. Other Architectures gathers the families beyond standard deep nets: self-organizing maps, neuro-evolution, RBFNs, and signal-processing networks. Representations is the embedding page, from entity embeddings and *2vec techniques to vector similarity search. Meta Learning (AutoML) ends the part by automating model choice, hyperparameters, and architecture search.

Modalities

The architectures then meet a kind of data. Deep Neural Machine Vision covers detection, recognition, and segmentation in machine vision. Deep Neural Audio collects models for speech, sound classification, and related audio tasks. Deep Neural Tabular points at TabNet and at surveys of deep learning on tabular data. Deep Neural Time Series points at NeuralProphet for deep neural time-series forecasting.