Vanilla Neural Networks

A neural network is a mathematical function of the form y = FNN(x), composed of nested layers where each layer f(x) = g(Wx + b) uses a weight matrix W, bias vector b, and activation function g, with parameters learned via gradient descent.

Vanilla Neural Networks A neural network is just a math function: y = FNN x . FNN has a nested form. Think of it as a stack of layers. A 3-layer neural network that returns a scalar value looks like this: y = FNN x = f₃ f₂ f₁ x Each f — f₁ , f₂ , … fₙ — has the same form: f x = g Wx + b W the weight matrix and b a bias vector are the learned parameters, usually trained via gradient descent. g is the activation function, and it can be chosen differently for each layer.