Multi-layer perceptron (MLP)

Multi-layer perceptron (MLP)

In MLP each node is connected to every node in the next layer.Nodes in the hidden and output layers are connected to a bias node (feeding a constant value). Input data are weighted, then added up and finally transformed using an activation function. 

 Figure 8‑16 MLP 3-5-2 with bias nodes

When each node in the network is connected to every node in the next layer by a feedforward link, it is commonly referred to as a multi-level perceptron (MLP, Figure 8‑16). The perceptron part of this description is a reference to its early development in a simpler form as a model of neuron triggering in the retina of an eye.

By convention all nodes in the hidden layer and the output layer are also connected to a bias node. This feeds a constant input value, 1, into the set of weights, and acts in a similar manner to the constant term in a regression model. Bias node weights are treated in the same manner as per other node weights. In the MLP, with 1 bias node associated with each feedforward layer, n input nodes, one hidden layer with m hidden nodes and p output nodes, the W and Z weight matrices have dimensions (n+1)x(m) and (m+1)x(p). The MLP then has the architecture shown in Figure 8‑16. The effective number of parameters, λ, for such a network is the sum of the number of feedforward connections (including the bias connections), so in this example λ=20+12=32.

Input data are weighted according to their wij values and combined (typically summed) in the hidden layer. This weighted sum is then modified by what is known as an activation function. This two-step process of summation of inputs and then modification of this sum by an activation function, g, to create the output value can be illustrated at the node level as shown in Figure 8‑17.

Figure 8‑17 ANN hidden node structure

How to

ANNs: Training/ Learning

Backpropagation training algorithm:

Forward phase – the inputs are presented to the network

Backpropagation phase – the outputs are compared with the targets and the weights are adjusted

Neural Network Learning problem

• Adjusting the connection weights so that the network generates
• the correct prediction on the training data.
• Start with random weights for all the connections in the neural network.
• Input data is fed into the input layer.
• The data is passed through the network layer by layer.
• Each neuron computes a weighted sum of its inputs and applies an activation function to produce its output.
• The outputs of one layer become the inputs to the next layer.
• This process continues until the final output layer produces the network's prediction.
• Compare the network’s prediction with the actual target values (labels) using a loss
• function.
• Adjust the weights in the direction that reduces the loss.
• Repeat for multiple iterations (epochs) until the loss converges to a minimum value.