Table of Contents
Neural networks are widely used in image recognition tasks due to their ability to learn complex patterns. This case study explores the implementation process, including detailed calculations involved in training a neural network for image classification.
Designing the Neural Network
The network consists of an input layer, one hidden layer, and an output layer. For simplicity, consider a small dataset with images represented by 4 pixels each. The input layer has 4 neurons, the hidden layer has 3 neurons, and the output layer has 2 neurons representing two classes.
Initial Weights and Biases
Weights are initialized randomly. For example, the weights between input and hidden layer are:
- W11 = 0.2
- W12 = -0.3
- W13 = 0.4
- W21 = -0.5
- W22 = 0.1
- W23 = -0.2
- W31 = 0.3
- W32 = -0.4
- W33 = 0.2
Biases are set to zero initially for simplicity.
Forward Propagation and Calculations
Given an input image with pixel values [0.5, 0.2, 0.1, 0.7], the hidden layer inputs are calculated as:
H1 = (0.5)(0.2) + (0.2)(-0.3) + (0.1)(0.4) = 0.1 – 0.06 + 0.04 = 0.08
H2 = (0.5)(-0.5) + (0.2)(0.1) + (0.1)(-0.2) = -0.25 + 0.02 – 0.02 = -0.25
H3 = (0.5)(0.3) + (0.2)(-0.4) + (0.1)(0.2) = 0.15 – 0.08 + 0.02 = 0.09
Applying an activation function, such as sigmoid, the hidden layer outputs are:
OH1 = 1 / (1 + e-0.08) ≈ 0.52
OH2 = 1 / (1 + e0.25) ≈ 0.43
OH3 = 1 / (1 + e-0.09) ≈ 0.52
The outputs are then used as inputs to the output layer, where weights are similarly applied to compute class probabilities.
Calculating Error and Updating Weights
Suppose the true class is class 1, represented as [1, 0]. The output layer computes:
O1 = 1 / (1 + e-z1)
Where z1 is the weighted sum of hidden outputs. The error is calculated as:
Loss = 0.5 * (target – output)2
Using gradient descent, weights are updated based on the error and learning rate, for example, 0.1.
Summary of Calculations
This process repeats for multiple epochs, adjusting weights to improve accuracy. Detailed calculations involve matrix multiplications, activation functions, and error derivatives, which are essential for training neural networks effectively.