MACHINE LEARNING ALGORITHMS

Build an Artificial Neural Network from scratch using Python

In recent years, we've seen numerous breakthroughs in artificial intelligence algorithms. Many of these are simply modern types of neural networks such as Recurrent Neural Network, Convolutional Neural Network, Radial Basis Functional Neural Network, etc, each designed for specific types of structured or unstructured data. Understanding the underlying principle of a simple neural network would help you in breaking down these new complex algorithms.

Concept of Neural Networks – to mimic the human brain

Neurons are the basic units of our nervous system. They use electrical impulses and chemical signals to transmit information between different areas of the brain, and between the brain and the rest of the nervous system.

The human brain has an average of 86 billion neurons. Neurons have three basic parts:

• a cell body: this contains the nucleus which controls the cell’s activities and contains the cell’s genetic material.

• axon: transmits messages from the cell.

• dendrite: receives messages for the cell.

Coding a simple Neural Network from scratch

Before proceeding, you should be:

• familiar with the logistic regression algorithm

• able to code in Python

Terminologies

• Neurons: 

• Layers: A layer is made up of multiple neurons. 

• Weights: 

• Vectors: 

• Activation functions: 

• Forward and Back Propagation: 

• Cost function: 

• Gradient descent: 

Step 1 - Determine the architecture

There are several varieties of ANNs. The number of hidden layers, input parameters and output varies across different architectures. When creating a Neural Network, it is important to know the number of inputs there are, the number of outputs needed, and the number of hidden layers that would work best. It is also a good practice to ensure that all hidden layer uses equal number of neurons.

Step 3 - Define the activation function

There are many types of activiation functions that can be used in a Neural Network. The most popular of them are sigmoid, relu and softmax. For this ANN we are building, we would use the sigmoid activation funtion type. It outputs values in the range of 0 to 1.

Step 4 - Implement Forward Propagation

Forward Propagation is the process of moving from the input layer (left) to the output layer (right) in the neural network. The input data is fed in the forward direction through the network. Each hidden layer accepts the input data, processes it and passes to the successive layer until it gets to the output layer.

Step 5 - Compute Cost Function

The cost function evaluates the performance of a neural network. It compares the predicted outputs and actual outputs and calculates the error of the model. A cost function is a single value, not a vector. The higher the cost function, the farther away our prediction is from the actual value. This is why it is important to minimize the cost function.

Step 6 – Implement backpropagation to compute partial derivative of the cost function

Backpropagation is the opposite of forward propagation. It is the process of moving from the output layer (right) to the input layer (left). During backpropagation (also referred to as backprop), the total loss is propagated back into the neural network to know how much of the loss every node is responsible for, and subsequently updating the weights in such a way that minimizes the loss by giving the nodes with higher error rates lower weights and vice versa.