Neural Network from Scratch using C

Hello There, 

Around a year ago we talked. Now several things have changed. I recently joined as a PhD scholar 😃. And decided to soil my hands in AI/ML. During my M.Tech days(2011-2013), we are taught Soft computing. Though I did my Thesis on Network On Chip (NOC) which is related to putting different IPs on to a silicon substrate. I devised a Genetic Algorithm based best mapping  as well as a deterministic heuristic based mapping using graphs. both the codes are available on my github page. But I have never paid much attention to ML till 2016, when I was developing a traffic detection  application for a client. thought it was never went beyond PoC stage.

Now I thought to put all my knowledge to earn a degree (Yes a Doctorate). To refresh my understandings of my master's degree time, I decided to go step by step implementing building blocks to learn this topic.

We should start by putting a single neuron first. The artificial neurons, the counter part of natural neurons share similar characteristic.

 

 Dendrites are inputs and axon is output. But one thing we must put here which is Activation. It is the strength we provide as output.

in case of perceptrons input are applied through some weights. This means different input channel have different weights. the perceptron sums the weighted inputs and apply some sort of activation before output the result.

So mathematically we can write 

output = activation(sum ( inputs x weights)) 

output = activation(sum ( input array * weights array))

we can say we need 2 one dimensional matrices for inputs and weights.  Then upon the result of the multiplication we apply activation function.

Lets assume we have 3 inputs i₁, i₂, i₃ with weights w₁,w₂ and w_3. Now output will be \begin{equation} z=\sum_{1}^{n} i_mw_m \end{equation} next activation of z is output. 

There are several activation functions used to activate the output of perceptron. the most used are Binary activation, ReLU(0,x) and Sigmoid(0,1). 

We will use sigmoid for our application. \begin{equation}\sigma(z)=\frac{1}{1+e^{-z}}\end{equation}

 Next we will see the simple implementation of this perceptron.

Bye Bye for now.