Neural Network from Scratch Using C (Part-3)

 

XOR Problem

In part 2, AND gate and OR gate could be trained using a single perceptron. But XOR gate could not be as it is not linearly separable. The XOR function can be represented as \begin{align} y &= x_{1}\bar{x_{2}}+\bar{x_{1}}x_{2} \\ z_{1} &= x_{1}\bar{x_{2}} \\ z_{2} &= \bar{x_{1}}x_{2} \\ y &= z_{1}+z_{2} \end{align} Looking at the above equations \begin{align} z_{1} &= x_{1} \land \bar{x_{2}} \\ z_{2} &= \bar{x_{1}} \land x_{2} \\ y &= z_{1} \lor z_{2} \end{align} y is splited into 2 equation z1 and z2, 2 linearly separable functions hence it can not be realized using a single neuron. Then the solution will be a multiple neurons. The organization of multiple neurons stacked in different layers is known as "Multi Layer Perceptron". In this part we will see a very basic multi layer perceptron to solve XOR problem. This chapter involves calculus differential. For Information purpose basic things will be discussed but no in-depth calculus will be used. It is assumed the reader is well aware of derivatives.

Multi layer Perceptron

A Multi layer perceptron (MLP) is a network of neurons arranged in more than one layer. That is Output, Hidden and Input. Generally Input Layer is a direct connecting layer. Hidden layer is where the real magic happens and the output layer is interface to user. A hidden layer is called as such because user doesn't interact with that layer. In an MLP multiple hidden layers may be present. But there is a limitation which is due to "Vanishing Gradient Problem". If interested then There are enough material present on internet. In an MLP with one input layer, one hidden layer and one output layer, and the network is fully connected that is each neuron of a layer is connected each neuron of immediate nearby layer. Please See the picture Bellow.

One thing to observe here is Hidden layer. Each the neurons in this layer has connections from each neuron in Input layer, and they are connected to output neurons. This is a simple fully connected neural network with 4 inputs, 5 hidden and one output neuron. In Future multiple output networks will be discussed and used.

The connections between layers are weights. These weights are generally represented using matrices. for convenience we will represent each weight as w_ij where i is i^th neuron of next layer and j is j^th neuron of current layer. In above case weight matrix between input and hidden layer should be defined as a matrix Wih[5,4]=> 5 rows and 4 columns and the weight matrix between hidden and output is Who[1,5]. For Input, hidden output and final output the matrix will be a column matrix.

Forward Propagation

Input data moves through the network and creates some output. This is called forward propagation. Previously in single neuron, the calculation was easier. Here in case of multiple layers it is a bit involved. It involves matrix multiplication. Consider above Image.input arriving at i^th neuron of hidden layer can be expressed as: \begin{equation} h_{i} = w_{i0}x_{0}+w_{i1}x_{1}+w_{i2}x_{2}+w_{i3}x_{3} \end{equation} or in general terms it can be expressed as \begin{equation} h_{i} = \sum_{j=0}^{n}w_{ij}x_{j} \end{equation} If set of equation for h_i is observed closely then it is seen that it is result of matrix multiplication. $$ \begin{bmatrix} h_{0} \\ h_{1} \\ h_{2} \\ h_{3} \\ h_{4} \end{bmatrix} = \begin{bmatrix} w_{00}^{ih}&w_{01}^{ih}&w_{02}^{ih}&w_{03}^{ih} \\ w_{10}^{ih}&w_{11}^{ih}&w_{12}^{ih}&w_{13}^{ih} \\ w_{20}^{ih}&w_{21}^{ih}&w_{22}^{ih}&w_{23}^{ih} \\ w_{30}^{ih}&w_{31}^{ih}&w_{32}^{ih}&w_{33}^{ih} \\ w_{40}^{ih}&w_{41}^{ih}&w_{42}^{ih}&w_{43}^{ih} \end{bmatrix} \times \begin{bmatrix} x_{0} \\ x_{1} \\ x_{2} \\ x_{3} \end{bmatrix} $$ $$H = W^{ih}.X$$

This resultant matrix need to be applied activation function of hidden layer to produce output. Thus generated values is input to the out put layer(in our case output neuron). As previous layer we need to do a matrix multiplication to find the input to the output layer and apply activation function of output layer to get the network output. $$ \begin{bmatrix} O_{0} \end{bmatrix} = \begin{bmatrix} w_{0}^{ho}&w_{1}^{ho}&w_{2}^{ho}&w_{3}^{ho}&w_{4}^{ho} \end{bmatrix} \times \begin{bmatrix} h_{0} \\ h_{1} \\ h_{2} \\ h_{3} \\ h_{4} \end{bmatrix} $$ $$ O = W^{ho}.H $$

Back Propagation

The Output O is dependent on weights W^ho and H. H is dependent on weights W^ih. As the selection of initial weights are random, hence they are unlikely to be perfect. Hence the output O is different than expected or target. The difference of Target and Output is known as Error. This error has to be compensated by adjusting weights W^ho and W^ih. The adjustment of weights starts from last layer and goes down to beginning. Hence it is called Back Propagation.

Back propagation of errors has to be adjusted among different layers, depending on how much each layer contributed. To find out contribution of error a partial derivative of error has to be found out with respect to different contributing elements. Description and derivation of this is beyond the scope as it will lengthen the article. For reference one can watch this video

We can represent the updated weight matrix of a layer as follows $$ W_{new}=W_{cur}+\alpha[target-actual output].X$$ where 

• X is input to the layer 
• α is learning rate

 

Implementation

To adjust weights in W^ho>, we need to find error in output, then apply derivative of activation function to it. Next multiply by scalar learning rate(α). This finish scalar calculations. The resultant multiplied by the Input to the layer that is activated output of hidden layer will give us delta weights. this delta weights added to the weight matrix W^oh gives updated weights in Hidden to Output layer.

Same procedure have to be repeated for input to hidden layer.  To begin with we need to know error at hidden layer first. This can be found by multiplying (W^oh)^T with error at output. Rest procedures are same as above.

	// find error in output
	matrix err_out=subtract_matrix(expect,final_out);
	matrix whoT = transpose(nn.who);
	matrix err_hid=mat_mult(whoT,err_out);
	
	// calculate delta errors 
	matrix_map(&final_out,sigmoid_prime);
	mat_mult_hadamard(&err_out,&final_out);
	matrix hidT= transpose(hidden_out);
	matrix delta_who = mat_mult(err_out,hidT);
	mat_mult_scalar(nn.lr,&delta_who);
	
	matrix_map(&hidden_out,sigmoid_prime);
	mat_mult_hadamard(&err_hid,&hidden_out);
	matrix inpT = transpose(inp);
	matrix delta_wih = mat_mult(err_hid,inpT);
	mat_mult_scalar(nn.lr,&delta_wih);
	// update weights
	matrix tempa =add_matrix(nn.who,delta_who);
	assign_matrix(&nn.who,&tempa);
	matrix tempb = add_matrix(nn.wih,delta_wih);
	assign_matrix(&nn.wih,&tempb);

With this we are ready to make a very small neuralnet and train it for XOR problem.  MNIST dataset can also be trained but needs patience and time( I achieved with 91.6% accuracy). Iris Dataset and wheat seed data set are very easily trained. Bellow is XOR problem implementation.

#include <stdio .h >
#include "../mlp1.c"
int main(int argc, char *argv[])
{
	srand(0);
	nnet my_nn =create_nn(2,3,1);

	// training
	
	for(int n = 0; n< 30000; n++)
	{
		matrix inp = create_matrix(2,1);
		matrix expected = create_matrix(1,1);
		
		set_cell(&inp,0,0,1.0);
		set_cell(&inp,1,0,0.0);
		set_cell(&expected,0,0,1);
		train(my_nn,inp,expected);
		
		set_cell(&inp,0,0,1.0);
		set_cell(&inp,1,0,1.0);
		set_cell(&expected,0,0,0);
		train(my_nn,inp,expected);
		
		set_cell(&inp,0,0,0.0);
		set_cell(&inp,1,0,1.0);
		set_cell(&expected,0,0,1);
		train(my_nn,inp,expected);
		
		set_cell(&inp,0,0,0.0);
		set_cell(&inp,1,0,0.0);
		set_cell(&expected,0,0,0);
		train(my_nn,inp,expected);
		
		delete_matrix(&expected);	
		delete_matrix(&inp);
	}
	// save the trained netork
	save_nn(my_nn,"xor2.net");

	// load nn from file
	my_nn = load_nn("xor2.net");
	// test trained network
	matrix inp = create_matrix(2,1);
	set_cell(&inp,0,0,1.0);
	set_cell(&inp,1,0,1.0);
	matrix out = predict(my_nn,inp);
	display_matrix(out);
	delete_matrix(&out);
	set_cell(&inp,0,0,0.0);
	set_cell(&inp,1,0,0.0);
	out = predict(my_nn,inp);
	display_matrix(out);
	delete_matrix(&out);
	set_cell(&inp,0,0,0.0);
	set_cell(&inp,1,0,1.0);
	out=predict(my_nn,inp);
	display_matrix(out);
	delete_matrix(&out);
	set_cell(&inp,0,0,1.0);
	set_cell(&inp,1,0,0.0);
	out = predict(my_nn,inp);
	display_matrix(out);
	
	delete_matrix(&out);
	delete_nn(my_nn);
	delete_matrix(&inp);
	return 0;
}
=======output====
[ashok@fedora xor]$ ./xor
[ 0.00559855 ]

[ 0.06528170 ]

[ 0.96165163 ]

[ 0.96165793 ]

Though I may put all the code here or on github, but I would like the reader to implement his own version analogous to this implementation. In any problem Contact me for the Full code its free But before asking please try to implement yourself.

Till now we discussed

• What is an artificial neuron
• How to implement it
• What are the areas it can be used(linear separable)
  
• How to train it
• What is XOR problem
• What is Multi layer neural network
• How to implement one
• How to train
• Example of XOR implementation

This is the end of 3 parts for simple neural network. Part1 and Part2 can be found here. In some future article we will discuss Convolutional Neural Network(CNN)
 

Neural Network from Scratch Using C (Part-2)

 

Implementing a Perceptron

In this part we will start implementing a single perceptron and do some experiments. 

first part can be found here.

A single Perceptron

A single Perceptron(neuron) can do a lot of jobs. Yes Like a Neuron in Our nervous system it can do a lot of things. As One grows from Infants to toddler the neurons in our brain get trained(learned) to coordinate between our limbs, coordinate our activities etc...
Similarly the Perceptrons can be trained to classify data as per the training data. but we can only classify linearly separable data.

What is a linearly separable?
Linear separability can be termed as, if the plot of data can be separated by a single straight line. This means depending on the separation criterion, there exists a straight line dividing the data points in to 2 regions.

Now lets see a small code implementing it step by step. NOTE: We need a matrix manipulation library for all our Neural network adventure. For this purpose my tmatlib on github is quite suitable. Though one should not expect it to perform like blas. In my code i used a similar version known as smatlib. As it is continuously growing I have not published it on github. But stay assure the functionality and performance of both are same. 

// structure of our perceptron.

typedef struct perceptron
{
	int n_inp;   // number inputs
	matrix W;    // weight matrix
	double lr;   // learning rate
	double bias; // bias
}perceptron;

A straight line on a 2-dimentional plain can be expressed as y=mx+c. The Bias parameter in our perceptron is related to c. Our training data is x and y, x is input and y is expected or target. Our perceptron learns the value of m. Learning rate(lr) determines how fast or slow the perceptron should learn. If Learning rate is too low then it may be trapped in local minima. If it is too high then it may oscillate and may not reach the solution. So Bottom line is one has to experiment with learning rate(lr) value to achieve better result. Weight matrix is initialized with random values. Gradually with training it is adjusted/updated to achieve better result. 

 I uploaded all the codes at Simple-NeuralNetwork.

In the repo there are 4 files perceptron.h, linearsep.c, and_gate.c and or_gate.c. We will discuss bits from those files and see how they work.

In perceptron.h file, there are functions/methods to create, manipulate, train, display and delete the perceptron. General sequence is 

• create a perceptron
• prepare data set
• train
• test with predict function

This is simple, is not it? Now we will look into 2 functions, namely predict and train. These  functions are the core of this perceptron.

predict:

double predict(perceptron p, double inp[])
{
	double res=0.0;
	
	for(int r = 0; r < p.n_inp; r++)
	{
		res += inp[r]*get_cell(p.W,0,r);
	}
	res += p.bias;
	double ret= sigmoid(res);
	return(ret);
} 

This function multiplies inputs with corresponding weights and adds them. Next, to activate the output apply sigmoid function. A sigmoid function ranges from 0 to 1. Please see the first part here where the sum process and sigmoid function is defined.

Train:

  void train(perceptron *p,double *data,double desired)
  {
	double guess = predict(*p,data);
	
	double err = desired - guess;
	for(int n = 0; nn_inp; n++)
	{
		double cur_w = get_cell(p->W,0,n);
		set_cell(&p->W,0,n,cur_w+p->lr*err*data[n]);
	}
	p->bias += p->lr *err;
  } 

In this function, perceptron is trained by adjusting weights and bias.The steps can be as follows.

• calculate output
• find error by subtracting output from target
• find delta weight by multiplying error, data(input) and learning rate
• find delta bias by multiplying error and learning rate
• update weights
• update bias

In step 3 we are calculating the delta weights. This is the most important step in whole code. How and why delta value is multiplication of error with input is a topic itself.  As an Hint it is gradient, hence a derivative of error term with inputs and weights are to be considered.

Testing

To tryout this implementation, 3 code files were provided. which adheres to the general sequence. First one is linear_sep.c, In this demo program, random data sets are created and the perceptron is trained; then with a known data set it is compared. the output on my system is  as bellow.

************Perceptron:**************

number of inputs = 2
Weights:
[ 1.00000000 ]
[ 1.00000000 ]

Bias = 1.000000
*************************************
************Perceptron:**************

number of inputs = 2
Weights:
[ 61.70302487 ]
[ -31.13024636 ]

Bias = 0.622900
*************************************

is (2.000000 < 2 x 20.000000 + 1) predicted = 1.000000

For other 2 demo codes. Can this artificial neuron mimic reliably a digital gate? Well that was the motive when artificial neuron was proposed. It is observed that AND and OR gate are possible to simulate but not XOR gate. WHY???  from bellow diagrams it is obvious that OR function and AND function are linear separable but not the XOR function.

// OR gate implementation.

    // x1 x2 y
    //  0  0 0
    //  0  1 1
    //  1  0 1
    //  1  1 1
    //
    //  0,1-------1,1
    //   |          |
    //  \|          |
    //   |\         |
    //   | \        |
    //  0,0-\-----1,0
    //       \-separating line
    
    // AND gate implementation.
	// x1 x2 y
	//  0  0 0
	//  0  1 0
	//  1  0 0
	//  1  1 1
	//
	//  0,1---\----1,1
	//   |      \    |
	//   |        \  |
	//   |          \|
	//   |           |\-line separating  
	//  0,0-------1,0
// XOR function
// x1  x2  y
//  0   0  0
//  0   1  1
//  1   0  1
//  1   1  0
//        /----------|----- 2 lines separates
//  0,1--/-----1,1   |
//   |  /       |    |
//   | /        |    |
//   |/         |/---|
//   /          /
//  /|         /|
//  0,0-------/-1,0
    

Output of OR gate 

************Perceptron:**************

number of inputs = 2
Weights:
[ 0.00000000 ]
[ 0.00000000 ]

Bias = 1.000000
*************************************
************Perceptron:**************

number of inputs = 2
Weights:
[ 11.48282021 ]
[ 11.48242093 ]

Bias = -5.281243
*************************************
inputs 0, 0 predicted = 0.005060
inputs 1, 0 predicted = 0.997978
inputs 0, 1 predicted = 0.997977
inputs 1, 1 predicted = 1.000000

 

Output of AND gate

************Perceptron:**************

number of inputs = 2
Weights:
[ 0.00000000 ]
[ 0.00000000 ]

Bias = 1.000000
*************************************
************Perceptron:**************

number of inputs = 2
Weights:
[ 10.22930534 ]
[ 10.22993658 ]

Bias = -15.512550
*************************************
inputs 0, 0 predicted = 0.000000
inputs 1, 0 predicted = 0.005050
inputs 0, 1 predicted = 0.005053
inputs 1, 1 predicted = 0.992943

In the above result if we consider 0.99 as 1 and anything less that 0.005 as 0 then our results are at par with the truth table.

As we saw above XOR is not linearly separable, we can not simulate it with a single perceptron. we need more than one layer to simulate it we will do it in a future post.

Till then happy coding.

Neural Network from Scratch using C

Hello There, 

Around a year ago we talked. Now several things have changed. I recently joined as a PhD scholar 😃. And decided to soil my hands in AI/ML. During my M.Tech days(2011-2013), we are taught Soft computing. Though I did my Thesis on Network On Chip (NOC) which is related to putting different IPs on to a silicon substrate. I devised a Genetic Algorithm based best mapping  as well as a deterministic heuristic based mapping using graphs. both the codes are available on my github page. But I have never paid much attention to ML till 2016, when I was developing a traffic detection  application for a client. thought it was never went beyond PoC stage.

Now I thought to put all my knowledge to earn a degree (Yes a Doctorate). To refresh my understandings of my master's degree time, I decided to go step by step implementing building blocks to learn this topic.

We should start by putting a single neuron first. The artificial neurons, the counter part of natural neurons share similar characteristic.

 

 Dendrites are inputs and axon is output. But one thing we must put here which is Activation. It is the strength we provide as output.

in case of perceptrons input are applied through some weights. This means different input channel have different weights. the perceptron sums the weighted inputs and apply some sort of activation before output the result.

So mathematically we can write 

output = activation(sum ( inputs x weights)) 

output = activation(sum ( input array * weights array))

we can say we need 2 one dimensional matrices for inputs and weights.  Then upon the result of the multiplication we apply activation function.

Lets assume we have 3 inputs i₁, i₂, i₃ with weights w₁,w₂ and w_3. Now output will be \begin{equation} z=\sum_{1}^{n} i_mw_m \end{equation} next activation of z is output. 

There are several activation functions used to activate the output of perceptron. the most used are Binary activation, ReLU(0,x) and Sigmoid(0,1). 

We will use sigmoid for our application. \begin{equation}\sigma(z)=\frac{1}{1+e^{-z}}\end{equation}

 Next we will see the simple implementation of this perceptron.

Bye Bye for now.

GitHub Update

 After a Long time another post.

Added 2 small projects, one is a single header Matrix library and another is a CPU Simulator.

The matrix library (tmatlib) is capable enough to solve simultaneous linear equation. Though it was written several months back but I was Lazy.

The CPU simulation code was my tech-itch :p it is very minimal and for fun. It has small set of instructions, it has only 8 bit wide registers and a very small memory only 1kb which includes stack, data and code.

my github link here.

Laravel Application deployment on Fedora 34 server

 Recently I have to migrate an old Laravel application of a client to a newer OS release Fedora 34. When it was developed Fedora-29 was Latest 😁.

 

The procedure is as bellow.

• Install Fedora 34
• Update the installation (sudo dnf update -y)  
• Install Apache (sudo dnf install httpd)
• Install Mariadb, Mariadb server (sudo dnf install mariadb mariadb-server)
• Install PHP, PHP Mbstring, PHP zip, PHP Mysqlnd, PHP Mcrypt, PHP XML, PHP Json, PHP suMod ( sudo dnf install composer php php-mbstring php-zip php-mysqlnd php-mcrypt php-xml php-json mod_suphp -y)

• check if PHP is properly working or not by using following script

phpinfo.php

<?php

phpinfo();

?> 

this script should be in /var/www/html/ directory which is by default  document root for httpd server.

Point the browser to this file and see if it shows a page about PHP version and other details. If not then check if apache server is running. 

• As super user create a laravel.conf file under /etc/httpd/conf.d directory with following content

<VirtualHost *:80>

    ServerName your server name

    DocumentRoot /var/www/html/laravel_project/public

    <Directory /var/www/html/laravel_project>

        AllowOverride All

    </Directory>

</VirtualHost>

•  Copy the laravel project to /var/www/html/ as super user and cgange the owner by chown -R apache:apache laravel_project 
• Start Mariadb (sudo systemctl start mariadb)

This is almost complete. For your specific cases different services or tweaking may be required. 

Laravel is a good framework for web-app development but for sometimes it is not getting more acceptance. Now the buzz word is Node . In a future article I may write on settingup and using Express framework for Web-App.

Thank you.

Deploying Django application on Fedora 33

Deploying a "Hello, World" Django app on Fedora 33
Requirement
1) Fedora server
2) NGINX web server
3) Django application framework
4) Gunicorn

A "Hello, World" application using Django

Django framework based upon Python. Current Python version is 3.7.
First check if python is installed. If not install it.

$ sudo dnf install python3

once Python is installed using pip3 django should be installed

$ pip3 install django~=3.1.2

Any version of django is good but it is personal preference.

next Gunicorn should be installed.

$ pip3 install gunicorn

We are almost ready for Django application development.

Soiling your hand
To start our development, we need to create a directory and put all our requirement there

$ mkdir myapp && cd myapp

I choose myapp but any thing is ok

$ django-admin startproject mapp

mapp is project name. Django will create some files for us and lessen our job. It will create
mapp directory and few files

.
└── mapp
    ├── manage.py
    └── mapp
        ├── asgi.py
        ├── __init__.py
        ├── settings.py
        ├── urls.py
        └── wsgi.py

Next we create a small application to run on top of it.

$ cd mapp
$ django-admin startapp hello

.
├── hello
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── manage.py
└── mapp
    ├── asgi.py
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

now open the mapp/settings.py

$ vi mapp/settings.py

add following lines

INSTALLED_APPS = [
    ...
    ...
    'hello',
]

next open views.py

$ vi hello/views.py


...
...
from django.http import HttpResponse
def homePageView(request):
    return HttpResponse('Hello, World!')

create a file urls.py under hello and open it

$ vi hello/urls.py

add the following code to it.

from django.urls import path
from .views import homePageView
urlpatterns = [
    path('', homePageView, name='home')
]

Update mapp/urls.py as bellow

from django.contrib import admin
from django.urls import path, include
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('hello.urls')),
]

Now test our app by running

$ python3 manage.py runserver

In browser, open http://127.0.0.1:8000
and enjoy your app

Ok you might have seen some warnings here. So please don't worry. Execute

$ python3 manage.py migrate

some messages will be displayed and now again execute

$ python3 manage.py runserver

This time there are no warnings, right? Ok good...

Deployment part

Open settings.py

$ vi mapp/settings.py

and change following

...
ALLOWED_HOSTS = [ '*' ]
...
...
STATIC_ROOT = '/var/www/static'
MEDIA_ROOT = '/var/www/media'

and make sure you created those directories

$ sudo mkdir /var/www/static
$ sudo mkdir /var/www/media

Next run

$ sudo python3 manage.py collectstatic

we are almost done.

Running guincorn

First copy mapp/wsgi.py as mapp/mapp.wsgi

$ cp mapp/wsgi.py mapp/mapp.wsgi
$ sudo gunicorn mapp.wsgi --workers=5 --bind=127.0.0.1:8888

now gunicorn is running. you can check with http://127.0.0.1:8888 and it will show you hello world page.

But we need nginx to serve for us on internet so we have to tell nginx to serve for us.

use following code as a .conf file in /etc/nginx/conf.d/ directory
we assume it as myapp.conf

server {
    # the port your site will be served on
    listen      80;
    # the domain name it will serve for
    server_name 192.168.1.3;   # substitute by your FQDN and machine's IP address
    charset     utf-8;

    #Max upload size
    client_max_body_size 75M;   # adjust to taste

    # Django static files. You should run 'django manage.py collectstatic'
    location /static  {
        alias /var/www/static;      # your Django project's media files
    }
    location /media {
    alias /var/www/media;
    }
    # Finally, send all non-media requests to the Django server.
    location / {
        proxy_pass http://127.0.0.1:8888;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

restart nginx server by using

$ sudo systemctl restart nginx

now check from a remote machine using http://IP.address.of.server/

do you get it properly??? if not then check firewall settings on server. next check access on files etc...


goodluck.

Fedora 32 on Google Cloud

Google doesn't provide a Fedora image for its cloud platform neither Fedora provides one. but this wont stop us. It is a nice learning and adventure for orthodox people like me.

Those who have started there career in Computer in late 80's and early 90's can understand my pain 😬

Well during this time of pandemic COVID-19 I thought of taking an adventure ride into the world of Virtualization. Though Virtualization is not new to me, as during my days at RedHat I did some virtualization stuff by porting drivers, but that is a very short lived assignment for me. Even I was fascinated by Vmware's ESX server Community edition. Even to tryout my toy Operating System NanoOS i need some platform to try out so Was using qemu and BOCHS.
But setting up a VM in a cloud is just a new thing for me. If you like to learn along and how to do it then read along.

Prerequisites 

As I don't know much about Windowz, will restrict to Google's cloud console as much as possible and at some times Linux desktop.

• Google account
• Machine with Fedora Desktop(any Linux distro will do but I like Fedora)
• Google SDK for Linux installed on your machine.

Deploying a VM needs availability of an Image. Though there is no image for Fedora 32 but similar images are available for CentOS and RHEL. We could choose any but I would prefer CentOS over RHEL.

Plan of Action 

• We will create a VM using CentOS image. This will be a very minimum one as we will use it for creating Fedora 32 image.
• Download Fedora cloud image from Fedora download site
• Tweek Fedora image to make it suitable for Google.
• Upload the image
• Create an instance of it 

Creating a VM 

Here you need a google account your Gmail Account wont work here. You need to have cloud permission in your account. Please refer to google site for procedure. its not difficult but its not relevant here hence omitted.

Similarly install Google cloud SDK (gcloud and gsutil). Information about the procedure is available on Google SDK site.

• Login to Google cloud, the bellow screen will be displayed

• Click on menu (3 horizontal line)

• from drop down select Compute Engine

• Next it will display a page with all vm instances if available. for us it should be empty. Next on the top there is an icon with a plus

• Now a form will be displayed, it will ask to enter a name for your VM instance . We will put boiler-plate there. Dont worry about other options except Boot Disk.

• On Boot Disk option by default it will be debian. There is a button change next to it. Clicking it will display several options.

• On clicking change something like this will appear. We will select CentOS. On next screen it will ask which version to use. we will use CentOS 8 as it has dnf command. But If you prefere yum can choose other older versions too.
  
  

• Allow it for some time say 5 Minutes. then on VM Instance page our new VM will be displayed.

Ok we have successfully created a VM on Google Cloud now we can enjoy it for some time🙆 .Relax get a coffee and think what we have achieved.
After this point our journey will be some what tough as we will be doing some sysadmin job (yes SysAdmins are really great people They do a lot of hectic work to ease our life).

Click on ssh button at the end of the line of our newly created VM instance. It will open a newbrowser window with shell prompt of our VM. Check if all your commands are accepted or not.

Download Fedora cloud image

As we know there is no Google compatible image for Fedora available but Cloud images are available. We can download it and adjust  for Google compatibility. 
On the ssh terminal shell prompt check if wget is available or not. Else install it by using dnf or yum.

~]$ sudo dnf install wget

So go ahead and download  Fedora Cloud image. Choose raw image for convenience. Remember we are downloading into VM on google not onto our Desktop.

~]$ wget -c"https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.raw.xz"

it will be downloaded to the home folder in VM. Give it some time as it is a long file , but don't worry it will be downloaded in few minutes say 2 to 3 minutes.

Preparing the image

Now the raw file is downloaded as *.tar.xz format. It has to be extracted  first 

~]$ tar -xvf Fedora-Cloud-Base-32-1.6.x86_64.raw.xz

It is better to copy this file to a smaller name like disk.raw and work on that.

• mount the image

~]$ sudo losetup  /dev/loop0 disk.raw
~]$ kpartx -a /dev/loop0

mount the disk

$ sudo mkdir /mnt/disk
$ sudo mount /dev/mapper/loop0p1 /mnt/disk

• Bind mount some filesytem

~]$ sudo mount --bind /dev /mnt/disk/dev
~]$ sudo mount --bind /sys /mnt/disk/sys
~]$ sudo mount --bind /proc /mnt/disk/proc

besides this another file has to be created that is /etc/resolv.conf content of that file is very small and as follows.

search domain.name
nameserver 8.8.8.8

Its now time to go into chroot

$ sudo chroot /mnt/disk

After chrooting check if ping works, that is if network works else adjust your network to work in chroot environment.

First thing to do is update the image. by

~]$ sudo dnf update

Next we have to install 

~]$ sudo dnf -y remove cloud-init
~]$ sudo dnf -y install google-compute-engine-tools
~]$ sudo dnf clean all

Now we are almost done. Google need some service to be enabled.

~]$ sudo systemctl enable google-accounts-daemon google-clock-skew-daemon \
    google-instance-setup google-network-daemon \
    google-shutdown-scripts google-startup-scripts

Well our Image is almost ready. Now we should leave chroot environment unbind and unmount the filesystems we have mounted earlier.

~]$ sudo umount /mnt/disk/dev /mnt/disk/sys /mnt/disk/proc
~]$ sudo umount /mnt/disk
~]$ sudo losetup -d /dev/loop0

In above steps, unmounting disk will throw "resource busy error" dont worry. Give it some time to finish writing stuffs, 5 to 10 minutes maximum. then unmount the disk and delete the loop0. We are done.

Google need its image to be in *.tar.gz format so we will compres the image in tar.gz format.

~]$ tar cvzf fedora-32-google-cloud.tar.gz disk.raw

Now we will upload the image just created "fedora-32-google-cloud.tar.gz"

We have created in side Fedora VM. We have to put it into Google cloud bucket. Using Web console

• Goto Storage
• Create a bucket

remeber its name. Now from within VM we can copy this image to bucket.

~]$ gsutil cp "fedora-32-google-cloud.tar.gz" "gs://[name of the bucket]/"

Replace with your bucket name in above comand. Well it will be uploaded to the bucket in a while. It can be seen listed under the bucket created.

From your desktop fedora fire this command.

~]$ gcloud compute images create --source-uri \
    gs://[your bucket]/fedora-32-google-cloud.tar.gz \
    --licenses "https://compute.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx" \
    fedora-32-google-cloud-image

Wait for few seconds, an image will be created in images section of console. it can be seen under Images section of Web console. Note the --licenses calause. it is optional. you can omit it. but if you omit it then the created image wont be able to run virtual machines under it. Yes you read it correctly a Nested VM.

I think you enjoied your second coffee ... Jocking...

Now the image is ready and we can create an instance out of it as we created our boiler-plate only difference will be in case of Boot Image we will choose our image and we will choose a multi CPU configuration. After the instance is created check it using SSH and enjoy your Multi CPU Fedora Server on Google cloud.

Well This is enough for Now. Please contact me if you need any help.

73's
DE VU3VFU
Ashok.

Sources :
1) major io
2) linuxmint
3) media com

Covid-19 Lockdown Update

Hi friends, in this time of global pandemic, all are forced to be inside. To avoid the boredom of self exile some thing extra has to be done to dissipate stored energy. I adapted few of my hobbies to do the same. Today I want to share how and what I was doing during past few weeks.

2m Flower pot Antenna

For my VHF activity I made VK2ZOI design flowerpot antenna. The first one i made is a described here. This one worked exceptionally well it outperformed my existing (refurbished) 5/8 GP. i could reach 30 km with only 5 watts with 59 report from 2 stations and 57 from other stations. Earlier it was not possible to reach them from my QTH (location). After tremendous success, i decided to make its big brother hi-gain dual band antenna described here. I tried it 2 times but i failed miserably. this designed didn't work for me (Might be i missed something). So i kept experimenting on that design on hold 🌝🌝🌝.

SDR on Raspberry PI 3B

SDR is my passion. I am a fan of this Software Defined Radio(SDR) technology. During this boring lock-down, I thought to put SDR softwares on a RPI-3B. Well yes there images available to do SDR on RPI. But they are all based on Raspbian. Though i can work on it as I worked on Ubuntu and Armbian. But Fedora is my choice as I have worked with Redhat earlier.

For Experimenting I need to change image often hence first I enabled USB booting on RPI-3B. Then  Tried some images. If any of you trying or replicating then do remember fedora is a heavier OS for RPI.

Installing Fedora.

REMEMBER  Never jump to load a GUI image directly by downloading.

You know Why I am telling 😅 ......... Yes you are right. So download Fedora minimal image for armhfp from here.  Now either use "dd" or use "disk image writer" to create a USB. USB pendrive should be more than or equals to 8GB. 

(I downloaded Fedora 31 but you can go for latest Fedora 32 also.)

After making the disk boot from it. Next is to enable wifi. the key is you have only nmcli to configure and setup your network. After you are connected to Internet you need to update the system.

sudo dnf update

next is installing SDR software. basic thing is to install RTL-SDR driver. Its easy just use dnf to install if you are ok with 28MHz and above reception. If you need  HF reception without any up-converter , then stock RTL-SDR from fedora repo wont work. For that purpose I will recomend AB9IL . down load it and compile and install it. After installation try rtl_test and rtl_tcp. if you want to use this as remote then you can stop here. Execute 

rtl_tcp -a <ip address of RPI> -p <port default is 1234> -s12000 

Yes keep sampling rate small as RPI doesn't have much power and wifi is unreliable 😛😛😛. From another machine on the same network use gqrx to connect to your RPI and tune FM stations and of course tune  HF to see if you get any... I am sure you will get plenty of Chinese and Arabic stations. I could receive our Local Medium wave station on 972.0KHz only. so you can try your luck.( Note: the sensitivity and frequency range varies from Dongle to Dongle)

Well if you want to venture in to a standalone SDR, then stay with me. The Next step is to put a GUI Desktop environment. One must choose as per his/her personal choice but my experience is bitter with LXQt Gnome and XFCE on RPI-3B. Only LXDE worked smoothly and without much problem. Though i prefere LXQt on my Desktop. 

Next is you have to install SDR Radio software like gqrx or CubicSDR. For any SDR software One need a DSP library. liquidDSP andGnuradio are best in my view. GNURadio available from Fedora repo installs Python-2. But GNURadio 3.8 from github uses Pythom-3 hence no old dependencies. So be prepare to compile GNURadio by yourself. 

How to compile for RPI? 

Well You can compile it on RPI itself but it is never intended. So 

1- copy everything to a folder on your desktop PC 

2- install qemu-user-static on your pc

3- locate qemu-arm-static

4- copy it in to the folder in point 1

5- use chroot and compile on your PC

Though it is standard method this is slow in comparison to compiling directly on RPI.( Atleast on my Intel(R)Core(TM)2Duo CPU E6750@2.66GHz )

using this process GNURadio, gqrx, LiquidDSP, WxWidget, SoapySDR, SoapySDRserver, SoapySDRremote,soapy-rtlsdr and CubicSDR are compiled.

Please refer to individual compilation process described for respective  packages. 

Now Copy back the compiled apps to your Pen drive. Boot from it and try executing gqrx and CubicSDR. 

Note: May complain about Library not found even after executing ldconfig so adjust /etc/ld.so.conf to include /usr/local/lib  

now everything will be fine.

Executing gqrx

Try finding optimum sample rate, band width. Next adjust FFT buffer and speed. Reasonable sound quality will be heard.

Executing CubicSDR

Here multiple settings at multiple areas should be adjusted. Sampling rate, Audo sampling rate and gain to get a good sound quality.

 Cheers 73's

DE VU3VFU 

How to setup LAMP using Fedora 31

There are plenty of documents available on internet and you may be a pro in this field. Well I am new and its for my references and someone like me 😃 trying to learn by doing myself. 

Here I will put the key points to setup a basic LAMP (Linux Apache Mariadb PHP) server all by yourself on your hardware 😃.

Begin 

To start with we have to have a Fedora server or Worksation  ready. To achieve this one can use Live ISO files from Fedora site. Please refer to Documentation on how to do so. Here I assume knowledge of fedora server installation and setup.

The system may be installed on real hardware or on VM(Virtual Machine).

Basic LAMP packages need to be installed

After the bare minimum Fedora server installation, it is required to install the following packages.

php php-cli php-php-gettext php-mbstring php-mcrypt php-mysqlnd php-pear
php-curl php-gd php-xml php-bcmath php-zip mariadb-server httpd.

Use dnf install as root to install the packages and its dependencies. 

Configuration

It is assumed that the network is configured properly and it is accessible from outside world. For Virtual machine, Network setup is bit involved but Internet has many resource. ( Hint: Install a Bridge and go on)

1) SELinux should be set to permissive. (Please do not dissable it)

• sudo setenforce 0
• open /etc/selinux/config as root
• sudo nano /etc/selinux/config
• change SELINUX=enforcing to SELINUX=permissive
• save the file

2) PHP setup

• set timezone in /etc/php.ini 
• date.timezone = Asia/Kolkata

3) MySQL configuration

• open /etc/my.cnf.d/mariadb-server.cnf as root
• in [mysqld] section change characterset to utf8 

character-set-server=utf8

• start mariadb service
• sudo systemctl strat mariadb
• enable the service to run automatically when system boots
• sudo systemctl enable mariadb
• setup root password for mariadb by using mysql_secure_installation
• remove anonymous user
• disallow remote root login
• remove test databases and access to it
• reload previledge tables

4) Firewall setup

• allow access for HTTP/HTTPS connections

    sudo firewall-cmd --add-service={http,https} --permanent

• allow mysql access through firewall

    sudo firewall-cmd --add-service=mysql --permanent

• reload firewall

    sudo firewall-cmd --reload

now you are done. Enjoy your adventure.

My Experience with Cyclone Fani

The extremely severe cyclonic storm #Fani recently visited Odisha. It is Peculiar to have a Cyclone in this time of year. Though history says earlier there are some cyclones hit Odisha coast during this summer season.
This Cyclone traveled a very long distance to destroy Odisha. This Cyclone originate near equator in Indian Ocean. it crossed east of Srilanka and gradually headed towards Tamilnadu. Without entering into Tamilnadu coast it headed northwards in Bay of Bengal. .

[Path Of Cyclone Fani]

It touched Land in the morning of 3rd May 2019 near Puri( rather On Lake Chilika). It devastated the surrounding. and proceed further to destroy Khurda, Bhubaneswar and Cuttack.

The Major damage it done is to the Lord Jagannath Temple at Puri. Several Stone carvings fell off from the temple The Age old banyan tree KalpaBata is severely damaged. Water sipped into the inner sanctum and deities were not worshiped.

I have seen several Cyclones including #1999SuperCyclone . Though Super cyclone has done a lot of damage, this one is no less. Wind speed of 200kmph with gust 220 to 250 kmph ripped everything in its path. major damages to infrastructures like power grids, cell towers under construction buildings are name a few.

In case of 1999SuperCyclone, Most of the trees are uprooted including several banyan trees. bent most of the electric poles. But this one ripped branches from trunks at some places trees are uprooted. in case of 1999SuperCyclone wind speed reached more than 300 kmph, but Fani reached upto 220 kmph maximum.

At Cuttack Wind started at 9am gradually intensified. Wind blew from North East direction till 1.30pm. Wind Direction suddenly changed due West. This wind caused major damage. 

My Preparedness

See this SWLING article

I was runner-up in that challenge. My preparation for Cyclone Fani was as per I have described. Well Not all. I have charged emergency light, arranged candles and lantern. Stocked Biscuits and Flat rice. Stored around 15Liters of drinking water. Emergency medicines and fully Charged 3 Baofeng Walkie Talkies. The GP5SSB I got as a gift from SWLINGPost, put new battery in it.   I have downed external antennae for HF and VHF. Baofeng handys are quite good as scanner. I monitored HAM band so I could know the situation in surrounding area and also if situation arises I could give a distress call to Nearby monitoring person. But that situation didn't arise. 

HAM volunteers from WBRC started their communication service since 5th May. They Have setup a VHF in state control room at Bhubaneswar, ADM office Puri and Khurda.

Since 5th may onward I was monitoring and in QSO with VU2IPL(Suresh). VU2FTP VU2XRY VU3YDA and VU3OXI are handling the communication between Puri, Khurda and Bhubaneswar. 

Going ahead

Gradually Normalcy return to Cuttack. water supply was restored from 6th may. And Electricity supply resumed in my locality on 8th May evening. 

Lessons Learnt 

"How much you be prepared for any natural calamity it is less."

My preparation was good no doubt. But the season is summer season. Every season have there advantage and disadvantages. Cyclone in summer brings very uncomfortable climate. Hot and Humid climate makes sweating like hell and dehydrates. So water stock should be more for a family of 8. 

Power backup is not sufficient. Hence a Solar power system or Genset is required to survive after effects of summer cyclones.

#FaniCyclone

#HAM

#HAMRescue

My Experiment with Compiler Writing. Part-2

Sorry for a long long delay

As promised I did some experiment but due some other involvement I could not post. Sorry again...

But now i can share some code which translate very few keywords from BASIC to C. Why I choose these 2 language is ease of understanding.

BASIC is like normal English. Even a newcomer can understand the code. C is more professional and has a compiler to compile. BASIC is an interpretive language, Most implementation are interpreter, though some BASIC compilers are available.
Bellow is the code sofar :)

compile under Linux using gcc.
Use a small BASIC file like

REM remarks
FOR i = 0 TO 10
PRINT "Hello World"
NEXT i
a = 5234.9876
b = 123.789
IF a>b  THEN b=a
IF a>=b THEN c = a+b
IF a<>b THEN c = b
IF a<=b THEN c = a
IF a<b THEN c =b
PRINT a,b,c
END 

and see the generated file and enjoy.

/*
 * Parser for BASIC like language
 * 
 * Copyright 2014 Ashok Shankar Das 
 * 
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 * 
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 * MA 02110-1301, USA.
 * 
 * 
 */
#include
#include
#include
#include
//#include "symtab.c"

FILE *source=NULL, *target=NULL, *subf=NULL;

/* ============= Boolean Defination =============== */
typedef enum { false=0,true} bool;

/* ==============Token types ====================== */
typedef enum { 	t_invalid_token=0, t_ident,t_number, t_literal, 
		t_addop, t_mulop, t_lparen, t_rparen, t_less, 
		t_leq, t_neq, t_eql, t_geq, t_gt, 
		t_punctuation, t_keyword,t_dtyp, t_eol,t_skip	} type_of_token;

char *tok_typ[]={"INVALID","IDENTIFIER","NUMBER","LITERAL",
		  "ADDOP", "MULOP", "LPAREN", "RPAREN", "LESS",
		 "LESSEQL", "NOTEQL", "EQUAL", "GREATEQ", "GREATER",
			"PUNCT", "KEYWORD", "DTYP", "EOL", "skip"};
/* ================== Keywords ==================== */

char *keywords[] = { "LET", "FOR", "TO", "STEP", "NEXT", "DIM",
	       "IF", "THEN", "ELSE", "ENDIF", "READ",
	       "DATA", "END", "FUNCTION", "RETURN", "PROGRAM",
	       "INPUT", "PRINT", "INT", "LEFT$", "RIGHT$",
	       "MID$", "CHR", "ASC", "SUB", "RANDOMIZE",
	       "REM", "RND", "AS"};
#define keyword_count sizeof(keywords)/sizeof(char *)

char *type_specifiers[]={"BYTE","INTEGER","LONG","DOUBLE"};
#define dtype_count sizeof(type_specifiers)/sizeof(char *)

/* ========= Double linklist Implementation ====== */
// define a doubly linked list to represent a token.
typedef struct token
{
	char value[50];		// string value
	type_of_token t_type;	// type
	struct token *prev;	// previous token to see previous
	struct token *next;	// next token 
}token;
token *token_list = NULL;		// list which represents the tokens of the program
// Function create_token takes a string and tokentype
// returns pointer to token
token *create_token(char *val, type_of_token t_typ)
{
	token *t =NULL;
	t=(token*)malloc(sizeof(struct token));
	if(t == NULL)
	{
		printf("Error: Unable to allocate memory for token\n");
		exit(-1);
	}
	bzero(t,sizeof(struct token));
	strcpy(t->value,val);
	t->t_type = t_typ;
	t->prev = NULL;
	t->next = NULL;
	return(t);
}

// function push_back puts a token in token_list
// accepts a pointer to a token
// returns a pointer to a token
token *push_back(token *t)
{
	token *temp = token_list;
	// add at the end
	if (token_list)
	{
		temp = token_list;
		while(temp->next != NULL)
			temp=temp->next;
		temp->next=t;
		t->prev = temp;
	}
	else
	{
		token_list = t;
	}
	return (token_list);
}
// Function push_front puts a token at the front of a list
// accepts a pointer to token
// returns a pointer to token
token *push_front(token *t)
{
	if(token_list)
	{
		t->next=token_list;
		token_list->prev = t;
		t->prev = NULL;
		token_list = t;
	}
	else
	{
		token_list = t;
	}
	return(token_list);
}

// Function display_list traverses the token_list and prints the token information
void display_list()
{
	token *list;
	list = token_list;
	int line = 1;
	//printf("DEBUG: token_list at %p\n",token_list);
	while(list!=NULL)
	{
		printf("%s [ %s ]\n",list->value,tok_typ[list->t_type]);
		if(list->t_type == t_eol)
			line +=1;
		list = list->next;
	}
	printf("Total Lines scanned = %d\n",line-1);
}

/* ================ lookahead ==================== */
// Function look look for the next character in the stream
// accepths a FILE pointer
// return a character
char look(FILE *source)
{
	char ch='\0';
	if(!feof(source))
	{
		ch = fgetc(source);
		ungetc(ch,source);
	}
	return ch;
}
/* ============== sacnner Proper =================== */
// Function Parse, actual parser implementation actually it is lexer and parser combined
// accepts FILE pointer
// returns boolean value success or failure
bool scanner(FILE *source)
{
	char peekc, inpc;
	char tmpstr[50];
	bzero(tmpstr,sizeof(tmpstr));
	token *curr_token =NULL;
	type_of_token t_typ = 0; //invalid token 
	
	while(!feof(source))
	{
		peekc= look(source);
		//while(peekc == ' ' || peekc == 0x09)
		//	peekc = (char)fgetc(source);
		inpc = (char)fgetc(source);
		while(true)
		{
			if(isalpha(inpc))	/* symbol */
			{
				int i=0;
				tmpstr[i]=inpc;
				i++;
				bool kw = false;
				while(true)
				{
					peekc = look(source);
					if(isalnum(peekc) || peekc == '_')
					{
						inpc = (char)fgetc(source);
						tmpstr[i] = inpc;
						i++;
					}
					else
					{
						tmpstr[i]='\0';
						break;
					}
				}
				
				for(i =0; i< keyword_count; i++)
				{
					if(strcasecmp(tmpstr,keywords[i])==0)
					{
						kw = true;
						break;
					}
				}
				if(kw == false)
				{
					t_typ = t_ident;
					if(strcasecmp(tmpstr,"AS")==0)
						t_typ = t_skip;
					for(i=0;i')
							{
								t_typ = t_neq;
								inpc = fgetc(source);
								tmpstr[i] = inpc;
								i++;
							}
							else if( peekc == '=')
							{
								t_typ = t_leq;
								inpc = fgetc(source);
								tmpstr[i] = inpc;
								i++;
							}
							break;

					case '>':	t_typ = t_gt;
							peekc = look(source);
							if(peekc == '=')
							{
								t_typ = t_geq;
								inpc = fgetc(source);
								tmpstr[i] = inpc;
								i++;
							}
							break;
					case '+':
					case '-':	t_typ = t_addop;
							break;
					case '*':
					case '/':	t_typ = t_mulop;
							break;
					case '=':	t_typ = t_eql;
							break;
					case '(':	t_typ = t_lparen;
							break;
					
				}
				
				break;
			}			
			break;
		}
		if(t_typ != 0)
		{
			curr_token = create_token(tmpstr,t_typ);
			token_list = push_back(curr_token);
			bzero(tmpstr,sizeof(tmpstr));
			curr_token = NULL;
			t_typ = 0;
		}
	}
	fclose(source);
	return true;
}
void do_header(FILE *target)
{
	if(target == NULL)
		exit(-1);
 
	fprintf(target,"#include \n");
	fprintf(target,"#include \n");
	fprintf(target,"#include \n");
	fprintf(target,"#include