Press "Enter" to skip to content

Making a simple neural-network in R

I’ll assume you have installed R and have it open (enter R in the terminal),
everything else should be covered and keywords will be hyper-linked to the relevant
information. You’ll need another terminal open for Perl.
First off, let’s use Perl to create some training and test data in the form of two csv
files. train.csv will contain the numbers 1 to 10 and their squares, test.csv will
contain a list of the squares of 11 to 20. We’re going to try and use a neural-network
to predict numbers from their squares.

perl -e 'print "Numbers,Squares\n";
print "$_," . $_ ** 2 . "\n" foreach 1..10' > train.csv
perl -e 'print "Numbers, Squares\n";
print "$_," . $_ ** 2 . "\n" foreach 11..20' > test.csv

Next we need to install the package nerualnet in R:

install.packages("neuralnet")

You can load it with:

library("neuralnet")

Now we need to load the csvs we created into a data frame:

train = read.csv("train.csv")
test = read.csv("test.csv")

You can view the training data by entering it’s name, for example:

train

Now it’s time to train a neural-network on the train data-frame:

nn = neuralnet(Numbers~Squares, data=train)

This means call the function neuralnet with our arguments, which are seperated by
commas, inside the brackets. The first part is our equation, we want to work out
the Numbers column by looking at the Sqaures column, and the next argument is
that the columns are to be found in the train dataframe. The results are saved to
nn. The predictions made for the numbers column based on the Squares column are
saved in the value net.result (we are looking for the numbers 1 to 10 here). This can
be accessed like so:

nn$net.result

Which outputs:

[[1]]
      [,1]

1 1.451933393
2 1.941955387
3 2.717870764
4 3.718855026
5 4.863158653
6 6.056196214
7 7.203804950
8 8.227693615
9 9.077706509
10 9.736236374

A visualization of the neural-network can be seen using:

plot(nn)

Which shows:

if you want to save the plot you can enter these commands while the window is open:

dev.copy(png,filename="nnplot1.png")
dev.off()

We’re close, ish. We can improve the accuracy of the neural-network by changing
parameters such as the number of hidden units:

nn = neuralnet(Numbers~Squares, data=train, hidden=5)

The neural-network now looks like this:

and gives these results:

[[1]]
          [,1]
1  1.000191765
2  1.999393931
3  3.002453611
4  3.994537580
5  5.005497580
6  6.001400284
7  6.997401654
8  7.995421386
9  9.008071063
10 9.995713027

Better, but how accurate is the neural-network on data it hasn’t seen yet? We can
make predictions on the testing data like this:

compute(nn,test[,2])

Where nn is where we stored the neural-network and test is the data-frame we read
the testing data in to. [,2] means we’re making predictions based on the second column. The predictions (remember we’re looking for the numbers 11
to 20 here) outputted were:

$net.result
             [,1]
 [1,] 10.75356770
 [2,] 11.16308276
 [3,] 11.33522546
 [4,] 11.40045343
 [5,] 11.42471473
 [6,] 11.43381937
 [7,] 11.43725379
 [8,] 11.43853622
 [9,] 11.43900303
[10,] 11.43916683

Terrible! So what’s going on here? This neural network is no good at making predictions on data it hasn’t seen.  It’s not a good thing that our Squares
column is rising exponentially while our numbers column is not. A simple way to smooth the data is to log the “Squares” column:

test$Squares = log(test$Squares)

Which means, make the values in the Squares column in the train data-frame equal
to the value of the log of the values in the squares column in the train data-frame.
Do the same for the test data-frame and make a neural-network for the revised
data. The prediction on the test data now looks like:

$net.result
             [,1]
 [1,] 10.78986615
 [2,] 11.62057801
 [3,] 12.41011568
 [4,] 13.15946564
 [5,] 13.86999593
 [6,] 14.54331424
 [7,] 15.18116526
 [8,] 15.78535712
 [9,] 16.35770917
[10,] 16.90001553

That’s an improvement, but we’re still missing something. Having only 10 instances
of data to train on is appalling. Let’s create some new data with Perl, but this time
the training data will have 100 numbers and their squares:

perl -e 'print "Numbers,Squares\n"; print "$_,", $_ ** 2, "\n" foreach 1..100' > train.csv
perl -e 'print "Numbers,Squares\n"; print $_ ** 2, "\n" foreach 101..110' > test.csv

You’ll have to load the new data into data-frames as earlier, and alter the Squares
columns. Make a neural-network with the new data. Of course there is now a new
problem:

Warning message:
algorithm did not converge in 1 of 1 repetition(s)

You can pass the stepmax argument to neuralnet in the usual way, note you can
enter numbers in their e-notation:

nn = neuralnet(Numbers~Squares, data=train, hidden=5, stepmax=2e6)

The predictions (we’re now looking for the numbers 101 to 110) are now:

$net.result
             [,1]
 [1,] 100.9460653
 [2,] 101.9297390
 [3,] 102.9112996
 [4,] 103.8906277
 [5,] 104.8676041
 [6,] 105.8421098
 [7,] 106.8140262
 [8,] 107.7832355
 [9,] 108.7496205
[10,] 109.7130649

Pretty good for a simple example! The take away lessons are to have as much data
as possible and exercise data scaling.

Further Reading:

How to use neuralnet
Data Cleaning

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *