I'm trying to create a neural network of two hidden layers on Java. The data would be separated into two sets, training set and testing set. Both sets has 75 set of 4 data each row, and each class has their desired output (target). In terms of calculation, it will be doing forward and back propagation respectively, using Sigmoid function as its' activation function. Doing so, I have the initialization part in the following :
public static void main(String[] args) {
// TODO Auto-generated method stub
// Declaration and Initiation
double[][] training = new double[75][4];
double[][] testing = new double[75][4];
double[][] target = new double[75][3];
BufferedReader br = null;
BufferedReader br2 = null;
try {
br = new BufferedReader(new FileReader("filepathhere\\training.csv"));
br2 = new BufferedReader(new FileReader("filepathhere\\testing.csv"));
} catch (FileNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
String line = "";
int row = 0, i;
try {
while ((line = br.readLine()) != null) {
String[] token = line.split(",");
for (i = 0; i < 4; i++) {
training[row][i] = Double.parseDouble(token[i]);
}
if (row < 75) {
row++;
} else {
break;
}
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
br.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String line2 ="";
int row2 = 0, j;
try {
while ((line2 = br2.readLine()) != null) {
String[] token = line2.split(",");
for (j = 0; j < 4; j++) {
testing[row2][j] = Double.parseDouble(token[j]);
}
if (row2 < 75) {
row2++;
} else {
break;
}
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
br2.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Defining desired output on each classes
for (int k = 0; k < 25; k++) {
target[k][0] = 1.0;
target[k][1] = 0.0;
target[k][2] = 0.0;
}
for (int k = 25; k < 50; k++) {
target[k][0] = 0.0;
target[k][1] = 1.0;
target[k][2] = 0.0;
}
for (int k = 50; k < 75; k++) {
target[k][0] = 0.0;
target[k][1] = 0.0;
target[k][2] = 1.0;
}
...Input is as follows for training.csv (https://drive.google.com/open?id=1zwdcnm_tVcPZD5N0Mt2O9DXE9tBEGVBh) and testing.csv (https://drive.google.com/open?id=1OGMAygObZVwvwGB6uQffmGX3Zie-GpIw).
For output, there will be three : base value, followed by values on first & second layer respectively. I have this one that is unsatisfactory :
Results are follows :
1 : 1.0 00
1 : 1.0 00
1 : 1.0 00
2 : 1.0 00
2 : 1.0 00
2 : 1.0 00
3 : 1.0 00
3 : 1.0 00
3 : 1.0 00
4 : 1.0 00
4 : 1.0 00
4 : 1.0 00
5 : 1.0 00
5 : 1.0 00
5 : 1.0 00
6 : 1.0 00
6 : 1.0 00
6 : 1.0 00
.
.
.
.
75 : 1.0 00
75 : 1.0 00
75 : 1.0 00How can I achieve target output, which will become three different values each, and shows how each neural networks differs each in terms of their output values?
Here is the core programming, including the Sigmoid and Derivative-Sigmoid part. I am convinced that something is wrong with the propagations' part:
// Forward Propagation
int bias = 1;
double learningrate = 0.03;
for (int count = 1; count < 10000; count++) {
for (int k = 0; k < 75; k++) {
for (int l = 0; l < 3; l++) {
op_H1[l] = bias;
for (int m = 0; m < 4; m++) {
op_H1[l] += training[k][m] * weight_I_H1[l][m];
op_H1[l] = Sigmoid(op_H1[l]);
}
}
for (int l = 0; l < 3; l++) {
op_H2[l] = bias;
for (int m = 0; m < 3; m++) {
op_H2[l] += op_H1[l] * weight_H1_H2[l][m];
op_H2[l] = Sigmoid(op_H2[l]);
}
}
for (int l = 0; l < 3; l++) {
op_O[l] = bias;
for (int m = 0; m < 3; m++) {
op_O[l] += op_H2[l] * weight_H2_O[l][m];
op_O[l] = Sigmoid(op_O[l]);
}
// Back Propagation
}
for (int l = 0; l < 3; l++) {
delta3[l] = (target[k][l] - op_O[l] * DSigmoid(op_O[l]));
}
for (int l = 0; l < 3; l++) {
for (int m = 0; m < 3; m++) {
weight_H2_O[l][m] += learningrate * op_H2[m] * delta3[l];
}
}
for (int l = 0; l < 3; l++) {
for (int m = 0; m < 3; m++) {
delta2[l] = DSigmoid(op_H2[l]) * weight_H2_O[m][l] * delta3[m];
}
}
for (int l = 0; l < 3; l++) {
for (int m = 0; m < 3; m++) {
weight_H1_H2[l][m] += learningrate * op_H1[m] * delta2[l];
}
}
for (int l = 0; l < 3; l++) {
for (int m = 0; m < 3; m++) {
delta1[l] = DSigmoid(op_H1[l]) * weight_H1_H2[m][l] * delta2[m];
}
}
for (int l = 0; l < 3; l++) {
for (int m = 0; m < 4; m++) {
weight_I_H1[l][m] += learningrate * training[l][m] * delta1[l];
}
}
}
}
// Results Displaying
System.out.println("Results are follows :");
System.out.println("");
for (int k = 0; k < 75; k++) {
for (int l = 0; l < 3; l++) {
op_H1[l] = 1;
for (int m = 0; m < 4; m++) {
op_H1[l] += testing[k][m] * weight_I_H1[l][m];
op_H1[l] = Sigmoid(op_H1[l]);
}
}
for (int l = 0; l < 3; l++) {
op_H2[l] = 1;
for (int m = 0; m < 3; m++) {
op_H2[l] += op_H1[m] * weight_H1_H2[l][m];
op_H2[l] = Sigmoid(op_H2[l]);
}
}
for (int l = 0; l < 3; l++) {
op_O[l] = 1;
for (int m = 0; m < 3; m++) {
op_O[l] += op_H2[m] * weight_H2_O[l][m];
}
op_O[l] = Sigmoid(op_O[l]);
System.out.print((k + 1) + " : " + op_O[l] + "\t");
System.out.println("00");
}
System.out.println("");
}
}
static double Sigmoid(double output) {
return 1 / (1 + Math.pow(Math.E, -output));
}
static double DSigmoid(double output) {
return output * (1 - output);
}
}