Neural Network Not Learning Correctly

TensorFlow
1

My Neural Network Model is not Training Accurately. Please help me in this regard. My model is as follows

import tensorflow as tf
import numpy as np
import random
from tensorflow import keras
from tensorflow.keras import layers, optimizers, losses, Sequential
from keras.losses import MeanSquaredError
from tensorflow.keras.layers import Dense
import matplotlib.pyplot as plt

x = np.array(range(0,2000),dtype=float)
z = (x + x)**2

minz, maxz = min(z), max(z)
meanz = (minz+maxz)/2
for i in range(0,len(x)):
z[i] = (z[i] - meanz)/(maxz-minz)

x_train = tf.gather(x, indices= range(0,1000))
y_train = tf.gather(z, indices=range(0,1000))

model = keras.Sequential([
keras.layers.Dense(units=64, activation = ‘linear’, input_shape=[1]),
keras.layers.Dense(units=64, activation = ‘linear’),
keras.layers.Dense(units=64, activation = ‘linear’),
keras.layers.Dense(units=32, activation = ‘linear’),
keras.layers.Dense(units=1, activation = ‘linear’)
])
model.summary()
model.compile(keras.optimizers.Adam(learning_rate=0.001), loss = ‘mean_squared_error’, metrics=‘accuracy’)
model.fit(x_train,y_train,epochs=500)

zz = model.predict([10.0])
zp = print((zz*(maxz-minz) + meanz))

with the above input x=10, prediction should be (2*10)**2 but it comes out to be a very large negative value. Please help me in fixing this model.

3

Hi @MUHAMMAD_KAMRAN_BUTT, I have executed the code in colab as see that the y_train data is inaccurate. If you want to train a model to get predictions as (2*input)**2 the training data should also be with that relation. For example

x_train = np.arange(0, 100,dtype=int)
y_train = np.array([(2 * i) ** 2 for i in x_train])

Thank You.

1 Like
4

Thank you for your kind suggestion. I tried, but the problem still persists. If I take less number of data points, it gives prediction close to the original but still error is huge. but if I use x in the range from 0 to 2000 and accordingly y=(2*x)^2, the predicted values are erroneous. I am posting again my code would you please see that. Thank you

import tensorflow as tf
import numpy as np
import random
from tensorflow import keras
from tensorflow.keras import layers, optimizers, losses, Sequential
from keras.losses import MeanSquaredError
from tensorflow.keras.layers import Dense
import matplotlib.pyplot as plt

defining the Data set for X and Y: Y = (2*x)^2

x_train = np.arange(0, 1000,dtype=float)
y_train = np.array([(2 * i) ** 2 for i in x_train])
#print(x_train,‘\n\n’,y_train)

Normalization/ Data Scaling

minx, maxx = min(x_train), max(x_train)
miny, maxy = min(y_train), max(y_train)
for i in range(0,len(x_train)):
x_train[i] = (x_train[i] - minx)/(maxx - minx)
y_train[i] = (y_train[i] - miny)/(maxy - miny)
print(x_train,x_train.shape,y_train,y_train.shape)

Model Defining

model = keras.Sequential([
keras.layers.Dense(units=64, activation = ‘linear’, input_shape=[1]),
keras.layers.Dense(units=64, activation = ‘linear’),
keras.layers.Dense(units=64, activation = ‘linear’),
keras.layers.Dense(units=32, activation = ‘linear’),
keras.layers.Dense(units=1, activation = ‘linear’)
])
model.summary()

model.compile(keras.optimizers.Adam(learning_rate=0.001), loss = ‘mean_squared_error’, metrics=‘accuracy’)
model.fit(x_train,y_train,epochs=500, verbose=2, batch_size = 100)

Prediction: Normalizing x and then Denomalizing the output value

xp = 10
xp = (xp - minx)/(maxx - minx)
zz = model.predict([xp])
zp = print((zz*(maxy-miny) + miny))

6

Hi @MUHAMMAD_KAMRAN_BUTT, I you plot x_train, y_train
image

you can see the data is non linear. You can try using non linear activation like relu in the model layers.For example

model = tf.keras.Sequential([
    tf.keras.layers.Dense(units=1,input_shape=[1]),
    tf.keras.layers.Dense(units=64,activation='relu'),
    tf.keras.layers.Dense(units=1,),
])

You can try with different model architectures for better results. Please refer to this gist for working code example. Thank You.

8

Thank you very much for your kind suggestions. It worked, I changed activation function to “exponential” and now it fits the data.