How to improve the accuracy of the model?

Question

Python

I'm trying to implement a linear regression model in keras. To do this, I use one input neuron and a bias neuron (although there is no bias in the training sample), a sigdoid function for activation. But the results don't make me happy. How can accuracy be increased? Am I normalizing the input correctly?

import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import *
import numpy as np

#hyperparameters
epochs = 60
epo = np.arange(0, epochs, 1)

n_samples = 40

train_x = np.linspace(0, 20, n_samples) #входы
train_y = -2 * np.linspace(0, 20, n_samples) #+ 4 * np.random.rand(n_samples) #правильные ответы

x = train_x.reshape(-1, 1)
y = train_y.reshape(-1, 1)

leng = 10 ** len(str(int(max(y)))) #для нормализации

x = x / leng
y = y / leng

model = Sequential()
model.add(Dense(units = 1, input_shape = (1, )))
model.add(Activation('sigmoid'))

model.compile(loss='mean_squared_error',
              optimizer='sgd',
              metrics=['mse', 'acc'])
history = model.fit(x = x, y = y, batch_size = 1, epochs = epochs)

test_x = np.linspace(0, 20, n_samples)
w, b = model.get_weights()

plt.subplot(211)
plt.scatter(train_x, train_y)
plt.plot(train_x, train_x * w[0] + b)
plt.title('result')

print(history.history['acc'])

plt.subplot(212)
plt.plot(history.history['loss'], epo)
plt.title('loss')

plt.show()

The plot of the predicted line against the background of the sample is shown at the top, and the plot of the loss is at the bottom.

PS I am new to Keras.

Answer 1

Yours train_yis in [-40, 0] and Activation('sigmoid')casts all values ​​to [0, 1]. Remove the activation, it should converge.

Answer 2

Normalization is easier to do through sklearn.

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
x = sc.fit_transform(x)
y = sc.fit_transform(y)