Why, when creating models in Keras sequentially, do they somehow depend on each other, although, probably, they should not?

Question

Neural networks

I have a code in which I create a neural network model in a function with the given parameters (the number of layers and neurons in them, and the like). I use this function to iterate through models in order to find the best model (such a small study). For example, I start with 2 layers of 11 neurons, 2 layers of 12 neurons, ... , 3 layers of 11 neurons ... and so on.
At the same time, after training each model, I build a graph of changes in the loss function and the quality index of the model by epochs. It would seem that I just have to look at the graphs generated during the enumeration and choose the best model that has learned better in the minimum number of epochs. But! When I select the best model from the ones iterated over and call the same function only with the already given parameters of the selected model, I get a different result (not the one I saw on the graph during the enumeration)! And I have no idea why.
When iterating, I use the model.summary () method, which displays the characteristics of the model: Layer_name_number ---- size of the input / output, and the like ...
So, when the model creation function is called for the first time in the loop, then model.summary() displays the names of the layers with a number, starting from 1 and, say, the last one with number 5. Then the second time this function is called in the loop with other parameters, model .summary() outputs the name of the first layer with a number starting from 6. As if they are somehow connected, although when the function exits, all created objects should be lost (deleted).
The general picture: sequential enumeration of the model, starting from some model x1 and ending with model xN, and separately create and train a network with parameters like those of xN, then the results are different, namely, the model we came to by enumeration has better indicators on the graph. It seems that Keras, when building the next model, uses the previous training, that is, it is, as it were, pre-trained, and each time it shows the result better and better. Why is this happening?

def plot_res(result, n_epoch, n_hidden, c_hidden, batch_size):
    fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(8, 4))

    ax[0].set_ylim(ymin=-0.05, ymax=1.05)
    ax[0].plot(range(1, n_epoch + 1), result.history['val_loss'],
               linestyle='--', color='g', label='validation')
    ax[0].plot(range(1, n_epoch + 1), result.history['loss'], color='y', label='train')
    ax[0].set_xlabel('Эпохи')
    ax[0].set_ylabel('Функция потерь')
    ax[0].legend(loc='best')
    ax[0].set_title('Изменение функции потерь')

    ax[1].set_ylim(ymin=-0.05, ymax=1.05)
    ax[1].plot(range(1, n_epoch + 1), result.history['val_acc'],
               linestyle='--', color='g', label='validation')
    ax[1].plot(range(1, n_epoch + 1), result.history['acc'], color='y', label='train')
    ax[1].set_xlabel('Эпохи')
    ax[1].set_ylabel('Показатель качества')
    ax[1].legend(loc='best')
    ax[1].set_title('Изменение показателя качества')
    # fig.show()
    fig.savefig('NN_eph{}_c_hd{}_n_hd{}_btch{}.png'.format(n_epoch, c_hidden, n_hidden, batch_size), dpi=300)


# Для тестирования
def neural_network(n_epoch=10,
                   batch_size=64,
                   verbose=0,
                   n_classes=2,
                   optimizer=Adam(),
                   n_hidden=38,
                   validation_split=0.2,
                   reshaped=11,
                   c_hidden=2):

    x_train, x_test, y_train, y_test = prepare_data_base(n_classes, "DATA_BASE.csv")

    model = Sequential()
    model.add(Dense(n_hidden, input_shape=(reshaped,)))
    model.add(Activation('relu'))

    for _ in range(c_hidden):
        model.add(Dense(n_hidden))
        model.add(Activation('relu'))

    model.add(Dense(n_classes))
    model.add(Activation('softmax'))
    model.summary()

    model.compile(loss="binary_crossentropy", optimizer=optimizer, metrics=["accuracy"])

    result = model.fit(x_train, y_train, batch_size=batch_size, epochs=n_epoch,
                       verbose=verbose, validation_split=validation_split)

    score = model.evaluate(x_test, y_test, verbose=verbose)
    print("Test score: ", score[0])
    print("Test accuracy: ", score[1])

    plot_res(result, n_epoch, n_hidden, c_hidden, batch_size)

    model.save("model_eph{}_c_hd{}_n_hd{}_btch{}.json".format(n_epoch, c_hidden, n_hidden, batch_size))

Answer 1

Perhaps the global increment in the layer numbers may be due to the fact that the Tensorflow session is stored globally in Keras in tensorflow_backend.py and the default variable names must be unique within sessions. This is if the backend is TF.