Overfit and model saving issues

TensorFlow General Discussion
,
1

Hello, my model may have encountered overfitting issue as during the training process, the final accuracy on both the validation and training sets are 1. Moreover, the accuracy on the test set is also 1. However, I am not quite sure whether overfitting has occurred. Here is my log:

Epoch 1/10
2023-05-15 09:30:41.426293: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 3s 97ms/step - loss: 0.3733 - accuracy: 0.9377 - val_loss: 0.6043 - val_accuracy: 0.9208
Epoch 2/10
2023-05-15 09:30:42.535313: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 53ms/step - loss: 0.2549 - accuracy: 0.9836 - val_loss: 0.4312 - val_accuracy: 0.9875
Epoch 3/10
2023-05-15 09:30:43.200824: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.1609 - accuracy: 0.9967 - val_loss: 0.3159 - val_accuracy: 1.0000
Epoch 4/10
2023-05-15 09:30:43.872236: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.1158 - accuracy: 1.0000 - val_loss: 0.2585 - val_accuracy: 1.0000
Epoch 5/10
2023-05-15 09:30:44.548382: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.0969 - accuracy: 1.0000 - val_loss: 0.2022 - val_accuracy: 1.0000
Epoch 6/10
2023-05-15 09:30:45.220167: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.0788 - accuracy: 1.0000 - val_loss: 0.1666 - val_accuracy: 1.0000
Epoch 7/10
2023-05-15 09:30:45.886257: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 55ms/step - loss: 0.0648 - accuracy: 1.0000 - val_loss: 0.1404 - val_accuracy: 1.0000
Epoch 8/10
2023-05-15 09:30:46.562473: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 56ms/step - loss: 0.0542 - accuracy: 1.0000 - val_loss: 0.1242 - val_accuracy: 1.0000
Epoch 9/10
2023-05-15 09:30:47.245306: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 57ms/step - loss: 0.0472 - accuracy: 1.0000 - val_loss: 0.1059 - val_accuracy: 1.0000
Epoch 10/10
2023-05-15 09:30:47.934772: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.0407 - accuracy: 1.0000 - val_loss: 0.0936 - val_accuracy: 1.0000

This is my first question

my second question is:
when I use the statement tf.keras.models.save_model(model, './my_model.h5') to save the model, I get a warning.
CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
warnings.warn('Custom mask layers require a config and must override ’

Will this affect the accuracy of my saved model? How can I solve this problem? Here is my code:

import tensorflow as tf
import os
import matplotlib.pyplot as plt

base_dir='./daten/dash'
train_dir=os.path.join(base_dir,'train')
val_dir=os.path.join(base_dir,'validation')

BATCH_SIZE=30
IMG_SIZE=(160,160)

train_dataset = tf.keras.utils.image_dataset_from_directory(train_dir,
                                                            shuffle=True,
                                                            batch_size=BATCH_SIZE,
                                                            image_size=IMG_SIZE)

validation_dataset = tf.keras.utils.image_dataset_from_directory(val_dir,
                                                                 shuffle=True,
                                                                 batch_size=BATCH_SIZE,
                                                                 image_size=IMG_SIZE)

val_batches=tf.data.experimental.cardinality(validation_dataset)
test_dataset=validation_dataset.take(val_batches//5)
validation_dataset=validation_dataset.skip(val_batches//5)


AUTOTUNE = tf.data.AUTOTUNE

train_dataset = train_dataset.prefetch(buffer_size=AUTOTUNE)
validation_dataset = validation_dataset.prefetch(buffer_size=AUTOTUNE)
test_dataset = test_dataset.prefetch(buffer_size=AUTOTUNE)

data_augmentation=tf.keras.Sequential([
    tf.keras.layers.RandomFlip('horizontal'),
    tf.keras.layers.RandomRotation(0.2),
    tf.keras.layers.RandomCrop(160,160),
    tf.keras.layers.RandomZoom(.5, .2)
])


preprocess_input=tf.keras.applications.mobilenet_v2.preprocess_input


IMG_SHAPE= IMG_SIZE+(3,)
base_model=tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
                                             include_top=False,
                                             weights='imagenet')




image_batch,label_batch=next(iter(train_dataset))

feature_batch=base_model(image_batch)


base_model.trainable=False


base_model.summary()


global_average_layer = tf.keras.layers.GlobalAveragePooling2D()
feature_batch_average=global_average_layer(feature_batch)


prediction_layer=tf.keras.layers.Dense(1)
prediction_batch=prediction_layer(feature_batch_average)



inputs=tf.keras.Input(shape=(160,160,3))

x=data_augmentation(inputs)

x=preprocess_input(x)

x=base_model(x,training=False)

x=global_average_layer(x)

x=tf.keras.layers.Dropout(0.2)(x)

outputs=prediction_layer(x)
model=tf.keras.Model(inputs,outputs)


base_learning_rate = 0.0001
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),
              loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=['accuracy'])

model.summary()


initial_epochs = 10

loss0, accuracy0 = model.evaluate(validation_dataset)


history = model.fit(train_dataset,
                    epochs=initial_epochs,
                    validation_data=validation_dataset)


loss, accuracy = model.evaluate(test_dataset)
print('Test accuracy :', accuracy)


acc = history.history['accuracy']
val_acc = history.history['val_accuracy']



loss = history.history['loss']
val_loss = history.history['val_loss']


tf.keras.models.save_model(model, './my_model.h5')

plt.figure(figsize=(8, 8))
plt.subplot(2, 1, 1)
plt.plot(acc, label='Training Accuracy')
plt.plot(val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.ylabel('Accuracy')
plt.ylim([min(plt.ylim()),1])
plt.title('Training and Validation Accuracy')

plt.subplot(2, 1, 2)
plt.plot(loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.ylabel('Cross Entropy')
plt.ylim([0,1.0])
plt.title('Training and Validation Loss')
plt.xlabel('epoch')
plt.show()
3

@yunpeng_huo,

Yes, your model is overfitting to the dataset and it might fail in cross dataset generalisation.

One possible reason is that only one dense neuron is being trained and the base model is freezed. You can try to add one or more layers before the prediction layer and see the results.

Thank you!