First steps in Keras - Error

Keras
,
1

Hello,

I’m trying to get into Keras and tried this example: Image classification from scratch

Sadly, after ~15min of training the model I get this error and can’t find why:

Epoch 1/50
 51/586 [=>............................] - ETA: 22:58 - loss: 0.7538 - accuracy: 0.5607
Corrupt JPEG data: 1153 extraneous bytes before marker 0xd9
 54/586 [=>............................] - ETA: 22:51 - loss: 0.7506 - accuracy: 0.5625
Corrupt JPEG data: 396 extraneous bytes before marker 0xd9
121/586 [=====>........................] - ETA: 20:21 - loss: 0.7217 - accuracy: 0.5837
Corrupt JPEG data: 65 extraneous bytes before marker 0xd9
156/586 [======>.......................] - ETA: 19:08 - loss: 0.7039 - accuracy: 0.5990
Corrupt JPEG data: 239 extraneous bytes before marker 0xd9
277/586 [=============>................] - ETA: 14:13 - loss: 0.6671 - accuracy: 0.6269
Corrupt JPEG data: 2226 extraneous bytes before marker 0xd9
321/586 [===============>..............] - ETA: 12:15 - loss: 0.6515 - accuracy: 0.6395
Corrupt JPEG data: 162 extraneous bytes before marker 0xd9
323/586 [===============>..............] - ETA: 12:09 - loss: 0.6510 - accuracy: 0.6399
Warning: unknown JFIF revision number 0.00
347/586 [================>.............] - ETA: 11:04 - loss: 0.6477 - accuracy: 0.6428
Corrupt JPEG data: 128 extraneous bytes before marker 0xd9
384/586 [==================>...........] - ETA: 9:23 - loss: 0.6404 - accuracy: 0.6475
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
/tmp/ipykernel_407/871483020.py in <module>
     11     metrics=["accuracy"],
     12 )
---> 13 model.fit(
     14     train_ds, epochs=epochs, callbacks=callbacks, validation_data=val_ds,
     15 )

/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb

/usr/local/lib/python3.9/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     52   try:
     53     ctx.ensure_initialized()
---> 54     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     55                                         inputs, attrs, num_outputs)
     56   except core._NotOkStatusException as e:

InvalidArgumentError: Graph execution error:

Unknown image file format. One of JPEG, PNG, GIF, BMP required.
	 [[{{node decode_image/DecodeImage}}]]
	 [[IteratorGetNext]] [Op:__inference_train_function_6829]

I looked into the files but there are only the .jpg files provided by the tutorial.

3

Maybe files are corrupted. Did you try downloading them again?

4

I tried it two times, but I can try again.
At first, I thought its the two Thumb.db files in the two folders and deleted them, but this didn’t work.

5

Deleting corrupted files is mentioned in the example.
Did you try running this script? It should remove 1500 files.

image
image745×581 23.5 KB

7

I did run it, but is says zero images deleted, this is my whole notebook, maybe there is something I oversaw: https://filebox.fhooecloud.at/index.php/s/YCXDrEZz7bYp4Zq

8

First thing I would try is to debug this script.
You can try printing file name in this loop to see if theres something wrong with path.

import os

num_skipped = 0
for folder_name in ("Cat", "Dog"):
    folder_path = os.path.join("PetImages", folder_name)
    for fname in os.listdir(folder_path):
        print(fname) # <---- add this
        fpath = os.path.join(folder_path, fname)
        try:
            fobj = open(fpath, "rb")
            is_jfif = tf.compat.as_bytes("JFIF") in fobj.peek(10)
        finally:
            fobj.close()

        if not is_jfif:
            num_skipped += 1
            # Delete corrupted image
            os.remove(fpath)

print("Deleted %d images" % num_skipped)
9

I tried it again completely from the beginning and it is deleting the files (but not the same amount as in the example) so maybe this is not the problem: https://filebox.fhooecloud.at/index.php/s/Go6maP6ZMnsqEdd

11

After many trial an error it worked with this code to delete corrupted files (it deleted two more files compared to the previous one):

#Filter out corrupted images

from PIL import Image

num_skipped = 0

for folder_name in ("Cat", "Dog"):
    folder_path = os.path.join("PetImages", folder_name)
    for fname in os.listdir(folder_path):
        fpath = os.path.join(folder_path, fname)
        if fname.endswith('.jpg'):
            try:
                img = Image.open(fpath) # open the image file
                #print("%s", fpath)
                exif_data = img._getexif()
                img.verify() # verify that it is, in fact an image
            except:
                num_skipped += 1
                # Delete corrupted image
                os.remove(fpath)
print("PIL deleted %d images" % num_skipped)
1 Like
13

@Tropaion,

Because of corrupted files, you received this error. Before training your model, you have to filter out corrupted images as shown here.

I am successfully able to run code on Colab using TF v2.8. Please find the gist here for reference.Thanks!

15

@chunduriv I don’t want to be rude, but always giving me the same answer doesn’t help me. Since I used exactly the same code without any difference and it still doesn’t work, it has to be something wrong.

1 Like