Train a model using 3d float tensors instead of jpg images?

TensorFlow General Discussion
, ,
1

Hello, as the title says I am trying to train a model using a dataset of 3d float tensors as .npy files. I am following the tutorial of the object detection api (Training Custom Object Detector — TensorFlow 2 Object Detection API tutorial documentation)

My problem is that I cannot generate the record files. Running the generate_tfrecord.py script i get an error that says
“cannot identify image file %r” % (filename if filename else fp)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x0000023864E34168>.

which makes sense, really. I tried to work around this by editing the script and changing line 118, where instead of Image.open I put np.load. This successfully creates the record files. But when I try to train the model I get a different error before training starts:

tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [Unable to decode bytes as JPEG, PNG, GIF, or BMP]

Note that when I follow the exact same steps when using a dataset of .jpg files everything works fine. So how can I adapt the code to work directly with the tensors, without converting to and from image files?

3

Hello @John_Couc

Thank you for using TensorFlow,
In the generate_tfrecord.py we need to add the functionality to handle .npy files like this

from PIL import Image
image = Image.fromarray(tensor.astype(np.uint8))

then save the image buffer to jpeg, and return that value to serialization function.

'image': _bytes_feature(tensor_to_jpeg(image))

Another workaround would be to convert .npy files to .jpg beforehand and check the files integrity and then convert them to tf_records.

Thank you.