Error in tensor shape when calling model.fit()

TensorFlow
1

Function to apply histogram equalization to each image individually

def histogram_equalization(image, label):
# Convert the TensorFlow tensor to a NumPy array
image_np = image.numpy()

# Apply histogram equalization to each image in the batch
equalized_images = []
for img in image_np:
    # Ensure the image is in the range 0-255 and type uint8
    img = img.astype('uint8')
    img = img.squeeze()
    # Apply OpenCV's histogram equalization (which expects a 2D array of type uint8)
    img_eq = cv2.equalizeHist(img)
    
    img_eq = np.expand_dims(img_eq, axis=-1)
    equalized_images.append(img_eq)

# Convert the list of equalized images back to a tensor
equalized_images = np.array(equalized_images)

return tf.convert_to_tensor(equalized_images, dtype=tf.float32), label

Apply histogram equalization to the dataset

train_data_eq = train_ds.map(lambda image, label: tf.py_function(histogram_equalization, [image, label], [tf.float32, tf.float32]))

The error is
ValueError Traceback (most recent call last)
Cell In[27], line 2
1 epochs=5
----> 2 history = model.fit(
3 train_data_eq,
4 validation_data=val_ds,
5 epochs=epochs
6 )

File /opt/conda/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py:122, in filter_traceback..error_handler(*args, **kwargs)
119 filtered_tb = _process_traceback_frames(e.traceback)
120 # To get the full stack trace, call:
121 # keras.config.disable_traceback_filtering()
→ 122 raise e.with_traceback(filtered_tb) from None
123 finally:
124 del filtered_tb

File /opt/conda/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py:122, in filter_traceback..error_handler(*args, **kwargs)
119 filtered_tb = _process_traceback_frames(e.traceback)
120 # To get the full stack trace, call:
121 # keras.config.disable_traceback_filtering()
→ 122 raise e.with_traceback(filtered_tb) from None
123 finally:
124 del filtered_tb

ValueError: as_list() is not defined on an unknown TensorShape.

1 Like
2

Hi @Hnin_Daiwee, After applying the map function to the train_ds could you please let us know the shape of the elements in the mapdataset using

for element in train_data_eq.take(1):
    print(element.shape)

Also please let us know how you are creating this train_ds dataset. Thank You.

5

@Kiran_Sai_Ramineni , thank you for your attention and time, here are you want to know

train_ds = tf.keras.utils.image_dataset_from_directory(
  train_data_dir,
  seed=123,
  color_mode = 'grayscale',
  label_mode = 'categorical',
  image_size=(img_height, img_width),
  batch_size=batch_size)

  print(image_batch.shape)
  print(labels_batch.shape)
  break

output shape is (32, 450, 450, 1) (32, 4)

7

Hi @Hnin_Daiwee, I suspect that in the map function using tf.py_function is not providing the shape properly. I have tried to set the shape manually and did not get that error.

def apply_histogram_equalization(image, label):
    image, label = tf.py_function(histogram_equalization, [image, label], [tf.float32, tf.float32])
    image.set_shape((None, 256, 256, 1))
    label.set_shape((None, 4))
    return image, label
train_data_eq = train_ds.map(apply_histogram_equalization)

please refer to this gist for a complete code example. Thank You.

1 Like
8

Hello @Kiran_Sai_Ramineni, thank you so much. Your support makes it fixed.