Why does fit() recalculate num_batches no matter what I specify for steps_per_epoch?

TensorFlow General Discussion
, ,
1

I’m having issues where TF can’t calculate the cardinality of my custom dataset in graph execution mode (its dimension is set to None). But specifying None, 1, or a random int like 5 doesn’t change that it’s always trying to calculate it. I’m totally missing any docs on why this is a problem.

model.fit(
        dataset_choice.dataset, 
        epochs=args.epochs, 
        steps_per_epoch = 5, 
        callbacks=[tensorboard_callback])

Traceback (most recent call last):
    history = model.fit(dataset_choice.dataset, epochs=args.epochs, callbacks=[tensorboard_callback]) 
    packages/keras/src/trainers/data_adapters/tf_dataset_adapter.py", line 73, in num_batches
    **cardinality = int(self._dataset.cardinality())**
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'SymbolicTensor'
3

Hi @BreckEmert, Could you please let us know if your custom dataset is a finite dataset or infinite dataset. Does this code run fine in eager mode. If possible please provide the standalone code to reproduce the issue. Thank You.