Tensorflow.data.experimental.load not working inside tensorflow.data.Dataset().flatmap

TensorFlow General Discussion
,
1

I am trying to load data inside flat_map. Inside map_func used in flat_map, Im unable to get file name from input argument.
`filenames = [“data4/0/31”, “data4/0/32”]
dataset = tf.data.Dataset.from_tensor_slices(filenames)
def parse_fn(filename):
print(filename)
return tf.data.experimental.load(filename)
dataset = dataset.interleave(lambda x:
parse_fn(x),
cycle_length=4, block_length=16)

for item in dataset.as_numpy_iterator():
print(item)`

I already posted a question in stackoverflow.
python - Trying to load dataset inside flat_map got error ‘TypeError: expected str, bytes or os.PathLike object, not Tensor’ - Stack Overflow

3

Hi @Dinesh_Kuruba, Could you please try with the below modified code and let us know if the problem is resolved or not.

filenames = ['data4/0/31', 'data4/0/321']
dataset = tf.data.Dataset.from_tensor_slices(filenames)
def parse_fn(filename):
  return tf.data.Dataset.load(filename,element_spec=tf.TensorSpec(shape=[None,]))
dataset = dataset.interleave(lambda x:parse_fn(x),cycle_length=4, block_length=16)

Thank You.