Access S3 on tensorflow

TensorFlow General Discussion
,
1

I have installed tensorflow 2.8 as well as tensorflow-io 0.24.0 based on discussions on https://tensorflow-prod.ospodiscourse.com/t/how-can-i-access-to-hdfs-file-system-in-the-latest-tensorflow-2-6-0/3876

and am trying to read tfrecords stored on s3 as:

import tensorflow as tf
import tensorflow_io as tfio
pattern = ‘s3://***.tfrecord’
print(tf.io.gfile.glob(pattern))

This throws a familiar:
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme ‘s3’ not implemented (file:****)

Is there a straightforward way of reading/ writing tfrecords from/to s3?

1 Like
3

tensorflow-io package needs to be installed separately as shown below

pip install tensorflow-io

Thank you!