Create TensocrflowTFRecords at this format

TensorFlow General Discussion
,
1

Greetings!

I need to write Tensorflow TFRecords files in this specific format, for input to an object detection model:

PrefetchDataset element_spec={
    'image': TensorSpec(shape=(None, None, 3), dtype=tf.uint8, name=None), 
    'image/filename': TensorSpec(shape=(), dtype=tf.string, name=None), 
    'image/id': TensorSpec(shape=(), dtype=tf.int64, name=None), 
    'objects': {
        'area': TensorSpec(shape=(None,), dtype=tf.int64, name=None), 
        'bbox': TensorSpec(shape=(None, 4), dtype=tf.float32, name=None), 
        'id': TensorSpec(shape=(None,), dtype=tf.int64, name=None), 
        'is_crowd': TensorSpec(shape=(None,), dtype=tf.bool, name=None), 
        'label': TensorSpec(shape=(None,), dtype=tf.int64, name=None)}
}

I want to implement like this implementation, but at above specified format (where ‘ojbects’ key have another subkeys in it)

Thanks in advance!
Allan S.

3

Hello @allanfreitas

Thank you for using TensorFlow

In the documentation of tf.train.example it is mentioned how to deal with features for example [here] (TFRecord and tf.train.Example  |  TensorFlow Core) you can make create_example function like this example.

object_features = {
        'area': tf.train.Feature(int64_list=tf.train.int64List(value=objects['area'])),
        #similarly we can define for other elements.
    }

    feature_dict['objects'] = tf.train.Feature(bytes_list=tf.train.BytesList(value=[tf.io.serialize_tensor(tf.convert_to_tensor(object_features)).numpy()]))
    
    # Create an Example
    example = tf.train.Example(features=tf.train.Features(feature=feature_dict))

for writing TFRecords we can iterate through every entry and write as binary data.

        for entry in data:
            image = entry['image'] 
            filename = entry['filename']  
            image_id = entry['id']  
            objects = entry['objects'] 
            
            example = create_example(image, filename, image_id, objects)
            writer.write(example.SerializeToString())

For reading TFRecords also you may need to follow the same way to retrieve data while training.