Tflite-model-maker DataLoader Bug

TensorFlow General Discussion
, , ,
1

So long story short, makesense.ai csv export is not compatible with AutoML, so I am trying to load my data using VOC XML (which I assume is equivalent to PASCAL VOC).

When I try to load the label map, I get this issue:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-b16cb19650e1> in <module>()
----> 1 train_data, validation_data, test_data = object_detector.DataLoader.from_pascal_voc('/content/drive/MyDrive/Colab\ Notebooks/example_data\ frames\ 16fps','/content/drive/MyDrive/Colab\ Notebooks/labels_deep-learning_2021-08-26-03-32-39',{1: 'person', 2: 'notperson'})

TypeError: cannot unpack non-iterable DataLoader object

Pretty sure my arguments are correct according to this documentation: tflite_model_maker.object_detector.DataLoader  |  TensorFlow Lite

I have passed the label_map as a copy/paste from the documentation, my images_dir argument is my folder with all of my images, and my annotations_dir argument is my folder with all of my xml files.

Note, I have more images than annotations - I don’t know if this causes problems but I assume that all that matters is that the images referenced in annotations exist in the images_dir - extra images not referenced by the xml annotation files will just be ignored.

Thoughts?

3

Erik, do you have a short snippet on a Colab with what you are trying to do? that would make it easier to help you.

@Yuqi_Li might be able to help

5

I’m trying to utilize one of the two (one, two) Colab tutorials (they are very similar but have slight differences - is there a reason to use one over the other?) that retrain efficientDet-lite on a custom dataset and compile for coral edge tpu.

All I am changing from these tutorials is the data to be trained on - I am uploading my files to drive, mounting the drive in the notebook, and then trying to use DataLoader on the VOC xml data, and I get this issue.

The issue I have, is that I can find no annotation tool that is open source that exports directly to csv in the AutoML format (labelimg does but I cannot get that to work on macOS).

So I am trying to load annotation data from makesense.ai exported as VOC xml, but this is not working in the Colab, so my model cannot get trained :frowning:

Here’s my modified code from the tutorial referenced as “two” above:

from google.colab import drive

drive.mount('/content/drive')

images_dir = '/content/drive/MyDrive/Colab\ Notebooks/example_data\ frames\ 16fps'

annotations_dir = '/content/drive/MyDrive/Colab\ Notebooks/labels_deep-learning_2021-08-26-03-32-39'

label_map = {1: 'person', 2: 'not-person'}

train_data, validation_data, test_data = object_detector.DataLoader.from_pascal_voc(images_dir,annotations_dir,label_map)

As mentioned before, I have more images referenced than are annotated - but I would think that those images that are not referenced in the annotation would just be ignored.

If my label_map doesn’t match the labels in my annotations, will that cause this error?

6

Hi Erik,

regarding why two samples, is that they have different audience, a basic one on how to use Model Maker (2) e the specific one for Edge TPU.

regarding the other questions I prefer to defer to @Yuqi_Li

cc: @Tian_LIN @khanhlvg

8

The error indicates that you cannot unpack the tuple train_data, validation_data, test_data. Actually, the API only returns a DataLoader.

So, it should be:

dataloader = object_detector.DataLoader.from_pascal_voc(...)
2 Likes
10

You may find this tutorial helpful regarding your question: https://youtu.be/kjuStyfl6yk