How to train a custom object detection model which is compatible with ml kit library

TensorFlow General Discussion
, , ,
1

I want to train a custom object detection model (hand detection).
I tried using Tensorflow’s Colab but it is not detecting any thing.
I used it in my android app but it says model is not compatible with ml kit library.
so can I make this model compatible with ml kit?
or how can I train a new model which is compatible with ml kit.
Thank you!

3

@Abdullah_Ilyas Welcome to tensorflow forum!

While your existing model not compatible with ML Kit might not work immediately, there are options to make it compatible or train a new model specifically for ML Kit.

Making your existing model compatible with ML Kit:

  1. Quantization: Quantize your model from FP32 to TFLite format.
  2. Model conversion: Use the TensorFlow Lite Converter to convert your TFLite model to a format compatible with ML Kit.
  3. Model optimization: Optimize the TFLite model for on-device inference using techniques like quantization-aware training or pruning.
  4. Input and output compatibility: Ensure your model’s input and output formats are compatible with ML Kit’s input image format and expected output format for hand detection.

Training a new model compatible with ML Kit:

  1. Choose a compatible model architecture: Select a model architecture supported by ML Kit, such as MobileNet v2 or EfficientNet.
  2. Pre-trained weights: Use pre-trained weights for the chosen architecture available on TensorFlow Hub.
  3. Custom training: Fine-tune the pre-trained model on your hand detection dataset using the TensorFlow Object Detection API.
  4. Export to TFLite: Export the trained model to TFLite format using the TensorFlow Lite Converter.
  5. Optimize for on-device inference: Apply optimization techniques like quantization-aware training or pruning to the TFLite model.

Here are some additional resources that can help you:

Specific considerations for hand detection:

  • Ensure your training data includes diverse hand poses, positions, and lighting conditions.
  • Adjust the model’s input size and resolution based on the desired hand detection accuracy and performance on your device.
  • Consider using data augmentation techniques like random cropping and flipping to improve the model’sgeneralizability.

Let us know if this helps!

5

Hi there @Tanya.

I was trying to find ways to use efficientdet models with MLKit and I can not find any comprehensive solution to adjust the output of the models to be compatible with what MLKit expects.
I get:

 com.google.android.gms.tasks.RuntimeExecutionException: com.google.mlkit.common.MlKitException: Failed to initialize detector. Unexpected number of dimensions for output index 0: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1).

The output of efficientdetD0 model is as follows:

StatefulPartitionedCall:1600
name: StatefulPartitionedCall:1
tensor: float32
denotation: Feature
The scores of the detected boxes.
location: 600
StatefulPartitionedCall:3598
name: StatefulPartitionedCall:3
tensor: float32
denotation: BoundingBox
The locations of the detected boxes.
location: 598
StatefulPartitionedCall:0601
name: StatefulPartitionedCall:0
tensor: float32
denotation: Feature
The number of the detected boxes.
location: 601
StatefulPartitionedCall:2599
name: StatefulPartitionedCall:2
tensor: float32
denotation: Feature
The categories of the detected boxes.
location: 599

I was trying to encapsulate the pre-trainned model as a layer and add a final outputlayer that would get the data into what MLkit is expecting, however, so far I could not make it work.

Please, Any Hints or guidance would be great!
Thanks in advance!