Input tensor 0 is missing TensorMetadata

hissain · November 27, 2023, 4:34am

This article shows how to add meta_data into a tflite model before running in Android. However, it only shows example for Image classification. I am trying to add meta data for Audio classification. How can I get guide.

I am getting error in Android when running a model.tflite which has no meta data.

Error getting native address of native library: task_audio_jni
java.lang.IllegalArgumentException: Error occurred when initializing AudioClassifier: input tensor 0 is missing TensorMetadata.
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.initJniWithModelFdAndOptions(Native Method)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.access$000(AudioClassifier.java:74)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier$1.createHandle(AudioClassifier.java:143)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier$1.createHandle(AudioClassifier.java:136)
at org.tensorflow.lite.task.core.TaskJniUtils$1.createHandle(TaskJniUtils.java:70)
at org.tensorflow.lite.task.core.TaskJniUtils.createHandleFromLibrary(TaskJniUtils.java:91)
at org.tensorflow.lite.task.core.TaskJniUtils.createHandleFromFdAndOptions(TaskJniUtils.java:66)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.createFromFileAndOptions(AudioClassifier.java:134)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.createFromFile(AudioClassifier.java:91)
at com.samsung.classification.MainActivity.startAudioClassification(MainActivity.java:86)
at com.samsung.classification.MainActivity.lambda$showActivitySelectionDialogue$0$com-samsung-classification-MainActivity(MainActivity.java:51)
at com.samsung.classification.MainActivity$ExternalSyntheticLambda0.onClick(Unknown Source:2)
at com.android.internal.app.AlertController$AlertParams$3.onItemClick(AlertController.java:1485)
at android.widget.AdapterView.performItemClick(AdapterView.java:376)
at android.widget.AbsListView.performItemClick(AbsListView.java:1295)
at android.widget.AbsListView$PerformClick.run(AbsListView.java:3571)
at android.widget.AbsListView$3.run(AbsListView.java:4751)
at android.os.Handler.handleCallback(Handler.java:942)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loopOnce(Looper.java:226)
at android.os.Looper.loop(Looper.java:313)

George · November 27, 2023, 2:12pm

Hi @hissain

Is your python code working ok before trying to convert it to .tflite? What example have you used to train the model? Have you used TF Model Maker or a different approach?
The code for inserting the metadata is the same as the one you have provided. Can you share more info about the normalization and the other info for your model that must be added?

George · November 27, 2023, 2:13pm

Check here some examples with the TF Lite Model Maker that adds the metadata automatically

George · November 27, 2023, 2:24pm

Here is the example how TF Lite Model Maker is adding the parameters inside the model.

github.com

tensorflow/examples/blob/master/tensorflow_examples/lite/model_maker/core/task/model_spec/audio_spec.py

# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Audio model specification."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import abc

This file has been truncated. show original

You can se also which are the parameters based on if it is YamnetSpec or BrowserFftSpec

hissain · November 28, 2023, 6:29am

It was working OK in the same Notebook as I have found evaluation score at around 90% accuracy. I was trying to install tflite_model_maker however, it is failing to install because of dependency conflicts with tflite_support. Is there any guideline on it?

Please refer to the issue I am facing. python - ERROR: Cannot install tflite-model-maker (The conflict is caused by other modules) - Stack Overflow
@George_Soloupis

George · November 28, 2023, 9:14am

It is an known issue. Please refer to http://discuss.ai.google.dev/t/future-of-tflite-model-maker-and-mediapipe-model-maker/17375 or take a look a recent example I wrote
Use TensorFlow Lite Model Maker with a custom dataset | by George Soloupis | Medium
Inside there you can find a python notebook with example of using a conda env inside Colad.

Regards

hissain · November 29, 2023, 1:51pm

@George_Soloupis your blog provides examples for unix systems. I was trying to run same but tflite model maker installing for indefinite time and storage gets full. Any other suggestions?

George · November 29, 2023, 2:36pm

@hissain The example is working OK inside a Colab session.
No other suggestions I can think of.

hissain · November 29, 2023, 2:53pm

@George_Soloupis can you share any working notebook? I have same issue in colab notebook as well.

George · November 29, 2023, 3:20pm

Inside the blog post there is a link to a notebook…just before the code snippets

hissain · November 29, 2023, 5:17pm

@George_Soloupis could you please help from where can I get the /gdrive/MyDrive/JWick/gun_shot_wav.zip file? I was trying to run the colab and it failed to unzip because I have no zip file in my drive. Here is my notepad copied from your git.

George · November 30, 2023, 4:01am

All you need to use is inside the blog post!

faiaz_amin · June 25, 2024, 12:24pm

Hi @hissain
In this case you need to add tensor flow metadata manually for audio classifications. Here is how I did it.

from tflite_support import flatbuffers
from tflite_support import metadata as _metadata
from tflite_support import metadata_schema_py_generated as _metadata_fb

""" ... """
"""Creates the metadata for an audio classifier."""

# Creates model info.
model_meta = _metadata_fb.ModelMetadataT()
model_meta.name = "model name"
model_meta.description = "Description"
model_meta.version = "v1"
model_meta.author = "Your Name"
model_meta.license = ("Apache License. Version 2.0 "
                      "http://www.apache.org/licenses/LICENSE-2.0.")

# Creates input info.
input_meta = _metadata_fb.TensorMetadataT()

# Creates output info.
output_meta = _metadata_fb.TensorMetadataT()

input_meta.name = "name"
input_meta.description = "description"
input_meta.content = _metadata_fb.ContentT()
input_meta.content.contentProperties = _metadata_fb.AudioPropertiesT()
input_meta.content.contentProperties.channels = no. channels
input_meta.content.contentProperties.sample_rate = rate
input_meta.content.contentPropertiesType = (
    _metadata_fb.ContentProperties.AudioProperties)
input_stats = _metadata_fb.StatsT()
input_meta.stats = input_stats

# Creates output info.
output_meta = _metadata_fb.TensorMetadataT()
output_meta.name = "name"
output_meta.description = "description"
output_meta.content = _metadata_fb.ContentT()
output_meta.content.content_properties = {}
output_meta.content.contentPropertiesType = (
    _metadata_fb.ContentProperties.FeatureProperties)
output_stats = _metadata_fb.StatsT()
output_stats.max = [1.0]
output_stats.min = [0.0]
output_meta.stats = output_stats

label_file = _metadata_fb.AssociatedFileT()
label_file.name = os.path.basename("label_file location")
label_file.description = "Description"
label_file.type = _metadata_fb.AssociatedFileType.TENSOR_AXIS_LABELS
output_meta.associatedFiles = [label_file]

# Creates subgraph info.
subgraph = _metadata_fb.SubGraphMetadataT()
subgraph.inputTensorMetadata = [input_meta]
subgraph.outputTensorMetadata = [output_meta]
model_meta.subgraphMetadata = [subgraph]
print(subgraph.inputTensorMetadata[0].content.contentProperties.sample_rate)
print(subgraph.inputTensorMetadata[0].content.contentProperties.audio_format)
b = flatbuffers.Builder(0)
b.Finish(
    model_meta.Pack(b),
    _metadata.MetadataPopulator.METADATA_FILE_IDENTIFIER)
metadata_buf = b.Output()
print(metadata_buf)
populator = _metadata.MetadataPopulator.with_model_file("/lmodel location")
populator.load_metadata_buffer(metadata_buf)
populator.load_associated_files(["label file location"])
populator.populate()

Change the file location as per your system.

Topic		Replies	Views
Input tensor 0 is missing TensorMetadata error General Discussion tflite , raspberry_pi , help_request	3	1589	November 27, 2023
TensorFlow Lite Metadata General Discussion models , help_request , tflite	5	2337	December 22, 2023
Facing issue while loading tensorflow lite model General Discussion tflite , models	6	2081	May 2, 2023
Converting keras to .tflite - unreal classifying results General Discussion android , tflite	0	574	May 15, 2023
TFlite Android Studio Signal Processing Error General Discussion tflite	1	163	March 11, 2024

Input tensor 0 is missing TensorMetadata

Related topics