Measure Inference time of TensorFlow Lite

TensorFlow General Discussion
, ,
1

I want to measure the inference time of TensorFlow Lite implemented on a Microcontroller (Nano Sense 33). I am beginner to TFLite and would be thankful if anyone can suggest me about the followig. I don’t know about the structure of the TensorFlow Lite C++ Library. I just trained a sine-value predictor model using a Google Colab and then used a Hellow_World example from Arduino to run that example on Nano 33. The trained model is imported into the Arduino using C Array. We have implemented a DWT time measurement class which produces a time elapsed between different parts of the code/program during exexution. But I am not very familiar with the TensorFlow Lite architecture and need help about:

  1. How can we determine/calculate the inference time of the TFlite? I mean where to initiate the timestamping in the C++ library of the TensorFlow Lite?
  2. Is it possible to know how much layers are being used inside TFlite? and How to measure that what is the time consumed by each layer?

Thanks in advance! Please guide!

3

I guess you can use some of the tools presented here: Performance measurement  |  TensorFlow Lite

Last time I tried this was for Android and it worked super well.