Confusion Matrix - Image Classification

Hagen_Hagen · October 5, 2021, 3:01am

Is there a way to create a confusion Matrix on Tflite Model Maker - Image Classification that is similar to Audio Classification?

George · October 5, 2021, 4:46am

Hi @Hagen_Hagen

Take a look at those notebooks for showing confusion matrix with sklearn and matplotlib:

github.com

farmaker47/Yamnet_Speech_commands/blob/main/Yamnet_Speech_commands.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Yamnet_Speech_commands.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {

This file has been truncated. show original

and

github.com

farmaker47/Yamnet_Speech_commands/blob/main/Yamnet_Plain_Speech_commands.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Yamnet_Plain_Speech_commands.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {

This file has been truncated. show original

Finally this one with Model maker classes:

github.com

farmaker47/Yamnet_Speech_commands/blob/main/Model_Maker_Speech_commands.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Model_Maker_Speech_commands.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {

This file has been truncated. show original

I think by combining you can print confusion matrix also for Image Classification task.

Best

Hagen_Hagen · October 5, 2021, 5:45am

Hi I followed the 3rd notebook and I get this error.

AttributeError: 'ImageClassifier' object has no attribute 'confusion_matrix'

George · October 5, 2021, 6:04am

Maybe ‘ImageClassifier’ object has no attribute 'confusion_matrix then. So create it customly by following the 2 first notebook with sklearn and matplotlib.

Best

lgusm · October 5, 2021, 10:36am

What I would, while it’s not available on the ImageClassifier, is to copy what AudioClassifier is doing: examples/tensorflow_examples/lite/model_maker/core/task/audio_classifier.py at 1ff6d7aa9bdf9749ada28bf67ad435ca518b6232 · tensorflow/examples · GitHub

the code might be easy to copy/adapt

Hagen_Hagen · October 5, 2021, 2:00pm

Where will I put the code? Inside the image_classifier.py? or just in another cell in Jupyter Notebook?

lgusm · October 5, 2021, 2:58pm

I’d just create another cell with a function.
that is the easiest solution

Hagen_Hagen · October 5, 2021, 2:59pm

I put it inside the image_classifier.py and it worked. Will that be fine?

lgusm · October 5, 2021, 4:10pm

I guess so, if it’s working for you that’s enough, right?

George · October 5, 2021, 5:49pm

@Hagen_Hagen
Create a PR with that fix.

Hagen_Hagen · October 6, 2021, 12:59am

I am getting the same error “AttributeError: ‘ImageClassifier’ object has no attribute ‘confusion_matrix’” when I put it in another cell, I think putting it inside image_classifier.py would be fine since there’s no error and ‘model.confusion_matrix’ worked since I put it inside the image_classifier.py. I just thought it will break the package, but it worked. Thank you so much for the help guys!

Bhuvnesh_Saini · October 9, 2021, 7:49am

Hi @Hagen_Hagen
Checkout this link https://tensorexamples.com/2020/07/23/Quantize-your-model-using-tflite.html