I trained a Keras CNN model for a classification task using grayscale images (48x48x1). After training and saving the model, I now would like to use the model in an Android application, so I converted the trained model to a TFLite model.
After conversion, I did two things in order to confirm that the converted model works properly:
I tested running inferences on some grayscale test images using the TFLite model in Python, and got proper and expected results (using the same dimensions for the input as I used for training).
I inspected the TFLite model in Netron and confirmed that it expects an input of the following format: float32[1, 48, 48, 1].
The issue I am facing now is with loading input images to the TFLite model using Java in Android Studio. Since my model expects grayscale input images (1 channel), the total input bytes for one frame are supposed to be 48x48x1x4= 9216 bytes (the last 4 multiplied is because of the input being of FLOAT32 format). The problem is that the TensorImage input format, which is what the TFLite model supports, only supports RGB input images, which isnāt compatible with single-channel input images. In addition, Bitmap doesnāt even support storing grayscale images, so I wouldnāt know how to store any grayscale input images into a Bitmap and not have an extra number of bytes that would later cause issues.
I am not an expert in Android development and have slowly been learning and building things, so I understand there might be easy solutions that Iām overlooking. I would appreciate any help that comes my way!
Thanks for the detailed explanation. This issue has been taken care of about a year ago with the introduction of TensorFlow Lite Support TransformToGrayScaleOp.java operator. Check inside the file to see how this is handled.
After finding out about the ImageProcessor support in TFLite, I tried to check if there was any op that would take care of the conversion to grayscale. However, I couldnāt find any, and I just learned that this was due to my using āorg.tensorflow:tensorflow-lite-support:0.1.0ā in my build.gradle file instead of āorg.tensorflow:tensorflow-lite-support:0.2.0ā - only the latter supports the TransformToGrayscaleOp.
One comment that I do have is that no matter how long and hard I searched for a solution online, I was not able to come across any documentation for the TransformToGrayscaleOp online. Iām not sure if it is anything to do with my way of searching, but it would definitely be more convenient if the documentation for things like these were more easily āfindableā.
But again, I really appreciate your help! I was stuck on this for a few days and your answer solves my issue!
One comment that I do have is that no matter how long and hard I searched for a solution online, I was not able to come across any documentation for the TransformToGrayscaleOp online
Hey @ahmadchalhoub99 - Mark from the TF Docs team here. Sorry this wasnāt optimal for you - can you provide some examples of how you tried to find it (e.g. ādid a google search for X, Y, Zā, ābrowsed this repo: ā¦ā)? Be as verbose as you would like
There are many paths to the many docs, and hearing your experiences, even anecdotally, helps us with our content strategy, whether it be adding the right metadata, or building better supplementary content (codelabs, etc).
I wouldnāt be able to remember specifically what phrases I used to search, but it was things along the lines of: āload grayscale image into tensorflow lite model java androidā, ācreate grayscale TensorImageā, āconvert bitmap to grayscale TensorImageā, and things like that.
Thank you so much! It would be really beneficial to also fully understand all the steps of conversion by doing it manually, so I appreciate your very useful response!
Just as a thought, since I only recently started working with Android development (Java), would you recommend that I stick to Java and improve in it, or do you think switching over to Kotlin would be a better thing to do?
@ahmadchalhoub99
Well generally in the future you will get involved into projects that will require both skills. Surely Kotlin is better, with easier functions , null handling and more but you will always get into a situation where you will try to understand a block of code in Java that happens generally in the documentation. Also in the future you will like to contribute to a github library that probably it is going to be in Java. Android Studio has an easy way to insert Java code in Kotlin files by interpreting it and 99% of the cases it does a pretty good job.
So I believe that it is necessary to continue now with Java, have a good understanding of the language and when you feel ready switch to Kotlinā¦ Try to get in touch once in a while with Java though in the futureā¦ Java will always be there in some projects and documentation.