Inaccurate Classification in MobileNet – "Safety Pin" as Top Result

TensorFlow
,
1

Hello, community!

I’m developing a React Native application that uses the TensorFlow.js MobileNet model to classify images taken with the camera or selected from the gallery. The app is working well, but I’m facing an issue with the accuracy of the results.

The problem occurs when classifying certain images, where the highest probability result does not match the detected object. In a recent test, for example, the classification returned “safety pin” as the most likely option, even though the image was not a safety pin.

Here are some details about my image processing flow:

  1. Capture an image via camera or gallery.
  2. Convert the image to base64 and then to a tensor using decodeJpeg.
  3. Resize the image to [224, 224] using resizeBilinear.
  4. Normalize pixel values by dividing by 255.
  5. Pass the image to MobileNet for classification.

My classification code is as follows:

await tensorFlow.ready();
const model = await mobilenet.load();
const imageBase64 = await FileSystem.readAsStringAsync(uri, {
  encoding: FileSystem.EncodingType.Base64,
});
const buffer = tensorFlow.util.encodeString(imageBase64, "base64").buffer;
const raw = new Uint8Array(buffer);
const tensorImage = decodeJpeg(raw);
const resizedTensor = tensorFlow.image.resizeBilinear(tensorImage, [224, 224], true);
const tensorNormalized = resizedTensor.div(255.0);
const result = await model.classify(tensorNormalized as tensorFlow.Tensor3D);
console.log("Classification result:", result);

My Questions:

  1. Could this inaccuracy be related to image preprocessing? Am I handling the input correctly?
  2. Is there any way to improve accuracy, such as fine-tuning the model or a technique to ensure better class matching?
  3. Could any other factors be affecting the results?

I appreciate any help or suggestions! Thank you!