Help Needed: Converting Fine-Tuned Gemma 3 Vision Model to TensorFlow Lite (TFLite)

Telli_Koroma · July 21, 2025, 3:49am

Hi everyone,

I’ve successfully fine-tuned the Gemma3ForConditionalGeneration model and have been getting great results. My goal now is to deploy this model on mobile devices for offline use, which requires converting it to the TensorFlow Lite (TFLite) format.

I’ve tried several standard conversion methods, but I’m running into challenges, likely due to the model’s complex multimodal architecture. I’m looking for a reliable workflow or script to handle this conversion.

Key Details:

Model Architecture: Gemma3ForConditionalGeneration
Special Tokens: The model uses several special tokens, including <bos> (ID: 2), <image_soft_token> (ID: 262144), (ID: 255999), and <end_of_image> (ID: 256000).
Input Format: The model expects a specific input sequence combining text and image tokens. Each image is represented by 256 image tokens.

Has anyone successfully converted a fine-tuned Gemma 3 vision model (or a similar multimodal model like PaliGemma) to TFLite? Any scripts, tutorials, or guidance on the correct process would be extremely helpful.

Thank you in advance for your help!

George · July 21, 2025, 5:26am

Can you check this procedure?

github.com/google-ai-edge/ai-edge-torch

ai_edge_torch/generative/examples/paligemma/convert_to_tflite.py

main

# Copyright 2024 The AI Edge Torch Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

"""Example of converting a PaliGemma model to multi-signature tflite model."""

from absl import app
from ai_edge_torch.generative.examples.paligemma import paligemma
from ai_edge_torch.generative.utilities import converter

This file has been truncated. show original

Having instructions from this:

I do not see a way to convert the gemma3n though (for now). This .task file consists of several tflite files inside.

Telli_Koroma · July 21, 2025, 8:48am

Thanks george. I tried this route but couldn’t get it to work. I might take a second look. thanks!

BalakrishnaCh · July 25, 2025, 10:13am

Hi @Telli_Koroma ,

To covert the Gemma models to TFLite format, you can utilize the MediaPipe ai-edge conversion script from the following GitHub repository. The example code script which is provided is for PaliGemma. If you would like to explore for Gemma2 or Gemma3 please visit the Gemma and Gemma3 packages under example package in the same GitHub repo.

Thanks.

Telli_Koroma · August 15, 2025, 11:22am

Thanks! This approach worked!

Topic		Replies	Views
How to convert trained HuggingFace PaliGemma Model to TFLite? General Discussion models , tflite , tensorflow	6	570	July 1, 2025
Help converting tflite models with mediapipe Google AI Edge tflite-support , mediapipe	2	407	January 22, 2025
Export/Build tf2 ssd-resnet model from ckpt.data to tflite General Discussion tflite , model_garden , help_request	5	2968	June 17, 2021
Issues in deciphering TensorFlowLite Interpreter output General Discussion models , android , tflite , help_request	10	1348	June 22, 2021
Change Dimension of tflite model General Discussion tflite , help_request	8	3944	September 15, 2021

Help Needed: Converting Fine-Tuned Gemma 3 Vision Model to TensorFlow Lite (TFLite)

Related topics