Gemini Fine-Tuned Model - Document Processing Error and Capability Inquiry

Hello,

I am encountering an error when prompting a fine-tuned model with a PDF for analysis, both in AI Studio and using the Gemini API. The entire chat session ceases to function after I prompt it with a document.

I would like to know if a Gemini fine-tuned model has document understanding capabilities. I have been unable to find this information in the official documentation.

Please advise.

Thank you,
Zak

Hi @zak,

Thanks for flagging. I am getting the same error in AI Studio. I will get back to you on this.

1 Like

Hey @zak,

Currently,Tuning only support text input-output pair. Once tuned, it’s text input text output model. A tuned model doesn’t support image input. A pdf might contain image or you can say it will not support pdf as well, so that’s why you are getting this error.

Thanks.

Hello @Govind_Keshari,

Thank you for the clarification. I understand that the fine-tuned Gemini model only supports text input-output pairs.

Using AI Studio and the Gemini API, I have noticed the lack of support for a “system prompt” with fine-tuned models. While the documentation (Gemini Model Tuning Limitations ) mentions the absence of chat-style multi-turn conversations, it doesn’t explicitly address the use of a system prompt to guide the model’s behavior.

The absence of a system prompt would prevent the fine-tuned model from effectively utilizing RAG (Retrieval-Augmented Generation) or processing text extracted from PDF documents, as these techniques typically rely on providing contextual instructions via a system prompt.

Could you please confirm whether system prompts are indeed unsupported for fine-tuned Gemini models? If so, are there any alternative strategies recommended for incorporating contextual information or instructions when using RAG or PDF-derived text with a fine-tuned model?

Thanks,
Zak

Hey @zak,

Unfortunately, system instruction is also not supported with tuned model. It is also not mentioned in the doc. Here is the some limitations.

1 Like