HTTP 500 when using custom tuned model

Gemini API
1

Hi there,

I have been trying to fine-tune a Gemini model for use in a production environment. I followed the suggestions here for service-account authentication (the section “Use a service account to authenticate”) and here for fine-tuning.

The fine-tuning RPCs succeed and when I call list_tuned_models, I get:

TunedModel(name='tunedModels/fine-tuned-003',
           source_model='models/gemini-1.0-pro-001',
           base_model='models/gemini-1.0-pro-001',
           display_name='',
           description='',
           temperature=0.9,
           top_p=1.0,
           top_k=1,
           state=<State.ACTIVE: 2>,
           create_time=datetime.datetime(2024, 4, 27, 15, 9, 47, 340406, tzinfo=datetime.timezone.utc),                                                      update_time=datetime.datetime(2024, 4, 27, 15, 9, 47, 340406, tzinfo=datetime.timezone.utc),
           tuning_task=TuningTask(start_time=None,
                                  complete_time=None,                                                                                                                               snapshots=[],
                                  hyperparameters=Hyperparameters(epoch_count=100,
                                                                  batch_size=4,
                                                                  learning_rate=0.001)))

However, if I try to call inference like so:

genai.GenerativeModel(model_name="tunedModels/fine-tuned-003").generate_content("asdf")

I get and HTTP 500 with:

An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting

This is not a very specific error message! How should I debug?

2

This seemed to be due to using an older pip of google-generativeai. Weird.

But, fixed!

3

Thanks for sharing the cause of the issue - appreciate it. Glad it’s fixed!