Hi there,
I have been trying to fine-tune a Gemini model for use in a production environment. I followed the suggestions here for service-account authentication (the section “Use a service account to authenticate”) and here for fine-tuning.
The fine-tuning RPCs succeed and when I call list_tuned_models
, I get:
TunedModel(name='tunedModels/fine-tuned-003',
source_model='models/gemini-1.0-pro-001',
base_model='models/gemini-1.0-pro-001',
display_name='',
description='',
temperature=0.9,
top_p=1.0,
top_k=1,
state=<State.ACTIVE: 2>,
create_time=datetime.datetime(2024, 4, 27, 15, 9, 47, 340406, tzinfo=datetime.timezone.utc), update_time=datetime.datetime(2024, 4, 27, 15, 9, 47, 340406, tzinfo=datetime.timezone.utc),
tuning_task=TuningTask(start_time=None,
complete_time=None, snapshots=[],
hyperparameters=Hyperparameters(epoch_count=100,
batch_size=4,
learning_rate=0.001)))
However, if I try to call inference like so:
genai.GenerativeModel(model_name="tunedModels/fine-tuned-003").generate_content("asdf")
I get and HTTP 500 with:
An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting
This is not a very specific error message! How should I debug?