Fine-tuning gemini-1.5-flash

Gemini API
1

i am trying to fine-tune gemini-1.5-flash model;
say, i put 350 QA pairs and set 5 epochs and lr = 1e-5 and batch is 4.
(and using code from tutorial Tutorial de ajuste  |  Gemini API  |  Google AI for Developers for losses charts etc.)
and it trains and shows loss decreased from 229 points to 65 points
then i take name from output like modelapi-db94jne5e29u and add prefix tunedModels/ to it and start fine-tuning with it as base model . it fine-tunes , shows loss again from ~225 to ~65 points.
① what are those points?
② why in 2nd step loss not starts from around 65 points?

3

Hi @Al_Shayda,

  • The points you are observing are the values of the loss function during training.
  • The reason why the loss doesn’t start at around 65 points in second fine tuning, the model starts from the point of the previous fine-tuning and continues learning from there, and this might cause the loss to start higher than 65, depending on the difficulty of the new data.

Could you please share reproducible code for your use case for further assistance.

Thank You