Need Help with JSONL Format for finetuning Vertex AI for Paraphrasing (Gemini 1.5 Flash)

Google AI Studio
, ,
1

What I Did

  1. AIS Studio (Google Sheet Approach)
  • I created a Google Sheet with two columns: “Input” and “Output.”
  • Collected around 170 examples of original text vs. paraphrased text.
  • Trained with 6 epochs, a batch size of 4, and everything worked exactly as intended. The model reliably outputs a paraphrased version of whatever I input.
  1. Vertex AI (JSONL Approach)
  • Tried the same examples, but now they have to be uploaded as a JSONL file.
  • Each record in my JSONL file looks like this (based on what I found in documentation and other examples):

json

CopyEdit

{"contents": [{"role": "user", "parts": [{"text": "Leonardo da Vinci: (examining a sketch) Ah, the wings must be stronger, perhaps fashioned from silk rather than parchment."}]}, {"role": "model", "parts": [{"text": "Leonardo da Vinci: (viewing a sketch) Ah, the wings must be stronger, possibly made of silk rather than parchment."}]}]}
{"contents": [{"role": "user", "parts": [{"text": "The air resists, but I am determined to make man fly like the birds."}]}, {"role": "model", "parts": [{"text": "The air resists, but I am determined to make man fly like a bird."}]}]}
{"contents": [{"role": "user", "parts": [{"text": "Niccolò Machiavelli: (leaning back, smirking) A noble ambition, Leonardo."}]}, {"role": "model", "parts": [{"text": "Niccolò Machiavelli smirked as he said, \"A noble ambition, Leonardo.\""}]}]}
  • I used the same hyperparameters (6 epochs, batch size=4).
  • However, once trained, the model doesn’t behave as expected. It doesn’t just output a paraphrased version of the user input. Sometimes it returns entirely different text or extraneous details.

Goal: When I input a sentence or paragraph, I want the model to output a reliable paraphrased version

1 Like