I was playing around the Gemini api’s context caching and batch requests in order to put something together for project 4 of this years DeepMind AI’s GSoC projects.
However, I can’t find a way to pass any reference of the created cache to the batch request. Currently what I’m doing is, create a cache and put that cache name in the input.jsonl and upload it to GCS and point this URI in my batch job. However, batch job gives me some default answers(Ones that are irrelevant to my question).
Once I go view the predictions.jsonl from my storage, I get to see the error.
This is the input.jsonl
{"request": {"contents": [{"role": "user", "parts": [{"text": "What is a turing machine?"}]}], "generationConfig": {"temperature": 0.4}, "tools": [{"cachedContent": "cachedContents/ckhjh0c70hgw"}]}}
{"request": {"contents": [{"role": "user", "parts": [{"text": "What is the main topic of the video that is in the cache context given to you?"}]}], "generationConfig": {"temperature": 0.4}, "tools": [{"cachedContent": "cachedContents/ckhjh0c70hgw"}]}}
I ger irrelevant responses in my terminal like
0 2025-03-07 20:49:51.165000+00:00 {'contents': [{'parts': [{'file_data': None, 'text': 'List objects in this image.'}, {'file_data': {'file_uri': 'gs://cloud-samples-data/generative-ai/image/office-desk.jpeg', 'mime_type': 'image/jpeg'}, 'text': None}], 'role': 'user'}], 'generationConfig': {'temperature': 0.4}} {'candidates': [{'avgLogprobs': -0.181431194521346, 'content': {'parts': [{'text': 'Here are the objects in the image:\n- Globe\n- Eiffel Tower\n- Airplane\n- Tablet\n- Shopping cart\n- Present\n- Coffee cup\n- Keyboard\n- Mouse\n- Passport\n- Sunglasses\n- Money\n- Notebook\n- Pen'}], 'role': 'model'}, 'finishReason': 'STOP'}], 'createTime': '2025-03-07T20:49:51.453664Z', 'modelVersion': 'gemini-2.0-flash-001@default', 'responseId': '71vLZ6DYG6mUhMIPyqGIkAs', 'usageMetadata': {'candidatesTokenCount': 53, 'candidatesTokensDetails': [{'modality': 'TEXT', 'tokenCount': 53}], 'promptTokenCount': 1812, 'promptTokensDetails': [{'modality': 'IMAGE', 'tokenCount': 1806}, {'modality': 'TEXT', 'tokenCount': 6}], 'totalTokenCount': 1865}} -0.181431 STOP [{'text': 'Here are the objects in the image:
- Globe
- Eiffel Tower
- Airplane
- Tablet
- Shopping cart
- Present
- Coffee cup
- Keyboard
- Mouse
- Passport
- Sunglasses
- Money
- Notebook
- Pen'}]
In prediction.jsonl, I see the 400 error.
{"status":"Bad Request: {\"error\": {\"code\": 400, \"message\": \"Invalid JSON payload received. Unknown name \\\"cachedContent\\\" at 'tools[0]': Cannot find field.\", \"status\": \"INVALID_ARGUMENT\", \"details\": [{\"@type\": \"type.googleapis.com/google.rpc.BadRequest\", \"fieldViolations\": [{\"field\": \"tools[0]\", \"description\": \"Invalid JSON payload received. Unknown name \\\"cachedContent\\\" at 'tools[0]': Cannot find field.\"}]}]}}","processed_time":"2025-03-13T18:58:33.522+00:00","request":{"contents":[{"parts":[{"text":"What is a turing machine?"}],"role":"user"}],"generationConfig":{"temperature":0.4},"tools":[{"cachedContent":"cachedContents/ckhjh0c70hgw"}]},"response":{}}
{"status":"Bad Request: {\"error\": {\"code\": 400, \"message\": \"Invalid JSON payload received. Unknown name \\\"cachedContent\\\" at 'tools[0]': Cannot find field.\", \"status\": \"INVALID_ARGUMENT\", \"details\": [{\"@type\": \"type.googleapis.com/google.rpc.BadRequest\", \"fieldViolations\": [{\"field\": \"tools[0]\", \"description\": \"Invalid JSON payload received. Unknown name \\\"cachedContent\\\" at 'tools[0]': Cannot find field.\"}]}]}}","processed_time":"2025-03-13T18:58:33.522+00:00","request":{"contents":[{"parts":[{"text":"What is the main topic of the video that is in the cache context given to you?"}],"role":"user"}],"generationConfig":{"temperature":0.4},"tools":[{"cachedContent":"cachedContents/ckhjh0c70hgw"}]},"response":{}}
No mentor is yet assigned for the task so any help is appreciated, thanks!