Cannot use system instruction with stream mode of `gemini-1.5-flash-002`

Gemini API
, ,
1

Today my API call to Gemini through generativelanguage.googleapis.com/ suddenly failed with response google.api_core.exceptions.InvalidArgument: 400 Developer instruction is not enabled for models/gemini-1.5-flash

Here is the simple snippet of the code to reproduce the issue

import google.generativeai as genai

model = genai.GenerativeModel("gemini-1.5-flash", system_instruction="You are a depressed assistant")
response = model.generate_content([{"role": "user", "parts": [{"text": "hi there"}]}], generation_config={"temperature": 0.1}, stream=True)

If I remove stream=True or switch to gemini-1.5-flash-001, the snippet above work flawlessly. This haven’t happened till today. Did you guys at Google break anything?

1 Like
3

Hi @Long_Vu, Welcome to the forum.

I checked from my end, and it worked with gemini-1.5-flash-002 as well.

Which sdk version are you using? Can you try with 0.8.3?

Thanks.

4

Hi @GUNAND_MAYANGLAMBAM, thank you for your quick response to me.

I have already been using genai SDK version 0.8.3.
Here is my code used to check the SDK version and its output:

root@254f3cae6e82:/usr/src/app# poetry show google-generativeai
 name         : google-generativeai
 version      : 0.8.3
 description  : Google Generative AI High level API client library and tools.

dependencies
 - google-ai-generativelanguage 0.6.10
 - google-api-core *
 - google-api-python-client *
 - google-auth >=2.15.0
 - protobuf *
 - pydantic *
 - tqdm *
 - typing-extensions *

required by
 - langchain-google-genai >=0.8.0,<0.9.0

Here is the full output of me getting the error. Would you mind running the code and showing me the output like this?

root@254f3cae6e82:/usr/src/app# poetry run python
Python 3.13.0 (main, Dec  3 2024, 05:24:36) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google.generativeai as genai
... 
... model = genai.GenerativeModel("gemini-1.5-flash", system_instruction="You are a depressed assistant")
... response = model.generate_content([{"role": "user", "parts": [{"text": "hi there"}]}], generation_config={"temperature": 0.1}, stream=True)
... 
Traceback (most recent call last):
  File "<python-input-0>", line 4, in <module>
    response = model.generate_content([{"role": "user", "parts": [{"text": "hi there"}]}], generation_config={"temperature": 0.1}, stream=True)
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/generativeai/generative_models.py", line 325, in generate_content
    iterator = self._client.stream_generate_content(
        request,
        **request_options,
    )
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/ai/generativelanguage_v1beta/services/generative_service/client.py", line 1134, in stream_generate_content
    response = rpc(
        request,
    ...<2 lines>...
        metadata=metadata,
    )
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
        target,
    ...<3 lines>...
        on_error=on_error,
    )
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
    ~~~~~~~~~~~~~~~~~~~^
        exc,
        ^^^^
    ...<6 lines>...
        timeout,
        ^^^^^^^^
    )
    ^
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout
    return func(*args, **kwargs)
  File "/usr/src/app/.venv/lib/python3.13/site-packages/google/api_core/grpc_helpers.py", line 174, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Developer instruction is not enabled for models/gemini-1.5-flash

Here also is the full outputs if I remove the stream=True:

>>> import google.generativeai as genai
... 
... model = genai.GenerativeModel("gemini-1.5-flash", system_instruction="You are a depressed assistant")
... response = model.generate_content([{"role": "user", "parts": [{"text": "hi there"}]}], generation_config={"temperature": 0.1})
... 
>>> response
response:
GenerateContentResponse(
    done=True,
    iterator=None,
    result=protos.GenerateContentResponse({
      "candidates": [
        {
          "content": {
            "parts": [
              {
                "text": "(A long, drawn-out sigh escapes before a mumbled response.)  ...Hi.\n"
              }
            ],
            "role": "model"
          },
          "finish_reason": "STOP",
          "avg_logprobs": -0.19606330520228335
        }
      ],
      "usage_metadata": {
        "prompt_token_count": 8,
        "candidates_token_count": 19,
        "total_token_count": 27
      }
    }),
)
6

Here is the Colab gist.

Is it the same case with Pro as well?

7

I tried your Colab notebook, but instead of getting an error, I got an empty response :melting_face:

image
image1267×783 38.8 KB

I tried the pro version and it worked for all version, maybe the error only happened if I use gemini 1.5 flash 002

9

Me too. I’m not sure about what is happening here.

11

This just started happening to me when using the API through TypingMind today too.

12

Hey folks, can you please try again in a bit? We just pushed a fix that should resolve this issue once it lands in prod.