Gemini 2.5 Pro with empty response.text

Gemini API
1

Running Gemini 2.5 Pro with grounded search sometimes returns empty response.text with finish_reason of STOP and no other reason. When I inspect the response dict it shows evidence of the search with some web meta information, but nothing else.

Any ideas on what is going on?

Here is my code:

def get_response(seed, model, system_prompt, user_prompt):
    api_key = os.environ.get('GEMINI_API_KEY')
    google_search_tool = Tool(
        google_search=GoogleSearch()
    )

    response = None
    client = genai.Client(api_key=api_key)
    try:
        response = client.models.generate_content(
            model=model,
            contents=user_prompt,
            config=GenerateContentConfig(tools=[google_search_tool],
                                         system_instruction=system_prompt,
                                         temperature=0,
                                         thinking_config=ThinkingConfig(include_thoughts=True),
                                         response_modalities=["TEXT"]))
        return response
    except Exception as e:
        print(f"\n    Caught exception: {e}.")
        if response is not None:
            usage = response.usage_metadata
            print(f'    Total Tokens: {usage.total_token_count}')
        return None

This is for an application that is focused on research. I am using 2.5 Pro stable.

10 Likes
3

Hi @Bryan_Hughes , Welcome to the forum.

Would you mind sharing system_prompt and user_prompt so that I can reproduce it on my end?

1 Like
5

I have exactly the same issue. The problem occurs only sometimes and single prompt does not cause this issue but the same prompt after several runs could cause it so it does not seem to be prompt specific

1 Like
6

Hi @GUNAND_MAYANGLAMBAM,

Unfortunately our system_prompt is very sensitive, as is our user_prompt, which I appreciate isnt helping here. I can say this, while it is sensitive it is also not doing anything crazy. We have rubric of questions which we are using grounded search to return high quality results.

Also, this seems to be a new behavior in the past week or so, which I am also seeing more 500 responses. The prompts are very reasonable is number of tokens so perhaps there is more overloading going on?

Cheers,
Bryan

1 Like
7

Not sure if this helps, but there response object even has None for candidates, so not sure what could cause the model to return with this response. I would expect any other errors to be a status code (like 500). Also, the system prompt clearly directs that if is no answer, to specifically state this.

image
image1759×606 104 KB

1 Like
8

there response object even has None for candidates, so not sure what could cause the model to return with this response.

Yeah, that doesn’t look right. There’s also no prompt_feedback, so there’s nowhere to get a finish_reason. So +1 this looks like a bug in the API.

1 Like
9

While I would be happy to share the system prompt privately, the challenge is that it doesnt happen all the time with the same user prompt (which is where the question is). Though it seems to be happening on the same question (out of a set), so I can experiment more in extracting that question from the over all application runs and see if I can get it to reproduce in colab.

Also what is messed up is that I think I am being charged for each of these non-responses. Unfortunately I think that the billing API only refreshes every 24 hours.

10

Some debugging notes. When the request fails where the elements in the response are all None (like in the screenshot), my code waits about a minute and then retries. Nothing is cached so it is calling client.models.generate_content each time, and each time nothing.

I decided to see if is the prompt so I set up a colab exactly like my code and executed with the same system and user prompt and worked perfectly, so I dont think it is the prompt, or if it, there is something random between colab and my python application where repeating the request in my python also has the same failed response.

11

Are you running the same SDK version in both?

12

Yeah. The colab is copy and past from the python application. Also another issue that seems to be new, I am now encountering a few requests that never return. I use Pycharm as my dev environment and when I hit pause and then stop (i.e. SIGINT), it always stops on return self._sslobj.read(len, buffer) in this function is ssl.py:

    def read(self, len=1024, buffer=None):
        """Read up to LEN bytes and return them.
        Return zero-length string on EOF."""

        self._checkClosed()
        if self._sslobj is None:
            raise ValueError("Read on closed or unwrapped SSL socket.")
        try:
            if buffer is not None:
                return self._sslobj.read(len, buffer)
            else:
                return self._sslobj.read(len)
        except SSLError as x:
            if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
                if buffer is not None:
                    return 0
                else:
                    return b''
            else:
                raise

When I switch back to 2.0-flash, no issues. I am going to try running with 2.5 Flash Preview next.

1 Like
14

I’m also facing the same issue with Gemini 2.5 Pro and Flash. The issue does not occur constantly and is very random.

I’m working with large prompts (unfortunately can’t share) and sometime the API runs for a long time, seems like the model generates long output, but the response is empty at the end!

15

Is anyone else getting billed for these calls? It is hard to tell. The response does have the total tokens from the system prompt + user prompt which can add up.

1 Like
16

Hopefully this helps. Now I am getting this pretty regularly - 7 times in the last 11 requests (which are running in sequence):

Caught exception: 500 INTERNAL. {'error': {'code': 500, 'message': 'An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting', 'status': 'INTERNAL'}}.```
1 Like
17

I’m seeing the exact same issues here. Once it returns None as input, it will consistently return that every time I call it via the Python packages. Oddly enough, if i do the exact same prompt via the AIStudio UI it will still work fine. My specific prompts are <1k text and ~4k in document tokens if that helps.

18

Hopefully this is fixed soon. I really need to be using 2.5. While incredibly SLOOOOOOW, it does produce much better results.

19

More observations. The bug seems to be in the python SDK. When I run the same system prompt and user prompt with the same python code (literally copy and paste) in colab, it works 100% of the time. After making some adjustments to my system prompt in colab (again running it back to back with no issues), I copy it back to my python code and it now has failed 5 times in a row.

1 Like
21

Bro it’s because the ai hit a page with a AI attack that generates random text slowly eventually linking back to a other loop then it just wastes resources or training time until it times out. They have a name one min I’ll look it up eh idk but it’s a response to ignoring robots.txt

22

same issue but i am using simple java request

23

Hi, the issue has been escalated to the engineering team. Thanks.

7 Likes
25

I am having the same issue. What was the engineering team feedback?

3 Likes