Help with Error "The input token count (35868) exceeds the maximum number of tokens allowed (32768)."}"

We are getting below error despite we are passing prompt with 1030 token size. I’m attaching the screenshot of the token also.

We are using Gemini 1.5 pro. With grounded search with chat functionality
Error:

Traceback (most recent call last):
File “/home/test/python3/lib/python3.10/site-packages/google/api_core/grpc_helpers.py”, line 75, in error_remapped_callable
return callable_(*args, **kwargs)
File “/home/test/python3/lib/python3.10/site-packages/grpc/_channel.py”, line 1161, in call
return _end_unary_response_blocking(state, call, False, None)
File “/home/test/python3/lib/python3.10/site-packages/grpc/_channel.py”, line 1004, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = “The input token count (35868) exceeds the maximum number of tokens allowed (32768).”
debug_error_string = “UNKNOWN:Error received from peer ipv4:74.125.134.95:443 {created_time:“2025-03-07T01:51:12.056793008-05:00”, grpc_status:3, grpc_message:“The input token count (35868) exceeds the maximum number of tokens allowed (32768).”}”

1 Like

Getting similar error. I am running grounded search with chat functionality on Gemini 1.5 pro.

2 Likes

Yeah, grounded search with chat functionality on Gemini 1.5 pro.

Looks like the error is coming from the grounded search with chat functionality. If I remove either option, the error disappears. Google team, something has started to break when we combine these two recently.

2 Likes

Hi @Vinay_Kumar

Welcome to Forum,

It would be great if you can you send me a sample prompt or how you are using it so that I can reproduce it.

Thanks!

1 Like

Yes, I also facing issue on same

The error is coming from the grounded search with chat functionality.

Prompt: Below is a list of questions for an TEST. Using this data, generate a list of 5-8 bullet points of primary business objectives the solution sought looks to address. Output in JSON with key primary_business_objectives. [{“question”: “Is the system easy to use for both frequent and occasional users, by being easy to learn, intuitive to new or occasional users? Please explain”, “category”: “Functional Requirements”, “sub_category”: “General Requirements”}, {“question”: “What are your product differentiators? Please provide specific sample/screenshot/documentation”, “category”: “Functional Requirements”, “sub_category”: “General Requirements”}, {“question”: “Can the platform be able to create WCAG 2.0 AA (and section 508) compliant in terms of accessibility?”, “category”: “Functional Requirements”, “sub_category”: “General Requirements”}, {“question”: “Is your company GDPR compliant?”, “category”: “Functional Requirements”, “sub_category”: “General Requirements”}, {“question”: “Can the platform provide a custom domain for customer?”, “category”: “Functional Requirements”, “sub_category”: “General Requirements”}, {“question”: “Can the platform send reminders?”, “category”: “Functional Requirements”, “sub_category”: “General Requirements”}, {“question”: “Can the platform upload background variables about participants?”, “category”: “Functional Requirements”, “sub_category”: “Sample/Recruiting”}, {“question”: “Can the platform upload email lists?”, “category”: “Functional Requirements”, “sub_category”: “Sample/Recruiting”}, {“question”: “Does the platform have the option to send reminders via SMS, either to US or International destinations?”, “category”: “Functional Requirements”, “sub_category”: “Sample/Recruiting”}, {“question”: “Does the platform have an intelligent handling of invitation bounce backs so that undelivered messages are marked as uncontactable but that autoresponder such as out of the office are separated and handled appropriately?”, “category”: “Functional Requirements”, “sub_category”: “Sample/Recruiting”}, {“question”: “Does the platform have integration with panel management module - either your own or a recommended company?”, “category”: “Functional Requirements”, “sub_category”: “Questionnaire Design”}, {“question”: “Does the platform have the capability to copy/past questions”, “category”: “Functional Requirements”, “sub_category”: “Questionnaire Design”}, {“question”: “Do you support for incentive and reward programs”, “category”: “Functional Requirements”, “sub_category”: “Panel Management”}, {“question”: “Do you have the ability to organize the panel database into multiple virtual panels, and for panelists to belong to more than one panel?”, “category”: “Functional Requirements”, “sub_category”: “Panel Management”}, {“question”: “Ability to import or upload new panelist data?”, “category”: “Functional Requirements”, “sub_category”: “Panel Management”}, {“question”: “Ability to export panel database?”, “category”: “Functional Requirements”, “sub_category”: “Panel Management”}, {“question”: “Does it allow survey creation and publishing through API’s?”, “category”: “Functional Requirements”, “sub_category”: “API”}, {“question”: “Does it allow for customization of survey contents?”, “category”: “Functional Requirements”, “sub_category”: “API”}, {“question”: “Does the platform allow data export in real time?”, “category”: “Services Requirements”, “sub_category”: “Integrations and Outputs”}, {“question”: “Does the system expose metadata for any data that are made accessible for exchange or export, or that may be imported or exported?”, “category”: “Services Requirements”, “sub_category”: “Integrations and Outputs”}, {“question”: “Can the platform export data to CSV, Excel, and SPSS?”, “category”: “Services Requirements”, “sub_category”: “Integrations and Outputs”}, {“question”: “Does it have the ability to export a survey project and import survey questions and skip the platform logic?”, “category”: “Services Requirements”, “sub_category”: “Migration support of survey projects”}, {“question”: “Does it have the ability to upload exported raw data files from the incumbent and upload them safely to a new survey platform?”, “category”: “Services Requirements”, “sub_category”: “Migration support of survey projects”}, {“question”: “Please provide fixed pricing structure for the ongoing managed services (ie dedicated TAM and detailed scope)”, “category”: “Services Requirements”, “sub_category”: “Managed Services”}, {“question”: “Does the System offer high levels of resilience in the event of a server or database failure? Does it include capabilities for server clustering, server and data storage redundancy and database replication? Please explain.”, “category”: “Technical Requirements”, “sub_category”: “Security”}, {“question”: “Does your product fully align to MVSP controls?”, “category”: “Technical Requirements”, “sub_category”: “Security”}]

This issue is coming from yesterday only.

I’m encountering the same issue with all Gemini Pro models when using grounded search with Vertex. However, I’ve found a workaround by changing the Data store documents from multiple separate documents to a single merged document.

That said, I still haven’t figured out the root cause of the issue :fearful:!"

My enterprise project uses Vertex AI Grounded Search, and I’ve been encountering the same error for a few days, causing a delay in my release timeline. I’ve tried reporting the issue to Google through every channel, but I still haven’t received any action. :fearful:

@Siva_Sravana_Kumar_N Seems like the error is when grounded search is enabled with large prompt (token size of 25k) seems to break. When grounding is removed it works fine. Can you please prioritize this fix? Its causing release issues for many of your customers.

@Siva_Sravana_Kumar_N This issue seems to come up only with API. It seems to work just fine in console.