I’ve implemented the url context functionality of Gemini 2.5 Flash in my app to parse the content from a url. I found that this feature only worked occasionally. Does it mean it essentially works as a web crawler that certain websites may have implemented guardrails to prevent crawling? What can I do to ensure the success rate of content parsing when using this feature?
2 Likes
thank you for reaching out in the forum, please note that the api cannot scrape the websites which need login or any this that is stopping the access. make sure that the URLs are not blocking usign any cached logins and make sure you can access the links in an incognito tab.
Build error handling into your application to gracefully handle cases where URL context parsing fails.
Consider providing users with options to enter text directly or upload documents as a fallback if URL parsing fails.
1 Like
yip, agreed, getting soo many errors with the url context tool. 400 errors today on vertex_ai, tried gemini and the exact same requests went through! Occasionally get 503 and 429 errors only when using url-context too.
I’m also facing similar issues. Inconsistent responses sometimes it goes through other times it fails to access the URL. I also notices higher failure rates when using the API vs AI studio. I hope this can be improved