Error 500 with video content when using the Swift SDK

Gemini API
1

I was directed here by the Swift SDK team on Github. When supplying a video regardless of size, I’m seeing error code 500 with the gemini-1.5-flash-latest model. Request(truncated) & response are below:

Request:
curl -H 'x-goog-api-key: AIza' -H 'x-goog-api-client: genai-swift/0.5.4' -H 'Content-Type: application/json' 'https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:streamGenerateContent?alt=sse' -d '{"contents":[{"parts":[{"text":"What’s going on in this video?"},{"inline_data":{"mime_type":"video\/mp4","data":"AAAAHGZ0eXBtcDQyAAAAAWlzb21tcDQxbXA0MgAAAAFtZGF0AAAAAAAGHigAACC2ZYiCAF9tD+CpeFgSic1MJlFb+kje1p6UbSkRhmtb....

Response:
[GoogleGenerativeAI] The server responded with an error: <NSHTTPURLResponse: 0x60000022ee80> { URL: https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent } { Status Code: 500, Headers { "Alt-Svc" = ( "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000" ); "Cache-Control" = ( private ); "Content-Encoding" = ( gzip ); "Content-Length" = ( 173 ); "Content-Type" = ( "application/json; charset=UTF-8" ); Date = ( "Fri, 16 Aug 2024 05:53:05 GMT" ); Server = ( "scaffolding on HTTPServer2" ); Vary = ( Origin, "X-Origin", Referer ); "server-timing" = ( "gfet4t7; dur=7421" ); "x-content-type-options" = ( nosniff ); "x-frame-options" = ( SAMEORIGIN ); "x-xss-protection" = ( 0 ); } } [GoogleGenerativeAI] Response payload: { "error": { "code": 500, "message": "An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting", "status": "INTERNAL" } }

I see this happening for every video content request with videos ranging in size from a few KB to a few MB. Image & document data work fine. Any help in identifying the root cause is appreciated.

2

Welcome to the forum.

This is a known problem, which Google has failed to document despite us developers having notified them in Video content supplied *inline* causes server fault since May 16th.

The video content can not be posted as inlineData, which is what the request you show is doing. That is guaranteed to generate a 500 error return code. To use video content, you have to use the File API. There is a technical reason, the File API performs preprocessing on the video, sampling it at fixed time intervals and saving the sampled images in a set. That preprocessing is missing when the video is transmitted inline.

Hope that helps!

1 Like
3

Thank you @OrangiaNebula for the reply. I’m relieved that it’s not something I was doing incorrectly with the SDK. What you say also makes sense as video upload to the API through the Firebase Extension & Vertex AI work fine, so I can only presume they’re using file API internally. I hope Google fixes this soon for the native SDK.

1 Like