Vertex AI API can't generate with video

Gemini API
, ,
1

I need help with my Vertex AI API. I can’t generate anything with video. I’ve been following the documentation but anytime i try to generate with video I get this cryptic 500 error from Gemini

Error generating content: GoogleGenerativeAIError: [VertexAI.GoogleGenerativeAIError]: got status: 500 Internal Server Error. {“error”:{“code”:500,“message”:“Internal error encountered.”,“status”:“INTERNAL”}}

I would use the AI studio API but I can’t because of limitation from vercel. My app is supposed to launch in 2 days and at this rate I dont think I can make it.

this is the code that generates:

 const generativeModel = vertexAI.getGenerativeModel({
                        model: 'gemini-1.5-flash-001',
                        systemInstruction: {
                          parts: [
                            {text: "You are a documentation writing AI."},
                               LONG system prompt...
                          ],
                        },
                      });

                      console.log('instantiated generative model')
                    
                      const filePart = {
                        fileData: {
                            fileUri: fileUri, mimeType: 'video/mp4'
                        }
                    };

                    

                      const textPart = {
                        text: `
                        brand: ${brand}, goal: ${goal},`,
                      };
                    
                      const req = {
                        contents: [{role: 'user', parts: [filePart, textPart]}],
                      };
                    
                      console.log('generating content with Vertex AI')
                    
                      const resp = await generativeModel.generateContent(req);
                      console.log('sent request to generative model')
                      const contentResponse = await resp.response;
                        console.log('awaiting response from generative model')
                      console.log(JSON.stringify(contentResponse));

It generates text-to-text fine, but completely falls apart when I add video.

2

Nevermind. I found out that one of my items that was supposed to be passed into the generation function, specifically the GCS file URI, was undefined for some reason.

For people in the future trying to solve this problem, make sure to console.log EVERYTHING that is going into your prompt, including text, media, and anything else that isn’t explicitly written,