I’m trying to use the video extension feature with the Veo 3.1 model using the Google Generative Language API. I am sending requests to the endpoint:
https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-generate-preview:predictLongRunning
My goal is to provide an initial video segment for the model to extend. However, every time I include the video object in my instances payload, the API rejects the request. It always complains about the video object, suggesting my format is wrong.
"`bytesBase64Encoded` isn't supported by this model. Please remove it or refer to the Gemini API documentation for supported usage."
Here is the basic structure of the JSON payload I’m sending. I’ve left the bytesBase64Encoded string empty here for brevity.
{
"instances": [
{
"prompt": "People from a card are becoming alive",
"video": {
"bytesBase64Encoded": "..."
}
}
],
"parameters": {
"sampleCount": 1,
"resolution": "720p",
"aspectRatio": "9:16",
"durationSeconds": 4
}
}
I have tried several variations for the video object based on how other Google APIs work, but none are successful:
"video": { "bytesBase64Encoded": "..." }
"video": { "gcsUri": "gs://my-bucket/my-video.mp4" }
"video": { "uri": "gs://my-bucket/my-video.mp4" }
My main issue is that I cannot find any official Google Cloud documentation for this specific endpoint or the veo-3.1 model. All available documentation seems to be for different models or the Vertex AI endpoints, which appear to use a different structure.
Has anyone successfully used the video extension feature with this veo-3.1-generate-preview endpoint? What is the correct JSON structure (or Maybe aother endpoint for this) for passing a video (either as bytes or a GCS URI) in the instances payload?Could you share any documentation or a working example payload for this specific Veo 3.1 API? Any help would be greatly appreciated.
1 Like
This is unrelated to your issue, but I wanted to ask, how did you get access to the Veo 3.1 video extension feature? I’m unable to find any documentation on how to get access to it.
I’m using Vertex AI with the Python SDK, and I keep getting this error whenever I try to use the video extension feature:
‘error’: {‘code’: 400, ‘message’: ‘Async process failed with the following error: Video extension is not allowlisted for project xxxxxxxxxx.’, ‘status’: ‘FAILED_PRECONDITION’}}
I was getting the same issue when I was using this endpoint
POST ``https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/veo-3.1-generate-preview:predictLongRunning
Then I tried with this one https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-generate-preview:predictLongRunning And it is not giving that error anymore, but I cannot upload the video, every time it complains about the structure
I found an issue, don’t use a veo-3.1-generate-preview. As I understood for a video extension, you can use only veo-2.0-generate-preview. Basically you can generate a first video with veo-3.1-generate-preview but extend it you can only using 2.0
You know I am using Vertex Ai through n8n, and I am able to create a video with Veo 3.0 and extend with Veo 2. But it is access the “video”: { “gcsUri”: “gs://my-bucket/my-video.mp4” }
“video”: { “uri”: “gs://my-bucket/my-video.mp4” } like you mentioned.
I was reading that you can access the do this same operation with Gemini Api. So it appears that I am able to create a video with Veo 3.1; just tried to copy of the sample payload and changed the prompt and executed video. But now I am trying to load an image as a reference, and I am getting an error on my payload:
This one works:
{
“instances”: [{
“prompt”: “A close up of two people staring at a cryptic drawing on a wall, torchlight flickering. A man murmurs, “This must be it. That’''s the secret code.” The woman looks at him and whispering excitedly, “What did you find?””
}
]
}’ | jq -r .name)
But when I try to load the following payload it fails, gives me an internal error with the api:
{
“instances”: [
{
“prompt”: “A close up of two people staring at a cryptic drawing on a wall, torchlight flickering. A man murmurs, “This must be it. That’''s the secret code.” The woman looks at him and whispering excitedly, “What did you find?””,
“image”: {
“bytesBase64Encoded”: “….”,
“mimeType”: “image/jpeg”
}
}
],
“parameters”: {
“aspectRatio”: “9:16”,
“sampleCount”: 1,
“durationSeconds”: 8,
“storageUri”: “gs://angel_1/veo-output/”
}
}
Any recommendations on how to properly format the payload. The node prior had to convert the image into a Base64Encoded file so I get a long list of characters so that I can properly load the image file.
Hi Ivan,
I previously posted using Google Vertex Endpoints.
Below is the process using Google Gemini API to extend the video. This is assuming you were successful creating the first video.
Extending Video Node (I did it in n8n).
URL: https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-generate-preview:predictLongRunning
Please note the uri should come from your previously generated veo 3.1 video, for example in my case: https://generativelanguage.googleapis.com/v1beta/files/g3d8wka7wqcg:download?alt=media
Payload:
{
"instances": [
{
"prompt": "you prompt for your extended video goes here",
"video": {
"uri": "{{ $json.response.generateVideoResponse.generatedSamples[0].video.uri }}"
}
}
],
"parameters": {
"aspectRatio": "9:16",
"seed": 47219
}
}
You will be required to create a poll node,
Endpoint: https://generativelanguage.googleapis.com/v1beta/{{ $json.name }}
in this example, https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-generate-preview/operations/2dg5y87iuvqz
You must create poll node, as it will provide your new uri for the extended video. for example in my case - https://generativelanguage.googleapis.com/v1beta/files/g3d8wka7wqcg:download?alt=media
You must then create a download node, with the endpoint:
{{$json.response.generateVideoResponse.generatedSamples[0].video.uri}}
in my case this is the actual url - https://generativelanguage.googleapis.com/v1beta/files/g1dlxy0qcs8r:download?alt=media
in your node in n8n for example, you will select options, response format and enter file, and under put output in field and enter data, so that you can download the video file in a binary form and you will be able to view your extended file.
good luck, and I hope it helps.