@Synduex The media upload even though not fully documented at Method: media.upload | Google AI for Developers | Google for Developers has two parts. The part you’re trying to achieve is part 1 found at that API reference. It won’t return anything in body, and you should not send file stream for the first part. What happens is that first part only accepts the meta data which is optional AND THESE HEADERS:
return [
'X-Goog-Upload-Protocol' => 'resumable',
'X-Goog-Upload-Command' => 'start',
'X-Goog-Upload-Header-Content-Length' => [THE_FILE_SIZE]
'X-Goog-Upload-Header-Content-Type' => [THE_MIME_TYPE],
];
Forgive me for the php but am sure you are able to send these headers in Scala as well.
WHAT YOU DO FOR THAT REQUEST:
- Calculate the total byte size of the file.
- Obtain the mime type. I doubt
application/octet-stream will be accepted.
Google will return empty body for that initial request, but the header will contain a url you should upload the file to. That will take you to second part.
The header you need will be in X-Goog-Upload-URL.
Part 2 is some how tricky but I’ll try and explain from what I did.
We already have the url to upload to, this is an equivalent in my php code:
$handle = fopen($filePath, 'rb');
$chunkSize = self::CHUNK_SIZE;
$offset = 0;
while (!feof($handle)) {
$chunkData = fread($handle, $chunkSize);
$end = $offset + strlen($chunkData);
$command = ($end < $fileSize) ? 'upload' : 'upload, finalize';
$chunkRequest = new UploadMediaChunkRequest($uploadUrl, $chunkData, $offset, $command);
$response = $this->connector->send($chunkRequest);
if ($response->failed()) {
fclose($handle);
throw new Exception('Chunk upload failed: ' . $response->body());
}
$offset = $end;
}
fclose($handle);
This is what happens:
- Open the file to be uploaded in binary mode.
- Define chunk size (8 * 1024 * 1024) (8mbs) to be uploaded
- Initialize an offset variable to keep track of the position in the file.
- Start a loop that continues until you’ve read the entire file:
a. Read a chunk of data from the file.
b. Calculate the end position of this chunk.
c. Determine the upload command:
- Use ‘upload’ if there’s more data to come.
- Use ‘upload, finalize’ for the last chunk.
Send a POST request to the upload URL you got from step 1 with the following:
- The chunk data in the request body.
- Pass these headers with the request:
return [
'Content-Length' => strlen($this->chunkData),
'X-Goog-Upload-Offset' => [THE_OFFSET],
'X-Goog-Upload-Command' => [THE_COMMAND],
];
Content-Length in my request is calculating the length of the chunk which is the body of the request.
The last loop that sends command upload, finalize will receive the File instance.
Not sure that’s going to help you enough. I got the idea from this cookbook (shell+curl): cookbook/quickstarts/file-api/sample.sh at main · google-gemini/cookbook · GitHub
Here’s an AI generated pseudocode representation of my code that might help you translate to Scala:
uploadUrl = // URL obtained from initial request
fileSize = // Total file size
chunkSize = 8 * 1024 * 1024 // 8MB chunks
offset = 0
openFile(filePath)
while (not endOfFile) {
chunkData = readChunk(chunkSize)
endPosition = offset + chunkData.length
command = if (endPosition < fileSize) "upload" else "upload, finalize"
size = readSizeofChunk(chunkData)
response = sendRequest(
method: POST,
url: uploadUrl,
headers: {
"Content-Length": size,
"X-Goog-Upload-Command": command,
"X-Goog-Upload-Offset": offset
},
body: chunkData
)
if (response.isError) {
handleError(response)
}
offset = endPosition
}
closeFile()
return finalResponse // Contains uploaded file metadata (File instance)
Note that I didn’t use PUT, both parts used POST.
Recap: Part 1 uses the endpoint in Method: media.upload | Google AI for Developers | Google for Developers but Part 2 makes request to url returned to you in Part 1.
Haven’t tried my luck with defining a larger chunk size to speed things up which could also break. Be sure to check and terminate upload early if an error occurs.
Also note I didn’t stream both requests.
