Hey all, I’m wondering if it’s possible to stream directly into the Gemini file manager Node package?
I need to have files streamed instead of saving to the filesystem because I am deploying on Vercel, and you can only store up to 512 MB because Vercel runs on serverless architecture. This won’t work for me because I am uploading videos, sometimes up to 20 minutes long.
However, it seems that looking at the filesystem is hardcoded, and the package would have to be updated with a new feature to allow streaming
I’ve contacted one of my CE’s at GCP but maybe this feature exists and is just undocumented, which isn’t too much of a reach considering i’ve already had to discover undocumented Gemini features/limitations.
If an engineer from Google is seeing this, please reply or email me! It’s kinda urgent.
You can’t do it with the Node library, but you can certainly access the Gemini File API directly.
The API supports Google’s standard file upload semantics, which include multipart resumable uploads. So you can upload it one chunk at a time. (Which isn’t quite streaming, but sounds like it will work well for you.)
If you’re already familiar with GCP, it may be easier for you to use the Vertex AI Gemini AI. Instead of uploading the file to the File API, you’d upload it to a GCS bucket in the same project and provide the gs: URI, which may be easier for you to handle.
I might have to migrate to the Vertex AI API instead of the Gemini API so I can use GCS. That will probably take half a week - a full week to do. I’ll have to do it after my app’s launch, since https://makethedocs.com launches in 4 days. I might have to just host it on GCE until I can migrate.
(A) It shouldn’t be that big a change. If you’re using the node libraries, they’re mostly compatible. The biggest difference is in how you do authentication. You can certainly use the Vertex AI Gemini API and GCS from Vercel.
(B) If you really want to use the File API, you can look at the File API media upload doc for the URL and general method, and at Perform resumable uploads | Cloud Storage | Google Cloud for the general structure of how to do resumable uploads. (You should use the URL from the first doc, plus your API key, when applying the second doc’s methodologies.)
(However, I hear you at the frustration of the library’s limitation. It is why I’m working on Langchain.js support for the feature without Google’s libraries. Good luck!)
It won’t be that big of a change, no. But it is big enough for me to hold out on doing this right before launch. I wouldn’t want to put out an untested mess that might break the core feature of my app if it meant I could get the feature out in time.
But looking at the Vertex AI docs, if I can kick ass tomorrow and get it done, I can test it enough and make sure that it’s production ready just in time for launch.