What exactly is the minimal authentication setup for interacting with the corpora related REST API calls? I am getting lost in documentation maze if I try to understand anything beyond the key based access at this point.
Here is how I access the API normally, say to generate content.
curl -X POST https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent?key=
-H ‘Content-Type: application/json’
-d ‘{ “contents”: [{ “parts”: [{ “text”: “I am testing. Can you respond?” }] }] }’
Obviously, there is no need to specify a project name anywhere or do anything on the client side like gcloud tokens etc. Looks like the API key is sufficient.
The same thing doesn’t work - as it seems like CORPORA needs more permissions (401 error saying UNAUTHENTICATED)
curl -X GET https://generativelanguage.googleapis.com/v1beta/corpora?key=
I tried to use gcloud auth token like I have done while trying the vertex AI before trying this simpler API mechanism; as shown below:
curl -X GET https://generativelanguage.googleapis.com/v1beta/corpora?key=
-H “Authorization: Bearer $(gcloud auth print-access-token)”
-H “x-goog-user-project: ”
This fails with 403 error with PERMISSION_DENIED & ACCESS_TOKEN_SCOPE_INSUFFICIENT.
Now, the account I set up during gcloud cli is my email that is the owner of the project and I am not sure at this point where to do what. I only end up messing around with different screens all over google console through API screens and OAuth setup and what not.
I have to declare my ignorance and impatience with all things google now, as all my brain power now is needed for dealing with 10+ PDF books ranging from 50-500 pages of text that just don’t convert to text chunks properly and I have so much work to chunk that text into a more meaningful set of chunks; and creating a web page that can deal with these API calls.
And the entire google authentication mechanism seems so much more complicated once I get into the non API key stuff. It is just pages of text jumping from one location to the next and after a few jumps I feel totally tired and want to look for alternatives like pinecone (currently evaluating), SOLR search (planning to use) etc.
With pinecone, this is just a simple API KEY based authentication, if I use pinecone to store these texts and do vector search using embedding vector data I can get from
https://generativelanguage.googleapis.com/v1beta/{model=models/*}:embedContent
With SOLR, no authentication stuff because I already have one doing plain text search; just need to add the vector search to it.
Am I missing something so simple here? Isn’t being the owner of the project sufficient with all the access scopes needed to deal with CORPORA? Do I have to go all the way into configuring an OAUTH application and so on? I originally went through so many different ways to authenticate with Vertex API, gcloud, ADC, service account and so on and now I am so confused.
What is the least necessary step to enable access for a) user account b) a service account allowing it to use the corpora sections of the REST API to be used with my semantic search assisted prompting (RAG)?
Thanks a lot.
Shankar.