OpenAI Compatibilioty with JNodeJS - Error

Hugo_Lebegue · November 22, 2024, 6:07am

I am experimenting with the use of OpenAI library to communicate with Gemini and follow the documentation at

When using the openai libraries on NodeJS and executing a simple await openai.models.list() I receive

generativelanguage.googleapis.com

/v1beta/openai/
404 status code (no body) NotFoundError: 404 status code (no body)

/v1beta/
401 Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential.

I supply a valid gemini API key I generated for personal use.

I wonder if anyone else has the same issue or has recommendations to resolve this.

Thank you

jkirstaetter · November 22, 2024, 6:43am

Hi there,
IIRC, the list models functionality is not available for the moment. For now, chat completions and embeddings are working.

Hugo_Lebegue · November 22, 2024, 7:10am

Thank you for your response. I assume that /v1beta/openai/ is the proper end point then.

Regarding the streaming, is this endpoint supported?

openai.beta.chat.completions.stream(inboundS);

If not, do you have a full sample of how to use Streaming with tools (and respond to tool use)

On a side note, I am trying this simple code:

 const completion = await openai.chat.completions.create({
            model: engine,
            user: user._id,
            messages: [
                { role: "system", content: [{ type: "text", text: systemPrompt}] },
                { role: "user", content: [{ type: "text", text:userPrompt}] },
            ],
            temperature: 0.3,
            response_format: { "type": "json_object" }
        })

And getting
BadRequestError: 400 status code (no body)

Any help is appreciated

Thank you for all your help

jkirstaetter · November 25, 2024, 11:18am

Hi @Hugo_Lebegue

The link / resource you provided has streaming documented

See here for working examples, constraints and roadmap of using Gemini with the OpenAI API

Also from there: “We will initially support the Chat Completions API and Embeddings API, with plans for additional compatibility in the weeks and months to come.”

Your code sample seems to miss the actual client to specify the Gemini API Key and the base url.

Cheers, JoKi

Hugo_Lebegue · November 26, 2024, 4:04am

Thank you for your feedback

I have seen and read the 2 documentations links you mentioned.,I was asking the community if someone had a complete example of using tools+stream combined, specifically detailing how to respond to tool use while processing stream chunks. I realize that this is probably more a question for OpenAI.

I took note that the beta section of the APIs are not supported, (for those that are still using Beta), and that the end- point for the client is created by overwriting the endpoint URL and the API key in the config section.

const openai = new OpenAI(config)

Thank you for your help.

jkirstaetter · November 26, 2024, 4:22am

Hello Hugo,
You’re most welcome. Glad you got it working.
Kindly mark a response as “solution” if it solved your issue. Thanks.

Hugo_Lebegue · November 26, 2024, 12:18pm

Hi Jochen,

Unfortunately, I can not get this end point to work .
While the code works flawlessly with open AI, using the chat completion, I get a 404 No Body .

If you know a way I can turn debugging on for the open ai library so I can send additional logs if this can be helpful.

I am using the exact same code as the one you sent earlier.

 const result = await openai.chat.completions.create({
        model: engine,
        messages: [
            {"role": "system", "content": `${prompt}`},
            {"role": "user", "content": `${inputText}`}
        ],
        temperature: 0.3,
    })

I am really puzzled by this

pria:middleware:openai streamConversation For user admin@praxislxp.com model gemini-1.5-flash +653ms
  pria:middleware:openai Connecting with config GEMINI {
   "apiKey": "GOCxxxxxxxxxxxxxxi1PthP6iuytt7T",
   "baseURL": "https://generativelanguage.googleapis.com/v1beta/openai/"
} +0ms
PRIA ERROR BadRequestError: 400 status code (no body)
    at Function.generate (C:\_svn\_praxis\code\pria_ui\pria-ui-1\node_modules\openai\error.js:45:20)
    at OpenAI.makeStatusError (C:\_svn\_praxis\code\pria_ui\pria-ui-1\node_modules\openai\core.js:293:33)
    at OpenAI.makeRequest (C:\_svn\_praxis\code\pria_ui\pria-ui-1\node_modules\openai\core.js:337:30)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async doStreamConversation (C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\middlewares\openai.js:440:28)
    at async Object.streamConversation (C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\middlewares\openai.js:385:5)
    at async rag.chat (C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\middlewares\rag.js:484:13)
    at async C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\ai\personal\qanda.js:90:3 {
  status: 400,
  headers: {
    'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000',
    'content-encoding': 'gzip',
    'content-type': 'application/json; charset=UTF-8',
    date: 'Tue, 26 Nov 2024 12:15:57 GMT',
    server: 'scaffolding on HTTPServer2',
    'server-timing': 'gfet4t7; dur=1951',
    'transfer-encoding': 'chunked',
    vary: 'Origin, X-Origin, Referer',
    'x-content-type-options': 'nosniff',
    'x-frame-options': 'SAMEORIGIN',
    'x-xss-protection': '0'
  },
  request_id: undefined,
  error: undefined,
  code: undefined,
  param: undefined,
  type: undefined
}

Let me know if you can see anything obvious. Thank you

jkirstaetter · November 26, 2024, 1:25pm

Hi @Hugo_Lebegue

What’s the actual value of the model name, you are using a variable “engine” in your code?
Do you use an existing model, like they are offered in Google AI Studio? Eg. something like “gemini-1.5-flash”

Hugo_Lebegue · November 26, 2024, 1:33pm

YEs, this is what I am using: gemini-1.5-flash

jkirstaetter · November 26, 2024, 1:45pm

Did you already try a plain HTTP request?
Replace the {{apiKey}} variable with your key value. I’m using VS Code with REST Client extension to query the OpenAI API. This gives me an HTTP 200 response.

POST https://generativelanguage.googleapis.com/v1beta/openai/chat/completions
Authorization: Bearer {{apiKey}}
Content-Type: application/json

{
  "model": "gemini-1.5-flash",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant."},
    { "role": "user", "content": "Explain to me how AI works"}
  ]
}

(openai.http)

The error response in your code says that it’s an HTTP 400 Bad Request. Meaning, there is something wrong with the data in your request sent to the OpenAI API. Are you able to debug/trace the JSON before it goes over the wire?

Eventually, you are missing escaping of characters in your system message and user message? Your prompt and inputText are wrapped in single quotes only. Shouldn’t that be double-quotes and then have any double-quote in your value escaped?

Hugo_Lebegue · November 26, 2024, 2:46pm

Sae problem using a simple API tester plugin from Chrome . Nothing spectacular, as if the End point is simply not listening

jkirstaetter · November 26, 2024, 2:49pm

No, in your API tester you are using the wrong URL which gives you an HTTP 404 Not Found.

See my REST call regarding the actual API endpoint!

https://generativelanguage.googleapis.com/v1beta/openai/chat/completions

jkirstaetter · November 26, 2024, 2:50pm

Hi @Hugo_Lebegue
You forgot to mask your API KEY in the screenshot! Please revoke and create a new one.

Hugo_Lebegue · November 26, 2024, 2:51pm

You are correct, it worked when I corrected.

This also validates my API Key (modified for posting the image here) SO Open AI is not composing the valid end poiunt URL when I specify the root as

    config = {
        apiKey: user?.institution?.gemini_api_key || process.env.GEMINI_API_KEY,
        baseURL: `https://generativelanguage.googleapis.com/v1beta/openai/`
    }

How can I trace what URL is openAI trying to get to ?

jkirstaetter · November 26, 2024, 2:54pm

That URL is correct!
Because it says “baseUrl” and the actual method you are calling, like eg. openai.chat.completions.create, adds the remaining path to that baseUrl.

But that’s not the problem. The URL seems to be correct between your code and the API tester. The problem is the data / payload your library sends as the body. This causes the HTTP 400 Bad Request because it contains information not accepted by the Gemini endpoint.

Hugo_Lebegue · November 26, 2024, 3:11pm

I see. I use NodeJS 16, with “openai”: “^4.73.1”.

I modified my locole version of core.js so I can debug code execution. This is what is now logged in the console:

It is going to the right URL
https://generativelanguage.googleapis.com/v1beta/openai/chat/completions

The path ‘/chat/completions’ is correct.

OpenAI:DEBUG:request https://generativelanguage.googleapis.com/v1beta/openai/chat/completions {
  method: 'post',
  path: '/chat/completions',
  body: {
    model: 'gemini-1.5-flash',
    messages: [
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object],
      [Object], [Object], [Object], [Object]
    ],
    tools: [
      [Object], [Object], [Object],
      [Object], [Object], [Object],
      [Object], [Object], [Object],
      [Object], [Object], [Object],
      [Object], [Object], [Object],
      [Object], [Object], [Object],
      [Object], [Object], [Object]
    ],
    n: 1,
    user: '65cfebc32f5e1b37d4e52327',
    frequency_penalty: 0,
    presence_penalty: 0,
    temperature: 0.2,
    stream: true,
    stream_options: { include_usage: true },
    tool_choice: 'auto'
  },
  stream: true
} {
  'content-length': '64504',
  accept: 'application/json',
  'content-type': 'application/json',
  'user-agent': 'OpenAI/JS 4.73.1',
  'x-stainless-lang': 'js',
  'x-stainless-package-version': '4.73.1',
  'x-stainless-os': 'Windows',
  'x-stainless-arch': 'x64',
  'x-stainless-runtime': 'node',
  'x-stainless-runtime-version': 'v16.20.2',
  authorization: 'Bearer xxxxxx',
  'x-stainless-retry-count': '0'
}
OpenAI:DEBUG:response (error; (error; no more retries left)) 400 https://generativelanguage.googleapis.com/v1beta/openai/chat/completions {
  'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000',
  'content-encoding': 'gzip',
  'content-type': 'application/json; charset=UTF-8',
  date: 'Tue, 26 Nov 2024 15:06:24 GMT',
  server: 'scaffolding on HTTPServer2',
  'server-timing': 'gfet4t7; dur=374',
  'transfer-encoding': 'chunked',
  vary: 'Origin, X-Origin, Referer',
  'x-content-type-options': 'nosniff',
  'x-frame-options': 'SAMEORIGIN',
  'x-xss-protection': '0'
} undefined
PRIA LOGIC ERROR BadRequestError: 400 status code (no body)
    at Function.generate (C:\_svn\_praxis\code\pria_ui\pria-ui-1\node_modules\openai\error.js:45:20)
    at OpenAI.makeStatusError (C:\_svn\_praxis\code\pria_ui\pria-ui-1\node_modules\openai\core.js:293:33)
    at OpenAI.makeRequest (C:\_svn\_praxis\code\pria_ui\pria-ui-1\node_modules\openai\core.js:337:30)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async doStreamConversation (C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\middlewares\openai.js:441:28)
    at async Object.streamConversation (C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\middlewares\openai.js:386:5)
    at async rag.chat (C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\middlewares\rag.js:484:13)
    at async C:\_svn\_praxis\code\pria_ui\pria-ui-1\routes\ai\personal\qanda.js:90:3 {
  status: 400,
  headers: {
    'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000',
    'content-encoding': 'gzip',
    'content-type': 'application/json; charset=UTF-8',
    date: 'Tue, 26 Nov 2024 15:06:24 GMT',
    server: 'scaffolding on HTTPServer2',
    'server-timing': 'gfet4t7; dur=374',
    'transfer-encoding': 'chunked',
    vary: 'Origin, X-Origin, Referer',
    'x-content-type-options': 'nosniff',
    'x-frame-options': 'SAMEORIGIN',
    'x-xss-protection': '0'
  },
  request_id: undefined,
  error: undefined,
  code: undefined,
  param: undefined,
  type: undefined
}

Updates:

I was able to make the library work until I removed from following properties the json

frequency_penalty,
presence_penalty,
tool_choice, and
user

Additionally, the library won’t work either if you declare more than 1 too in the tools section

  "tools": [
         {
            "type": "function",
            "function": {
               "name": "get_browser",
               "description": "You have the 'get_browser' tool. Use 'get_browser' tool to search the web for up-to-date information ",
               "parameters": {
                  "type": "object",
                  "properties": {
                     "argument_1": {
                        "type": "string",
                        "description": "The specific information, event, or topic to search (e.g. Latest financial news)."
                     }
                  },
                  "required": [
                     "argument_1"
                  ]
               }
            }
         },
		  {
            "type": "function",
            "function": {
               "name": "get_foo",
               "description": "You have the 'get_foo' tool. Use 'get_foo' tool to get the users location",
               "parameters": {
                  "type": "object",
                  "properties": {
                     "argument_2": {
                        "type": "string",
                        "description": "The specific location."
                     }
                  },
                  "required": [
                     "argument_2"
                  ]
               }
            }
         }
      ],

Finally I suggest to enhance the API libs to communicate the reason for error, instead of an opaque 400.

Where can we log these bugs so they get addressed in future versions of openAI lib or Gemini ?

Thank you for all your help

Hugo_Lebegue · November 26, 2024, 4:48pm

I found the problem.

The issue is that the API doesn’t communicate the cause/source of the error.

in the JSON sent through the APIs, It doesn’t support the following properties:

user
frequencey_penalty
prsence_penalty
tool_choice

Additionally, Gemini sends a 400 when I specify more than 1 tool

"tools": [
        
         {
            "type": "function",
            "function": {
               "name": "get_browser",
               "description": "You have the 'get_browser' tool. Use 'get_browser' tool to search the web for up-to-date information ",
               "parameters": {
                  "type": "object",
                  "properties": {
                     "argument_1": {
                        "type": "string",
                        "description": "The specific information, event, or topic to search (e.g. Latest financial news)."
                     }
                  },
                  "required": [
                     "argument_1"
                  ]
               }
            }
         },
	{
            "type": "function",
            "function": {
               "name": "get_foo",
               "description": "You have the 'get_foo' tool. Use 'get_foo' tool to get the users location",
               "parameters": {
                  "type": "object",
                  "properties": {
                     "argument_1": {
                        "type": "string",
                        "description": "The specific location."
                     }
                  },
                  "required": [
                     "argument_1"
                  ]
               }
            }
         }
      ],

jkirstaetter · November 26, 2024, 5:10pm

Hi @Hugo_Lebegue
Yes, that’s kind of described in the blog article when Google announced their beginning of OpenAI API compatibility. There are lots of things in active development right now. There are already other threads about the problem re. multiple tools not working. They just started…

Dear @GUNAND_MAYANGLAMBAM please forward this to the eng. team. Thanks.

As for those properties named, they are already in the types, like eg.

        /// <summary>
        /// Optional. Controls whether the model should use a tool or not, and which tool to use.
        /// Can be either:
        /// - The string \"none\", to disable tools.
        /// - The string \"auto\", to let the model decide.
        /// - The string \"required\", to force the model to use a tool.
        /// - A function name descriptor object, specifying the tool to use. The last option follows the following schema: { \"type\": \"function\", \"function\": {\"name\" : \"the_function_name\"} }
        /// </summary>
        public string? ToolChoice { get; set; }

        /// <summary>
        /// Optional. Presence penalty applied to the next token's logprobs if the token has already been seen in the response.
        /// </summary>
        /// <remarks>
        /// This penalty is binary on/off and not dependant on the number of times the token is used (after the first).
        /// Use frequencyPenalty for a penalty that increases with each use. A positive penalty will discourage
        /// the use of tokens that have already been used in the response, increasing the vocabulary. A negative
        /// penalty will encourage the use of tokens that have already been used in the response, decreasing
        /// the vocabulary.
        /// </remarks>
        public float? PresencePenalty { get; set; }
        /// <summary>
        /// Optional. Frequency penalty applied to the next token's logprobs, multiplied by the number of times each token has been seen in the respponse so far.
        /// </summary>
        /// <remarks>
        /// A positive penalty will discourage the use of tokens that have already been used, proportional to the number
        /// of times the token has been used: The more a token is used, the more difficult it is for the model to use
        /// that token again increasing the vocabulary of responses.
        /// Caution: A negative penalty will encourage the model to reuse tokens proportional to the number of times
        /// the token has been used. Small negative values will reduce the vocabulary of a response.
        /// Larger negative values will cause the model to start repeating a common token until it hits the
        /// maxOutputTokens limit: "...the the the the the...".
        /// </remarks>
        public float? FrequencyPenalty { get; set; }

Cheers, JoKi

Hugo_Lebegue · November 26, 2024, 5:20pm

Thank you for forwarding these findings to the Eng Team. It would be great if the API could communicate the reason for the 400. This will help save time debugging.

Have a great one, and again, thank you for your help

Hugo_Lebegue · November 28, 2024, 5:44am

Finally, using the gemini flash-1.5, you will get a 400 when

you are trying to analyze a picture that is too large (exceeds something lie 1 MB I think (not completely sure since the error code is not received) and
the file_url can not be an actual URL, but MUST be a base 64. In Open AI you can send a standard web url, (Modified by moderator) but for Gemini, it must be a small Base 64

I will continue to add my findings here.

Thank you

Topic		Replies	Views
OpenAI compatibility not working anymore Gemini API gemini-15 , api	5	197	November 19, 2024
OpenAI list models returns HTTP 401 Gemini API api	0	34	December 3, 2024
New endpoints Chat, Embedding and OpenAI Gemini API api , ai	9	144	November 25, 2024
Usage not showing for OpenAI compatibility Gemini API open-models	1	68	November 22, 2024
OpenAPI spec File Gemini API gemini-15 , api , models	1	114	August 22, 2024

OpenAI Compatibilioty with JNodeJS - Error

Any help is appreciated

Related topics