gemini-2.5-pro, although being a superb coder, cannot help with it, since the new ts/js sdk is after its training cutoff.
this is possible with openai/anthropic api:
#!/usr/bin/env -S npm run tsn -T
import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic(); // gets API Key from environment variable ANTHROPIC_API_KEY
async function main() {
const stream = client.messages
.stream({
messages: [
{
role: 'user',
content: `Hey Claude! How can I recursively list all files in a directory in Rust?`,
},
],
model: 'claude-3-5-sonnet-latest',
max_tokens: 1024,
})
// Once a content block is fully streamed, this event will fire
.on('contentBlock', (content) => console.log('contentBlock', content))
show original
#!/usr/bin/env -S npm run tsn -T
import OpenAI from 'openai';
const openai = new OpenAI();
async function main() {
const runner = openai.beta.chat.completions
.stream({
model: 'gpt-3.5-turbo',
messages: [{ role: 'user', content: 'Say this is a test' }],
})
.on('message', (msg) => console.log(msg))
.on('content', (diff) => process.stdout.write(diff));
for await (const chunk of runner) {
console.log('chunk', chunk);
}
const result = await runner.finalChatCompletion();
show original
and even possible with gemini model via openai compatible api.
even possible with the generativeai api?
{
role: "user",
parts: [
{
text: "Count from 1 to 10, put each number into square brackets and on a separate line",
},
],
},
],
});
const finalResponse = await result.response;
expect(finalResponse.candidates.length).to.be.equal(1);
const text = finalResponse.text();
expect(text).to.include("[1]");
expect(text).to.include("[10]");
});
it("stream true, invalid argument", async () => {
const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY || "");
const model = genAI.getGenerativeModel({
model: "gemini-1.5-flash-latest",
safetySettings: [
(with full respect and the recognition that the new api is still being built in public)
Hi @Sirui_Lu ,
Apologies for the late response. If you want to generate a non-streaming response, you can use the ai.models.generateContent API. You can check the following example . Thank you!
yes but somehow that is less stable and results in more time out than the streaming version…
1 Like