AI Studio and safety settings

Google AI Studio
,
1

So I noticed that AI Studio can make libraries for Imagen and Gemini, yet it now fails to pass the safety settings correctly to the API. As a result, they are now being ignored.
I work on a story-telling App with several female characters as main characters. In the stories they do fun stuff like going to the beach in bikini or being “friends with benefits”. Nothing erotic, but still sensitive enough to need a lower safety setting. (The word “bikini” must pass the censorship!)
While looking at the generated code, I noticed that AI Studio is putting the safety settings within the config block. Previously, it had to be outside the confiig block! So in the past, I would have something like this:

const request = { 
    model, 
    contents: { parts: [imagePart, textPart] }, 
    config: { 
        responseMimeType: "application/json", 
        responseSchema: landscapeSchema, 
        temperature: 0.5, }, 
    safetySettings };

And safetySettings are put outside the config.
Now I get something like this:

const response = await ai.models.generateContent({
        model,
        contents: [{ role: "user", parts: [{ text: prompt }] }],
        safetySettings,
    });

And yes, this now breaks my app when the word “bikini” appears as that is a sensitive word. (Who came up with that?)
Plenty of people are already complaining that these safety settings are now messed up and are causing problems. A lot of apps are breaking when AI Studio changes how these Gemini and Imagen APIs are called. This is a serious issue now and it needs to be properly fixed.

In the old versions, I would use @google/genai and have the safety settings outside the config block. The word “Bikini” thus gets accepted. But now AI Studio puts it inside the config block and “Bikini” explodes the app. Well, it just generates errors. Make it work again!

1 Like
3

Hi @Katje , Welcome to Forum
I have reviewed your issue and confirmed that safety settings remain part of the config.


import {
  GoogleGenAI,
  HarmBlockThreshold,
  HarmCategory,
} from '@google/genai';

async function main() {
  const ai = new GoogleGenAI({
    apiKey: process.env.GEMINI_API_KEY,
  });
  const config = {
    thinkingConfig: {
      thinkingBudget: 0,
    },
    imageConfig: {
      imageSize: '1K',
    },
    safetySettings: [
      {
        category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
        threshold: HarmBlockThreshold.BLOCK_NONE,  // Block none
      },
    ],
  };
  const model = 'gemini-flash-latest';
  const contents = [
    {
      role: 'user',
      parts: [
        {
          text: `INSERT_INPUT_HERE`,
        },
      ],
    },
  ];

  const response = await ai.models.generateContentStream({
    model,
    config,
    contents,
  });
  let fileIndex = 0;
  for await (const chunk of response) {
    console.log(chunk.text);
  }
}

main();

Please try different safety settings and share the updates with us.

1 Like