So I noticed the line “You are not to generate simulated data or simulate anything!” in the AI thinking process and realized why an important part of my app keeps failing! As it turns out, the code isn’t doing what I asked for, which was to find bounding boxes around all persons in an image using Gemini. Instead, it started to simulate this process and created some fake boxes for each person. Thus, I got the right number of boxes, but when it had to cut each person out, I got a piece of sky or something else.
So I had to tell AI Studio that any data simulation is absolutely forbidden. But it amazes me that the AI is even trying to simulate data when it is requested to extract data. This will confuse a lot of people.
The reason for this seems to be a 429 error so the AI thinks it’s better to please the user and simulate data than just to provide an error message and tell me it can’t do it.
And another point: while I told the AI to only do what I asked and not change things unrelated to my prompt, it keeps changing stuff anyway. As a result, it keeps damaging the code as functions disappear or are altered and rewritten. Even the user interface keeps changing.
For example, I had a console log. Poof, gone! I had a list of documents in the help menu of my app. Poof, gone. I had an option to enter an API key. Poof, dead. This is happening more and more often, making the whole AI Studio completely useless as it keeps damaging what I’m trying to build.
Trying to replicate it is challenging as I don’t even always notice that the AI is adding a simulation. In this case, I am feeding an image to my App, which then uses Gemini to find bounding boxes around all the persons in the image. It then has to make cutouts of each person and store that in a gallery. In further actions, I can select one or more persons and ask Gemini to complete the person by adding any missing arms or legs that weren’t visible in the original picture, plus do some other improvements. However, AI Studio decided that this call to get the bounding boxes should be a simulation, so I never got a correct image of any person.
After some trial and error, it finally did what I asked for, but I’m not sure if this simulation repeats if I ask the app again to make these bounding boxes.
As for AI Studio damaging the app instead of fixing it. That is a regular occurrence when app.tsx grows to be large. (Over 750 to 1,000 lines.) It truncates this file, and thus I lose functionality. So now I told AI Studio to make a minimal app.tsx file and put all the logic into separate modules in the /app folder. This seems to work okay, although AI Studio still tends to truncate files.
I have learned to just roll back instead of asking AI Studio to fix the code. That’s because it lost code because of the truncation, which it would need to regenerate from my “Fix it” prompt. It then becomes “creative”. But splitting all modules into smaller modules helps a lot to minimize this damage.
Still leaves the problem of AI Studio doing things I did not ask for. Often just minor things like changing captions of buttons or adding or removing icons on buttons. My app allows me to add names of people to pictures, and AI Studio thought they would also need a profile picture. But it has no profile pictures, so instead it gave each name a circle with their first name in it. My app is not meant to make profiles of people.
As I said, the AI is creative. Sometimes too creative…