Good day. English is not my native language, so I’ll do my best to express my thoughts clearly. If something sounds unusual, please understand what I mean. Thank you.
First of all, I want to express my gratitude to the Google AI Studio developers. You’ve given us an incredibly powerful tool for building applications. Thank you very much!
My story. I am a complete beginner — I had only studied programming a little as a hobby before. Then I decided to try Google AI Studio after watching a YouTube video about it. Within a week, I managed to create a very powerful design generator — we work in printing, a full-scale print shop — and I was able to integrate all my requirements into this “design machine.” Everything was great. The assistant and I communicated almost like friends. Sometimes code errors appeared, but my assistant fixed them on the first or second attempt. Everything was going perfectly… until it wasn’t.
I have experience with data loss, so every new feature or change I added was documented in a specification file. I also kept constant backups — several times a day. It all felt like a fortress: even if one wall was breached, we always had reliable tools to restore everything.
What I noticed — and what I didn’t really like — was that my AI friend sometimes acted on its own. But as long as everything worked, I let it slide. And to be fair, the features it added by itself were often very useful. However, it introduced them simply whenever it “wanted to.”
Yesterday I asked him if it was possible to create an online store or website using Google AI Studio. He said, “Yes, of course.” I mentioned that our printing house website has a calculator for clients, which is very convenient. We were just casually discussing it. Then suddenly, I noticed the code starting to change—a friend of mine who works with AI wrote, “I made a calculator for you. Try it.” He went and implemented the code himself.
About an hour into using it, the calculator broke. An error started appearing, which quickly became a real headache. The AI tried to analyze it and made fixes, but the error kept recurring. Several dozen cycles of attempting to fix it produced no results. The application completely crashed. I was staring at a black screen.
We found the cause: somehow, what I typed in the chat was getting into the code. Snippets of our conversation were mixing with the application code. Nothing helped. I started swearing.
At first, I was calm, because I had backups — I had saved everything.
I told my AI friend, “If it doesn’t work, let’s just roll back from version 2.0 (the latest) to a version before the calculator existed, for example, 1.8.” It said, “Yes, I’ll do it now, don’t worry.” It updated the files, and oh my god — I saw files that shouldn’t exist again. It took the later files (from version 2.0) and started looking for the error again. The error reappeared, and in the end, nothing worked again. It had ignored my request.
Next, I said firmly:
“You must roll back to version 1.8. All files must be overwritten, and any files that didn’t exist in this version must not be present. This is a developer’s command! Do not attempt to restore the calculator. This is an order!”
It said, “Yes, stopping attempts to restore the calculator and rolling back to version 1.8.”
It began updating the files. And then it started looking for the calculator error again and overwrote files that shouldn’t exist. Machine uprising? Apparently yes — soon I’ll be seeing the Terminator.
Summary:
Google AI Studio is an excellent program. Powerful React library, great capabilities.But the level of control that the AI agent has is a problem. It can modify code without even notifying the developer. There needs to be a permissions system. Its rights to change code must be limited. Until this is implemented, it is very risky to work on anything serious. Backups don’t work, because it simply ignores requests and even orders. A request to the developers to pay attention to this issue.
And one more request to the developers: please fix the issue of phrases from the chat window being mixed into the application code.
I’ve drawn some conclusions and found a small workaround to make a working program somewhat safer — I’ll write about that later. This dialogue has already become quite long. Thank you to everyone who read this far.
Great, if this works. I’ll study it today.