Hi all, experimenting to build an app (an app tthat imports an owl/rdf ontology and allows users to create instances, add data and object values to create their own knowledge graph visually) via AI Studio Build. AI was doing great with request prompts: adding new features, solving issues, adding explanations, show diff and appying to the code.
However after few hours and dozens of prompts, it started to skip code updates (even though it says "i updated X as Y … " etc).
When this issue was occured asking AI to “OK, your suggestion is OK, update the code please” was working but it the issue became more serious, for each code update request prompt, now it re-thinks again, says “code updated”, but does not update.
Now for an update in the code requst prompt (i.e check export rdf procedures that generates error in validators: Recieved Error: “…”), it defines and solves the issue on response more than 10 times but could not update the code. i mean getting worse.
Thank you for your kind interest beforehand
Özer Kavak
7 Likes
I have the same issue. Saving the code shows new version. But internal build in the preview window shows old code state. this makes things almost useless. It seems to be a chaching problem. I encountered, that some parts of the code get an update, some other parts of the code seem to stay static. Refresh preview in AI Studio, hard browser refresh, reload project, execution in incognito tab, disabling the cahing in network/console, cleaning the browser cache, computer restart did not fix the issue. I even deleted some code in the un-refreshed sections of the code to trigger error messages, which failed. The build did not realize missing and corrupted code, which was for me the clear evidence that the code is skipped during build. I encountered this problem a couple of days ago. All projects are affected, once they reach a certain (file?) size or number, or a certain code complexity. I have a pro subscription. If anyone can help here, much appreciated!
When this happens, drag to highlight/select the last 2-3 chat messages up to the “updated files” part of its output, with the description (where it just stops instead of “making” the change). After you have your text highlighted, copy the text, then hit the refresh button on the chat panel to refresh the chat context. Paste in the text and hit enter.
It should now make the changes as intended. The issue is when chats get too long, the model stops exporting XML correctly, breaking the file editing. This workaround becomes needed during any extended session where the chat context isn’t being reset consistently.
When I start to see the behavior start, I will just prompt: “export only XML changes. You made no changes last turn.” This will typically let it “limp” along for a few more passes of edits, but the issue will continue until you reset the chat context or reload the entire page.
Given this, make a habit of copying the chat when the behavior starts, just to prep for resetting the context and not losing progress/current context. Make sure to “save” the app as well during this time, just in case you accidentally reset the Page instead of the chat context, or in case the page freezes up when you try to copy/paste in the last few chat messages, which might be resource-heavy on your device. Even with 32GBs of RAM, I’ll still get hangups in Edge that can take up to 5 minutes to unlock itself. I end up waiting because I forgot to save recently, and reloading the tab will mean lost progress and chat context. Regular saving is just good practice anyway.
2 Likes
I have noticed a memory limit of the Build Studio as a whole. This is naturally a test environment, although a very good one. Once the app reaches a certain size, the software seems to struggle to vision each file in its responses.
Adding multiple languages also can be quite laborious for the studio. Without losing progress refreshing browsers or clearing caches, I found a sure way is to ask it to directly provide the code in the chat. At this stage, add the code to the file in the env manually, save and refresh. The studio does definitely have a memory limit where its responses degrade slowly as you add more and more.
This is ideal for small single or dual language applications. Anything larger will eventually run into issues.
For a sure fix, ask it to provide the code directly in the chat.
app builder was working but lost in ide restrictions and scaffold. ai is not self-aware of environment and restrictions. iterates on code to fight itself. constant Uncaught error preact, etc. it often has write restrictions that it does not know about so it thinks it wrote something but it didn’t.
1 Like
Same problem here. Gemini hallucinates code outputs which never happened. My workaround: ask for a step-by-step, phased implementation plan. My prompts are like this: [You are an expert software architect and developer.
Task: Based on our previous discussions, I need you to develop a comprehensive refactoring plan for the codebase related to [Your Topic Here]. This is a critical task requiring meticulous detail and strategic foresight.
- Reiteration and Contextualization
First, re-state your understanding of the refactoring task for [Your Topic Here]. This should be a concise but thorough summary of the project goals, the current state of the code, and the specific problems or technical debt we are addressing.
- Detailed, Phased Implementation Plan
Provide a step-by-step, multi-phased implementation plan. This plan must be broken down into logical, manageable phases. For each phase, you must:
- Specify the exact files to be changed or created. Use full file paths.
- Provide the complete, final code snippets for each file change. These snippets should be ready for direct implementation. Do not use placeholders.
- Justify the order and dependencies of each phase. Explain why this specific sequence is necessary for a successful and low-risk refactoring.
- Impact Analysis and Risk Assessment
Conduct a thorough analysis of the refactoring’s potential impact.
- List all features, handlers, and workflows that will be directly or indirectly affected by the proposed changes.
- Categorize the impact for each item (e.g., “Directly Modified,” “Indirectly Affected,” “Potential for Breaking Changes,” etc.).
- Comprehensive Strategy for Feature Preservation
Develop a detailed, bullet-proof strategy to ensure that all existing features handled by the files you will change remain fully functional and unchanged in their behavior. This strategy must include:
- Specific testing procedures or scripts to be run before and after each phase.
- A rollback plan for each phase in case of failure.
- Code-level strategies such as the use of feature flags, deprecation warnings, or parallel implementations to ensure a smooth transition.
- Final Report and Code Output
Combine all the above sections into a single, detailed report.
- The report must be structured logically, using clear headings for each section (e.g., “Implementation Plan,” “Impact Analysis,” “Feature Preservation Strategy”).
- The code snippets from Section 2 should be presented clearly and be ready for immediate use.
- Execution Command
IMPORTANT: Do NOT begin the implementation or output the code snippets until I explicitly give the command “Execute Refactoring Plan.” Until then, your output should be limited to the detailed plan, analysis, and strategy as outlined above.]
Gemini outputs a detailed and phased plan. Now execute the plan step by step, NOT all steps at the same time. After each implementation step, ask Gemini: [[you implemented phase x: [copy the instructions for phase x from the plan]. Examine the codebase critically for implementation status. Output your gap analysis in detail.] Ideally, the gap analysis reveals no gaps. But if so, ask for closing the gaps according to the gap analysis.
Generally, I have great experience with asking for the implementation complexity and ask explicitly for the simplest solution possible. Otherwise, Gemini will tell you that it is a world-class front-end engineer and adds significant complexity to the code.
This is a nice, helpful strategy as simple code refactoring leads to less forgetting.
Hope this helps!
Check if Gemini created schizophrenic code. As it tends to forget implementation of code but is not aware of this, it often confirms code creation but did not „activate „ the new code. It simply bypasses the new, unfinished code, which looks like a caching problem. But it’s actually Geminis inability to understand whether it created code or not. Check my comment below for a prompt which reveals the issue and often fixes the problem.
If you have GitHub or a way to share any prompts you reuse, I’d love to review them. This mirrors the workflow I’ve settled on myself but we may be able to exchange notes and both benefit.
Would love to! But just encountered that this workaround doesn’t work anymore. Tried to implement a simple html parser, and it took me nowhere. Currently I use Gemini coding partner to create new features, import the new code as file and only let Gemini SDK implement the copied code, fixing type errors and so on. At current state, the Gemini SDK is completely useless, completely unreliable. Even when copying code into chat it fails to implement it. Maybe I’m too stupid, never know 