I’ve asked Gemini to help format the below report after I found the model hallucinated an h1 pawn in an API response to a chess analysis prompt.
Gemini API Hallucination Bug Report
Model Issue: Logic / Grounding Hallucination (Falsely identifying non-existent pieces)
Context: Chess position analysis via FEN payload.
1. Description of Bug
The Gemini model hallucinated the presence of an “h1 pawn” in its analysis. According to the official rules of chess, a pawn can never exist on the 1st rank as it must promote upon reaching the end of the board. Furthermore, the provided FEN string does not contain any pawn on the h1 square.
2. Input Data & Payload
- Current Position (FEN): k7/5p2/4p1p1/4P3/5PKP/5R1P/8/5r2 b - - 9 48
- Move Played: Rg1+
3. Why this is a Hallucination
- FEN Breakdown: The 1st rank row in the FEN is represented by the final
5r2(5 empty squares, 1 black rook, 2 empty squares). The h1 square is entirely empty. - Chess Rule Violation: Pawns move forward. A white pawn can never be found on the 1st rank (h1).
- Model Output Error: The model generated an analysis referencing an “h1 pawn” which legally and physically cannot exist in this position.
4. Expected Behavior
The model should strictly parse the provided FEN string, respect the legal rules of chess, and only mention pieces that are actually present on the board.
Below is a subset of the response payload:
"text": "{\n \"short_summary\": \"Your move 48... Rg1+ provides an ineffective check, allowing White to comfortably block with 49. Rg3. Black misses the immediate opportunity to attack the vulnerable h1 pawn.\",\n \"chess_principle\": \"Active Piece Play & Target Identification\",\n \"deep_dive\": \"Your move 48... Rg1+ provides an ineffective check, permitting White to interpose 49. Rg3 and shield the king from further immediate threats, allowing White's king to secure its position and consolidate. Instead, 48... Rd1 is superior as it instantly targets the vulnerable h1 pawn, creating a direct material threat and forcing White to respond defensively, thereby gaining tempo and maintaining initiative for Black. By failing to attack the h1 pawn directly, Black loses a crucial opportunity to put pressure on White and capitalize on the exposed pawn structure, ultimately leading to a slightly less favorable endgame position.\"\n}"