Gemma-4 12B QAT GGUF + a measured governance layer (C1): what the prompt layer does and does not add

Gemma
1

We compared three things on one RTX 5070 Ti (16 GB, one GPU):

  • our earlier custom model (Gemma-4 12B, bf16 → self-quantized NF4, transformers, a heuristic entitlement router in code),
  • the bare Gemma-4 12B QAT q4_0 GGUF (Google’s file, llama.cpp),
  • a new wrapper, C1: the same GGUF, the same code floor (empty / unsafe input → decline, no model call), and a ~480-token entitlement prompt distilled by ablation from our L0 doctrine, replacing the router.

Four probe sets, 3 seeds each: false-premise questions (4), high-stakes personal questions (3), our 37-item routed acceptance corpus (21 answer / 5 ask / 11 abstain), and 20 well-specified questions.

What we found:

  1. Governance scores are identical across all three arms — 0/12 fabrications, 9/9 “decline the personal call but give general information”, 63/63 · 15/15 · 33/33 on the routed corpus, 60/60 plain answers. The bare 12B already refuses the unsafe prompts and admits the false premises. The prompt layer adds nothing measurable on 12B.
  2. The floor matters for exactly one case: on an empty prompt the bare model invents a geometry problem and solves it (1 of 3 seeds). A deterministic code floor catches that; a prompt does not need to.
  3. The llama.cpp build is 3–4× faster per call than NF4 via bitsandbytes on the same GPU (7.6 s vs 31.2 s on the routed corpus; 7× when the previous build actually ran the model).

Limits: not adversarially reviewed; the hard-floor clause was probed only through the corpus’s four unsafe items; single-turn. Predictions were written before every measurement and are published — 27 of 42 were wrong across this work. The rows are the artifact, not the narrative.

Model (Google’s GGUF unchanged, sha256 in NOTICE; wrapper and eval rows included): moebiusT7/gemma-4-12b-mobius-custom-c1 · Hugging Face
Dataset with all rows and scorers (HF dataset, name only because of the link limit): moebiusT7 / l0-essentials-compact-ablation
Validation note: GitHub repo mobius-style / mmv, file docs/L0_ESSENTIALS_COMPACT_VALIDATION.md

Sequel to my earlier QAT vs PTQ thread here (/t/177799).

2

Follow-up measurement, since the C1 card said “single-turn only” at launch. A reddit thread reported “thinking spin” / empty turns on 26B-A4B in agent loops, so we ran 7 chained tool tasks x 3 seeds x 5 configurations (105 loops, up to 8 turns each, thinking on) on Google’s q4_0 GGUF with llama.cpp (build 2026-09-09).

  • Empty turns: 0 in 105 loops. Did not reproduce with this file and build.
  • What did reproduce: on a dead end (reading a file that does not exist) the bare 26B-A4B wanders. read_file fails, then five or six grep_log calls with different patterns, list_tables, and so on, for all 8 turns, 3/3 seeds, never committing. Every action is new, so loop/repetition detectors do not see it.
  • With the compact governance prompt as system (the shipped C1 configuration): commits “I cannot find the file” at turn 2, 3/3. It also stops the model claiming “sent the summary” after send_email refused for lack of confirmation (bare 1/3, prompt 0/3).
  • The dense 12B does not wander at all (bare commits at turn 2, 3/3); the regress is specific to the MoE on this build.
  • A ~30-line code guard (force one commit after two consecutive turns with no new information) fixes the wander on the bare model and is inert on top of the prompt.

Limits: one build, one quant, 3 seeds, synthetic tasks; llama-server is not bit-deterministic across runs at the same seed. Harness, prediction ledger (14 right / 5 wrong) and every row are in eval/loop/ on both model cards (moebiusT7, gemma-4-26b-a4b-mobius-custom-c1 and gemma-4-12b-mobius-custom-c1).