# Using Gemini structured output to turn messy scraped HTML into clean JSON — sharing my pipeline

**URL:** <https://discuss.ai.google.dev/t/using-gemini-structured-output-to-turn-messy-scraped-html-into-clean-json-sharing-my-pipeline/179642>\
**Category:** Gemini API\
**Tags:** gemini\
**Created:** [August 25, 2026, 2:19am UTC](https://discuss.ai.google.dev/t/using-gemini-structured-output-to-turn-messy-scraped-html-into-clean-json-sharing-my-pipeline/179642 "2026-08-25T02:19:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Greta](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/greta/32/144215_2.png) [@Greta](https://discuss.ai.google.dev/u/Greta)\
**Post date:** [August 25, 2026, 2:19am UTC](https://discuss.ai.google.dev/t/using-gemini-structured-output-to-turn-messy-scraped-html-into-clean-json-sharing-my-pipeline/179642/1 "2026-08-25T02:19:48Z")

</div>

Hi all — sharing a pattern that’s been working really well for me, in case it helps anyone building data pipelines on top of the Gemini API.

The problem: web pages are hostile input for downstream systems. I collect product/pricing data from public pages across multiple markets (via a geo-targeted residential proxy layer, since most of these pages serve different content per country), and the raw HTML-to-structured-data step was always the fragile part. CSS selectors break on every redesign; dedicated parser maintenance became a second project.

What changed: I stopped parsing and started asking Gemini to do it.

The flow:

1. Fetch page HTML (requests + residential proxies for geo-appropriate vantage points)
2. Strip scripts/styles, keep semantic HTML (cuts tokens roughly 60–70%)
3. Send to Gemini with a response\_schema, so output is guaranteed JSON

The key piece is structured output — a Pydantic model defines the schema once, and every response comes back validated:

```python
from pydantic import BaseModel
from google import genai

class Product(BaseModel):
    name: str
    price: float
    currency: str
    availability: str

client = genai.Client()
resp = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=f"Extract product data from this HTML:\n{clean_html}",
    config={"response_mime_type": "application/json",
            "response_schema": Product},
)
product = Product.model_validate_json(resp.text)

```

Notes from running this in production for a couple of months:

- gemini-2.5-flash handles this task well and keeps per-page cost negligible
- Stripping scripts/styles before sending is the single biggest cost/quality lever
- Structured output eliminates the “almost valid JSON” failure mode entirely
- For multilingual markets, I keep the prompt in English but let the model extract in the page’s language, then translate in a second pass — cheaper than prompt + page both being multilingual

Curious whether others here have hit the “LLM as parser” pattern, and what you do about hallucinated fields on sparse pages — my current mitigation is a confidence check where the model must also return an “evidence” substring I verify against the raw HTML.ge