Using Gemini API to Categorize Junk Removal Photos Before Creating Customer Estimates

Gemini API
1

Hi everyone,

I’m experimenting with Gemini API for a small local service workflow.

The idea is to let customers upload photos of items they want removed, then use Gemini to:

  • identify common household items
  • group them into categories (furniture, appliances, yard waste, electronics, etc.)
  • estimate pickup complexity
  • return structured JSON that can be passed to another application for quote generation

For those who have built similar vision workflows:

  • Which model has worked best?
  • Do you rely on structured JSON output?
  • How do you reduce misclassification when multiple items appear in one image?
  • Any prompt engineering tips for more consistent results?

I’d appreciate hearing about real-world implementations.

2

We’ve been running a Gemini vision workflow in production for our real estate photo editing app picaid.app and here’s what’s worked for us:

Model: We use gemini-3.5-flash-lite for analysis. It’s fast and handles multi-item scenes well. We also have gemini-3.5-flash and gemini-2.5-flash available as options when we need more accuracy, at the cost of speed.

Structured JSON: Yes, absolutely. Define the schema explicitly in the prompt and pass a JSON example as reinforcement. Gemini is quite reliable at staying within it.

Reducing misclassification with multiple items: Instruct the model to enumerate each distinct item individually before categorizing, rather than summarizing the scene. Also be explicit about uncertainty: “if unsure, default to X category” rather than leaving it open.

For certain scenarios you are better to give a multiple choice query always leaving an out for when it can be none of the above. Other scenarios a simple is or isn’t works.

Prompt tips:

  • Provide the category list explicitly — don’t let it invent its own
  • For cluttered images, add “focus only on items clearly intended for removal, ignore background” — cuts noise significantly

The explicit enumeration + predefined category list combo has been the most impactful change for consistency.

I can’t give away too much as we’ve got thousands of hours invested in development and honing the right approaches to specific tasks, but you’re asking the right questions.

You can try the app for free for a few photos and see how it works, check the “Objects” tab under each image’s settings and you’ll see that we even give the user the ability to toggle on or off our app’s acknowledgement of those items. On means it can be considered for editing, Off means it is to keep its hands off of those items no matter what (leave them as they are).

3

Thanks for sharing your experience. The point about explicitly enumerating each item before categorizing it makes a lot of sense.

One thing I’m still wondering about is how you handle confidence levels when the image quality is poor or objects overlap.

Do you return a confidence score for each detected item and trigger a manual review below a certain threshold, or do you rely entirely on prompt instructions?