Building an Agentic "Assistant Producer": Turning Vertex AI Gemini 2.5 Conversations into 1-Click Database Commits

Community
, , ,
1

Hey everyone,

We are building FAST (Film & Media Operating System) on Google Cloud Platform, and wanted to share an architectural pattern we recently deployed that solved a major user friction point: bridging unstructured conversational AI with strict, transactional database writes.

The Problem: The “Chat-to-Action” Gap

In film and commercial production management, users often converse with AI to brainstorm schedules, log equipment overages, or adjust call times. Traditional chat assistants output great Markdown text, but force the user to manually copy/paste details across 5 different database forms (schedules, budgets, call sheets, contact directories).

The Architecture: Structured Staged Actions

To make our AI assistant (Prodigi) act like a veteran Assistant Producer, we upgraded our pipeline on Vertex AI using gemini-2.5-flash in us-central1:

  1. System Persona & Structured Payload:

    We instruct Gemini to return conversational reasoning alongside an optional typed array of up to 3 stagedActions:

TypeScript

interface StagedAction {
  id: string;
  pillar: 'dashboard' | 'who' | 'what' | 'where' | 'when' | 'goal' | 'messages';
  actionType: 'create_task' | 'add_event' | 'create_contact' | 'add_budget_line' | 'add_location' | 'send_message';
  title: string;
  payload: {
    title?: string;
    dueDate?: string;
    amount?: number;
    department?: string;
    recipient?: string;
    [key: string]: any;
  };
}

  1. Frontend Interactivity (React Native Web):

    Instead of blindly executing database mutations or dumping raw text, our UI parses the payload and renders Interactive Staged Action Cards directly beneath the chat bubble.

  • The user can edit dates, dollar amounts, or recipient tags inline.

  • A single click on [ Commit Action ✓ ] triggers an authenticated POST request to our domain endpoints (/api/tasks, /api/events, /api/budget), writing atomically to our database and updating navigation badges in real time.

┌────────────────────────────────────────────────────────────────────────┐
│ PRODIGI ASSISTANT                                                      │
│ "I've logged the lighting package adjustment for the night shoot.      │
│  We also need to flag the overtime meal penalty."                      │
├────────────────────────────────────────────────────────────────────────┤
│ ⚡ STAGED ASSISTANT ACTIONS (2 DETECTED)                               │
│ [ 📅 When ]  Night Shoot Call: Scene 14 (Oct 14 @ 18:00) [ Commit ✓ ]  │
│ [ 📈 Goal ]  Budget Variance: +$650 (Meal Penalty)       [ Commit ✓ ]  │
└────────────────────────────────────────────────────────────────────────┘
  1. Performance Gains with Gemini 2.5 on Vertex AI:

  • Sub-second structured extraction: Fast response times even with complex film production prompts.

  • Zero schema hallucination: Combining system instructions with JSON schema constraints yields reliable parsing across web and mobile clients.