Built a local knowledge layer on macOS with EmbeddingGemma 2 + MCP

Gemma
1

Hi everyone,

I’ve been experimenting with EmbeddingGemma 2 and built a small local knowledge layer for macOS.

The goal is to keep the architecture simple and reusable:

local files
→ EmbeddingGemma 2
→ local vector index
→ semantic search service
→ MCP
→ ChatGPT / Codex / other AI agents

The search layer is independent from the LLM, so multiple AI clients can query the same local knowledge base.

The current scope is intentionally limited:

  • local-first
  • read-only
  • semantic search over local files
  • no cloud dependency for retrieval
  • reusable through MCP
  • not tied to one specific assistant

What I find especially interesting is using EmbeddingGemma 2 as the retrieval layer while keeping the LLM/agent completely separate.

I published the project here:

Github

I’d really appreciate feedback from people working with Gemma, local retrieval, semantic search, or MCP.

Do you think this “local knowledge layer” approach makes sense as a reusable component, rather than building a full RAG stack around one specific model or assistant?