Trustworthy RAG: Bounding-Box Citations and Conditional OCR

Every RAG chatbot says "according to the document." Mine draws a box on the actual page and makes you look.

Most "chat with your PDF" demos stop at extract → embed → retrieve → answer. The parts that matter in production are the ones they skip: scanned pages, source verification, and what happens when a dependency fails.

flowchart TD
    subgraph Ingest
        pdf[PDF page] --> chk{Page earns OCR? under 50 native chars}
        chk -->|no — born digital| blk[Text blocks + bounding boxes]
        chk -->|yes — scanned| ocr[Tesseract OCR] --> blk
        blk --> chnk[Chunks carrying bbox metadata]
        chnk --> emb[MiniLM embeddings] --> idx[(FAISS)]
    end
    subgraph Query
        q[Question] --> vs[Vector search] --> idx
        vs --> ans[Answer + matched chunks]
        ans --> ov[Red overlays on the rendered page]
    end

Bounding boxes ride the whole pipeline as chunk metadata — the query path never re-opens the PDF.

Every fallback has a scar

None of the resilience here is speculative — each fallback was added after a real failure.

src/config.py

MODEL_NAME = "gemma-3-27b-it"          # free tier, ~30 RPM
EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
CHUNK_SIZE = 1000                       # chars
CHUNK_OVERLAP = 200
TOP_K = 4
OCR_MIN_NATIVE_CHARS = 50               # below this, a page earns OCR
THROTTLE_SECONDS = 2

The store is in-memory FAISS on purpose — self-contained, zero infrastructure — with any persistent swap (Chroma, pgvector) isolated behind a single module. Multi-turn memory and per-page rendering round out the tour.

A citation you can't inspect is just a confident tone of voice.