How It Works
Two-corpus RAG with a live Box integration. Here's what's actually happening under the hood.
The pipeline
Scrape FDA enforcement corpus
870+ CDER/CBER warning letters (2019–present) scraped from FDA.gov, categorized into 10 violation areas using Claude, chunked, and embedded into Pinecone.
Connect to Box via JWT
Internal quality documents live in a Box folder. The server-to-server JWT connector downloads files on demand — no migration, no export.
Embed internal documents
Each document is chunked with section context preserved and embedded into a separate Pinecone namespace. Box webhooks trigger automatic re-ingestion when files change.
Cross-corpus retrieval
For each violation category, semantic search runs against both corpora in parallel — retrieving the most relevant warning letter passages and internal document sections.
Risk signal generation
Claude analyzes the enforcement patterns and document evidence to produce a structured signal: enforcement frequency, document coverage assessment, and a specific review prompt for the team.
Stream results in real time
Signals appear as they complete — 10 categories processed in parallel batches, streamed via SSE so users see results progressively rather than waiting for the full scan.
Architecture
Who uses it and why
Same engine. Different use cases.
The two-corpus RAG architecture adapts to any domain where external reference data needs to be cross-referenced against internal documents.
| Pharma Intelligence | Compliance Copilot | Rules Expert | |
|---|---|---|---|
| Document corpus | FDA warning letters + quality SOPs | 21 CFR Part 11 + policy documents | 2023 Rules of Golf |
| Retrieval | Two-namespace cross-corpus | Single-namespace requirement matching | Hybrid vector + BM25 |
| Output | Risk signals with coverage assessment | Gap analysis with requirement status | Cited rule answers |
| External integration | Box JWT connector | Static document upload | None |
| Streaming | SSE (signal-by-signal) | SSE (requirement-by-requirement) | UI message stream |
Ready to explore?
Start with enforcement trends or run a full risk scan against Meridian's documents.