The Challenge
Apex Capital maintained a private regulatory repository consisting of 850,000 SEC filings, compliance briefs, and private wealth research reports. Their internal advisor assistant suffered from severe semantic drift: naive chunking broke tables across pages, causing standard embeddings to hallucinate yield numbers and misquote regulatory guidance.
The Architecture & Technical Solution
ThatWare built a hardened Hybrid Retrieval-Augmented Generation (RAG) pipeline:
-
Context-Aware Semantic Chunking: Replaced fixed-token chunking (512 tokens) with recursive structural parsing that preserves markdown tables, parent-child section relationships, and entity headers intact.
-
Dense & Sparse Hybrid Reranker: Deployed a dual-retrieval pipeline combining Qdrant Vector Embeddings (dense) with BM25 Lexical Matching (sparse), followed by a Cohere Rerank layer to return top-5 precision matches.
-
Metadata Capsule Injection: Every retrieved document chunk received injected JSON metadata tags (issuer, date, compliance tier, audit hash) prior to LLM synthesis, enabling deterministic source citation.
The Transformation & Results
-
99.4% Extraction Precision: Hallucinations on regulatory and yield metrics dropped to near zero across 250,000 monthly internal queries.
-
76% Reduction in LLM Inference Cost: Tighter, reranked context payloads drastically reduced input token overhead per prompt.
-
48ms Average Search Latency: Multi-index caching delivered real-time answers directly inside compliance advisor dashboards.