We've all built toy RAG demos that work perfectly on a single clean text file. But when you deploy RAG into production to parse actual client documents, the ingestion pipeline breaks first.
The Reality of Document Ingestion
In our client projects, we frequently encounter scanned PDF files, multi-column reports, and embedded database tables. Standard character-based chunking schemes shred these tables, destroying the relationship between columns. To fix this, we utilize unstructured layout detection libraries to partition documents into semantic tables and narrative blocks before converting them into vectors.
// Python example: Preprocessing tables before chunking
from unstructured.partition.pdf import partition_pdf
elements = partition_pdf(
filename="client_financials.pdf",
strategy="hi_res",
infer_table_structure=True,
chunking_strategy="by_title",
max_characters=1000,
new_after_n_chars=800
)
# This preserves table structures instead of shredding them into noise.Hybrid Search and Cross-Encoder Reranking
Vanilla vector search is a poor match for exact keyword lookups and complex user intents. To achieve production accuracy, we implement a hybrid approach: BM25 sparse search for keyword matching, paired with dense vector search (using pgvector) for semantic context. Finally, we run the results through a Cross-Encoder Rerank model (such as BGE Reranker) to evaluate the top-k retrieved chunks before sending them to the LLM.
- Query Expansion: Generating synonyms and sub-queries to capture search intent
- Hybrid Retrieval: Merging sparse and dense search indices with reciprocal rank fusion
- Reranking: Recalculating query-to-chunk relevance using a deep transformer model
Continuous Evaluation Loops
User needs evolve, and documentation changes. We monitor context recall, answer relevance, and faithfulness by integrating evaluation tools (such as Ragas, open-sourced by Exploding Gradients) into our deployment workflows.
Frequently Asked Questions
How do you handle tables in RAG?
We parse tables as raw HTML/markdown, summarize the structural data using an LLM, index the summary in the vector database, and return the original HTML table context to the LLM during generation.