Postgres Internals I
This document discusses a few high-level concepts of PostgreSQL’s internals. Even though these are not needed in day-to-day activities, knowing them would certainly help understand queries and ...
Gems of Coding
System Design - Rate Limiter
Rate limiter is a vital component in large-scale web applications, as it controls traffic to the application servers to ensure a smooth experience for the majority of the users. It has become a cri...
DDIA - Chap01 - Reliable, Scalable and Maintainable Applications
A majority portion of web applications are largely data-intensive, and not compute-intensive. The speed of fetching the overall data to process the requests is generally the limiting factor as is a...
RAFT Consensus Algorithm
One of the foundation problems of designing a distributed system is how to store some data in a distributed system. One basic approach is to share or duplicate all the data between the nodes. Howev...
Production RAG - Part 8: Evaluation and Operations
Seven posts of machinery, and none of it is trustworthy until you can answer one question: how do you know it works? RAG systems are unusually hostile to intuition here, because a broken one keeps ...
Production RAG - Part 7: Grounded Generation
Retrieval can succeed completely and the answer can still be wrong. The model receives five relevant chunks and asserts a sixth thing that none of them support — a plausible number, a policy that d...
Production RAG - Part 6: Entities and Knowledge Graphs
At some point in every RAG project, a category of question stops working and no amount of tuning fixes it. Better chunking does not help. A stronger reranker does not help. The reason is that two c...
Production RAG - Part 5: Images and Multimodal Retrieval
Text retrieval works because both sides of the comparison are text living in one embedding space. An image breaks that symmetry, and the way you restore it is one of the few RAG decisions that is g...
Production RAG - Part 4: Hybrid Search, Fusion and Reranking
Ask a RAG system “how do I shut down an instance” and keyword search returns nothing useful, because the documentation says “halt” and “stop”. Ask it about CEAMVCSRE3110002C and semantic search dri...
Production RAG - Part 3: Chunking for Retrieval
A chunk is the smallest thing retrieval can return. You never get half a chunk, and never a chunk plus a bit of its neighbour. Whatever boundaries you draw at ingestion time are permanent — they de...