A chunk is the smallest thing retrieval can return. You never get half a chunk, and never a chunk plus a bit of its neighbour. Whatever boundaries you draw at ingestion time are permanent — they de...

A PDF is not a document in any structured sense. Rooted in PostScript, it is a list of drawing instructions: place this glyph at these coordinates, fill this rectangle, draw this image. There is no...

Retrieval-augmented generation looks deceptively simple. Split the documents, embed the pieces, search for the closest ones, paste them into a prompt. A working prototype takes an afternoon. What t...

The Log-Structured Merge Tree, also known as an LSM tree, is one of the most popular storage engines, focusing on performant write operations. The benefit over the B+ Tree with this mechanism is th...

The Byzantine consensus problem is an interesting problem in distributed systems, even though it dates back to medieval times. The idea originates in medieval battle strategy, where all loyal gener...

The transaction isolation level is one of the vital aspects to understand how transactions influence each other when run concurrently. Isolation level establishes a balance between consistency and ...

Distributed systems consist of a set of loosely coupled services that end users perceive as a single coherent system. However, due to the loose coupling between components and communication over th...

Concurrency and Paralleism - these terms sound similar; however, from a software engineer’s perspective, they denote two different concepts. Concurrency is the ability of a system to perform multi...

In this wiki, we will explore an approach to designing a notification service. Part of the System Design Case Studies series (7 parts) System Design - Rate Limiter Introduction to Consistent Ha...

In this wiki, we will explore an approach to designing a Web-crawling service. Part of the System Design Case Studies series (7 parts) System Design - Rate Limiter Introduction to Consistent Ha...