TLA+ and PlusCal from First Principles
Many programmers learn TLA+ through PlusCal, with a model checker that keeps telling them they’re wrong. The syntax is never the hard part. What takes time is the mean...
Latest posts
Page 1 of 11
Low-Level Design: Class Relationships and Design Patterns in Go and Python
Low-level design rests on two skills: telling how two classes are related, and recognizing design patterns, which are those same relationships arranged on purpose to s...
Inference Engineering - Part 6: Diffusion, and What Ties It Together
Every earlier part of this series assumed autoregressive generation: the model writes one token at a time, left to right, and each token depends on all the ones before...
Inference Engineering - Part 5: Sampling, From the API Down to the Bits
Three parameters sit between a trained model and its output: temperature, top_k and top_p. They are widely used and rarely defined precisely. All three act at the same...
Inference Engineering - Part 4: It's Memory Bandwidth All the Way Down
The idea here is simple, but memory bandwidth comes up so often in inference engineering that it deserves a post of its own. It rests on the split Part 2 drew between ...
Inference Engineering - Part 3: Where the Representation Lives
Many times, you would come across the claim that “the encoder builds an internal representation of the input.” However, most modern LLMs are decoder-only. So do they j...
Inference Engineering - Part 2: What Actually Runs the Model
Part 1 ended with a few hundred gigabytes of weights and an architecture description. This post is about what happens to them next, and it is where the series title st...
PostgreSQL - Making Double Bookings Impossible
Every hotel reservation system starts with the same line of code: SELECT 1 FROM reservation WHERE room_id = $1 AND check_out > $2 AND check_in < $3; -- no rows?...
Inference Engineering - Part 1: How a Model Gets Made
I kept running into large language model (LLM) systems and finding that I could use them without understanding them. That is an uncomfortable place to be: you end up a...
Production RAG - Part 9: Splitting an Answer into Claims
Part 7 argued that a groundedness gate has to score atomic claims and take the minimum, because a five-sentence answer with one fabrication averages out to a passing s...
