Blog
Longer technical posts and deep dives into software engineering topics. For shorter notes, see my Today I Learned collection.
47 articles
The Streaming Backbone Under Generative AI
July 31, 2026
Everyone draws the GenAI diagram with the model in the middle and arrows pointing at it. Nobody draws the pipe those arrows actually are. That pipe is a stream: a durable, ordered, replayable log that sits between the things producing events and the LLMs, embedders, and agents consuming them. This is the plumbing under real-time RAG, live grounding, and agent-to-agent messaging, and it decides whether your system stays up when traffic spikes.
Tags: #ai #streaming #kafka #generative-ai #architecture #agents
Streaming LLM Output: Tokens and Tool Calls
July 31, 2026
An LLM writes its answer one token at a time either way. Streaming just shows you those tokens as they land instead of hiding them behind a blank screen. Same total time, wildly different feeling. Here’s how it works over the wire, and why streaming through an agent loop is the part that quietly breaks.
Tags: #ai #streaming #llm #sse #generative-ai #architecture
Streaming Data Into RAG: Keeping the Index Live
July 31, 2026
A normal RAG index is a photograph. You embed your documents once, and from that second on the index is frozen while the world keeps moving. That’s fine for a policy handbook and useless for anything that changes: tickets, prices, inventory, chat. This post is about closing that gap, wiring a stream of changes straight into your vector index so retrieval reflects now, not last night. AWS in the loop, with the GCP and Azure equivalents at the end.
Tags: #ai #rag #streaming #aws #real-time #architecture
Streaming Live Context Into LLMs and AI Agents
July 31, 2026
Most agents wait to be asked. A human types a question, the agent answers, and nothing happens until the next question. The more useful pattern is the opposite: the agent is asleep until something in the world happens, and the event itself wakes it up. This post is about event-driven agents, where the event is the perception, and how you keep the context window fed with what just happened without letting it overflow.
Tags: #ai #agents #streaming #event-driven #generative-ai #architecture
The Three AI Systems That Are Secretly the Same Design
July 24, 2026
RAG chatbots, agents, and recommendation engines look like three different problems. Build a few of them and you notice they’re the same machine wearing different clothes: cast a cheap wide net, spend real compute narrowing it, then apply the rules a model won’t learn on its own. Here’s the shared shape, why it keeps showing up, and what it tells you about where these systems actually break.
Tags: #ai #system-design #rag #agents #recommendations #architecture
Designing an Agent That Doesn’t Go Off the Rails
July 24, 2026
Some tasks can’t be one prompt: investigate a failing customer, pull their tickets, check billing, guess the cause, draft a reply. That needs planning, several tool calls, and reacting to what comes back. This is a full walk through how I’d design an agentic system end to end, the loop, the tools, the memory, the guardrails, and the cloud services that fill each box on AWS, GCP, and Azure, with the honest reasons you should reach for an agent as late as you possibly can.
Tags: #ai #agents #orchestration #system-design #langgraph #architecture
Designing a Recommendation System: The Funnel
July 24, 2026
You open an app and the ‘up next’ somehow nails it, out of millions of things, in the time it took the page to paint. That’s not one clever model. It’s a funnel: narrow millions to a handful in stages, spending more compute the fewer items you have left. Here’s the whole thing end to end, the two-tower trick at its core, and the cloud services that fill each box on AWS, GCP, and Azure.
Tags: #ai #recommendations #ranking #system-design #embeddings #architecture
Designing a RAG System That Actually Retrieves
July 24, 2026
Everyone can bolt a vector database onto an LLM and call it RAG. The hard part is the retrieve step actually finding the right paragraph, out of thousands of internal docs that changed this morning, with the user’s permissions respected and a citation on the answer. This is the full design, box by box, plus the cloud services that fill each box on AWS, GCP, and Azure.
Tags: #ai #rag #system-design #retrieval #embeddings #architecture
Use Case: When AI Reads Your Forms and Quietly Gets the Numbers Wrong
July 19, 2026
A client wants AI to pull the right numbers out of their forms: insurance claims, bank applications, statements. Point normal RAG at them and it starts getting values wrong, the policy number becomes the account number, one amount becomes another. Here’s how I’d actually build it, why the naive approach fails on forms, the approaches that fix it, and the honest side effect and accuracy cost of each.
Tags: #ai #rag #chunking #forms #use-case #explainer
Open Weights: What It Really Means (and Why It’s Not Open Source)
July 19, 2026
You keep hearing that models like GLM, DeepSeek, and Llama are ‘open source’ AI. Most of them aren’t, not really. They’re open weight, which is a different, more limited thing that people constantly mix up. Here’s what open weights actually means, how it differs from open source and closed models, in plain language, with a simple analogy anyone can follow.
Tags: #ai #open-weights #open-source #models #explainer
Designing a Support Chatbot That Doesn’t Make Things Up
July 18, 2026
A company hands you a pile of help docs and asks for a chatbot that answers customer questions from them, accurately, with sources, and without inventing policies that don’t exist. It sounds simple. It is not. This is a full walk through how I’d design it: every approach with its honest pros and cons, and, at the end, how DoorDash and LinkedIn actually built theirs, with links to their own write-ups.
Tags: #ai #rag #chatbot #architecture #use-case #explainer
LangGraph: When Your AI Needs a Flowchart That Actually Runs
July 17, 2026
A straight-line pipeline is fine until your AI needs to loop, branch, pause for a human, or survive a crash halfway through. That’s the wall LangChain’s simple agent hits, and exactly where LangGraph begins. This is what LangGraph is, why it uses graphs, its pieces, real use cases, the honest trade-offs, and how it differs from LangChain.
Tags: #ai #langgraph #langchain #agents #orchestration #explainer
LangChain: The Toolkit for Building With AI, and When to Skip It
July 17, 2026
If you want to build a real app around an AI model, you quickly hit a wall of plumbing: connecting the model, giving it memory, letting it use tools, chaining steps together. LangChain is the toolkit that handles that plumbing. Here is what it actually is, its pieces, real things people build with it, the honest pros and cons, and the times you’re better off without it.
Tags: #ai #langchain #frameworks #agents #tools #explainer
Building an Agent to Sift Years of Messy Data Into a Plan
July 17, 2026
A client hands you years of advertising data. Most of it is noise. You need a tool that reads all of it, keeps only what matters, and hands back a real plan. This is a walk through how I’d actually build that: the three approaches you’d weigh, why the obvious one is a trap, and the design I’d ship, with the honest trade-offs.
Tags: #ai #agents #rag #architecture #use-case #explainer
What Is an Agent, Really?
July 14, 2026
Everyone throws the word around now. Agents, subagents, multi-agent systems, spawning agents. But strip away the buzz and ask the simple question: what actually makes something an agent, and not just a chatbot with extra steps? Here is the clear answer, the one idea that separates an agent from everything else, and the vocabulary people use loosely, explained properly.
Tags: #ai #agents #concepts #autonomy #llm #explainer
Understanding the Agent Harness, From Scratch
July 13, 2026
A language model can only do one thing: read text and write text back. It can’t open a file, run your code, or check a result. So how do AI agents actually do real work? The answer is a piece of software called the harness. This is a from-zero walkthrough of exactly what it is and how it works, built for anyone with just basic AI knowledge.
Tags: #ai #harness #agents #beginner #llm #explainer
OpenEnv: The Gym Where AI Agents Learn to Act
July 11, 2026
You can’t teach an agent to do a job by lecturing it. It has to try, fail, get feedback, and try again, thousands of times, safely. That safe practice space is an environment, and OpenEnv is the new shared standard for building them. Here is what it is, how the reset-step-loop works, and why Meta, Hugging Face, Nvidia and others all lined up behind it.
Tags: #ai #openenv #reinforcement-learning #agents #rl #explainer
How a Giant AI Model Fits on Your Laptop
July 09, 2026
A serious AI model is billions of numbers, tens of gigabytes at full size. Your laptop has nowhere near the room. Yet it runs, smoothly. The trick is quantization: storing each number in fewer bits, like saving a photo as a JPEG instead of a giant RAW file. Here is exactly how it works, why it barely hurts quality, and where it finally breaks.
Tags: #ai #quantization #local-llm #gguf #llm #explainer
Why AI Hallucinates, and How We Actually Stop It
July 08, 2026
An AI will look you in the eye and invent a fact, a citation, a court case that never existed, with total confidence. It’s the single biggest reason people don’t trust AI. But in 2025 researchers finally explained why it happens (it’s not a bug, it’s an incentive), and the industry built a real toolkit to fight it. This is the complete picture: what a hallucination is, why it happens, and exactly how the best companies stop it.
Tags: #ai #hallucination #rag #reliability #llm #explainer
Fine-tuning vs RAG vs Prompting: How to Actually Make an AI Model Yours
July 08, 2026
You want the AI to know your company’s data, or sound like your brand, or stop getting one thing wrong. There are three ways to do it, and people constantly pick the wrong one, spending months fine-tuning when a good prompt would have done, or prompting forever at a problem only retrieval can fix. Here is the one mental model that tells you which lever to pull, and when.
Tags: #ai #rag #fine-tuning #prompting #llm #explainer
Prompting Methods: The Different Ways We Talk to AI
July 07, 2026
The exact same model can give you a mediocre answer or a brilliant one, depending only on how you asked. Over the last few years people worked out a real toolkit of prompting methods: zero-shot, few-shot, chain-of-thought, self-consistency, ReAct, and more. Here is what each one is, when to reach for it, and the honest 2026 twist: why reasoning models quietly changed the rules.
Tags: #ai #prompting #prompt-engineering #chain-of-thought #llm #explainer
The Harness: Why the Model Is the Smallest Part of Your AI Agent
July 06, 2026
Everyone obsesses over which model an agent uses. But teams keep making agents dramatically better without touching the model at all. One team deleted 80% of their agent’s tools and success climbed from 80% to 100%. The thing they changed has a name: the harness. Here is what it is, why it suddenly matters more than the model, and how to use it to your advantage.
Tags: #ai #agents #harness #harness-engineering #tool-use #explainer
Ollama: Running Real AI on Your Own Machine
July 06, 2026
One command, and a capable language model is running on your laptop. No API key, no cloud, no per-token bill, no data leaving your machine. Ollama made local AI feel as easy as docker run. Here is what it actually is, how it shrinks a giant model to fit your hardware, the commands that matter, who really uses it, and where it stops making sense.
Tags: #ai #ollama #local-llm #llama-cpp #quantization #explainer
Pi: The Coding Agent That Wins by Doing Less
July 05, 2026
Every coding agent is racing to add features: more tools, more modes, more built-in everything. Pi went the other way. Four tools. The shortest system prompt in the business. No MCP, no plugins to hunt for. And if it can’t do something? You ask it to build the ability itself. Here is why that radical minimalism works, and why it quietly became the engine inside OpenClaw.
Tags: #ai #agents #pi #coding-agents #open-source #explainer
OpenClaw: The AI That Lives in Your Chat Apps
July 05, 2026
Most AI assistants make you come to them, a website, a tab, a login. OpenClaw flips it: the agent comes to you, on WhatsApp, Telegram, Slack, wherever you already talk, and it can actually do things: browse, run commands, test your app, send the report. Here is what it is, how the architecture works, and what people are really building with it, from developers to QA to your family group chat.
Tags: #ai #agents #openclaw #automation #open-source #explainer
How AI Agents Actually Work
July 04, 2026
A chatbot answers and stops. An agent keeps going: it thinks, does something, looks at what happened, and decides what to do next, over and over, until the job is done. That single loop is the whole idea. Here is how it works, where it came from (a 2022 paper called ReAct), and why the hardest part isn’t making it smart. It’s making it stop.
Tags: #ai #agents #react #agent-loop #tool-use #explainer
Hermes: The AI Agent That Learns From Itself
July 04, 2026
Most AI agents wake up every morning with amnesia. You teach them something Monday, and Monday’s lesson is gone by Tuesday. Hermes, from Nous Research, made forgetting optional: it writes its own skills from experience and gets better at what you actually do. Here is what it is, why people are talking about it, how it stacks up against OpenClaw, and where it genuinely falls short.
Tags: #ai #agents #hermes #nous-research #open-source #explainer
Context Engineering: Why AI Forgets, and How We Fight It
July 04, 2026
Every AI model has a working memory with a hard edge. Fill it and the oldest things fall off, and even before that, it quietly stops paying attention to the middle. Context engineering is the craft of deciding what the model gets to see on every single call. Here is why forgetting happens, the research that proved it, and the real techniques teams use to fight it.
Tags: #ai #context-engineering #llm #memory #agents #explainer
Embeddings: How AI Knows Two Things Mean the Same
July 03, 2026
Type ‘my laptop keeps freezing’ and get back a doc titled ‘system hangs on boot.’ Not one word matches, yet it’s the right answer. That quiet trick is embeddings: turning meaning into numbers you can measure the distance between. Here is how it actually works, from the first intuition to the parts senior engineers still find beautiful.
Tags: #ai #embeddings #vectors #semantic-search #rag #explainer
MCP: The Port That Let AI Finally Touch the World
June 30, 2026
For a while, the smartest models on earth were trapped behind glass. They could think, but they couldn’t reach. MCP is the standard plug that changed that. Here is what it actually is, why it isn’t just another REST API, the three things a server can offer, and how you build one yourself.
Tags: #ai #mcp #agents #protocols #developer-tools #explainer
Skills: The Quiet Standard Every AI Coding Tool Agreed On
June 28, 2026
A folder, a Markdown file, two lines of frontmatter. That is a Skill. Somehow it became the one thing Claude, Cursor, Bob, Codex, and a dozen other tools all read the same way. Here is how Skills actually work, why the design is clever, and where they earn their keep.
Tags: #ai #skills #agents #developer-tools #explainer
Open the Hood: Why Almost Every AI Code Editor Is Secretly the Same Editor
June 26, 2026
Cursor, Windsurf, Antigravity, IBM Bob. They look like rivals from different worlds. Pop the hood and you find the same engine in nearly all of them. Once you see it, the whole confusing landscape suddenly makes sense.
Tags: #ai #code-editors #cursor #windsurf #developer-tools #explainer
Roo Code: The Tool That Taught AI to Wear Different Hats
June 26, 2026
Roo Code hit three million installs, gave the AI in your editor a clever idea called modes, then bet its whole future against the very thing that made it famous. It is gone now, but the lessons inside it are the foundation of how agentic coding works today.
Tags: #ai #roo-code #cline #coding-agents #developer-tools #explainer
How Your Editor Reads Your Mind: ASTs, Code Splitting, and the Trick Behind AI Suggestions
June 24, 2026
Your editor finishes your code before you do. It feels like mind reading. It is not. It is two clever ideas working together: turning code into a tree, and learning to fill a gap from both sides. Here is the whole thing, from scratch.
Tags: #ai #ast #code-completion #developer-tools #copilot #explainer
From Vibe Coding to Spec Kit: How to Tell an AI What to Build (and Get It Right)
June 23, 2026
Letting an AI just wing it feels amazing for about three days. Then it starts guessing, and the guesses pile up. Spec-driven development, and GitHub’s Spec Kit, is the calm, structured way out. Here is how it works, explained from scratch.
Tags: #spec-kit #spec-driven-development #ai #developer-tools #vibe-coding #explainer
AGENTS.md vs Skills vs Plain Scripts: What Goes Where, and Why It Matters
June 22, 2026
Three ways to teach an AI agent how to work in your project, and they are not interchangeable. Get the wrong one in the wrong place and your agent quietly gets worse, not better. Here is how to think about it.
Tags: #ai #agents #agents-md #skills #developer-tools #explainer
What Tokens Actually Are (And Why They Cost You)
June 21, 2026
Tokens are the hidden unit that every AI bill, every context limit, and every weird counting mistake comes down to. Here is what they really are, why providers count them differently, and why a model cannot tell you how many r’s are in strawberry.
Tags: #llm #tokens #tokenization #ai #explainer
RAG, Part 3: Making It Actually Good (and Why It Still Fails)
June 21, 2026
A basic RAG system retrieves by meaning and calls it a day. The good ones do more: they search by keyword and meaning together, re-judge the results, and respect the strange ways a model loses focus. This is the part that separates a demo from something you would trust.
Tags: #rag #retrieval #reranking #hybrid-search #ai #llm #explainer
RAG, Part 2: Chunking, the Decision That Quietly Decides Everything
June 21, 2026
Before a single answer is retrieved, you have to cut your documents into pieces. Cut them badly and nothing else can save you. Here is every chunking strategy, shown on real text, with honest numbers on what actually works.
Tags: #rag #chunking #retrieval #ai #llm #explainer
RAG, Part 1: What It Really Is (and the Problem It Solves)
June 21, 2026
A language model that has read the whole internet still cannot tell you what is in your own company handbook. RAG is how we fix that. This is the foundation: the problem, the idea, and how a machine decides what counts as relevant.
Tags: #rag #retrieval #embeddings #ai #llm #explainer
How LLMs Actually Work
June 20, 2026
No math, no jargon walls. Just a plain, honest walk through what is really happening when you type a question into ChatGPT and words come back.
Tags: #llm #ai #machine-learning #explainer #beginners
When Your Database Goes Down at 3 AM: A MongoDB War Story
December 28, 2025
How a full /var/log partition crashed MongoDB in production, what I learned about truncate vs rm, and the permanent fix using LVM.
Tags: #mongodb #linux #troubleshooting #devops #incident-response #lvm #xfs
Building a Smart Caching System That Cut AI API Costs by 85%
October 28, 2025
How I built an in-memory cache for llmswap that reduced repeat API calls from 3 seconds to 0.66 seconds using thread-safe LRU eviction.
Tags: #python #caching #performance #llm #api #threading #memory-management #optimization
Why Our Users Were Complaining About Slow Search (And How 4 Lines Fixed It)
October 05, 2025
How MongoDB indexes transformed our infrastructure search from 15 seconds to under 100ms, fixing customer complaints overnight.
Tags: #mongodb #performance #indexing #database #infrastructure #optimization
What I Learned Building a CLI That People Actually Use
September 15, 2025
Lessons from building llmswap’s command-line interface - from argparse chaos to a tool with 30,000+ downloads.
Tags: #python #cli #argparse #ux #developer-tools
I Finally Stopped Alt-Tabbing to ChatGPT While Coding
September 04, 2025
How llmswap 4.1.1’s generate command changed my entire development workflow - AI assistance right in the terminal.
Tags: #ai #llm #developer-productivity #coding #cli-tools #openai #anthropic #workflow #terminal #llmswap #python #automation #infrastructure #devops #machine-learning #artificial-intelligence #software-engineering #programming #coding-tools #vim
From Hackathon to Open Source: Why I Extracted My LLM Switching Logic
August 30, 2025
How a hackathon project that didn’t win became a PyPI package with 30,000+ downloads.
Tags: #hackathon #open-source #ai #llm #python #openai #anthropic #gemini #watsonx #ollama #pypi #developer-tools #infrastructure #rag #mcp #cli-tools #machine-learning #artificial-intelligence #software-engineering #devops #cloud-computing #openstack #programming #coding #tech-story #startup #innovation #python-package #api #chatbot #automation