How Your Editor Reads Your Mind: ASTs, Code Splitting, and the Trick Behind AI Suggestions

You are typing a function. You get as far as function calculateTotal(items) { and pause. Before your fingers reach the next key, faded grey text appears, suggesting the entire body, and it is roughly what you were about to write. You press Tab. It was right.

That moment feels like the editor read your mind. I want to take that feeling apart, gently, until there is no magic left, only a couple of genuinely clever ideas that you will understand completely by the end of this. There are two of them. The first is how a computer turns your messy text into something it actually understands. The second is the surprisingly elegant trick that lets an AI fill a gap in the middle of your code. Let us build both from nothing.

Code is just text, until it becomes a tree

Here is the uncomfortable truth a beginner rarely hears: to a computer, your beautiful code is, at first, a meaningless string of characters. f, u, n, c, a space, and so on. It has no idea that function is special or that a curly brace opens a block. Before anything smart can happen, the computer has to find the structure hiding in that text. It does this in two steps.

The first step is called tokenization. The computer scans your text left to right and groups the characters into meaningful chunks called tokens, throwing away the noise like extra spaces and comments. The word function becomes one token. A number becomes a token. A + becomes a token. Take a tiny expression, 2 + (z - 1), and watch it break apart.

2 + ( z - 1 )

Tokenization: the raw text becomes a flat list of meaningful pieces. Each token knows what kind of thing it is (a number, an operator, a parenthesis). Whitespace and comments are dropped here, because they do not change the meaning.

A flat list of tokens is better than raw text, but it is still flat. It does not capture that the z - 1 part has to happen before the addition. So the second step, called parsing, arranges those tokens into a tree that captures the structure. This tree has a name you will hear constantly once you start noticing it: the Abstract Syntax Tree, or AST.

For 2 + (z - 1), the tree looks like this. The addition sits at the top. Its two branches are the number 2 and the whole subtraction. And here is the detail I love: the parentheses have vanished.

The AST for 2 + (z - 1). The shape itself encodes the order of operations, so the parentheses are no longer needed. The tree does not care how you spaced things or where you put brackets. It captures only what the code means. That is the whole point of the word "abstract."

This is genuinely one of the most important ideas in all of programming, so let me say it plainly. An AST is your code’s meaning, stripped of its appearance. Every node is either an operation (like add, or “call this function,” or “loop over this”) or a value (like a number, a variable, a piece of text). The connections between nodes say how they relate. Once your code is a tree, a computer can finally reason about it instead of just reading letters.

And this is not some niche AI thing. ASTs have quietly powered your tools for years. When your editor reformats your code on save, it parsed it into a tree and printed the tree back out neatly. When a linter warns you about an unused variable, it walked the tree and noticed a branch nothing points to. When you hit “rename symbol” and it correctly renames the right variable everywhere without touching a same-named one in a different scope, that is the tree knowing what belongs where. The AST is the unsung hero behind half the things you already take for granted.

Why AI tools cut your code along the branches

Now we can connect this to AI. A language model has a limit on how much it can read at once, so a big codebase has to be chopped into pieces before the model can work with it. The naive way is to cut every fifty lines, like slicing a loaf without looking. The problem is obvious the moment you picture it: you will slice straight through the middle of a function, leaving its head in one piece and its body in another. Now neither piece makes sense on its own.

Because the AST knows where a function actually starts and ends, the smarter tools cut along the branches of the tree instead. A function stays whole. A class stays whole. Each piece is a complete, meaningful unit. Watch the difference.

Cutting by line count

function getUser(id) {
const row = db.find(id)
✂ cut at line 50 return format(row)
}

Cutting by the tree

function getUser(id) {
const row = db.find(id)
return format(row)
}

Same function, two ways to chop it for an AI. On the left, a blind line cut splits the function in half and ruins both pieces. On the right, AST-aware chunking keeps the whole function together because the tree knows where it begins and ends. If you read my post on chunking, this is that idea applied to code specifically, and it is why structure-aware splitting beats blind splitting every time.

So when an AI assistant pulls in “relevant code” to help answer you, the good ones are pulling whole functions and classes that the AST identified, not random fifty-line windows. Better pieces in means better suggestions out.

The real magic: filling the gap from both sides

Here is the part that genuinely surprised me when I learned it, and it is the heart of why completions feel like mind reading.

A normal language model writes left to right. It reads what came before and predicts what comes next. That is fine when your cursor is at the end of a line. But think about what you actually do when coding: very often your cursor is in the middle of existing code. There is code above it and code below it. A plain left-to-right model would ignore everything below your cursor, which is half the clue about what belongs in the gap.

The fix has a wonderfully descriptive name: Fill-in-the-Middle. The editor takes the code before your cursor, the code after your cursor, and the empty gap between them, and it asks the model to fill that gap using both sides as context.

function calculateTotal(items) {

return items.reduce((a, b) => a + b.price, 0)

}

prefix (code above) + suffix (code below) → predict the middle

The accent text is what the model proposed for the gap. It could write it because it saw both the opening line above and the closing brace below, and reasoned about what fits between them. The editor knows the function is not finished, so it fills it in.

Now, the clever bit underneath. How do you make a left-to-right model fill a middle gap? You do not rebuild the model. You play a trick with the order. During training, code is split into three parts, prefix, middle, and suffix, and then reshuffled into the order prefix, then suffix, then middle. The model still does the only thing it knows how to do, predict the next token left to right, but because the middle now comes last, predicting “the next part” means predicting the gap, with both sides already in front of it.

It is a beautifully lazy idea. Rather than invent a new kind of model, you just rearrange the puzzle so that “fill the middle” becomes “continue from the end.” And it works: GitHub reported that adding this Fill-in-the-Middle ability lifted the rate of suggestions people actually accepted by around ten percent. A reordering trick, ten percent more useful completions.

How the editor actually builds the question

So your editor understands structure (AST) and knows how to fill a gap (FIM). The last piece is what it actually sends to the model each time, because it cannot send your whole project. It has a token budget, the same idea from my post on tokens, and it has to spend that budget wisely.

Behind the scenes there is a step often called the prompt assembler. Every time you pause typing, it quickly gathers the most useful context it can find, ranks it by importance, and packs as much as fits into the budget, dropping the least important things when space runs low.

highestthe code right around your cursor (your prefix and suffix)

highrelevant snippets from your other open files and nearby code

mediumimports and key definitions, so it knows what is available

mediumyour project's instruction file, if you have one

loweverything else, trimmed away first when the budget runs out

The prompt assembler, simplified. It is a ranking-and-trimming game played against a token budget, every single time you pause. This is also why what you have open and how you have named things matters so much: it directly shapes what the model gets to see.

Putting the whole thing together

Let me stitch it into one picture, the journey from your keystrokes to that grey suggestion.

you typeraw code text

→

tokenizesplit into tokens

→

parsebuild the AST

→

chunkwhole functions, by the tree

→

assembleprefix + suffix + context, FIM

→

suggestthe grey text you Tab

The full path. Structure understanding (tokenize, parse, chunk) feeds context assembly (FIM prompt within a token budget), which produces the completion. No mind reading. Just a parser, a tree, a reordering trick, and a careful budget.

Why knowing this makes you better at using it

Here is the practical payoff, because understanding the machine changes how you drive it.

Once you know the editor leans on the code around your cursor and in your open files, you start helping it on purpose. You open the file with the function you are about to call, so its shape is in the context. You give variables clear, honest names, because those names are tokens the model reads and reasons about. You write a clear comment or function signature before the body, because that becomes the prefix the suggestion is built from. You are no longer hoping the AI guesses right. You are feeding it the clues it actually uses.

That is the quiet shift from being surprised by your tools to being in command of them. The grey text was never magic. It was a tree and a trick and a budget, and now you know all three. The next time a completion lands perfectly, you will not just press Tab. You will know exactly why it knew.