Streaming Live Context Into LLMs and AI Agents
It’s 3am. A latency alarm on the checkout service crosses its threshold and fires. Nobody’s awake. By the time the on-call human blinks at their phone, there’s already a thread waiting for them: the error rate over the last twenty minutes, the deploy that went out at 2:47, the three services downstream that started timing out right after, and a one-line hypothesis about which change is the likely culprit. A human didn’t assemble that. An agent did, and it started the moment the alarm fired, because the alarm was the thing that woke it up.
That’s the pattern this post is about. Most agents you’ve seen are built the other way round: a person asks, the agent answers, and between questions the agent doesn’t exist. That’s fine for a chatbot. It’s the wrong shape for anything that’s supposed to notice things.
Waiting to be asked vs waking up
Here’s the split, and it’s more fundamental than it looks.
Request-response (the default)
The agent sits idle until a human types a question. Or it polls: it wakes on a timer, checks "anything new?", finds nothing, sleeps, checks again. Most of its life is spent asking a question nobody answered yet.
Event-driven (the useful one)
The agent is asleep. Something happens in the world, a ticket, an alert, a webhook, a new file, and that event is delivered to the agent, which wakes, reacts, and sleeps again. No polling. The event is the trigger.
Polling is what you write when you don’t have events. You loop, you check, you sleep, you check again, and you pay for every check that found nothing. Webhooks were the first fix (let the source call you), but a raw webhook still lands in your app and forces you to route it. The clean version is an event backbone: sources publish events, a bus fans them out, and an agent is one of the subscribers that gets woken. AWS’s own guidance for serverless agents puts this bluntly, event-driven architecture is the backbone, and the events are the agent’s observations.
That word, observations, is the whole idea. If you’ve read what an agent really is, you know the loop: perceive, reason, act. Textbook diagrams draw “perceive” as the agent going out and looking at the world. Event-driven flips it. The world comes to the agent. The event is the perception layer.
The architecture, mapped to the loop
Draw it out and the shape is simple: a source emits an event, a bus carries it, and that delivery is the agent’s perceive step. Everything after is the same reason-and-act loop you’d build for a chatbot.
The sources are whatever your world produces: a webhook from GitHub or Stripe, a message on a queue, an object landing in S3, a row changing in a database, an alert from your monitoring stack. The bus is the load-bearing middle: EventBridge on AWS, Pub/Sub on Google Cloud, Event Grid on Azure. It’s durable (the event survives if the agent is down), it fans out (multiple agents can subscribe to the same event), and it routes (rules decide which events reach which agent). AWS Lambda’s docs split event sources into push (the source invokes you, like EventBridge or SNS) and pull (the runtime polls a queue like SQS or a Kinesis stream on your behalf), and either way the point is the same: you stop writing the polling loop yourself.
Then the agent wakes. On the clouds this is a serverless runtime: an Azure Functions trigger that starts an agent on any event, a Cloud Run service fired per Pub/Sub message through Eventarc, a Lambda invoked by EventBridge. Scale to zero when nothing’s happening, spin up when an event lands. That economic shape (pay only when the world does something) is exactly why serverless and event-driven agents fit together so well.
Deterministic or LLM-native? Pick per step
Once the agent is awake, “reason and act” isn’t one choice. There are two orchestration styles, and the mistake is treating it as either-or.
A deterministic orchestrator (Step Functions on AWS, Workflows on Google Cloud, Logic Apps on Azure) runs a fixed graph: do this, then this, branch here. You know the steps in advance, so you hard-code them. An LLM-native agent (Bedrock AgentCore, Vertex agents, Azure AI Foundry agents) hands the model a set of tools and lets it decide the order at runtime. AWS’s serverless guidance describes both, and both get triggered the same way, by an event off the bus.
The useful framing is per step, not per system. Deterministic where the path is known and you want it auditable and cheap. LLM-native where the input is messy and the right move genuinely depends on what the event says. A triage agent might use a deterministic wrapper to fetch a fixed set of signals, then hand those to an LLM step to actually reason about them. Same idea as picking the right tool inside an agent that doesn’t go off the rails: give the model latitude where judgment is needed, and none where it isn’t.
A worked example: the alert that triages itself
Back to 3am. Here’s the sequence, which is roughly the shape of PagerDuty’s SRE agent, an alert event triggers an agent that gathers context and posts a timeline, described generically.
Latency on checkout crosses threshold. The monitoring system publishes an incident event to the bus. No human involved yet.
A routing rule matches the event and invokes the triage agent, handing it the alert payload as its perception.
It calls tools: recent logs, the error-rate metric, deploy history, related past incidents. This is the act phase feeding the reason phase.
The LLM correlates: error rate jumped right after the 2:47 deploy, three downstream services timed out in sequence. It forms a hypothesis.
Into the incident channel: what changed, when, the likely cause, and what it already checked. A starting point, not a verdict.
Instead of a blank alert, the responder opens a thread that's already done the boring first ten minutes of investigation.
The thing to notice: “investigate this alert” never waited for a human to ask. In a request-response world, someone gets paged, reads the alert, opens a chat window, pastes it in, and only then does the agent start. Event-driven collapses all of that. The alert is the request. It’s the same reason GitHub’s Copilot coding agent can be kicked off by assigning it an issue or applying a label, the webhook for that label is the “go do this” signal. No human types “please start.”
Keeping the window fed with what just happened
Here’s where event-driven agents get genuinely hard, and it’s a generative-AI problem, not a plumbing one.
The agent reasons over its context window, and the window is bounded. As events stream in (more alerts, more log lines, more tool results from the last three checks), they pile into the context. Left alone, that context overflows, and a full window is a slow, expensive, confused window. So you can’t just keep appending “what just happened” forever. You have to decide, continuously, what stays in the window and what gets thrown out.
Anthropic’s context management work names both halves of this. Context editing automatically clears stale tool results as you approach the limit, so old, already-digested observations stop taking up room. The memory tool lets the agent write durable facts to a store outside the window and read them back on demand. Put together: the window holds what’s happening now, and the things worth remembering get pushed to memory so they survive the eviction.
The real design question isn’t “how do I stream events in.” It’s what belongs in-window versus what gets promoted to memory. Keep too much in the window and you overflow and pay for it. Promote too aggressively and the model loses the thread of what’s happening right now. There’s no universal answer, it’s the same context-engineering judgment call from why AI forgets, just under a live stream instead of a static prompt.
And the payoff for getting it right is real-time grounding: the agent answers from the live state of the world, not a snapshot baked in at startup. If you’ve read the companion post on streaming into RAG, it’s the same instinct from the retrieval side, don’t let the model reason over a photograph of a world that’s already moved on.
Agents that talk through events, not HTTP
One more shift worth flagging. Once you’ve got an event backbone, multi-agent systems change shape. Instead of agent A calling agent B over HTTP and waiting, agent A publishes an event (“triage complete, cause identified”) and whichever agents care about that (the remediation agent, the incident-comms agent) are subscribed and wake up on it. Publish and subscribe instead of point-to-point calls. It’s looser, it’s more durable (the event survives if a consumer is briefly down), and it’s how streaming-agent platforms like Confluent’s frame agents that run directly on live event streams. Worth noting the vendor tilt there: Confluent sells Kafka, so of course the answer is a stream. The underlying idea holds regardless.
The same shape on the three clouds
Every cloud has a full stack for this. Different names, identical roles.
| Role | AWS | GCP | Azure |
|---|---|---|---|
| Event source | S3 / webhooks / CloudWatch alarms | Cloud Storage / webhooks / Cloud Monitoring | Blob / webhooks / Monitor alerts |
| Bus / trigger | EventBridge / SNS / SQS | Pub/Sub / Eventarc | Event Grid / Service Bus |
| Agent runtime | Lambda / Bedrock AgentCore | Cloud Run / Vertex agents | Azure Functions / Foundry agents |
| Short-term memory | In-window + DynamoDB / ElastiCache | In-window + Firestore / Memorystore | In-window + Cosmos DB / Redis |
| Long-term memory | AgentCore Memory / vector store | Vertex Memory Bank / vector store | Foundry memory / AI Search |
The gotchas nobody warns you about
Event-driven agents fail in ways request-response agents never do, because now the input arrives on its own schedule, possibly twice, possibly in the wrong order, possibly a thousand at once.
Out-of-order events. The event bus doesn't promise the order things happened is the order you receive them. An "incident resolved" can land before the "incident opened" it resolves. Carry a timestamp or sequence number on every event and let the agent reason about ordering itself, rather than trusting arrival order.
Duplicate events. Most buses deliver at-least-once, which is a polite way of saying "sometimes twice." If the same alert fires your agent twice, you get two triage threads, or worse, two remediation actions. Make the agent idempotent: dedupe on an event ID so a repeat delivery is a no-op, not a second run.
Context overflow. A long-running incident streams in more events than the window can hold. If you never evict, you overflow and the run degrades. Lean on context editing to clear stale tool results and a memory store to hold the durable facts, so the window stays about *now*.
Event storms. One outage can fire a thousand alerts in a minute, and a naive setup wakes a thousand agent runs, each burning tokens on the same incident. Debounce and rate-limit at the bus: collapse related events, batch them into one invocation, and cap how often the agent can be woken for the same source.
That last one bites hardest. The whole appeal of event-driven is that events trigger the agent automatically, which is exactly why an event storm is dangerous: automatic triggering with no throttle is a token bonfire waiting for a bad night. Debounce before you deploy, not after the bill.
The takeaway
The default agent waits to be asked. It’s a very good chatbot and a very bad colleague, because a colleague notices things without being prompted. Event-driven flips the polarity: the agent sleeps until the world does something, and the event itself is the perception that starts the perceive-reason-act loop. “Investigate this alert” stops being a sentence a tired human types at 3am and becomes the alert firing, directly.
The plumbing is the easy half, sources, a bus, a serverless runtime, and every cloud sells the whole stack. The hard half is generative-AI-shaped: keeping the context window fed with what just happened without drowning it, deciding what stays in-window and what gets promoted to memory, and surviving duplicate, out-of-order, storm-shaped events without doing the wrong thing twice. Get those right and you’ve got an agent that’s already working by the time you wake up.
References
I built the argument and the diagrams myself. The patterns aren’t mine to claim, and the docs below are the real sources if you’re implementing this. Nothing here is copied from them.
This is one of a set of streaming posts. See the streaming backbone under generative AI, streaming LLM output tokens and tool calls, and the companion streaming data into RAG. For the loop mechanics under all of it, LangGraph, and for the wider design series, three AI systems, same design.