Disclosure: I work on Logster, a threat detection platform that uses a language model to read endpoint telemetry. So I have a stake in this. I’m writing it anyway because “just paste logs into an LLM” is the first idea everyone has, including us, and the reasons it doesn’t work are more interesting than the idea itself.
The pitch writes itself. Language models are good at reading text. Logs are text. Security analysts spend their lives reading logs. Point one at the other and the problem is solved.
Every security team with an engineer and a weekend has tried to paste logs into a model. It demos beautifully. You paste in a suspicious process chain, ask the model what it thinks, and it produces a genuinely insightful answer — better than a junior analyst’s, sometimes better than a senior one’s.
Then you try to paste logs into it from a real environment, and the whole thing falls apart on arithmetic.
The arithmetic
Take a mid-sized estate: 10,000 endpoints. Assume 100 process events per endpoint per day, which is conservative — a busy server does that before lunch. That’s a million events a day, and I’m deliberately ignoring file, network, registry, and authentication telemetry to keep the number friendly.
Now serialize one event. An enriched process creation record — image path, command line, parent image, parent command line, hashes, user, integrity level, working directory, timestamps — runs about 250 tokens as JSON. Call it 250 to stay conservative.
| Events per day | 1,000,000 |
| Tokens per event | ~250 |
| Tokens per day | 250,000,000 |
| Typical frontier context window | 1,000,000 |
| Largest advertised context window | 10,000,000 |
A one-million-token context window holds 0.4% of a single day of telemetry from one mid-sized estate. Put differently: fill it entirely with process events and you have captured roughly six minutes.
You can go bigger. Thirteen models now advertise a million tokens or more, and Llama 4 Scout advertises ten million. That buys you 4% of one day — about an hour. It is a tenfold improvement on a number that needed to improve by a factor of two hundred and fifty.
And it is a generous reading, because advertised context and usable context are not the same thing. Long-context evaluations consistently show retrieval accuracy falling off well before the stated limit — the failure mode now generally called context rot. The effective window is smaller than the number on the box, and the gap widens the more of it you fill.
The cost side is no kinder. At current frontier pricing of roughly $5 per million input tokens, reading that day once costs about $1,250. Call it $456,000 a year, for one estate, to read each event exactly once with no re-reading, no follow-up questions, and no output tokens. A cheaper model brings it down — a mid-tier model at $2 per million lands near $180,000 a year — but you haven’t changed the shape of the problem, only its slope.
And notice what that spend buys: a model that has seen each event once, in isolation, with no memory of last Tuesday.
The four fixes everyone tries
Once you accept that you cannot simply paste logs into a context window, the same four ideas come up, in roughly this order. Each one is reasonable. Each one fails for a specific reason worth understanding.
1. Just use a recent window
Feed the model the last N events. Slide the window forward. Simple, cheap, obvious.
This fails because intrusions don’t happen inside a window. Initial access, then a quiet period, then privilege escalation, then lateral movement, then the objective — that sequence routinely unfolds over days or weeks. The phishing email that started it is long outside any window you can afford to keep open.
A sliding window can tell you that a process spawned a shell. It cannot tell you that the process was installed by a service that was created by a session that authenticated with credentials stolen nineteen days ago. That chain is the detection. The window has already forgotten the part that mattered.
2. Sample
Take a representative fraction. Standard practice everywhere else in observability.
But sampling assumes the interesting signal is statistically represented in the population. In security it is the opposite: the events you need are, by definition, a vanishingly small fraction of the total. Sample at 1% and you have a 99% chance of discarding any given malicious event. You have built a system that works beautifully except on the days it matters.
3. Summarize first
Compress each hour into a paragraph. Paste logs into the summarizer first, and feed the model summaries instead of raw events.
The problem is that summarization is lossy in exactly the wrong direction. What survives summarization is what’s frequent and typical. What gets dropped is the singular and the odd — which is precisely the detection signal.
A summary that says “1,247 process creations, mostly routine software updates” has technically described the hour. It has also thrown away the one rundll32 invocation with an unusual command line, because one event out of 1,247 is noise to a summarizer and the whole story to an analyst.
4. Embed everything and retrieve
The sophisticated answer: vector-embed the logs, retrieve the semantically relevant ones at query time, feed those to the model. Standard RAG.
This is the one that seems like it should work, and it’s the most instructive failure.
Two things break it. First, log lines are near-duplicates of each other. Ten thousand process events from the same estate occupy a tiny, dense region of embedding space — they’re all “a process started on a Windows host.” Cosine similarity has almost nothing to discriminate on. Retrieval returns a thousand things that look equally relevant, which is the same as returning nothing.
Second, and more fundamentally: semantic similarity is not the relationship you need. The events that matter for a given process are not the events that resemble it. They’re its parent. Its parent’s parent. The file it wrote three hours ago. The connection its sibling opened. Those events look nothing alike. A credential-access event and a network connection have no semantic similarity worth speaking of — they’re related causally, and embeddings don’t encode causality.
RAG retrieves things that are about the same topic. Detection needs things that are connected to the same incident. Different question.
What relevance actually means here
Every attempt to paste logs into a model fails the same way wearing a different hat: each one selects context by the wrong criterion. Recency, frequency, or resemblance. None of those is what makes an event relevant to an investigation.
Relevance in security telemetry is structural. An event matters to another event because of a causal relationship between them — process ancestry, file handles, socket ownership, credential inheritance, session lineage. Those relationships form a graph. And a graph is a thing you can traverse deterministically, in code, before a model ever sees a token.
This reframes the whole problem. The question stops being how to paste logs into a context window at all, and becomes “which forty events, out of two hundred and fifty million, does the model need in order to answer this question.”
Forty events is about ten thousand tokens. That fits anywhere. It fits in a small model. It fits several times over with room for instructions, prior context, and the model’s own reasoning.
The context window was never the constraint. Selection was.
What context building actually involves
Calling it “selection” undersells it. In practice the work breaks into a few distinct jobs, none of which a language model does for you:
Maintaining the graph. Process identity that survives PID reuse. Lineage captured at exec time, because on Linux an orphaned process is reparented and its real ancestry is destroyed. Relationships that persist after the processes themselves have exited, since attack chains are full of dead intermediate nodes.
Deciding traversal depth. Every relationship you follow multiplies the context. Two hops from an interesting process might be forty events; four hops might be four thousand. Knowing when to stop is most of the engineering.
Including negative context. The model needs to know what’s normal for this host to judge what’s abnormal. That means deliberately including counter-examples — the fifty times this service started benignly — which is context that looks wasteful and isn’t.
Compressing the stable parts. Full detail on the events under suspicion, aggressive compression on the ambient activity that establishes the baseline. Uniform detail across the whole context is a waste of the budget.
None of this is machine learning. It’s data structures, graph traversal, and a set of judgment calls about what matters. It’s ordinary engineering — and it’s where the difficulty actually lives.
The model is the easy part
Here’s what I find genuinely surprising, having built this.
Give a current frontier model forty well-chosen events with the causal relationships made explicit, and it is very good at the security question. It identifies the technique. It explains why the chain is suspicious in terms an analyst can verify. It notices things a rule wouldn’t, because it’s reading the sequence as a narrative rather than pattern-matching each line.
That capability is real, and it’s roughly the same across the frontier models. It is not the differentiator, and anyone telling you their model is what makes their detection work is either not being straight with you or hasn’t hit the scaling problem yet.
Anyone can paste logs into a prompt. The differentiator is whether you can find those forty events. Everything upstream of the prompt — the collection model, the process identity, the graph, the traversal heuristics, the compression strategy — is where the actual work is, and it’s unglamorous in a way that doesn’t fit a press release.
Which is, I suspect, why so much of the “AI for security” conversation is about models, and so little of it is about context.
Logster is a threat detection platform for Windows and Linux. Rather than paste logs into a prompt and hope, it collects endpoint telemetry, builds a behavioural graph, and uses that graph to assemble the context a model reads. Technical detail is in the documentation.
Numbers above are illustrative and deliberately conservative — real estates generally produce more telemetry than the example, not less. If your figures differ substantially I’d be interested to hear it.