Open-weight models cut CTI agent costs to under a tenth if given more turns

For a small security team this means you can run and audit an agent yourself and reduce model spend by over 90%, but expect to allow longer agent sessions and tune message limits.

What actually changed

Researchers ran CTI-REALM, a live threat-intelligence benchmark where an agent turns a report into a working detection by querying telemetry, under a single harness, scorer and panel of LLM judges. They evaluated 26 models: 19 open-weight and 7 closed-frontier models. The central result is that an open-weight model can match the strongest closed models "at less than a tenth of their cost" when it is allowed enough turns. The study also found that the benchmark's published agent-message limit distorted results: tasks that hit the limit scored zero at every checkpoint, so the limit partly measured which models finished first rather than which produced better detections.

Who it affects

• Small security teams and SOCs choosing whether to deploy an open-weight model they can run and audit, or a closed frontier model bought as a service.
• Procurement and engineering leads who budget for model inference and operational costs; the paper ties capability parity to the number of agent turns and model type rather than list price alone.
• Teams that use agent-driven automation tied to live telemetry, because the benchmark shows agent-session limits can change which model appears stronger.

What it costs or what it replaces

• The study reports open-weight models achieved parity at less than a tenth of the closed models' cost; the announcement does not state pricing for the models tested. Do not assume specific price points or subscription fees from this paper alone.
• Practically, the implication is that open-weight models can replace more expensive closed-frontier offerings for detection-generation workflows, provided you accept potential increases in agent interaction time and the operational overhead of running and auditing models in-house.

What we don't know

• Which exact model names and versions were part of the 19 open-weight and 7 closed models evaluated.
• The absolute compute, latency or wall-clock time required for the open-weight models to reach parity when given more turns.
• How many additional agent turns are typically needed in real-world telemetry settings to match closed models' outputs.
• Whether the result generalises beyond CTI-REALM to other security tasks or non-security agent workflows.
• The paper summary truncates one claim about price as a predictor of capability; the full text is needed to confirm that statement.

What to do next

1. Re-run a focused CTI-REALM-like test this week with one open-weight and one closed model on a representative sample of your telemetry. Measure cost-per-session, number of agent turns to completion and detection quality under your own message limits.
2. If you control the harness, remove or relax per-session message limits and compare outcomes again; record checkpoints so truncated runs do not lose prior progress as the paper describes.
3. If open-weight models match your quality targets, plan a small pilot to run and audit the model in-house; budget for slightly longer agent runtimes and for engineering time to instrument the telemetry queries and scoring pipeline.