At INT21, we are all-in on self-improving multi-agent systems. We have built SwarmOS, our cloud-native platform for running specialized agents, and our first product, PTX Kernel Factory. The biggest change is not simply a larger context window. It is what frontier models make possible when they are orchestrated: turning one enormous context problem into a coordinated team of smaller, evidence-seeking tasks.
About a year ago, I was exploring how AI agents could generate CUTLASS C++ kernels, NVIDIA’s building blocks for high-performance GPU computation. By my count, the entire CUTLASS codebase represented roughly five million tokens. At the time, the best production model available to us offered a one-million-token context window.
The central blocker was never code generation. It was finding and preserving the right evidence across the repository.
Rather than wait for a magical five- or ten-million-token model, I ran the one-million-token model several times in parallel. Each agent studied a different portion of the codebase, and combined their findings as the final step.
It was a simple architecture, but it established the principle behind our work today:
When context stops fitting vertically, scale it horizontally.
A Bigger Window Is Not Better Context
Even when millions of tokens technically fit inside a model, the model must still separate signal from noise. A bigger window introduces more irrelevant information, more intermediate output, and more competition for attention.
Multi-agent systems address this structurally. Specialized agents explore different parts of a codebase, investigate the same question from independent angles, and return distilled findings to a coordinating agent. When a subproblem is still too large, it gets divided again.
The goal is not infinite context. It is effective context.
A General Solution for Complex Problems
At INT21, we use SwarmOS not only for hard engineering problems, such as expert-level PTX generation, but also to understand complex business landscapes.
To test this outside a codebase, we pointed SwarmOS at a different kind of long-context problem:
The research ran autonomously using public information. The system involved 27 agents, performed 166 web searches, visited more than 200 web pages across 73 unique domains, and ran for about two hours.
In total, it consumed 119 million tokens.
We are sharing the report in this article because we believe this is a topic many people will want to understand more deeply. But the report is also a demonstration of the broader point: multi-agent orchestration is the real long-context breakthrough.
Long Context Is Becoming a Systems Problem
So, are the latest AI generations solving long contexts?
Not by making context infinite, but in an agentic way.
It is helping solve long context by making it divisible, searchable, and composable.
That is why INT21 is all-in on multi-agent systems. At INT21, we are building Self-Improving Compute Infrastructure, and SwarmOS is the operating system behind a massive number of agents.
PTX Kernel Factory is now in beta for teams working on GPU kernel generation and AI compute infrastructure. Accepted participants receive limited-time free access and $100 in credits. Join the beta ->.