MIT Researchers Just Solved Context Rot

The phenomenon where LLM effectiveness degrades significantly after approximately 100,000 tokens regardless of stated context window size (0:30). Chroma's research demonstrated that whether a model claims 200K or 1M token capacity, performance drops predictably at this threshold.

MIT's solution that treats prompts as part of the environment rather than direct neural network input (5:45). The LLM writes code to programmatically decompose large documents and recursively invokes smaller versions of itself to process manageable chunks.

The RLM system uses Python to perform 'reconnaissance' on large documents (6:00), extracting metadata like length, structure (chapters), and relevant sections before processing. This allows intelligent chunking rather than blind splitting.

The recursive element where the main LLM (GPT-5) creates tool calls to smaller versions (GPT-5 mini) to handle individual document sections (7:30). Each sub-agent processes its chunk and returns summarized answers to the parent model.