Having big context windows is critical for LLM’s, but fitting a decent sized context on a GPU is rough work. Even when you do get a decent budget, the reality is that you’ll be running out of context more than a few times in an 8 hour session with a 120,000 or even 250,000 token context window.

It’s worse than that since research seems to indicate contexts start to degrade and lose coherence after you use up about 60% of your context budget. You can try /compress commands, but eventually things degrade enough you have to close and restart – which means you lose all that great context and have to retrain the LLM.

There’s a couple ways to fight this.

On Claude, you can persist sessions to external storage and resume sessions.

On Copilot CLI, the memory already exists. Copilot CLI writes every session to a local SQLite database — ~/.copilot/session-store.db. Desi Villanueva wrote a simple python script that will re-load that context if you use Copilot. Handy if you use it for Visual Studio Code.