Having big context windows is critical for LLM’s, but fitting a decent sized context on a GPU is rough work. Even when you do get a decent budget, the reality is that you’ll be running out of context more than a few times in an 8 hour session with a 120,000 or even 250,000 token context window. <\/p>\n\n\n\n
It’s worse than that since research seems to indicate contexts start to degrade and lose coherence after you use up about 60% of your context budget. You can try \/compress commands, but eventually things degrade enough you have to close and restart – which means you lose all that great context and have to retrain the LLM.<\/p>\n\n\n\n
There’s a couple ways to fight this. <\/p>\n\n\n\n
On Claude, you can persist sessions to external storage<\/a> and resume sessions.<\/p>\n\n\n\n On Copilot CLI, the memory already exists.<\/strong>\u00a0Copilot CLI writes every session to a local SQLite database \u2014\u00a0 Having big context windows is critical for LLM’s, but fitting a decent sized context on a GPU is rough work. Even when you do get a decent budget, the reality is that you’ll be running out of context more than a few times in an 8 hour session with a 120,000 or even 250,000 token context window. It’s worse than that since research seems to indicate contexts start to degrade and lose coherence after you use up about 60% of…<\/p>\n~\/.copilot\/session-store.db<\/code>. Desi Villanueva wrote a simple python script that will re-load that context if you use Copilot<\/a>. Handy if you use it for Visual Studio Code.<\/p>\n","protected":false},"excerpt":{"rendered":"