class
ion7.llm.kv.ContextManager
Functions
ContextManager.new
Build a manager.
ContextManager:set_hook
Install a hook. See module docstring for the list of names.
ContextManager:set_system
Set or replace the system prompt. Does the right thing for both multi-seq (encode into the reserved prefix slot once, copy to every session) and single-seq (remember text only) modes. A change in system text invalidates the radix cache : every stored blob was taken under the previous system, restoring it now would mix two different priors.
ContextManager:prepare
Make `ctx` ready for the next decode of `session`. After this call the session's row contains exactly the encoded prefix + conversation history up to (but not including) the next reply.
ContextManager:make_room
Reclaim at least `n_needed` cells in `session`'s KV row by evicting old tokens. Used by the engine and pool to push back the n_ctx wall mid-generation without re-rendering the prompt. The session's `n_past` is updated in place ; the caller must keep its sampler state coherent across the call (penalty history is window-relative, so a small post-eviction off-by-one is tolerable in practice — a deliberate eviction means the model is already at the edge of its useful context anyway).
ContextManager:release
Drop the session's KV row + return its slot to the pool. Call when a conversation is permanently closed.
ContextManager:fork
Fork a session : duplicate its KV row onto a fresh slot. The child's seq_id is assigned here ; on first prepare the child will snapshot from the copy, no prefix re-encoding needed.
ContextManager:stats
Snapshot of the manager's runtime stats.
ContextManager:clear_radix
Force-clear the radix cache. Useful between unrelated workloads when blobs from prior runs would no longer be useful.