ion7-llm / kv.eviction

module

kv.eviction

Functions

M.handle_overflow

Try to free at least `overflow` tokens from `seq_id`. The function mutates the KV cache and the session's bookkeeping (`n_past`, `_msg_kv_ends`) ; it returns a structured report of what happened.

M.handle_overflow(ctx, seq_id, session, opts)
ctxion7.core.Context
seq_idinteger
sessionion7.llm.Session
optstable
→ table