module
kv.eviction
Functions
M.handle_overflow
Try to free at least `overflow` tokens from `seq_id`. The function mutates the KV cache and the session's bookkeeping (`n_past`, `_msg_kv_ends`) ; it returns a structured report of what happened.
M.handle_overflow(ctx, seq_id, session, opts)
ctxion7.core.Context
seq_idinteger
sessionion7.llm.Session
optstable
→ table