module
context.kv
Functions
M.kv_can_shift
True if this context's KV cache supports position shifting. Returns `false` for recurrent models (Mamba, RWKV) — they have no positional index to shift.
M.kv_seq_pos_min
Smallest KV position currently held by `seq_id`, or `-1` when the sequence is empty / the memory accessor is missing.
M.kv_seq_pos_max
Largest KV position currently held by `seq_id`, or `-1` for empty sequences.
M.kv_clear
Wipe the entire KV cache and reset our Lua-tracked `n_past` mirror to 0. This is the brutal "start over" reset used after big context changes (e.g. switching prompt entirely).
M.kv_seq_rm
Drop KV entries for `seq_id` in `[p0, p1)`.
M.kv_seq_cp
Copy KV entries from `src_seq` to `dst_seq` in `[p0, p1)`. Used to fork sequences for parallel decoding or beam search.
M.kv_seq_keep
Drop EVERY sequence in the KV cache except `seq_id`.
M.kv_seq_shift
Add `delta` to every position in `[p0, p1)` for `seq_id`. Positive shifts the window forward, negative drops it back. Only valid when `kv_can_shift()` returns true.
M.kv_seq_div
Divide every position in `[p0, p1)` for `seq_id` by `d`. Used for context compression (group multiple positions together to fit a longer effective prompt in the same window).