ion7-llm / kv.prefix

class

ion7.llm.kv.Prefix

_ctx ion7.core.Context
_vocab ion7.core.Vocab
_seq integer? Reserved seq_id, or nil when unavailable.
_text string? Cached prompt text (for `:set` short-circuit).
_n integer Token count of the encoded prompt.

Functions

Prefix.new Prefix:has_slot Prefix:seq_id Prefix:set Prefix:copy_to Prefix:n_tokens Prefix:text Prefix:clear

Prefix.new

Build a prefix manager.

Prefix.new(ctx, vocab)
ctxion7.core.Context
vocabion7.core.Vocab
→ ion7.llm.kv.Prefix

Prefix:has_slot

True when this manager owns a real prefix slot (i.e. the context supports `n_seq_max >= 2`). When false, `:copy_to` is a no-op and callers should account for the prompt at session-prefill time.

Prefix:has_slot()
→ boolean

Prefix:seq_id

The seq_id that holds the encoded prefix, or nil when the manager runs in degraded mode (n_seq_max == 1).

Prefix:seq_id()
→ integer?

Prefix:set

Encode `system_text` into the prefix slot. No-op when called with the same text twice (the cache is idempotent on identical input). Has no effect when the manager is in degraded mode — the prompt is only remembered ; the session will re-encode it itself.

Prefix:set(system_text)
system_textstring
→ integerToken count after encoding (0 in degraded mode).

Prefix:copy_to

Copy the prefix into `dst_seq_id`. The destination's pre-existing KV (if any) is wiped first so a stale prefix from a prior session cannot leak. Returns the number of tokens copied (0 in degraded mode).

Prefix:copy_to(dst_seq_id)
dst_seq_idinteger
→ integer

Prefix:n_tokens

Token count of the encoded prefix. Always 0 in degraded mode.

Prefix:n_tokens()
→ integer

Prefix:text

Cached prompt text, or nil when never set.

Prefix:text()
→ string?

Prefix:clear

Drop the cached prefix. Wipes the prefix slot's KV and zeroes the token count ; subsequent `:set` calls will re-encode.

Prefix:clear()