API Reference

Look up syntax, argument types and evaluation rules here. For the complete operational semantics, see The language.

API index

Page	Contents
Program	let, def, use, pool, stage, workload, session, server, gauge, claim
Pool	the options of pool
Stage	fifo, ps, delay
Engine	device, engine … on, tokens cap, schedule, execute, pool … on
Workload	arrive, trace, init, turn, hidden
Statements	the kernel: hold, run, grow, branch, …
Serving vocabulary	transfer, tool
Functions	arithmetic functions and observables
Distributions	~exp, ~det, ~uniform, ~erlang, ~h2, ~bernoulli
Context variables	tokens, age, remaining, … and the moment each exists at
Attributes	the built-in session attributes

Each entry gives its syntax, argument types, defaults and evaluation rules. Functions and distributions also state their return value. The Examples sections include complete runnable programs; syntax blocks with placeholders describe a form.

For the Python package, see the Python Reference. For commands and report fields, see the CLI Reference.

Types

Type	Meaning
expr	An expression. Every value is a number (f64); comparisons and ! give 0 or 1, and a non-zero operand is true. inf is a number.
Size, Value	Ordinary quantities: workload-owned request/client attributes and server bookkeeping, respectively. See attribute types.
Cost(resource)	An explicitly interpreted resource quantity. Primitive resource operations require their own cost type. See cost.
const	An expression that folds at link time: numbers, let constants, arithmetic and the arithmetic functions. It may not read a session attribute, a pool or a stage, or draw.
pool	The name of a pool, with an index (kv[j], any expr) when it is an array.
stage	The name of a stage, indexed the same way.
observe	The name of an observation, used by end-of-run aggregates.
block	{ stmt* }.
NAME	An identifier. What it names (a session attribute, a constant, a pool, a stage) depends on where it is written.

An expr is evaluated at one moment, and the context variables it may read depend on which. The moment of each argument is stated in its entry.

A function signature names each argument's kind: expr, pool or stage, or observe for an observation aggregate. Passing a pool name where an expr is expected (or the reverse) is a link error.

By what you are modelling

Start from the part of a deployment you want to write; each row names the constructs that say it and a program that uses them.

To model…	Read	In a program
a request queue and a server	stage (fifo, ps, delay), run	examples/single-turn/mg1.sq; tutorial 1
arrivals, turns and think time	arrive, turn, session; think time as tool, on the workload's own delay	the workloads use case
replaying a production trace	trace	examples/replay/vllm_replay.sq (measured replay)
memory a request holds while it runs	pool, hold	tutorial 2
what the scheduler reads at admission	at admission, moments	examples/multi-turn/vllm.sq (known, reused)
a prefix cache across turns	cache, evict, cached (attributes), drop; cachedin for the lookup at admission	tutorial 4; examples/multi-turn/vllm.sq for cachedin
an offload tier for evicted prefixes	spill	tests/pool_semantics.rs
continuous batching and chunked prefill	engine (tokens cap, each at most, execute), run E prefill, run E decode, growing	the vLLM use case; tutorial 5
the order requests are admitted and served in	queue by, advance running (decode first, by (…))	examples/multi-turn/replica.sq (decode first); examples/papers/kong_svf.sq (queue by)
preemption and recompute	preempt lifo, computed (attributes)	examples/multi-turn/vllm.sq (known = computed …)
the scheduler's waiting loop and its budget	pool … on, admit waiting, budget_left	examples/multi-turn/vllm.sq (pool reqs on vllm, budget_left)
what the scheduler may not read	hidden	every vLLM program hides the output length (o; out in the replay)
prefill/decode disaggregation	lease, release, load, transfer … from … to	the P/D use case
a routing policy	choose, observables	examples/multi-turn/routing.sq
definitions shared by several programs	def, use	lib/vllm.sq and the programs that use it
what to measure or check	observe, gauge, claim	the Kong use case