Different workloads

These examples keep the same engine and vary the client: single requests, chat, tool-using agents and an approximation of delegated subagent traffic. The deployment uses a device, the engine vllm on it, its pools and server; workload supplies arrivals and turns. Client delays (tool, user, delegate) represent time outside the engine.

All four programs share the engine constants, resource declarations and server body of examples/multi-turn/vllm.sq, checked by tests/workloads.rs. They hide output length o from scheduling expressions. See the vLLM use case for the engine's validation scope.

	single turn	multi-turn chat	multi-turn agent	agent with subagents
program	vllm_single_turn.sq	vllm_chat.sq	vllm.sq	vllm_subagents.sq
a session is	one request	a conversation	a task	a task and the subtasks it hands out
carried to the next turn	nothing	the conversation, K = K + n + o	the same	the same
between turns	—	a person, run user (cost(user, ~exp(Z)))	a tool, tool (~exp(Z))	a tool, or waiting for the subagents
who makes arrivals	the environment	the environment	the environment	a Poisson approximation of parent and child traffic
the prefix cache	never read back	the last turn's prefix	the last turn's prefix	same, and a subagent misses the context it copied

Single turn

examples/single-turn/vllm_single_turn.sq

  // A request arrives, is served, and its session ends: no context carries
  // over, so `K` stays 0 and the prefix cache is never read back (entries
  // are per session, docs/language.md §9).
  workload {
    arrive poisson(Lambda);
    hidden o;              // the scheduler knows max_tokens, not the length (sched/utils.py:98-119)
    init { set K = 0; }
    turn {
      set n = ceil(~uniform(1000, 3000));
      set o = floor(~exp(200)) + 1;
    }
  }

No session declaration is needed: each arrival makes one turn and then leaves. K stays 0 because nothing carries over, so the server's prompt = K + n is just the new tokens. The cache (prompt + o) in the server still caches the prefix when the request finishes, but no later request of that session will read it, so here the cache only takes up space until LRU evicts it. The hit rate is 0.

This does not model shared system prompts. vLLM identifies cached blocks by content and prefix hashes (kv_cache_utils.py:650-680, block_pool.py:197-223), so a later request can reuse a matching cached prefix from another request. serQ's cache is per session; see limitations.

Multi-turn chat

examples/multi-turn/vllm_chat.sq

  // Each turn resends the conversation so far (`K`) with a short message;
  // the gap between turns is a person, not a tool.
  workload {
    arrive poisson(Lambda);
    hidden o;              // the scheduler knows max_tokens, not the length (sched/utils.py:98-119)
    init { set K = 0; }
    turn {
      set n = ceil(K == 0 ? ~uniform(200, 1000) : ~exp(100));
      set o = floor(~exp(300)) + 1;
      set more = ~bernoulli(p);
    }
    session {
      turn;
      while (more) {
        set K = K + n + o;
        run user (cost(user, ~exp(Z)));
        turn;
      }
    }
  }

Each turn sends the conversation K plus a new message of mean 100 tokens. After the request, set K = K + n + o carries its input and output into the next turn. run user (cost(user, ~exp(Z))) models reading and typing time on a delay stage, with mean Z = 20 seconds.

Multi-turn agent

examples/multi-turn/vllm.sq

  // The client: sessions arrive, each turn brings a prompt of n new tokens
  // on a context of K and wants o tokens back, and after a turn the session
  // calls a tool and comes back with probability p. Nothing here is vLLM.
  workload {
    arrive poisson(Lambda);
    hidden o;              // the scheduler knows max_tokens, not the length (sched/utils.py:98-119)
    init { set K = 0; }
    turn {
      set n = ceil(K == 0 ? ~uniform(1000, 3000) : ~exp(500));
      set o = floor(~exp(200)) + 1;
      set more = ~bernoulli(p);
    }
    session {
      turn;
      while (more) {
        set K = K + n + o;
        tool (~exp(Z));
        turn;
      }
    }
  }

The agent has the same turn structure as chat, with a 3-second mean tool delay and 500 mean new tokens per turn. Its initial prompt is a longer task description. The workload determines these differences.

Agent with subagents

This example approximates subagents as independent sessions. It does not create child sessions in response to a parent's delegation or join their actual completion times.

examples/subagent/vllm_subagents.sq

  // An arrival is a subagent with probability kids / (1 + kids), so the
  // stream carries both at the rate the parents would create them. A
  // subagent starts from a context of its own: the parent's context it
  // copies is not shared across sessions (docs/language.md §9).
  workload {
    arrive poisson(Lambda * (1 + kids));
    hidden o;              // the scheduler knows max_tokens, not the length (sched/utils.py:98-119)
    init {
      set sub = ~bernoulli(kids / (1 + kids));
      set K = sub ? floor(~uniform(2000, 6000)) : 0;
      set t_s = now;
    }
    turn {
      set n = ceil(K == 0 ? ~uniform(1000, 3000) : ~exp(500));
      set o = floor(~exp(200)) + 1;
      set more = ~bernoulli(sub ? pc : p);
      set fan = sub ? 0 : ~bernoulli(q);
    }
    session {
      loop {
        turn;
        set K = prompt + o;
        branch (more) {
          branch (fan) { run delegate (cost(delegate, W)); } else { tool (~exp(Z)); }
        } else {
          branch (sub) { observe subagent = now - t_s; }
          end;
        }
      }
    }
  }

The approximation has three parts:

• The children arrive on their own. A parent's turn that continues delegates with probability q to k subagents. A parent continues p / (1 − p) times on average, so it creates kids = q · k · p / (1 − p) subagents. The program folds them into the one Poisson stream at rate Lambda · (1 + kids), and marks an arrival as a subagent (sub) with probability kids / (1 + kids).
• The parent waits a constant. A delegating turn runs run delegate (cost(delegate, W)). The supplied W = 15.4 seconds is a model input; it does not track the simulated children's completion times. It was calibrated as the mean maximum of k subagent durations collected with --dump, iterated until it stopped moving (seed 1, 10 000 s, 1 000 s of warm-up); seeds 1–5 give 15.22 ± 0.25 s.
• A subagent's context is its own. It starts from K ~ uniform(2000, 6000), which stands for the parent's context it copies. The cache does not share entries across sessions, so the subagent misses that prefix.

--set q=0 gives the same agents with no delegation. Here are both next to the other three (seed 1, 1 800 s after warm-up; requests/s is the number of response observations over those 1 800 s; each program runs at its own arrival rate, so compare the rows within a pair, not across the table):

program	requests/s	hit rate	prefill tokens	TTFT (ms)	response (ms)	engine busy
vllm_single_turn	2.98	0.00	1 995	43	96	0.23
vllm_chat	1.62	0.80	221	5	66	0.10
vllm	2.94	0.89	727	16	62	0.16
vllm_subagents --set q=0	0.90	0.89	665	13	58	0.05
vllm_subagents	3.77	0.68	1 997	53	112	0.29

Delegation increases request traffic and the mean prefill work in this approximation. The first turn of each subagent misses its copied context; a shared content-addressed cache could reuse that prefix. These are illustrative model results, not measurements of a subagent runtime.

Limits of the approximation

• Parent wait W is fixed, so slower child responses do not delay the parents. The model cannot reproduce that feedback dynamically.
• The parent holds no engine resources while it waits. This example cannot expose deadlocks caused by a parent retaining resources its child needs.
• Prefixes are not shared across sessions.
• Children have no parent identity. Scheduling and statistics cannot follow an actual task tree.

The language's fork/join runs parallel legs of one request; it does not create the independent child sessions this workload approximates.