Flag one or more workers as canary and give the pool a canary percentage: that share of callers, assigned consistently by an identity hash rather than per-request coin flips, prefer the canary worker while everyone else stays on stable <a href="#ref-1">[1]</a>. Both groups remain each other's failover, so the change is reversible and contained.
Try new models on real traffic
New model versions, new quantizations, and new daemon builds can be trialled on a controlled slice of real traffic before the fleet commits to them <a href="#ref-1">[1]</a>. This is how you run an A/B-style comparison of a new model against the incumbent using live requests rather than a synthetic bench that may not match your workload.
Coherent, not flapping
Caller-consistent slicing means a user is either fully on the trial or fully off it, so feedback is coherent and comparisons are meaningful with no flapping mid-conversation <a href="#ref-1">[1]</a>. A user who is in the canary group stays there for their whole session, which is what makes the resulting comparison trustworthy.
Fail-safe by construction
If the canary box dies or misbehaves, its slice fails over to stable workers automatically <a href="#ref-1">[1]</a>. The blast radius is the slice, and only until failover. Pair a canary with a drain-aware rolling upgrade, which is the mechanical rollout, and a fleet-wide model change becomes a graduated decision rather than a leap <a href="#ref-2">[2]</a>.