SageMaker routes shared prompts to warm caches

Proof: You need at least two instances. With one instance, all requests go to the same place regardless of strategy.

Impact: The existing Invoke API still works without changing model requests. At least two instances and serving-framework prefix caching are needed. Overloaded instances can spill requests elsewhere. AWS's reported latency improvement depends on its specific shared-prefix benchmark, not every prompt.

Watch next: Compare cache-hit rate and latency in a SageMaker routing benchmark using consistent request serialization.