Updates
1. AWS compares model cost per successful outcome
AWS published an open benchmarking harness comparing practical model deployments by cost per correct answer and completed task.
Proof: This is a comparison of practical deployment configurations, not a controlled estimate of intrinsic model capability.
Impact: The results do not isolate intrinsic model capability.
Watch next: Compare model cost per accepted deliverable in a benchmark that includes failed attempts and review work.
Canonical host: aws.amazon.com