Standard and Pro subscriptions add allowances for search, webpage parsing, code execution, speech and image tools through MCP. Subscribe on the Qianwen AI platform, obtain the Token Plan endpoint and API key, then configure a compatible tool. Included allowances vary by tool and plan. After they run out, additional calls are billed; video generation and several managed services are pay-as-you-go rather than included unlimited usage.
Watch next
Check the Standard or Pro tool quota before a workflow; monthly tool allowances are separate from model Credits.
Claude plugin eval compares with and without your plugin
Claude Code added plugin behavior evaluation with a no-plugin baseline.
Use evals to measure how reliably your plugin steers Claude to the right outcome, to catch regressions when you change the plugin or a new model ships, and to see what the plugin contributes compared with no plugin at all.
Plugin authors need Claude Code 2.1.269 or later and a plugin manifest. Run claude plugin eval init at the plugin root, then claude plugin eval dot to test prompts against graders. By default each case runs three times with the plugin and three without. These and judge calls consume plan usage or API billing. A high score alone does not prove the plugin helped; compare the two arms.
Watch next
Inspect failed graders and usage-limit errors before interpreting a score change as a regression.
Cognition released SWE-2, post-trained from Kimi K3, and made it available in Devin Desktop and CLI.
SWE-2 is available starting today in Devin Desktop and CLI. We’re also rolling it out on Devin Web and Fusion.
Engineers can use SWE-2 in those clients for coding work; Devin Web and Fusion are still described as rolling out. Cognition reports fewer detours and earlier implementation in its task benchmarks. Those are vendor evaluations under specific effort levels and task sets. Its pricing and benchmark comparisons are not guarantees that an individual project will be cheaper or correctly completed.
Watch next
Compare accepted code and total cost in a SWE-2 benchmark using the same project and test suite.
Sakana separates cost-focused Fugu Max from Ultra v2
Sakana released Fugu Max and Fugu Ultra v2, two configurations of its model-orchestration architecture.
Both models are available today via our standard OpenAI-compatible API.
The benchmark comparisons are Sakana's claims, and SWEFish is its internal coding benchmark. They do not establish a guaranteed advantage on a customer's workload.
Watch next
Compare Max and Ultra v2 on a workload benchmark using accepted outputs and total task cost.
AgentCore separates working infrastructure from successful tasks
AWS published a dual-monitoring reference for an airline-reservation agent system.
Infrastructure monitoring and agent effectiveness monitoring require different approaches.
AgentCore Evaluations scores sampled interactions for helpfulness, correctness and goal completion. AWS DevOps Agent investigates infrastructure failures such as a missing model-invocation permission. Setup requires AWS account permissions and the Boto3 SDK. Model judges can be wrong, and asynchronous sampling can miss or detect a bad response only after delivery. This is an architecture example, not guaranteed autonomous incident resolution.
Watch next
Compare evaluation judgments against known failures in an AgentCore benchmark; inspect missed goals despite healthy infrastructure.
AgentCore turns MCP responses into interactive booking cards
AWS published an AgentCore MCP Apps example that returns interactive HTML widgets inside compatible AI hosts.
If the tool has an associated resource URI (for example, ui: //widget/unicorn-list ), the AI host initiates this phase. Tools without an associated widget, such as view_bookings and return_unicorn, return text-only content and skip this phase entirely.
The sample uses an unauthenticated Gateway fronted by WAF; production access controls still need deliberate design.
Watch next
Check the next AgentCore MCP Apps release note for rollout, availability, or quota changes.
NVIDIA NIM benchmark depends on workload and hardware
NVIDIA published a Nemotron 3 Ultra NIM serving configuration and performance comparison.
The published curves are a starting point, not a promise that every application will see the same result.
The reported throughput uplift combines caching, scheduling, precision and speculative decoding. It is not a general model-quality improvement, and the component gains cannot simply be added together.
Watch next
Compare NIM configurations on a benchmark with the same hardware and per-user latency target.
AWS added prefix-aware routing for SageMaker real-time inference endpoints.
You need at least two instances. With one instance, all requests go to the same place regardless of strategy.
The existing Invoke API still works without changing model requests. At least two instances and serving-framework prefix caching are needed. Overloaded instances can spill requests elsewhere. AWS's reported latency improvement depends on its specific shared-prefix benchmark, not every prompt.
Watch next
Compare cache-hit rate and latency in a SageMaker routing benchmark using consistent request serialization.
Salesforce Newsroom reports: Salesforce says the platform gives companies faster, more flexible ways to automate customer service and deliver measurable outcomes. That is the company’s claim; it does not establish measured customer savings.
The business use is customer service automation. For companies using the platform, the financial question is whether that automation reduces the cost of handling customer requests while maintaining service quality.
Watch next
After Salesforce Completes Acquisition of Fin, watch ACWI, EAFE, emerging markets, and official updates tied to those regions.
NVIDIA has one of the largest and most complex supply chains in the world, and its performance is measured from wafer-out to first token. The interval is in two...
Multi-agent systems fail in ways traditional monitoring misses. This post presents a dual-layer approach to monitoring production agents: Amazon Bedrock AgentCore Evaluations for continuous quality scoring and AWS DevOps Agent for autonomous infrastructure inv
Comparing models on dollars per million tokens misses what production workloads actually pay for: outcomes. This post shares an open-source benchmarking harness that measures cost per correct answer, agent trajectory cost, and rubric-graded deliverable quality