Sonnet 5.5 migration: test the full task
Practical perspective from an IT leader working across operations, security, automation, and change.
9 minute read with practical, decision-oriented guidance.
Leaders and operators looking for concise, actionable takeaways.
Topics covered
A Sonnet 5.5 migration should proceed when the complete integration produces acceptable business results at a cost you can justify. A response that looks correct is insufficient if the application never saves the required draft, loses an exception or repeats an action. Compare the current and proposed versions on the same work before deciding which part of the workflow to change.
Claude is an AI model that an application can call through an API. In a business system, that call may be only one step between receiving a request and producing a record a member of staff can use. This article applies to that integration decision. It does not assume that a change to a chat subscription requires the same work.
Check the migration path before estimating the benefit
Anthropic's Sonnet 5.5 announcement describes faster output and lower task costs in its testing. Those findings do not establish a saving for your workflow. Use the current API pricing table for the route you actually buy, including applicable cache writes, cache reads, tool charges and pricing modifiers. API charges and subscription prices are different costs.
The migration guide distinguishes the Messages API from Claude Managed Agents. It describes integration changes for the Messages API that depend on the starting model, while saying Managed Agents requires only a model-name change. A business should first identify its integration path and existing settings. Otherwise a supplier's estimate may cover changing a name while leaving application behaviour untested.
One consequential Messages API change concerns tool selection. A tool is an action the application makes available to the model, such as preparing a draft order record. The guide says forced tool-selection types are rejected and automatic selection can return without a tool call. Constraining the fields in a tool request does not guarantee that the request happens. The integration owner must check completion in the destination as well as validating the model response.
Decide what counts as one accepted task
Consider a fictional parts distributor whose application converts an incoming order request into a draft for staff approval. Priya owns order operations; Sam owns the integration. The model cannot approve an order, charge a customer or send a message. Their migration question is whether Sonnet 5.5 can replace the existing model for this draft-preparation task.
They agree that acceptance requires the correct customer reference, items and quantities in the test draft store, linked to the original request identifier. A readable answer without that saved record fails. A duplicated record also fails. Missing or contradictory information must produce a visible exception for a person, rather than an invented quantity or customer reference.
They prepare 100 synthetic requests with known expected outcomes. The set includes ordinary requests, conflicting quantities, missing details and unsupported items. An accepted task can be a correct exception record. It does not have to be a completed order draft. Priya checks the business result against those expectations, and Sam checks that the intended record actually exists.
The test store has no connection to production customer records, payment services or email delivery. Simulated destination failures exercise error handling there. The team records the prompt, model settings, tool definitions and application version for each run, so a later improvement can be traced to the change that produced it.
A cheaper first run can still fail the decision
The figures below are invented planning assumptions for this example, not measured results, Anthropic benchmarks or a quote. Each run uses the same 100 requests. Accepted-after-correction counts are additional to first-pass acceptances; a retried request is still one business task.
| Outcome across 100 requests | Current integration | Initial proposed integration | Fixed proposal, full retest |
|---|---|---|---|
| Accepted first time | 90 | 94 | 97 |
| Accepted after correction or retry | 8 | 2 | 3 |
| Failed or unresolved | 2 | 4 | 0 |
| Total accepted | 98 | 96 | 100 |
| Assumed API and tool charges, including failed attempts and retries | USD 12 | USD 8 | USD 9 |
| Human review and correction time | 150 minutes | 120 minutes | 90 minutes |
The initial proposal appears cheaper. Its API and tool cost per accepted task is USD 8 divided by 96, or about USD 0.083. The current integration costs USD 12 divided by 98, or about USD 0.122. Looking only at that comparison would favour the proposal.
Priya rejects it for release. Four requests lack the required destination result, despite the application showing them as complete. A member of staff relying on that status could assume an order had reached the review queue when there was nothing to review. The two failures in the current integration also need attention, but their existence does not justify introducing a new silent failure.
Sam checks the retained responses and destination records. In this fictional implementation, the four failures share one cause: the application marks a successful API response as task completion even when automatic selection returns no draft-creation tool call. It checks the response status but not the business action. This is an application defect exposed by the migration test, not evidence that every Sonnet 5.5 integration has the same defect.
Fix completion tracking, then repeat the comparison
Sam changes the application so it records completion only after the expected destination record is confirmed. If no tool call occurs, the task stays incomplete and goes to a visible exception queue. Any retry uses the original request identifier and checks for an existing draft before another write. Priya agrees which exceptions staff can resolve and which must stay on hold.
They first repeat the four failed cases, then run the complete 100-request set again against the fixed proposal. Sam also simulates a destination that saves a draft but loses the acknowledgement. The repeated attempt finds the existing draft under the same request identifier and creates no second record. A simulated refusal to save remains visible as an unresolved exception, rather than a completed task. These are acceptance tests of the proposed application behaviour, not capabilities supplied automatically by changing the model.
The full retest produces the third column. Ninety-seven requests meet their expected outcome first time. Three require correction or retry and then pass; none remains unresolved. Priya signs off the results for the defined draft-preparation task. Sam retains the failed evidence and the changed application version, so the team can explain why the initial proposal was rejected and the revised one accepted.
Passing this set supports a limited next step. It does not prove that an unfamiliar order format, a different tool or a higher workload will behave the same way. The team must expand the tests before expanding that scope.
Include the review work in the migration decision
For a fuller comparison, assume review time costs USD 30 an hour in this example. This is a fictional costing assumption, not a wage recommendation. Using the same currency avoids hiding an exchange-rate assumption inside the calculation.
The current run costs USD 12 in API and tool charges plus USD 75 for 2.5 hours of review. USD 87 divided by 98 accepted tasks is about USD 0.888 per accepted task. The initial proposal costs USD 8 plus USD 60 for two hours of review, or about USD 0.708 per accepted task. It remains unacceptable despite that apparent saving because the completion defect is unresolved.
The fixed proposal costs USD 9 plus USD 45 for 1.5 hours of review. USD 54 divided by 100 is USD 0.54 per accepted task. All failed attempts and corrections stay in the totals. If a run accepts no tasks, there is no meaningful cost-per-accepted-task figure to report; it is not zero.
One-off investigation, changes and test design belong in a separate migration budget. These small runs cannot establish how quickly that investment would pay back. The AI ROI guide explains the wider investment question. For an actual decision, replace the assumed charges with billing evidence or clearly identified estimates, and record elapsed time too. Lower review effort may be valuable, but a longer wait for a draft could still disrupt a time-sensitive operation.
Accept a limited change with an owner and a way back
Priya approves a supervised pilot for the tested draft-preparation task. Staff approval remains mandatory, and order confirmation, payment and messaging stay outside the model's authority. Sam owns the release and retains the previous working configuration, with the return procedure checked in the test environment. Priya owns business acceptance and the daily comparison of submitted requests with draft or exception records.
An unexplained missing or duplicate record stops the pilot and returns affected work to the established manual process while Sam investigates. The team checks destination records before replaying anything, because switching back does not itself undo a partially completed task. If the pilot requires more review than the business can sustain, Priya can retain the existing workflow rather than treating the migration as inevitable.
That decision accepts a defined application change under supervision. It does not approve every future use of the model or promise the illustrative saving. Retaining the current integration avoids migration effort, but also leaves its known defects and operating costs to address. The choice depends on which problems the proposed change actually resolves.
Bring the failed task to an integration review
If your comparison shows a correct-looking response but no usable business record, the next useful step is to examine the application around the model. Bring a redacted task example, the expected destination result, the failed result and the current ownership of retries and exceptions. Avoid sharing customer data or credentials in an initial enquiry.
My bespoke websites and business systems service can cover an agreed integration review or application change, launch checks and documented handover. We can scope the connection between model responses, destination records and staff approval before deciding whether a migration is worth implementing. The service does not guarantee an AI saving.
Discuss your Sonnet migration workflow with the business task you need to complete and the failure your test has exposed. That gives us a specific basis for agreeing the work and its acceptance evidence.
Share this post
About the author
Daniel J Glover
IT Leader with experience spanning IT management, compliance, development, automation, AI, and project management. I write about technology, leadership, and building better systems.
Continue exploring
Keep building context around this topic
Jump to closely related posts and topic hubs to deepen understanding and discover connected ideas faster.
Explore topic hubs
Related article
AI ads phishing: check before connecting
AI ads phishing makes account connections a business decision. Verify the supplier, challenge requested access and agree a narrower reporting workflow.
Related article
Claude Cowork: cloud tasks for business
Claude Cowork changes new Pro and Max tasks on 6 October. Check cloud file copies, desktop dependencies and who approves continuing each business task.
Related article
Zammad security: helpdesk risk checks
Zammad security questions extend to connected business systems. Review DIVD's qualified case guidance, integration access, ownership and support continuity.
Related article
SQL linked queries: test beyond restart
SQL linked queries can work after a restart and still slow down over repeated runs. Agree what evidence your business needs before accepting normal operation.
Ready to improve your IT operations?
Request a free 30-minute consultation to discuss your IT challenges. Send a short outline and I will reply to arrange a suitable time. No obligation.
Request a free 30-minute consultationGet Occasional IT Leadership Insights
IT leadership insights, occasionally. No fluff. Unsubscribe any time.
No spam. Unsubscribe any time.