Your team ships faster than it has ever shipped. The AI writes the feature, writes the tests, the suite goes green, the pull request gets merged. So the question turns up in planning now, usually from whoever signs the invoices: do we still need people clicking through the app by hand, and do we still need a human acceptance step at the end?
Yes to both. And the human step deserves more room than you gave it two years ago, not less.
Automate everything that answers "does this code do what it was told to do." Keep humans for the question no test suite can ask: does using this thing actually feel right. Those are two different questions. Most teams have quietly stopped asking the second one, because the first one became so cheap to answer.
What AI generated code actually gets wrong
The data on this got much better over the past year, and it does not say what the tooling vendors hoped.
CodeRabbit analysed 470 real open source pull requests in December 2025, comparing 320 marked as AI coauthored against 150 human only. Critical and major defects came in up to 1.7 times higher on the AI side. Logic and correctness issues, meaning business logic errors and unsafe control flow, rose 75 percent. Performance problems like excessive I/O appeared nearly eight times more often.
Lightrun polled 200 SRE and DevOps leaders across the US, UK and EU for its 2026 report. In 43 percent of cases, AI generated code changes still needed manual debugging in production after passing QA and staging. Read that one twice, because it is the number that decides this argument. The gate was there. It did not catch it.
SmartBear asked 273 software leaders in May 2026 whether quality had held up. The answer from 70 percent of them was that it had already degraded, because code creation outran testing capacity.
None of that is an argument against AI assisted development. We use it every day and we are not giving it back. It is an argument that the volume of code your team produces went up sharply while the thing checking that code stayed exactly the same size.
The question no test asserts
Here is where I think most teams have it backwards.
AI is genuinely good at turning business logic into code. Give it a clear rule and it will implement the rule, cover the branches, and write a test proving the rule holds. That part is close to solved, and pretending otherwise is nostalgia.
What it cannot do is understand the world your app lives in. It has never held a phone one handed on a tram with two bars of signal. It does not know that your primary action sits exactly where a thumb cannot reach on a six inch screen, or that the loading state is technically correct and still makes the app feel broken, or that the form is fine except no human would ever fill it in that order.
There is no assertion for "this feels wrong." No test fails when a screen is confusing.
Almost everything you build is built for humans. So a human has to be the one who signs off. That is not a process preference, it is the shape of the problem.
Your four options
1. Automated suites only, including AI written tests
The cheapest setup and by far the most common right now. Unit tests, integration tests, CI, and increasingly the tests themselves generated by the same assistant that wrote the feature.
Fast, nearly free at the margin, and it does catch real regressions. It also has a structural hole. When the same model writes the implementation and the test, both come out of the same reading of the requirement. If that reading was wrong, you get a confident green suite proving the wrong thing works perfectly.
2. Automated suites plus emulators and cloud device farms
Add BrowserStack or similar, run across device and browser matrices, collect screenshots. Nothing to buy, scales to whatever you pay for, and it genuinely catches layout breakage.
What it does not give you is a person using the product. A screenshot on a virtual Pixel tells you the button rendered. It says nothing about whether the app stutters once the device is warm, whether the notification lands while you are looking at the screen, or whether the flow is pleasant.
3. Developers test their own work
Common in small teams and better than nothing. It is also the weakest of the four, for a reason that has nothing to do with skill.
The person who built the feature cannot see it fresh. They know which button to press, in which order, with which data. They unconsciously avoid the paths that feel wrong, because those paths felt wrong to build. Ask a developer to test their own screen and you get the happy path, verified beautifully.
4. A dedicated QA person, on real devices, on a fixed cadence
Someone whose job is quality, testing on the hardware users actually own, on a schedule rather than at the end of a project.
This is what we run. One QA expert, one full day per week per app, on 16 physical devices covering Android, iOS, iPadOS, macOS, Windows and Linux. Not automated runs on those devices. A person holding them and using the product the way a user would.
The cost is honest: a day a week, every week, that produces no features. What you get back is seeing your product the way your users will see it, roughly a week before they do.
Where each one breaks
Option 1 breaks silently, which is what makes it the dangerous one. Nothing looks wrong until a user tells you, and by then it is production and it is a support conversation.
Option 2 breaks on anything physical. Real networks, real batteries, thermal throttling on a three year old midrange Android, real hands.
Option 3 breaks on familiarity, and it gets worse the more experienced the developer is.
Option 4, the one I am recommending, breaks in one specific and very common way: when the QA person is outside the build loop. A tester who receives a finished build, runs through it, and files tickets into a backlog is a filter, not a feedback loop. Everything they find is expensive by the time they find it, and the team stops feeling responsible for how the product feels because quality became somebody else's column on the board. The cadence is what prevents this. Weekly means findings land while the code is still warm and while the person who wrote it still remembers why.
And the honest concession: if your product is an internal tool, an admin panel or an operations dashboard used by 20 trained employees who have no alternative, this is overspending. Those users will tell you it is ugly and keep using it, because it is their job. Automate the correctness, protect the data, skip the ceremony.
The decision table
- Automated only. What it catches: Regressions, and logic your spec described correctly. What it misses: Anything the spec got wrong, all UX and feel. Suits: APIs, back end services, internal tools.
- Plus cloud devices. What it catches: Layout breakage across screens and browsers. What it misses: Real world conditions, whether the flow makes sense. Suits: Web products with wide browser spread.
- Developers self test. What it catches: Obvious breakage, the happy path. What it misses: Everything a fresh pair of hands would hit first. Suits: Very early stage, pre revenue.
- Dedicated QA on real devices, weekly. What it catches: Feel, flow, and the wrong thing built correctly. What it misses: Nothing structural, but it costs a day a week. Suits: Anything where users can leave.
Pick this
If your users have a choice, meaning consumer apps, marketplaces, anything where a bad first session ends in an uninstall, use option 4. A real person, on real hardware, on a weekly cadence, who did not write the code. This is the recommendation and I do not think it is close.
If your users have no choice, internal tools and back office systems, use option 1 or 2 and spend the saved time on correctness and data integrity. Nobody churns from an ERP because the animation is stiff.
If you are regulated, healthcare, fintech, anything with an audit trail, you need the documented QA function regardless of how it feels to use. "The AI wrote it" is not an answer auditors accept, and it will not become one.
One rule holds across all four: never let the thing that writes the code be the only thing that checks the code. A human reviews the acceptance criteria before the AI touches them, a different human confirms the result. Otherwise you are grading your own homework, at machine speed.
You probably do not need to hire for this
Here is the part most founders get wrong in the other direction. Having read the above, the instinct is to hire a QA engineer. For most teams shipping one or two products, that is a full salary for work that genuinely takes a day a week.
We built the setup for our own projects and it turned out to be underused, so we offer it to other teams. One QA expert, one day a week, 16 devices across Android, iOS, iPadOS, macOS, Windows and Linux, testing your builds on the hardware your users actually own. You get the findings while the code is still warm, without carrying a permanent hire or buying a shelf of phones that will be obsolete in three years.
At Conimex IT we build custom web and mobile products in Laravel, React and React Native, as a Laravel Community Partner, and we use AI assistance heavily throughout. That is exactly why the manual testing day is protected rather than optional.
If you are shipping AI assisted work faster than you can check it, and you are not certain whether the last human gate in your process is real or ceremonial, we will look at it with you. Get in touch at [email protected].