Today’s QA teams mix two kinds of engineers. Some learned testing before AI assistants existed. Others learned it with an AI assistant open from day one. Neither is better. They break things differently, they miss different bugs, and the best teams use both.
I’m the first kind. I learned test automation with Java and Selenium, by writing every locator and every wait by hand. Today an AI writes much of my test code. So I’ve seen both ways of working up close, and I’ve watched what each one catches and what each one lets through.
01The QA who learned before AI
We learned the slow way. A test failed, and we read the stack trace line by line. A test was flaky, and we spent an afternoon finding the race condition. Nobody generated our test data or explained the codebase to us. We read it.
What that gave us:
- A debugging instinct. When a build goes red, we know where to look first, because we’ve looked in the wrong places many times before.
- Healthy distrust of green. We’ve seen tests that pass while checking nothing, so a passing suite is a question, not an answer.
- Test design fundamentals. Boundary values, equivalence classes, state transitions, risk-based priorities. We learned these because we couldn’t afford to test everything.
- Knowing what’s under the UI: HTTP, the DOM, SQL, how a browser actually waits for a page.
Where we fall short:
- Spending hours on boilerplate an assistant would write in a minute, out of habit or pride.
- Using AI as fancy autocomplete instead of rethinking the workflow around it.
- Expecting every test to be deterministic. AI features aren’t, and pass/fail thinking alone doesn’t fit them.
02The QA who learned with AI
This generation never stared at a blank spec file. Ask for a Playwright framework with page objects, and it exists before lunch. Ask what a confusing function does, and the answer comes back in seconds.
What that gives them:
- Speed. A first framework in a day instead of a week, and far less fear of an unfamiliar codebase.
- Comfort with AI products. They test chatbots and agents without being thrown by answers that change from run to run.
- Natural use of AI for the boring parts: test data, edge-case brainstorming, refactoring, documentation.
Where they fall short:
- Tests that can’t fail. AI is very good at writing a test that passes. A test that has never failed proves nothing.
- Stuck when the AI is wrong. If the assistant’s fix doesn’t work and its second fix doesn’t either, there’s no debugging habit to fall back on.
- Flakiness hidden by retries. Adding a retry makes the build green, so the real timing bug ships.
- Coverage that looks great on a dashboard while the risky paths stay untested, because nobody asked which paths were risky.
03Side by side
| Situation | Learned before AI | Learned with AI |
|---|---|---|
| Writing a new test | Slower, but every assertion is deliberate | Fast, but assertions need a second look |
| A red build | Reads the trace, forms a theory, checks it | Pastes the error into the assistant |
| A flaky test | Hunts the race condition | Risk of adding a retry and moving on |
| Testing an AI feature | Has to unlearn pure pass/fail | Already thinks in scores and baselines |
| A new codebase | Reads it, slowly | Gets a tour in minutes |
| Deciding what to test | Risk-based, from habit | Whatever the prompt suggested |
These are tendencies, not rules. I’ve met careful engineers in both groups, and careless ones too.
04What each should borrow from the other
If you learned before AI:
- Let the assistant write the boilerplate, and spend the saved hours on test design and exploratory testing.
- Learn to test non-deterministic systems: evaluation metrics, adversarial scenarios, and a baseline the scores can’t drop below.
- Use AI to get through a new codebase fast, then check its explanation against the code.
If you learned with AI:
- Make every new test fail once, on purpose, before you trust it.
- When a build breaks, write down your own guess before asking the assistant. Then compare.
- Learn the fundamentals the assistant leans on: HTTP status codes, the DOM, SQL joins, how waits work.
- Never fix flakiness with a retry. Find out why it’s flaky.
05What I look for when hiring
When I’ve helped with QA hiring, the question that matters isn’t when someone learned. It’s whether they can tell when the AI is wrong. Three questions surface that quickly:
- “Show me a test you wrote that caught a real bug.” It shows whether their tests check things that matter.
- “Make this test fail.” Hand over a passing test with a weak assertion. Good candidates find the weakness fast.
- “How would you test something that answers differently every time?” It shows whether they can move past pure pass/fail.
06Where I stand
The fundamentals I learned the slow way are exactly what let me use AI safely now. I can let it write fast because I can tell when what it wrote is wrong. I wrote more about that workflow in How I test with Claude Code.
The short version: AI made writing tests cheap. It didn’t make judging them cheap. The QA engineers who do best, whichever way they learned, are the ones who still ask: would this test fail if the feature broke?