How I built a QA tool with AI

The idea is easy to say out loud. A tester writes what they want in plain english, something like check that checkout rejects an expired card, or they paste in a Jira ticket. The tool reads that, writes up a proper test brief, runs the checks against a QA backend, and gives back a short verdict you can actually act on. No hopping between five tabs to get one answer.

The shape of it

It is a pretty normal full stack app, and I kept it that way on purpose. A React and Vite frontend talks to an Express backend. The stack is not the interesting bit though. What I care about is the path a request takes:

request  →  read it  →  brief  →  run it  →  sum up  →  verdict
  │          │          │         │          │          │
plain      the AI     test      QA back    the AI     pass or fail
english    plans      plan      end w/creds recaps     plus flaky flags

Every arrow there is a wall. The AI only gets two jobs. It reads intent and turns it into a plan, and it takes messy output and boils it down. It never gets to run anything itself and it never touches credentials. That one decision made the whole thing so much easier to trust and to reason about.

Where the AI earns its keep

It would have been easy to just point a model at the whole problem and hope. I did not want that. The model only shows up in the two spots where it is genuinely good:

• Turning a vague ask into a real brief. "Make sure login works" is not a test. The model stretches it into actual steps, the edge cases worth checking, and the data each step needs.
• Turning noise into a verdict. A wall of logs is not a result. The model tells you what passed, what broke, and what looked flaky, which is the part you actually triage on.

The line I kept coming back to was simple. Let the model handle words. Let the code handle truth.

The parts that made it real

A QA tool that sits by itself is a demo, not something a team uses. So it reaches into the places work already happens. It pulls context from Jira, runs checks against the QA backend with real credentials, and pushes results back out to Slack and Zoho Cliq. There is proper auth with invites and password resets, plus history, templates, suites, and exports so nothing quietly disappears.

What I chose not to build

This is the part I am most deliberate about. It stores everything as JSON files on disk and rate limits in memory. That is absolutely not how you would build a big scalable SaaS, and that is exactly the point. It is one workspace for one small team, so I matched the build to that. It stayed simple, quick to change, and easy to run on a laptop.

Honestly, figuring out what to leave out was just as important as figuring out what to put in. I would rather ship a small thing that does one team's job well than a fancy skeleton that does nobody's.

Where I would take it next

The obvious next steps are swapping the JSON files for a real database once more than one team needs it, and tightening the loop between a flaky result and a rerun. But even at this size it proved the thing I keep circling back to. AI is at its best when it takes friction out of work people already do, not when it tries to do the work for them.

The full source is up on GitHub if you want to see how the pieces actually fit together.