Plenty of mid-sized companies now have an AI search tool pointed at their own documents: contracts, policies, price books, work instructions, supplier files. The demo goes well. Then someone in finance asks it a real question, gets a confident answer from an outdated document, and trust drops fast.
The fix is not a better model. It is a test set, and a person who owns it.
What a test set is
A test set is a fixed list of real questions your staff ask, each paired with the document or passage that holds the right answer. You run the list against the tool, score the results, change something, and run the list again. Because the questions never change, you can tell whether the tool got better or worse.
Without one, quality is judged by whoever tried the tool last. That produces opinions, not decisions.
Why the retrieval step deserves the attention
Most internal AI assistants work in two steps. They first find the passages that look relevant, then write an answer from them. If the first step pulls the wrong page, the second step writes a fluent answer from the wrong material.
That is why testing only the final wording misses the problem. You need to check what the system found, not just what it said.
What a good question set looks like
- Drawn from real work. Pull questions from help desk tickets, email threads and the queries your team types into the current search bar.
- Spread across departments. Include finance, operations, HR, sales and the plant floor, not just the area your sponsor cares about.
- Hard on purpose. Add questions where two documents disagree, where a policy changed last year, or where the answer sits in a table inside a long PDF.
- Marked with a correct answer. Each question needs a source and an expected result agreed by someone who knows the subject.
- Reviewed on a schedule. Documents change, so the set must change with them.
A set of a few dozen questions is enough to start. Several hundred is realistic once more than one department depends on the tool. Building it takes weeks of steady effort, not an afternoon.
The people this work needs
This is where most projects stall. The skills are split across several roles, and few companies have all of them in one seat.
| Role | What they do | Where they come from |
|---|---|---|
| Subject matter reviewer | Writes questions and confirms right answers | Finance, operations or technical staff |
| Data or AI analyst | Runs tests, scores results, reports patterns | AI and data hiring |
| Engineer or developer | Fixes indexing, chunking and ranking issues | Software or data engineering |
| Project owner | Sets priorities and keeps the cycle going | Operations leadership |
The subject matter reviewer is the hardest to find and the one most often overlooked. A controller who can say whether an answer about revenue recognition is correct is worth more to this project than another generalist developer. For a short, defined build, contract talent often fits better than a permanent hire, since the heavy question-writing phase ends.
How to run it as a routine
Treat testing as a recurring job, not a launch gate.
- Build the first question set before you expand access to the tool.
- Score every run the same way and keep the results.
- Sort failures into groups: wrong document found, right document but wrong passage, outdated source, missing source.
- Hand each group to the person who can fix it. Some fixes belong to engineering. Others belong to whoever owns the documents.
- Re-run the full set after each change, and again whenever a major batch of documents is added.
Many failures turn out to be content problems. Duplicate policies, old versions left in the folder and inconsistent naming cause more bad answers than the software does.
Signs you are not ready to roll out wider
- No one can say how often the tool is right.
- Complaints are the only feedback you receive.
- Nobody owns the source documents.
- The person who built the pilot is also the only person who tests it.
If two or more of those are true, hold the wider rollout and build the test set first. It costs far less than rebuilding trust after a bad answer reaches a customer or an auditor.
What to do next
Pick one department and write 30 real questions with agreed answers this month. Run them through your tool and see what comes back. The result will tell you which roles you are missing, whether that is a reviewer with deep subject knowledge, a data analyst or an engineer.
If you need AI and data talent, contract or permanent, to build and run that testing, a Traverse recruiter can help with this.