How to Test an AI Tool Without Using Customer Data
Create realistic fictional examples, including ordinary work and tricky exceptions. Decide what a good answer looks like before running the test and record where the tool gets it wrong.
Make cases realistic without copying records
A cleaning business could invent three customer requests with varying site sizes and access needs. Include one missing a detail the business would need, and see whether the tool asks a useful question.
Compare outcomes side by side
Give the same inputs to each candidate tool and note factual errors, unsupported details and time spent correcting the answer. Try again with a fresh example so you can see whether the result is repeatable.
See the guide to using AI to write a business procedure to see how to prompt the tool and check a draft. For more support, see Gecko Gully’s AI guides.
What you can do today
Test one fictional example that leaves out an important detail.
Frequently asked questions
Can I just change the customer’s name?
Details such as addresses, project descriptions or combinations of facts can still identify someone. A fully fictional example is preferable for early testing.
When can I move to real data?
First confirm the organisation’s permissions, tool settings and data requirements. Start with only the information needed for the approved task.
