Back to AI techniques. What to do when the source is silent and AI answers anyway. Here is where the human in the loop comes in.
A fabricated citation is the easy failure to catch, once you know to look. Ask for the exact quote, it doesn’t match, you’ve caught it. The harder failure is when the source is real, the quote is accurate — and the specific question you asked isn’t actually answered by it. AI fills the gap with the most reasonable-sounding interpretation and hands it to you with the same confident tone as a settled answer. There’s no fabrication to catch. Just a judgment call wearing the outfit of a citation.
This is the one that gets past “quote it exactly,” because the quote is real. The problem isn’t the citation — it’s the distance between what the source actually says and what you asked.
Before you close out a verified answer, ask one more question:
“Does the source directly answer this, or are you interpreting it? If you’re interpreting, say what assumption you’re making and what a different reasonable interpretation would be.”
That question forces AI to separate three things that a confident answer blends together: what the source states outright, what AI inferred from it, and where someone could reasonably land somewhere else. Only the first one is verified. The second and third are exactly where you want to spend your own judgment — not because AI got it wrong, but because there wasn’t a single right answer to get.
A citation tells you AI found something real. It doesn’t tell you the real thing settles your question. Knowing the difference is what separates a checked answer from a defensible one.
About the Author
Russ Hissom, CPA is a principal of UtilityEducation.com, a firm that provides power and utilities rate and expert witness services, and on-demand professional education classes in co-op and utility accounting, finance, ratemaking, artificial intelligence, and management.
Russ was a partner in a national accounting and consulting firm for 20 years. He works with electric investor-owned and public power utilities, electric cooperatives, and gas, water, and wastewater utilities. His goal is to share industry best practices to help your business perform effectively and efficiently and meet the challenges of the changing power and utilities industry.
Contact Russ at [email protected]
The material in this article is for informational purposes only and should not be taken as legal or accounting advice provided by Utility Accounting & Rates Specialists, LLC or UtilityEducation.com. You should seek formal advice on this topic from your accounting or legal advisor.