AI Hallucination

Short answer: In practice, a hallucination is an invented figure, a policy that does not exist, a wrong date or a reference to a document that was never written. It happens because language models write text that sounds likely, and they do not check it unless the system is built to. Hallucinations are a known feature of the technology, so any business use has to plan for them.

The risk is highest when someone asks the model about specific facts it was not given, such as company prices, contract terms or recent events. Hallucinations are hard to spot because they sound as fluent and confident as correct answers. The business impact depends on where the output goes. A wrong suggestion in an internal first draft is cheap to fix. A wrong answer sent to a customer, entered into a business system or used in a compliance decision can be expensive. Good design lowers the rate, but every business use should expect some wrong answers.

Several practices reduce hallucinations. Give the model the relevant source material instead of relying on what it learned in training. This is what retrieval-augmented generation does. Tell the model to answer only from that material and to say when information is missing. Ask it to show its sources, so a reviewer can check each claim. Use fixed answer formats and automatic checks, such as confirming that an extracted total matches the line items. Test on real cases and track the error rate over time. Finally, put a person in the review step wherever a mistake would be costly.

A company tests an AI assistant on 50 questions about its own returns policy. Without the policy, the model answers 31 correctly and invents return periods for the other 19. The team then gives it the policy document and tells it to answer only from it and to name the section it used. Now 46 answers are correct, 3 say the information is missing and 1 is wrong. Because of that remaining error, the company adds a review step for refund questions.

The risk of hallucination decides where AI output needs checking before anyone uses it. The process owner sets the level of human checking for each workflow. They base it on three things: what a wrong answer would cost, the error rate measured on real cases and which controls catch errors before they cause harm.

An overall error rate from testing can hide much higher rates on rare or unusual questions, so break the results down by type of question. Sources can be wrong too, because a model can point to the wrong document, so spot checks are still needed. Where output affects customers, contracts, financial records or health, the company remains accountable for errors.