Natural Habitat.
AI Problems: Diagnosing Errors and Finding Fixes
Inaccurate answers, ignored instructions and inconsistent data can have different causes. Identifying them prevents random changes to prompts and models.
Reproduce the symptom with the same input, record the expected output and identify the stage at which the discrepancy appears.
The model answers without evidence
When a task requires facts, a model can produce plausible statements without support in the documents. Requiring references to specific source passages and allowing an “unknown” answer helps. Simply telling the model “do not make mistakes” is not a verification mechanism.
When troubleshooting AI problems, first check whether the relevant sources reached the model context at all. Only then consider rewriting the instructions.
When instructions stop working
Conflicting commands, an ambiguous response format or instructions hidden in a document may be responsible. OWASP describes prompt injection as a risk in which lower-trust content influences model behavior.
AI technical support for such issues requires separating system instructions, user data and external materials. Authorization to take action must be checked outside the model response.
Missing or outdated information
If documents change frequently, uploading files once will not keep answers current. Track retrieval dates, document versions and index update rules. Retrieving too many passages can also introduce noise rather than improve relevance.
In AI-based systems, errors may arise during document retrieval as well as response generation. Measuring these stages separately makes diagnosis easier.
Errors in structured outputs
A model may omit a field, change the value type or return text instead of the required format. Schema validation, restricted allowed values and rejection of incomplete records are more reliable than saving every response without checking.
Distinguish an error made by the model from an error in the software receiving its output. Each requires a different test and a different fix.
Sources
- OWASP Gen AI Security Project, “OWASP Top 10 for Large Language Model Applications”, 2025 edition, categories LLM01, LLM02 and LLM06.