Minimizing hallucinations
While Claude is incredibly powerful and versatile, it can sometimes generate text that is factually incorrect, inconsistent, or irrelevant to the given context. This phenomenon is known as "hallucination" and can occur when the model tries to fill in gaps in its knowledge or when the input is ambiguous.
Let's explore various strategies to minimize hallucinations and improve the accuracy and reliability of Claude's outputs. By implementing these techniques, you can ensure a better user experience and more trustworthy results.
Allow Claude to say "I don't know"
One effective way to reduce hallucinations is to explicitly give Claude permission to say "I don't know," especially when asking fact-based questions (also known as "giving Claude an out"). This allows Claude to acknowledge its limitations and avoid generating incorrect information.
Here's an example prompt that encourages Claude to admit when it doesn't have the answer:
| Content | |
|---|---|
| System | Please answer the following question to the best of your ability. If you are unsure or don't have enough information to provide a confident answer, simply say "I don't know" or "I'm not sure." |
| User | What is the capital of Burkina Faso? |
By giving Claude an explicit "way out," you can reduce the likelihood of it generating inaccurate information.
Reminder
While Claude has read a lot on the internet and knows things about the real world, it does not have internet access. Claude was trained on data that can be over a year out of date. It also does not know anything about current events.
Ask Claude for direct quotes
When working with long documents, asking Claude to extract word-for-word quotes relevant to a specific question can help minimize hallucinations. This approach is more effective for longer documents (>300 words) and may be less reliable for shorter ones.
Here's an example prompt that requests direct quotes:
| Role | Content |
|---|---|
| User | Consider the following document: <document> {{DOCUMENT}} <document> Please extract word-for-word quotes from this document that are relevant to the question <question>{{QUESTION}}</question>. If there are no quotes in this document that seem relevant to this question, say "I can't find any relevant quotes." |
By requesting direct quotes, you can more easily verify the accuracy of the information provided by Claude (and furthermore get Claude to use quotes for inline citations). You can also perform string matching or calculate the percentage of overlap between the quotes and the original document to ensure the model is not hallucinating.
Quote extraction accuracy note
You might not get 100% reproduction of the original document text when asking for quotes, but you want fidelity to be high. For example, the model might add "[sic.]" if there is an error in the document, or add context to quotes like
he [Brian] asked her [Diana] to dinner, which is fine as long as the added content is accurate.If you think Claude is adding inaccurate content, you may want to filter for a very high degree of overlap and make the instructions more rigorous, e.g., by adding something like
Please ensure your quotes are directly taken verbatim from the document. Do not add any additional content like disambiguations or comments.
You can further utilize the direct quotes that Claude extracts by telling Claude to answer only if there is relevant information found on those quotes, or to ground its answer only in the information found in the extracted quotes. See this example citation-based prompt from our prompt library:
| Content | |
|---|---|
| System | You are an expert research assistant. Here is a document you will answer questions about: <document> {{DOCUMENT}} </document> First, find the quotes from the document that are most relevant to answering the question, and then print them in numbered order. Quotes should be relatively short. If there are no relevant quotes, write "No relevant quotes" instead. Then, answer the question, starting with "Answer:". Do not include or reference quoted content verbatim in the answer. Don't say "According to Quote [1]" when answering. Instead make references to quotes relevant to each section of the answer solely by adding their bracketed numbers at the end of relevant sentences. Thus, the format of your overall response should look like what's shown between the <example></example> tags. Make sure to follow the formatting and spacing exactly. <example> Quotes: [1] "Company X reported revenue of $12 million in 2021." [2] "Almost 90% of revene came from widget sales, with gadget sales making up the remaining 10%." Answer: Company X earned $12 million. [1] Almost 90% of it was from widget sales. [2] </example> If the question cannot be answered by the document, say so. |
| User | {{QUESTION}} |
Other strategies to minimize hallucinations
Here are some additional techniques to help reduce hallucinations:
- Tell Claude to answer only if it is very confident in its response.
- Give Claude room to think before responding.
- Ask Claude to generate multiple outputs and check for consistency across the responses. If there are inconsistencies, it may indicate that one or more of the outputs contain hallucinations.
By implementing these strategies, you can significantly reduce the occurrence of hallucinations and improve the overall reliability of Claude's outputs. Remember to test these techniques with your specific use case to ensure they meet your needs and expectations.
If you continue to encounter issues or have questions, don't hesitate to reach out to our customer support team.
Updated about 2 months ago