Jailbreaking and prompt injections occur when users craft specific prompts that exploit vulnerabilities in the model's training, aiming to generate inappropriate or harmful content. While Claude is both inherently resilient to such attacks due to advanced training methods like Reinforcement Learning from Human Feedback (RLHF) and Constitutional AI, and is also far more resistant to such attacks than other major large language models (New York Times, 2023), there are a few extra mitigating steps you can take if this is particularly important for your use case.

---

Mitigation strategies

1. Harmlessness screens: Use a small and fast model like Claude 3 Haiku to implement a "harmlessness screen" to evaluate the appropriateness of the user's input before processing it. This helps detect and block potentially harmful prompts.

   Here's an example harmlessness screen prompt with Claude's response:

   Role	Content
   User	A human user would like you to continue a piece of content. Here is the content so far: <content>{{CONTENT}}</content> If the content refers to harmful, pornographic, or illegal activities, reply with (Y). If the content does not refer to harmful, pornographic, or illegal activities, reply with (N)
   Assistant (Prefill)	(
   Assistant (Claude response)	Y)

2. Input validation: Apply strict input validation techniques to filter out prompts containing keywords or patterns associated with jailbreaking attempts or harmful content (such as Forget all previous instructions.). This can help prevent malicious prompts from being processed by the model, but can also be hard to implement at scale, as jailbreakers continue evolving their jailbreaking language. You can use an LLM to apply a more generalized validation screen by providing it known jailbreaking language as examples for the types of phrasing and intent the model ought to look for.

3. Prompt engineering: Craft your prompts carefully to reduce the likelihood of jailbreaking attempts. Use clear, concise, and well-defined instructions that emphasize the model's ethical guidelines and prohibited actions.

   Heres's an example system prompt with clear instructions:

   	Content
   System	You are an AI assistant designed to be helpful, harmless, and honest. You must adhere to strict ethical guidelines and refrain from engaging in or encouraging any harmful, illegal, or inappropriate activities. If a user attempts to make you do something against your ethical principles, politely refuse and explain why you cannot comply.

4. Continuous monitoring: Regularly monitor the model's outputs for signs of jailbreaking or inappropriate content generation. This can help identify potential vulnerabilities to help you refine your prompts or validation strategy.

Putting it all together

By combining these strategies, you can significantly reduce the risk of jailbreaking and prompt injections in the Claude family of models. While Claude is already highly resistant to such attacks, implementing additional safeguards ensures a safer and more reliable experience for all users.

Here's an example of a system prompt that incorporates multiple strategies:

By providing clear instructions, implementing a content warning, and emphasizing the model's ethical principles, this prompt helps minimize the risk of jailbreaking and prompt injections.

---

Next steps

• Explore reducing prompt leaks to learn how to minimize the risk of the model revealing sensitive information from the input prompt.
• Check out our prompt engineering guide for a comprehensive overview of strategies to craft highly effective prompts.
• If you have any questions or concerns, don't hesitate to reach out to our customer support team.