As we announced in our blog post, Anthropic’s Claude models are now generally available through Amazon Bedrock.

Calling Claude through Bedrock slightly differs from how you would call Claude when using Anthropic’s client SDK’s. This guide will walk you through the process of completing an API call to Claude on Bedrock in Python.

Note that this guide assumes you have already signed up for an AWS account and configured programmatic access.

📘

Messages in Amazon Bedrock

The Amazon Bedrock API currently only supports Text Completions, not Messages. We'll be adding support for Messages soon.

Install and configure the AWS CLI

  1. Install a version of the AWS CLI at or newer than version 2.13.23
  2. Configure your AWS credentials using the AWS configure command (see Configure the AWS CLI) or find your credentials by navigating to “Command line or programmatic access” within your AWS dashboard and following the directions in the popup modal.
  3. Verify that your credentials are working:
    aws sts get-caller-identity
    

Install an SDK for accessing Bedrock

It is also possible to access Bedrock via the AWS CLI but we generally recommend using an official SDK.

pip install anthropic-bedrock
npm install @anthropic-ai/bedrock-sdk
pip install boto3>=1.28.59

Accessing Bedrock

List available models

The following examples show how to print a list of all the Claude models available through Bedrock:

aws bedrock list-foundation-models --by-provider anthropic --query "modelSummaries[*].modelId"
import boto3

bedrock = boto3.client(service_name="bedrock")
response = bedrock.list_foundation_models(byProvider="anthropic")

for summary in response["modelSummaries"]:
    print(summary["modelId"])

Making requests

The following examples shows how to generate text from Claude 2 on Bedrock:

import anthropic_bedrock
from anthropic_bedrock import AnthropicBedrock

client = AnthropicBedrock(
    # Authenticate by either providing the keys below or use the default AWS credential providers, such as
    # using ~/.aws/credentials or the "AWS_SECRET_ACCESS_KEY" and "AWS_ACCESS_KEY_ID" environment variables.
    aws_access_key="<access key>",
    aws_secret_key="<secret key>",
    # Temporary credentials can be used with aws_session_token.
    # Read more at https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html.
    aws_session_token="<session_token>",
    # aws_region changes the aws region to which the request is made. By default, we read AWS_REGION,
    # and if that's not present, we default to us-east-1. Note that we do not read ~/.aws/config for the region.
    aws_region="us-east-2",
)

completion = client.completions.create(
    model="anthropic.claude-v2:1",
    max_tokens_to_sample=256,
    prompt=f"{anthropic_bedrock.HUMAN_PROMPT} Tell me a funny joke about outer space! {anthropic_bedrock.AI_PROMPT}",
)
print(completion.completion)
import AnthropicBedrock from '@anthropic-ai/bedrock-sdk';

const client = new AnthropicBedrock({
  // Authenticate by either providing the keys below or use the default AWS credential providers, such as
  // using ~/.aws/credentials or the "AWS_SECRET_ACCESS_KEY" and "AWS_ACCESS_KEY_ID" environment variables.
  awsAccessKey: '<access key>',
  awsSecretKey: '<secret key>',

  // Temporary credentials can be used with awsSessionToken.
  // Read more at https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html.
  awsSessionToken: '<session_token>',

  // awsRegion changes the aws region to which the request is made. By default, we read AWS_REGION,
  // and if that's not present, we default to us-east-1. Note that we do not read ~/.aws/config for the region.
  awsRegion: 'us-east-2',
});

async function main() {
  const completion = await client.completions.create({
    model: 'anthropic.claude-v2:1',
    max_tokens_to_sample: 256,
    prompt: `${AnthropicBedrock.HUMAN_PROMPT} Tell me a funny joke about outer space! ${AnthropicBedrock.AI_PROMPT}`,
  });
}
main().catch(console.error);
aws bedrock-runtime invoke-model \
  --model-id anthropic.claude-v2:1 \
  --cli-binary-format raw-in-base64-out \
  --body '{"prompt": "Human: Tell me a funny joke about outer space!\n\nAssistant:", "max_tokens_to_sample": 50, "anthropic_version": "bedrock-2023-05-31"}'  \
  /dev/stdout
import boto3
import json

bedrock = boto3.client(service_name="bedrock-runtime")
body = json.dumps(
    {
        "prompt": "\n\nHuman: Tell me a funny joke about outer space\n\nAssistant:",
        "max_tokens_to_sample": 100,
        "anthropic_version": "bedrock-2023-05-31"
    }
)

response = bedrock.invoke_model(body=body, modelId="anthropic.claude-v2:1")

response_body = json.loads(response.get("body").read())
print(response_body.get("completion"))

Making streaming requests

The following examples shows how to generate text from Claude 2 on Bedrock via the streaming interface:

from anthropic_bedrock import AnthropicBedrock, HUMAN_PROMPT, AI_PROMPT

client = AnthropicBedrock()

stream = client.completions.create(
    prompt=f"{HUMAN_PROMPT} Write a very short essay about space travel to Mars{AI_PROMPT}",
    max_tokens_to_sample=300,
    model="anthropic.claude-v2:1",
    stream=True,
)
for completion in stream:
    print(completion.completion, end="", flush=True)
import AnthropicBedrock from '@anthropic-ai/bedrock-sdk';

const client = new AnthropicBedrock();

const stream = await client.completions.create({
  prompt: `${AnthropicBedrock.HUMAN_PROMPT} Write a very short essay about space travel to Mars${AnthropicBedrock.AI_PROMPT}`,
  model: 'anthropic.claude-v2:1',
  stream: true,
  max_tokens_to_sample: 300,
});
for await (const completion of stream) {
  console.log(completion.completion);
}
import boto3
import json

bedrock = boto3.client(service_name="bedrock-runtime")
body = json.dumps(
    {
        "prompt": "\n\nHuman: Write a very short essay about space travel to Mars\n\nAssistant:",
        "max_tokens_to_sample": 200,
        "anthropic_version": "bedrock-2023-05-31"
    }
)
response = bedrock.invoke_model_with_response_stream(
    modelId="anthropic.claude-v2:1", body=body
)

stream = response.get("body")

if stream:
    for event in stream:
        chunk = event.get("chunk")
        if chunk:
            print(json.loads(chunk.get("bytes").decode()))

You can view the official Bedrock docs here.