在建立訊息時,您可以設置 "stream": true 來使用 server-sent events (SSE) 逐步流式傳輸回應。

使用 SDK 進行流式傳輸

我們的 PythonTypescript SDK 提供多種流式傳輸方式。Python SDK 允許同步和異步流。詳情請參閱每個 SDK 的文檔。

import anthropic

client = anthropic.Anthropic()

with client.messages.stream(
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
    model="claude-3-opus-20240229",
) as stream:
  for text in stream.text_stream:
      print(text, end="", flush=True)

事件類型

每個 server-sent 事件都包含一個命名的事件類型和相關的 JSON 數據。每個事件都將使用 SSE 事件名稱(例如 event: message_stop),並在其數據中包含匹配的事件 type

每個流使用以下事件流程:

  1. message_start:包含一個 Message 對象,其中 content 為空。
  2. 一系列內容塊,每個內容塊都有一個 content_block_start、一個或多個 content_block_delta 事件以及一個 content_block_stop 事件。每個內容塊都有一個 index,對應於其在最終 Message content 數組中的索引。
  3. 一個或多個 message_delta 事件,表示對最終 Message 對象的頂層更改。
  4. 最後一個 message_stop 事件。

Ping 事件

事件流還可能包含任意數量的 ping 事件。

錯誤事件

我們偶爾可能會在事件流中發送錯誤。例如,在使用量高的時期,您可能會收到一個 overloaded_error,這通常對應於非流式上下文中的 HTTP 529:

錯誤示例
event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}

其他事件

根據我們的版本控制策略,我們可能會添加新的事件類型,您的代碼應該優雅地處理未知的事件類型。

Delta 類型

每個 content_block_delta 事件都包含一個 delta,其類型更新給定 index 處的 content 塊。

文本 delta

text 內容塊 delta 如下所示:

文本 delta
event: content_block_delta
data: {"type": "content_block_delta","index": 0,"delta": {"type": "text_delta", "text": "ello frien"}}

輸入 JSON delta

輸入 JSON delta 塊是 tool use beta 的一部分。

tool_use 內容塊的 delta 對應於塊的 input 字段的更新。為了支持最大的粒度,delta 是 部分 JSON 字符串,而最終的 tool_use.input 始終是一個 對象

您可以累積字符串 delta,並在收到 content_block_stop 事件時使用像 Pydantic 這樣的庫進行部分 JSON 解析,或者使用我們的 SDK,它們提供了訪問解析後的增量值的幫助程序。

tool_use 內容塊 delta 如下所示:

輸入 JSON delta
event: content_block_delta
data: {"type": "content_block_delta","index": 1,"delta": {"type": "input_json_delta","partial_json": "{\"location\": \"San Fra"}}}

注意:我們當前的模型只支持一次發出 input 的一個完整的鍵和值屬性。因此,在使用工具時,流式事件之間可能會有延遲,因為模型正在工作。一旦累積了一個 input 鍵和值,我們就會將它們作為多個 content_block_delta 事件發出,其中包含分塊的部分 json,以便該格式可以自動支持未來模型中更細粒度的內容。

原始 HTTP 流響應

我們強烈建議在使用流式模式時使用我們的客戶端 SDK。但是,如果您正在構建直接的 API 集成,則需要自己處理這些事件。

流響應由以下部分組成:

  1. 一個 message_start 事件
  2. 可能有多個內容塊,每個內容塊包含: a. 一個 content_block_start 事件 b. 可能有多個 content_block_delta 事件 c. 一個 content_block_stop 事件
  3. 一個 message_delta 事件
  4. 一個 message_stop 事件

響應中可能還會分散 ping 事件。有關格式的更多詳細信息,請參閱事件類型

基本流式請求

請求
curl https://api.anthropic.com/v1/messages \
     --header "anthropic-version: 2023-06-01" \
     --header "anthropic-beta: messages-2023-12-15" \
     --header "content-type: application/json" \
     --header "x-api-key: $ANTHROPIC_API_KEY" \
     --data \
'{
  "model": "claude-3-opus-20240229",
  "messages": [{"role": "user", "content": "Hello"}],
  "max_tokens": 256,
  "stream": true
}'
響應
event: message_start
data: {"type": "message_start", "message": {"id": "msg_1nZdL29xx5MUA1yADyHTEsnR8uuvGzszyY", "type": "message", "role": "assistant", "content": [], "model": "claude-3-opus-20240229", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 25, "output_tokens": 1}}}

event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}

event: ping
data: {"type": "ping"}

event: content_block_delta
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "Hello"}}

event: content_block_delta
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "!"}}

event: content_block_stop
data: {"type": "content_block_stop", "index": 0}

event: message_delta
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence":null}, "usage": {"output_tokens": 15}}

event: message_stop
data: {"type": "message_stop"}

使用工具的流式請求

在此請求中,我們要求 Claude 使用工具告訴我們天氣。

請求
  curl https://api.anthropic.com/v1/messages \
    -H "content-type: application/json" \
    -H "x-api-key: $ANTHROPIC_API_KEY" \
    -H "anthropic-version: 2023-06-01" \
    -H "anthropic-beta: tools-2024-05-16" \
    -d '{
      "model": "claude-3-opus-20240229",
      "max_tokens": 1024,
      "tools": [
        {
          "name": "get_weather",
          "description": "Get the current weather in a given location",
          "input_schema": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA"
              }
            },
            "required": ["location"]
          }
        }
      ],
      "tool_choice": {"type": "any"},
      "messages": [
        {
          "role": "user",
          "content": "What is the weather like in San Francisco?"
        }
      ],
      "stream": true
    }'
響應
event: message_start
data: {"type":"message_start","message":{"id":"msg_014p7gG3wDgGV9EUtLvnow3U","type":"message","role":"assistant","model":"claude-3-haiku-20240307","stop_sequence":null,"usage":{"input_tokens":472,"output_tokens":2},"content":[],"stop_reason":null}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: ping
data: {"type": "ping"}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Okay"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" let"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"'s"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" check"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" the"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" weather"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" for"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" San"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" Francisco"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" CA"}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":":"}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_01T1x1fJ34qAmk2tNTrN7Up6","name":"get_weather","input":{}}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"{\"location\":"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" \"San"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" Francisc"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"o,"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" CA\""}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":", "}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"\"unit\": \"fah"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"renheit\"}"}}

event: content_block_stop
data: {"type":"content_block_stop","index":1}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":89}}

event: message_stop
data: {"type":"message_stop"}