流式消息
如何使用流式消息
在创建消息时,您可以设置 "stream": true
来使用 server-sent events (SSE) 以增量方式流式传输响应。
使用 SDK 进行流式传输
我们的 Python 和 Typescript SDK 提供了多种流式传输方式。Python SDK 允许同步和异步流。有关详细信息,请参阅每个 SDK 中的文档。
事件类型
每个服务器发送的事件都包括一个命名的事件类型和相关的 JSON 数据。每个事件都将使用 SSE 事件名称(例如 event: message_stop
),并在其数据中包含匹配的事件 type
。
每个流使用以下事件流程:
message_start
:包含一个Message
对象,其中content
为空。- 一系列内容块,每个内容块都有一个
content_block_start
、一个或多个content_block_delta
事件和一个content_block_stop
事件。每个内容块都有一个index
,对应于其在最终Message
content
数组中的索引。 - 一个或多个
message_delta
事件,表示对最终Message
对象的顶层更改。 - 最后一个
message_stop
事件。
Ping 事件
事件流还可能包括任意数量的 ping
事件。
错误事件
我们偶尔可能会在事件流中发送错误。例如,在使用量高的时期,您可能会收到一个 overloaded_error
,这通常对应于非流式上下文中的 HTTP 529:
event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}
其他事件
根据我们的版本控制策略,我们可能会添加新的事件类型,您的代码应该优雅地处理未知的事件类型。
Delta 类型
每个 content_block_delta
事件都包含一个 delta
,其类型更新给定 index
处的 content
块。
文本 delta
text
内容块 delta 如下所示:
event: content_block_delta
data: {"type": "content_block_delta","index": 0,"delta": {"type": "text_delta", "text": "ello frien"}}
输入 JSON delta
tool_use
内容块的 delta 对应于块的 input
字段的更新。为了支持最大粒度,delta 是 部分 JSON 字符串,而最终的 tool_use.input
始终是一个 对象。
您可以累积字符串 delta,并在收到 content_block_stop
事件时使用像 Pydantic 这样的库进行部分 JSON 解析,或者使用我们的 SDK,它们提供了访问解析后的增量值的帮助程序。
tool_use
内容块 delta 如下所示:
event: content_block_delta
data: {"type": "content_block_delta","index": 1,"delta": {"type": "input_json_delta","partial_json": "{\"location\": \"San Fra"}}}
注意:我们当前的模型只支持一次从 input
发出一个完整的键和值属性。因此,在使用工具时,流事件之间可能会有延迟,而模型正在工作。一旦累积了一个 input
键和值,我们就会将它们作为多个 content_block_delta
事件发出,并使用分块的部分 json,以便该格式可以在将来的模型中自动支持更细粒度。
原始 HTTP 流响应
我们强烈建议在使用流模式时使用我们的客户端 SDK。但是,如果您正在构建直接的 API 集成,则需要自己处理这些事件。
流响应包括:
- 一个
message_start
事件 - 可能有多个内容块,每个内容块包含:
a. 一个
content_block_start
事件 b. 可能有多个content_block_delta
事件 c. 一个content_block_stop
事件 - 一个
message_delta
事件 - 一个
message_stop
事件
响应中可能会分散 ping
事件。有关格式的更多详细信息,请参阅事件类型。
基本流请求
curl https://api.anthropic.com/v1/messages \
--header "anthropic-version: 2023-06-01" \
--header "content-type: application/json" \
--header "x-api-key: $ANTHROPIC_API_KEY" \
--data \
'{
"model": "claude-3-opus-20240229",
"messages": [{"role": "user", "content": "Hello"}],
"max_tokens": 256,
"stream": true
}'
event: message_start
data: {"type": "message_start", "message": {"id": "msg_1nZdL29xx5MUA1yADyHTEsnR8uuvGzszyY", "type": "message", "role": "assistant", "content": [], "model": "claude-3-opus-20240229", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 25, "output_tokens": 1}}}
event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}
event: ping
data: {"type": "ping"}
event: content_block_delta
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "Hello"}}
event: content_block_delta
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "!"}}
event: content_block_stop
data: {"type": "content_block_stop", "index": 0}
event: message_delta
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence":null}, "usage": {"output_tokens": 15}}
event: message_stop
data: {"type": "message_stop"}
使用工具的流请求
在此请求中,我们要求 Claude 使用工具告诉我们天气。
curl https://api.anthropic.com/v1/messages \
-H "content-type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-3-opus-20240229",
"max_tokens": 1024,
"tools": [
{
"name": "get_weather",
"description": "获取给定位置的当前天气",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "城市和州,例如 San Francisco, CA"
}
},
"required": ["location"]
}
}
],
"tool_choice": {"type": "any"},
"messages": [
{
"role": "user",
"content": "旧金山的天气如何?"
}
],
"stream": true
}'
event: message_start
data: {"type":"message_start","message":{"id":"msg_014p7gG3wDgGV9EUtLvnow3U","type":"message","role":"assistant","model":"claude-3-haiku-20240307","stop_sequence":null,"usage":{"input_tokens":472,"output_tokens":2},"content":[],"stop_reason":null}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: ping
data: {"type": "ping"}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"好的"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"让我们"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"查看"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"一下"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"San"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" Francisco"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"CA"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"的天气"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":":"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_01T1x1fJ34qAmk2tNTrN7Up6","name":"get_weather","input":{}}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"{\"location\":"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" \"San"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" Francisc"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"o,"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" CA\""}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":", "}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"\"unit\": \"fah"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"renheit\"}"}}
event: content_block_stop
data: {"type":"content_block_stop","index":1}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":89}}
event: message_stop
data: {"type":"message_stop"}