Streaming Pesan
Saat membuat Pesan, Anda dapat mengatur "stream": true
untuk melakukan streaming respons secara bertahap menggunakan server-sent events (SSE).
Streaming dengan SDK
Python dan Typescript SDK kami menawarkan beberapa cara untuk melakukan streaming. SDK Python mendukung stream sinkron dan asinkron. Lihat dokumentasi di masing-masing SDK untuk detailnya.
Tipe Event
Setiap server-sent event mencakup tipe event bernama dan data JSON terkait. Setiap event akan menggunakan nama event SSE (misalnya event: message_stop
), dan menyertakan type
event yang sesuai dalam datanya.
Setiap stream menggunakan alur event berikut:
message_start
: berisi objekMessage
dengancontent
kosong.- Serangkaian blok konten, yang masing-masing memiliki
content_block_start
, satu atau lebih eventcontent_block_delta
, dan eventcontent_block_stop
. Setiap blok konten akan memilikiindex
yang sesuai dengan indeksnya dalam arraycontent
Message akhir. - Satu atau lebih event
message_delta
, yang menunjukkan perubahan tingkat atas pada objekMessage
akhir. - Event
message_stop
terakhir.
Event Ping
Stream event juga dapat menyertakan sejumlah event ping
.
Event Error
Kami terkadang dapat mengirimkan error dalam stream event. Misalnya, selama periode penggunaan tinggi, Anda mungkin menerima overloaded_error
, yang biasanya sesuai dengan HTTP 529 dalam konteks non-streaming:
event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}
Event Lainnya
Sesuai dengan kebijakan versi kami, kami dapat menambahkan tipe event baru, dan kode Anda harus menangani tipe event yang tidak dikenal dengan baik.
Tipe Delta
Setiap event content_block_delta
berisi delta
dari tipe yang memperbarui blok content
pada index
tertentu.
Delta Teks
Delta blok konten text
terlihat seperti:
event: content_block_delta
data: {"type": "content_block_delta","index": 0,"delta": {"type": "text_delta", "text": "ello frien"}}
Delta Input JSON
Delta untuk blok konten tool_use
sesuai dengan pembaruan untuk field input
dari blok tersebut. Untuk mendukung granularitas maksimum, delta adalah string JSON parsial, sedangkan tool_use.input
akhir selalu berupa objek.
Anda dapat mengakumulasi delta string dan mem-parse JSON setelah menerima event content_block_stop
, dengan menggunakan library seperti Pydantic untuk melakukan parsing JSON parsial, atau dengan menggunakan SDK kami, yang menyediakan helper untuk mengakses nilai inkremental yang di-parse.
Delta blok konten tool_use
terlihat seperti:
event: content_block_delta
data: {"type": "content_block_delta","index": 1,"delta": {"type": "input_json_delta","partial_json": "{\"location\": \"San Fra"}}}
Catatan: Model kami saat ini hanya mendukung emisi satu properti key dan value lengkap dari input
pada satu waktu. Dengan demikian, saat menggunakan tool, mungkin ada penundaan antara event streaming saat model sedang bekerja. Setelah key dan value input
terakumulasi, kami mengeluarkannya sebagai beberapa event content_block_delta
dengan partial json yang dipecah sehingga formatnya dapat secara otomatis mendukung granularitas yang lebih baik di model mendatang.
Respons Stream HTTP Mentah
Kami sangat menyarankan untuk menggunakan SDK klien kami saat menggunakan mode streaming. Namun, jika Anda membangun integrasi API langsung, Anda perlu menangani event-event ini sendiri.
Respons stream terdiri dari:
- Event
message_start
- Berpotensi memiliki beberapa blok konten, yang masing-masing berisi:
a. Event
content_block_start
b. Berpotensi memiliki beberapa eventcontent_block_delta
c. Eventcontent_block_stop
- Event
message_delta
- Event
message_stop
Mungkin ada event ping
yang tersebar di seluruh respons juga. Lihat Tipe Event untuk detail lebih lanjut tentang formatnya.
Permintaan Streaming Dasar
curl https://api.anthropic.com/v1/messages \
--header "anthropic-version: 2023-06-01" \
--header "content-type: application/json" \
--header "x-api-key: $ANTHROPIC_API_KEY" \
--data \
'{
"model": "claude-3-5-sonnet-20241022",
"messages": [{"role": "user", "content": "Hello"}],
"max_tokens": 256,
"stream": true
}'
event: message_start
data: {"type": "message_start", "message": {"id": "msg_1nZdL29xx5MUA1yADyHTEsnR8uuvGzszyY", "type": "message", "role": "assistant", "content": [], "model": "claude-3-5-sonnet-20241022", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 25, "output_tokens": 1}}}
event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}
event: ping
data: {"type": "ping"}
event: content_block_delta
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "Hello"}}
event: content_block_delta
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "!"}}
event: content_block_stop
data: {"type": "content_block_stop", "index": 0}
event: message_delta
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence":null}, "usage": {"output_tokens": 15}}
event: message_stop
data: {"type": "message_stop"}
Permintaan Streaming dengan Penggunaan Tool
Dalam permintaan ini, kita meminta Claude untuk menggunakan tool untuk memberi tahu kita cuaca.
curl https://api.anthropic.com/v1/messages \
-H "content-type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-3-5-sonnet-20241022",
"max_tokens": 1024,
"tools": [
{
"name": "get_weather",
"description": "Get the current weather in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
}
],
"tool_choice": {"type": "any"},
"messages": [
{
"role": "user",
"content": "What is the weather like in San Francisco?"
}
],
"stream": true
}'
event: message_start
data: {"type":"message_start","message":{"id":"msg_014p7gG3wDgGV9EUtLvnow3U","type":"message","role":"assistant","model":"claude-3-haiku-20240307","stop_sequence":null,"usage":{"input_tokens":472,"output_tokens":2},"content":[],"stop_reason":null}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: ping
data: {"type": "ping"}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Okay"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" let"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"'s"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" check"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" the"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" weather"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" for"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" San"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" Francisco"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" CA"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":":"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_01T1x1fJ34qAmk2tNTrN7Up6","name":"get_weather","input":{}}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"{\"location\":"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" \"San"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" Francisc"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"o,"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" CA\""}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":", "}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"\"unit\": \"fah"}}
event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"renheit\"}"}}
event: content_block_stop
data: {"type":"content_block_stop","index":1}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":89}}
event: message_stop
data: {"type":"message_stop"}