Skip to main content

stream_turn

Function stream_turn 

Source
fn stream_turn(
    pool: PgPool,
    app_config: ApplicationConfiguration,
    conversation_id: Uuid,
    chat_request: LLMRequest,
    user_context: ChatbotTurnContext,
) -> Pin<Box<dyn Stream<Item = ChatbotResult<Bytes>> + Send>>
Expand description

Runs the request rounds of one turn against the LLM and streams its events as NDJSON.

Keeps asking the LLM as long as a round ends in tool calls it answered itself, and ends the turn on a text answer, an error, a suspension, or the iteration limit. Owns the cancellation guard, so a client that disappears mid-turn still gets what arrived saved.