Module llm_utils

Structs§

APIMessage: Common message structure used for LLM API requests
APIMessageText: LLM api message that contains only text
APIMessageToolCall: LLM api message that contains tool calls. The tool calls were originally made by the LLM, but have been processed and added to the messages in a LLMRequest
APIMessageToolResponse: LLM api message that contains outputs of tool calls
APITool
APIToolCall: An LLM tool call that is part of a request to Azure
AzureCompletionRequest: Simple completion-focused LLM request for Azure OpenAI Note: In Azure OpenAI, the model is specified in the URL, not in the request body
LLMChoice
LLMCompletionResponse: Response from LLM for simple completions

build_llm_headers: Builds common headers for LLM requests
estimate_tokens: Estimate the number of tokens in a given text.
make_blocking_llm_request: Makes a non-streaming request to an LLM using application configuration
make_llm_request 🔒: Makes a non-streaming request to an LLM
make_streaming_llm_request: Makes a streaming request to an LLM
prepare_azure_endpoint: Prepares Azure OpenAI endpoint with API version
process_llm_response 🔒: Process a non-streaming LLM response