Skip to main content

conversation_prompt_cache_key

headless_lms_chatbot::azure_chatbot::request

Function conversation_prompt_cache_key 

Source 
fn conversation_prompt_cache_key(
    model_type: &ModelType,
    conversation_id: Uuid,
) -> Option<String>
Expand description

Routes every request of one conversation to the same prompt cache, or None for a model that has no Azure prompt cache.

The transcript is append-only, so a turn’s prefix contains the previous turn’s, and the entry worth routing to is the one that turn wrote.