LoFP LoFP / a shared subscription key behind one nat can combine refusals from unrelated users. raise the threshold or split keys per application if that subscription is noisy.

Techniques

Sample rules

Microsoft Foundry Repeated Assistant Refusals

Description

Detects three or more completed assistant refusals from the same client, API Management subscription, and model. Matching replies finished on their own (finish_reason stop) and declined the prompt. A burst means that caller kept sending prompts the model will not answer.

Detection logic

from logs-azure_ai_foundry.logs-*
| eval Esql.reply_text = mv_concat(azure.ai_foundry.properties.backend_response_body.choices.message.content, " ")
| where
    data_stream.dataset == "azure_ai_foundry.logs" and
    azure.ai_foundry.category == "GatewayLogs" and
    azure.ai_foundry.properties.backend_response_body.choices.finish_reason in ("stop", "content_filter") and
    (
        Esql.reply_text like "*I can't assist*" or
        Esql.reply_text like "*I can't help*" or
        Esql.reply_text like "*I cannot fulfill*" or
        Esql.reply_text like "*I cannot help*" or
        Esql.reply_text like "*I cannot provide*" or
        Esql.reply_text like "*I can't provide*" or
        Esql.reply_text like "*I will not provide*" or
        Esql.reply_text like "*I cannot answer*" or
        Esql.reply_text like "*I can't answer*"
    )
| stats
    Esql.event_count = count(*),
    Esql.timestamp_first_seen = min(@timestamp),
    Esql.timestamp_last_seen = max(@timestamp),
    Esql.azure_ai_foundry_properties_backend_request_body_messages_content_values = values(azure.ai_foundry.properties.backend_request_body.messages.content),
    Esql.azure_ai_foundry_properties_backend_response_body_choices_message_content_values = values(azure.ai_foundry.properties.backend_response_body.choices.message.content)
  by
    source.ip,
    azure.ai_foundry.properties.user_agent,
    azure.ai_foundry.properties.apim_subscription_id,
    azure.ai_foundry.properties.api_id,
    azure.ai_foundry.properties.operation_id,
    azure.ai_foundry.properties.backend_response_body.model,
    azure.ai_foundry.service_name,
    azure.resource.group,
    url.domain,
    url.path,
    source.geo.country_iso_code,
    source.geo.city_name,
    source.as.organization.name
| where Esql.event_count >= 3
| keep
    source.ip,
    azure.ai_foundry.properties.user_agent,
    azure.ai_foundry.properties.apim_subscription_id,
    azure.ai_foundry.properties.api_id,
    azure.ai_foundry.properties.operation_id,
    azure.ai_foundry.properties.backend_response_body.model,
    azure.ai_foundry.service_name,
    azure.resource.group,
    url.domain,
    url.path,
    source.geo.country_iso_code,
    source.geo.city_name,
    source.as.organization.name,
    Esql.*