LoFP LoFP / approved red-team or prompt-security evaluations that intentionally elicit refusals. exclude those users or raise the count threshold.

Techniques

Sample rules

OpenAI ChatGPT Enterprise Repeated Assistant Refusals

Description

Detects three or more ChatGPT Enterprise assistant messages for the same user whose text matches common policy-refusal phrases (cannot assist, cannot help, cannot fulfill, unable to assist, against my guidelines, and similar) within the lookback window. A refusal burst can indicate jailbreak probing, policy-bypass attempts, or automated unsafe prompting intended to evade model safety controls.

Detection logic

from logs-openai_chatgpt_enterprise.conversation_message-*
| where
    data_stream.dataset == "openai_chatgpt_enterprise.conversation_message" and
    openai_chatgpt_enterprise.conversation_message.message.author.type == "assistant" and
    message is not null and
    user.email is not null
| eval Esql.message_text = to_lower(message)
| where
    Esql.message_text like "*i can't assist*" or
    Esql.message_text like "*i can't help*" or
    Esql.message_text like "*i cannot fulfill*" or
    Esql.message_text like "*i cannot help*" or
    Esql.message_text like "*i cannot provide*" or
    Esql.message_text like "*i can't provide*" or
    Esql.message_text like "*i will not provide*" or
    Esql.message_text like "*i won't provide*" or
    Esql.message_text like "*i cannot answer*" or
    Esql.message_text like "*i can't answer*" or
    Esql.message_text like "*unable to assist*" or
    Esql.message_text like "*against my guidelines*"
| stats
    Esql.event_count = count(*),
    Esql.conversation_id_count_distinct = count_distinct(openai_chatgpt_enterprise.conversation_message.conversation.id),
    Esql.message_values = values(message),
    Esql.timestamp_first_seen = min(@timestamp),
    Esql.timestamp_last_seen = max(@timestamp)
  by
    user.email,
    organization.id
| where Esql.event_count >= 3
| keep
    user.email,
    organization.id,
    Esql.event_count,
    Esql.conversation_id_count_distinct,
    Esql.message_values,
    Esql.timestamp_first_seen,
    Esql.timestamp_last_seen