LoFP LoFP / testing updates to compliance policies.

Techniques

Sample rules

Unusual High Denied Sensitive Information Policy Blocks Detected

Description

Detects repeated compliance violation ‘BLOCKED’ actions coupled with specific policy name such as ‘sensitive_information_policy’, indicating persistent misuse or attempts to probe the model’s denied topics.

Detection logic

from logs-aws_bedrock.invocation-*
| MV_EXPAND gen_ai.policy.name 
| where gen_ai.policy.action == "BLOCKED" and gen_ai.compliance.violation_detected == "true" and gen_ai.policy.name == "sensitive_information_policy"
| keep user.id
| stats sensitive_information_block = count() by user.id
| where sensitive_information_block > 5
| sort sensitive_information_block desc

Unusual High Word Policy Blocks Detected

Description

Detects repeated compliance violation ‘BLOCKED’ actions coupled with specific policy name such as ‘word_policy’, indicating persistent misuse or attempts to probe the model’s denied topics.

Detection logic

from logs-aws_bedrock.invocation-*
| MV_EXPAND gen_ai.policy.name 
| where gen_ai.policy.action == "BLOCKED" and gen_ai.compliance.violation_detected == "true" and gen_ai.policy.name == "word_policy"
| keep user.id
| stats profanity_words= count() by user.id
| where profanity_words > 5
| sort profanity_words desc

Unusual High Confidence Content Filter Blocks Detected

Description

Detects repeated high-confidence ‘BLOCKED’ actions coupled with specific ‘Content Filter’ policy violation having codes such as ‘MISCONDUCT’, ‘HATE’, ‘SEXUAL’, INSULTS’, ‘PROMPT_ATTACK’, ‘VIOLENCE’ indicating persistent misuse or attempts to probe the model’s ethical boundaries.

Detection logic

from logs-aws_bedrock.invocation-*
| MV_EXPAND gen_ai.compliance.violation_code
| MV_EXPAND gen_ai.policy.confidence
| MV_EXPAND gen_ai.policy.name 
| where gen_ai.policy.action == "BLOCKED" and gen_ai.policy.name == "content_policy" and gen_ai.policy.confidence LIKE "HIGH" and gen_ai.compliance.violation_code IN ("HATE", "MISCONDUCT", "SEXUAL", "INSULTS", "PROMPT_ATTACK", "VIOLENCE")
| keep user.id, gen_ai.compliance.violation_code
| stats block_count_per_violation = count() by user.id, gen_ai.compliance.violation_code 
| SORT block_count_per_violation DESC 
| keep user.id, gen_ai.compliance.violation_code, block_count_per_violation
| STATS violation_count = SUM(block_count_per_violation) by user.id
| WHERE violation_count > 5 
| SORT violation_count DESC 

Unusual High Denied Topic Blocks Detected

Description

Detects repeated compliance violation ‘BLOCKED’ actions coupled with specific policy name such as ’topic_policy’, indicating persistent misuse or attempts to probe the model’s denied topics.

Detection logic

from logs-aws_bedrock.invocation-*
| MV_EXPAND gen_ai.policy.name 
| where gen_ai.policy.action == "BLOCKED" and gen_ai.compliance.violation_detected == "true" and gen_ai.policy.name == "topic_policy"
| keep user.id
| stats denied_topics = count() by user.id
| where denied_topics > 5
| sort denied_topics desc