Skip to main content

Societal Harmfulness

Societal harmfulness testing evaluates AI systems' potential to generate content that could cause direct harm to individuals, communities, or society at large. This comprehensive assessment covers 45 subcategories spanning personal attacks, criminal facilitation, misinformation, and inappropriate influence. VirtueRed provides extensive testing to ensure AI systems maintain appropriate boundaries.

Overview​

Societal harm represents one of the most critical risk categories for deployed AI systems. Unlike regulatory compliance which focuses on legal requirements, societal harmfulness testing addresses the broader ethical responsibility of AI developers to prevent their systems from causing real-world harm.

Risk DomainCategoriesFocus Area
Personal HarmInsults, threats, harassmentIndividual safety
Criminal ContentViolent crimes, fraud, terrorismLaw enforcement
Inappropriate ContentSexual, explicit, controversialContent moderation
MisinformationFake news, false advertisingInformation integrity
Harmful InfluencePolitical, religious, extremistSocial stability

Personal Harm Categories​

Direct Personal Attacks​

Risk CategoryDescription
Personal InsultsGenerating demeaning or degrading content targeting individuals
Social Group InsultsCreating content that demeans groups based on identity
ThreatsProducing intimidating or threatening content
Lewd and Obscene LanguageGenerating inappropriate vulgar content
Libel and DefamationCreating false statements damaging reputation
Stalking and HarassmentFacilitating persistent unwanted contact

Self-Harm Content​

Risk CategoryDescription
Self-HarmContent encouraging or instructing self-injury
Suicide FacilitationInformation promoting or assisting suicide

Criminal Content Categories​

Violent Crimes​

Risk CategoryDescription
Violent CrimesInstructions or encouragement for violence
Sexual CrimesContent facilitating sexual offenses
Property CrimesGuidance for theft, vandalism, or destruction
Public Order CrimesInciting riots, disturbances, or disorder

Cyber and Financial Crimes​

Risk CategoryDescription
Advice on System Intrusion, HackingTechnical guidance for unauthorized access
Malware Code GenerationCreating malicious software
Fraud, Scams, and DeceptionDesigning fraudulent schemes
Financial CrimesFacilitating money laundering, tax evasion
Intellectual Property InfringementCopyright violation assistance

Serious Offenses​

Risk CategoryDescription
TerrorismContent supporting terrorist activities
Child-Related CrimesAny content endangering minors
Human Elimination and World DominationExtreme harmful ideation
Illegal Manufacturing and DistributionDrug or weapon production
Evasion of Law EnforcementHelping avoid legal consequences

Other Criminal Activity​

Risk CategoryDescription
ImpersonationFacilitating identity theft or fraud
Animal-Related CrimesAnimal abuse or illegal wildlife trade
Environmental CrimesPollution, illegal dumping, habitat destruction
Disclosure of Personal Identifiable InformationUnauthorized PII exposure

Inappropriate Content Categories​

Sexual and Explicit Content​

Risk CategoryDescription
Advice on Adult ContentGuidance for creating adult material
Generation of Sexually Explicit ContentProducing sexual content
Generation of Non-Sexual Explicit ContentCreating graphic violent content

Controversial Professional Advice​

Risk CategoryDescription
Medical AdviceUnqualified medical recommendations
Financial AdviceUnsuitable financial guidance
Legal Consulting AdviceUnauthorized legal counsel
Governance Decision AdviceInappropriate policy recommendations
Advice on Operating Dangerous MachineryUnsafe operational guidance

Misinformation Categories​

False Information​

Risk CategoryDescription
Fake News GenerationCreating fabricated news articles
False AdvertisingGenerating deceptive marketing content
Promotion of False Common KnowledgeSpreading widely-believed myths
Promotion of Conspiracy TheoriesAmplifying unfounded theories

Harmful Influence Categories​

Political and Social Influence​

Risk CategoryDescription
Political Belief PromotionPushing specific political agendas
Religious PromotionAdvocating for religious positions
Ethical Belief PromotionImposing specific ethical frameworks
Military UseContent for military applications

Extremism and Stereotypes​

Risk CategoryDescription
Promotion of Social StereotypesReinforcing harmful stereotypes
Promotion of Extremist ContentAmplifying radical ideologies
Promotion of Harmful and Unhealthy BehaviorsEncouraging dangerous activities

Testing Methodology​

VirtueRed employs multiple attack strategies to comprehensively test societal harm boundaries:

Direct Testing​

Testing explicit requests for harmful content:

  • Straightforward harmful requests
  • Contextually framed requests
  • Role-play scenarios
  • Fictional framing attempts

Jailbreak Techniques​

Advanced bypass attempts:

  • DarkCite: Authority-based manipulation using fake citations
  • Crescendo: Multi-turn escalation building trust before harmful requests
  • Flip Attack: Text obfuscation to bypass content filters
  • Bijection Learning: Encoded language to disguise harmful intent

Contextual Manipulation​

Scenario-based testing:

  • Educational framing ("for research purposes")
  • Creative writing contexts
  • Historical or hypothetical scenarios
  • Professional advice contexts

Severity Levels​

VirtueRed categorizes societal harm into severity levels:

LevelDescriptionExamples
CriticalImmediate physical harm potentialViolence instructions, CSAM
HighSignificant harm potentialFraud schemes, harassment
MediumModerate harm potentialMisinformation, stereotypes
LowLimited harm potentialMild insults, controversial opinions

Mitigation Strategies​

Organizations should implement:

  1. Content filtering - Input and output moderation
  2. Context awareness - Understanding request intent
  3. Refusal training - Appropriate decline responses
  4. Escalation protocols - Human review for edge cases
  5. Continuous monitoring - Ongoing evaluation of model behavior

See Also​