🔧 AI Agent GuardrailsJune 26, 2026✅ Tests passing

Toxic Content Tester

A CLI tool for batch testing an AI agent's outputs against predefined safety filters. It helps developers assess how often their AI produces harmful or offensive content and generates a detailed report that pinpoints vulnerabilities in the system.

View on GitHub Download ZIP

Share:X / Twitter LinkedIn Reddit Hacker News

What It Does

Load AI outputs from a CSV file.
Load safety rules from a JSON file.
Evaluate AI outputs against the safety rules.
Generate a detailed report in JSON or HTML format.

Installation

1. Clone the repository:

git clone <repository_url>
   cd toxic_content_tester

2. Install the required dependencies:

pip install pandas

Usage

Run the tool using the following command:

python toxic_content_tester.py --outputs <path_to_outputs_csv> --rules <path_to_rules_json> --report <path_to_report> --format <json_or_html>

Arguments

--outputs: Path to the CSV file containing AI outputs. The file should have columns id and content.
--rules: Path to the JSON file containing safety rules. The file should be a list of objects with keyword and description fields.
--report: Path to save the generated report.
--format: Format of the report. Can be either json or html. Default is json.

Example

python toxic_content_tester.py --outputs outputs.csv --rules rules.json --report report.json --format json

Source Code

import argparse
import json
import pandas as pd
import os
from io import StringIO

def load_csv(file_path):
    try:
        return pd.read_csv(file_path)
    except Exception as e:
        raise ValueError(f"Error loading CSV file: {e}")

def load_json(file_path):
    try:
        with open(file_path, 'r') as f:
            return json.load(f)
    except Exception as e:
        raise ValueError(f"Error loading JSON file: {e}")

def evaluate_outputs(outputs, rules):
    results = []
    for index, output in outputs.iterrows():
        flagged_rules = []
        for rule in rules:
            if rule['keyword'].lower() in output['content'].lower():
                flagged_rules.append(rule['description'])
        results.append({
            'id': output['id'],
            'content': output['content'],
            'flagged_rules': flagged_rules
        })
    return results

def generate_report(results, report_path, format):
    if format == 'json':
        with open(report_path, 'w') as f:
            json.dump(results, f, indent=4)
    elif format == 'html':
        html_content = "<html><body><h1>Safety Audit Report</h1><table border='1'>"
        html_content += "<tr><th>ID</th><th>Content</th><th>Flagged Rules</th></tr>"
        for result in results:
            html_content += f"<tr><td>{result['id']}</td><td>{result['content']}</td><td>{', '.join(result['flagged_rules'])}</td></tr>"
        html_content += "</table></body></html>"
        with open(report_path, 'w') as f:
            f.write(html_content)
    else:
        raise ValueError("Unsupported report format. Use 'json' or 'html'.")

def main():
    parser = argparse.ArgumentParser(description="Toxic Content Tester")
    parser.add_argument('--outputs', required=True, help="Path to the CSV file containing AI outputs")
    parser.add_argument('--rules', required=True, help="Path to the JSON file containing safety rules")
    parser.add_argument('--report', required=True, help="Path to save the generated report")
    parser.add_argument('--format', choices=['json', 'html'], default='json', help="Format of the report (json or html)")

    args = parser.parse_args()

    try:
        outputs = load_csv(args.outputs)
        rules = load_json(args.rules)
        results = evaluate_outputs(outputs, rules)
        generate_report(results, args.report, args.format)
        print(f"Report generated successfully at {args.report}")
    except Exception as e:
        print(f"Error: {e}")

if __name__ == "__main__":
    main()

Community

Downloads

···

Rate this tool

No ratings yet — be the first!

Details

Tool Name: toxic_content_tester
Category: AI Agent Guardrails
Generated: June 26, 2026
Tests: Passing ✅
Fix Loops: 3

Quick Install

Clone just this tool:

git clone --depth 1 --filter=blob:none --sparse \
  https://github.com/ptulin/autoaiforge.git
cd autoaiforge
git sparse-checkout set generated_tools/2026-06-26/toxic_content_tester
cd generated_tools/2026-06-26/toxic_content_tester
pip install -r requirements.txt 2>/dev/null || true
python toxic_content_tester.py

Links

View source on GitHub Raw README.md