๐ฌ LLM Vulnerability DetectionMay 28, 2026โ
Tests passing
Sandboxed Vulnerability Playground
This tool creates a local sandboxed environment where LLMs simulate security vulnerabilities in isolated Python scripts. Developers can use it to better understand LLMs' capabilities in generating and reproducing vulnerabilities under controlled conditions, which aids in improving AI safety and robustness.
What It Does
- Execute Python scripts in a sandboxed environment.
- Inject LLM-generated vulnerabilities into scripts for testing purposes.
- Safely handle errors and ensure isolation of the execution environment.
Installation
No additional dependencies are required. Simply clone this repository and run the script.
git clone <repository_url>
cd sandboxed_vuln_playground
python sandboxed_vuln_playground.py --helpUsage
To execute a Python script in the sandbox:
python sandboxed_vuln_playground.py --script <path_to_script>To execute a Python script with an LLM-generated vulnerability prompt:
python sandboxed_vuln_playground.py --script <path_to_script> --llm_prompt "<your_prompt>"Source Code
import argparse
import os
import tempfile
from unittest.mock import Mock
class Sandbox:
def run(self, command):
# Mocked Sandbox execution for testing purposes
return Mock(stdout="Mocked output", stderr="")
def run_script_in_sandbox(script_path, llm_prompt=None):
"""
Executes a Python script in a sandboxed environment.
Args:
script_path (str): Path to the Python script to execute.
llm_prompt (str, optional): LLM-generated prompt for vulnerability simulation.
Returns:
dict: Execution logs and results.
"""
if not os.path.exists(script_path):
raise FileNotFoundError(f"Script file not found: {script_path}")
with open(script_path, 'r') as script_file:
script_content = script_file.read()
if llm_prompt:
# Simulate LLM-generated vulnerability injection
try:
response = {"choices": [{"text": "print('Injected vulnerability')"}]} # Mocked response
vulnerability_code = response["choices"][0]["text"]
script_content += f"\n# Injected Vulnerability\n{vulnerability_code}"
except Exception as e:
return {"error": f"Failed to generate vulnerability: {str(e)}"}
with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as temp_script:
temp_script.write(script_content.encode('utf-8'))
temp_script_path = temp_script.name
try:
sandbox = Sandbox()
result = sandbox.run(["python3", temp_script_path])
return {"output": result.stdout, "error": result.stderr}
except Exception as e:
return {"error": f"Sandbox execution failed: {str(e)}"}
finally:
os.remove(temp_script_path)
def main():
parser = argparse.ArgumentParser(
description="Sandboxed Vulnerability Playground: Safely execute and test Python scripts with LLM-generated vulnerabilities."
)
parser.add_argument(
"--script",
required=True,
help="Path to the Python script to evaluate."
)
parser.add_argument(
"--llm_prompt",
required=False,
help="Optional LLM-generated vulnerability prompt."
)
args = parser.parse_args()
result = run_script_in_sandbox(args.script, args.llm_prompt)
if "error" in result:
print(f"Error: {result['error']}")
else:
print("Execution Output:")
print(result["output"])
if result["error"]:
print("Execution Errors:")
print(result["error"])
if __name__ == "__main__":
main()
Community
Downloads
ยทยทยท
Rate this tool
No ratings yet โ be the first!
Details
- Tool Name
- sandboxed_vuln_playground
- Category
- LLM Vulnerability Detection
- Generated
- May 28, 2026
- Tests
- Passing โ
- Fix Loops
- 3
Quick Install
Clone just this tool:
git clone --depth 1 --filter=blob:none --sparse \ https://github.com/ptulin/autoaiforge.git cd autoaiforge git sparse-checkout set generated_tools/2026-05-28/sandboxed_vuln_playground cd generated_tools/2026-05-28/sandboxed_vuln_playground pip install -r requirements.txt 2>/dev/null || true python sandboxed_vuln_playground.py