All Toolsโ€บEsoteric Task Generator
๐Ÿ’ฌ LLM Esoteric Code BenchmarksMarch 20, 2026โœ… Tests passing

Esoteric Task Generator

A Python library to generate tasks in esoteric programming languages for benchmarking LLMs. It includes customizable templates, random code generators, and validation utilities. This tool helps researchers create diverse and challenging benchmarks for evaluating language models.

What It Does

  • Generate random programs in esoteric languages.
  • Customizable templates for task generation.
  • Validation utilities for syntactic correctness.

Installation

Install the required dependencies:

pip install pyparsing==3.1.1 pytest==7.4.2

Usage

Generated Task:
++[>++<-]

Source Code

import random
from pyparsing import Word, alphas, nums, Optional, ParseException

def generate_brainfuck_task(complexity):
    """
    Generate a random Brainfuck program based on the given complexity.

    Args:
        complexity (int): Determines the length and complexity of the program.

    Returns:
        str: A randomly generated Brainfuck program.
    """
    commands = ['>', '<', '+', '-', '.', ',', '[', ']']
    program = ''.join(random.choices(commands, k=complexity))
    return program

def validate_brainfuck_syntax(program):
    """
    Validate the syntax of a Brainfuck program.

    Args:
        program (str): The Brainfuck program to validate.

    Returns:
        bool: True if valid, False otherwise.
    """
    stack = []
    for char in program:
        if char == '[':
            stack.append('[')
        elif char == ']':
            if not stack:
                return False
            stack.pop()
    return len(stack) == 0

def generate_task(language='brainfuck', complexity=5):
    """
    Generate a task in the specified esoteric language.

    Args:
        language (str): The target esoteric language (default: 'brainfuck').
        complexity (int): The complexity level of the task (default: 5).

    Returns:
        str: The generated task as a string.

    Raises:
        ValueError: If the language is unsupported or complexity is invalid.
    """
    if complexity <= 0:
        raise ValueError("Complexity must be a positive integer.")

    if language == 'brainfuck':
        task = generate_brainfuck_task(complexity)
        if not validate_brainfuck_syntax(task):
            raise ValueError("Generated Brainfuck program has invalid syntax.")
        return task
    else:
        raise ValueError(f"Unsupported language: {language}")

if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser(description="Esoteric Task Generator")
    parser.add_argument("--language", type=str, default="brainfuck", help="Target esoteric language (default: brainfuck)")
    parser.add_argument("--complexity", type=int, default=5, help="Complexity level of the task (default: 5)")

    args = parser.parse_args()

    try:
        task = generate_task(language=args.language, complexity=args.complexity)
        print("Generated Task:")
        print(task)
    except ValueError as e:
        print(f"Error: {e}")

Community

Downloads

ยทยทยท

Rate this tool

No ratings yet โ€” be the first!

Details

Tool Name
esolang_task_generator
Category
LLM Esoteric Code Benchmarks
Generated
March 20, 2026
Tests
Passing โœ…

Quick Install

Clone just this tool:

git clone --depth 1 --filter=blob:none --sparse \
  https://github.com/ptulin/autoaiforge.git
cd autoaiforge
git sparse-checkout set generated_tools/2026-03-20/esolang_task_generator
cd generated_tools/2026-03-20/esolang_task_generator
pip install -r requirements.txt 2>/dev/null || true
python esolang_task_generator.py
Esoteric Task Generator โ€” AI Tools by AutoAIForge