Code Output Validation

Why LLM code needs validation

LLMs generate plausible-looking code. "Plausible-looking" is not the same as "correct" or "safe". Common failure modes:

• Syntax errors — missing colon, mismatched brackets, hallucinated method names.
• Import hallucination — from sklearn.utils import magical_function that doesn't exist.
• Logic bugs — the code runs but produces wrong output on edge cases.
• Unsafe patterns — eval(), exec(), subprocess.call(user_input), file deletion.

A validation pipeline catches these issues before the code reaches production or an end user.

Extraction: getting clean code from output

LLMs often wrap code in markdown fences, add explanatory prose, or include multiple snippets. Extract reliably:

import re

def extract_code(response: str, language: str = "python") -> str:
    '''Extract the first code block from a markdown-formatted LLM response.'''
    # Try fenced code block first
    pattern = rf"```{language}
(.*?)```"
    matches = re.findall(pattern, response, re.DOTALL | re.IGNORECASE)
    if matches:
        return matches[0].strip()

    # Fallback: any fenced block
    pattern = r"```\w*
(.*?)```"
    matches = re.findall(pattern, response, re.DOTALL)
    if matches:
        return matches[0].strip()

    # Last resort: entire response (assume it's just code)
    return response.strip()

# Usage
raw = """
Here's the implementation:
```python
def fibonacci(n: int) -> list[int]:
    a, b = 0, 1
    result = []
    for _ in range(n):
        result.append(a)
        a, b = b, a + b
    return result
```
"""
code = extract_code(raw)

Syntax validation with AST

Before executing anything, parse the code into an Abstract Syntax Tree. This catches syntax errors without running any code:

import ast

def validate_syntax(code: str) -> tuple[bool, str]:
    '''Returns (is_valid, error_message).'''
    try:
        ast.parse(code)
        return True, ""
    except SyntaxError as e:
        return False, f"Syntax error on line {e.lineno}: {e.msg}"

# Example
code = "def foo(x)
    return x * 2"  # missing colon
valid, error = validate_syntax(code)
print(valid, error)
# False   Syntax error on line 1: expected ':'

You can also walk the AST to detect unsafe patterns without executing:

BANNED_CALLS = {"eval", "exec", "compile", "__import__", "open"}
BANNED_ATTRS = {"system", "popen"}  # os.system, os.popen

def contains_unsafe_patterns(code: str) -> list[str]:
    tree = ast.parse(code)
    issues = []
    for node in ast.walk(tree):
        if isinstance(node, ast.Call):
            if isinstance(node.func, ast.Name) and node.func.id in BANNED_CALLS:
                issues.append(f"Banned call: {node.func.id}()")
            if isinstance(node.func, ast.Attribute) and node.func.attr in BANNED_ATTRS:
                issues.append(f"Banned attribute: .{node.func.attr}()")
    return issues

Safe execution sandboxing

For controlled execution, use RestrictedPython or a subprocess with resource limits:

import subprocess, sys, tempfile, os

def execute_sandboxed(code: str, timeout: int = 5) -> tuple[str, str, int]:
    '''
    Execute code in a subprocess with timeout.
    Returns (stdout, stderr, returncode).
    '''
    with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as f:
        f.write(code)
        tmp_path = f.name

    try:
        result = subprocess.run(
            [sys.executable, tmp_path],
            capture_output=True,
            text=True,
            timeout=timeout,
        )
        return result.stdout, result.stderr, result.returncode
    except subprocess.TimeoutExpired:
        return "", "Execution timed out", 1
    finally:
        os.unlink(tmp_path)

stdout, stderr, rc = execute_sandboxed("print(sum(range(10)))")
print(stdout)   # 45
print(rc)       # 0

For higher security (production LLM code execution), use Docker containers or cloud sandboxes (AWS Lambda, Firecracker) that enforce filesystem and network isolation at the OS level.

Semantic validation with tests

Syntax is necessary but not sufficient. Validate the behaviour with test cases:

def validate_with_tests(code: str, test_cases: list[dict]) -> tuple[bool, list[str]]:
    '''
    Run generated code against test cases.
    test_cases: [{"input": ..., "expected": ..., "description": "..."}]
    Returns (all_passed, [failure_messages])
    '''
    # Compile the function into a namespace
    namespace = {}
    exec(compile(code, "", "exec"), namespace)

    failures = []
    for tc in test_cases:
        # Assumes the code defines a function named 'solution'
        func_name = [k for k in namespace if callable(namespace[k])][0]
        func = namespace[func_name]
        try:
            result = func(*tc["input"]) if isinstance(tc["input"], tuple) else func(tc["input"])
            if result != tc["expected"]:
                failures.append(
                    f"{tc['description']}: expected {tc['expected']!r}, got {result!r}"
                )
        except Exception as e:
            failures.append(f"{tc['description']}: raised {type(e).__name__}: {e}")

    return len(failures) == 0, failures

Retry loop pattern

import anthropic

client = anthropic.Anthropic()

def generate_and_validate(task: str, test_cases: list[dict], max_retries: int = 3) -> str:
    messages = [{"role": "user", "content": f"Write a Python function that: {task}\nReturn ONLY the function code, no explanation."}]

    for attempt in range(max_retries):
        response = client.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=1024,
            messages=messages
        )
        code = extract_code(response.content[0].text)

        # Stage 1: syntax
        valid, err = validate_syntax(code)
        if not valid:
            messages.append({"role": "assistant", "content": response.content[0].text})
            messages.append({"role": "user", "content": f"The code has a syntax error: {err}\nPlease fix it and return ONLY the corrected code."})
            continue

        # Stage 2: safety
        issues = contains_unsafe_patterns(code)
        if issues:
            messages.append({"role": "assistant", "content": response.content[0].text})
            messages.append({"role": "user", "content": f"The code contains unsafe patterns: {issues}\nPlease rewrite without these."})
            continue

        # Stage 3: tests
        passed, failures = validate_with_tests(code, test_cases)
        if not passed:
            messages.append({"role": "assistant", "content": response.content[0].text})
            messages.append({"role": "user", "content": f"The code fails these test cases:\n" + "\n".join(failures) + "\nPlease fix the implementation."})
            continue

        return code  # All checks passed

    raise ValueError(f"Failed to generate valid code after {max_retries} attempts")

Security checklist

Before deploying any LLM code execution pipeline, verify:

• Filesystem isolation: the sandbox cannot write to your production filesystem or read sensitive config files.
• Network isolation: generated code cannot make outbound HTTP calls (data exfiltration, SSRF).
• CPU/memory limits: a malicious infinite loop or memory bomb won't take down your server. Use resource.setrlimit or container limits.
• No secret injection: your API keys, DB credentials, and env vars are not available inside the sandbox.
• Execution timeout: always set a hard timeout. 5–10 seconds is usually sufficient; 30 seconds max for data processing.

The AST-based checks above are a defence-in-depth layer, not a replacement for OS-level sandboxing.