A Read-Only AI Audit Loop for the Command Line

When I'm working through a plugin or a config file and want a quick second set of eyes, I want that check in the same pipe as everything else — no browser tab, no paste buffer, no copying half a file into a chat window. So I wrote a small bash function around llm, Simon Willison's command-line tool for language models, and it has become an optional first pass before I commit anything I'm not sure about.

The function is deliberately read-only. It pipes a file into a model, asks for findings, and prints them to the terminal. It does not run tools, does not write files, and does not get a shell. The model suggests; I decide. That boundary is the entire point, and a good portion of this article is about protecting it.

What you need

  • llm installed (pipx recommended) and a model provider key configured
  • glow for rendered markdown output — optional, the script falls back to plain text without it
  • Comfort with a shell — this lives in ~/.bashrc, not a dashboard
  • The judgment to say no to the model's output, which is the only part no tool can do for you

The scripts below are a starting point, not a product. They carry my specifics — DeepSeek model names, a Bludit context detector, a particular FLAG / DO NOT FLAG contract — because that's what I actually run. If your stack differs, you'll want to edit those parts, and the context detection in particular is meant to be extended: nothing stops you from adding a catch for WordPress or a homegrown framework and attaching a reference file that corrects whatever the model gets wrong about it. None of this is difficult bash, but it does assume you're comfortable reading a shell function and changing what it matches before you lean on it. Offered in that spirit: useful, and also specific.


Installing llm

pipx keeps llm and its dependencies out of your system Python — worth doing on a server where PHP-FPM and other services already have opinions about Python versions.

pipx install llm

# Store the key in llm's own keyring (not in a script, not in shell history)
llm keys set deepseek

# Confirm the model names your install actually exposes
llm models

pip install llm and brew install llm both work. The Homebrew build pins Python 3.12, which can complicate plugins that depend on PyTorch — irrelevant for the read-only functions below, but worth knowing if you later install heavier plugins. The functions never reference an API key. llm reads the key from its own store or the provider's environment variable, which keeps credentials out of your .bashrc and out of your shell history. Install glow from your package manager or its releases page if you want rendered output.


The ai_audit function

Drop this in ~/.bashrc and reload with source ~/.bashrc:

# ==============================================
# AI Audit — targeted code review via llm
# ==============================================

AI_AUDIT_REFS="${HOME}/.config/ai-audit/refs"

# ------ Single-file audit: bugs, security, targeted tasks ------
ai_audit() {
    local model="deepseek-v4-pro"
    if [ "$1" = "-f" ]; then
        model="deepseek-v4-flash"
        shift
    fi

    local file="$1"; shift || true
    if [ ! -f "$file" ]; then
        echo "Usage: ai_audit [-f] <file> [specific instructions]"
        echo ""
        echo "Examples:"
        echo "  ai_audit ./login.php"
        echo "  ai_audit ./form.php \"Can an unauthenticated user inject anything?\""
        echo "  ai_audit -f ./style.css \"Minify this and output the result\""
        echo "  ai_audit ./functions.php \"Which functions are unused?\""
        return 1
    fi

    # Refuse files that commonly hold secrets. Name matching is coarse —
    # it catches the files you'll actually reach for, not every case.
    case "$file" in
        *wp-config.php*|*.env|*.key|*.pem|*.p12|*id_rsa*|*id_ed25519*|*id_ecdsa*)
            echo "Refusing: credential-bearing file." >&2
            return 1 ;;
    esac

    # Local syntax check — advisory, not a gate. The output joins the
    # prompt so the model can diagnose a parse failure instead of being
    # blocked by it. Files with no checker pass through unchecked.
    local syntax_note=""
    case "${file##*.}" in
        php)     syntax_note=$(php -l "$file" 2>&1) ;;
        sh|bash) syntax_note=$(bash -n "$file" 2>&1) ;;
        py)      syntax_note=$(python3 -m py_compile "$file" 2>&1) ;;
    esac

    # --- Context detection ---
    local context=""
    local content_sample
    content_sample=$(head -100 "$file" 2>/dev/null)
    local filename_lower
    filename_lower=$(basename "$file" | tr '[:upper:]' '[:lower:]')
    local filepath_lower
    filepath_lower=$(realpath "$file" 2>/dev/null | tr '[:upper:]' '[:lower:]')

    if echo "$content_sample" | grep -qiE 'bludit|bl-kernel|bl-themes|bl-plugins' \
       || echo "$filename_lower" | grep -qE '^bl-|bludit' \
       || echo "$filepath_lower" | grep -qE '/bl-themes/|/bl-plugins/|/bl-kernel/|/bludit/'; then
        if [ -f "${AI_AUDIT_REFS}/bludit.md" ]; then
            context=$(cat "${AI_AUDIT_REFS}/bludit.md")
            echo "Attached Bludit reference (~$(wc -c < "${AI_AUDIT_REFS}/bludit.md" | numfmt --to=iec))"
        fi
    fi
    # --- End context detection ---

    echo "Auditing $file with $model..."

    local system_prompt="You are a code reviewer. Find only what is actually broken or dangerous.

FLAG if:
- Syntax error or undefined reference that would cause a fatal error
- Security hole exploitable by an unauthenticated visitor
- Logic that would silently produce wrong output for end users

DO NOT flag:
- Code style, organization, or architecture preferences
- Admin-only concerns (admins have full code execution access)
- Performance speculation without measurements
- Missing optional files or \"best practices\" that don't affect functionality

If you find nothing, say so briefly. Do not invent problems."

    if [ -n "$context" ]; then
        system_prompt="${system_prompt}

---
# VERIFIED FRAMEWORK REFERENCE (source-code-verified)
This documents the exact framework version this code targets.
- THEME_DIR_* constants are defined by the core boot sequence
  before any theme file loads — do not flag them as undefined.
- init.php is OPTIONAL for themes.
- Admin users have full code execution; admin-supplied HTML is
  not a vulnerability.
- Trust this reference over your training data for API details.

${context}
---
"
    fi

    local user_prompt="$*"
    if [ -n "$syntax_note" ]; then
        user_prompt="Local syntax check output:

${syntax_note}

If the check reported an error, treat it as the first finding and show the minimal fix.
${user_prompt}"
    fi

    # Render with glow if present, plain text otherwise.
    local -a cmd=(llm -m "$model" -s "$system_prompt")
    if [ -n "$user_prompt" ]; then
        cmd+=("$user_prompt")
    fi

    if command -v glow >/dev/null 2>&1; then
        "${cmd[@]}" < "$file" | glow -
    else
        "${cmd[@]}" < "$file"
    fi
}

Walking through it from top to bottom:

Secret guard. Before anything reaches a model, the name-matching check refuses the files you're most likely to be tempted by: wp-config.php, .env, key files, certificates, SSH private keys. This is deliberately coarse — a file named database.php can hold credentials just as easily. It's a tripwire for the obvious cases, not a scanner for every case.

Advisory syntax preflight. PHP, shell, and Python files get a local syntax check, but the result never blocks the audit. It's captured and folded into the prompt instead. A file that parses cleanly gets a "no problems" note the model can ignore; a file that doesn't gets the exact parser error, and the model's first job becomes explaining and fixing it. That's the behavior you actually want when you're staring at a plugin that won't load — the point is diagnosis, not gating. Files with no checker (.conf, .css, .js) skip this entirely and go straight to the model.

Context detection. The block that greps the filename, path, and first hundred lines for Bludit markers is not there to tell the model what kind of file it's looking at — the model can infer an nginx config from the word proxy_pass. It's there to attach verified reference material when the model's training data is likely to be wrong. Bludit is niche enough that a model will confidently hallucinate hook signatures and constants, so the function hands it a source-verified reference and says "trust this over your training data." For nginx, WordPress, or plain PHP, training data is good enough that no reference is needed — reserve this pattern for the frameworks where the model demonstrably lies.

The FLAG / DO NOT FLAG prompt. This is the feature that makes the tool usable. Most AI code review fails because the model treats it as an opportunity to perform — style opinions, architecture lectures, best-practice sermons about files it hasn't seen. The prompt forbids all of that and narrows the model to three concrete failure classes: fatal errors, unauthenticated-exploitable security holes, and silently wrong output. "If you find nothing, say so briefly" is the line that turns this from a suggestion engine into an audit.

glow fallback. The final block renders markdown with glow if it's installed and falls back to plain text if it isn't. Piping unconditionally to a missing command would swallow the entire audit.