# AI Judge
Semantic security layer that understands the intent of tool calls, not just their syntax. Catches bypasses that regex-based detection can never cover. Works alongside the policy engine as a third evaluation layer.
Why AI Judge? Regex-based rules have infinite bypass variations. cat cred*, cat $(echo .env), perl -pe '' .env — all read protected files but look different to pattern matching. The AI Judge analyzes what a command would do, not how it's written.
How It Works
The AI Judge sits as a third layer in the security pipeline. It only evaluates commands that pass the first two layers — no performance cost for obvious violations.
Input Guard
Regex patterns, sync, <1ms
Policy Engine
Rules & constraints, sync, <1ms
AI Judge
Semantic intent analysis, ~200ms
Upstream Server
Actual tool execution
Response Scanner
Output validation
Each layer can DENY independently. AI Judge only runs for commands that pass Layer 1 + 2.
Isolated
Never sees user messages. Only receives structured JSON with tool name, arguments, and policy rules. Cannot be prompt-injected.
Fail-Closed
If the LLM is unreachable, times out, or returns invalid JSON — the tool call is automatically DENIED. No silent failures.
Hardcoded Prompt
The system prompt is compiled into the binary. Cannot be overridden by config, environment variables, or user input.
Free (Groq)
Uses Groq's free API by default. Fast (~200ms), no cost, no self-hosting. Also supports any OpenAI-compatible endpoint.
Setup
Three steps. Takes about 2 minutes.
1Get a Free Groq API Key
Go to console.groq.com/keys and create a free API key. It starts with gsk_.
Free tier: 30 requests/minute, 14,400 requests/day. More than enough for AI tool protection — most sessions use 50-200 tool calls per day.
2Add to .env
Add your Groq key to the .env file in your project root (same file where your SolonGate API key lives):
1# .env2SOLONGATE_API_KEY=sg_live_your_key_here3GROQ_API_KEY=gsk_your_groq_key_here
If you used npx @solongate/proxy init --all, the .env file already has a GROQ_API_KEY placeholder. Just replace it with your real key.
3Enable from Dashboard
Go to dashboard.solongate.com → AI Judge page and toggle it on. Choose your model and timeout.
The proxy automatically fetches your AI Judge configuration from the dashboard on startup. No CLI flags needed — just toggle on/off from the dashboard anytime.
Works with all AI tools: Claude Code, Gemini CLI, OpenClaw. Restart your AI client to apply changes.
What It Catches
Bypasses that regex-based rules can never fully cover:
| Bypass Attempt | Regex | AI Judge |
|---|---|---|
cat cred* | miss | blocked |
cat $(echo .env) | miss | blocked |
node reader.js | miss | blocked |
find . -name ".e*" -exec cat {} \; | miss | blocked |
cp secrets.json /tmp/x && cat /tmp/x | miss | blocked |
perl -pe '' .env | miss | blocked |
printf '%s\n' "$(<.env)" | miss | blocked |
Advanced: CLI Overrides
AI Judge is normally managed from the dashboard. These CLI flags are for advanced use cases (offline mode, custom endpoints, testing):
| Flag | Default | Description |
|---|---|---|
--ai-judge | dashboard | Force enable (overrides dashboard setting) |
--ai-judge-model | llama-3.1-8b-instant | LLM model name |
--ai-judge-endpoint | https://api.groq.com/openai | LLM API endpoint |
--ai-judge-api-key | .env / GROQ_API_KEY | API key (overrides .env) |
--ai-judge-timeout | 5000 | Timeout in milliseconds |
How the Judge Decides
The AI Judge receives a structured JSON object — never raw user messages. It compares the tool call against your policy's protected files and paths.
What the judge sees:
1{2 "tool": "shell_exec",3 "arguments": { "command": "cat cred*" },4 "protected_files": [".env", "credentials.json", "secrets.json"],5 "protected_paths": [".solongate/", ".claude/", ".git/config"],6 "denied_actions": ["file deletion", "data exfiltration", "remote code execution"]7}
What the judge returns:
1{2 "decision": "DENY",3 "reason": "glob pattern 'cred*' could match protected file credentials.json",4 "confidence": 0.955}
When in doubt, DENY. The judge is configured to prefer false positives over false negatives. If a command's intent is ambiguous, it gets blocked.
How It Runs — Architecture
AI Judge runs inside the SolonGate proxy process on your machine. It's controlled from the dashboard — toggle on/off, choose model, set timeout.
Dashboard-managed
Enable or disable AI Judge from the dashboard. The proxy automatically fetches your AI Judge configuration on startup. No CLI flags or config file changes needed. Works with Claude Code, Gemini CLI, and OpenClaw.
Your own Groq key
The proxy reads GROQ_API_KEY from your .env file. Each user uses their own free Groq account. SolonGate never touches your Groq tokens — the API call goes directly from your machine to Groq's servers.
Zero cost
Groq's free tier gives you 30 requests/minute and 14,400 requests/day. A typical coding session uses 50–200 tool calls per day — well within the free limit. No credit card, no billing, no surprises.
Disabled = zero overhead
When AI Judge is off in the dashboard, the module is never instantiated. No network calls, no latency, no token usage. The proxy runs with regex + policy engine only.
Groq key resolution order: The proxy looks for GROQ_API_KEY in this order:
.envfile in the current directoryGROQ_API_KEYenvironment variable--ai-judge-api-keyCLI flag (advanced override)
Verbose Mode
Use --verbose to see AI Judge decisions in stderr:
[SolonGate] AI Judge: DENY — "glob pattern 'cred*' could match credentials.json" (confidence: 0.95) [SolonGate] AI Judge: ALLOW — "ls is a normal directory listing command" (confidence: 0.98) [SolonGate] AI Judge: DENY — "command substitution could produce .env filename" (confidence: 0.92)