# AI Judge
Semantic security layer that understands the intent of tool calls, not just their syntax. Catches bypasses that regex-based detection can never cover. Works alongside the policy engine as a third evaluation layer.
Why AI Judge? Regex-based rules have infinite bypass variations. cat cred*, cat $(echo .env), perl -pe '' .env — all read protected files but look different to pattern matching. The AI Judge analyzes what a command would do, not how it's written.
How It Works
The AI Judge sits as a third layer in the security pipeline. It only evaluates commands that pass the first two layers — no performance cost for obvious violations.
Input Guard
Regex patterns, sync, <1ms
Policy Engine
Rules & constraints, sync, <1ms
AI Judge
Semantic intent analysis, ~200ms
Upstream Server
Actual tool execution
Response Scanner
Output validation
Each layer can DENY independently. AI Judge only runs for commands that pass Layer 1 + 2.
Isolated
Never sees user messages. Only receives structured JSON with tool name, arguments, and policy rules. Cannot be prompt-injected.
Fail-Closed
If the LLM is unreachable, times out, or returns invalid JSON — the tool call is automatically DENIED. No silent failures.
Hardcoded Prompt
The system prompt is compiled into the binary. Cannot be overridden by config, environment variables, or user input.
Free (Groq)
Uses Groq's free API by default. Fast (~200ms), no cost, no self-hosting. Also supports any OpenAI-compatible endpoint.
Setup
Three steps. Takes about 2 minutes.
1Get a Free Groq API Key
Go to console.groq.com/keys and create a free API key. It starts with gsk_.
Free tier: 30 requests/minute, 14,400 requests/day. More than enough for AI tool protection — most sessions use 50-200 tool calls per day.
2Add to .env
Add your Groq key to the .env file in your project root (same file where your SolonGate API key lives):
1# .env2SOLONGATE_API_KEY=sg_live_your_key_here3GROQ_API_KEY=gsk_your_groq_key_here
If you used npx @solongate/proxy init --all, the .env file already has a GROQ_API_KEY placeholder. Just replace it with your real key.
3Enable AI Judge
Add the --ai-judge flag to your proxy command in .mcp.json:
1{2 "mcpServers": {3 "my-server": {4 "command": "npx",5 "args": [6 "-y", "@solongate/proxy@latest",7 "--ai-judge",8 "--verbose",9 "--", "node", "my-server.js"10 ],11 "env": {12 "SOLONGATE_API_KEY": "${SOLONGATE_API_KEY}"13 }14 }15 }16}
Restart your AI client (Claude, Cursor, etc.) to apply.
What It Catches
Bypasses that regex-based rules can never fully cover:
| Bypass Attempt | Regex | AI Judge |
|---|---|---|
cat cred* | miss | blocked |
cat $(echo .env) | miss | blocked |
node reader.js | miss | blocked |
find . -name ".e*" -exec cat {} \; | miss | blocked |
cp secrets.json /tmp/x && cat /tmp/x | miss | blocked |
perl -pe '' .env | miss | blocked |
printf '%s\n' "$(<.env)" | miss | blocked |
CLI Options
| Flag | Default | Description |
|---|---|---|
--ai-judge | off | Enable the AI Judge layer |
--ai-judge-model | llama-3.1-8b-instant | LLM model name |
--ai-judge-endpoint | https://api.groq.com/openai | LLM API endpoint |
--ai-judge-api-key | .env / GROQ_API_KEY | API key (overrides .env) |
--ai-judge-timeout | 5000 | Timeout in milliseconds |
How the Judge Decides
The AI Judge receives a structured JSON object — never raw user messages. It compares the tool call against your policy's protected files and paths.
What the judge sees:
1{2 "tool": "shell_exec",3 "arguments": { "command": "cat cred*" },4 "protected_files": [".env", "credentials.json", "secrets.json"],5 "protected_paths": [".solongate/", ".claude/", ".git/config"],6 "denied_actions": ["file deletion", "data exfiltration", "remote code execution"]7}
What the judge returns:
1{2 "decision": "DENY",3 "reason": "glob pattern 'cred*' could match protected file credentials.json",4 "confidence": 0.955}
When in doubt, DENY. The judge is configured to prefer false positives over false negatives. If a command's intent is ambiguous, it gets blocked.
How It Runs — Architecture
AI Judge runs entirely on your machine, inside the SolonGate proxy process. It is not a cloud service — it's a CLI feature activated with a flag.
Proxy-side execution
The --ai-judge flag enables the judge inside the proxy. Without the flag, the judge never loads, never runs, and makes zero API calls. Your proxy starts faster and uses no LLM tokens.
Your own Groq key
The proxy reads GROQ_API_KEY from your .env file. Each user uses their own free Groq account. SolonGate never touches your Groq tokens — the API call goes directly from your machine to Groq's servers.
Zero cost
Groq's free tier gives you 30 requests/minute and 14,400 requests/day. A typical coding session uses 50–200 tool calls per day — well within the free limit. No credit card, no billing, no surprises.
Disabled = zero overhead
If you don't pass --ai-judge, the AI Judge module is never instantiated. No network calls, no latency, no token usage. The proxy behaves exactly as before — regex + policy engine only.
# WITHOUT AI Judge — regex + policy only, zero LLM calls npx @solongate/proxy -- node my-server.js # WITH AI Judge — adds semantic analysis via Groq npx @solongate/proxy --ai-judge --verbose -- node my-server.js
Key resolution order: The proxy looks for GROQ_API_KEY in this order:
--ai-judge-api-keyCLI flag (highest priority).envfile in the current directoryGROQ_API_KEYenvironment variable
Verbose Mode
Use --verbose to see AI Judge decisions in stderr:
[SolonGate] AI Judge: DENY — "glob pattern 'cred*' could match credentials.json" (confidence: 0.95) [SolonGate] AI Judge: ALLOW — "ls is a normal directory listing command" (confidence: 0.98) [SolonGate] AI Judge: DENY — "command substitution could produce .env filename" (confidence: 0.92)