CLICommands
arbiter
snug arbiter <COMMAND>Arbiter LLM-as-a-judge reasoning and evaluation.
Subcommands
| Command | What it does |
|---|---|
health | Service health check |
evaluate | One-shot evaluation |
rubric | Rubric management |
session | Session management |
eval | Evaluation results management |
compare | Pairwise comparison of two content items |
calibrate | Submit a calibration sample |
calibration | Get calibration statistics |
experiment | Experiment management |
stats | Session score statistics |
Run snug arbiter <command> --help for that command's own options.
The global options apply everywhere.
Examples
snug arbiter evaluate --criteria "Rate accuracy 0-1" --content '{"response":"No refunds."}' --api-key sk-ant-...
snug arbiter rubric create --file rubric.json
snug arbiter rubric list
snug arbiter rubric get --rubric-id rub_123
snug arbiter rubric update --rubric-id rub_123 --file rubric-v2.json
snug arbiter rubric delete --rubric-id rub_123
snug arbiter session create --file session.json
snug arbiter session list --status active
snug arbiter session start --session-id ses_123
snug arbiter session push-gt --session-id ses_123 --file ground-truth.json
snug arbiter session push-content --session-id ses_123 --file content.json
snug arbiter session evaluate --session-id ses_123 --api-key sk-ant-...
snug arbiter eval list --session-id ses_123
snug arbiter eval get --evaluation-id eval_123
snug arbiter eval appeal --evaluation-id eval_123 --reason "Score too low"
snug arbiter eval override --evaluation-id eval_123 --file scores.json --reason "Correct per policy"
snug arbiter compare --session-id ses_123 --content-a '{}' --content-b '{}' --api-key sk-ant-...
snug arbiter calibrate --session-id ses_123 --file calibration.json
snug arbiter calibration --session-id ses_123
snug arbiter experiment create --file experiment.json
snug arbiter experiment list
snug arbiter experiment get --experiment-id exp_123
snug arbiter stats --session-id ses_123
snug arbiter healthNext
- arbiter guide - what the service does and when to reach for it.
- API reference - every endpoint these commands call.
- CLI reference - global options, output modes, and exit codes.