Screens / Judging the output

Eval scorecard

Show how an agent version did on a fixed suite, against the baseline, with what to look at first.

For: The people who build and release the agent.

Markdown

3 ways to lay it out

1. Scorecard against a baseline

Pick it when: The default: one run against the one before it.

An overall rate, then each criterion's rate and its change from the baseline, over failure clusters.

Eval scorecardProAn eval run against its baseline: the overall pass rate, then each criterion with its change.
Failure clustersProFailing runs grouped by what went wrong, largest first.
Stat cardProOne metric with its change against a named period.

Install scorecard against a baseline

Pro and Premium items need a licence key; free ones install as they are.

2. Two runs side by side

Pick it when: Deciding between two versions or two models.

Two columns of measures with the change between them.

Comparison viewProBefore and after, numerically, coloured by whether the change is good.
Subscore cardProWeighted subscores behind an overall score.
Eval scorecardProEach side's scorecard.

Install two runs side by side

Pro and Premium items need a licence key; free ones install as they are.

3. Quality over time

Pick it when: Watching a live agent drift or recover after a change.

A line-like column chart of scores with markers for what changed alongside.

Quality trendProScores over time with what changed alongside: a model switch, an instruction edit.
Version historyProWhich version was live when.

Install quality over time

Pro and Premium items need a licence key; free ones install as they are.

Rules that matter most here

Unknown stays unknown
A criterion with too few cases shows its count and no rate; a missing baseline is "no baseline".
Motion never says something false
A change is coloured good or bad only by what it is, and a flat change isn't coloured.
Reasoning is believed, not verified
A judge's reasons are shown as the judge's, not as verified.

Already covered elsewhere

Builds the whole screen with an agent route and a store: Eval dashboard (Premium), with a -claude twin for the Claude Agent SDK.

Brief for a coding agent

Paste this into the agent that builds the screen. The docs MCP server returns the same text from get_screen.

Build an eval scorecard with Candor UI. Start from the eval-dashboard block (eval-dashboard-claude for the Claude Agent SDK): each version of an agent runs a fixed suite of cases checked in code, and a page compares the runs.
EvalScorecard shows the overall pass rate and each criterion's rate and change from the baseline; FailureClusters groups the failures by what went wrong, largest first.
Show counts next to rates. A criterion with too few cases shows its count and no rate; no baseline says "no baseline".
ComparisonView puts two runs side by side; colour a change only when it is good or bad.
QualityTrend shows scores over time with what changed alongside.
Pass scores as plain data; Candor does not run the evals.

Start from the "Scorecard against a baseline" layout: An overall rate, then each criterion's rate and its change from the baseline, over failure clusters.
Candor items: eval-scorecard (https://candor.themethodic.com/docs/eval-scorecard.md), failure-clusters (https://candor.themethodic.com/docs/failure-clusters.md), stat-card (https://candor.themethodic.com/docs/stat-card.md).
Install: npx candor-ui add eval-scorecard failure-clusters stat-card
Read each item's docs before using it and take its props from there; don't guess them. Pro and Premium items need the person's licence key, which they add with `npx candor-ui key <key>`; don't ask for it.