Structured interview evaluation: a rubric you can run this week
In short: Unstructured interviews are among the least predictive and least consistent hiring tools in common use; structured interviews — same questions, anchored scoring, independent ratings — are among the most. The rubric below can be adopted without software. An evaluation agent helps by drafting the scored evaluation from the interview notes, with the sentence it relied on beside each score, for the interviewer to confirm or change.
Why structure
Decades of selection research point the same way: interviews predict job performance much better when every candidate for a role gets the same questions, answers are scored against written anchors, and interviewers rate independently before they discuss. The reason is simple — structure removes most of the room for first impressions, similarity bias and the strongest-personality-in-the-debrief effect. It also produces a record, which matters for fairness and for compliance.
A rubric that fits most roles
Five criteria is about the limit of what an interviewer can score reliably in one conversation. For each, write three anchors: what a 1, a 3 and a 5 sound like. Example for a customer-support lead role:
| Criterion | 1 — below bar | 3 — at bar | 5 — clearly above |
|---|---|---|---|
| Role-specific skill | Describes tasks without method or outcome. | Describes a method and one measurable outcome. | Describes method, outcome, and what they changed afterwards. |
| Problem solving | Jumps to a solution; cannot explain alternatives. | Names the problem, one alternative, and a reason for the choice. | Frames the problem, weighs trade-offs, names what would change their mind. |
| Working with others | Examples are solo or blame others. | Concrete example of resolving a disagreement. | Example shows changing their own view based on a colleague's input. |
| Communication | Answers are hard to follow; needs repeated prompting. | Clear, answers the question asked. | Clear, concise, adapts detail to the listener. |
| Motivation and fit for this role | Generic reasons; little knowledge of the role. | Specific reasons tied to the role's actual work. | Specific reasons plus realistic view of the role's downsides. |
Calibrating interviewers
Before the first real interview, have every interviewer score the same two recorded or written mock interviews independently, then compare. Where scores differ by two or more points, rewrite the anchor until they converge. Repeat quarterly; drift is normal. Keep debriefs after independent scoring, never before.
Where an agent helps — and where it must not
An interview evaluation agent reads the interviewer's notes or the transcript and drafts a scored evaluation: for each criterion, a proposed score and the one or two sentences from the interview it relied on. The interviewer sees the draft beside their own impression, confirms or changes each score, and adds anything the notes missed. Time per evaluation drops from twenty minutes to five; consistency between interviewers rises because everyone is scored against the same anchors with the evidence visible.
What the agent must not do: decide. It does not compare candidates, does not rank, does not recommend advancing or rejecting. Those are the interviewer's and hiring manager's actions, logged under their names. This separation is a fairness principle and, in several jurisdictions, close to a legal requirement — see our 2026 compliance checklist.
Measuring whether it works
- Inter-rater agreement: how often two interviewers' confirmed scores for the same candidate are within one point.
- Agent agreement: how often the interviewer keeps the agent's proposed score. Below about 70% the rubric or the anchors need work.
- Time per evaluation, measured, not estimated.
- Adverse impact: selection rates by protected group at each stage, reviewed by HR and counsel.
These four numbers are exactly what the weekly log in a recruiting pilot tracks.
Run this on your own hiring data
A six-week recruiting pilot measures exactly these things — parser accuracy, reviewer agreement, time per application and compliance records — on your role family, with your reviewers, before you decide anything.