Interview scorecards: how a panel scores one candidate against the same criteria

Three interviewers sit through the same interview. Afterwards, one calls the candidate strong, one calls them mixed, and the third cannot reconstruct from their notes why they leaned either way. Nothing went wrong in the room. What is missing is an instrument: a single scorecard, identical for every candidate and every interviewer, that fixes in advance what is being scored, what each score means, and the point at which scores may be compared. The U.S. Office of Personnel Management (OPM) describes structured interviews as employing rules for eliciting, observing, and evaluating responses, and notes that this structure increases interviewers' agreement on their evaluations by limiting the amount of discretion an interviewer is allowed [1]. The scorecard is where those rules become paperwork.
Internal link suggestion (editorial note, not part of the published text): this page is the companion to the site's rubric method note, "Structured interview evaluation: a rubric you can run this week" [3], which sets the criteria and anchors; this page covers the panel paperwork that uses them. It also pairs with "Resume parsing errors reviewers should check by hand," which applies the same discipline — same fields, checked the same way, against the source — one stage earlier, at resume review.
The blank scorecard, field by field
A scorecard has two halves: the individual form each panellist completes alone, and the group form the panel completes together. OPM's practical guide publishes samples of both — an individual rating form and a group rating form [2] — and the fields below follow that division.
The header. Role, candidate name or identifier, date, interviewer's name, and the version of the question set and rubric in use. The version line looks bureaucratic until the rubric changes mid-round; without it, two scorecards from different weeks cannot be compared because they were not measuring against the same anchors [3].
The criterion block, repeated once per criterion. Each block carries five things: the criterion's name and a one-line definition taken from the job analysis — OPM's guidance ties interview questions to competencies identified through job analysis as critical to job success [1]; the questions mapped to that criterion, so every candidate is scored on the same evidence; a notes space reserved for what the candidate actually said, in behavioural terms; the anchors printed on the form itself, at 1, 3, and 5, so the standard is in front of the rater at the moment of scoring rather than in a document they read last week [3]; and the score line. Nothing else goes in the block. A comment about the candidate's manner, confidence, or fit has no field to land in, which is the point.
The sign-and-submit line. The guide's instruction is that each panellist rates the candidate immediately after the candidate leaves the room, forming an independent evaluation without discussion with the other panel members, and signs and dates the form [2]. Ratings should be specific, defensible, and supported by behavioural examples from the notes [2]. A score that is not written down before the debrief is not an independent score; it is a recollection of the debrief.
The group half. One row per criterion for the consensus score, a space beside any changed score for the reason, and a signature line for the whole panel. The guide's group form works this way: individual ratings are transferred onto it, the panel justifies each rating in discussion, changes made during consensus are initialled, and a final group consensus rating is entered for each competency [2]. Below the scores sits a separate recommendation line, completed after the consensus scores — so the recommendation argues from the scores instead of replacing them.
The worked example: one candidate, three scorecards
Illustrative example — a composite, not a real hiring round. The role is operations coordinator at a regional delivery firm. The panel has three members, the size OPM's guide works with: the hiring manager, the team lead the role reports alongside, and a supervisor from an adjacent team [2]. The rubric has five criteria — planning and prioritisation, problem solving, working across teams, communication, and follow-through and accuracy — each with anchors written at 1, 3, and 5 [3]. Candidate B is the second of four candidates. What follows is the panel's paperwork for Candidate B, criterion by criterion.
Planning and prioritisation — scores 5, 4, 4; consensus 4
Asked about a day when the plan fell apart before noon, the candidate described a vehicle breakdown at nine in the morning: forty deliveries re-sequenced, drivers called in a new order, the two time-critical drops protected. The hiring manager scored 5; the other two scored 4. A one-point spread does not need rescuing, but it does need one check: read the top score against the 5 anchor, which asks for the re-plan plus a check-back loop with the people affected by it. The notes showed re-sequencing and driver calls, but no check-back with the customers whose windows moved. Consensus: 4, with the gap named in the group form — the score describes the answer that was given, not the answer one more probe might have produced.
Problem solving — scores 4, 2, 4; consensus 4
The situational question put a supplier missing its delivery window twice in one week. The candidate's answer came in two parts: an immediate fix (move the standing order), then a comparison — stay with the supplier against a service credit, or move the volume — and a stated trigger: a third miss would change the call. The team lead scored 2. The hiring manager and the supervisor scored 4. This is the discrepancy the panel discussion exists for: OPM's guide has panellists compare notes and thoroughly explore the basis for discrepancies before reaching consensus [2]. The basis here was in the notes. The team lead's notes stopped at the first fix, because they were still writing up the previous answer when the comparison was given; the other two sets of notes quoted it. The team lead revised to 4, initialled the change on the group form, and recorded the reason: initial rating based on incomplete notes. Note what the consensus is not: it is not the average of 4, 2, and 4. It is the score the full evidence supports, reached after the weakest record was repaired.
Working across teams — scores 3, 3, 3; consensus 3
Asked for a disagreement with another team about priorities, the candidate gave a concrete case: a shared vehicle pool, a clash settled by trading deadlines with the other team's lead. All three panellists scored 3 independently, and their notes pointed at the same evidence — a real example, a workable resolution, and nothing showing the candidate changing their own position in response to the other side, which is what the 5 anchor asks for. A unanimous 3 with matching evidence is a finding, not fence-sitting. It is worth distinguishing from the rating error OPM's guide calls central tendency — rating everything at the middle of the scale, all 3s on a 5-point scale, regardless of performance [2]. The difference is in the notes: three identical scores backed by three matching pieces of evidence are agreement; three identical scores backed by nothing in particular are a habit.
Communication — scores 5, 4, 4; consensus 4
The hiring manager scored 5, as they had on planning — and that repetition is itself a prompt. OPM's guide lists the halo effect among the common rating errors: allowing a rating in one competency to influence ratings in others, as when a high rating on oral communication biases the rating on problem solving [2]. So the panel checked the communication evidence on its own. It showed clear, well-structured answers throughout. What it did not show was the candidate adapting their level of detail to the listener — explaining the same delay differently to a driver, a client, and a manager — which is what this rubric's 5 anchor describes. The hiring manager accepted 4, and the reason went on the group form: evidence supports clarity, not adaptation. The check is not an accusation; it is the panel asking whether a score is about this criterion or about the general impression the candidate made.
Follow-through and accuracy — scores 3, 4, 3; consensus 3
The candidate's example described catching a recurring error in the delivery manifest, correcting it, and setting up a weekly check to catch it again. The team lead scored 4, crediting the weekly check. The other two scored 3: the rubric's 3 anchor describes a completed loop — spot it, fix it, confirm the fix landed — and in the answer as given, the confirmation step was a plan, not something already done. The panel scored against the anchor as written and recorded the reasoning. If the panel believes the anchor undervalues setting up the check, that is a rubric revision for the next round, argued in calibration [3] — not an exception made mid-round for one candidate, because an exception for one candidate is a different standard for the rest.
The completed group form for Candidate B reads: planning and prioritisation 4, problem solving 4, working across teams 3, communication 4, follow-through and accuracy 3 — each change from the individual cards initialled, each reason recorded, and the form signed by all three panellists, as the guide prescribes [2].

Reading a finished set of scorecards
Across a full round, the scorecards become data about the panel as well as about the candidates. Three patterns are worth looking for. A rater who sits a point above or below the rest of the panel on most criteria is showing what the guide calls leniency or strictness — a tendency to rate all candidates high or low irrespective of their actual performance [2]. A criterion that splits the panel by two points or more on candidate after candidate is usually not a rater problem but an anchor problem; the site's calibration practice is to rewrite the anchor until independent scores converge, and to measure agreement by how often two interviewers' confirmed scores fall within one point of each other [3]. And comparisons only hold if the panel held: where feasible, the same interviewers should be used across all candidates, to ensure consistency in ratings [2].
What the scorecard never does is make the decision. It produces comparable, evidence-backed scores that a named hiring manager decides from — and a record that shows how: the question set and rubric version used, the independent scores recorded before discussion, any score changed afterwards with its reason, and the final decision with the name of the person who made it [2][3]. That record is what makes one candidate's 4 mean the same thing as another candidate's 4, three weeks and four interview rooms later.
Sources
- U.S. Office of Personnel Management — Structured Interviews (assessment methods guidance: structured interviews employ rules for eliciting, observing, and evaluating responses and increase interviewers' agreement by limiting interviewer discretion; interviews with higher degrees of structure show higher validity, rater reliability, rater agreement, and less adverse impact; the most structured form asks all applicants the same pre-defined lead and probe questions, scored according to benchmarks of proficiency; questions are based on competencies identified through job analysis; structured interviews are usually scored by a panel in which each member rates responses individually and then participates in a group discussion to resolve significant scoring discrepancies). https://www.opm.gov/policy-data-oversight/assessment-and-selection/other-assessment-methods/structured-interviews/
- U.S. Office of Personnel Management — Structured Interviews: A Practical Guide, September 2008 (a panel of two or three interviewers; each panel member individually observes, records, and evaluates; ratings made independently immediately after the candidate leaves, specific, defensible, and supported by behavioral examples, with forms signed and dated; after independent rating, panel members compare notes, thoroughly explore the basis for discrepancies, and reach consensus, recording the consensus rating on a group rating form and signing it; changes to individual ratings during consensus discussion are initialed; common rating errors include rater bias, halo effect, central tendency, leniency, strictness, and similar-to-me; sample individual rating form, Appendix G, and sample group rating form, Appendix H; documentation of the interview process to retain). https://www.opm.gov/policy-data-oversight/assessment-and-selection/structured-interviews/guide.pdf
- FlourishAIHR — Structured interview evaluation: a rubric you can run this week (method note: five criteria with anchors written at 1, 3, and 5; interviewers rate independently before they discuss; calibration by having every interviewer score the same mock interviews and rewriting the anchor where scores differ by two or more points; inter-rater agreement measured as how often two interviewers' confirmed scores are within one point; the record each interview leaves — question set and rubric version, independent scores recorded before the debrief, any score changed after discussion with the reason, and the final decision with the name of the person who made it). Published October 18, 2026; updated November 2, 2026. https://flourishaihr.com/insights/structured-interview-evaluation-rubric.html
Run this on your own hiring data
A six-week recruiting pilot measures exactly these things — parser accuracy, reviewer agreement, time per application and compliance records — on your role family, with your reviewers, before you decide anything.