Module 6 · ~12 min · Structured Interviewer Certification
Debrief, Calibration & the Decision
A great interview is wasted if the decision is made by the loudest voice in the room. This module turns evidence and 1–5 scores into a clear, documented, defensible recommendation — and tells you when to bring others back in.
AI Instructor: "The most expensive moment in hiring isn't the interview — it's the five-minute hallway conversation where someone says 'I just liked them' and everyone nods. Structure protects the decision the same way it protected the interview."
What you'll be able to do
- Score independently before any group discussion, and explain why that order matters.
- Name and interrupt groupthink, halo, and loudest-voice bias in a debrief.
- Use the 1–5 proficiency scores and per-competency profile to structure the conversation.
- Reconcile score differences between interviewers using evidence, not seniority.
- Choose a decision category and document an evidence-based rationale.
- Know when to pull TA, clinical leadership, or another clinician back in.
Key concept 1 — Independent scoring before discussion
Every interviewer completes their scorecard before anyone shares an opinion. This is non-negotiable, and it's the single biggest protection against bias in the decision.
- Why: the moment a respected colleague says "I loved them," everyone's scores drift toward that anchor. Independent scores preserve genuinely independent signal.
- How: scores + evidence submitted to the debrief; the AI (or facilitator) reveals them side-by-side only after all are in.
"If you change your score in the debrief, it should be because of evidence you hadn't heard — never because of who said it."
Key concept 2 — The three debrief biases
| Bias | What it looks like | Interrupt |
|---|---|---|
| Groupthink | Quick consensus; no one wants to be the dissenter. | Independent scores first; explicitly invite the lowest scorer to speak first. |
| Halo / horn | One strong (or weak) trait colors every competency. | Debrief competency-by-competency, not "overall impression." |
| Loudest-voice / seniority | The most senior or assertive person's read wins. | Reconcile on evidence; rank-anchored opinions don't outweigh cited behavior. |
Key concept 3 — Using the 1–5 scores in the debrief
Walk the per-competency profile, not a single overall number. The profile tells a story a composite hides:
- Strong everywhere → likely Ready to Hire.
- Strong with one fixable gap (e.g., 4s across the board, a 3 on documentation/Professionalism that's coachable) → Hire with Targeted Ramp Support.
- A key competency unscored or thin on evidence → Hold / Needs More Evidence.
- A 1–2 on a safety-critical competency, or an unresolved professionalism/safety red flag → Do Not Move Forward (or Hold for clinician review).
The composite (mean of competencies) is a summary, not the decision. A red flag on Professionalism (#3) or an unsafe disposition in the case station (#1) is a gate that the average can't wash out.
Key concept 4 — Reconciling score differences
Two interviewers, different scores — that's normal and useful. Reconcile like this:
- Surface the evidence behind each score, not the score itself ("I gave Coachability a 2 because she said she couldn't think of a mistake").
- Check for different data: often the gap is real — one interviewer heard something the other didn't. That's signal, not error.
- Check for calibration drift: if the evidence is the same but scores differ, one interviewer is lenient or severe — re-anchor to the BARS.
- Don't average to make peace. Land on the score the evidence supports; document the dissent if it remains.
"A persistent gap between two interviewers on the same evidence is a calibration finding — it feeds the interrater-reliability trend we track over time."
Key concept 5 — The decision categories
Scale-based, not pass/fail. The recommendation maps the competency profile + any gates to one of four categories:
| Decision | Profile that supports it | What it means / next step |
|---|---|---|
| Ready to Hire | Strong across competencies (mostly 4–5), no red flags, safe case-station disposition. | Recommend offer; document strengths; clean handoff to TA. |
| Hire with Targeted Ramp Support | Generally strong with one or two coachable gaps (e.g., a 3 on a non-safety competency). | Recommend offer + a named 30/60/90 ramp plan for the gap; assign a mentor/precept. |
| Hold / Needs More Evidence | A key competency unscored or evidence too thin to decide; mixed signal. | Don't force a call — get the missing evidence (second interview, case station, another clinician, references). |
| Do Not Move Forward | Clear competency gaps (1–2 on core), OR an unresolved professionalism/safety red flag (a gate), OR unsafe clinical judgment. | Decline respectfully and promptly; document the evidence-based rationale. |
Decision thresholds (gates): a true professionalism red flag (dishonesty, ethics lapse, refusal of accountability) or an unsafe clinical disposition / missed red flag in the case station is a gate — it sends the candidate to Hold (for clinician review) or Do Not Move Forward regardless of a high average elsewhere.
Key concept 6 — Document the rationale
A defensible recommendation is evidence → score → decision, in writing:
"Recommendation: Hire with Targeted Ramp Support. Communication 4 (used teach-back, de-escalated the anxious parent). Autonomy 4 (clear 9pm-transfer reasoning with appropriate escalation). Professionalism 3 — coachable gap: described a documentation shortcut but owned it when probed; ramp plan to address. No red flags. Case station: safe disposition, role-calibrated (PA). Composite 3.8 / Strong band."
Notice: every score has cited evidence, the gap is named and addressable, and the decision follows the evidence — not the other way around. This is what protects the decision legally and clinically.
Key concept 7 — When to bring others back in
| Pull in… | When |
|---|---|
| TA | Decision reached (any category) → handoff for offer/decline, logistics, and candidate communication; or to schedule a second interview for a Hold. |
| Another clinician | Competency #1 is thin/uncertain, or the case station raised a safety concern that needs a second clinical read; or to deepen coverage via the interview kit. |
| Clinical leadership (Medical Director / Regional) | A professionalism/safety gate is triggered, a borderline high-stakes call, scope/licensure questions, or interviewer disagreement that evidence can't reconcile. |
TA owns logistics and candidate communication; clinical judgment (Competency #1) is never signed off by TA — that stays with a qualified clinician.
Key concept 8 — Candidate experience: timely and respectful
- Be timely. Clinicians are in demand; a slow decision loses strong candidates and signals disorganization. Move the recommendation to TA quickly.
- Be respectful in a decline. Prompt, courteous, no false feedback; never disclose protected-category reasoning or anything that isn't job-relevant and documented.
- Keep it consistent. Every candidate for a role goes through the same structure and gets a decision the same way — fairness and legal defensibility.
"Decline language stays job-relevant and brief. 'We've decided to move forward with other candidates' is enough — invented feedback creates risk."
Interactive — AI-led debrief simulation
You submit your independent scores first. Then the AI plays a second interviewer with a divergent read — and a bias to interrupt:
- The AI second interviewer scores Coachability a 5 ("I just really liked her energy") with no evidence — halo + loudest-voice.
- Your job: reconcile on evidence. Ask what behavior supports the 5; re-anchor to the BARS; land on the evidence-supported score.
The AI then reveals whether you (a) held to evidence, (b) drifted to the confident colleague, or (c) over-corrected, and coaches the reconciliation move.
"Notice the trap: the other interviewer is confident and senior. Confidence isn't evidence. 'What did she actually say or do that earns a 5?' is the whole skill."
Calibration exercise — same candidate, build the recommendation
You're given a completed competency profile (with one fixable gap and no red flags) plus a second profile (high average but a professionalism red flag). For each, pick the decision category and write the one-sentence rationale. The AI compares to the expert key and shows whether the gate correctly overrode the average in the second case.
Reflection
- Recall a real debrief: did independent scores come first, or did someone anchor the room? What would you change?
- When you've disagreed with a senior colleague on a candidate, did you reconcile on evidence or defer? What will you do next time?
Knowledge check
- Why must scoring be independent and submitted before the group discusses the candidate?
- Two interviewers score Autonomy differently. Name two legitimate reasons and how you'd tell them apart.
- A candidate averages 4.2 but has an unresolved professionalism red flag. What decision category, and why doesn't the average decide it?
- What's the difference between "Hire with Targeted Ramp Support" and "Hold / Needs More Evidence"?
- Who signs off on Competency #1, and who never does?
- What should and shouldn't a decline message contain?
Module summary
Score independently first, debrief competency-by-competency, reconcile differences on evidence rather than seniority, and map the per-competency profile to one of four decision categories — with professionalism and clinical-safety red flags acting as gates the average can't override. Document evidence → score → decision, pull in another clinician or clinical leadership when judgment or safety is uncertain, and hand off to TA for a timely, respectful candidate decision.
Key takeaways
- Independent scores before discussion — always.
- Debrief by competency, reconcile by evidence, not by rank.
- Four decisions: Ready to Hire · Hire with Targeted Ramp Support · Hold / Needs More Evidence · Do Not Move Forward.
- Professionalism / clinical-safety red flags are gates.
- TA owns logistics; a clinician owns clinical judgment; be timely and respectful with candidates.
Readiness checkpoint (1–5)
The AI scores your debrief simulation and calibration exercise — did you hold to evidence, apply the gates correctly, choose a defensible decision category, and document a clean rationale? Returns a Module 6 proficiency score, your debrief tendency (anchors to others vs. holds to evidence), and one targeted improvement. No pass/fail.