Module 5 · ~22 min · Structured Interviewer Certification
Practice Lab
You've learned the competencies, the questions, the probes, and the scoring. Now you do it — live, against AI candidates who won't always make it easy. This is where reading becomes skill.
AI Instructor: "Reading about probing doesn't make you a better interviewer. Probing a defensive candidate, catching yourself writing a verdict instead of a quote, and scoring against the anchor when your gut says otherwise — that does. Let's run the reps."
What you'll be able to do
- Conduct a full structured interview — behavioral + situational — against an AI candidate.
- Probe in real time to convert vague answers into scorable evidence.
- Capture evidence-not-inference notes during a live exchange.
- Run and score the clinical case / chart-talk station (Competency #1).
- Complete a full eight-competency scorecard and check it against the expert key.
How the lab works
Five AI agents run this lab in concert:
| Agent | Its job in the lab |
|---|---|
| Interview Simulation | Sets the role/level, supplies the competency focus, paces you, prompts your notes. |
| Candidate Role-Play | Plays the candidate at a configured proficiency (strong / mixed / weak; humble / defensive). Won't volunteer perfect answers. |
| Case Simulation (Agent #8) | Runs the clinical case / chart-talk station for Competency #1. |
| Scoring/Calibration | Scores your interviewing and your scoring; reports your calibration gap. |
| Feedback Coach | Turns scores into 1–3 prioritized, concrete improvements. |
The lab loop: Watch an annotated example → run a rep → take notes → complete the scorecard → compare to the expert key → get coached → run the next rep.
Warm-up — annotated excerpts: weak vs. strong interviewing
Same candidate answer, two interviewers. Read what the strong one does differently.
| Weak interviewer | Strong interviewer | |
|---|---|---|
| Candidate | "We had a tough patient once and it worked out fine." | |
| Response | "Great, sounds like you handle pressure well." (accepts surface answer; infers a verdict) | "What was tough about it, and what did you specifically do?" (depth + evidence probe; goes for the 'I') |
| Note written | "Handles pressure well." (inference) | "Patient escalating re: wait; candidate de-escalated by acknowledging, gave time estimate, re-triaged." (evidence) |
| Score basis | Gut feel. | Evidence matched to the BARS anchor. |
"The weak interviewer already 'decided.' The strong one is still gathering evidence. That difference is the whole course."
Rep 1 — Full behavioral + situational interview
Setup: You'll interview an AI candidate. Pick or be assigned a role (MD/DO, NP, or PA) and a hidden persona (strong / mixed / weak; humble / defensive). Focus competencies: #4 Teamwork, #5 Coachability, #6 Autonomy, #7 Adaptability (plus #2/#8 as they surface).
- Ask your planned behavioral and situational questions from the Module 4 bank.
- Probe every thin answer — the candidate is configured not to hand you clean STAR answers.
- Take evidence notes in the note panel as you go (quotes, not verdicts).
- When you've gathered enough, close the interview and move to the scorecard.
"I stay in character. If you don't probe, you won't get the detail. If you ask an illegal question, I'll answer in character — but the moment gets flagged for the Legal & Bias Coach."
"I'll nudge you if you skip probing or start scoring on vibe, and I'll capture a clean transcript + your notes for scoring."
Rep 1 — Complete the scorecard
Score the candidate on each focus competency (1–5), citing the evidence from your notes. Then the AI reveals:
- The expert key for this persona (the ground-truth signals it was configured to show).
- Your calibration gap per competency (lenient / on / severe).
- Where your notes were inference instead of evidence.
- One to three prioritized improvements from the Feedback Coach.
"What you did well first, with evidence — then the single highest-leverage fix and a try-this probe. Never 'probe more' with no example."
Rep 2 — The clinical case / chart-talk station (Competency #1)
Setup: The Case Simulation Agent (Agent #8) presents a common UC case for your candidate's role. You run it as the clinician rater.
- Present the case; ask the candidate to think aloud (differential, work-up, disposition).
- Reveal information only as the candidate asks for it.
- When they commit, the agent introduces one wrinkle — watch whether they re-reason or anchor.
- Probe reasoning, not recall: "What would change your mind?" "What's your safety net?"
- Score Competency #1 on the BARS (1–5), role-calibrated — appropriate consultation/escalation is a green flag, not a deduction.
Then the agent returns: the BARS score, safety strengths, safety concerns, a readiness signal (ready / targeted reinforcement / additional coaching), and model reasoning (differential → red-flag screen → disposition → safety net → escalation).
Reminder: a qualified clinician owns final interpretation of Competency #1 — TA does not score clinical judgment.
Calibration round — rate the anchor, close the gap
You'll score two short anchor transcripts from the Calibration Anchor Library (one a mixed-signal candidate, one with a hidden red flag) and compare to the consensus key. The AI reports your direction and size of gap and whether it's consistent across reps (your scoring tendency).
"The red-flag anchor is the one to watch. Lenient interviewers explain away the red flag ('they were probably just nervous'). Score the evidence, not the excuse you invent for them."
Reflection
- In Rep 1, where did you almost accept a surface answer? What probe saved it — or didn't?
- Did your scores skew lenient or severe vs. the key? In which direction will you correct next time?
- In the case station, did you score the candidate's reasoning or their recall? Be honest.
Knowledge check
- The AI candidate gives a clean, polished first answer. What should you assume, and what do you do?
- Why does the Candidate Role-Play Agent not volunteer perfect STAR answers?
- In the case station, a PA says they'd consult their SP before transferring. Deduction or green flag — and why?
- Your scores are consistently a point above the expert key. What is that called, and how do you correct it?
- You wrote "seems disorganized" in your notes. What's the problem, and what should it say?
Module summary
The Practice Lab runs the full loop: interview an AI candidate with behavioral and situational questions, probe to evidence, take evidence-not-inference notes, complete the eight-competency scorecard, run and score the clinical case station, and calibrate against expert anchors — with scored feedback after every rep on the 1–5 proficiency scale.
Key takeaways
- The first answer is rehearsed — reps build the reflex to probe.
- Notes are quotes, not verdicts — even under live time pressure.
- Score against the anchor, not your gut; the key shows your direction of error.
- The case station scores reasoning and safety, role-calibrated — clinician-owned.
- Calibration is a skill you build rep by rep, not a trait you have.
Readiness checkpoint (1–5)
The AI aggregates your Rep 1 interview + scorecard, the case-station rep, and the calibration round into a Module 5 proficiency score — with your interviewing strengths, your calibration tendency, any development flags (e.g., used a prohibited question, scored an unsafe disposition as acceptable), and your top focus heading into the final proficiency assessment. No pass/fail.