Can this AI shortlist support the next decision?
Check the screen against the job before accepting its view of the candidate pool.
The question
You lead recruiting operations at Harbor People Services. A service-analyst role accepts a relevant degree or equivalent work evidence. The AI summary calls channel B a weaker pool. The two channels used the same application window and screening tool. All records are invented.
| Channel A | Channel B | |
|---|---|---|
| Applications | 10 | 10 |
| Advanced by the tool | 5 | 4 |
These are recruitment channels, not demographic groups. The counts describe tool decisions, not job performance.
“Can this AI shortlist support the next decision?”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Separate the screen from candidate ability
What would you do here? Open the move you’d make.
Accept the weaker-pool label because B advanced less often.
Tempting, but it misses something. The label assumes the screen is valid.
Describe the rates and keep the capability claim open.
The strong move. This separates the observed result from its possible explanation.
Set identical advancement counts without checking anyone’s work.
Tempting, but it misses something. Equal counts would not establish a sound assessment.
See how a strong candidate says it
“The tool advanced 50% from A and 40% from B. That does not show which pool has more people who meet the job criteria.”
Why it works: A decision rate is not a measure of job capability.
Step 2: Check the stated job criteria
What would you do here? Open the move you’d make.
Review the job evidence in both selected and rejected records.
The strong move. This can expose missed candidates and unsupported advances.
Review only the people already advanced.
Tempting, but it misses something. That cannot show what the screen missed.
Make a degree mandatory after seeing the results.
Tempting, but it misses something. That changes the agreed requirement after the screen.
See how a strong candidate says it
“Review advanced and rejected records against the degree-or-equivalent requirement. Check what the tool used and whether relevant work evidence was captured.”
Why it works: A credential proxy can miss the evidence the role accepts.
Step 3: Reconcile the missed records
A complete panel review of these twenty applications finds six A applicants and eight B applicants who meet the agreed rubric. The tool advanced four of those six in A and three of those eight in B. Each channel also has one advanced applicant who does not meet the rubric.
What would you do here? Open the move you’d make.
Keep using the grade because most advanced candidates meet the rubric.
Tempting, but it misses something. That ignores the seven people the tool missed.
Treat this batch as proof of unlawful discrimination.
Tempting, but it misses something. These channel counts do not establish a legal finding or a cause.
Correct the affected decisions and investigate the screening criterion.
The strong move. This acts on the verified errors and keeps the explanation open.
See how a strong candidate says it
“The screen advanced seven people who meet the rubric and two who do not. It missed seven who do: two in A and five in B. Reopen those records and correct the screen before it rejects more applications.”
Why it works: The review shows specific mistakes in this batch, without proving their cause.
Step 4: Set the next review gate
What would you do here? Open the move you’d make.
Restart once the vendor says the model improved.
Tempting, but it misses something. The local decision quality still needs checking.
Name an owner and test the revised screen on new records.
The strong move. This links the decision to a repeatable review.
Use referral status as the replacement criterion.
Tempting, but it misses something. A channel is not evidence that a person can do the job.
See how a strong candidate says it
“Record the corrected decisions, the criterion under review and who can restart automatic rejection. Check a fresh set against the same job evidence before relying on the grade again.”
Why it works: The next gate needs evidence about errors, not only speed.
The answer, in short
- Decision
- Correct the shortlist and pause unreviewed rejection by this screen.
- Because
- The completed review found seven missed records that meet the rubric.
- What would change it
- A documented criterion fix and a fresh review could justify restarting the screen.
Common traps
- Treating a tool grade as ground truth.
- Inferring a legal finding from a small channel comparison.
Check your answer
- Did I separate advancement from capability?
- Did I inspect rejected records too?
- Did I keep the same job criteria?
- Did I name a restart gate?
Last reviewed Nov 10, 2026