Instructions for use: the AI assessment

Intended to help you choose which solutions to your case you read, and to give participants feedback on their work.

Not intended for:

• using a score or advice as the only reason to hire or reject someone;
• comparing scores between different cases: each case has its own yardstick;
• saying anything about someone's personality or about more than this one solution.

• When the case closes, the AI assesses all submitted solutions at the same time, each one twice, independently. Until then, participants can edit their solution; the AI assesses the latest version. The advice is usually ready within an hour of the case closing; until then, you do not choose yet. The AI measures how plausible the solution makes it that it meets your success criteria (the solution score): a plan cannot prove the future, so what counts is a reasoned estimate based on the baseline, with the assumptions behind it. It also gives a level for each competency according to a fixed rubric, with a verbatim quote as evidence.
• While participants are working on the case, the AI responds once to the first version of each participant's solution: what works, what is missing or does not quite fit, and what questions a decision-maker would ask. Without a score or judgement; the participant writes for themselves what they do with it. If the AI cannot respond at that moment, that step may be left empty.
• Just before submitting, a participant can have the AI check whether the solution covers what you ask for: what the participant is to submit and each of your success criteria, and whether they work out the figures from the baseline, with the assumptions behind them. At most twice. The AI only says whether something is covered, not how well. That result is for the participant only: you do not see it, and it does not count in the assessment.
• If you answer a question in the Q&A round, or add something yourself, the AI receives it as part of the case brief: for the assessment, everything that was there when the case closed, the same for every solution, and also for the response and the check. That way, the AI judges with the same brief that the participants had. The AI never receives who asked a question.
• The AI does not receive any name, email address, school or photo. Participants are asked not to put their name, origin or religion in their solution. The instruction given to the AI says that who someone is never counts.

• For each solution, the core, which the participant writes themselves: one or two sentences. Participants may use AI themselves for this. The core counts in the assessment.
• For each solution, also a summary in two short parts: what the participant proposes, and why it works. TalentProof's AI drafts it; the participant edits and approves it. If the AI did not manage this, the participant wrote it themselves, or left it empty. It says who wrote it. The summary does not count in the assessment. Read it as the participant's pitch, not as a judgement: you read the work itself after you have confirmed the shortlist.
• For each solution, advice in the form of a range, for example “place 2–4”, instead of a score. A range shows how certain the AI is: if two ranges overlap, the AI cannot really tell those solutions apart.
• Until the results are final, no names, only “Solution 1”, “Solution 2”, and so on.
• In the full solution, also the AI's response to the first version of the solution (the step “The AI's take”), with what the participant did with it underneath. Read it as help for the participant, not as a judgement of the solution: that judgement is yours.
• No competency scores or levels from the AI. You assess participants' competencies yourself, with the same rubric. To help you get started, a competency sometimes shows one or more passages that the AI found relevant to it: the paragraph from the solution containing the AI's quote, with the part of the solution it is in. Without a judgement: you decide whether a passage is strong or weak.
• After the final results, if the participant filled it in: how they used AI themselves for the solution. This is voluntary, does not count and is not sent to the AI. You only read it after the results, so that it does not colour your choice.

• Two assessments of the same text sometimes differ slightly. That is why the advice works with a margin: half the spread between the rounds, and at least 0.25 points.
• Every quote from the AI is compared with what the participant actually wrote. A quote that is not in it does not count as evidence.
• A bias test checks whether the AI judges differently in the case of spelling mistakes, simple Dutch, gender, origin, religion, age or a disability. Spelling deliberately counts only towards the Communication competency. TalentProof sends the latest result on request.
• The AI only reads written text: it sees no conversation, no experience and no personality. Someone who writes less fluently may score lower on Communication.
• The advice is only as good as your success criteria and your baseline: vague criteria, or a baseline without figures, produce vague advice. The AI receives the baseline as part of the case brief.
• If the assessment of a solution fails, you still see the solution, without advice. If TalentProof finds a serious problem with the assessment, we switch off the advice for everyone (the emergency stop). You will see this on your case page, and you then continue choosing and ranking without advice.

• Choose for yourself. The advice is a starting point. You choose your own shortlist, as many as your case has winners; nothing is ticked in advance. Read the solutions you choose in full.
• Assess for yourself. Assess the competencies with the rubric, before the results and without names. To do this, read the whole solution, not just the passages the AI points to.
• Answer questions on time, without changing the assignment. During the Q&A round, participants ask questions without their name. Answer within three days; your answer applies to everyone and is sent to the AI. Clarify, but do not add new requirements or criteria: otherwise the AI measures the early submissions against a different bar than the late ones.
• Leave it to someone who can do it. Whoever chooses and assesses on behalf of your company knows the case and knows what this AI can and cannot do (this page).
• Report what is wrong. If you see advice that makes no sense, or a pattern in which a group consistently scores lower, report it to TalentProof. In the event of a serious incident, we report it to the supervisory authority.
• Inform those affected. TalentProof informs participants on your behalf: in the privacy statement, the terms of use and when they submit. If you run a case for your own employees, inform them and your works council (ondernemingsraad) yourself in advance.
• Explanation on request. If a participant asks for an explanation of a decision, forward the request to TalentProof; we provide the data.