Have You Encountered This Problem?
Early candidates in a high-stakes Assessment Center (AC) for selection or promotion,
give information to later candidates.
Situation:
A large number of candidates for promotion participate in an assessment center over a period of two or more days.
Problem:
Despite warnings to keep information confidential, candidates in early days tell some candidates in later days about the content of simulations and other procedures. As a result, the process is not standardized, equal, and fair to all candidates. Later candidates who get information have an advantage over other candidates.
Solutions:
1. Level the Playing Field with Orientation.
Require all candidates to attend an orientation session prior to the AC in which all facets of the AC are described. Tell candidates the dimensions/competencies that will be assessed, including the names and definitions. Do not tell the specific preferred behaviors in any rating standard for any exercise. Tell candidates about each type of simulation exercise, for example "You will participate in a group discussion with other candidates to analyze problems in an organization and write suggestions for management." Do not state the actual content of the problems. Give candidates tips and advice on how to approach exercises. For example, "Thoroughly read the instructions. Plan your work so you complete each exercise in the time allowed."
2. Assess Maximum Performance Competencies with Difficult Exercises.
Assess only those competencies that require candidates to demonstrate their best level of performance. Make the simulations difficult for all candidates. Use rating standards that require high level competency. Thus, even though a candidate may know the case study will involve analyzing a complex marketing challenge, the complex information and data in the case will require skill in analysis, decision making, and judgment. Another example: just because a candidate knows she is supposed to deal with a role player who is a poor performer, doesn't mean she has the ability to be a good coach.
3. Build Two or More Parallel (Comparable) Sets of Exercises.
Build parallel sets of each exercise. Administer Form A on Day 1 to one group of candidates, and Form B on Day 2 to another group of candidates. Parallel forms are the same type, have the same complexity and difficulty, and measure the same competency. But they have different content. Examples: in the in-box, change the names of characters or requests for advice; in the case study, change the financial data which must be analyzed.
4. Maintain Complete Separation of Candidates.
Example, say you have three simulation exercises. Administer each exercise on 3 consecutive days to all candidates. On day 1 have the first wave of candidates arrive at 8:00 a.m., take the In-Box, then go to a sequestration room. Wave 2 arrives at 10:00, takes the In-box and goes to a second sequestration room. On day 2 and 3 follow the same procedure for the other simulations.
Variation: have waves 1 to 4 arrive at 8:00 and go to sequestration. Wave 1 takes the In-Box, then goes to another sequestration room. Waves 5 to 8 arrive before Waves 1 – 4 are released.
NOTE: while in sequestration, candidates can complete other assessment techniques, such as psychometric tests or personality questionnaires, OR they can be engaged in training activities.
These solutions can be used singly or in combinations.
George Thornton
George.thornton@colostate.edu
22 November 2021
A large number of candidates for promotion participate in an assessment center over a period of two or more days.
Problem:
Despite warnings to keep information confidential, candidates in early days tell some candidates in later days about the content of simulations and other procedures. As a result, the process is not standardized, equal, and fair to all candidates. Later candidates who get information have an advantage over other candidates.
Solutions:
1. Level the Playing Field with Orientation.
Require all candidates to attend an orientation session prior to the AC in which all facets of the AC are described. Tell candidates the dimensions/competencies that will be assessed, including the names and definitions. Do not tell the specific preferred behaviors in any rating standard for any exercise. Tell candidates about each type of simulation exercise, for example "You will participate in a group discussion with other candidates to analyze problems in an organization and write suggestions for management." Do not state the actual content of the problems. Give candidates tips and advice on how to approach exercises. For example, "Thoroughly read the instructions. Plan your work so you complete each exercise in the time allowed."
2. Assess Maximum Performance Competencies with Difficult Exercises.
Assess only those competencies that require candidates to demonstrate their best level of performance. Make the simulations difficult for all candidates. Use rating standards that require high level competency. Thus, even though a candidate may know the case study will involve analyzing a complex marketing challenge, the complex information and data in the case will require skill in analysis, decision making, and judgment. Another example: just because a candidate knows she is supposed to deal with a role player who is a poor performer, doesn't mean she has the ability to be a good coach.
3. Build Two or More Parallel (Comparable) Sets of Exercises.
Build parallel sets of each exercise. Administer Form A on Day 1 to one group of candidates, and Form B on Day 2 to another group of candidates. Parallel forms are the same type, have the same complexity and difficulty, and measure the same competency. But they have different content. Examples: in the in-box, change the names of characters or requests for advice; in the case study, change the financial data which must be analyzed.
4. Maintain Complete Separation of Candidates.
Example, say you have three simulation exercises. Administer each exercise on 3 consecutive days to all candidates. On day 1 have the first wave of candidates arrive at 8:00 a.m., take the In-Box, then go to a sequestration room. Wave 2 arrives at 10:00, takes the In-box and goes to a second sequestration room. On day 2 and 3 follow the same procedure for the other simulations.
Variation: have waves 1 to 4 arrive at 8:00 and go to sequestration. Wave 1 takes the In-Box, then goes to another sequestration room. Waves 5 to 8 arrive before Waves 1 – 4 are released.
NOTE: while in sequestration, candidates can complete other assessment techniques, such as psychometric tests or personality questionnaires, OR they can be engaged in training activities.
These solutions can be used singly or in combinations.
George Thornton
George.thornton@colostate.edu
22 November 2021