Scoring the In-box Exercise
The in-basket/in-box has been used effectively for over 50 years, as a stand-alone assessment technique and as a key part of assessment centers. There are two methods commonly used to score this organizational simulation. Both have been researched and proven effectively. I will mention another method sometimes used, but one I do not endorse.
The first commonly used scoring method involves the assessor reading only the written responses by the candidate, and then rating performance on the specified dimensions. The rationale for this method is that the written responses are what the characters in the scenario would actually see. The scenario is a simulation of what would go back to the person who initiated the item, that is, the subordinate, boss, colleague, person outside the organization, etc. The response is the actual behavior the candidate displayed; it is what really reflects the candidate's thinking and action. The disadvantage of this method is that the assessor must make some inferences about some decision-making processes, and thus assessment of some dimensions, such as problem analysis is difficult.
The second commonly used scoring method involves two steps. First the assessor follows the first method and records a preliminary initial score on the dimensions. Second, the assessor conducts a follow-up interview with the candidate and asks a series of questions. There are two options here: (1) the questions (challenges) can be a standardized list of questions which are asked by each assessor of each candidate. I prefer this option. (2) the questions can be altered to pursue certain issues arising in the initial review and scoring. The rationale for this option is that it is more informative and time-efficient to ask questions that focus on special issues for each candidate. Advocates say this scoring method gives some insight into the thought processes of the candidate and provides information to score some dimensions, such as the breadth of information in the candidate's decision making. The disadvantage of this method is that it reflects the candidate's thinking long after the behavioral responses have been shown. The candidate has had time to re-think the situation, and his/her explanations may not reflect thinking when actually sending out the reply to each item. In addition, the candidate may not be aware of the process he/she actually followed.
Some in-box exercises involve a multiple choice format for the responses. The developer of the in-box generates a list of 4 alternatives and the candidate picks one. The advantage is that the alternatives can be pre-scored and thus a score can be immediately generated. I do not favor this method, because it does not involve the overt behavior generated by the candidate…just a choice of one of the options provided. And the candidate may have wished to take some behavior that is not among the 4 offered.
George Thornton
George.thornton@colostate.edu
22 November 2021
The first commonly used scoring method involves the assessor reading only the written responses by the candidate, and then rating performance on the specified dimensions. The rationale for this method is that the written responses are what the characters in the scenario would actually see. The scenario is a simulation of what would go back to the person who initiated the item, that is, the subordinate, boss, colleague, person outside the organization, etc. The response is the actual behavior the candidate displayed; it is what really reflects the candidate's thinking and action. The disadvantage of this method is that the assessor must make some inferences about some decision-making processes, and thus assessment of some dimensions, such as problem analysis is difficult.
The second commonly used scoring method involves two steps. First the assessor follows the first method and records a preliminary initial score on the dimensions. Second, the assessor conducts a follow-up interview with the candidate and asks a series of questions. There are two options here: (1) the questions (challenges) can be a standardized list of questions which are asked by each assessor of each candidate. I prefer this option. (2) the questions can be altered to pursue certain issues arising in the initial review and scoring. The rationale for this option is that it is more informative and time-efficient to ask questions that focus on special issues for each candidate. Advocates say this scoring method gives some insight into the thought processes of the candidate and provides information to score some dimensions, such as the breadth of information in the candidate's decision making. The disadvantage of this method is that it reflects the candidate's thinking long after the behavioral responses have been shown. The candidate has had time to re-think the situation, and his/her explanations may not reflect thinking when actually sending out the reply to each item. In addition, the candidate may not be aware of the process he/she actually followed.
Some in-box exercises involve a multiple choice format for the responses. The developer of the in-box generates a list of 4 alternatives and the candidate picks one. The advantage is that the alternatives can be pre-scored and thus a score can be immediately generated. I do not favor this method, because it does not involve the overt behavior generated by the candidate…just a choice of one of the options provided. And the candidate may have wished to take some behavior that is not among the 4 offered.
George Thornton
George.thornton@colostate.edu
22 November 2021