Many organizational stakeholders often assume that a highly customized test containing items written specifically for their company is the only viable solution to their assessment needs. They may believe this even when there are some very good off-the-shelf solutions that would cost much less and be just as effective, if not more effective, in predicting job success.
This can occur for a number of reasons. One reason stems from the belief that the target role is more unique in the capabilities needed for success than it actually is. A good example of this is first or second line people manager roles. Across many different industries and business units, early career people managers tend to struggle with the same issues: coaching and delegating effectively, managing performance, developing talent, etc.
Nevertheless, stakeholders seeking an assessment tool to help address their people manager problems often feel that the tool will not be effective unless it is customized to their particular organization’s culture, business model, or some other unique facet of their company. As a result, they are not willing to consider off-the-shelf people manager assessments, even when those assessments are supported by strong validation evidence from a variety of companies in various industries.
Another issue that sometimes creates challenges when customizing an assessment is determining which portion of the assessment that needs to be modified. Consider that modifications can be made to any or all of the following:
Look and feel of the assessment platform (if it is an online assessment)
How the assessment is scored
From the list above, assessment items are by far the most challenging, time-consuming, and costly to customize. This is because it is unclear how well the assessment items are working until they are validated through research studies. It takes considerable time and effort to administer the items to enough people to obtain the needed sample size for a validation study, not to mention to analyze and summarize the data, draw the appropriate conclusions, and determine what additional steps, if any, need to be taken to further improve the items.
On the other hand, the remaining items on the list above require relatively less time, effort, and cost to customize. For an online assessment, changes to the platform (if cosmetic and/or minor) and to the scoring are often completed through relatively simple programming work. Changes to the report content may involve a competency mapping if new competency labels are desired. Feedback provided on the reports may also need to be re-written to better align with the target organization’s language. While this work does require some time, it can typically be done within weeks as compared to months for a typical validation study.
In addition, off-the-shelf assessment tools with strong validation evidence are often preferred over a building a new, highly customized assessment when considering legal defensibility, particularly when you can demonstrate the target job requires the same skills or competencies as the other jobs for which the assessment tool has already been validated. This is because there are validation strategies (e.g., validity generalization, transportability studies) that can establish legal defensibility for an assessment that typically require less time, effort, and cost than designing and validating a new assessment tool.
In summary, many stakeholders insist upon building highly customized assessments because they feel it is the only way for it to work effectively in their organization. Often this is simply not the case, particularly for roles in which the capabilities needed for success remain consistent across different industries or organizations.
In addition to the people manager example noted above, this also applies to roles such as Sales and Customer Service Representatives, and entry level production and maintenance roles within manufacturing. There are good assessment tools available for these kinds of roles that effectively predict success in many different organizations, and it simply makes good business sense to consider these off-the-shelf solutions before deciding that a highly customized assessment is the only solution.