2023 Scoring/ Norm Launch FAQ
For internal use only. Please use the external version of this document when sharing with partners and clients.
Question
Response
How did the scoring change for the HPI and HDS and what makes this scoring approach better?
The HPI and HDS both originally used a two-point True/False response scale. In September 2018 Hogan started collecting responses on the HPI and HDS using a four-point response scale (Strongly Agree, Agree, Disagree, and Strongly Disagree). The upgrade to a four-point response scale increases the range of possible responses, which allows participants to choose answers that better reflect how they feel about each HPI and HDS item. This approach allows for more precise scoring of the main scales and subscales, which further enhances the already considerable interpretive power of the assessments.
What is a norm? How is it used?
A norm is a set of data collected from people who have completed the Hogan assessments. We use these norms to convert a person’s raw scores into more meaningful percentile scores, which tell us how that person’s scores compare to the scores of other people in the norm.
For example, if a person has a percentile score of 45, that person scored the same as or higher than 45% of the norm group.
With the move to a four-point response scale, a new data set was required in order to provide the appropriate corresponding percentile scores. We started collecting these norm data in 2018 and finalized this new global norm in late 2022.
What are the benefits of using the upgraded scoring and norm?
By moving from a two-point to a four-point response scale, we are increasing the precision with which we can measure and generate main scale and subscale scores. Think of this as adding more degrees on a thermometer, more marks on a ruler, or more notches on a belt.
Primarily, we expect this increased precision will result in an improved feedback experience for participants. Because we can now make finer distinctions in how people score, participants should find their results to be even more “on target,” particularly at the subscale level.
Additionally, there are multiple benefits in using the related upgraded norm. Because this norm is based on the latest data gathered by Hogan, it provides better representation of work-relevant demographic groups, including age, gender, job, industry, and assessment language. Some notable changes are:
Gender representation improved by 8 points
- Previous Norm: 42% women/58% men
- New Norm: 46% women/54% men
Number of countries/territories increased by 11
- Previous Norm: 179
- New Norm: 190
Number of languages and dialects up to 48 total, with Lithuanian, Montenegrin, and Slovene being new additions
Should I change how I interpret the results?
No. You can interpret main scale percentile scores and subscale “brick” counts in the exact same way you currently do. We are changing nothing about the underlying dimensions or constructs that our assessments measure or predict.
When will the new scoring and norms become available?
Starting June 1, 2023, the new norms will be in place for clients who use off-the-shelf reports and those clients whose custom report scoring was not meaningfully affected by the upgrade.
How do I start using the new scoring and norms?
The move to the new scoring and norms will be automatic for most clients, so it’s likely you won’t have to do anything!
For a small set of clients whose custom report scoring is affected by the upgrade, Hogan will make scoring recommendations and will implement the changes once approved by the client. Any clients using a web service integration should also confirm with Hogan that they are using the latest norm ID.
Is there an additional cost for using the new scoring and norms?
No.
Where can I find more information?
Your Hogan representative can provide you with the full technical documentation for the upgraded scoring and norm.
Why didn’t the MVPI move to a four-point scale?
Four response options will be available as part of the future MVPI revision project (MVPI 3.0).
Where am I likely to see the biggest shift in scores?
Interpersonal Sensitivity (HPI) is the most likely scale to see changes. However, these changes are still smaller than the score changes we typically see when someone takes the current HPI twice.
When does the previous norm go away?
Previous norms will be removed from accounts when the new 2023 norms are added. We will retire all old norms from HALO in 2024.
How does this affect local norms?
Local norms were also updated as part of this process.
When should someone be rescored?
When comparing assessment results across people, we recommend that everyone be scored using the same norm.
People who took the assessments using the four-point response option but who were scored using True/False norms may need to have their reports rescored using the new norms. There will be no additional charge in these cases.
Keep in mind that a person can be rescored using the new norms only if they took the assessments using the four-point response option, which started in September 2018. Otherwise, the person will need to retest if they need to be scored on the 2023 norm.
When would retesting be required?
Our retesting policy remains the same. Although we do not anticipate large shifts in scores, we do expect small shifts to occur. Like most interventions, having the most recent data will usually yield the best results. As such, we recommend people reassess after about five years.
However, major life or work events (e.g., changing companies, getting a new boss, having kids) may have an impact on a person’s reputation and values over time; in such cases, it might be useful to reassess sooner than five years.
Can people be compared if they were scored on two different norms?
We recommend using the same norm any time you compare people. This will help you make the most accurate and fair comparisons.
That said, please note that people’s personality scores are similar across the norm groups (i.e., most people tend to score in the same interpretive ranges on most scales despite the version of the norm they are scored on).
Can people be combined into groups or teams if they were scored on two different norms?
All data being combined for a Group or Team report must be scored using the same norm. Data from people who took the assessments using the four-point response option, which includes most people since September 2018, can be combined into Group or Team reports.
People who previously took the assessments using the four-point response option but who were scored using True/False norms may need to have their reports rescored using the new norms. There will be no additional charge in these cases.
Did any of the OTS selection profiles shift for the 2023 norm?
Yes, although we note many of these shifts are good ones. However, for one profile, AI Professionals, we did see a shift we will address by updating that profile's scoring.
How will this affect a client’s fit levels/pass rates (where applicable)?
In most instances, pass rates or fit levels will show a less than 10% pass rate differences between using the 2023 and 2019 norms.
For any custom scoring with greater than 10% pass rate differences, the Talent Analytics team will recommend scoring adjustments to bring the pass rates within their initial range.
What are the criteria to be classified as a yellow client?
Yellow clients are those who require minimal effort from Hogan and/or the client to be compatible for the scoring and norm update.
Examples are clients with a custom profile that shows no significant impact when run on the 2023 norms and/or has an API integration to get reports or normed scores. See here for more information on communicating with yellow clients.
What are the criteria to be classified as a red client?
Red clients are those who require more meaningful effort from Hogan and/or the client to be compatible for the scoring and norm update.
Examples are clients with a custom profile that shows a significant impact when run on the 2023 norms and/or are hosting the assessment items. See here for more information on communicating with red clients.