Cultural Sensitivity Panel Conducts Review of BOC Exam

One of the 2023 achievements for the BOC was to facilitate a cultural sensitivity review to ensure fairness of the BOC exam to the extent possible. This process is completed approximately every five years, but a recommendation by the BOC’s Inclusion, Diversity, Equity and Advocacy Strategies (IDEAS) Committee prompted this most recent undertaking of a cultural sensitivity review of the certification exam for Athletic Trainers (ATs) to ensure exam content is appropriate and fair for all candidates, regardless of their background.

The primary purpose of the cultural sensitivity review was to eliminate content issues related to race, ethnicity, gender, socioeconomic status, language, age and/or other factors not pertinent to the assessment. Items that are culturally sensitive use language that is fair for all candidates, regardless of their personal characteristics, and do not impede any group of candidates from demonstrating their knowledge of athletic training principles. Items that exhibit bias can provide an advantage or disadvantage to certain groups.

CULTURAL SENSITIVITY REVIEW

To facilitate this review, the BOC recruited a diverse and expert panel of ATs to conduct a systematic review of the examination forms to be used during the 2024 testing year. Their goal, following review was to suggest revisions to address any specific issues found and make recommended changes to the BOC Exam Style Guide that is used to ensure editorial consistency and to carry out recommendations related to inclusion, diversity and equity.

The panel meeting held September 22 and 23, 2023, at BOC headquarters was facilitated by James P. Henderson, Ph.D., of Credentialing Examination Consulting, LLC, with support from BOC Director of Psychometrics and Examinations Ian Hembry, Ph.D. and BOC Exam Development Manager Nathan Burns, MS, ATC.

REVIEW PROCESS

After an initial overview of the purpose and process of the meeting, each panelist answered 250 questions from the full BOC exam without the answer key so they could experience the items from the perspective of candidates and engage in the logical processes involved in selecting responses. For security purposes, panelists used BOC-owned computers.

After panelists completed the exam, there was a training session on the threat to validity posed by culturally insensitive and biased test content. A meeting booklet prepared and distributed to panelists in advance of the meeting provided a written explanation of key concepts related to the exam and the cultural sensitivity review. A guided discussion addressed these key concepts, which included a definition for and consequences of item bias, as well as potentially biasing content to look for when reviewing items. Panelists worked in pairs to conduct their reviews and record observations guided by detailed considerations.

Once the pairs had completed reviews and recorded observations, the full group reconvened to review identified issues, determine if change was necessary and recommend solutions during an item-by-item discussion. Every question that any of the pairs had made a note about was displayed for the whole panel and discussed, with recommendations documented for the BOC Exam Style Guide. Panelists identified changes to be implemented in less than 10% of the 250 questions they reviewed, about half the number of issues found in the prior sensitivity review. Recommendations were integrated into the BOC Exam Style Guide so future forms of the test will include content that is not biased among the cultural dimensions of concern.

This article was originally published in the 2024 summer “Cert Update” newsletter.