Putting Social Work Values to the Test — ASWB’s Commitment to Diversity, Equity, and Inclusion
By Lavina Harless, MSW, LCSW
Social Work Today
Vol. 20 No. 3 P. 24

The role of the Association of Social Work Boards is to maintain the high quality and rigor of social work licensing exams, but it is also tasked with ensuring that social work’s mission, values, and ethics are reflected in the exams.

The licensing examinations for social workers across the country assess competencies in a wide range of areas, including the profession’s values embracing diversity, equity, and inclusion. But can those values be lived out in the process of creating the tests? As a social worker who oversees development of our profession’s licensing examinations, I believe the answer is yes.

“The role of the Association of Social Work Boards examination program is to identify and assess knowledge that entry-level social workers need to demonstrate to ensure that the public is protected. That includes knowledge about cultural competency and ethics,” says ASWB Chief Operating Officer Dwight Hymans, MSW, LCSW. “We strive to carry out these tasks in a way that reflects the very values and ethics we are testing.”

While ASWB’s core mission is to provide services to the regulatory community to protect the public from incompetent and unethical practice, its signature service is examination development. In providing this service, ASWB also aims to adhere to the core values of the profession while it carries out that mission (ASWB, 2018b). Just as the social work profession promotes diversity, equity, and inclusion in society, ASWB strives to embrace those values as it creates and administers the social work licensing examinations.

Touchpoints exist at every step of the examination development process to ensure fairness for test-takers and to make real ASWB’s commitment to diversity, equity, and inclusion.

The Practice Analysis
Long before a test-taker views a test question, or “item,” on a screen in a testing center, a survey—the first step of the analysis of the practice of social work—has assessed what social workers are actually doing in the field. This survey is conducted every five to seven years and helps to ensure the widest possible range of perspectives on what social workers need to know in order to practice safely and competently (ASWB, 2017). The last time it was completed in 2016, more than 23,000 social workers responded.

The survey responses serve as the building blocks of the final examination blueprints, the content outlines for the licensing exams. The path from survey analysis to final blueprint publication is further strengthened by a panel of subject matter experts representing a diverse range of demographics, practice settings, and geography.

Roxroy Reid, PhD, MSW, LCSW, who currently serves on ASWB’s board of directors, says his work with examination development as a member of the 2009 practice analysis task force opened his eyes to the thoroughness of the practice analysis. “The process has rigorous psychometric requirements that must be followed, and the task force was able to balance that with our social work subject matter expertise,” Reid says. “Having a demographically diverse task force of social workers involved helped to ensure that the practice analysis not only captured the breadth of professional social work activities but [also] the breadth of the social workers actually performing those activities.”

Item Writing and Approval
The nearly 100 practicing social workers across the United States and Canada who write items for the social work licensing exams are selected to represent the full diversity of the profession. They write according to guidelines that help them understand factors that could make a question easier or harder for any subgroup of test-takers. These guidelines cover word choice, tacit assumptions, and stereotyping.

Once an item is written, it is evaluated by multiple reviewers, beginning with item development consultants, who are seasoned academicians and/or practitioners. They are selected in part for their ability to ensure that the profession’s values of diversity, equity, and inclusion are upheld in every test question, even when the question does not directly test knowledge of those values. Once items are approved by the consultants, they are reviewed by the ASWB examination committee, a panel of subject matter experts carefully selected to ensure diversity across a range of factors, including demographics and practice settings (ASWB, 2019). The panels work through each item individually and must reach consensus for an item to move into approved status.

Pretesting and Ongoing Analysis
Items approved by the examination committee are not immediately included as scored items on the licensing exams; instead they are auditioned as nonscored pretest items alongside scored items on an ASWB examination. Candidates are presented with 170 items, but only 150 of them are counted toward the final score. The additional 20 items, woven into the exam so they are indistinguishable from scored questions, are monitored to ensure that each meets strict standards for reliability and quality.

That monitoring includes analysis of whether a pretest question advantages or disadvantages one subgroup of test-takers by showing differential item functioning (DIF). Items are also measured to determine whether high-performing, mid-performing, and low-performing candidates across measurable demographic groups are answering individual items correctly at similar rates.

After enough data are gathered, pretest items that fail to meet standards—including those that show DIF and may demonstrate bias—are returned to the examination committee for review and analysis. The committee may decide to further edit the item—in which case it must begin the pretest process all over again—or remove it from the bank of active items. When an item is edited and returned for another pretest audition, it must meet the same statistical goal as before—there are no shortcuts (ASWB, 2018a, p. 10).

Monitoring of item performance doesn’t end once an item moves out of pretest status. Scored items are continually monitored to ensure that performance doesn’t slip. If a scored item demonstrates a statistically significant drop in performance, it is taken out of use and returned to the examination committee for review. Should the committee decide to edit and keep the item, it returns to pretest status.

Form Review
The examinations presented to candidates are assembled from items that have met performance standards to match the test blueprint. Each “form”—a version of a particular examination (e.g., Bachelors, Masters, or Clinical) with different test items—measures the same range of knowledge, skills, and abilities. And while the compilation of a form is largely driven by numerical ratings and the content outline, this process is not purely automatic. Pairs of social work subject matter experts who have experience as item writers and examination committee members review each form. The form reviewers are tasked with approving each item included in every version of every ASWB examination, a process that includes review for potential bias.

ASWB and the Testing Industry
In addition to meeting the social work profession’s expectations that exams will be fair and reliable, ASWB must also ensure that the exams are meeting testing industry standards that apply to all high-stakes licensing exams across disciplines. ASWB meets and often exceeds those standards—and its commitment to professional values and ethics fuels its efforts to do even more.

Thorough subject matter expert review is one process that sets the ASWB exams apart from other similar exams, according to senior scientist Gordon Waugh, PhD, ASWB’s psychometric consultant at Human Resources Research Organization (HumRRO). He says, “ASWB’s test development is essentially equivalent with other professional testing organizations; however, ASWB does tend to gather more information on its tests than most other examination programs. Activities such as its detailed quarterly reporting of test results and item statistics, special studies, and thorough subject matter expert review of new items and items with poor pretest statistics are not typical.”

Another senior scientist at HumRRO, Kevin Bradley, PhD, the industrial organizational psychologist who served as the project manager for ASWB’s 2016 practice analysis, sees ASWB’s attention to individual test question construction as a key tool in its development of reliable and fair examinations.

“One of the most widely accepted strategies to help ensure fairness in testing is to minimize what we in the psychometrics world call ‘construct-irrelevant variance,’ basically, the term for what happens when a test question gives some candidates an advantage if they possess irrelevant knowledge,” Bradley says. An example might be a question that requires social work knowledge but uses a sports analogy. “Test-takers who are sports fans would have an unfair advantage because they possess construct-irrelevant knowledge.”

That’s an obvious example. In reality, the presence of construct-irrelevant knowledge can be much more subtle, requiring a self-aware, strategic approach to test development. Bradley adds that ASWB minimizes construct-irrelevant variance throughout its test development process. “And for ASWB, that strategy involves including subject matter experts from all segments of the social work population,” he says.

Both Bradley and Waugh note that ASWB goes to extra lengths to root out construct-irrelevant variance. “For example, ASWB recently commissioned an analysis of the examinations’ reading levels to ensure that test questions weren’t being written at a complexity level that exceeds what’s required in practice,” Bradley says. “That additional scrutiny adds to ASWB’s tools to minimize the possibility that one segment of test-takers has an advantage over other segments based on factors other than their social work knowledge.”

Attention to Diversity
“Unless you have actually sat in on an ASWB examination committee meeting or served as an item writer,” says Fran Franklin, PhD, MSW, LCSW, “it is hard to get a complete picture of what is involved in the examination development process.” Franklin is a past member of the ASWB board of directors who served as the board liaison to the examination committee. She has also been an ASWB item writer, practice analysis task force member, and standard setting participant. “Every single test question is painstakingly reviewed and rereviewed to make sure it is testing what it is supposed to test,” Franklin says. “The attention to the potential for bias is not just about the subject matter addressed in a question; it is about the smallest details, right down to actual word choice, even including incorrect options on a question.” And though she acknowledges the process is cumbersome, Franklin says, “I believe the results are worth it.”

Gatekeepers of the Profession: A Perspective
I view my job through the lens of our profession’s values and ethics. It’s the only way I can approach my responsibility to the people we serve as social workers, no matter the setting.

I believe it’s the same lens that should be applied to all the gatekeeping processes associated with our profession and beyond. As members of the social work profession, we know that questions about equality and fairness must be a part of the conversation in the widest possible context.

Ultimately, licensing examination programs have a limited scope. They are meant to evaluate minimum competence for safe practice on Day One of the job, alongside other measures that include education and experience. They are, in a sense, snapshots in time of competence. The social work licensing exams cannot serve as outcome measures for social work education, nor can they verify excellence in a particular area of practice. In other words, passing the social work licensing exam cannot substitute for the learning required to be a social worker, nor for the skills we gain through hard-earned experience. In the end, the exams are only a small part of a much bigger picture.

The same is true when it comes to our profession’s commitment to diversity, equity, and inclusion. An examination cannot fix a broader societal issue, though it serves as a mirror that can reflect those issues. Its creators can and should do everything possible to ensure that the exam operates according to the profession’s values. That obligation is made clear in our Code of Ethics, which directs us to, among other things, “promote policies and practices that demonstrate respect for difference, support the expansion of cultural knowledge and resources, [and] advocate for programs and institutions that demonstrate cultural competence” (NASW, 2017, 6.04).

In that and multiple other passages, our code makes it clear that social workers should promote conditions that encourage respect for cultural and social diversity within the United States and around the world. This demands a holistic approach, one in which all stakeholders—including testing organizations, educational programs, and professional associations—look carefully at what they are doing within their own spheres. All must acknowledge the need to address their own piece of a much larger issue and accept the opportunity for leadership in the social work community and the larger society. After all, creating a more diverse, equitable, and inclusive profession requires working toward creating a more diverse, equitable, and inclusive society. ASWB is intent on doing its part.

Mary Jo Monahan, MSW, LCSW, CEO of ASWB, put it this way: “Social workers are especially well placed to understand that our society’s journey toward equity is not yet complete, but we are committed to helping make progress in that direction. ASWB takes seriously the goal of moving the social work profession toward greater inclusion, diversity, and equity.”

— Lavina Harless, MSW, LCSW, is director of examination development for the Association of Social Work Boards.

References
Association of Social Work Boards. (2017). 2017 analysis of the practice of social work final report. Retrieved from https://www.aswb.org/wp-content/uploads/2017/04/2017-Tech-Report.pdf

Association of Social Work Boards. (2018a). ASWB guide to the social work exams (2nd ed.). Association of Social Work Boards.

Association of Social Work Boards. (2018b). Strategic framework 2019–2021. Retrieved from https://www.aswb.org/wp-content/uploads/2018/11/strategic-framework.pdf

Association of Social Work Boards. (2019). Examination program 2019 yearbook. Retrieved from https://www.aswb.org/wp-content/uploads/2018/10/2019-Yearbook.pdf

National Association of Social Workers. (2017). NASW Code of Ethics. Retrieved from https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-of-Ethics-English   

COVID-19 UPDATE
ASWB is maintaining a COVID-19 page (aswb.org/covid-19) with information about changes to exam processes and regulatory provisions. We are encouraging the social work community to access this page or our regulatory provisions page (aswb.org/regulatory-provisions) to get updates while the pandemic continues. We offer a FAQ section on the COVID-19 page about the exams and how to reschedule testing appointments among other exam-related topics.

For social workers, emergency orders are being issued by states to reduce barriers and provide the public with additional access to care. The regulatory provisions page has date-stamped information about the current status of these emergency orders by state. If a state is not listed, there are no emergency provisions as of the date listed. Social workers should go directly to their state board to get the most accurate information and to review the legal and regulatory language specific to social work for their jurisdiction. Our laws and regs database (aswbsocialworkregulations.org/licensingWebsitesReportBuilder.jsp) provides links to all United States and Canadian regulatory boards

— Source: ASWB