This op-ed was originally published in Deseret News on April 11, 2023.
Recently, ChatGPT performed at or near the threshold of 60% accuracy on the United States Medical Licensing Exam without specialized input. This artificial intelligence program has also passed the bar exam and earned passing grades on law school essays. A list of other challenging exams passed by AI can be found here.
The performance of ChatGPT on examinations often used to gatekeep entry to professions reveals a few things. First, these exams don’t do a great job of accounting for the hands-on training or experience often required to become a licensed professional (see ChatGPT passing the medical licensing exam without clinician input). Second, these exam results display that prospective professionals can recall the same fact-based information, but often at a much slower pace than AI.
If ChatGPT has the potential to pass exams used in occupational licensing without any specialized input, are these tests really an effective way to guarantee consumer safety or ensure a qualified workforce?
Ensuring Competency
Undoubtedly, making sure professionals are competent and well-trained enough to safely perform their professional duties is imperative. Nobody wants a clueless surgeon or an untrained pilot. However, current licensing exams for many professions may not be the indicator they are touted to be. Instead, these exams may actually present unneeded barriers to entry for professionals and can disproportionately exclude minority groups from licensure.
An example of this is demonstrated through social worker examination requirements. Many states, including Utah, require or have required various types of social workers to pass a licensing examination administered by the Association of Social Work Boards in order to become fully licensed professionals.
These exams are intended to gauge a social worker’s ability to practice ethically and safely. They are currently mandatory for gaining licensure in nearly every U.S. state at various levels of licensure. However, the necessity and value of these exams have been challenged.
Recent pass-rate data from the social work association has revealed significant disparities in passing rates between white test-takers and those of other racial and ethnic groups. This data suggests that the exams administered by the association may be biased and, as a result, harm hopeful Black, Latino/Hispanic and Indigenous social workers. Instead of serving their intended purpose, these exams appear to inadvertently promote economic protectionism and insulate those already licensed from the potential competition of newly licensed professionals.
It is worth noting that HB250, passed during the 2023 Utah legislative session, removed some of these examination requirements for licensure as a certified social worker or social service worker.
Clearly, these examination requirements need to be rethought in their use or ability to demonstrate competency. Many examination requirements could likely be removed without causing any danger, and for those that remain in place, a greater emphasis should be placed on uniquely human components, not just memorization.
The Silver Lining
Policy Associate
Ben was a Policy Associate at Libertas Institute.