This op-ed was originally published in Deseret News on April 11, 2023.
Recently, ChatGPT performed at or near the threshold of 60% accuracy on the United States Medical Licensing Exam without specialized input. This artificial intelligence program has also passed the bar exam and earned passing grades on law school essays. A list of other challenging exams passed by AI can be found here.
The performance of ChatGPT on examinations often used to gatekeep entry to professions reveals a few things. First, these exams don’t do a great job of accounting for the hands-on training or experience often required to become a licensed professional (see ChatGPT passing the medical licensing exam without clinician input). Second, these exam results display that prospective professionals can recall the same fact-based information, but often at a much slower pace than AI.
If ChatGPT has the potential to pass exams used in occupational licensing without any specialized input, are these tests really an effective way to guarantee consumer safety or ensure a qualified workforce?
Ensuring Competency
Undoubtedly, making sure professionals are competent and well-trained enough to safely perform their professional duties is imperative. Nobody wants a clueless surgeon or an untrained pilot. However, current licensing exams for many professions may not be the indicator they are touted to be. Instead, these exams may actually present unneeded barriers to entry for professionals and can disproportionately exclude minority groups from licensure.