Bar Exam Monitoring Software Raises Concerns About Bias, Access
The COVID-19 pandemic set up roadblocks in the path that law students typically take as they enter the legal profession. Traditional law schools rapidly moved to online classes with varying degrees of success. Internships, mentorships, and study abroad programs were canceled or delayed. Law firms abandoned on-campus interviews.
Amid this turmoil and frustration, graduating law students faced another hurdle: the bar exam. Several state bar associations and licensing boards decided they could not herd hundreds of test takers into a large facility for a 2-day exam. After months of delays, many opted for an online test.
In some cases, the online testing software had problems and user support was slow. But for some law school graduates, a bigger worry was the facial-recognition software used to proctor the exams. Facial-recognition technology has a well-documented history of misidentifying people of color, particularly women. Students with disabilities also faced significant challenges.
These and other issues prompted the American Civil Liberties Union (ACLU), the Electronic Frontier Foundation (EFF), and deans of major California law schools to express concerns about the process. Some state bar associations even adopted provisional licensing programs and diploma privileges that allowed graduates to enter the legal field without taking the bar exam.
How the Remote Bar Exam Works
The bar exam is held simultaneously across multiple jurisdictions, typically on the last Tuesday and Wednesday of July and February. In 2020, the pandemic prompted 14 states, the District of Columbia, and the Virgin Islands to delay the July exam until October and administer it remotely. Four states offered the October remote exam in addition to an earlier in-person exam.
ExamSoft testing software was used to administer the October exam. Each test taker was required to take a selfie using the software and complete a practice exam. On the day of the exam, the test taker had to take another selfie with the software before beginning the exam. The two images were compared to ensure it was the same person.
During the test, ExamSoft recorded video and audio of the test taker, whose eyes had to stay within camera range throughout each 60- to 90-minute section of the exam. The multimedia file was uploaded along with the completed exam, and analyzed by artificial intelligence (AI) software for abnormal behavior. The videos were also reviewed by human proctors who provided the jurisdiction with time-stamped incident reports. It was then up to the jurisdiction to determine which cases warranted further investigation or sanctions.
Was the Process Fair?
However, some people of color reported that during the practice bar exam, the software displayed an alert saying that it could not identify them due to “poor lighting.” These test takers began to suspect that their dark skin tone was the problem. Some were so concerned that they took the exam with bright lights shining in their faces to reduce the risk that the software might flag their exams for investigation.
Their concerns were warranted. In 2019, the National Institute of Standards and Technology conducted a study of 189 facial-recognition software algorithms to detect potential bias. Researchers analyzed how well the algorithms performed one-to-one matches (comparing two different photos of the same person) and one-to-many matches (comparing a person’s photo to others in a database). Some algorithms were highly accurate. Overall, however, the algorithms had higher rates of false positives for African-Americans and Asians compared to Caucasians with one-to-one matches, and for African-American women with one-to-many matches.
Facial-recognition software has also exhibited higher error rates for people with certain craniofacial disabilities, and some disabled individuals are simply unable to meet the requirements of the remote test. As a result, the California State Bar required some disabled test takers to participate in an in-person administration of the exam. Several sued the state bar and the National Conference of Board Examiners for allegedly violating the Americans with Disabilities Act and California’s Unruh Act.
Alternatives to the Bar Exam
Some jurisdictions were sympathetic to the challenges facing prospective lawyers. The California Supreme Court permanently lowered the passing score for the bar exam from 1440 to 1390 (while this decision wasn't driven by COVID-19, the pandemic likely influenced the timing). The California State Bar also implemented a provisional licensing program in which graduates practice under the supervision of a qualified attorney.
Louisiana, Oregon, Utah, Washington, and the District of Columbia granted emergency diploma privileges for recent graduates of accredited law schools in lieu of the October 2021 exam. Groups such as United for Diploma Privilege are pushing for more widespread, permanent adoption of the practice. Today, however, Wisconsin is the only jurisdiction that automatically admits graduates of in-state, accredited law schools to the state bar.
The majority of jurisdictions administered the February 2021 bar exam remotely and on schedule. Fewer problems were reported. However, biases associated with race and disability—and even crowded, noisy living conditions and lack of reliable internet access—may potentially prevent capable students from entering the practice of law. These problems also extend to other fields that have licensing requirements. By disrupting longstanding processes, the COVID-19 pandemic has encouraged new conversations about fairness in the licensing of professionals.
Earn a Law Degree Online With Purdue Global Law School
Purdue Global Law School, formerly Concord Law School, is the nation’s first fully online law school, and we are accredited by the State Bar of California. We offer two online legal degrees:
The Juris Doctor, which trains you to become a California-licensed attorney
The Executive Juris Doctor, a doctorate in law for those who wish to build their legal expertise without becoming a practicing attorney
We also offer individual law courses for professionals working in business, IT, health care, government, human resources, education, and other fields who wish to brush up their skills.
To learn more about our online law degrees and courses, request more information today.