AI in Legal Exams Controversy Over California Bar Exam Sparks Debate

To save money and allow remote testing, the California State Bar created an exam rather than rely on the questions from the National Conference of Bar Examiners (“NCBE”), as it had done for over 50 years.

The transition from a national exam format to a state-developed one was ripe with technical problems, typos, and poorly written questions. Some test-takers reported that multiple-choice questions had more than one correct answer or were missing essential facts. Several examinees sued Meazure Learning, the vendor hired to administer the test remotely.

While initial criticism focused on the technical issues and vendor performance, the situation escalated once the Bar disclosed that AI had been used to draft some of the exam content without the knowledge or approval of the California Supreme Court, the Committee of Bar Examiners, law school deans, or the test takers.

AI’s Role in the California Bar Exam

The California State Bar incorporated artificial intelligence into its exam development process to modernize test creation and reduce costs.

It relied on a contractor that used AI to help generate multiple-choice questions later reviewed by subject matter experts. AI-generated content, reviewed by legal experts, replaced questions that had traditionally been developed through human drafting.

This shift occurred alongside a $3.8 million cost-saving effort that also included transitioning to remote test delivery and moving away from the national bar exam.

Law Schools and Legislators Demand Transparency

Law school deans, students, and state legislators expressed serious concerns about the fairness and transparency of the February 2025 California bar exam.

More than a dozen deans from ABA-accredited California law schools sent a letter to California Supreme Court Chief Justice Patricia Guerrero, raising “serious concerns about the exam’s fairness and validity” and urging the court to release all 200 multiple-choice questions, disclose authorship of items lacking attribution, and consider returning to the national bar exam system .

Senator Thomas Umberg, chair of the Senate Judiciary Committee, criticized the exam’s rollout as “an unmitigated disaster” and expressed concern that the State Bar may have acted without proper authority.

AI Undermines Confidence in Legal Licensing

The February 2025 bar exam revealed how quickly the use of artificial intelligence can erode confidence in professional licensing when transparency and oversight are lacking.

Although the State Bar claimed the AI-generated multiple-choice questions—23 of the 171 scored items—met statistical reliability benchmarks, many examinees reported that questions were vague, contained typos, or had more than one correct answer .

Despite internal validation processes, this disconnect between technical performance and user experience led to widespread concern.

Legislative Action in Response to California Bar Exam Failures

In response to the backlash, lawmakers acted quickly. Senator Thomas Umberg introduced Senate Bill 47, which would authorize an independent audit of the February 2025 bar exam by the California State Auditor.

The purpose of the audit is to examine both the technical failures during the administration of the exam and the internal decision-making that led to the use of artificial intelligence in drafting exam questions.

In addition, Umberg is advancing Senate Bill 253, the annual bill that authorizes license fees for the State Bar. This bill gives the legislature an opportunity to press for reform by tying funding approval to improvements in transparency and oversight. Together, these legislative actions reflect a growing recognition that maintaining public trust in legal credentialing requires clear processes, strong accountability, and full transparency whenever AI is introduced.

Lessons from the California Bar Exam AI Debacle

This legislative response points the changing landscape of artificial intelligence in professional systems. Rather than waiting for problems to surface, lawmakers are pushing for proactive measures.

With the California bar exam, the lack of stakeholder engagement has become a clear warning to other institutions. Legal educators and elected officials are emphasizing that any effort to modernize licensing or regulatory practices must include all key stakeholders in the process.

The California bar exam debacle offers a crucial lesson: new technology must not be adopted at the expense of professional integrity. Institutions must lead with transparency and prioritize the public interest if they hope to unlock the benefits of AI without undermining trust.