UNC Law’s October 22 AI courtroom exercise gave students a simulated appellate argument with AI judicial characters. The public program proposed comparing the AI’s reasoning and result with a previously decided U.S. Supreme Court case. That establishes a setting for examining generated legal reasoning. It doesn’t establish that students learned how to use AI responsibly in legal practice.
That distinction matters. A simulated judge can reach the same result as the real Court and still offer little evidence about whether students can spot weak reasoning, check legal support, or decide when an AI answer should be rejected. In the public materials checked through October 26, I have not located a transcript, student assessment, debrief report, or rubric showing those skills were taught or measured.
A weak AI answer could still teach something
WRAL reported that the ChatGPT-powered judicial roles used faculty-supplied case documents and persona files, could question advocates and refer to the other simulated justices, and ultimately reached the same result as the real Court. The report describes the aim as examining AI’s legal reasoning and how lawyers could use it, rather than replacing judges. It also says an official at the law school gave the system a 1.5-out-of-10 assessment, particularly because of its conversation with advocates.
That 1.5 figure is the official’s judgment about the simulation, not a student grade or an accuracy rate. And a flawed exchange needn’t make the class a failure. If students had to identify where an AI justice missed an argument, return to the case record, and explain a correction, the model’s weakness could become the point of the lesson.
But that is a possible teaching design, not a documented outcome. The official program describes what the session was meant to do: use three AI justices with different approaches, let students argue a decided case, see whether advocacy changed AI positions, and compare the result and reasoning with the real Court. It does not say how students’ work was evaluated.
The useful test is student judgment
A classroom exercise doesn’t need to prove that AI can handle legal work generally, and it doesn’t need to become a formal research trial to be useful. Still, a claim about preparing future lawyers calls for evidence about their judgment, not only the machine’s performance.
The clearest evidence would be student work showing that participants could identify an unsupported claim, verify a legal authority, explain why an answer failed, and say when they would not rely on it. A before-and-after assessment or a published debrief could help answer that question. Without it, the public can see a demonstration of AI-generated judicial reasoning, not proof that students acquired transferable professional judgment.
There is one technical limit worth keeping in view. WRAL reported that the system had no live internet access or outside documents during the exercise. That does not settle whether the underlying model had encountered the case before the event. The checked sources also do not identify the case, model version, prompts, persona files, or full exchange, so outside readers cannot independently assess why the AI reached its result.
UNC Law’s exercise may have given students a useful reason to challenge an AI rather than defer to it. The public materials checked so far don’t establish whether that happened. A transcript, debrief, or assessment of student work would turn the open learning question into something readers could evaluate.