Snell v. United Specialty Insurance (Judge Newsom)

What happened

Snell v. United Specialty Insurance Co., No. 22-12581, arose from a landscaper's claim for insurance coverage after he was sued over his installation of an in-ground trampoline. One question was whether that work was "landscaping" within the policy. Judge Kevin Newsom wrote a concurrence describing how he had approached that question. He wrote: "I said to one of my clerks, 'I wonder what ChatGPT thinks about all this.'" He then set out the questions he put to ChatGPT and to Google's Bard, including "What is the ordinary meaning of 'landscaping'?" and "Is installing an in-ground trampoline 'landscaping'?"

He described his proposal as modest: that courts "consider—consider—whether and how AI-powered large language models like OpenAI's ChatGPT, Google's Gemini, and Anthropic's Claude might—might—inform the interpretive analysis."

In United States v. Deleon, No. 23-10478, decided September 5, 2024, he wrote a second concurrence on the phrase "physically restrained" in a sentencing enhancement. He put the same question to ChatGPT, Claude and Gemini several times. He reported that identical queries returned "ever-so-slightly different" answers, and he reproduced the responses in an appendix. He explained that the variation could itself be informative, because "a series of wildly different answers would probably suggest a lack of confidence" in the model's response.

What the court did

In Snell, the panel of Judges Newsom, Branch and Luck affirmed for the insurer on May 28, 2024. The majority resolved the case under an Alabama statute governing insurance applications and did not decide the landscaping question. In Deleon, the panel affirmed the sentence, and the concurrence noted that the court was bound by circuit precedent. In neither case did the majority adopt a model's output as a basis for decision.

Where it stands

Both decisions are final. The concurrences set out, in a published appellate opinion, what a sitting judge asked language models on a legal question and what they returned, and they have drawn commentary from practitioners and scholars. They remain concurrences. They are not a rule of decision in the 11th Circuit.

The obligation it engaged

These are examples of use that met Clause 8 of the Expert Record Standard, disclosure, and Clause 7, contemporaneous record. The judge stated which systems he used, what he asked and, in Deleon, what each returned. The model responses appeared in concurrences, and the judgments rested on other grounds. The use was offered as one input for readers to weigh, which is consistent with Clause 2, non-delegation.

His observation that the same question produced different answers bears directly on Clause 5, reproducible analysis. A chat response is not a re-runnable procedure. Another examiner asking the same question may receive a different answer, which is why the concurrence reproduced the responses rather than one. An expert who consults a model on a definitional point should keep the same kind of record, and should not present a single response as if it were a fixed result.

Sources

• U.S. Court of Appeals for the 11th Circuit · May 28, 2024
  James Snell v. United Specialty Insurance Company, No. 22-12581
• U.S. Court of Appeals for the 11th Circuit · September 5, 2024
  United States v. Deleon, No. 23-10478
• Greenberg Traurig · October 2024
  'Sequel' Concurrence on AI Use in Legal Interpretation

What could not be confirmed

The number of responses reproduced in the Deleon appendix was counted as 30 (10 per model); that count is not stated in this entry. The Snell facts about the landscaper's work are simplified from the opinion.