Courts Beginning to Set Standards for Evidence Relying upon Artificial Intelligence

In recent years, some have coined the mainstream rise of generative artificial intelligence (AI) a modern-day space race. The legal profession, and especially litigators, have not been unaffected. As the use of generative AI becomes more commonplace in the practice of law, courts have been hesitant to regulate its use in litigation. A recent decision from the Surrogate’s Court in Saratoga County, however, suggests that trend may be about to change.

For better or worse, many of the headlines about the use of generative AI in litigation have focused on the negative. For example, the case of Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023) received a lot of attention after Judge Kevin Castel of the Southern District of New York slapped two attorneys appearing before him with a $5,000 sanction for filing a motion with citations to fictitious cases. Specifically, the court found that the attorneys violated Rule 11 of the Federal Rules of Civil Procedure by failing in their gatekeeping roles and citing multiple fictitious cases created by ChatGPT without authenticating them, and then attempting to justify the citations rather than coming clean about their provenance.

In doing so, the court concluded that the attorneys acted in bad faith based on their acts of conscious avoidance and false and misleading statements to the court. In addition to the monetary sanctions, the court also required the attorneys to provide written notice of the order to their client and each judge falsely identified as authoring one of the fake decisions cited by the attorneys.

There has been less attention given by courts, however, regarding how to handle the use of generative AI when there is no readily apparent defect in its use. A recent decision in Matter of Weber, 2024 NY Slip Op 24258, 2024 N.Y. Misc. LEXIS 8609 (Sur. Ct. Oct. 10, 2024), however, has addressed this question head-on. In Weber, the court conducted a hearing on a claim for breach of fiduciary duty. In connection with that hearing, both parties presented evidence relating to the proper measure of damages, including reports and testimony from damages experts. One party’s damages expert testified that he replied upon Microsoft Copilot, a large language model generative AI chatbot, in crosschecking his damages calculations resulting from the alleged breach.

Although the court independently found that damages expert not credible, it separately addressed the reliance upon generative AI. First, the court focused on the damages expert’s inability to detail the specific process by which he utilized Microsoft Copilot. The damages expert could not recall the specific input or prompt that he utilized and did not know what sources Microsoft Copilot relied upon or how it arrived at the output relied upon by the damages expert.

Given the lack of any evidence as to the reliability of Microsoft Copilot in general, or how it was applied by the damages expert in this case, and coupled with the court’s own endeavors to use Microsoft Copilot to reproduce the damages expert’s conclusion, which reportedly produced three different results when the same query was entered on three different computers, the court found that it could not blindly accept calculations performed by AI as accurate.

Ultimately, the court concluded that, due to the nature of the rapid evolution of artificial intelligence and its inherent reliability issues, counsel has an affirmative duty to disclose the use of AI in connection with any evidence sought to be admitted and any such evidence is subject to a Frye hearing prior to admission.

In sum, there are numerous issues and potential risks to weigh when employing generative AI into legal practice. While the Weber case, on its face, applies only to Surrogate’s Court practice, litigators in all New York courts should take note of its findings and be prepared when intending to rely upon evidence incorporating generative AI to (i) affirmatively disclose the use of generative AI, and (ii) defend such evidence under the Frye standard. Regarding the latter, attorneys should be prepared to establish how the generative AI tool utilized works, what sources it considers, how it was actually utilized in connection with the proffered evidence, and how the generative AI output was verified.

Reprinted with permission from the January 10, 2025, issue of the New York Law Journal^©, ALM Media Properties, LLC. Further duplication without permission is prohibited. All rights reserved.