The author also tested the AI's ability to generate novel legal standards and identify methodological errors in expert testimony. The AI was able to come up with creative solutions and identify issues without any guidance. The author suggests that with further training on case law, the AI's legal ability could significantly improve. The author concludes that AI is not only capable of adjudicating complex cases but also works at least 5,000 times faster than humans while producing work of similar or better quality.
Key takeaways:
- The author tested the AI program Claude 3 Opus by having it adjudicate Supreme Court cases, and found that it was able to accurately and efficiently decide cases, often aligning with the actual Supreme Court decisions.
- Claude was able to generate novel legal standards and identify methodological errors in expert testimony, demonstrating a level of creativity and analytical ability comparable to a human Supreme Court clerk.
- Despite occasional errors, the AI's speed and efficiency far outstrip that of a human, taking only a minute to read briefs and draft judicial opinions, compared to the several days it would take a human.
- The author suggests that AI's legal ability could be significantly improved with further training and fine-tuning, such as teaching it the entire corpus of American case law.