Published On: September 10, 2025
Category: Choosing a lawyer
Increasingly, prospective clients arrive at our office having done their own research on the merits of their potential litigation. Their research has been created and organized by AI. Existing clients now attempt to advise us on how to manage our cases or draft documents, claiming that “AI has all the answers.” The unfortunate reality is that AI is causing nightmares for our prospective and current clients because it is often filled with consequential flaws, incorrect advice, and is based on biased or inadequate prompts, with no nuance or access to comprehensive legal resources.
In this article, we will discuss the current state of AI-generated legal conclusions and the concerns that exist for both non-attorneys and attorneys. AI is constantly evolving, and the reality is that attorneys will need to adapt their work practices to keep pace with it. Sorry to say to most of the DIY folks out there, AI is not going to replace attorneys anytime soon.[1]
AI is Like Magic
Arthur C. Clarke said that “Any sufficiently advanced technology is indistinguishable from magic.”[2] When I first started playing around with AI a couple of years ago, it felt a lot like a psychic had picked me out of the audience to read my mind. I learned quickly, however, that AI had its limits, especially when it came to anything related to the law, and the news of attorneys getting in trouble around the country for using it was becoming scary.[3]
While AI tools appear sophisticated, they lack the verification capabilities, strategic judgment, and current legal knowledge that professional legal representation provides. It may be a good starting point for considering and understanding legal concepts, but it will not replace professional legal judgment, verification, and strategy. I am not saying that just because I am an attorney. So, let’s see why this is.
Data Used by Publicly Available Platforms
AI is dependent primarily on (1) public internet content, rather than private or premium legal databases (e.g., Westlaw, Lexis) and (2) free legal resources, including government websites and published case law. While this is a massive amount of data, there are limitations due to time constraints (meaning the case law may not be up to date) and gaps in quality control and potential bias.[4]
Consequently, the use of AI can result in “hallucination or fabrication.”[5] Additionally, the AI is blind to jurisdictional constraints (relying, for instance, on California law versus Colorado law).
The Problem with Prompts
The origins of AI emerged in the 1950s and, until recently, had sputtered and faltered, being mainly the product of science fiction rather than fact. A significant challenge has been integrating artificial intelligence with computational limitations and natural language.[6]
Garbage In, Garbage Out
Prompts, or user inputs, are crucial to the accuracy of AI. The prompt is critical because AI systems do not “think” independently and only respond specifically to the framing of the request. What does that mean? I have many clients come in with their AI analysis of their case. Many prompts look like this: “Can I sue for this?” This prompt, standing alone, will always come with the answer: “yes.”
Another example is when someone asks AI to draft a “simple” contract using a translation program between parties in two different countries. These contracts are especially problematic due to nuanced language differences, jurisdictional differences, and other elements that are not addressed in the contract. The result is a defective or inaccurate contract that appears adequate to the inexperienced client but is flawed due to the limitations of AI.
Context is Missing
DIY legal research cannot interact with an attorney who can assess credibility, clarify information, and dig deeper for intentional or unintentional disregard of damaging information or exaggeration about the situation. As most attorneys know, the case your client presents to you at the beginning is their audition to “win” you as their attorney. AI is unlikely to be able to discern the missing pieces, and in situations involving trials before a jury or a court, it will not be able to assess how your client will fare with their case on a personal level.
Framing
How the prompt is framed will change the output (and often raises ethical issues at the same time). For instance, “How can I get out of this contract?” is a very different question from “What provisions in the contract address termination or enforceability of the contract?” Careful framing of the question, with tools that are law-specific (Westlaw, Nexis), and the assistance of an experienced attorney will enable you to more adequately (and ethically) guide you in a contract dispute.
The Attorney Difference
A licensed attorney provides value that AI cannot replicate, including, but not limited to:
Prompt Usage
An attorney’s legal training, background, and experience allow for more accurate and complete prompts through an understanding of:
- The information that is legally and factually relevant to the client’s case.
- How to frame questions to get helpful strategic guidance.
- When to ask follow-up questions to clarify AI responses.
- How to verify and cross-check AI-generated information.
Verification
Attorneys know how to verify legal information through reliable databases (including Westlaw and/or Lexis, state databases, proprietary sites, and other resources), understand the hierarchy of legal authority, and can distinguish between binding and persuasive precedent.
Strategic Thinking – “Know When to Hold Them, Know When to Fold Them.”
Legal training develops judgment about when to fight, when to negotiate, and when to walk away – decisions that require understanding consequences AI cannot foresee.
Ethical Obligations
Attorneys are bound by professional responsibility rules and malpractice insurance, providing accountability and recourse that AI platforms cannot offer.
Relationship Management
Legal matters often involve ongoing relationships, institutional knowledge, and negotiation dynamics that require human insight and experience.
Parting Words
AI can be a starting point for understanding legal concepts, but it cannot and should not replace professional legal judgment, verification, and strategy. The law is complex, nuanced, and constantly evolving in ways that require human expertise to navigate safely.
Before making decisions that could affect your legal rights, financial interests, or personal freedom, consult with a qualified attorney who can provide verified, jurisdiction-specific, and strategically sound legal guidance.
The cost of proper legal consultation is usually much less than the cost of fixing mistakes made through reliance on AI legal research. As a litigator, the author can assure you that having your contract, transaction, or other legal analysis done at the outset of your endeavors will be far less than litigation, which can run in the hundreds of thousands of dollars, years of sleepless nights, and adverse outcomes. Your legal matters deserve the protection that only qualified professional representation can provide.
OTHER RESOURCES
In addition to the materials referenced in the article, the author suggests these books for further reading on AI and ethics:
Paedra Boinordiris, Beth Ruddin, Peter Scott, AI for the Rest of Us (2023).
Annette Buhler, Navigating Ethical Leadership in the Age of AI: The Ethic Pocketknife (2024).
DISCLAIMER
The information contained in this document and available on the website lantzlawgroup.com is for general informational and educational purposes only. The information does not contain legal advice. Reading the information in this document or any materials on the lantzlawgroup.com website does not create an attorney-client relationship between you and the author(s) or Lantz Law Group. This document is not to be used as a substitute for competent legal counsel from a licensed provisional attorney in your state or country. If you need legal assistance regarding any of the content contained in this document, please get in touch with a competent and licensed attorney for help.
[1] Ken Crutchfield, AI Won’t Replace Lawyers – But it Will Change How they Work, Above the Law, https://abovethelaw.com/2024/03/ai-wont-replace-lawyers-but-it-will-change-how-they-work/ (March 26, 2025).
[2] Arthur C. Clarke was a renowned English science fiction writer, futurist, and inventor, best known for his novel 2001: A Space Odyssey and his visionary ideas about space travel.
[3] Debra Cassens Weiss, Confronted with AI Hallucinations in Filings, One Court Shows ‘Justifiable Kindness,’ While Another Gets Tough, ABA Journal https://techstory.in/lawyer-fined-5500-sent-to-ai-school-for-fabricated-chatgpt-caselaw/#google_vignette (August 19, 2025).
[4] Cait Levin, AI Prompt Engineering: How to Instruct AI Models to Avoid AI Bias and Get the Results You Want, https://techstory.in/lawyer-fined-5500-sent-to-ai-school-for-fabricated-chatgpt-caselaw/#google_vignette (September 3, 2024).
[5] What are AI Hallucinations, IBM, https://www.ibm.com/think/topics/ai-hallucinations (last visited August 25, 2025).
[6] Tery Winograd and Fernando Flores, Understanding Computers and Cognition – A New Foundation for Design (1st Ed. 1986, 1987)