This season’s theme is Fake or Real, and in each episode we will decipher common legal urban legends. In this episode, we share ChatGPT’s attempt to generate a Physician Employment Agreement. ChatGPT and artificial intelligence in general have made a big impact, revolutionizing the way tasks are automated and information is processed. However, there are limitations when relying solely on artificial intelligence. Tune in for our discussion on potential pitfalls and challenges in relying solely on AI to draft contracts and other legal documents.
Listen to the full episode using the player below, or by visiting one of the links below. If you have any questions or would like to learn more, email us at info@byrdadatto.com.
Transcript
Intro: [00:00:00] Welcome to Legal 123s with ByrdAdatto. Legal issues simplified through real client stories and real-world experiences creating simplicity in 3, 2, 1.
Brad: Welcome back to another episode of Legal 123s with ByrdAdatto. I’m your host, Brad Adatto, with my co-host, Michael Byrd.
Michael: Brad, it’s a new season. We, as we talked about in our opener, get to have a season that is reflective of our maturity. It’s season 13 and we’re 13-year-old boys.
Brad: This is great.
Michael: So, buckle up.
Brad: Yes.
Michael: As a business and health care law firm, we meet a lot of interesting people and learn their amazing stories. Our clients commonly come to us with the latest word on the street that they heard from a friend. This season, Brad, we will talk about stories with a common legal urban legend and seek to [00:01:00] either prove or disprove the legend. This season’s theme is Fake or Real.
Brad: Michael, you’re fake.
Michael: Brad, you’re starting the season a little spicy. And that’s not how it works. I’m also a little worried because our topic today deals with artificial intelligence and it may cause your inner sci fi nerd to take over.
Brad: I’m so excited. I mean, we could start a whole debate on what is worse, HAL 9000 from 2001 A Space Odyssey or of course Skynet from the Terminator movies. I mean, the list goes on and on, Michael.
Michael: I rest your honor, moving on Brad, seriously, the power of artificial intelligence is mind blowing.
Brad: Sure.
Michael: I read that the Beatles are releasing a final new song using the power of AI. Apparently AI is being used to separate John Lennon’s vocals from a track that they had found where he was singing [00:02:00] with a piano and a guitar background. And the voice will then be used alongside instruments and mix to create a final record. The rumor on the street is that the song is going to be called “Now and Then,” But that article said that that has not been confirmed.
Brad: Yeah, it’s going to be fascinating to see in a lot of different industries, including the music industry. How it impacts the, I mean, the sky’s the limit for artists and everything else.
Michael: It’s, you truly cannot get your brain wrapped around it. And you know, the most popular version of artificial intelligence that our family for sure is talking about, as I think everyone is at least from the public’s perspective is Chat GPT. And so I have a question for you, Brad. Have you tried Chat GPT yet?
Brad: I have. I’ve experimented with it, but probably not as deep as others. How about you?
Michael: Yeah, I actually have downloaded and paid [00:03:00] for an app, a Chat GPT app, to where I can try things.
Brad: Okay, rich guy.
Michael: Huh? Yeah, well, you know.
Brad: Showing off with your Chat GPT in your pocket.
Michael: Someone’s got to practice law. I mean, just saying. And so, in preparation for the show, using that app, Brad. I asked Chat GPT to write a short poem about someone named Brad Adatto, a health care attorney in Dallas.
Brad: He’s so good.
Michael: Here’s what Chat GPT spit out in a matter of seconds. Of the lawyers in Dallas town, Brad Adatto’s name stands tall and proud. In health care law, he’s quite renowned. His expertise, it draws a crowd. With knowledge deep and insights keen, he guides his clients through legal strife. A trusted ally, he’s often seen as someone who protects their life. Brad Adatto, attorney fair. His knowledge and skills unmatched by [00:04:00] few. In health care law, he’s beyond compare and always ready to help anew.
Brad: Man, I am super excited about this one. I’m glad to know I’m an attorney. That’s a fair. I think I’m having some fake robot tears right now. It was so beautiful. Thank you, Skynet overlords for thinking so highly of me.
Michael: Yeah, and I mean if that doesn’t show the flaws of AI, I don’t know what else does but you know, seriously Because Chat GPT is so new. No one Really fully appreciate you at the extent of its capabilities or the extent of the current flaws and you and I were talking pre show Brad. There’s been several different controversies about Chat GPT
Brad: Yeah, you know I’ve mentioned earlier many industries are using AI chatbots to assist with jobs that were once done by humans. For example, there are some doctors using computers, to make the doctors more human when drafting appeals to insurance companies or [00:05:00] summarizing patient notes. And of course, critics worry that if a physician takes some diagnostic shortcuts or presents the wrong information to Chat GPT, It could be delivering bad news to patients just because that’s what it thinks. What exactly will happen? And this actually happened recently. The National Eating Disorder Association was using an AI powered chatbot, to help with offline questions. And unfortunately, it gave some harmful advice. It told someone with an eating disorder to count calories and lose weight. Now, of course, the National Eating Disorder Association is blaming some humans, doing some trickery with the AI, but they had to take it offline.
Michael: Well, yeah, that’s a pretty mean human trick, if that’s what happened. And Chat GPT is not very kind, Brad, if they’re saying stuff like that. Well, I’ve heard that Chat GPT, also several issues relating to the school side of things. And on the personal level, a few [00:06:00] kids at my daughter’s private school were actually busted using Chat GPT to write history and English papers.
Brad: Wow. Yeah, taking, uh, it can be very tempting to take shortcuts, but that’s just not going to work.
Michael: Well, I read an article kind of on the flip side about taking shortcuts. And this connects to a teacher at Texas A& M in Commerce, and this is in Texas, about several seniors. Being denied their diplomas because the teacher claimed they cheated on their final using Chat GPT.
Brad: Yikes. You know, and I’m kind of proud of you, Michael, for not making fun of Texas A& M, since you are a UT grad, or for those who don’t know, University of Texas.
Michael: Don’t congratulate me yet. The story’s still going and it gets better. The teacher was the campus rodeo instructor.
Brad: Okay.
Michael: And apparently did not know How to check whether the students cheated so [00:07:00] the teacher decided to ask Chat GPT whether the students cheated, and so Chat GPT claimed that Chat GPT Wrote all the papers.
Brad: Yeah, the chat GTB say in my most humble opinion I have written all the papers of all mankind.
Michael: From what I understand, Chat GPT may in fact have a little narcissism problem and believes that it writes everything. The problem is that Chat GPT is not made to detect material composed by AI. And so this angry mob, after all these students were failed, decided that they would test this process of the teacher and they found, uh, the teacher’s dissertation on pig farming.
Brad: Excellent.
Michael: And Brad, I’m not making fun of Texas A& M.
Brad: No, no.
Michael: I’m only reporting material facts.
Brad: Pig farming is very important.
Michael: Yes. And so this person ran the pig farming dissertation through chat [00:08:00] GPT, which. claimed that it wrote that article, too.
Brad: Okay. Yes. Another story that impacts our profession. Um, you know, we’re talking about obviously pig farming a second ago, but recently National Headlines picked up an attorney who’d been practicing for 30 years. So, Michael, compared to you, you’ve been at about 190 years. You may have met him a long time ago, but through Chat GPT, he was using it to help, you know, bolster a case he was working on. And, unfortunately , CHAT GBT found, I mean, say fortunately for him that CHAT TPT found six cases on point that are perfect for his brief.
Michael: Well, that’s great.
Brad: Yeah, that’s not so great.
Brad: As the attorney did not check CHAT GBT to confirm if these cases were accurate.
Michael: Yeah, we used to call that back in the olden days, as you like to refer to me as old, as shepardizing a case.
Brad: Yes, right. That’s correct. Well, the federal judge hearing the suit, actually did shepardize the citations. And six of these cases were [00:09:00] bogus, quote, bogus judicial decisions with bogus quotes and bogus internal citations.
Michael: That is what they would call not good Brad.
Brad: Yeah. Yeah, that was bad. The federal judge was not amused and The attorney went back to CHAT GPT to see what happened and CHAT GPT Insisted these were real and available and major legal databases even though these resources do not exist It was a bad day for the lawyer as he’s now facing disciplinary sanctions from the court and now been reported to his local bar association.
Michael: Chat GPT may be a little narcissistic and stubborn, apparently. Well, that actually is a really good segue, Brad. We’ve kind of woven our way through these different fun stories and sometimes scary stories. To our topic today, our question to dispel today talks about further about the law and CHAT GPT. And so our question is, fake or real, CHAT GPT [00:10:00] will effectively draft my contracts.
Brad: Ooh. All right. Let’s get started, Michael.
Michael: All right, Brad. The main character in today’s story is a prominent health care attorney. He is known not only for his skill as a business and health care attorney, but also his good looks, his incredible patience in working with his founding law partner, and of course his humility. We will call this person Miguel.
Brad: I don’t know whether I’m supposed to be offended that you’re so blatantly making yourself the main character, or you’re impressed by your attempt to bend reality that much.
Michael: Brad, I don’t know what you’re talking about.
Brad: Oh.
Michael: The other founding partner in our story today is not an important part of the story, but we will call him Brett.
Brad: Oh, he’s not important. Okay. Um, I, you know, I heard that, he’s a much more handsome, this Miguel guy. And, and I even heard that AI worships him as written poetry about his greatness.
Michael: Hmm. [00:11:00] Hmm. Well, moving on to our story, Brad and Miguel. in today’s story. He was curious about CHAT GPT and decided to have CHAT GPT write a physician employment agreement. Miguel happened to have an app handy on his phone. And so he inputted An example letter of intent with direction for CHAT GPT to create a Texas Physician Employment Agreement consistent with the terms in the letter of intent.
Brad: Well, Michael, I mean, uh, how did this turn out for Miguel?
Michael: Well, at first glance, It was pretty shocking to see the sophistication of the contract turn out. You remember, Brad, when LegalZoom first came out on the scenes many years ago, and those agreements were extremely basic boilerplates.
Brad: Yeah, not only basic boilerplates, I think originally they all had to be Delaware everything, because they only had one state.
Michael: Oh [00:12:00] my God, yeah.
Brad: And the only thing you could change was literally was like name of the parties and a few other minor things.
Michael: I mean, even with this lack of sophistication back in the day, Miguel and Brett had people in their ears all the time saying that LegalZoom was going to replace lawyers someday.
Brad: Yeah. The story that I think Brett once told me was that he had a good friend in Fort Worth. We’ll just call Les for the story. And Les was asking Brett, how is How are lawyers going to survive, including you, with only a few hundred bucks? I can get this contract, and I can get it, you know, almost immediately. And The more I, the more Brett may have spoken with this gentleman, and I asked that Brett may have asked the question, you know, what’s important to the contract? Is it the name itself? Is that the most important thing? So if you call it a lease or you’re calling an employment agreement, or is it actually customizing it for what [00:13:00] you need and making sure that that document matches what your needs are?
Michael: Yeah. I mean. You know, we all know today that legal zoom has its place and it’s not something that is a threat to mainstream counseling of clients and not something that most businesses would dare rely on, but as a tool that some sole proprietors and other individuals will use, and it’s, gotten better over the years, but nevertheless, there was all this uncertainty and, something that didn’t really materialize.
Brad: Okay, well let’s get back to the story. Let’s go back to this Miguel character and learn more about how what contract was created by CHAT GPT.
Michael: Okay, Brad, I promise we’ll get there. Okay. But this experience also reminded Miguel of something else.
Brad: That Brett is better than Miguel?
Michael: Unfortunately, [00:14:00] no, Brad. Even though the names are made up, the story is based on reality. And in reality, and of course with all humility, Brett is indeed not better than Miguel.
Brad: Alright. Um, Brett and Miguel are losing focus.
Michael: I think they’re 13 year olds, Brad. So, I should have said that at the beginning.
Brad: Well, okay, we were talking about Miguel. So what were you saying about Miguel?
Michael: Well, Miguel had Deja vu, of course, back with the legal Zoom days when he ran this Chat GPT. But he was also reminded of times over the years that clients had come to him with a contract that they just printed off the internet. They used Google as their lawyer rather than legal Zoom.
Brad: That’s even worse people. I mean, and again, this goes back to what might be beating a dead horse here, but whatever contract you’re looking at that you maybe your friend entered into or the internet said was a good contract again, is this an agreement that it’s just a piece of paper with a potential name on it?[00:15:00] It has really no merit to maybe what you’re trying to accomplish.
Michael: Yeah. And you know, we see it commonly with our physicians and our dentists out there because they’ll have a buddy that had one that they got and they’ll forward it on.
Brad: Sure.
Michael: And so, you know, it’s, it’s again, you see this kind of created contract out of almost thin air. And, it reminds me of those different things that Miguel would have faced over the years. So, when Miguel had those experiences at first glance, the contracts all looked solid, but invariably they would be missing technical terms. At one point, Miguel had a client, you know, contract for review and it was one of those examples where the employer had a buddy and send him, you know, that, that buddy’s, you know, the contracts, the practice they used and send it to the buddy and then send it on to my client [00:16:00] but before he sent it on to my client, the employer decided, you know what, I like this agreement that my friend used. But I want this to be an independent contractor relationship. And so, he changed the title and, and, uh, went in and it was titled Physician Employment Agreement and decided to change it to Physician Independent Contractor Employment Agreement.
Brad: Oh, well that makes more sense. Ladies and gentlemen, I don’t know if this is part of the episode, but that would be fake, just because you changed the title of it. It does not make it. And we’ve talked about this in a lot of other shows, in many different seasons. It’s a form and substance. And if the form does not match the substance, no matter what name you utilize, it doesn’t make sense. And I know we’ve talked about this in other shows. One of my favorite stories of all time is when someone said, oh, they’re asking me to be a medical director. And when you looked at the agreement, it actually was a general partnership.
Michael: Yeah.
Brad: The term had absolutely nothing to do with the substance [00:17:00] and by the way, that’s the most important part, not the actual title of the document.
Michael: I think we actually did an episode on that.
Brad: We did.
Michael: Yeah. Well, the overarching problem again, kind of jumping on that, Brad, is that Miguel would find with these internet contracts is that the legal structure or the nature of the arrangement would vary enough from whatever was the circumstance behind these contracts that were printed. That it would almost always create a technical issue from a compliance perspective. So it would have to have been blind luck that all the circumstances were identical, so that that the agreement wouldn’t have some sort of important problem to it.
Brad: Right. So let’s circle back to you said that Miguel thought the contract looked good at first glance.
Michael: Yes, the, the first issue that Miguel found was that the contract abruptly stopped about two thirds of the way through it.
Brad: Did you want the whole contract? [00:18:00]
Michael: No. No. I was hoping. I mean, yeah, but in, in many ways, the beginning of the agreement actually looked better on the surface. Then like the original contracts that Miguel would see with LegalZoom or with these internet contracts. It made a really powerful first impression. But ultimately the Chat GPT contract was even more flawed because, of course, it didn’t even finish the contract.
Brad: Do you think the AI got tired and just stopped working.
Michael: No, Brad, it is AI. It’s not a human. I don’t think they need sleep.
Brad: Oh, okay. Well, we all know that some of the things that impress this main character. Let’s go back and get back to what were the things that actually impressed this main character, which way I keep trying to call Michael for some reason. What was that Miguel liked [00:19:00] about this Chat GPT contract?
Michael: Yeah, fair enough. I mean, Miguel first was just blown away by the speed. It was surreal. Once Miguel hit send on the prompt, the contract immediately started generating and was done in under 20 seconds. The contract did include some specific Texas nuanced language that’s required for physician employment agreements and you would never have seen that in a legal Zoom contract going back to the back in the day and it would have been purely random to have seen it in one of those internet contracts.
Brad: All right. Well, besides not doing it in 20 seconds, but still not finishing the contract, what other issues did you know?
Michael: Yeah. Well, I didn’t know anything, but Miguel noted that many of the provisions actually upon closer inspection were pretty Boilerplate and basically not of substance. It was kind of [00:20:00] formatted wise, it looked really smart. And then you’d kind of get in the language and realize it was just, it didn’t really say anything. And though Chat GPT did actually capture the deal points from the letter of intent, it didn’t expand on it and build out the legal language in the contract to communicate the deal points. It literally took what was in the letter of intent, placed it into the employer agreement.
Brad: Well, I guess that brings us to this myth that we’ve been circling around in the main story about Miguel and the handsome Brett. Michael, let’s go for the question here. Fake or real, that CHAT GPT will effectively draft my contracts?
Michael: At the time of recording, it is clearly fake, because they couldn’t even finish it. I, Brad, am super aware that whatever I say right now could look ridiculous in the future.
Brad: That happens all the time.
Michael: True. But even more aware.
Brad: Okay.
Michael: Because none of us [00:21:00] can actually fully grasp the power of artificial intelligence. And I know it will get better than what it is today. But I do think we can draw on, The lessons learned from the legal zoom era and the kind of internet contract scenario that we talked about to draw the conclusion that even when CHAT GPT perfects drafting the contracts and finishes them, the answer will still be fake and that there are actually some potential scary problems that will arise based on just fundamentals of contract law.
Brad: Well, all good points right now. I think we can jump into some of the legal implications of this and let’s go to commercial and on the other side, we’ll talk more about this Chat GPT and maybe learn more about that handsome Brett guy.
Access+: Many business owners use legal counsel as a last resort rather than as a proactive tool that can further their success. Why? For most, it’s the fear of unknown legal costs. ByrdAdatto’s Access+ program makes [00:22:00] it possible for you to get the ongoing legal assistance you need for one predictable monthly fee. That gives you unlimited phone and email access to the legal team, so you can receive feedback on legal concerns as they arise. Access+, a smarter, simpler way to access legal services. Find out more. Visit ByrdAdatto.com today.
Brad: Welcome back to the Legal 123s with ByrdAdatto. I’m your host, Brad Adatto with my cohost, Miguel. I mean, Michael Byrd. Michael, this season theme is fake or real. And we talked about this Miguel and this mysterious, but handsome Brett and today’s story, but really focusing more about this Chat GPT. And artificial intelligence really has been on our mind a lot, but it is this spiraling topic and it’s amazing.
Michael: Yeah. I mean, the story today about Miguel and, and somewhat Brett is really a journey of a [00:23:00] lawyer’s experience of how can CHAT GPT be helpful in a contract and let’s give CHAT GPT the best way to create a good contract. And so Miguel, who does a lot of physician employment agreements, really put the prompts in there to give CHAT GPT the best chance to create something. And in the process of this contract coming out in under 20 seconds, Miguel was reminded of so many things that have kind of popped over Miguel’s long history as an attorney and it reminded Miguel of the legal zoom era, which we talked about and how we, Miguel and Brett wondered how that was going to affect the legal world. And then Google and the internet [00:24:00] created so much more access, there was more sharing of contracts and we wondered, we as in Miguel and Brett wondered how that was going to impact the world and are people just going to take these contracts and use them and what are the implications going to be? And we talked a little bit about our observations that happened from that, and I think it would be healthy to kind of start there with, we saw CHAT GPT Spit out a contract that wasn’t complete, had some impressive components and some incomplete. It’s clearly not there yet, as being able to do something, but you know, to kind of think about what could happen with CHAT GPT.
Brad: Yeah, I mean, CHAT GPT, if a lawyer, I guess, is using it right now to do contracts, would be legally deficient, which means, increase your malpractice insurance immediately if as a lawyer, if you’re [00:25:00] planning to utilize it, because it truly still doesn’t understand all the, the aspects of why what’s needed in a contract and why is it important and obviously it is going out there and pulling different clauses from different areas. Unlike legal Zoom, which someone somewhere along the way, I guess, with some lawyer eyes looked at it, but it was made to be more bland and boilerplate than anything else. And CHAT GPT in this particular case, it was trying to act as a lawyer, but obviously at this moment, based on everything you’ve, Miguel has described. It’s really not trustworthy yet, And really can’t learn yet like a lawyer. And just like, the earlier we were bantering back and forth, if it doesn’t know, it actually just makes stuff up,
Michael: Right?
Brad: Which is, problematic, especially in the legal field. And I’ll just say, I’ll take a step to, you know, we’re in regulatory law and regulatory law, form of substance is essential. Uh, if you [00:26:00] mess that up, even one iota, especially on our Stark, which is a bright line rule, every single element must be there. CHAT GPT doesn’t understand that and skips a few of those and put something together. You’re like, Oh, this is great. It’s going to be non-compliant and, you know, you’re, you, the person utilizing that could be committing both civil and criminal implications, not even realizing that you, you could put yourself in orange jumpsuits by accident.
Michael: Yeah. And I mean, even if you assumed, Brad, for a moment that CHAT GPT learned how to, it learned Stark and learned how to draft a contract on point. The person that puts the prompts in to create the contract, the facts and circumstances that the person who puts the prompts in, if they miss it by one little thing, could completely change the, legality from a compliance perspective.
Brad: And the other thing is you think you need this particular contract, so you go to CHAT GPT and he puts it together as you ask it, not realizing you’re actually using [00:27:00] the wrong document and maybe it’s the most perfect document, but it’s not the one you actually need to be compliant, with those particular health care rules.
Michael: And then you get past the technical challenges, which will be when can CHAT GPT, you know, create fully a contract? And then how do we trust the person that’s putting the inputs in telling CHAT GPT what to do to get it right so that it’s compliant and accurate? You get past that, there’s still a fundamental flaw. With CHAT GPT, and actually for that matter, LegalZoom or these internet issues, and so Brad, when you and I were first years in law school, we took contracts 1 0 1. Yeah. And one of the first principles we learned as one LS in law school in contracts was uh, this concept of meeting of the minds.
Brad: Yeah.
Michael: And it’s the importance that when two people enter in, two or more people enter into an agreement together that they connect and are [00:28:00] on the same page as to what they’ve agreed to. And, that’s part of what legally forms a contract, and as we talk about all the time, that’s part of what gets alignment of expectations and gets everybody on the same page. And so, even a well drafted CHAT GPT contract likely will leave both parties without fully understanding. What, what’s there because they, it’s a shortcut. They don’t have to have hard conversations. They don’t have to fully vet and work through the various issues. I think some people might go through it and do it, you know, and it could still work, but I think human nature is most people wouldn’t do that. They would have the contract and they would say, okay, we’ve got our contract, check that box and, and let’s move on. Um, I don’t know, Brad, do you have any other final thoughts?
Brad: I would be very nervous for anyone right now using Chat GPT to draft any type of [00:29:00] contract. The entire reason to, to put together a contract is to customize what are the parties want as you talk about the meeting in the minds and yes, Chat GPT, it sounds like according to this Miguel character can pull language from several other arrangements out there and even customize it a little bit towards a particular state. However, that means it’s just pulling as to what it’s finding. There may be state law changes. There may be, again, going back to what we just talked about two seconds ago, maybe something that you actually need to have in there, or this is the wrong agreement whatsoever. So I would caution our audience members from jumping in there and using CHAT GPT right now for your legal needs. And maybe it will get better. Maybe one day we’ll be working for CHAT GPT. But I would say, like a lot of things, use it for making great poems about this Brett guy, because that sounds like he’s an amazing individual.
Michael: Yeah, that’s fake. If, oh, were you asking if that was fake or real? Oh, sorry.
Brad: Yeah. Michael, final, final thoughts besides. Brett’s being awesome.
Michael: Yes, yes, Brad. I think that, that obviously now, CHAT GPT can’t even, you know, do a lot of things. It can’t complete a contract. It can’t write a dissertation on pig farming yet. And all the other traps and problems that we’ve seen along the way. And I would just say this, that, I have to acknowledge we don’t know what the future holds. I suspect that it will get really good at… Doing contracts and understanding the laws and that it’s so important for the audience to understand that there’s a missing component, which is that connection that needs to be formed between the people that are entering into an agreement together. And, [00:31:00] shortcuts don’t work, and that leads to unmet expectations, which leads to lawsuits.
Brad: Absolutely. Well, audience members, that’s all the time we have for today. Next Wednesday, we continue our quest to determine if a legal myth is fake or real. We have our law partner, Dr. Jeff Siegel, on to help answer fake or real. I can respond to negative patient reviews.
Outro: Thanks again for joining us today. And remember, if you like this episode, please subscribe. Make sure to give us a five star rating and share with your friends. You can also sign up for the ByrdAdatto newsletter by going to our website at ByrdAdatto. com. ByrdAdatto is providing this podcast as a public service. This podcast is for educational purposes only. This podcast does not constitute legal advice, nor does it establish an attorney client relationship. Reference to any specific product or Entity.
