Name: Can AI help manage risk? Bet on it.
Uploaded: 2025-09-15T21:16:09.143Z
Duration: 53 min 57 s
Description: Can AI help manage risk? Bet on it.

Transcript for "Can AI help manage risk? Bet on it.": Good morning, good afternoon, or good evening to everybody. I know we have people from all around the world joining us. So thrilled, you were able to come and, spend a little time with myself and Nate here. Nate, special thanks to you for making time. Thanks for having me. You and I have already spoken, so I know you're fascinating. Why don't you just share a little bit with the, with the audience about who you are, where you come from, and and kind of what your expertise is here. Sure. Thank you. So, I'm Nate Kostelnick. I'm a lawyer. I'm in Pittsburgh, and I'm currently with Oncology Nursing Society. We are a international nonprofit, trade group representing everybody in the oncology nursing space. And, in my day to day, I work a lot with contracts, especially for our our purposes here, more involved with AI, and IT as well. So thanks again for having me. I'm looking forward to the discussion. Fantastic. And and thank you for all the work you do. I thank your company for like, you know, I sit here and make software. Actually, I don't even make software. I tell people about software that somebody else makes, and you guys are saving lives. So, appreciate having someone on here whose whose work really is is making a difference. That all being said, we're here to talk about AI and, and, risk. So we sort of made ourselves some agenda here, just to keep us on track. So what we can do is we can talk about, sort of what risk is in legal, the past, present, and the future. We are gonna talk about consistency and how building a playbook, can really help us to deliver consistent and low risk results. We're gonna talk about a concept that Nate Kostelnick absolutely came up with the last time we were talking, responsible playbooking. I wrote don't ask dumb questions. I think that's just me being, being a marketing guy, but I think that it's I think you're gonna really love it. And if you take away nothing else, responsible playbooking will be the best. I will tell you a little bit about trends and evolution, the value of tracing, how clauses show up from a risk perspective, and then sort of some next steps. How can we, you know, make this real? What can we do? What are the steps that people can take when you when you go back to the office or back to your your computer on the corner of the dining room table as as they be at this point? And it'll it's gonna really revolve around automation, standardization, and optimization. Alright? Again, as Jennifer said at the beginning, if you have any questions, things you want to drop in, please do drop something in the q and a. We're gonna have fun chatting, but we'd love to answer your questions as we go. Alright. So step one, risk in legal, the past, the present, and the future. Nate, you and I were talking about this just the other day. Tell us a little bit about what you think about sort of from the perspective of what are the risks that you're always watching out out out for and and how some of them are just sort of always there. Sure. I mean, we're we're trained to to spot risks everywhere and to find them, even when they may not be there. So we're always staying busy looking for things that could go wrong, the worst case scenarios, and even the the smaller, more operational failures, so collapsing all of that kind of under the umbrella of risk. Fortunately, today, we have a whole new, garden variety of risks that we're gonna see with AI. We're seeing new ones come out almost every single day. Someone has a new issue, they're raising, hey. Here's something to think about. Here's something new, and that's only gonna accelerate. So, more risk going forward. Right? Yeah. I think that's I think that's true. And I think one of the interesting things is how risk changes over time. Right? As well as those classic adages, you know, the only thing we know the only thing that's consistent is inconsistent or is change. We're about to release, I think next week, a survey that we ran around tariffs, actually. And and what was fascinating was that at this stage, tariffs have become one of those huge issues that nobody was thinking about. You know, like, a year ago, maybe a year ago, we started to get a bit worried. But, you know, tariffs in in The United States particularly have been around about two to 3% for a really long time, and we sort of got complacent. Right? But now we're hearing some folks that what have we got here? Something like, 54% of the respondents we had to our survey said, yep. Tariffs have become like a fundamental part of their life as they try to not only, negotiate appropriately into new contracts, but as they try to assess the risk that they have in their current corpus. I'm assuming you don't have a whole lot of of, tariff risks in in your Yeah. We personally don't work too much with tariffs. We were much busier, in the COVID days, like everybody else. But one of the one of the takeaway we had from that is, not just considering the risk from our perspective, but seeing the risks that our counterparties will identify and try and mitigate, in in their contracts. So being able to see what, the trends seem to be, what certain vendors are doing and asking for, that enables you to see also maybe where another vendor is taking a position that's that's this is something entirely new. This is this is way off the market. So you can kinda gather that new information and see where things are changing, across all your different counterparties. Yeah. I mean, I think, you know, I COVID's a good example. Right? It was a it was a risk none of us have really taken into account. And it appeared, and all of a sudden, it was the only risk that we had to talk about. So I think the unpredictability of of risk is is really, you know, it's it's kind of part of the game. Right? Risk is not just constant. Risk is ever changing, which makes it, I guess, by turn even riskier. I I I do know that, in fact, when you and I were talking the other day, we were talking about, you know, we're going to discuss how AI can help you with risk. But actually, AI even introduces some risk. Right? A lot of risk. Right? It as we said before, it seems like there's a new issue cropping up, every day that's that's AI related and isn't necessarily something that AI can solve. It it's just a new risk presented by AI. So we're all familiar with, you know, is this AI tool training on our data? How secure is the the information we input, into the tool? What else is it doing with our our, outputs and our prompts? So those are new questions that we have to now evaluate on a case by case basis with every new vendor we interact with. Not all of them are taking the same positions and security posture, so it's, it's learning as we go and finding out what some vendors are willing to commit to, what some aren't willing to commit to, and and there's risk right there. We we aren't able to have kind of consistency, across all the the counterparties. So that's one one big risk. And then there's there's a whole bunch of new ones. One of the ones I'm reading a bit about is prompt injection and how if you, you you know, we're all familiar with the chat box and whatnot, but you can get, a prompt that will kind of inject new instructions into the way you interact, with the tool, and it can kind of go haywire. So there's a lot of risks that are popping up with with stuff like that too. So, brave new world. Right? Yeah. Yeah. And and that on top of it hallucinating case law, which is my personal favorite. That seems to hit the news every every week or two, which I think really sort of speaks to where we're going next, which is, you know, putting a ring fence around what it can do. Right? And one of the things that we've we were talking about last week is how consistency is what breeds security. So just throwing your contract at some random chat GPT type thing is almost by definition gonna get you different results every time. And so we were talking last week, about how you use playbooks when you're using AI, with your contracts. Give us a give us a sort of a an overview of how you go about that. Right. So, contract play booking has been around for a long time. Many of us have a big long table in Excel or Word where it lists clauses or issues and it takes you know, then it has our preferred position, you know, indemnification. Here's what we we absolutely wanna see in an indemnification clause. You may have some fallbacks where the the counterparty won't go with our preferred position. Here's our fallback. You probably have some notes for how to present that position to your counterparty and and, argue for it. And then maybe, an approval and escalation path where, hey, if if the first few options didn't work and the counterpart is really digging in, what do we do? So that's really where things were. As I said, you know, Word document or an Excel table, and you'd kind of, or at least what I used to do, I have my double monitors. I'd pull up a contract I'm reviewing on one screen and my playbook on the other, and you, you start kinda manually going down and saying, here's what I need to put in. Here's where I can push back. And so you you do it a little bit manually. But now, with AI, we're seeing ways to, kind of implement playbooking at scale. And so you can take that manual manual playbook and use an AI tool that will kind of scan your contracts using that as a guide, so you get, much faster review, much more more efficient. You'll get red lines and you'll get some comments. So now instead of me doing that manual review, I have some playbooks that I run on certain contract types where I click a button or two, and I'll get almost instant feedback on what I would consider our preferred positions. We'll get some markups, and that first pass review is now done. And then I'll step in and I'll do a second pass review. I'll look at what the AI has done, and then I'll read, maybe the other clauses to see if there's anything else in there that I should should tweak. I go through the the red line comments that the AI has made. I said, okay. Maybe this one isn't that important. We don't need to make that change. If there's another one that's more significant, we'll keep that, and then pass along to the counterparty. And before you know it, that has saved me, quite a bit of time. Yeah. No. That's great. And I and, you know, obviously, I was fascinated to hear you building these, where you are. It's a it's a key element of one of the products that, you know, we offer here here at Agiloft, a little product called screens. And I'm not gonna I'm not gonna go into a sales pitch for everybody, but I think that what I found fascinating, about screens is that it took a very different approach to contract review and redlining to what almost all of the rest of the CLA industry was doing at the time to quit. To your point, you create a you create a playbook. You have standards. And that basically, yes, no questions. And every time you get a contract, you just run a a playbook against the screen, as we call it. And it pumps out your answer to your questions along with some suggestions for improvement or or for, you know, sort of realigning, I guess. And I think that that kind of shook us all up because it turns out that that really is a great way to standardize the approach you take and to make it appropriate to the type of contract. Right? Because different kinds of contracts, you get different kinds of results. Right? Or you you you want to get different kinds of results. And we were talking the other day about, you know, kind of having a distinction between playbooks you use for kind of I I wanna be whatever you wanna call it. So standard types of contracts you get all the time versus the more complicated one. So do you think there's a difference here in how we use AI based on the type of contract we're looking at? Yeah. Absolutely. And that's that's how I have it set up and and a lot of others do as well. So you'll have maybe an NDA playbook. You'll have a a SAS playbook if you're the vendor and if you're the customer. You'll also have probably a MSA or something like that. Again, vendor and customer, those are kind of the standards. Depending on your industry, you probably have some other ones that are more specialized. We work a lot, with health care institutions. So we have a business associate, playbook as well that we can kind of run, and make sure that we're consistent, when we work with all those different customers because, not all of the the counterparties are gonna have the same requirements and standards in their contracts. And so a playbook allows us to kind of develop a sense of consistency to know that the clauses and positions we really care about are always gonna be, checked for, and reviewed. So that really enhances, the consistency as you mentioned. Interesting. Yeah. Yeah. So it's, you know, one of the things that we we we talk with our with our customers about a lot is how to get the right set of questions and the right sort of depth, for each individual thing. I think with the sort of the simpler style, your NDA end of the pool, if you will, you can be quite specific and you can be quite direct. And as you move your way up into the deep end where you've got larger, more complex, more, want of a better word, creative contracts. Now you're really getting more guideposts rather than answers. Yep. Nonetheless, I think that there's a a real benefit in taking a consistent approach to each class of contract. Right? It's it's you're not just teaching an AI. You're teaching yourself, I guess. Right? You still have to have that human sense that you understand what's going on. Right. And especially, you know, for for some of the small teams that, I'm a team of one. There may be some other smaller teams. You may have one or five. And then you have the larger teams on the other end, you know, 50, a 100. And one of the challenges is, getting the consistency across those teams, and also getting the consistency every time the same person reviews a contract. You know, the one thing you don't want is to review, for me to review a contract on Monday and make certain changes and then review the same type of contract on Friday and and maybe not take the same consistent approach. So, the the benefit here is standardization and consistency so you can ensure that everybody's kind of reviewing with the same perspective, and and the same checklist, so to speak, of what you're screening for, but also that that individually, you're doing that as well. So it kinda keeps you honest from that perspective as well. Yeah. It's sort of just bringing your your your dual monitor thing you talked about a moment ago to life. Right? It's like it's really committing to that to that that set of clauses you have in the background. And and that actually brings us, I think, really to this this concept you came up with the other day, which I just thought was was genius of responsible playbook. When you rather than me guessing, why don't you explain what you were thinking about when we were talking about that? Sure. So, I'm sure we've all been in the situation where, you have your your standard contract, you send it to the other side, and instead of getting markups or anything like that back, you just get an entirely new document. And it's well, we don't wanna use yours. Here's our standard MSA. So now you're starting from square one. And we all know that that that's frustrating. It slows things down. Their template isn't customized to your business operations. So it it's just a very frustrating experience, and you're never gonna get, rid of that entirely. But with playbooking, especially with AI, I think it's our our our chance to work to avoid that. And so when you create your playbook, I think the last thing you wanna do is just use that as a stand in for forcing your own template on the other side, because that's not really gonna get you everywhere you wanna be. And so we all obviously want to include certain concepts and certain provisions in our playbook. But I think as we're creating our playbook, it's really valuable to take the time and figure out what we don't necessarily care about and what we don't want in our playbooks. Because Yeah. The more stuff we put in I mean, we could have, you know, 70 pages or 70 clauses that we're gonna check for, but all that means is that we're gonna send 70 red lines back to the other side, and now it may come back to us, and we may have now 70 issues to negotiate. If we don't care about all of those, let's take the time upfront when we're creating our playbook and eliminate some of them. Do we really need this? Do we really wanna check for it? Do we really wanna force this red line on the other side? Do we really wanna negotiate it? Because now if we create our playbook and we do it responsibly and we put the thought and effort into it upfront, we can reduce the issues we're gonna screen for. And now we're gonna have, you know, faster deal time, fewer issues to negotiate, and and hopefully a smoother transaction. Because when you exclude something from your playbook, every time your playbook runs on a contract, you get that benefit of that time saved and that issue being not negotiated at scale. And so it's a it's a real time saver, and that's really where some of that efficiency lies in taking your judgment upfront and saying, this really is an issue we care about, or do we really wanna waste our time on this every contract that we review? So that's that's what I do with responsible playbooking. It's a challenge because we're always tempted to just include more, but even just taking that time upfront, I think pays off. I I mean, I think I just think that's brilliant lateral thinking. Right? I think as a as a lawyer, you're trained to look for risk. You're looking to absolutely minimize the the the potential for bad things to happen, but recognizing that sometimes over negotiating is actually the risk, is is quite fascinating. I I would share a anecdote I shared with a chef who's one of our customers, a couple of months ago. And one of his biggest challenges is getting all of the contracts into the CLM system, which if anybody has CLM, I'm sure you're all gonna go, yeah. No kidding. It's everybody's problem. Right? Because somebody down at the other end, one of your business partners, has, like, a teeny tiny little contract, and they just don't wanna deal. And so what he's come up with is a plan to use the screens screening that I was talking about just a couple of minutes ago and essentially devolve that down to his business partners for certain kinds of contracts. So now he can say to them, listen. You don't have to wait for me to to deal with it. All you gotta do is run it against the screen, assuming there's nothing crazy out there. It's gonna tell you, yes. Please go ahead and sign. Right? So if you're gonna have someone who's gonna turn up during the autumn months and sweep the leaves from the front of the building and you're gonna pay them $300 a month, yes. We should have a contract because, you know, this is America, but I don't need to spend legal time on it. As a result, the, the deal he's made essentially with these people is you are officially good to go. All you gotta do is stick it in the machine and run the stand run run this this playbook. Fantastic. Everybody out there not only gets to get their contract signed almost instantly, they also get to be in compliance with the company, and the legal team has the contract in the file just in case. And that also brings value, obviously, beyond legal into the procurement team who now have visibility over all of these teeny tiny little contracts that are getting signed across, the whole organization. So everybody wins with responsible playbook. Right? When you think about under certain sets of circumstances, what are the things that I need to know? What are the things I need to worry about, what are the things that really just are ancillary to the way we're running the business. I thought it was I thought it was fascinating way of getting it done. Yeah. I mean, anything you can get into your CLM as opposed to having the rogue contracts floating out there that, you know, haven't been seen by anybody. Who knows what's out there. You can't track them down. So anything you can do to use that consistency to just get them and get them into the CLM, is a big win. Absolutely. One of the first things I learned when I when I came to, AdTalk a couple of years ago was one of the great signs that CLM implementation is working. And this was with a few customers who've done it specifically, actually, to deal with NDAs. They said one of the first signs it's working is we see more NDAs than we've ever seen before. I think that's everybody which is taking the last one and going, I'm not waiting for three days. I'm just gonna switch out the names and make a copy and ship it out. Yeah. And now it's just out there floating. So, you know, this is why I think responsible playbooking is such a great concept you've come up with here. It's really not just how do I take my old set of clauses that were kind of my, you know, my defacto standards and see if they fit. It's how do I make sure I'm only looking after things that are really gonna make a difference. It's just a brilliant, brilliant concept. Yeah. And so I think there's some some practical, things you could do with that. One, as you mentioned, I think an easy easy lift, to get people started and to get stuff into the CLM is have a a screen for the right party's name. You know, you're you're gonna be having people change their party's names anyway. I work for an organization where we have a few different affiliated organizations that all sound the same, And so it's really easy for, you know, the wrong party to just get inserted, but that's an easy fix. That's an easy lift for for AI, where you can have that caught ensure consistency, and that's something the business team will love to know that it's an easy, error that that they caught, they can replace, and then you get it in the CLM. So that's that's a great way to get started. And then some of the other stuff is, you know, kind of review your boilerplate. I mean, if you're asking for 10 or 15 issues on your boilerplate in your playbook, that's a great place to go to say, do we do we really wanna ask about all these issues? Let's see if we can narrow it down. So there's another, way to kinda trim your playbook or think about trimming your Yeah. It's a great idea. And I mean, the last the last thought that I put up there, because I I did pop it off the screen. I forgot it was there. The lean into expert guidance is there are, people out there who have shared kind of what you might call canonical or templated or example playbooks. Certainly in screens, we have a a library of I think it's up to about 50 pre built scripts that I'm gonna guess almost nobody uses exactly as written, because because we're all creative people. None of us really believe we wanna just take somebody else's stuff on, but they make a wonderful template. If you're writing your, to your point, like your SaaS playbook and you've never written one before, great place to sort of pick up a little bit of expertise from someone who does these all all day in and day out. If I'm gonna have to see if I can get you to build some, Now I come to think about it, Nate. That's then then then then then they have to be my next little my next little foray into calling you up. Yeah. Absolutely. And and screens is a great example because you've you've leveraged kind of that, expertise with with people who have, tons of experience, working on a particular contract type where they able to go in and do that responsible playbooking where they've said, here are the the 10 issues we care about. We're not even gonna put these other issues in. This is really what matters. So I I think that's a great use case or benefit, from screens. Yeah. Yeah. Well and and actually, it sort of brings us to our next topic, which is the talking about trends and evolution and sort of the value of tracing the path of clauses. And the reason for that is that when screens first built the the reason built the bridge, if you will, is that when screens first came up with these these community screens, as they're called, they added a little system where it counts over time how many of the standards pass and fail and how many of the contracts that put pass and fail. It's completely anonymized. Nobody knows who did what. None of the details are getting shared around, I hasten to add, given that I'm talking to lawyers. It's literally just a counter. And the reason that it was built that way was to help people to understand kind of, should I be arguing this this case? Right? So if if I'm I don't know. If I get a, a lease, so they haven't done a lease in a while, so I use the expert built screen, and I say, maybe the screen says, you know, the the landlord must pay for the electricity. Pick a pick a poison. Right? And it comes back. He says, well, it failed, but, actually, 87% of the time this fails. I might say to myself, maybe I'm not gonna get that one, and I could use my I could use my limited time and efforts differently. Do you do you find yourself doing something along those same lines by, you know, sort of as you use a standard playbook, understanding trends and how things are changing? Yeah. So I have a past example and then, a current example with AI. So, we do a lot of, events, and so we're always negotiating with hotels and convention centers and and, service providers for events around certain times. And that was great for a long time, and then COVID happened. And, obviously, everybody was was concerned about force majeure clauses. And pre COVID, we used to have great success negotiating our force majeure clauses with hotels. We could ask for, anything, and and they would be more than willing to to change the language for what would allow us to get out of, a large commitment for a hotel block, a room block. But with COVID, after COVID, that that did a one eighty. And now the the changes that we used to be able to ask for and have a lot of flexibility with, really evaporated. And now we rarely get them. We rarely see them. We get a lot of pushback, because the hotel industry, especially, has really dug in on their learning of what worked and what didn't. And so there's a very clear trend where, something that worked in the past no longer, is able to fly anymore. So we know, we can't really ask for the same stuff we used to be able to ask for. To your point, we know that that we're probably not gonna get that. So maybe we wanna focus our negotiation capital kind of somewhere else, or think about other ways to to kinda mitigate that risk. And so pulling that through to today, everybody's working with with AI, and it's still early. So we're not quite seeing what certain AI vendors, kind of across the board are willing to commit to. I don't know necessarily if if they all know. It's it's still kind of, how are we managing this risk. You know, one example is do vendors want to kind of put any warranties on the quality of their their output? Or do they just wanna take a hands off approach and say, hey. We're you kinda know what you're getting with AI, and it's it's just predicting stuff, and so we're not gonna make any warranties about it. And so kind of tracing that as a trend if you're a customer to say, what can I ask for, from an AI vendor for a commitment? What what can't I ask for? What are they gonna give? You know, some vendors may be willing to commit to it. Others will say, not in a thousand years. And so kind of tracing that trend, as as the AI contracting space kinda moves forward, is is really interesting. That is interesting. Yeah. I mean, I think that, you know, this this concept of the emerging shift. Like, I it's interesting. We started sort of talking about the the emergence of tariffs and then the kind of, you know, comparing it to the arrival of COVID and then looking at the sort of the knock on effect from COVID of the of the force majeure, right, which I'm gonna go out on a limb and guess that they haven't been too fussed about negotiating it with you because no real force majeure event had occurred in an awfully long time. And as we know, history is a series of swing of swinging bells. They're just like, oh, we don't care at all, goes all the way to we care a lot. I guess one of the questions I would have though is, you know, once you work out that they're not going to negotiate on force majeure, does that mean that you and you mentioned this a moment ago, you're negotiating capital. Does that mean you do try to negotiate something else? Or does it just mean there's one less thing to worry about? Yeah. You could go both ways, depending on, kind of what you've been given from from them on the front. You might be able to say, okay. Well, if I if I can't negotiate this this clause that may be more of a legal clause, indemnification or something like that, maybe there's something that we can have for the business team where we wanna concentrate on pricing or something like that, maybe a discount here or there, where we might be able to get, another exception. And so, you're always dealing with the situation where, you know, might as well make the red line to the force majeure and have them say no and then say, okay. Well, if you won't give us on that clause, can we get, you know, this this other clause elsewhere, maybe a discount? Or on the other hand, do you even wanna say, I know they're gonna say no. Maybe I'm not even gonna ask for it and just dive in and focus on what you really, can see a benefit from maybe the discount or the pricing. So I I think that's a balancing act and something you kinda need to, have a read on based on the vendors you typically work with, but it's, something always think about. Yeah. I mean, that that's an interesting that's an interesting way of thinking about it. Also, like, knowing that they're going to say no almost gives you a way to to sort of game out the next two steps of negotiation, I I guess. Right? It's not just that you, you know, you either decide not to fight it or you can go, I know they're going to say no, so I'm ready right here, right now with what I'd like to have instead. It's really an interesting way of thinking about tactics of negotiation. And as you and I had talked about on the on the other call, you always have the challenge where, you make all your red lines and you you even hold back on a few, but then you send it over and they say, oh, we accepted all of them. Then you have the the moment of pause where you say, well, maybe I should have made some other stuff. I should have been super aggressive. So Yeah. That's always something to to think about. Right? Yes. Yes. I don't if they said yes that quickly, what could I have got Darn it. But you got the stuff you cared about. So Exactly. Sometimes sometimes you gotta what do they say in Vegas? You gotta be a good winner, not just a good loser. Right. Okay. So this this is really interesting. And I and I do think that, you know, what we see on the CLM side, is that as, this contract review business has has slowly been absorbed into the CLM system. Right? So now our our customers are running these screens against the contracts and the data is coming back into the Agiloft CLM, we are now able to see people building dashboards and reports and tracking things over time. Because, again, you know, the moment you think you've got it squared away, everything changes again. I think it's really important to keep that eye on the data, so that you are seeing what's happening in the market. So I guess what we're saying here is AI is awesome, and it's doing all these good things, and it's speeding us along, and it's finding things without having to read it. But in some sense, even more important is it's it's giving us a window into what's happening in the market as it does the work. Even though I I think you and I talked about this the other day. It's it's not so much that it saves you all this time. It saves some time, and it makes you better and allows you to use your time better, but it's giving you this data, which I think is really a key element going forward. It's not just tapping a chat GPT. It's understanding how the market's developing in front of you. Yeah. And not just your external market too. I mean, it's it's kind of internal business operations. You know, we've all had a time or maybe some out there still do, where you you don't have a CLM. And, it's tough to imagine, what that is like or or was like for me, because you just don't have access to all sorts of information that, you you can proactively rely on to to help your team. I mean, you think about stuff, internally for price increases and renewals, stuff like that, where you kind of lose the visibility if you don't have a way to track that in in your without a CLM. Oh, gosh. Yeah. I mean, I I I hear from people every day. It's like literally the phone rings, internal legal team says, hello. Somebody says, how much do I have to buy from that particular vendor before we get a credit? And just, you know, all hell breaks loose, and five minutes later, they're calling around every CLM vendor in the world trying to work at helping. And, you know, data availability is is is fundamental. And that's like I say, AI looks flashy and exciting, and it's like talking to a to a human robot. But where the rubber really hits the road is getting consistency, getting that standardization, and getting the data out so that you can track things over time. And then once you get that type of feedback, where you had your internal customer saying, you know, what what's my license requirements or what what seat requirements do I have? That's something you turn into, an item in your playbook that now you can screen your future contracts to say, hey. Is there a minimum requirement for, our seats or something like that? Where now you can proactively address that going forward. Yeah. Yeah. Absolutely. And I mean, and this is we've sort of, without without meaning to do so, we've sort of dipped into our last little slide, right, which is if we can see that using AI saves us time, allows us to use our, you know, our best skills, what do we do next? And I I I wonder if you wanna go back a second because it's something that our, chief strategy officer often says. It's talking about how it's not just saving you time, it's actually creating kind of mind space. Right? So you you don't think of it as it used to take me two hours, then it only takes me ten minutes because that's the better way to think about it is the time that I spend is not spent doing basic stuff that, like, takes up my brainpower but isn't really moving the needle. Let's have the time be used intelligent. But I think that the key here is is twofold. Right? It's get things done quicker or better, and then stick don't stick the data in the bit band, which is what we were just talking about. Wanted to, ask you a little bit about kind of how you've experienced things changing as you've gotten deeper into using AI with Playbooks and CLM versus what it was like before. Well, again, the the the great thing about AI is that we're able to say, you know, even if you don't start using it today and, achieve perfection, that's not what to shoot for. I mean, part of it is that iterative process of trying things out, seeing not just what works. You know, it's great if it catches a red line for you, or make something easier. But it's also, maybe more important, I think, to to see where, it may fail for you, where where, it red lines an issue that you say, I I it shouldn't red line that, but I can just quickly fix it and move on. Or maybe it misses something or you can kinda look at it and say, I'm not sure why it did that. Those failures are really where you can step in and, really probe and say, well, was it the playbook, the way I drafted the the playbook rule? Is it something that that was missed for a different reason? And really learn from those failures so that you can, again, leverage that and correct it at scale. And for your own benefit, understand maybe, you know, AI isn't isn't, always necessarily kind of the the magic magic wand that'll you just touch it on something, it'll, complete it. It it may not be perfect. And so learning those imperfections and how you can work with it is really valuable and can also inform your strategic work because you'll say, okay. This is maybe something that I don't wanna delegate yet to AI. This is something I wanna keep keep for me and think about how we can better address that going forward. So I I think, you know, kind of those small iterative changes, to to just manage the development of AI as it's going forward. Right. Yeah. No. A 100%. So I think what we're seeing here is kind of the next steps that we would recommend. Right? First step is you you gotta have a CLM of some sort. We'll we'll be kind. We won't say it has to be Agiloft even though, obviously, it has to be Agiloft, but, you know, you gotta have something in place. Right? So you've got your processes managed, you've got your repository available for for use, then you gotta drop your AI on top of it and standardize. Right? Using playbooks that are responsibly created so that you're asking the questions that you, you know, that you need to ask and not the ones that you don't. And then you're extracting and following the data over time so that you can learn both about what your company or your organization wants to do, but also what the market is willing to do. Yeah. And that really enhances kind of consistency, past, present, and future. So you get consistency in how you're able to retrieve, your contracts and clauses that you care about, how you review contracts in the present as they they come in. And then looking forward, knowing that all the future contracts, you're gonna have kind of, an understanding of the risk that you want to identify, you know, that they will be reviewed with a certain, perspective, and everything. So you kind of get consistency, again, past, present, and future. Yeah. Yeah. So I I so I have a random question for you, but I think it's important. You know, we've mentioned a couple of times as we've chatted long that, you know, AI isn't always perfect. How often do you see it do something that is, let's call it unexpected, but in a in a creative way? Like, it doesn't you see a hallucination or it just answers something really oddly. How often do you see that? How hard is it it to to catch? Yeah. It's it's hard because, I mean, you see it here and there and and maybe frequently depending on how, often you work with it. But it's hard because, I think there's a tendency to think, okay. I can just re prompt it or tweak it just a little bit, and it's easy then to forget that it missed something or that it didn't give you the feedback that you were looking for or expecting. So I think it's really important to identify kind of any little issue where it isn't, kind of the immediate response you were looking for because those are are where you wanna dig in. And so, I I like to experiment with it and, you know, give it creative clauses and say, I wonder if I can I can trick it? And so, one example that I had is I had a mutual indemnification clause. You know, each party will identify the other for blah blah blah. So it's mutual. And then I tuck in an indemnity at the the last sentence in the paragraph that's unilateral, and it'll say, in addition, vendor will identify customer for x. And so if you read it, it's not elegant. Right? It's it's this whole clause that's mutual all the way up until the last sentence where it's an addition. And you never wanna see that in an agreement, that's that's for drafting. But here and there. Right? Some some agreements do it, and it's it's really you can see that that last sentence is a throw in, where somebody was drafting. So I I gotta include this too. But when you get a clause like that and you ask your AI, is the indemnification mutual? Sometimes you'll get responses that will say, yes, there's mutual indemnification, in this agreement. And, technically, that's right. But, you know, legal, you want it to also catch that last sentence and say, well, wait a minute. It's it's not really all mutual. There's this other clause. And so then you also see some other tools that that will catch it, and they'll say this is mutual. However, there's a a an additional indemnification that's not mutual. And so I like to experiment with stuff like that because I think that's a great example of something that, is that is that the way that I prompt it? Is there a a better way to prompt it? You know, is indemnification mutual? Is it only mutual? And and maybe that would have uncovered, kind of that inconsistency. Is it maybe something in my playbook where I have to rewrite the rule to capture that? But but that's a great learning experience that I had where, I mean, you really get into the weeds and you can see where it didn't quite give give me the output I expected. And so now I can dive in more, and see how I can I can work to to resolve that? Yeah. I think that's brilliant. And I I I gotta be honest. I think lawyers are the perfect people to be putting AI through its through its paces on the prompting side. I often think of AI as being kinda like a genie, but not the cute blue Robin Williams genie, but the scary genie of the of the old tales where you you make a wish and it grants it in the worst possible way possible. Right? You know, you say I want you know, I I wish for a million dollars, so it kills off half your family and gives you the insurance money. That sort of thing. You know? And I think you you you you raise a really interesting point. When you build the prompt, you have to tell it you have to be incredibly specific. And and, you know, playing around with it, if there's anybody who's really good at getting language to do exactly what they wanted to do, I'm gonna say it's attorneys pretty much. Yeah. And and Go ahead. Go ahead. No. No. Please. I was gonna say another another great example, I think are, any any clauses with numbers. If you have something that's, you know, within thirty days, you know, sixty, ninety, whatever it is, anything like that, I'd play around with kind of clever ways to describe a time period. It's just interesting the way that AI will sometimes interpret that and redline that. So that's another, great rule if you if you have your playbook, you know, dive into anything where you have a time period, or something with numbers to see if you can, come up with a clever way to draft the clause that that wouldn't be caught, by that rule because that's easy for for you to spot, and to have something jump jump out. We're gonna have to start calling you the prompt doctor. And then that's gonna be your that's gonna be your new name, prompt doctor, the guy who worked out how to confuse the AI. I love it. Alright. Listen. We have time for just a a a question or two, and then we need to toddle along. So I have this question here. The question here is, what are the limitations for additional risk management approaches that AI is helping overcome? I guess the question is, you know, we talked about how AI is great, but, like, what problem are we solving that existed before? Right. So, I have a a conversation that I I think back to. I'm a fan of of standardized agreements where they're not drafted by one party or the other. They're kind of independently created, by third parties. They kind of say, here's what both sides should really settle on. And so, it's kind of a a reflection of the market. And I was I was talking with somebody that was having a challenge with their their NDA process and all the red lines. And I said, well, you know, have you ever thought about using a standardized approach? And they said, yeah. But but then what's the point of AI? And, it made me laugh because, you know, the the the point is to kinda get the agreement signed on terms everybody can agree to. And so if you could do that without AI, may maybe pursue that. You know, it it doesn't necessarily have to be a solution in and of itself. And so I think it is helping us think of creative ways, to get things done, but, you know, it doesn't have to be, the answer to everything. I don't know if that You know what? I think that's a very wise answer, I have to say. Yeah. I mean, it it does things. It speeds things up. It standardizes and automates, but it doesn't, you know, solve world peace or beat all the children at Christmas. It's one has to have a sense of measure, which I think takes us back to the whole responsible playbook. Right? Don't just because the AI can do everything, don't do everything. Alright. I'm gonna give you one more question, and then we're gonna move along because this just came in, and I think it's really a clever, clever a clever question. Cheers. This is straight to you. Don't do me. Straight to you. How confident are you in AI's ability to spot risk better than humans? Fantastic question. I wanna say both. And I know that's a confusing answer, but one of the fun things, about AI is there are there are plenty of studies about clauses that it would catch, that, they tested the same contract against lawyers, and AI will catch some stuff that lawyers don't, but they will also have the inverse where, lawyers will catch something that AI won't. And there's an example, it's called the VALS study, v a l s, where there is a most favored nation clause. And, I think a few AI tools missed it, but some but not all lawyers caught it. And so, on the one hand, I think that gives us confidence to say there are some risks we're gonna be able to identify because, you know, contracts are still about words and Mhmm. AI may not have the same, way of reading the words that we do. And so we'll be able to identify if someone is trying to cleverly hide a most favored nation clause, in in a clause that you wouldn't expect. And so I think there will be still some instances where, we catch some risks that AI may not identify. And on the other hand, I think AI is also effective at spotting, you know, stuff that that we may not even think about. If you if you haven't done a a playbook, if you don't have a checklist, and you're just trying to do it from your head, maybe you'll miss something. Maybe you won't think about, kind of the downstream effect of a certain clause that AI may be able to call out for you. So, I don't know. Maybe the easy way out to answer your question, to say both, But I I think it's it is a little bit of both because I think there are some areas where, we're still gonna come out ahead, but others where where AI may may have the the edge. I think I think that's I think that's exactly the right answer. And I the way I think about it is just it's AI is gonna spot a whole bunch of stuff real quick. And even if it's stuff that you would have spotted yourself, it's done that work now, and it gives you the opportunity to use that big brain, finding the really tricky bits. Alright. At that point, that's sort of taken us to the end. I wanna thank you so much. It's been so interesting. Thank you. And I wanna thank everybody who stuck with us. For those of you who are interested in just giving this a bit of a go, please head over to screens.ai. That's screens.ai, and you can get a a trial version of screens. Give it a go. It's free gratis and for nothing, and it's a bit of fun. And if you have any interest in in the CLM, please drop us a line. We're over here at agiloft.com. Nate, thank you again. I think we're gonna call it. Everybody have a wonderful day, and we hope to see you again soon. Thank you.