Transcript
Hi, I’m Josh. I’m the founder and CEO of Nomio, and I love contract repositories so much that I am the only person in the world who has spent the last seven years doing pretty much nothing but building and maintaining them, and perfecting the science of how to do so.
So, I’m now going to be sharing a bunch of learnings with you through a series of videos. If you haven’t already seen them, check out the previous two videos, and we’re going to jump in and recap what we’ve learned so far.
So, if you naively plug Claude or ChatGPT onto your contracts and try and get that to answer questions for you, you’re going to be paying a huge amount of money and time. And that’s because you’re going to be putting all of the contents of all of your contracts into the LLM every single time.
That’s really expensive. It gets even worse when you stick agents on top of your contracts, rather than you asking the questions.
So, the way that we fix this is that we figure out, hey, for most questions, I only need to look at certain parts of the contract.
So, let’s build a system that does that for us. It breaks up contracts into various pieces and then, when we have a question, we just look at those particular pieces to answer the question.
But, as we saw in the last video, there are still loads of problems with this. So, what if, instead of simply trying to cut out the amount of stuff that we look through and all of the uncertainty that comes with that, we do something that we’ve been doing for literally decades and we say, hey, why don’t we just calculate in advance the answers to a whole lot of questions, we do the work once and then every time we have the question, we can look at the same data that we’ve already got. And this is really good for two reasons. The obvious one is that we don’t repeat work, which means we don’t pay multiple times for the same piece of work.
But second, it means that your answers are going to be very consistent and they’re going to be the same every time, which is a big problem when you’re doing dealing with AI.
You ask the same question, you get slightly different answers every single time. So this is actually a really good starting point and loads of people like to dunk on spreadsheets, but I think when you look at the available options, they’re quite high up there in terms of what you could, solve this problem with. Now, the big problem with spreadsheets is that you’ve got the no link to the documents that they came from or that the information they contain came from.
So, say you’re looking at a cell in the spreadsheet and you think, ah, that looks a bit funny. How do I know that that’s right?
There’s no easy way to go from that cell to the part of the contract that defines the information that the cell is summarising.
And that’s really bad for trust. If you cannot trust the spreadsheet, even if it’s theoretically 100% correct, it’s only actually going to get used in your business by you and your colleagues if you can trust that it is correct.
The real killer here is that the cells that contain weird-looking values, the ones that don’t look like they belong, the ones that look like they’re wrong, are probably the most important ones to actually keep track of, but they’re also the ones that you’re going to least trust because they are different to all of the others, and if you can’t quickly validate that that one is actually correct, you’re going to just stop using the spreadsheet tool together.
You’re going to go straight back to the document, which is something that you were trying to avoid by using the spreadsheet. The other problem is that, even if you did trust it, you can’t see the surrounding context. So, say we have termination for convenience, yes.
Well, this tells us, yes, we have termination for convenience, but it doesn’t tell us anything like what’s the notice period for terminating, or are there early termination fees, or any other conditions around that termination for convenience.
So, a spreadsheet doesn’t tell you the full story, it’s just a really useful summary, but it’s no replacement for the actual contract text itself. And you can really see this in the next problem. Which is where, you only have the information that you’ve done the work to prepare in advance.
If you think of a new piece of information that you care about, someone needs to go and do the work to populate that column, and until that happens, you’re just going to be stuck doing the basic stuff that we try to avoid with this process, which is either just stick Claude directly on top of your contracts, or, even worse, go and manually, you know, Control-F your way around your contracts to try and find that information.
So, spreadsheets don’t cover everything in advance, but what they do cover, can be really convenient. There are other problems here as well, the whole, the whole benefit that we get from a spreadsheet is contingent on someone or something in your business actually populating and maintaining that spreadsheet on an ongoing basis. So, this information won’t just magically appear by itself, and it won’t magically stay updated. Contracts are evolving.
They get amended. New contracts come in. Contracts get terminated. Every time you’ve got a change like that in your business, the spreadsheet needs to reflect that.
And a big problem is that, as these contracts evolve, as it happens over time, your business also evolves, and the people, in your business evolve, and that means that you’ve got different people all contributing to this spreadsheet.
People are going to interpret the same kind of information in different ways, and it means you’re going to get inconsistency in how this information is expressed.
And then, of course, you’ve got this horrible key person risk, where the person, who’s maintaining your spreadsheet could leave at any moment, either on holiday or permanently, or they’re off sick for whatever reason, and now you’ve got a real mess, for the next person who’s going to come in, who, by the way, is probably not going to trust the spreadsheet, because there’s no way of validating that it’s correct. And we see this pattern time and time again.
In fact, our most common source of customers is companies that have attempted to do something like this, and then they stop trusting the thing, and then they have to rebuild it, or they just don’t do anything at all, because, the project never seems to work out.
And then, finally, it’s very, very hard to, lock down access control when you’ve got a spreadsheet. Row-based permissions are pretty tough to do, and then, because someone’s got to go in and maintain it and add new stuff to it all the time, they’ve got to remember all the permissions across the org. That’s really onerous. And then, of course, we’ve got to think about the fact that, rows and columns are columns aren’t actually how contracts work in real life.
Remember that a contract isn’t just a single document. A contract is one or more documents that, together, combine to make up the contract, and different documents can have different relationships to each other, and this really compounds the complexity of managing a spreadsheet, which is just fundamentally incapable of modelling the kind of information that you need to model in order to have things be really accurate and maintainable in the long term.
But that is a topic for the next video, so thank you very much for paying attention, and I will see you next time.
