Transcript
Good afternoon, and welcome back. I am Josh, founder and CEO of Nomio, and I am the only person in the world who has spent the last seven years building and maintaining contract repositories so that you don’t have to.
And I am recording this video in the middle of a UK heatwave, because guess what? That’s what you do when you’re committed to contract management.
Now, speaking of contract management, in front of me, we can see a contract. And I put this here because if you’re watching this, you have probably taken a single contract and you’ve fed it into AI, whether it’s Claude, Copilot, ChatGPT, whatever AI you’ve got available.
And what you’ve probably found is that if you ask a question about that contract, you’re going to get really, really good answers back.
These large language models are incredible.
However, If you stick that same AI on top of your big pile of 500 contracts, you ain’t getting a good answer. It’s not gonna work.
And the reason why is something that is very frustratingly not talked about. So let’s say we’ve got a question.
We feed the question into our AI, but the AI has got to get the data from somewhere as well. And we’re so focused on what AI are we using, ooh, how fancy is the AI, that we completely forget that a way more important input is, what is it that we’re feeding our AI with?
What is the shape of the data? Because when we’re just taking our lazy, disorganised file system, our pile of 500 contracts, we’re not giving AI any of the help that it needs to actually do a good job.
And so what you’re to find is that doing this is going to leave you with a bit of very expensive, slow, unreliable system.
And I wanted to represent those three words with a single image, so I picked the London Underground. If you are not from London, then don’t worry. All this means is expensive, slow, and unreliable.
Now, I’m going to illustrate that to you by showing you a diagram.
So, this big white box represents all of your contracts, those 500 contracts, every word, across every page, across every document.
This is all of the raw material that we’re going to work through. But the thing is, whenever you have a question about your contracts, or even an individual contract, you only really have a very small amount of information across all of those words that’s relevant to answering that question.
This is the needle in the haystack. And the reason why, naively sticking AI on top of your messy file system is so expensive and so slow is because you’re forcing the AI to try to read through this entire big white rectangle when all it needs to read through is that small square, that small red square.
And this is why it’s so unreliable as well. Because when you’ve got so much information, you’ve got so much noise it pollutes the data that you’re feeding into the AI.
And if you look at this orange square, this represents what we call the context window, or the total amount of information that, that the AI can hold in its memory at any one time.
Remember that there’s no additional storage mechanism for all of this stuff, it’s just the file system. So, it makes it very hard, if you’ve got a system that only has this much space, to have to traverse something that is many times larger than what it can hold, and come up with a good answer.
And what makes this worse, is that, and that’s why we’ve got this circular arrow, is that every single time you ask a question, even if you ask the same question again and again, you’re gonna repeat all of this unnecessary work to try and come up with a new answer.
And this is also what makes things unreliable, because when you ask an LLM the same thing again and again, you’re gonna get slightly different answers each time.
Certainly if you phrase the question in a different way, you’re gonna get different answers. This is gonna kill reliability, because you can’t trust that you have same question, same response.
And it means you’re just gonna spend way more tokens than you need to.
Now, if you’re doing this, it’s bad.
But think about a future where agents are the ones who are asking questions of your contract database, or however you decide to store your contracts, agents are gonna hammer the data store far more frequently than you will.
Maybe a hundred times, maybe a thousand times more. And so that’s going to really rack up the costs for you.
It’s bad enough when it’s just a person doing it. Imagine when you’ve got autonomous systems. Asking questions of this thing.
And then, because the results are less reliable, They’re going to be compounding this unreliability across their entire reasoning chain, their entire task chain, for whatever they’ve been sent off to do.
So, ironically, getting AI to work well on your contracts means it’s even more important to have your data in good shape.
And the solution in theory is pretty simple. We just have to change our diagram and insert this layer in between the raw data and the AI.
And what this layer does is it simply reorganises the raw information, the 500 messy contracts, into something that is much much friendlier for both humans and AI to ask a question about.
And the great thing here is that we only have to do the work once to organise the data. And then every single time we have a question, we benefit.
When we repeat the same question again, we benefit. We do the work twice. We work once, we benefit every single time in the future.
And the way that the layer does this is by taking that big haystack and shrinking it down super small before the AI even sees them.
We shrink the haystack right down so that it only just surrounds the needle. But this isn’t as easy as it sounds.
Firstly, we have to be able to shrink the haystack enough so that we get rid of enough noise. And keep things as small as possible that we’re not too big compared to how much information the AI can even hold.
But the thing is, the more we shrink things down, the harder it is to be confident that we’ve shrunk ourselves around the red dot.
So this might happen.
We might just look at completely different things. See the wrong stuff, and give you a bad answer.
So what we’re trying to achieve is to take the haystack, to shrink it just around the needle, and that shrunken haystack that’s left fits comfortably inside our AI.
Now, I’ve already done some videos explaining what this layer needs to look like, but I’m going to do more. So stay tuned for future videos that actually tell you what’s in the layer.
