Transcript
Hi, I’m Josh, I’m the founder and CEO of Nomio, and I believe I’m the only person in the world who has spent the last 7 plus years building and maintaining contract repositories.
I want to share some learnings with you, so that you don’t make the same mistakes that I’ve had to learn along the way.
This one is all about what you need to look out for before naively sticking AI on top of your big pile of contracts.
So, what you’ve probably found is that if you’ve put a contract into Claude or ChatGPT, it’s amazing if you want to ask questions about that contract. It’s really good.
And the same is true if you put in a few, like five. But just because it works for one or five doesn’t mean it’s going to work for a much bigger collection.
So, say you’ve got 500 contracts, it’s naive to try to extrapolate and expect you’re going to get the same good results out.
And the reason for this is that LLMs can only hold a certain amount of information at once before they stop working.
So, a contract, between about 10 and 20 pages, will be around 20,000 tokens. A token is roughly a word, and LLMs use tokens as their currency.
It’ll ingest a contract: 20k tokens that will contribute to the budget that the LLM has. In the case of the latest Claude models, as of May 2026, we’re looking at about a million tokens, which means that in the best possible case, you can store about 50 contracts. So you can chuck about 50 contracts into Claude.
In reality, it’s a lot less than that because the degradation happens around 25% of the way to filling up the context window.
But there is a solution.
We can parallelise the use of Claude across our contracts, and we can do this because most of our contracts are independent of each other, so we can get multiple instances of Claude to do the same thing on a whole load of different contracts, give us a bunch of answers, and then we combine all those answers to give us the answer we’re looking for across all 500.
So, a little bit more involved, but it’s still possible. The real issue is that, however you do this, you’re going to run into really big issues with cost and time.
So, if you take Anthropic’s mid-range model, Sonnet, it costs $3 per million input tokens. Now, remember, a contract is 20,000 tokens.
So, across our 500 contracts, we’re looking at 10 million tokens. This means every time you want to ask a question, it’s costing you $30.
This is going to get expensive really, really fast, and it takes time. You’re going to be waiting several minutes.
So, compared to a spreadsheet, which is basically free to ask a question of and happens instantly, this is much less convenient.
And it only gets worse when you consider that, in the future, we’re going to be having agents ask all these questions much more frequently than people.
So, that $30 per question becomes extraordinarily expensive, and that slowdown you get from having to wait several minutes is extraordinarily long.
And so, we don’t want to directly stick one of these models on top of our contracts.
There are many smarter ways we can do this, where we can still leverage the really incredible power of these models, but we have to be a bit smarter.
I’m going to walk through that in another video.
