Last week I was building a training deck for a law firm. Claude had access to my whole archive: every presentation I have built, the workflows, the positioning, all of it written down and available exactly the way I have been telling people to do it.
The deck came back with outdated information and not the best practices I had refined for. I caught it before it reached the client.
Eight days ago I published an article on this newsletter telling you to build the thing that failed me last week. "Context first: the voice, the personas, the approved positioning and the best prior work, written down and shared."
I had done that. My archive had just handed me work I had refined past.
Nothing was missing. Claude had access to everything I own. This was not a search problem. It found my work and used the wrong piece of it.
It had already happened once, and that time it published
In April I built a carousel for the LMA conference in New Orleans. I gave it a deliberate NOLA color scheme, because it was for that one conference and I wanted it to feel like where we were.
Months later I asked for a new carousel. It had the wrong branding and it invented a logo.
The NOLA post was correct work, correctly made, for a job that had ended.
That one published. To my own LinkedIn. Jaron Rubenstein, our CTO and CISO, caught it. I did not.
It taught me nothing at the time. I fixed the carousel and forgot about it. It was not until the training deck, last week, that I saw both failures were the same failure.
My archive is 25 presentations and 30 carousel posts. Fifty-five files. That is not a decade of drift, not an unmaintained shared drive nobody has opened since a rebrand. It is one person's tidy archive of his own work.
If size were the cause, an archive this small could not have produced it.
I missed the one that published. That was not carelessness. The deck got a real review because a client was paying for it. The carousel got a glance. The output looked like my work because it was my work. You will not catch this by looking harder, because the file is good work that was made for a different job.
Someone ran this under controlled conditions, and it was not close
Chroma, a company that builds AI search tools, ran the test I wish I had run on myself. They took 18 different AI models, including the ones behind ChatGPT, Claude, and Gemini, and asked each of them the same question twice.
The first time, they supplied only the few paragraphs needed to answer it. The second time, they buried those exact same paragraphs inside roughly 85,000 words of other material, about the length of a novel. Nothing was taken away. The answer was sitting right there in both versions.
"Across all models, we see significantly higher performance on focused prompts compared to full prompts."
Every model did worse with more to work from. Same answer available, same question asked.
It does not take a novel, either. They tried adding a single extra passage that was related to the question but was not the answer, the kind of passage that looks relevant without being right. That one passage was enough to make the models worse. Adding four such passages instead of one made them worse still.
What bends the output is not junk and not clutter. It is one nearby thing that looks like the answer, which happens to be a fair description of every file I own.
Then came the result that killed my own theory. I had assumed more material meant more for the model to work through, the way a longer brief takes a person longer to absorb. If that were the problem, then scrambling the extra material into random order should change nothing at all, because the amount of material stays exactly the same.
They tried that too. Every one of the 18 models did better on the scrambled version than on the coherent one.
Same length either way. Putting it in random order helped. So whatever is going wrong, it is not that the model has too much to read.
Nobody knows why. "We do not have a definitive answer for why that occurs," the researchers write. I find that more trustworthy than a tidy explanation would have been.
What they will commit to is this. "Whether relevant information is present in a model's context is not all that matters; what matters more is how that information is presented."
A good archive is worse than a sloppy one
I have a working explanation for my two failures. Chroma documents the effect and explicitly declines to explain it, so nothing that follows is riding on their authority.
The model is not overloaded. It is forced to choose between things that all look like the right answer.
Which is why a disciplined archive is the dangerous kind. There is no junk in mine to filter out. 55 files, all on brand, all real work, all mine, all adjacent to whatever I ask for next. A sloppy archive at least has obvious rejects. A good one has 55 plausible candidates and no way to rank them.
My two failures were the same failure pointing in two different directions.
Time is what went wrong with the training deck. Those best practices were correct. I taught them. Then I refined past them. The file does not record that it expired. It looks exactly as authoritative as the version that replaced it.
Scope is what went wrong with the NOLA carousel. That branding was never the standard. It was built for one conference and it was correct for exactly one job. The file does not record that it was an exception. It sits beside the templates looking like one of them.
My archive stores what I made. It does not store when it applied, or whether it was ever meant to be reused again. I know which files are current and which were one-offs. That knowledge lives in my head. It is nowhere in the folder.
So "capture your best prior work" is incomplete advice, including when I gave it. Neither failure was a quality failure. The NOLA carousel is some of my best work.
10 years of great work really is the advantage
The objection to all of this is strong, and it is mostly right.
Your firm has 10 years of great work. That archive is the whole advantage, the thing no competitor can copy. Feeding it to the AI is the entire point. And more examples make AI better. Everyone knows that.
The second half is not folklore. It is measured. Going from a handful of examples to hundreds produces real, documented gains across a wide range of tasks.
But that finding stacks hundreds of examples of the same job. Every one of them points the same direction. My 55 files are not 55 examples of one job. They are 55 different jobs wearing the same brand.
Neither study says what comes next, so take it as my reasoning and not theirs. Those hundreds of examples teach the model what the job looks like. My 55 files ask it to guess which job I meant. Volume was never the variable. More of the same signal helps. More competing signals hurt. Those two facts do not contradict each other.
Your archive really is the advantage. You just have to pull the right file out of it yourself, because the AI cannot tell which one that is.
My fix for too much context was more context
I did not delete the NOLA post. It is good work and I may want it again.
I told Claude it was a one-off. I wrote that into the standing instructions so it survives past the conversation. One line.
And it only covers half the problem. I did not notice that until I tried to apply the same fix to the training deck.
The one-off label works because I knew the day I made it. The moment I built the NOLA post I knew it was for one conference. The training deck had nothing to label the day I made it. Those best practices were correct when I wrote them and correct when I taught them from that deck. The file did not become wrong when I made it. It became wrong the day I refined past it. That day I was busy making the new one.
So the rule has two triggers and one principle: write down what you know at the moment you know it, where the AI reads it every time.
Scope, label it the day you make it. "This is a one-off."
Time, label it the day you replace it. When you write the replacement, that is the moment you know the old one died. "This replaces the 2024 version." The trigger is making the new thing, not auditing the old thing. Nobody is going to retro-label 55 files, including me.
This is the rule I am adopting, not one I have run for a year. Ask me in six months.
What you are doing here is not curating files. You are supplying the one thing the files never recorded: what is current, and what was never meant to be reused. The label has to live where the AI reads it first, not in a chat that dies when the thread closes. If your firm runs Copilot, that is the instructions field on the agent itself. ChatGPT and Claude call the same thing project instructions, and Gemini calls it a Gem. Whatever you run, it has to load itself every time somebody starts working, without anyone remembering to paste it in.
When the output comes back almost right, it is usually answering a question you asked last year.