Why did the same coding agent see the whole problem one day and miss the obvious the next?
Note: I drafted and edited both this article and this disclosure using an iterative back-and-forth with AI assistants. The narrative, personal experiences, framing, and opinions are my own.
A Sudden Switch
The new session had everything it should have needed. It was the same coding agent, using the same underlying model and reasoning level. I gave it a detailed handoff document from the previous session. It should have known what we decided, what we did, and what came next.
Within minutes, it missed the first question we needed to answer. It was obvious the day before.
I’m a data scientist. I was working with the agent on a machine learning model using a large training dataset, but only a small fraction of its records represented the outcome we cared about. We needed to decide how to handle that imbalance before building anything. The numbers were in the handoff, but the agent missed what they implied and charged ahead.
It felt as if a colleague who deeply understood the whole problem left and was replaced overnight by a bright new graduate who read the notes but did not think through what they implied.
I regularly work with several coding agents, and each has different strengths. I saw the same unsettling discontinuity across all of them. So when I recently read a LinkedIn post about AI being a mind-reader one day and missing the point the next, I knew the feeling.
Maybe AI has good and bad days too.
Where Did the Judgment Go?
The old session was unusually good. The agent and I moved in lockstep, but not because we agreed all the time. We kept improving each other’s ideas. By the end, the agent behaved like a colleague with good judgment. It understood the problem, anticipated risks, proposed solutions, and challenged me when needed.
So I went back to the old conversation and asked what had happened. Why did the new session feel as if I hired a junior engineer straight out of college?
The answer was unexpected and very direct: “What made this session feel senior wasn’t the model. It was the accumulated context and your corrections.”
That was the agent’s diagnosis of its own behavior, not a scientific finding about how language models work. An AI is not necessarily a reliable narrator of its internal machinery.
But the diagnosis matched what I saw. I supported abstract arguments with real data. I offered counterexamples that broke approaches which looked elegant on paper. I asked whether a solution was general enough to survive the next use case. I set boundaries when the agent wandered into unnecessary complexity. I kept returning us to the real goal. Most importantly, I invited criticism instead of agreement. Each exchange made the next one better.
The handoff kept the facts, the artifacts, and the next task. What it lost was harder to write down: the shared understanding we built and the instinct to pause and ask the right question before diving in.
The uncomfortable insight was that I turned out to be part of the
system producing the good answers.
It wasn’t because I stumbled upon a magical prompt. Those exchanges progressively established what kind of conversation this was and how much scrutiny the work required. The new session inherited the work, but not the working relationship.
Tilting the Odds
Instead of trying yet another elaborate opening prompt, I gradually turned the conditions of good collaboration into a repeatable working process. It took many weeks. Whenever the level of interaction dropped, I did a quick retrospective with the agent: What was missing, and how could we preserve it next time? Over time, those retrospectives produced a simple set of rules for maintaining the quality of the conversation:
- Before consequential work, agree on the problem, desired outcome, and constraints. Write it down. Ask the agent to own the outcome, not merely follow the proposed steps.
- Plan first. Keep implementation plans and important decisions in current project documents that future sessions are expected to read, rather than leaving them only in chat history or completed work records.
- Keep a short current scope document that tells every session which outcome matters now, what work comes next, and what is deliberately parked.
- Use session handoff documents to preserve hard-to-reconstruct pointers, open judgment calls, and context at risk of being compressed or lost.
- Separate “deep” work that requires judgment from “ops” work that executes settled decisions. Treat the first substantial request in a new session as deep, and return to deep whenever execution exposes a new decision. Require a recommendation, checked facts versus assumptions, the strongest alternative, and what could make the answer wrong.
- Review consequential decisions with fresh eyes.
- After context compression, reread the operating instructions and current decision documents.
None of this is magic or exotic. We have access to technology that does incredible things, but the standard processes for development and communication have only become more important.
Striving for More Good Days
Do these rules make every session brilliant? Unfortunately, not. AI systems are variable, and some disappointing answers really are just disappointing answers. I learned to expect it, read as much as I can of the agent’s output, and challenge it with questions. My goal is modest: shorten the cold start, catch shallow reasoning earlier, and stop paying the same frustration tax every time a new conversation starts.
AI still has good and bad days. We can only help the bad days be fewer and shorter.
Image created by the author with an AI image generator.