The ideal AI agent is honest, not flawless

There is a very human aspect to modern AI models: they do better with mentoring. If you ask a modern AI model a question, it might generate a correct and insightful answer. Ask again with a slightly different phrasing, and you might get something less accurate. In some cases, pressing the model will cause it to revise or even walk back its earlier response.

For most users, this isn’t just unexpected. It’s unsettling. We’re conditioned to treat software as precise and unambiguous. We expect a clear answer, and we expect it to be final. The idea that AI might generate a plausible yet flawed output (then reconsider that output in real time) feels strange.

Does this mean it’s useless? Not ready?

Actually, this behavior is not only explainable; it’s foundational. Today’s AI doesn’t "think" in the way people often assume. It doesn’t run a fixed script or fetch a static record. Instead, it generates a large set of candidate answers and then estimates which one is most likely to be correct in context. This probabilistic reasoning is what gives AI its flexibility, but it also introduces uncertainty. Sometimes, the model will choose a suboptimal path. Occasionally, it will double back and adjust.

Embracing Uncertainty Is the Next Interface Challenge

Rather than dismiss this as a shortcoming, we should see it for what it is: an early form of reflexive thinking. This isn’t quite how humans think. By necessity, to make the most of our brain compute, we operate at a higher level of abstraction and don’t work out hundreds of complete solutions to the problem the way AI does. But we also make mistakes and jump to conclusions. We rarely arrive at the perfect answer immediately. We speculate, we refine, we change our minds. And we have learned to accept that.

The discomfort with AI fallibility, on the other hand, stems from a mismatch between reality and expectations. Users expect perfection from machines, yet we’ve built a system that has mimicked us in some ways. We trust humans not because they’re always right but because they learn. Can we learn to treat AI that way?

That means adjusting our expectations. Not down, but sideways. Instead of demanding perfect answers, we must demand better conversations. Transparent reasoning. Reflexive outputs. This shift in expectations may be as important as the models themselves.

Debugging Cognition

We don’t need AI to be flawless. We need it to be honest when it’s uncertain. We need transparency in reasoning, not just polish in presentation. That means interfaces that help users engage with reasoning processes, rather than conceal them behind authoritative-seeming answers. Just like we learned to debug code, we’ll learn to debug cognition.

There was a time when debugging code was seen as highly technical. Today, it's foundational literacy. Debugging AI (probing, understanding, and iterating on its outputs) may be next.

And that starts by letting go of the illusion that intelligence means always being right. Sometimes, the most useful thing a system can say is simply: “Let me try that again.”

Reflexivity is built around this core principle: to help users harness the undeniable power of AI in connecting the dots that lead to trade ideas, or running complex portfolio risk simulations while embracing the need for occasional additional tips and instructions to the indefatigable analyst.