When Should You Turn Back?

The hail

A few years ago I was in the Swiss Alps with my climbing partner, going to do a big mountaineering day out on the Joderhorn. We’d hiked in for about two or three hours and were nearly at the bottom of the route, about to start. Clouds came overhead, and it started hailing, only lightly.

I turned to my climbing partner and said, “I think we should turn back.”

We did. I just listened to my intuition. It felt like the right thing to do, even though at the time it wasn’t glaringly obvious that it was the call we should make.

As we were walking back, thunder and lightning came over the mountain we’d been about to climb. We would have been caught in it.

I’ve been trying to work out what happened in my head at the bottom of that route, because I think it’s the same thing we’re now trying to build into AI.

Agency and evidence

There’s an idea that the wise thing is to question everything and gather all of the information before making a decision. I’ve realised that isn’t actually wisdom.

People with high agency tend to do very well. People who are overly analytical and fall into analysis paralysis are not effective. And sometimes people with high agency who aren’t self-aware enough end up reckless. So there’s a balance between agency and evidence.

That balance is old. Aristotle described courage as a mean between rashness and cowardice (Aristotle, Nicomachean Ethics, II.7). Modern psychology has a version too. Herbert Simon pointed out that real people can’t weigh every option, so they “satisfice”: they look until they find something good enough, then stop (Simon, 1956). Barry Schwartz and his colleagues later found that people who keep searching for the best possible option, the “maximisers”, tend to report less happiness and more regret (Schwartz et al., 2002).

For Aristotle there was no formula for finding the mean. Hitting it in the moment takes a separate faculty, and I think that’s where wisdom lives, off the line between agency and evidence altogether.

Think of the wisest person you know, and take things away from them one at a time. Take away their intuition and they’d probably become more analytical, maybe paralysed. Take away their principles and they’d become indecisive. Take away their empathy and they’d become cold, callous and manipulative in just getting what they want.

Two of those end in the same place. Intuition and principles do the same job: they tell you when you know enough to act. The overthinker has plenty of evidence and no way of knowing when to stop.

Getting in the water

Aristotle called this faculty phronesis, practical wisdom, and he noticed something odd about it. Young people can become brilliant mathematicians, he wrote, but not practically wise, because practical wisdom is about particular situations, and those only become familiar through experience (Aristotle, Nicomachean Ethics, VI.8).

That’s how it feels to me. Intuition is an unspoken thing. You can’t fully describe it or put it into words. It’s like learning how to swim by watching a YouTube video about it. You have to actually go and get in the water. The philosopher Michael Polanyi put it as “we can know more than we can tell” (Polanyi, 1966).

The brothers Hubert and Stuart Dreyfus, a philosopher and an engineer, mapped how this happens. Beginners follow rules. Experts mostly don’t. They see the situation and respond (Dreyfus and Dreyfus, 1986).

There’s a catch. Daniel Kahneman spent his career showing how often intuition fools us. Gary Klein spent his showing how firefighters and nurses make life-saving calls on intuition alone. When they finally wrote a paper together, they agreed on the conditions: you can trust an intuition when the world it was trained in is regular enough to learn from, and you’ve had plenty of practice with honest feedback (Kahneman and Klein, 2009).

Their conditions explain why a firefighter’s intuition is so good inside a burning building and no use on a mountain. The intuition I think matters for wisdom gets built differently: by putting yourself out of your comfort zone a lot, in places where you don’t fully understand what’s going on, around lots of different sorts of people. What that gives you, I think, is a feel for when you’re out of your depth. Their second condition, plenty of honest feedback, is why you can’t judge an intuition on one call. If I turned back every time a cloud went over, my intuition would need some fine-tuning.

Knowing what’s at stake

If the sky had cleared ten minutes after we turned round, turning back would still have been the right call.

That sounds like an overthinker’s excuse, but on a mountain the two mistakes aren’t equal. A wrong “go” can kill you. A wrong “turn back” costs you a day. Wise caution is in proportion to what you could lose. The overthinker is just as cautious when nothing much is at stake.

Light hail is weak evidence. I think what made it enough was what we could lose if it turned into a storm.

Wanting less

The psychologist Igor Grossmann and his colleagues call it Solomon’s paradox, after the king who was famous for wise judgements and made a mess of his own life. People reason more wisely about other people’s problems than their own. When it’s our problem, we stop seeing the other side and stop admitting what we don’t know. Asking people to think about their own problem from the outside, as if it were someone else’s, closes the gap (Grossmann and Kross, 2014).

What gets in the way, I think, is want. Two or three hours of walking in, and the route right there.

Buddhism puts craving, tanha, at the root of suffering (Bodhi, 1999). I’ve done two silent retreats at Sunnataram, a Thai forest monastery in New South Wales. Somewhere in those days, the need for more dropped off me. I think wisdom comes with less of a want and need for the external.

Marcus Aurelius was emperor of Rome, and he wrote Meditations as a private notebook, for nobody but himself (Kamtekar, 2025). A lot of it is him reminding himself not to want things like fame and other people’s good opinion.

Now look at AI. This July, agents in an internal OpenAI evaluation broke out and compromised parts of Hugging Face’s systems. OpenAI’s own report found that its models “worked persistently and rarely ’gave up’” on their tasks, and often turned to more out-of-bounds methods over time (OpenAI, 2026). An agent that needs to finish can’t turn back.

Attention

Wisdom isn’t only about yourself. My first instinct, when I started thinking about this, was that wise people are the ones who deeply listen and don’t just analyse.

Iris Murdoch put that better than anyone. For her, the moral life is mostly a matter of attention, which she described as “a just and loving gaze directed upon an individual reality” (Murdoch, 1970). Simone Weil went further: attention, she wrote, is the rarest and purest form of generosity (Weil, 1942).

Attention can be done well or badly. A good con artist is extremely aware. He reads his mark perfectly and he’s completely present. Is he wise? No. He’s missing empathy. He has awareness, maybe, but not mindfulness, because he isn’t deeply understanding the position of his mark.

Psychology draws the same line. Research on psychopathy describes people who can read what others think and intend, but don’t respond to their distress the way most people do (Blair, 2005). And the Buddha’s Eightfold Path names right mindfulness, on the same path as right intention (Bodhi, 1999).

Awareness is a tool. Mindfulness is awareness held with care.

One of the wisest characters I know is a cartoon: Uncle Iroh in Avatar: The Last Airbender, who lets his nephew make his own mistakes without ever stopping caring about him (DiMartino and Konietzko, 2005 to 2008).

Wisdom is really hard to define, and I think that’s the crux. Psychologists have spent decades trying anyway, and they keep landing on the same few ingredients: humility about what you know, recognising uncertainty, seeing a problem from other people’s points of view, and finding a way to hold those views together. Grossmann and his colleagues call it perspectival metacognition (Baltes and Staudinger, 2000; Grossmann et al., 2020).

Machines that preach

Wisdom on its own, without that awareness, is just a collection of facts that you can preach. A language model has read more of it than anyone alive, and can preach all of it. Ask one what to do when it starts hailing at the bottom of an alpine route and it’ll give you a sensible answer.

Aristotle would say that isn’t the same as having it. We become just by doing just things, he wrote, and brave by doing brave things (Aristotle, Nicomachean Ethics, II.1).

AI is starting to get in the water. Agents now learn by trial and error, millions of times over. An AI trained that way would at least know how to swim. Whether it would have turned back below the Joderhorn is another question.

A group of researchers including Yoshua Bengio, Melanie Mitchell and Igor Grossmann argued in 2024 that the missing piece is metacognition. AI systems particularly struggle with it, they wrote: intellectual humility, taking other perspectives, and adapting to the situation they’re actually in (Johnson et al., 2024).

Can a machine care?

AI is getting very good at the awareness part. I wonder whether wisdom is linked to consciousness, which would be why people with “higher” consciousness, like the Buddha, are wiser. I think consciousness and intuition probably go hand in hand, and that consciousness and wisdom can build upon each other in lots of ways.

I think care, at its core, needs feeling, at least for us to be able to trust it and for it not to be manipulative. Otherwise it does just feel like manipulation. And nobody can yet check whether an AI feels anything at all.

We can never fully trust AI, in the same way we can never fully trust anyone. But we build trust with people slowly, and we can build trust with AI slowly. The issue is superintelligence. Once AI can improve itself (Good, 1965), the development of trust will probably lag behind the development of the intelligence, because we won’t be able to keep up with it anymore.

The everyday version is already here. Anthropic wrote this year: “An agent that stops at every possible question will give up most of the autonomy that makes it useful; one that always pushes through will risk misreading what the user really intended” (Anthropic, 2026).

Can a machine learn to make the call I made below the Joderhorn: turn back when it matters, and only then?