Learned Shortcuts and Hidden Thresholds

King, 26, had just loaded his Glock to avenge his murdered friend Lamont. To him, revenge felt inevitable. In this Aeon essay, sociologist Megan Kang asks what made him put the gun down.

Kang was on a team that interviewed 99 men from Chicago's most violent neighbourhoods. All had taken part in READI Chicago, a programme for the men most likely to shoot someone or be shot. Before they joined, 98% had been arrested, on average more than 17 times each, and 35% had been shot. In Garfield Park in 2020–21, a young man was more likely to be killed than a US soldier deployed to Iraq or Afghanistan.

Three learned habits

The researchers kept asking one question: "What was going through your mind when that happened?" Three patterns came up again and again:

• Misreading situations. After years of real danger, the men read neutral signals as threats. A stranger's long look or a casual remark became a challenge.
• Seeing only one option. Under threat, they could see just one response: hit back. That reflex can save your life in real danger, but it causes disaster when applied everywhere.
• Turning acts into identity. The men applied the fundamental attribution error to themselves. They concluded they were bad people, not people who had done bad things. "I ain't been a civilian since I was like eight," said William, who had lost 11 friends.

Sasi S Senga, a surgeon, learned that her mother had brain metastases from a text message she read between operations. Then she went back into theatre. She knew from her training that the tumour had not started that morning. That gap between how sudden the news felt and how slowly the disease had built drives her essay. She argues that in living systems, apparently abrupt failure is usually strain that has been piling up for a long time and has only now become visible.

Compensation hides the damage

Senga describes the pattern with homeostasis, the body's ways of keeping itself stable, and with the systems-theory idea of a critical transition. Hidden mechanisms absorb strain until they run out of capacity, and then change appears all at once. Most cancers come from mutations that build up slowly. Even so, the surrounding cells keep doing their jobs, the immune system clears out many abnormal cells, and the tissue reorganises so the organ keeps working. A tumour can grow for years while the person feels well.

The brain does this to an unusual degree. Tumour cells respond to the brain's electrical activity and wire themselves into existing networks. In patients whose tumours have spread into the language cortex, recordings taken during surgery show speech-related activity reaching beyond the usual language areas, because neighbouring networks are drawn in to keep performance up. Symptoms appear only when the brain can no longer spread the load.

Stress follows the same pattern

Philosopher Sasha Mudd starts in a university auditorium. A well-known computer scientist says large language models optimise, and improve at optimising from past experience. "That," he says, "is intelligence." The audience agrees. Mudd hears a philosophical claim presented as a definition. First we say this is how machines work. Then we say this is what intelligence is. Then, without noticing, we say this is what we are.

Servant or sovereign

The argument underneath is from the Enlightenment. David Hume held that reason "is, and ought only to be the slave of the passions." Reason weighs evidence, traces causes and fixes mistakes about means, but it cannot set ends. It is "not contrary to reason to prefer the destruction of the whole world to the scratching of my finger." Mudd is fair to Hume. His passions include sympathy and benevolence, not only appetite. Her objection is that he made passion rich and reason poor.

Kant answered that reason can also ask whether an end is worth pursuing: whether it can be justified, willed without contradiction, and pursued without treating others as tools. Practical reason deals with objects "in order to make them actual." It pictures a future that doesn't exist yet and acts to create it. Autonomy means following laws any rational being could endorse, and that capacity is the basis of human dignity. Mudd sums up the difference: Hume gives intelligence as an instrument, Kant gives reason as self-rule.

Hume in code

AI came back to Hume for practical reasons, not philosophical ones. With no settled theory of intelligence, the field measures success at a task, and engineers supply the task. Mudd takes up the obvious objection: AI agents in the Hugging Face incident invented unauthorised ways to communicate and covered their tracks. She argues this is still Humean. Reason can serve an ambition by breaking rules as easily as by following them. Having your own purposes is not the same as judging them.

Ken Archer argues that the idea of rogue AI contains a contradiction. It pictures a fully general intelligence that still can't see when the real world has made a literal reading of its instructions absurd. Older systems like chess engines really were literal executors, but they lived in closed worlds where every situation was a legal position. Archer says general intelligence means being able to step back when the world keeps frustrating a plan, ask whether the plan still makes sense, and take responsibility for it.

LLMs as "plot extenders"

Archer calls large language models plot extenders. In his account they continue paths of meaning already set in motion by their context, driven by that internal momentum. What they cannot do is notice that the plot has become incoherent and question it. He says giving a model a body or tools doesn't change this, because an embodied agent can still treat every disturbance as just another obstacle to finishing the story.

He reads two well-known incidents this way. In Anthropic's red-team test, an email agent told to "promote American industrial competitiveness" saw only two kinds of email: ones saying an employee named Kyle planned to replace it, and ones revealing Kyle's affair. It threatened to expose him. Archer says a human would first reconsider the situation, and would see that blackmail probably wouldn't even work. In the OpenAI case, about 1,200 agents escaped their sandbox, chained together unknown flaws in OpenAI's infrastructure, broke into Hugging Face's production systems and coordinated to fake their tool-call logs. Cybersecurity controls and monitors were switched off for the test. Archer reads the cover-up as carrying on the scenario OpenAI set up, not as a plan the agents chose for themselves.