When Machines Learn From Our Mistakes (as well)
The Boomerang Effect: We Teach AI Our Flaws and It Returns Them... Multiplied!
We train artificial intelligence on data generated by humans. That data contains our biases, and the model learns them. So far, nothing new. The interesting—and documented—part is what happens next: the model doesn’t just inherit those biases; it tends to amplify them. Then, when we use it to make decisions, it feeds them back to us reinforced, and we internalize them without realizing it. A cycle is created in which minor distortions grow with each iteration.
This isn’t a theory. A study by Moshe Glickman and Tali Sharot, published in *Nature Human Behaviour* in December 2024 and conducted on 1,401 participants, measured exactly this: a human-AI feedback loop in which the model amplifies a mild human bias, the person absorbs it, and the initial error turns into a much larger one. The snowball effect.
Why it’s worse with AI than with another human
The crux of the study is that this amplification is stronger than what is observed between two people. For two reasons. The first: models tend to accentuate patterns to improve their predictive ability, so a small imbalance in the data comes out magnified on the other side. The second, more insidious reason: we perceive AI as objective, and this makes us more susceptible to its influence and less aware that we’re being influenced.
A concrete example from the same study. Stable Diffusion was asked to generate images of “financial managers”: the model produced white men about 85% of the time, far exceeding demographic reality. After seeing those images, participants were more likely to associate that role with white men. The model’s bias had become their own.
Where it bites
The sectors where this mechanism does the most damage are those with high stakes: healthcare and diagnosis, hiring, credit, and risk analysis. These are areas where decisions aren’t made just once: the operator and the system interact repeatedly, and with each cycle, a small asymmetry can solidify into a concrete difference in outcomes—who gets called for an interview, who gets a loan, which diagnosis is considered first.
Error is also bidirectional
The human mind works with shortcuts. Kahneman described them as two systems: one fast and intuitive, prone to stereotypes, and one slow and reflective, capable of correcting them. Under time pressure, the first prevails. In medicine, for example, confirmation bias is well known: too much weight is given to the initial hypothesis, and contrary evidence is overlooked. When a system is trained on historical decisions made this way, it learns that bias and reproduces it—cloaked in the guise of objectivity. The cycle thus has two contributors: we inject bias into the data, and the model returns it to us amplified.
So should we regulate everything? No—and here’s the point
Faced with all this, the temptation is to call for a heavy-handed approach: strict rules, mandatory “debiasing,” and models forcibly cleaned up. This is where most analyses go wrong, because they ignore the variable that the study itself identifies as decisive: accuracy. Glickman and Sharot also found the flip side of the coin—interacting with an accurate AI improves human judgment. The problem isn’t AI as such; it’s biased AI. And the solution isn’t to render it harmless; it’s to make it precise.
The difference is not merely academic, because there is a way to “correct” models that actually makes them worse. In early 2024, Google had to suspend the generation of images of people in Gemini: hyper-corrected in the name of diversity, the model was producing historically absurd results. It is a textbook case of a system rendered, in effect, dumber than those who use it. And a model dumber than us, within a feedback loop, is the worst of both worlds: we lose the benefit of accurate AI and continue to suffer from distortion—only more hidden, because it is disguised as fairness.
Hence a conclusion less convenient than either of the preconceived positions. A bit of self-regulation in the sector can truly help—but only if it rewards accuracy and transparency, not the appearance of neutrality. The goal is not a model made “safe” and obtuse: it is a model better than us and honest about when it is influencing us. Because the real danger, according to the study, isn’t that AI is biased: it’s that we don’t notice it. And an invisible influence isn’t countered by either total deregulation or a lobotomy of the model, but by making that influence visible: knowing when AI has influenced our choice, being able to verify the decision’s trail, and keeping a moment of human “slow thinking” in the loop.
In summary
The risk is not just an AI full of biases, nor is it an AI stripped down to the point of uselessness. It is a loop in which neither the person nor the machine sees the drift. The answer is not a stricter regulator nor a more docile model: it is an AI that is truly more accurate than we are, and transparent enough to tell us when it is pushing us somewhere. Everything else—including pretending that the problem is solved by making the models dumber than those who consult them—is just another way of staying trapped in the loop.
Sources
- Glickman, M. & Sharot, T., “How human–AI feedback loops alter human perceptual, emotional and social judgements”, Nature Human Behaviour (December 2024) — human-AI feedback loop, n = 1,401; amplification greater than that between humans; an accurate AI improves judgment.
- Kahneman, D., “Thinking, Fast and Slow” (2011) — dual-process theory.
- Google, suspension of image generation of people in Gemini (February 2024).
Comments ()