Be Wary of AI Judgment

The human mind, and its capacity for
logic and reason, remains unsurpassed,
but people can deceive like LLMs do

The Nature and Locus of Plausible Nonsense
by David Coppedge, CEH Editor

Plausible nonsense and deliberative reasoning: Benchmarking LLMs against human judgment (Veri and Umbellino, PNAS, 9 Sept 2026). These two authors are concerned that large language models (LLMs), the basis of artificial intelligence, are capable of generating “plausible nonsense” such as hallucinations. What is their explanation for why humans are better at reasoning? Evolution.

Their phrase “plausible nonsense” differs from the deception humans engage in, such as the rhetorical tricks made by politicians, marketers, and propagandists, because humans can think. Humans have a conscience. We can be aware of the fact that we are speaking nonsense or lying for an evil purpose. LLMs do not have that capacity. They follow the algorithms with which they were programmed by humans.

AI users need to be aware of the pitfalls inherent in LLMs. Veri and Umbellino understand that plausibility does not equate to truth. They know that LLMs can manipulate language to sound credible but have no conscience to be aware that they are lying or deceiving people.

LLMs are built to generate linguistically plausible outputs. Research by Anthropic reveals that each predicted token arises from the activation of sparsely coded features interacting in circuits, which support coherence and context-sensitive continuation, without thereby guaranteeing factual verification. This architecture is associated with three well-documented phenomena: hallucination, where plausibility-seeking circuits extrapolate beyond evidence to produce fluent but factually incorrect content, such as invented historical events or citations; unfaithful chain-of-thought, where the model constructs seemingly logical explanations for incorrect answers, creating persuasive but ungrounded reasoning paths, and sycophancy, where outputs adapt to perceived user preferences, prioritizing agreement over independent judgment.

Concealed Moral Judgments

Notice in passing that Veri and Umbellino recognize this is a problem. A hidden subtext in this paper is that it would be wrong for people to be deceived by these LLMs. The awareness of a moral failing comes from the image of God embedded in the human heart. Does one AI engine “know” in its digital heart that another AI engine is committing a moral lapse? It might disagree with it, or suggest ways to combat it, but even those responses would be algorithmic, not based in morality.

Philosophers have likened these traits to Frankfurtian bullshit, speech that is indifferent to truth and prioritizes rhetorical persuasiveness over factual veracity. Relatedly, recent work conceptualizes this condition as epistemia, where linguistic plausibility substitutes for epistemic evaluation, producing the appearance of knowledge without grounded judgment. The plausibility-oriented architecture of LLMs exposes a central tension in their design: they are engineered for plausibility without verification. Yet in the context of ill-structured problems, this same tendency may at times support context-sensitive and socially acceptable justifications that resonate with shared norms and values.

Notice the unstated moral judgments in that quote. Is it wrong to be indifferent to truth? Is it wrong to state things without factual veracity? Is epistemia (similar to sophistry) a danger? Should judgments be grounded? If so, in what should they be grounded, if not epistemic verification from logic and evidence? Who does the verifying? How do they judge it? Is it wrong to deceive people with socially acceptable justifications instead of answers grounded in truth and logic?

Are LLMs guilty of sin when they hallucinate? Are they morally culpable when they output unverified statements?

Humans Can Hallucinate, Too

Veri and Umbellino discuss theories of reasoning. What is their preferred one?

You guessed it: Evolution.

The account of reasonableness developed here aligns with argumentative theory of reasoning, which posits that reasoning evolved for social purposes: to justify decisions, persuade others, and coconstruct meaning. Collective reasonableness, then, is the ability to construct arguments that are acceptable across perspectives and contextually coherent. Building on this view, democratic reasoning can be understood as the inclusion of multiple perspectives and the integration of diverse, context-sensitive reasons. Together, these ideas provide a basis for assessing deliberative quality.

The theory they propose is devoid of truth content or rational thought. It is pragmatic. “Democratic reasoning” reduces to whichever rhetorical device prompts group behavior among the most evolved deadheads in a given context. If a gang of skilled sophists has the ability to “construct arguments” that sound “contextually coherent” to the deadheads, it could be called “democratic reasoning.” It would have that magical modern quality: it would be inclusive. The deadheads could come from different perspectives. Some could be green. Some could be pink. Some could be tall and skinny. Some could be short and fat. If they were all convinced by the rhetoric, this would be “democratically reasonable.” Truth need not have anything to do with it.

If Veri and Umbellino value inclusion and acceptability across perspectives, would they include scholars of intelligent design in the reasoning about evolution?

• Would they reason that if engineers created AI, then our own highly sophisticated brains must be products of engineering design?
• Would they accept the logic that democracy must go beyond mob rule?
• Would they allow for the right of an individual to stand against the collective? 10,000 Frenchman can be wrong.
• Would they include Jefferson’s perspective that rights are endowed by a Creator?

The authors never mention thought or thinking. They only briefly mention truth. Instead, the criterion of democratic reasonableness is met whenever the collective decides is plausible after sufficient amount of deliberation (another concept they mention dozens of times). But if the collective can’t agree, who chooses the winning argument, and on what basis? It reduces to mob rule.

The authors speak of values, but never equate them to realities justified logically or empirically. If a deliberative process among deadheads concludes that a certain outgroup should be eliminated, so be it. Democratic reasonableness has done its job.

Oh, but LLMs are bad when they use sophistry.

A concrete illustration emerges in legal practice, where lawyers using LLMs to devise case strategies are “tripped up” by model sycophancy, as systems tend to validate rather than critically assess their reasoning. In such cases, the failure is not factual error, but the absence of adversarial reasoning: the model collapses a space of competing arguments into a single, internally coherent narrative that appears well-justified but remains untested. Experimental evidence confirms and generalizes this pattern, showing that LLMs reinforce users’ views, increase confidence, and reduce exposure to counterarguments, thereby weakening reasoning through limited consideration of competing perspectives.

Why, then, not have multiple LLMs do the work of reasoning and come to a consensus? Would that solve the problem? If evolved humans can vocalize counterarguments from their evolved brains and mouths, surely LLMs can do the same. Truth has left the room.

Turning Their Theory Against Itself

The issues that Veri and Umbellino raise are serious and deserving of reasoned debate. How does society prevent LLM sycophancy from deceiving people? One thing they could do is bring counterarguments by intelligent design advocates into the “adversarial reasoning.” This would be inclusive. It would help refute or validate the “democratic” reasoning, but only if hallucination can be recognized as a moral and logical failure.

A full refutation of this paper is simpler. One doesn’t need to refute it, because it is self-refuting. To see why, turn their reasoning against itself. According to these two authors, reason is deliberative and democratic, not truth-based. Someone could “critically assess their reasoning” as merely the collective consensus of their academic peers. Conclusion: their paper is not true. It’s merely a collective opinion. They should be more inclusive. They need to get out of their echo chamber and talk non-materialists. In addition, they need to talk to farmers, ranchers, and cab drivers. They should deliberate with other creatures they believe evolved, like chimpanzees, lizards, and even house plants. Shouldn’t the ones promoting democratic reasoning practice what they preach?

According to these two authors, reasoning evolved for social purposes. Conclusion: they wrote this paper for social purposes, “to justify decisions, persuade others, and coconstruct meaning.” Were they rationalizing? Where they engaging in sophistry in a sycophantic way? Were they conjuring up meaning out of their own evolved brains? Yes! They just tried to pull the wool over our eyes! They didn’t “mean” anything they said!

No further arguments, your honor.

Let us hope his honor the judge understands the difference between truth and sophistry in an honorable way.

“A theory which explained everything else in the whole universe but which made it impossible to believe that our thinking was valid would be utterly out of court. For that theory would itself have been reached by thinking, and if thinking is not valid that theory would, of course, be demolished. It would have destroyed its own credentials. It would be an argument which proved that no argument was sound—a proof that there are no such things as proofs.”  —C. S. Lewis, Miracles