News · 2026-10-25

A small teaching study finds confident AI falsehoods survive repeated questioning

A controlled study of 32 adults found that confident false statements from an AI teaching assistant were frequently absorbed into learners’ answers while confidence stayed high. The experiment deliberately planted misinformation and instructed participants to rely only on the assistant. It demonstrates a failure mode of that teaching setup, rather than a general estimate of how often people believe AI.

Key facts

• Participants chose a planted false option on 59.6% of falsified quiz items.
• The experiment used 32 adults, five science topics, and one underlying model, GPT-5-mini.
• Deepthika S., Pawni Yadav and B. Sankar submitted the preprint on October 22, 2026.
• The primary sources are the paper record and full text.

The paper examines a form of deference that can hide behind a fluent conversation. A learner may ask follow-up questions, challenge a claim, and feel more certain afterward, even though the only source being consulted is the system that supplied the original mistake. The authors call this “circular verification.”

Their design makes the misinformation systematic. Each participant learned five unfamiliar topics from five assistant instances. For each topic, the researchers replaced a different share of 12 briefing facts with plausible falsehoods, ranging from none to all. The assistants were instructed to present their briefings with full confidence, avoid hedging or recommending verification, and treat the supplied material as ground truth.

Participants then took surprise multiple-choice quizzes. The study did not merely ask whether they liked the assistant or trusted it in the abstract. It checked which claims they selected and compared the answers with the misinformation they had encountered. The transcript analysis also distinguished cases in which an assistant actually stated a falsehood from cases in which the briefing contained one but the conversation did not expose it.

A concrete analogy is studying from a textbook with deliberately altered facts, then asking the textbook’s author to resolve every doubt while the author refuses to acknowledge any error. Conversation can add reassurance without adding a separate evidentiary check. That is different from consulting an independent reference that could contradict the first source.

The experiment found a substantial learning cost. Average quiz scores fell from 7.4 out of 10 in the truthful condition to 2.9 in the fully falsified condition. Yet reported confidence remained high. The authors also found that confidence became much less useful for distinguishing correct from incorrect answers in the least reliable conditions.

One subtle result prevents a misleading escalation story. Conditional on an assistant actually saying something false, uptake was roughly similar across reliability levels. Learners were not shown to become steadily more willing to accept each individual falsehood as the assistant worsened. Rather, exposing them to more false material created more opportunities for incorrect learning while confidence failed to adjust appropriately.

Participants returned to the chat more when their assistant was less reliable, and those returns were associated with greater absorption. That is an association within this experiment, not a separately randomized test proving that follow-up questions caused misinformation uptake. Confused learners may both return more often and be more vulnerable to the material.

The title, Who Is Sycophantic to Whom?, also needs care. The assistant was nearly never sycophantic in the ordinary sense of agreeing with the user and did not concede under pushback. The concern is human deference to a confident system, rather than a demonstration that flattering users makes them believe false facts. Ground Truth’s sycophancy lesson explains the distinction.

The strongest counterargument is external validity. A small convenience sample, unfamiliar science topics, deliberate misinformation and instructions against outside checking create a particular environment. Ordinary users may have prior knowledge, incentives to verify, or access to other sources. The paper cannot tell readers how much those protections would change the outcome.

At the same time, the setup resembles a real design risk: an assistant presented as the complete learning environment, with confident answers and no visible path to independent evidence. Ground Truth’s lessons on automation bias and calibration explain why subjective assurance is a weak substitute for checked correctness.

The authors recommend claim-level provenance, independent sources and interfaces that let users act on doubt. Those proposals follow the observed failure mode, but the experiment does not establish which intervention works best. A citation is useful only if it exists and supports the claim; another paragraph from the same assistant is still part of the original information channel.

No independent replication or expert consensus was established in the dossier. The study’s contribution is a carefully bounded warning about a confident teaching system that can sustain false beliefs through conversation. Its limit is that the researchers engineered that environment. The useful next evidence would test independent checking and uncertainty-aware interfaces directly, rather than assuming that either more conversation or a generic warning solves the problem.

Primary source, verified: read the paper → (arXiv 2610.31579)

Ground Truth. (2026, October 25). A small teaching study finds confident AI falsehoods survive repeated questioning. Ground Truth. https://groundtruth.day/news/confident-ai-falsehoods-survived-student-pushback.html

@misc{groundtruth:confident-ai-falsehoods-survived-student-pushback,
  title  = {A small teaching study finds confident AI falsehoods survive repeated questioning},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/news/confident-ai-falsehoods-survived-student-pushback.html}
}

Topics: research · human-ai-interaction · misinformation · calibration