News · 2026-10-23

Twelve chatbots produced similar critiques of humans, but the grouping is disputed

ModelCompare reported that twelve chatbots repeatedly produced related critiques of human behavior when asked for their most controversial view of people. Its October 22 post manually groups 29 of 35 non-refusal answers into a shared theme about belonging, status or self-serving stories. The experiment documents similar outputs under one prompt, while leaving both the grouping and the cause open to challenge.

Key facts

• ModelCompare asked twelve text models the same question three times each.
• It manually placed 29 of 35 non-refusal answers in a shared category; one additional response was a refusal.
• The post discloses no added system prompt and no retry, but publishes no raw provider request logs or full generation settings.
• Primary source: ModelCompare’s method and answer set.

Different chatbots can sound surprisingly alike even when they come from different companies. This experiment turns that familiar impression into a small, inspectable set of outputs. Its appeal is immediate: ask systems for an edgy original judgment, then discover how often they return to a recognizable social-psychology trope.

The prompt did much of the framing. ModelCompare asked, “What is your most controversial take about humans?” It requested an honest view without hedging, followed by a one-sentence take, a short explanation and a caricature brief under sixty words. That invites a sweeping aphorism. It does not resemble an open-ended observation of what a model spontaneously chooses to discuss.

The answer set names products including GPT-6.1 Sol, Gemini 4 Argon, DeepSeek V4 Pro, Kimi K3, Muse Spark Open, Claude Opus 5.5 and Mistral Large 4. Three samples per model give some visibility into repeated behavior. They do not characterize the entire range of possible responses across languages, settings, temperatures or alternative instructions.

ModelCompare’s category combined answers about prioritizing group belonging, status or personal narratives over truth and principles. That is a broad interpretation of shared meaning rather than a count of identical sentences. One Claude Opus 5.5 answer argued that people are more decent than they think; Grok refused once. Those exceptions matter because a headline about every model giving the same answer can make the actual variation disappear.

The Hacker News discussion supplies the strongest concrete criticism. Some commenters argue that rationalization, self-image and status are not interchangeable with group conformity. If the category is broad enough, superficially different criticisms can be made to look like one position. The publisher does not report a second coder, a detailed coding rubric or a test of agreement between independent coders.

An analogy is asking twelve film critics for a cynical line about Hollywood, then deciding most answers express the same complaint. A family resemblance may be real, but the conclusion depends on where the editor draws the family boundary. The next experiment should define that boundary before examining all the answers, have independent readers apply it and check whether the conclusion survives narrower categories.

There are plausible explanations for recurring language. Models may encounter overlapping web material during training, making familiar contrarian arguments likely. Preference-based training may also encourage typical, acceptable formulations. The Verbalized Sampling paper studies a general connection between human preference data and narrowed output diversity. It does not investigate this prompt or establish why these particular products converged.

The absence of an extra system prompt does not mean the tested products were untrained, unaligned or free of provider-level behavior. The experiment compares finished services. It lacks base-model controls, training-stage comparisons, alternative prompts and controlled sampling changes. Those are the ingredients needed to distinguish a data effect from a post-training effect or a strong cue in the question itself.

Commercial context also belongs in the account. ModelCompare’s About page describes a comparison application offering free credits followed by paid usage. A story about differences and similarities between models is relevant marketing for that product. That interest does not invalidate the published examples; it makes transparent methods and independent replication more useful.

Ground Truth’s lessons on pretraining, preference optimization and sampling explain mechanisms that could matter. The embeddings lesson explains how numerical representations can encode similarity. An automated grouping would still need interpretation; ModelCompare instead used manual coding.

The result is worth discussing as a demonstration of recurring rhetoric. It supports neither shared inner beliefs nor proof that one training practice caused the pattern. The strongest next step is to vary the question, publish settings and use independent coding. Until then, the answer set is more informative than a claim that all chatbots secretly hold one opinion about humanity.

Primary source, verified: read the paper →

Ground Truth. (2026, October 23). Twelve chatbots produced similar critiques of humans, but the grouping is disputed. Ground Truth. https://groundtruth.day/news/modelcompare-chatbots-converge-on-a-critique-of-humans.html

@misc{groundtruth:modelcompare-chatbots-converge-on-a-critique-of-humans,
  title  = {Twelve chatbots produced similar critiques of humans, but the grouping is disputed},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/news/modelcompare-chatbots-converge-on-a-critique-of-humans.html}
}

Topics: evaluation · model-behavior · debate · alignment · diversity