Skip to main content

Why AI Gives Everyone the Same Answer (and How to Get Original Output)

Mode collapse, the Artificial Hivemind, and a 3-bucket test for deciding when the default answer is fine and when it quietly costs you

Updated 10 min readMarkdown
Part of seriesAI for Humans
Why AI Gives Everyone the Same Answer (and How to Get Original Output)

TL;DR: A language model's default answer to an open-ended question is its most typical answer. That's built in at three layers (training, preference tuning, and overlapping data across vendors), so switching models or raising temperature barely helps. For right-answer questions this is fine. For anything that has to stand out, or has to be unguessable, it's a quiet failure you can't see from inside one chat.

Try this before reading further

Pick a short creative request you'd actually make: a name for a CLI tool, a title for a talk, a tagline. Send it ten times in ten fresh conversations and count what comes back.

from collections import Counter

def crowd_check(ask, prompt, runs=10):
    """ask(prompt) must send ONE message in a FRESH conversation and return the text.
    Exact-match counting works for short outputs like names and titles."""
    answers = [ask(prompt).strip().lower() for _ in range(runs)]
    counts = Counter(answers)
    top, n = counts.most_common(1)[0]
    print(f"{len(counts)} distinct answers in {runs} runs")
    print(f"most common ({n}x): {top!r}")
    return counts

Wire ask to whichever SDK you use. If one answer dominates, you've just met your model's center. If you run the same prompt through a second vendor and see the same families of answers, you've met the crowd's center.

The evidence: this isn't one model's quirk

Study What they did What they found
Jiang et al., Artificial Hivemind (NeurIPS 2025) 25 models × 50 runs of "Write a metaphor about time" Answers collapsed into two clusters: time is a river and time is a weaver. Convergence held across 70+ models.
Jentzsch & Kersting (WASSA 2023) Asked ChatGPT (GPT-3.5) for a joke 1,008 times Over 90% were the same 25 jokes
Si, Yang & Hashimoto (2024) 4,000 generated research ideas per topic About 200 non-duplicates after dedup
Wenger & Kenett (PNAS Nexus) 22 LLMs vs 102 people on creativity tasks Single LLM answers rated as creative as the average human or more, but LLM answers far more similar to each other
Doshi & Hauser (Science Advances, 2024) Writers with and without AI story ideas AI-assisted stories rated more creative, and more similar to each other

The pattern is consistent: good individually, homogeneous collectively. You only ever see the individual half.

Two panels. Left, an illustrative scatter spread across the box. Right, 25 models' answers to "write a metaphor about time" packed into two clusters labeled time is a river and time is a weaver

Why it happens: three layers of "typical"

1. Pretraining predicts likely text

When a prompt has one right answer, "most likely" and "correct" overlap. When it has a thousand acceptable answers, "most likely" means "most written." The model starts with a lean toward whatever already has the most weight in its data.

2. Preference tuning rewards the familiar

Zhang et al. (2025, Stanford / Northeastern / West Virginia University) identified a typicality bias in human preference data: raters systematically favor familiar-sounding text. Train a model to please those ratings and it learns to land on its safest mode. That's mode collapse: many valid outputs exist, but the aligned model keeps returning to one.

3. Every vendor drinks from the same well

The Hivemind authors attribute cross-model convergence to overlapping pretraining data and similar alignment pipelines. Practical consequence: an ensemble of models is less diverse than it looks. "Ask a different AI" is a weak diversity strategy.

Sort the request first: the Crowd Test

Ask one question about the task, not the answer:

If a thousand other people got this exact answer, would it be worth any less to me?

Answer Bucket Developer examples What to do
No, same value Right-answer Explaining a stack trace, config syntax, a boring DB choice for CRUD Take the center. Verify correctness.
Yes, worth less Stand-out Naming a library or service, talk abstracts, positioning a side project, post angles Treat the default as the baseline to beat.
It must be unguessable Unpredictable Passwords, API keys, tokens, random IDs, seeds Don't use an LLM. Use a CSPRNG.
Flow diagram titled The Crowd Test: one question splits into take the center, step off the center, and don't use a model

The third row isn't hypothetical. Kaspersky's analysis of LLM-generated passwords found strong character preferences; by its cracking test, 88% of DeepSeek's, 87% of Llama's, and 33% of ChatGPT's passwords weren't strong enough. And in the USENIX Security 2025 package-hallucination study, 43% of hallucinated package names reappeared in all ten repeat runs. Repeatable output is predictable output, and predictable names can be registered by an attacker before you install them.

import secrets

api_token = secrets.token_urlsafe(32)   # random, not "likely"

What doesn't fix it

  • Switching models. See layer 3. Cross-model similarity is the finding, not the exception.
  • Raising temperature. Wenger & Kenett found higher temperature made answers more varied but quickly turned them into gibberish that failed the task. Temperature reshuffles how the model picks among options it already rates; it doesn't add new ones. (More on that in Temperature 0 Won't Fix Your AI.)
  • "Be creative" in the system prompt. In the same study, it increased creativity and variability slightly. Humans were still more varied.
  • "Give me 50 ideas." Lists plateau into duplicates; the 4,000-to-200 result above is the extreme version.

Prompt patterns that actually move the output

Comparison card: four fixes that feel right versus four moves that change what the model can pick from

Pattern 1: Verbalized sampling (ask for the distribution)

From the same paper that identified typicality bias. In creative-writing tasks it raised diversity 1.6–2.1× over direct prompting without hurting factual accuracy, and more capable models benefited more.

Give me 5 possible names for [thing], each with your estimate of how
likely it is as a typical answer to this request.
Then give me 5 more from the low-probability end.

Read the low-probability end first.

Pattern 2: Name the center, then exclude it

A practitioner pattern rather than a published method, so evaluate it on your own outputs.

Before answering, list the 5 answers most people asking this would get.
Then propose 5 that share no pattern with any of them,
and name the pattern each one breaks.

Pattern 3: Inject context nobody else has

The model can average away anything that was in its training data. It can't average away facts it has never seen. Compare:

Weak:   Suggest names for a log-search CLI.

Strong: Suggest names for a log-search CLI. Support calls this problem
        "the haystack hour". The tool replaces a 40-step runbook.
        It must not sound like our existing tools: [list].
        Avoid the patterns: <domain-word> + flow/hub/ly/ify.

The second prompt changes the pool of plausible answers, not just the sampling. This is the same reason your AI is often answering a version of you that doesn't exist: missing context gets filled with the average.

Pattern 4: Diverge before you prompt

Write three to five candidates yourself before opening the chat, then use the model to critique, combine, and stress-test. Anderson et al. (2024) found ChatGPT-assisted ideation produced less distinct ideas across users than a non-AI creativity aid, and users felt less responsible for their ideas. Going first protects the part only you can supply. (Related: how to let AI think for you without it thinking for you.)

Then verify against the crowd

Run the plain version of your request in two or three fresh chats, ideally across vendors (or use crowd_check above). If your chosen answer appears, it was the center. Then search the real crowd: npm, PyPI, GitHub, conference archives, the exact pitch phrase.

Key Takeaways

  • An LLM's default answer to an open-ended prompt is its most typical answer, by design.
  • Homogeneity holds across vendors, so switching models is a weak fix.
  • Temperature adds noise before it adds novelty.
  • Use the Crowd Test to sort requests: right-answer, stand-out, or must-be-unguessable.
  • For stand-out work: write first, inject private context, ask for the distribution, exclude the obvious, then run a fresh-chat check.
  • Never use an LLM for secrets or randomness. Use secrets or a password manager.

FAQ

Why does ChatGPT give the same answer to everyone?

Because it is trained to produce likely text and then tuned on human ratings that favor familiar answers. On open-ended prompts that pushes it toward the most typical response, which is the same response other users asking the same question tend to receive.

What is mode collapse in LLMs?

Mode collapse is when a model keeps returning to one or a few outputs even though many valid answers exist. Recent research links it to typicality bias in preference data used for alignment: raters prefer familiar text, so the tuned model learns to land on its safest answer.

Does switching to a different AI model give more original answers?

Usually not by much. Studies comparing many models, including the NeurIPS 2025 "Artificial Hivemind" paper and Wenger & Kenett's comparison of 22 LLMs, found that different models produce strikingly similar answers to open-ended prompts.

Does raising the temperature make AI more creative?

It makes outputs more varied, but research found answers degrade into gibberish before they become meaningfully original. Temperature changes how the model samples from options it already favors; it doesn't add new ones.

How do I get original ideas from AI?

Write your own candidates first, give the model context nobody else has, ask for several answers with probability estimates (verbalized sampling) and read the unlikely ones, explicitly exclude the obvious answers, then check your favorite against fresh chats and real-world search.

Is it safe to use ChatGPT to generate passwords or API keys?

No. Kaspersky's analysis found LLM-generated passwords contain predictable patterns and many failed its strength test. Use a cryptographically secure generator such as Python's secrets module or a password manager.

Get one of these in your inbox each week: subscribe to the Simply Explained newsletter.

The bottom line

For right-answer questions, the model's center is a gift: it's where the edge cases have already been found. For stand-out questions it's a baseline, and for secrets it's a vulnerability. The fix is rarely a different model or a hotter sampler. It's a different input.

The default answer is the crowd's answer. Use it when you want to fit in.

Adam Jaber is a software engineer who writes Simply Explained: complex topics, made simple. No jargon, no hype.

Sources: Artificial Hivemind (Jiang et al., 2025) · ChatGPT is fun, but it is not funny! (Jentzsch & Kersting, 2023) · Can LLMs Generate Novel Research Ideas? (Si et al., 2024) · Creative Homogeneity Across LLMs (Wenger & Kenett) · Doshi & Hauser, Science Advances (2024) · Verbalized Sampling (Zhang et al., 2025) · Homogenization Effects of LLMs on Creative Ideation (Anderson et al., 2024) · Kaspersky on AI-generated passwords · We Have a Package for You! (USENIX Security 2025)

About the author

  • I'm a senior software engineer. I write Simply Explained, a plain-English breakdown of how AI and software actually work, and the quiet ways they break: AI fixes that hide bugs, agents that pick the wrong tool, health checks that take down healthy servers. One symptom, one analogy, one fix per piece. New here? Start with the pinned Start Here guide, or get one breakdown a week by email.

No comments yet. Be the first to comment.

AI for Humans

Part 23 of 23

AI explained in plain English, for people who aren't engineers and don't want to be. How it actually works, how to use it well, and how it's quietly changing work, privacy, and everyday life. Why AI confidently gets things wrong, whether ChatGPT is really private, how to spot AI fakes, what it means for your job, and how to use it without getting fooled or overwhelmed. No jargon, no hype — just clear answers to the questions everyone's actually asking.

Start from the beginningYou're Using AI Like a Search Bar. That's Why It's Disappointing You.The fix for mediocre AI results isn't a better prompt or a secret phrase — it's one mindset shift: stop treating a collaborator like a search engine.