Expert Personas in Prompts: What They Change and What They Don't

Try this before you read anything else. In a repo that ships an LLM feature, run:

grep -rniE "you are (a|an) " --include="*.py" --include="*.ts" --include="*.txt" --include="*.md" --include="*.yaml" .

If you get hits, you're in good company. "You are a world-class senior engineer" (or analyst, or support agent, or security auditor) is one of the most common first lines in system prompts.

This post is about what that line actually does. Short version: it changes the voice of the output. Two systematic studies found it doesn't reliably change the accuracy. And in production it has a few side effects most teams never measure.

TL;DR

• Expert personas ("You are an expert X") did not reliably improve factual accuracy across hundreds of roles, thousands of questions, and multiple model families.
• What a persona does move: tone, vocabulary, confidence, length, assumed audience. That makes answers sound checked when they aren't.
• In production, personas can trigger off-domain refusals, lower accuracy when the role is low-knowledge, and behave differently after a model upgrade.
• Replace the title with a job description: Audience, Standard, Checks, When unsure, Deliverable.
• Verify with the Strip Test: run the prompt with and without the persona on inputs where you already know the right answer.

What the evidence says

Read the table as a whole and the pattern is clear: persona effects exist, but they're model-specific, task-specific, and unpredictable. That's the signature of a wording perturbation, not a capability you're unlocking. The Wharton authors' own recommendation is to put the effort into task-specific instructions, examples, and evaluation instead.

Why a title can't add knowledge

The model's knowledge is fixed in its weights before your request arrives. The prompt conditions which existing patterns are likely in the output. It doesn't load new facts.

"You are a senior security engineer" conditions toward text that looks like a senior security engineer wrote it: domain vocabulary, a terse register, low hedging. There's no hidden expert mode behind the title. If the model can spot an IDOR bug, it can spot it without being told it's an expert. If it can't, the title won't help.

The useful part of a human expert was never their title. It was their job description: who the work is for, what "good" means, what they check, what they do when unsure, and what they hand back.

The left column is visible in every output, which is why personas feel like they work. The right column is what you actually need, and it's the one the studies found no reliable movement in.

That's also the risk. Tone is how humans judge expertise. A confident, jargon-correct answer reads like it was reviewed, so you check it less, especially in areas where you couldn't check it well anyway. (This is the same gap covered in Why AI Feels Smarter About Topics You Don't Know, and the same "changes the look, not the knowledge" trap as Temperature 0 Won't Fix Your AI.)

Three production failure modes

1. Off-domain refusals

In the Wharton tests, Gemini 2.5 Flash given an expert persona from a field unrelated to the question often declined to answer: on average about 10.6 of 25 attempts per question. If your assistant's system prompt says "You are an expert in our billing system," an adjacent question about account security may get treated as out of scope. Whether it does depends on model, wording, and version. That's a lot of product behavior riding on one adjective-heavy sentence.

2. Low-knowledge roles lower accuracy

Trying to simplify output with "act as a layperson" or similar is risky. In the same report, all three low-knowledge personas (layperson, young child, toddler) scored below baseline on o4-mini, and the toddler persona significantly hurt GPT-4o. Set the audience instead of the identity:

- You are a beginner. Explain how DNS works.
+ Explain how DNS works to a developer who knows HTTP but has never configured a domain.

3. Upgrade drift

Gemini 2.0 Flash got modest gains from expert personas on MMLU-Pro; Gemini 2.5 Flash didn't. A persona added because it "seemed better" on an older model is an untested assumption on the current one, and it's still being sent on every call.

The refactor: title to job description

Before:

You are a world-class senior security engineer with 20 years of experience.
Review this code for security issues.

After:

Review this pull request for security problems before it is merged.

Audience: the author, a mid-level developer who will fix issues today.

Standard: this endpoint is public and handles customer account data.
Anything that could expose another customer's data is a blocker.

Check specifically:
- SQL built with string concatenation or f-strings
- object IDs from the request used without checking the caller owns them
- secrets, tokens, or personal data written to logs
- user input passed to redirects, file paths, or shell commands

When unsure: if a check depends on code you can't see (middleware,
auth config), say what you'd need to see instead of assuming it's safe.

Deliverable: findings, most severe first. For each: file and line,
the risk in one sentence, and the fix.

The second prompt makes no claim about who the model is. It carries everything the title was implying, in a form your team can review in a PR. The "Checks" block is the highest-value part: it's the accumulated list of what goes wrong, which is what experience actually is. (For more on reviewing model-written code, see AI Writes Code That Looks Right.)

The "When unsure" line matters more than it looks. Without it, gaps get filled with plausible defaults in the same confident tone. It's the prompt-level version of giving a schema an honest escape hatch, as in Structured Outputs Still Hallucinate.

The Strip Test as a tiny eval

The trick is to use inputs where you already know the right answer, for example three PRs with bugs you've already found and fixed. Then "better" means "caught more of the known issues," not "sounded more senior."

# strip_test.py: compare a prompt with and without its persona line
PERSONA = "You are a world-class senior security engineer."
TASK = open("review_prompt.txt").read()

def call_model(system: str, user: str) -> str:
    """Wrap your provider's SDK here. Keep model, temperature,
    and max tokens identical for both variants."""
    raise NotImplementedError

# inputs with known answers: diffs whose bugs you already found
cases = {
    "pr_101.diff": ["unscoped order lookup by id"],
    "pr_117.diff": ["auth token written to request log"],
    "pr_122.diff": ["string-built SQL in search filter"],
}

RUNS = 3  # outputs vary run to run; don't judge on one sample

for path, known_issues in cases.items():
    diff = open(path).read()
    for label, system in [("with", PERSONA + "\n\n" + TASK), ("without", TASK)]:
        for r in range(RUNS):
            out = call_model(system, diff)
            print(f"\n=== {path} | {label} | run {r+1} ===\n{out}")
    print(f"\nKnown issues for {path}: {known_issues}")

Score each output by hand against the known issues. Then:

• Same substance: the persona is a costume. Keep it only if you want that tone.
• Worse without it: the persona was smuggling in an assumption (audience, depth, focus). Find it and write it as an explicit line.
• Better without it: delete it.

Re-run whenever you change models. Persona effects can shift between versions.

When a role is still worth keeping

Roles are fine for voice and stance: "write as a patient teacher," "argue as a skeptical reviewer." Even then, notice that the useful part of "act as a skeptical reviewer whose job is to find the weakest assumption" is the task clause, not the noun. The Wharton authors also note personas can help with tone, perspective, and approach. Just don't count on them for correctness.

Key Takeaways

• A persona line conditions style, not knowledge. Systematic studies found no reliable accuracy gain from expert personas.
• Persona effects are model-specific and can flip between versions, so treat any "it works" as untested until measured.
• Personas can cause off-domain refusals; low-knowledge roles can reduce accuracy.
• Replace titles with five explicit lines: Audience, Standard, Checks, When unsure, Deliverable.
• Validate with the Strip Test on inputs with known answers, multiple runs, scored on substance.

FAQ

Does telling ChatGPT to "act as an expert" make it more accurate?

Not reliably. Zheng et al. tested 162 personas on 2,410 factual questions across four model families and found no improvement over a no-persona prompt. Wharton's December 2025 report found no expert persona reliably beat baseline on GPQA Diamond across six models. Some models showed gains in specific cases, but they weren't consistent.

What is role prompting?

Role prompting means assigning the model an identity at the start of a prompt, such as "You are a senior data scientist." It's usually done to steer tone and focus, and is often assumed to improve accuracy. The evidence supports the first use much more than the second.

Why do some people say persona prompts work?

Because some results show gains: Kong et al. (2023) reported AQuA accuracy rising from 53.5% to 63.8% on ChatGPT with a designed role-play setup. But other studies found the opposite (role-play hurt Llama 3 on 7 of 12 datasets), and the effect varies by model and version. Also, persona outputs sound more expert, which is easy to mistake for being more correct.

Should I remove "You are a..." from my system prompt?

Test it first. Run the Strip Test: the same prompt with and without the line, on inputs with known answers, several runs each. If substance is the same, the line is only setting tone. If it's worse without, write down what the persona was implying. If it's better without, remove it.

What should I write instead of an expert persona?

A job description in five lines: the audience and what they'll do with the output, the standard for "good," the specific checks to perform, what to do when information is missing, and the exact deliverable format. These are the things expertise actually consists of.

Do expert personas help with anything?

Yes: voice, perspective, and stance. A role is a quick way to get a patient teaching tone or a critical reviewer's posture. Keep roles for how the answer should sound or what position to take, not for making it more correct.

Related reading

• Why AI Feels Smarter About Topics You Don't Know
• Temperature 0 Won't Fix Your AI. It Just Hides the Doubt.
• AI Writes Code That Looks Right. That's Exactly the Danger.
• Why Your AI Answers Are Mediocre (And the 7 Prompt Fixes That Change Everything)
• New here? Start Here: A Plain-English Map to Understanding AI

The bottom line

The model already brings everything it knows to every request. What it can't bring is your context: who it's for, what good looks like, what usually breaks, and what to do with doubt. A title gestures at all of that. A job description actually says it.

Titles set the voice. Descriptions set the work. Only one of them shows up in the accuracy.

Get one of these in your inbox each week: subscribe to the Simply Explained newsletter.

Adam Jaber is a software engineer who writes Simply Explained: complex topics, made simple. No jargon, no hype.