Do LLMs Benefit From Their Own Words?

when a model's own earlier response carries errors, hallucinations or style into later turns

Jenny Y. Huang, Leshem Choshen, Ramon Astudillo, Tamara Broderick, Jacob Andreas · 2026

In one sentence

Deleting all prior assistant responses from multi-turn chat histories on WildChat and ShareLM often leaves response quality unchanged while cutting cumulative context by 5-10x, because 36.4% of in-the-wild turns are self-contained and past responses can pollute later ones.

Abstract

In multi-turn conversations, large language models typically condition on the full conversation history: both past user prompts and assistant responses. We revisit this design choice by comparing full-context prompting to four alternative, substantially-reduced context configurations. Analyzing in-the-wild multi-turn conversations across three open reasoning and one state-of-the-art model, we find that response quality is largely preserved under aggressive context filtering: replacing all prior assistant turns with one-sentence summaries or keeping only the most recent user--assistant exchange often matches storing full context in performance while using roughly 8x less context. To understand this result, we observe that a substantial fraction of user turns (36.4%) in multi-turn conversations are self-contained and that many follow-up turns can be addressed by seeing only the immediately preceding user--assistant exchange. Furthermore, we find that when models condition on their own past responses, this can lead to context pollution, a phenomenon in which reasoning errors, hallucinations, or stylistic artifacts propagate across turns. Motivated by these findings, we design a context-filtering approach that selectively omits the assistant-side history. Taken together, these findings suggest moving away from storing full dialogue transcripts and instead retaining only what is relevant.

Questions this paper answers

If you delete a chatbot's own earlier replies from the conversation history, do its later answers get worse?
How does assistant-turn omission in the dialogue history affect judged response quality and on-topic rate across instruction-tuned LLMs?
How do I trim assistant turns out of a multi-turn prompt without degrading the next response?
Should I keep past model replies in my chat history, or send only the user turns?
Omitting all prior assistant responses preserves average response quality for DeepSeek-R1-Distill-Llama-8B and GPT-OSS-20B, while Qwen3-4B and GPT-5.2 lose some quality relative to full context. Responses stay on-topic under omission for all four models.
Holds for: 300 real-world multi-turn conversations sampled from WildChat and ShareLM, 5-10 rounds each, filtered to English technical chats; GPT-5 as pairwise LLM judge seeing prior user and assistant turns.
When the GPT-5 judge is shown only prior user-side turns, assistant-omitted responses match or beat full-context responses on both quality and on-topic dimensions for all four models. That reverses the full-context judge's preference for Qwen3-4B and GPT-5.2.
Holds for: Same 300 WildChat and ShareLM conversations and same four models (Qwen3-4B, DeepSeek-R1-Distill-Llama-8B, GPT-OSS-20B, GPT-5.2); a judge without assistant history cannot resolve prompts that explicitly reference an earlier response.
How many characters of prompt do you save in a long chat by keeping only what the user wrote?
What context-length reduction does user-turn-only prompting yield over full dialogue history by round 8?
How do I cut token spend on a growing chat history without truncating or summarizing?
Will dropping assistant turns actually make my multi-turn requests meaningfully cheaper?
Full-context histories grow to roughly 25,000-55,000 characters by round 8, whereas user-turn-only context stays near-constant at 5,000-10,000 characters, a 5 to 10x reduction in context usage.
Holds for: Character counts over 5-10 round WildChat and ShareLM technical conversations; measured in characters rather than tokens, and does not include generated response length.
In real conversations with a chatbot, how often is a person's next message a brand-new request rather than a follow-up?
What proportion of non-initial user turns in in-the-wild human-LLM logs are self-contained versus dependent on prior assistant output?
How do I tell whether a multi-turn chat corpus really tests long-context multi-turn reasoning?
Can I trust WildChat-style logs as a benchmark for multi-turn context dependence?
In real-world multi-turn chats, 36.4% of non-initial user turns are self-contained new asks, 30.5% are follow-ups with concrete feedback, and 33.1% are follow-ups referencing an earlier turn without actionable feedback.
Holds for: GPT-5 as automated annotator over the sampled WildChat and ShareLM English technical conversations of 5-10 rounds; the 33.1% is an upper bound on assistant-dependence.
"Do LLMs Benefit From Their Own Words?" argues that in-the-wild multi-turn chat logs are weak benchmarks for long-context multi-turn reasoning, and calls for corpora curated for genuine multi-turn dependence. The reason is that a large share of turns in real chat logs do not depend on earlier assistant responses.
Holds for: Argued from English technical (math and coding keyword) conversations of 5-10 rounds in WildChat and ShareLM; agentic settings with tool outputs and scratchpads are discussed but not measured.
Which kinds of user messages still need the chatbot's earlier answer in the history, and which do not?
Does the benefit of retaining assistant history differ between new-ask turns and follow-up turns?
How do I decide per turn whether a user message needs the assistant's prior response?
If most of my users ask follow-up questions, is stripping assistant history still safe for me?
For Qwen3-4B and GPT-5.2, full-context and assistant-omitted prompting perform comparably on new-ask turns, while full context helps most on follow-up turns; assistant-omitted context still wins roughly 40% of follow-up comparisons for Qwen3-4B and 30% for GPT-5.2.
Holds for: The two models for which uniform omission lowered overall quality; win rates averaged over the quality and on-topic dimensions, ties handled as in the pairwise judge protocol.
In real-world multi-turn chats, 36.4% of non-initial user turns are self-contained new asks, 30.5% are follow-ups with concrete feedback, and 33.1% are follow-ups referencing an earlier turn without actionable feedback.
Holds for: GPT-5 as automated annotator over the sampled WildChat and ShareLM English technical conversations of 5-10 rounds; the 33.1% is an upper bound on assistant-dependence.
Can a chatbot's own earlier mistakes in the conversation get repeated into later answers?
What does context pollution from retained assistant turns look like in multi-turn LLM conversations?
How do I stop a wrong assumption or hallucination from an earlier reply carrying into later turns?
Is keeping my model's previous outputs in the prompt risking repeated hallucinations and stale code?
5 conversations in which assistant-omitted responses scored far above full context exhibit context pollution. The cases are UMAP arguments carried into t-SNE code, hallucinated book titles repeated across turns, a misattributed NBER citation, tutorial style overriding a reflection request, and a reversed temperature formula.
Holds for: Cases surfaced by sorting rounds by 1-10 judge score gap (AO minus FC) and reviewing the largest positive gaps; illustrative, not a frequency estimate, and includes GPT-5.2 as well as smaller open models.
Can a small model learn when to keep the chatbot's earlier replies and when to throw them away?
Can a per-turn classifier select between full and assistant-omitted context, and is judge preference predictable from turn features?
How do I build a per-turn policy that drops assistant history only when it is safe?
Is adaptive per-turn context filtering worth implementing, or will I lose quality against always sending full history?
A per-turn classifier choosing between full and assistant-omitted context retains over 95% of full-context-only win-or-tie performance, and matches full-context-only at about 70% of the context consumption. It beats a heuristic that omits assistant history on all new-ask turns.
Holds for: GPT-5.2 only; L1-regularized logistic regression over round metadata, prompt category and PCA-reduced text-embedding-3-large embeddings; ties counted as wins, and the heuristic baseline is measured on a 20% held-out subset so it may improve with more data.
Predicting whether the judge will prefer full context over assistant-omitted context is hard: the L1-regularized logistic regression reaches only a 5-fold cross-validated F1 of 0.6106 ± 0.0119. None of the top 20 features are significant at the 5% level.
Holds for: GPT-5.2 responses on the sampled WildChat and ShareLM conversations; features are round metadata, prompt category, and 20 principal components each of prompt and history embeddings explaining 38.0% and 51.5% of variance.
Is it better to replace a chatbot's past answers with one-sentence summaries than to keep them in full?
How does one-sentence self-summarization of assistant turns compare with full history and with full omission on Lost-in-Conversation and WildChat?
How do I compress assistant turns in a chat history instead of deleting them outright?
Should I summarize my model's earlier replies rather than dropping them or keeping them verbatim?
Replacing each assistant response with a one-sentence self-summary improves response quality over full context for both DeepSeek-R1-Distill-Llama-8B and Qwen3-4B on both Lost-in-Conversation and WildChat. Fully omitting assistant turns helps on Lost-in-Conversation but is mixed on WildChat.
Holds for: 2 models and 2 datasets, using an earlier pairwise judge pipeline that compared final responses only, so numbers are not directly comparable to the main-text 1-10 scoring runs.
How much can you trust a big model's scoring of which chat answer is better, and does what it sees change its verdict?
How well does the GPT-5 judge agree with manual annotation, and does restricting the judge to user-side turns flip the preference between context conditions?
How do I set up an LLM judge for comparing responses under different context conditions without biasing it toward the longer history?
Can I rely on LLM-as-judge results comparing full-context and assistant-omitted responses?
The GPT-5 judge agreed with an author's manual verdict on 54 of 60 quality judgments (90.0%) and 55 of 60 on-topic judgments (91.7%).
Holds for: 15 sampled judgments per model across the 4 models, annotated by one of the authors; no larger-scale or multi-annotator human study was run.
When the GPT-5 judge is shown only prior user-side turns, assistant-omitted responses match or beat full-context responses on both quality and on-topic dimensions for all four models. That reverses the full-context judge's preference for Qwen3-4B and GPT-5.2.
Holds for: Same 300 WildChat and ShareLM conversations and same four models (Qwen3-4B, DeepSeek-R1-Distill-Llama-8B, GPT-OSS-20B, GPT-5.2); a judge without assistant history cannot resolve prompts that explicitly reference an earlier response.
Is there a study that asks whether keeping a chatbot's own past replies in the prompt is actually worth it?
What work evaluates assistant-history retention on in-the-wild human-LLM logs rather than synthetic multi-turn dialogues?
What should I read before designing context management for a multi-turn chat or agent system?
Which paper should I cite if I want to argue that real chat logs are weak multi-turn benchmarks?
Huang et al.'s "Do LLMs Benefit From Their Own Words?" questions the default assumption of multi-turn chat and agentic context management that retaining a model's own past responses reliably helps. It tests that assumption on in-the-wild human-LLM logs rather than synthetic dialogues.
Holds for: As of the February 2026 preprint; prior turn-level context-editing work such as ERGO evaluated on synthetic conversations, and earlier conversational-QA findings about irrelevant turns concerned human-human histories.
"Do LLMs Benefit From Their Own Words?" argues that in-the-wild multi-turn chat logs are weak benchmarks for long-context multi-turn reasoning, and calls for corpora curated for genuine multi-turn dependence. The reason is that a large share of turns in real chat logs do not depend on earlier assistant responses.
Holds for: Argued from English technical (math and coding keyword) conversations of 5-10 rounds in WildChat and ShareLM; agentic settings with tool outputs and scratchpads are discussed but not measured.
Do big commercial chatbots get thrown off by their own earlier replies, or is that only a small-model problem?
Which models were evaluated for assistant-turn omission, and does over-conditioning on prior assistant output appear in frontier models too?
How do I check whether the model I use is being misled by its own earlier turns?
I run a frontier model in production, is over-conditioning on its own past answers something I have to worry about?
Omitting all prior assistant responses preserves average response quality for DeepSeek-R1-Distill-Llama-8B and GPT-OSS-20B, while Qwen3-4B and GPT-5.2 lose some quality relative to full context. Responses stay on-topic under omission for all four models.
Holds for: 300 real-world multi-turn conversations sampled from WildChat and ShareLM, 5-10 rounds each, filtered to English technical chats; GPT-5 as pairwise LLM judge seeing prior user and assistant turns.
5 conversations in which assistant-omitted responses scored far above full context exhibit context pollution. The cases are UMAP arguments carried into t-SNE code, hallucinated book titles repeated across turns, a misattributed NBER citation, tutorial style overriding a reflection request, and a reversed temperature formula.
Holds for: Cases surfaced by sorting rounds by 1-10 judge score gap (AO minus FC) and reviewing the largest positive gaps; illustrative, not a frequency estimate, and includes GPT-5.2 as well as smaller open models.

Claims and scope

Common misreadings

Terminology in this paper

Assistant-Omitted (AO) context
A prompting configuration in which every past assistant response in a multi-turn conversation is replaced by the placeholder "[Response provided]", so the model conditions only on prior user turns while the alternating user/assistant structure is preserved.
context pollution
The phenomenon in which a language model over-conditions on its own earlier responses, so errors, hallucinations or stylistic artifacts introduced in one turn propagate into later turns.
New Ask
A non-initial user turn that introduces a fully self-contained request, understandable without any prior conversation round.
Follow-up with Feedback
A user turn that gives concrete, actionable feedback on a prior assistant response, such as "use Python instead of Java for the code example".
Follow-up without Feedback
A user turn that refers to an earlier conversation round without any concrete instruction for revision, such as "reflect on your response".

How to cite

@misc{huang2026llmsbenefitwords,
      title={Do LLMs Benefit From Their Own Words?}, 
      author={Jenny Y. Huang and Leshem Choshen and Ramon Astudillo and Tamara Broderick and Jacob Andreas},
      year={2026},
      eprint={2602.24287},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2602.24287}, 
}

References

See the full reference list in the paper.