Human Learning by Model Feedback: The Dynamics of Iterative Prompting with Midjourney

Shachar Don-Yehiya; Leshem Choshen; Omri Abend

doi:10.18653/V1/2023.EMNLP-MAIN.253

Human Learning by Model Feedback: The Dynamics of Iterative Prompting with Midjourney

Shachar Don-Yehiya, Leshem Choshen, Omri Abend · EMNLP 2023 · 2023

In one sentence

A dataset of 107,051 iterative Midjourney prompting threads shows that user prompts drift predictably along an interaction — growing longer, more magic-word-laden and lower in GPT-2 perplexity — evidence that humans both add missing details and adapt to the model's own language preferences.

Abstract

Generating images with a Text-to-Image model often requires multiple trials, where human users iteratively update their prompt based on feedback, namely the output image. Taking inspiration from cognitive work on reference games and dialogue alignment, this paper analyzes the dynamics of the user prompts along such iterations. We compile a dataset of iterative interactions of human users with Midjourney. Our analysis then reveals that prompts predictably converge toward specific traits along these iterations. We further study whether this convergence is due to human users, realizing they missed important details, or due to adaptation to the model's ``preferences'', producing better images for a specific language style. We show initial evidence that both possibilities are at play. The possibility that users adapt to the model's preference raises concerns about reusing user data for further training. The prompts may be biased towards the preferences of a specific model, rather than align with human intentions and natural manner of expression.

Questions this paper answers

when someone keeps rewriting a prompt for an image generator, do the rewrites change in a predictable direction?
how do linguistic features of text-to-image prompts shift with turn index within an interaction thread?
how can I tell whether my prompt edits across a Midjourney session follow a systematic pattern rather than random trial and error?
if I look at my own history of retried image prompts, should I expect them to be getting longer and more model-friendly over time?: Along a Midjourney interaction thread prompt length, magic-words ratio, repeated-words ratio, sentence rate and syntactic tree depth rise approximately monotonically with prompt index, while perplexity falls.
Holds for: 645 threads of at least 10 prompts, averaged at each index 1 to 10; the magic-words curve is noisiest and does not saturate within 10 prompts.; Rising prompt length, sentence rate and tree depth support Midjourney users adding omitted details. The rising magic-words and repeated-words ratios and falling perplexity support users adapting to the model's language preferences.
Holds for: An interpretive reading of the feature trends; the paper does not adjudicate between the two explanations, and notes falling perplexity can be partly a byproduct of increasing length.
what is different about the wording of image prompts whose results people actually keep?
which linguistic features separate upscaled from non-upscaled Midjourney prompts, and which fail to reach significance?
how should I word a text-to-image prompt so the output is one I want to keep?
is it worth making my image prompts longer and adding style keywords, or does concreteness matter more?: Midjourney prompts whose image the user upscaled are longer (16.67 vs 14.78 words), have a higher magic-words ratio (0.109 vs 0.096) and lower GPT-2 perplexity (2173 vs 2855) than non-upscaled prompts.
Holds for: 169,620 cleaned English Midjourney prompts, 25% of them upscaled; Mann-Whitney U test, significant after Bonferroni correction. Upscaling is a proxy for satisfaction.; Concreteness is the one linguistic feature that does not separate upscaled from non-upscaled Midjourney prompts, at 3.2628 versus 3.2629 with p = 0.123. Length, magic words, perplexity, repeated words, sentence rate and tree depth all differ significantly.
Holds for: Prompt-level average of Brysbaert et al. word concreteness ratings over 169,620 cleaned English Midjourney prompts; on BLIP-2 captions of the generated images the same feature does reach significance.
can you guess from the wording alone whether someone was happy with the picture an image generator made?
how well does a GPT-2 prompt-only classifier predict upscale events compared with a ResNet18 on the image grid?
how do I predict from logged prompt text whether a user will keep a generated image?
if I only have prompt text and no click data, can I get a usable signal about image satisfaction?: A GPT-2 classifier predicts whether a Midjourney image was upscaled from the prompt text alone at 58.2% accuracy, 8.2 points above random on balanced data. A ResNet18 given only the generated image grid reaches 55.6%, 5.6 points above random.
Holds for: 80/20 train-test split, classes balanced by sampling, standard deviations 0.26 and 0.21 over 3 seeds, no hyperparameter search. Accuracy is far too low for practical use.
is there a public collection of people's repeated attempts at prompting an image generator?
what corpus of multi-turn text-to-image interaction threads with upscale labels is available, and how long are the threads?
where do I get data to study how users revise image prompts across turns?
is a Midjourney Discord prompt corpus big enough for me to study multi-turn revision, or are most sessions single-shot?: The Midjourney threads dataset compiles 107,051 iterative human-model interaction threads scraped from the Midjourney Discord. Each prompt is paired with its generated image grid, upscale flags, timestamps and anonymized user ids.
Holds for: One 'newbies' Discord channel, English-language prompts only, collected 23 January to 1 March 2023; 169,620 prompts after cleaning. Threads are assigned automatically, not by human annotation.; Midjourney interaction threads are mostly very short: the average thread runs 1.58 prompts and only 645 of 107,051 threads (0.6%) contain 10 or more, with the longest reaching 77.
Holds for: Automatic intersection-over-union splits over 169,620 cleaned English prompts from 30,394 users; 6578 threads have at least 4 prompts, 2485 at least 6 and 1214 at least 8.
which study looks at how people change their language to suit an image generator?
what work frames iterative text-to-image prompting as a repeated reference game with one-sided human adaptation?
what should I read first about humans adapting their language to a text-to-image model over turns?
I need a citation for users drifting toward a generative model's preferred phrasing — which paper covers that?: Human Learning by Model Feedback frames iterative text-to-image prompting as a repeated reference game in which only the human adapts. It studies the linguistic dynamics of prompts across turns rather than the quality of any single prompt.
Holds for: As of publication in 2023 the authors state they know of no prior work examining prompt dynamics between iterations; the closest prior prompt-log analysis identified sessions by a 30-minute timeout.; Human Learning by Model Feedback argues that upscaled prompt-image pairs from text-to-image logs are risky as free RLHF preference data. Prompts may drift toward one model's stylistic preferences rather than natural human expression.
Holds for: A concern raised from adaptation trends observed on Midjourney and DiffusionDB, not an experiment: no model is trained on such data and no resulting degradation is measured.
is it safe to train a model on which pictures users chose to keep from their own prompts?
what is the risk of treating upscaled prompt-image pairs from generation logs as RLHF preference data?
how do I avoid bias if I build a preference dataset out of logged image-generation prompts and upscale clicks?
should I use Midjourney-style upscale logs as free human preference data for my image model?: Human Learning by Model Feedback argues that upscaled prompt-image pairs from text-to-image logs are risky as free RLHF preference data. Prompts may drift toward one model's stylistic preferences rather than natural human expression.
Holds for: A concern raised from adaptation trends observed on Midjourney and DiffusionDB, not an experiment: no model is trained on such data and no resulting degradation is measured.; Rising prompt length, sentence rate and tree depth support Midjourney users adding omitted details. The rising magic-words and repeated-words ratios and falling perplexity support users adapting to the model's language preferences.
Holds for: An interpretive reading of the feature trends; the paper does not adjudicate between the two explanations, and notes falling perplexity can be partly a byproduct of increasing length.
do the same patterns in repeated image prompts show up on data from a different generator?
do the thread-level feature trends replicate on DiffusionDB Stable Diffusion prompts, and which feature fails?
can I expect prompt-revision trends measured on Midjourney to carry over to Stable Diffusion logs I have?
if my logs come from Stable Diffusion rather than Midjourney, do the reported iteration trends still apply?: The thread-dynamics trends replicate on DiffusionDB Stable Diffusion prompts for every feature except the magic-words ratio, which does not stay approximately monotone.
Holds for: First 250,000 prompts of the DiffusionDB 2M subset, 105,644 after cleaning, 14,927 threads of which 1045 have at least 10 prompts. DiffusionDB lacks upscale labels.
how do you work out which consecutive image prompts are all attempts at the same picture?
how accurate is word intersection-over-union thread segmentation of consecutive prompts against manual annotation, versus a BERTScore threshold?
how do I segment a stream of logged prompts into separate interaction threads?
should I use a word-overlap rule or an embedding-similarity rule to group my logged prompts into sessions?: Splitting consecutive Midjourney prompts into threads by word intersection-over-union above 0.3 matches 500 manual annotations at F1 0.87 with WindowDiff 0.24, beating a BERTScore-threshold split at F1 0.84 and WindowDiff 0.30.
Holds for: Manual annotation by one author on users with at least 4 prompts, two further authors re-annotating 70 prompts each, Fleiss' kappa 0.815. Both methods assume non-overlapping threads.
do people end up at a similar prompt length no matter how long their first prompt was?
do lengthening and shortening Midjourney threads converge on a common prompt-length range?
how long should my image prompts be if I want to land where experienced users end up?
my image prompts are very long — is trimming them toward a middle length what other users converge on?: Midjourney threads whose prompts get longer start relatively short and threads whose prompts get shorter start relatively long, with both groups converging toward the same length range.
Holds for: Threads split into two sets by whether the last prompt's feature value exceeds the first; group means only, with no causal test of why the range is preferred.
does using more concrete, physical wording make an image generator give you something you keep?
does mean word concreteness differ significantly between upscaled and non-upscaled text-to-image prompts?
should I swap abstract words for concrete ones when revising an image prompt?
is it worth rewriting my prompts to be more concrete, or does that not affect whether I keep the image?: Concreteness is the one linguistic feature that does not separate upscaled from non-upscaled Midjourney prompts, at 3.2628 versus 3.2629 with p = 0.123. Length, magic words, perplexity, repeated words, sentence rate and tree depth all differ significantly.
Holds for: Prompt-level average of Brysbaert et al. word concreteness ratings over 169,620 cleaned English Midjourney prompts; on BLIP-2 captions of the generated images the same feature does reach significance.
do phrases like '8K' and 'highly detailed' really go with image prompts people keep, and do users add more of them as they go?
how do magic-words ratios relate to upscale outcomes and to prompt index within an interaction thread?
should I be stacking aesthetic quality keywords into my image prompts, and do users pile on more over a session?
are style booster keywords worth adding to my image prompts, or is that just a habit users pick up as a session drags on?: Midjourney prompts whose image the user upscaled are longer (16.67 vs 14.78 words), have a higher magic-words ratio (0.109 vs 0.096) and lower GPT-2 perplexity (2173 vs 2855) than non-upscaled prompts.
Holds for: 169,620 cleaned English Midjourney prompts, 25% of them upscaled; Mann-Whitney U test, significant after Bonferroni correction. Upscaling is a proxy for satisfaction.; Along a Midjourney interaction thread prompt length, magic-words ratio, repeated-words ratio, sentence rate and syntactic tree depth rise approximately monotonically with prompt index, while perplexity falls.
Holds for: 645 threads of at least 10 prompts, averaged at each index 1 to 10; the magic-words curve is noisiest and does not saturate within 10 prompts.

Claims and scope

The Midjourney threads dataset compiles 107,051 iterative human-model interaction threads scraped from the Midjourney Discord. Each prompt is paired with its generated image grid, upscale flags, timestamps and anonymized user ids. (Section 3 and Section 4.3)
Scope: One 'newbies' Discord channel, English-language prompts only, collected 23 January to 1 March 2023; 169,620 prompts after cleaning. Threads are assigned automatically, not by human annotation.
Human Learning by Model Feedback frames iterative text-to-image prompting as a repeated reference game in which only the human adapts. It studies the linguistic dynamics of prompts across turns rather than the quality of any single prompt. (Section 1 and Section 10)
Scope: As of publication in 2023 the authors state they know of no prior work examining prompt dynamics between iterations; the closest prior prompt-log analysis identified sessions by a 30-minute timeout.
Midjourney prompts whose image the user upscaled are longer (16.67 vs 14.78 words), have a higher magic-words ratio (0.109 vs 0.096) and lower GPT-2 perplexity (2173 vs 2855) than non-upscaled prompts. (Table 1)
Scope: 169,620 cleaned English Midjourney prompts, 25% of them upscaled; Mann-Whitney U test, significant after Bonferroni correction. Upscaling is a proxy for satisfaction.
Concreteness is the one linguistic feature that does not separate upscaled from non-upscaled Midjourney prompts, at 3.2628 versus 3.2629 with p = 0.123. Length, magic words, perplexity, repeated words, sentence rate and tree depth all differ significantly. (Table 1)
Scope: Prompt-level average of Brysbaert et al. word concreteness ratings over 169,620 cleaned English Midjourney prompts; on BLIP-2 captions of the generated images the same feature does reach significance.
A GPT-2 classifier predicts whether a Midjourney image was upscaled from the prompt text alone at 58.2% accuracy, 8.2 points above random on balanced data. A ResNet18 given only the generated image grid reaches 55.6%, 5.6 points above random. (Section 6.1)
Scope: 80/20 train-test split, classes balanced by sampling, standard deviations 0.26 and 0.21 over 3 seeds, no hyperparameter search. Accuracy is far too low for practical use.
Along a Midjourney interaction thread prompt length, magic-words ratio, repeated-words ratio, sentence rate and syntactic tree depth rise approximately monotonically with prompt index, while perplexity falls. (Figure 3)
Scope: 645 threads of at least 10 prompts, averaged at each index 1 to 10; the magic-words curve is noisiest and does not saturate within 10 prompts.
Midjourney threads whose prompts get longer start relatively short and threads whose prompts get shorter start relatively long, with both groups converging toward the same length range. (Figure 4 and Figure 7)
Scope: Threads split into two sets by whether the last prompt's feature value exceeds the first; group means only, with no causal test of why the range is preferred.
Rising prompt length, sentence rate and tree depth support Midjourney users adding omitted details. The rising magic-words and repeated-words ratios and falling perplexity support users adapting to the model's language preferences. (Section 7)
Scope: An interpretive reading of the feature trends; the paper does not adjudicate between the two explanations, and notes falling perplexity can be partly a byproduct of increasing length.
Human Learning by Model Feedback argues that upscaled prompt-image pairs from text-to-image logs are risky as free RLHF preference data. Prompts may drift toward one model's stylistic preferences rather than natural human expression. (Section 9)
Scope: A concern raised from adaptation trends observed on Midjourney and DiffusionDB, not an experiment: no model is trained on such data and no resulting degradation is measured.
The thread-dynamics trends replicate on DiffusionDB Stable Diffusion prompts for every feature except the magic-words ratio, which does not stay approximately monotone. (Figure 5)
Scope: First 250,000 prompts of the DiffusionDB 2M subset, 105,644 after cleaning, 14,927 threads of which 1045 have at least 10 prompts. DiffusionDB lacks upscale labels.
Splitting consecutive Midjourney prompts into threads by word intersection-over-union above 0.3 matches 500 manual annotations at F1 0.87 with WindowDiff 0.24, beating a BERTScore-threshold split at F1 0.84 and WindowDiff 0.30. (Section 4.2)
Scope: Manual annotation by one author on users with at least 4 prompts, two further authors re-annotating 70 prompts each, Fleiss' kappa 0.815. Both methods assume non-overlapping threads.
Midjourney interaction threads are mostly very short: the average thread runs 1.58 prompts and only 645 of 107,051 threads (0.6%) contain 10 or more, with the longest reaching 77. (Figure 2 and Section 4.3)
Scope: Automatic intersection-over-union splits over 169,620 cleaned English prompts from 30,394 users; 6578 threads have at least 4 prompts, 2485 at least 6 and 1214 at least 8.

Common misreadings

The linguistic features reported for upscaled versus non-upscaled Midjourney prompts are statistically significant but explain only a small proportion of the variance; they are not a recipe for writing good prompts.
The rise in magic words and fall in perplexity along a Midjourney thread are evidence consistent with users adapting to the model's preferences, not a demonstration that such prompts cause better images.
Human Learning by Model Feedback does not decide between 'users add omitted details' and 'users adopt model-like language', nor quantify their relative contribution; it presents evidence that both are at play.
Upscaling is a proxy for satisfaction, not a ground-truth quality label: users sometimes upscale images because they are amusingly bad or to record the creation process.
The claim that reusing upscaled prompts as RLHF data would make models more 'model-like' is a concern argued from the observed drift, not a trained-and-measured result.
Thread boundaries in the released Midjourney dataset come from an automatic intersection-over-union heuristic validated against 500 manual annotations, not from human labelling of the full corpus.
The Midjourney dynamics analysis rests on the 645 threads with at least 10 prompts, a 0.6% tail of the corpus, since the average thread is only 1.58 prompts long.

Terminology in this paper

thread: A sequence of consecutive prompts by one user that are all attempts to generate the same target image or scene, ending when the described scene or main subject changes intrinsically.
magic words: Words that add no real content to a text-to-image prompt but are popular among practitioners, such as 'beautiful', '8k' and 'highly detailed'; operationally, words appearing at least 1000 times in the Midjourney prompt corpus whose corpus probability exceeds their Google-ngrams probability by a factor of at least 100.
upscale: A Midjourney Discord command requesting a higher-resolution version of one image from the generated 4-image grid, used as a proxy signal that the user was satisfied with that image.
semi-reference game: A repeated reference game in which only one participant can adapt: the human user revises prompts based on the generated image while the text-to-image model stays frozen.

How to cite

@inproceedings{DBLP:conf/emnlp/Don-YehiyaCA23,author       = {Shachar Don{-}Yehiya and
                  Leshem Choshen and
                  Omri Abend},
  editor       = {Houda Bouamor and
                  Juan Pino and
                  Kalika Bali},
  title        = {Human Learning by Model Feedback: The Dynamics of Iterative Prompting
                  with Midjourney},
  booktitle    = {Proceedings of the 2023 Conference on Empirical Methods in Natural
                  Language Processing, {EMNLP} 2023, Singapore, December 6-10, 2023},
  pages        = {4146--4161},
  publisher    = {Association for Computational Linguistics},
  year         = {2023},
  url          = {https://doi.org/10.18653/v1/2023.emnlp-main.253},
  doi          = {10.18653/V1/2023.EMNLP-MAIN.253},
  timestamp    = {Fri, 12 Apr 2024 01:00:00 +0200},
  biburl       = {https://dblp.org/rec/conf/emnlp/Don-YehiyaCA23.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

References

See the full reference list in the paper.