As noted by ITmedia, a research team from the University of Washington and the University of Colorado Boulder recently released a study entitled “AI Fiction in the Wild,” which delves into over 500,000 anonymous English-language ChatGPT conversation records. Interestingly, among its conclusions, the research highlights a particularly active user who spent months producing vast amounts of Doki Doki Literature Club! pregnancy-themed fanfiction. The study discovered that there was a pattern among a certain group of users to repeatedly ask for fanfiction of particular intellectual properties, as well as a noticeable trend of requesting explicit material. The findings come at a relevant moment given the ongoing discussion about AI-generated content in fandom and fanfiction.
The “AI Fiction in the Wild” research investigates how AI users employ ChatGPT to create fiction, considering both volume and subject matter. It utilizes the WildChat dataset, which comprises conversations gathered through a no-cost chatbot on Hugging Face, where participants gained access to GPT-3.5 Turbo and GPT-4 without needing an OpenAI account after agreeing to their conversations possibly being used and anonymously shared for research purposes.
The researchers examined approximately 573,000 English conversations gathered between April 2023 and May 2024. Since the subject focused specifically on fiction, they first determined which conversations qualified by filtering only content involving hypothetical or fictional scenarios (AI was also used for this process, though humans manually verified the accuracy of filtering based on a sample of 300 selected conversations).
Of the approximately 573,000 conversations, around 195,000 were categorized as fiction, and of those, roughly 52,000 included “sexually explicit material.” Additionally, another 67,000 conversations were marked “toxic” by the dataset. This suggests that almost 30% of the fiction produced by AI users contained sexual content.
Another intriguing detail is that the researchers found that fiction creation was heavily concentrated among a very small group of “intensive users.” According to the data, the top 2% of fiction-creating users accounted for more than 80% of all fiction-related conversation logs. The total number of users producing fiction was estimated at around 10,000, which implies that roughly 200 people were responsible for over 150,000 fiction prompts.
Among these intensive users, the researchers identified a couple of unique patterns of behavior. One type, termed the “story recyclers,” repeatedly generated versions of the same narrative for a certain period before shifting to another subject. Others, labeled “endless story requesters,” spent extended periods repeatedly asking for nearly identical stories with only slight modifications.
The aforementioned Dok Doki Literature Club! fanfic creator is given as a prime example of an endless story requester. Over several months, this individual prompted ChatGPT to produce fanfiction based on the game thousands of times, with the very specific premise of character Natsuki unexpectedly going into labor and ChatGPT being asked to continue the story from there.
In response, ChatGPT generated a wide range of outcomes, such as a scenario where emergency services arrive in time, and both mother and baby survive safely. While the paper describes this user as an extreme exception, they observe that many intensive fiction users showed similar inclinations. Among prompts submitted by the top 2% of users, 69% were repetitive prompts, with users repeatedly trying to refine or re-run nearly identical requests.
The study also ranked the franchises most frequently mentioned in these fiction-related conversations. Doki Doki Literature Club! led the list with 22,381 mentions, followed by Freedom Planet (5,204), League of Legends (4,514), and Naruto (4,342). However, it’s important to note for all the reported findings that the WildChat dataset is not representative of all ChatGPT users. Since it originates from a free chatbot on Hugging Face, its users were likely more tech-savvy and more involved in online culture than the typical AI chatbot user. Even so, the researchers contend that WildChat provides a rare view into real-world interactions with ChatGPT.