Wikipedia talk:WikiProject AI Cleanup

Main pageDiscussionGuideResourcesPoliciesResearch

Any way to track users who have received an AI use warning?

As long as the off-wiki world continues to have widespread LLM use, we are still going to get new users who hop on an immediately start making LLM edits. As our AI policy gains awareness, recent changes/new page patrollers are going to start doing a lot of drive-by templating of these users, who absent further intervention/policy explanation will probably make dozens or hundreds more LLM edits before ending up at ANI and most likely blocked. Is there a way for those of us who do a lot of LLM cleanup to see when this happens, so we can intervene as soon as possible to prevent further damage to the wiki and salvage new editors before they dig themselves into a indef-bound hole? Since warning templates are substituted rather than transcluded I can't just use the "what links here" tool. -- LWG ^{talk (VOPOV)} 19:53, 27 April 2026 (UTC)

The standard templates add the use talk page to Category:User talk pages with large language model notices, which currently has ca. 5200 pages. --Gurkubondinn (talk) 20:21, 27 April 2026 (UTC)

Oh perfect, I didn't know about that category. Thanks! -- LWG ^{talk (VOPOV)} 20:48, 27 April 2026 (UTC)

There's also this search. Gnomingstuff (talk) 22:52, 27 April 2026 (UTC)

wow I didn't know about either of these thanks! Dr vulpes (Talk) 00:39, 28 April 2026 (UTC)

On a related note [1], is it possible to have an automated log of editors that have been warned for AI use (both using {{uw-ai}} and manually at AINB) with an edit counter for when their most recent edit was for people to monitor? Accounts could then be manually removed from it when they’re deemed not to be using LLMs anymore or are blocked Kowal2701 (talk, contribs) 00:56, 28 April 2026 (UTC)

I bet I could vibe-code up something to that effect with Claude ;). My ideal solution would automatically populate whenever a user is warned via a template, and would have space for a responding editor to assess likely date of first AI use, outcome of interaction with the editor, and date range of edits still needing to be checked. -- LWG ^{talk (VOPOV)} 14:46, 28 April 2026 (UTC)

That'd be really great, Dr vulpes said something similar re automating the creation of clean-up subpages Kowal2701 (talk, contribs) 18:14, 28 April 2026 (UTC)

just be careful because people will yell at you if you deign to add the dreaded, horrible, awful death sentence that is a cleanup tag Gnomingstuff (talk) 04:16, 29 April 2026 (UTC)

I think that's a fantastic proposition. JTtheOG (talk) 00:15, 28 May 2026 (UTC)

Unregistered accounts making LLM based edits

I have a suspicion that User:~2026-24855-28 is making LLM based edits based on their edits on List of United States technological universities (see intro and how they changed the table) and a now deleted post on r/wikipedia reddit bragging about using ChatGPT to make edits. Admittedly, I have no hard evidence. - Wil540 art (talk) 07:39, 5 May 2026 (UTC)

Checked this edit, it does have close paraphrasing. It also has spinning up apparent OR based on 1,000+ page patent documents. This may be a case of handwritten draft rejected, then run it through an llm to resubmit (looks like they made five such expansions within two minutes). CMD (talk) 08:00, 5 May 2026 (UTC)

@Chipmunkdavis Good points. This seems like a motivated editor. What’s the best way to proceed? - Wil540 art (talk) 15:10, 5 May 2026 (UTC)

Unless llm-text is found from earlier (I had a quick look and didn't see any obvious flags), reverting the edits from 4 May onwards is probably enough. Can drop a note on their talkpage. CMD (talk) 15:49, 5 May 2026 (UTC)

I've gone ahead and done this for the five drafts in question. CMD (talk) 04:08, 8 May 2026 (UTC)

They're back on a new TA. This included an llm expansion to a live article, here. What seems curious is that this expansion actually removes two offline sources. I'm beginning to think that the initial drafts are AI as well, although they're stubby and almost entirely sourced to www.uspto.gov/, which I assume is PD anyway? CMD (talk) 16:02, 8 May 2026 (UTC)

Now they've jumped to TA. They seem to be jumping TAs when an old one gets a warning. This needs admin intervention. CMD (talk) 00:29, 10 May 2026 (UTC)

Looking further, this appears to be the same person as this TA, which was separately noticed for AI use by Gurkubondinn. This is a big cleanup job. CMD (talk) 00:37, 10 May 2026 (UTC)

Ohh, this is the Draft:Age and mathematical productivity editor...

• User talk:Gurkubondinn/Archives/2026 §§ Fixed DOI error at "Age and mathematical productivity" article​ and Oh come on...

I think there were some other talk page conversations (maybe even on my talk page), but I remember this.. --Gurkubondinn 09:47, 10 May 2026 (UTC)

Yikes. This is a mess. How can we get an admin involved here? - Wil540 art (talk) 13:38, 10 May 2026 (UTC)

@Admins willing to patrol AINB: Kowal2701 (talk, contribs) 00:45, 11 May 2026 (UTC)

For now, I have semiprotected List of United States technological universities for 1 month. ~Anachronist (who / me) (talk) 04:38, 11 May 2026 (UTC)

Also I have reverted the recent unexplained changes. ~Anachronist (who / me) (talk) 04:45, 11 May 2026 (UTC)

• ~2026-21863-02 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-21576-71 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-21099-87 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-20189-58 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-18394-74 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-18615-40 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-18662-15 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-14941-00 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-13174-26 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-13092-59 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2025-40641-16 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-90461-0 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-78471-2 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-24855-28 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-20213-20 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-28180-72 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-27853-24 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-28014-93 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-25694-95 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-26971-07 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-26163-21 (talk · contribs · deleted contribs · logs · filter log · block user · block log)
• ~2026-24865-26 (talk · contribs · deleted contribs · logs · filter log · block user · block log)

These accounts are showing "ip: unavailable" when I use my TAIP. What is that supposed to mean? Somepinkdude (talk) 16:50, 17 May 2026 (UTC)

@Somepinkdude: the IP data is only kept for 90 days before it is deleted. --gurkubondinn 21:57, 3 June 2026 (UTC)

Only three of those are showing unavailable for me, likely because they haven't edited more recently than 90 days ago. ~Anachronist (who / me) (talk) 20:52, 17 May 2026 (UTC)

@Anachronist, @Somepinkdude, @Gurkubondinn, @Kowal2701, @Chipmunkdavis

Thanks for taking a look at these. It appears many of the LLM based edits are still live, for example: Youth March for Integrated Schools (1958). Should I bring these to the Wikipedia:AI noticeboard? I don't see much guidance on how to report LLM use by Temporary accounts. - Wil540 art (talk) 20:39, 3 June 2026 (UTC)

Revert and move on. Or salvage anything useful and move on. I reverted the edits to Youth March for Integrated Schools (1958) because the text appeared to engage in WP:SYNTHESIS of some of the sources. And the changes were oddly written in British English, for a US-centric article.

As for reporting, this page or the AI noticeboard are fine to call for cleanup. Ping administrators if you believe administrative action is required. ~Anachronist (who / me) (talk) 22:19, 3 June 2026 (UTC)

Alternate system of tracking AI-generated articles

Based on the discussion on WP:VPP -- where people are saying that they are going to remove any AI-generated template without a discussion on the talk page, even if that rationale is in the parameter reason=y, which the whole point of is to contain the rationale -- we should probably consider some alternate way to track AI-generated articles besides the template, because it is useless if people can just remove the template at any time, because people apparently hate the very idea of AI cleanup, despise the people who do it, and are dead-set on obstructing them and wasting their time at any turn, in any way possible. I am so, so, so, so, so tired. I really hate that no matter what you do, it's wrong. You get told to do X, and then you're told "haha! all this time you should have done Y! I will now undo your work, you fool, you rube!" Gnomingstuff (talk) 00:57, 8 May 2026 (UTC)

Could we make something automated that lists tags that have been removed, the date, and who by, which people could patrol (ie. glance at)? Kowal2701 (talk, contribs) 09:25, 8 May 2026 (UTC)

An edit filter might be able to do that. It would obviously track all removals, good, bad and irrelevant (e.g. page blanking vandalism). Thryduulf (talk) 10:24, 8 May 2026 (UTC)

Open cleanup cases

Now that WP:LLMPROD is a guideline, should we start reverting the edits in Category:WikiProject AI Cleanup open cases, especially since I don't think we're using cleanup cases anymore? InfernoHues (talk) 01:42, 8 May 2026 (UTC)

We need to start using cleanup cases again and continue to do, see above topic. Gnomingstuff (talk) 03:11, 8 May 2026 (UTC)

Definitely, although at the same time cleanup cases should still be used for more difficult issues (for example, disentangling articles with both AI and human edits). Chaotic Enby (talk · contribs) 04:00, 8 May 2026 (UTC)

If doing so, remember to be careful to revert only LLM-generated text that has not been subsequently significantly modified by humans. If there have been subsequent edits that don't improve the article for reasons unrelated to LLM-use, I recommend you be very explicit with edit summaries and any talk page comments that you are reverting for two separate reasons and not accusing the subsequent editor of using an LLM. If the subsequent edits are substantial improvements, don't just revert them. If it's unclear whether subsequent substantial edits are or are not improvements use the talk page first.

This will reduce the friction and so reduce the pushback and reduce the likelihood of overreach accusations and similar. Thryduulf (talk) 04:09, 8 May 2026 (UTC)

Yep, of course! InfernoHues (talk) 04:25, 8 May 2026 (UTC)

I've done one example from this case, see my reverts to April 1941 Italian offensive in Epirus. InfernoHues (talk) 04:26, 8 May 2026 (UTC)

Found a completely AI generated articles.

Church of the Holy Archangels Michael and Gabriel, Sarajevo.

Tagged by someone else prior, Shows the signs in the guide, ran it through 0gpt just to be sure, showed 100% for every paragraph. Should cleanup be done, or should I just speedy it. Starlet! (Need to talk?) (Library) (Sandbox) 15:43, 8 May 2026 (UTC)

Looks like most of it was added in this edit, and later remixed throughout the article. I've reverted to the pre-AI version, feel free to check if any similar contributions by the same author need cleanup. Chaotic Enby (talk · contribs) 15:59, 8 May 2026 (UTC)

Thanks. Starlet! (Need to talk?) (Library) (Sandbox) 16:28, 8 May 2026 (UTC)

Was this AI?

I hatted this comment a while ago, but another user disagreed. I might have been a bit over-zealous there, and want to check with someone else. I think there are some AI signs present and a very different writing style from the TA's other edits. [link] InfernoHues (talk) 23:44, 8 May 2026 (UTC)

Besides the rule of three, there's the odd switch between curly and straight apostrophes, as well as the striking difference in grammar and style between it and the next comment by the TA. I'd point them to WP:LLM?, as non-answers such as That was the most eloquent way of saying what needed to be said are less than ideal here. Chaotic Enby (talk · contribs) 23:49, 8 May 2026 (UTC)

Yeah this is probably AI, “focused on (promotional crap)”, “rooted in (promotional crap)”, X rather than Y, etc Gnomingstuff (talk) 23:54, 8 May 2026 (UTC)

AI edit summary log

Hi, I made a bot which goes through live edit summaries and then sends them to an LLM to try and figure out whether the edit summary was generated by an LLM in order to facilitate early detection of the most obvious of AI users. It's at User:Fermiboson/AIlog and the bot account User:FermibosonAIlogbot updates it every so often (whenever my python script hasn't crashed).

The LLM is really bad at its job, so it does have a lot of false positives. However, compared to directly patrolling RCP, the proportion of actually AI edits is already much increased so I believe even in its current form it can be useful to those interested in AI patrol. The main issue is the volume of edits it flags - I cannot review all the edits alone so for those of you who enjoy reading hundreds of kilobytes of text with at best tangential connections to your interests, please help out with reviewing the log. Fermiboson (talk) 15:18, 9 May 2026 (UTC)

This is great stuff. Is your script on github? If not do you mind emailing to me? I may have a few ideas about how to improve classification performance. NicheSports (talk) 15:55, 9 May 2026 (UTC)

The prompt part of the script is at User:Fermiboson/AIlog/source-code, though I'll update that after I get back from something tonight. Fermiboson (talk) 17:23, 9 May 2026 (UTC)

Updated @NicheSports - the prompt has been moved to a file which I'll keep off wiki, so I'll email it to you if you still have interest.

For everyone else - I think I've improved the false positive rate. It now makes three API calls to the LLM which in my testing set reduces the false positive rate to 1.1% +- 0.5% (95% CI). I'm fairly sure however that in operation the false positive rate will still be much higher if only for the reason that edit summaries that sound AI don't necessarily imply an AI edit. Fermiboson (talk) 22:48, 27 May 2026 (UTC)

Naive Idea

Anecdotally, it seems like a lot of people using LLMs for UPE are doing so for Google's Knowledge Panel. If there was a way to make the possible AI tag to show up in the panel itself, then once that becomes known among paid editors it might stop some of the LLM usage. I don't really know a way to do that besides just adding text to the first line of the page, but maybe the more techy people here have other ideas? Or know if this is feasible at all? InfernoHues (talk) 00:04, 10 May 2026 (UTC)

Wikipedia:Wikipedia is written by humans, for humans listed at Requested moves

A requested move discussion has been initiated for Wikipedia:Wikipedia is written by humans, for humans to be moved to Wikipedia:Wikipedia is written by humans. This page is of interest to this WikiProject and interested members may want to participate in the discussion here. —RMCD bot 16:15, 11 May 2026 (UTC)

To opt out of RM notifications on this page, transclude {{bots|deny=RMCD bot}}, or set up Article alerts for this WikiProject.

Template:AI-generated span

Would it be possible for someone to create {{AI-generated span}}, which could be used like {{AI-generated inline}} but more specific (like {{Citation needed span}})? Thank you, Wracking ^talk! 19:46, 17 May 2026 (UTC)

 Done! Chaotic Enby (talk · contribs) 20:17, 17 May 2026 (UTC)

don't forget to put in the documentation that some people are saying that you have to fill out 3 different locations in order to use it because people want to make AI cleanup as arduous as possible

also, someone please get me a probate lawyer because I am pretty sure that I am not going to finish all these fucking talk page sections within my lifetime. obstruction tactics worthy of the terrible trivium Gnomingstuff (talk) 20:34, 17 May 2026 (UTC)

@Chaotic Enby, thank you so much!

@Gnomingstuff, I'm not sure what all three places are, but I wonder if WP:Twinkle could help here? For example, when you check "COI" in Twinkle, it prompts you to provide Explanation for COI tag (will be posted on this article's talk page): I recently used Twinkle for this, and it definitely saved me time: article diff, talk diff Wracking ^talk! 21:14, 17 May 2026 (UTC)

It would be great if you could add the reason parameter to the tag using Twinkle; currently you can only add an explanation to the edit summary. As for the three different places, they mean the edit summary, the reason parameter in the tag, and the talk page. But I still believe that's absolutely ridiculous and any one of those places is good enough. If someone is actually going to mass revert AI tags because the explanation for them is not in more than one location, they should be blocked for disruptive editing. I₂Overcome ^talk 21:34, 17 May 2026 (UTC)

I will leave a message at Wikipedia talk:Twinkle. Wracking ^talk! 21:47, 17 May 2026 (UTC)

don't worry, there is now a new and fun variation: reverting even though you did put it in three places, by just claiming you didn't! whether it's disruptive or not is kind of irrelevant because people just do it anyway Gnomingstuff (talk) 08:45, 18 May 2026 (UTC)

Hi Gnomingstuff! I genuinely understand your frustration, but it feels like a bit of... burnout maybe? Please don't get this get to your nerves, you've done some amazing work and I really don't want small incidents to discourage you like this. Chaotic Enby (talk · contribs) 14:03, 18 May 2026 (UTC)

it's almost as if being compelled to create over 6,000 repetitive and utterly pointless talk page sections to satisfy someone's whims has an effect on a person, especially when every one of those 6,000 is an invitation for people to yell at you more, and when the tags are just going to get removed anyway because no one trusts that someone could know what the fuck they are talking about Gnomingstuff (talk) 04:45, 19 May 2026 (UTC)

I understand your frustration, but swearing is not prohibited in Wikipedia project spaces. RotatingPirateShip (talk) 09:27, 27 May 2026 (UTC)

@RotatingPirateShip there is no such rule, why make things up? fifteen thousand two hundred twenty four (talk) 09:57, 27 May 2026 (UTC)

I found that rule in Wikipedia:Swearing_is_permissible RotatingPirateShip (talk) 09:58, 27 May 2026 (UTC)

Ah, I misread your comment, struck. fifteen thousand two hundred twenty four (talk) 10:01, 27 May 2026 (UTC)

It's okay, we all make mistakes :) RotatingPirateShip (talk) 10:03, 27 May 2026 (UTC)

Suggestion: we all make fucking mistakes :) --gurkubondinn 10:10, 27 May 2026 (UTC)

Wikipedia:Swearing_is_permissible states that swearing is not prohibited in Wikipedia project spaces. RotatingPirateShip (talk) 13:52, 27 May 2026 (UTC)

AI detection

Hi all, I work as the Wikimedian in Residence at the University of Edinburgh and I notice a page edited by one of our students was flagged as containing AI-generated content, namely Classic Lolita Fashion. As this was part of an extracurricular project we were supporting the student on we just want to understand a bit more about how and why the content was flagged in this way. i.e. if it was just immediately and abundantly obvious to the naked eye or if there were some useful tools and scripts employed to aid you (and whether they have been known to flag false positives are are generally reliable). What we are trying to ascertain is how we can better detect and protect against AI-generated content in student work ourselves using any best practice from reviewing editors like yourself and the community and the Foundation. I also want to check that, as the student is a non-native English speaker from China, whether online tools that would naturally have helped her improve her written English fluency would equally be flagged as AI? The flagging editor has intimated that "a report on GPTZero gave 87% probability of AI generated content" which is a great start to help my understanding but I wanted to widen my question out about the resources/tools/scripts being used by WikiProject AI Cleanup and any thoughts on their general focuses, usefuless and reliability and any learning points we can take from this... both in terms of improving our own workflow of making sure student work is quality assured and making sure that students & staff are aware of the efforts being undertaken in combating AI generated content on Wikipedia and any approaches/difficulties/opportunities we can flag here at the University to improve AI literacy and usage and to better encourage greater academic rigour & referencing in sharing knowledge to Wikipedia. I realise this a big question but any places to start with understanding the tools, approaches and discussions being had to maintain Wikipedia's integrity would be super helpful. Any thoughts or advice do let me know. Stinglehammer (talk) 15:40, 19 May 2026 (UTC)

The best place to start would be Wikipedia:Signs of AI writing, and in particular the studies and/or preprints it cites. There are some caveats here (some of what's on that page is still anecdotal or based on informal data collection; some of it is more relevant to older LLMs; some of it is specific to "Wikipedia style articles") but a good amount of it has some corroboration in research.

I wasn't the person who flagged this article -- it seems like it has gone through AI editing -- but I can walk you through my process, with the original version.

• The first thing I do is look for common AI vocabulary. In AI writing, this will often co-occur, and will co-occur in a way consistent with when it was produced (i.e. the 2023 stuff will appear together with other 2023 stuff, the 2025 stuff will go with 2025 stuff, etc). This one is more subtle (consistent with it being possibly AI rewritten) but we have the style minimises the emphasis on the body’s natural shape and instead highlights a structured and balanced silhouette (which is also a variation on negative parallelism), emphasise classical colour palettes, etc. We also have a table that is essentially advertising various brands (known for designs that reflect the elegance and aesthetic ideals, known for refined tailoring and elegant silhouettes, etc), a common issue with AI writing where such subtle promotional tone sneaks in even when someone isn't trying to create an ad. Undue emphasis on contributing to broader trends -- played a role in shaping (extremely common AI phrasing, especially if it is a "key role" or "pivotal role" or "crucial role"), contributed to the development, etc.
• Next, if I'm doing a spot-check for verifiability, those are what I start with. Taking the first example ("minimizes the emphasis on blah blah blah"), in this case the article is real and accessible on academia.edu. The text -- as is frequent with AI-generated content -- does not seem to be fully backed up by the source; I believe this is an attempt to "summarize" the sentence "Most Lolita styles hide the body shape underneath layers of slips, petticoats, and panniers," which is not really the same thing (no mention of balance or structure) and also turns it into AI slop phrasing ("hides" becomes "minimizes the emphasis," mere existence becomes "highlighting", etc). Spot-checking some more stuff, we have some enthusiasts regard Madame de Pompadour as an important symbolic figure, but the source this is cited to only mentions one person drawing any connection to Madame de Pompadour (AI writing often turns one person into multiple in this way), and that person is novelist Novala Takemoto. This stuff keeps happening; contemporary Lolita styles—particularly Classic Lolita—tend to adopt more modest designs is also cited to the Younker paper, but the Younker paper does not connect "Classic Lolita" specifically to that. Another: classic Lolita usually employs softer, lower-saturation tones such as pink, beige, and light purple to create a retro, elegant overall style -- the source (which is dubious anyway as it's a store blog) doesn't mention pink at all and only mentions purple in saying that Gothic Lolita uses dark purple. The last few words, meanwhile, are more promotional opinion. Labels frequently associated with the style include Victorian Maiden, Mary Magdalene, Excentrique, and Innocent World - the book does not mention Excentrique at all and only mentions the other three brands as having "a Lolita touch" (i.e., not associated with Classic Lolita specifically). Unlike Gothic Lolita [...] Classic Lolita developed more gradually through the influence of multiple brands -- again, the source does not verify this, the only thing it seems to say about classic lolita fashion specifically is that it occurs "in solid shades, with ribbons and trims." This is the stuff Wiki Education Foundation was talking about when they said that almost all of the statements in AI-generated text failed verification.

And sure, obviously people misinterpret sources all the time, but they usually don't misinterpret them by claiming they mention proper names that they just don't; if you mention a brand, and cite it to a book that doesn't name the brand anywhere, that suggests that you didn't actually read the book.

As far as detectors, I don't personally use them except if I'm really on the fence, but most stuff I've read suggests Pangram is the gold standard right now (~99% accuracy) GPTZero isn't bad either, and when they get it wrong it's usually in the direction of false negatives. There is research about flagging non-native English speakers more often, but as far as I know much of this research involves older LLMs (which, to be fair, most of the research does, very little has caught up to GPT-5), and there are a lot of confounding variables (their writing being flagged by people who are not skilled in AI detection specifically; people who don't speak a language well potentially being more likely to use AI tools for communication in that language; machine translation tools like Google Translate now using generative AI and not always being transparent to the user about it). This recent preprint suggests that when detectors flagged writers' recent work as AI, they almost never flagged those same writers' older work as AI, although the sample size is 10 and since they're "veteran reporters" in English-language publications they are probably fluent speakers. Gnomingstuff (talk) 16:49, 19 May 2026 (UTC)

Possible LLM use found

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

On the article Wonderoos, I saw that the writing style is very reminiscent of an LLM, especially in the Premise section. Also there are mistakes, like calling the in-universe pet species (Poofy Schmoop) "Schmoopsy" I've added Template:AI-generated to the article, but is it okay if you could review it? RotatingPirateShip (talk) 09:20, 27 May 2026 (UTC)

Such a post would be best made at WP:AINB where there are more eyes. Fermiboson (talk) 22:49, 27 May 2026 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Updating G15

With the recent amendment to WP:NEWLLM to prohibit the use of LLMs to generate or rewrite article content, I think its time to update G15 to encapsulate AI-generated articles generally, not simply unreviewed ones.

Some clear cut signs I'm thinking of: presence of turn0search0, attributableIndex, oaicite, or having a majority of citations include utm_source=chatgpt.com.

It might also be worth emphasizing that WP:AfD or getting consensus for WP:LLMPROD are better for less clear-cut cases. Ca ^{talk to me!} 00:44, 29 May 2026 (UTC)

Yep, all the unambiguous cases (like turn0search0 or oaicite) should definitely go there at the very least. I was thinking of also revisiting the Markdown criterion, but it doesn't seem to be a thing with newer models anymore, so I'm not sure how relevant it would be. Chaotic Enby (in solidarity · talk · contribs) 00:49, 29 May 2026 (UTC)

Markdown is something that I still routinely see, mostly as **markdown bolding**. But there's one specific Markdown element that I want to see added as a G15 criteria, when an edit is surrounded in a markdown code block:

```wiki

```

There's usually perfectly valid Wikitext between ```wiki and ```, and I'm pretty sure that this happens when someone asks a chatbot to generate a page in wikitext markup and the bot delivers it in a formatted code block for the user.

The leading ```wiki isn't always present (it's more obvious and so users are more likely to see and remove it), so the trailing ``` would need to be sufficient on it's own for G15. --gurkubondinn 10:21, 29 May 2026 (UTC)

Huh, didn't expect it to still be so common, but that's definitely a good catch and should be added! Chaotic Enby (in solidarity · talk · contribs) 10:27, 29 May 2026 (UTC)

I don't really know how common it is, because the edit filter 1369 (hist · log) doesn't match for triple backticks (and I think it would be better to have a separate filter for these code block markers). This is one of the reasons that I've been considering requesting permissions for edit filters, but the granting criteria seems hard to meet. Finding these is almost always a surefire way of spotting an AI-wielding user though, yesterday I ended up draftifing c. ~25 AI-generated articles after I spotted an edit with a markdown codeblock. --gurkubondinn 10:37, 29 May 2026 (UTC)

By coincidence I spoke to @Daniel Quinlan about the triple backticks just a couple of days ago and he added it to 1369, so hopefully that should catch these going forward. There are a very few existing cases where it's used in legitimate wikitext (usually to mark up an ASCII code sample) but overwhelmingly it seems to appear in definitely or at least plausibly AI edits.

I think most of the examples I found had only the start or end backticks, not both. Interestingly, some were relatively small edits including the ``` - so presumably some people are using it to generate individual snippets (eg single paragraphs or references) and not just entire pages. Andrew Gray (talk) 12:50, 29 May 2026 (UTC)

The utm_source could have come from someone using ChatGPT to find sources, and not to write content, so I don't think that should be in G15. Adding the rest of the things you mentioned makes a lot of sense. InfernoHues (talk) 01:41, 29 May 2026 (UTC)

Forgot to mention -- G15 really needs a disclaimer about the need to check Wayback Machine to ensure citations are genuinely nonexistent. Ca ^{talk to me!} 05:37, 29 May 2026 (UTC)

That depends on when the article was created/when the citation was added. If someone adds a link today to somewhere that existed five years ago, that is still a nonsensical citation (probably hallucinated), and they did not read the source themselves. Because if you are reading a now-dead source, then you are reading it on Wayback Machine (or on any other internet archive), so you would copy that link. You wouldn't strip out the https://web.archive.org/... part first. --gurkubondinn 10:10, 29 May 2026 (UTC)

Not necessarily, e.g. you could be copying the reference from somewhere else (on or off Wikipedia), you could have viewed it a few days ago when it was working, etc. It's not something reliable enough for speedy deletion. Thryduulf (talk) 10:26, 29 May 2026 (UTC)

That's why I said "five years" in my reply, because I meant in a timeframe that is unreasonable between reading a source and then using it in an edit. Fair point on copying from elsewhere, though. But I think that this would be reliable enough for speedy deletion if the criteria would require multiple (maybe 3+) instances of this? But just on its own, I agree with you that it doesn't work as a speedy deletion criteria (thought it is an AISIGN in and of itself, and it most likely also means that the user has not read the source that they are citing). --gurkubondinn 10:41, 29 May 2026 (UTC)

I never seen AI cite the Wayback Machine before. Ca ^{talk to me!} 11:16, 29 May 2026 (UTC)

It seems to have "learned" about it somewhat recently. But afaik, The Internet Archive are not super happy about being scraped by these companies (and neither should they, their scrapers are ridiculously aggressive and badly implemented). Maybe some of them are using the API to search for archived snapshots now? --gurkubondinn 11:24, 29 May 2026 (UTC)

Never mind, I just sent ChatGPT the prompt search up in Wayback Machine for 2008 version of wikipedia's page on Cats and it dutifully gave this Wayback Machine link. So, yeah, I agree it seems to know how to cite WM.

However, I do suspect its hallucinated (instead of making use of RAG/actually searching in WM) as the WM does not appear in the "sources" list and the timestamp on the url doesn't seem to match the one on the actual snapshot. Ca ^{talk to me!} 11:40, 29 May 2026 (UTC)

Yep, the Wayback Machine will redirect you to the "nearest" snapshot if you give it a non-existing timestamp.

$ https --headers "https://web.archive.org/web/20080705195056/https://en.wikipedia.org/wiki/Cat" | grep -E "^(HTTP|location)"
HTTP/1.1 302 FOUND
location: https://web.archive.org/web/20080625213031/http://en.wikipedia.org/wiki/Cat

Examining the timestamps

$ AI_TIMESTAMP="20080705195056"
$ https --headers "https://web.archive.org/web/${AI_TIMESTAMP}/https://en.wikipedia.org/wiki/Cat" \
    | grep "^location" 
    | awk '{ print $2 }' 
    | awk -F'/' '{ print $5 }'
20080625213031

$ WAYBACK_TIMESTAMP=$(https --headers "https://web.archive.org/web/${AI_TIMESTAMP}/https://en.wikipedia.org/wiki/Cat" 2>/dev/null \
    | grep "^location" \
    | awk '{ print $2 }' \
    | awk -F'/' '{ print $5 }')
    
$ if [[ "${AI_TIMESTAMP}" == "${WAYBACK_TIMESTAMP}" ]]; then echo "Same timestamp"; else echo "Different timestamps"; fi
Different timestamps

I've noticed this with AI-generated references to Wayback Machine (haven't seen AI-generated references to any other internet archives yet) for a while now. --gurkubondinn 12:03, 29 May 2026 (UTC)

that is still a nonsensical citation

Just to clarify my own words here, I didn't mean this as "meets the 'nonsensical references' criteria for G15". I just meant it in the sense "that still doesn't make sense and therefore it is nonsensical". Didn't make the connection to how it was basically the name of the G15 criteria, probably could have phrased this better. --gurkubondinn 21:13, 29 May 2026 (UTC)

Anything listed under WP:OAICITE, WP:AISIGNS § turn0search0 and WP:AISIGNS § attribution and attributableIndex are valid G15 criteria already because they are all "nonsensical references" that no human would ever write down (especially the byte sequence), but it would not hurt to explicitly list them as examples. You sometimes get people incorrectly removing {{db-g15}} over this, even for the very common OAICITE. I've had this happen a handful of times for all of them, but I've requested CSD for hundreds of pages under all of these criterias.

For the UTM parameters, those only "count" when every single reference has utm_source=chatgpt.com (or the second most common example that I see, copilot.com). But these should still be tagged with {{AI-retrieved source}}, though people are very prone to just removing them. If it is just a handful of them it doesn't even really reliably mean that the user is even using AI as a search engine anymore, because these links have spread over the internet by now (in no large part due to how commonly they appear on Wikipedia). You can even find them in Google search results now.

As a sidenote, the m:Cite Unseen userscript is very useful, and it marks references with the UTM tags with a little orange robot symbol. That's also why they shouldn't be removed from citations. --gurkubondinn 10:04, 29 May 2026 (UTC)

Oh my god, how have I not seen Cite Unseen before. That's awesome, thanks for sharing. EatingCarBatteries _{(contribs | talk)} 01:46, 30 May 2026 (UTC)

I'd support adding the more "smoking gun" AI signs. I don't fully agree with Gurkubondinn that these are already covered by "nonsensical references", as in many cases the links work and back up the text they're cited for, which is far from the original intent of that phrase. Regarding old sources or bizarre access dates, while they can be an indicator, I'm not sure that they're enough for G15. At a minimum, we'd have to require checking whether the article is a translations from other Wikipedias. Toadspike [Talk] 20:49, 29 May 2026 (UTC)

Oh, I meant that stuff like OAICITE, grok_card, lenticular brackets + dagger sign, turn0search0, and the 0xee 0xa8 0x81 byte-sequence is already covered by "nonsensical references". I agree with the dates etc (I think we should do something about that but I don't know what). Sorry that wasn't clear, --gurkubondinn 21:03, 29 May 2026 (UTC)

While I agree G15 should be expanded, as it is very narrow, adding utm_source to the criteria isn't a good idea. If someone types in "find me reliable, secondary sources for X topic" and ChatGPT spits out valid references, but they have the tracking tag, that is totally fine. It doesn't mean that the chatbot wrote the article. EatingCarBatteries _{(contribs | talk)} 01:43, 30 May 2026 (UTC)

References

Uw-ai1 wording

The warning template {{uw-ai1}} assumes good faith of the editor, but is still written on the assumption that the templater has strong enough concerns about the addition to raise the subject, and that the text could be so bad that someone may have already removed it - it says that the user's text seemed to be generated using a large language model, that they should instead use their own words, and that their contribution may have been reverted.

In ambiguous cases where I'm not sure if AI was used, I've generally avoided using this template and (if I have time) handwritten a talk page message instead, to avoid the risk of the user being insulted by template's phrasing, and (if they were using AI) potentially going into one of several unhelpful defensive spirals about it.

Would Wikipedia benefit from having a milder "zero" level warning (or {{welcome-ai}}), that frames the issue as a non-judgmental question, or as a "did you know" statement about Wikipedia's AI policy? Or could ai1 be given a gentler wording so that a user receiving it for a minor, ambiguous edit won't be on the back foot? Belbury (talk) 11:56, 29 May 2026 (UTC)

There is a {{welcome-llm}} message (we could redirect Template:welcome-ai there). I often leave it for newer editors, sometimes along with a {{uw-ai1}} notice if that feels warranted as well. --gurkubondinn 12:22, 29 May 2026 (UTC)

 Done. Chaotic Enby (in solidarity · talk · contribs) 12:32, 29 May 2026 (UTC)

i've created upper-cased Template:Welcome-AITemplate:Welcome-AI as well. --gurkubondinn 12:40, 29 May 2026 (UTC)

We could have a template, which only includes the first paragraph of Template:Welcome-LLM, so as to have an introductory message against generative ai which isn't also a welcome message. — EarthDude (Talk) 12:35, 29 May 2026 (UTC)

LLM use across multiple Wikipedias

Hi, after noticing Gold market in India, (now reported and deleted, see discussion) the user also created the same article (also LLM-generated) in several other language Wikipedias. Now, the AI notice template was available in some of the Wikipedias, which I added, and was quickly noticed by other editors and deleted (Spanish, German). I also managed to add it to the Arabic, French, and Hindi Wikipedias (however so far it seems that they haven't been noticed). The other to Wikipedias (mr.wiki, ta.wiki) have no AI notice template, so I couldn't do anything.

Now, my concern is that cases like these were a user generates the articles across multiple different Wikipedias will be a problem. Of course, WP:EMBASSY was slightly helpful with contacting an Italian editor (no response so far), however, the embassy doesn't have editors for every Wikipedia. And, every Wikipedia has there own policies (and may or may not have a policy for LLM articles).

So, any ideas of what we can do to make it easier to report these articles for the foreign language Wikipedias? I am actually willing to do everything I can to take care of situations like these, so it may be helpful listing my name somewhere for people to ping me in these cases (please let me know if I can and where). Also, are there any useful tools for such cross-wiki incidents, such as deletion? Since I was actually able to install Twinkle on the Spanish Wikipedia, and nominate a speedy deletion myself. Fortek67 (talk) 17:04, 30 May 2026 (UTC)

A German editor just recommended me XReport – this tool should be very useful however I'm not exactly sure if I would report or nominate the article for deletion. I will assume that the wikis that have an AI notice template do prohibit LLM, so I will nominate them. Fortek67 (talk) 17:11, 30 May 2026 (UTC)

meta:Artificial intelligence/Policies by project could be of help! Not too sure about the tool situation, however. Chaotic Enby (in solidarity · talk · contribs) 17:17, 30 May 2026 (UTC)

What do you think of XReport? Fortek67 (talk) 17:21, 30 May 2026 (UTC)

Looks good! Chaotic Enby (in solidarity · talk · contribs) 17:25, 30 May 2026 (UTC)

A bit unrelated but there seems to be no userbox for participants of the WikiProject. Could I possibly make one and promote it on the project page, potentially under the participants section? Fortek67 (talk) 17:30, 30 May 2026 (UTC)

It's at {{User WP AI Cleanup}}, but feel free to add it there for visibility! Chaotic Enby (in solidarity · talk · contribs) 17:31, 30 May 2026 (UTC)

Didn't see that, thank you for showing me! Added the template. Fortek67 (talk) 17:37, 30 May 2026 (UTC)

Wikipedia:Sharing AI chatbot sessions

I created this page to help guide editors share their AI chat sessions.

I think more people should be asking those accused of improper AI use to share their chat sessions. It's an easy way for the AI-accused to demonstrate their compliance with WP:NOLLM and for reviewers to check if their characterization of their AI use isn't misleading.

Of course, it's possible to just fake a compliant session on the fly, but it should at least be useful for good-faith editors. Ca ^{talk to me!} 01:21, 1 June 2026 (UTC)

I don't know what good this would do. I see a lot of AI-generated text posted in good faith, that doesn't come from a chatbot, it comes from proofreading tools like Grammarly, which pollute your prose with many of the WP:AISIGNS. These people swear that they aren't using an AI chatbot, and they're telling the truth as far as they're concerned. For those that are actually using an AI chatbot, it seems to me that at least half of them lie about it, as if that would accomplish anything. ~Anachronist (who / me) (talk) 22:26, 3 June 2026 (UTC)