Wikipedia:Bots/Requests for approval/AutoPatrollerTrials

AutoPatrollerTrials

Operator: CocaPopsRather (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 12:19, Saturday, June 6, 2026 (UTC)

Automatic, Supervised, or Manual: automatic

Programming language(s):

Source code available:

Function overview: Monitors changes for likely vandalism/spam, using algorithm first and GenAI review as a fallback. It reverts only obvious vandalism or spam, can post standard user warnings, and avoids content disputes, BLP/source nuance, edit wars, and other cases requiring human review

Links to relevant discussions (where appropriate):

Edit period(s): Continuous

Estimated number of pages affected: unlimited/all

Exclusion compliant (Yes/No): Yes

Already has a bot flag (Yes/No): No

Function details: -reliable vandlism detection, -for cases when a normal algorithm would not revert vandlism, a gen-ai model is used to compare the diffs and decide the action.

Discussion

Note: The user account this request is for is also listed as the Operator, but the account name does not clearly indicate that the account is a bot and the account has very few edits. Please note that WP:Bot policy states that a bot account's username should make it immediately clear that the account is in fact a bot, which is normally done by having the account name end with the word "Bot". Also note that a bot may not operate itself, so the Operator field should identify the account of the human running the bot. AnomieBOT ⚡ 12:23, 6 June 2026 (UTC)[reply]
I did make an account with the word BOT In it, however it was soft-blocked, and I was advised to change the name. In-turn resulting in "AutoPatrollerTrials", instead of, "AutoPatrollerBot". I will change the operator field now. AutoPatrollerTrials (talk) 12:26, 6 June 2026 (UTC)[reply]

I would like to provide a example test of when the bot would not normally revert for obvious vandlism, but promote the request to Gen-AI to decide whether or not it should be reverted.

The bot will also use a confidence scale, if the confidence of the AI is not above 97% that an edit should be reverted it is ignored to prevent malfunction. CocaPopsRather 12:39, 6 June 2026 (UTC)[reply]

Is there a reason why User:ClueBot NG is not sufficient for this task? Why do we need a second anti-vandal bot? Primefac (talk) 13:22, 6 June 2026 (UTC)[reply]

ClueBot NG is excellent for high-confidence, obvious vandalism, and this bot is not intended to duplicate or replace it. ClueBot uses ClueBot Core, which only reverts when its model reaches a very high threshold (e.g really obvious vandlism). This bot is aimed at the edits that fall below that threshold or require more contextual judgement. It uses additional context such as the diff, page history, user history, warning history and policy-specific reasoning to decide whether an edit should be reverted.

The intent is not to create a second ClueBot, but to handle a different class of edits: cases that are suspicious or damaging but not obvious enough for to be detected by cluebot core. CocaPopsRather 13:37, 6 June 2026 (UTC)[reply]

A few items of note:

The bot account should be used only for bot edits. You appear to be using the account for manual edits as well, such as filing this BRFA.
Considering that this is stated to use generative AI, has there been any discussion, such as at a Village pump or a talk page dedicated to anti-vandalism work, that indicates the community at large is willing to trust generative AI with this task?

Anomie ⚔ 13:34, 6 June 2026 (UTC)[reply]

On the account-use point: I understand that the bot account should be used only for bot edits. Filing or discussing the BRFA from the bot account was a mistake, therefore I have moved the discussion to my main account.

On the gen AI point: I agree that community confidence is important,especially when regarding gen ai. My thinking was that a tightly limited trial would provide concrete evidence about whether the system is accurate enough, rather than asking the community to evaluate it only as theoretical.

That said, if the committee feels wider discussion should happen before any trial, I am happy to start that discussion at Village pump or alternative.

The intended trial would be conservative: human-review first, logging decisions, and demonstrating performance before any fully autonomous use is in place. CocaPopsRather 13:41, 6 June 2026 (UTC)[reply]

Non-BAG member here, but I am a bot operator. I do have a few questions:

Which AI model will you be using to analyze the edits? Additionally, are you willing to provide the entire prompt text for transparency's sake?
Who is paying for the tokens consumed by this bot? English Wikipedia generates roughly 160,000 edits per day. Even if your bot is only sending about 3% of the edits to the AI, that's still nearly 5,000 edits per day being reviewed by an LLM. That would consume many tokens, and rack up a sizable bill. If the tokens are subsidized, what is the motivation of the organization providing such subsidy?
In your own words, you used AI to write the code for WP:VandalHandle. Did you use AI to write the code for this bot? If so, has said code been thoroughly reviewed by a human?
Where is the community consensus asking for such a bot? WP:BOTREQUIRE point 4 specifically requires that there be consensus established for a task.

Thanks! phuzion (talk) 16:45, 6 June 2026 (UTC)[reply]

Thanks for the questions, Ill answer.

On the AI model: the has been designed so that the model provider is configurable rather than hard-coded. It can use API based models from providers: OpenAI, Google, or Anthropic, and it can also use local Ollama models. For any approved trial, I am willing to specify exactly which model is being used and provide the full prompt text for transparency.

On token costs: any API costs would be paid by me. There is no external organisation subsidising the bot, and therefore no outside motivation or influence. I also do not intend to send every edit to an LLM. The intended design is that normal algorithmic checks filter edits first, and only a much smaller number of suspicious cases would be escalated for model review.

On AI assistance in the code: the diff you linked refers to the WP:VandalHandle documentation rather than the VandalHandle code itself. However, to answer the wider question clearly: No, AI has not been used to "write the bot", however, AI assistance has of-course been used for identifying logic errors, suggesting fixes, and improving code. To clarify: No, the program has not simply been written by AI and deployed without human understanding. The code has been human reviewed by me, and the output/functionality has been tested.

On consensus: I understand that WP:BOTREQUIRE point 4 requires consensus for the task. My initial view was that anti-vandalism bot work is already a well-established task area on enwiki, but I accept that this proposal adds a new element because it involves GenAI-assisted review. Because of that, I agree that wider community confidence is important before any autonomous reverting or warning is considered.

In light of Anomie’s suggestion, I think the best next step is to run a logging-only test first, where the bot records what it would have reverted or warned for, without making edits. Those logs can then be inspected by BAG and the community, and used as evidence in a later discussion about whether this specific approach has consensus. CocaPopsRather 18:14, 6 June 2026 (UTC)[reply]

Note that, when it comes time for the community discussion, you'd probably do better to hold it at WP:Village pump (proposals) or the like than trying to do it here. Anomie ⚔ 23:28, 6 June 2026 (UTC)[reply]

A logging-only trial can be conducted easily enough without approval by having the bot list actions it would take somewhere where the log can be inspected, for example a page in the bot's userspace or a page on an external site. Note, if writing to the bot's userspace, you'd likely want to batch updates rather than updating with each potential action. That log should be sufficient to hold a community discussion. Anomie ⚔ 17:46, 6 June 2026 (UTC)[reply]

Thank you for that suggestion, I'll conduct that. CocaPopsRather 17:59, 6 June 2026 (UTC)[reply]

On hold. Feel free to disable this tag when the necessary information has been obtained. Primefac (talk) 10:59, 7 June 2026 (UTC)[reply]

Although at that point they may want to go back to using User:AutoPatrollerBot, and rename this page accordingly. Anomie ⚔ 15:43, 7 June 2026 (UTC)[reply]