Wikipedia talk:Creating a bot

From Wikipedia, the free encyclopedia

This is not the place to request bots, or to request permission to run a bot. Please use:

Wikipedia:Bots/Requests for approvals to ask for permission to run a bot;
Wikipedia:Bot requests to request that someone write a bot to do something.

This is the talk page for discussing improvements to the Wikipedia:Creating a bot page.
Please sign and date your posts by typing four tildes (`~~~~`). Put new text under old text. Click here to start a new topic. New to Wikipedia? Welcome! Ask questions, get answers.	Be polite Assume good faith No personal attacks Be welcoming

[edit] Things to expand/add

We should add more info on:

bot speed and timers
more specifics on getting the bot approved
pages for a bot to be disabled (ie if User:Bot/disableme contains "end" then stop editing)
checking for things such as edit conflicts, being blocked, non-existant pages
Different modules other than bot frameworks, like http clients, xml parsers, etc...

GeorgeMoney (talk) 03:11, 10 December 2006 (UTC)

The article is still very very new so there is loads of stuff like the above yet to add. i am going to be working to expand the article further today - PocklingtonDan 09:35, 10 December 2006 (UTC)

[edit] stuff

I know this idea might be a litte over-the-top and pointless, so feel free to say how stupid doing this would be, etc... but does anyone think we should add syntax highlighting to the example code using http://tools.devshed.com/webmaster-tools/syntax-highlighting/ ? GeorgeMoney (talk) 21:56, 10 December 2006 (UTC)

No, I think that is an excellent idea. I think perhaps we should try and standardise the code at some point in the future so that each code chunk produces exactly the same output, to give a good example of how each language achieves the same output. We don't have to do this right now though, there's other areas that need expanding first. - PocklingtonDan 08:17, 11 December 2006 (UTC)

[edit] Process

I think it goes 1. Do something 2. Do it several times 3. Get fed up 4. Think about automating it 5. Realise you can 6. Try - fail - try 7. Suceed! (with lots of manual testing) 8. Ask permission 9. Get permission to test 10. Test 11. Get permission to run 12. Run 13. Laurels and brickbats.

Rich Farmbrough, 21:43 12 December 2006 (GMT).

[edit] Requesting the building of a bot

Is it possible for someone to build me a wiki bot? I just need it for editing articles in another wiki. Is that at all possible?--H*bad 07:16, 1 January 2007 (UTC)

See Wikipedia:Bot_requests -- SatyrTN (talk | contribs) 16:02, 31 January 2007 (UTC)

[edit] PHP Bot - RfC

I think this is (now) the place to put this, but if not, let me know.

I've put together a PHP bot for WikiProjects. It reviews all articles that are tagged with a project's tag, determines which ones have cleanup tags, and puts them all on a "full list of things to-do". It also pulls a random subset of that list suitable for using in a project's "to-do" list. I created it for WP:LGBT, but I've set it up to be used by other WikiProjects. You can see the current results at User:SatyrBot/Maintenance results full and User:SatyrBot/Maintenance results short. And those results have been copied manually to the LGBT project's page, if you want to see how they'll be used.

What I'm looking for is some input as to how it works and if it needs anything. Right now it has to be manually fired, but I'd like to set a cron job so it will fire weekly for any projects that want it - but I don't want to do that until I've gotten some input.

Thanks in advance, -- SatyrTN (talk | contribs) 16:14, 31 January 2007 (UTC)

Doh! I guess I should mention that, since it's currently manually fired, it's accessible through a web page: [1] That page is password protected, so email me for the login info. -- SatyrTN (talk | contribs) 16:32, 31 January 2007 (UTC)

I'm not sure if I understood you correctly: Do you want to know how cronjobs work and where you can get them? I'm using free cronjobs from cronjob.de. Setting them up is very easy. Your cronjob provider will explain everything. You might need to include ignore_user_abort(TRUE); because the cron job provider might not want to wait until your script finished and set_time_limit(???) to stop your server from aborting it early. I hope that answered your question. If not, ask again. You can use my talk page, too. By the way, don't forget to request approval for your bot / major changes! — Ocolon 18:20, 22 March 2007 (UTC)

[edit] Need some useful content

They weren't kidding with the "needs an overhaul" tag. This article needs some serious work.

To start, the title of the article is "creating a bot". One of the very first things mentioned is that this article is geared towards experienced programmers. However, the majority of the article talks about programming techniques; testing, testing again, development cycles, etc. While there is a place for that, there's way to much of it. On the other hand, there are no examples of code to show what a bot may look like. It describes what an edit token is, but not how to use one, or even how to make the changes (I assume through some form of regular expressions, but its not discussed at all.)

IMHO, without having any experience progamming bots, here's what I would expect this article to look like:

intro... what a bot is, some examples of how to use one
a short discussion on the administrative process required to make a bot. Note that many folks are - probably the majority of folks coming to this page - are not making wikipedia bots, but rather bots for their own wikis.
a programming language-nonspecific discussion of how a bot should look. Use lots of pseudocode here.
some short examples of bots in different languages, with links to existing bots in those languages.
links to useful stuff... development cycle discussions, "intro to programming" links, FAQ link, etc.

That would be what I would expect. I would love to see some of that implemented. At the very least, the pseudocode section is crucial. I hope this is helpful in rewriting this article.

I would love to see this article be useful, but I know nothing about programming bots. However, if I can be helpful in any other ways, please let me know on my talk page. Thanks! -- eykanal talk 14:43, 14 February 2007 (UTC)

ok... nnnow what did you say ???

From what i saw it looks like you have to be a technology expert or have a masters degree from Harvard or be a 3 yrs old!:P to figure out how to make one of these things! --Takaomi I. Shimoi 16:03, 20 March 2007 (UTC)

Hi eykanal. I'm afraid you have been mislead. This is not an ecyclopedic Wikipedia article! This is a Wikipedia internal project page. Your ideas are not bad. You can write an article on the bot creating process under Creating a bot. But not here. This page might be helpful: Bot. There is a great diversity of bots, as is among computer programs in general. There is no typical look of a bot, as far as I know. — Ocolon 16:39, 20 March 2007 (UTC)

[edit] Software?

Do you need specal software to make a bot?! please no!--Takaomi I. Shimoi 16:14, 22 March 2007 (UTC)

Nope, no "special" software. Programming languages, interpreters, compilers are free and you can get free development environments too — or use Notepad. — Ocolon 18:08, 22 March 2007 (UTC)

[edit] API section

This whole section is misleading and could do with a clean up/removing.

"An API is an Application Programming Interface." - True.
"It describes how a bot should interact with wikipedia." - Not true. While there are guidelines for how a bot should interact with wikipedia (under "guidelines" lower down), they are not related to any API.

There is an API which I /think/ can be used to interact with wikipedia, which you can find here: http://meta.wikimedia.org/wiki/API

It's not helpful to call techniques like "screen scraping" APIs, because they're not. Perhaps this section could be replaced by something like "Ways bots can interact with wikipedia". 58.106.147.155 04:32, 18 April 2007 (UTC)

[edit] Why there is no wsdl?

Could you tell me, why you give web api, instead of just releasing wsdl. http://www.google.com/search?hl=uk&q=define:wsdl Why should we write for each platform/language new wrapper, instead of just using wsdl, which is implemented and tested on each platform? uk:User:Alex_Blokha

[edit] Question

So when I edit, my bot will automatically edit?? Just a question. PNiddy Go! 0 16:53, 3 August 2007 (UTC)

I'm not sure what you mean. If your code includes a login to Wikipedia, all edits will be performed under the bot's account. — Madman bum and angel (talk – desk) 17:49, 3 August 2007 (UTC)

[edit] Bot Exchange?

I'm a technology freak, but not advanced enough to create a bot. Is there any place where users give away bots? If so, press "Sonic" on my signature to leave a note. Thanks, Hyper Sonic Boom 02:22, 9 October 2007 (UTC)

Not really, only because bot operators should be experienced in the bot's language, and expert enough to fix any problems that may arise. — madman bum and angel 02:32, 9 October 2007 (UTC)

[edit] PyWikipedia test script

I'm having trouble with m:Pywikipedia's test.py file - it's not printing any output from it's normal output functions (although manually adding a print statement works fine.) Is anyone familiar with what would cause that and how to fix it?

As far as I know, it's still a stock installation, and I know that Windows Vista wouldn't cause problems for the standard output. --Sigma 7 16:12, 15 November 2007 (UTC)

[edit] wpStarttime not served

Hey everybody,

I'm writing a bot from scratch (yes, I know, reinventing the wheel -- I'm a control freak that way :-)). My poison is PHP. Everything's working great, except the edit page I retrieve in order to get the token and the rest of the variables doesn't contain wpStarttime and wpEdittime. I get this:

<input type='hidden' value="" name="wpStarttime" />
<input type='hidden' value="" name="wpEdittime" />

If I retrieve the page with wget, I do get them (but of course with wget I'm not logged in; my script does get authenticated first). If I retrieve the page with a browser, I do get them. But if I retrieve the page with PHP's curl, nada. That obviously leads to edit conflicts and nil results. So, what gives?

Thanks,
Gutza ^{T T+} 21:33, 23 November 2007 (UTC)

You likely missed to send the right cookies with curl. -- Purodha Blissenbach (talk) 09:56, 15 March 2008 (UTC)

Thank you for the reply – I was able to identify the problem and fix it though, after much trial and error. The problem was way more insidious than I had expected. I ended up grafting my code on SxWiki (the Pro version) – and, lo and behold, the same problem resurfaced! I am using a pre-fabricated method to do all CURL requests. The method accepts two parameters: the URL and the POST variables (the latter is optional). If I get POST variables, I do a POST – if I don't, I do a GET. Or so I thought. In order to clear all POST stuff from my existing CURL object (which might have just done a POST) in the GET branch, I used to do the following code:

<?
  curl_setopt($ch, CURLOPT_POST, 0);
  curl_setopt($ch, CURLOPT_POSTFIELDS, '');
?>

After a lot of hair pulling I was able to determine that the second line above activated the CURL object's POST method (yes, even though I was passing nothing). You can appreciate how difficult it was to come from "not receiving wpStarttime" to "CURL does a POST when you set the POSTFIELDS, and MediaWiki doesn't like that, so it doesn't serve wpStarttime". Anyway, problem solved, and SxWiki Pro is on its way out, if only I could find some testers (see the news on SourceForge if you're interested). --Gutza ^{T T+} 21:37, 15 March 2008 (UTC)

[edit] Planning to make a new bot

Hi, I am planning to make a new bot and I need some help. Please try not to bite me because this is my first time planning to make a bot. Currently featured portal candidates is undergoing a revamp and we want to make a bot to help streamline the promotion process (like adding categories, the feature star, etc.) I'm planning to use AWB for this bot. Can someone give me a quick step-by-step of what I need to do? OhanaUnited^{Talk page} 01:20, 20 December 2007 (UTC)

[edit] BasicBot Problem

Hello, i am trying to make my first bot using the BasicBot template in PHP, however i can't get it to log in, i wondered if anyone else has used this code. I have put my username and password in the code but i keep getting the 'Unable to connect' error. I wondered whether this due to the fact that my account is not an authorised bot, i am giving it the details of my second account CAGBot, which i created to become a bot. I havent submitted a bot request yet because i want to do a bit of initial testing in my namespace sandbox first I can't think of any other reason why it won't log in thank you in advance PiTalk - Contribs 23:51, 25 January 2008 (UTC)

Same with me too. I entered in further code to debug, and found out its not able to save the cookies, or proper cookies, thats (one of the reason) why probably its failing to login, for editing, but viewing mode was okay up until 2 weeks ago, but i tried yesterday, after developing my bot's actual function codes, it failed even for viewing mode ! (wikimedia probably changed even more codes), i fixed couple of bugs to reach that point, where it was at least trying for editing, but with "200"(editing failed), not the "302"(editing success). This "BasicBot" is using the "Snoopy" class library, both are outdated and still have many more bugs in it. Hope this helps, and, any help will be much appreciated. -- Tarikash 03:57, 8 February 2008 (UTC).

I have the same problem. I get an "unable to connect" error every time. If I look at the Snoopy variables, it says that the server is returning a "403 Forbidden" response code. Also the following shows up in the response headers: "X-Squid-Error: ERR_ACCESS_DENIED". Any ideas? Kaldari (talk) 23:37, 3 June 2008 (UTC)

[edit] Move this page to another wiki?

Since almost all said on this project page applies to bots operating on any mediawiki wiki, and the rest applies to all WikiMedia Foundation owned wikis, nearly nothing is specific to the English language Wikipedia, I suggest, to move the content of this page to either Meta Wiki or the MediaWiki wiki, and leave only a short summary and a link here.

I believe the added benefit of being found in a central place and becoming more available to every other community outweighs the cost of an additional click for English language Wikipedians.

Also, unburdoning the English Wikipedia from stuff that is not genuine its own, is imho a good idea, as well as having this good material at a place where it is more likel sought for. -- Purodha Blissenbach (talk) 09:53, 15 March 2008 (UTC)

[edit] French template bot

Can anyone assess the possibility of making a bot to add Template:French commune to any article lacking it in Category:Cities, towns and villages in France. Template data would be access through the French interwiki and would be taken from fr:Modèle:Infobox Commune de France already in place on the French pages which uses the same parameters. If the template already existed on EN (under any of its redirect) or didn't exist on FR then no edit would be made. One small caveat is that the French use:

arrondissement=[[Arrondissement de Bordeaux]] canton=[[Canton de Saint-André-de-Cubzac]]

Whereas on EN we remove the Arrondisement de (or d') and Canton de (or d'). We have hundreds of Canton articles with easily accessible data from FR that could/should be transferred and sanitized. Obviously they need sourcing but that can be fixed later since most efforts so far to transfer the data don't deal with that. gren グレン 06:46, 19 April 2008 (UTC)

[edit] Perl packages

I want to write a new bot in Perl. This page mentions several Perl packages for bots, but there no comparison between them. Can anyone recommend once of them?

Also, there's a package called MediaWiki, which looks quite impressive, but is not mentioned. Is it intentional or was it just missed? --Amir E. Aharoni (talk) 08:10, 30 May 2008 (UTC)

[edit] Notes on using SxWiki Lite

If you're trying to use SxWiki Lite, there are a couple of things to note. First, make sure you create a cookiejar.txt file in the same directory as your bot and make it world writable. Secondly, if you're trying to use the sxGetCat() function, you'll need to update the code to reflect recent API changes. See the bug report. Kaldari (talk) 17:55, 4 June 2008 (UTC)

[edit] WikiFunctions.dll or DotNetWikiBot

I'm planning to rewrite my old bot in .NET, well at least implement similar functionality. Will probably not even bother to look at the old code. Anyway my simple but I guess very hard to answer question is should I make a AWB plugin and use WikiFunctions.dll or use the DotNetWikiBot frame work? The bot will not need lots of GUI, but seeing what it is doing would be nice so I guess AWB is better from that point of view, not sure if DotNetWikiBot have help for that? How is the functionality in the 2 frameworks? Are they very similar or is one generally much better. Any suggestions? --Stefan ^talk 12:58, 8 June 2008 (UTC)