[Wikia-l] New Repo - Wikia-Pywikibot

Sun Apr 27 21:33:07 UTC 2008

Yes, I know about that extension... I found out about it before it was 
put up by browsing some of Wikia's source code.

I personally had some issues with the extension. The family files were 
largely redundant. All of them prefixed with wikia_ (which breaks all my 
bots which use 'anime', etc...) which is redundant when targeting for 
wikia. And because of how they were generated they had a large redundant 
list of namespaces which don't need to be specified because they're 
already part of family.py.
I created a wikia_basefamily.py which all my family files now inherit 
from. This means that we only need to specify special flags like the 
required self.wikia['projectns'] which sets both the project namespace 
and it's talk (Note that this is geared towards English wiki only, other 
languages use a different format, I added this cause most of Wikia's 
wiki are English). But more importantly flags like, self.wikia['smw'] 
which when set to True automatically populate the list of SMW 
namespaces, starting at 300. A different starting namespace can also be 
specified instead.

Actually, I find how pywikibot does things largely pointless. The API 
itself could be used to grab all of the data that pywiki forces you to 
manually enter into family files. All you would need to do would be to 
give it a base url, and it should be able to generate everything else by 
itself.

Honestly, I'm thinking of trying to make it work more like SVN/GIT commands:

*Remote name aliases:*

$ pywiki remote "anime" http://en.anime.wikia.com
> Added name alias "anime" to base url [http://en.anime.wikia.com]
$ pywiki remote "anime" http://anime.wikia.com
> Error: The name alias "anime" already exists.
$ pywiki remote -f "anime" http://anime.wikia.com
> Changed name alias "anime" from [http://en.anime.wikia.com] to [http://anime.wikia.com]
$ pywiki remote -r "anime"
> Removed name alias "anime" from base url [http://anime.wikia.com]

*Getting the site data:*

$ pywiki sitedata show "anime"
> There is no site data saved for "anime"
$ pywiki sitedata update "anime"
> Getting site data for "anime" from http://anime.wikia.com/api.php?action=query&meta=siteinfo&siprop=general|namespaces&format=xmlfm
$ pywiki sitedata show "anime"
> Version: 1.12alpha
> Lang: en
> Namespaces:
> (-2) "Media"
> (-1) "Special"
> (0) ""
> (1) "Talk"
> (2) "User"
> (3) "User talk"
> (4) "Animepedia"
> (5) "Animepedia talk"
> (6) "Image"
> (7) "Image talk"
> (8) "MediaWiki"
> (9) "MediaWiki talk"
> (10) "Template"
> (11) "Template talk"
> (12) "Help"
> (13) "Help talk"
> (14) "Category"
> (15) "Category talk"
> (110) "Forum"
> (111) "Forum talk"
> (112) "World"
> (113) "World talk"
> (300) "Relation"
> (301) "Relation talk"
> (302) "Property"
> (303) "Property talk"
> (304) "Type"
> (305) "Type talk"

*Setting up users:*

$ pywiki user show "anime"
> There is no user set for "anime"
$ pywiki user set "anime" "AnimeBot"
> Set [[User:AnimeBot]] as normal user for "anime"
$ pywiki user set -s "anime" "AnimeBotSys"
> Set [[User:AnimeBotSys]] as sysop user for "anime"
$ pywiki user show "anime"
> Normal: AnimeBot
> Sysop: AnimeBotSys
$ pywiki user clean "anime"
> Unset users for "anime"
$ pywiki user set "anime" "AnimeBot" "AnimeBotSys"
> Set [[User:AnimeBot]] as normal user for "anime"
> Set [[User:AnimeBotSys]] as sysop user for "anime"

And so on... All configured via cli instead of by messing with a config 
file you shouldn't need to.

Or I could always make it interactive and instead of using "pywiki 
command ..." use the interactive Python shell to create a bot shell.

~Daniel Friesen(Dantman) of:
-The Gaiapedia (http://gaia.wikia.com)
-Wikia ACG on Wikia.com (http://wikia.com/wiki/Wikia_ACG)
-and Wiki-Tools.com (http://wiki-tools.com)

Jesús Martínez wrote:
> About family bot configuration, maybe I'm wrong, but some time ago I
> found on [[Special:Specialpages]] one special page about bot
> configuration, with namespaces and other stuff, like a family file.
> Maybe Wikia was testing some extension, but it's gone now.
>
> An special page like this would be useful, though.
>
> 2008/4/27 DanTMan wrote:
>   
>>  Took the tip. Though I did the monobook force manually myself. That patch
>> may have aged and missed new things, also that one for Export was redundant
>> (Export is a raw page with no skin).
>>
>>  Actually, I was planning on doing something one up on what you thought of.
>> Though, when I first thought of it, the pywikipediabot guys rejected it.
>>  I was going to tweak the function that grabs family files. In a way so that
>> when you use -family:wikia_name and it cannot find a Wikia family, it will
>> fallback to a special class which will use the API to grab namespace and
>> other information about the wiki. Then it will save that data to a flat file
>> (What's the point of building a brand new family file for each wiki, just
>> store it all in a flatfile) and thus the framework will work all over wikia
>> without ever needing a family file, unless your wiki is desynced from Wikia
>> (ie: Uncyclopedia), or is one of the few which use the /w format cause they
>> moved to Wikia from another place and need link compatibility.
>>  Of course, that would be changed so that now you would simply use
>> -family:name when I get around to building that for this repo.
>>
>>  As for your patch.
>>  I did some file searching before I made my change.
>>  But I did it in the reverse of you and found some other info:
>>  - export_address is the function which gives out the export page address.
>>  -- _GetAll.getData is the only thing which calls for that address (besides
>> perhaps one of my scripts or something real odd we don't need to worry
>> about)
>>  --- getall is the function which scripts actually use to grab multiple
>> pages of data.
>>  I don't know what tree leads replace.py to getall cause I didn't do enough
>> hunting. But however, you know the "Getting ## pages from somesite..." that
>> shows up as output in replace.py to indicate data being grabbed. That is
>> part of getall, and that's the only place which has that output.
>>
>>  The issue with editing get other than the fact that you are also applying
>> that replacement to the edit page text you should not be applying it to, is
>> that there are 7 other scripts which call getall. Not only that, but that
>> includes the pagegenerators (which are likely what replace.py is actually
>> using to get the data), and even the disambiguation scripts. So by editing
>> get instead of _GetAll the scripts which use getall or a page generator like
>> they are supposed to are all still broken.
>>
>>  ~Daniel Friesen(Dantman) of:
>> -The Gaiapedia (http://gaia.wikia.com)
>> -Wikia ACG on Wikia.com (http://wikia.com/wiki/Wikia_ACG)
>> -and Wiki-Tools.com (http://wiki-tools.com)
>>
>>  C Stafford wrote:
>>  re:
>> http://svn.nadir-point.com/viewvc/wikia-pywikibot/wikipedia.py?r1=10&r2=9&pathrev=10
>> see
>> https://sourceforge.net/tracker/index.php?func=detail&aid=1885569&group_id=93107&atid=603139
>>
>> also, in general
>> https://sourceforge.net/tracker/index.php?func=detail&aid=1916496&group_id=93107&atid=603141
>>
>>
>> since i was doing random bot work all around wikia, i got tired of
>> manually creating a family file for each wiki, especially when i knew
>> i was only going to be botting there this one time.
>> i figured it would be worth the time to write something to generate
>> the family files dynamically (well, with some cache) from the wiki's
>> current namespace structure (yay Special:Export), than to have an
>> archive of only certain ones.
>>
>> so i wrote one. works quite nicely too, random sample url from the
>> wiki goes in, family file come out.
>>
>> sadly, its only here on my local machine right now (i had it online
>> briefly for some testing, thanks again manticore and jack). i'll see
>> what i can do it get it cleaned up (and protected against attacks) and
>> with a better interface and put online for people.
>>
>> -ubrfzy
>> _______________________________________________
>> Wikia-l mailing list
>> Wikia-l at wikia.com
>> http://lists.wikia.com/mailman/listinfo/wikia-l
>>
>>
>>
>> _______________________________________________
>>  Wikia-l mailing list
>>  Wikia-l at wikia.com
>>  http://lists.wikia.com/mailman/listinfo/wikia-l
>>
>>
>>     
> _______________________________________________
> Wikia-l mailing list
> Wikia-l at wikia.com
> http://lists.wikia.com/mailman/listinfo/wikia-l
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/wikia-l/attachments/20080427/5bf107c9/attachment-0004.html 