[Grub-dev] Mistery of Wikipedia ban revealed
Angela
beesley at gmail.com
Sat Aug 2 15:15:50 UTC 2008
On Sun, Aug 3, 2008 at 12:04 AM, Balinny <balinny at gmail.com> wrote:
>
> I always assumed that Wikipedia simply has blocked queries with the
> substring Grub in the User-Agent.
That is still the case. The exact user-agent they block is
"grub-client". It was added many years ago and should be easy to
appeal if proof can be shown that the problem has been fixed.
http://en.wikipedia.org/robots.txt is where it's blocked.
What's perhaps more interesting is that it's blocked by Wikia:
http://www.wikia.com/robots.txt
Like many wiki sites, we just copied Wikipedia's robots.txt a few years ago.
Angela
More information about the Grub-dev
mailing list