[Grub-dev] Mistery of Wikipedia ban revealed

Angela beesley at gmail.com
Sat Aug 2 15:15:50 UTC 2008


On Sun, Aug 3, 2008 at 12:04 AM, Balinny <balinny at gmail.com> wrote:
>
> I always assumed that Wikipedia simply has blocked queries with the
> substring Grub in the User-Agent.

That is still the case. The exact user-agent they block is
"grub-client". It was added many years ago and should be easy to
appeal if proof can be shown that the problem has been fixed.

http://en.wikipedia.org/robots.txt is where it's blocked.

What's perhaps more interesting is that it's blocked by Wikia:
http://www.wikia.com/robots.txt
Like many wiki sites, we just copied Wikipedia's robots.txt a few years ago.

Angela


More information about the Grub-dev mailing list