[Grub-dev] C# Grubng 0.1 and few questionsHi everyone I create today first alpha version of graphic Grub client. It have all options of first release - minimal GUI, unstable program and ugly source code ; ) But i think it can be useful for test p

[Jason Pump]

Here's a tool to search robots.txt. Alexa's is gone afaik.

http://botseer.ist.psu.edu/botseer.jsp?query=grub-client&submit=Search

The "summaries" seem to just be the top of the robot page.


On 1/28/08, Mark (Markie) <newsmarkie at googlemail.com> wrote:
> heheh begging :-)
>
> okay cool, as long as the name doesnt hold us back :-)
>
> cheers
>
> mark
>
>
> On Jan 28, 2008 9:07 AM, jer <jeremie at jabber.org> wrote:
>
> >
> > > wouldn't it be better to just set the user agent as something
> > > completely different like WikiaSearchBot or something, so we can
> > > loose the Grub reputation?? and then be able to crawl many sites??
> >
> > I don't think it's blocked at all that many, not that I've seen, just
> > that the wiki*.org projects are high profile and we can work on
> > getting that corrected.
> >
> > Grub was started as this vision of a shared crawler, and it's finally
> > getting there, I personally want to see the name stand for that and
> > for something good that benefits everyone.  Just because there was
> > one version many years ago that was buggy, that isn't enough for me
> > to want to abandon the original tie between the name and the vision :)
> >
> > Besides, it's good motivation to make this work *SO* well that
> > everyone is /begging/ to be in it :)
> >
> >
> >
> >
> > Jer
> >
> > _______________________________________________
> > Grub-dev mailing list
> > Grub-dev at wikia.com
> > http://lists.wikia.com/mailman/listinfo/grub-dev
> >
>
>
> _______________________________________________
> Grub-dev mailing list
> Grub-dev at wikia.com
> http://lists.wikia.com/mailman/listinfo/grub-dev
>
>