[Grub-dev] Why grub is blacklisted so widely.

[Bartek Jasicki]

>> My English is little rusty, thus I'm not sure if i good understanding ;)
>> Client fetch only pages, don't look at robots.txt? Server check
>> robots.txt before add url's to workunit?
>>
>> Bartek
>>     
> Yes.
> It's a bit strange that precisely you are asking that, given that the C# 
> client (or any of them) don't check robots.txt ;-)
>
>   

Only a little ;) I think ahead about add check robots.txt to C# client
thus i ask about this.
And if I good remember, all clients are in early stage ;)

Bartek