[Grub-dev] problem of understanding "crawl-corruption" - anyone care to explain?

[jer]

> this needs to be a lot more foolprof and failsafe i think. make use of
> every reply/data the servers receive, as much as possible.

Balinny already replied but I wanted to reiterate the need for the  
server to be cautious.  One of the most important things we have to  
do is keep people from injecting junk into the system, and the  
current system does that by enforcing that the client do *exactly* as  
it was told in the workunit.  Any well behaved client can make an  
entry for every single url it was given in the workunit, any  
incomplete or incorrect results have to be rejected to be safe.

This at least prevents URL injection... dealing with content  
injection under the assigned URLs is a different problem altogether.

Jer