[Grub-dev] problem of understanding "crawl-corruption" - anyone care to explain?

[ab]

i am fighting with this problem of so-called corrupted crawls.

i have already extensively reported and tried to track down reasons and 
details about this issue, but is somehow really dont understand this 
whole idea of how a set of httpserver-replies can get "corrupted", and 
therefore denied from the soap-server (upload).

i would be really glad if someone would care to enlighten me:

see comments onwards from:
<http://dev.grub.org/cgi-bin/trac.cgi/ticket/9#comment:31>

this "corruption" is driving me crazy, and the grub-crawler is simply a 
failed experiment to me at its present state and design if its really 
supposed to be like this. :(

cheers.