[Grub-dev] back working on some grub stuff :)
Bartek Jasicki
thindil2 at gmail.com
Sun May 11 09:14:46 UTC 2008
On 2008-05-10 at. 18:14:38
Jeremie Miller <jeremie at jabber.org> wrote:
> I've been toying with re-generating a bunch of workunits, and I'd
> like to include this header:
>
> Accept: text/html
>
> Is there any other headers that we should include by default?
Maybe change this on:
Accept: text/html, text/*
Then crawler can get all text type but prefer html. Later client or
server can convert for example .pdf, .doc or .odf files on .html files.
Plus here is still problem with servers which not send any content type
headers.
Maybe add:
Accept-Encoding: gzip,deflate
too? For now C# client has it by default. This can save bandwith and if
i good saw, some servers give content only when it is set (example:
Wikipedia return other page on no compressed connection and other for
compressed)
> Eventually we'll have if-modified-since and etag, but that's a ways
> off yet :(
>
I think header if-modified-since is good idea, this can save bandwidth
and work both for Grub and net servers.
> I'll also be playing with varying the PUT destination (different
> kinds of workunits go to different places), which shouldn't affect
> anything...
>
I wish to test it but for now server with workunits again not works
(HTTP 500 error) ;) Maybe add to client options proposed by Balinny:
"Spam mailing-list on server error"? :D
Bartek
More information about the Grub-dev
mailing list