[Grub-dev] Max amount of bytes per request

[Bartek Jasicki]

On 2009-03-19, at 12:39:55
Giorgos Logiotatidis <seadog at sealabs.net> wrote:

> Hello everyone,

Hi ;)

> 	I just saw the latest grub's c# client release notes (hey
> congrats thindil! :) 

Thanks ;)

>and I noticed that line:
> 
> ""- if max amount of bytes allowed by one request exceed, wrote
> partial response to file, instead report as error""
> 
> According to the clients page [1] the max amount bytes per request is
> 25Mb. If the request exceeds that should we just write the partial
> response and pretend that everything went fine or should we also
> report that with some extra headers in the arc file?
> 

Write only partial response - that's all. Nothing special is needed to
add to .arc file.

> BTW how did we came with the value 25MB? Probably one will not find a
> 25Mb text/html file on the internet. So this is for binary files, like
> pdfs and images, right? I 'm wondering that's the point to write
> partial pdf data or partial image data to the arc. They wont be of
> any value, would they?
> 

This was Jeremie decision ;) Earlier, C# client report that long files
as a error in .arc file. Partial data can be useful, because Nutch can
search (if have enabled this plugins) in for example pdf, mp3, .doc,
.odf. There no sense to load something bigger to database, because
AFAIK, Nutch read only first 200 KB of each record.
Value 25 MB was set as a example - probably we can drop it more (even
to 1-5 MB). But this is question to people who care about IOI ;)

> 
> Regards,
> Giorgos
> 
> [1] http://grub.org/?q=en/node/140
> 

Bartek

-- 
Grub Next Generation: http://grub.org
Mailing List: grub-dev at wikia.com
IRC: #wikia-search at irc.freenode.net
Jabber: thindil at jabberpl.org