[Grub-dev] Url dispatching and failed upload

[Bartek Jasicki]

Dnia 2009-01-12, o godz. 02:18:53
"chris hall" <followingthepath at gmail.com> napisał(a):

> > On dispatch server, workunit is forgotten. This don't mean that urls
> > from this workunit disappear. They can be put to another workunit
> > for another client.
> 
> 
> But how long until the dispatch server packs them into another
> workunit? The dispatch server does not know if the upload was a
> success or not, so the urls have to wait the same amount of time as
> the ones that suceeded.
> 

Dispatch server generate a bunch of workunits in this same time.
Because it get urls randomly from database then delay between use url
in different workunits is various. Workunits are generated before
client connect to server. If client connect, server add only
last record (with info about upload server address and .arc filename) in
workunit (last lines) and put sent workunit to other directory.
So, if client cannot crawl selected workunit or upload .arc file, only
client lost some time (and unfortunately bandwidth), but urls are not
disappears.

Bartek

-- 
Grub Next Generation: http://grub.org
Mailing List: grub-dev at wikia.com
IRC: #wikia-search at irc.freenode.net
Jabber: thindil at jabberpl.org