[Search-l] Search-l Digest, Vol 9, Issue 4

[Gérard Dupont]

Hi,
Your self scale architecture appear to be promising and your figure too !

Unfortunately, I can't find any paper on it and your abstract does not
describe enough to see if it can be applied to search. Do you have any link
?

G.Dupont



>
> Hi,
>
>
>
> I have only recently been lurking in this community. I hope my question
> isn't out of school or already well understood.
>
>
>
> I have been developing / researching architectures to support very large
> scale reliable computing at low cost. It would appear search is a good
> candidate application. The purpose of this note is to solicit interest in
> collaborating with me to flesh out and prototype a specific Search
> architecture for Search.
>
>
>
> The architectural model I have being used is self-scaling (as per figure
> below). The goal is being able to treat a network of computers as a
> logical
> computing space. The implementation integrates HTTP servers & clients into
> a
> single execution space using an asynchronous I/O engine. With this
> approach
> the capability to serve 10,000+ concurrent HTTP connections and maintain a
> multi-gigabyte throughput has been demonstrated on a single device.
> Combined
> with technologies such as ZFS, means terra bytes of content can be cheaply
> stored, search and served. This enables 100-1000x lowering in the cost to
> serve a web request, but with a highly reliable configuration. The
> capability to serve 1/7th of Australia internet traffic with a single
> $2,500
> PC was recently demonstrated.
>
>
>
> My thinking for search was that if a proxy server & client was deployed on
> each device, content could be pre-downloaded and served locally. Anyone
> using the search service would have a much faster internet experience.
> This
> content could also be processed and monitored in real time for actual
> usage.
> The communities accessing sites would do most data aggregation, only
> finally
> transferring source data to a much smaller number of root computers.
> Network
> & computing costs for search are thus mostly offloaded to the edge of the
> network. A better service that improves the more people use it, but at a
> decreasing or static cost.
>
>
>
> If this is an approach worth pursuing, I would appreciate the feed back.
>
>
>
> Regards,
>
> Phil Haynes
>
>
>
> Figure 1 Self Scaling computing
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.wikia.com/pipermail/search-l/attachments/20070809/23461f66/attachment.html
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: not available
> Type: image/jpeg
> Size: 15101 bytes
> Desc: not available
> Url :
> http://lists.wikia.com/pipermail/search-l/attachments/20070809/23461f66/attachment.jpe
>
> ------------------------------
>
> _______________________________________________
> Search-l mailing list
> Search-l at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-l
>
>
> End of Search-l Digest, Vol 9, Issue 4
> **************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-l/attachments/20070809/3f2af963/attachment.html