Its a wonderful architecture.. After seeing this mail, i suddenly charted out a work flow diagram to understand it..<br><br><div><span class="gmail_quote">On 7/5/07, <b class="gmail_sendername">jer</b> <<a href="mailto:jeremie@jabber.org">
jeremie@jabber.org</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">This is a brief overview of a large vision: enabling search to become
<br>a part of the Internet's infrastructure. Building on Atlas as an<br>open protocol, search can become a fully distributed and<br>interoperable world-wide community. All of the participants can<br>interact openly and in any role where they believe they can add value
<br>to the network.<br><br>A search engine can be constructed from many independent entities<br>serving different roles instead of one monolithic system. These<br>entities are exchanging aggregate information, or knowledge, and can
<br>decide with whom they want to work with. To design this working<br>economy based on knowledge, there must be balance between these<br>various entities. Each actor must have incentive to act both for<br>their own benefit and for the benefit of the whole, and enough
<br>information to make and validate those decisions. Reputations and<br>relationships are the essential fabric of Atlas, just as they are in<br>a real-world free market.<br><br>There are three primary roles within Atlas:
<br><br> Factory - Responsible to the content.<br> Collector - Responsible to the keyword.<br> Broker - Responsible to the searcher.<br><br>Each of these actors must interact with the others to complete any<br>
search request. Any two roles could be performed by a single entity<br>(whereas if all three are performed by one entity, the result would<br>be a traditional, monolithic search engine).<br><br>A Factory is akin to a crawler in today's search engines. An Atlas
<br>Factory must fetch and process the content as intelligently as<br>possible, performing analysis (such as Natural Language Processing)<br>and normalizing it into distinct units. A Factory shares its highly<br>refined and processed output with one or more Collectors based on who
<br>they believe is best utilizing it.<br><br>A Collector absorbs and indexes output from one or more Factories,<br>with one primary goal: ranking. An Atlas Collector must provide the<br>most intelligent ranking and relationship analysis possible. A
<br>Collector has to compete for the output of a Factory, as well as<br>compete to provide the best ranking quality for Brokers.<br><br>A Broker must provide a searcher with the best possible results. It<br>does so by combining diverse ranking results from Collectors and also
<br>by retrieving content from the original Factories. This last step, a<br>Broker interacting with a Factory, is critical to maintaining a<br>balanced ecosystem. All Factories must be aware of and approve how<br>their results are being used and by whom.
<br><br>Reputation and reward is bi-directional between all parties (Factory-<br>Collector, Collector-Broker, and Broker-Factory).</blockquote><div><br>Basically it seems like a N-N connection between different entities with N-N relationships... When it goes to implementation, this requires discovery and monitoring services.. Don't we need a central entity to do all these ??
<br><br>This central entity is also distributed over regions, organizations.. Like in Grids, we have VO (Virtual Organisations) and in P2P, we have trackers.. But they will be like mentors..<br><br> Why not we have a grouping between F/C/B ?? ie, F/C/B can be considered as a unit which goes into some other higher level structure.. or, we can also group it separately like Factories, Collectors, Brokers or a combinations of them..
<br><br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Each entity may<br>choose to interact on principle (free, Commons), attribution (results
<br>provided by), or commercially (as a paid service), the Atlas protocol<br>is purely a facilitator and does not restrict how the relationships<br>between any entities are formed. In considering these motives for<br>the various entities, it's likely that the free-based networks will
<br>tend to become more specialized, commercial ones will compete on<br>quality, and attribution based networks will mature in both directions.<br><br>This simple yet powerful division of roles, responsibilities, and<br>relationships will result in a distributed economic foundation for an
<br>Internet Search Infrastructure. The wire protocol and further<br>definition of the interactions between these entities is openly<br>evolving, anyone interested is welcomed to join the discussions and<br>see the initial proposals at
<a href="http://lists.wikia.com/mailman/listinfo/">http://lists.wikia.com/mailman/listinfo/</a><br>atlas-l over the coming weeks. <br></blockquote><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Thanks, looking forward to a radically different search ecosystem in<br>the coming years :)</blockquote><div><br>Thanks for giving such a fantastic architecture definition.. <br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Jer<br><br>_______________________________________________<br>Search-l mailing list<br><a href="mailto:Search-l@wikia.com">Search-l@wikia.com</a><br><a href="http://lists.wikia.com/mailman/listinfo/search-l">http://lists.wikia.com/mailman/listinfo/search-l
</a><br>Change options or unsubscribe: <a href="http://lists.wikia.com/mailman/options/search-l">http://lists.wikia.com/mailman/options/search-l</a><br></blockquote></div><br><br clear="all"><br>-- <br>Pushparajan V<br><a href="http://www.vprajan.org">
http://www.vprajan.org</a><br>- - - - - - - - <br>Know me: <a href="http://www.hackerkey.com/decrypt.php?hackerkey=v4sw57BCHJUY$hw3/5ln2pr6AFOPSck3ma4u7FLMSw7DTWXm6l6FGIKLRSU$i862NLJ0CAe6$t3b4en4a23Ns3MSr9g5AGO">http://www.hackerkey.com/decrypt.php?hackerkey=v4sw57BCHJUY$hw3/5ln2pr6AFOPSck3ma4u7FLMSw7DTWXm6l6FGIKLRSU$i862NLJ0CAe6$t3b4en4a23Ns3MSr9g5AGO
</a><br>- - - - - - - -