[Search-l] Processing from mulitple indexes...

[Jeremie Miller]

Definitely publish a simple JSON api and it'll become really easy to  
mash it up.

I'm starting to see so many JSON RESTful APIs out there that I wonder  
if the real semantic web isn't happening already via this kind of  
growth :)

Jer

On Sep 4, 2008, at 2:50 PM, Aerik Sylvan wrote:

> Sweet!  That's beautiful.  I can picture hundreds of of little  
> brokers serving up interesting bits of information, and being pulled  
> into meta-search results in the client, providing a richer and more  
> up-to-date experience for the searcher... very cool.  Tell you what,  
> if there's any interest, I've got small datasets at both  
> wikidweb.com (domain level, mostly, about 40k listings, GFDL) and  
> tagthis.info (page level, don't know how many off the top of my  
> head, no license selected yet... gotta do that...) that I'd be happy  
> to publish via js api call, to help feed such a thing.  Getting data  
> from larger vendors (del.ico.us, stumbleupon, whoever) would be  
> better of course, but... :-)
>
> Cool...
>
> I've got this whole vision (I continue to think it's pretty close to  
> your vision) of a much more interconected, cross-pollinating  
> internet, with data shared, aggregated, ranked, and shared again  
> much more freely...  Somewhat off topic, I'm working on something  
> simple and waaay compelling: RSS type feeds for events, and then an  
> aggregator/search engine to find events.  (Think of it, garage  
> sales, concerts, sports events, all published in an atom feed, and  
> searchable by date and location...) - I'm working on a prototype/ 
> promotional website at http://eventfeed.org - I plan to open source  
> the (relatively simple) code in an effort for greater adoption.   
> (It's really simple and rough right now, but basically works.)
>
> Best Regards,
> Aerik
>
>
> On Thu, Sep 4, 2008 at 9:44 AM, Jeremie Miller <jeremie at jabber.org>  
> wrote:
> Heh, that's almost exactly how the JS behind the current re.search
> stuff works, it's pulling in and merging between two indexes currently
> (nutch and KT contribs, and another recent idea is to build a third
> index of just page summaries so we can fit more into our limited
> resources).  One of our important principles is to try and use only
> open-source-built and freely-licensed indexes too :)
>
> The "widgets" (or whatever you want to call them) discussion is very
> close to this also, triggering the merge of other custom results into
> any search, and there's some working going on now to clean up the JS
> to support this even better.
>
> Jer
>
> On Sep 3, 2008, at 3:45 PM, Aerik Sylvan wrote:
>
> > Crossposting this - not sure of the best home for it...
> >
> > So, we have the the beginnings of code and a framework that really
> > facilitates multiple indexes, with factories and brokers and
> > collectors... How about moving the processing of the final result to
> > the client?  In other words, meta search via javascript.  You get a
> > lot of nifty things with that architecture - one of the big bits is
> > that it reduces the server resources needed to aggregate results.
> > It increases bandwidth, becuase you'd need to server perhaps the top
> > 100 results from each index you're querying, and let the client
> > whittle them down to the top 10 combined result.
> >
> > Anybody working/thinking in that direction?
> >
> > (Had this thought for awhile, but the news of Google Chrome's js
> > engine being 10+ times faster makes it even more intriguing).
> >
> > Best Regards,
> > Aerik
> >
> > --
> > http://www.wikidweb.com - the Wiki Directory of the Web
> > http://tagthis.info - Hosted Tagging for your website!
> > _______________________________________________
> > Wikia Search mailing list
> > http://re.search.wikia.com/
> > Change options or unsubscribe: http://lists.wikia.com/mailman/options/search-l
>
> _______________________________________________
> Wikia Search mailing list
> http://re.search.wikia.com/
> Change options or unsubscribe: http://lists.wikia.com/mailman/options/search-l
>
>
>
> -- 
> http://www.wikidweb.com - the Wiki Directory of the Web
> http://tagthis.info - Hosted Tagging for your website!
> _______________________________________________
> Wikia Search mailing list
> http://re.search.wikia.com/
> Change options or unsubscribe: http://lists.wikia.com/mailman/options/search-l