[Search-l] Fwd: more than just interoperability

[Aerik Sylvan]

On 6/2/07, Jason Calacanis <jason at calacanis.com> wrote:
>
>
> The even bigger problem is that the folks who have the best
> information to crawl--folks like delicious and google--do not allow
> metasearch and would take action if you sucked their data into another
> dataset that competed with them.
>
> They would, of course, have a very good point: it's not very fair to
> build a business overnight by indexing their information. Folks have
> been trying to do this to Craigslist and Craig Newmark has blocked
> them.


Exactly the point I'm making.  There's a tremendous amount of pretty good
human generated data out there ( I would argue that, at a minimum, the
categorization of urls in dmoz is of some value, anyway), and integrating it
would be a great way to get a decent signal to noise ratio.  I think the
wikia search has the capability to create some novel and interesting
filtering and weighting mechanisms, but all by our lonesome, it will be
difficult and timeconsuming to develop a really interesting dataset.

So, since Wikia is a for-profit venture, perhaps it makes sense to look into
licensing some data from closed providers (stumbleupon, for instance - since
del.icio.us/yahoo is unlikely to want to feed a competing search engine).

Aerik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-l/attachments/20070602/b10481c1/attachment.html