[Search-l] more than just interoperability

[Jason Calacanis]

On 6/1/07, peter burden <peter.burden at gmail.com> wrote:
> However the sites mentioned would be excellent sources of crawler seeds,
> although effective crawling of
> dynamic (Web 2.0) database/CMS driven sites poses some significant
> problems - especially if they're using
> Ajax.

Peter,

The even bigger problem is that the folks who have the best
information to crawl--folks like delicious and google--do not allow
metasearch and would take action if you sucked their data into another
dataset that competed with them.

They would, of course, have a very good point: it's not very fair to
build a business overnight by indexing their information. Folks have
been trying to do this to Craigslist and Craig Newmark has blocked
them.

The DMOZ is open, but their data isn't clean enough at this point to
be of any real value. No fault to the DMOZ editors, but due to--from
what they tell me--AOL ignoring them/not supporting them. For the
record, I tried to buy DMOZ from AOL when I was leaving and they had
no interest in selling it.

best regards,

Jason
-----------
http://www.mahalo.com
http://www.calacanis.com