[Search-l] Short interview with Jeremie Miller

[jer]

> It is early here and I'm not sure if I understand this correctly.
>
> a: What you are saying is that the snapshots of crawled webpages  
> will be available for download by anyone?
>
> b: Wikiasearch denies any responsibility for the end use or abuse  
> of this freely available set of the (copyrighted) data of others?
>
> c: Wikiasearch hopes that people will be nice with the data and not  
> abuse it and if someone abuses it, the community will admonish them  
> after the fact?
>
> d: Wikiasearch will be repackaging the copyrighted works of others  
> and making them available for download?

Wow, you really tried hard to make it sound so awful :)

Yes to all of the above, it'll of course follow all the rules like  
current cached results and the Wayback Machine do as far as copyright  
goes, and anyone can already crawl, I just want to lower the barrier  
and increase the collaboration between those trying to do good with  
it, be it search or research.

>> A great ecosystem to be part of by the way, if any of the work we  
>> do  helps any of them succeed then I'll be very happy :)
>
> The whole "ecosystem" term is one that is greatly abused. The  
> search industry is more like a group of medieval warring city  
> states and countries. Each one is desperately fighting for its own  
> survival in a highly competitive and dangerous environment. Many  
> will remain at the warring city state level for years and only a  
> few will rise to the empire stage of Google or Yahoo.

And what a horrible state to be in, if an open economy can't make a  
dent here to improve this situation in a few years then so be it, I  
still feel it really important to try (and it's obviously needed).

> I think that you still do not understand the mentality of a search  
> engine developer. Most of us would consider the market in which we  
> operate in terms of threats and opportunities. These search  
> startups are all busy working on search products. Some will succeed  
> and many will fail. It is a business with a very high attrition  
> rate. Those who survive tend to be somewhat cynical about new  
> projects. We have to develop a kind of survival instinct that  
> enables us to quickly determine what will work in the market and  
> what will not.

If any of the open source tools or services don't provide a  
competitive advantage then they don't deserve to succeed either.   
Ultimately, I'd rather see more search startups all enjoyably working  
on adding unique value to the whole network, not trying to re-build  
another entire empire.

> A compliment as dreadful as my poor grammar. :) Perhaps the manage,  
> crawl and repackage aspect is the only innovative angle.
>
> But in the end, what makes Wikiasearch different from those who  
> compile spam lists? One of the main problems that search engines  
> have with their indices is dealing effectively with spam. Your  
> Wikiasearch project will, realistically, add to that search engine  
> spam problem by providing the MFAs with the copyright content of  
> others. Has Wikiasearch really thought about the implications both  
> for the search ecosystem and from a legal point of view?

If an open community of enthusiasts can't collectively add value  
here, and can't monitor for abuse, then we've done something wrong...  
perhaps only time will tell.

Jer