:-D thanks<br><br>mark<br><br><div class="gmail_quote">On Thu, May 15, 2008 at 3:25 PM, Jimmy Wales <<a href="mailto:jwales@wikia.com">jwales@wikia.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="Ih2E3d">Paul Vixie wrote:<br>
> 1. why is ISC's the only backend? jer's vision is backend syndication, so,<br>
> if his XML schema is stable and if there's at least one f/l/oss implementation<br>
> of crawling and of indexing, then, why aren't there more crawlers and more<br>
> indexers, conforming to jer's XML, possibly flooding data between each other<br>
> and possibly dividing up the workload so that all crawlers don't have to<br>
> crawl all sites? ISC ought to have peers, and we ought to be able to have<br>
> gentlemen's agreements, like, "we'll do [a-l].com, you do [m-z].com", etc.<br>
<br>
</div>We strongly support this notion of backend syndication, and we are<br>
hopeful that as the infrastructure and protocols mature, we will get<br>
more and more people working with us on this.<br>
<div class="Ih2E3d"><br>
> 2. why is Wikia's the only frontend? again referring to the syndication<br>
> model, and knowing that there are other "social search engines", when will<br>
> we see someone other than wikia use ISC's backend, or any other backend whose<br>
> data can be reached using jer's XML?<br>
<br>
</div>Hopefully people can start on this soon... as has already been pointed<br>
out, a great start might be to use our code... sounds like Jer is<br>
deciding on the license now.<br>
<div class="Ih2E3d"><br>
> 3. who is driving the syndication model? it's clear that ISC knows how to<br>
> provide network and power, and that jer knows how to design the system and<br>
> build various parts of it, but who is the champion for jer's vision -- who<br>
> will drive us to better answers for #1 and #2 above? who ought to be in here<br>
> answering critics and beating the drum, which is a distraction to jer (and<br>
> candidly he's too busy to do this part well unless he drops other stuff<br>
> that's already late)? remembering that jimbo keeps this issue alive in the<br>
> press, the overall project still lacks a day to day "programme manager".<br>
<br>
</div>We just transitioned our New York office to fulltime work on the search<br>
project, and Dan Lewis is being put fulltime on the task of community<br>
outreach: answering critics, beating the drum, and doing the detailed<br>
work of working with inbound inquiries from potential partners who are<br>
already interested, outreach to potential partners who are not yet<br>
interested, etc.<br>
<div class="Ih2E3d"><br>
> 4. what else is jer working on? has wikia dedicated him to this project or<br>
> does he also handle day to day fire fighting on wikia's existing services to<br>
> justify his paycheck? and while we're on that topic, what other personnel<br>
> has wikia dedicated to this -- how seriously are they really taking it, in<br>
> terms of cash on the barrel head?<br>
<br>
</div>Jer is fulltime on search, as are several others. Dennis, Seth,<br>
Jeffrey, David, Aaron, Dan... I feel that I am forgetting someone.<br>
<br>
We are prepared to ramp up our commitment as we start to get traction,<br>
as well. At the present time, every time I ask the team what we need to<br>
buy, they say "not yet, we are coding". :)<br>
<div class="Ih2E3d"><br>
> 5. who else is working on this, outside of wikia? what outside volunteers<br>
> or wikia competitor's employees have commit access to the source pool for<br>
> the crawler, or indexer, or front end, or have root access to the donated<br>
> back end machines hosted by ISC? if the answer is nobody, then is that due<br>
> to lack of outreach (see #3 above) or is it wikia's preference that outsiders<br>
> contribute content rather than code and sysops? (is that written anywhere?)<br>
<br>
</div>Strong preference that we get lots of people coding on a fully open<br>
system, as they like it. I think so far we have not done a great job of<br>
outreach, but then again, we have not had everything in place to get<br>
people oriented and started.<br>
<br>
Also, we view ourselves as a "good neighbor" part of the existing Nutch<br>
project: Dennis is a Nutch committer who is starting to work on a set of<br>
ideas he is calling "Nutch 2.0".<br>
<div class="Ih2E3d"><br>
> 6. where are the mini-articles stored? if outside volunteers are mostly<br>
> contributing data, is that data stored on wikia's front end? if so, what are<br>
> the redistribution terms -- would wikia flood this data to competing front<br>
> end operators, and accept incoming floods of similar data from competitors?<br>
> or, is this the "secret sauce", there's no way to get access to contributed<br>
> data of this kind except one article at a time, inside wikia's advertising<br>
> system?<br>
<br>
</div>It's all GFDL, and we make available database dumps. We would have to<br>
consider a "flood" of incoming data from a community/editorial point of<br>
view, but totally welcome it, and are totally committed to sharing<br>
everything extremely liberally.<br>
<div class="Ih2E3d"><br>
> 7. given that the idea of "taking on google" is silly, given their size and<br>
> focus and ambition and brand strength and so on, and that what we can<br>
> actually hope to achieve with this project is to change the game and make<br>
> search part of the internet infrastructure, where are the white papers,<br>
> journal articles, and outreach glossies explaining what the new world of<br>
> internet search could look like, and what effect this change will have on<br>
> google, microsoft, yahoo, and the current market hierarchy, and the rest of<br>
> the "social search" scene?<br>
<br>
</div>I think this is a really great question. :)<br>
<br>
One of the things I have been arguing is that we are no threat to google<br>
even if we are wildly successful at "making search part of the internet<br>
infrastructure" as you put it...<br>
<br>
Google's brand is tied up with search, but Google's business is not<br>
searhc, per se, but the matching of advertisements to user actions and<br>
intentions online. The threat to google is not an open source<br>
alternative that helps 1,000 small competitors to flourish, but a single<br>
large proprietary competitor (Powerset?) that captures enough market<br>
share to take away the advertising marketplace.<br>
<br>
1,000 small competitors are much more likely to simply partner with<br>
Google for ad revenues, because buyers go where the sellers are, and<br>
sellers go where the buyers are.<br>
<div class="Ih2E3d"><br>
> 8. has anybody reached out to yahoo and microsoft to see if they'd like to<br>
> join this effort or at least sponsor it, since as #2 and #3 in internet<br>
> search today, they're the ones with the most to gain if we change the game.<br>
> and if nobody's doing this now, and i did it, what would wikia say about<br>
> sharing the sponsorship burden with other players, perhaps larger players?<br>
<br>
</div>We have done some of this, and would be eager to support you if you want<br>
to help us with it. We can talk privately about the status of current<br>
talks but there is nothing to report and nothing likely to happen right<br>
away... but there are a lot of interested parties in the industry.<br>
<div class="Ih2E3d"><br>
> this list of questions isn't meant to be exhaustive. but as in my own<br>
> controversial efforts over the years, i find the quality of criticism here<br>
> somewhat low.<br>
<br>
</div>:-) Quality criticism is extremely valuable.<br>
<div class="Ih2E3d"><br>
> also for the record, ISC's hosting of this project has been a cash neutral<br>
> event for us, which is important since we don't have cash for this kind of<br>
> thing. the 15-ton air handler wikia bought feeds a room that has other<br>
> projects in it too, and our network is a fixed cost, and wikia has agreed<br>
> to pay for the power we use for search, and the servers were all donated,<br>
> and that donation was targetted for this project, and we got a lot more<br>
> servers than we needed, and we've been passing the excess along to other<br>
> f/l/oss and internet security projects. so no matter whether this project<br>
> changes the world, ISC is already winning.<br>
<br>
</div>:-)<br>
<br>
--Jimbo<br>
<div><div></div><div class="Wj3C7c">_______________________________________________<br>
Wikia Search mailing list<br>
<a href="http://alpha.search.wikia.com/" target="_blank">http://alpha.search.wikia.com/</a><br>
Change options or unsubscribe: <a href="http://lists.wikia.com/mailman/options/search-l" target="_blank">http://lists.wikia.com/mailman/options/search-l</a><br>
</div></div></blockquote></div><br>