On Mon, Mar 31, 2008 at 8:08 PM, Dennis Kubes <<a href="mailto:kubes@apache.org">kubes@apache.org</a>> wrote:<br>> <br><snip lots of context>. One of the<br>> problems I see is being able to determine if a change actually improves<br>
> search results (besides just looking at the results). Any ideas on how<br>> to determine this?<br>> <br><br>At the risk of being dense/overly stubborn: direct feedback coupled with indirect feedback? The direct feedback would be a score given by willing participants. Indirect feedback is clickthroughs, stuff like that. I really think the area where wikia search has a chance to shine is in how we engage participants in the process. I'm *sure* google and the other big guys collect indirect feedback. They've got an army of scientists. We probably won't beat them at that game, but we shouldn't ignore indirect feedback and pagerank style algorithms either.. BUT, where we can be diiferent - maybe even better - is in engaging the direct involvement of the community. I don't mean picking the entrenched results for a given search, as Mahalo is doing (BORING! I *still* say it's "just" a spin on the dmoz approach - not that it's bad, it just isn't new or impressive any more) but instead engage the community to continually provide feedback on results and keep the results *fresh*. Incorporate ideas like Alexas <a href="http://www.alexa.com/site/ds/movers_shakers">movers and shakers</a>. I have always like how netfilx asks you how you liked a movie, after you've returned it. Maybe we can think of better methods.<br>
<br>The trick is going to be volume. If you have 10 million searches, and 1 percent of the searches have active feedback, that's 100,000 data points. Given an assumed very wide spread (that 100,000 points will have a very small head and a very long tail), it is hard to draw statistically significant conclusions. So, you need either a) more searches or b)a higher percentage of feedback. Not sure how to tackle that, but if any project ever was going to do it, it's this one.<br>
<br>Best Regards,<br>Aerik<br><br><br>-- <br><a href="http://www.wikidweb.com">http://www.wikidweb.com</a> - the Wiki Directory of the Web<br><a href="http://tagthis.info">http://tagthis.info</a> - Hosted Tagging for your website!<br>