[Search-l] Old markets and methods vs. new

[Aerik Sylvan]

A bunch of vaguely related thoughts, that seem like a reasonable fit for
this mailing list:

<rant>So I was reading my wife's "Women First" magazine and saw that Mahalo
was in their "top 100 useful websites".  I suppose I have to continue to
give Jason credit where credit is due:  he has apparently very successfully
tweaked the open directory model into a search application.  But at the same
time, I am disgruntled and somewhat disgusted at this - in my humble
opinion, it is 5% inspiration, and 95% marketing... </rant>

Therein lies a dynamic that continues to baffle and intrigue me:  The
internet, more so than any other medium or mechanism, gives people the
opportunity to very quickly disseminate new content and ideas... it is
*possible* for a web page or application to literally become famous
overnight.  Because any page is potentially accessible to anyone on earth,
and because it is possible to scale very very quickly, the internet levels
the playing field more than anything the world has ever seen.  And we see
evidence of this frequently - look at Google, Facebook, Fark... all the
countless applications that were started in garages, dorms, etc.

Yet, the old world dynamic continues to dominate.  If you're connected to
money and people with status, you have a *huge* advantage.  And that
advantage is 99% marketing - anyone can write code or publish some awesome
content, but if you've got money and connections, you can spread your ideas
faster and more thoroughly.  But this is entirely a manufactured limitation
- it is not inherent in the architecture of the net itself.  It's not like
you have to build a million widgets and get them onto store's shelves, or
even print and distribute thousands of newspapers... yet the playing field
still favors the entrenched players.

I believe that in the not too distant future, we will see more disruptive
technologies that will attempt to level the field more, but that the guys
with the money and connections will always fight to stay on top.  I'm
looking forward to seeing (and maybe even helping create) some of those
disruptive technologies.  Distributed search is one possibility.

Other things of interest:  Microsoft getting into social bookmarking:
http://www.informationweek.com/news/internet/social_network/showArticle.jhtml?articleID=208400248(and
many more).  People are still trying to figure out how to really make
tags ("social bookmarks") really work - it's not going to stop at
Del.icio.us.  I still believe this is a space where an open, distributed
framework makes tremendous sense, and also a space that has strong ties to
community driven search.

Lastly, and similarly: the data being built in Wikipedia (and similar
projects) is huge and is under-utilized.  Some tools exist to mine it, but I
think we're only scratching the surface.  My favorite possibility is
category intersections.  A category in Wikipedia is essentially a tag -
someone has said that this chunk of information should be associated with
this concept.  I'd like to share a story I wrote on the developer's mailing
list:

I was looking for multi-player roleplaying games for my kids (age 6 and 8)
to play over our LAN.  I wanted something rated E for everyone.  So I
googled.  I looked at Amazon.  I read reviews.  Then I stumbled on to a
Wikipedia article and had an inspired thought:  I can look for video games
by category on Wikipedia.  So dusted off my prototype categories Lucene
index and did some searches, like "+Multiplayer_online_games
+Windows_games".  It was awesome.  In the end, the dataset in Wikipedia
wasn't quite large enough or complete enough to get me the answers I wanted,
but it *WAS* better quality than the searches I had done in Google.  It was
a terrific test case that showed 1) categories or tags can be a very
powerful way to find content 2) intersections (and more sophisticated means,
like semantic analysis) are powerful and viable ways of utilizing human
entered data and 3) and inclusive content policy on Wikipedia is an absolute
must from a data-mining perspective.

So let's get on with the truly revolutionary and disruptive technologies!

Best Regards,
Aerik

-- 
http://www.wikidweb.com - the Wiki Directory of the Web
http://tagthis.info - Hosted Tagging for your website!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-l/attachments/20080527/4fc908fd/attachment.html