[Search-l] June 3, 2008 Search update

[Dan Lewis]

Here is what the Search team worked on this week.

Operations:
* Fought a lot of index searching code
* Improving mulicast code
* Began exploring building a newer version of hbase
* Added caching and fixed misc errors in search JSON.

Index:
* Improved Wikipedia results widget
* Starting human improvement on top keywords

Nutch:
* Helped fix a tika bug in Nutch 1
* Changed Nutch 2 crawler to use commons httpclient 4.x
* Implemented gzip processing
* Implemented https processing
* Implemented manual handling and storage of redirect info
* Improved Nutch 2 serialization libraries -- allowed arbitrary object
creation, created finctions to allow non-default constructor object
creation, and allowed serialing of object fields from serialization
implementations
Social Tools:
* Added RSS feeds for user activity
* Re-worked notifications

Search Tools:
* Re-worked main page
* Made related searches graphic
* Changed the "See results from" into favicons
* Added language support
* Added RSS feeds for search results
* Drafted copyright policy
* Annotations now give instant feedback
* Links can be added into annotations
* Added sorting based on star rating
* Re-incorporated and improved handling of Mini articles
* Added fun stuff -- Mapped Changes, Live Changes, The Faviconnery,
Micro Results, Bloom, Clippy, Texter
* Lots of cross-browser testing and bug swatting