[Search-l] May 29, 2008 Search Update

[Dan Lewis]

Here is an updated on what the Search team did last week.

Grub:
•       Grub's "open-loop" is now available.  Anyone can input URLs
and download the compressed crawls
•       Seth is working with Markie and Bartek Jasincki on the Grub
website.  They volunteered to maintain it and keep it current.
(Thanks to you both!)  You can see what they're working on at
http://grub.new.swlabs.org

Operations:
•       Continued optimizing operational support for the project

Index:
•       Finished arbitrary boost indexer and began its implementation
•       Integrated an experimental Wikipedia and Freebase result widget
•       Began supporting language-based searches
•       Built a general list of regexes to demote URLs in the current index

Nutch 2:
•       Finished new XML configuration functionality
•       Tested integration with Hadoop serialization functionality
•       Working on new crawler and HTML parser

Social Tools
•       Imported data and profile images from the alpha site
•       Created social use metrics toolset
•       Set up ReCaptcha
•       Allowed activity.js to display activity for anonymous users
•       Merged new code with existing Wikia code

Search Tools:
•       Added links to try the search on other sites
•       Added background selector dialogue
•       Set up help, legal, about, and "fun" sections
•       Set up the front page
•       Fixed the search results header
•       Began adding tooltips
•       Set up dialogue for "related results" area
•       Began working on cleaning up page history
•       Added real-time global changes