[Search-l] Introducing Wikia Evolution

[Linas Vepstas]

2008/8/7 Jeremie Miller <jeremie at jabber.org>:
>> ...
>> I'm not sure what "ui" has to do with tagging ... are you
>> trying to implement a del.icio.us or digg type tagging UI
>> (as opposed to automated tagging methods)?
>
> While I would *love* to explore all the automated and directed "tagging" in
> relation to NLP, the context was specifically how to search just a
> manually-directed subset of the whole index, as in the example given, search
> only "education" sites (but we don't have a user tool to create that kind of
> manual direction yet).
>
> The same mechanisms used to create these filters can of course also be used
> once there's more rich tag datasets, or based on importing the DBs of other
> tagging sites.

OK. I'll post updates in a month or two. My current tasks are not
tagging per-se, but rather sense-similarity measures. I'm pretty
sure I can use these for tagging, but first I have to build my
similarity indexes and infrastructure.

(You may be tickled to know that one of the core similarity
algos runs the page-rank algo on the individual words of
sentences/paragraphs. (with "hyperlinks" created between
words by parsing, mutual information, etc.)  This provides
a fairly shallow, but  still state-of-the-art mechanism for
identifying what a paragraph is talking about.)

--linas