[Search-l] Using Grid Computing for Wikia Project

[Marc .]

I will try to summarize some selected ideas that appears in this list in the
last days.

Maxim Zakharov reported the Microsoft research for a searching engine that
"combine" IM communities and "classic web search" to produce a "personalized
search" http://research.microsoft.com/displayArticle.aspx?0rc=n&id=1720

I suppose that this can be adopted in our approach of a search engine, (and
we have a IM expert here, don't you think Jer? ;~)

Joel reported the "scoring concept" system and defend (correctly) that:
"1) Such a system can serve as a complement to existing search engine (at
least initially), rather than as a replacement. Hence gaining a critcal
mass of users is less difficult.
2) The approach doesn't require the same amount of infrastructure as a
full-scale search engine, since it suffices to store only ratings.
Storing whole documents is not strictly necessary."

Grahame Gould answered to my mail and points:
"It seems to me that the simplest way, at least initially, is to provide
users with the choice.  Some may like to get involved in communities of
users who "think like they do", but me I'll stick with "try a search and
see what happens and try again".  It's nice to sometimes get distracted
by unexpected search results."

Beside of it, we are discussing that the main problem in a P2P net may be
the large bandwidth demand and Gérard seems to be preoccupied about the
concern that the users may have of a  "spyware" engine that report their
habituates to some "central" servers.

I assume that all the argument are valid, and that are my points, (i will
explain why I put them here later)
1- let make people share what they want to share.
2- let use the "score concept" in a P2P net.
3- to make a "social net" we could use a "modified Jabber IM"
4- the P2P net should only have one address per user (like almost all P2P
nets) but the user in this net need to have the liberty to choose his own
address.

Ok, now I can explain why I think this could be "glued" all together.
1- if the people can choose an address in a P2P net, they need to choose an
address that is close to people who indicate him "good links" (that why a
"social net" can be very helpful and the Jabber IM very useful to build it
quickly --using the friends list.)
2- if the person doesn't share anything good (or anything a all), he will
stay alone, because others people will go "far" from him.
3- I believe that using the "scoring concept" the engine should not use a
lot of bandwidth: the only thing share in this case are "scored links" and
"contacts degree".
4- give "options to users" is, in this system as simple as make the user
"choose his address" with N variables.
5- the only person who receives the final result of a search based in a
system like this is the user himself, no one can see what he see. (Even if a
user can use a program to classify his received links, that is his problem!)

Note: I used the word "choose" because I don't have a better one: even if
the address in the p2p net is automatically chosen by a software in the
computer of the user, it will remain "chosen"

That all I have to say here, sorry to insist in the same way I present on
the mail of the 5th may.

What does you (everybody) think?

BR
Marc Rosenfeld



2007/5/16, Gérard Dupont <ger.dupont at gmail.com>:
> Your proposal is feasible : using user and community experience in order
to
> improve te relevance of query results. It has already proved to be
reliable
> in information filtering application. However, two main concern will tends
> to make me think that it will be hard to make it works well. First the
users
> don't like to be under surveillance of their behavior or interests. That's
> not a technical limitation but if we want to make the search engine
popular,
> a particular explanation has to be correctly build in order to explain
that
> the engine won"t be a spyware. Then, technically, a "full" peer to peer
> architecture has many drawbacks which will be hard to tackle : long time
> before answering and ever partial view of the wwWeb.
> There is no limitation at all, and I like the idea, I just wanted to point
> some difficulties.
>
> G.Dupont
>
> PS : excuse me for all mistakes in english...
>
> _______________________________________________
> Search-l mailing list
> Search-l at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-l
> Change options or unsubscribe:
> http://lists.wikia.com/mailman/options/search-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-l/attachments/20070517/81436c88/attachment.html