many thanks for this post, this is the kind of stuff i like to hear
:-D i dont mind spurts and lulls, even expect them, but if we could
have an indication of spurts, or little tasters of whats to come.. :-D<br>
<br>
and as for the lunatics, welll ill join you there :-p /me will bring the straight jackets?<br>
<br>
thanks<br>
<br>
mark<br><br><div class="gmail_quote">On Thu, Mar 27, 2008 at 1:49 AM, jer <<a href="mailto:jeremie@jabber.org">jeremie@jabber.org</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Ask my kids would say, "burrrrrn!" ;-)<br>
<br>
I feel and share your frustration, I know at least for my part I've<br>
done a poor job keeping going a steady stream of updates on the<br>
things I've been playing with. Of course, I've never personally been<br>
good at regularity, tend to step back and push things out in<br>
spurts... all I can do though is try to catch up for the last month<br>
or so of various chaotic activities.<br>
<br>
First, Grub is partially stalled right now as we began in Feb the<br>
migration to the NG version, who's server side is back-ended by<br>
hbase, a very alpha open source bigtable clone written atop hadoop.<br>
That process has been painfully sputtery, a cluster of ~7 servers is<br>
maxing out at a few mil docs, not nearly good enough to push GrubNG<br>
to grow significantly. Just last week though I think I came up with<br>
a way to partially work around the challenges of using hbase before<br>
it's ready, so with luck in the next week or so we can get that<br>
distributed crawler growing again :)<br>
<br>
I mentioned Grub/hbase first as while working on it I spent the time<br>
to really think about the bigtable model with hbase and how we can<br>
use it, I even went so far as to design and build a system I called<br>
KT (for Keyword Tuple, it's in the re.search svn repo), that should<br>
be able to scale up while supporting a huge variety of really<br>
exciting features. I even prototyped an implementation of both the<br>
server and a search result interface with a bunch of features which<br>
those that I IM with regularly have seen. I've hesitated to show it<br>
off to a wider audience as the prototype was both horribly insecure<br>
and unusable unless someone showed you first.<br>
<br>
There's a light at the end of the tunnel though, I've attached a pdf<br>
of a ROUGH DRAFT that I hope we can actually let everyone start<br>
playing with in as little as a week. Here's a quick list of the<br>
various features that KT will be enabling in our search results:<br>
<br>
- add new result<br>
- edit any result's title/summary (and see revisions, revert)<br>
- highlight any single result as the best one<br>
- star ratings (already exist)<br>
- delete/trash any result (leaves the title but grey'd out, can be<br>
undeleted easily)<br>
- full change history for that set of results, and rss feed to watch<br>
for changes<br>
- add comments to any result<br>
- select text, images, or input forms from the target site to be<br>
shown below it's search result<br>
- alternative related searches and did-you-mean (user driven)<br>
- custom backgrounds in the header for given searches<br>
<br>
I've found myself doing a bunch of these things because it's simply<br>
*fun*, so if anyone just can't wait and is willing to suffer through<br>
some UI or random breakage pain, just ping me on IM and I can<br>
accommodate some you've-been-warned early access to play.<br>
<br>
It's possible to do lots more than what's above as well, KT is a very<br>
simple and flexible framework.<br>
<br>
Besides helping with the things above, I know Dennis and Seth have<br>
been working hard on improving the index, the result quality has<br>
improved significantly and our infrastructure/systems are a lot<br>
healthier than how they ended up in the mad rush to get it ready for<br>
that crazy first week (we did 5mil queries the first day and didn't<br>
break, not bad).<br>
<br>
As a reminder in general though, this is overall a long term project,<br>
there's going to be lulls and storms. It's also a pretty insane<br>
project, and insane people seem to be rather poor organizers and<br>
communicators, lots will be broken and go un-addressed for way too<br>
long, guess we're well on track for something :)<br>
<font color="#888888"><br>
Jer</font><br><br>
On Mar 26, 2008, at 5:13 PM, Mark (Markie) wrote:<br>
<br>
> Right, im afraid the time has come once again where i have been<br>
> wondering to my self again, and i feel that things need to be said,<br>
> so here they are.<br>
><br>
> *Whats happening with the project. AFAIK overall (and i know<br>
> somethings have happened) but *very* little seems to have happened<br>
> since the launch. Now i know that things are probably happening<br>
> with the team, but any chance of actually telling the users about<br>
> this, cos its not looking good from here atm.<br>
><br>
> Ive copied in the so called pillars of search<br>
><br>
> Transparency - riiiiiiight :-(<br>
> Community - hmmm contribute to stale projects?<br>
> Quality - well....<br>
> Privacy - hmm yes that seems to have been done to an extent ( by<br>
> the community mind)<br>
><br>
> Ive been on the project since dec 2006, and so have been waiting<br>
> along time for this to happen, so its not purely a case of i want<br>
> everything to happen NOW, i just want it to look like SOMETHING<br>
> will happen SOON.<br>
><br>
> *This brings me onto the next topic of where is the project<br>
> going??? There has been practically no progress, and frankly i cant<br>
> see much being done from my point. The launch has happened, many<br>
> people were interested, contributed but have now left, because<br>
> NOTHING has happened. so overall the net gain of launching the<br>
> project?? bad press and a few (relative to the web) minis.<br>
><br>
> *Many things have been promised by various people, which havent<br>
> happened. Most specifically this has come from a certain member of<br>
> staff, one specifically, that has said that they will do many<br>
> things, but even the most basic of tasks seem to have not happened.<br>
> so Broken/missed promises. Well iirc (name here) said he would make<br>
> sure that the about pages etc were created, hmm... (http://<br>
> <a href="http://alpha.search.wikia.com/about.html" target="_blank">alpha.search.wikia.com/about.html</a> in case you forgot where those<br>
> were). This is a wikia project, any chance of getting ANY<br>
> involvement/input/co-ordination from the team who, ultimately, want<br>
> us to make them more successfull and a profit (if were being frank).<br>
><br>
> Now i know i havent been that active recently on the wiki, but i<br>
> have been reading the mailing lists and talking in irc, but the<br>
> main reason for me not being active on the wiki, is mainly the fact<br>
> that i just dont have the motivation to do anything because of the<br>
> above. Frankly atm its a stale project, but hopefully this rant<br>
> (which i hate doing) will mean that the project will hopefully<br>
> become better.<br>
><br>
> If i have offended anyone above then i am sorry, but i feel that<br>
> certain things need to be said right now, in order to make the<br>
> project better, which is my aim.<br>
><br>
> Many thanks and look forward to the responses to this, especially<br>
> from wikia staff<br>
><br>
> Regards<br>
><br>
> mark<br>
><br>
> (user:Markie)<br>
<br>
<br></blockquote></div><br>