From newsmarkie at googlemail.com  Wed Jun  4 20:01:24 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Wed, 4 Jun 2008 21:01:24 +0100
Subject: [search-ui] Screenshots
Message-ID: <a6424d870806041301q33cb1337hb61f16522f66b625@mail.gmail.com>

Incase anyone is interested ive uploaded screenshots in 46 diff browsers,
and onyl a few of them fail <shakes fist at msie> :-p

http://people.swlabs.org/~markie/Screenshots/

a few tweaks needed in some of them, ill try to set up a gallery in a mo

regards

mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080604/8b429e1c/attachment.html 

From jeremie at jabber.org  Wed Jun  4 21:43:36 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Wed, 4 Jun 2008 16:43:36 -0500
Subject: [search-ui] Screenshots
In-Reply-To: <a6424d870806041301q33cb1337hb61f16522f66b625@mail.gmail.com>
References: <a6424d870806041301q33cb1337hb61f16522f66b625@mail.gmail.com>
Message-ID: <BF7ADDFE-4458-4967-9A37-C47AF1BB6BD3@jabber.org>

NICE! Thanks!

On Jun 4, 2008, at 3:01 PM, Mark (Markie) wrote:

> Incase anyone is interested ive uploaded screenshots in 46 diff  
> browsers, and onyl a few of them fail <shakes fist at msie> :-p
>
> http://people.swlabs.org/~markie/Screenshots/
>
> a few tweaks needed in some of them, ill try to set up a gallery in  
> a mo
>
> regards
>
> mark


From borboleta at gmail.com  Wed Jun  4 22:12:27 2008
From: borboleta at gmail.com (Bani)
Date: Wed, 4 Jun 2008 19:12:27 -0300
Subject: [search-ui] Bug reports
Message-ID: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>

First bug report: the bug report page at
http://re.search.wikia.com/help/reportbug.html doesn't seem to work.
At least nothing happens after I click submit. My browser is
Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.14)
Gecko/2008040413 Firefox/2.0.0.14.

Second bug report (the one I've tried to submit there):
Profile update for work and education isn't working properly. On Work
it didn't set year and country, and on Education I couldn't add
anything at all.


Regards,
Bani

From rainer.blome at gmx.de  Thu Jun  5 19:37:17 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 05 Jun 2008 21:37:17 +0200
Subject: [search-ui] Bug reports
In-Reply-To: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
References: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
Message-ID: <4848406D.4090204@gmx.de>

Great new launch, applause!

Working links to substantial about pages, privacy policy, copyright and 
terms of use, yay!

The interface is a joy to use, and mostly easy to use.

Some issues:

Bani wrote:
 > First bug report: the bug report page at
 > http://re.search.wikia.com/help/reportbug.html doesn't seem to work.
 > At least nothing happens after I click submit. My browser is
 > Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.14)
 > Gecko/2008040413 Firefox/2.0.0.14.

o Same here, with
   Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13)

o In the right mouse button (context) menu, "Copy" is grayed out.
   It should be active.

o The different meanings of "Comment" and "Annotate"
   become clear only after one has used them.
   Tooltips for all of the "edit" links might help here.

o Every action should be easy to undo.
   - I did not see a way to "unspotlight" a result without spotlighting
     another result.
   - I did not see a way to "unhide" an image that i hid just a moment
     ago.
   - I did not see a way to remove a comment
     (this is probably intentional)

o Recent Changes:
   - sometimes have negative time stamps, as in "-15 seconds ago".
   - Recent Changes are sometimes not sorted by time stamp.
   - Individual Recent Changes sometimes vanish upon reload.
   - Recent Changes' time stamps are sometimes off by several hours.
     Maybe a time zone issue?

o Fun:Live Changes: Same issues as with Recent Changes.

o Fun:Faviconnery does not appear to do anything.
o Fun:Texter does not appear to do anything.
o Fun:Microresults does not appear to do anything.
o Fun:Clippy does not appear to do anything.

Rainer

From newsmarkie at googlemail.com  Thu Jun  5 20:33:09 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Thu, 5 Jun 2008 21:33:09 +0100
Subject: [search-ui] Bug reports
In-Reply-To: <4848406D.4090204@gmx.de>
References: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
	<4848406D.4090204@gmx.de>
Message-ID: <a6424d870806051333i4b44dc14g4db4d0117d3ac418@mail.gmail.com>

hmm i agree with most of these reports however patches have been submitted
for most of these, they just dont seem to be making their way out :-(

poking the team :-p

mark

On Thu, Jun 5, 2008 at 8:37 PM, Rainer Blome <rainer.blome at gmx.de> wrote:

> Great new launch, applause!
>
> Working links to substantial about pages, privacy policy, copyright and
> terms of use, yay!
>
> The interface is a joy to use, and mostly easy to use.
>
> Some issues:
>
> Bani wrote:
>  > First bug report: the bug report page at
>  > http://re.search.wikia.com/help/reportbug.html doesn't seem to work.
>  > At least nothing happens after I click submit. My browser is
>  > Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.14)
>  > Gecko/2008040413 Firefox/2.0.0.14.
>
> o Same here, with
>   Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13)
>
> o In the right mouse button (context) menu, "Copy" is grayed out.
>   It should be active.
>
> o The different meanings of "Comment" and "Annotate"
>   become clear only after one has used them.
>   Tooltips for all of the "edit" links might help here.
>
> o Every action should be easy to undo.
>   - I did not see a way to "unspotlight" a result without spotlighting
>     another result.
>   - I did not see a way to "unhide" an image that i hid just a moment
>     ago.
>   - I did not see a way to remove a comment
>     (this is probably intentional)
>
> o Recent Changes:
>   - sometimes have negative time stamps, as in "-15 seconds ago".
>   - Recent Changes are sometimes not sorted by time stamp.
>   - Individual Recent Changes sometimes vanish upon reload.
>   - Recent Changes' time stamps are sometimes off by several hours.
>     Maybe a time zone issue?
>
> o Fun:Live Changes: Same issues as with Recent Changes.
>
> o Fun:Faviconnery does not appear to do anything.
> o Fun:Texter does not appear to do anything.
> o Fun:Microresults does not appear to do anything.
> o Fun:Clippy does not appear to do anything.
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080605/41954e89/attachment.html 

From rainer.blome at gmx.de  Thu Jun  5 20:58:47 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 05 Jun 2008 22:58:47 +0200
Subject: [search-ui] Mini articles should be more visible
In-Reply-To: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
References: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
Message-ID: <48485387.7000500@gmx.de>

Mini articles are currently "psychologically invisible", to me at least.
I was half done writing a question about what should happen to the mini 
articles when I noticed that there are edits and looked closer.
Indeed, there they are, "hidden" above that separator line.

Loosely quoting the interview with Udi Manber:
"Q: What do people expect from search?
  A: They want an answer. The goal is to return that answer. "

Mini-articles have the potential to provide these answers immediately, 
without reference to a web search result.
This is a unique, invaluable feature of Wikia Search.

Does Wikia Search want to be a pure search engine?
If so, be done with the mini articles and drop them.

If not, and it seems so (and I hope so), the mini articles should be 
made more prominent.  For example by spotlighting them by default.
Or by highlighting them with a light gray background.

And a way to add new mini articles should be provided
(or is there one and I missed it?).

A star rating for mini articles might help.  The higher the rating, the 
more conspicuous the mini article could be, by giving it more screen 
space and maybe intensifying the background color.

Rainer

From rainer.blome at gmx.de  Thu Jun  5 21:30:30 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 05 Jun 2008 23:30:30 +0200
Subject: [search-ui] "Deleting" results
In-Reply-To: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
References: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
Message-ID: <48485AF6.7070507@gmx.de>

Currently, "deleting" a result will gray it out.
Is there anything else done to that result, for example is its rating 
changed?

The interactions left are to copy the URL or to undelete it.

Sometimes I think "Why, this URL looks related to the search term, why 
was it deleted?" and want to check.  Having to copy the URL and open a 
new page is tedious.  Undeleting the result, following the link and 
maybe deleting the result again is about equally tedious and causes 
unnecessary changes.

Frequently visited pages ("Jimmy Wales", for example) are quickly 
cluttered with "deleted" links, many of which are valid results in the 
sense that "Jimmy Wales" at least appears on the target page. "Bad" 
results should be demoted, not "deleted".

Therefore, I would prefer it if the "Delete" action was replaced by the 
ability to "award zero stars", meaning "In my opinion, this result is 
not related to the search term".

Rainer

From jeremie at jabber.org  Sat Jun  7 07:23:57 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Sat, 7 Jun 2008 02:23:57 -0500
Subject: [search-ui] "Deleting" results
In-Reply-To: <48485AF6.7070507@gmx.de>
References: <35b94d690806041512l142dd515g4e8055c4a846a7a5@mail.gmail.com>
	<48485AF6.7070507@gmx.de>
Message-ID: <D4DE19C9-35D8-4D06-9B3F-AE40B969DFDA@jabber.org>

I just checked in a patch to svn that will make sure deleted results  
are sorted below any rated ones (demoted), and when you mouse-over the  
"Undelete" it will show you the full result so you can decide if you  
want to or not.

On Jun 5, 2008, at 4:30 PM, Rainer Blome wrote:

> Currently, "deleting" a result will gray it out.
> Is there anything else done to that result, for example is its rating
> changed?
>
> The interactions left are to copy the URL or to undelete it.
>
> Sometimes I think "Why, this URL looks related to the search term, why
> was it deleted?" and want to check.  Having to copy the URL and open a
> new page is tedious.  Undeleting the result, following the link and
> maybe deleting the result again is about equally tedious and causes
> unnecessary changes.
>
> Frequently visited pages ("Jimmy Wales", for example) are quickly
> cluttered with "deleted" links, many of which are valid results in the
> sense that "Jimmy Wales" at least appears on the target page. "Bad"
> results should be demoted, not "deleted".
>
> Therefore, I would prefer it if the "Delete" action was replaced by  
> the
> ability to "award zero stars", meaning "In my opinion, this result is
> not related to the search term".
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>


From bernhard at janetzki.eu  Sun Jun  8 11:22:35 2008
From: bernhard at janetzki.eu (Bernhard Janetzki)
Date: Sun, 08 Jun 2008 13:22:35 +0200
Subject: [search-ui] help needed?
Message-ID: <484BC0FB.6010708@janetzki.eu>

Hello together,
yesterday i discovered the wikia project and i like this idea very much!
I wondering if there is some help needed to bring forward the ui search 
interface?

As an advanced software developer i can possibly contribute sth. to this 
great project!

Best regards from bavaria,
Bernhard

From jeremie at jabber.org  Wed Jun 11 00:54:20 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Tue, 10 Jun 2008 19:54:20 -0500
Subject: [search-ui] KT api v2
Message-ID: <0F2C0C0F-3D1B-44E5-A81D-842A197A94B8@jabber.org>

(now that I've written this and re-read it, it's rather dense and out  
of context and I apologize, it's a raw brain-dump)

I don't know of a better place to discuss the KT service/API so I  
figured the search-ui list is as good as any as it's the UI that  
depends deeply on it :)

So, now that we've got it in heavy daily use, we need to think again  
about KT and how to support a number of new functions we're seeing  
immediate need for:
	- keyword statuses: protected, loggedin
	- user authorization levels: anon, loggedin, admin
	- expiration of tokens
	- rate limiting of writes to prevent flooding
	- more granular selections (to reduce size of some kt loads)
	- more efficient single round trips (multiple things in one request)
	- hostname/url based lookups

Here's our current API in http://svn.swlabs.org/kt/kt-tools/ :

kt.js
	t[] = tuple name(s)
	k = keyword/search string
	f = callback function

user.js
	t[] = tuple name(s)
	u = user name
	n = number to return, default 10
	f = callback function

browse.js
	start = keyword to begin from (optional, starts from beginning  
otherwise)
	max = how many keywords to return
	f = callback function

tuple.js
	t = tuple name
	max = number to return, default 10
	f = callback function

new.js
	k = keyword/search string
	t = tuple name
	j = json object
	v = validation token (to apply against user attribute in json string)


Here's my proposed changes:

SOONER:

tuple.js:
	- allow multiple t's in one request (dunno why it wasn't from the  
start)
new.js:
	- validation token is required, "anon" is just a user=IP with a valid  
token
	- token format becomes "hash:UTC" and hashing is md5(USER+UTC+SECRET)
	- KT will only consider tokens with a UTC newer than X minutes
	- KT will track the number of writes per token and block flooding/ 
botting (token creator must take care in providing them)
	- each user type is a different SECRET and KT tries from list to  
determine user privilege level
	- only "admin" users will be able to write to the "admin" tuple
	- admin tuple is where keyword flags like protected or loggedin can  
be set, new.js must check this now too before saving
	f = callback function returning actual saved json (dunno why it  
doesn't have this yet)

NEXT:

url.js:
	t[] = tuple name(s)
	u = exact url
	n = number to return
	f = callback function

urls.js:
	- works just like browse.js but start=url
			

LATER:

For any t=name, allow the more granular t=name:X where X is an integer  
number of the newest X tuples to return for that name (versus the  
overall n= or max= that is indiscriminate).

Very rough idea, for any "read" call (any but new.js) allow a filter  
to reduce the json tuples returned, when specified then only matching  
json objects are returned, very simple format:
		&r={"id>":12345,"l=":"en"}&r={"user=":"foo"}
Each reducer is a flat json object where the name of each attribute  
must be one of = > < and a single reducer's attrib is "and'd" together  
so that an object must match all of them to match the reducer, but  
individual reducers are "or'd" as passing any one of them will allow  
the object through.

Jer


From jeremie at jabber.org  Wed Jun 11 01:49:11 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Tue, 10 Jun 2008 20:49:11 -0500
Subject: [search-ui] help needed?
In-Reply-To: <484BC0FB.6010708@janetzki.eu>
References: <484BC0FB.6010708@janetzki.eu>
Message-ID: <A3AA67D5-F9A7-4CFE-A667-CB0CED96BB79@jabber.org>

It's kind of a mess right now, but definitely take a look at the  
subversion repository:

http://svn.swlabs.org/re.search/cool/

If you're good with JS, it might not be too bad... but it needs a lot  
of re-org/cleanup before it'll be easy for anyone :)

Jer

On Jun 8, 2008, at 6:22 AM, Bernhard Janetzki wrote:

> Hello together,
> yesterday i discovered the wikia project and i like this idea very  
> much!
> I wondering if there is some help needed to bring forward the ui  
> search
> interface?
>
> As an advanced software developer i can possibly contribute sth. to  
> this
> great project!
>
> Best regards from bavaria,
> Bernhard
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>


From newsmarkie at googlemail.com  Wed Jun 11 11:02:46 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Wed, 11 Jun 2008 12:02:46 +0100
Subject: [search-ui] KT api v2
In-Reply-To: <0F2C0C0F-3D1B-44E5-A81D-842A197A94B8@jabber.org>
References: <0F2C0C0F-3D1B-44E5-A81D-842A197A94B8@jabber.org>
Message-ID: <a6424d870806110402p39064d6au89e603e80e4a3d5d@mail.gmail.com>

woooaaahhhhh overload :-p


On Wed, Jun 11, 2008 at 1:54 AM, Jeremie Miller <jeremie at jabber.org> wrote:

> (now that I've written this and re-read it, it's rather dense and out
> of context and I apologize, it's a raw brain-dump)
>
> I don't know of a better place to discuss the KT service/API so I
> figured the search-ui list is as good as any as it's the UI that
> depends deeply on it :)
>
> So, now that we've got it in heavy daily use, we need to think again
> about KT and how to support a number of new functions we're seeing
> immediate need for:
>        - keyword statuses: protected, loggedin

yes definately need this :-)

>        - user authorization levels: anon, loggedin, admin

see comments below about anons

>        - expiration of tokens
>        - rate limiting of writes to prevent flooding

yes :-)

>        - more granular selections (to reduce size of some kt loads)
>        - more efficient single round trips (multiple things in one request)
>        - hostname/url based lookups
>
> Here's our current API in http://svn.swlabs.org/kt/kt-tools/ :
>
> kt.js
>        t[] = tuple name(s)
>        k = keyword/search string
>        f = callback function
>
> user.js
>        t[] = tuple name(s)
>        u = user name
>        n = number to return, default 10
>        f = callback function
>
> browse.js
>        start = keyword to begin from (optional, starts from beginning
> otherwise)
>        max = how many keywords to return
>        f = callback function
>
> tuple.js
>        t = tuple name
>        max = number to return, default 10
>        f = callback function
>
> new.js
>        k = keyword/search string
>        t = tuple name
>        j = json object
>        v = validation token (to apply against user attribute in json
> string)
>
>
> Here's my proposed changes:
>
> SOONER:
>
> tuple.js:
>        - allow multiple t's in one request (dunno why it wasn't from the
> start)
> new.js:
>        - validation token is required, "anon" is just a user=IP with a
> valid
> token

it was recently discussed in the irc channel, why do anons need to be an
IP?  couldnt they maybe be a hash or a number, there is no need to keep
their actual ip at the UI level?  maybe it could be kep in the backend table
for blocking, but if privacy is a key i dont think we should be publishing
IP's so widely (or failing this we need to have a HUGE warning message for
anon editors)

>        - token format becomes "hash:UTC" and hashing is
> md5(USER+UTC+SECRET)
>        - KT will only consider tokens with a UTC newer than X minutes
>        - KT will track the number of writes per token and block flooding/
> botting (token creator must take care in providing them)
>        - each user type is a different SECRET and KT tries from list to
> determine user privilege level
>        - only "admin" users will be able to write to the "admin" tuple
>        - admin tuple is where keyword flags like protected or loggedin can
> be set, new.js must check this now too before saving
>        f = callback function returning actual saved json (dunno why it
> doesn't have this yet)
>
> NEXT:
>
> url.js:
>        t[] = tuple name(s)
>        u = exact url
>        n = number to return
>        f = callback function
>
> urls.js:
>        - works just like browse.js but start=url
>
>
> LATER:
>
> For any t=name, allow the more granular t=name:X where X is an integer
> number of the newest X tuples to return for that name (versus the
> overall n= or max= that is indiscriminate).
>
> Very rough idea, for any "read" call (any but new.js) allow a filter
> to reduce the json tuples returned, when specified then only matching
> json objects are returned, very simple format:
>                &r={"id>":12345,"l=":"en"}&r={"user=":"foo"}
> Each reducer is a flat json object where the name of each attribute
> must be one of = > < and a single reducer's attrib is "and'd" together
> so that an object must match all of them to match the reducer, but
> individual reducers are "or'd" as passing any one of them will allow
> the object through.
>
err yeah <nod and smile> :-p

thanks for all the info :-)

regards

mark


>
> Jer
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080611/135448d0/attachment.html 

From rainer.blome at gmx.de  Thu Jun 12 19:34:22 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 12 Jun 2008 21:34:22 +0200
Subject: [search-ui] i18n, l10n [Was: Re: [Search-l] May 20, 2008 Update]
In-Reply-To: <485003D1.8000906@wikia.com>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>
	<485003D1.8000906@wikia.com>
Message-ID: <48517A3E.90108@gmx.de>

Jimmy Wales wrote:
> But we need help translating the string tables... Dan is looking to 
> coordinate people who are eager to help with this.

The project has a lot on its plate.  Limiting the effects of abuse 
should be at least as high on the list as localization.

That said, it does not look like http://svn.swlabs.org/re.search/cool/
has been internationalized (but my Javascript is dusty and I might be 
mistaken).  Can you use help there?

As for localization, I can help with the German one.

Rainer

From jeremie at jabber.org  Thu Jun 12 19:41:37 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Thu, 12 Jun 2008 14:41:37 -0500
Subject: [search-ui] i18n, l10n [Was: Re: [Search-l] May 20, 2008 Update]
In-Reply-To: <48517A3E.90108@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>
	<485003D1.8000906@wikia.com> <48517A3E.90108@gmx.de>
Message-ID: <1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>

It's definitely a heaping full plate :)

Abuse-related tools are making progress and there's a decent  
understanding of what needs to be done there, but I don't know if we  
can say the same for i18n/l10n.

I think the "general" idea is to use the wiki to manage a list of  
translated strings, and build javascript stringpacks that the  
interface loads depending on what language you choose.  It's quite a  
bit of work and the interface is still in quite a bit of flux, so it  
might be hard to nail this process down to remain stable enough to  
build on...  That said, maybe just identifying a few major places in  
the interface that will be stable and should be translated at a  
minimum to make it useful, would be a good place to start?

What kind of localization things should we consider beyond  
translations, anything major/structural? (I definitely don't have  
enough experience here to know)

Thanks,

Jer


On Jun 12, 2008, at 2:34 PM, Rainer Blome wrote:

> Jimmy Wales wrote:
>> But we need help translating the string tables... Dan is looking to
>> coordinate people who are eager to help with this.
>
> The project has a lot on its plate.  Limiting the effects of abuse
> should be at least as high on the list as localization.
>
> That said, it does not look like http://svn.swlabs.org/re.search/cool/
> has been internationalized (but my Javascript is dusty and I might be
> mistaken).  Can you use help there?
>
> As for localization, I can help with the German one.
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>


From rainer.blome at gmx.de  Thu Jun 12 20:54:15 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 12 Jun 2008 22:54:15 +0200
Subject: [search-ui] i18n, l10n [Was: Re: [Search-l] May 20, 2008 Update]
In-Reply-To: <1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>
	<48517A3E.90108@gmx.de>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
Message-ID: <48518CF7.6040307@gmx.de>

Jeremie Miller wrote:
> Abuse-related tools are making progress and there's a decent  
> understanding of what needs to be done there, 

That's good to hear.

> but I don't know if we can say the same for i18n/l10n.

Does this mean that you have no internationalization API of choice yet 
(not that I have one)?

Translation appears to be the main i18n issue for Wikia Search now.
I have experience with i18n only with text translation (in C/C++).

> I think the "general" idea is to use the wiki to manage a list of  
> translated strings, and build javascript stringpacks that the  
> interface loads depending on what language you choose.

Stringpack, yes. Keeping the translation tables in the Wiki, a very 
interesting idea. In JSON format? Anonymous editing would have to be 
disabled, or it would become a "defacement service" :-).

> ...the interface [...must...] remain stable enough to build on.

Change to the internationalized code is a problem only for the 
localizers that need to keep pace, isn't it?
The i18n (instrumentation) itself should not be directly affected.

For translation, you need a text lookup API.
And that API needs to be stable, before the whole of the code is 
instrumented.
For example: text= stringpack.lookup(123, "Recent Changes")

The english text is the default value, returned when a stringpack could 
not be loaded or the text not found in the pack.

Some prefer to use only a numeric id (to save a bit of memory),
but that makes the code hard to read and maintain:
text= stringpack.lookup(123).
What's more is that this makes the system less resilient. As long as the 
default texts are in the code, you don't even need a working string pack.

Others leave out the numeric ID and use the default value as the key:
text= stringpack.lookup("Recent Changes")
This makes it harder to change the text of the default language 
(english, in this case), since the key value must be changed in all 
string packs.  Or an explicit english pack is used, and the english text 
is changed only there. But then the display value might differ from the 
default text, which complicates maintenance.

 > What kind of localization things should we consider beyond
 > translations, anything major/structural? (I definitely don't have
 > enough experience here to know)

What do you mean by "localization things"?  Currency?  Collation?
The timezone might be useful to know.

Rainer

From balinny at gmail.com  Thu Jun 12 21:14:41 2008
From: balinny at gmail.com (Balinny)
Date: Thu, 12 Jun 2008 23:14:41 +0200
Subject: [search-ui] i18n, l10n
In-Reply-To: <1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>
	<48517A3E.90108@gmx.de>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
Message-ID: <485191C1.1010306@gmail.com>

Jeremie Miller wrote:
> What kind of localization things should we consider beyond  
> translations, anything major/structural? (I definitely don't have  
> enough experience here to know)
>
> Thanks,
>
> Jer
Use placeholders, don't hardcode the items order. It is tempting to use
translate("save edit for ") + result_name, but in other languages it may 
be 'for result save edit',
or 'result edit shall be saved'

From rainer.blome at gmx.de  Thu Jun 12 21:54:06 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 12 Jun 2008 23:54:06 +0200
Subject: [search-ui] i18n, l10n
In-Reply-To: <485191C1.1010306@gmail.com>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
	<485191C1.1010306@gmail.com>
Message-ID: <48519AFE.5080508@gmx.de>

Balinny wrote:
> Use placeholders, don't hardcode the items order. It is tempting to use
> translate("save edit for ") + result_name, but in other languages it may 
> be 'for result save edit',
> or 'result edit shall be saved'

Yes, like gettext does: gettext("Hello {0}!", name);

Here's an Apache-style-licensed server-side ECMAScript implementation: 
http://dev.orf.at/trac/jala/browser/trunk/code/I18n.js
whose usage is described here:
http://dev.orf.at/trac/jala/wiki/JalaModules/I18n

I suspect that it would not run in browsers, but the gettext-like API 
looks good to me.

Found it via http://re.search.wikia.com/search.html#javascript%20i18n
-> http://en.wikipedia.org/wiki/Comparison_of_web_application_frameworks
-> Look in that table for those implemented with JavaScript (only four)
-> http://en.wikipedia.org/wiki/Helma_Object_Publisher
-> http://en.wikipedia.org/wiki/Helma_Object_Publisher#External_Links
-> http://dev.orf.at/trac/jala
-> http://dev.orf.at/trac/jala/wiki/JalaModules
-> http://dev.orf.at/trac/jala/wiki/JalaModules#I18n
-> http://dev.orf.at/trac/jala/wiki/JalaModules/I18n

There are bound to be others like this out there.

From rainer.blome at gmx.de  Thu Jun 12 22:20:45 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 13 Jun 2008 00:20:45 +0200
Subject: [search-ui] i18n, l10n
In-Reply-To: <48519AFE.5080508@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>	<485191C1.1010306@gmail.com>
	<48519AFE.5080508@gmx.de>
Message-ID: <4851A13D.2020606@gmx.de>

Rainer Blome wrote:
> Here's an Apache-style-licensed ... ECMAScript implementation [of gettext]:
 > ...
> There are bound to be others like this out there.

Such as Maxime Haineault's MIT-licensed gettext-js:
http://code.google.com/p/gettext-js/source/checkout

That looks more like it should work in browsers.

From david.pean at gmail.com  Fri Jun 13 17:37:40 2008
From: david.pean at gmail.com (David Pean)
Date: Fri, 13 Jun 2008 13:37:40 -0400
Subject: [search-ui] i18n, l10n
Message-ID: <e49dbf0d0806131037s7d31c590v4dbe7f31df522a9f@mail.gmail.com>

I started to do some initial testing on generating i18n javascript
files from the Wiki.   Managing the language text in the wiki seems
like a good way to go, since its already set up for it, can track
revisions etc -- we can even tie in the search site and have a simple
interface for editing messages.  So I tested out english -- If every
Mediawiki message is included, its about 247K in JSON format :(

http://re.search.wikia.com/BEWARE/JSON/Messagesen.js

Obviously we will not need every message here, so I was wondering how
we should handle which messages we want to use in the search
interface.

Should we keep a simple list of messages on its own page on the wiki?
Or just a simple text file on the svn somewhere to parse through?

Dave

From rainer.blome at gmx.de  Fri Jun 13 22:23:24 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Sat, 14 Jun 2008 00:23:24 +0200
Subject: [search-ui] [Search-l] why is Wikia's the only frontend? [Was:
	wrong	questions]
In-Reply-To: <51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>
References: <g31w481mhw.fsf@sa.vix.com>
	<482B4F00.70209@gmx.de>	<482B5545.6090207@gmx.de>
	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>
Message-ID: <4852F35C.9070506@gmx.de>

Jeremie Miller wrote on 15.05.2008 05:07:
> ... "svn co http://svn.swlabs.org/re.search"
Did that now.

> and particularly looking in the "cool" folder
 > then modding/hacking away,
That's what I want to try.

 > the results will load/ work from the html files loaded locally even.

In the browser, I just opened file:///C:/re.search/cool/index.html .
The page loads, of course, but the form's search button leads to
http://re.search.wikia.com/search.html#suchen , while I expected
file:///C:/re.search/cool/search.html#suchen instead.
Did I misunderstand you?

Rainer

From newsmarkie at googlemail.com  Fri Jun 13 22:30:25 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Fri, 13 Jun 2008 23:30:25 +0100
Subject: [search-ui] [Search-l] why is Wikia's the only frontend? [Was:
	wrong questions]
In-Reply-To: <4852F35C.9070506@gmx.de>
References: <g31w481mhw.fsf@sa.vix.com> <482B4F00.70209@gmx.de>
	<482B5545.6090207@gmx.de>
	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>
	<4852F35C.9070506@gmx.de>
Message-ID: <a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>

iirc you need to change the values in /JSON/servertouse.js which will
currently be set to use the wikia settings.

this should solve the problem

regards

mark

On Fri, Jun 13, 2008 at 11:23 PM, Rainer Blome <rainer.blome at gmx.de> wrote:

> Jeremie Miller wrote on 15.05.2008 05:07:
> > ... "svn co http://svn.swlabs.org/re.search"
> Did that now.
>
> > and particularly looking in the "cool" folder
>  > then modding/hacking away,
> That's what I want to try.
>
>  > the results will load/ work from the html files loaded locally even.
>
> In the browser, I just opened file:///C:/re.search/cool/index.html .
> The page loads, of course, but the form's search button leads to
> http://re.search.wikia.com/search.html#suchen , while I expected
> file:///C:/re.search/cool/search.html#suchen instead.
> Did I misunderstand you?
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080613/13a29a13/attachment.html 

From david.pean at gmail.com  Fri Jun 13 22:58:52 2008
From: david.pean at gmail.com (David Pean)
Date: Fri, 13 Jun 2008 18:58:52 -0400
Subject: [search-ui] [Search-l] why is Wikia's the only frontend? [Was:
	wrong questions]
In-Reply-To: <a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>
References: <g31w481mhw.fsf@sa.vix.com> <482B4F00.70209@gmx.de>
	<482B5545.6090207@gmx.de>
	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>
	<4852F35C.9070506@gmx.de>
	<a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>
Message-ID: <e49dbf0d0806131558k6e9a2566icb5e74748793d4f6@mail.gmail.com>

Yep...there are a few paths in there...you'll want to change:

var search_server_to_use = "file:///C://Desktop/SEARCH/";

On Fri, Jun 13, 2008 at 6:30 PM, Mark (Markie)
<newsmarkie at googlemail.com> wrote:
> iirc you need to change the values in /JSON/servertouse.js which will
> currently be set to use the wikia settings.
>
> this should solve the problem
>
> regards
>
> mark
>
> On Fri, Jun 13, 2008 at 11:23 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
>>
>> Jeremie Miller wrote on 15.05.2008 05:07:
>> > ... "svn co http://svn.swlabs.org/re.search"
>> Did that now.
>>
>> > and particularly looking in the "cool" folder
>>  > then modding/hacking away,
>> That's what I want to try.
>>
>>  > the results will load/ work from the html files loaded locally even.
>>
>> In the browser, I just opened file:///C:/re.search/cool/index.html .
>> The page loads, of course, but the form's search button leads to
>> http://re.search.wikia.com/search.html#suchen , while I expected
>> file:///C:/re.search/cool/search.html#suchen instead.
>> Did I misunderstand you?
>>
>> Rainer
>> _______________________________________________
>> Search-UI mailing list
>> Search-UI at wikia.com
>> http://lists.wikia.com/mailman/listinfo/search-ui
>
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
>

From rainer.blome at gmx.de  Fri Jun 13 23:06:53 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Sat, 14 Jun 2008 01:06:53 +0200
Subject: [search-ui] Setting up a test environment
In-Reply-To: <a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>
References: <g31w481mhw.fsf@sa.vix.com>
	<482B4F00.70209@gmx.de>	<482B5545.6090207@gmx.de>	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>	<4852F35C.9070506@gmx.de>
	<a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>
Message-ID: <4852FD8D.9080208@gmx.de>

Mark (Markie) wrote:
> iirc you need to change the values in /JSON/servertouse.js which will 
> currently be set to use the wikia settings.
> 
> this should solve the problem

Has the expected effect, thanks.


From aaron.wright at gmail.com  Sat Jun 14 01:50:43 2008
From: aaron.wright at gmail.com (Aaron Wright)
Date: Fri, 13 Jun 2008 21:50:43 -0400
Subject: [search-ui] Setting up a test environment
In-Reply-To: <4852FD8D.9080208@gmx.de>
References: <g31w481mhw.fsf@sa.vix.com> <482B4F00.70209@gmx.de>
	<482B5545.6090207@gmx.de>
	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>
	<4852F35C.9070506@gmx.de>
	<a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>
	<4852FD8D.9080208@gmx.de>
Message-ID: <d0fd42530806131850g3322d6a7q96620e79ca73c615@mail.gmail.com>

great.  if you have any other questions, give us a shout.  We'll help out.

--Aaron

On Fri, Jun 13, 2008 at 7:06 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> Mark (Markie) wrote:
>> iirc you need to change the values in /JSON/servertouse.js which will
>> currently be set to use the wikia settings.
>>
>> this should solve the problem
>
> Has the expected effect, thanks.
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From rainer.blome at gmx.de  Sun Jun 15 23:06:00 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 16 Jun 2008 01:06:00 +0200
Subject: [search-ui] i18n, l10n
In-Reply-To: <1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>
	<48517A3E.90108@gmx.de>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
Message-ID: <4855A058.7080707@gmx.de>

To freshen my JavaScript (unused for seven years, I think), I translated 
the front page to German (and to French, probably badly so, just so I 
had a different translation table for testing).

index.html loads the new file js/i18n/i18n.js, which currently contains 
both the translation functions and the tables (which should be put in a 
separate file, but I don't know how to "#include" in JavaScript).

The links and texts in index.html I have fitted with ids so that their 
texts can be replaced (via assignment to innerHTML).  I did it this way 
just to see if it could be done without switching to fully generated 
HTML, which would be cleaner (but which I did not dare try at first).

header.js already generates the contents, wrapping the english strings 
in gettext() calls was enough.

Currently, the choice of locale is hard-coded in i18n.js, I haven't been 
able to switch my browser's locale from en-US to something else yet.

Are you interested in patches? If so, in what form?

Rainer

From rainer.blome at gmx.de  Sun Jun 15 23:08:50 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 16 Jun 2008 01:08:50 +0200
Subject: [search-ui] Setting up a test environment
In-Reply-To: <d0fd42530806131850g3322d6a7q96620e79ca73c615@mail.gmail.com>
References: <g31w481mhw.fsf@sa.vix.com>
	<482B4F00.70209@gmx.de>	<482B5545.6090207@gmx.de>	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>	<4852F35C.9070506@gmx.de>	<a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>	<4852FD8D.9080208@gmx.de>
	<d0fd42530806131850g3322d6a7q96620e79ca73c615@mail.gmail.com>
Message-ID: <4855A102.5070506@gmx.de>

Aaron Wright wrote:
> great.  if you have any other questions, give us a shout.  We'll help out.
> 
> --Aaron

The static pages work locally, but searching does not.  Should it?  To 
test internationalization of the result page, the search would have to 
work.  Did you recently test that it works locally?
Rainer


From rainer.blome at gmx.de  Mon Jun 16 22:17:10 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Tue, 17 Jun 2008 00:17:10 +0200
Subject: [search-ui] Setting up a test environment
In-Reply-To: <4855A102.5070506@gmx.de>
References: <g31w481mhw.fsf@sa.vix.com>	<482B4F00.70209@gmx.de>	<482B5545.6090207@gmx.de>	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>	<4852F35C.9070506@gmx.de>	<a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>	<4852FD8D.9080208@gmx.de>	<d0fd42530806131850g3322d6a7q96620e79ca73c615@mail.gmail.com>
	<4855A102.5070506@gmx.de>
Message-ID: <4856E666.5000303@gmx.de>

Rainer Blome wrote:
> The static pages work locally, but searching does not.

With some tweaking, it works now (had to disable the showAds() call, 
haven't seen why yet).

> test internationalization of the result page,

Works with as glitch under IE6, Seamonkey and Firefox2 (the "add result" 
area has moved to the very bottom, don't know why yet).

Rainer

From aaron.wright at gmail.com  Mon Jun 16 23:31:27 2008
From: aaron.wright at gmail.com (Aaron Wright)
Date: Mon, 16 Jun 2008 19:31:27 -0400
Subject: [search-ui] Setting up a test environment
In-Reply-To: <4856E666.5000303@gmx.de>
References: <g31w481mhw.fsf@sa.vix.com> <482B4F00.70209@gmx.de>
	<482B5545.6090207@gmx.de>
	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>
	<4852F35C.9070506@gmx.de>
	<a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>
	<4852FD8D.9080208@gmx.de>
	<d0fd42530806131850g3322d6a7q96620e79ca73c615@mail.gmail.com>
	<4855A102.5070506@gmx.de> <4856E666.5000303@gmx.de>
Message-ID: <d0fd42530806161631n66114868n1b5d59e1f50d02b7@mail.gmail.com>

Hey Rainer -

I updated search.html in the svn; if you just update the svn and
change the paths in server to use to local paths, you should be good
to go without the need for tweaks and without the glitches in IE6.

--Aaron

On Mon, Jun 16, 2008 at 6:17 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> Rainer Blome wrote:
>> The static pages work locally, but searching does not.
>
> With some tweaking, it works now (had to disable the showAds() call,
> haven't seen why yet).
>
>> test internationalization of the result page,
>
> Works with as glitch under IE6, Seamonkey and Firefox2 (the "add result"
> area has moved to the very bottom, don't know why yet).
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From rainer.blome at gmx.de  Tue Jun 17 08:00:57 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Tue, 17 Jun 2008 10:00:57 +0200
Subject: [search-ui] Setting up a test environment
In-Reply-To: <d0fd42530806161631n66114868n1b5d59e1f50d02b7@mail.gmail.com>
References: <g31w481mhw.fsf@sa.vix.com>
	<482B4F00.70209@gmx.de>	<482B5545.6090207@gmx.de>	<51EEB276-E1E9-479B-A967-92312020A15D@jabber.org>	<4852F35C.9070506@gmx.de>	<a6424d870806131530p739dfbfj62456ab4cf58d756@mail.gmail.com>	<4852FD8D.9080208@gmx.de>	<d0fd42530806131850g3322d6a7q96620e79ca73c615@mail.gmail.com>	<4855A102.5070506@gmx.de>
	<4856E666.5000303@gmx.de>
	<d0fd42530806161631n66114868n1b5d59e1f50d02b7@mail.gmail.com>
Message-ID: <48576F39.8040204@gmx.de>

Aaron Wright wrote:
 > I updated search.html in the svn;

That fixed the "Add result" box (had one little conflict, but easy to 
resolve), and it now works with the showAds() call.

Now that the result history is visible, I can see more text to translate :).

 > if you just update the svn
I assume that SVN reports updates with "U", as CVS does, right?

So, again, are you interested in patches, and if so, in what form?

Rainer

From rainer.blome at gmx.de  Tue Jun 17 08:21:29 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Tue, 17 Jun 2008 10:21:29 +0200
Subject: [search-ui] Use of Google and other third-party sites
Message-ID: <48577409.3090000@gmx.de>

I see with regret that some basic open source components (prototype.js, 
scriptaculous.js) are now loaded from a Google site instead of from 
Wikia. Why has this been done?

Every time someone uses Wikia Search, Google can now potentially see 
this.  This conflicts with your privacy goal.

Yes, I could patch the JavaScript to use local files, but would prefer 
it if the path was relative, so that the files are loaded from wherever 
the Wikia-specific files are loaded, like it was last week.

Along the same line, the uses of Google ads conflicts with the privacy 
goal.  Wikia may not record the search terms, but the Google 
advertisements service may well do.  Can the ads be switched off at the 
user's option?

While I'm at it, is the freebase.com extra turned on by default?  It 
looks like it is, but I never noticed any links being offered.

In general, access to third-party sites should be made transparent 
(documented), optional, and ideally opt-in.

Rainer

From balinny at gmail.com  Tue Jun 17 10:34:41 2008
From: balinny at gmail.com (Balinny)
Date: Tue, 17 Jun 2008 12:34:41 +0200
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <48577409.3090000@gmx.de>
References: <48577409.3090000@gmx.de>
Message-ID: <48579341.9020009@gmail.com>

Rainer Blome wrote:
> I see with regret that some basic open source components (prototype.js, 
> scriptaculous.js) are now loaded from a Google site instead of from 
> Wikia. Why has this been done?
>
> Every time someone uses Wikia Search, Google can now potentially see 
> this.  This conflicts with your privacy goal.
>
> Along the same line, the uses of Google ads conflicts with the privacy 
> goal.  Wikia may not record the search terms, but the Google 
> advertisements service may well do.  Can the ads be switched off at the 
> user's option?
>
> In general, access to third-party sites should be made transparent 
> (documented), optional, and ideally opt-in.
>
> Rainer
>   
These are bad news
*scriptaculous is fetch from ajax.googleapis.com:
prototype.js, builder.js, effects.js, dragdop.js, controls.js, 
slider.js, sound.js...
It should be mirrored locally.

*But it's much worse that search.js is adding Google ads, as it's also 
loaded
http://www.google.com/afsonline/show_afs_ads.js
http://www.google.com/custom...

I have checked and Google is able to retrieve your search terms (they're 
sent to google to
provide adequate ads).
So i think this is a bad move. Also, there's no option to disable them 
in your preferences
(only slightly better).

 From about.html:


    Privacy

A searcher's privacy must be protected and respected, on both a 
technological and social level.

As we increasingly turn to the Web for answers to all of our problems 
and concerns, we as searchers need to be assured that the details of our 
private lives - medical questions, financial advice, a job search, etc. 
- are not subject to scrutiny by others.

Unfortunately, the current Web climate is running in the opposite 
direction...


And thus Wikia Search is improving it by using its own index... and 
giving the search to one of the major search engines!!



From newsmarkie at googlemail.com  Tue Jun 17 15:24:01 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Tue, 17 Jun 2008 16:24:01 +0100
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <48579341.9020009@gmail.com>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
Message-ID: <a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>

I TOTALLY agree with both these issues and have raised them previously.  re
the scripts, that was raised both in the forums (
http://search.wikia.com/wiki/Forum:Googleapis.com) and straight to the
devs.  they kinda fixed the issue and they now load from
re.search.wikia.comon the main page, but then go to
googleapis.com on search.html, maybe additional fixes needed :-p

 the google ads have also been raised in the forum (
http://search.wikia.com/wiki/Forum:Google_Ads%3F) and i was told by dan that
he would get me a reply to what i said there.  i havent been on the/his case
recently :-p (due to exams) but am disappointed to still see these on there
and no explanation

i have copied dan in, who is the main wikia contact for these kind of
issues, so hopefully we could get his/wikia's input

many thanks

mark

On Tue, Jun 17, 2008 at 11:34 AM, Balinny <balinny at gmail.com> wrote:

> Rainer Blome wrote:
> > I see with regret that some basic open source components (prototype.js,
> > scriptaculous.js) are now loaded from a Google site instead of from
> > Wikia. Why has this been done?
> >
> > Every time someone uses Wikia Search, Google can now potentially see
> > this.  This conflicts with your privacy goal.
> >
> > Along the same line, the uses of Google ads conflicts with the privacy
> > goal.  Wikia may not record the search terms, but the Google
> > advertisements service may well do.  Can the ads be switched off at the
> > user's option?
> >
> > In general, access to third-party sites should be made transparent
> > (documented), optional, and ideally opt-in.
> >
> > Rainer
> >
> These are bad news
> *scriptaculous is fetch from ajax.googleapis.com:
> prototype.js, builder.js, effects.js, dragdop.js, controls.js,
> slider.js, sound.js...
> It should be mirrored locally.
>
> *But it's much worse that search.js is adding Google ads, as it's also
> loaded
> http://www.google.com/afsonline/show_afs_ads.js
> http://www.google.com/custom...
>
> I have checked and Google is able to retrieve your search terms (they're
> sent to google to
> provide adequate ads).
> So i think this is a bad move. Also, there's no option to disable them
> in your preferences
> (only slightly better).
>
>  From about.html:
>
>
>    Privacy
>
> A searcher's privacy must be protected and respected, on both a
> technological and social level.
>
> As we increasingly turn to the Web for answers to all of our problems
> and concerns, we as searchers need to be assured that the details of our
> private lives - medical questions, financial advice, a job search, etc.
> - are not subject to scrutiny by others.
>
> Unfortunately, the current Web climate is running in the opposite
> direction...
>
>
> And thus Wikia Search is improving it by using its own index... and
> giving the search to one of the major search engines!!
>
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080617/b52b2a71/attachment.html 

From newsmarkie at googlemail.com  Tue Jun 17 15:45:28 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Tue, 17 Jun 2008 16:45:28 +0100
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
Message-ID: <a6424d870806170845tbb6a7d8wf463c92a5e177ba3@mail.gmail.com>

whoops forgot to cc dan in :-p

mark

On Tue, Jun 17, 2008 at 4:24 PM, Mark (Markie) <newsmarkie at googlemail.com>
wrote:

> I TOTALLY agree with both these issues and have raised them previously.  re
> the scripts, that was raised both in the forums (
> http://search.wikia.com/wiki/Forum:Googleapis.com) and straight to the
> devs.  they kinda fixed the issue and they now load from
> re.search.wikia.com on the main page, but then go to googleapis.com on
> search.html, maybe additional fixes needed :-p
>
>  the google ads have also been raised in the forum (
> http://search.wikia.com/wiki/Forum:Google_Ads%3F) and i was told by dan
> that he would get me a reply to what i said there.  i havent been on the/his
> case recently :-p (due to exams) but am disappointed to still see these on
> there and no explanation
>
> i have copied dan in, who is the main wikia contact for these kind of
> issues, so hopefully we could get his/wikia's input
>
> many thanks
>
> mark
>
>
> On Tue, Jun 17, 2008 at 11:34 AM, Balinny <balinny at gmail.com> wrote:
>
>> Rainer Blome wrote:
>> > I see with regret that some basic open source components (prototype.js,
>> > scriptaculous.js) are now loaded from a Google site instead of from
>> > Wikia. Why has this been done?
>> >
>> > Every time someone uses Wikia Search, Google can now potentially see
>> > this.  This conflicts with your privacy goal.
>> >
>> > Along the same line, the uses of Google ads conflicts with the privacy
>> > goal.  Wikia may not record the search terms, but the Google
>> > advertisements service may well do.  Can the ads be switched off at the
>> > user's option?
>> >
>> > In general, access to third-party sites should be made transparent
>> > (documented), optional, and ideally opt-in.
>> >
>> > Rainer
>> >
>> These are bad news
>> *scriptaculous is fetch from ajax.googleapis.com:
>> prototype.js, builder.js, effects.js, dragdop.js, controls.js,
>> slider.js, sound.js...
>> It should be mirrored locally.
>>
>> *But it's much worse that search.js is adding Google ads, as it's also
>> loaded
>> http://www.google.com/afsonline/show_afs_ads.js
>> http://www.google.com/custom...
>>
>> I have checked and Google is able to retrieve your search terms (they're
>> sent to google to
>> provide adequate ads).
>> So i think this is a bad move. Also, there's no option to disable them
>> in your preferences
>> (only slightly better).
>>
>>  From about.html:
>>
>>
>>    Privacy
>>
>> A searcher's privacy must be protected and respected, on both a
>> technological and social level.
>>
>> As we increasingly turn to the Web for answers to all of our problems
>> and concerns, we as searchers need to be assured that the details of our
>> private lives - medical questions, financial advice, a job search, etc.
>> - are not subject to scrutiny by others.
>>
>> Unfortunately, the current Web climate is running in the opposite
>> direction...
>>
>>
>> And thus Wikia Search is improving it by using its own index... and
>> giving the search to one of the major search engines!!
>>
>>
>> _______________________________________________
>> Search-UI mailing list
>> Search-UI at wikia.com
>> http://lists.wikia.com/mailman/listinfo/search-ui
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080617/f99360d6/attachment-0001.html 

From jeremie at jabber.org  Tue Jun 17 16:05:36 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Tue, 17 Jun 2008 11:05:36 -0500
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
Message-ID: <E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>

There are two separate issues here, one is loading remote scripts, and  
the other is remote "services" that may be able to track/learn more  
about the searcher.

The former, how we're using googleapis.com to load prototype/ 
scriptaculous, doesn't share any searcher information or have any  
context on the page (the search terms are *not* passed), it's simply  
using google's file-serving api to provide those common resources that  
many sites use, and ultimately should be cached and far faster for  
everyone to load them.  It's pretty trivial to use local copies, svn  
has copies in it even, it was simply a way to try and make the loading  
of the results faster.

The second and more challenging issue is the remote services, adwords,  
freebase, wikipedia's api, flickr, and others in the future.  We've  
discussed running all of these and anything that gets passed the  
search terms through a proxy, so that they cannot track based on IP or  
cookie and only see the service requests.  This is possible to do but  
both fairly daunting technically and could actually slow down things  
(when we're trying to make them faster).  We've been investing our  
time into fixing other pretty glaring bugs while these larger issues  
are looming above.

I like the idea of making these options, it'd be interesting to say "I  
trust [x] Google, [x] Wikipedia, [x] Flickr,..." and so on, and then  
selectively load the un-trusted ones via an anonymizing proxy  
(slightly slower would be the penalty).  Again, pretty challenging,  
but overall this is something I think everyone is definitely painfully  
aware of and wanting to figure out soon :)

Jer

On Jun 17, 2008, at 10:24 AM, Mark (Markie) wrote:

> I TOTALLY agree with both these issues and have raised them  
> previously.  re the scripts, that was raised both in the forums (http://search.wikia.com/wiki/Forum:Googleapis.com 
> ) and straight to the devs.  they kinda fixed the issue and they now  
> load from re.search.wikia.com on the main page, but then go to  
> googleapis.com on search.html, maybe additional fixes needed :-p
>
>  the google ads have also been raised in the forum (http://search.wikia.com/wiki/Forum:Google_Ads%3F 
> ) and i was told by dan that he would get me a reply to what i said  
> there.  i havent been on the/his case recently :-p (due to exams)  
> but am disappointed to still see these on there and no explanation
>
> i have copied dan in, who is the main wikia contact for these kind  
> of issues, so hopefully we could get his/wikia's input
>
> many thanks
>
> mark
>
> On Tue, Jun 17, 2008 at 11:34 AM, Balinny <balinny at gmail.com> wrote:
> Rainer Blome wrote:
> > I see with regret that some basic open source components  
> (prototype.js,
> > scriptaculous.js) are now loaded from a Google site instead of from
> > Wikia. Why has this been done?
> >
> > Every time someone uses Wikia Search, Google can now potentially see
> > this.  This conflicts with your privacy goal.
> >
> > Along the same line, the uses of Google ads conflicts with the  
> privacy
> > goal.  Wikia may not record the search terms, but the Google
> > advertisements service may well do.  Can the ads be switched off  
> at the
> > user's option?
> >
> > In general, access to third-party sites should be made transparent
> > (documented), optional, and ideally opt-in.
> >
> > Rainer
> >
> These are bad news
> *scriptaculous is fetch from ajax.googleapis.com:
> prototype.js, builder.js, effects.js, dragdop.js, controls.js,
> slider.js, sound.js...
> It should be mirrored locally.
>
> *But it's much worse that search.js is adding Google ads, as it's also
> loaded
> http://www.google.com/afsonline/show_afs_ads.js
> http://www.google.com/custom...
>
> I have checked and Google is able to retrieve your search terms  
> (they're
> sent to google to
> provide adequate ads).
> So i think this is a bad move. Also, there's no option to disable them
> in your preferences
> (only slightly better).
>
>  From about.html:
>
>
>    Privacy
>
> A searcher's privacy must be protected and respected, on both a
> technological and social level.
>
> As we increasingly turn to the Web for answers to all of our problems
> and concerns, we as searchers need to be assured that the details of  
> our
> private lives - medical questions, financial advice, a job search,  
> etc.
> - are not subject to scrutiny by others.
>
> Unfortunately, the current Web climate is running in the opposite
> direction...
>
>
> And thus Wikia Search is improving it by using its own index... and
> giving the search to one of the major search engines!!
>
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui


From balinny at gmail.com  Tue Jun 17 22:17:36 2008
From: balinny at gmail.com (Balinny)
Date: Wed, 18 Jun 2008 00:17:36 +0200
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
References: <48577409.3090000@gmx.de>
	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
Message-ID: <48583800.4010305@gmail.com>

Jeremie Miller wrote:
> There are two separate issues here, one is loading remote scripts, and  
> the other is remote "services" that may be able to track/learn more  
> about the searcher.
>
> The former, how we're using googleapis.com to load prototype/ 
> scriptaculous, doesn't share any searcher information or have any  
> context on the page (the search terms are *not* passed), 
<paranoic mode="on">
Right, but google should not even know how much people is using wikia 
search, much less
being able to inject javascript on a downsample of its users.
</paranoic>


> I like the idea of making these options, it'd be interesting to say "I  
> trust [x] Google, [x] Wikipedia, [x] Flickr,..." and so on, and then  
> selectively load the un-trusted ones via an anonymizing proxy  
> (slightly slower would be the penalty).  Again, pretty challenging,  
> but overall this is something I think everyone is definitely painfully  
> aware of and wanting to figure out soon :)
>
> Jer
>   

Why would you need to trust into wikipedia or flick? I'd expect Search 
Wikia to have them on its index.
I think you only need to trust into other crawlers (Google, Yahoo, MSN 
Search...) for which
the current system of click here to see their results seems quite good. 
If i follow that link, i'm
allowing it to view my ip and search terms. On the future, that will 
need to be customizable, but
we don't have so many for now.


From rainer.blome at gmx.de  Wed Jun 18 08:32:56 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Wed, 18 Jun 2008 10:32:56 +0200
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
References: <48577409.3090000@gmx.de>
	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
Message-ID: <4858C838.80101@gmx.de>

Jeremie Miller wrote:
 > There are two separate issues here

At least two, so I'll answer in different mails.

 > ... how we're using googleapis.com to load prototype/
 > scriptaculous, doesn't share any searcher information

It does "share searcher information", in my view.  It sure doesn't 
provide the account id or search term, but that's not what I meant.

It provides at the very least the IP address and the time that requests 
are made.  From IP address and latencies, the geographical location can 
be narrowed down surprisingly well (if done right, which certain 
organisations certainly have the resources to do, if they want to).  The 
time tells you something about activity patterns, if you're willing to 
analyze these. For files of this size, it should also be possible for 
the server to infer the user's connection bandwidth.  This in turn is an 
indicator for the economical or social resources available to the user. 
Yes, I'm being deliberately paranoid here.

For the average user, the HTTP request also provides operating system 
and browser type and version (in the User-Agent header), languages 
spoken (Accept-Languages header) and of course the requesting page 
(re.search.wikia.com/search.html, in this case).

Furthermore, organisations such as Google can correlate information 
obtained from all sites of their own, be it File Servers (GoogleAPIs), 
Search, Ads, Maps, Blogs, Mail, Video (YouTube), Groups, GIS (Google 
Earth alias Keyhole), or, not least, traffic analysis (Google Analytics 
alias Urchin).  A staggering combination, if you think about it.  From 
this perspective, re.search using GoogleAPIs can add more precision to 
Google's potential picture of the user.  It would be nice if Google 
ruled out this type of correlation in their privacy policy, but they do 
the converse: "We may combine the information you submit under your 
account with information from other Google services or third parties".

All this taken together is a lot of "searcher information" in my view.
That's what I meant when I wrote "Every time someone uses Wikia Search, 
Google can now potentially see this.".

Rainer

From rainer.blome at gmx.de  Wed Jun 18 09:34:32 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Wed, 18 Jun 2008 11:34:32 +0200
Subject: [search-ui] Speeding up re.search
In-Reply-To: <E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
References: <48577409.3090000@gmx.de>
	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
Message-ID: <4858D6A8.9050700@gmx.de>

Issues discussed in this mail:

o What can be done to speed up re.search?

o What should be done to speed up re.search?

o Does loading GoogleAPIs scripts help to speed up re.search?

o What are the costs of the first query, in terms of both number
   of requests and amount of data?
   What are the costs of a subsequent query, in terms of both
   requests and data?

o How many requests does a single use of re.search actually need,
   as opposed to actually use?

o Bonus: Analysis of the requests used by an actual query.

Rainer


---

o What can be done to speed up re.search?

Want faster results?  Issue less requests and ask for less data. :-)
In other words, reduce the number of default features.


o What should be done to speed up re.search?

Switch off by default or drop entirely the features with the worst 
cost-benefit ratio.  These are, in my opinion: favicons, annotation 
images, freebase and automatic search for related search terms, in that 
order.  Why these? See the comparison table below under "What are the 
costs...?", and the detailed analysis still further down for my 
feature-specific opinions.

These features do not only use bandwidth for the requests that implement 
the features at the protocol level.  They also require code, which 
requires bandwidth of its own when the code is not found in the browser 
cache.  If the code can be partitioned so that optional features are 
loaded on demand, bandwidth consumption can be further reduced.

Switch off the worst two or three of these features in terms of 
cost-benefit ratio, and you're down to one third of the original amount 
of requests and data.


o Does loading GoogleAPIs scripts help to speed up re.search?

Jeremie Miller wrote:
 > using googleapis.com to load prototype/ scriptaculous, ...,
 > was simply a way to try and make the loading of the results faster.

The difference in question is between accessing a search.wikia.com URL 
and a googleapis.com URL.  Your argument is that a cache miss is less 
likely for a Google URL, do I understand you correctly?  It would be 
interesting to see a measurement of these likelihoods.

Even if the difference in likelihood is significant, would it make a 
noticeable difference?  That depends on the user's bandwidth, the file 
sizes, and what else is going on when using re.search, like network 
latencies and time to run the scripts themselves.

The libraries' size is about 250kB.  For a user on 768kbit DSL, loading 
these takes about 3 seconds, no big issue there.
For a user on 56kbit dial-up, loading these libraries should take about 
40 seconds.  So, if you're on dial-up, use a large cache ;-).  These 40 
seconds would be a nice saving, if the cache hit rate is indeed 
noticeably higher for a Google URL than for a Wikia URL.

Another question to ask is whether these libraries are needed in their 
entirety.


o What are the costs of the first query, in terms of both number
   of requests and amount of data?
   What are the costs of a subsequent query, in terms of both
   requests and data?

Some re.search features can be identified with a particular type of 
request, for example "Mini Article" request or "annotation image" 
request.  The following table shows for each request type the total 
sizes of all responses and the numbers of requests (column "n").  It 
does this for two queries, "test" with an empty cache and then "wikia", 
with the cache having been filled with the static files.  To get the 
data, I used the Burp Proxy to watch traffic.  I analyzed the log by 
visual inspection and manual calculation, so there will be some errors 
in the numbers.  The table is ordered by the sum of the sizes for the 
two queries.

"test"   "wikia"
Size  n  Size  n Object Type
---------------------------------
  350  4   113 15 annotation images
  354 20     0  0 static content (html, js, img)
  250  8     0  0 third-party libraries
   50 24   120 26 favicons
   87  6    45  8 index and kt requests for related terms
   35  3    69  3 index and kt requests for search term
   25  4     9  4 Freebase
   11  4     3  3 Ads
    1  1     1  1 notifications
    1  1     1  1 Wikipedia
    1  1     1  1 Mini Article
    1  1     1  1 user block check
----------------------------------
1166 77   363 63 total

Looking at a total of 1.2 MB for the first query (counting only 
downstream), we have here a search engine that strongly favors 
broad-band users (1.2MB at 56kbit takes about 3 minutes).


o How many requests does a single use of re.search need?

The first use needs about 80 requests, in the example shown below.
Subsequent queries for the same term, assuming that caching is 
effective, need about 50 requests.

How many of these requests are really needed, when no cached data is 
available?  I'd say about 17.  21, if ads are included.

How many of these are really needed when caching is effective?
I'd say about seven.  Nine, if ads are included (not that I want them).


---
Detailed analysis of a worst-case scenario follows.

--
Situation: Firefox 2, fresh start, clean cache, no favicons,
  no persistent cookies, session cookies allowed (my default setup).

--
Accessing the front page re.search.wikia.com:

No, it is not necessary to go to the front page to issue a search
request, but let's have a look anyway, just to get a feel for it:

http://re.search.wikia.com/
http://re.search.wikia.com/kt_files/structure.css
http://re.search.wikia.com/kt_files/style.css
http://re.search.wikia.com/kt_files/index.css
http://re.search.wikia.com/js/prototype.js
http://re.search.wikia.com/js/scriptaculous.js
http://re.search.wikia.com/js/builder.js
http://re.search.wikia.com/js/effects.js
http://re.search.wikia.com/js/dragdrop.js
http://re.search.wikia.com/js/controls.js
http://re.search.wikia.com/js/slider.js
http://re.search.wikia.com/js/sound.js
http://re.search.wikia.com/js/util.js
http://re.search.wikia.com/JSON/servertouse.js
http://re.search.wikia.com/JSON/serverCall.js
http://re.search.wikia.com/JSON/login.js
http://re.search.wikia.com/header.js
http://re.search.wikia.com/kt_files/front-logo.png
http://search.wikia.com/index.php?action=ajax&rs=wfgetNotificationsJSON
http://re.search.wikia.com/kt_files/browser_search_ff2_win.png
http://search.wikia.com/index.php?action=ajax&rs=wfCheckNewNotificationsJSON
------------
21 requests.

Eight of these are for third-party scripts (prototype.js to
sound.js).  Do they matter?  Maybe, as long as they have not been
cached.  They could be combined into one and minimized, if
bandwidth is a concern.  Personally, I think the page loads
quickly, in less than five seconds.

--
Further instances of the notification request keep on being issued
every few seconds:

http://search.wikia.com/index.php?action=ajax&rs=wfCheckNewNotificationsJSON

What are these used for?  I'm not logged in.

--
Now for the search proper

Typing "test" in the search form and hitting "Search":
(any "#test" suffixes in the URLs have not been logged)

Buckle up, goggles on!                 (and scroll down)

http://re.search.wikia.com/search.html
http://sb.google.com/safebrowsing/update?client=navclient-auto-ffox&appver=2.0.0.14&version=goog-white-domain:1:480,goog-white-url:1:371,goog-black-url:1:22312,goog-black-enchash:1:53182
http://re.search.wikia.com/kt_files/search.css?10
http://ajax.googleapis.com/ajax/libs/prototype/1.6.0.2/prototype.js
http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.1/scriptaculous.js
http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.1/builder.js
http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.1/effects.js
http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.1/dragdrop.js
http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.1/controls.js
http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.1/slider.js
http://ajax.googleapis.com/ajax/libs/scriptaculous/1.8.1/sound.js
http://re.search.wikia.com/search.js?8
http://kt.search.isc.org/kttest/now.js?f=itsNow(&r=0.9561252224630629
http://re.search.wikia.com/header.js?9
http://search.wikia.com/index.php?action=ajax&rs=wfgetNotificationsJSON
http://re.search.wikia.com/ads.html
http://re.search.wikia.com/kt_files/edit.jpg
http://www.google.com/afsonline/show_afs_ads.js
http://www.google.com/custom?client=pub-4086838842346968&sa=Search&safe=on&channel=&alternate_ad_url=&cof=DIV%3AFFFFFF%3BBGC%3AFFFFFF%3BLC%3A0000CC%3BVLC%3A0000CC%3BALC%3A0000CC%3BT%3A000000%3BGALT%3A008000%3BFORID%3A4&adskip=0
http://re.search.wikia.com/ads.html?q=test
http://search.wikia.com/index.php?action=ajax&rs=wfcheckBlocksJSON&r=0.5718468320186741
http://re.search.wikia.com/kt_files/lblue.png
http://index.search.isc.org/nutchsearch?query=test%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0
http://kt.search.isc.org/kttest/kt.js?t=del&t=add&t=sug&t=spot&t=stars&t=edit&t=bg&t=com&t=selection&t=also&k=test&r=0.9352628394972157&f=processKT(
http://search.wikia.com/index.php?action=ajax&rs=wfGetArticleJSON&rsargs%5B%5D=Mini:test&rsargs%5B%5D=maRender&rsargs%5B%5D=1&rsargs%5B%5D=100
http://en.wikipedia.org/w/api.php?action=query&list=search&srwhat=text&srsearch=test&format=json&callback=wpRender
http://www.freebase.com/api/service/search?limit=3&callback=fbRender&prefix=test
http://www.google.com/custom?client=pub-4086838842346968&q=test&sa=Search&safe=on&channel=&alternate_ad_url=&cof=DIV%3AFFFFFF%3BBGC%3AFFFFFF%3BLC%3A0000CC%3BVLC%3A0000CC%3BALC%3A0000CC%3BT%3A000000%3BGALT%3A008000%3BFORID%3A4&adskip=0
http://index.search.isc.org/nutchsearch?query=test%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=10
http://www.freebase.com/api/service/mqlread?callback=wpfb&queries=%7B%22wp%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%7D%5D%2C%20%22%2Fcommon%2Ftopic%2Fimage%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22a%3Akey%22%3A%20%22Test_cricket%22%7D%5D%7D%7D
http://re.search.wikia.com/do/favicon/google.com/
http://index.search.isc.org/nutchsearch?query=java%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0
http://kt.search.isc.org/kttest/kt.js?t=del&t=add&t=sug&t=spot&t=stars&t=edit&t=bg&t=com&t=selection&t=also&k=java&r=0.5902580851328586&f=processKT(
http://index.search.isc.org/nutchsearch?query=IQ%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0
http://kt.search.isc.org/kttest/kt.js?t=del&t=add&t=sug&t=spot&t=stars&t=edit&t=bg&t=com&t=selection&t=also&k=iq&r=0.003843534732156151&f=processKT(
http://index.search.isc.org/nutchsearch?query=sat%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0
http://kt.search.isc.org/kttest/kt.js?t=del&t=add&t=sug&t=spot&t=stars&t=edit&t=bg&t=com&t=selection&t=also&k=sat&r=0.39648440404580454&f=processKT(
http://www.freebase.com/api/service/mqlread?callback=fbBack&queries=%7B%22a0%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22guid%22%3A%20%22%239202a8c04000641f800000000003c943%22%2C%20%22name%22%3A%20null%2C%20%22type%22%3A%20%22%2Fcommon%2Ftopic%22%2C%20%22webpage%22%3A%20%5B%7B%22uri%22%3A%20null%7D%5D%7D%5D%7D%2C%20%22a1%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22guid%22%3A%20%22%239202a8c04000641f8000000000318346%22%2C%20%22name%22%3A%20null%2C%20%22type%22%3A%20%22%2Fcommon%2Ftopic%22%2C%20%22webpage%22%3A%20%5B%7B%22uri%22%3A%20null%7D%5D%7D%5D%7D%2C%20%22a2%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22guid%22%3A%20%22%239202a8c04000641f8000000003cb0288%22%2C%20%22name%22%3A%20null%2
C%20%22type%22%3A%20%22%2Fcommon%2Ftopic%22%2C%20%22webpage%22%3A%20%5B%7B%22uri%22%3A%20null%7D%5D%7D%5D%7D%7D
http://www.freebase.com/api/trans/blurb/guid/9202a8c04000641f800000000003c949?maxlength=600&callback=wpsum
http://re.search.wikia.com/do/favicon/search.yahoo.com/
http://re.search.wikia.com/do/favicon/celdt.cde.ca.gov
http://re.search.wikia.com/do/favicon/star.cde.ca.gov
http://re.search.wikia.com/do/favicon/www.rertr.anl.gov
http://re.search.wikia.com/do/favicon/www.vehicletest.state.ma.us
http://re.search.wikia.com/kt_files/star_on.gif
http://www.rertr.anl.gov/images/RERTRnukeATR2.jpg
http://www.test.de/filestore/1678889.jpg?path=/24/61/695d018c-702d-4e38-9d37-c53a115660dd-jpg.jpg&key=3DADA1A776D959BFA3AF8660606A112D2CC56207
http://z.about.com/d/pregnancy/1/8/0/c/3/iStock_000004752386XSmall.jpg
http://www.esdim.noaa.gov/apache_pb.gif
http://re.search.wikia.com/do/favicon/wsd.nlm.nih.gov
http://re.search.wikia.com/do/favicon/en.wikipedia.org
http://images.wikia.com/search/images/photos/static/e78/71.240.162.37-s.jpg
http://images.wikia.com/search/images/photos/static/d8f/66.114.51.22-s.jpg
http://re.search.wikia.com/kt_files/feed.png
http://www.rertr.anl.gov/images/cpd.gif
http://z.about.com/d/pregnancy/1/0/m/a/3/07bruni211st.jpg
http://re.search.wikia.com/kt_files/star_off.gif
http://www.ftwtv.com/gfx/upload/media_many_1212500485.jpg
http://www.ftwtv.com/gfx/upload/media_1132.jpg
http://www.ftwtv.com/gfx/upload/media_1257.jpg
http://www.ftwtv.com/gfx/upload/media_many_1207857375.jpg
http://www.ftwtv.com/gfx/upload/media_many_1212453014.jpg
http://www.ftwtv.com/gfx/upload/media_1166.jpg
http://www.freebase.com/api/trans/image_thumb/wikipedia/images/commons_id/8840340
http://re.search.wikia.com/do/favicon/java.about.com
http://re.search.wikia.com/do/favicon/rex.nci.nih.gov
http://re.search.wikia.com/do/favicon/rucsoundings.noaa.gov
http://re.search.wikia.com/do/favicon/worldwind.arc.nasa.gov
http://re.search.wikia.com/do/favicon/www.iq.harvard.edu
http://re.search.wikia.com/do/favicon/children.webmd.com
http://re.search.wikia.com/do/favicon/www.fashioniq.com
http://re.search.wikia.com/do/favicon/iqweb.moc.edu
http://re.search.wikia.com/do/favicon/testprep.about.com
http://re.search.wikia.com/do/favicon/www.kbnp.com
http://re.search.wikia.com/do/favicon/blogworldexpo.com
http://re.search.wikia.com/do/favicon/java.sun.com
http://images.wikia.com/search/images/photos/default_s.png
http://re.search.wikia.com/do/favicon/www.collegeboard.com
http://re.search.wikia.com/do/favicon/www.cabinet.iq
http://farm2.static.flickr.com/1381/753839600_b378c962b9_t.jpg
http://re.search.wikia.com/kt_files/lgreen.png

---
Wow, 81 requests, by my line count.

[On a subsequent request with a different search term, with effective 
caching, about 30 requests less would be sent.  The point is that the 
remaining 50 are still a lot].

To put this into perspective, even given the enormous number of
requests, the results are displayed after about eight seconds (*without* 
the proxy, on a broadband connection in Europe).  Loading appears to 
stop after about 30 seconds.  When shift-reloading the page, it appears 
to stop loading after about ten seconds.

Let's break the above requests down by request purpose.

Favicons

The most frequent request type seems to be the favicon, with a
count of 22.  Who really needs these?  What is the benefit?  These
logos do not help much to judge results.
Make this opt-in, and you have only 59 requests left.

Annotation images

There are about 15 images loaded beyond Wikia's own ones.  It is
possible that using "test" as the query will not give a typical
result, because someone added multiple annotation images to a
single result (which is even "deleted").  However, I assume that
happens for any popular search term, so it should be not too far
off.  Make annotation image loading opt-in and you have only 44
requests left.

Third-party Scripts

Eight requests are for third-party scripts (prototype.js to
sound.js).  Do they matter?  Rarely so. In many people's browser
session, only the first batch of URL accesses would be served by
the URL target, subsequent ones would be served from the browser
cache.  In the long run, the same URLs will be used for the .js
files in both index.html and search.html, so that in the scenario
here, there would have been no requests counted for these files.
We are now down to 36 requests.

Images

Seven images from re.search.wikia.com and images.wikia.com, used as
background colors (lblue.png, lgreen.png) and symbols (edit.jpg,
feed.png, star_on.gif, star_off.gif, default_s.png) on the results
page.  Maybe the color PNGs might be replaced by style sheet
definitions.  default_s.png could be replaced by a white rectangle.
edit.jpg could be replaced by text, the way it was before
(localized text, in the future).  For now, I'll count them as core
functionality, not discounting any requests.

Freebase

There are five requests to www.freebase.com. One of these is huge,
1100 bytes in its URL-encoded form.  Whatever these
requests do, I haven't noticed it yet.  If the purpose is to see
whether freebase has anything to say about the query, doesn't one
short request suffice?
We are now down to 31 requests.

index.search.isc.org

http://index.search.isc.org/nutchsearch?query=test%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0
http://index.search.isc.org/nutchsearch?query=test%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=10
http://index.search.isc.org/nutchsearch?query=java%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0
http://index.search.isc.org/nutchsearch?query=IQ%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0
http://index.search.isc.org/nutchsearch?query=sat%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0

The first one is the core functionality, "give me ten results for
'test'".  The second one is probably done because more than ten
results fit in my window, so it's core functionality, too.  The
last three are made for related search terms.  This functionality
looks unfinished, the queries should be for "test+sat" instead of
just "sat", "test+iq" instead of just "iq" etc., shouldn't they?
These queries could be done on demand.  Doing them automatically
should be opt-in or at least opt-out.  Discounting 3 makes 28.

kt.search.isc.org

The first does what?:

http://kt.search.isc.org/kttest/now.js?f=itsNow(&r=0.9561252224630629

The later four access meta-information and "decoration" of the
results (the star rating, annotations, etc.), right?
Discounting three (for java, iq and sat) leaves 25.


HTML and style sheets

Four requests to re.search.wikia.com, for search.html,
kt_files/search.css, ads.html and ads.html?q=test... a second time?
Why?  Discounting the two ads.html-requests requests leaves 23.


Google Ads

Three requests to www.google.com appear to implement Google ads.
At least one of them transmits the search query.
Discount them all, arriving at 19.


ECMAScript

Two scripts from re.search.wikia.com, header.js and search.js.
Apparently the individual files (main.js, ui.js etc.) have combined
batched into a single one?  These perform the magic of the user
interface.  These are necessary, and it makes sense to keep them
separate.


Blocking

One request to check whether the use can save (independent of
the query).  Is the block restricted to the mini articles or a
general write block?


Mini Article

One request to the search.wikia wiki, for a mini article with a
title like the query.


Another Notifications Request...

...mixed in between the other requests.  Discounting one, leaving 18.


Google "Safe Browsing"

http://sb.google.com/safebrowsing/update?client=navclient-auto-ffox&appver=2.0.0.14&version=goog-white-domain:1:480,goog-white-url:1:371,goog-black-url:1:22312,goog-black-enchash:1:53182

http://www.google.com/support/firefox/bin/topic.py?topic=11815
tells me that "Safe Browsing" is an anti-phishing feature, built
into Firefox 2.  This request probably downloads a blacklist.  This
is done maybe once per browser session? No, not for every session.
An interesting topic on its own, another opt-out I at least did not
know about. Let's turn that off in the browser's options.
Gee, is there no end?  But I digress.  In any case,
it's not done for every search request.  Leaves 17.


Wikipedia

Similar to the freebase requests, Wikipedia is always searched,
transmitting the query to Wikipedia.  For Wikipedia, only one
request is used to implement this, nice.


For this scenario, the functionality that I like can thus be implemented 
with about 17 requests, plus or minus a few. With ads, it would still be 
only 21.

Rainer


From balinny at gmail.com  Wed Jun 18 13:19:32 2008
From: balinny at gmail.com (Balinny)
Date: Wed, 18 Jun 2008 15:19:32 +0200
Subject: [search-ui] Speeding up re.search
In-Reply-To: <4858D6A8.9050700@gmx.de>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<4858D6A8.9050700@gmx.de>
Message-ID: <48590B64.3050709@gmail.com>

Rainer Blome wrote:
> o What should be done to speed up re.search?
>
> Switch off by default or drop entirely the features with the worst 
> cost-benefit ratio.  These are, in my opinion: favicons,
Favicons can't be dropped. Some browsers try to *always* fetch them, 
even if they don't
have a reason to think it exists. Waht you can do is place a great 
expiry time. Now it's quite
good, 6 days.

> If the code can be partitioned so that optional features are 
> loaded on demand, bandwidth consumption can be further reduced.
>
>   
That would be the best way :)

>
> o Does loading GoogleAPIs scripts help to speed up re.search?
>
> Jeremie Miller wrote:
>  > using googleapis.com to load prototype/ scriptaculous, ...,
>  > was simply a way to try and make the loading of the results faster.
>
> The difference in question is between accessing a search.wikia.com URL 
> and a googleapis.com URL.  Your argument is that a cache miss is less 
> likely for a Google URL, do I understand you correctly?  It would be 
> interesting to see a measurement of these likelihoods.
>
> Even if the difference in likelihood is significant, would it make a 
> noticeable difference?  That depends on the user's bandwidth, the file 
> sizes, and what else is going on when using re.search, like network 
> latencies and time to run the scripts themselves.
>   
I don't think it's so noticeable. If there's some difference, i see more 
likely it's due to being
far from swlabs servers, but nearer to a google server (i think it uses 
akamai).

I would optimize the library load by joining it on a single file 
(they're 7 http requests!) and
"compressing" it.

> o What are the costs of the first query, in terms of both number
>    of requests and amount of data?
>    What are the costs of a subsequent query, in terms of both
>    requests and data?

I copied you and also took some measures :)
Note that other browsers may have a diffrent number of parallel connections.
The sized are the amount of data downloaded, headers included.

Main page:
1 redirect if you go to http://search.wikia.com/
http://re.search.wikia.com/    Compressed, 1189 bytes
http://re.search.wikia.com/favicon.ico    Uncompressed. Would compress 
very well.
http://re.search.wikia.com/js/builder.js Compressed. Same connection as 
favicon, 13747 bytes total
http://re.search.wikia.com/kt_files/structure.css    Compressed, 4012 bytes
http://re.search.wikia.com/kt_files/index.css    Uncompressed!
http://re.search.wikia.com/js/scriptaculous.js    Compressed. Same 
connection as index.css, 2970 bytes total
http://re.search.wikia.com/kt_files/style.css    Compressed, 2023 bytes
http://re.search.wikia.com/js/prototype.js    Compressed, 29125 bytes
http://re.search.wikia.com/js/effects.js        Compressed, 9435 bytes
http://re.search.wikia.com/js/dragdrop.js        Compressed, 8134 bytes
http://re.search.wikia.com/js/controls.js        Compressed, 9504 bytes
http://re.search.wikia.com/js/slider.js        Compressed, 3106 bytes
http://re.search.wikia.com/js/sound.js        Compressed, 1299 bytes
http://re.search.wikia.com/js/util.js        Compressed, 4059 bytes
http://re.search.wikia.com/JSON/servertouse.js    Uncompressed. I 
recommend placing this inline.
http://re.search.wikia.com/JSON/servercall.js    Uncompressed, same 
connection as servertouse. What's it's purpose? Seem to be trying to 
*avoid* caching
http://re.search.wikia.com/JSON/login.js        Compressed, same 
connection as servertouse. Total size 4485 bytes.
http://re.search.wikia.com/header.js        Compressed, 5151 bytes
http://search.wikia.com/index.php?action=ajax&rs=wfgetNotificationsJSON    
Compressed.
http://search.wikia.com/index.php?action=ajax&rs=wfCheckNewNotificationsJSON 
x 9 times    Compressed. Same connection as wfgetNotificationsJSON. 
Total size 7923. Could the server headers be reduced? Only the Via 
header is bigger than the content. The X-* headers are also redundant. 
Plus it's mime is not text/html as stated. Using a zero-size reply 
instead of "//no new notifications" would allow to get rid of headers 
Content-Encoding, Vary and X-Vary-Options. And you would avoid the 
overheard of the compression (the body is 42 bytes, for sending a 22 
bytes comment). Total size 7923 bytes
/kt_files/front-logo.png    I think pngcrush would do a good work on it.
/kt_files/browser_search_ff3_win.png    Same connection as 
front-logo.png Total size 18940


Total download size: 125102 bytes = 122 Kb
Without search anything yet.



Now, i look for 'Wikia':
http://re.search.wikia.com/search.html    Compressed, 2014 bytes
http://re.search.wikia.com/kt_files/search.css?10 Compressed, 5221 bytes
http://ajax.googleapis.com/ajax/libs/prototype/1.6.0.2/prototype.js    
Compressed, 64411 bytes
http://re.search.wikia.com/search.js?8    Compressed, 26153 bytes
http://kt.search.isc.org/kttest/now.js?f=itsNow(&r=0.5439882331330094    
Uncompressed, why not sending it inline?
http://kt.search.isc.org/kttest/kt.js?t=del&t=add&t=sug&t=spot&t=stars&t=edit&t=bg&t=com&t=selection&t=also&k=wikia&r=0.30684323004272973&f=processKT(    
Same connection. Uncompressed while it should be (52089 bytes). Total 
size 52767
http://re.search.wikia.com/header.js?9    Compressed, 5105 bytes. The 
same file was cached for the main page!!
http://search.wikia.com/index.php?action=ajax&rs=wfgetNotificationsJSON   
http://search.wikia.com/index.php?action=ajax&rs=wfcheckBlocksJSON&r=0.6886680380373874    
Same connection as wfgetNotificationsJSON Total size 1793. Note that 
wfcheckBlocksJSON is providing the ip and time, which is precisely the 
purpose of now.js! Instead of using a random number, better use a unix 
timestamp rounding up the last digit.
http://re.search.wikia.com/ads.html    Uncompressed!
http://re.search.wikia.com/ads.html?q=Wikia    Same connection, same 
uncompressed file. One query isn't needed.
http://re.search.wikia.com/kt_files/star_off.gif    Same connection, 
uncompressed, wouldn't compress too bad.
http://re.search.wikia.com/do/favicon/es.starcraft.wikia.com    Same 
connection, uncompressed, would compress very well. Total size 14630 bytes.
http://re.search.wikia.com/kt_files/edit.jpg    Uncompressed.
http://re.search.wikia.com/kt_files/lblue.png    Same connection.
http://re.search.wikia.com/do/favicon/es.wikipedia.org    Same connection.
http://re.search.wikia.comdo/favicon/es.elderscrolls.wikia.com    Same 
connection.
http://re.search.wikia.com/kt_files/lgreen.png    Same connection, total 
size 21949
http://www.google.com/afsonline/show_afs_ads.js    Compressed, size 1570
http://search.wikia.com/index.php?action=ajax&rs=wfGetArticleJSON&rsargs%5B%5D=Mini:Wikia&rsargs%5B%5D=maRender&rsargs%5B%5D=1&rsargs%5B%5D=100    
Compressed, 949 bytes
http://search.wikia.com/do/favicon/google.com/    Uncompressed, would 
compress well.
http://search.wikia.com/do/favicon/en.blogging.wikia.com    Same 
connection. Uncompressed, would compress well. Total size 12840
http://search.wikia.com/do/favicon/search.yahoo.com/    Uncompressed, 
Would compress well. 1424 bytes.
http://en.wikipedia.org/w/api.php?action=query&list=search&srwhat=text&srsearch=Wikia&format=json&callback=wpRender    

http://index.search.isc.org/nutchsearch?query=Wikia%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=0    
Compressed
http://index.search.isc.org/nutchsearch?query=Wikia%20lang%3Aen&hitsPerSite=1&lang=en&hitsPerPage=10&type=json&start=10 
Same connection, compressed. Total size 5901 bytes
http://www.google.com/afsonline/show_afs_ads.js     Compressed
http://www.google.com/custom?client=pub-4086838842346968&q=Wikia&sa=Search&safe=on&channel=&alternate_ad_url=&cof=DIV%3AFFFFFF%3BBGC%3AFFFFFF%3BLC%3A0000CC%3BVLC%3A0000CC%3BALC%3A0000CC%3BT%3A000000%3BGALT%3A008000%3BFORID%3A4&adskip=0    
Same connection, compressed. Total size 2474 bytes
http://www.freebase.com/api/service/search?limit=3&callback=fbRender&prefix=Wikia    
Compressed.
http://www.freebase.com/api/service/mqlread?callback=fbBack&queries=%7B%22a0%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22guid%22%3A%20%22%239202a8c04000641f8000000000315964%22%2C%20%22name%22%3A%20null%2C%20%22type%22%3A%20%22%2Fcommon%2Ftopic%22%2C%20%22webpage%22%3A%20%5B%7B%22uri%22%3A%20null%7D%5D%7D%5D%7D%2C%20%22a1%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22guid%22%3A%20%22%239202a8c04000641f800000000776ad83%22%2C%20%22name%22%3A%20null%2C%20%22type%22%3A%20%22%2Fcommon%2Ftopic%22%2C%20%22webpage%22%3A%20%5B%7B%22uri%22%3A%20null%7D%5D%7D%5D%7D%2C%20%22a2%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22guid%22%3A%20%22%239202a8c04000641f8000000006f6f405%22%2C%20%22name%22%3A%20null%2C%20%22type%22%3A%20%22%2Fcommon%2Ftopic%22%2C%20%22webpage%22%3A%20%5B%7B%22uri%22%3A%20null%7D%5D%7D%5D%7D%7D    
Same connection, compressed body smaller than query string.
http://www.freebase.com/api/trans/blurb/guid/9202a8c04000641f8000000006f6f409?maxlength=600&callback=wpsum    
Same connection, compressed. Total size 4161 bytes
 http://www.freebase.com/api/service/mqlread?callback=wpfb&queries=%7B%22wp%22%3A%20%7B%22query%22%3A%20%5B%7B%22%2Fcommon%2Ftopic%2Farticle%22%3A%20%5B%7B%22id%22%3A%20null%7D%5D%2C%20%22%2Fcommon%2Ftopic%2Fimage%22%3A%20%5B%7B%22id%22%3A%20null%2C%20%22optional%22%3A%20true%7D%5D%2C%20%22a%3Akey%22%3A%20%22Wikia%22%7D%5D%7D%7D    
Compressed
http://www.freebase.com/api/trans/image_thumb/guid/9202a8c04000641f8000000004a74fb7    
Same connection, uncompressed. Total size 9526 bytes

http://re.search.wikia.com/do/favicon/videojuego.wikia.com    
Uncompressed, would compress very well.
http://re.search.wikia.com/kt_files/feed.png    Same connection, 
uncompressed. Suggest pngcrunch.
http://re.search.wikia.com/kt_files/star_voted.gif    Same connection, 
uncompressed. Would compress well. Total size 13475 bytes
http://www.freebase.com/api/trans/blurb/guid/9202a8c04000641f800000000031596b?maxlength=600&callback=wpsum    
Compressed, 1260 bytes
http://images.wikia.com/search/images/photos/static/c54/Angela-s.jpg    
Uncompressed. 812 bytes
http://images.wikia.com/search/images/photos/static/... <myip>-s.jpg    
Uncompressed 404. There's no point in looking for the picture for an anon!
http://images.wikia.com/search/images/photos/default_s.png    Same 
connection. Uncompressed. Total size 1556 bytes
http://images4.wikia.nocookie.net/central/images/archive/b/bf/20071011194317%21Wiki_wide.png    
Uncompressed, 4665 bytes

+5 dns queries and answers

Total download size 254656 bytes = 248Kb



From rainer.blome at gmx.de  Wed Jun 18 21:36:26 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Wed, 18 Jun 2008 23:36:26 +0200
Subject: [search-ui] Speeding up re.search
In-Reply-To: <48590B64.3050709@gmail.com>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<4858D6A8.9050700@gmx.de>
	<48590B64.3050709@gmail.com>
Message-ID: <48597FDA.5040606@gmx.de>

Balinny wrote:
> Rainer Blome wrote:
>> Switch off by default or drop entirely the features with the worst 
>> cost-benefit ratio.  These are, in my opinion: favicons,
> Favicons can't be dropped. Some browsers try to *always* fetch them, 
> even if they don't have a reason to think it exists.

The ones that you mean are fetched from the target sites of bookmarks, 
for example worldwind.arc.nasa.gov, right?

re.search fetches the favicons I mean from re.search.wikia.com, for example
http://re.search.wikia.com/do/favicon/worldwind.arc.nasa.gov .
For the "test" query, 22 of these requests are made.
re.search.wikia.com is in effect working as a proxy server for these 
favicons.

> I would optimize the library load by ... "compressing" it.
Looks good.  What server did you use?


From jeremie at jabber.org  Wed Jun 18 22:02:24 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Wed, 18 Jun 2008 17:02:24 -0500
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <4858C838.80101@gmx.de>
References: <48577409.3090000@gmx.de>
	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<4858C838.80101@gmx.de>
Message-ID: <CB4DE307-0694-4918-92AD-3935C9B69B6E@jabber.org>

Definitely see your point, I really don't think Google is doing  
anything this evil (at least it's 'evil' IMO) but for the minor shared- 
resource improvements it's really not worth it.  It was more a  
deployment convenience decision during the flurry of new-version- 
launch updates, and I agree and think we should switch back to using  
our own local copies.

It all sits behind the (most awesome) Panther CDN anyway, so it should  
be highly cached anyway.

Jer

On Jun 18, 2008, at 3:32 AM, Rainer Blome wrote:

> Jeremie Miller wrote:
>> There are two separate issues here
>
> At least two, so I'll answer in different mails.
>
>> ... how we're using googleapis.com to load prototype/
>> scriptaculous, doesn't share any searcher information
>
> It does "share searcher information", in my view.  It sure doesn't
> provide the account id or search term, but that's not what I meant.
>
> It provides at the very least the IP address and the time that  
> requests
> are made.  From IP address and latencies, the geographical location  
> can
> be narrowed down surprisingly well (if done right, which certain
> organisations certainly have the resources to do, if they want to).   
> The
> time tells you something about activity patterns, if you're willing to
> analyze these. For files of this size, it should also be possible for
> the server to infer the user's connection bandwidth.  This in turn  
> is an
> indicator for the economical or social resources available to the  
> user.
> Yes, I'm being deliberately paranoid here.
>
> For the average user, the HTTP request also provides operating system
> and browser type and version (in the User-Agent header), languages
> spoken (Accept-Languages header) and of course the requesting page
> (re.search.wikia.com/search.html, in this case).
>
> Furthermore, organisations such as Google can correlate information
> obtained from all sites of their own, be it File Servers (GoogleAPIs),
> Search, Ads, Maps, Blogs, Mail, Video (YouTube), Groups, GIS (Google
> Earth alias Keyhole), or, not least, traffic analysis (Google  
> Analytics
> alias Urchin).  A staggering combination, if you think about it.  From
> this perspective, re.search using GoogleAPIs can add more precision to
> Google's potential picture of the user.  It would be nice if Google
> ruled out this type of correlation in their privacy policy, but they  
> do
> the converse: "We may combine the information you submit under your
> account with information from other Google services or third parties".
>
> All this taken together is a lot of "searcher information" in my view.
> That's what I meant when I wrote "Every time someone uses Wikia  
> Search,
> Google can now potentially see this.".
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>


From newsmarkie at googlemail.com  Wed Jun 18 22:06:49 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Wed, 18 Jun 2008 23:06:49 +0100
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <CB4DE307-0694-4918-92AD-3935C9B69B6E@jabber.org>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<4858C838.80101@gmx.de>
	<CB4DE307-0694-4918-92AD-3935C9B69B6E@jabber.org>
Message-ID: <a6424d870806181506p48f2c08dlbc94c0257dc0f32b@mail.gmail.com>

i dont think thats its really a case of being evil (whether they are or not
:-p) but more a case of privacy being one of the key/unique (selling?)
points of wikia search, and therefore things like this and other things
(google ads) really are against the key pillars on which the project stand.
in the case of this im sure it was an honest mistake, which has already been
partially recitified.

also panther rocks :-p

regards

mark

On Wed, Jun 18, 2008 at 11:02 PM, Jeremie Miller <jeremie at jabber.org> wrote:

> Definitely see your point, I really don't think Google is doing
> anything this evil (at least it's 'evil' IMO) but for the minor shared-
> resource improvements it's really not worth it.  It was more a
> deployment convenience decision during the flurry of new-version-
> launch updates, and I agree and think we should switch back to using
> our own local copies.
>
> It all sits behind the (most awesome) Panther CDN anyway, so it should
> be highly cached anyway.
>
> Jer
>
> On Jun 18, 2008, at 3:32 AM, Rainer Blome wrote:
>
> > Jeremie Miller wrote:
> >> There are two separate issues here
> >
> > At least two, so I'll answer in different mails.
> >
> >> ... how we're using googleapis.com to load prototype/
> >> scriptaculous, doesn't share any searcher information
> >
> > It does "share searcher information", in my view.  It sure doesn't
> > provide the account id or search term, but that's not what I meant.
> >
> > It provides at the very least the IP address and the time that
> > requests
> > are made.  From IP address and latencies, the geographical location
> > can
> > be narrowed down surprisingly well (if done right, which certain
> > organisations certainly have the resources to do, if they want to).
> > The
> > time tells you something about activity patterns, if you're willing to
> > analyze these. For files of this size, it should also be possible for
> > the server to infer the user's connection bandwidth.  This in turn
> > is an
> > indicator for the economical or social resources available to the
> > user.
> > Yes, I'm being deliberately paranoid here.
> >
> > For the average user, the HTTP request also provides operating system
> > and browser type and version (in the User-Agent header), languages
> > spoken (Accept-Languages header) and of course the requesting page
> > (re.search.wikia.com/search.html, in this case).
> >
> > Furthermore, organisations such as Google can correlate information
> > obtained from all sites of their own, be it File Servers (GoogleAPIs),
> > Search, Ads, Maps, Blogs, Mail, Video (YouTube), Groups, GIS (Google
> > Earth alias Keyhole), or, not least, traffic analysis (Google
> > Analytics
> > alias Urchin).  A staggering combination, if you think about it.  From
> > this perspective, re.search using GoogleAPIs can add more precision to
> > Google's potential picture of the user.  It would be nice if Google
> > ruled out this type of correlation in their privacy policy, but they
> > do
> > the converse: "We may combine the information you submit under your
> > account with information from other Google services or third parties".
> >
> > All this taken together is a lot of "searcher information" in my view.
> > That's what I meant when I wrote "Every time someone uses Wikia
> > Search,
> > Google can now potentially see this.".
> >
> > Rainer
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
> >
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080618/d79ca34e/attachment-0001.html 

From newsmarkie at googlemail.com  Wed Jun 18 22:21:13 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Wed, 18 Jun 2008 23:21:13 +0100
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
Message-ID: <a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>

hmmm this does seem to be an answer of sorts to some of the questions, and i
agree with some of what is put below.  however what i fail to see is an
obvious and blatant answer to how google ads are acceptable on wikia search
in their current form.

on http://re.search.wikia.com/legal/privacy.html im not seeing anywhere that
covers and says that "we will pass your data (which can personally identify
you) onto other third parties".  overall i think the use of advertising is
inevitable, and as i said before i would even be bold enough to support the
current and maybe more subtle adverts (ie text only non banners/popups etc)
on the project if it helps to pay for the development costs and help the
project in future.  however in their currrent state, passing data onto
google and going against the privacy policy, which is one of the four key
pillars of the project is NOT acceptable.

is there likely to be any change, either led by or accepted by wikia in
terms of the adverts that are hosted and how they are hosted, or is this an
issue that you would like to quietly die away and be swept under the carpet?

regards

mark

On Tue, Jun 17, 2008 at 5:05 PM, Jeremie Miller <jeremie at jabber.org> wrote:

> There are two separate issues here, one is loading remote scripts, and
> the other is remote "services" that may be able to track/learn more
> about the searcher.
>
> The former, how we're using googleapis.com to load prototype/
> scriptaculous, doesn't share any searcher information or have any
> context on the page (the search terms are *not* passed), it's simply
> using google's file-serving api to provide those common resources that
> many sites use, and ultimately should be cached and far faster for
> everyone to load them.  It's pretty trivial to use local copies, svn
> has copies in it even, it was simply a way to try and make the loading
> of the results faster.
>
> The second and more challenging issue is the remote services, adwords,
> freebase, wikipedia's api, flickr, and others in the future.  We've
> discussed running all of these and anything that gets passed the
> search terms through a proxy, so that they cannot track based on IP or
> cookie and only see the service requests.  This is possible to do but
> both fairly daunting technically and could actually slow down things
> (when we're trying to make them faster).  We've been investing our
> time into fixing other pretty glaring bugs while these larger issues
> are looming above.
>
> I like the idea of making these options, it'd be interesting to say "I
> trust [x] Google, [x] Wikipedia, [x] Flickr,..." and so on, and then
> selectively load the un-trusted ones via an anonymizing proxy
> (slightly slower would be the penalty).  Again, pretty challenging,
> but overall this is something I think everyone is definitely painfully
> aware of and wanting to figure out soon :)
>
> Jer
>
> On Jun 17, 2008, at 10:24 AM, Mark (Markie) wrote:
>
> > I TOTALLY agree with both these issues and have raised them
> > previously.  re the scripts, that was raised both in the forums (
> http://search.wikia.com/wiki/Forum:Googleapis.com
> > ) and straight to the devs.  they kinda fixed the issue and they now
> > load from re.search.wikia.com on the main page, but then go to
> > googleapis.com on search.html, maybe additional fixes needed :-p
> >
> >  the google ads have also been raised in the forum (
> http://search.wikia.com/wiki/Forum:Google_Ads%3F
> > ) and i was told by dan that he would get me a reply to what i said
> > there.  i havent been on the/his case recently :-p (due to exams)
> > but am disappointed to still see these on there and no explanation
> >
> > i have copied dan in, who is the main wikia contact for these kind
> > of issues, so hopefully we could get his/wikia's input
> >
> > many thanks
> >
> > mark
> >
> > On Tue, Jun 17, 2008 at 11:34 AM, Balinny <balinny at gmail.com> wrote:
> > Rainer Blome wrote:
> > > I see with regret that some basic open source components
> > (prototype.js,
> > > scriptaculous.js) are now loaded from a Google site instead of from
> > > Wikia. Why has this been done?
> > >
> > > Every time someone uses Wikia Search, Google can now potentially see
> > > this.  This conflicts with your privacy goal.
> > >
> > > Along the same line, the uses of Google ads conflicts with the
> > privacy
> > > goal.  Wikia may not record the search terms, but the Google
> > > advertisements service may well do.  Can the ads be switched off
> > at the
> > > user's option?
> > >
> > > In general, access to third-party sites should be made transparent
> > > (documented), optional, and ideally opt-in.
> > >
> > > Rainer
> > >
> > These are bad news
> > *scriptaculous is fetch from ajax.googleapis.com:
> > prototype.js, builder.js, effects.js, dragdop.js, controls.js,
> > slider.js, sound.js...
> > It should be mirrored locally.
> >
> > *But it's much worse that search.js is adding Google ads, as it's also
> > loaded
> > http://www.google.com/afsonline/show_afs_ads.js
> > http://www.google.com/custom...
> >
> > I have checked and Google is able to retrieve your search terms
> > (they're
> > sent to google to
> > provide adequate ads).
> > So i think this is a bad move. Also, there's no option to disable them
> > in your preferences
> > (only slightly better).
> >
> >  From about.html:
> >
> >
> >    Privacy
> >
> > A searcher's privacy must be protected and respected, on both a
> > technological and social level.
> >
> > As we increasingly turn to the Web for answers to all of our problems
> > and concerns, we as searchers need to be assured that the details of
> > our
> > private lives - medical questions, financial advice, a job search,
> > etc.
> > - are not subject to scrutiny by others.
> >
> > Unfortunately, the current Web climate is running in the opposite
> > direction...
> >
> >
> > And thus Wikia Search is improving it by using its own index... and
> > giving the search to one of the major search engines!!
> >
> >
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
> >
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080618/8721a89f/attachment.html 

From jeremie at jabber.org  Wed Jun 18 22:34:40 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Wed, 18 Jun 2008 17:34:40 -0500
Subject: [search-ui] Speeding up re.search
In-Reply-To: <4858D6A8.9050700@gmx.de>
References: <48577409.3090000@gmx.de>
	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<4858D6A8.9050700@gmx.de>
Message-ID: <CCB1CEBA-DF8B-43AF-9250-5FDBA37EE4E9@jabber.org>

Before I respond to some of the details, I want to majorly fork this  
discussion...

The current results interface and almost all of re.search is for  
*research*, it's supposed to be an experimental playground where we  
can try new tools and experiment with new features, give users more  
options to see if they're useful or not.

This has been poorly (or perhaps not at all) communicated, so I'm  
saying it now hoping it's not too late :)

For *production* style discussion where we're really regression  
testing and streamlining things, getting it stable/fast/secure/etc,  
that should be a separate branch or even different repository for that  
interface code.  I also think it's a little too soon to head fully  
down that path, as we only just now learning how people are using all  
the myriad of tools and what ones might be missing/good/bad.

It's going to be messy for a while, lots of different pressures and  
needs from everyone, but that's a healthy process to encourage when  
we're trying to learn and discover and grow.

That said, I *would* like to start to build a streamlined interface, I  
think that's a research project all in it's own in learning how to  
optimize a lot of these things.  Also, the JS code in main.js and  
ui.js is in sore need of refactoring, pulling out the core stable  
functions into something solid and reusable.

Anyway, I'll touch on a few things below (taking into account my  
perspective on this all from above):

> o What can be done to speed up re.search?
>
> Want faster results?  Issue less requests and ask for less data. :-)
> In other words, reduce the number of default features.

We need a new/clean different interface code tree where it starts out  
minimal.

> o What should be done to speed up re.search?
>
> Switch off by default or drop entirely the features with the worst
> cost-benefit ratio.  These are, in my opinion: favicons, annotation
> images, freebase and automatic search for related search terms, in  
> that
> order.  Why these? See the comparison table below under "What are the
> costs...?", and the detailed analysis still further down for my
> feature-specific opinions.

You just listed all the things that my wife loves and the reason she  
is actually using our search :)

Like above, I think this all belongs in a "streamlined" new code  
tree.  Maybe instead of the "cool" folder we should have a "fast"  
folder for building this out?

> ... snip'd ...
> Further instances of the notification request keep on being issued
> every few seconds:
>
> http://search.wikia.com/index.php?action=ajax&rs=wfCheckNewNotificationsJSON
>
> What are these used for?  I'm not logged in.

You can send messages to anon users, it's both an experiment and  
actually quite useful :)

> To put this into perspective, even given the enormous number of
> requests, the results are displayed after about eight seconds  
> (*without*
> the proxy, on a broadband connection in Europe).  Loading appears to
> stop after about 30 seconds.  When shift-reloading the page, it  
> appears
> to stop loading after about ten seconds.

Usually results are *displayed* sub-second here in the US where the  
servers are local, but even still, by using Panther CDN we'd like to  
get most/all of the /display-critical/ requests being cached, which  
means they should serve "local" via the CDN almost anywhere in the  
world.  All of the misc data loads (notification checks, images, some  
of the KT stuff, etc) would keep it from *finishing* but shouldn't  
impact the usability as they load in the background.

> There are five requests to www.freebase.com. One of these is huge,
> 1100 bytes in its URL-encoded form.  Whatever these
> requests do, I haven't noticed it yet.  If the purpose is to see
> whether freebase has anything to say about the query, doesn't one
> short request suffice?

Wikipedia's API doesn't have any way whatsoever to get summary text,  
so it comes from Freebase.  Also, Freebase has a bunch of "direct"  
urls and it asks for any of those, but because of the way their API  
works you have to do multiple queries to get meta-data and again for  
summaries :(

Again, it's an experiment, and I'm not sure if we can draw any  
conclusions about how well the Freebase stuff has worked yet, if it's  
been worth it, dunno :/

> index.search.isc.org:  The
> last three are made for related search terms.  This functionality
> looks unfinished, the queries should be for "test+sat" instead of
> just "sat", "test+iq" instead of just "iq" etc., shouldn't they?
> These queries could be done on demand.  Doing them automatically
> should be opt-in or at least opt-out.  Discounting 3 makes 28.

Yeah, more experimenting.  More useful things should be done with the  
"related" searches to make it worth pulling them in. Again, should all  
be background and not impact result usability :/

> kt.search.isc.org
>
> The first does what?:
>
> http://kt.search.isc.org/kttest/now.js?f=itsNow(&r=0.9561252224630629

We have to get the current time on the server to compare all  
timestamps to, unavoidable, but only needs to happen once and the  
"skew" can be stored in a cookie or something to compare to client  
local time.  It can be greatly optimized by doing that.

I'm looking forward to getting the "fast" folder started on the  
re.search svn, if that sounds good to everyone :)

Jer


From jeremie at jabber.org  Wed Jun 18 22:42:56 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Wed, 18 Jun 2008 17:42:56 -0500
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>
Message-ID: <2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>

Personally I think "advertising" in the context of a search result  
(when it's done right) can be a very *useful* thing and often provides  
good results... I frequently find what I need in the right bar on  
google's results (moreso when I'm actually looking for a product/ 
service).

I'd also personally like to have our own advertising solution, but  
this isn't really the forum or time yet for that discussion :)

We're going to add some controls to let people decide who/what  
services they trust, advertising or supplemental results or widgets or  
otherwise, it's going to be a frequent thing as the web is becoming a  
very intermixed medium, and I know I'd love to experiment with pulling  
in /more/ services, not less, *grin*.

The only thing I want to sweep under the carpet is spam, and with  
people deleting stuff, I've seen some stellar results pages, it's  
really exciting :)

Jer

On Jun 18, 2008, at 5:21 PM, Mark (Markie) wrote:

> hmmm this does seem to be an answer of sorts to some of the  
> questions, and i agree with some of what is put below.  however what  
> i fail to see is an obvious and blatant answer to how google ads are  
> acceptable on wikia search in their current form.
>
> on http://re.search.wikia.com/legal/privacy.html im not seeing  
> anywhere that covers and says that "we will pass your data (which  
> can personally identify you) onto other third parties".  overall i  
> think the use of advertising is inevitable, and as i said before i  
> would even be bold enough to support the current and maybe more  
> subtle adverts (ie text only non banners/popups etc) on the project  
> if it helps to pay for the development costs and help the project in  
> future.  however in their currrent state, passing data onto google  
> and going against the privacy policy, which is one of the four key  
> pillars of the project is NOT acceptable.
>
> is there likely to be any change, either led by or accepted by wikia  
> in terms of the adverts that are hosted and how they are hosted, or  
> is this an issue that you would like to quietly die away and be  
> swept under the carpet?
>
> regards
>
> mark
>
> On Tue, Jun 17, 2008 at 5:05 PM, Jeremie Miller <jeremie at jabber.org>  
> wrote:
> There are two separate issues here, one is loading remote scripts, and
> the other is remote "services" that may be able to track/learn more
> about the searcher.
>
> The former, how we're using googleapis.com to load prototype/
> scriptaculous, doesn't share any searcher information or have any
> context on the page (the search terms are *not* passed), it's simply
> using google's file-serving api to provide those common resources that
> many sites use, and ultimately should be cached and far faster for
> everyone to load them.  It's pretty trivial to use local copies, svn
> has copies in it even, it was simply a way to try and make the loading
> of the results faster.
>
> The second and more challenging issue is the remote services, adwords,
> freebase, wikipedia's api, flickr, and others in the future.  We've
> discussed running all of these and anything that gets passed the
> search terms through a proxy, so that they cannot track based on IP or
> cookie and only see the service requests.  This is possible to do but
> both fairly daunting technically and could actually slow down things
> (when we're trying to make them faster).  We've been investing our
> time into fixing other pretty glaring bugs while these larger issues
> are looming above.
>
> I like the idea of making these options, it'd be interesting to say "I
> trust [x] Google, [x] Wikipedia, [x] Flickr,..." and so on, and then
> selectively load the un-trusted ones via an anonymizing proxy
> (slightly slower would be the penalty).  Again, pretty challenging,
> but overall this is something I think everyone is definitely painfully
> aware of and wanting to figure out soon :)
>
> Jer
>
> On Jun 17, 2008, at 10:24 AM, Mark (Markie) wrote:
>
> > I TOTALLY agree with both these issues and have raised them
> > previously.  re the scripts, that was raised both in the forums (http://search.wikia.com/wiki/Forum:Googleapis.com
> > ) and straight to the devs.  they kinda fixed the issue and they now
> > load from re.search.wikia.com on the main page, but then go to
> > googleapis.com on search.html, maybe additional fixes needed :-p
> >
> >  the google ads have also been raised in the forum (http://search.wikia.com/wiki/Forum:Google_Ads%3F
> > ) and i was told by dan that he would get me a reply to what i said
> > there.  i havent been on the/his case recently :-p (due to exams)
> > but am disappointed to still see these on there and no explanation
> >
> > i have copied dan in, who is the main wikia contact for these kind
> > of issues, so hopefully we could get his/wikia's input
> >
> > many thanks
> >
> > mark
> >
> > On Tue, Jun 17, 2008 at 11:34 AM, Balinny <balinny at gmail.com> wrote:
> > Rainer Blome wrote:
> > > I see with regret that some basic open source components
> > (prototype.js,
> > > scriptaculous.js) are now loaded from a Google site instead of  
> from
> > > Wikia. Why has this been done?
> > >
> > > Every time someone uses Wikia Search, Google can now potentially  
> see
> > > this.  This conflicts with your privacy goal.
> > >
> > > Along the same line, the uses of Google ads conflicts with the
> > privacy
> > > goal.  Wikia may not record the search terms, but the Google
> > > advertisements service may well do.  Can the ads be switched off
> > at the
> > > user's option?
> > >
> > > In general, access to third-party sites should be made transparent
> > > (documented), optional, and ideally opt-in.
> > >
> > > Rainer
> > >
> > These are bad news
> > *scriptaculous is fetch from ajax.googleapis.com:
> > prototype.js, builder.js, effects.js, dragdop.js, controls.js,
> > slider.js, sound.js...
> > It should be mirrored locally.
> >
> > *But it's much worse that search.js is adding Google ads, as it's  
> also
> > loaded
> > http://www.google.com/afsonline/show_afs_ads.js
> > http://www.google.com/custom...
> >
> > I have checked and Google is able to retrieve your search terms
> > (they're
> > sent to google to
> > provide adequate ads).
> > So i think this is a bad move. Also, there's no option to disable  
> them
> > in your preferences
> > (only slightly better).
> >
> >  From about.html:
> >
> >
> >    Privacy
> >
> > A searcher's privacy must be protected and respected, on both a
> > technological and social level.
> >
> > As we increasingly turn to the Web for answers to all of our  
> problems
> > and concerns, we as searchers need to be assured that the details of
> > our
> > private lives - medical questions, financial advice, a job search,
> > etc.
> > - are not subject to scrutiny by others.
> >
> > Unfortunately, the current Web climate is running in the opposite
> > direction...
> >
> >
> > And thus Wikia Search is improving it by using its own index... and
> > giving the search to one of the major search engines!!
> >
> >
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
> >
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui


From newsmarkie at googlemail.com  Wed Jun 18 22:46:11 2008
From: newsmarkie at googlemail.com (Mark (Markie))
Date: Wed, 18 Jun 2008 23:46:11 +0100
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>
	<2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>
Message-ID: <a6424d870806181546v2f359f85v929bb572b9b712d2@mail.gmail.com>

this gunna sound arsey now, and its not meant to so sorry if it does, but
until these new features are there, we/you are going to continue to hand our
information out widly to google?

i have no problem with ads and widgets etc, as long as they are appropriate
to the aims of wikia search in terms of what the project aims to do, in this
case privacy.

also on the spam front, i agree with you there.  also as an aside on this i
plan to have a MAJOR clearout of the white list, which i fear has become a
major bed of spam links etc.

regards

mark

On Wed, Jun 18, 2008 at 11:42 PM, Jeremie Miller <jeremie at jabber.org> wrote:

> Personally I think "advertising" in the context of a search result
> (when it's done right) can be a very *useful* thing and often provides
> good results... I frequently find what I need in the right bar on
> google's results (moreso when I'm actually looking for a product/
> service).
>
> I'd also personally like to have our own advertising solution, but
> this isn't really the forum or time yet for that discussion :)
>
> We're going to add some controls to let people decide who/what
> services they trust, advertising or supplemental results or widgets or
> otherwise, it's going to be a frequent thing as the web is becoming a
> very intermixed medium, and I know I'd love to experiment with pulling
> in /more/ services, not less, *grin*.
>
> The only thing I want to sweep under the carpet is spam, and with
> people deleting stuff, I've seen some stellar results pages, it's
> really exciting :)
>
> Jer
>
> On Jun 18, 2008, at 5:21 PM, Mark (Markie) wrote:
>
> > hmmm this does seem to be an answer of sorts to some of the
> > questions, and i agree with some of what is put below.  however what
> > i fail to see is an obvious and blatant answer to how google ads are
> > acceptable on wikia search in their current form.
> >
> > on http://re.search.wikia.com/legal/privacy.html im not seeing
> > anywhere that covers and says that "we will pass your data (which
> > can personally identify you) onto other third parties".  overall i
> > think the use of advertising is inevitable, and as i said before i
> > would even be bold enough to support the current and maybe more
> > subtle adverts (ie text only non banners/popups etc) on the project
> > if it helps to pay for the development costs and help the project in
> > future.  however in their currrent state, passing data onto google
> > and going against the privacy policy, which is one of the four key
> > pillars of the project is NOT acceptable.
> >
> > is there likely to be any change, either led by or accepted by wikia
> > in terms of the adverts that are hosted and how they are hosted, or
> > is this an issue that you would like to quietly die away and be
> > swept under the carpet?
> >
> > regards
> >
> > mark
> >
> > On Tue, Jun 17, 2008 at 5:05 PM, Jeremie Miller <jeremie at jabber.org>
> > wrote:
> > There are two separate issues here, one is loading remote scripts, and
> > the other is remote "services" that may be able to track/learn more
> > about the searcher.
> >
> > The former, how we're using googleapis.com to load prototype/
> > scriptaculous, doesn't share any searcher information or have any
> > context on the page (the search terms are *not* passed), it's simply
> > using google's file-serving api to provide those common resources that
> > many sites use, and ultimately should be cached and far faster for
> > everyone to load them.  It's pretty trivial to use local copies, svn
> > has copies in it even, it was simply a way to try and make the loading
> > of the results faster.
> >
> > The second and more challenging issue is the remote services, adwords,
> > freebase, wikipedia's api, flickr, and others in the future.  We've
> > discussed running all of these and anything that gets passed the
> > search terms through a proxy, so that they cannot track based on IP or
> > cookie and only see the service requests.  This is possible to do but
> > both fairly daunting technically and could actually slow down things
> > (when we're trying to make them faster).  We've been investing our
> > time into fixing other pretty glaring bugs while these larger issues
> > are looming above.
> >
> > I like the idea of making these options, it'd be interesting to say "I
> > trust [x] Google, [x] Wikipedia, [x] Flickr,..." and so on, and then
> > selectively load the un-trusted ones via an anonymizing proxy
> > (slightly slower would be the penalty).  Again, pretty challenging,
> > but overall this is something I think everyone is definitely painfully
> > aware of and wanting to figure out soon :)
> >
> > Jer
> >
> > On Jun 17, 2008, at 10:24 AM, Mark (Markie) wrote:
> >
> > > I TOTALLY agree with both these issues and have raised them
> > > previously.  re the scripts, that was raised both in the forums (
> http://search.wikia.com/wiki/Forum:Googleapis.com
> > > ) and straight to the devs.  they kinda fixed the issue and they now
> > > load from re.search.wikia.com on the main page, but then go to
> > > googleapis.com on search.html, maybe additional fixes needed :-p
> > >
> > >  the google ads have also been raised in the forum (
> http://search.wikia.com/wiki/Forum:Google_Ads%3F
> > > ) and i was told by dan that he would get me a reply to what i said
> > > there.  i havent been on the/his case recently :-p (due to exams)
> > > but am disappointed to still see these on there and no explanation
> > >
> > > i have copied dan in, who is the main wikia contact for these kind
> > > of issues, so hopefully we could get his/wikia's input
> > >
> > > many thanks
> > >
> > > mark
> > >
> > > On Tue, Jun 17, 2008 at 11:34 AM, Balinny <balinny at gmail.com> wrote:
> > > Rainer Blome wrote:
> > > > I see with regret that some basic open source components
> > > (prototype.js,
> > > > scriptaculous.js) are now loaded from a Google site instead of
> > from
> > > > Wikia. Why has this been done?
> > > >
> > > > Every time someone uses Wikia Search, Google can now potentially
> > see
> > > > this.  This conflicts with your privacy goal.
> > > >
> > > > Along the same line, the uses of Google ads conflicts with the
> > > privacy
> > > > goal.  Wikia may not record the search terms, but the Google
> > > > advertisements service may well do.  Can the ads be switched off
> > > at the
> > > > user's option?
> > > >
> > > > In general, access to third-party sites should be made transparent
> > > > (documented), optional, and ideally opt-in.
> > > >
> > > > Rainer
> > > >
> > > These are bad news
> > > *scriptaculous is fetch from ajax.googleapis.com:
> > > prototype.js, builder.js, effects.js, dragdop.js, controls.js,
> > > slider.js, sound.js...
> > > It should be mirrored locally.
> > >
> > > *But it's much worse that search.js is adding Google ads, as it's
> > also
> > > loaded
> > > http://www.google.com/afsonline/show_afs_ads.js
> > > http://www.google.com/custom...
> > >
> > > I have checked and Google is able to retrieve your search terms
> > > (they're
> > > sent to google to
> > > provide adequate ads).
> > > So i think this is a bad move. Also, there's no option to disable
> > them
> > > in your preferences
> > > (only slightly better).
> > >
> > >  From about.html:
> > >
> > >
> > >    Privacy
> > >
> > > A searcher's privacy must be protected and respected, on both a
> > > technological and social level.
> > >
> > > As we increasingly turn to the Web for answers to all of our
> > problems
> > > and concerns, we as searchers need to be assured that the details of
> > > our
> > > private lives - medical questions, financial advice, a job search,
> > > etc.
> > > - are not subject to scrutiny by others.
> > >
> > > Unfortunately, the current Web climate is running in the opposite
> > > direction...
> > >
> > >
> > > And thus Wikia Search is improving it by using its own index... and
> > > giving the search to one of the major search engines!!
> > >
> > >
> > > _______________________________________________
> > > Search-UI mailing list
> > > Search-UI at wikia.com
> > > http://lists.wikia.com/mailman/listinfo/search-ui
> > >
> > > _______________________________________________
> > > Search-UI mailing list
> > > Search-UI at wikia.com
> > > http://lists.wikia.com/mailman/listinfo/search-ui
> >
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
> >
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080618/bcfe6b8f/attachment.html 

From jeremie at jabber.org  Wed Jun 18 23:05:48 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Wed, 18 Jun 2008 18:05:48 -0500
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <a6424d870806181546v2f359f85v929bb572b9b712d2@mail.gmail.com>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>
	<2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>
	<a6424d870806181546v2f359f85v929bb572b9b712d2@mail.gmail.com>
Message-ID: <75924C73-5D8C-4342-B556-35E59CBC6077@jabber.org>

> this gunna sound arsey now, and its not meant to so sorry if it  
> does, but until these new features are there, we/you are going to  
> continue to hand our information out widly to google?

>
> i have no problem with ads and widgets etc, as long as they are  
> appropriate to the aims of wikia search in terms of what the project  
> aims to do, in this case privacy.

It's more than just the ads "leaking" bits of data, so we definitely  
need to get the "allowed 3rd party" preferences in soon to plug all  
the holes, that's the only real solution, to make sure a framework  
exists so privacy is protected now and in the future as best we can.

That said, we should really put a caveat up somewhere that while the  
pillars of the *project* include privacy, there is a *large* amount of  
very open development going on and if you're playing with any of this  
alpha/beta/testing stuff, things are going to be messy.

I fundamentally believe in privacy through independence and choice,  
not a preference pane on some service.  As long as all the tools and  
services are open and transparent, privacy can be absolutely protected  
since anyone can easily build their own or choose from a plethora of  
providers.  This is a Jabber vs. IM silos perspective though, privacy  
is ultimately the ability to run my own :)

Jer

From rainer.blome at gmx.de  Thu Jun 19 07:37:06 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 19 Jun 2008 09:37:06 +0200
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <CB4DE307-0694-4918-92AD-3935C9B69B6E@jabber.org>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<4858C838.80101@gmx.de>
	<CB4DE307-0694-4918-92AD-3935C9B69B6E@jabber.org>
Message-ID: <485A0CA2.8090205@gmx.de>

Jeremie Miller wrote:
> Definitely see your point, I really don't think Google is doing  
> anything this evil (at least it's 'evil' IMO)

I don't either. Let's not tempt them ;-)  For the record, I find many of 
Google's individual services terrific, especially the lean search 
interface.  It's the combination that I'm wary of.

> It all sits behind the (most awesome) Panther CDN anyway, so it should  
> be highly cached anyway.

What's behind Panther?  re.search.wikia.com?

Rainer

From rainer.blome at gmx.de  Thu Jun 19 08:03:24 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 19 Jun 2008 10:03:24 +0200
Subject: [search-ui] Document re.search features [Was: Re: Use of Google and
 other third-party sites]
In-Reply-To: <2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>
References: <48577409.3090000@gmx.de>
	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>
	<2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>
Message-ID: <485A12CC.6000707@gmx.de>

Jeremie Miller wrote:
> I frequently find what I need in the right bar on  
> google's results (moreso when I'm actually looking for a product/ 
> service).

You're more lucky, then :) I'd say "sometimes" instead of "frequently", 
even when I'm looking for a product.

> I'd love to experiment with pulling in /more/ services, not less, *grin*.

There are so many features in re.search.  Many are easy to see, but some 
are not, and some are invisible (for example the freebase queries).  You 
should at least mention each feature somewhere.
Maybe add a little question mark that links to each non-obvious 
feature's documentation.

For example,
o What do the colored bars underneath the related search terms mean?
o What do the colors mean?
o How are the related search terms chosen?

o For the header background, how are the offered images chosen from flickr?
o Is there a way to see "the flickr page that the image lives on" (not 
sure if this is a unique or well-defined one).   For example, what does 
the header image for the search term "wikia" show?

 > some stellar results pages, it's really exciting :)

Totally agree :)

Rainer

From jeremie at jabber.org  Thu Jun 19 17:26:36 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Thu, 19 Jun 2008 12:26:36 -0500
Subject: [search-ui] Use of Google and other third-party sites
In-Reply-To: <485A0CA2.8090205@gmx.de>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<4858C838.80101@gmx.de>
	<CB4DE307-0694-4918-92AD-3935C9B69B6E@jabber.org>
	<485A0CA2.8090205@gmx.de>
Message-ID: <846BBCBC-5AF5-458C-A2F8-C52DA58D547C@jabber.org>

>> It all sits behind the (most awesome) Panther CDN anyway, so it  
>> should
>> be highly cached anyway.
>
> What's behind Panther?  re.search.wikia.com?

Yep:

jer$ dig re.search.wikia.com | grep CNAME
re.search.wikia.com.	600	IN	CNAME	g1.panthercdn.com.

And so is kt/index.search.isc.org, though the kt stuff isn't  
(currently) using any of the caching features.

Jer


From jeremie at jabber.org  Thu Jun 19 17:33:40 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Thu, 19 Jun 2008 12:33:40 -0500
Subject: [search-ui] Document re.search features [Was: Re: Use of Google
	and other third-party sites]
In-Reply-To: <485A12CC.6000707@gmx.de>
References: <48577409.3090000@gmx.de>
	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>
	<2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>
	<485A12CC.6000707@gmx.de>
Message-ID: <F63ADCC7-A089-4D8E-A0A7-1AB9678D2652@jabber.org>

Most of these are all great things for the help pages, who Dan has  
been instrumental in helping pull together, I think he's on this list  
but I cc'd him anyway :)

And yeah, I agree that it'd be nice to jump "out" to the flickr page,  
not sure if that's easy or not but it'll get added to the list of nice- 
things-to-do.

Jer

On Jun 19, 2008, at 3:03 AM, Rainer Blome wrote:

> Jeremie Miller wrote:
>> I frequently find what I need in the right bar on
>> google's results (moreso when I'm actually looking for a product/
>> service).
>
> You're more lucky, then :) I'd say "sometimes" instead of  
> "frequently",
> even when I'm looking for a product.
>
>> I'd love to experiment with pulling in /more/ services, not less,  
>> *grin*.
>
> There are so many features in re.search.  Many are easy to see, but  
> some
> are not, and some are invisible (for example the freebase queries).   
> You
> should at least mention each feature somewhere.
> Maybe add a little question mark that links to each non-obvious
> feature's documentation.
>
> For example,
> o What do the colored bars underneath the related search terms mean?
> o What do the colors mean?
> o How are the related search terms chosen?
>
> o For the header background, how are the offered images chosen from  
> flickr?
> o Is there a way to see "the flickr page that the image lives on" (not
> sure if this is a unique or well-defined one).   For example, what  
> does
> the header image for the search term "wikia" show?
>
>> some stellar results pages, it's really exciting :)
>
> Totally agree :)
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>


From rainer.blome at gmx.de  Thu Jun 19 19:04:43 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 19 Jun 2008 21:04:43 +0200
Subject: [search-ui] Document re.search features
In-Reply-To: <F63ADCC7-A089-4D8E-A0A7-1AB9678D2652@jabber.org>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>	<2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>	<485A12CC.6000707@gmx.de>
	<F63ADCC7-A089-4D8E-A0A7-1AB9678D2652@jabber.org>
Message-ID: <485AADCB.6060105@gmx.de>

The existing help page is good, thanks Dan.  Concerning the colored bars 
behind the related results, it's already documented. It's my fault that 
I did not think of looking there.  On the new launch I even read it, but 
apparently not thoroughly enough :-).

Rainer

Jeremie Miller wrote:
 > Most of these are all great things for the help pages, who Dan has
 > been instrumental in helping pull together, I think he's on this list
 > but I cc'd him anyway :)
 >
> On Jun 19, 2008, at 3:03 AM, Rainer Blome wrote:
>> You should at least mention each feature somewhere.
>> Maybe add a little question mark that links to each non-obvious
>> feature's documentation.
>>
>> For example,
>> o What do the colored bars underneath the related search terms mean?
>> o What do the colors mean?
>> o How are the related search terms chosen?
>>
>> o For the header background, how are the offered images chosen from  
>> flickr?
>> o Is there a way to see "the flickr page that the image lives on" (not
>> sure if this is a unique or well-defined one).   For example, what  
>> does
>> the header image for the search term "wikia" show?


From aaron.wright at gmail.com  Thu Jun 19 19:07:03 2008
From: aaron.wright at gmail.com (Aaron Wright)
Date: Thu, 19 Jun 2008 15:07:03 -0400
Subject: [search-ui] Document re.search features
In-Reply-To: <485AADCB.6060105@gmx.de>
References: <48577409.3090000@gmx.de> <48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>
	<a6424d870806181521x3882df41x6d341be1c4a8fa29@mail.gmail.com>
	<2C75390A-3B45-4F55-8675-60676F6C1722@jabber.org>
	<485A12CC.6000707@gmx.de>
	<F63ADCC7-A089-4D8E-A0A7-1AB9678D2652@jabber.org>
	<485AADCB.6060105@gmx.de>
Message-ID: <d0fd42530806191207k48532a22r4cb6d0628a37147b@mail.gmail.com>

We are also making some help videos that will explain how to use each
of the features.  Their in production now, once they are ready we will
incorporate them on the site.

--Aaron

On Thu, Jun 19, 2008 at 3:04 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> The existing help page is good, thanks Dan.  Concerning the colored bars
> behind the related results, it's already documented. It's my fault that
> I did not think of looking there.  On the new launch I even read it, but
> apparently not thoroughly enough :-).
>
> Rainer
>
> Jeremie Miller wrote:
>  > Most of these are all great things for the help pages, who Dan has
>  > been instrumental in helping pull together, I think he's on this list
>  > but I cc'd him anyway :)
>  >
>> On Jun 19, 2008, at 3:03 AM, Rainer Blome wrote:
>>> You should at least mention each feature somewhere.
>>> Maybe add a little question mark that links to each non-obvious
>>> feature's documentation.
>>>
>>> For example,
>>> o What do the colored bars underneath the related search terms mean?
>>> o What do the colors mean?
>>> o How are the related search terms chosen?
>>>
>>> o For the header background, how are the offered images chosen from
>>> flickr?
>>> o Is there a way to see "the flickr page that the image lives on" (not
>>> sure if this is a unique or well-defined one).   For example, what
>>> does
>>> the header image for the search term "wikia" show?
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From rainer.blome at gmx.de  Thu Jun 19 23:24:35 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 20 Jun 2008 01:24:35 +0200
Subject: [search-ui] Git [Was: Re:  Speeding up re.search]
In-Reply-To: <CCB1CEBA-DF8B-43AF-9250-5FDBA37EE4E9@jabber.org>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<4858D6A8.9050700@gmx.de>
	<CCB1CEBA-DF8B-43AF-9250-5FDBA37EE4E9@jabber.org>
Message-ID: <485AEAB3.8090309@gmx.de>

Jeremie Miller wrote:
> The current results interface and almost all of re.search  is for
> *research*, ... 
> For *production* style discussion where we're really regression  
> testing and streamlining things, getting it stable/fast/secure/etc,  
> that should be a separate branch or even different repository for that  
> interface code. 
 > ... a "streamlined" new code
> tree.  Maybe instead of the "cool" folder we should have a "fast"  
> folder for building this out?

Have you considered using "Git" for code management?
Since about half a year I use it daily and love it.
Sure you need to get used to it, but it is *well* worth it.
Seeing is believing, best have someone show "gitk" to you.

I think Git would make it easier for others to participate.
Git makes it especially easy to experiment on a new branch and keep that 
experimental branch up to date with respect to any other branch, for 
example the master or a "stable" branch.  In addition, it has excellent 
support for distributed development.  In particular, private or public 
branches can be created and advanced locally, while still updating them 
from remote repositories.  And it does not need a database.

Many have written about why to git, how to git and how it works.
These are my favorites:

[http://eagain.net/articles/git-for-computer-scientists/
  Git for Computer Scientists] is a very compact "introduction to git 
internals for people who are not scared by words like Directed Acyclic 
Graph". It explains how Git conceptually works.

[http://www.newartisans.com/blog_files/git.from.bottom.up.php
  Git from the bottom up] covers similar topics as "Git for Computer 
Scientists", but explains more concretely how Git achieves the 
conceptual goals, and how to "manually" replicate this.

Looking for the second tutorial, I found this, but have not read it yet:

[http://utsl.gen.nz/talks/git-svn/intro.html An introduction to git-svn 
for Subversion/SVK users and deserters].  Looks like a way to use a git 
interface with a SVN repo.  Quote: "This article is aimed at people who 
want to contribute to projects which are using Subversion as their 
code-wiki."  This may be my ticket as long as re.search is using svn.

Sure, it is still possible to shoot yourself in the foot with Git, but 
it's less likely than with CVS to do so inadvertently.  With CVS, in my 
experience, merging is dangerous, you can easily lose or duplicate 
changes  - this hasn't happened with Git yet, although with Git I have 
merged more often in half a year than with CVS in ten years.
With SVN I have no merge experience.

Rainer

From jeremie at jabber.org  Fri Jun 20 00:28:03 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Thu, 19 Jun 2008 19:28:03 -0500
Subject: [search-ui] Git [Was: Re:  Speeding up re.search]
In-Reply-To: <485AEAB3.8090309@gmx.de>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<4858D6A8.9050700@gmx.de>
	<CCB1CEBA-DF8B-43AF-9250-5FDBA37EE4E9@jabber.org>
	<485AEAB3.8090309@gmx.de>
Message-ID: <AB979E66-EDEF-468D-A2EC-FE653A6A9EAF@jabber.org>

I have no strong affinity towards any SCM (except maybe mercurial,  
which is very git-like anyway)... SVN has the advantage of being just  
about anywhere out of the box :/

So, git would be cool, just gotta find the time to play with it a  
little now :)

On Jun 19, 2008, at 6:24 PM, Rainer Blome wrote:

> Jeremie Miller wrote:
>> The current results interface and almost all of re.search  is for
>> *research*, ...
>> For *production* style discussion where we're really regression
>> testing and streamlining things, getting it stable/fast/secure/etc,
>> that should be a separate branch or even different repository for  
>> that
>> interface code.
>> ... a "streamlined" new code
>> tree.  Maybe instead of the "cool" folder we should have a "fast"
>> folder for building this out?
>
> Have you considered using "Git" for code management?
> Since about half a year I use it daily and love it.
> Sure you need to get used to it, but it is *well* worth it.
> Seeing is believing, best have someone show "gitk" to you.
>
> I think Git would make it easier for others to participate.
> Git makes it especially easy to experiment on a new branch and keep  
> that
> experimental branch up to date with respect to any other branch, for
> example the master or a "stable" branch.  In addition, it has  
> excellent
> support for distributed development.  In particular, private or public
> branches can be created and advanced locally, while still updating  
> them
> from remote repositories.  And it does not need a database.
>
> Many have written about why to git, how to git and how it works.
> These are my favorites:
>
> [http://eagain.net/articles/git-for-computer-scientists/
>  Git for Computer Scientists] is a very compact "introduction to git
> internals for people who are not scared by words like Directed Acyclic
> Graph". It explains how Git conceptually works.
>
> [http://www.newartisans.com/blog_files/git.from.bottom.up.php
>  Git from the bottom up] covers similar topics as "Git for Computer
> Scientists", but explains more concretely how Git achieves the
> conceptual goals, and how to "manually" replicate this.
>
> Looking for the second tutorial, I found this, but have not read it  
> yet:
>
> [http://utsl.gen.nz/talks/git-svn/intro.html An introduction to git- 
> svn
> for Subversion/SVK users and deserters].  Looks like a way to use a  
> git
> interface with a SVN repo.  Quote: "This article is aimed at people  
> who
> want to contribute to projects which are using Subversion as their
> code-wiki."  This may be my ticket as long as re.search is using svn.
>
> Sure, it is still possible to shoot yourself in the foot with Git, but
> it's less likely than with CVS to do so inadvertently.  With CVS, in  
> my
> experience, merging is dangerous, you can easily lose or duplicate
> changes  - this hasn't happened with Git yet, although with Git I have
> merged more often in half a year than with CVS in ten years.
> With SVN I have no merge experience.
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>


From rainer.blome at gmx.de  Fri Jun 20 07:10:05 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 20 Jun 2008 09:10:05 +0200
Subject: [search-ui] Internationalization patch [Was: Re:  i18n, l10n]
In-Reply-To: <4855A058.7080707@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
	<4855A058.7080707@gmx.de>
Message-ID: <485B57CD.7060402@gmx.de>

Here you go.

This is what I did to create the patch:

svn update
At revision 262.
svn diff > i18n.r262.patch
gzip i18n.r262.patch
gzip -c cool/js/i18n/i18n.js > i18n.js.gz

This is what you need to do:

svn checkout -r 262  # Untested by me, but should work
cd re.search
mkdir cool/js/i18n

To my embarrassment, I have not tested applying this patch, not enough 
space in the file system at the moment to make a copy and try it.  I 
assume it goes like this:

gunzip -c i18n.r262.patch.gz | patch
gunzip -c i18n.js.gz > cool/js/i18n/i18n.js

Edit re.search/cool/JSON/servertouse.js:
Change  search_server_to_use = "file:///C:/re.search/cool/"; to match 
your working tree.

Open file:///C:/re.search/cool/index.html in your browser.
While not logged in, you should see no (Wikia-Search-specific) English 
strings on the front and result pages, unless you click something.

DONE

   o Make results page work locally.

   o Internationalized: index.html, header.js, search.html, main.js,
     ui.js, hist.js, expand_ma.js.

   o Determine browser's language setting. Unfortunately under my
     Firefox,
     the value of navigator.language is always "en-US", no matter which
     languages I have configured under Tools-> Options-> Advanced->
     General-> Languages-> Choose...

TODO

   o Make element modification work on IE, or use something else that's
     better than document.write()?

   o Determine preferred content language as set in the user's
     preferences or browser's options, or use the language set by
     util.js/langselect.js.

   o Move the translation tables to separate files.

   o Internationalize further html pages and code.

Rainer


Rainer Blome wrote:
> To freshen my JavaScript (unused for seven years, I think), I translated 
> the front page to German (and to French, probably badly so, just so I 
> had a different translation table for testing).
> 
> index.html loads the new file js/i18n/i18n.js, which currently contains 
> both the translation functions and the tables (which should be put in a 
> separate file, but I don't know how to "#include" in JavaScript).
> 
> The links and texts in index.html I have fitted with ids so that their 
> texts can be replaced (via assignment to innerHTML).  I did it this way 
> just to see if it could be done without switching to fully generated 
> HTML, which would be cleaner (but which I did not dare try at first).
> 
> header.js already generates the contents, wrapping the english strings 
> in gettext() calls was enough.
> 
> Currently, the choice of locale is hard-coded in i18n.js, I haven't been 
> able to switch my browser's locale from en-US to something else yet.
> 
> Are you interested in patches? If so, in what form?
> 
> Rainer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: i18n.r262.patch.gz
Type: application/unix-tar
Size: 8524 bytes
Desc: not available
Url : http://lists.wikia.com/pipermail/search-ui/attachments/20080620/cb634b35/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: i18n.js.gz
Type: application/unix-tar
Size: 4224 bytes
Desc: not available
Url : http://lists.wikia.com/pipermail/search-ui/attachments/20080620/cb634b35/attachment-0003.bin 

From balinny at gmail.com  Fri Jun 20 10:40:32 2008
From: balinny at gmail.com (Balinny)
Date: Fri, 20 Jun 2008 12:40:32 +0200
Subject: [search-ui] Git [Was: Re:  Speeding up re.search]
In-Reply-To: <485AEAB3.8090309@gmx.de>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<4858D6A8.9050700@gmx.de>	<CCB1CEBA-DF8B-43AF-9250-5FDBA37EE4E9@jabber.org>
	<485AEAB3.8090309@gmx.de>
Message-ID: <485B8920.3090202@gmail.com>

Rainer Blome wrote:
> Sure you need to get used to it, but it is *well* worth it.
>   
> I think Git would make it easier for others to participate.
>   
Do the advanced features of git overcome the learning curve ?


What's its current support/performance on Windows? It (used to?) be a 
big disadvantage against it.

From rainer.blome at gmx.de  Fri Jun 20 14:13:14 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 20 Jun 2008 16:13:14 +0200
Subject: [search-ui] Git
In-Reply-To: <485B8920.3090202@gmail.com>
References: <48577409.3090000@gmx.de>	<48579341.9020009@gmail.com>
	<a6424d870806170824w3c33c375p55156fdc17a6b768@mail.gmail.com>
	<E5216DB5-6E85-41C0-BC78-49087DA7B37D@jabber.org>	<4858D6A8.9050700@gmx.de>
	<CCB1CEBA-DF8B-43AF-9250-5FDBA37EE4E9@jabber.org>	<485AEAB3.8090309@gmx.de>
	<485B8920.3090202@gmail.com>
Message-ID: <20080620141314.229120@gmx.net>

Balinny <balinny at gmail.com> wrote:
> Rainer Blome wrote:
> > Sure you need to get used to it, but it is *well* worth it.
> Do the advanced features of git overcome the learning curve ?

Never ask me a general question - I might try to give an all-encompassing answer ;-).

The benefits of Git outweighed the costs of switching to it, for me.

The main benefits are not any "advanced" features, it's the basic features
o reliability,
o community-oriented design and
o speed.

A little about my experience in each of these regards below,
but before I bore you with these:

To minimize learning costs, Git can be used with a work flow that
uses a central shared repository.  For development
with only a handful, maybe ten developers, for example, this may be
the best choice anyway.

Basic Git Benefits

o Reliability - With CVS, I had data corruption.  It was not CVS's fault,
  but it didn't notice either, and it took a while until we noticed it. 
  Git has been designed to detect corruption right away.

o Speed - With CVS, some operations, especially branching (tagging),
  are slow. I already talked about merging being dangerous in the last mail.
  So I branched only when absolutely necessary (fix branches).
  Git makes branching fast and merging painless.
  Git also has slow use cases, but they didn't bother me so far.

o Community-oriented design - I sometimes sync my editor configuration
  between private and occupational environments.
  With CVS this was difficult to do because I'd had to choose
  a central repository location.
  Since it's my code, I want to keep the "conceptually central"
  repository on one of my machines (not on some client's machine).
  And that is difficult because I don't want to run a server and
  connect from outside.
  SVN, as far as I know, also requires some central repository.
  It may make this easier, but I don't know.

  Git, by contrast, caters especially to distributed, sporadically
  connected development.  Except by explicit configuration or by
  convention, a Git repository clone is just as powerful as its origin(al).
  This makes it feasible (economical) to independently advance multiple
  distributed branches and still keep them mostly in sync.

o Fun - There is an enormous amount of OSS brainpower behind Git,
  and it shows, in many ways.
  Something that is important but very hard to justify with words
  is that for me, it is plain fun to use Git.
  The reason probably is that it solves many problems that I had with other VCSs.

o Further benefits: good tools out of the box, compact repository,
  good support (in my experience).

> What's its current support/performance on Windows? It (used to?) be a 
> big disadvantage against it.

Under cygwin it works well, out of the box.  Without cygwin, no idea.
 Just use it under cygwin.

Rainer

-- 



GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf at gmx

From rainer.blome at gmx.de  Fri Jun 20 21:09:44 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 20 Jun 2008 23:09:44 +0200
Subject: [search-ui] Internationalization patch [Was: Re:  i18n, l10n]
In-Reply-To: <485B57CD.7060402@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>	<4855A058.7080707@gmx.de>
	<485B57CD.7060402@gmx.de>
Message-ID: <485C1C98.2000206@gmx.de>

It should have been "svn update -r 262"
(instead of "svn checkout -r 262"), sorry.


Further items to do:

o Decide whether the language selection dialog should switch the
   UI language in synchrony with the desired results language.

o Make it work with different character sets.
   Currently, I used HTML/entities to get non-ASCII characters,
   but this does not work with texts displayed in list boxes (why not?).

   Would utf-8 be the right way to go?
   Is it a good idea to look at the source of http://www.wikipedia.org/
   and try to learn from it?

o Is it OK to use JSON format for the translation table or are
   there strong reasons to use PO files?

o Support positional message parameters.

o Create tar file that runs locally out of the box.

o Support right to left writing (direction: rtl) .

o Support top to bottom writing direction (block-progression: rl).

o Scale, localize for more languages.

Have a nice weekend, Rainer


From borboleta at gmail.com  Sun Jun 22 02:17:32 2008
From: borboleta at gmail.com (Bani)
Date: Sat, 21 Jun 2008 23:17:32 -0300
Subject: [search-ui] Internationalization patch [Was: Re: i18n, l10n]
In-Reply-To: <485C1C98.2000206@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>
	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>
	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>
	<485003D1.8000906@wikia.com> <48517A3E.90108@gmx.de>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
	<4855A058.7080707@gmx.de> <485B57CD.7060402@gmx.de>
	<485C1C98.2000206@gmx.de>
Message-ID: <35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>

I've tried this patch but it didn't work well here. Maybe I've applied
it wrong. But when I open the index page all I get is a page with 3
commas and a link to wikia.

From rainer.blome at gmx.de  Sun Jun 22 18:26:41 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Sun, 22 Jun 2008 20:26:41 +0200
Subject: [search-ui] Internationalization patch [Was: Re: i18n, l10n]
In-Reply-To: <35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>
	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>	<4855A058.7080707@gmx.de>
	<485B57CD.7060402@gmx.de>	<485C1C98.2000206@gmx.de>
	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>
Message-ID: <485E9961.7060507@gmx.de>

Bani wrote:
> when I open the index page all I get is a page with 3
> commas and a link to wikia.

Looks like anything but the index.html page could not be loaded.
In re.search/cool/JSON/servertouse.js, the value of search_server_to_use 
must end in "cool/".

The following mail I sent on Friday night is apparently stuck in 
moderation.  I'll send you the tar file.

Jer, could you lift the moderation threshold of 40k a bit?

Rainer

---
Rainer Blome wrote:
 > To my embarrassment, I have not tested applying this patch, not 
enough > space in the file system at the moment to make a copy and try it.

If something can go wrong, it will.  After cleaning up a bit, I tried
to apply that patch.
Well, svn diff apparently does not output something understandable by 
Cygwin patchutils' patch.  Hunks fail, paths have to be manually 
specified, no good.

Instead, I prepared and attached a tar file that can be unpacked over a 
re.search r262 checkout.  This time I tested it, it works.

cd re.search
svn update -r262
cd ..
tar xvzf re.search-i18n-mod.tar.gz
cd re.search/cool
# Configure search_server_to_use (value must end in cool/):
emacs JSON/servertouse.js
firefox

Rainer


From borboleta at gmail.com  Sun Jun 22 18:50:06 2008
From: borboleta at gmail.com (Bani)
Date: Sun, 22 Jun 2008 15:50:06 -0300
Subject: [search-ui] Internationalization patch [Was: Re: i18n, l10n]
In-Reply-To: <485E9961.7060507@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>
	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>
	<485003D1.8000906@wikia.com> <48517A3E.90108@gmx.de>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
	<4855A058.7080707@gmx.de> <485B57CD.7060402@gmx.de>
	<485C1C98.2000206@gmx.de>
	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>
	<485E9961.7060507@gmx.de>
Message-ID: <35b94d690806221150t1326dcebg1cefd2e466916170@mail.gmail.com>

Ah, ok. Now with the tar file it really worked :)

On Sun, Jun 22, 2008 at 3:26 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> Bani wrote:
>> when I open the index page all I get is a page with 3
>> commas and a link to wikia.
>
> Looks like anything but the index.html page could not be loaded.
> In re.search/cool/JSON/servertouse.js, the value of search_server_to_use
> must end in "cool/".
>
> The following mail I sent on Friday night is apparently stuck in
> moderation.  I'll send you the tar file.
>
> Jer, could you lift the moderation threshold of 40k a bit?
>
> Rainer
>
> ---
> Rainer Blome wrote:
>  > To my embarrassment, I have not tested applying this patch, not
> enough > space in the file system at the moment to make a copy and try it.
>
> If something can go wrong, it will.  After cleaning up a bit, I tried
> to apply that patch.
> Well, svn diff apparently does not output something understandable by
> Cygwin patchutils' patch.  Hunks fail, paths have to be manually
> specified, no good.
>
> Instead, I prepared and attached a tar file that can be unpacked over a
> re.search r262 checkout.  This time I tested it, it works.
>
> cd re.search
> svn update -r262
> cd ..
> tar xvzf re.search-i18n-mod.tar.gz
> cd re.search/cool
> # Configure search_server_to_use (value must end in cool/):
> emacs JSON/servertouse.js
> firefox
>
> Rainer
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From rainer.blome at gmx.de  Mon Jun 23 13:04:54 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 23 Jun 2008 15:04:54 +0200
Subject: [search-ui] Encoding  [Was: Re:  Internationalization patch]
In-Reply-To: <35b94d690806221150t1326dcebg1cefd2e466916170@mail.gmail.com>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>
	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>	<4855A058.7080707@gmx.de>
	<485B57CD.7060402@gmx.de>	<485C1C98.2000206@gmx.de>	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>	<485E9961.7060507@gmx.de>
	<35b94d690806221150t1326dcebg1cefd2e466916170@mail.gmail.com>
Message-ID: <485F9F76.10904@gmx.de>

Bani wrote:
> Ah, ok. Now with the tar file it really worked :)

Great :).

UTF-8

Encoding documents and strings in UTF-8 is the right thing to do,
as fas as I can see.
Having found how to use Emacs 22 with Cygwin (instructions are here: 
http://sinewalker.wordpress.com/2008/04/04/cygwin-and-emacs-22/), I can 
finally edit UTF-8 files and converted the entities used in i18n.js to 
UTF-8 characters.

As long as the browser's encoding is set to UTF-8, this solves the issue
of literal HTML entities being visible in list boxes, for example
"l & o u m l ; s c h t e" (without the spaces) instead of "l?schte".
It also makes it easier to write and debug translation tables.

Encoding specified by META-tags

When the encoding is auto-detected (in IE: View-> Encoding-> Auto-select 
option is active; in Firefox: View-> Character Encoding-> Auto-Detect-> 
Universal), it chooses "Western European" for some pages and 
consequently displays the non-ASCII characters wrong.

This is apparently because the following line is present in those pages 
that get auto-detected as UTF-8, and it is missing in those that get 
detected as "Western European":

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

If I am not mistaken, there should be a slash at the end, but it looks 
like it works anyway.


Rainer


From rainer.blome at gmx.de  Mon Jun 23 13:07:36 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 23 Jun 2008 15:07:36 +0200
Subject: [search-ui] Language Selection
Message-ID: <485FA018.6090008@gmx.de>

How do we set the language?

Browser idiosyncracy - It is tempting to elide the "syncra" here :)

In IE, the value of both navigator.systemLanguage and 
navigator.userLanguage is "de" for me, although I have an english XP and 
configured "fr" as my preferred content language in IE (just for 
testing).  The OS language at least shows in the value of 
navigator.browserLanaguage being "en-us".

To allow the user to choose the desired interface language at runtime,
I added a call to i18n.setlanguage() to hide_langselect() (which saves 
the user's choice of results language) and use getCookie("ws_lang") when 
the i18n object is initialized.  The user must reload the page for this 
to take effect.  Can and should the reload be automated?

At least one of the displayed texts changes immediately after saving, 
the one for "Language:".  This is because show_lang_header() is called 
by hide_langselect().  Appropriate redisplay functions for the rest of 
the page might help, but for now a page reload is fine by me.

In IE, getCookie("ws_lang") unfortunately returns "false", no matter 
what privacy settings I use.  In such cases, the interface uses 
browser_language() instead.

Rainer

From balinny at gmail.com  Mon Jun 23 21:35:22 2008
From: balinny at gmail.com (Balinny)
Date: Mon, 23 Jun 2008 23:35:22 +0200
Subject: [search-ui] Language Selection
In-Reply-To: <485FA018.6090008@gmx.de>
References: <485FA018.6090008@gmx.de>
Message-ID: <4860171A.4060201@gmail.com>

Rainer Blome wrote:
> To allow the user to choose the desired interface language at runtime,
> I added a call to i18n.setlanguage() to hide_langselect() (which saves 
> the user's choice of results language) and use getCookie("ws_lang") when 
> the i18n object is initialized.  The user must reload the page for this 
> to take effect.  Can and should the reload be automated?
>   
document.reload()
But i find the partial reloading preferable.

From david.pean at gmail.com  Mon Jun 23 21:49:09 2008
From: david.pean at gmail.com (David Pean)
Date: Mon, 23 Jun 2008 17:49:09 -0400
Subject: [search-ui] Internationalization patch [Was: Re: i18n, l10n]
In-Reply-To: <485E9961.7060507@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>
	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>
	<485003D1.8000906@wikia.com> <48517A3E.90108@gmx.de>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
	<4855A058.7080707@gmx.de> <485B57CD.7060402@gmx.de>
	<485C1C98.2000206@gmx.de>
	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>
	<485E9961.7060507@gmx.de>
Message-ID: <e49dbf0d0806231449p356a69b2mf9935727e4e2cf96@mail.gmail.com>

For some reason I didn't get a file attached to this email.  I also
tried the previous 2 files and got a weird index page

Dave

On Sun, Jun 22, 2008 at 2:26 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> Bani wrote:
>> when I open the index page all I get is a page with 3
>> commas and a link to wikia.
>
> Looks like anything but the index.html page could not be loaded.
> In re.search/cool/JSON/servertouse.js, the value of search_server_to_use
> must end in "cool/".
>
> The following mail I sent on Friday night is apparently stuck in
> moderation.  I'll send you the tar file.
>
> Jer, could you lift the moderation threshold of 40k a bit?
>
> Rainer
>
> ---
> Rainer Blome wrote:
>  > To my embarrassment, I have not tested applying this patch, not
> enough > space in the file system at the moment to make a copy and try it.
>
> If something can go wrong, it will.  After cleaning up a bit, I tried
> to apply that patch.
> Well, svn diff apparently does not output something understandable by
> Cygwin patchutils' patch.  Hunks fail, paths have to be manually
> specified, no good.
>
> Instead, I prepared and attached a tar file that can be unpacked over a
> re.search r262 checkout.  This time I tested it, it works.
>
> cd re.search
> svn update -r262
> cd ..
> tar xvzf re.search-i18n-mod.tar.gz
> cd re.search/cool
> # Configure search_server_to_use (value must end in cool/):
> emacs JSON/servertouse.js
> firefox
>
> Rainer
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From rainer.blome at gmx.de  Mon Jun 23 23:09:53 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Tue, 24 Jun 2008 01:09:53 +0200
Subject: [search-ui] Internationalization patch [Was: Re: i18n, l10n]
In-Reply-To: <e49dbf0d0806231449p356a69b2mf9935727e4e2cf96@mail.gmail.com>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>
	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>	<4855A058.7080707@gmx.de>
	<485B57CD.7060402@gmx.de>	<485C1C98.2000206@gmx.de>	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>	<485E9961.7060507@gmx.de>
	<e49dbf0d0806231449p356a69b2mf9935727e4e2cf96@mail.gmail.com>
Message-ID: <48602D41.9080406@gmx.de>

David Pean wrote:
> For some reason I didn't get a file attached to this email.  

There was none attached, I sent the file just to Bani.
I'll send a newer one to you.

The mail that I actually attached the tar file to has still not
been approved to appear on the list (the tar file is 55kB).

For the language selection to persist across page changes (in other , 
you have to pass the index page, it seems?  Not sure if this is due to 
my modifications.  Where is the ws_lang cookie created?

Rainer


From david.pean at gmail.com  Tue Jun 24 20:07:04 2008
From: david.pean at gmail.com (David Pean)
Date: Tue, 24 Jun 2008 16:07:04 -0400
Subject: [search-ui] Internationalization patch [Was: Re: i18n, l10n]
In-Reply-To: <48602D41.9080406@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>
	<48517A3E.90108@gmx.de>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
	<4855A058.7080707@gmx.de> <485B57CD.7060402@gmx.de>
	<485C1C98.2000206@gmx.de>
	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>
	<485E9961.7060507@gmx.de>
	<e49dbf0d0806231449p356a69b2mf9935727e4e2cf96@mail.gmail.com>
	<48602D41.9080406@gmx.de>
Message-ID: <e49dbf0d0806241307t17f5c2a6k58dbb2e84cd49531@mail.gmail.com>

Cool stuff!

We have merged your changes with our latest development work (on our
dev server).   There are some bugs that popped up.  Once we get
everything working and tested, we will put it up on the re.search site
-- probably the next day or so.

Oh, the lang cookie is set in main.js

Dave

On Mon, Jun 23, 2008 at 7:09 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> David Pean wrote:
>> For some reason I didn't get a file attached to this email.
>
> There was none attached, I sent the file just to Bani.
> I'll send a newer one to you.
>
> The mail that I actually attached the tar file to has still not
> been approved to appear on the list (the tar file is 55kB).
>
> For the language selection to persist across page changes (in other ,
> you have to pass the index page, it seems?  Not sure if this is due to
> my modifications.  Where is the ws_lang cookie created?
>
> Rainer
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From jeffrey.tierney at gmail.com  Tue Jun 24 20:17:33 2008
From: jeffrey.tierney at gmail.com (Jeffrey J Tierney)
Date: Tue, 24 Jun 2008 16:17:33 -0400
Subject: [search-ui] Internationalization patch [Was: Re: i18n, l10n]
In-Reply-To: <e49dbf0d0806241307t17f5c2a6k58dbb2e84cd49531@mail.gmail.com>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>
	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>
	<4855A058.7080707@gmx.de> <485B57CD.7060402@gmx.de>
	<485C1C98.2000206@gmx.de>
	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>
	<485E9961.7060507@gmx.de>
	<e49dbf0d0806231449p356a69b2mf9935727e4e2cf96@mail.gmail.com>
	<48602D41.9080406@gmx.de>
	<e49dbf0d0806241307t17f5c2a6k58dbb2e84cd49531@mail.gmail.com>
Message-ID: <9131eace0806241317o595612bby7cb068622c75c484@mail.gmail.com>

There are actually two places where the ws_lang cookie could be set from.

We left in what Jer had originally put in  (in the "search" function of
main.js, if you type lang: xx in the search box it gets set), but we also
added in the case for the language select dropdown in the lightbox, which is
in js/langselect.js (the hide_langselect function).

Jeff


On Tue, Jun 24, 2008 at 4:07 PM, David Pean <david.pean at gmail.com> wrote:

> Cool stuff!
>
> We have merged your changes with our latest development work (on our
> dev server).   There are some bugs that popped up.  Once we get
> everything working and tested, we will put it up on the re.search site
> -- probably the next day or so.
>
> Oh, the lang cookie is set in main.js
>
> Dave
>
> On Mon, Jun 23, 2008 at 7:09 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> > David Pean wrote:
> >> For some reason I didn't get a file attached to this email.
> >
> > There was none attached, I sent the file just to Bani.
> > I'll send a newer one to you.
> >
> > The mail that I actually attached the tar file to has still not
> > been approved to appear on the list (the tar file is 55kB).
> >
> > For the language selection to persist across page changes (in other ,
> > you have to pass the index page, it seems?  Not sure if this is due to
> > my modifications.  Where is the ws_lang cookie created?
> >
> > Rainer
> >
> > _______________________________________________
> > Search-UI mailing list
> > Search-UI at wikia.com
> > http://lists.wikia.com/mailman/listinfo/search-ui
> >
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080624/eaa7708f/attachment.html 

From rainer.blome at gmx.de  Tue Jun 24 21:53:39 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Tue, 24 Jun 2008 23:53:39 +0200
Subject: [search-ui] cookies are matched in a case-sensitive fashion
In-Reply-To: <9131eace0806241317o595612bby7cb068622c75c484@mail.gmail.com>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>	<4855A058.7080707@gmx.de>
	<485B57CD.7060402@gmx.de>	<485C1C98.2000206@gmx.de>	<35b94d690806211917j127871degc1dcfd2b113aa5c8@mail.gmail.com>	<485E9961.7060507@gmx.de>	<e49dbf0d0806231449p356a69b2mf9935727e4e2cf96@mail.gmail.com>	<48602D41.9080406@gmx.de>	<e49dbf0d0806241307t17f5c2a6k58dbb2e84cd49531@mail.gmail.com>
	<9131eace0806241317o595612bby7cb068622c75c484@mail.gmail.com>
Message-ID: <48616CE3.40102@gmx.de>

Something to be aware of while testing locally on a Windows box:

Even if the OS does not care about drive letters or path names
being upper or lower case, the browser may care (Firefox does).
In particular, cookies are matched in a case-sensitive fashion.

This means that a cookie set on a page whose URL starts with
  file:///c:/
is not found from a page whose URL starts with
  file:///C:/.

In the context of testing locally,
the case of the letters (at least the drive letter) in the initial URL
(for example file:///c:/re.search/cool/search.html#test)
should match the case of the letters used in the
search_server_to_use value.  Otherwise the cookie(s) set by the
initial page will not be found by the referred-to pages.

Rainer


PS: In case you are interested, I had this scenario almost
fully written and was going to send it as a question when
I finally understood what was happening.
Here is how to reproduce the problem:

What's puzzling me is this:
o In cool/JSON/servertouse.js:
   search_server_to_use="file:///C:/re.search/cool/";
o Open a fresh Firefox, no cache, no cookies.
o Go to the local search page,
   "file:///c:/re.search/cool/search.html#test".
   Here, it comes up in English,
   because navigator.language's value is "en-us".
o Using the Language dialog, choose a different language, save.
o Reload the page.
   It should now display in the newly selected language.  Good.
o Click on the Help header link.  Arriving at the help.html page,
   the header displays in the default language (English here). Bad :-(.
   Why isn't the cookie used?  See above.

o Click on the Wikia Search link at the top.  Arriving at the
   index.html page, it displays in the default language.
o Type test and hit Search.
   The search page displays in the default language.
   ["file:///C:/re.search/cool/search.html#test"]
o Using the Language dialog, choose a different language, save.
o Reload the page.
   It should now display in the newly selected language.  Good.
o Click on the Help header link.  Arriving at the help.html page,
   the header now displays in the newly selected language. Good.
   Why is the cookie now used?
   Because the second time around at the search page,
   the URL was different.

Jeffrey J Tierney wrote:
> On Tue, Jun 24, 2008 at 4:07 PM, David Pean <david.pean at gmail.com> wrote:
>> Oh, the lang cookie is set in main.js
>
> There are actually two places where the ws_lang cookie could be set from.
> 
> We left in what Jer had originally put in  (in the "search" function of 
> main.js, if you type lang: xx in the search box it gets set), 

That one I missed, but it has an effect only when "lang:xy" is used.

> we
> also added in the case for the language select dropdown in the lightbox, 
> which is in js/langselect.js (the hide_langselect function).

That is the one I used to add the i18n.setlanguage() call.

Rainer


From rainer.blome at gmx.de  Tue Jun 24 22:28:35 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Wed, 25 Jun 2008 00:28:35 +0200
Subject: [search-ui] Language Selection
In-Reply-To: <4860171A.4060201@gmail.com>
References: <485FA018.6090008@gmx.de> <4860171A.4060201@gmail.com>
Message-ID: <48617513.4070201@gmx.de>

Balinny wrote:
> Rainer Blome wrote:
>> Can and should the reload be automated?  
> document.reload()
document.reload() does not work with Firefox.
window.location.reload() works.

When the language is selected on the index page, reloading the page is 
much less distracting than on the search page.

> But i find the partial reloading preferable.

That requires a way to regenerate all affected content.
Currently, the HTML pages are not set up for this.

To be able to select the language on the index page,
  the style sheets need some massage:

kt_files/index.css: // Disabled:
//#site-language {
//	display:none;
//}

kt_files/structure.css:
// Please improve:
#site-language {
	position:absolute;
	top:0px;
	// Moved to the left:
	//right:10px;
	right:100px;
}

Rainer


From rainer.blome at gmx.de  Wed Jun 25 08:38:22 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Wed, 25 Jun 2008 10:38:22 +0200
Subject: [search-ui] Language Selection
In-Reply-To: <485FA018.6090008@gmx.de>
References: <485FA018.6090008@gmx.de>
Message-ID: <486203FE.9090600@gmx.de>

In Mozilla, navigator.language always returns en-US:
https://bugs.launchpad.net/firefox/+bug/117915 
https://bugzilla.mozilla.org/show_bug.cgi?id=285267 .
A solution is to go to about:config and change the value of 
general.useragent.locale from it default en-US to the desired value, for 
example de-DE. The second issue record lists other possible solutions.

When I do this, navigator.language is de-DE, which i18n.js picks up
(no need to use the dialog to change the language).

 > In IE, getCookie("ws_lang") unfortunately returns "false", no matter
 > what privacy settings I use.  In such cases, the interface uses
 > browser_language() instead.

I'm still stumped here, so can't test interactively setting the language 
(the dialog presentation works, but it has no effect).

In IE, the value of navigator.userLanguage reflects the locale setting 
under Control Panel -> Regional and Language Options -> Standards and 
Formats.  Until the cookie works, this is my way of changing the language.

Rainer


From rainer.blome at gmx.de  Thu Jun 26 11:20:36 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Thu, 26 Jun 2008 13:20:36 +0200
Subject: [search-ui] Language Selection
In-Reply-To: <485FA018.6090008@gmx.de>
References: <485FA018.6090008@gmx.de>
Message-ID: <48637B84.7030102@gmx.de>

In order be able to load only the needed translation table(s),
I've separated the tables from the code.
I've also separated determination of which language is
desired (i18n.desired) and actually setting the
translation table (i18n.language).
This is what is done now:

o If the desired language, for example "pt-BR", is not available, fall
   back to the trimmed desired language, "pt" in the example.

o If the trimmed desired language is not available, fall back to the
   browser's default language (for example "fr-FR").

o If the browser's default language is not available, fall
   back to the browser's trimmed default language (for example "fr").

o If the browser's trimmed default language is not available,
   fall back to the application's default language ("en").

o Request the desired language translation table, and,
   to enable immediate fallback, the translation table for the
   browser's default language.

o If a requested language name has a dash in its name, also
   request the translation table for the trimmed language name.

   FIXME: For simplicity, currently all potentially needed tables are
   requested (up to four), even though we do not know which ones
   of these exit.  It would be better if we knew which tables
   exist and requested only those known to exist.

What do you think?

Rainer


From david.pean at gmail.com  Thu Jun 26 16:06:50 2008
From: david.pean at gmail.com (David Pean)
Date: Thu, 26 Jun 2008 12:06:50 -0400
Subject: [search-ui] Updates
Message-ID: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>

Hey everyone,

We've rolled out a few changes to re.search.wikia.com

* Merged Rainer's i18n functionality for the interface.  Currently
German and partial French are supported.  We have some people already
interested in helping out for other languages, so we can keep building
on this.  The next change we should include would be seperating each
language file out (I think Rainer has been working on this)

Note: Jeff made some cool tweaks on the language selector so you can
preview/see the language changes on the fly.  Try out Deutsch

* # of queries and contributions listed on homepage (updates live)

* New Add URL functionality -- will auto fill in the HTML title and
meta description for the url added.

Please update your SVN accordingly, as there were alot of file changes :)

Dave

From balinny at gmail.com  Thu Jun 26 18:52:41 2008
From: balinny at gmail.com (Balinny)
Date: Thu, 26 Jun 2008 20:52:41 +0200
Subject: [search-ui] Language Selection
In-Reply-To: <48637B84.7030102@gmx.de>
References: <485FA018.6090008@gmx.de> <48637B84.7030102@gmx.de>
Message-ID: <4863E579.4020503@gmail.com>

Rainer Blome wrote:
>    FIXME: For simplicity, currently all potentially needed tables are
>    requested (up to four), even though we do not know which ones
>    of these exit.  It would be better if we knew which tables
>    exist and requested only those known to exist.
>
> What do you think?
>
> Rainer
>   
Why not having an index of available translations at that file?
It requires to edit it on each language addition, but seems better overall.


From jwales at wikia.com  Thu Jun 26 20:39:47 2008
From: jwales at wikia.com (Jimmy Wales)
Date: Thu, 26 Jun 2008 16:39:47 -0400
Subject: [search-ui] Updates
In-Reply-To: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
Message-ID: <4863FE93.5080407@wikia.com>

David Pean wrote:
> * New Add URL functionality -- will auto fill in the HTML title and
> meta description for the url added.

This one is my favorite. :)

From rainer.blome at gmx.de  Thu Jun 26 22:34:12 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 27 Jun 2008 00:34:12 +0200
Subject: [search-ui] overview.mov
Message-ID: <48641964.2000902@gmx.de>

Could you please move cool/help/overview.mov elsewhere,
for example to wikimedia Commons?  Together with its .svn copy,
it takes up a large space (20MB), compared to the rest of the sources.
Or is there a need for it to be kept with the source code?

By the way, the "overview" page with the movie crashes my
browsers. This may be the plugin's fault, however most
quicktime movies display OK.  Quicktime movies are configured
to display using the VLC 0.8.6d plugin.  Seamonkey crashed upon
starting the movie, while Firefox survived playing the movie
and crashed when I tried to close the tab with the
overview.html page.


From rainer.blome at gmx.de  Thu Jun 26 23:31:50 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 27 Jun 2008 01:31:50 +0200
Subject: [search-ui] Updates
In-Reply-To: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
Message-ID: <486426E6.1040607@gmx.de>

David Pean wrote:
> We've rolled out a few changes to re.search.wikia.com

> * Merged Rainer's i18n functionality for the interface.
Woohooo!

> Currently German and partial French are supported.

German ist partial, too.  The community menu, anything with profiles, 
the background dialog, metrics, all need to be instrumented.

> The next change we should include would be seperating each
> language file out (I think Rainer has been working on this)

Yes.  Although I'm having trouble loading them on demand.  Currently, 
they are loaded via doc.write(<script src="...js">...).  I tried the 
following, but could not get that to work (I think it is loaded too late):

function loadJSfile(filename) {
     var elt= document.createElement('script');
     elt.setAttribute("type", "text/javascript");
     elt.setAttribute("src", filename);
     if (undefined != elt) {
	document.getElementsByTagName("head")[0].appendChild(elt);
     }
}

Something like XMLHTTPRequest combined with eval seems promising to me. 
  Given all the dynamic loading re.search already does, there's got to 
be a function that loads a string from a server somewhere.  I looked for 
one, but have not found one that I could use yet. Where is it :)?

Rainer


From jwales at wikia.com  Fri Jun 27 00:04:21 2008
From: jwales at wikia.com (Jimmy Wales)
Date: Thu, 26 Jun 2008 20:04:21 -0400
Subject: [search-ui] overview.mov
In-Reply-To: <48641964.2000902@gmx.de>
References: <48641964.2000902@gmx.de>
Message-ID: <48642E85.9020506@wikia.com>

Yeah, I support this too.  I am planning on making little movies like 
this pretty often to show new features as we roll them out, no sense in 
this being in svn at all.

I have no clue about the crashing, I didn't intend for it to be fancy in 
any way. :)

Rainer Blome wrote:
> Could you please move cool/help/overview.mov elsewhere,
> for example to wikimedia Commons?  Together with its .svn copy,
> it takes up a large space (20MB), compared to the rest of the sources.
> Or is there a need for it to be kept with the source code?
> 
> By the way, the "overview" page with the movie crashes my
> browsers. This may be the plugin's fault, however most
> quicktime movies display OK.  Quicktime movies are configured
> to display using the VLC 0.8.6d plugin.  Seamonkey crashed upon
> starting the movie, while Firefox survived playing the movie
> and crashed when I tried to close the tab with the
> overview.html page.
> 
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
> 
> 


From david.pean at gmail.com  Fri Jun 27 00:21:35 2008
From: david.pean at gmail.com (David Pean)
Date: Thu, 26 Jun 2008 20:21:35 -0400
Subject: [search-ui] overview.mov
In-Reply-To: <48641964.2000902@gmx.de>
References: <48641964.2000902@gmx.de>
Message-ID: <e49dbf0d0806261721u7b92abf2k274d6528d75983bf@mail.gmail.com>

I've been meaning to do this..had a slow connection one day and was
trying to check out the code :)

We'll take care of it

On Thu, Jun 26, 2008 at 6:34 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> Could you please move cool/help/overview.mov elsewhere,
> for example to wikimedia Commons?  Together with its .svn copy,
> it takes up a large space (20MB), compared to the rest of the sources.
> Or is there a need for it to be kept with the source code?
>
> By the way, the "overview" page with the movie crashes my
> browsers. This may be the plugin's fault, however most
> quicktime movies display OK.  Quicktime movies are configured
> to display using the VLC 0.8.6d plugin.  Seamonkey crashed upon
> starting the movie, while Firefox survived playing the movie
> and crashed when I tried to close the tab with the
> overview.html page.
>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From rainer.blome at gmx.de  Fri Jun 27 00:38:44 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 27 Jun 2008 02:38:44 +0200
Subject: [search-ui] Updates
In-Reply-To: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
Message-ID: <48643694.8040904@gmx.de>

The search.html in svn r279 is older than the one on re.search.wiki.com?

From rainer.blome at gmx.de  Fri Jun 27 01:03:11 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 27 Jun 2008 03:03:11 +0200
Subject: [search-ui] Updates
In-Reply-To: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
Message-ID: <48643C4F.8050803@gmx.de>

David Pean wrote:
> Note: Jeff made some cool tweaks on the language selector so you can
> preview/see the language changes on the fly.  Try out Deutsch

Far out!  Clever.  Works well.  A bit obfuscated, maybe.
403 character in one line.  Is your carriage return key out of 
commission? ;-)

xlateOnFly can be simplified, no eval needed:

function xlateOnFly(lang) {
	for (var ii=0; ii<i18n_arr.length; ii++) {
		var txt= gettextonfly(i18n_arr[ii].i_id, lang);
		var a= i18n_arr[ii].i_args;
		if ( ! a) {
			var elt= document.getElementById("i18n_"+ii);
			if (elt) { elt.innerHTML = txt; }
		}
		else {
			if (a.id && a.param) {
			    var elt= document.getElementById(a.id);
			    if (elt) {
				elt[a.param]= (a.pre ? a.pre : "")
				    + txt + (a.post ? a.post : "");
			    }
			}
		}
	}
}

Why "ii" instead of just "i"?
Why "i_id" instead of just "id", and "i_args" instead of just "args"? 
(these may be JS newbie questions)?

Are i_args.pre or i_args.post set anywhere?

gettextonfly(id, lang) should take optional arguments, to be used in 
formatting.  Same for xlate and gettext.  If a special treatment is 
needed, such as for the dialog buttons (attribute values in general), a 
separate function should be used.

The reload solution always works, why not keep that for those pages 
where xlateOnFly does not work?

Rainer

From david.pean at gmail.com  Fri Jun 27 01:11:00 2008
From: david.pean at gmail.com (David Pean)
Date: Thu, 26 Jun 2008 21:11:00 -0400
Subject: [search-ui] Updates
In-Reply-To: <48643694.8040904@gmx.de>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
	<48643694.8040904@gmx.de>
Message-ID: <e49dbf0d0806261811i2c337a6co7d1163038a7b366d@mail.gmail.com>

We use a different copy of search.html on the server vs in SVN.

The live version includes one minified file of all the js files used,
plus a concatenated css file.

Dave

On Thu, Jun 26, 2008 at 8:38 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> The search.html in svn r279 is older than the one on re.search.wiki.com?
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From jeffrey.tierney at gmail.com  Fri Jun 27 01:34:14 2008
From: jeffrey.tierney at gmail.com (Jeffrey J Tierney)
Date: Thu, 26 Jun 2008 21:34:14 -0400
Subject: [search-ui] Updates
In-Reply-To: <48643C4F.8050803@gmx.de>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
	<48643C4F.8050803@gmx.de>
Message-ID: <9131eace0806261834j5b95d582icf8f4aa976a0b8d0@mail.gmail.com>

heh... obfuscation is my specialty.  kind of like my signature on the code
:)

Actually I was planning on going back and cleaning this stuff up over the
next couple of days, but then when I saw your email about selectively
loading the different language files, i wanted to wait to clean it up until
we could try to integrate that, since I figured that the functionality might
have to change a bit if you dont have all of the translations pre-loaded.

I will take a look at replacing the 403 char 1 line function with what you
have below.
*

Why "ii" instead of just "i"?
Why "i_id" instead of just "id", and "i_args" instead of just "args"?
(these may be JS newbie questions)?*

no real reason for this... just that since i was playing around with getting
this to work I used ii this time instead of i, and "i_" before those names
for easier differentiation in debugging, and I forgot to switch them back
before we pushed the code live.  (and because obfuscation is a personal
hobby :))
*
gettextonfly(id, lang) should take optional arguments, to be used in
formatting*

gettext still takes the optional arguments that it always did, and that is
still the function that is used throughout the pages to handle getting the
text from the appropriate language and outputting it.  gettextonfly is not
called anywhere except from in the xlateOnFly function, at which point it is
only reading through the array of i18n messages that was created as the page
loaded (through the modified gettext function).  when that array is created,
it is populated with objects that contain the original optional arguments
passed into the gettext function for each item, so those are then used in
the gettextonfly function without needing to pass them as their own
argument.

* Are i_args.pre or i_args.post set anywhere?
*these can be included as part of the optional argument object that gets
passed into gettext.  the only example of where they are being used right
now is in hist.js.  as the different history actions get loaded, the
different dropdowns load the actions, and keep count of the number of each
action that was executed, which gets added to the text of the select
option.  Because of that, when the translation on the fly occurs, the text
that needs to be replaced is different than what it was when it loaded.
With the pre and post options, the additional text can be added back in
after the translation:
gettext(txt, {id:sel.name, param:'options[' + i + '].text', pre:o.count+"
"});

The reload solution should be used for pages where xlate on fly is not an
option.  if that is not the case now, then I will fix it to include that.

Please let me know if you want any further clarification on any of the
specific points of this email, and I'd be happy to explain further.  Also, I
would be happy to help out with getting the dynamic loading of the various
i18n files set up.

Thanks,
Jeff

*
*
On Thu, Jun 26, 2008 at 9:03 PM, Rainer Blome <rainer.blome at gmx.de> wrote:

> David Pean wrote:
> > Note: Jeff made some cool tweaks on the language selector so you can
> > preview/see the language changes on the fly.  Try out Deutsch
>
> Far out!  Clever.  Works well.  A bit obfuscated, maybe.
> 403 character in one line.  Is your carriage return key out of
> commission? ;-)
>
> xlateOnFly can be simplified, no eval needed:
>
> function xlateOnFly(lang) {
>        for (var ii=0; ii<i18n_arr.length; ii++) {
>                var txt= gettextonfly(i18n_arr[ii].i_id, lang);
>                var a= i18n_arr[ii].i_args;
>                if ( ! a) {
>                        var elt= document.getElementById("i18n_"+ii);
>                        if (elt) { elt.innerHTML = txt; }
>                }
>                else {
>                        if (a.id && a.param) {
>                            var elt= document.getElementById(a.id);
>                            if (elt) {
>                                elt[a.param]= (a.pre ? a.pre : "")
>                                    + txt + (a.post ? a.post : "");
>                            }
>                        }
>                }
>        }
> }
>
> Why "ii" instead of just "i"?
> Why "i_id" instead of just "id", and "i_args" instead of just "args"?
> (these may be JS newbie questions)?
>
> Are i_args.pre or i_args.post set anywhere?
>
> gettextonfly(id, lang) should take optional arguments, to be used in
> formatting.  Same for xlate and gettext.  If a special treatment is
> needed, such as for the dialog buttons (attribute values in general), a
> separate function should be used.
>
> The reload solution always works, why not keep that for those pages
> where xlateOnFly does not work?
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikia.com/pipermail/search-ui/attachments/20080626/2184d83d/attachment-0001.html 

From rainer.blome at gmx.de  Fri Jun 27 02:04:55 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 27 Jun 2008 04:04:55 +0200
Subject: [search-ui] Updates
In-Reply-To: <e49dbf0d0806261811i2c337a6co7d1163038a7b366d@mail.gmail.com>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>	<48643694.8040904@gmx.de>
	<e49dbf0d0806261811i2c337a6co7d1163038a7b366d@mail.gmail.com>
Message-ID: <48644AC7.5040403@gmx.de>

The search.html in SVN is missing the changes that I submitted.
The live one has them.

David Pean wrote:
> We use a different copy of search.html on the server vs in SVN.
> 
> The live version includes one minified file of all the js files used,
> plus a concatenated css file.
> 
> Dave
> 
> On Thu, Jun 26, 2008 at 8:38 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
>> The search.html in svn r279 is older than the one on re.search.wiki.com?
>> _______________________________________________
>> Search-UI mailing list
>> Search-UI at wikia.com
>> http://lists.wikia.com/mailman/listinfo/search-ui
>>
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
> 


From rainer.blome at gmx.de  Fri Jun 27 10:48:44 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 27 Jun 2008 12:48:44 +0200
Subject: [search-ui] Translate on the fly
In-Reply-To: <9131eace0806261834j5b95d582icf8f4aa976a0b8d0@mail.gmail.com>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
	<48643C4F.8050803@gmx.de>
	<9131eace0806261834j5b95d582icf8f4aa976a0b8d0@mail.gmail.com>
Message-ID: <20080627104844.304770@gmx.net>

The attachment is a tar of a new i18n/ directory (just that).
It should work when unpacked in r279/re.search/cool/js.
Contents are now:
 de.js - Updated translation table for German (tons of translated language names)
 fr.js - Translation table for French, nothing new
 init.js - Just a call to i18n.init(), this should be loaded last.
 i18n.txt - Documentation, todo list, no need to load those
 i18n.js - integrated xlateOnFly with "static" loading of translation tables, BEWARE status

> Von: "Jeffrey J Tierney" <jeffrey.tierney at gmail.com>
> ... cleaning [xlateOnFly] up ...

gettext now stores a suitable "update" closure instead of the i_args objects.
This makes it much easier to see what is going on.
xlateOnFly becomes a two-liner with this.
(Your stuff is still there, so it's not a true two-liner yet.)
(At the moment, I think i_args is still stored, but it's not used any more).
xlateOnFly does not have to know how to update an element any more.
If new types of elements are needed, xlateOnFly does not have to change.

> ... selectively loading the different language files...

The state of this is as described in my previous mail.
The improved way of choosing the effective language is described there as as well.

> I would be happy to help out with getting the dynamic loading of the various
> i18n files set up.

If you could get request_table("fr") to truly dynamically load fr.js, 
(get contents as string, eval it, i18n.fr is then available),
that would be swell.

> ... try to integrate [xlateOnFly] with [request_table] ...

xlateOnFly should get (load) the language table and
pass it to the update closure (or to gettextonfly, if we don't use closures)
or maybe just set i18n.language.
This idea is not yet reflected in the attached tar.
Currently, the language name is passed to gettextonfly.
This leaves the table-finding to gettextonfly, which has to do that
over and over again, hundreds of times for a single page.  
Yes, there are hundreds of items in the i18n_arr.


>> gettextonfly(id, lang) should take optional arguments, to be used in
>> formatting

> the original optional arguments passed into the gettext function ...
> are then used in the gettextonfly function without needing to pass them as their own
> argument.

This would do what is needed, but it is not done, neither in SVN nor in live i18n.js.
Currently gettextonfly has no way to get at the orginal args (live code below).
The closure knows the original args and could pass them to gettextonfly.
 gettextonfly(translationtable, textid [optional arguments])

The current gettext should be renamed to getspan, to make clearer what it does.
xlate should use a dedicated function (not getspan) that directly uses the already known element id.

Rainer

function gettextonfly(id, lang) {
	var text= id;
	var langExists = eval("i18n." + lang);
	if (langExists) {
		text= langExists[id];
		if (undefined == text) { text= id; }
	}
	// FIXME: Format the message with the given args
	return text;
}

-- 



Psssst! Schon vom neuen GMX MultiMessenger geh?rt?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: i18n.r279.tgz
Type: application/octet-stream
Size: 11209 bytes
Desc: not available
Url : http://lists.wikia.com/pipermail/search-ui/attachments/20080627/6c9cd001/attachment.obj 

From rainer.blome at gmx.de  Fri Jun 27 19:23:19 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 27 Jun 2008 21:23:19 +0200
Subject: [search-ui] get_involved.html states wrong search-ui URL
In-Reply-To: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
Message-ID: <48653E27.9070403@gmx.de>

In the "Search Interface" section of get_involved.html,
it says "please join the Search UI mailing list 
http://lists.wikia.com/mailman/listinfo/searchwiki".

This suggests that the searchwiki list is the Search UI's list, which it 
is not.

The correct URL is:
http://lists.wikia.com/mailman/listinfo/search-ui

Rainer

From aaron.wright at gmail.com  Sat Jun 28 21:19:10 2008
From: aaron.wright at gmail.com (Aaron Wright)
Date: Sat, 28 Jun 2008 17:19:10 -0400
Subject: [search-ui] get_involved.html states wrong search-ui URL
In-Reply-To: <48653E27.9070403@gmx.de>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>
	<48653E27.9070403@gmx.de>
Message-ID: <d0fd42530806281419j2f918584p2895e832e9d0a55f@mail.gmail.com>

Fixed.

--Aaron

On Fri, Jun 27, 2008 at 3:23 PM, Rainer Blome <rainer.blome at gmx.de> wrote:
> In the "Search Interface" section of get_involved.html,
> it says "please join the Search UI mailing list
> http://lists.wikia.com/mailman/listinfo/searchwiki".
>
> This suggests that the searchwiki list is the Search UI's list, which it
> is not.
>
> The correct URL is:
> http://lists.wikia.com/mailman/listinfo/search-ui
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>

From rainer.blome at gmx.de  Mon Jun 30 13:21:26 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 30 Jun 2008 15:21:26 +0200
Subject: [search-ui] "Deprecated method document.getSelection() called.
 Please use window.getSelection() instead."
In-Reply-To: <20080627104844.304770@gmx.net>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>	<48643C4F.8050803@gmx.de>	<9131eace0806261834j5b95d582icf8f4aa976a0b8d0@mail.gmail.com>
	<20080627104844.304770@gmx.net>
Message-ID: <4868DDD6.8070300@gmx.de>

When I delete or undelete a result, Firefox shows this informational 
message in the error console:
"Deprecated method document.getSelection() called.  Please use 
window.getSelection() instead.

From rainer.blome at gmx.de  Mon Jun 30 14:24:16 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 30 Jun 2008 16:24:16 +0200
Subject: [search-ui] Loading tables on demand
In-Reply-To: <20080627104844.304770@gmx.net>
References: <e49dbf0d0806260906o4c9f0b8cjdfd0926f07f4c708@mail.gmail.com>	<48643C4F.8050803@gmx.de>	<9131eace0806261834j5b95d582icf8f4aa976a0b8d0@mail.gmail.com>
	<20080627104844.304770@gmx.net>
Message-ID: <4868EC90.2070009@gmx.de>

Got it working (locally).  Translation tables are now loaded on demand.
To allow for timeouts and error handling, this is done asynchronously.
This complicates the code a bit, but is worth it, in my opinion.
The attachment is a tar of js/ with the new files.

cd re.search;
tar xvf cool.i18n.r279.ondemand.tgz

As always, edit cool/JSON/servertouse.js.

Contents are now:
search.html - internationalized version
JSON/servertouse.js - loads js/i18n/i18n.js
js/i18n/de.js - Current translation table for German
js/i18n/fr.js - Translation table for French, nothing new
js/i18n.txt - Documentation, todo list, no need to load those
js/langselect.js - Adapted to new functionality
js/i18n/i18n.js -
o Loads translation tables on demand.
o Debugging alerts can be turned on by setting logger's "verbosity" to
   higher values (6 turns them all on).
o xlateOnFly shares the new property i18n.onfly_table with gettextonfly.
o i18n_arr replaced by i18n.updaters, which stores just closures.
o BEWARE status,
o not tested via HTTP (local file system only)

In case you used the previous tar file, js/i18n/init.js is no longer needed.

Rainer

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cool.i18n.r279.ondemand.tgz
Type: application/octet-stream
Size: 23779 bytes
Desc: not available
Url : http://lists.wikia.com/pipermail/search-ui/attachments/20080630/94590062/attachment-0001.obj 

From rainer.blome at gmx.de  Mon Jun 30 16:06:46 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 30 Jun 2008 18:06:46 +0200
Subject: [search-ui] Loading tables on demand (2)
Message-ID: <20080630160646.5960@gmx.net>

[This is a resend of this mail, using a different way to send it,
 as the original did not turn up for at least an hour.]

Got it working (locally).  Translation tables are now loaded on demand.
To allow for timeouts and error handling, this is done asynchronously.
This complicates the code a bit, but is worth it, in my opinion.
The attachment is a tar of js/ with the new files.

cd re.search;
tar xvf cool.i18n.r279.ondemand.tgz

As always, edit cool/JSON/servertouse.js.

Contents are now:
search.html - internationalized version
JSON/servertouse.js - loads js/i18n/i18n.js
js/util.js - now uses gettextonfly instead of gettext, where appropriate
js/langselect.js - Adapted to new functionality
js/i18n/de.js - Current translation table for German
js/i18n/fr.js - Translation table for French, nothing new
js/i18n.txt - Documentation, to do list, no need to load those
js/i18n/i18n.js -
o Loads translation tables on demand.
o Debugging alerts can be turned on by setting logger's "verbosity" to
  higher values (6 turns them all on).
o xlateOnFly shares the new property i18n.onfly_table with gettextonfly.
o i18n_arr replaced by i18n.updaters, which stores just closures.
o BEWARE status,
o not tested via HTTP (local file system only)

In case you used the previous tar file, js/i18n/init.js is no longer needed.

Rainer

-- 
GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
Jetzt dabei sein: http://www.shortview.de/wasistshortview.php?mc=sv_ext_mf at gmx
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cool.i18n.r279.ondemand.tgz
Type: application/octet-stream
Size: 23779 bytes
Desc: not available
Url : http://lists.wikia.com/pipermail/search-ui/attachments/20080630/e6458978/attachment-0001.obj 

From rainer.blome at gmx.de  Mon Jun 30 19:11:54 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Mon, 30 Jun 2008 21:11:54 +0200
Subject: [search-ui] Loading tables on demand (2)
In-Reply-To: <20080630160646.5960@gmx.net>
References: <20080630160646.5960@gmx.net>
Message-ID: <48692FFA.1040606@gmx.de>

In i18n.js, in the logger object literal, there is a trailing comma 
before the closing parenthesis.  It prevents the script from loading in 
IE.  Remove the comma and it should work in IE.

By the way, it should work when applied to r281 (that's what I tested it 
with), r279 is just what I started from.

Rainer

From jeremie at jabber.org  Mon Jun 30 19:18:17 2008
From: jeremie at jabber.org (Jeremie Miller)
Date: Mon, 30 Jun 2008 14:18:17 -0500
Subject: [search-ui] Loading tables on demand (2)
In-Reply-To: <48692FFA.1040606@gmx.de>
References: <20080630160646.5960@gmx.net> <48692FFA.1040606@gmx.de>
Message-ID: <E3830606-BB85-4E83-9D0F-402DA0605403@jabber.org>

I love this stuff, thanks Rainer, being able to dynamically load any  
language seems so futuristic, yet it's here :)

On Jun 30, 2008, at 2:11 PM, Rainer Blome wrote:

> In i18n.js, in the logger object literal, there is a trailing comma
> before the closing parenthesis.  It prevents the script from loading  
> in
> IE.  Remove the comma and it should work in IE.
>
> By the way, it should work when applied to r281 (that's what I  
> tested it
> with), r279 is just what I started from.
>
> Rainer
> _______________________________________________
> Search-UI mailing list
> Search-UI at wikia.com
> http://lists.wikia.com/mailman/listinfo/search-ui
>


From rainer.blome at gmx.de  Fri Jun 20 22:04:17 2008
From: rainer.blome at gmx.de (Rainer Blome)
Date: Fri, 20 Jun 2008 22:04:17 -0000
Subject: [search-ui] Internationalization patch [Was: Re:  i18n, l10n]
In-Reply-To: <485B57CD.7060402@gmx.de>
References: <aad1535c0805201239v7c836991t2b93f87fb5628661@mail.gmail.com>	<a6424d870805201242y2eca4952x191c5d141eb8ee28@mail.gmail.com>	<4f8a4c2c0805221204p4196c5a2o8045baf148c28e88@mail.gmail.com>	<485003D1.8000906@wikia.com>	<48517A3E.90108@gmx.de>	<1EF6B671-659D-4A0B-98B5-E33FFEA4FEA3@jabber.org>	<4855A058.7080707@gmx.de>
	<485B57CD.7060402@gmx.de>
Message-ID: <485C290F.10008@gmx.de>

Rainer Blome wrote:
> To my embarrassment, I have not tested applying this patch, not enough 
> space in the file system at the moment to make a copy and try it.

If something can go wrong, it will.  After cleaning up a bit, I tried
to apply that patch.
Well, svn diff apparently does not output something understandable by 
Cygwin patchutils' patch.  Hunks fail, paths have to be manually 
specified, no good.

Instead, I prepared and attached a tar file that can be unpacked over a 
re.search r262 checkout.  This time I tested it, it works.

cd re.search
svn update -r262
cd ..
tar xvzf re.search-i18n-mod.tar.gz
cd re.search/cool
# Configure search_server_to_use (value must end in cool/):
emacs JSON/servertouse.js
firefox

Rainer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: re.search-i18n-mod.tar.gz
Type: application/unix-tar
Size: 41130 bytes
Desc: not available
Url : http://lists.wikia.com/pipermail/search-ui/attachments/20080620/07fc7e3f/attachment.bin