from_name: Ray Saintonge

subject: [Wikitech-l] French Wikipedia and utf-8

body:
Vincent Ramos wrote:

> * improve interwiki links: any non latin-1 (ISO-8859-1 = default charset)
> link would be possible without any transformation. Many non latin-1
> links are copied and pasted in raw format by interwiki wanderers but
> not checked afterwards; they are always miscoded, which results in loss
> of time to fix them;

Most French accented characters are interchangeable between Unicode,
ISO-8859-1, and the more recent ISO-8859-15 (which was a modification
that allowed for such things as the Euro symbol).  It's the other
renditions that seem to cause the problem.

> * improve orthography (and articles naming): French uses the famous <oe>
> digraph that is not encoded by latin-1 (latin-9 does); every editor
> must either type the HTML entity œ or prefer not to encode it,
> resulting in misspelled words (one of our bots, Orthogaffe, when it
> was used for orthography purpose, had many "oeuvre -> œuvre"
> replacements to do);

I can use Alt+0156 to create the ? ligature, but I believe that this is
in the unstable area of coding.  It is not normally on the keyboard.
 Simply using "oe" without a digraph is not a spelling error, but a
breach of typographical convention.  Unfortunately most books that I do
use for French language reference use the digraph but do not discuss the
problem at all.  The book by L?andre Bergeron, "Dictionnaire de la
langue qu?b?coise", uses the two letter format in its listings, but also
without explanation.  Alphabetical lists treat the ligature as if it
were two letters, so it should be treated as optional, and the
initiating author's choice should be respected.  I would simply use the
two letter form and would object if it were changed.  In article titles
the author's choice should also be respected, but a redirect should be
set up from the alternative.

> * terminate transcodage problems: many editors do not use Windows
> and its codepages; other do, but with Win-1252 or Unicode as default
> charset. When some text is pasted from an application not using strict
> latin-1 (but Win-1252, MacRoman, etc.) to some wiki editing area,
> it is badly transcoded by the Wiki-soft, resulting in many raw
> quotation marks and <oe> ligatures being replaced by question marks.

Yes, I would be glad to be rid of the anoying question marks, squares or
diamonds.

>     Cons:
> * any text containing non ASCII characters would increase
> its weight : instead of one byte for a single <c with cedilla>, it
> would require two; French uses lots of non ASCII characters, as
>
The diamonds that appeared in your original letter did not reproduce at
all when I quoted the letter for this answer. The c-cedilla is a part of
ISO-8859-1 and I enter it with Alt+0231.  The two byte encoding should
not be a worry.  It is unavoidable for all chinese characters.

>     Would it be possible, thus, to make utf-8 default charset
> for the French Wikipedia?

I believe that it should be the standard for all the Wikis.

Ec


list: wikien-l

source: pipermail-mbox

from_name: Arvind Narayanan

subject: [WikiEN-l] WRONG LIST!!! (was: Copyright violation)

in_reply_to:
<3FB81A0F.2040907@telus.net>;
	from saintonge@telus.net on Sun, Nov 16, 2003 at 04:45:03PM -0800

body:
On Sun, Nov 16, 2003 at 04:45:03PM -0800, Ray Saintonge wrote:
> Daniel Mayer wrote:
>
> >Ec wrote:
> >
> >>This issue will affect more people than those who sit on the
> >>Wikilegal list.
> >>
> >
> >Oh, so that is why you are OK with this thread being on the English mailing
> >list? Do all the other people on all the other language versions of Wikipedia
> >along with those people involved with other Wikimedia projects not matter?
> >Please.
> >
> I didn't start the thread but would likely have used wikipedia-l if I
> had.  You're right to say that it isn't an English-only problem..  That
> would be a good place to start something with such broadly applicable
> implications.
>
I started the thread; sorry I didn't know about wikilegal-l. I've been
CCing my followup mails to wikilegal.

Arvind
--
Its all GNU to me


from_name: Daniel Mayer

subject: [WikiEN-l] Copyright violation

body:
Toby wrote:
>[Posting to <wikiEN-L> since we could change the text
>only on [[en:]] -- and we may need to only on [[en:]] if that's
>all that's being violated. [[fr:]] et al can then copy us if they
>find it necessary, or desirable; or even do their own thing,
>within reason.]

If it makes sense for en.wikipedia then it makes sense to translate it. And
what about Wikibooks and Wiktionary?

-- mav


list: wikitech-l

from_name: Tom Lubbock

subject: [Wikitech-l] Re: Web space request

body:
i have the requisite space and bandwidth to host your dumps
tom lubbock
tom.lubbock at bnc.ox.ac.uk

"Alfio Puglisi" <puglisi at arcetri.astro.it> wrote in
message news:Pine.GSO.4.58.0311171011350.12323 at hercules...
> Hello,
> for a while I've been generating html dumps of Wikipedia, that I would
> like now to make public (since similar things are popping up left and
> right, and it seems that it would be useful for legitimate mirroring).
>
> Unfortunately, I lack the web space/bandwidth to host it myself. The
> complete, compressed dump of the English wikipedia is around 750MB,
> including all images. Adding the other languages (the ones using MediaWiki
> software) it's about 1 GB. Removing images it goes down to maybe 200
> megabytes.
>
> Is there anyone with a suitable hosting? It needs aboud 1GB of space (and
> growing a bit week after week :-), and reasonable bandwidth. Also, should
> the dump include images or copyright issues are still unresolved?
>
> Ciao,
> Alfio


from_name: clist at cox.net

subject: [Wikitech-l] Danish wiki is messed up

body:
Somebody has messed with the installation of the danish wiki.
Yesterday I asked on Brion Vibbers todo list for an update to LanguageDa.php, but this morning somebody has replaced it with the english one, and apparently transformed it to UTF-8.
The result is that all the danish letters are broken when somebody tries to edit an article, and on most pages the danish letters look funny.

I have blocked the database for editing on danish until it is fixed, so we don't loose to many danish letters.

Could somebody please re-install version 1.4.2.6 of LanguageDa.php and put it back to ISO-8859-1 coding?

Regards
Christian List


list: wikipedia-l

from_name: Jimmy Wales

subject: [Wikipedia-l] Server update

body:
The part that penguincomputing has been waiting on is now due to be in
tomorrow.  They will build the system, do the 2 day burn-in, and ship
it on Thursday, assuming nothing goes wrong.

This will mean, I think, that Jason will have it Friday or Monday.
It'll be up to him exactly when he goes to San Diego to install it,
but early next week anyway.

--Jimbo


subject: [Wikitech-l] Server update


subject: [Wikipedia-l] bank balances

body:
SunTrust account is unchanged since last update, unless some bank
fees were charged:
$1,862.23

Paypal has various amounts in various currencies, with a USD value of
$1,533.20 -- this is up just over $200 from the last time I reported,
1 week ago.

I can absolutely that removing the donation text from all the pages
has resulted in a significant drop in incoming funds, as expected.

--Jimbo


from_name: Poor, Edmund W

subject: [WikiEN-l] Open letter to Ed Poor: On heavy-handed sysops

body:
Viajero,

Thank you for an outstandingly sensitive and penetrating analysis.
You've really given me food for thought.

I'm going to need time to digest your comments, which could mean that I
better not spend any of that precious time editing articles for a while.

Ed Poor


from_name: Brion Vibber

subject: [Wikipedia-l] Re: Getting rid of admin status

references:
<sh65hi99zs.fsf@tux.gnu.franken.de>
	<20031117184045.GA6279@tomk32.homelinux.org>
	<200311172011.15555.mail@ulrich-fuchs.de>
	<20031117194335.GC6279@tomk32.homelinux.org>
	<shoeva7miq.fsf@tux.gnu.franken.de>

body:
On Nov 17, 2003, at 12:44, Karl Eichwalder wrote:

> "Thomas R. Koll" <tomk32 at gmx.de> writes:
>
>> No, but if, then he wasn't correct enough.
>
> Yes, my mail was ambiguous.  Uli got it right :)

What's your username?

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20031117/abc2afed/attachment.pgp


subject: [Wikitech-l] Re: French Wikipedia and utf-8

references:
<opryskgacgo7frcm@localhost> <3FB91709.70500@telus.net>
	<opryswsfylo7frcm@localhost>

body:
Vincent Ramos wrote:

> Le Mon, 17 Nov 2003 10:44:25 -0800, Ray Saintonge
> <saintonge at telus.net> a rit:
>
>> I can use Alt+0156 to create the ligature, but I believe that this is
>> in the unstable area of coding.  It is not normally on the keyboard.
>>  Simply using "oe" without a digraph is not a spelling error, but a
>> breach of typographical convention.
>
> The question is not clearly answered and many French linguists and
> grammarians do not agree. If you read French, I may suggest you these
> documents:
> <http://www.gutenberg.eu.org/pub/GUTenberg/publicationsPDF/25-andre.pdf>
> <http://www.gutenberg.eu.org/pub/GUTenberg/publicationsPDF/22-yannis.pdf>.
>

I've looked at them.  I find the first to be terribly pedantic, and the
second to really be concerned about much broader issues about digraphs
in general.

Some of the French arguments are hilarious.  Being outman?vred by the
Icelanders who wanted to insure that the thorn was included?
 Considering the digraph as a separate letter so that Georges Perec
could avoid considering the e of the digraph as a real e in a novel that
he wrote without using the letter e?  WOW!!!

Assuming that my facts are not mistaken, the strongest argument against
the digraph is that it does not even appear on a standard French keyboard.

Ec


subject: [Wikitech-l] Re: [Mediawiki-l] Image Deletion Huh?

body:
> On Nov 17, 2003, at 12:38, Evan Prodromou wrote:
>> Am I nuts, or did Image deletion get horked somewhere between 20030829
>> and 20031107? I can't delete images anymore.
>
> Yeah, that's broken. Sorry. :(
>
> It should be a fairly easy fix, but just hasn't got gotten to.

Okay, got to it. In stable CVS, patch attached.

I'll put out a 20031118 point release with these little fixes soon. :)

One note: it used to be that any logged-in user could delete a past
revision of an uploaded file. There's no way to restore them, and no
logging is done except the web server's logs. Until this gets cleaned
up, past revisions are only deletable by sysops. Any user can still
revert an old version to the current version, which is equivalent to
re-uploading a copy.

-- brion vibber (brion @ pobox.com)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-imagedelete.diff
Type: application/text
Size: 675 bytes
Desc: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031117/68cbfe4d/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031117/68cbfe4d/attachment.pgp>


subject: [Wikipedia-l] Switching everything to UTF-8

body:
On Nov 17, 2003, at 15:02, Tomasz Wegrzanowski wrote:
> Staying so long with ISO 8859 was a mistake.
>
> So I propose converting all Wikipedias that aren't using UTF-8 yet to
> UTF-8.
> Procedure should be like that:
[...]

How about we do the conversion when installing the new big database
server? (Tentatively next week if they actually ship the machine on
time.)

We'll have to go down to read-only mode while copying stuff over
anyway, so this consolidates downtime. The conversion itself can be
done by simply piping the database dump through iconv as it's being
copied into the new db.

Easy as pie!

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20031117/fced47f1/attachment.pgp


subject: [Wikitech-l] Re: [Wikipedia-l] Switching everything to UTF-8

body:
On Nov 17, 2003, at 15:02, Tomasz Wegrzanowski wrote:
> Staying so long with ISO 8859 was a mistake.
>
> So I propose converting all Wikipedias that aren't using UTF-8 yet to
> UTF-8.
> Procedure should be like that:
[...]

How about we do the conversion when installing the new big database
server? (Tentatively next week if they actually ship the machine on
time.)

We'll have to go down to read-only mode while copying stuff over
anyway, so this consolidates downtime. The conversion itself can be
done by simply piping the database dump through iconv as it's being
copied into the new db.

Easy as pie!

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031117/fced47f1/attachment.pgp>


from_name: Christopher Mahan

subject: [Wikitech-l] Server update

body:
--- "Poor, Edmund W" <Edmund.W.Poor at abc.com> wrote:
> I'm not sure everyone realizes how much of a sacrifice in time and
> money
> it is, for Jimbo to do all the shopping and follow-up - and for him
> to
> send employees of Bomis to set up equipment.

I for one do, since I have to do it where I work and it's a royal P
in the proverbial A.

=====
Christopher Mahan
chris_mahan at yahoo.com
818.943.1850 cell
http://www.christophermahan.com/

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


from_name: Delirium

body:
Tomasz Wegrzanowski wrote:

>It would be better if numeric entities were converted too.
>The code is somewhere in Phase1->Phase2 conversion script,
>and in konwert program too.
>
>
I'm not sure what other peoples' feelings on this are, but I personally
would prefer using numeric entities on the English wikipedia for
characters that aren't standard in English.  Many of us don't have
keyboard configurations that allow the literal characters to be typed,
so keeping them as numeric entities makes editing them easier.

As for the UTF-8 switch itself, I can see some advantages, but I think
you gloss over the disadvantages too quickly.  On Mozilla Firebird on
Linux, at least, the UTF-8 wikis look _much_ worse than the ISO-8859-x
wikis, because they're forced to use a Unicode font, of which only
fairly ugly ones with strange proportioning are Freely available.

-Mark


from_name: Peter Gervai

subject: [Wikipedia-l] Downloading Wikipedia (was: please help me)

body:
On Fri, Nov 14, 2003 at 07:00:31PM +0100, Axel Boldt wrote:
> Brion Vibber wrote:
>
> >Please do *not* use programs such as Webstripper. Especially on dynamic
> >sites like Wikipedia, they create a huge amount of load on the servers by
[...]

> While I of course agree with Brion's statement, I think we should also
> try to understand the other side. When I was still on a dial-up line
> that charged by the minute, I also several times tried to download
> interesting sites all at once, so that I could read them leisurely
> off-line. That seems to be quite natural behavior. You can't really
> learn stuff when the clock is ticking.
>
> Don't we have a nice compressed static HTML tree by now that we could
> offer people under the "Download Wikipedia" heading on the main page?

I see no problem about the possibility to generate a whole Wikipedia
(mirror) site as a static snapshot. Probably it could be done with minimal
modificaiton of the scripts. Static content can be retrieved by bots
without causing any trouble.

Consider it as a special mirror.

I'm not volunteering though ;-)

g.


subject: [Wikipedia-l] RE: [Wikitech-l] Server update

body:
Thanks for arranging this, Jimbo.

I'm not sure everyone realizes how much of a sacrifice in time and money
it is, for Jimbo to do all the shopping and follow-up - and for him to
send employees of Bomis to set up equipment.

Ed Poor


subject: [Wikitech-l] Server update


list: textbook-l

from_name: Karl Wick

subject: [Textbook-l] wikibooks written up in The Hindu

body:
This is an Indian newspaper that I just noticed wrote an
article about our site a couple of weeks ago:

http://www.hindu.com/biz/2003/10/27/stories/2003102700010200.htm

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


from_name: Vincent Ramos

subject: [Wikitech-l] French Wikipedia and utf-8

body:
	I was wondering about the pros and cons of utf-8 for the French
Wikipedia:

	Pros:
* improve interwiki links?: any non latin-1 (ISO-8859-1 = default charset)
link would be possible without any transformation. Many non latin-1
links are copied and pasted in raw format by interwiki wanderers but
not checked afterwards; they are always miscoded, which results in loss
of time to fix them;

* improve orthography (and articles naming): French uses the famous <oe>
digraph that is not encoded by latin-1 (latin-9 does); every editor
must either type the HTML entity œ or prefer not to encode it,
resulting in misspelled words (one of our bots, Orthogaffe, when it
was used for orthography purpose, had many "oeuvre -> œuvre"
replacements to do);

* terminate transcodage problems: many editors do not use Windows
and its codepages; other do, but with Win-1252 or Unicode as default
charset. When some text is pasted from an application not using strict
latin-1 (but Win-1252, MacRoman, etc.) to some wiki editing area,
it is badly transcoded by the Wiki-soft, resulting in many raw
quotation marks and <oe> ligatures being replaced by question marks.

	========

	Cons:
* any text containing non ASCII characters would increase
its weight : instead of one byte for a single <c with cedilla>, it
would require two; French uses lots of non ASCII characters, as
? ? ? ? ?;
* I do not see other cons.

	Would it be possible, thus, to make utf-8 default charset
for the French Wikipedia?

			Vincent Ramos


from_name: Tomasz Wegrzanowski

subject: [Wikitech-l] French Wikipedia and utf-8

body:
On Mon, Nov 17, 2003 at 06:06:00PM +0100, Vincent Ramos wrote:
> 	Cons:
> * any text containing non ASCII characters would increase
> its weight : instead of one byte for a single <c with cedilla>, it
> would require two; French uses lots of non ASCII characters, as
> ? ? ? ? ?;

It's not obvious it's going to be bigger at all - you need only 2-3 chars
instead of about 8 (ሴ) for characters not in Latin 1.

French main page:
Latin 1 (as is):		28035
Naively converted to UTF-8:	28235 (0.7% bigger)
With all &entities; UTF-8-ized	27867 (0.6% smaller)

The last number is smaller than Latin 1, but some of it is cheating -
French main page contained &codes; for some characters that could be represented
directly (ü), and some of conversions weren't completely legal (&->&).

Still, there's no reason to believe UTF-8 is going to be significantly worse
spacewise than ISO-8859-1. For English Wikipedia it's going to be obviously smaller,
as it uses even less characters from 128-255 range.


from_name: Karl Eichwalder

subject: [Wikipedia-l] Getting rid of admin status

body:
Please help me to get rid of the so called Admin status (German
WP). Who is able to tweak the database system accordingly?

--
                                                         |      ,__o
                                                         |    _-\_<,
http://www.gnu.franken.de/ke/                            |   (*)/'(*)


from_name: ray at ganymede.org

subject: [Wikitech-l] number/values for namespaces?

body:
I see the numeric values in the database for the namespaces. I do not
see documentation anywhere which describes, in human terms, to what
each number corresponds.

Should I just be looking in the source for this stuff, or is there doc
I have missed?

thanx - ray


from_name: Chris Seaton

subject: [Wikitech-l] French Wikipedia and utf-8

body:
On Mon, 2003-11-17 at 18:33, Luc Van Oostenryck wrote:
> Vincent Ramos wrote:
> >
> >
> >     I was wondering about the pros and cons of utf-8 for the French
> > Wikipedia:
> >
>
> %<--
>
> >
> >     Would it be possible, thus, to make utf-8 default charset
> > for the French Wikipedia?
> >
> >             Vincent Ramos
> >
>
> Please don't! This is totally unneeded for the french charset and will only causing problems
> with browser which don't support UTF-8.

Don't you ever use mathematical symbols in France?

> -- Looxix
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikitech-l


body:
On Nov 17, 2003, at 18:47, ray at ganymede.org wrote:
> I see the numeric values in the database for the namespaces. I do not
> see documentation anywhere which describes, in human terms, to what
> each number corresponds.

Right at the top of Language.php you'll see the number <-> name
assignments.

Don't trust what documentation there is! :) It's woefully out of date.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031117/53d6576d/attachment.pgp>


from_name: viajero at quilombo.nl

body:
Dear Ed,

I hope you had a nice weekend.

About the Mother Teresa article.

I got involved in that article not out profound interest in the subject
but because I was distressed to see that Adam Carr withdrew (figuratively
speaking!)  from Mother Teresa because of his frustration at the stubborn
insistence of an indignant Catholic user, Aplank (now known as
Alexandros), that all criticisms of her be removed from the article.  As
you may know, Adam Carr, who only joined Wikipedia in early September, has
proven to be an outstanding historian.  He prepared an alternative version
of the MT article at the end of October to resolve a contentious debate
over the existing one (and whose version it later replaced).  As you also
know, Aplank, for reasons now well understood, has had a lot of trouble
understanding Wikipedia collaborative editing conventions and accepting
alternate, critical points of view.

At the beginning of last week, it was protected by a brand new sysop,
Secretlondon (duly anointed just a day or two before) for the third time
in several weeks after yet again an edit war involving Aplank broke out.

We began the slow, arduous process of trying to find a way out of the
impasse.  We tried to define some ground rules. Aplank slowly began to
appreciate that he  was out of line and, crucially, we had finally
impressed on him that changes to the article *must be* discussed on the
Talk page first.  Another Catholic user, Pfortuny, agreed to restoring the
now heavily mutilated article to Adam's original, and Adam -- much my
pleasure -- took an interest in becoming involved again, only he refused
to do so unless Aplank was banned from the article.  I pointed out that
there is no precedent for "banning" a user from an article and made an
alternative proposal, which didn't work out.  But nonetheless, we were
working on a solution and we were *just* about at the point of restarting
the article when "Uncle Ed" arrived on Friday afternoon.

You unilaterally decided to unprotect that article.  You decided to RENAME
it (!!!)  Teresa of Calcutta.  You dismissed all the proceeding discussion
sight  unseen as as "unproductive talk" and "bickering" and simply ignored
it.  (I am not neutral because I was involved but I thought the percentage
of "bickering was modest.)  You made a dozen or so substantial edits to
article.  Pfortuny and Secretlondon both expressed their polite surprise
at all this.  I was less polite.

When Adam asked whether the situation with Aplank clarified, you
responded:

  Aplank has a clean slate. Why? Because I said so, and no one objected
:-)

  He doesn't need clearance from anyone to edit the article; it's been
unprotected.

(In a similar way, several days before you arbitrarily removed the
discussion of Aplank from Problem users -- a page whose usefulness I have
already questioned here -- without resolving in any way the underlying
problem. But that is also a general problem with that page:  it offers no
way of resolving issues.)

Ed, please, what *were* you thinking Friday afternoon? Did you need to
tick off  a couple more Good Deeds on your To Do list before you left for
the weekend?  Where you trying to rescue Aplank the "persecuted Christian"
from the hands of infidels???

Now the reason I am posting this is a open letter is to make the following
points and raise a question:

There are a lot of problems on Wikipedia right now and I think this is
because there are so few established procedures for dealing with conflict.
 At one end of the spectrum, one can IP-block an anonymous vandal.  At the
other, an abusive user can be hard-banned by Jimbo.  In between that,
there is nothing, except hunkering down in an edit war or slinging mud on
the Problem users page. For better or worse, a kind of anarchy reins, and
since there is so little precedence for what we are trying to build, we
have have to stumble along, experimenting as we go.

Ideally, a perfectly formed system for dealing with conflict would spring
from the brain of Jimbo Wales like Athena from the head of Zeus.  However,
in the absence of such a miracle happening, I see two alternatives:

1) a coterie of super-syops -- like yourself  -- solve problems based
personal authority and boldness of action.  Obviously useful for "putting
out fires".

2) strategies for dealing with conflict evolve organically and spread
through the Wikipedia community like Khronus's memes (!). This obviously
take time and patience and forbearance on the part of trigger-happy
sysops.

So, which way forward?

I personally would prefer the second option, even though it means a lot
more work. The first lacks transparency and lends itself to arbitrary
decisions.  It means also that Wikipedia does not evolve in a structured
way as an environment but simply relies on a few strong personalities to
impose solutions, thereby glossing over weaknesses in the system.  But
perhaps it is naive of me to think that it can be any other way.  I hope
that others here will share their viewpoints on this.

Adam Carr has since abandoned Mother Teresa, which is his choice, but I
think it is a great pity. In our slow, stumbling way, we *might* have been
able to reach  a compromise agreement with him had you not interfered. Had
we been able to, those of us involved -- Secretlondon, Pfortuny, Aplank,
myself and others -- might have learned from the experience and used it to
help resolve  similar such conflicts in the future. How else will useful
precedents for dealing with conflict evolve?

None of my comments above diminishes my respect for you as someone who is
gifted at articulating and implementing WP's NPOV philosophy.  It is just
that I don't believe that that moral authority should be construed as --
for the lack of a better term -- political power.  In the long run, it is
simply counterproductive.

-- Viajero


from_name: Dante Alighieri

subject: [WikiEN-l] Test email, ignore at will

body:
This is a test of my new email client.

-----
Dante Alighieri
dalighieri at digitalgrapefruit.com

"The darkest places in hell are reserved for those who maintain their neutrality in times of great moral crisis."
   -Dante Alighieri, 1265-1321


from_name: Luc Van Oostenryck

subject: [Wikitech-l] French Wikipedia and utf-8

body:
Vincent Ramos wrote:
>
>
>     I was wondering about the pros and cons of utf-8 for the French
> Wikipedia:
>

%<--

>
>     Would it be possible, thus, to make utf-8 default charset
> for the French Wikipedia?
>
>             Vincent Ramos
>

Please don't! This is totally unneeded for the french charset and will only causing problems
with browser which don't support UTF-8.

-- Looxix


from_name: Thomas R. Koll

body:
On Mon, Nov 17, 2003 at 06:32:23PM +0100, Karl Eichwalder wrote:
> Please help me to get rid of the so called Admin status (German
> WP). Who is able to tweak the database system accordingly?

There's still a Vote running at
http://de.wikipedia.org/wiki/Wikipedia_Diskussion:Administratoren#Bezeichnungsfrage
an as it seems the majority wants to keep the current names. Wait until a decision is
settled.

Anyways changing the naming ain't no database thing but translation issue.

Changing 532 articles (all namespaces) containing the word "Administrator", "Admin" or "Sysop"
should be done by hand.

ciao, tom
--
http://www.tomk32.de - just a geek trying to change the world
   -.-       http://de.wikipedia.org/wiki/Benutzer:TomK32
   /|>	           http://tomk32.bookcrossing.com
   / \                 http://tinyurl.com/u6de


from_name: Ulrich Fuchs

body:
> There's still a Vote running at
> http://de.wikipedia.org/wiki/Wikipedia_Diskussion:Administratoren#Bezeichnu
>ngsfrage an as it seems the majority wants to keep the current names. Wait
> until a decision is settled.

Thomas, you misunderstood: Karl want's to get rid of ''his'' admin status, so
his user record needs to be changed in the database.

Uli


body:
On Mon, Nov 17, 2003 at 08:11:15PM +0100, Ulrich Fuchs wrote:
> > There's still a Vote running at
> > http://de.wikipedia.org/wiki/Wikipedia_Diskussion:Administratoren#Bezeichnu
> >ngsfrage an as it seems the majority wants to keep the current names. Wait
> > until a decision is settled.
>
> Thomas, you misunderstood: Karl want's to get rid of ''his'' admin status, so
> his user record needs to be changed in the database.

No, but if, then he wasn't correct enough.

Karl: Please help me to get rid of the so called Admin status

your interpretation would be: Please help me to get rid of MY Admin status

ciao, tom
--
http://www.tomk32.de - just a geek trying to change the world
   -.-       http://de.wikipedia.org/wiki/Benutzer:TomK32
   /|>	           http://tomk32.bookcrossing.com
   / \                 http://tinyurl.com/u6de


from_name: Rick

subject: [WikiEN-l] User:Caius2ga

body:
[[User:Caius2ga]] is repeatedly involved in edit wars over the useage of the names of Polish cities, insisting on calling every city that ever was or ever might have been in Poland, by its current name, regardless of what it might have ever been called.  He has now begun to refer to those who disagree with him as Nazis.

RickK

---------------------------------
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/wikien-l/attachments/20031117/550d67f0/attachment.htm


from_name: Erik Moeller

subject: [Wikipedia-l] Re: [Wikitech-l] Server update

body:
Jimmy-
> The part that penguincomputing has been waiting on is now due to be in
> tomorrow.  They will build the system, do the 2 day burn-in, and ship
> it on Thursday, assuming nothing goes wrong.

> This will mean, I think, that Jason will have it Friday or Monday.
> It'll be up to him exactly when he goes to San Diego to install it,
> but early next week anyway.

That sounds promising. Can you give us an update on what our current total
donations account balance is, minus the costs for the server?

Regards,

Erik


subject: [Wikitech-l] Server update


from_name: Jake Nelson

subject: [WikiEN-l] Hijack

body:
Ray Saintonge wrote:
> It doesn't mean that the scan is copyright either; that depends on
> whether there is a creative act in making that scan.

True, though the standard for what constitutes a creative act varies widely.

> A photograph of public property is copyrightable of course, but, at
> least in Canada, taking a picture of an permanently placed building or
> outdoor statue is not a breach of copyright, even if you intend to
> publish the photograph.

Sure, of course not.... wasn't what I was saying. The issue is that many
people will agree that photographs can be creative works, but fail to see
that there's no clear distinction between scans and other photographs.

-- Jake


from_name: Anthere

body:
> >>     Would it be possible, thus, to make utf-8
> default charset
> >> for the French Wikipedia?
>
> > I believe that it should be the standard for all
> the Wikis.
>
> 	So do I. Thanks for your answer,
>
> 	Vincent

I understand your need as a linguist for these
letters. I just hope I was the last editor to ever
come to wikipedia with a browser not supporting utf8;
because any other one will just be excluded from the
project.

I fear a bit that some newbie will come one day, and
that you will have to fix everything behind him, and
then explain to him he is just not welcome. Hopefully,
it will never happen :-)

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


body:
"Thomas R. Koll" <tomk32 at gmx.de> writes:

> No, but if, then he wasn't correct enough.

Yes, my mail was ambiguous.  Uli got it right :)

Best,
Karl

--
                                                         |      ,__o
                                                         |    _-\_<,
http://www.gnu.franken.de/ke/                            |   (*)/'(*)


subject: [Wikipedia-l] bank balances

body:
Jimmy-
> I can absolutely that removing the donation text from all the pages
> has resulted in a significant drop in incoming funds, as expected.

We should set a clear new goal once the DB server is set up, and then we
can do a real fundraiser. Such a fundraiser needs one thing most
critically: live or at least daily updates of the current balance so
people get motivated to "reach the goal". Once we have acquired the
necessary funds, we can remove the donation text again. The "Donations"
link in the sidebar will always be there.

What do we need most critically? I still feel that hiring a DB expert to
look over parts of the code, especially the search, could greatly help our
performance, and restore the search functionality (which I miss a lot!).
Of course it would be nice to get this expertise for free, but so far it
hasn't been forthcoming. More web servers would be good and there are
plenty of non-techie uses for money, but I think we should try not to be
pushy asking for money. Let's try to refine our product first -- approval
mechanism, mediation process, yadda yadda yadda.

Maybe what we really need is a decent roadmap, not necessarily for the
nebulous "Wikipedia 1.0", but for Wikimedia as a whole.

Regards,

Erik


from_name: Brion Vibber

body:
On Nov 17, 2003, at 20:06, Lars Aronsson wrote:
> Erik Moeller wrote:
>> perhaps in the 5% range. Given that a single edit by such a person
>> will
>> break an entire page, it might not be so wise to switch (but perhaps
>> I'm
>> missing something -- is Meta running UTF-8?).
>
> Would it be possible to let the database run on UTF-8 internally, but
> to let the PHP script analyze and convert data to and from certain
> browsers?  Perhaps the majority of users are using UTF-8-capable
> browsers, so the conversion would use a minimum of resources.

Certainly possible, as long as care is taken to keep round-trips clean.

Another possibility is simply to 'blacklist' known problem browsers by
printing a notice/link to better browsers on the edit page warning that
they may have problems, as we now have a warning on long pages that
some browsers may have problems. (Though in that case we aren't
checking specific browsers.)

The main problem browser these days is Internet Explorer for Mac; it's
years out of date and the most recent version still doesn't grok UTF-8
for editing. The most recent Macs ship with Safari as the default, but
most existing Macs out there are going to have IE or (shudder) Netscape
4.x as the default browser.

> All I know is that MySQL has better UTF-8 support from version 4.1.x,
> as described in chapter 9, http://www.mysql.com/doc/en/Charset.html
> The same goes for Perl version 5.8, but what about PHP?

PHP currently has pretty much no UTF-8 support aside from some
conversion functions. Strings are treated as arbitrary-length byte
sequences, and we've got some custom functions to deal with case
changing and the like.

There are some multibyte character set support functions which may or
may not be suitable for replacing the Utf8Case functions, that should
get looked into.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031117/0845e72e/attachment.pgp>


from_name: Daniel Mayer

body:
Erik wrote:
>Is this true? All I know is that we had a *lot* of problems
>with broken special chars on the Meta-Wiki during the logo
>contest. I have no idea which browser broke them, but it
>seems to be a not totally uncommon one, perhaps in the
>5% range. Given that a single edit by such a person will
>break an entire page, it might not be so wise to switch
>(but perhaps I'm missing something -- is Meta running UTF-8?).

IIRC meta is. And that fact has created some of the problems you mention. I
therefore see no compelling need to convert Latin-1 languages to UTF-8 and in
fact think such a switch would be harmful. It is also wrong-headed to state
(as Tomasz did) that if people have non-UTF-8-friendly browsers that they
should upgrade. That is not the attitude we should have when things work just
fine the way they are (at least on the English Wikipedia - others may have
more compelling reasons to use UTF-8 that outweigh the negatives).

The only place where UTR-8 would be very useful is with interlanguage links.
But that could better be solved by placing all interlanguage links outside of
the regular wiki text of pages. That separate edit window could support UTF-8
and be shared by all Wikipedia's. This should minimize damage done by
non-UTF-8-compliant browsers and as an added benefit could be part of an
easier way to add language links to articles (inputing the links once would
create language links in every article listed in the common meta space).

-- Daniel Mayer (aka mav)


body:
Le Mon, 17 Nov 2003 10:44:25 -0800, Ray Saintonge
<saintonge at telus.net> a ?crit:

> I can use Alt+0156 to create the ? ligature, but I believe that this is
> in the unstable area of coding.  It is not normally on the keyboard.
>  Simply using "oe" without a digraph is not a spelling error, but a
> breach of typographical convention.

[snip]

The question is not clearly answered and many French linguists and
grammarians do not agree. If you read French, I may suggest you these
documents:
<http://www.gutenberg.eu.org/pub/GUTenberg/publicationsPDF/25-andre.pdf>
<http://www.gutenberg.eu.org/pub/GUTenberg/publicationsPDF/22-yannis.pdf>.

> The c-cedilla is a part of ISO-8859-1 and I enter it with Alt+0231.  The
> two byte encoding should not be a worry.  It is unavoidable for all
> chinese characters.

Yes; articles with non latin-1 writing systems, linguistics texts
and other are such a burden to format; moreover, html entities make
the file quite not readable?:
You see what I mean...

>>     Would it be possible, thus, to make utf-8 default charset
>> for the French Wikipedia?

> I believe that it should be the standard for all the Wikis.

	So do I. Thanks for your answer,

	Vincent


body:
Le Mon, 17 Nov 2003 18:31:40 +0100, Tomasz Wegrzanowski
<taw at users.sf.net> a ?crit:

> Still, there's no reason to believe UTF-8 is going to be significantly
> worse
> spacewise than ISO-8859-1. For English Wikipedia it's going to be
> obviously smaller,
> as it uses even less characters from 128-255 range.

I agree with you.

	Vincent


body:
Le Mon, 17 Nov 2003 19:33:19 +0100, Luc Van Oostenryck
<luc.vanoostenryck at easynet.be> a ?crit:

> Please don't! This is totally unneeded for the french charset and will
> only causing problems
> with browser which don't support UTF-8.

	Do you know what percentage do this trick?

	The point is: why do other Wikimedia projects use utf-8
and not fr:? Are there so many utf-8 non compliant browsers?

	Vincent


from_name: Ray Saintonge

body:
Anthere wrote:

>>>>    Would it be possible, thus, to make utf-8
>>>>
>>default charset
>>
>>>>for the French Wikipedia?
>>>>
>>>I believe that it should be the standard for all
>>>
>>the Wikis.
>>
>>	So do I. Thanks for your answer,
>>
>>	Vincent
>>
>
>I understand your need as a linguist for these
>letters. I just hope I was the last editor to ever
>come to wikipedia with a browser not supporting utf8;
>because any other one will just be excluded from the
>project.
>
>I fear a bit that some newbie will come one day, and
>that you will have to fix everything behind him, and
>then explain to him he is just not welcome. Hopefully,
>it will never happen :-)
>
As much as I may be strongly in favour of the UTF-8 system, I think that
it is also important that we find ways of dealing sensitively with the
kind of difficulties that you raise.

Ec

>


references:
<opryskgacgo7frcm@localhost> <3FB9146F.6000500@easynet.be>
	<opryswzz0bo7frcm@localhost>

body:
Vincent Ramos wrote:
>
> Le Mon, 17 Nov 2003 19:33:19 +0100, Luc Van Oostenryck
> <luc.vanoostenryck at easynet.be> a ?crit:
>
>> Please don't! This is totally unneeded for the french charset and will
>> only causing problems
>> with browser which don't support UTF-8.
>
>
>     Do you know what percentage do this trick?
>
>     The point is: why do other Wikimedia projects use utf-8
> and not fr:? Are there so many utf-8 non compliant browsers?
>
>     Vincent

I don't know how much browser are non-UTF-8 compliant,
but I'm sure there will problems.
My point of view is more like the following:
   why should we need UTF-8 on fr when 99.9999% of text are fairly happy with
   ISO8859-1 (the charset that most european use and french in particular)
   and named or numeric HTML entities are OK for the rest.

Note, Vincent, that I understand your needs for special characters.

-- Looxix


subject: [Wikitech-l] French Wikipedia and utf-8: Yes, but in TeX :-)

references:
<opryskgacgo7frcm@localhost> <3FB9146F.6000500@easynet.be>
	<1069095271.5585.0.camel@	william

body:
Chris Seaton wrote:
> On Mon, 2003-11-17 at 18:33, Luc Van Oostenryck wrote:
>
>>Vincent Ramos wrote:
>>
>>>
>>>    I was wondering about the pros and cons of utf-8 for the French
>>>Wikipedia:
>>>
>>
>>%<--
>>
>>
>>>    Would it be possible, thus, to make utf-8 default charset
>>>for the French Wikipedia?
>>>
>>>            Vincent Ramos
>>>
>>
>>Please don't! This is totally unneeded for the french charset and will only causing problems
>>with browser which don't support UTF-8.
>
>
> Don't you ever use mathematical symbols in France?
>

Yes, but in TeX :-)

-- Looxix


references:
<opryskgacgo7frcm@localhost> <3FB9146F.6000500@easynet.be>
	<opryswzz0bo7frcm@localhost> <3FB94BC3.4090306@easynet.be>

body:
On Mon, Nov 17, 2003 at 11:29:23PM +0100, Luc Van Oostenryck wrote:
> I don't know how much browser are non-UTF-8 compliant,
> but I'm sure there will problems.
> My point of view is more like the following:
>   why should we need UTF-8 on fr when 99.9999% of text are fairly happy with
>   ISO8859-1 (the charset that most european use and french in particular)
>   and named or numeric HTML entities are OK for the rest.

Why should we need ISO 8859-N with named or numeric HTML entities
when 100% of text works with UTF-8 ? The number is much smaller than 99.9999%.
It's probably much smaller than 99% too.

I worked with articles about Japanese language when Polish Wikipedia
used ISO 8859-2 and it wasn't fun. I wasted lot of time trying to locate
which &-entity corresponds to which character to fix a typo or make some
other change. In the end I installed local mirror Wikipedia with UTF-8,
and make a bunch of Perlscripts that converted from ISO 8859-2 + &codes;
to readable representation.


subject: [Wikitech-l] Re: Wikitech-l UTF

body:
Message: 10
Date: Tue, 18 Nov 2003 07:02:56 +0100
From: Vincent Ramos <siva-nataraja at alussinan.org>
Subject: [Wikitech-l] Re: French Wikipedia and utf-8
To: wikitech-l at wikipedia.org
Message-ID: <oprytke6ylo7frcm at localhost>
Content-Type: text/plain; format=flowed;
charset=iso-8859-15

Le Mon, 17 Nov 2003 21:35:42 -0800 (PST), Anthere
<anthere6 at yahoo.com> a ?crit:

> I understand your need as a linguist for these
> letters. I just hope I was the last editor to ever
> come to wikipedia with a browser not supporting
utf8;
> because any other one will just be excluded from the
> project.

	Yes, but non utf-8 compliant browsers are not very
common, nowadays; there should be less and less.

> I fear a bit that some newbie will come one day, and
> that you will have to fix everything behind him, and
> then explain to him he is just not welcome.
Hopefully,
> it will never happen :-)

	It is already the case with pages featuring raw
<oe> digraphs or quotation marks (which are not
latin-1
characters but, I do not know how, there are editors
who
can save pages with them): when someone with another
system edits them, every <oe> and quotation mark
is replaced by <?>. Some browsers do not respect raw
non breakable spaces and turn them into normal spaces,
etc.
There is always someone to correct these pages.

	At least, with utf-8 these problems should be less
frequent, because <oe> and <'> could be directly
encoded in a
charset which possesses them.

	Vincent

------

true Vincent

But in this case, it will not be just a few caracters
on a few pages, it will be *all* accentuated
caracters, ?, ?, ?, ?, ?, ?, ? and so on, on *all*
pages. That makes quite a difference. The user would
be listed problematic whithin less than 20 articles.

And Safari does not work on OS 9 :-(

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


body:
Staying so long with ISO 8859 was a mistake.

So I propose converting all Wikipedias that aren't using UTF-8 yet to UTF-8.
Procedure should be like that:
1. new LanguageXX.php prepared and put under some name
2. make backups
3. create tables curutf8 and oldutf8
4. disable write access
5. convert all data - numeric HTML codes are going to be replaced by UTF-8 characters too.
6. rename tables cur and old to cur88591 and cur88591
7. rename tables curutf8 and oldutf8 to cur and old
8. replace old LanguageXX.php with utf8-enabled version
9. reenable write access

The conversion script should be tested on test.* Wikipedia first.

During step 5 Wikipedia is going to be read only. It may take some time,
especially with English Wikipedia, so it's better to do conversion of each Wikipedia
separately. During steps 6-8 Wikipedia may not work at all, but it's going to
take less than a minute.

Does anybody have any really good reason why shouldn't I proceed ?
These reasons aren't good enough:
* broken URLs     - all old URLs are going to work after upgrade
* size increase   - size is going to stay about the same
* broken browsers - they should be upgraded, if someone has browser so old
			that it doesn't grok UTF-8, it's not going to grok CSS,
			PNGs, and other things we're using either.
			Unless we want to remove all CSS and PNGs, there's
			no point in not using UTF-8.
* ISO 8859-N is good enough - no, it's not. Not if someone wants to write about
			people and places from countries where non-8859-1 Latin
			characters are used, or about linguistics, or math, etc.


subject: [Wikitech-l] Switching everything to UTF-8


body:
Here's a breakdown of browsers used to save edits over a day on
de.wikipedia.org. I checked de because it's the largest non-English
non-UTF-8-using Wikipedia.

Total posts in the day: 2861

Browsers known to work with editing UTF-8:
   MSIE/Win: 1146
   Gecko: 966 [549 Windows; 379 Linux/Unix; 31 Mac]
   Opera 6+: 331 [291 Windows; 40 Linux]
   Konqueror: 216
   Safari: 137
total: 2796 (98%)

Browsers which can be problematic with UTF-8:
   old Netscape: 25
   ELinks: 20 (not tested, but it's a text-mode browser and these tend
to have problems in this area)
   MSIE/Mac: 10
   Lynx: 4 (text-mode again)
   Opera <6: 2
total: 61 (2%)

There's some margin of error; my regexps lost track of 4 hits or so
somewhere in that mess.

I'm actually surprised that the Internet Explorer/Mac quotient is so
low, though it's a pleasant surprise. :)

Anyway, adding a notice and/or doing some extra conversion for those 2%
shouldn't be a huge server burden.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031118/44f0c352/attachment.pgp>


body:
Tomasz-
> * broken browsers - they should be upgraded, if someone has browser so old
> 			that it doesn't grok UTF-8, it's not going to grok CSS,
> 			PNGs, and other things we're using either.
> 			Unless we want to remove all CSS and PNGs, there's
> 			no point in not using UTF-8.

Is this true? All I know is that we had a *lot* of problems with broken
special chars on the Meta-Wiki during the logo contest. I have no idea
which browser broke them, but it seems to be a not totally uncommon one,
perhaps in the 5% range. Given that a single edit by such a person will
break an entire page, it might not be so wise to switch (but perhaps I'm
missing something -- is Meta running UTF-8?).

Regards,

Erik


from_name: Toby Bartels

subject: [WikiEN-l] Copyright violation

body:
Maveric149 wrote:

>Toby wrote:

>>[Posting to <wikiEN-L> since we could change the text only on [[en:]] --
>>and we may need to only on [[en:]] if that's all that's being violated.
>>[[fr:]] et al can then copy us if they find it necessary, or desirable;
>>or even do their own thing, within reason.]

>If it makes sense for en.wikipedia then it makes sense to translate it. And
>what about Wikibooks and Wiktionary?

If other 'pedias and other projects want to use that text or a translation,
then I am certainly not about to object!  But [[wp:en:]] needn't wait on them.
Hence the justification of keeping to <wikiEN-L>, where something might happen.
If you like it and want to suggest it to, say, Wikibooks,
then by all means post a message to <textbook-L>!

(I hope that that clarifies what I meant.)

-- Toby


from_name: Tomasz Wegrzanowski

body:
On Mon, Nov 17, 2003 at 03:28:46PM -0800, Brion Vibber wrote:
> On Nov 17, 2003, at 15:02, Tomasz Wegrzanowski wrote:
> >Staying so long with ISO 8859 was a mistake.
> >
> >So I propose converting all Wikipedias that aren't using UTF-8 yet to
> >UTF-8.
> >Procedure should be like that:
> [...]
>
> How about we do the conversion when installing the new big database
> server? (Tentatively next week if they actually ship the machine on
> time.)
>
> We'll have to go down to read-only mode while copying stuff over
> anyway, so this consolidates downtime. The conversion itself can be
> done by simply piping the database dump through iconv as it's being
> copied into the new db.
>
> Easy as pie!

It would be better if numeric entities were converted too.
The code is somewhere in Phase1->Phase2 conversion script,
and in konwert program too.


body:
Mav, do you have a constructive suggestion for improving compatibility
with older browsers? If so, please post it on wikitech-l, where we're
discussing the issue. We already know you don't like the idea of
breaking older browsers.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20031118/aaa07122/attachment.pgp


body:
On Nov 18, 2003, at 01:34, Daniel Mayer wrote:
> I don't care about breaking old browsers, I care about breaking
> articles and
> not being at least minimally accessable to people who use old
> browsers. One
> thing is certain, however: Switching to UTF-8 will not improve
> compatibility
> with old browsers or improve the experience of the people who use them.

I'll take that as a hypothetical acceptance of a UTF-8 solution that
explicitly retains compatibility with older browsers and thus avoids
breaking articles.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20031118/225f4a6f/attachment.pgp


subject: [Wikitech-l] MediaWiki stable release 20031118 (minor fixes)

body:
Fixes the broken maintenance scripts from yesterday's security fix
release, image deletion works again, and longstanding problems with
rebuildlinks have been more or less resolved.

Release notes:
http://sourceforge.net/project/shownotes.php?release_id=198311

Download:
http://prdownloads.sourceforge.net/wikipedia/mediawiki-20031118.tar.gz?
download

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031118/a915ea9c/attachment.pgp>


body:
Tomasz-
> * broken browsers - they should be upgraded, if
someone has browser
so old
> 			that it doesn't grok UTF-8, it's not going to
grok CSS,
> 			PNGs, and other things we're using either.
> 			Unless we want to remove all CSS and PNGs,
there's
> 			no point in not using UTF-8.

Is this true? All I know is that we had a *lot* of
problems with broken
special chars on the Meta-Wiki during the logo
contest. I have no idea
which browser broke them, but it seems to be a not
totally uncommon
one,  perhaps in the 5% range. Given that a single
edit by such a person will
break an entire page, it might not be so wise to
switch (but perhaps
I'm  missing something -- is Meta running UTF-8?).

Regards,

Erik

I confirm it is in utf 8
I broke many pages for months there, and refrained
from editing many others, and wrote all french pages
without any accents.

I also noticed during the logo contest, that some
people were messing pages as well, so obviously some
people are still using them (recommand Netscape 7 for
Mac OS 9).

I really fear we will have to check very carefully any
newbie contribution, and perhaps tell them to go away.
I hope we consider this carefully, before switching
everything. I understand there would be some benefits.
But perhaps some drawbacks as well. Any relevant
number (of non utf supporting browser) might be
interesting. Imho.

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


body:
Tomasz wrote:

>1.
>There are many reasons other than interwiki. ISO 8859-1  is
>broken by design - it doesn't even encode all Latin characters,
>and other characters are also needed for correct Latin-script
>typography.

Oh, which ones? All the Latin characters on my keyboard work just fine.
Extended Latin characters also work fine. If you want anything more fancy
then use &code;.

>2.
>Things are NOT fine the way they are. At least not for English
>Wikipedia.

That begs the question. What is wrong?

>3.
>And, as I said, we already break compatibility with very old
>browsers in many ways. Or do you maybe want to ban all PNGs,
>OGGs etc., and implement some converter from CSS to HTML3-
>compatible markup ?

Come on. Not being able to view PNGs, OGGs, fancy CSS tricks and HTML-3 only
stuff does not harm any article. However, somebody using an older browser can
really screw-up an article on at UTF-8 page. This already happens fairly
often on meta, even though meta has very low traffic compared with
en.wikipedia. You cannot compare the two.

UTF-8 is a non-starter for en.wikipedia, and probably for most other
Latin-based wikis as well. But the choice of charset should be up to each
wiki and that choice should reflect the needs of that wiki.

-- Daniel Mayer (aka mav)


subject:
[Wikipedia-l] UTF-8 - idea (maybe stupid,
	from a non-tech dreamer :-)

body:
>From: Brion Vibber <brion at pobox.com>

>Mav, do you have a constructive suggestion for
improving >compatibility
>with older browsers? If so, please post it on
>wikitech-l, where we're
>discussing the issue. We already know you don't like
the >idea of
>breaking older browsers.

He does not like because that is unnice to newbies,
and that generates bad vibes between oldbies (does
that exist ?) as well.

I am glad I do not break pages any more, either
because of utf (Opera), or because of the cut at 30 ko
(Opera), or by the freezing editing window (IE) or by
the random spaces additions in words (Netscape 4). I
still mess with html though :-)

An idea (perhaps totally stupid), would it be possible
that *some* pages are in utf, while most are not.
So, the pages that could benefit utf (such as the list
or arabic terms or such) are in utf, perhaps with a
box to check when saving. and the others are left as
is. Vincent et al could work nicely, most readers
would perhaps see more things, but the pages where
problems would occur would be limited ?

Is that possible ?

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


body:
Peter Gervai wrote:
>Could you point us to the page and revision of the problem?

couple examples:

http://meta.wikipedia.org/w/wiki.phtml?title=What_to_do_with_www.wikipedia.org&diff=4613&oldid=4612
http://meta.wikipedia.org/w/wiki.phtml?title=Main_Page&diff=20132&oldid=20130

This happens on meta's Main Page often. Ask Anthere and Erik for other
examples.

>I'm curious what kind of problem it might have been, as many
>of the Wikipedias are in UTF-8 from the start, and we had no
>problem whatsoever.

Probably because their browsers work nicely in UTF-8 because they have to. If
they didn't they would be useless for any language where UTF-8 is required.
In places where UTF-8 isn't required, browsers that can't support it tend to
slip by without being fixed or upgraded. If it ain't broke...

>However we *do* have problems with english wikipedia when
>pages contain unrepresentable literal characters, which makes
>the page break after editing. See "Budapest" article on wikitravel,
>where every special dash and curly quote marks became question
>marks. Truly ugly.

I don't understand. Is Wikitravel in UTF-8?

-- Daniel Mayer (aka mav)


body:
Brion wrote:
>Mav, do you have a constructive suggestion for
>improving compatibility with older browsers? If so,
>please post it on wikitech-l, where we're discussing
>the issue. We already know you don't like the idea of
>breaking older browsers.

I don't care about breaking old browsers, I care about breaking articles and
not being at least minimally accessable to people who use old browsers. One
thing is certain, however: Switching to UTF-8 will not improve compatibility
with old browsers or improve the experience of the people who use them.

-- Daniel Mayer (aka mav)


body:
Brion wrote:
>Another possibility is simply to 'blacklist' known problem
>browsers by printing a notice/link to better browsers on
>the edit page warning that they may have problems, as
>we now have a warning on long pages that some browsers
>may have problems. (Though in that case we aren't checking
>specific browsers.)

Those types of messages are not at all welcoming. I get pretty pissed when I
go to a website that informs me that I need to upgrade or change my browser.
And I /really/ get annoyed when those websites make suggestions on which
browser I should use.  I already have a browser that I am very comfortable
with thank-you-very-much. And yes I upgrade often. But many people don't
because they either don't know how, have a dial-up account that can't handle
more than a couple meg download or they have an outdated OS or computer
equipment.

We shouldn't be telling those people that they have to upgrade to use
Wikipedia if they can use Wikipedia just fine without UTF-8. The benefits
simply do not seem to be at all compelling enough to justify the negatives
(at least for en.wikipedia). But if you can figure out a way to have UTF-8
without the associated problems, then great.

--- Daniel Mayer (aka mav)


from_name: Lars Aronsson

body:
Erik Moeller wrote:
> perhaps in the 5% range. Given that a single edit by such a person will
> break an entire page, it might not be so wise to switch (but perhaps I'm
> missing something -- is Meta running UTF-8?).

Would it be possible to let the database run on UTF-8 internally, but
to let the PHP script analyze and convert data to and from certain
browsers?  Perhaps the majority of users are using UTF-8-capable
browsers, so the conversion would use a minimum of resources.

All I know is that MySQL has better UTF-8 support from version 4.1.x,
as described in chapter 9, http://www.mysql.com/doc/en/Charset.html
The same goes for Perl version 5.8, but what about PHP?

--
  Lars Aronsson (lars at aronsson.se)
  Aronsson Datateknik - http://aronsson.se/


body:
Vincent Ramos wrote:
>Yes, but non utf-8 compliant browsers are not very
>common, nowadays; there should be less and less.

And people who use Linux as a desktop OS are pretty rare as well. Should we
therefore do things that ignore their basic editing needs? In other words, if
people using these older browsers edit fine as is, then why in the world we
suddenly do something that made their every edit destructive and them labled
a vandal?

I must admit I accused one user of "destroying a page" after she edited a meta
page with an old browser. I'm deeply sorry for that and therefore don't think
we should have UTF-8 were there is not an overriding and very compelling need
for it (alas meta is such a place, but en.wikipedia isn't - perhaps other
Latin wikis as well).

-- Daniel Mayer (aka mav)


body:
On Tue, Nov 18, 2003 at 01:24:00AM +0100, Erik Moeller wrote:
> Tomasz-
> > * broken browsers - they should be upgraded, if someone has browser so old
> > 			that it doesn't grok UTF-8, it's not going to grok CSS,
> > 			PNGs, and other things we're using either.
> > 			Unless we want to remove all CSS and PNGs, there's
> > 			no point in not using UTF-8.
>
> Is this true? All I know is that we had a *lot* of problems with broken
> special chars on the Meta-Wiki during the logo contest. I have no idea
> which browser broke them, but it seems to be a not totally uncommon one,
> perhaps in the 5% range. Given that a single edit by such a person will
> break an entire page, it might not be so wise to switch (but perhaps I'm
> missing something -- is Meta running UTF-8?).

It's nothing like 5% - it's at least one order of magnitude smaller number.
On Polish Wikipedia there weren't any serious problems with UTF-8-incompatible browsers.


body:
On Mon, Nov 17, 2003 at 10:29:25PM -0500, Daniel Mayer wrote:
> Erik wrote:
> >Is this true? All I know is that we had a *lot* of problems
> >with broken special chars on the Meta-Wiki during the logo
> >contest. I have no idea which browser broke them, but it
> >seems to be a not totally uncommon one, perhaps in the
> >5% range. Given that a single edit by such a person will
> >break an entire page, it might not be so wise to switch
> >(but perhaps I'm missing something -- is Meta running UTF-8?).
>
> IIRC meta is. And that fact has created some of the problems you mention. I
> therefore see no compelling need to convert Latin-1 languages to UTF-8 and in
> fact think such a switch would be harmful. It is also wrong-headed to state
> (as Tomasz did) that if people have non-UTF-8-friendly browsers that they
> should upgrade. That is not the attitude we should have when things work just
> fine the way they are (at least on the English Wikipedia - others may have
> more compelling reasons to use UTF-8 that outweigh the negatives).
>
> The only place where UTR-8 would be very useful is with interlanguage links.
> But that could better be solved by placing all interlanguage links outside of
> the regular wiki text of pages. That separate edit window could support UTF-8
> and be shared by all Wikipedia's. This should minimize damage done by
> non-UTF-8-compliant browsers and as an added benefit could be part of an
> easier way to add language links to articles (inputing the links once would
> create language links in every article listed in the common meta space).

1.
There are many reasons other than interwiki. ISO 8859-1  is broken by design -
it doesn't even encode all Latin characters, and other characters are also needed
for correct Latin-script typography.

2.
Things are NOT fine the way they are. At least not for English Wikipedia.

3.
And, as I said, we already break compatibility with very old browsers in many ways.
Or do you maybe want to ban all PNGs, OGGs etc., and implement some converter
from CSS to HTML3-compatible markup ?


from_name: Vincent Ramos

body:
Le Mon, 17 Nov 2003 21:35:42 -0800 (PST), Anthere
<anthere6 at yahoo.com> a ?crit:

> I understand your need as a linguist for these
> letters. I just hope I was the last editor to ever
> come to wikipedia with a browser not supporting utf8;
> because any other one will just be excluded from the
> project.

	Yes, but non utf-8 compliant browsers are not very
common, nowadays; there should be less and less.

> I fear a bit that some newbie will come one day, and
> that you will have to fix everything behind him, and
> then explain to him he is just not welcome. Hopefully,
> it will never happen :-)

	It is already the case with pages featuring raw
<oe> digraphs or quotation marks (which are not latin-1
characters but, I do not know how, there are editors who
can save pages with them): when someone with another
system edits them, every <oe> and quotation mark
is replaced by <?>. Some browsers do not respect raw
non breakable spaces and turn them into normal spaces, etc.
There is always someone to correct these pages.

	At least, with utf-8 these problems should be less
frequent, because <oe> and <'> could be directly encoded in a
charset which possesses them.

	Vincent


from_name: Poor, Edmund W

subject: [WikiEN-l] User:Caius2ga

body:
Rick,

1. Please ask Caius not to refer to other contributors as "Nazis".

2. As for the "current name" issue, that comes under the category of
usage.

Let's deal with the first issue, first. After the conversational tone
gets calmed down, we can start to deal with the more complex issue.

Ed Poor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/wikien-l/attachments/20031118/56e86d45/attachment.htm


from_name: Nikola Smolenski

body:
On Tuesday 18 November 2003 00:57, Tomasz Wegrzanowski wrote:
> On Mon, Nov 17, 2003 at 03:28:46PM -0800, Brion Vibber wrote:
> > We'll have to go down to read-only mode while copying stuff over
> > anyway, so this consolidates downtime. The conversion itself can be
> > done by simply piping the database dump through iconv as it's being
> > copied into the new db.
>
> It would be better if numeric entities were converted too.
> The code is somewhere in Phase1->Phase2 conversion script,
> and in konwert program too.

Note also that cases like &#1071; must NOT be converted, as it is their
intention to appear like entities in displayed text. Perhaps it's the easiest
not to convert & at all.

It would also be nice if one who does the conversion tell us what was the size
of DB dump before and after the conversion :) I'm quite sure there'll be a
decrease.

Would it also make sense to convert [[sr:%D0...]] interlanguage links for
further decrease in size? I don't know is there a tool which could do it, but
if not I could make a C program for it in five minutes.


from_name: Mirko Thiessen

subject: [WikiEN-l] Re: User:Caius2ga

body:
RickK wrote:

> [[User:Caius2ga]] is repeatedly involved in edit wars
> over the useage of the names of Polish cities, insisting
> on calling every city that ever was or ever might have
> been in Poland, by its current name, regardless of what
> it might have ever been called.  He has now begun to
> refer to those who disagree with him as Nazis.

Most disgusting is the behaviour of almost all parties involved in these
edit wars. While I do not have a preference concerning the city names (I
don't care what they are called, there are indeed more important things),
the name calling and reversion wars are unacceptable.

The brave user Kosebamse mediated a compromise for the Silesia article.
The main combattants Nico and Szopen agreed to accept the final wording,
apologised to each other for previous behaviour and removed themselves
 from the problem user page. But there are users who refuse to discuss
while a page is protected. See the discussion on [[User talk:Wik]], where
Kosebamse tried to convince him of discussing his points of view. As soon
as a page is unprotected, this sort of users is starting to revert. The
Silesia page was unprotected for twelve hours (which is almost a record),
and after it was protected again, user Caius2ga started to call his
opponents Nazis.

PLEASE NOTE that this is not a complaint about the content, which is
inserted into the articles by certain users. It is a complaint about the
behaviour of the users involved. Their behaviour is a shame and as far
 from Wikiquette as it could be.

I am just an observer and not involved anymore, because I learnt that
mediation is useless with this sort of users. I guess, Kosebamse won't
give it one more try as well. This is so frustrating. I know of several
users who keep out of all Poland/Germany articles (me included), and I
know one user, who left Wikipedia because of this. My advice: Kick them
out. Wik and Caius2ga are not able to work in a cooperative project and
are poisoning the atmosphere.

Best regards,

Mirko.


from_name: Fred Bauder

subject: [WikiEN-l] Reversion warriors

body:
> From: "Poor, Edmund W" <Edmund.W.Poor at abc.com>
> Reply-To: English Wikipedia <wikien-l at Wikipedia.org>
> Date: Tue, 18 Nov 2003 10:35:20 -0500
> To: "English Wikipedia" <wikien-l at Wikipedia.org>
> Subject: RE: [WikiEN-l] Reversion warriors
>
> Mirko,
>
> I am frustrated, too. I don't see any easy solution to the "constant
> state of edit war". Worse, I suspect that the problem of Wikipedia is a
> microcosm of the problem of the entire world.
>
> For there to be stability, there must either be a force which prevents
> contending elements from clashing -- or those elements must come to a
> mutually satisfying agreement.
>
> As Viajero said or hinted (or probably both!), I have often been a
> "force" which settled disputes via some sort of authority. But this does
> not satisfy anyone: not me, and not Viajero, and not the contributors
> who wait a day, a week, or a month and just start tearing apart the
> battlefield (oops, I mean article!) again.
>
> I find to my shock that my "solutions" are often do not provide enduring
> satisfaction. I'm more like a Soviet premier using military and
> political power to suppress ethnic strife than a genuine peacemaker.
>
> This troubles me. This disturbs me. This is the number one factor that
> drives me to want to leave Wikipedia forever.
>
> Ed Poor

If people (with a number of contradictory viewpoints) will torture and kill
to get a viewpoint across, why is a surprise that they would revert a
Wikipedia article?

Fred

Fred


from_name: Peter Gervai

body:
On Mon, Nov 17, 2003 at 10:29:25PM -0500, Daniel Mayer wrote:
> Erik wrote:
> >Is this true? All I know is that we had a *lot* of problems
> >with broken special chars on the Meta-Wiki during the logo
[...]
>
> IIRC meta is. And that fact has created some of the problems you mention. I

Could you point us to the page and revision of the problem? I'm curious what
kind of problem it might have been, as many of the Wikipedias are in UTF-8
from the start, and we had no problem whatsoever.

However we *do* have problems with english wikipedia when pages contain
unrepresentable literal characters, which makes the page break after
editing. See "Budapest" article on wikitravel, where every special dash and
curly quote marks became question marks. Truly ugly.

--~~~~


subject: [Wikipedia-l] Transwiki

body:
The following is a point that I raised with Angela as part of my attempt
to deal with what I see as the ongoing problem of Votes for Deletion.  I
would like to seek constructive solutions to that.

When it comes to deletions my first rule would always be to resolve the
benefit of the doubt in favour of keeping.  Secondly I would put delete
only when an article falls squarely into a specified category.  The
first two (and there would be others) clear categories would be
    (a) Obvious bad-faith nonsens such as infantile comments, alphabet
soup and insults (the kind we all know when we see it), and
    (b) Clear tyupographical errors in an article title

The general class of articles that I want to consider now is that of
those articles that belong in another member project of the Wikimedia
family. Such an article will likely have valuable information, but the
contributor has just put it in the wrong place. That group has a number
of number of possible sub-classes:

   1. dictionary type entries; (They belong in Wiktionary)
   2. foreign language entries: (They belong in the Wiktionary for the
      relevant language)
   3. entire quoted books; (They belong in Wikibooks)
   4. memorials; (They belong in what is now the 9/11 Memorial, which
      could be expanded to cover forgetable victims of other disasters)

One observation that I have about the current listing of things to be
transferred to Wiktionary is that it is somewhat useless where it is. A
person that devotes his entire Wiki time to work on Wiktionary may not
see it and may not even know that it's there. To solve this I would
propose a Transwiki:
<http://en2.wikipedia.org/w/wiki.phtml?title=Transwiki:&action=edit>
namespace on all projects. I first thought of using the word "Transfer"
for this, but decided that a coined word would avoid any possible
ambiguities. Currently Thou <http://en2.wikipedia.org/wiki/Thou> is the
first entry on Wikipedia:Things to be moved to Wiktionary
<http://en2.wikipedia.org/wiki/Wikipedia:Things_to_be_moved_to_Wiktionary>
. To deal with this I would create an article in Wiktionary called
[[Transwiki:Thou]] to which everything that is now in Thou
<http://en2.wikipedia.org/wiki/Thou> could be copied as is. A
Wiktionarian who is looking for something to do could then work at
co-ordinating that article with what really should be on Wiktionary
according to Wiktionary rules. Once that is done the Transwiki article
could be deleted.

Since this idea can be applied to transfers between any two projects, I
would suggest that the name "Transwiki" be applied uniformly across all
projects.  This is important for foreign language entries   That way a
worker in the source project will know exactly where to put something in
the target project without knowing anything about the target language.

Ec


from_name: Andre Engels

body:
On Mon, 17 Nov 2003, Vincent Ramos wrote:

> 	Do you know what percentage do this trick?
>
> 	The point is: why do other Wikimedia projects use utf-8
> and not fr:? Are there so many utf-8 non compliant browsers?

I think it's historical. When switching over to the phase 3, ISO 8859-1
was used for the western languages, and UTF-8 only for those languages that
really could not dow ithout, like Japanese. Nowadays, all newly converted
languages are immediately put on UTF-8, while Polish (formerly 8859-4) and
Russian (formerly 8859-5) have also been switched to UTF-8, but the 'older'
ISO 8859-1 languages have all been kept on ISO 8859-1.

Andre Engels


subject: [WikiEN-l] Reversion warriors

body:
Mirko,

I am frustrated, too. I don't see any easy solution to the "constant
state of edit war". Worse, I suspect that the problem of Wikipedia is a
microcosm of the problem of the entire world.

For there to be stability, there must either be a force which prevents
contending elements from clashing -- or those elements must come to a
mutually satisfying agreement.

As Viajero said or hinted (or probably both!), I have often been a
"force" which settled disputes via some sort of authority. But this does
not satisfy anyone: not me, and not Viajero, and not the contributors
who wait a day, a week, or a month and just start tearing apart the
battlefield (oops, I mean article!) again.

I find to my shock that my "solutions" are often do not provide enduring
satisfaction. I'm more like a Soviet premier using military and
political power to suppress ethnic strife than a genuine peacemaker.

This troubles me. This disturbs me. This is the number one factor that
drives me to want to leave Wikipedia forever.

Ed Poor


from_name: Karl Eichwalder

subject: [Wikipedia-l] Re: Switching everything to UTF-8

body:
Peter Gervai <grin at tolna.net> writes:

> However we *do* have problems with english wikipedia when pages contain
> unrepresentable literal characters, which makes the page break after
> editing.

Yes, the same is valid for the German WP.  Pages are served as
iso-8859-1 but people don't hesitate to add iso-8859-15 (the EUR
symbol) or even windows-1252 ("smart quotes").  The next one, with a
conforming browser, edits such an article and the next reader will see
some quotations marks or other artefacts - this happens, not that
often, though.

> See "Budapest" article on wikitravel, where every special dash and
> curly quote marks became question marks. Truly ugly.

Yes, that's what I meant ;)

--
                                                         |      ,__o
                                                         |    _-\_<,
http://www.gnu.franken.de/ke/                            |   (*)/'(*)


body:
On Tuesday 18 November 2003 11:03, Daniel Mayer wrote:
> Brion wrote:
> >Another possibility is simply to 'blacklist' known problem
> >browsers by printing a notice/link to better browsers on
> >the edit page warning that they may have problems, as
> >we now have a warning on long pages that some browsers
> >may have problems. (Though in that case we aren't checking
> >specific browsers.)
>
> Those types of messages are not at all welcoming. I get pretty pissed when
> I go to a website that informs me that I need to upgrade or change my
> browser. And I /really/ get annoyed when those websites make suggestions on
> which browser I should use.  I already have a browser that I am very
> comfortable with thank-you-very-much. And yes I upgrade often. But many
> people don't because they either don't know how, have a dial-up account
> that can't handle more than a couple meg download or they have an outdated
> OS or computer equipment.
>
> We shouldn't be telling those people that they have to upgrade to use
> Wikipedia if they can use Wikipedia just fine without UTF-8. The benefits
> simply do not seem to be at all compelling enough to justify the negatives
> (at least for en.wikipedia). But if you can figure out a way to have UTF-8
> without the associated problems, then great.

The benefits of having non-ISO-8859-1 texts editable for everyone do not
outweight the negatives of not having them editable for 2% of users?

Further, there are browsers that have problems when UTF-8 is NOT used.
Konqueror, which I am using, for one: on ISO-8859-1 pages, all non-ISO-8859-1
characters typed in turn into question marks, so they have to be typed in as
entities, and most interwiki links also don't work; and Brion's survey show
that there are 3.5 times as much konqueror users as of all non-UTF-8 browsers
combined.


body:
>From Wikitech-l:

Erik wrote:
>If UTF-8 has enough advantages (and many people seem
>to think it does), then telling 2% of the userbase
>that their browser is outdated and corrupts pages
>when editing seems acceptable.

The pages will only get corrupted by their browsers
/if/ we switch to UTF-8! So why lock out 2% of our
user base just to make it a bit easier to have
non-Latin scripts on a Latin-based language wiki?
[NOTE: I agree that UTF-8 makes sense on meta though;
there the advantages /do/ outweigh the benefits.] But
if those 2% can be accommodated as is and we can have
UTF-8, then great. Otherwise my idea of having a
separate UTF-8-friendly edit window for language codes
would solve the language link problem (which, IMO, is
the only issue that has real merit here).

Sorry, but as a long-time Linux desktop user I get a
bit pissed whenever somebody brings up the "only x%"
arguments. It is often used as an excuse to prevent me
from using certain websites. When I see "You need to
upgrade your browser" I leave and never come back. We
needn't repeat the "only x%" type of attitude here.

-- mav

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


from_name: Erik Moeller

body:
Brion-
> Another possibility is simply to 'blacklist' known problem browsers by
> printing a notice/link to better browsers on the edit page warning that
> they may have problems,

For the record, I would support blocking problematic browsers from editing
entirely. If UTF-8 has enough advantages (and many people seem to think it
does), then telling 2% of the userbase that their browser is outdated and
corrupts pages when editing seems acceptable. Perhaps we can do it in a
smart way and provide links to alternatives for each blacklisted browser
(e.g. Netscape 4->Mozilla).

On the other hand, I would not support any solution that still leads to
corrupted pages.

Regards,

Erik


subject: [Wikipedia-l] Students!

body:
> From: Nicolas Weeger <weeger at noos.fr>
> Reply-To: wikipedia-l at Wikimedia.org
> Date: Tue, 18 Nov 2003 19:11:47 +0100
> To: wikipedia-l at Wikimedia.org
> Subject: Re: [Wikipedia-l] Students!
>
>
>> something like:
>> * we welcome it! :)
>> * make sure your students understand NPOV
>> * and so on
>
> (en:)
> I have one main concern about that, even if the idea is great :)
> Since grades will be given partly for the language, the contents & such,
> what happens if someone outside the students fixes typos, links, adds
> things? This, arguably, is the teacher's trouble to sort out. But what
> if the teacher asks 'ok, could non-students please do not change this
> page during the ''examen'' ?' ? What should we reply? It obviously is
> against Wikipedia's very nature to 'lock' (morally, at least :)) a page.

User contributions combined with page history for each student will return
the work they did. No need for anyone else to change what they ordinarily
do.

Fred

>
> (fr:)
> J'ai une seule crainte, ? ce propos, m?me si l'id?e est super :)
> Les notes ?tant attribu?es (sans doute) en fonction du langage, du
> contenu, etc, que se passera-t-il si quelqu'un qui n'est pas un de ces
> ?tudiants change la page, en corrigeant des fautes, des liens, etc ?
> C'est ?videmment le probl?me du professeur. Mais que r?pondre s'il nous
> demande 'bon, que les non-?tudiants ne touchent pas ? ces pages le temps
> de l'examen' ? C'est totalement contre la nature de Wikip?dia de
> 'bloquer' une page (m?me moralement :))
>
> Nicolas 'Ryo'
>
> _______________________________________________
> Wikipedia-l mailing list
> Wikipedia-l at Wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikipedia-l


body:
Peter wrote:
>Not at all! It clearly shows what happens when a page

>_IS_ 8859-1 encoded but editors want to use fancy
>characters. Same hppens when they do it with an old
>browser on UTF-8 pages. So, you get trash either way,

>other editors revert the same way, so you may well
>use utf-8, don't you? :-)

OIC. Well a person should /not/ be using fancy things
like curly quotes and long hyphens because many
browsers (especially on non-MS systems) display them
as question makes. These should be fixed, not allowed
to propagate. The fact that some browsers break these
codes should be a good hint that they should not be
used to begin with (esp. since ASCII quotes and
regular hyphens can be used instead).

But a way around the larger issue is to sniff whether
or not a browser is UTF-8-aware and then serve a page
in either UTF-8 or in Latin-1 (whatever the ISO) based
on that. When a UTF-8 page is displayed it shows the
actual non-Latin characters, when the Latin-1 page is
displayed it shows the codes the represent those
characters.

That at least will prevent pages from getting damaged,
but the special characters will still show up as
question marks for people with older browsers, so
things like curly quotes and long hyphens should be
automatically converted to their ASCII counterparts.

-- Daniel Mayer (aka mav)

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


body:
On Tue, 18 Nov 2003, Brion Vibber wrote:

> Here's a breakdown of browsers used to save edits over a day on
> de.wikipedia.org. I checked de because it's the largest non-English
> non-UTF-8-using Wikipedia.
>
> Total posts in the day: 2861
>
> Browsers known to work with editing UTF-8:
>    MSIE/Win: 1146
>    Gecko: 966 [549 Windows; 379 Linux/Unix; 31 Mac]
>    Opera 6+: 331 [291 Windows; 40 Linux]
>    Konqueror: 216
>    Safari: 137
> total: 2796 (98%)
>
> Browsers which can be problematic with UTF-8:
>    old Netscape: 25
>    ELinks: 20 (not tested, but it's a text-mode browser and these tend
> to have problems in this area)
>    MSIE/Mac: 10
>    Lynx: 4 (text-mode again)
>    Opera <6: 2
> total: 61 (2%)
>
> There's some margin of error; my regexps lost track of 4 hits or so
> somewhere in that mess.

I did the actual check for Lynx. Lynx does a commendable job at reading
UTF-8 (mapping each character to the closest transcription into Latin-1),
but editing with it will cause disasters (it will send the output in the
same way, thus with all non-Latin-1 characters changed to their closest
Latin-1 equivalent).

Although I do like to have UTF-8 (recently had to add dots that do not
belong there on 'Hisarlik' in my nl:-article on Schliemann), I also
think that 2% of our userbase is large enough a group to not just get
rid of or allow to inadvertently destruct pages.

Andre Engels


from_name: The Cunctator

subject: [WikiEN-l] Reversion warriors

body:
> Ed Poor wrote:
> I find to my shock that my "solutions" are often do not provide
enduring
> satisfaction. I'm more like a Soviet premier using military and
> political power to suppress ethnic strife than a genuine peacemaker.

Or a caring but underaged and immature parent trying to raise a child.
(Or a well-meaning, slightly erratic uncle who's clueless about
childcare...)

> This troubles me. This disturbs me. This is the number one factor that
> drives me to want to leave Wikipedia forever.

Do you think you would only enjoy participating in Wikipedia if you put
yourself in a role as sysop/mediator?

You might want to try not using your sysop powers (protecting pages,
renaming pages) to affect disputes for a while.

This doesn't mean you'd have to stop doing work to try to strengthen the
overall Wikipedia more than simply contributing and editing articles.

For example, you could instead actively recommend more people into the
sysop ranks, especially people who are boring editors who wouldn't
necessarily pipe up--those who simply contribute steadily and avoid edit
wars.

You could work to minimize the Votes for Deletion entries by improving
stubs (dull work, but critically important).

Etc.


from_name: Christopher Mahan

body:
--- Daniel Mayer <maveric149 at yahoo.com> wrote:
> OIC. Well a person should /not/ be using fancy things
> like curly quotes and long hyphens because many
> browsers (especially on non-MS systems) display them
> as question makes. These should be fixed, not allowed
> to propagate. The fact that some browsers break these
> codes should be a good hint that they should not be
> used to begin with (esp. since ASCII quotes and
> regular hyphens can be used instead).

Isn't there a "approved characters page" somwhere?

> But a way around the larger issue is to sniff whether
> or not a browser is UTF-8-aware and then serve a page
> in either UTF-8 or in Latin-1 (whatever the ISO) based
> on that. When a UTF-8 page is displayed it shows the
> actual non-Latin characters, when the Latin-1 page is
> displayed it shows the codes the represent those
> characters.

Please, no browser sniffing.

> That at least will prevent pages from getting damaged,
> but the special characters will still show up as
> question marks for people with older browsers, so
> things like curly quotes and long hyphens should be
> automatically converted to their ASCII counterparts.

I would rather see code that changes the latin chars to utf-8 chars
at article save.

=====
Christopher Mahan
chris_mahan at yahoo.com
818.943.1850 cell
http://www.christophermahan.com/

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree


from_name: Tim Starling

subject: [Wikitech-l] Re: number/values for namespaces?

body:
ray at ganymede.org wrote:
>
> I see the numeric values in the database for the namespaces. I do not
> see documentation anywhere which describes, in human terms, to what each
> number corresponds.
>
> Should I just be looking in the source for this stuff, or is there doc I
> have missed?
>
> thanx - ray

For human terms, see:

http://en2.wikipedia.org/wiki/Wikipedia%3ADatabase_queries

The master list is in Language.php, near the top:

http://cvs.sourceforge.net/viewcvs.py/wikipedia/phase3/languages/Language.php?rev=1.64&view=auto

-- Tim Starling


from_name: tarquin

subject: [Wikipedia-l] Students!

body:
I noticed on the fr: Bistro
(http://fr.wikipedia.org/wiki/Wikip%E9dia:Le_Bistro) that a university
lecturer is planning on getting students to write articles as part of
their course.
Some of you may remember this happened with the en: wikipedia a few
months ago.

I suggest that we create a page on Meta of guidelines for this sort of
thing, since I imagine we'll be seeing more of it.

something like:
* we welcome it! :)
* make sure your students understand NPOV
* and so on


from_name: Russell Jones

body:
This does not affect 2% of users, but 2% of sessions.  Some people use
more than one browser and will switch if need be. I suggest the switch
to UTF-8 is done quickly, and that the support for the minority browsers
is added later by someone who feels it is worth the effort.  This
somewhat peripheral issue should not impede development of the Wikipedia.

Russell

Daniel Mayer wrote:

>Brion wrote:
>
>
>>Another possibility is simply to 'blacklist' known problem
>>browsers by printing a notice/link to better browsers on
>>the edit page warning that they may have problems, as
>>we now have a warning on long pages that some browsers
>>may have problems. (Though in that case we aren't checking
>>specific browsers.)
>>
>>
>
>Those types of messages are not at all welcoming. I get pretty pissed when I
>go to a website that informs me that I need to upgrade or change my browser.
>And I /really/ get annoyed when those websites make suggestions on which
>browser I should use.  I already have a browser that I am very comfortable
>with thank-you-very-much. And yes I upgrade often. But many people don't
>because they either don't know how, have a dial-up account that can't handle
>more than a couple meg download or they have an outdated OS or computer
>equipment.
>
>We shouldn't be telling those people that they have to upgrade to use
>Wikipedia if they can use Wikipedia just fine without UTF-8. The benefits
>simply do not seem to be at all compelling enough to justify the negatives
>(at least for en.wikipedia). But if you can figure out a way to have UTF-8
>without the associated problems, then great.
>
>--- Daniel Mayer (aka mav)
>
>_______________________________________________
>Wikitech-l mailing list
>Wikitech-l at Wikipedia.org
>http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20031118/8f538611/attachment.html>


from_name: Delirium

body:
Peter Gervai wrote:

>On Tue, Nov 18, 2003 at 04:28:32AM -0500, Daniel Mayer wrote:
>
>
>>Peter Gervai wrote:
>>
>>
>>>Could you point us to the page and revision of the problem?
>>>
>>>
>>couple examples:
>>
>>http://meta.wikipedia.org/w/wiki.phtml?title=What_to_do_with_www.wikipedia.org&diff=4613&oldid=4612
>>http://meta.wikipedia.org/w/wiki.phtml?title=Main_Page&diff=20132&oldid=20130
>>
>>This happens on meta's Main Page often. Ask Anthere and Erik for other
>>examples.
>>
>>
>
>I see. First is not a good example, Opera 5 is _ancient_, you can't expect
>that anyone would support it, as upgrading is clearly painless.
>
>Second example is indeed valid, but it isn't a problem for you: if the page
>does not contain non-8859-1 characters, nothing gets garbled. If it does
>contain others then, well, you *need* utf-8 on that page anyway. (Embed
>codes are a little bit slow to type, don't you agree? If not, write your
>reply manually by using embeds. :))
>
>
I don't really see the problem with typing embed codes manually on, for
example, the English Wikipedia.  I presume the vast majority of the text
will be in English (that is the language, after all), with only a very
few words and phrases not in English (Greek or Chinese or etc. versions
of foreign place names).  It shouldn't be *too* much of a hassle to type
those few phrases with embed codes.

Actually, I think with the current setup, on en: at least, you can type
them literally and when you hit submit or preview it'll automatically
convert them to the numeric codes.  I seem to recall this happening with
some Greek text I pasted in (though I could be mistaken).

-Mark


from_name: Brion Vibber

body:
On Nov 18, 2003, at 14:49, Delirium wrote:
> Actually, I think with the current setup, on en: at least, you can
> type them literally and when you hit submit or preview it'll
> automatically convert them to the numeric codes.  I seem to recall
> this happening with some Greek text I pasted in (though I could be
> mistaken).

That's a behavior of some, but not all, browsers. A property of those
browsers that do this is they also properly submit UTF-8 data when the
pages are in UTF-8.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20031118/05d4e2e5/attachment.pgp


from_name: Anthere

body:
On Tue, Nov 18, 2003 at 04:28:32AM -0500, Daniel Mayer
wrote:
> Peter Gervai wrote:
> >Could you point us to the page and revision of the
problem?
> > couple examples:
> >
http://meta.wikipedia.org/w/wiki.phtml?title=What_to_do_with_www.wikipedia.org&diff=4613&oldid=4612
>
http://meta.wikipedia.org/w/wiki.phtml?title=Main_Page&diff=20132&oldid=20130
> > This happens on meta's Main Page often. Ask
Anthere and Erik for
other
> examples.

>I see. First is not a good example, Opera 5 is
>_ancient_, you can't expect that anyone would support
>it, as upgrading is clearly painless.

This is NOT true. I tried upgrading to Opera 6, and it
was unworkable, because it needed to much memory for
my computer to handle it. Sure enough, I did not break
anything any more, but the browser had great
slowliness, and crashed on average every 30 mn. This
is not precisely what I call *painless*

I kept it for a while, just to edit meta page, then
gave up because it was just too much hassle.

The upgrade is perfectly ok if you have a recent
computer. But you cannot expect every user to have so.
There was a big campaign about 4 years ago in France,
and many many people bought some imacs. I doubt very
much the casual user just bought a brand new computer
since then. Except for wikipedia, I did not meet any
problem with it, fine for power, fine for dvd, fine
for internet. Just problem on wikipedia. And I bought
a boosted one.

I know it is not possible for these users to use Opera
6, and not possible to switch to system X.

>Second example is indeed valid, but it isn't a
problem >for you: if the page does not contain
non-8859-1 >characters, nothing gets garbled. If it
>does contain others then, well, you *need* utf-8 on
>that page anyway. (Embed codes are a little bit slow
>to type, don't you agree? If not, write  your
>reply manually by using embeds. :))

I do not understand that comment.

French people suggested they could just by hand
correct broken caracters.
This is not an option I fear

I just made an example : this is what appear after my
edit : http://meta.wikipedia.org/wiki/Lorraine

Can you figure manually correcting each time after a
user ?

I have the plain answer : I saw a couple of reaction
on meta to my destructive edits; I was just reverted;
I know very well that if we switch to utf-8 on all
wikipedias without a technical tweak to automatically
insure "translation", the user of this browser,
perhaps just a mother at home with a 4 years old imac,
perhaps a student in Algeria, perhaps a kid in Brasil,
will just be kicked out.

Perhaps is it just 2%, and perhaps those editing
wikipedia right now are people technically better
equipped that the average human being connected to
internet, and perhaps we just do not want to keep it
that way, and perhaps we want liberty and openess.
Depends on what is important.

>I understand your problem, it is valid, and that's
>probably the reason it's topic on wikitech. Still I
>believe we can expect editors to use non-ancient
>browsers (remember, reading is not a problem).

But perhaps we do not want to exclude these people
from editing ?

>As far as I know most
>browsers handle this very well (including, for
>example, unix character
>mode browsers).

And perhaps most users know nothing about unix

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree