subject: [Wikipedia-l] Re: [Intlwiki-l] Review mechanism ? Stable pages ?

body:
On mar, 2002-05-21 at 10:40, Krzysztof P. Jasiutowicz wrote:
> Have you looked up recently the Wikipedia page
> http://www.wikipedia.com/wiki/Karl_Landsteiner as I did yesterday ?

Well, I did just now, and I replaced the garbage with a brief stub after
a 10-second google search turned up several informationful web pages
about the guy. You could have done that yourself!

> With growing number of pages we either need some review and approval
> mechanism or a stable namespace or even a stable version of Wikipedia
> (that would be so open for editing for everyone).
>
> What do you think ?

The system we have is based on two ideas:

a) Changes to articles will be seen and reviewed by other wikipedians
  and
b) Wikipedians will improve articles they find to be lacking

A is dependent on the number of wikipedians interested in any particular
topic (or with a mind to check out new articles), and B is dependent on
YOU.

Now, in the case of [[Karl Landsteiner]], the article only has some 47
hits listed at the moment. Most of those are probably spiders; a few
more for the idiot who put together the page, and a couple for you and
me. Maybe nobody else ever bothered to look at the article before, and
it thus survived for a month before being corrected.

If we required that pages be reviewed before they appear to mortal
users, would anyone still have ever looked at it? Would it be sitting in
a queue still, invisible? Maybe that would be good for this particular
article, but would we want that for all the little-seen articles that
*aren't* garbage?

I dunno.

-- brion vibber (brion @ pobox.com)


list: wikipedia-l

source: pipermail-mbox

from_name: Brion L. VIBBER

body:
On mar, 2002-05-21 at 11:19, Tomasz Wegrzanowski wrote:
> One thing - I hereby officially propose ;) that option "only minor
> changes" should be available only for registered users.

That's been the case for a couple of months on the new software. (And
furthermore, show ALL edits is the default setting. You have to go out
of your way to hide minor edits, and I don't know why anyone would want
to anyway.)

If the other wikis get converted sometime, problem solved!

By the way, MOST OF THE INTERNATIONAL WIKIS ARE LOSING THEIR EDITING
HISTORIES BECAUSE THE OLD WIKI SOFTWARE DELETES OLD PAGE VERSIONS AFTER
A COUPLE OF WEEKS!!!!!!!!! I *hate* destructive default settings.

> All cases of vandalism on Polish wikipedia that weren't immediately
> fixed were in case where nobody have seen it because it wasn't being
> shown on 'Ostatnie Zmiany' (recent changes) page.
>
> Of course a vandal could bother to register, but most of them don't.
> So maybe this feature should be turned off completely.
> It's very dangerous.

Agreed, that's why we did it. :)

-- brion vibber (brion @ pobox.com)


list: wikitech-l

from_name: Jason Richey

subject: [Wikitech-l] links in the sidebar

body:
I couldn't help but notice that the link in the sidebar that refers to
today's date doesn't have the spaces escaped.  So, instead of clicking
through to "http://www.wikipedia.com/wiki/May+21", you go to
"http://www.wikipedia.com/wiki/May 21,2002", which puts you at
"http://www.wikipedia.com/wiki/May".  Perhaps this was intentional...

--
"Jason C. Richey" <jasonr at bomis.com>


from_name: lcrocker at nupedia.com

subject: [Wikipedia-l] images and search rankings

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020521/bb4ca956/attachment.diff


subject: [Wikitech-l] links in the sidebar

body:
On mar, 2002-05-21 at 12:15, Jason Richey wrote:
> I couldn't help but notice that the link in the sidebar that refers to
> today's date doesn't have the spaces escaped.  So, instead of clicking
> through to "http://www.wikipedia.com/wiki/May+21", you go to
> "http://www.wikipedia.com/wiki/May 21,2002", which puts you at
> "http://www.wikipedia.com/wiki/May".  Perhaps this was intentional...

Oh, that whole thing is broken, broken, broken.

* Doesn't fit article naming conventions and no one wants such pages
* Not URL-escaped (the space can break on some browsers)
* No path! (Thus doesn't work from http://www.wikipedia.com/ or
  http://www.wikipedia.com/wiki.phtml?blahblahblah)

I've taken the liberty of removing it from wikiSkinStandard.php.

-- brion vibber (brion @ pobox.com)


subject: [Wikipedia-l] New codebase, testbed

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020521/0dca46c7/attachment.diff


subject: [Wikitech-l] New codebase, testbed

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020521/0dca46c7/attachment.ksh>


from_name: Chuck Smith

body:
> We could also use the name-piping feature to add alt
> text (short
> descriptions of the image for text-mode and speech
> browsers):
>
>   [[image:de-flag.jpg|German flag]]

Oooh!  This would definitely help our search engine
rankings too!  Go Brion!

Chuck

=====
Come to my homepage!  Venu al mia hejmpagxo!
http://amuzulo.babil.komputilo.org/
====
Venu al la senpaga, libera enciklopedio
esperanta reta! http://eo.wikipedia.com/

_________________________________________________________
Do You Yahoo!?
Informaci?n de Estados Unidos y Am?rica Latina, en Yahoo! Noticias.
Vis?tanos en http://noticias.espanol.yahoo.com


from_name: koyaanisqatsi at nupedia.com

subject: [Wikipedia-l] New codebase, testbed

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020521/2488837b/attachment.diff


from_name: Jimmy Wales

subject: [Wikitech-l] Please play with this

body:
http://www.3apes.com/

This is a strictly COMMERCIAL idea of mine.  I am pretty sure that the
data would be released under a free license, maybe the GNU FDL, but
more likely something a lot simpler, sort of a Berkeley-style license,
without all the complicated talk about "cover pages" and so forth.

If you don't feel like contributing to such a commercial idea, I
totally understand.

I'd just like to have people play with it and see if it is as fun for you
as it has been for me.

Basically, it is a wiki-style search engine.  There's a javascript
"magic button" you can download.  This enables you to quickly add neat
sites to the database, and they show up on Recent Changes.

The software is totally homegrown perl of my own making.  It's pretty
awful.  I'm planning a total rewrite sometime soon.

--Jimbo


from_name: Brion Vibber

body:
On Tue, 21 May 2002, Krzysztof P. Jasiutowicz wrote:
> On 21-05-2002, Brion L. VIBBER wrote thusly :
> > On mar, 2002-05-21 at 10:40, Krzysztof P. Jasiutowicz wrote:
> > > Have you looked up recently the Wikipedia page
> > > http://www.wikipedia.com/wiki/Karl_Landsteiner as I did yesterday ?
> > Well, I did just now, and I replaced the garbage with a brief stub after
> > a 10-second google search turned up several informationful web pages
> > about the guy. You could have done that yourself!
> OK. Next time when I spot such ugly article I certainly will.
> I just thought that some discussion would be helpful. It was an example
> of the problem.

Nic nie szkodzi. No harm done, but you could still have fixed it and then
brought it to our attention. :)

> > The system we have is based on two ideas:
> >
> > a) Changes to articles will be seen and reviewed by other wikipedians
> >   and
> Do we have enough people to look after 100000+ articles ? (+int'l)

If we have enough people to *write* 100000+ articles, then hopefully! If
contributions drop off in the future to lower levels, then there will be
fewer changes to sift through in the list, and new vandalism will be even
easier to find.

The worrying thing is not new vandalism -- it's *old* vandalism. Once
something drops out of RecentChanges, it may as well have fallen off
the face of the Earth until somebody stumbles into it. An article that
nobody fixed a month or a year ago might still be sitting around, and
making it the Stable version ain't gonna help that now. Unless you propose
that 30,000+ articles be individually checked before creating the
"stable" namespace?

> and the reformed Karl Landsteiner article can be (though rather unlikely)
> be vandalized again any minute now.

It can be, of course. And it can be fixed again...

The idea of a stable namespace/review process has been discussed from time
to time, but there's not really been consensus.

If we were to introduce such a system, I think basically this would mean
that for each article, some particular revision of it would be designated
the known good/stable version, and only that version would be shown to
random visitors / distributed in static CD versions. Logged-in users
and those who elected to see the latest unstable version could see and
edit the current revision... New edits would be (somehow) reviewed and,
if ok, designated the new stable version.

There are at least two big questions about how this ought to work:

1) How do we get new folks to dive in and contribute? Having your
improvements show up *immediately* is a big draw to the wikipedia
experience.

2) How does the review process work? If very few people are interested in
a topic, a new article or change might get completely ignored, and very
good articles may never be seen; so limiting reviewing to certain trusted
users would likely not be sufficient. On the other hand, it's child's
play for organized vandals to set up secondary accounts to mod up their
own work, as many discussion-oriented sites have discovered on
establishing user-run comment moderation systems.

> > b) Wikipedians will improve articles they find to be lacking
> >
> > A is dependent on the number of wikipedians interested in any particular
> > topic (or with a mind to check out new articles), and B is dependent on
> > YOU.

> OK. I admit I am amazed by overall quality and how little impact vandals
> have had so far. But is this just an exception or are there more articles
> that need mending ? How do we know ?

I would say that it's an exception (the vast majority of articles are
indeed not vandalized) *and* that there are more that need mending. Keep
looking -- watch the skies!

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] That darn cookie

body:
I've just submitted changes to special_userlog(in|out).php which should
fix the infamous cookie path problem (login cookie sets path of /wiki/ and
thus you aren't logged in when you look at / or /wiki.phtml -- yecch!)

People will have to re-login to make it take effect once installed...

-- brion vibber (brion @ pobox.com)


subject: [Wikipedia-l] New codebase, testbed

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020521/3e7bd334/attachment.diff


from_name: Vicki Rosenzweig

subject: [Wikipedia-l] The 'pedia seems to be broken

body:
Any attempt to open any page is getting me

Fatal error: Cannot instantiate non-existent class: skinstandard in
/home/wiki-newest/work-http/wikiPage.php on line 1142

I'd post this to "Bug reports" on the 'pedia, except I get the same error
there.
--
Vicki Rosenzweig
vr at redbird.org
http://www.redbird.org


from_name: Kurt Jansson

subject: [Wikitech-l] php-wiki de

body:
> I have set up the test site at http://test-de.wikipedia.com/,

Thanks Jason! I informed the German Wikipedians and hope they'll start
playing around with it soon.

I wrote down some bugs I found on
http://test-de.wikipedia.com/wiki/wikipedia:Beobachtete+Fehler

But that's in German, and I don't know anybody else but Magnus will
understand it. So shell I post a (bad) translated report to wikitech-l
from time to time?

* The search for words with umlauts doesn't seem to work. The script
doesn't find them, and when I search for more than one word (including
one with umlauts) I get MySQL errors.

* The conversion didn't correct the upper and lower case problem in
article titles. I thought someone made a script for this?

* And can't we prevent the loss of article histories when converting to
the new format?

BTW, maybe it's possible to keep an unlimited revision history (and
import the old ones) when the script is faster again, or when we change
to the new server.

More complaints coming soon ... ;-)

Kurt


subject: [Wikitech-l] That darn cookie

body:
On Tue, 21 May 2002 lcrocker at nupedia.com wrote:
> (Brion wrote:)
> > I've just submitted changes to special_userlog(in|out).php
> > which should fix the infamous cookie path problem (login
> > cookie sets path of /wiki/ and thus you aren't logged in when
> > you look at / or /wiki.phtml -- yecch!)
>
> This is installed at http://www.piclab.com/wiki/wiki.phtml now,
> so you can test it.

Seems okay so far... cookie path is /, so in theory it should work.

>  While the problem is fresh in your mind, it
> would be helpful if you could test /newwiki as well, since one
> of few things I didn't borrow any code at all for is the user
> login system--it's all new, and I think works the way I would
> expect, but it would be nice to have some other opinions on that.

(Random note: Try going to
  http://www.piclab.com/newwiki/wiki.phtml?title=:Richter_scale
You get [[Ichter scale]] instead.)

As far as the login: it's still setting a cookie path of /newwiki. (It
appears  to always set the current directory unless you specify otherwise
to setcookie(); a simple fix.)

A couple other notes on the login screen:
* Eliminating the separate login step after creating a new account is a
good thing; it saves confusion (and precious mouse clicks). Kudos!

* I don't really like the way the "login" is up top and "create new
account" down bottom. (I'm the one who made it do that, so blame me... I
split them up because it was difficult to figure out how to make them work
when they were merged together -- if you filled out both password fields
it would attempt to create a new account and would NOT log you in to an
existing one. But it's still ugly.) Perhaps we could put them
side-by-side, or just make "create new account" a separate screen which
you can click to from the login form (or from a failed login attempt). In
any case, that's mainly a cosmetic issue.

* I'd love it if, after logging in, I got sent back to where I came from!
If one is logging in in order to edit/deal with watchlist/whatever,
perhaps one is already at a particular page one wants to manipulate, and
doesn't wish to go find it again or back-back-back-reload or use-other-window-
switch-back-reload. (Same would be nice for the user preferences.) What
think you?

* Uhhh, I put another asterisk in here but I don't remember what for. I do
hope it wasn't important.

-- brion vibber (brion @ pobox.com)


from_name: Krzysztof P. Jasiutowicz

subject: [Wikipedia-l] Review mechanism ? Stable pages ?

body:
Hello all,

It might seem a little off-topic on the international mailing list
but it seems rather urgent and is of general interest (also non-English
Wikipedias).

Have you looked up recently the Wikipedia page
http://www.wikipedia.com/wiki/Karl_Landsteiner as I did yesterday ?

With growing number of pages we either need some review and approval
mechanism or a stable namespace or even a stable version of Wikipedia
(that would be so open for editing for everyone).

What do you think ?

regards,
kpj.


subject: [Wikitech-l] That darn cookie

body:
(Messages seem to be falling off the list again.)

On mer, 2002-05-22 at 01:27, lcrocker at nupedia.com wrote:
> >(Random note: Try going to
> >  http://www.piclab.com/newwiki/wiki.phtml?title=:Richter_scale
> >You get [[Ichter scale]] instead.)
>
> Fixed, thanks.
>
> > As far as the login: it's still setting a cookie path of /newwiki.
> > (It appears  to always set the current directory unless you specify
> > otherwise to setcookie(); a simple fix.)
>
> Fixed. Can I ask what you're using to test cookies?  I'm telnetting
> to port 80 and doing things manually which is a pain.

Oh, I'm a lazy one. I just try logging in with Netscape or Mozilla and
look at the resulting cookies file. Good enough for my purposes...

> >A couple other notes on the login screen:
> > * I don't really like the way the "login" is up top and "create
> > new account" down bottom. (I'm the one who made it do that, so
> > blame me... I split them up because it was difficult to figure out
> > how to make them work when they were merged together...
>
> I didn't much like it either, I just quickly copied the old one.  It's
> better now, I think.  Perhaps some text could be added as well.

Ahh, I like the new one! Compact, yet clear in function.

A couple further thoughts: I suspect we might want to explicitly mark
the e-mail field for new accounts as "optional" to appease the paranoids
out there.

It looks kind of like "Remember my password across sessions" is grouped
in with the 'create new account' function, but as far as I know it
applies both for new accounts and later logins. Should it be separated
by a blank line? Eh.

> > * I'd love it if, after logging in, I got sent back to where I
> > came from!
>
> Great idea!  Done.

Woohoo! Thanks.

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] New database scheme

body:
On mar, 2002-05-21 at 23:19, Axel Boldt wrote:
> Since Lee added a namespace field, I think now is the time to
> brainstorm and see if any other changes to the database scheme are
> needed or desirable.
>
> First, I think the cur_namespace should be indexed, since we are using
> it for finding articles. Should old also get a an old_namespace, or do
> you intend to store the namespace as part of the title?

I don't think we need a separate namespace field in old.

> Also, I was
> wondering about the cur_title/cur_ind_title situation. Would it be
> possible to declare cur_title not as binary and combine the two?

IIRC cur_ind_title contains spaces where cur_title contains underscores,
so that words in the title are treated as separate words for searching.

They could potentially be merged by taking the underscores out of
cur_title and doing s/_/ / magic instead of s/ /_/... but that leaves
the question of getting around limitations in MySQL's search indexing.
I'll have to grab version 4 and see what can be done as far as case
folding, accent matching etc; otherwise we may still end up wanting
separate real-content and search-index fields both for titles and
article text.

> Should we store the passwords in user in encrypted form? (Right now,
> any sysop with mysql access can read all passwords.)

Yes, that's been discussed and I think we agreed to do it, it just
hasn't gotten done yet. Since this would require an update script to be
run to make it work, we may as well stick it in with whatever other
database changes we may be making...

> Maybe we should store the user_rights as a bitstring?
>
> The tables linked, unlinked and ipblocks should have PACK_KEYS=1
> (makes reads faster and updates slower). Any other possible fields for
> site_stats? Maybe a date field?

I assume the point of the site_stats table is so that writing updates to
the page view figures etc can't block access to the cur table?

-- brion vibber (brion @ pobox.com)


from_name: Axel Boldt

subject: [Wikitech-l] New database scheme

body:
Since Lee added a namespace field, I think now is the time to
brainstorm and see if any other changes to the database scheme are
needed or desirable.

First, I think the cur_namespace should be indexed, since we are using
it for finding articles. Should old also get a an old_namespace, or do
you intend to store the namespace as part of the title? Also, I was
wondering about the cur_title/cur_ind_title situation. Would it be
possible to declare cur_title not as binary and combine the two?
Should we store the passwords in user in encrypted form? (Right now,
any sysop with mysql access can read all passwords.) Maybe we
should store the user_rights as a bitstring?

The tables linked, unlinked and ipblocks should have PACK_KEYS=1
(makes reads faster and updates slower). Any other possible fields for
site_stats? Maybe a date field?

Axel


subject: [Intlwiki-l] Re: [Wikitech-l] php-wiki de

body:
On mer, 2002-05-22 at 10:36, Kurt Jansson wrote:
> >> I wrote down some bugs I found on
> >> http://test-de.wikipedia.com/wiki/wikipedia:Beobachtete+Fehler
> >>
> >> But that's in German, and I don't know anybody else but Magnus will
> >> understand it. So shell I post a (bad) translated report to
> wikitech-l
> >> from time to time?
>
> The list is growing! Brion, how's your German? :-)

Nicht so gut, but I'll try to muddle through the list mit mein
W?rterbuch und verstehen ein bisse...

> Okay, my word fell under the minimum because of the chopping. "Berlin
> Fl?che" gives me
>
> Warning: Supplied argument is not a valid MySQL result resource in
> /home/wiki-de-php/fpw/special_dosearch.php on line 324

If I actually make the software check return codes and report useful
error messages, I see:

  You have an error in your SQL syntax near 'MATCH (cur_ind_title)
  AGAINST ("Fl?che") ) AND cur_t' at line 3

Looks like mismatched parentheses. I've replaced (what I think is) the
rest of the \w regexps with [\w\x80-\xff], and it looks improved, but as
much of the search code is a mystery to me, I can't guarantee anything!

(Some changes to special_dosearch.php now in CVS.)

> > Which articles are missing histories? I checked a few at random and so
> > far, so good.
>
> Sorry, forget about this. I was talking nonsense.

(Whew!)

> > > More complaints coming soon ... ;-)
> >
> > Vielen dank!
>
> And, sorry, I forgot to mention that the new script all in all REALLY
> KICKS ASS!!! I love it.

Oh, that's so much nicer than "it's too slow!" or "that's a well-meaning
feature but poorly implemented" or "make sure everything works 100%
before putting it online". You're my new favorite user! ;)

-- brion vibber (brion @ pobox.com)


subject: [Wikipedia-l] beta/stable ideas

body:
Chuck Smith wrote:
> This is of course the hardest part.  Let's form a
> Cabal!  :)

Dude, don't even joke about that.  :-)  We don't need that
war to break out again.

> We need to have a base idea to start a system like
> this, so I'm presenting one that we can discuss.

Maybe we should better define the objectives and constraints of such a system.

The most important constraint, I think, is that anything we do must
respect the purity of the wiki-nature.  YOU can edit THIS page NOW.
Wow.

--Jimbo


in_reply_to: <1022002700.2811.0.camel@rdaneel.moisty.org>; from brion@pobox.com on Tue, May 21, 2002 at 10:38:18AM -0700

body:
On Tue, May 21, 2002 at 10:38:18AM -0700, Brion L. VIBBER wrote:
> On mar, 2002-05-21 at 06:28, Kurt Jansson wrote:
> > > I have set up the test site at http://test-de.wikipedia.com/,
> >
> > Thanks Jason! I informed the German Wikipedians and hope they'll start
> > playing around with it soon.
> >
> > I wrote down some bugs I found on
> > http://test-de.wikipedia.com/wiki/wikipedia:Beobachtete+Fehler
> >
> > But that's in German, and I don't know anybody else but Magnus will
> > understand it. So shell I post a (bad) translated report to wikitech-l
> > from time to time?
> >
> > * The search for words with umlauts doesn't seem to work. The script
> > doesn't find them,
>
> Hmm, looks like the search needs another drubbing. I believe it's
> chopping up words at the boundaries of non-ASCII characters; if your
> word is big enough on either end (?bersetzung, terroranschl?ge) this
> does a good enough job anyway, but it's hardly the way it should work!

It doesn't (unless you have changed this in my code, I didn't check). The
parsing function usses the PERL regular expression \w to decide if something
is legal in a search word or not. If it thinks a character is illegal it
gives an error so you should have noticed that. If the result is empty and
you didn't get a syntax error there can be two problems:
- MySQL doesn't index that character. I wasn't able to find out which
characters are exactly indexed by the full-text index and which not. I know
that characters with umlauts get indexed (search for "G"odel" for example)
but I wouldn't know about the ringel-S (sp?) for example. The easiest way to
find out is probably simply trying.
- The special characters in the articles were written by using entities. The
search doesn't know that "o and ö are the same.

Btw, if you have questions about the parse function for the search, just
ask. I'm very busy at the moment so I don't have time to do any real
programming, but answering a few questions should not be a problem.

-- Jan Hidders


subject: [Wikipedia-l] [mjansen@fnl.nl: RE: Wikipedia logo]

body:
I don't recall who orignially designed the Wikipedia logo, but I got a
request for a higher resolution logo from a magazine editor who is
going to be printing a story about wikipedia.  If anybody knows who I
should contact, please let me know.

He wants the logo to be about 1 or 2 inches at 300 dpi.  So, we'll
need it to be about 540x532 pixels.  If we get one that is HUGE, then
I shouldn't ever have to ask for another one...

Jason Richey

----- Forwarded message from mjansen at fnl.nl -----

From: mjansen at fnl.nl
Date: Wed, 22 May 2002 11:01:36 +0200
To: jasonr at bomis.com
Subject: RE: Wikipedia logo

Op dinsdag 21 mei 2002 20:26, heeft Jason Richey [SMTP:jasonr at bomis.com]
geschreven:
> I'll see what I can come up with...  What dimensions are you looking
> for?  Our current logo is 135x133 pixels at 72 pixels/inch, or
> 0.450x0.443 inches at 300 dpi.  I'm sure we can generate the logo as
> you would like it, but it would be helpful to know how large you need
> it.

It would be perfect if the logo could be blown up to a width and height of
about 1 to 2 inches at 300 dpi.

Thanks in advance

Michael Janszen
Redacteur / Editor
c't Magazine
Postbus 31 331
6503 CH Nijmegen
Tel.: 024-3723659

----- End forwarded message -----

--
"Jason C. Richey" <jasonr at bomis.com>


subject: [Wikipedia-l] beta/stable ideas

body:
> There are at least two big questions about how this
> ought to work:
>
> 1) How do we get new folks to dive in and
> contribute? Having your
> improvements show up *immediately* is a big draw to
> the wikipedia
> experience.

Well, we could call it beta and stable if we wanted to
and people could know that the beta version was more
up-to-date than the stable version.  A lot of people
at Wikipedia are here just to read and they can keep
doing that and others are here to write and I suggest
we have a link on the bottom of each stable article
that says, "Edit this article in Beta".  I imagine all
of us would still just watch the beta anyway.  :)

> 2) How does the review process work? If very few
> people are interested in
> a topic, a new article or change might get
> completely ignored, and very
> good articles may never be seen; so limiting
> reviewing to certain trusted
> users would likely not be sufficient. On the other
> hand, it's child's
> play for organized vandals to set up secondary
> accounts to mod up their
> own work, as many discussion-oriented sites have
> discovered on
> establishing user-run comment moderation systems.

This is of course the hardest part.  Let's form a
Cabal!  :)  When there's enough positive reviews from
enough experienced users (maybe those signed up for
over 3 months) and *no* negative reviews (all negative
reviews should have a reason why they voted negative)
then the article makes it to stable.  If someone is
found abusing the system, they could be knocked back
to standard user with a 3 month waiting period to get
back.

This is only a suggestion!  The values of course could
be tweaked, but I think this could work.  It needs to
a) not cause ppl to try to do things to improve their
possibilities of getting in the "Cabal" (um, reviewing
committee), i.e. time works well for this and b) not
easily subject to abuse and c) it needs to be simple!

We need to have a base idea to start a system like
this, so I'm presenting one that we can discuss.

Chuck

=====
Come to my homepage!  Venu al mia hejmpagxo!
http://amuzulo.babil.komputilo.org/
====
Venu al la senpaga, libera enciklopedio
esperanta reta! http://eo.wikipedia.com/

_________________________________________________________
Do You Yahoo!?
Informaci?n de Estados Unidos y Am?rica Latina, en Yahoo! Noticias.
Vis?tanos en http://noticias.espanol.yahoo.com


body:
On mer, 2002-05-22 at 14:43, Jan.Hidders wrote:
> On Wed, May 22, 2002 at 11:14:06AM -0700, Brion L. VIBBER wrote:
> >
> > If I actually make the software check return codes and report useful
> > error messages, I see:
> >
> >   You have an error in your SQL syntax near 'MATCH (cur_ind_title)
> >   AGAINST ("Fl?che") ) AND cur_t' at line 3
> >
> > Looks like mismatched parentheses. I've replaced (what I think is) the
> > rest of the \w regexps with [\w\x80-\xff], and it looks improved, but as
> > much of the search code is a mystery to me, I can't guarantee anything!
>
> It looks Ok to me. Having said that can I as one of the contributers of the
> search code make a small protest here? [But feel free to ignore me because
> I've been away for too long without due notice.] I checked what \w actually
> matches on my system here and it matches the following ASCII codes
> (decimal):
>
>   48 - 57  ( '0' - '9' )
>   65 - 90  ( 'A' - 'Z' )
>   95       ( '_' )
>   79 - 122 ( 'a' - 'z' )
>   170
>   181
>   192 - 214
>   216 - 246
>   248 - 255
>
> So that includes all the German characters (or has our encoding scheme
> changed?)

If you'll recall, we're switching all the wikipedias to UTF-8 (if we
don't do it now, we'll just end up doing it in a few years and it'll be
more painful). The above covers some, but not all UTF-8 sequences that
encode valid letters.

In an ideal world, locale settings would apply (and work correctly and
consistently!) and \w would match everything necessary, but... (PHP
4.1.0 has some sort of special UTF-8 mode for regexps that might or
might not be useful here.)

> and it should have worked with simply \w (and it did, as I said,
> searching for "Go\"del" went fine. This is important because (1) our error
> reporting should be as tight as possible and not just give an empty search
> result if the user types a character that isn't indexed and (2) if something
> did really go wrong before then your quick-fix is now hiding a bug that may
> come to haunt us later.

Oh, I don't doubt that at all. :)

-- brion vibber (brion @ pobox.com)


body:
On mar, 2002-05-21 at 06:28, Kurt Jansson wrote:
> * The conversion didn't correct the upper and lower case problem in
> article titles. I thought someone made a script for this?

Okay, I finally got a chance to finish that up. I've just checked in
recaseLinks.php, which looks through the link tables with the critical
eye of the old capitalization function, counts up the various alternate
name forms in use for each page, and renames articles to the form most
frequently used in links. It also puts in redirects for other forms that
were used, so there shouldn't be breakage.

Jason, if you could give this a try on the German test site:
  php recaseLinks.php
  php rebuildLinks.php # The link tables then need to be rebuilt

(The Esperanto wiki also needs it, but it still needs to be converted
again to see if the alternate charset function works as I think it
should. I'd be mucho grateful if you could do that, so I can set people
loose on bug & message-translation checking!)

-- brion vibber (brion @ pobox.com)


from_name: Lee Daniel Crocker

subject: [Wikitech-l] Note on new codebase

body:
I did send a reply to Alex last night--something seems to have
dropped it on the floor; I don't know if it's on my side or the
list software.  Brion's mail seems to imply the latter, but
since it hasn't happened to me other times that I rememeber,
I'll reserve judgment.

At any rate, thanks, Alex, for pointing out that my new namespace
fields were not indexed.  I also strongly agree that NOW is
exactly the right time to think about tweaking the database
structure--all the infrastructure is in place now, but little
of the hairy structure-sensitive code of the special pages is
written, and the new codebase is well-enough encapsulated that
making database changes should be easy.  Finally, the new code
already requires a database upgrade, so that's no excuse not to
do other useful changes.  I'm updating the conversion script as
as I go.

To test my assumption of encapsulation I did in fact replace the
namespace fields in both the "cur" and "old" tables with integers.
This should keep the database smaller and speed it up.  Making the
change took a lot less time than updating the database itself, so
it looks like the new codebase passed that test.

I'm also not thrilled with the linked/unliked tables.  I'm thinking
of replacing them with a single "X links to Y" table using integer
indexes.

I'm also open to suggestions about how to change cur_ind_title.

Finally, I mostly agree with Axel's contention that "approval"
of articles is just metadata--the piece of information that person
X approved of article Y should just be stored somewhere, and used
when needed to make collections or whatever.  I would call the
users authorized to make such approvals "editors", rather than
"sysops"--the latter seems more like the folks who can do techie
things like direct database access, etc.

Doing that would be easier with yet another database change I
considered earlier--replacing the odd linked-list mechanism of
old page versions seems awkward.  It could be replaced by simply
using revision numbers that would be common to both the "cur" and
"old" tables.  Or an even more radical change--merge both of those
tables in to one, and create a "current_revision" table.  Ideas?

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


body:
>On mar, 2002-05-21 at 06:28, Kurt Jansson wrote:
>> > I have set up the test site at http://test-de.wikipedia.com/,
>>
>> Thanks Jason! I informed the German Wikipedians and hope they'll
start
>> playing around with it soon.
>>
>> I wrote down some bugs I found on
>> http://test-de.wikipedia.com/wiki/wikipedia:Beobachtete+Fehler
>>
>> But that's in German, and I don't know anybody else but Magnus will
>> understand it. So shell I post a (bad) translated report to
wikitech-l
>> from time to time?

The list is growing! Brion, how's your German? :-)

> > and when I search for more than one word (including
> > one with umlauts) I get MySQL errors.
>
> I'm not getting that, this seems to work for me. (Try for example
> "terroranschl?ge staaten".)
>
> Note of course that the dreaded 4-character minimum is still in place.

Okay, my word fell under the minimum because of the chopping. "Berlin
Fl?che" gives me

Warning: Supplied argument is not a valid MySQL result resource in
/home/wiki-de-php/fpw/special_dosearch.php on line 324

Warning: Supplied argument is not a valid MySQL result resource in
/home/wiki-de-php/fpw/special_dosearch.php on line 326

Warning: Supplied argument is not a valid MySQL result resource in
/home/wiki-de-php/fpw/special_dosearch.php on line 334

Warning: Supplied argument is not a valid MySQL result resource in
/home/wiki-de-php/fpw/special_dosearch.php on line 336

> Which articles are missing histories? I checked a few at random and so
> far, so good.

Sorry, forget about this. I was talking nonsense.

> > More complaints coming soon ... ;-)
>
> Vielen dank!

And, sorry, I forgot to mention that the new script all in all REALLY
KICKS ASS!!! I love it.

Kurt


from_name: Bryan Derksen

subject: [Wikipedia-l] beta/stable ideas

body:
At 01:17 AM 5/23/02 +0200, you wrote:
>That said, here's my minimalistic suggestion: everything works exactly
>as it does now, except that every page gets two additional links:
>"View last reviewed version of this article" and "I have reviewed this
>version of the article and I think it is ok". The history of every
>article would record who has reviewed which version of the article and
>when.

How about having a page which presents links to articles in decreasing
order of time since they were last reviewed? If everyone every once in a
while grabbed a few off the top to review, then eventually all of the
articles would be cycled through on a regular basis.

Personally, though, I think things are pretty good as they are right now.

--
"Let there be light." - Last words of Bomb #20, "Dark Star"


in_reply_to: <1022091247.2698.0.camel@rdaneel.moisty.org>; from brion@pobox.com on Wed, May 22, 2002 at 11:14:06AM -0700

body:
On Wed, May 22, 2002 at 11:14:06AM -0700, Brion L. VIBBER wrote:
>
> If I actually make the software check return codes and report useful
> error messages, I see:
>
>   You have an error in your SQL syntax near 'MATCH (cur_ind_title)
>   AGAINST ("Fl?che") ) AND cur_t' at line 3
>
> Looks like mismatched parentheses. I've replaced (what I think is) the
> rest of the \w regexps with [\w\x80-\xff], and it looks improved, but as
> much of the search code is a mystery to me, I can't guarantee anything!

It looks Ok to me. Having said that can I as one of the contributers of the
search code make a small protest here? [But feel free to ignore me because
I've been away for too long without due notice.] I checked what \w actually
matches on my system here and it matches the following ASCII codes
(decimal):

  48 - 57  ( '0' - '9' )
  65 - 90  ( 'A' - 'Z' )
  95       ( '_' )
  79 - 122 ( 'a' - 'z' )
  170
  181
  192 - 214
  216 - 246
  248 - 255

So that includes all the German characters (or has our encoding scheme
changed?) and it should have worked with simply \w (and it did, as I said,
searching for "Go\"del" went fine. This is important because (1) our error
reporting should be as tight as possible and not just give an empty search
result if the user types a character that isn't indexed and (2) if something
did really go wrong before then your quick-fix is now hiding a bug that may
come to haunt us later.

-- Jan Hidders


subject: [Wikipedia-l] beta/stable ideas

body:
Hi Chuck!

> > There are at least two big questions about how this
> > ought to work:
> >
> > 1) How do we get new folks to dive in and
> > contribute? Having your
> > improvements show up *immediately* is a big draw to
> > the wikipedia
> > experience.
>
> Well, we could call it beta and stable if we wanted to
> and people could know that the beta version was more
> up-to-date than the stable version.  A lot of people
> at Wikipedia are here just to read and they can keep
> doing that and others are here to write and I suggest
> we have a link on the bottom of each stable article
> that says, "Edit this article in Beta".

I really like the idea. The argument most people come up with in a
discussion about WP (after the ones we all know ...) is, that they can't
rely on the information they find, because too few people might have
proofread it or someone might have changed the text 5 seconds before the
accessed the page.

How about a debian-like scheme (as I understand it):

* unstable/sid (still in development) = beta = what we have now
* testing = proofread by enough wikipedians in a process like the one
Chuck suggested
* stable = Nupedia (in a few years or so ;-)

I'm a bit sceptic about Nupedia and the authority idea behind it (but I
admit that I don't know much about it), but I'm sure that it will
attract many scholars.

How do we take articles out of testing or stable if a proven mistake is
found? By voting in a short time period (3 days)? Or do we need elected
maintainers?

And we will need to vote in every change that is made in the rules of
the process. It'll get much (!) more complicated that WP, but the idea
is very attracting!

> I imagine all of us would still just watch the beta anyway.  :)

Of course! :-)

Bye,
Kurt

P.S.: A last association:
* unstable/sid (still in development) = anarchy
* testing = democracy
* stable = meritocracy?


subject: [Wikipedia-l] beta/stable ideas

body:
Honestly, I am not sure that anything in the direction of
review/stable release etc. should be done, at least as long as the
average article quality continues to rise, which I think it does.

People who complain that they can't trust anything they find in
Wikipedia should be commended for their healthy attitude; it is much
more dangerous to think "see, it says here, this is part of the stable
Wikipedia, so everything on this page must be right".

That said, here's my minimalistic suggestion: everything works exactly
as it does now, except that every page gets two additional links:
"View last reviewed version of this article" and "I have reviewed this
version of the article and I think it is ok". The history of every
article would record who has reviewed which version of the article and
when.

The set of all "last reviewed versions" could then be seen as the
"stable" Wikipedia and could be pressed on CD. This would at least
guard against vandalism, stupid jokes and blatant propaganda and
advertising that sometimes gets through.

The only issue is: who is allowed to review articles? The pragmatic
answer would be: all sysops. A code of honor is probably in order,
saying that no sysop should review an article that they themselves
substantially contributed to.

Axel


body:
On ?a?, 2002-05-23 at 02:21, Jan.Hidders wrote:
> On Wed, May 22, 2002 at 03:34:08PM -0700, Brion L. VIBBER wrote:
> >
> > If you'll recall, we're switching all the wikipedias to UTF-8 (if we don't
> > do it now, we'll just end up doing it in a few years and it'll be more
> > painful).
>
> Uh oh. I remember we had that discussion and as I already said then: moving
> to UTF-8 breaks the fulltext search.

Fulltext search is already broken in a million ways! It doesn't know
character references (ü, ĉ, į etc), it can't find
partial matches or sounds-likes, it can't find "X" when you search for
"Xs" or "Xs" when you search for "X", it doesn't return *ANY* results
for words it thinks are too common...

UTF-8 is the least of our problems; it just means that case-folding is a
little trickier (and if we had a decent $*#@%# database, it would take
care of that for us).

> This is because the indexing algorithm
> assumes we use Latin-1 and bases upon that its decision to collate and
> chooses which characters to index. For the German characters this will
> probably more or less work out, but if you go you beyond (why else use
> UTF-8?) that you will get into severe trouble.
>
> Do you realize how serious this situation is?

Yes, that's *exactly* why we have to break out of the "everything in the
world should be Latin-1, oh and by the way even though we have limited
support for other character sets -- but not the ones YOU need -- you can
only select one single character set for the whole database server!
MWOOHAAHAAHAAA!" rut and make it work for the rest of us too.

> > In an ideal world, locale settings would apply (and work correctly and
> > consistently!) and \w would match everything necessary, but... (PHP 4.1.0
> > has some sort of special UTF-8 mode for regexps that might or might not be
> > useful here.)
>
> It's not so much PHP that is the problem, as it is MySQL.

The particular problem I was discussing was the regexps, which were a
PHP problem. MySQL could do all the magic it wanted; if the words don't
get through the regexps in the PHP code they'll never get anywhere in
the database's fulltext search.

> Perhaps we should
> considering moving to PostgreSQL which really supports UTF-8 and is a better
> database anyway (some special pages could be implemented far more
> efficiently there).

Oh, it's not like that hasn't been suggested. If anybody knows how to go
about switching to Postgres, I sure as heck wouldn't object. I have no
emotional attachment to MySQL; as far as I know it's only being used
because Magnus was already familiar with it.

-- brion vibber (brion @ pobox.com)


in_reply_to: <1022150580.4700.0.camel@rdaneel.moisty.org>; from brion@pobox.com on Thu, May 23, 2002 at 03:42:59AM -0700

body:
Here's a brief history of why we use MySQL:

Bomis' first database experience was using the software that runs
slashdot.  At the time, it required MySQL (other databases are
supported now).  So, we installed it.  Our next database project
(Nupedia) was shortly thereafter, and we had no reason to learn a
whole different database setup.  So, when Wikipedia came along, and we
decided to make it database-driven, it just didn't make sense to
change databases, especially since it would mean we would have to
change Nupedia to the new format.

Now, the programmer who wrote Nupedia (for the most part) is no longer
with Bomis.  I'm not sure what, if any, MySQL specific properties
Nupedia uses (or Wikipedia, for that matter).  I am not opposed to
installing a different database, but I can't really spend a lot of
time on the conversion of the software.  Of course, it's not really my
decision anyway...

Jason Richey

Brion L. VIBBER wrote:

> Oh, it's not like that hasn't been suggested. If anybody knows how to go
> about switching to Postgres, I sure as heck wouldn't object. I have no
> emotional attachment to MySQL; as far as I know it's only being used
> because Magnus was already familiar with it.

--
"Jason C. Richey" <jasonr at bomis.com>


body:
Jan.Hidders wrote:
> It's not so much PHP that is the problem, as it is MySQL. Perhaps we should
> considering moving to PostgreSQL which really supports UTF-8 and is a better
> database anyway (some special pages could be implemented far more
> efficiently there).

A good opportunity for switching from MySQL to PostgreSQL is coming
soon, with the move from our current server to a new and much more
powerful server.  I'd be happy to install PostgreSQL over there
instead.  And we'll be giving out developer accounts to several
people, including root access where appropriate, so things are going
to be a lot easier for the development team.

I propose this as also a good time to switch to Lee's refactored
codebase, unless the consensus is that his code is not better than the
existiing codebase.

--Jimbo


body:
On ?a?, 2002-05-23 at 04:52, Jan.Hidders wrote:
> On Thu, May 23, 2002 at 03:42:59AM -0700, Brion L. VIBBER wrote:
> > Fulltext search is already broken in a million ways! It doesn't know
> > character references (ü, ĉ, į etc), it can't find
> > partial matches or sounds-likes, it can't find "X" when you search for
> > "Xs" or "Xs" when you search for "X", it doesn't return *ANY* results
> > for words it thinks are too common...
>
> All true, although the last problem will be solved when we move to the new
> MySQL and use the boolean search there. But at least there was a
> well-defined semantics: if a word shows up in the edit-text the search will
> find it. This is now no longer the case and no clever PHP programming can
> solve this.

What's not well-defined about "We're having some problems with the
search engine right now; if you don't find what you're looking for at
first, try capitalizing it and search again." ?

> > UTF-8 is the least of our problems; it just means that case-folding is a
> > little trickier (and if we had a decent $*#@%# database, it would take
> > care of that for us).
>
> Case folding is not the only problem. The problem is that the fulltext index
> does not index certain characters above 128. That means that words that
> contain multibyte characters that are represented using such characters will
> not be indexed. That's a bit harder to explain to the users than the
> previous problems.

If it is in fact having problems indexing chars over 128, install the
attached hacked character set definition file and reindex the database.

  my.cnf -> /etc/ (or wherever)
  Index, custom.conf -> /usr/share/mysql/charsets/ (or wherever)
  Run 'myisamchk -r -q' over the tables.

Works great on my machine...

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
# Configuration file for the latin1 character set

# ctype array (must have 257 elements)
  00
  20  20  20  20  20  20  20  20  20  28  28  28  28  28  20  20
  20  20  20  20  20  20  20  20  20  20  20  20  20  20  20  20
  48  10  10  10  10  10  10  10  10  10  10  10  10  10  10  10
  84  84  84  84  84  84  84  84  84  84  10  10  10  10  10  10
  10  81  81  81  81  81  81  01  01  01  01  01  01  01  01  01
  01  01  01  01  01  01  01  01  01  01  01  10  10  10  10  10
  10  82  82  82  82  82  82  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  10  10  10  10  20
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02
  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02  02

# to_lower array (must have 256 elements)
  00  01  02  03  04  05  06  07  08  09  0A  0B  0C  0D  0E  0F
  10  11  12  13  14  15  16  17  18  19  1A  1B  1C  1D  1E  1F
  20  21  22  23  24  25  26  27  28  29  2A  2B  2C  2D  2E  2F
  30  31  32  33  34  35  36  37  38  39  3A  3B  3C  3D  3E  3F
  40  61  62  63  64  65  66  67  68  69  6A  6B  6C  6D  6E  6F
  70  71  72  73  74  75  76  77  78  79  7A  5B  5C  5D  5E  5F
  60  61  62  63  64  65  66  67  68  69  6A  6B  6C  6D  6E  6F
  70  71  72  73  74  75  76  77  78  79  7A  7B  7C  7D  7E  7F
  80  81  82  83  84  85  86  87  88  89  8A  8B  8C  8D  8E  8F
  90  91  92  93  94  95  96  97  98  99  9A  9B  9C  9D  9E  9F
  A0  A1  A2  A3  A4  A5  A6  A7  A8  A9  AA  AB  AC  AD  AE  AF
  B0  B1  B2  B3  B4  B5  B6  B7  B8  B9  BA  BB  BC  BD  BE  BF
  C0  C1  C2  C3  C4  C5  C6  C7  C8  C9  CA  CB  CC  CD  CE  CF
  D0  D1  D2  D3  D4  D5  D6  D7  D8  D9  DA  DB  DC  DD  DE  DF
  E0  E1  E2  E3  E4  E5  E6  E7  E8  E9  EA  EB  EC  ED  EE  EF
  F0  F1  F2  F3  F4  F5  F6  F7  F8  F9  FA  FB  FC  FD  FE  FF

# to_upper array (must have 256 elements)
  00  01  02  03  04  05  06  07  08  09  0A  0B  0C  0D  0E  0F
  10  11  12  13  14  15  16  17  18  19  1A  1B  1C  1D  1E  1F
  20  21  22  23  24  25  26  27  28  29  2A  2B  2C  2D  2E  2F
  30  31  32  33  34  35  36  37  38  39  3A  3B  3C  3D  3E  3F
  40  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
  50  51  52  53  54  55  56  57  58  59  5A  5B  5C  5D  5E  5F
  60  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
  50  51  52  53  54  55  56  57  58  59  5A  7B  7C  7D  7E  7F
  80  81  82  83  84  85  86  87  88  89  8A  8B  8C  8D  8E  8F
  90  91  92  93  94  95  96  97  98  99  9A  9B  9C  9D  9E  9F
  A0  A1  A2  A3  A4  A5  A6  A7  A8  A9  AA  AB  AC  AD  AE  AF
  B0  B1  B2  B3  B4  B5  B6  B7  B8  B9  BA  BB  BC  BD  BE  BF
  C0  C1  C2  C3  C4  C5  C6  C7  C8  C9  CA  CB  CC  CD  CE  CF
  D0  D1  D2  D3  D4  D5  D6  D7  D8  D9  DA  DB  DC  DD  DE  DF
  E0  E1  E2  E3  E4  E5  E6  E7  E8  E9  EA  EB  EC  ED  EE  EF
  F0  F1  F2  F3  F4  F5  F6  F7  F8  F9  FA  FB  FC  FD  FE  FF

# sort_order array (must have 256 elements)
  00  01  02  03  04  05  06  07  08  09  0A  0B  0C  0D  0E  0F
  10  11  12  13  14  15  16  17  18  19  1A  1B  1C  1D  1E  1F
  20  21  22  23  24  25  26  27  28  29  2A  2B  2C  2D  2E  2F
  30  31  32  33  34  35  36  37  38  39  3A  3B  3C  3D  3E  3F
  40  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
  50  51  52  53  54  55  56  57  58  59  5A  5B  5C  5D  5E  5F
  60  41  42  43  44  45  46  47  48  49  4A  4B  4C  4D  4E  4F
  50  51  52  53  54  55  56  57  58  59  5A  7B  7C  7D  7E  7F
  80  81  82  83  84  85  86  87  88  89  8A  8B  8C  8D  8E  8F
  90  91  92  93  94  95  96  97  98  99  9A  9B  9C  9D  9E  9F
  A0  A1  A2  A3  A4  A5  A6  A7  A8  A9  AA  AB  AC  AD  AE  AF
  B0  B1  B2  B3  B4  B5  B6  B7  B8  B9  BA  BB  BC  BD  BE  BF
  41  41  41  41  5C  5B  5C  43  45  45  45  45  49  49  49  49
  44  4E  4F  4F  4F  4F  5D  D7  D8  55  55  55  59  59  DE  DF
  41  41  41  41  5C  5B  5C  43  45  45  45  45  49  49  49  49
  44  4E  4F  4F  4F  4F  5D  F7  D8  55  55  55  59  59  DE  FF
-------------- next part --------------
# sql/share/charsets/Index
#
# This file lists all of the available character sets.

big5 		   1
czech 		   2
dec8 		   3
dos 		   4
german1 	   5
hp8 		   6
koi8_ru 	   7
latin1		   8
latin2		   9
swe7 		  10
usa7 		  11
ujis 		  12
sjis 		  13
cp1251		  14
danish		  15
hebrew		  16
# The win1251 character set is deprecated.  Please use cp1251 instead.
win1251 	  17
tis620		  18
euc_kr		  19
estonia 	  20
hungarian 	  21
koi8_ukr 	  22
win1251ukr 	  23
gb2312		  24
greek 		  25
win1250 	  26
croat 		  27
gbk 		  28
cp1257		  29
latin5		  30
custom		  31
-------------- next part --------------
[server]
default-character-set=custom


subject: [Fwd: Re:  [Intlwiki-l] Re: [Wikitech-l] php-wiki de]

body:
(Dropped off the list. Hmm, I notice there's no "reply-to" header on
wikitech-l messages like there is on wikipedia-l. Is this what's
tripping people up?)

-----Forwarded Message-----

From: lcrocker at nupedia.com
To: BrionL. VIBBER <brion at pobox.com>
Subject: Re:  [Intlwiki-l] Re: [Wikitech-l] php-wiki de
Date: 23 May 2002 10:09:01 -0700

>> Perhaps we should considering moving to PostgreSQL which
>> really supports UTF-8...

> Oh, it's not like that hasn't been suggested. If anybody knows
> how to go about switching to Postgres, I sure as heck wouldn't
> object...

I'll look into installing it at Piclab so we can test it.
0


in_reply_to: <1022106849.3538.0.camel@rdaneel.moisty.org>; from brion@pobox.com on Wed, May 22, 2002 at 03:34:08PM -0700

body:
On Wed, May 22, 2002 at 03:34:08PM -0700, Brion L. VIBBER wrote:
>
> If you'll recall, we're switching all the wikipedias to UTF-8 (if we don't
> do it now, we'll just end up doing it in a few years and it'll be more
> painful).

Uh oh. I remember we had that discussion and as I already said then: moving
to UTF-8 breaks the fulltext search. This is because the indexing algorithm
assumes we use Latin-1 and bases upon that its decision to collate and
chooses which characters to index. For the German characters this will
probably more or less work out, but if you go you beyond (why else use
UTF-8?) that you will get into severe trouble.

Do you realize how serious this situation is?

> In an ideal world, locale settings would apply (and work correctly and
> consistently!) and \w would match everything necessary, but... (PHP 4.1.0
> has some sort of special UTF-8 mode for regexps that might or might not be
> useful here.)

It's not so much PHP that is the problem, as it is MySQL. Perhaps we should
considering moving to PostgreSQL which really supports UTF-8 and is a better
database anyway (some special pages could be implemented far more
efficiently there).

-- Jan Hidders


subject: [Wikipedia-l] beta/stable ideas

in_reply_to: <200205222317.BAA23139@linde.uni-paderborn.de>; from axel@uni-paderborn.de on Thu, May 23, 2002 at 01:17:39AM +0200

body:
On Thu, May 23, 2002 at 01:17:39AM +0200, Axel Boldt wrote:
> Honestly, I am not sure that anything in the direction of
> review/stable release etc. should be done, at least as long as the
> average article quality continues to rise, which I think it does.

I completely agree with Alex here. The openness and accesability of Wikpedia
are both the causes of its success and its vulnerability. If reviewing
didn't work for Nupedia, why do you think it will work now? How do you
expect to be able to review 30.000+ articles?

What bothers me is that this again a small extension that makes the
interface and the software a little more complex. I will repeat ad nauseam
that everything about Wikipedia should be kept as simple as possible.
Wikpedia has already strayed far too far from the path of WikiWiki.

-- Jan Hidders


subject: [Wikitech-l] Note on new codebase

body:
On Wed, May 22, 2002 at 07:17:12PM -0500, Lee Daniel Crocker wrote:
>
> To test my assumption of encapsulation I did in fact replace the
> namespace fields in both the "cur" and "old" tables with integers.
> This should keep the database smaller and speed it up.

For indexed fields this speed-up will be negligible.

> I'm also not thrilled with the linked/unliked tables.  I'm thinking
> of replacing them with a single "X links to Y" table using integer
> indexes.

I considered that when designing them but also here indices are defined so
the speed-up will be very small and having the full title there helps to
avoid certain joins which are not negligible.

> I'm also open to suggestions about how to change cur_ind_title.

What is wrong with it?

-- Jan Hidders


subject: [Wikitech-l] Missing mail

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020523/54c8204c/attachment.ksh>


subject: [Wikitech-l] vacation

body:
I'm leaving this afternoon for a vacation.  I'll be back on the fifth
of June...

--
"Jason C. Richey" <jasonr at bomis.com>


body:
On Thu, May 23, 2002 at 03:42:59AM -0700, Brion L. VIBBER wrote:
> On ??a??, 2002-05-23 at 02:21, Jan.Hidders wrote:
> >
> > Uh oh. I remember we had that discussion and as I already said then:
> > moving to UTF-8 breaks the fulltext search.
>
> Fulltext search is already broken in a million ways! It doesn't know
> character references (ü, ĉ, į etc), it can't find
> partial matches or sounds-likes, it can't find "X" when you search for
> "Xs" or "Xs" when you search for "X", it doesn't return *ANY* results
> for words it thinks are too common...

All true, although the last problem will be solved when we move to the new
MySQL and use the boolean search there. But at least there was a
well-defined semantics: if a word shows up in the edit-text the search will
find it. This is now no longer the case and no clever PHP programming can
solve this.

> UTF-8 is the least of our problems; it just means that case-folding is a
> little trickier (and if we had a decent $*#@%# database, it would take
> care of that for us).

Case folding is not the only problem. The problem is that the fulltext index
does not index certain characters above 128. That means that words that
contain multibyte characters that are represented using such characters will
not be indexed. That's a bit harder to explain to the users than the
previous problems.

> > It's not so much PHP that is the problem, as it is MySQL.
>
> The particular problem I was discussing was the regexps, which were a
> PHP problem. MySQL could do all the magic it wanted; if the words don't
> get through the regexps in the PHP code they'll never get anywhere in
> the database's fulltext search.

I know, but what I am saying is that this is something we can solve with PHP
programming. But if MySQL doesn't index it then we can we can let through all
we want but it won't turn up in the search results. Period.

> > Perhaps we should considering moving to PostgreSQL which really supports
> > UTF-8 and is a better database anyway (some special pages could be
> > implemented far more efficiently there).
>
> Oh, it's not like that hasn't been suggested. If anybody knows how to go
> about switching to Postgres, I sure as heck wouldn't object.

Unfortunately, I only have very limited hands-on experience with PostgreSQL
and no experience with its fulltext indexing. From the manuals I gather it
does have such a thing but it takes some configuring. On the up-side it
seems far more configurable and you can in fact change the parser that looks
for which words to index, how to collate, et cetera. It was written by
Russians, I believe, who are generally more aware of character set problems
anyway.

But the problem is that I don't have the time to research this; next month I
have a conference and I have two dead-lines for articles coming up.

-- Jan Hidders


from_name: The Cunctator

subject: [Wikitech-l] MySQL vs. Postgre

body:
On 5/23/02 1:34 PM, "Jimmy Wales" <jwales at bomis.com> wrote:

> Jan.Hidders wrote:
>> It's not so much PHP that is the problem, as it is MySQL. Perhaps we should
>> considering moving to PostgreSQL which really supports UTF-8 and is a better
>> database anyway (some special pages could be implemented far more
>> efficiently there).
>
> A good opportunity for switching from MySQL to PostgreSQL is coming
> soon, with the move from our current server to a new and much more
> powerful server.  I'd be happy to install PostgreSQL over there
> instead.  And we'll be giving out developer accounts to several
> people, including root access where appropriate, so things are going
> to be a lot easier for the development team.
>
I gotta say, I'm hardly convinced that Postgre is that astoundingly better
than MySQL for our purposes. As was pointed out, Slashdot runs on MySQL, and
it's a damn fast, high-traffic, complicated site.

If you guys want to change to Postgre, go ahead, but it should be recognized
that it's a mostly arbitrary decision.

At least that's how I see it.

--tc


from_name: Magnus Manske

body:
> -----Original Message-----
> From: wikitech-l-admin at nupedia.com
> [mailto:wikitech-l-admin at nupedia.com]On Behalf Of Jason Richey
> Sent: Thursday, May 23, 2002 6:58 PM
> To: Brion L. VIBBER
> Cc: Jan.Hidders; Wikitech-L
> Subject: Re: [Intlwiki-l] Re: [Wikitech-l] php-wiki de
>
> Now, the programmer who wrote Nupedia (for the most part) is no longer
> with Bomis.  I'm not sure what, if any, MySQL specific properties
> Nupedia uses (or Wikipedia, for that matter).  I am not opposed to
> installing a different database, but I can't really spend a lot of
> time on the conversion of the software.  Of course, it's not really my
> decision anyway...

The current Nupedia software seems pretty unusable at the moment (copyeditor
pages, among others). Personally, I doubt it will see the rebirth of
Nupedia. As I keep advertising ;) I'm rewriting the whole thing
(http://nunupedia.sourceforge.net). I doubt any MySQL-specific queries will
be necessary in that project, so there's no need to keep MySQL just for
Nupedia's sake.

As for Wikipedia, except for the search function, I don't think there's much
(if any) MySQL-specific stuff in there. Shouldn't be too hard to change to
something else. Maybe we could use the Lee's test server for trying???

Magnus


in_reply_to: <1022176755.2674.0.camel@rdaneel.moisty.org>; from brion@pobox.com on Thu, May 23, 2002 at 10:59:13AM -0700

body:
On Thu, May 23, 2002 at 10:59:13AM -0700, Brion L. VIBBER wrote:
>
> If it is in fact having problems indexing chars over 128, install the
> attached hacked character set definition file and reindex the database.

Wow! A customized character set! I didn't know that was possible in MySQL.
That looks indeed like a solution. I bow before you, oh master.

-- Jan Hidders

PS. I'd still favour a move to PostgreSQL though, which would probably be a
cleaner solution and has also some other benifits. But before we do that I
would like to be very very sure that fulltext indexing (which is now I
believe the only MySQL-specific thing we use) works well there.


subject: [Wikitech-l] MySQL vs. Postgre

in_reply_to: <B912C53F.104E%cunctator@kband.com>; from cunctator@kband.com on Thu, May 23, 2002 at 04:11:43PM -0400

body:
On Thu, May 23, 2002 at 04:11:43PM -0400, The Cunctator wrote:
>
> I gotta say, I'm hardly convinced that Postgre is that astoundingly better
> than MySQL for our purposes. As was pointed out, Slashdot runs on MySQL,
> and it's a damn fast, high-traffic, complicated site.

For a few special pages (recent changes and lonely pages) I needed nested
subqueries, but MySQL doesn't have those. As a consequence I had to write
some PHP code for some high volume data processing that can be done far more
efficiently by the database.

-- Jan Hidders


body:
On Thu, May 23, 2002 at 09:33:21PM +0200, Jan.Hidders wrote:
>
> PS. I'd still favour a move to PostgreSQL though, which would probably be a
> cleaner solution and has also some other benifits. But before we do that I
> would like to be very very sure that fulltext indexing (which is now I
> believe the only MySQL-specific thing we use) works well there.

An interesting alternative might be to do the indexing ourselves. This may
sound very inefficient but let me explain. One method/trick that was used in
PgSQL to get a fulltext index was to create someting like a separate table

   text_index( search_word, document_id )

with a conventional index on the column search_word. Now, every time a row
is added to the table that you want to be searchable you retrieve all the
search words (and their suffixes) and add the corresponding
pairs to the table text_index. So if the text in document with id '5' is
"Text Indexing" you add the following pairs to table text_index:

  ( "text", 5 )
  ( "ext", 5 )
  ( "xt", 5 )
  ( "indexing", 5 )
  ( "ndexing", 5 )
  ...

If you now want to search for an article containing "dex" you can query the
text_index table and use in your WHERE clause something like

  "search_word LIKE 'dex%'"

In this case the index for the search_word column is used because the LIKE
expression doesn't start with a wild-card. As you can see this will also
give you the partial matches. In PgSQL this is easy to implement because you
can define triggers that call certain functions whenever a row is inserted
or deleted in a table. But also in MySQL we could do this, although we
would then have to ensure ourselves that every time an article is
created/updated/deleted the text_index table is also updated.

The nice part is that we would have total control over what is indexed and
how it is indexed in the form of a PHP function that determines which pairs
are added to text_index. This function could take care of collation and
could even do nifty things like taking entity references into account or let
a-umlaut match with ae. The different Wikipedias would probably all have a
localized version of this function.

The down-side is of course some performance when articles are updates and
the extra work we need to do on writing this PHP function (although initial
version could be quite simple).

-- Jan Hidders


from_name: manning at bartlett.net

subject: [Wikipedia-l] Re: beta/stable discussion

body:
Hi all from extremely (and fatally) hot Hyderabad, India... 1037 dead so far in
this town alone. Anywhere else and this would be an international tragedy, here
it barely makes the papers.

Beta-stable: In the words of Mr Horse (from Ren and Stimpy) "No Sir, I don't
like it". One of LMS's early ideas was that when an article looked good enough
it would get moved to Nupedia. Problem with that was that no-one ever actually
went to Nupedia.

The same will happen here, if we create a Wikipedia that is static (can't be
edited) then no-one will go there. Even the people who are truly only here to
browse will gravitate to the dynamic site, simply because it is possible that
that site "is more current".

To prove this I'd recommend creating a one-off Static version, putting a link
to it from the dynamic site and then monitoring web-traffic over a period of 2-
3 months. Even after the site has been around for a while I'd be willing to
wager it would still be a very lonely place.

If I am wrong, and the site does get a lot of traffic, well then we should
spend the time and develop a proper management and quality control system. But
until we have proved its viability, we would just be wasting effort - effort
better spent on creating/editing articles.

Cheers
Manning
(Back in civilisation around June 20)

*Doomsayers have successfully forecasted 35 of the last 2 problems in
Wikipedia's
history*


subject: [Wikitech-l] Linked list in "old" table

body:
>replacing the odd linked-list mechanism of
>old page versions seems awkward.  It could be replaced by simply
>using revision numbers that would be common to both the "cur" and
>"old" tables.

Yes, I think the linked-list approach should go. Especially since the
current codebase doesn't seem to use it. We saw a while ago that the
[[Pim Fortuyn]] article has a broken linked list, probably produced by
some database timeout, and yet the history page shows up fine, only
the diffs are computed incorrectly.

Every article should have a clean revision number, period. That way,
there would be an unambiguous way to the n-th version of some article.
The current mixture of oldid and version (look at the links on a
history page) is a mess.

Axel


subject: [Wikitech-l] Linked list in "old" table

in_reply_to: <200205241459.QAA03569@linde.uni-paderborn.de>; from axel@uni-paderborn.de on Fri, May 24, 2002 at 04:59:18PM +0200

body:
On Fri, May 24, 2002 at 04:59:18PM +0200, Axel Boldt wrote:
> >replacing the odd linked-list mechanism of old page versions seems
> >awkward.  It could be replaced by simply using revision numbers that
> >would be common to both the "cur" and "old" tables.
>
> Yes, I think the linked-list approach should go. Especially since the
> current codebase doesn't seem to use it.

I couldn't agree more. The linked list is redundant if you assume that every
revision has a unique timestamp, i.e., pages are not updated within a
second. (In that case even a revision number would be redundant.) I had to
make that assumption to get some reasonable SQL on the history page, but
that could be easily adapted if a version number would be introduced.

Note by the way that the version number on the history page is in reverse,
i.e., the lowest number is the latest version, so that is probably not the
number you want to store in the database.

Hmmm, thinking about this it ocurred to me that since old_id is auto_inc I
could have used that to determine the order of the versions without making
any assumptions at all. :-/ So as far as I can tell we don't even need a
version number in the database.

-- Jan Hidders


from_name: Daniel Mayer

subject: [Wikipedia-l] beta/stable ideas

body:
On Thursday 23 May 2002 12:01 pm, Bryan wrote:
> How about having a page which presents links to articles in decreasing
> order of time since they were last reviewed? If everyone every once in a
> while grabbed a few off the top to review, then eventually all of the
> articles would be cycled through on a regular basis.

Great idea!

maveric149


from_name: Guardian Tor

subject: [Wikipedia-l] beta/stable ideas

body:
This sounds like a trust metric, which the folks over at advogato.org are very fond of. There's some interesting ideas in this article: http://www.advogato.org/article/454.html

- Stephen Gilbert

----- Original Message -----
From: Axel Boldt <axel at uni-paderborn.de>
Date: Thu, 23 May 2002 01:17:39 +0200 (MET DST)
To: wikipedia-l at nupedia.com
Subject: [Wikipedia-l] beta/stable ideas

Honestly, I am not sure that anything in the direction of
review/stable release etc. should be done, at least as long as the
average article quality continues to rise, which I think it does.

People who complain that they can't trust anything they find in
Wikipedia should be commended for their healthy attitude; it is much
more dangerous to think "see, it says here, this is part of the stable
Wikipedia, so everything on this page must be right".

That said, here's my minimalistic suggestion: everything works exactly
as it does now, except that every page gets two additional links:
"View last reviewed version of this article" and "I have reviewed this
version of the article and I think it is ok". The history of every
article would record who has reviewed which version of the article and
when.

The set of all "last reviewed versions" could then be seen as the
"stable" Wikipedia and could be pressed on CD. This would at least
guard against vandalism, stupid jokes and blatant propaganda and
advertising that sometimes gets through.

The only issue is: who is allowed to review articles? The pragmatic
answer would be: all sysops. A code of honor is probably in order,
saying that no sysop should review an article that they themselves
substantially contributed to.

Axel
[Wikipedia-l]
To manage your subscription to this list, please go here:
http://www.nupedia.com/mailman/listinfo/wikipedia-l

________________________________
Wikipedia: The Free Encyclopedia
   http://www.wikipedia.com

--

Powered by Outblaze


subject: [Wikipedia-l] Re: Manning's beta/stable comments

body:
On Friday 24 May 2002 12:01 pm, Manning wrote:
> Beta-stable: In the words of Mr Horse (from Ren and Stimpy) "No Sir, I
> don't like it". One of LMS's early ideas was that when an article looked
> good enough it would get moved to Nupedia. Problem with that was that
> no-one ever actually went to Nupedia.

My opinion on the reason why this particular idea of Larry's didn't work was
because there wasn't any practical (or otherwise) integration of
Nupedia with wikipedia and the two projects represented two different
cultures and methodologies -- not because the concept was inherently flawed.
If there is tight integration of this proposed functionality and it is made
very clear to viewers of the static pages that an editable version exists by
"clicking here", I think it just may work. Although to clarify, I think the
most current editable version of articles should be the one displayed by
default -- a user would have to click to view the reviewed version (the
difference between the two versions also needs to be made obvious and
intuitive).

> The same will happen here, if we create a Wikipedia that is static (can't
> be edited) then no-one will go there. Even the people who are truly only
> here to browse will gravitate to the dynamic site, simply because it is
> possible that that site "is more current".

Interesting idea, but I think this only really applies to casual visitors and
not to researchers or students who need reliable data. For example:

If I needed information about a topic I would put far more trust in a static
page that had been reviewed by 10 people with trusted status than I would
place on an "up to date" yet un-reviewed version. How could I trust the
validity of the world-editable version if I were witting a report? What use
would there be in citing and linking to the world-editable version if I can't
have a reasonable amount of confidence that the information I am citing will
be there when someone checking my work follows the link?

It would reflect very negatively on me if I cited something interesting I
found in a beautifully written and informative - yet world editable -
wikipedia article on say the Apollo Moon landings, and then a reviewer of my
work checked the link and instead of seeing what I saw, sees a totally
unrelated and unintelligible diatribe about CARROTS. Or worse, the version of
the article that the reviewer sees was rewritten by a pseudoscience freak who
is highly skeptical of NASA's "claim" that the Moon landings actually took
place. How is that going to make me and my work look to the reviewer?

> To prove this I'd recommend creating a one-off Static version, putting a
> link to it from the dynamic site and then monitoring web-traffic over a
> period of 2- 3 months. Even after the site has been around for a while I'd
> be willing to wager it would still be a very lonely place.

Sounds like a reasonable.proposal -- I too am not totally convinced that any
proposed "review process" can actually work. But my concern is that the size
of our current contributor base is not yet large enough to make this
practical. There doesn't seem to be enough eyes yet to keep reviewed versions
of articles reasonably up-to-date. And there is of course a HUGE backlog of
articles that haven't been touched by any editor for multiple months. But
then, starting this process wouldn't hurt the current "unreliable" status of
wikipedia articles and may in time lead to wikipedia being viewed as being a
more trusted source of information (if only for the handful of reviewed
articles).

> If I am wrong, and the site does get a lot of traffic, well then we should
> spend the time and develop a proper management and quality control system.
> But until we have proved its viability, we would just be wasting effort -
> effort better spent on creating/editing articles.

I agree in part -- if we decide to do this, then we need to give it a lot of
thought and not rush into things (best to do it right than to do it fast).
But because this idea has such great potential, I think it is
something we should investigate and give a chance. This is something that
won't really get a lot of traffic at first - there needs to be a critical
mass of reviewed articles before traffic to them in general increases
significantly (even a hundred reviewed articles would be totally lost in the
current database).

If done correctly this could finally begin to make wikipedia a critically
respected source of human knowledge that people can depend on and come back
to over and over again.

-- maveric149


subject: [Wikipedia-l] Comments on Axel's beta/stable ideas

body:
On Thursday 23 May 2002 12:01 pm, Axel wrote:
> That said, here's my minimalistic suggestion: everything works exactly
> as it does now, except that every page gets two additional links:
> "View last reviewed version of this article" and "I have reviewed this
> version of the article and I think it is ok". The history of every
> article would record who has reviewed which version of the article and
> when.
>
> The set of all "last reviewed versions" could then be seen as the
> "stable" Wikipedia and could be pressed on CD. This would at least
> guard against vandalism, stupid jokes and blatant propaganda and
> advertising that sometimes gets through.
>

Cool! What a fantastic idea!. I also like the idea of having two versions of
an article; but I would simply call them Reviewed and Development. The
reviewed one would be a static page that could be replaced with the
Development (editable) version of an article whenever a certain number of
reviewers give it the OK. It also would be most excellent to have an even
higher level of review called "Peer Review" that would be performed by
somebody with a related college degree.

This level of review would be roughly analogous to what Nupedia had (has?)
set-up but should done in a better way. But then the devil is in the details
here and we may need to temporarily freeze an article while it is being peer
reviewed so the expert can fix any glitches, submit the article as peer
reviewed and reopen the development version back to the masses (the
reviewer(s) should only have a very limited amount of time to review/fix the
article while the development version is locked). This might be a messy thing
to do in practice though and we should discuss this at length in order to
work out the details if we decide to do this at all.

So potentially we might have three versions of an article; one which is world
editable, one which was voted to be OK by users, and one that was tweaked and
made to conform to higher Nupedia-like standards.

> The only issue is: who is allowed to review articles? The pragmatic
> answer would be: all sysops.

Somewhat disagree: I tend to agree with Chuck's previous comment here in that
I think we should open this up to anybody who has been a user for over 3
months. I would add though that exceptions should be made for anybody who has
edited a reasonable set number of articles and who has the backing of at
least one sysop. We could update the software to give these user's special
additional rights automatically after three months and/or after they
edit more than a set number of articles (thus making them then eligible for
sysop promotion). We could then call these users "trusted hands"  (which is
already in the wikiware code -- but a "trusted hand" doesn't have any more
rights than a regular user does in the current set up -- correct me if I am
wrong Brion).

It would also be nice for a sysop to have the ability to "reset the clock" as
Chuck proposed for anybody who doesn't follow established policy after being
warned. The warning process could be started by a posting a message on a
special page by at least one person with "trusted hand" or greater
status. These warnings shouldn't do anything other than inform the user that
"they have been warned" with a reason why (there could be an automatic
temporary suspension of "trusted hand" status by being warned -- but I'm not
sure if I like that idea). These warnings could also automatically expire
after a set amount of time and/or number of edits unless somebody else renews
the warning. In addition, a "trusted hand" or sysop should be able to remove
the warnings if a mistake had been made. If the user doesn't get his or her
act together during the warning period, a sysop could then reset the user's
clock (which should have to be a different person than the one
starting the warning process).

Blatant trolling or VANDALISM could also be logged on these warning pages.
The warner could choose from a checklist of options when warning the warnee.
Such a list might include these options: Introduced bias, Rambling
contributions, Falsification of facts, Nonsensical/inappropriate
contributions, Introduction of propaganda, Sloppiness, Violation of naming
conventions or VANDALISM. If VANDALISM is selected then in addition to the
warning page a VANDALISM in PROGRESS page can also be displayed on
RecentChanges with the "contributions of" the warnee automatically placed on
that page.

There should, however, be some safeguards to help ensure that honest mistakes
by contributors are not used as the basis to warn a person (perhaps have a
silent first warning that only the warner and warnee know of... but then how
would you make that work?) . I'm concerned that such a system might be used
too much and contributors might think twice about hitting the save button. If
misused, such a system could have negative effects on the number and extent
of contributions wikipedia receives. So, if something like this is deemed
necessary, then great care must be taken to minimize possible negative
side-effects. But something similar to this will probably be needed in the
coming years as the number of active contributors becomes larger than anyone
(including users like me who contribute at least 3-4 hours a day on average)
is able to keep track of.

> A code of honor is probably in order,
> saying that no sysop should review an article that they themselves
> substantially contributed to.

In principle this might be a good idea but in practice I don't think this
would work with the size of our current contributor-base (not to mention
sysop-base).  As it is right now in many cases the people with the greatest
knowledge and ability to review an article have already significantly
contributed to it.

The "honor system" set-up may work after a few more years when wikipedia has
literally thousands of people contributing daily and has close to 100,000
articles (which is an arbitrary goal BTW -- I doubt we will all stop
submitting new articles when the project is "complete"). Then we could
reasonably expect that there would be enough "trusted hands" with sufficient
knowledge of the subject and who had not already contributed much to
the article to be able to vote it up or down.

Given all that -- I do have reservations that the power to warn and reset the
clocks of users may be abused and this could change the character of
wikipedia and discourage contributions. However, given that IP banning hasn't
<yet> been abused by any sysops I have hope that such a system might just
work.

If we could somehow ensure that we have reasonably competent reviewers then
Wikipedia could then (eventually) become what Nupedia was not able to be: an
extensive, trusted and useful source of human knowledge. We are almost at the
extensive stage -- Should we work on a framework for establishing reasonable
trust and therefore usefullness of our articles? I vote for yes.

Just my 1.5 cents

maveric149


from_name: Brion L. VIBBER

subject: [Wikipedia-l] Locking articles in a content war?

body:
As far as I know, the ability to protect pages from edits by non-sysop
users was added to protect against frequent vandalism of the main page.
Is it appropriate to be using this ability in a content war, for example
[[Reciprocal System of Theory]]?

-- brion vibber (brion @ pobox.com)


body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020525/e6a7676c/attachment.diff


in_reply_to: <1022348534.5821.2.camel@rdaneel.moisty.org>; from brion@pobox.com on Sat, May 25, 2002 at 10:42:13AM -0700

body:
Brion L. VIBBER wrote:
> As far as I know, the ability to protect pages from edits by non-sysop
> users was added to protect against frequent vandalism of the main page.
> Is it appropriate to be using this ability in a content war, for example
> [[Reciprocal System of Theory]]?

No!  That's very unwiki.  Very Very un-Wiki.

I just now unprotected it.

I looks like maveric149 protected the page.  I'm tempted to remove
your sysop status, mav.  But I don't want this to be a big fight or
anything.  Let's just not do this.

The only reason we have sysop status is (a) to gently counter some
kinds of vandalism and (b) allow users who 'know the ropes' to do some
technical things that are a bit too dangerous to leave open to
everyone.

Protecting pages over the long haul, like the front page for example,
should be based on the particular high profile of that particular
page, and verified instances of abuse.  (Someone was putting penis
pictures on the home page, ha ha, not so funny, so we had to lock it
away from anyone who isn't a known face.)

Protecting pages in a crisis would be fine, like if some jerk is going
through seriously vandalizing the site.  Banning would be o.k. at that
time, too.

But the single central principle that we have to adhere to, as sysops,
is that our "sysop jobs" are totally separate from our "participation
jobs".  This is the heart and soul of the wiki, the willingness to
meet on a level playing field and try hard, HARD, to find a way to
peacefully present the information in a way that's satisfactory to
everyone.

The biggest temptation will be in cases like this -- belligerent defense by
an unknown person of what appears to be kooky ideas.  I haven't actually
read the disputed material in detail yet, but I get the gist of what's going
on.

I'm sympathetic, but we have a code of honor that we need to follow.

--Jimbo


subject: [Wikipedia-l] Reciprocal system...

in_reply_to: <200205252100.g4PL0Fe11825@ross.bomis.com>; from lcrocker@nupedia.com on Sat, May 25, 2002 at 02:00:15PM -0700

body:
lcrocker at nupedia.com wrote:
> There's no real conflict; there's one anonymous crackpot with a
> pet theory, and the rest of the reasonable community.  The crackpot
> doesn't even /try/ to follow our guidelines here--he simply wants
> to make that article an advertisement for his pseudoscientific
> bullshit, and when we try to make a useful article out of it, he
> just restores his crap.  I don't think we should "lock" the article,
> but only because that prevents other reasonable people from editing
> it.  I /do/ think we should ban the crackpot if he doesn't give up.

I think that's right.  I think that a case could be made -- but we'd
need to get some consensus first -- that locking the article to
prevent him from overwriting it would be justified if we already had
banned him, and if he was evading the ban.  Removing the object of his
affection might help in a case like that.

> In general, though, I think we should not make hard-and-fast rules
> about use of sysop features.  We should use judgment, and do whatever
> is good for the project.  A content war between reasonable people
> who disagree is one thing, and we should treat such a conflict with
> respect.  But a crackpot spreading total nonsense is another, and we
> should use whatever tools we have to defend against his damage.

I agree with this, subject to a caveat that we should take the sysop
code of honor seriously, and try hard not to have it come to this.

The banning of 24 was done only after much soul searching.  It seems to have
worked.

--Jimbo


from_name: lcrocker at nupedia.com

subject: [Wikipedia-l] Reciprocal system...

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020525/b6c784e9/attachment.diff


subject: [Wikipedia-l] Re: Locking articles in a content war?

body:
On Saturday 25 May 2002 12:01 pm, Brion wrote:
> As far as I know, the ability to protect pages from edits by non-sysop
> users was added to protect against frequent vandalism of the main page.
> Is it appropriate to be using this ability in a content war, for example
> [[Reciprocal System of Theory]]?

As a general rule, most definitely yes. But in this case the eidt war had
degenerated in outright replacement with one totally different version with
another, then back again, then another revert, etc. This amounts to a type
vandalism and is counterproductive for all involved (although since this
"vandalism" is only being wrought onto a single article, it makes FAR more
sense to protect the page than even begining to consider banning an IP).

The article can just as easily be unprotected as soon as things settle down
(the period of protection was never meant to last more than a few days --
just enough time for the partisans to talk about what to do next). If a sysop
ever has to do this, then a reason why it had to be done should always be
placed in the talk for that the page in question. As was done.

If that sysop makes a mistake in judgement, then another syop can just as
easily reverse the protection. BUT the other sysops need to be informed that
such action is being taken. Perhaps it will be better to first bring this to
the mailing list instead of just placing a note on talk page....

Just read the talk and especially the old talk for [[Reciprocal System of
Theory]]: One inherently POV version was being advocated by a single person
named Doug and another NPOV version had been worked and agreed upon by a half
dozen others. After a previous version war in January Doug returns and places
his POV version in the place of the agreed upon NPOV version of January. Then
the version war started up again with one version being replaced by another
in kind.

Some discussion should take place as to when this ability should be used. If
it is only appropriate to use the protection function of the software on the
main page and maybe special and possibly on some wikipedia-specific pages,
then this function should be disabled by default to all but those pages.
Please reverse the protection of [[Reciprocal System of Theory]] if this
action was not appropriate.

maveric149


subject: [Wikitech-l] Database dumps

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020525/0cf78f72/attachment.ksh>


subject: [Wikipedia-l] Page cache still not fully updated?

body:
I'm still noticing some pages that appear different when viewed and when
edited. Also, while looking through stub articles one entry was:

116(43 chars) Pierre and Marie Curie

But when I looked at the article in question at
http://www.wikipedia.com/wiki/Pierre_and_Marie_Curie there were clearly
more than 43 characters in it.

Has the static page cache not been completely updated yet since being
re-enabled?

--
"Let there be light." - Last words of Bomb #20, "Dark Star"


subject: [Wikipedia-l] Re: Reciprocal system...

body:
On Sunday 26 May 2002 12:01 pm, : lcrocker wrote:
> There's no real conflict; there's one anonymous crackpot with a
> pet theory, and the rest of the reasonable community. ?The crackpot
> doesn't even /try/ to follow our guidelines here--he simply wants
> to make that article an advertisement for his pseudoscientific
> bullshit, and when we try to make a useful article out of it, he
> just restores his crap. ?I don't think we should "lock" the article,
> but only because that prevents other reasonable people from editing
> it. ?I /do/ think we should ban the crackpot if he doesn't give up.
>
Hum, I don't know about banning his IP; that seems to be a drastic measure
given that the "crackpot's" focus is on a single page. However, I think we
should at least try to make his version closer to NPOV by editing it and
placing in appropriate qualifiers. Such as; "Larson says X, however working
scientists in the field say Y."  The protection on the page has been lifted
and Doug has already been warned <not> to simply replace the current version
with his own. We should consider a ban <If> and <only> if, he reverts the
article again. Otherwise I think we should at least try to make his version
more NPOV.

> In general, though, I think we should not make hard-and-fast rules
> about use of sysop features. ?We should use judgment, and do whatever
> is good for the project. ?A content war between reasonable people
> who disagree is one thing, and we should treat such a conflict with
> respect. ?But a crackpot spreading total nonsense is another, and we
> should use whatever tools we have to defend against his damage.

Well since I was the one who protected the page, I guess I have to agree with
you Lee. ;) Actually, I totally agree with you - within the context of a
general framework, we should be able to exercise case-by-case judgement as to
when we use our sysop powers to do what we feel is the best <for the good of
the project.>.

With that said, I do think that a slightly different protocol for protecting
pages needs to be devised (otherwise the question would not have been posed
in the first place). Should we bother the list with requests for protecting
particular pages? If so, then it would also make sense to bother the list to
ban IPs since I feel this is even a more drastic measure than <temporarily>
protecting a page. However, a lot of damage can be done during the delay....
This may not be as much of an issue with protecting a single page though
(since it can just be restored later). On the other hand, others may make
valid edits to the POV version inputed by a troll which causes versioning
issues that would have to be worked-out later. My view, is that the sysop
doing the protecting should list the reasons why on the talk for the page and
the lock should be limited in duration (unless it is on the main or one of
the policy pages). It would also be nice to have a comment field on the
protection page and and an easy to find log of 'who has protected what' and
why.

In this particular case, I looked over the edit history of the page, and read
both the talk and old talk. After that, it was my personal <judgement> that
Doug's actions bordered on vandalism since he was replacing an agreed upon
version with his own biased one. But I did not feel that it would at all have
been appropriate to consider banning his IP <because> his actions were only
directed at a single page. This may have been an error in judgement on my
part since this had the effect of not allowing <anybody> but sysops to edit
the page (it would be nice to have the ability to limit certain classes of
contributors from editing a protected page - <only> blocking non-logged-in
users would have worked in this case). All I wanted to do was to force a
truce in order to stop the version war <so that> a compromise could be
reached.

I could have <easily> protected the page and not tell anybody about it.

PS PLEASE, PLEASE, PLEASE lets limit this discussion so that we can discuss
the FAR more interesting ideas on beta/stable. I fear that this will become a
red herring and trump that discussion...

maveric149


subject: [Wikipedia-l] Re: Jimbo's comment

body:
On Sunday 26 May 2002 12:01 pm, Jimbo wrote:
> I looks like maveric149 protected the page. ?I'm tempted to remove
> your sysop status, mav. ?

I have emailed Jimbo directly concerning this statement.

maveric149


subject: [Wikipedia-l] Doug's most recent response on [[talk:Reciprocal System of Theory]] --> encouraging start

body:
Below is a copy of the most recent talk on [[talk:Reciprocal System of
Theory]]:

Responding to some positive posts, maveric149 wrote:

Fantastic! I have been waiting for somebody else to chime in. Now maybe ?we
can continue the discussion so that the longer version can be tweaked some
more so that the people advocating the shorter version will be satisfied or
at the very least be able to live with a modified longer version. I would
like to hear from them -- especially since they have more complete knowledge
of the subject matter and arguments against it (which will be needed to make
the longer version closer to NPOV). I would like to second 130.94.121.26's
(is that you Jimbo?) statement about not simply replacing the current version
with the longer one until a compromise is reached. Perhaps if those with
knowledge of why this theory is pseudoscience heavily edited the longer
version, then it might be acceptable. I wish I had the appropriate knowledge
to edit the longer version myself -- but I don't. My goal all along with
protecting the page was to put, at least, a temporary stop to the outright
replacement of one version with another (a forced truce, if you will). Maybe
this can now be done. Although we might still decide that an external link to
a more complete treatment is more appropriate (which still tends to be my
vote -- especially if nobody has the time or energy to make the longer
version NPOV). Hopefully the authors of the shorter version can help us
decide what to do. --maveric149

The proponent of the longer version, Doug, responds:

I'm not sure I know quite what to say. I'm pleasantly surprised, certainly
pleased, and even somewhat moved by the show of reason and understanding
evident in these comments. I think what you chose to do maveric was the right
choice. I'm grateful to see the end of the brutal deletes, and the
expressions of a real desire to get to a genuine NPOV are certainly
encouraging. I'm ready to help all I can. I have reread the Wikipedia policy
and articles on etiquette and NPOV writing. I know I need help on writing so
as to restate the various views while not asserting the one I happen to agree
with, I've found that it's not as easy to do as one might suppose.

Doug

- end -

We should probably now continue the specific issue of what to do with this
particular article on that talk page. However, some framework on what to do
<in general> in these cases should be discussed here (again in a limited
fashion so that we can get on with more interesting matters).

-- maveric149