list: foundation-l

subject: [Foundation-l] Re: Wikimedia Foundation Website

body:
Jimmy (Jimbo) Wales a ?crit:
> Anthere wrote:
>
>>But for an american foundation, it makes sense to give priority to
>>english language.
>
>
> I don't want to disagree with Anthere, but I did want to add that I
> always think of the foundation as being international and located in
> the U.S. only due to me happening to live here.  I think that however
> we do approach the homepage issue (I have no strong opinion), we want
> to make sure not to be any more English-centric than absolutely
> necessary for whatever reasons.
>
> --Jimbo

Either way, I'll settle to what people are more confortable with :-)

In short, the first one to draft a gorgious home page for the website
will have my agreement :-)


list: wikimedia-l

source: pipermail-mbox

from_name: Anthere


subject: [Foundation-l] Re: Announcement mailing list

body:
one benefit it would have would be to be a central place where to find
this information. Right now, very few wikis think of updating this kind
of information in meta itself. I know not really how to foster this.

There are some wikis which are no more small wikis really, and we
extremely rarely hear of them.
How many people here are participating to the spanish wikipedia ? From
those not participating, how many here have any idea what is going on
there right now ? Do you have specific problems ? Are you having fun ?

Angela_ a ?crit:
> Sj wrote:
>
>>I would like to request once more an announcement mailing list
>
>
> What function would an announcement list serve that the Goings-on page
> does not already? It is a lot easier to make translations on the wiki
> than it would be on an announcements list. On Goings-on, anyone can
> read it without needing to subscribe, and translations can be linked
> to. How would that work for a mailing list and what benefits would
> that have over the wiki?
>
> Angela.


list: wikitech-l

from_name: Christian Semrau

subject: [Wikitech-l] Double database entries

body:
Hello,

I created lists of cur table entries which have the same namespace and title.
They were probably created by clicking "Save page" more than once on a newly
created article.

These lists lead to bogus entries in maintenance lists, e.g. orphaned articles
or short pages.

I uploaded three lists:
* http://en.wikipedia.org/wiki/User:SirJective/Double_entries
* http://de.wikipedia.org/wiki/Benutzer:SirJective/Doppeleintr%C3%A4ge
* http://nl.wikipedia.org/wiki/Gebruiker:SirJective/Double_entries

Every language I downloaded so far (hu, fr, ja) has double entries.

I used a query like this on a local dump:

 SELECT DISTINCT
  c1.cur_namespace, c1.cur_title, c1.cur_id, c1.cur_timestamp
 FROM
  cur c1,
  cur c2 USE INDEX (name_title_timestamp)
 WHERE
  (c1.cur_namespace = c2.cur_namespace
  AND c1.cur_title = c2.cur_title)
  AND c1.cur_id <> c2.cur_id
 ORDER BY c1.cur_namespace, c1.cur_title, c1.cur_timestamp
 LIMIT 1000;

At de-WP, Echoray and Peterlustig found that these entries can be removed by
deleting and undeleting them.
Are there any objections to handling these entries this way? If not: Is some
sysop willing to go through the lists and del/undel the entries?

Christian aka SirJective


from_name: Ævar Arnfjörð Bjarmason

subject: [Wikitech-l] Bugzilla bug report migration

body:
The former address does not work for me but the latter does.
"MediaZilla Main Page"

On Fri, 13 Aug 2004 19:18:27 -0700, Brion Vibber <brion at pobox.com> wrote:
> The bugzilla.wikimedia.org (with M) address still seems to be
> problematic for some people, but bugzilla.wikipedia.org seems to be
> pretty well working by now.
>
> Timwi's started on migrating still-open bug reports from the sourceforge
> tracker to Bugzilla, which will bring us these amazing benefits:
>
> * Easier to create a login than it is on Sourceforge
> * Easier to search for existing bugs
> * Less cluttered interface!
> * Bugmail notifications are less cluttered; it's easier to tell what's
> changed
> * URLs typed in comments are clickable
> * Wikilinks typed in comments are clickable (to Wikipedia wikis at least)
> * Easy links from Wikipedia pages to bug reports like this: [[bugzilla:13]]
>
> At some point we need to look at the Feature Requests too; there are a
> lot of duplicates and weird things in there...
>
> Note -- if you take assignment of a bug, be sure to add wikibugs-l to
> the CC list so notifications will still go to the mailing list and IRC
> bot. We'll try to make that automatic in the future.
>
> -- brion vibber (brion @ pobox.com)
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>
>
>


subject: [Wikitech-l] developer poll

body:
Seeing that I appear to be around, I have been asked about results from
the http://meta.wikimedia.org/wiki/Developer_payment_poll

It appears I am here, but I am not really :-)
I was here just for the night, and went back in the car for the ocean
about 2 hours ago (well, that was my schedule anyway :-))

I'll be back from holidays on the 22nd. Offline till then.

I will publish aggregated results (anonymised when necessary) in the
next few days, and *hope* that we can have some comments by all editors
on them.

A few people have not answered yet, but usually gave their opinion
offline. They still have a week to give more information.

I think the board will try to plan a meeting that last week of august or
very soon after (in hope that Angela will not decide to go on holidays
in turn), so we can take together a decision with regards to using money
or not on development issues, and if so, how and for which tasks.

In all cases, I would like to thank all developers for their work in the
past years.

I have been on Wikipedia for 2 years and a half now, and I have some
memories of my first months . My first memories were Brion (somewhere in
april or may 2002) and Magnus (and others as well, some are now gone).
I remember phase I on en, then the switch to phase II and the helpful
new features. I remember translating the interface to switch fr for
phase II (it was never used), then the switch for phase III and little
by little switch for the international wikipedias, where Brion helped so
much. I  remember when Erik Zachte set all the statistical pages. Most
helpful for press releases. And new people join and bring their help,
some appear now to have always been here. Some features which seem so
standard now, did not exist at all before :-) Perhaps do we tend to
forget it :-)

I think that for most people, and except for a few developers, what
developers do is quite mysterious. Some tasks are visible, while others
are invisible. It was very interesting for me to come to understand more
on what each of you have been focusing on, and I think this should be
more visible to anyone. I hope we can do this. Most contributions of
editors are displayed on their user pages, visible through links or in
article history. This is not such for you.

I think how you contribute should be listed somewhere (it will also help
communication between you and editors or newbies developers) and visible
in release notes.

Anyway, thank you to all of you for your work and for your answers.

Ant

PS : one day, could we set the article counter again ? :-)

Ant goes jump in the ocean


from_name: Michael Becker

subject: [Wikitech-l] An alternate parser

body:
I can help with the CFG (Context Free Grammar) too. I don't have as much experience as you, I've only taken compiler course, not taught them, but I definitely could help out. I think that having a CFG for out wiki markup is invaluable because it will give us the sort of flexibility that we don't have now. That, and the parsers that lexx/yakk give you are going to be much quicker, and more efficient than what we have now. I also think we should leave it in C, not php, since it will be more efficient, and it sounds like we could use all the efficiency that we can get.

On Sat, 14 Aug 2004 00:42:10 +0200, Jan Hidders <jan.hidders at pandora.be> wrote:
>
>
> On Friday 13 August 2004 20:59, Brion Vibber wrote:
> > Magnus Manske wrote:
> > > I therefore suggest a new structure:
> > > 1. Preprocessor
> > > 2. Wiki markup to XML
> > > 3. XML to (X)HTML
> >
> > This doesn't actually solve any of the issues with the current parser,
> > since it merely has it produce a different output format.
> >
> > The main problems are that we have a mess of regexps that stomp on each
> > other all the time.
>
> Are you kidding? That is exactly what it would solve! If you would let the
> preprocessor be generated with a lex/yacc type of tool then you would for the
> first time have a decent formal documentation of the wiki-syntax in the form
> of a context-free grammar. That not only would give you a better idea of what
> the wiki-syntax exactly is and tell you exactly whether any new mark-up
> interferes with old mark-up, but you could also more easily add
> context-sensitive rules (like replacing 2 dashes with — but only in
> normal text). Moreover it would give you the power to make small changes to
> the mark-up language because you could easily generate a parser that
> translates all old texts to the new mark-up. Finally, having an explicit
> grammar also makes it more easy to make sure that you actually generate
> well-formed and valid XHTML, or anything else that you would like to generate
> from it and that needs somehow to satisfy a certain syntax.
>
> It's simply a briliant idea, and frankly I think it is in the long run as
> unavoidable as the step to a database-backend. If there is performance
> problem you could even consider storing the XML in the database so you only
> need do the raw parse at write time and the xml parse at read time.
>
> That hard part is of course to come up with the contex-free grammar (it should
> probably be LALR(1) at that). Since I used to teach compiler theory I might
> be of some help there.
>
> -- Jan Hidders
>
> PS. You could even get rid of the OCaml code since the Latex parsing could be
> integrated in the general parser.
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>

--
Michael Becker


from_name: Krzysztof Kowalczyk

subject: [Wikitech-l] An alternate parser

body:
> The one I'm currently working on is "manual" C++, but I see no reason
> not to use an parser-generator. Question: Can a parser-generator ensure
> the output (no matter the input) is valid XML? Can it remove potential
> harmful HTML tags?

Looks like you mis-understood what a parser generator does.

Based on a formal grammar, parser generator (like yacc or bison)
generates parser for this grammar. It does nothing else. It's up to
you to write code/actions for grammar parts (in this outputting XML
text).

So the answer is: no.

Krzysztof Kowalczyk | http://blog.kowalczyk.info


subject: [Wikitech-l] [X] Yes i want to delete this,

body:
Could the [X] yes i want to delete this button one has to click to
delete pages/images be abolished? I dont think  anyone would by
accedent click 2 links in a row (delete -> delete) so i dont think the
checkbox is needed and just slows me and others down when doing mass
deletion.


subject: [Wikitech-l] An alternate parser

body:
On Sat, 14 Aug 2004 11:38:13 -0400, Michael Becker
<mbecker at jumpingjackweb.com> wrote:
> I also think we should leave it in C, not php, since it will be more efficient, and it sounds like we could use all the efficiency that we can get.

Doing stuff it C is actually a problem for many people who would like
to run WikiMedia code as their personal wiki but don't have the
ability to install C PHP extensions. That is a very common scenario.
Those people (e.g. me) would no longer be able to use WikiMedia.

I know that WikiMedia's main purpose is to serve the needs of
WikiPedia but I would like the impact on non-WikiPedia users to be
considered when making a decision.

One solution would be to provide both slow, fall-back PHP parser and
fast C parser.

Krzysztof Kowalczyk | http://blog.kowalczyk.info


from_name: Magnus Manske

subject: [Wikitech-l] An alternate parser

body:
Jan Hidders wrote:

>On Saturday 14 August 2004 00:49, Brion Vibber wrote:
>
>
>>Jan Hidders wrote:
>>[snip]
>>
>>
>>
>>>That hard part is of course to come up with the contex-free grammar (it
>>>should probably be LALR(1) at that). Since I used to teach compiler
>>>theory I might be of some help there.
>>>
>>>
>>Yes, and that's the *only* part that will help. Having two or three or
>>five intermediate formats doesn't do anything to help the problem --
>>making the actual parser actually work token by token will.
>>
>>IMHO putting a lot of emphasis on output formats is a mistake, since it
>>ignores the actual problem.
>>
>>
>
>FWIW I completely agree with that. Having a real parser in C/C++ generated by
>a parser-generator is IMHO the core of the idea, and if you need to interface
>that somehow with the rest of the PHP code then XML is a good solution. I had
>the impression that this is what Magnus meant anyway, but I may have been
>wishful reading there.
>
>
No, you got that right :-)

If we'd limit ourselves to PHP, we could avoid XML, but PHP is not very
well suited for this kind of text parsing, even with regexps, especially
when it comes to performance. Also, using XML as an itermediate at one
point opens potential for exchange with other services, either for
output (generating static dumps, statistic analyses, etc.) or input
(e.g., using other wiki markup in MediaWiki, for other projects).

The one I'm currently working on is "manual" C++, but I see no reason
not to use an parser-generator. Question: Can a parser-generator ensure
the output (no matter the input) is valid XML? Can it remove potential
harmful HTML tags? If not, is there a tool (tidyhtml comes to mind) that
can? The chain would then be preprocessing-parsing-XML-XHTML.

The XML could be cached, as all changes influenced by user options would
happen only in the final step. Caveat: Cache will have to be invalidated
for variables and templates that change (e.g., {{NUMBEROFARTICLES}} and
edited templates).

Once the #2 parser is basically working, we (I) can adapt a copy of the
parser class and run some benchmarks against the current parser. That
should give us some more facts to base a decision on.

Magnus


from_name: Ashar Voultoiz

subject: [Wikitech-l] Re: [X] Yes i want to delete this,

body:
?var Arnfj?r? Bjarmason wrote:
> Could the [X] yes i want to delete this button one has to click to
> delete pages/images be abolished? I dont think  anyone would by
> accedent click 2 links in a row (delete -> delete) so i dont think the
> checkbox is needed and just slows me and others down when doing mass
> deletion.

Hello,

At least it forces users to review their choices and to enter a comment.
  I am personally against removing this checkbox.

--
Ashar Voultoiz


from_name: evan at wikitravel.org

subject: [Wikitech-l] An alternate parser

body:
On Fri, Aug 13, 2004 at 08:46:20PM +0200, Magnus Manske wrote:

> * The XML-HTML-converter (#3) can focus on the pure context of the page:
> make a normal link, stub link, thumbnailed image, or a category or
> interwiki link, etc.

It'd probably be sufficient to do an XSL style sheet. You could even send
XML+XSL to the client for clients that support it, and do server-side XSL
for clients that don't.

~ESP


body:
It's not users but sysops and i think if they have that privelage they
have a pretty good clue of what to delete and what not to delete.

On Sat, 14 Aug 2004 12:10:27 +0200, Ashar Voultoiz <thoane at altern.org> wrote:
>
>
> ?var Arnfj?r? Bjarmason wrote:
> > Could the [X] yes i want to delete this button one has to click to
> > delete pages/images be abolished? I dont think  anyone would by
> > accedent click 2 links in a row (delete -> delete) so i dont think the
> > checkbox is needed and just slows me and others down when doing mass
> > deletion.
>
> Hello,
>
> At least it forces users to review their choices and to enter a comment.
>   I am personally against removing this checkbox.
>
> --
> Ashar Voultoiz
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>


from_name: Brion Vibber

subject: [Wikitech-l] An alternate parser

body:
evan at wikitravel.org wrote:
> On Fri, Aug 13, 2004 at 08:46:20PM +0200, Magnus Manske wrote:
>>* The XML-HTML-converter (#3) can focus on the pure context of the page:
>>make a normal link, stub link, thumbnailed image, or a category or
>>interwiki link, etc.
>
> It'd probably be sufficient to do an XSL style sheet. You could even send
> XML+XSL to the client for clients that support it, and do server-side XSL
> for clients that don't.

That unfortunately introduces another dependency. PHP has an optional,
non-default XSLT module which requires an external library, or you can
run through an external program as a filter (requires that one be
obtained and be runnable, etc).

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040814/425f3658/attachment.pgp>


body:
Furthermore i'm pretty much against this "could be used in a bad way"
mentality which often ends up in some "feature" which in effect makes
things take a lot longer time for the people who actually volunteer to
carry out these actions - like deletion.

On Sat, 14 Aug 2004 12:24:18 +0200, ?var Arnfj?r? Bjarmason
<avarab at gmail.com> wrote:
> It's not users but sysops and i think if they have that privelage they
> have a pretty good clue of what to delete and what not to delete.
>
>
>
> On Sat, 14 Aug 2004 12:10:27 +0200, Ashar Voultoiz <thoane at altern.org> wrote:
> >
> >
> > ?var Arnfj?r? Bjarmason wrote:
> > > Could the [X] yes i want to delete this button one has to click to
> > > delete pages/images be abolished? I dont think  anyone would by
> > > accedent click 2 links in a row (delete -> delete) so i dont think the
> > > checkbox is needed and just slows me and others down when doing mass
> > > deletion.
> >
> > Hello,
> >
> > At least it forces users to review their choices and to enter a comment.
> >   I am personally against removing this checkbox.
> >
> > --
> > Ashar Voultoiz
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > Wikitech-l at wikimedia.org
> > http://mail.wikipedia.org/mailman/listinfo/wikitech-l
> >
>


list: wikipedia-l

from_name: James R. Johnson

subject: [Wikipedia-l] Re: FW: Wikipedia for Old English

body:
Hello all,
	I've begun translations to Anglo-Saxon for the main page, and the
Edit the Main Page, for the hopeful Anglo-Saxon Wiki.  How should I go about
putting those up?

James


subject: [Wikitech-l] PHP character-by-character performance

body:
Quick hacky benchmark:
http://meta.wikimedia.org/wiki/PHP_string_looping

Executive summary: using the append operator ($result .= "*") is
relatively speedy. Copying strings ($result = $result . "*") is really
reaaaaally slow.

'Relatively speedy' is tempered by the general sluggishness of PHP;
per-iteration loop overhead is itself larger than an append operation.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040814/3fd0487a/attachment.pgp>


from_name: Jens Frank

subject: [Wikitech-l] Re: An alternate parser

body:
On Fri, Aug 13, 2004 at 09:38:34PM +0200, Ashar Voultoiz wrote:
> Brion Vibber wrote:
> >Magnus Manske wrote:
> >
> >>I therefore suggest a new structure:
> >>1. Preprocessor
> >>2. Wiki markup to XML
> >>3. XML to (X)HTML
> >
> >
> >This doesn't actually solve any of the issues with the current parser,
> >since it merely has it produce a different output format.
> >
> >The main problems are that we have a mess of regexps that stomp on each
> >other all the time.
> >
> >-- brion vibber (brion @ pobox.com)
>
> Can't we switch back to the tokenizer parser and try to optimize it ?
> The token approch Seems much easier to maintain.

Character-by-character string parsing in PHP is slow since there is too
much overhead. Tokenizing probably has to be done in a C(++) function.

Another point where the tokenizer was slow was the byte-by-byte composing
of the result string. I've been told that adding small strings to an array
and joining them in the end is much faster, worth a try.

JeLuF


from_name: Yann Forget

subject: [Wikipedia-l] Distributing fonts

body:
Hi,

Le Friday 13 August 2004 21:16, Magnus Manske a ?crit :
> Gerard.Meijssen wrote:
> >I have been amazed at the passions that were stirred up when I proposed
> > that we distribute free fonts.
>
> I was amazed as well. I didn't bother to look through the Credits page,
> but if the fonts are free as in GPL/GFDL, did anyone of the bickering
> crowd actually consider building TTFs from the files, upload these at
> wikipedia, and offering them for convenient download there, with proper
> credits?
>
> We could also upload a free OGG player or the like. [[Wikipedia:Software
> resources]] or something.
>
> Magnus

I was not asked to give my opinion, still here it is. ;o)

I think it's a nice idea to provide tools necessary to read and listen to
Wikipedia (free fonts and an OGG player). It doesn't harm anyone but it could
help many.

Yann

--
http://www.non-violence.org/ | Site collaboratif sur la non-violence
http://www.forget-me.net/ | Alternatives sur le Net
http://fr.wikipedia.org/ | Encyclop?die libre
http://www.forget-me.net/pro/ | Formations et services Linux


subject: [Wikitech-l] Images used in templates are shown as orphan

body:
Hi,

I added this to the new Bugzilla.
Bug #48
http://bugzilla.wikipedia.org/show_bug.cgi?id=48

Yann

Le Friday 13 August 2004 13:37, Yann Forget a ?crit :
> Hi,
>
> I posted this last Sunday, but no answer.
> Is this bug already known?
> It Bugzilla ready for use so I can add it?
>
> Thanks,
> Yann
> ==============
> Images used in templates are shown as orphan on fr:
> Is this bug already known?
>
> cf.
> 1. http://fr.wikipedia.org/wiki/Image:Shiva.jpg
> used in http://fr.wikipedia.org/wiki/Shiva
> 2. http://fr.wikipedia.org/wiki/Image:Narmada.jpg
> used in http://fr.wikipedia.org/wiki/Narmad?
>
> http://fr.wikipedia.org/w/wiki.phtml?title=Special:Unusedimages&limit=100&o
>ffset=1800 Image #1872
>
> This is the case for all images included in templates that I can see.
>
> Yann

--
http://www.non-violence.org/ | Site collaboratif sur la non-violence
http://www.forget-me.net/ | Alternatives sur le Net
http://fr.wikipedia.org/ | Encyclop?die libre
http://www.forget-me.net/pro/ | Formations et services Linux


subject: [Wikitech-l] Re: An alternate parser

body:
On Sat, Aug 14, 2004 at 09:01:04PM +0100, Timwi wrote:

> >It'd probably be sufficient to do an XSL style sheet. You could even send
> >XML+XSL to the client for clients that support it, and do server-side XSL
> >for clients that don't.
>
> I don't know much about XSL(T), but personally I highly doubt that it
> can give us the flexibility we need.

Do you make a habit of forming strong opinions based on near-total ignorance?

~ESP


subject: [Wikitech-l] Re: PHP character-by-character performance

body:
Ashar Voultoiz wrote:
> Can you benchmark the following code against the two aboves ?
>
>   while (something) {
>     $result[$i] = '*';
>     $i++;
>   }
>   $newresult = implode('', $result);

Where 'something' is '$i < TEST_SIZE' this performs about the same as
the for loop version.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040814/edef73ef/attachment.pgp>


subject: [Wikitech-l] Re: An alternate parser

body:
Jens Frank wrote:

> <>Another point where the tokenizer was slow was the byte-by-byte
> composing
> of the result string. I've been told that adding small strings to an array
> and joining them in the end is much faster, worth a try.

It is slow because for each "add", memory has to be re-allocated. In
C++/STL, one can reserve memory in advance for the string. Can that be
done in PHP as well? It would speed up things a lot with a single extra
line.

Magnus


from_name: Zinnmann

subject: [Wikipedia-l] Distributing fonts

body:
>>I don't think, we have to mirror every single GPL software. But maybe
>>a comprehensive catalogue of free software and data (like fonts) would
>>be a good idea.
>>
>>
MM> It fits into the emotional state of this discussion that suddenly "a
MM> free OGG player" has become "every single GPL software"...

Sorry, Magnus, my comment wasn't meant to be offensive. I apologize if
it sounds like that.

No, it's just, that IMHO theres no need to distribute software as
there enough download location. Of course a small Ogg-Player wouldn't
harm our bandwith, but it might be a signal into a different diretion.
Knowledge is our domain. And in my opinion we should stick to that.
It's absolutely ok, to guide people to the (GPL) software they need.
But we dont't have to offer it ourself.


subject: [Wikitech-l] MediaWiki 1.3.1 released

body:
1.3.1 fixes some of the remaining problems in 1.3.0:

* Watchlist parameters now work with register_globals off
* Fixed parsing of ''italics'' and '''bold''' mark-up (again)
* Special:Allpages display is more sensible on smaller wikis
* Fixed XHTML parsing error in classic skins
* Moved pages update watchlist correctly
* Fixed rebuildall.php on case-sensitive Unix filesystems
* Disabled file cache compression by default due to incompatibility
   with output buffer compression (ob_gzhandler)
* New magic word PAGENAMEE (URL-escaped version of PAGENAME)
* Installation avoids blank username; better message on missing
   XML module
* $wgWhitelistAccount no longer breaks all logins.

Anyone still running the 1.3 beta releases is _strongly_ recommended to
upgrade to the current code due to numerous fixed bugs and a security
vulnerability in the betas with some PHP configurations.

Note that while MediaWiki through 1.1 required register_globals to be
on, 1.2 and 1.3 *do not*. If you have register_globals on, you should
turn it off unless you are absolutely sure you require it for some other
package. See http://php.net/register_globals for general information.

Release notes:
http://sf.net/project/shownotes.php?group_id=34373&release_id=260607

Download:
http://prdownloads.sf.net/wikipedia/mediawiki-1.3.1.tar.gz?download

Wiki admin help mailing list:
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

Bug report system:
http://bugzilla.wikipedia.org/

Play "stump the developers" live on IRC:
#mediawiki on irc.freenode.net

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040814/81e79f71/attachment.pgp>


from_name: Roozbeh Pournader

subject: [Wikipedia-l] personal threats again

body:
On Sat, 14 Aug 2004 03:28:23 +0000, Steven Melenchuk <crazyr2 at shaw.ca> wrote:
> This doesn't seem to be the first time this anon has come up in a
> dispute - see some of the text at [[Talk:Azerbaijan]], [[Talk:South
> Azerbaijan]], etc. for details.

This is not the same user. This guys seems to be pro-Persia.

> I'm currently in the process of talking
> with this user about his apparent dispute and asking him to provide
> evidence for his claims.

> In the meantime I suggest you attempt not to,
> shall we say, "fuel his argument".

I only remove his personal attacks and threats. Will that be
considered "fueling"?

> Hopefully this will not escalate any further than it already has...

I can only hope.

roozbeh


subject: [Wikipedia-l] personal threats again

body:
On Fri, 13 Aug 2004 23:37:37 -0600, Fred Bauder <fredbaud at ctelco.net> wrote:
> Blocking is just that, a possibly ineffective remedy should he use a wide
> range of dynamic addresses. Following dispute resolution you can certainly
> see him banned and blocked as much as possible, indeed under our personal
> attacks policy he can be blocked immediately. I gather you don't want to do
> it yourself, but a new administrator will be elected and all that can be
> done on that front will probably be done.

I have sometimes done it myself a few times, and the other admin has
also done it a few times. But we seem to be in shortage of human time
if he continues.

One other question would be "is the personal attacks policy limited to
the English Wikipedia?"

> Blocking does not address the underlying issue that he can in the policially
> and religiously charged athmoshere in Iran potentially make good on his
> threats. This is an issue that needs to be handled through institutions such
> as the Persian Academy (if indeed you belong to such an organization).
> Absurd denunciations need to be recognized as such, not taken seriously by
> those in authority. This attitude takes a while to develop and requires
> exposure of this sort of thing in public forums.

I don't think he can do much with the Academy, but his threats simply
hurt me and possibly also others, they have just resulted in one good
contributor saying bye-bye because of "the dirty atmosphere".

(BTW, I'm not a "member" of the Persian Academy. That requires one to
be a real expert in the language, which I'm not, but I'm a member of
the its Language and Computers council.)

> If indeed you have made edits and expressed opinions which are vulnerable to
> successful attack by political and religious authorities themselves
> outsiders such as ourselves can do some good by publicizing the matter and
> exposing it. This can be small comfort in practical terms.

Nothing like this has ever been raised, since I usually stick to facts
and try to follow NPOV, but even the writing also about other side's
POV is sometimes very offensive to some of the people living or born
in this part of the world. On Wikipedia, I've been called a Persian
Chauvinist by Pan-Turkists, Anti-Islamic Republic and Anti-Khomeini by
a carpet dealer who possibly did business with the Iranian government,
pro-Khomeini by a person insisting of inserting some sexual rulings
that Ayatollah Khomeini shares with other Shia clerics into the
Khomeini article, and finally pro-Islamic Republic, anti-Persian,
pro-Turkic, and pro-Arab by this late guy. It seems that they oppose
neutrality. That may also be because I work openly on Wikipedia, and
don't hide my identity, so they can have a real face to attack.

Anyway, on the English Wikipedia, I would really appreciate if more
people can get active on Iran-related articles like [[Persian Gulf]],
[[South Azerbaijan]], and [[Ruhollah Khomeini]], so I can take a
little rest from being blamed.

roozbeh


subject: [Wikitech-l] Re: An alternate parser

body:
> Maybe we could make the parser a php extension ? That will solve the
> problem on the end-users side.

No, it won't. The problem is that many people can't compile or even
install php extensions. Everyone who uses shared hosting has very
little control over their Apache/php/mysql configuration.

Only people running (expensive) co-location boxes or lease whole
servers can do things like installing extensions, upgrading Apache or
PHP or MySQL.

Krzysztof Kowalczyk | http://blog.kowalczyk.info


subject: [Wikipedia-l] Re: personal threats again

body:
On Sat, 14 Aug 2004 09:22:35 +0200, Anthere <anthere9 at yahoo.com> wrote:
> I am distressed to know that this user has started again. I think it is
> very important that we take such matters very seriously, and I suggest
> that you mail us again about that, so that we see what we can do that
> would be acceptable to you from a personal point of view.

The person looks to have got a username on the English Wikipedia
(User:Amir1), and this is indeed very good news to me. Maybe we can
instruct and recruit him after all.

> As for the persian wikipedia itself, I recommand that you take it easy
> with the election process if there is an emergency. We can very well
> grant temporary sysophood to any person you think is okay (elections can
> proceed in parallel in the coming weeks). To have anyone named sysop, if
> you have no bureaucrat, ask any steward for emergency sysophood.
> I would recommand firm blocking and immediate deletions of threats.

Thank you very much for the recommendation. I am also a bureaucrat on
the Persian Wikipedia, so I just turned the other sysop and another
user who is still being voted on but has not received any negative
vote into bureaucrats, and instructed them what to do.

> Even if the the ip is changing, perhaps a range block could do the deal.

Unfortunately the range of IPs is too wide. He may be using different providers.

> However, I think your personal safety is more important than threats on
> the wiki itself.

Well, I don't think he can do much. The threat itself is more scary
and hurting than any probably consequences.

roozbeh


from_name: Timwi

subject: [Wikitech-l] Re: An alternate parser

body:
Magnus Manske wrote:

> If we'd limit ourselves to PHP, we could avoid XML, but PHP is not very
> well suited for this kind of text parsing, even with regexps, especially
> when it comes to performance. Also, using XML as an itermediate at one
> point opens potential for exchange with other services, either for
> output (generating static dumps, statistic analyses, etc.) or input
> (e.g., using other wiki markup in MediaWiki, for other projects).

I can sympathise with this line of reasoning. The lexing and parsing
needs to be done in C/C++ for efficiency, but then the generated parse
tree should be processed and compiled by PHP, so we need to "transfer"
it, and XML does that job.

> The one I'm currently working on is "manual" C++, but I see no reason
> not to use an parser-generator. Question: Can a parser-generator ensure
> the output (no matter the input) is valid XML?

Krzysztof Kowalczyk already tried to explain this, but I'll put it in
different words. The parser generator doesn't "ensure" the output is
valid XML; the programmer has to do that. But it's easy. A parser
generator generates the code to turn wikitext into a parse tree; to go
from a parse tree to a valid-XML representation of that parse tree is
like 1+1=2. It's trivial.

Do you know what a parse tree is? If not, let me know, and I'll try to
explain that to you too.

> Can it remove potential harmful HTML tags?

Don't think of it as "removing". Don't think of the whole process as
turning one text directly into another. It's not like that.

You give a parser generator a "grammar" (see [[formal grammar]]). If
this grammar says that "<marquee>" isn't a syntax element, then it will
be interpreted as text, and stored as such in the parse tree. Later
(much later), when the parse tree is compiled into actual (X)HTML, the
text would (obviously) be HTML-escaped. So it would become <marquee>.

Alternatively, of course, one can explicitly define <marquee> to be a
null syntax element in the grammar, so that the final output doesn't
contain it. But that shouldn't be necessary.

> The chain would then be preprocessing-parsing-XML-XHTML.

Uhm... no. See:
http://mail.wikipedia.org/pipermail/wikitech-l/2004-August/012135.html
where I describe the process.

You don't want to do anything to the text before parsing it (assuming
here that parsing includes lexing, although technically they're separate
steps). You want to do all processing after parsing.

Why? Well, because this is the purpose of parsing. We want to turn the
text into a data structure that computers can handle better than text.
It is much easier and much less error-prone to say "if this object is a
tree node representing template-inclusion, then do this" than to say
"search the string for some fuzzy pattern that looks a bit like a
template inclusion, but look out for nesting, and make sure you get the
parameters right, because they might contain pipes inside piped links,
and try not to mess things up."

> The XML could be cached, as all changes influenced by user options would
> happen only in the final step.

That is correct! But this XML would still be just the parse tree for the
wiki text.

> Caveat: Cache will have to be invalidated
> for variables and templates that change (e.g., {{NUMBEROFARTICLES}} and
> edited templates).

This is obvious. We already have to invalidate the cache for everything
that changes.

However, currently we also need to invalidate the parser cache for pages
that include a template we have edited. This should not be necessary. We
should be able to retrieve the parse tree for a page independently of
that for the included templates. To put them together at page-view time
is not costly, because we have to compile the parse tree into HTML
anyway (which is also not costly, but it means one sweep through the
parse tree).

> Once the parser is basically working, we can [...]
> run some benchmarks against the current parser. That
> should give us some more facts to base a decision on.

Personally, I'm almost inclined to say that having a proper parser is
more important than performance. :-) But I'm confident that it will
outperform the current parser by far.

Timwi


subject: [Wikitech-l] Re: An alternate parser

body:
evan at wikitravel.org wrote:

> On Fri, Aug 13, 2004 at 08:46:20PM +0200, Magnus Manske wrote:
>
>>* The XML-HTML-converter (#3) can focus on the pure context of the page:
>>make a normal link, stub link, thumbnailed image, or a category or
>>interwiki link, etc.
>
> It'd probably be sufficient to do an XSL style sheet. You could even send
> XML+XSL to the client for clients that support it, and do server-side XSL
> for clients that don't.

I don't know much about XSL(T), but personally I highly doubt that it
can give us the flexibility we need.


body:
?var Arnfj?r? Bjarmason wrote:

> Could the [X] yes i want to delete this button one has to click to
> delete pages/images be abolished? I dont think  anyone would by
> accedent click 2 links in a row (delete -> delete) so i dont think the
> checkbox is needed and just slows me and others down when doing mass
> deletion.

I agree. Deleted pages can always be undeleted, so there's no real
security problem in removing the checkbox.


subject: [Wikitech-l] Re: Double database entries

body:
Christian Semrau wrote:

> I created lists of cur table entries which have the same namespace and title.
> [...]

You mentioned that they can be fixed by sysops. As such, I don't think
the tech list is the right place to ask if there is any objection to
them doing so. I'd rather ask on wikipedia-l. However, I don't see any
real harm in it. All it does is add a revision to the history. We always
want that! We want it to look like lots of people are working on lots of
articles :-))))

Timwi


subject: [Wikitech-l] Re: An alternate parser

body:
Fennec Foxen wrote:
> That doesn't mean you can't do it both ways- PHP for those with shared
> hosting, an extension for those with their own.
>
> Of course that means you have to keep two codebases up to date. :b
> Didn't say there were no drawbacks.

That's something we should avoid if we can. There are two main ways to
do so:
* Use a PHP parser only. Maximize performance by minimizing the need to
_use_ the parser.
* Use a parser generator that generates both C and PHP code.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040814/d87cb257/attachment.pgp>


body:
Timwi writes:

> Christian Semrau wrote:
>
> > I created lists of cur table entries which have the same namespace and title.
> > [...]
>
> You mentioned that they can be fixed by sysops. As such, I don't think
> the tech list is the right place to ask if there is any objection to
> them doing so. I'd rather ask on wikipedia-l. However, I don't see any
> real harm in it. All it does is add a revision to the history. [...]

As I don't know the mechanisms of deletion and undeletion, I wanted to ask if
there are technical objections to this procedure. Since you have none, I will
promote my lists :-)

Christian
de:Benutzer:SirJective


body:
Brion Vibber wrote:
> Quick hacky benchmark:
> http://meta.wikimedia.org/wiki/PHP_string_looping
>
> Executive summary: using the append operator ($result .= "*") is
> relatively speedy. Copying strings ($result = $result . "*") is really
> reaaaaally slow.
>
> 'Relatively speedy' is tempered by the general sluggishness of PHP;
> per-iteration loop overhead is itself larger than an append operation.
>
> -- brion vibber (brion @ pobox.com)

Hello brion

Can you benchmark the following code against the two aboves ?

   while (something) {
     $result[$i] = '*';
     $i++;
   }
   $newresult = implode('', $result);

Maybe even :
   $result = (string) implode('', $result);

--
Ashar Voultoiz


subject: [Wikitech-l] disambiguation enhancement

body:
Hello,

I have some ideas (certainly grabbed from this list though) to enhance
disambiguation pages. Actually the software doesn't really now if a page
is a disambiguation and that prevent me from implementing new feature.

Seems like the best way is to add in the database a way to know what
kind an article is. I came up with two solutions :

1/ add a 'cur_is_disambiguation' just like we have a 'cur_is_redirect'

2/ add a 'cur_kind' with a new table 'kind' that list the different
possible articles (null, 'redirect', 'disambiguation').

I will personally go for #2 that will help us implement new features
later if it's needed (like adding 'stub', 'quality' type of articles).

--
Ashar Voultoiz


subject: [Wikitech-l] Re: An alternate parser

body:
Timwi wrote:

> Magnus Manske wrote:
>
>> The one I'm currently working on is "manual" C++, but I see no reason
>> not to use an parser-generator. Question: Can a parser-generator
>> ensure the output (no matter the input) is valid XML?
>
>
> Krzysztof Kowalczyk already tried to explain this, but I'll put it in
> different words. The parser generator doesn't "ensure" the output is
> valid XML; the programmer has to do that. But it's easy. A parser
> generator generates the code to turn wikitext into a parse tree; to go
> from a parse tree to a valid-XML representation of that parse tree is
> like 1+1=2. It's trivial.
>
> Do you know what a parse tree is? If not, let me know, and I'll try to
> explain that to you too.

I do know what it is. I already wrote a compiler or two during my
informatics classes. The way I put the questions in the original mail
was confusing, and I apologize. It was rather meant rhetorical, like in
"I don't think it does".

>
>> The chain would then be preprocessing-parsing-XML-XHTML.
>
>
> Uhm... no. See:
> http://mail.wikipedia.org/pipermail/wikitech-l/2004-August/012135.html
> where I describe the process.
>
> You don't want to do anything to the text before parsing it (assuming
> here that parsing includes lexing, although technically they're
> separate steps). You want to do all processing after parsing.

By "preprocessing", I mean replacing the {{template}} things with the
appropriate text stored in the database. The external parser can't do
that, and it can't be done afterwards, when everything is XML already
(see below). So, we'll have to do it as the very first step, just like
C/C++ does.

> Why? Well, because this is the purpose of parsing. We want to turn the
> text into a data structure that computers can handle better than text.
> It is much easier and much less error-prone to say "if this object is
> a tree node representing template-inclusion, then do this" than to say
> "search the string for some fuzzy pattern that looks a bit like a
> template inclusion, but look out for nesting, and make sure you get
> the parameters right, because they might contain pipes inside piped
> links, and try not to mess things up."

Example :
{| {{template}}
| stuff
|}
with {{template}} being "bgcolor=#FFFFFF". Wouldn't that be filtered out
if we do the template replacement when we're already in XML? Because,
XML probably would look something like
<table <wikitemplate>template</wikitemplate>>

stuff

if we let the parser loose on the template inclusion, which is *not*
valid XML.
C/C++ also does "#define" before the actual parser. It is exactly the
same as our templates.

Of course, we can explicitly forbid cases like the above.

>
>> The XML could be cached, as all changes influenced by user options
>> would happen only in the final step.
>
>
> That is correct! But this XML would still be just the parse tree for
> the wiki text.

Yes, the XML->(X)HTML step (compiling) would have to be done again each
time. But I figure that will take way less time than parsing does now.

> Personally, I'm almost inclined to say that having a proper parser is
> more important than performance. :-) But I'm confident that it will
> outperform the current parser by far.

All in due time (pun intended:-)

Magnus


subject: [Wikitech-l] Re: An alternate parser

body:
Magnus Manske wrote:

> Timwi wrote:
>
>> Do you know what a parse tree is? If not, let me know, and I'll try to
>> explain that to you too.
>
> I do know what it is. I already wrote a compiler or two during my
> informatics classes. The way I put the questions in the original mail
> was confusing, and I apologize. It was rather meant rhetorical, like in
> "I don't think it does".

Does that mean the compilers you wrote in your informatics classes
didn't produce valid executable code? ;-)

I'm sorry. I'm being a bit silly here. Please ignore that. :)

>> You don't want to do anything to the text before parsing it (assuming
>> here that parsing includes lexing, although technically they're
>> separate steps). You want to do all processing after parsing.
>
> By "preprocessing", I mean replacing the {{template}} things with the
> appropriate text stored in the database.

Yes, I know what you meant. But I still don't think it's necessary to do
that before the parsing.

Personally I think your example ("{| {{template}}") should be disallowed
because it's clearly evil. The correct way is, of course, to have one
whole table in one template, and not parts of scattered around places.
But it's still possible:

> Because, XML probably would look something like
> <table <wikitemplate>template</wikitemplate>>
>
stuff

No, of course it wouldn't. If it does, then you've done something
horribly wrong with your generator code. It should instead look like this:

  <attrs><wikitemplate>template</wikitemplate></attrs>

stuff

Then the processing step expands all templates:

  <attrs>bgcolor="white" border="1"</attrs>

stuff

Then the processing step expands all attrs:

  <bgcolor>white</bgcolor>
  <border>1</border>

stuff

But as I said, I think it should be disallowed. I think *at most*
something like
{| bgcolor="{{template}}"
should perhaps be allowed, and that would yield:

  <bgcolor><wikitemplate>template</wikitemplate></bgcolor>

stuff

Then, of course, the processing step will find the "include" block and
replace it with, say, "white".

(Of course, it should be noted here that the "table", "tr" and "td"
thingies aren't HTML tags, and are conceptually different from the
corresponding HTML tags. I'm sure you know this, but I'm pointing this
out for our other readers.)

> C/C++ also does "#define" before the actual parser. It is exactly the
> same as our templates.

That doesn't mean it's the right thing to do. C/C++ macro pre-processing
is widely considered evil. No other programming language uses it to any
such significant extent. I am rather opposed to re-using that horrible
idea in MediaWiki's syntax.

> Of course, we can explicitly forbid cases like the above.

Which I would very much prefer we did, but we don't have to.

Greetings,
Timwi


subject: [Wikitech-l] Re: An alternate parser

body:
evan at wikitravel.org wrote:

> On Sat, Aug 14, 2004 at 09:01:04PM +0100, Timwi wrote:
>
>>>It'd probably be sufficient to do an XSL style sheet. You could even send
>>>XML+XSL to the client for clients that support it, and do server-side XSL
>>>for clients that don't.
>>
>>I don't know much about XSL(T), but personally I highly doubt that it
>>can give us the flexibility we need.
>
> Do you make a habit of forming strong opinions based on near-total ignorance?

Do you make a habit of forming arguments based on ad-hominem attacks?


subject: [Wikitech-l] Re: disambiguation enhancement

body:
Ashar Voultoiz wrote:

> I have some ideas (certainly grabbed from this list though) to enhance
> disambiguation pages. Actually the software doesn't really now if a page
> is a disambiguation and that prevent me from implementing new feature.

We don't want the software to know or care if a page in Wikipedia is a
disambiguation page. Other wikis might not have, need, or want, to use
disambiguation pages (or stubs or substubs or anything other arbitrary
classification of article-namespace pages).

> 1/ add a 'cur_is_disambiguation' just like we have a 'cur_is_redirect'
>
> 2/ add a 'cur_kind' with a new table 'kind' that list the different
> possible articles (null, 'redirect', 'disambiguation').

Both ideas are infeasible because the 'cur' table is huge. Adding a
column to it would take quite some time of site downtime (maybe one of
the server admins can tell us how much exactly). Add to that the
concerns I mentioned above. The database should not contain information
that is specific to Wikipedia.

Timwi


from_name: Fennec Foxen

subject: [Wikitech-l] Re: An alternate parser

body:
On Sat, 14 Aug 2004 20:04:01 -0700, Krzysztof Kowalczyk
<kkowalczyk at gmail.com> wrote:
> > Maybe we could make the parser a php extension ? That will solve the
> > problem on the end-users side.
>
> No, it won't. The problem is that many people can't compile or even
> install php extensions. Everyone who uses shared hosting has very
> little control over their Apache/php/mysql configuration.

That doesn't mean you can't do it both ways- PHP for those with shared
hosting, an extension for those with their own.

Of course that means you have to keep two codebases up to date. :b
Didn't say there were no drawbacks.


references:
<cflt0k$e7u$1@sea.gmane.org> <cfm2uj$b8f$1@sea.gmane.org>
	<200408150805.30306.smolensk@eunet.yu>

body:
Nikola Smolenski wrote:
> How costly it would be to change type of cur_is_redirect to int?

It's presently a tinyint, with valid values from -128 to 127. Altering
its meaning would require being careful not to break existing
assumptions, however, and neither 'disambiguation' nor 'stub' are imho
useful or objectively calculable.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040814/e1fea0df/attachment.pgp>


subject: [Wikitech-l] disambiguation enhancement

body:
Ashar Voultoiz wrote:

> Hello,
>
> I have some ideas (certainly grabbed from this list though) to enhance
> disambiguation pages. Actually the software doesn't really now if a
> page is a disambiguation and that prevent me from implementing new
> feature.
>
> Seems like the best way is to add in the database a way to know what
> kind an article is. I came up with two solutions :
>
> 1/ add a 'cur_is_disambiguation' just like we have a 'cur_is_redirect'
>
> 2/ add a 'cur_kind' with a new table 'kind' that list the different
> possible articles (null, 'redirect', 'disambiguation').
>
>
> I will personally go for #2 that will help us implement new features
> later if it's needed (like adding 'stub', 'quality' type of articles).
>
Sounds good. #2 does make more sense. Should we use binary combination?
It makes no sense for an article to be redirect and disambiguation at
the same time, but maybe for future tags?
So:
1 = redirect
2 = disambiguation
4 = stub
etc.

Magnus


subject: [Wikitech-l] Re: An alternate parser

body:
Krzysztof Kowalczyk wrote:
> On Sat, 14 Aug 2004 11:38:13 -0400, Michael Becker
> <mbecker at jumpingjackweb.com> wrote:
>
>>I also think we should leave it in C, not php, since it will be more efficient, and it sounds like we could use all the efficiency that we can get.
>
>
> Doing stuff it C is actually a problem for many people who would like
> to run WikiMedia code as their personal wiki but don't have the
> ability to install C PHP extensions. That is a very common scenario.
> Those people (e.g. me) would no longer be able to use WikiMedia.
>
> I know that WikiMedia's main purpose is to serve the needs of
> WikiPedia but I would like the impact on non-WikiPedia users to be
> considered when making a decision.
>
> One solution would be to provide both slow, fall-back PHP parser and
> fast C parser.
>
> Krzysztof Kowalczyk | http://blog.kowalczyk.info

Lo,

Maybe we could make the parser a php extension ? That will solve the
problem on the end-users side.

--
Ashar Voutloiz


body:
Magnus Manske wrote:
<snip>
> Sounds good. #2 does make more sense. Should we use binary combination?
> It makes no sense for an article to be redirect and disambiguation at
> the same time, but maybe for future tags?
> So:
> 1 = redirect
> 2 = disambiguation
> 4 = stub
> etc.
>
> Magnus

Seems the best option. With the given proposal (stub, disambiguation,
redirect) I have to see an article with 2 or more types.

--
Ashar Voultoiz


subject: [Wikitech-l] Re: An alternate parser

body:
Fennec Foxen wrote:
>
> Of course that means you have to keep two codebases up to date.

Which, of course, would be trivial if the only code that needs to be
kept synchronised is that which is generated by the parser generator.


body:
Nikola Smolenski wrote:

> On Sunday 15 August 2004 00:14, Timwi wrote:
>
>>Ashar Voultoiz wrote:
>>
>>>I have some ideas (certainly grabbed from this list though) to enhance
>>>disambiguation pages. Actually the software doesn't really now if a page
>>>is a disambiguation and that prevent me from implementing new feature.
>>
>>We don't want the software to know or care if a page in Wikipedia is a
>>disambiguation page. Other wikis might not have, need, or want, to use
>>disambiguation pages (or stubs or substubs or anything other arbitrary
>>classification of article-namespace pages).
>
> This could be made configurable: other wikis could appoint different meanings
> to those numbers, different logic for their calculation, and different link
> styles to link to them.

For completely general stuff, one could use the 'links' table to see
what templates a page includes.

Then you could have features like "Average length of pages that include
a given template" (and on Wikipedia, we would type 'dab' and 'stub' into
the field, etc.).

So far you haven't made your actual suggestion, or told us what your
original idea was. An idea should ideally not start out as an
implementation plan. :) You should first have an idea of what should
change in the user interface. If you describe that to us, maybe someone
can come up with an implementation plan that doesn't require huge
changes to the DB.

Timwi


from_name: Jimmy (Jimbo) Wales

subject: [Wikipedia-l] personal threats again

body:
Roozbeh Pournader wrote:
> One other question would be "is the personal attacks policy limited to
> the English Wikipedia?"

No, it is global.  Personal attacks are not helpful in the creation of a
high quality encyclopedia and a healthy, caring community.

I don't know if there has been a solution to the lack of sysops in the
Persian wikipedia while I was hiding from the hurricane, but if that's
a problem I recommend just making a few people sysops on a temporary
emergency basis and then let the usual methods be used to ratify.

--Jimbo


list: wikien-l

subject: [WikiEN-l] Image:TrangBang.jpg

body:
Michael Becker wrote:
> I too look forward to Jimmy's comments on the issue, however due to
>the Hurricane currently flooding his house with up to 5 feet of
>water, I somehow doubt he will be replying any time soon :(.

Just to update everyone, I'm home now and absolutely nothing happened
to my house -- I, and the entire Tampa Bay area, got very lucky this
time.

I will be back to normal tomorrow; today I am going to a birthday
party for one of my daughter's friends, and then also cleaning up the
house.  The storm didn't mess up the house, but we did in our rush to
try to move everything valuable upstairs.  (We don't have any
particularly valuable possessions, just ordinary stuff, but like
anyone, we have a lot of stuff of sentimental value.)

--Jimbo


subject: [WikiEN-l] Image:TrangBang.jpg

body:
I'd be open to other well-reasoned viewpoints, but it sounds to me
like Anthony's analysis is right.  In any event, it's certainly the
_sort_ of analysis that I think we can and must do for images like
this.

I also agree with Anthony on the details of which articles _need_ the
picture, but I do think that reasonable people might differ on the
details, so of course as usual I'm reluctant to make any unnecessary
decrees.

Anthony DiPierro wrote:

> Jens Ropers asked whether this is likely one of the "few rare cases" where
> we desperately need an image which is unavailable under fair use.
>
> Looking at the details, this image is used in [[Nudity]], [[Vietnam War]],
> [[Strategic bombing]], [[Kim Phuc Phan Thi]], [[History of the United States
> (1964-1980)]], and [[Huynh Cong Ut]].  I don't see any arguable "deperate
> need" for the image in any of the articles other than [[Kim Phuc Phan Thi]]
> and [[Huynh Cong Ut]], and the image is most likely fair use in those
> articles.
>
> I think we should remove the image from [[Nudity]], [[Vietnam War]],
> [[Strategic bombing]], and [[History of the United States (1964-1980)]].
> I'd suggest that we only link to it in the other articles (assuming it is
> available somewhere online outside of Wikipedia), but I could see an
> argument for keeping it, as it is essentially impossible to replace, the
> cost to buy a free license would be prohibitive (at least at Wikipedia's
> current stage of development), and it is most likely usable by at least the
> noncommercial reusers.  I don't see a problem with asking for permission to
> use the image in those two articles if we decided to keep it.
>
> Just my opinion, though.  I'd like to here Jimbo's assessment.
>
> Anthony
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l
>


from_name: Nikola Smolenski

body:
On Sunday 15 August 2004 00:14, Timwi wrote:
> Ashar Voultoiz wrote:
> > I have some ideas (certainly grabbed from this list though) to enhance
> > disambiguation pages. Actually the software doesn't really now if a page
> > is a disambiguation and that prevent me from implementing new feature.
>
> We don't want the software to know or care if a page in Wikipedia is a
> disambiguation page. Other wikis might not have, need, or want, to use
> disambiguation pages (or stubs or substubs or anything other arbitrary
> classification of article-namespace pages).

This could be made configurable: other wikis could appoint different meanings
to those numbers, different logic for their calculation, and different link
styles to link to them.

> > 1/ add a 'cur_is_disambiguation' just like we have a 'cur_is_redirect'
> >
> > 2/ add a 'cur_kind' with a new table 'kind' that list the different
> > possible articles (null, 'redirect', 'disambiguation').
>
> Both ideas are infeasible because the 'cur' table is huge. Adding a
> column to it would take quite some time of site downtime (maybe one of

How costly it would be to change type of cur_is_redirect to int?


from_name: Alwin Meschede

body:
Timwi schrieb:
>> I created lists of cur table entries which have the same namespace and
>> title.
>> [...]
>
> You mentioned that they can be fixed by sysops. As such, I don't think
> the tech list is the right place to ask if there is any objection to
> them doing so.

You know, deleting more than 200 articles by hand can be a painful
job... Maybe some developer could invent a SQL query that gets rid of
all those duplicates at once? And maybe the software can be somehow
improved to make sure it updates articles correctly and does *not*
create a duplicate in some weird corner-cases.


subject: [Wikitech-l] Mailing lists (was Re: Double database entries)

body:
Christian Semrau wrote:
> PS: What's up with the wikipedia-l list archives (pipermail and gmane)? I can't
> believe Jimbo is the only one writing to this list today. I have posted a
> message via gmane (~12:00 UTC), but that didn't show up yet.

There was also a spammer advertising Cialis, but the list didn't let
that go through. :)

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040815/ff01b9d6/attachment.pgp>


from_name: Christian Semrau

body:
Alwin Meschede writes:
>
> You know, deleting more than 200 articles by hand can be a painful
> job... Maybe some developer could invent a SQL query that gets rid of
> all those duplicates at once? And maybe the software can be somehow
> improved to make sure it updates articles correctly and does *not*
> create a duplicate in some weird corner-cases.

At de I saw a bot deleting redirects with nonexistent target (controlled by
de:Benutzer:Head), it may be possible to use a bot to delete and undelete these
items.

Apart from that it would be good to improve the software to stop the creation of
duplicates.

Christian
de:Benutzer:SirJective


subject: [Wikipedia-l] Re: Request for the second Serbian Wikipedia

body:
Philip Newton wrote:
> On Sun, 08 Aug 2004 15:28:36 -0700, Ray Saintonge wrote:
>>I suppose this all depends on there being a reversible one-to-one
>>relationship between the two versions of Serbian.
>
> This is possible, to the best of my knowledge.
>
> Some sounds which have one letter in Cyrillic have two letters in Latin,
> but AFAIK those two-letter sequences always map to that one Cyrillic
> letter, to the translation is unambiguous in both directions.

Except, of course, when you have foreign words and names to deal with.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20040815/7a6a8d2a/attachment.pgp


subject: [Wikitech-l] Re: An alternate parser

body:
Brion Vibber wrote:

> * Use a parser generator that generates both C and PHP code.

Is there such a thing? Or do I have to write one ;-)

Magnus


body:
Alwin Meschede wrote:

> You know, deleting more than 200 articles by hand can be a painful
> job... Maybe some developer could invent a SQL query that gets rid of
> all those duplicates at once? And maybe the software can be somehow
> improved to make sure it updates articles correctly and does *not*
> create a duplicate in some weird corner-cases.

There's a "INSERT ... ON DUPLICATE KEY UPDATE" function in MySQL which
should avoid double creation. Alternatively, we could make the key pair
cur_title,cur_namespace unique.

Magnus


body:
At the german WP, user Fristu took care of all ~100 listed entries, within a few
hours. I know the ~250 en: entries will not be cleaned within a day, but can
surely be manually cleaned within a week, given the entries are linked properly
(be careful with redirects).

Christian
SirJective

PS: What's up with the wikipedia-l list archives (pipermail and gmane)? I can't
believe Jimbo is the only one writing to this list today. I have posted a
message via gmane (~12:00 UTC), but that didn't show up yet.


from_name: Ray Saintonge

body:
?var Arnfj?r? Bjarmason wrote:

>It's not users but sysops and i think if they have that privelage they
>have a pretty good clue of what to delete and what not to delete.
>
>On Sat, 14 Aug 2004 12:10:27 +0200, Ashar Voultoiz <thoane at altern.org> wrote:
>
>
>>?var Arnfj?r? Bjarmason wrote:
>>
>>
>>>Could the [X] yes i want to delete this button one has to click to
>>>delete pages/images be abolished? I dont think  anyone would by
>>>accedent click 2 links in a row (delete -> delete) so i dont think the
>>>checkbox is needed and just slows me and others down when doing mass
>>>deletion.
>>>
>>>
>>Hello,
>>
>>At least it forces users to review their choices and to enter a comment.
>>  I am personally against removing this checkbox.
>>
>>
A little safety valve is always a good idea.  One additonal keystroke
should not seriously make demands on a person's.  The argument in
support of it has nothing to do with trusting sysops; one operating is
bad faith is no prevented from action by this one keystroke.  Sysops can
make mistakes as much as anyone else.

Ec


from_name: Ævar Arnfjörð Bjarmason

body:
So will the change be made in a future version?

On Sat, 14 Aug 2004 21:02:28 +0100, Timwi <timwi at gmx.net> wrote:
>
>
> ?var Arnfj?r? Bjarmason wrote:
>
> > Could the [X] yes i want to delete this button one has to click to
> > delete pages/images be abolished? I dont think  anyone would by
> > accedent click 2 links in a row (delete -> delete) so i dont think the
> > checkbox is needed and just slows me and others down when doing mass
> > deletion.
>
> I agree. Deleted pages can always be undeleted, so there's no real
> security problem in removing the checkbox.
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>


from_name: Magnus Manske

subject: [Wikitech-l] Re: An alternate parser

body:
Tim Starling wrote:

> Buying processors for parsing wikitext is shaping up to be the single
> largest use of Wikimedia funds. We shouldn't lightly dismiss the
> possibility of a signficant improvement in speed, even if it is
> detrimental to non-Wikimedia users. We can reduce resource usage with
> better caching, but that's not mutually exclusive with improving the
> base speed.

Yup.

> If we need to, we could fork the syntax. The PHP parser could be
> rarely updated, with features equivalent to 1.3 or 1.4 for the
> forseeable future. We could introduce a new parser written in (or
> compiled into) a low-level language, and conduct most development
> there. Although this hurts non-Wikimedia users, we have to consider
> this option rationally. We have to keep Wikimedia's interests, and the
> interests of the people who donate to Wikimedia, in mind.

I would prefer if this initiative wouldn't lead to a fork, partial or
not. IIRC, a parser-generator uses some language syntax file (in EBNF or
whatever is the flavor of the month now). If this is a more-or-less
standard format, we won't need a single parser generator that generates
both C and PHP; we could do with two separate ones which can read the
same format.

<fun>Or we'd have to translate from C to PHP manually; some regexps
should do. "$" in front of all the variables; replace "void" with
"function"; there you go!</fun>

Magnus


from_name: Rowan Collins

subject: [Wikitech-l] Re: An alternate parser

body:
> <fun>Or we'd have to translate from C to PHP manually; some regexps
> should do. "$" in front of all the variables; replace "void" with
> "function"; there you go!</fun>

Or perhaps we should move away from using regexps and use a parser
generator to create a parser for the output of the parser generator in
order to then process it in reverse and produce PHP...

Or not.

;-)

Slightly more seriously, Bison, for instance, doesn't seem to claim
standardisation of input format with anything other than itself, so it
is possible that you would have to convert the input grammar files
from that format into the input of some as-yet-unlocated PHP
outputting parser generator.

--
Rowan Collins BSc
[IMSoP]


from_name: Philip Newton

body:
On Sun, 08 Aug 2004 15:28:36 -0700, Ray Saintonge wrote:

> I suppose this all depends on there being a reversible one-to-one
> relationship between the two versions of Serbian.

This is possible, to the best of my knowledge.

Some sounds which have one letter in Cyrillic have two letters in Latin,
but AFAIK those two-letter sequences always map to that one Cyrillic
letter, to the translation is unambiguous in both directions.

Cheers,
Philip


from_name: Ralesk Ne'vennoyx

body:
Brion Vibber wrote:

> Philip Newton wrote:
>
>> On Sun, 08 Aug 2004 15:28:36 -0700, Ray Saintonge wrote:
>>
>>> I suppose this all depends on there being a reversible one-to-one
>>> relationship between the two versions of Serbian.
>>
>>
>> This is possible, to the best of my knowledge.
>>
>> Some sounds which have one letter in Cyrillic have two letters in Latin,
>> but AFAIK those two-letter sequences always map to that one Cyrillic
>> letter, to the translation is unambiguous in both directions.
>
>
> Except, of course, when you have foreign words and names to deal with.
>

Question: how is the Belarusan Wikipedia dealing with Lacinka and Cyrillic?


from_name: Brion Vibber

subject: [Wikitech-l] Two markets (was Re: An alternate parser)

body:
Tim Starling wrote:
> Magnus Manske wrote:
>> I would prefer if this initiative wouldn't lead to a fork, partial or
>> not.
>
>
> What's wrong with forking? We have two different markets for our
> software, and these markets have conflicting needs. Why not produce a
> customised product for each one?

It's additional work to maintain two versions of the code.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040815/b613c511/attachment.pgp>


from_name: Krzysztof Kowalczyk

body:
> What's wrong with forking? We have two different markets for our
> software, and these markets have conflicting needs. Why not produce a
> customised product for each one?

Possible dilution of dev resources, for one. If there are N developers
working on unified code-based, in the worst-case scenario after the
split each project might end up with, say, N/2 developers, testers,
users etc.

Krzysztof Kowalczyk | http://blog.kowalczyk.info


subject: [Wikitech-l] Re: An alternate parser

body:
On Mon, 16 Aug 2004 06:51:21 +0200, Nikola Smolenski <smolensk at eunet.yu> wrote:
> One question: wouldn't that PHP parser be slower than current parser, and how
> much?

No-one can know that unless the new parser is written and benchmarked
against the current one.

Krzysztof Kowalczyk | http://blog.kowalczyk.info


subject: [Wikitech-l] Re: Embed mediawiki ?

body:
Timwi wrote:
> Manuel VACELET wrote:
>
> > I investigate to embed a wiki engine into another web application. As
> > wikipedia user I think mediawiki is one of the best engine and I would
> > integrate it. But I would have your developer point of view :
> > Do you think it is a good idea to embed mediawiki as wiki engine or
> > should I look for another tool ?
>
> I am unsure what exactly you mean by "embed".  It is always possible to
> install MediaWiki in a subdirectory of your server root, and have other
> Web applications in different directories.

I think "embed" means combining them such that certain features would
work seamlessly as though it was all one application - most
importantly, I would expect, logins. IIRC, somebody else mentioned
something very similar to this just recently...

Ah yes, someone was considering merging the databases for MediaWiki
and phpBB: http://mail.wikipedia.org/pipermail/mediawiki-l/2004-August/000903.html

Basically, there doesn't seem to be a lot to stop you - if you do want
to combine the database with something else the same issues apply:
make sure table names don't conflict, formats match appropriately,
etc. Other than that, it's really a case of customising skins and
interface features to make it look and feel like part of the same
thing [hopefully fairly straight-forward], and adding any particular
magic features you want for your own needs.

Of course, I could have completely misunderstood the original message,
in which case please ignore my babblings...

--
Rowan Collins BSc
[IMSoP]


subject: [Wikitech-l] Re: An alternate parser

body:
Magnus Manske wrote:
>
> I would prefer if this initiative wouldn't lead to a fork, partial or
> not. IIRC, a parser-generator uses some language syntax file (in EBNF or
> whatever is the flavor of the month now). If this is a more-or-less
> standard format, we won't need a single parser generator that generates
> both C and PHP; we could do with two separate ones which can read the
> same format.

The file formats for both lex (resp. flex) and yacc (resp. bison)
include C code, so it is probably not possible in general to have a
parser generator that outputs both C and PHP from the same input file.

*However*, at least in the lexer I have written so far, there isn't
anything spectacularly impressive. This one will be trivial to keep a C
and PHP version synchronised of. As for the parser itself, I won't make
any promises just yet, but from what I've seen, it seems similarly
promising.

Timwi


body:
?var Arnfj?r? Bjarmason wrote:

> So will the change be made in a future version?

You are the one who proposed this. The onus lies with you to set up a
vote on Meta and convince us that the community at large really does
want this change. :-)


body:
Magnus Manske wrote:

> Alwin Meschede wrote:
>
>> You know, deleting more than 200 articles by hand can be a painful
>> job... Maybe some developer could invent a SQL query that gets rid of
>> all those duplicates at once? And maybe the software can be somehow
>> improved to make sure it updates articles correctly and does *not*
>> create a duplicate in some weird corner-cases.
>
> There's a "INSERT ... ON DUPLICATE KEY UPDATE" function in MySQL which
> should avoid double creation.

Ah, but that would over-write the older 'cur' row, no?

> Alternatively, we could make the key pair
> cur_title,cur_namespace unique.

That sounds much better. :-)


body:
Ralesk Ne'vennoyx wrote:

> Question: how is the Belarusan Wikipedia dealing with Lacinka and Cyrillic?

At the moment, be.wikipedia.org is pretty inactive. The few users who
are working on it ([[be:User:EugeneZelenko]] and
[[be:User:195.50.4.174]] today) use Cyrillic only. [[en:Lacinka]] says,
"Since the collapse of the Soviet Union and the subsequent independence
of Belarus some groups have advocated reviving the Lacinka alphabet, but
as of 2004 very few people use it."

Timwi


subject: [Wikitech-l] External diff engine

body:
Steinar H. Gunderson has been working on a C++ diff engine that can be
plugged into MediaWiki without too much fuss. Current 1.4 CVS has
drop-in support for it thanks to JeLuF.

My very unscientific test with a couple of ~134kb revisions of the
Village Pump with some number of paragraphs changed:

Plain page view:    2798ms
Classic diff:       4856ms (2798ms + 2058ms)
Diff with wikidiff: 3342ms (2798ms +  544ms)

Man, those long pages _drag_. :) Turning on memcached & the parser cache
brings down the page view time to 1501 ms, but doesn't significantly
alter the diff views, which go through the full render cycle.

[Current CVS head, no special parser caching options enabled, MySQL
4.0.20, PHP 5.0.1, Athlon XP 2400+. Times are median of 10 successive
hits run with 'ab' on the local host.]

This seems to be roughly a 4x improvement in the actual diffing speed
for a typical long page change a bit case.

The code for the differ is in the 'extensions' module in CVS, under
'wikidiff' subdirectory. It should compile on Linux; I've added a more
or less sensible Makefile. Requires SWIG (http://www.swig.org/).

Once php_wikidiff.so is installed, set $wgUseExternalDiffEngine = true;
and _magic_ happens!

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040816/908506d5/attachment.pgp>


from_name: N.T. Riche

subject:
[WikiEN-l] Mintguy is an ad-hominem,
	British Upper Class POV revert war monger

body:
I agree with you Mr. Sokolov....i've seen his edits as very well

Nick aka User:Plato

Abe Sokolov <abesokolov at hotmail.com> wrote:
coxcustomer3 at cox.net wrote:

"Mintguy has this stuffy, toffee-nosed, Upper Class, British POV, Canonical
Fascism about every article and doesn't allow for American contributions on
British and British related topics by me."

I'm an American, I've never been accused of an 'aristocratic' bias, and I
use American English spellings. As the main author of a number of UK-related
articles (e.g., British Empire), I've only gotten to know Mintugy as a
helpful, diligent, and competent user.

If he is reverting American contributors, I bet that it's because their
edits are unconstructive, not because they're Americans.

-User:172

_________________________________________________________________
Check out Election 2004 for up-to-date election news, plus voter tools and
more! http://special.msn.com/msn/election2004.armx

_______________________________________________
WikiEN-l mailing list
WikiEN-l at Wikipedia.org
http://mail.wikipedia.org/mailman/listinfo/wikien-l

---------------------------------
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.


from_name: James R. Johnson

body:
Now THAT sounds like a good idea - that would improve the smaller language's
wikis with possibly a translation link to an article in another language,
such as a Lance Armstrong article that was translated into German or
Swedish, and being on the German/Swedish wiki, being linked to by the
English wiki.  Good thinking.

James

-----Original Message-----
From: wikipedia-l-bounces at Wikimedia.org
[mailto:wikipedia-l-bounces at Wikimedia.org] On Behalf Of Peter Shaw
Sent: Sunday, August 15, 2004 10:12 PM
To: wikipedia-l at wikimedia.org
Subject: [Wikipedia-l] Suggestions for better cross-language collaboration

The following is a mediawiki feature request concerning improvements to make
cross-language cooperation and translation more attractive. I also posted it
to the enormous feature-request- and bug-report-wiki on meta, which is
probably better to read, because of the HTML formatting.
(http://meta.wikimedia.org/wiki/MediaWiki_1.3_comments_and_bug_reports#Featu
re_request:_Tools_for_cross-language_colaboration)

The unfortunate fact is that today the wikipedias reinvent the wheel for
every language. The only relationship that sometimes exists, is that
articles get translated from one language to another. Each of the changes
that are then made in both versions of the article benefits only the
wikipedia of one langauge.

This is not only a problem for the smaller wikipedias. Especially local
information is much better represented in the wikipedia of the regional
language, and it would be beneficial for the english wikipedia, if that
information were to be imported. Also some pages (I only know about german)
in foreign language versions are written from an interesting other
perspective or just have more work done, because some people speaking that
language happened to be very interested in the subject.

The reason why I personally don't translate wikipedia articles, is because I
feel that instead of benefiting wikipedia, I split up the manpower between
the english and the german version. This would change, if the work in one
language would benefit the wikipedia of another.

What I suggest is:

(1) The possibility to maintain a relationship to the link of the article in
other languages, like
- translated from (this article is a translation of the article in that
language)
- translated to (this article has been translated to that language)
- independent (the content of the article is language specific)
- unmerged (no translation in either direction known)
- merged/corresponding (people work on the article in both languages,
changes in both articles get applied to the other one)

(2) A counter that counts up from the last edit made with a special keyword,
an trans-lingual edit could then be called something like "cross-language
[german] in: Added new section, out: rearrangement of introduction". Such a
counter could either count the changes made in the other language, the
changes made in this edit, both added up, or preferably both beside each
other to get a glimpse on which one is more dominant.

(3) A special page, that lists the topics that have the highest counter
between this and another language, for people interested in translating
articles.

The better the articles get, the more sense does it make to translate the
content. Making translation more attractive is beneficial for both the main
(adding local information) and the language (more/higher quality content)
wikipedias.

_______________________________________________
Wikipedia-l mailing list
Wikipedia-l at Wikimedia.org
http://mail.wikipedia.org/mailman/listinfo/wikipedia-l


body:
A related idea - I don't know if it's already done, but how about when
starting a new wiki, have a template main page and welcome page in a
language specified by the person requesting the wiki, so that person can
then translate into the new language, such as Manx Gaelic from Scots Gaelic,
or the like?

James

-----Original Message-----
From: wikipedia-l-bounces at Wikimedia.org
[mailto:wikipedia-l-bounces at Wikimedia.org] On Behalf Of Peter Shaw
Sent: Sunday, August 15, 2004 10:12 PM
To: wikipedia-l at wikimedia.org
Subject: [Wikipedia-l] Suggestions for better cross-language collaboration

The following is a mediawiki feature request concerning improvements to make
cross-language cooperation and translation more attractive. I also posted it
to the enormous feature-request- and bug-report-wiki on meta, which is
probably better to read, because of the HTML formatting.
(http://meta.wikimedia.org/wiki/MediaWiki_1.3_comments_and_bug_reports#Featu
re_request:_Tools_for_cross-language_colaboration)

The unfortunate fact is that today the wikipedias reinvent the wheel for
every language. The only relationship that sometimes exists, is that
articles get translated from one language to another. Each of the changes
that are then made in both versions of the article benefits only the
wikipedia of one langauge.

This is not only a problem for the smaller wikipedias. Especially local
information is much better represented in the wikipedia of the regional
language, and it would be beneficial for the english wikipedia, if that
information were to be imported. Also some pages (I only know about german)
in foreign language versions are written from an interesting other
perspective or just have more work done, because some people speaking that
language happened to be very interested in the subject.

The reason why I personally don't translate wikipedia articles, is because I
feel that instead of benefiting wikipedia, I split up the manpower between
the english and the german version. This would change, if the work in one
language would benefit the wikipedia of another.

What I suggest is:

(1) The possibility to maintain a relationship to the link of the article in
other languages, like
- translated from (this article is a translation of the article in that
language)
- translated to (this article has been translated to that language)
- independent (the content of the article is language specific)
- unmerged (no translation in either direction known)
- merged/corresponding (people work on the article in both languages,
changes in both articles get applied to the other one)

(2) A counter that counts up from the last edit made with a special keyword,
an trans-lingual edit could then be called something like "cross-language
[german] in: Added new section, out: rearrangement of introduction". Such a
counter could either count the changes made in the other language, the
changes made in this edit, both added up, or preferably both beside each
other to get a glimpse on which one is more dominant.

(3) A special page, that lists the topics that have the highest counter
between this and another language, for people interested in translating
articles.

The better the articles get, the more sense does it make to translate the
content. Making translation more attractive is beneficial for both the main
(adding local information) and the language (more/higher quality content)
wikipedias.

_______________________________________________
Wikipedia-l mailing list
Wikipedia-l at Wikimedia.org
http://mail.wikipedia.org/mailman/listinfo/wikipedia-l


from_name: Tim Starling

subject: [Wikitech-l] Re: An alternate parser

body:
Brion Vibber wrote:
> Fennec Foxen wrote:
>
>> That doesn't mean you can't do it both ways- PHP for those with shared
>> hosting, an extension for those with their own.
>>
>> Of course that means you have to keep two codebases up to date. :b
>> Didn't say there were no drawbacks.
>
>
> That's something we should avoid if we can. There are two main ways to
> do so:
> * Use a PHP parser only. Maximize performance by minimizing the need to
> _use_ the parser.
> * Use a parser generator that generates both C and PHP code.
>

Buying processors for parsing wikitext is shaping up to be the single
largest use of Wikimedia funds. We shouldn't lightly dismiss the
possibility of a signficant improvement in speed, even if it is
detrimental to non-Wikimedia users. We can reduce resource usage with
better caching, but that's not mutually exclusive with improving the
base speed.

If we need to, we could fork the syntax. The PHP parser could be rarely
updated, with features equivalent to 1.3 or 1.4 for the forseeable
future. We could introduce a new parser written in (or compiled into) a
low-level language, and conduct most development there. Although this
hurts non-Wikimedia users, we have to consider this option rationally.
We have to keep Wikimedia's interests, and the interests of the people
who donate to Wikimedia, in mind.

-- Tim Starling


from_name: Peter Shaw

subject: [Wikipedia-l] Suggestions for better cross-language collaboration

body:
The following is a mediawiki feature request concerning improvements to make
cross-language cooperation and translation more attractive. I also posted it
to the enormous feature-request- and bug-report-wiki on meta, which is
probably better to read, because of the HTML formatting.
(http://meta.wikimedia.org/wiki/MediaWiki_1.3_comments_and_bug_reports#Feature_request:_Tools_for_cross-language_colaboration)

The unfortunate fact is that today the wikipedias reinvent the wheel for every
language. The only relationship that sometimes exists, is that articles get
translated from one language to another. Each of the changes that are then
made in both versions of the article benefits only the wikipedia of one
langauge.

This is not only a problem for the smaller wikipedias. Especially local
information is much better represented in the wikipedia of the regional
language, and it would be beneficial for the english wikipedia, if that
information were to be imported. Also some pages (I only know about german)
in foreign language versions are written from an interesting other
perspective or just have more work done, because some people speaking that
language happened to be very interested in the subject.

The reason why I personally don't translate wikipedia articles, is because I
feel that instead of benefiting wikipedia, I split up the manpower between
the english and the german version. This would change, if the work in one
language would benefit the wikipedia of another.

What I suggest is:

(1) The possibility to maintain a relationship to the link of the article in
other languages, like
- translated from (this article is a translation of the article in that
language)
- translated to (this article has been translated to that language)
- independent (the content of the article is language specific)
- unmerged (no translation in either direction known)
- merged/corresponding (people work on the article in both languages, changes
in both articles get applied to the other one)

(2) A counter that counts up from the last edit made with a special keyword,
an trans-lingual edit could then be called something like "cross-language
[german] in: Added new section, out: rearrangement of introduction". Such a
counter could either count the changes made in the other language, the
changes made in this edit, both added up, or preferably both beside each
other to get a glimpse on which one is more dominant.

(3) A special page, that lists the topics that have the highest counter
between this and another language, for people interested in translating
articles.

The better the articles get, the more sense does it make to translate the
content. Making translation more attractive is beneficial for both the main
(adding local information) and the language (more/higher quality content)
wikipedias.


subject:
[Wikitech-l] Changing the "Powered by" and "GNU FDL" images at
	the	bottom of pages.

body:
?var Arnfj?r? Bjarmason wrote:
> Is it possible to get the source files for these two images somewhere?:
>
> http://en.wikipedia.org/style/images/poweredby_mediawiki_88x31.png

This was posted to this list a few months ago. My search isn't turning
it up immediately, sorry. I don't think source files got posted, I'm not
sure what the format is.

> http://en.wikipedia.org/style/images/gnu-fdl.png

http://en.wikipedia.org/style/images/gnu-fdl.xcf

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040816/4a3f9ec6/attachment.pgp>


body:
On Sunday 15 August 2004 08:31, Brion Vibber wrote:
> Nikola Smolenski wrote:
> > How costly it would be to change type of cur_is_redirect to int?
>
> It's presently a tinyint, with valid values from -128 to 127. Altering
> its meaning would require being careful not to break existing
> assumptions, however, and neither 'disambiguation' nor 'stub' are imho
> useful or objectively calculable.

I'm also not sure how useful it is, but it is objectively calculable :)

$kind=substr($article,"{{disambig}})<<2|substr($article,"{{stub}}")<<1|substr($article,"#REDIRECT");
// (more or less...)


references:
<cflt0k$e7u$1@sea.gmane.org> <200408150805.30306.smolensk@eunet.yu>
	<cfn1ve$nuu$1@sea.gmane.org>

body:
On Sunday 15 August 2004 09:04, Timwi wrote:
> Nikola Smolenski wrote:
> > On Sunday 15 August 2004 00:14, Timwi wrote:
> >>Ashar Voultoiz wrote:
> >>>I have some ideas (certainly grabbed from this list though) to enhance
> >>>disambiguation pages. Actually the software doesn't really now if a page
> >>>is a disambiguation and that prevent me from implementing new feature.
> >>
> >>We don't want the software to know or care if a page in Wikipedia is a
> >>disambiguation page. Other wikis might not have, need, or want, to use
> >>disambiguation pages (or stubs or substubs or anything other arbitrary
> >>classification of article-namespace pages).
> >
> > This could be made configurable: other wikis could appoint different
> > meanings to those numbers, different logic for their calculation, and
> > different link styles to link to them.
>
> For completely general stuff, one could use the 'links' table to see
> what templates a page includes.

Hey, yes, that would be even easier.

> So far you haven't made your actual suggestion, or told us what your
> original idea was. An idea should ideally not start out as an
> implementation plan. :) You should first have an idea of what should
> change in the user interface. If you describe that to us, maybe someone
> can come up with an implementation plan that doesn't require huge
> changes to the DB.

Me? I don't have an original idea. I was just pondering on whether Ashar's
idea is possible to implement.


subject: [Wikitech-l] Re: An alternate parser

body:
On Monday 16 August 2004 00:32, Timwi wrote:
> *However*, at least in the lexer I have written so far, there isn't
> anything spectacularly impressive. This one will be trivial to keep a C
> and PHP version synchronised of. As for the parser itself, I won't make
> any promises just yet, but from what I've seen, it seems similarly
> promising.

One question: wouldn't that PHP parser be slower than current parser, and how
much?


body:
On Monday 16 August 2004 05:57, Tim Starling wrote:
> Magnus Manske wrote:
> > I would prefer if this initiative wouldn't lead to a fork, partial or
> > not.
>
> What's wrong with forking? We have two different markets for our
> software, and these markets have conflicting needs. Why not produce a
> customised product for each one?

In this case, interoperability. If parsers diverge, it might happen that other
MediaWiki wikis couldn't include Wikipedia's contents, and vice versa.


subject: [Wikitech-l] Wiki branches, voting

body:
I strongly support explorations of new ideas like this, but I caution
that many people (and I am among them) are very skeptical of _voting_
as a solution to wiki social issues.  Voting brings with it a whole
host of incentive problems, many of them much worse than the incentive
problems that we face today.

Voting can be a reasonable way to resolve issues for which absolutely
no other solution can be found.  But I very much doubt that
introducing a system where people vote on article content is a good
idea.

Let's imagine that 90% of the people editing a particular page are of
one opinion, and 10% are of a different opinion.  So long as the
participants are all reasonable and friendly, this poses no obstacle
to the production of an article that is satisfactory to all -- this is
the normal wiki way of producing quality encyclopedic content.

The majority has no choice but to respect the viewpoint of the
minority, because the minority can always just _edit the article_.

If branching and voting are introducted, we've destroyed the incentive
structure that makes wiki work.  The 90% now has no reason to respect
the viewpoint of the minority, because the vote is so heavily
one-sided that the minority can be safely ignored.

Voting is not necessarily the enemy of compromise -- but it certainly
*can* be, if used carelessly.

--Jimbo


body:
Timwi wrote:

> Magnus Manske wrote:
>
>> There's a "INSERT ... ON DUPLICATE KEY UPDATE" function in MySQL
>> which should avoid double creation.
>
>
> Ah, but that would over-write the older 'cur' row, no?
>
Yes, but AFAIK, this "double creation" only happens if a new article is
submitted and saved twice (thrice etc.) in rapid-fire mode. So, it would
basically overwrite the information with the same information again.

Magnus


subject: [Wikitech-l] Distributing wikimedia hardware

body:
mbecker wrote:
> I've just now subscribed to this list, so please stop me if this has
>been already discussed... After hearing about the storm coming toward
>the wikimedia servers. I thought to myself, wouldn't it be a good idea
>to have backup servers elsewhere in the world?

I think it would be a good idea, but only if the expense made sense.

Here are three things I wanted to pass along:0

1.  Keep in mind that our hosting facility remained confident
throughout the ordeal that there would be no loss of connectivity due
to this storm.  It's a strong building, they have good power
contingency plans, etc.  We are not in a position to evaluate their
confidence, of course.  I merely pass it along as information...

It's pretty hard to take down a data center like this one.

2.  We will soon have functional squids in France.  As we move forward
with that, getting them tested, gaining confidence in our ability to
manage that, then we should probably consider putting a db/backup
server there, or something similar.

3.  We are widely beloved, and it is not difficult for us to get very
fast offers of hosting on a temporary basis.  In this particular case,
the wonderful people at Oregon State University's Open Source Labs
offered us "3-4 hefty machines" available at a moment's notice.  I am
going to talk to them some more, but it seems likely that if we pay
the cost of a machine, they would be happy to host on a permanent
basis a slave db machine which we could keep synced to have a constant
realtime backup.

--Jimbo


from_name: Christiaan Briggs

subject: [WikiEN-l] When can an article be considered public domain?

body:
On 14 Aug 2004, at 6:39 am, Jim Cecropia wrote:

> First IANAL

Would such jargon stand up in a court of law?

Christiaan


body:
On 14 Aug 2004, at 6:39 am, Jim Cecropia wrote:

> To go to an extreme, you obviously cannot quote an entire article and
> then make some comments and call it fair use.

Could you quote the whole thing verbatim and sign it author unknown?

Christiaan


subject: [Wikitech-l] External diff engine

body:
Ivan Krstic wrote:
> Brion Vibber wrote:
>> Plain page view:    2798ms
>> Classic diff:       4856ms (2798ms + 2058ms)
>> Diff with wikidiff: 3342ms (2798ms +  544ms)
>
> That's a very long time. Is there any chance you can post some more
> fine-grained timings (perhaps profiling output) for these pages? What of
> the three seconds is network latency, what's overhead, and what's parsing?

Ah, you noticed it's kind of slow? :)

Here's an extract of the biggest offenders from the $wgProfiling output
for the with-memcached page view, which is the 'best case' with things
already cached out and no diff to do:

-total                         1 1481.058 1481.058 100.000%
Article::view                  1 1236.875 1236.875  83.513%
OutputPage::replaceLinkHolders 2 1213.474  606.737  81.933%
Setup.php                      1  161.677  161.677  10.916%
Title::secureAndSplit        752  122.670    0.163   8.283%
Skin::makeBrokenLinkObj      693   89.009    0.128   6.010%
OutputPage::output             1   78.290   78.290   5.286%

Numbers of calls, total time in ms, per-call time in ms, percentage of
total time. (Percentages do not add up to 100% because subroutine
execution time is included.)

Clearly there's still room for improvement in replaceLinkHolders.

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20040816/aad6e753/attachment.pgp>