from_name: Ray Saintonge

subject: [Foundation-l] Wikiversity=>Wikisophia

body:
Erik Moeller wrote:

> During and after our visit to the free software conference in South
> Africa, Angela and I talked a lot about the Wikiversity project, and
> the potential Wikimedia has to develop a truly global, free
> institution of learning as a new project.
>
> As you probably know, these discussions are currently focused on
>     http://meta.wikimedia.org/wiki/Wikiversity
> and the talk page.
>
> Don't panic: We're still a long way from launching anything. I'm not
> going to push this until we have the server situation under control
> and the existing projects have stabilized a bit. Nevertheless, there's
> one issue that I'd like to resolve now, which is the naming of the
> project.

Good, I think that a lot of groundwork needs to be laid before we get there.

> Angela, Jimbo, Daniel Mayer and I agree that the name Wikiversity is
> problematic in that it ties the project very strongly to the idea of
> traditional universities. This may lead to certain expectations as to
> its structure and the services it will provide (e.g. faculties,
> degrees), but also limit the project in other ways, e.g., by being
> perceived primarily or only as an institution of teritary learning.

I still prefer the term wikiversity.  I don't feel the strong
association with traditional universities, nor any of the other
perceptions that you have of it.  There was a time when university was
another name for a corporation.   What I find attractive about
"wikiversity" of turning toward a totality in a more classical approach
to learning.

> I'd like us to look at ideas for primary and secondary education as
> well, and I don't want to run into a wall because the established
> people of the Wikiversity community will say "It's an electronic
> university, this doesn't belong here."

Absolutely.  I believe that education is a seemless lifelong process.
Separate buildings for different levels of education makes sense in the
brick and mortar world.  Just as much as Wikipedia is not paper,
Wikiversity is not bricks and mortar.  Certainly there will be people to
raise the kind of objection that you mention, but I don't expect them to
be very influential.

> Angela suggested the name Wikisophia.org/.com, which is currently
> owned by Peter Danenberg (WikiTeX). I loved the idea immediately: the
> Greek sophia means "wisdom", but also has many other meanings in the
> area of
> learning. It is specific enough to be useful and vague enough to not
> limit the project very early in its nature or scope.

I find "sophia" to be a little pompous, and I also link it with the
concept of sophistry.  Teaching wisdom is a bit akin to teaching common
sense.  There is also the possibility of confusion with "Sophia
University", the Jesuit university in Japan.

> So, after discussing this in a small circle, I'd like to announce my
> intention to move the relevant pages on Meta and edit the summary to
> reflect the name change.

This action seems premature.

Ec


list: wikimedia-l

source: pipermail-mbox

subject: [Foundation-l] Wikiversity=>Wikisophia


list: wikitech-l

from_name: Brion Vibber

subject: [Wikitech-l] Longterm software strategy

body:
Lee Daniel Crocker wrote:
> On Mon, 2005-05-09 at 12:43 -0700, Brion Vibber wrote:
>>A possible middle road is to rewrite the core wiki engine to a separate
>>daemon, and adapt the existing PHP user interface to call into it for
>>much of the backend work that actually touches data.
>
>
> That's the approach I'd favor.  The existing PHP code represents a lot
> of good user-interface work for which PHP is perfectly suited.  The
> underlying stuff could easily be split up into multiple daemons (say,
> one for wikitext, one for images, one for equations,...) that could
> feed the PHP front-end.
>
> This would also allow incremental development, since each of the daemons
> could be written and attached indicidually without disturbing the rest
> of the codebase.

<nod>

>>Our biggest, nastiest burden is with internal communications: the
>>database changes too much. We have to wait on things getting sent,
>>received, applied, and copied around, and lagging databases send
>>everything into the toilet fast.
>
>
> Yep.  That's why I think the bulk of the text should be stored in a
> plain filesystem, where those problems are already well-known and
> solved, for the most part.  That would reduce the database to just
> metadata, which would be much smaller and more efficient.

Replication's still an issue with filesystems; it seems like every
network filesystem a) sucks and b) is a SPOF, and every distributed
filesystem a) sucks and b) sucks.

This is one reason we've been talking about using an external object
store rather than a filesystem. But a filesystem might work too; anyway
that's a separate issue.

The 1.5 schema separates the text storage backend from the revision
metadata for just this purpose; the metadata tables are smaller, and we
can more easily transition to putting bulk text elsewhere entirely. In
fact we can do a gradual transition by storing serialized reference
objects in the text table.

> Yes!  There's only one tricky part for which we may have to consider
> creative implementations: I tried as much as possible to take style
> markup (especially skin-specific) out of the rendered wikitext to
> allow it to be cached, but there's one case that's still a problem:
> red links (i.e., links to non-existent pages).  Users shouted at me
> that this was a sine-qua-non feature, and so I had to leave it in.
> But it makes caching rendered wikitext hard, and slows down rendering.

That doesn't really make output caching harder at all; you just follow
the link tables back a step and purge the affected pages.

Currently this is done 'inline' during save; we could return from
certain saves a little faster if we can shunt the purging to a
background daemon, and it could handle the occasional mass-change case
more gracefully than a giant 30,000-row update.

(There are two steps here: updating the page_touched timestamp on page
records, and sending PURGE requests to the squid proxies. The updated
touched time invalidates the parser cache and client-side cached output
on the next visit, while making explicit squid purges lets the squids
serve the cached pages they do have to anonymous visitors without having
to check with the master servers on every hit.)

At one point I had started to write a purge daemon to handle the squid
purge requests, but put it off after various improvements were made to
how they're handled which sped it up quite a bit. (Currently we're using
some kind of multicast thing, IIRC.)

Note that updates to template pages share roughly the same issue here,
exacerbated by the possibility that a changed template will contain
different links. Currently we make no attempt to update the link
references from pages containing a template when the template changes,
as I recall.

Somewhat different but worth mentioning is the updating of link tables
on save: we've tended to flip-flop between deleting and rewriting all
the rows, and ending up locking things during the save which turned
ugly. Should make sure those are maintained cleanly, swiftly, and
without too much pain.

[snip various other stuff mostly addressed by Tim & co]

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20050510/216aaed7/attachment.pgp>


from_name: Ævar Arnfjörð Bjarmason

subject: [Wikitech-l] Longterm software strategy

body:
> Making it a goal now ensures that we *can* take our time "doing it
> right," *before* it becomes an immediate need.  A comprehensive test
> suite is a must for this, and I know a lot of work has been put into
> improving the one we have over the past few weeks.  And, of course,
> there's still room for optimization in the current codebase.

Agreed, but our test suite isn't nearly as good as it should be, we
only have a few tests for each feature most of which test only basic
functionality of that feature as opposed to testing it in some
situations which are used "in the wild" and/or expected.

The parser tests are currently alot better than nothing, and if
something passes them it's certainly close to being a drop-in
replacement for the current parser, but they aren't quite perfect at
the moment.


subject: [Wikitech-l] Longterm software strategy

body:
> Making it a goal now ensures that we *can* take our time "doing it
> right," *before* it becomes an immediate need.  A comprehensive test
> suite is a must for this, and I know a lot of work has been put into
> improving the one we have over the past few weeks.  And, of course,
> there's still room for optimization in the current codebase.

Fully agreed, I just hope the whole doesn't disappear in some colossal
programming language war ( and that I haven't just started one right
now ).


subject: [Wikitech-l] Longterm software strategy

body:
> That's the approach I'd favor.  The existing PHP code represents a lot
> of good user-interface work for which PHP is perfectly suited.  The
> underlying stuff could easily be split up into multiple daemons (say,
> one for wikitext, one for images, one for equations,...) that could
> feed the PHP front-end.

The math module actually works alot like this at the moment, it's not
a daemon but it would be easy to make it one since it communicates
with the rest of the code through STDOUT and STDIN anyway.


subject: [Wikitech-l] Parser (was Re: Longterm hosting strategy)

body:
> like Reiserfs would probably speed it up, and also speed up access to
> the rest of the metadata in the database which would become much
> smaller.

How about something like a version control system, subversion for
example, I don't know how it would do speed wise for something like
this but with that you'd get

* Free version control ( as in we wouldn't have to write our own
custom system to keep track of versions )
* Versions stored as diffs

It does have some drawbacks of course, for one it would be a central
repository so it would suffer from the current problems of having a
central database for wikitext, but there are distributed version
control systems available ( arch, monotone ), something like this
should at least be considered and tested too.


from_name: Fredrik Johansson

subject: [Wikitech-l] Longterm software strategy

body:
On 5/10/05, Lee Daniel Crocker <lee at piclab.com> wrote:
>
> Yes!  There's only one tricky part for which we may have to consider
> creative implementations: I tried as much as possible to take style
> markup (especially skin-specific) out of the rendered wikitext to
> allow it to be cached, but there's one case that's still a problem:
> red links (i.e., links to non-existent pages).

> [...] Yet another is to optimize the hell out of discovering
> the simple existence of a page, so that it's not a bottleneck in
> rendering (say, by having a daemon that keeps a one-bit field for
> every page using a spell-checker data structure)

This seems like the best method.

Maybe then you could also send the cached text to the user along with
a separate list of which links should be red, using a simple
JavaScript to modify the HTML on the browser side? Not sure this would
be better than just modifying an existing cached version on the server
side though.

Fredrik


list: foundation-l

from_name: Erik Moeller

subject: [Foundation-l] Wikiversity=>Wikisophia

body:
Lee Daniel Crocker-
> I don't see how it's acceptable for us to agree to any conditions that
> affect our prerogatives for software use.  We need to use whatever
> software best suits our purposes.  WikiTeX is nice work, and we may
> very well end up using it or something similar.

Of course we're not going to install WikiTeX if we decide that there's a
better option, but that seems unlikely. WikiTeX is one of the most
requested features from our users. What has been holding up its setup
are not criticisms of its functionality, but security considerations. As
I see it, if we do this, the Board will ask Brion, as a paid Wikimedia
employee, to assist Peter in the security review and evaluation of
WikiTeX and if, and only if, it meets our needs and our strict security
requirements, it's going to be installed. Peter is working on a
chroot-jail based approach already, which should help to address the
concerns about possible shell escapes in certain LaTeX macros.

I would never want to make compromises about security or freedom of
choice. This is more a matter of exchanging favors within the limits of
sanity.

> Secondly, what's wrong with good old fashioned plain-English multiple
> word names for things like "Wikimedia Schools Project" or "Wikimedia
> Offline" or something?

Wikimedia is not an English language project, for starters. This makes
it necessary for English language derived names like Wiktionary to be
localized, which then leads to confusion when you go to a domain like
pl.wiktionary.org and end up on a site called Wikis?ownik. Of course, to
a certain extent, you will never be able to avoid this, but at least in
the Latin languages, you can strive for a name which doesn't require
localization. Wiktionary is not too bad because it's still a neologism,
whereas something like "Wikimedia Dictionary" would be much worse.

Secondly, if you want a fully descriptive name, this would make the name
unwieldy and impractical. "Wikimedia Computer Assisted Learning,
Teaching, Certification and Index of Resources" would obviously be
unacceptable. You can try making a nice acronym out of it, but then you
end up with the English language problem mentioned above.

A short and unique name is also useful for searching the project and
marketing it. That is why companies, including those working in
academia, come up with short and catchy names for their products.
Anything unsexy or unwieldy will not be found or linked as frequently as
something short and memorable. You can trust capitalism when it comes to
  the mechanisms of meme optimization.

Finally, consistency is important. We have an established naming scheme
-- our projects have unique names that usually begin with "Wiki-" -- and
in order to maintain our corporate identity, new projects should follow
the same pattern. (Incidentally, I named the "Wikimedia Commons" using a
different pattern because it is not a regular Wikimedia project, but an
umbrella project used by all others. Regardless, it is often abbreviated
as "Wikicommons", indicating a strong desire of users to have names
following this pattern..)

FWIW, the "cute" part of "Wikisophia" is not "-sophia", but "Wiki", and
we're pretty much stuck with that no matter what we do.

Erik


subject: [Foundation-l] Wikiversity=>Wikisophia


from_name: Delirium

subject: [Foundation-l] Wikiversity=>Wikisophia

body:
Ray Saintonge wrote:

> I find "sophia" to be a little pompous, and I also link it with the
> concept of sophistry.  Teaching wisdom is a bit akin to teaching
> common sense.

I mostly agree with this as well; I find "Wikisophia" a kind of
weird-looking name, and a bit pretentious.  I might be in the minority
(?), but I never interpreted "Wikiversity" as actually a university, but
in a more metaphorical sense---sort of like [[en:Wikipedia:Sandbox]].  I
wouldn't be averse to some other name I suppose, but I can't think of
one offhand that I like better.

-Mark


subject: [Foundation-l] Wikiversity=>Wikisophia


from_name: Daniel Mayer

subject: [Foundation-l] Re: Wikiversity=>Wikisophia

body:
--- Anthere <anthere9 at yahoo.com> wrote:
> I fear I must say the name wikiversity does speak much more to me, and
> wikisophia is confusing. I am not very keen on this name change.

OK - given that along with the other objections, I now withdraw my weak support
for the name change. All we need to do is say, up front, that we will not limit
Wikiversity to college level and above courses; that all levels of learning
will be allowed.

We should also invest in getting those security concerns for WikiTeX addressed.
Not to get a domain name, but because it makes sense to do so in light of the
very strong desire in users to have this functionality.

-- mav

__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail


body:
> Waaaay too slow (have you ever used Subversion?)  But it might not be
> a bad idea to put a WebDAV/DeltaV front end on whatever we create to
> make it possible for third-party tools to access it.

I've used it since early betas, however not in a production
enviroment, just my personal repository, so I haven't felt much need
for speed, however as David pointed out you might get more speed out
of fsfs than dbd and I just mentioned svn as an example, there are
more version control systems in the world.

Regardless, using a VCS would bring diff-based storage, and when it's
all said and done making a custom implementation of vcs-like features
might end up in something not much faster or even slower than a "real"
version control system.


list: wiktionary-l

from_name: Anthere

subject:
[Wiktionary-l] Re: Creation of Local Wiktionary mailing list for
 the PORTUGUESE wiktionary...

body:
hello Iuri

Until recently, meta user Fire took care of such creation. I am not sure
he does it anymore. I recommand you ask either on wikitech-l mailing
list, or ask user brion Vibber on meta.

Cheers

Ant

Iuriatan Muniz a ?crit:
> Hello... I didn't find a explicit explanation of the procedure to open
> a local wiktionary mailing list... But the fact is we of the
> Portuguese wiktionary are in need of such a list...
>
> Who can create the list ??? I could be the responsible, or moderator,
> or whatever is necessary...
>
> If i'm writing to the wrong address, sorry... =(
>
> []'s
>                                        Iuri


subject: [Foundation-l] Wikiversity

body:
>--- Anthere <anthere9 at yahoo.com> wrote:
>> I fear I must say the name wikiversity does speak much more to me, and
>> wikisophia is confusing. I am not very keen on this name change.

>OK - given that along with the other objections, I now withdraw my weak support
>for the name change. All we need to do is say, up front, that we will not limit
>Wikiversity to college level and above courses; that all levels of learning
>will be allowed.

>We should also invest in getting those security concerns for WikiTeX addressed.
>Not to get a domain name, but because it makes sense to do so in light of the
>very strong desire in users to have this functionality.

>-- mav

Hmmmm

Wikielearning might be a solution. E-learning has the benefit of being a rather framed word at least...

I must confess I did not give any feedback to Erik on the (short) naming discussion... essentially because it is unclear in my mind what the project is exactly about. And...  I thought Erik was asking first our opinion on the matter, then the discussion would come on the list... and I would have the opportunity to better make up my mind... or not make it up depending on people reaction... :-)

This said, aside from any naming issue...
*naming* should really be one of the *last* issue when thinking to create a project. Humm, with logo :-)
Most projects-to-be have a little temp name for a while.

It seems that we could let the envelop issue for later and discuss the core of the fruit first. It does not matter right now if we keep the working name later or change it really. What is potentially problematic is that giving a name to a project tends to set a delineation of it. If the delineation is not agreed upon, this is calling for troubles.

So, what is very important is first to agree on the project itself... what this project is about... who will be the audience... what the goals are... the means required...

It seems not everyone is in agreement about what the project is about... I am not sure I understood well where the disagreement is and I would love to hear more about it. *Before* any vote of any sort. Vote should only be the last resort.

ant

---------------------------------
Yahoo! Mail
 Stay connected, organized, and protected. Take the tour


subject: [Foundation-l] Wikiversity


list: wikien-l

from_name: Violet/Riga

subject: [WikiEN-l] Usernames advertising websites

body:
If it's a commercial website then it's dodgy.  If it's a personal one
then I don't think it's so bad.  There's certainly a gap in the
username policy.

~~~~ Violet/Riga

On 5/10/05, blankfaze <blankfaze at gmail.com> wrote:
> I have just noticed http://en.wikipedia.org/wiki/User:GordonWattsDotCom ...
> It seems to me to be somewhat inappropriate to create a username advertising
> a personal website, but I didn't see anything in the Wikipedia username
> policy ( http://en.wikipedia.org/wiki/Wikipedia:Username ) that says
> anything about adverts...
>
> So I thought I'd pose the issue to the mailing list, see what you all think.


list: wikipedia-l

from_name: Mark Williamson

subject: [Wikipedia-l] North Frisian wikipedia - arbitrary mention

body:
I am under the impression that at least a couple of them are not
elderly, and that there are definitely for example university
professors who speak teh language fluently.

In addition, I was sure these people spoke German fluently. Why
couldn't you start with a German interface, like the Bambara wikipedia
started with a French interface?

Mark

On 10/05/05, Wouter Steenbeek <musiqolog at hotmail.com> wrote:
> >Whatever success you have had, you have not indicated.
> >
> >I know from experience that if people are serious, they will be
> >willing to work on it right away - the people who say "I may work on
> >it someday" or "I will work on it after so-and-so date" are the people
> >who end up never helping.
> >
> >Regardless of who speaks it, it has been much longer than "one week",
> >and there has been 0 signs of activity. I don't expect a
> >thousand-article Wikipedia, but I think the community expects some
> >measure of success (at least 10 or so good articles by native
> >speakers) or at least one on-list endorsement by a fluent speaker (if
> >you have more than one, people will listen more).
> >
> >It has been a long time. I think - and I'm sure the community agrees,
> >or feels more strict than do I - that you should withhold a request
> >for a North Frisian Wikipedia until you can meet at least one of the
> >things I've outlined above.
> >
> >If these people are motivated, you should be able to get them to
> >support your Seeltersk request on-list, or add content to the test WP.
>
> Mark,
> I am talking about elderly people, who are -thank God- connected to the
> internet, but read no English (yes, you're reading it rightly!) and have no
> experience with wikis or whatsoever. So I have to translate the interface
> and other structure files and explain how how it all works before they can
> show their existence on wikimedia at all.
>
> W.
>
> _________________________________________________________________
> Speel online games via MSN Messenger http://messenger.msn.nl/
>
> _______________________________________________
> Wikipedia-l mailing list
> Wikipedia-l at Wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikipedia-l
>

--
SI HOC LEGERE SCIS NIMIVM ERVDITIONIS HABES
QVANTVM MATERIAE MATERIETVR MARMOTA MONAX SI MARMOTA MONAX MATERIAM
POSSIT MATERIARI
ESTNE VOLVMEN IN TOGA AN SOLVM TIBI LIBET ME VIDERE


subject:
no PD in France (was: Re: [Wikipedia-l] Wikipedia, Emergence,
	and The Wisdom of Crowds)

body:
Wow...

That is totally... strange.

On 10/05/05, Jean-Baptiste Soufron <jbsoufron at gmail.com> wrote:
> >>>
> >>> A simpler response would be to cite the clause in the statute that
> >>> says this.  To me a fundamental principal of law is that anything
> >>> which is not specifically forbidden is allowed.
> >>
> >> We're talking about emergence here... what do you mean ? ? ?
> >
> > The statement (which was cut off at some point) was about your guess
> > that French law did not allow a work to be put into the public domain.
> >  Citing a specific passage in the French statutes would be much more
> > effective than speculating about the law.
>
> Here it is:
>
> The article L-121-1 explains that moral rights are : "perp?tuel,
> inali?nable et imprescriptible", it lasts forever, they can't renounce
> to it and they can sue even years after a copyvio.
>
> And when talking about patrimonial rights, there are various provisions
> implying that the author cannot contribute his work to PD : L-122-7
> precises very strong formalities for any release of his representation
> or reproduction rights, L-122-8 precises that authors possess a forever
> right to get 3% of any auction of their work and that they can't
> renounce to it.
>
> But more importantly, L-131-2 states that : "Les contrats de
> repr?sentation, d'?dition et de production audiovisuelle d?finis au
> pr?sent titre doivent ?tre constat?s par ?crit. Il en est de m?me des
> autorisations gratuites d'ex?cution" which means that any contract to
> sell their representation/edition rights must be a written one, even
> when they authorizes free use of their work !
>
> Article L-131-5 even precises that if the author released his rights
> but that he got more than 7/12 less than what he should have been
> normally awarded, he can sue to get it.
>
> etc. etc. etc.
>
> _______________________________________________
> Wikipedia-l mailing list
> Wikipedia-l at Wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikipedia-l
>

--
SI HOC LEGERE SCIS NIMIVM ERVDITIONIS HABES
QVANTVM MATERIAE MATERIETVR MARMOTA MONAX SI MARMOTA MONAX MATERIAM
POSSIT MATERIARI
ESTNE VOLVMEN IN TOGA AN SOLVM TIBI LIBET ME VIDERE


from_name: Phil Boswell

subject: [WikiEN-l] Re: RE: Corrected

body:
<slimvirgin at gmail.com> wrote in message
news:4cc603b0505051626226b944d at mail.gmail.com...
> On 5/5/05, Stan Shebs <shebs at apple.com>
> wrote:
> > I would say anybody who posts to Usenet automatically becomes
> > a "public figure"; you can hardly spew your thoughts and opinions
> > into a million computers around the world, then try to claim
> > "privacy". If you want to be private, start by keeping your
> > mouth shut, eh?
> >
> It's an interesting point, whether sitting in your own home typing
> material onto the Web should in and of itself make you a public
> figure. I'm not aware of any relevant case law. Using your real name
> would certainly make it harder to argue that you'd intended to retain
> your privacy.

So adding your full name, address and contact details to the end of each and
every post, as is Mr Wollmann's habit, would seem to fit this description:
this guy has been doing his darndest to *become* a public figure for years.
--
Phil
[[en:User:Phil Boswell]]


from_name: Tony Sidaway

subject: [WikiEN-l] Re: RE: Corrected

body:
Phil Boswell said:
>
> So adding your full name, address and contact details to the end of
> each and  every post, as is Mr Wollmann's habit, would seem to fit this
> description:  this guy has been doing his darndest to *become* a public
> figure for years. --

The public/private distinction only applies in US jurisdictions, and with
limited effect.  Do not assume that you can say just anything about a
person just because he publishes his address.


from_name: Wouter Steenbeek

body:
>Your "success" with Seeltersk?? Are you kidding? The Test Wikipedia
>has 0 articles, and there is no hint of support from native speakers.

Did I claim I had success in /building/ a wikipedia? I meant I had success
in contacting scolars and, yes, in attracting native speakers. You expected
me to have already built something, but that was too optimistic. With such a
small language, spoken particularly by elder people, you can't expect the
rise of a wikipedia within one week. Moreover, I was quite busy over the
last months, and only recently the first real efforts towards building a
Seeltersk wikipedia were done at all. If I intended tio say that I had
success with /building/ a Seeltersk test-wp, I'd have said that.

Wouter

_________________________________________________________________
Nieuw: Download nu MSN Messenger 7.0 http://messenger.msn.nl/


body:
>Whatever success you have had, you have not indicated.
>
>I know from experience that if people are serious, they will be
>willing to work on it right away - the people who say "I may work on
>it someday" or "I will work on it after so-and-so date" are the people
>who end up never helping.
>
>Regardless of who speaks it, it has been much longer than "one week",
>and there has been 0 signs of activity. I don't expect a
>thousand-article Wikipedia, but I think the community expects some
>measure of success (at least 10 or so good articles by native
>speakers) or at least one on-list endorsement by a fluent speaker (if
>you have more than one, people will listen more).
>
>It has been a long time. I think - and I'm sure the community agrees,
>or feels more strict than do I - that you should withhold a request
>for a North Frisian Wikipedia until you can meet at least one of the
>things I've outlined above.
>
>If these people are motivated, you should be able to get them to
>support your Seeltersk request on-list, or add content to the test WP.

Mark,
I am talking about elderly people, who are -thank God- connected to the
internet, but read no English (yes, you're reading it rightly!) and have no
experience with wikis or whatsoever. So I have to translate the interface
and other structure files and explain how how it all works before they can
show their existence on wikimedia at all.

W.

_________________________________________________________________
Speel online games via MSN Messenger http://messenger.msn.nl/


from_name: Brent 'Dax' Royal-Gordon

subject: [Wikitech-l] [patch] Category headers change

body:
The attached patch (6KB, unified diff) against CategoryPage.php in
MediaWiki 1.4.4 changes the way headings are inserted into category
pages.

Previously, each letter would get its own heading.  This was a pet
peeve of mine, as on some pages you'd get several headings with only
one item:

A
* Apple
B
* Banana
F
* Food
* Fruit
M
* Melon
O
* Orange
T
* Tangerine

The patch combines consecutive sections with only one item in them,
and rewrites the header to the form "A–B".  So with the patch
applied, the above would be more like:

A-B
* Apple
* Banana
F
* Food
* Fruit
M-T
* Melon
* Orange
* Tangerine

The patch also changes how pages are inserted into each column.
Previously, categories with six pages or less were given one column,
while those with more than six had three columns; pages were then
divided evenly between the three columns.  With this change, all
categories (notionally) have three columns, but each column is
guaranteed at least six pages before the next column gets any.  The
columns are also represented as fixed-width <div>s instead of tables;
this could be trivially changed.

The patch is 6KB, likely mainly because I'm not very familiar with
PHP; it would probably need to be made more idiomatic before it could
be accepted.  It's in use on one of my wikis if you want to see the
result:

    <http://mt.brentdax.com/wiki/Special:Categories>

--
Brent 'Dax' Royal-Gordon <brent at brentdax.com>
Perl and Parrot hacker


from_name: Cormac Lawler

subject: [Foundation-l] Wikiversity=>Wikisophia

body:
On 5/10/05, Ray Saintonge <saintonge at telus.net> wrote:
> Erik Moeller wrote:
>
> > During and after our visit to the free software conference in South
> > Africa, Angela and I talked a lot about the Wikiversity project, and
> > the potential Wikimedia has to develop a truly global, free
> > institution of learning as a new project.
> >
> > As you probably know, these discussions are currently focused on
> >     http://meta.wikimedia.org/wiki/Wikiversity
> > and the talk page.
> >
> > Don't panic: We're still a long way from launching anything. I'm not
> > going to push this until we have the server situation under control
> > and the existing projects have stabilized a bit. Nevertheless, there's
> > one issue that I'd like to resolve now, which is the naming of the
> > project.
>
> Good, I think that a lot of groundwork needs to be laid before we get there.
>
> > Angela, Jimbo, Daniel Mayer and I agree that the name Wikiversity is
> > problematic in that it ties the project very strongly to the idea of
> > traditional universities. This may lead to certain expectations as to
> > its structure and the services it will provide (e.g. faculties,
> > degrees), but also limit the project in other ways, e.g., by being
> > perceived primarily or only as an institution of teritary learning.
>
> I still prefer the term wikiversity.  I don't feel the strong
> association with traditional universities, nor any of the other
> perceptions that you have of it.  There was a time when university was
> another name for a corporation.   What I find attractive about
> "wikiversity" of turning toward a totality in a more classical approach
> to learning.
>
> > I'd like us to look at ideas for primary and secondary education as
> > well, and I don't want to run into a wall because the established
> > people of the Wikiversity community will say "It's an electronic
> > university, this doesn't belong here."
>
> Absolutely.  I believe that education is a seemless lifelong process.
> Separate buildings for different levels of education makes sense in the
> brick and mortar world.  Just as much as Wikipedia is not paper,
> Wikiversity is not bricks and mortar.  Certainly there will be people to
> raise the kind of objection that you mention, but I don't expect them to
> be very influential.
>
> > Angela suggested the name Wikisophia.org/.com, which is currently
> > owned by Peter Danenberg (WikiTeX). I loved the idea immediately: the
> > Greek sophia means "wisdom", but also has many other meanings in the
> > area of
> > learning. It is specific enough to be useful and vague enough to not
> > limit the project very early in its nature or scope.
>
> I find "sophia" to be a little pompous, and I also link it with the
> concept of sophistry.  Teaching wisdom is a bit akin to teaching common
> sense.  There is also the possibility of confusion with "Sophia
> University", the Jesuit university in Japan.
>
> > So, after discussing this in a small circle, I'd like to announce my
> > intention to move the relevant pages on Meta and edit the summary to
> > reflect the name change.
>
> This action seems premature.
>
> Ec
>

How about Wikilearning? Or Wikiducation? Another idea could be to have
a parallel project to Wikiversity and call it Wikischools.

I'd like to get involved - have long had it in mind - and will try to
give some practical input when I'm not so busy with my own studies. I
have some experience and knowledge that might be useful.

Cormac


subject: [Foundation-l] Wikiversity=>Wikisophia


from_name: coro at pcmagic.net

subject: [WikiEN-l] Re: "block" considered too harsh a word?

body:
If it is only for a day or two, 'Time Out' with an explanation might do the
job.

Timwi writes:

>
> As we have seen numerous times in the past, users who are new to the
> Wikipedia tend to vastly overreact to being blocked. Some people throw
> quite a fit even when they're blocked for a mere 24 hours.
>
> A friend of mine suggested that perhaps the word should be changed to
> something more euphemistic and something less harsh than "block". I can
> sympathise with this thinking, because if I saw a page telling me I'm
> "blocked", not knowing what it means I would probably interpret it to mean
> "blocked indefinitely", and as we know people don't tend to read the
> entire message and hence don't notice where it mentions the time limit.
>
> Unfortunately, he didn't have any ideas what to call it instead. So I'm
> wondering what everyone else here thinks?
>
>


subject: [WikiEN-l] "block" considered too harsh a word?

body:
Timwi wrote:

> As we have seen numerous times in the past, users who are new to the
> Wikipedia tend to vastly overreact to being blocked. Some people throw
> quite a fit even when they're blocked for a mere 24 hours.
>
> A friend of mine suggested that perhaps the word should be changed to
> something more euphemistic and something less harsh than "block". I
> can sympathise with this thinking, because if I saw a page telling me
> I'm "blocked", not knowing what it means I would probably interpret it
> to mean "blocked indefinitely", and as we know people don't tend to
> read the entire message and hence don't notice where it mentions the
> time limit.

It isn't a question of what you name something.  As long as a newbie
isn't causing damage or otherwise acting in a hostile way, there should
be some latitude given.  Before any blocking takes place there should be
good faith attempts to contact and reason with him.  It may be
frustrating to see today's newbie do the same stupid things as last
week's newbie, but one cannot assume that he saw the warnings that were
given lto ast week's newbie.

A sysop who ignores this and peremptorily and impatiently blocks a
newbie deserves the same treatment that he would mete out to the newbie.

Ec


from_name: Lee Daniel Crocker

subject: [Wikitech-l] Re: Longterm software strategy

body:
On Tue, 2005-05-10 at 13:01 +0000, Dori wrote:
> On 5/10/05, Tim Starling <t.starling at physics.unimelb.edu.au> wrote:
> >
> > Lee Daniel Crocker wrote:
> > > Another thing to consider: at least some of the wikipedia-driven
> > > development will be totally unnecessary for mediawiki as a general-
> > > purpose open source project. We may want to decouple those projects
> > > at some point.
> >
> > Brion doesn't want to.
> >
> > -- Tim Starling
>
>
> We've gotten pretty good debugging/testing because they're coupled though
> right?

That's probably true; leaving them coupled just means that there will
need to be more configuration work, because there will certainly be
configurations needed for the largest wiki on the planet that would
be a nuisance to any other wiki.  Testing is a good argument in favor,
though.

--
Lee Daniel Crocker <lee at piclab.com>  <http://www.piclab.com/lee/>
<http://creativecommons.org/licenses/publicdomain/>


subject: [Wikitech-l] Longterm software strategy

body:
On 5/9/05, Brion Vibber <brion at pobox.com> wrote:
> There are a couple possibilities here. One is to rewrite MediaWiki in
> some other language entirely, say Python or C# or whathaveyou.
...
> In particular, PHP tends to impose an architecture where each request is
> served by an entirely new script invocation: you have to build any
> information up from scratch on each hit, and sharing things like
> localization tables between invocations is kind of hard. In another
> language it might be easier to run MediaWiki as a standlone server,
> which can keep shared data in memory and use it transparently from each
> thread.

Blaming the language is rarely a productive way to fix such a problem.
 In particular, Python usually uses the same initialize-and-run model
you complain about PHP using, and mod_mono doesn't seem to be widely
used.  (And I assume Wikimedia has no interest in switching to Windows
servers.)

Most reasonably mature languages are fast enough that the performance
bottlenecks are usually in user code.  And at least one of the PHP
problems--lack of a JIT--will be solved when PHP-on-Parrot is
available.  (There's at least one project to do this; the interpreter
itself is already quite fast and has JITs for several platforms,
although much of it still has to be written.)

> The other is to keep refactoring the PHP codebase (and it's been much
> changed since you left it, Lee) and, optionally, rewrite particular
> hotspots in another language.

I like the idea of porting hotspots, but keep in mind that we want
people to be able to use this even if they don't have access to a C
compiler.

> A possible middle road is to rewrite the core wiki engine to a separate
> daemon, and adapt the existing PHP user interface to call into it for
> much of the backend work that actually touches data.

I like this idea, although it carries its own costs.  In particular,
communicating between processes has inherent overhead; we'd have to be
reasonably sure that the caching would outweigh that.  Also note that
IPC and process-management mechanisms tend to vary across operating
systems; we might lose Windows support, for example.

One advantage of this split is that we could rewrite one of the
components in another language if we want, without affecting the other
one. If the front end becomes little more than a shell around a
daemon, we could provide a version written in C as an Apache module,
which is about as fast as it gets.

On the back end, Perl 6 is specced to have one of the most powerful
pattern-matching engines ever shipped with a language; it should be
able to eat wikicode for breakfast.  It's also designed to allow easy
interoperability with other languages, so just the wikicode parser
could be written in it while the rest is left as PHP.  Implementation
is moving quickly, with the backend Parrot Grammar Engine in progress
and the Pugs (Perl 6 in Haskell) compiler just starting on the syntax.
 (And, of course, you'd have a very happy Perl hacker here.)

--
Brent 'Dax' Royal-Gordon <brent at brentdax.com>
Perl and Parrot hacker


from_name: Kevin Carillo

subject: [Wikitech-l] Wikipedia and mysql 4.1

body:
Hi.
I'd like to know whether it is possible to run Wikipedia with mysql 4.1.
Before installing the whole environment with mysql 4.0.20a, I tried to set
it up with mysql 4.1 and I remember having problems installing mediawiki
which did not accept this version of mysql.
However, my research project requires a lot of queries that contain
subqueries, and it is almost impossible to rewrite them all as JOINS (one of
the reason is that I not only use SELECT queries but also UPDATE and
DELETE). Using Mysql 4.1 would help me considerably.

In case, it is possible to use mysql 4.1, is there any way to upgrade the
version of mysql by keeping my current wikipedia database?
Thank you.

Kevin Carillo

  _____

From: wikitech-l-bounces at wikimedia.org
[mailto:wikitech-l-bounces at wikimedia.org] On Behalf Of
wikitech-l at wikimedia.org
Sent: May 10, 2005 4:34 AM
To: wikitech-l at wikimedia.org
Subject: Wikitech-l Digest, Vol 22, Issue 26
Importance: Low

Send Wikitech-l mailing list submissions to
        wikitech-l at wikimedia.org <mailto:>

To subscribe or unsubscribe via the World Wide Web, visit
        http://mail.wikipedia.org/mailman/listinfo/wikitech-l
or, via email, send a message with subject or body 'help' to
        wikitech-l-request at wikimedia.org <mailto:>

You can reach the person managing the list at
        wikitech-l-owner at wikimedia.org <mailto:>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Wikitech-l digest..."

Today's Topics:

   1. Re: Parser (was Re: Longterm hosting strategy) (Tim Starling)
   2. Re: Parser (was Re: Longterm hosting strategy)
      (Lee Daniel Crocker)
   3. Re: Longterm software strategy (Tim Starling)
   4. Re: Parser (was Re: Longterm hosting strategy)
      (David A. Desrosiers)
   5. Re: Parser (was Re: Longterm hosting strategy)
      (?var Arnfj?r? Bjarmason)
   6. New machines installed, killed in record 9.5 hours (Brion Vibber)
   7. Re: Longterm software strategy (Brion Vibber)
   8. Link table updates (was Re: Longterm software strategy)
      (Tim Starling)

----------------------------------------------------------------------

Message: 1
Date: Tue, 10 May 2005 13:55:48 +1000
From: Tim Starling <t.starling at physics.unimelb.edu.au <mailto:> >
Subject: [Wikitech-l] Re: Parser (was Re: Longterm hosting strategy)
To: wikitech-l at wikimedia.org <mailto:>
Message-ID: <d5pau9$7q4$1 at sea.gmane.org <mailto:> >
Content-Type: text/plain; charset=ISO-8859-1

Lee Daniel Crocker wrote:
> I agree, I don't think the parser's a big issue, although it would be
> nice for a bit snappier response.  In hindsight, storing the wikitext in
> a database was a mistake.  There's already a wonderful piece of software
> highly optimized and scalable for storing randomly accessed variable-
> sized chunks of text with lots of tools for backup, replication, and so
> on; it's called a file system.  Storing the wikitext itself in something
> like Reiserfs would probably speed it up, and also speed up access to
> the rest of the metadata in the database which would become much
> smaller.

That's what ExternalStore is for. Moving the bulk out of the database,
or at least to a different database, is a pressing need. We need to
separate bulk, rarely accessed data from hot data, so that we can save
the highly redundant storage on the DB master for hot data. Domas has
been working on it. We're running out of disk space on Ariel again, and
another compression round is obviously only a stopgap solution.

-- Tim Starling

------------------------------

Message: 2
Date: Mon, 09 May 2005 21:04:55 -0700
From: Lee Daniel Crocker <lee at piclab.com <mailto:> >
Subject: Re: [Wikitech-l] Parser (was Re: Longterm hosting strategy)
To: ?var Arnfj?r? Bjarmason <avarab at gmail.com <mailto:> >, Wikimedia
developers
        <wikitech-l at wikimedia.org <mailto:> >
Message-ID: <1115697895.5779.25.camel at shuttle.piclab.com <mailto:> >
Content-Type: text/plain; charset=utf-8

On Tue, 2005-05-10 at 03:44 +0000, Cvar ArnfjC6rC0 Bjarmason wrote:
> > like Reiserfs would probably speed it up, and also speed up access to
> > the rest of the metadata in the database which would become much
> > smaller.
>
> How about something like a version control system, subversion for
> example, I don't know how it would do speed wise for something like
> this but with that you'd get

Waaaay too slow (have you ever used Subversion?)  But it might not be
a bad idea to put a WebDAV/DeltaV front end on whatever we create to
make it possible for third-party tools to access it.

--
Lee Daniel Crocker <lee at piclab.com <mailto:> >
<http://creativecommons.org/licenses/publicdomain/>

------------------------------

Message: 3
Date: Tue, 10 May 2005 14:35:03 +1000
From: Tim Starling <t.starling at physics.unimelb.edu.au <mailto:> >
Subject: [Wikitech-l] Re: Longterm software strategy
To: wikitech-l at wikimedia.org <mailto:>
Message-ID: <d5pd7q$cie$1 at sea.gmane.org <mailto:> >
Content-Type: text/plain; charset=ISO-8859-1

Lee Daniel Crocker wrote:
> Yes!  There's only one tricky part for which we may have to consider
> creative implementations: I tried as much as possible to take style
> markup (especially skin-specific) out of the rendered wikitext to
> allow it to be cached, but there's one case that's still a problem:
> red links (i.e., links to non-existent pages).  Users shouted at me
> that this was a sine-qua-non feature, and so I had to leave it in.
> But it makes caching rendered wikitext hard, and slows down rendering.
> One alternative is to simply tolerate them being out of date for the
> life of the cache.  Another is to possibly update the cache in some
> cheaper way.  Yet another is to optimize the hell out of discovering
> the simple existence of a page, so that it's not a bottleneck in
> rendering (say, by having a daemon that keeps a one-bit field for
> every page using a spell-checker data structure)

We already optimised it, didn't we? In the last public profiling run:

http://meta.wikimedia.org/wiki/Profiling/20050328

....it came in at 2.6% for the non-stub bundled query and 0.4% for the
stub query. I'd hardly call that a bottleneck. Individual link existence
tests came in at 5.9%, mostly due to special pages, but I've largely
fixed that in 1.5 by bundling the existence tests for commonly requested
special pages. It wasn't so long ago it was taking 15% for individual
queries and 15% for LinkCache::preFill():

http://meta.wikimedia.org/wiki/Profiling/Live_aggregate_20040604

....so we've come a long way.

> I'm all for your method, and I agree it's not an urgent need.  But I
> think we can slip the timeline even more.  The existing codebase will
> eventually be a liability, but I think we can throw hardware at it for
> a year or two.  Also, if we go the route of making independent daemons
> linked into the existing UI code, we don't have to deploy all at once.
> We could, for example, make and deploy the math daemon as a proof-of-
> concept, work out bugs with that, then do the others afterward.

We've already got two proof-of-concept daemons: the Chinese word
segmenter and Lucene.

There is a technical problem with Lucene at the moment: it uses file()
to fetch the result over HTTP, but that has an unconfigurable 3 minute
timeout. If the search daemon goes down, we hit apache connection limits
within a minute and the site stops working. We can either patch PHP to
use default_socket_timeout in this case, or switch to another method
like DIY pfsockopen or curl.

> Another thing to consider: at least some of the wikipedia-driven
> development will be totally unnecessary for mediawiki as a general-
> purpose open source project.  We may want to decouple those projects
> at some point.

Brion doesn't want to.

-- Tim Starling

------------------------------

Message: 4
Date: Tue, 10 May 2005 00:50:18 -0400 (EDT)
From: "David A. Desrosiers" <desrod at gnu-designs.com <mailto:> >
Subject: Re: [Wikitech-l] Parser (was Re: Longterm hosting strategy)
To: Wikimedia developers <wikitech-l at wikimedia.org <mailto:> >
Message-ID: <Pine.LNX.4.62.0505100049060.8023 at angst.gnu-designs.com
<mailto:> >
Content-Type: TEXT/PLAIN; charset=US-ASCII

> Waaaay too slow (have you ever used Subversion?)  But it might not
> be a bad idea to put a WebDAV/DeltaV front end on whatever we create
> to make it possible for third-party tools to access it.

        Only if you're using it with the default (and horribly slow)
bdb backend. If you use fsfs, you'll see performance several orders of
magnitude faster (and it also doesn't wedge or break like bdbd does
all the time).

        http://svn.collab.net/repos/svn/trunk/notes/fsfs

David A. Desrosiers
desrod at gnu-designs.com <mailto:>
http://gnu-designs.com

------------------------------

Message: 5
Date: Tue, 10 May 2005 07:22:10 +0000
From: ?var Arnfj?r? Bjarmason <avarab at gmail.com <mailto:> >
Subject: Re: [Wikitech-l] Parser (was Re: Longterm hosting strategy)
To: Lee Daniel Crocker <lee at piclab.com <mailto:> >
Cc: Wikimedia developers <wikitech-l at wikimedia.org <mailto:> >
Message-ID: <51dd1af805051000224ffb81c7 at mail.gmail.com <mailto:> >
Content-Type: text/plain; charset=ISO-8859-1

> Waaaay too slow (have you ever used Subversion?)  But it might not be
> a bad idea to put a WebDAV/DeltaV front end on whatever we create to
> make it possible for third-party tools to access it.

I've used it since early betas, however not in a production
enviroment, just my personal repository, so I haven't felt much need
for speed, however as David pointed out you might get more speed out
of fsfs than dbd and I just mentioned svn as an example, there are
more version control systems in the world.

Regardless, using a VCS would bring diff-based storage, and when it's
all said and done making a custom implementation of vcs-like features
might end up in something not much faster or even slower than a "real"
version control system.

------------------------------

Message: 6
Date: Tue, 10 May 2005 00:54:47 -0700
From: Brion Vibber <brion at pobox.com <mailto:> >
Subject: [Wikitech-l] New machines installed, killed in record 9.5
        hours
To: Wikimedia developers <wikitech-l at wikimedia.org <mailto:> >
Message-ID: <428068C7.30203 at pobox.com <mailto:> >
Content-Type: text/plain; charset="iso-8859-1"

All the new boxen (srv11-srv30) died mysteriously while Domas and I were
trying to restart Apache after installing PHP's CURL library extension
so a proper timeout could be used on the Lucene search.

By dead, I mean "Destination Host Unreachable". They're off the network,
kaput. That _shouldn't_ happen. :) All the other machines seem just
fine; only the spanking new ones exploded, and the reason for it is not
too clear. (Freak library incompatibility -> killing machines? That
_shouldn't_ happen.)

It may be necessary for somebody to flip the switches and reboot.

-- brion vibber (brion @  <mailto:> pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url :
http://mail.wikipedia.org/pipermail/wikitech-l/attachments/20050510/053dd5bf
/signature-0001.bin

------------------------------

Message: 7
Date: Tue, 10 May 2005 02:19:00 -0700
From: Brion Vibber <brion at pobox.com <mailto:> >
Subject: Re: [Wikitech-l] Longterm software strategy
To: Wikimedia developers <wikitech-l at wikimedia.org <mailto:> >
Message-ID: <42807C84.1080505 at pobox.com <mailto:> >
Content-Type: text/plain; charset="iso-8859-1"

Lee Daniel Crocker wrote:
> On Mon, 2005-05-09 at 12:43 -0700, Brion Vibber wrote:
>>A possible middle road is to rewrite the core wiki engine to a separate
>>daemon, and adapt the existing PHP user interface to call into it for
>>much of the backend work that actually touches data.
>
>
> That's the approach I'd favor.  The existing PHP code represents a lot
> of good user-interface work for which PHP is perfectly suited.  The
> underlying stuff could easily be split up into multiple daemons (say,
> one for wikitext, one for images, one for equations,...) that could
> feed the PHP front-end.
>
> This would also allow incremental development, since each of the daemons
> could be written and attached indicidually without disturbing the rest
> of the codebase.

<nod>

>>Our biggest, nastiest burden is with internal communications: the
>>database changes too much. We have to wait on things getting sent,
>>received, applied, and copied around, and lagging databases send
>>everything into the toilet fast.
>
>
> Yep.  That's why I think the bulk of the text should be stored in a
> plain filesystem, where those problems are already well-known and
> solved, for the most part.  That would reduce the database to just
> metadata, which would be much smaller and more efficient.

Replication's still an issue with filesystems; it seems like every
network filesystem a) sucks and b) is a SPOF, and every distributed
filesystem a) sucks and b) sucks.

This is one reason we've been talking about using an external object
store rather than a filesystem. But a filesystem might work too; anyway
that's a separate issue.

The 1.5 schema separates the text storage backend from the revision
metadata for just this purpose; the metadata tables are smaller, and we
can more easily transition to putting bulk text elsewhere entirely. In
fact we can do a gradual transition by storing serialized reference
objects in the text table.

> Yes!  There's only one tricky part for which we may have to consider
> creative implementations: I tried as much as possible to take style
> markup (especially skin-specific) out of the rendered wikitext to
> allow it to be cached, but there's one case that's still a problem:
> red links (i.e., links to non-existent pages).  Users shouted at me
> that this was a sine-qua-non feature, and so I had to leave it in.
> But it makes caching rendered wikitext hard, and slows down rendering.

That doesn't really make output caching harder at all; you just follow
the link tables back a step and purge the affected pages.

Currently this is done 'inline' during save; we could return from
certain saves a little faster if we can shunt the purging to a
background daemon, and it could handle the occasional mass-change case
more gracefully than a giant 30,000-row update.

(There are two steps here: updating the page_touched timestamp on page
records, and sending PURGE requests to the squid proxies. The updated
touched time invalidates the parser cache and client-side cached output
on the next visit, while making explicit squid purges lets the squids
serve the cached pages they do have to anonymous visitors without having
to check with the master servers on every hit.)

At one point I had started to write a purge daemon to handle the squid
purge requests, but put it off after various improvements were made to
how they're handled which sped it up quite a bit. (Currently we're using
some kind of multicast thing, IIRC.)

Note that updates to template pages share roughly the same issue here,
exacerbated by the possibility that a changed template will contain
different links. Currently we make no attempt to update the link
references from pages containing a template when the template changes,
as I recall.

Somewhat different but worth mentioning is the updating of link tables
on save: we've tended to flip-flop between deleting and rewriting all
the rows, and ending up locking things during the save which turned
ugly. Should make sure those are maintained cleanly, swiftly, and
without too much pain.

[snip various other stuff mostly addressed by Tim & co]

-- brion vibber (brion @ pobox.com <mailto:> )
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url :
http://mail.wikipedia.org/pipermail/wikitech-l/attachments/20050510/216aaed7
/signature-0001.bin

------------------------------

Message: 8
Date: Tue, 10 May 2005 20:32:11 +1000
From: Tim Starling <t.starling at physics.unimelb.edu.au <mailto:> >
Subject: [Wikitech-l] Link table updates (was Re: Longterm software
        strategy)
To: wikitech-l at wikimedia.org <mailto:>
Message-ID: <d5q25i$7he$1 at sea.gmane.org <mailto:> >
Content-Type: text/plain; charset=ISO-8859-1

Brion Vibber wrote:
> Somewhat different but worth mentioning is the updating of link tables
> on save: we've tended to flip-flop between deleting and rewriting all
> the rows, and ending up locking things during the save which turned
> ugly. Should make sure those are maintained cleanly, swiftly, and
> without too much pain.

Currently we're not locking, it's patched live to do a non-locking read
and then an incremental update. So the site keeps running, but
inconsistencies will inevitably be introduced.

The problem with locking reads turns out to be a bug in InnoDB. Well,
I'm calling it a bug anyway. The problem is that if you select all links
from a particular article for update, every link table row linking to
any of those pages is locked. So if one person is updating a stub,
nobody else can update a stub. This is easily repeatable in a controlled
situation with the following test. With two threads labelled T1 and T2:

T1> create table ltest (f int, t int, unique key (f,t), key t(t))
type=innodb;
T1> insert into ltest values (1,10),(2,10),(3,10);
T1> begin;
T1> select * from ltest where f=1 for update;

T2> begin;
T2> select * from ltest where f=2 for update;
(T2 waits for T1)

Last time I reported a locking problem to the MySQL bug tracker, I was
told that it wasn't a bug, and that I just needed to enable the
reassuringly-named innodb_locks_unsafe_for_binlog option. I'm happy to
concentrate on public ridicule if that's what they prefer.

The other problem was also related to link table updates. If you do
locking reads, you get many unnecessary deadlocks.

-- Tim Starling

------------------------------

_______________________________________________
Wikitech-l mailing list
Wikitech-l at wikimedia.org <mailto:>
http://mail.wikipedia.org/mailman/listinfo/wikitech-l

End of Wikitech-l Digest, Vol 22, Issue 26
******************************************


from_name: Austin Hair

subject: [Wikitech-l] Longterm software strategy

body:
> See my previous comments about Mr Torvald's lock-free "git" system. The
> ideas behind this look ideal as a way of running an object store for
> article versions, images and other BLOBs, and the whole linux-kernel
> list are kicking the **** out of it as real-life beta-testers as we
> speak. So far, no-one's found any show-stopping problems with this
> radical approach to large-scale version management. (or rather, version
> lack-of-management, since it more-or-less discards all previous wisdom
> about version control).

I played with an early version, and whatever else I can say about it,
it's a nice proof of concept.  Fast, too.  Not sure how well it'd
adapt to this situation, though.

> -- Neil

-- Austin


from_name: Jean-Baptiste Soufron

body:
>>>
>>> A simpler response would be to cite the clause in the statute that
>>> says this.  To me a fundamental principal of law is that anything
>>> which is not specifically forbidden is allowed.
>>
>> We're talking about emergence here... what do you mean ? ? ?
>
> The statement (which was cut off at some point) was about your guess
> that French law did not allow a work to be put into the public domain.
>  Citing a specific passage in the French statutes would be much more
> effective than speculating about the law.

Here it is:

The article L-121-1 explains that moral rights are : "perp?tuel,
inali?nable et imprescriptible", it lasts forever, they can't renounce
to it and they can sue even years after a copyvio.

And when talking about patrimonial rights, there are various provisions
implying that the author cannot contribute his work to PD : L-122-7
precises very strong formalities for any release of his representation
or reproduction rights, L-122-8 precises that authors possess a forever
right to get 3% of any auction of their work and that they can't
renounce to it.

But more importantly, L-131-2 states that : "Les contrats de
repr?sentation, d'?dition et de production audiovisuelle d?finis au
pr?sent titre doivent ?tre constat?s par ?crit. Il en est de m?me des
autorisations gratuites d'ex?cution" which means that any contract to
sell their representation/edition rights must be a written one, even
when they authorizes free use of their work !

Article L-131-5 even precises that if the author released his rights
but that he got more than 7/12 less than what he should have been
normally awarded, he can sue to get it.

etc. etc. etc.


from_name: Jim Redmond

body:
Timwi wrote:
 >A friend of mine suggested that perhaps the word should be changed to
something more euphemistic >and something less harsh than "block".

Hmm... I can't come up with a single word that does better than
"block".  What about "administrative review"?  "Temporary leave"?
"Enforced hiatus"?

Jim Redmond
[[User:Jredmond]]


from_name: Jack Lutz

body:
"You are blocked for 24 hours" is honest. Euphamisms seek to present
something as what it is not, and so are dishonest. I would very much
object to any change.


from_name: Fastfission

subject: [WikiEN-l] RE: Corrected

body:
Our policy is NPOV, not "Caution and kindness." If we do NPOV
correctly, then it shouldn't be a problem -- nobody will blame
Wikipedia itself for what it says, they'll blame the Usenet forum or
academic book or whatever.

The lack-of-lawyers is actually a serious concern which I suspect will
become a major problem at some point in the future if Wikipedia
continues to become larger and larger. The structure of many laws
relevant to what we do here is such that Wikipedia can be up shit
creek pretty quickly if somebody with money and influence wanted us to
be. For example, "fair use" is only a *defensive* clause in the USA --
you can only really claim it if you are sued for copyright violation.
It is not an offensive statement -- we cannot tell people they are
"violating our fair use." What this means is that, even if we are in
the right, it would be extremely easy for a company like Disney to
grind us to a halt through a lawsuit -- the lawyer's fees alone would
cost hundreds of thousands of dollars. "We removed them immediately
after you sent us a letter," WP plead to Walt. "Well, that's too bad
-- I want back-royalties," he replies. And so in the best of
situations, WP hires lawyers, they plead fair use and win, and then WP
just owes the lawyers hundreds of thousands of dollars. Worst of
situations, WP owes both Disney AND the lawyers money, who knows how
much.

I don't see any good way around this. It is less a problem of
Wikipedia itself (though I'd be very careful of "fair use" because of
the way it is structured in US law), as it is with the ability of the
big and powerful to wage legal war even on principles they are in the
wrong on. It's more of a maddening thought than it is a sobering one.

FF

On 5/5/05, slimvirgin at gmail.com <slimvirgin at gmail.com> wrote:
> Wikipedia has become an incredibly powerful medium but we don't have
> any kind of an ethical code for cases like this. We have no lawyers,
> no heirarchy of editors, no fact-checking procedures: none of the
> infrastructure of large, powerful news organizations, and yet we have
> arguably as much power as some of them. It's a sobering thought. All
> I'm arguing is that we should take that power seriously, and never
> abuse it; and if a private person complains that we have, where
> there's no public-interest issue involved, we should err on the side
> of caution and kindness, because we lose nothing by doing that, but
> the person being criticized might lose a lot if we don't (regardless
> of the details of this particular case: I'm talking generally).
>
> Sarah
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l
>


from_name: Mirco Nanni

subject: [Foundation-l] Wikipedia Web Logs for scientific research

body:
Dear Wikimedia community,

   please have a look at the following proposal of
collaboration I sent to the Wikimedia board right yesterday.
The board (and myself too!) would like to hear your opinion
on that, before taking a decision.
   I would like to specify that such collaboration will have
scientific- and academic-only purposes, without any
commercial involvement.
   Finally, the analysis software we are developing (and
going to apply to Wikipedia data, if the proposal will be
accepted) will be distributed in the scientific community as
open source.

   Of course, I will be glad to provide you any detail and
explanation you will think necessary.

   Thank you for your attention. Best regards,

	- Mirco

----------------------------------------------------------

Dear Sirs,

   I am writing to you on behalf of the KDD-Lab (Laboratory
on Knowledge Discovery and Delivery:
http://www-kdd.isti.cnr.it), a branch of the ISTI institute
of the Italian National Research Centre (CNR).

   Our group is working (among the others) on a project
regarding the analysis of the logs of web servers, and in
recent days we are working on analysis techniques that seem
to be best suited for "content-rich sites". Our first
thought obviously went to Wikipedia...

   We would like to have the opportunity to apply our
analysis techniques to the web logs of Wikipedia. Looking to
the Wikipedia access statistics, we believe that an optimal
amount of data would be the following: (1) the (raw) weblogs
of the English section covering a few days of usage, or (2)
a few weeks for the Italian section.

   Do you think it could be possible to start this kind of
collaboration?

   Of course, we are willing to provide you all the legal
agreements you will consider necessary, especially those
regarding privacy. And, obviously, we will properly
acknowledge your contribution in any of our scientifical
publications and reports where we use it.

[Addendum: the sensible information in web logs is
essentially located in the "client IP" field ("who visited
that page"). However, for our research purposes such field
is not strictly needed as an encrypted version of it would
be enough, thus avoiding most of the privacy issues.]

   Thank you for your attention.
   Looking for receiving your answer and opinion, I send you
my best regards,

                      - Mirco Nanni

  ====================================
   http://ercolino.isti.cnr.it/mirco
  ====================================


from_name: Rick

body:
--- blankfaze <blankfaze at gmail.com> wrote:
> I have just noticed
> http://en.wikipedia.org/wiki/User:GordonWattsDotCom
> ...
> It seems to me to be somewhat inappropriate to
> create a username advertising
> a personal website, but I didn't see anything in the
> Wikipedia username
> policy (
> http://en.wikipedia.org/wiki/Wikipedia:Username )
> that says
> anything about adverts...
>
> So I thought I'd pose the issue to the mailing list,
> see what you all think.

Mr. Watts HAS been inserting links to his personal
website in the [[Terri Schiavo]] article, but his
personal website is a geocities page, and not
gordonwatts.com.  I don't know if there realy is a
gordonwatts.com, I haven't bothered to find out.

RickK

__________________________________
Do you Yahoo!?
Make Yahoo! your home page
http://www.yahoo.com/r/hs


from_name: Fred Bauder

body:
Holiday or vacation?

Fred

> From: Timwi <timwi at gmx.net>
> Reply-To: English Wikipedia <wikien-l at Wikipedia.org>
> Date: Tue, 10 May 2005 17:05:14 +0100
> To: wikien-l at wikipedia.org
> Subject: [WikiEN-l] "block" considered too harsh a word?
>
>
> As we have seen numerous times in the past, users who are new to the
> Wikipedia tend to vastly overreact to being blocked. Some people throw
> quite a fit even when they're blocked for a mere 24 hours.
>
> A friend of mine suggested that perhaps the word should be changed to
> something more euphemistic and something less harsh than "block". I can
> sympathise with this thinking, because if I saw a page telling me I'm
> "blocked", not knowing what it means I would probably interpret it to
> mean "blocked indefinitely", and as we know people don't tend to read
> the entire message and hence don't notice where it mentions the time limit.
>
> Unfortunately, he didn't have any ideas what to call it instead. So I'm
> wondering what everyone else here thinks?
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l


body:
Suspension of editing privileges

Fred

> From: Jim Redmond <jim at scrubnugget.com>
> Reply-To: English Wikipedia <wikien-l at Wikipedia.org>
> Date: Tue, 10 May 2005 11:16:43 -0500
> To: English Wikipedia <wikien-l at Wikipedia.org>
> Subject: Re: [WikiEN-l] "block" considered too harsh a word?
>
> Timwi wrote:
>> A friend of mine suggested that perhaps the word should be changed to
> something more euphemistic >and something less harsh than "block".
>
> Hmm... I can't come up with a single word that does better than
> "block".  What about "administrative review"?  "Temporary leave"?
> "Enforced hiatus"?
>
> Jim Redmond
> [[User:Jredmond]]
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l


from_name: Richard Holton

body:
On 5/10/05, Anthere <anthere9 at yahoo.com> wrote:
> Let's make a contest for political talk :-)
>
> I'd say... in comparison to the usual "you can edit !"
>
> make it "you can not edit !" :-)
>
> Ant
>

How about something along the lines of "your privilege to edit has
been temporarily suspended."

This is quite direct about what exactly has happened, while reminding
the user that they have no inherent right to edit. It also emphasizes
the temporary-ness of the action.

-- Rich Holton

[[W:en:User:Rholton]]


from_name: Brion Vibber

subject: [Wikitech-l] Longterm software strategy

body:
Brent 'Dax' Royal-Gordon wrote:
> Blaming the language is rarely a productive way to fix such a problem.
>  In particular, Python usually uses the same initialize-and-run model
> you complain about PHP using, and mod_mono doesn't seem to be widely
> used.  (And I assume Wikimedia has no interest in switching to Windows
> servers.)

Just a note: I've specifically mentioned standalone daemons as an
alternative possibility, *not* CGI programs or ASP.NET.

PHP really is much more strongly tied to the CGI-bin script execution
model than say Python, which is fairly routinely used to write network
servers and GUI programs.

There's nothing in PHP-the-language that makes a standalone daemon
written in PHP _impossible_, but it's rather awkward with PHP-the-
implementation's limitations -- no threading, awkward subprocess
control, limited exception handling. (Domas has experimented a bit with
this.)

(A complaint about PHP-the-language I do have though is its total lack
of Unicode string support. The only sensible way to deal with non-ASCII
material ends up being using byte-oriented strings in UTF-8 encoding,
and you have to worry about character boundaries and invalid character
sequences yourself where relevant. Not insurmountable, but it's
certainly an annoyance. Python carries the legacy of transitioning from
this model, and now has both byte-oriented and Unicode string types;
perhaps one day PHP will make the jump too.)

> Most reasonably mature languages are fast enough that the performance
> bottlenecks are usually in user code.  And at least one of the PHP
> problems--lack of a JIT--will be solved when PHP-on-Parrot is
> available.  (There's at least one project to do this; the interpreter
> itself is already quite fast and has JITs for several platforms,
> although much of it still has to be written.)

Well, we'll see when it gets there. :)

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20050510/db4c1553/attachment.pgp>


from_name: Joseph Reagle

references:
<20050510003821.B8F361AC17DD@mail.wikimedia.org>
	<d5qlln$etq$1@sea.gmane.org> <4280DE6B.4090103@scrubnugget.com>

body:
On Tuesday 10 May 2005 12:16, Jim Redmond wrote:
> Hmm... I can't come up with a single word that does better than
> "block".  What about "administrative review"?  "Temporary leave"?
> "Enforced hiatus"?

Sabbatical, mini-WikiHoliday?


from_name: Anthere

body:
Hi :-)

I fear I must say the name wikiversity does speak much more to me, and
wikisophia is confusing. I am not very keen on this name change.

Ant


from_name: Delphine Ménard

subject: [Foundation-l] Proposal: a cross-local chapters list?

body:
As local chapters tend to flourish here and there, I am thinking that
it would be a good idea that we start a new mailing list (yeah, don't
scream ;) ) something like wikichapters-l.

The idea behind this would be:
- Exchange of best practices (how to set up a local chapter, how it
works, what it does, what works and what doesn't)
- Common chapter projects (How can we work together to set up
cross-language and cross-country projects)

Why? for example, we have just been made aware of a very interesting
European programme on the French mailing list (see
http://europa.eu.int/information_society/activities/econtentplus/docs/programme/econtentplus_decision_en.pdf
for more info), and this should be pursued by all European Chapters
together.

Sending that kind of information to separate language or chapter
mailing list is good, so that debates can go on in a preferred
language, but a "general" chapter list would allow to keep everybody
who's interested in what goes on in different areas of the world
informed at the same time.

I am thinking of an open mailing list, where anyone interested in
cross-cultural and cross-country projects can register, preferred
language English, or at least partial translations in English of
proposed projects/ideas etc.

What do you think? Does is sound necessary? Or should everything be
kept here on Foundation?

Cheers,

--
Delphine
~notafish


from_name: Leif Knutsen

subject: [WikiEN-l] Reactions to blocking

body:
> Timwi wrote:
>Yes, but isn't that all the more reason to assume that we won't de-admin
>two long-time trusted contributors just to save one contributor whom we
>don't even know? I just don't get why so many newbies instantly think
>that they're the most valuable Wikipedian ever born and that established
>admins must leave to make room for them.

I don't believe I ever expressed or implied that I was the most valuable wikipedian ever born, nor that everything will fall apart as a result of my departure.  But I would submit that the entire enterprise depends on people like me, and that it's not a good idea to piss us off unnecessarily.

As for being a newbie, I have 2400 edits under my belt - a tenth of yours but still respectable, if I may say so myself.  I'll admit it was the first time I was ever blocked, so in that sense I am new to the experience.  It would have taken you a couple of minutes to find out who I was and what I'd done, so don't complain that you don't know me.

I do think that complaints against admins should be taken seriously.  Nobody should be de-admined because of one editor's grievances, but if there's a pattern of bad and borderline abusive behavior, action should be taken.


from_name: Dori

body:
On 5/10/05, Tim Starling <t.starling at physics.unimelb.edu.au> wrote:
>
> Lee Daniel Crocker wrote:
> > Another thing to consider: at least some of the wikipedia-driven
> > development will be totally unnecessary for mediawiki as a general-
> > purpose open source project. We may want to decouple those projects
> > at some point.
>
> Brion doesn't want to.
>
> -- Tim Starling

We've gotten pretty good debugging/testing because they're coupled though
right?


body:
On 5/10/05, Mirco Nanni <mirco.nanni at isti.cnr.it> wrote:
>
> Of course, we are willing to provide you all the legal
> agreements you will consider necessary, especially those
> regarding privacy. And, obviously, we will properly
> acknowledge your contribution in any of our scientifical
> publications and reports where we use it.
>
> [Addendum: the sensible information in web logs is
> essentially located in the "client IP" field ("who visited
> that page"). However, for our research purposes such field
> is not strictly needed as an encrypted version of it would
> be enough, thus avoiding most of the privacy issues.]

The problem is if you substitute the IP with a unique number, and you still
show accesses to user pages, you can probably identify the logged in users.
I'd be OK if the IPs were masked AND accesses to non-article namespace pages
were not given out.


subject: [Foundation-l] Back... and Africa

body:
Hiya !

I am back from my holidays in Marroco. I took less pictures than usual,
because I lost my camera... borrowed my husband one (being less used to
it, I messed a lot of pictures), then run out of batteries in a rather
desertic environnment...

Well... inch' allah.
I put two of myself on commons. I need to label the other ones. One is
taken in Agadir medina (a nice reconstitution awarded sustainable
tourism), the other in Aglou, 80 km south of Agadir (nothing but big
sand beaches, kids playing football, and women covered from head to
toe), with at best a face visible, at worse, nothing really. One
hotel... closed.

http://commons.wikimedia.org/wiki/User%3AAnthere

Anyway, my holidays were real nice...
I did not look at a computer, and made only one use of phone from a
telecabine (amazingly widespread in Marroco).

Two comments related to wikipedia possibly...

First, I met a journalist from l'economist.com over there. A woman. She
was actually interviewing my husband :-) So... I hope we can expand our
coverage over there in the next few weeks :-)

Second, I was invited to a congress, not directly related to wikipedia,
but to my professional background. This meeting is a bit special, since
it is the meeting of women geoscientists in Africa. Took place in South
africa once, in Uganda the second time, Marocco should be third time.
Well, I met two of the organisers, shared oriental danses with them, as
well as 4 nice evenings actually... and here it is.

This meeting is in great part a way of reminding that women can also be
involved in research, or generally in their countries development. I
could join it as my real self (and suggest a couple of good friends
working on soil fertility in Africa to join as well)... but I have been
thinking that a conference on "sharing knowledge", "giving power to
people to educate" and similar issues... might be of interest...
especially if we succeed to make a little bit of PR around that. It
would be a nice mixing of people and communities... as well as great
support of really truely amazing women.

I must say I was very saddened by the condition of women there... in
regression compared to what it was 10 years ago... most of them veiled
(which is okay), but many also quite visibly "hidden" behind the man and
in huge pieces of clothes which left little possibility to move.
I actually had a bad experience one evening as I was walking alone in
the streets, in spite of being very much covered that day, and trying
not to be noticed at all. I managed to go back to the hotel by refusing
to talk/answer, keeping eyes low, trying to pretend I was with other
groups... and navigating from one policeman to the next.

I must confess I found it delightful to eat only pastries at lunch...
but it is unfortunate that I had to stick to pastries due to the fact a
woman alone in a restaurant is just NOT an option.

Anyway... there is a lot to do there and I hope we can participate.

Anthere


subject: [Foundation-l] Back... and Africa


subject: [Wikitech-l] Longterm software strategy

body:
> I like the idea of porting hotspots, but keep in mind that we want
> people to be able to use this even if they don't have access to a C
> compiler.

That's going to be a problem with any non-PHP code, to a greater or
lesser extent,  but there's no reason someone can't maintain a (now
modularized) native-PHP parser.

> > A possible middle road is to rewrite the core wiki engine to a separate
> > daemon, and adapt the existing PHP user interface to call into it for
> > much of the backend work that actually touches data.
>
> I like this idea, although it carries its own costs.  In particular,
> communicating between processes has inherent overhead; we'd have to be
> reasonably sure that the caching would outweigh that.  Also note that
> IPC and process-management mechanisms tend to vary across operating
> systems; we might lose Windows support, for example.

I'd like to see as much portability maintained as possible, but to be
honest, I'm not personally worried about it.  MediaWiki is, first and
foremost, the software that runs the Wikimedia projects.  I can't see
us running Windows on the cluster anytime soon.

> One advantage of this split is that we could rewrite one of the
> components in another language if we want, without affecting the other
> one. If the front end becomes little more than a shell around a
> daemon, we could provide a version written in C as an Apache module,
> which is about as fast as it gets.

There are numerous options available to us, and that's one of them.
Making MediaWiki the server itself is at the far end of that extreme,
and the are countless possibilities between it and the other.

> On the back end, Perl 6 is specced to have one of the most powerful
> pattern-matching engines ever shipped with a language; it should be
> able to eat wikicode for breakfast.  It's also designed to allow easy
> interoperability with other languages, so just the wikicode parser
> could be written in it while the rest is left as PHP.  Implementation
> is moving quickly, with the backend Parrot Grammar Engine in progress
> and the Pugs (Perl 6 in Haskell) compiler just starting on the syntax.
>  (And, of course, you'd have a very happy Perl hacker here.)

I've yet to be sold on Perl 6, but that's just my personal opinion.
It may yet turn out to be the best option, especially if its
pattern-matching capabilities are all the evangelists say they are.

> --
> Brent 'Dax' Royal-Gordon <brent at brentdax.com>
> Perl and Parrot hacker

At least you're open about your bias.  ;)

-- Austin


from_name: Tim Starling

subject: [Wikitech-l] Re: Parser (was Re: Longterm hosting strategy)

body:
Lee Daniel Crocker wrote:
> I agree, I don't think the parser's a big issue, although it would be
> nice for a bit snappier response.  In hindsight, storing the wikitext in
> a database was a mistake.  There's already a wonderful piece of software
> highly optimized and scalable for storing randomly accessed variable-
> sized chunks of text with lots of tools for backup, replication, and so
> on; it's called a file system.  Storing the wikitext itself in something
> like Reiserfs would probably speed it up, and also speed up access to
> the rest of the metadata in the database which would become much
> smaller.

That's what ExternalStore is for. Moving the bulk out of the database,
or at least to a different database, is a pressing need. We need to
separate bulk, rarely accessed data from hot data, so that we can save
the highly redundant storage on the DB master for hot data. Domas has
been working on it. We're running out of disk space on Ariel again, and
another compression round is obviously only a stopgap solution.

-- Tim Starling


body:
Lee Daniel Crocker wrote:
> Yes!  There's only one tricky part for which we may have to consider
> creative implementations: I tried as much as possible to take style
> markup (especially skin-specific) out of the rendered wikitext to
> allow it to be cached, but there's one case that's still a problem:
> red links (i.e., links to non-existent pages).  Users shouted at me
> that this was a sine-qua-non feature, and so I had to leave it in.
> But it makes caching rendered wikitext hard, and slows down rendering.
> One alternative is to simply tolerate them being out of date for the
> life of the cache.  Another is to possibly update the cache in some
> cheaper way.  Yet another is to optimize the hell out of discovering
> the simple existence of a page, so that it's not a bottleneck in
> rendering (say, by having a daemon that keeps a one-bit field for
> every page using a spell-checker data structure)

We already optimised it, didn't we? In the last public profiling run:

http://meta.wikimedia.org/wiki/Profiling/20050328

...it came in at 2.6% for the non-stub bundled query and 0.4% for the
stub query. I'd hardly call that a bottleneck. Individual link existence
tests came in at 5.9%, mostly due to special pages, but I've largely
fixed that in 1.5 by bundling the existence tests for commonly requested
special pages. It wasn't so long ago it was taking 15% for individual
queries and 15% for LinkCache::preFill():

http://meta.wikimedia.org/wiki/Profiling/Live_aggregate_20040604

...so we've come a long way.

> I'm all for your method, and I agree it's not an urgent need.  But I
> think we can slip the timeline even more.  The existing codebase will
> eventually be a liability, but I think we can throw hardware at it for
> a year or two.  Also, if we go the route of making independent daemons
> linked into the existing UI code, we don't have to deploy all at once.
> We could, for example, make and deploy the math daemon as a proof-of-
> concept, work out bugs with that, then do the others afterward.

We've already got two proof-of-concept daemons: the Chinese word
segmenter and Lucene.

There is a technical problem with Lucene at the moment: it uses file()
to fetch the result over HTTP, but that has an unconfigurable 3 minute
timeout. If the search daemon goes down, we hit apache connection limits
within a minute and the site stops working. We can either patch PHP to
use default_socket_timeout in this case, or switch to another method
like DIY pfsockopen or curl.

> Another thing to consider: at least some of the wikipedia-driven
> development will be totally unnecessary for mediawiki as a general-
> purpose open source project.  We may want to decouple those projects
> at some point.

Brion doesn't want to.

-- Tim Starling


from_name: John Fader

body:
On 5/10/05, Daniel Mayer <maveric149 at yahoo.com> wrote:
> OK - given that along with the other objections, I now withdraw my weak support
> for the name change. All we need to do is say, up front, that we will not limit
> Wikiversity to college level and above courses; that all levels of learning
> will be allowed.

Then "wikilearn" makes sense.

But then again it's a shame to have yet another english (or
european-derived, for that matter) term.  What is sanskrit for
"learn", I wonder?

--
John Fader


subject: [Wikitech-l] DNS weirdness today?

body:
Twice today I've requested a page (in one case the search page) on
en.wikipedia and ended up at some page in what I think is
ja.wikipedia.

I can't reproduce it reliably.  I can mail a screenshot, but it's not
really useful for debugging.

Perhaps we have a DNS (or squid) mixup ?

--
John Fader


from_name: Phil Boswell

subject: [Wikitech-l] Re: DNS weirdness today?

body:
"John Fader" <jfader at gmail.com> wrote in message
news:42eaa3d05051007001274a9ba at mail.gmail.com...
> Twice today I've requested a page (in one case the search page) on
> en.wikipedia and ended up at some page in what I think is
> ja.wikipedia.
>
> I can't reproduce it reliably.  I can mail a screenshot, but it's not
> really useful for debugging.
>
> Perhaps we have a DNS (or squid) mixup ?

I tried to jump from [[en:talk:Sluggy Freelance]] to [[en:Sluggy Freelance]]
using the standard tab link at the top of the page, and ended up on some
other language (I *think* it was some sort of Scandinavian) without any clue
as to what the article was about.

At first I figured it was some smartarse vandal inserting a cross-wiki
REDIRECT, but ISTR those have been disabled.
--
Phil
[[en:User:Phil Boswell]]


from_name: Timwi

body:
Tim Starling wrote:
>
> In my opinion, our biggest performance problem at the moment is not
> the parser, it's replication lag and database load balancing. There's
> lots of work that has to be done there.

How many people have access to the servers so they can work on this?

How many people who could potentially work on this, have asked for
access and have been refused?

Timwi


subject: [Wikitech-l] Re: DNS weirdness today?

body:
On 5/10/05, Phil Boswell <phil.boswell at gmail.com> wrote:
> At first I figured it was some smartarse vandal inserting a cross-wiki
> REDIRECT, but ISTR those have been disabled.

It's noteworthy that the browser's address bar still shows the correct
(i.e. desired)  en URL (which it wouldn't after any kind of
crosswiki).  The mapping (in my case) of en's search page to the ja
page about the Yoshimoto Kogyo comedy club surely can't be a DNS
confusion, leaving squid mixups or general doom.

--
John Fader


subject: [Wikitech-l] Re: Our own IRC server

body:
Georg Jaehnig wrote:
>
> As I wrote in the thread "Wikipedia Zeitgeist" it could be useful to
> have a log of the recent changes.  Not only to realize the Zeitgeist
> but also to do some other statistics for everybody.
>
> Is it possible to make a tar.gz file for every day and channel
> available?

What would such a file provide that the database dumps don't already
provide?


from_name: Delirium

subject: [Wikitech-l] Re: Longterm hosting strategy

body:
Neil Harris wrote:

> Now, if a connection was being limited by TCP startup alone, this
> means that data would be delivered somewhere between four and ten
> times faster, and in particular that page-rendering would be able to
> start four to ten times as quickly. In practice, this is not the sole
> factor involved, so observed gains will not be as great as this, but
> they will certainly be very noticeable by users.

I still don't see how that's the case.  I just timed some page views on
en:, de:, and fr:, and all took over 2000 milliseconds from request to
the end of page-load.  The en: Main Page seems to take around 4000-5000
milliseconds.  How is an 80-ms latency difference in TCP startup due to
geographical server location going to be noticeable on these scales?

-Mark


from_name: Jason Richey

subject: [Wikitech-l] Wikipedia and mysql 4.1

body:
We use MySQL 4.1 at Wikicities without any problem.  We did have a
problem in transitioning between versions though.  Just be careful to
include "--default-character-set=latin1" when you run your mysqldumps
(if that applies to you, of course).

Jason Richey

Kevin Carillo wrote:

>Hi.
>I'd like to know whether it is possible to run Wikipedia with mysql 4.1.
>Before installing the whole environment with mysql 4.0.20a, I tried to set
>it up with mysql 4.1 and I remember having problems installing mediawiki
>which did not accept this version of mysql.
>However, my research project requires a lot of queries that contain
>subqueries, and it is almost impossible to rewrite them all as JOINS (one of
>the reason is that I not only use SELECT queries but also UPDATE and
>DELETE). Using Mysql 4.1 would help me considerably.
>
>In case, it is possible to use mysql 4.1, is there any way to upgrade the
>version of mysql by keeping my current wikipedia database?
>Thank you.
>
>Kevin Carillo
>
>
>  _____
>
>From: wikitech-l-bounces at wikimedia.org
>[mailto:wikitech-l-bounces at wikimedia.org] On Behalf Of
>wikitech-l at wikimedia.org
>Sent: May 10, 2005 4:34 AM
>To: wikitech-l at wikimedia.org
>Subject: Wikitech-l Digest, Vol 22, Issue 26
>Importance: Low
>
>
>Send Wikitech-l mailing list submissions to
>        wikitech-l at wikimedia.org <mailto:>
>
>To subscribe or unsubscribe via the World Wide Web, visit
>        http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>or, via email, send a message with subject or body 'help' to
>        wikitech-l-request at wikimedia.org <mailto:>
>
>You can reach the person managing the list at
>        wikitech-l-owner at wikimedia.org <mailto:>
>
>When replying, please edit your Subject line so it is more specific
>than "Re: Contents of Wikitech-l digest..."
>
>
>Today's Topics:
>
>   1. Re: Parser (was Re: Longterm hosting strategy) (Tim Starling)
>   2. Re: Parser (was Re: Longterm hosting strategy)
>      (Lee Daniel Crocker)
>   3. Re: Longterm software strategy (Tim Starling)
>   4. Re: Parser (was Re: Longterm hosting strategy)
>      (David A. Desrosiers)
>   5. Re: Parser (was Re: Longterm hosting strategy)
>      (?var Arnfj?r? Bjarmason)
>   6. New machines installed, killed in record 9.5 hours (Brion Vibber)
>   7. Re: Longterm software strategy (Brion Vibber)
>   8. Link table updates (was Re: Longterm software strategy)
>      (Tim Starling)
>
>
>----------------------------------------------------------------------
>
>Message: 1
>Date: Tue, 10 May 2005 13:55:48 +1000
>From: Tim Starling <t.starling at physics.unimelb.edu.au <mailto:> >
>Subject: [Wikitech-l] Re: Parser (was Re: Longterm hosting strategy)
>To: wikitech-l at wikimedia.org <mailto:>
>Message-ID: <d5pau9$7q4$1 at sea.gmane.org <mailto:> >
>Content-Type: text/plain; charset=ISO-8859-1
>
>Lee Daniel Crocker wrote:
>
>
>>I agree, I don't think the parser's a big issue, although it would be
>>nice for a bit snappier response.  In hindsight, storing the wikitext in
>>a database was a mistake.  There's already a wonderful piece of software
>>highly optimized and scalable for storing randomly accessed variable-
>>sized chunks of text with lots of tools for backup, replication, and so
>>on; it's called a file system.  Storing the wikitext itself in something
>>like Reiserfs would probably speed it up, and also speed up access to
>>the rest of the metadata in the database which would become much
>>smaller.
>>
>>
>
>That's what ExternalStore is for. Moving the bulk out of the database,
>or at least to a different database, is a pressing need. We need to
>separate bulk, rarely accessed data from hot data, so that we can save
>the highly redundant storage on the DB master for hot data. Domas has
>been working on it. We're running out of disk space on Ariel again, and
>another compression round is obviously only a stopgap solution.
>
>-- Tim Starling
>
>
>
>------------------------------
>
>Message: 2
>Date: Mon, 09 May 2005 21:04:55 -0700
>From: Lee Daniel Crocker <lee at piclab.com <mailto:> >
>Subject: Re: [Wikitech-l] Parser (was Re: Longterm hosting strategy)
>To: ?var Arnfj?r? Bjarmason <avarab at gmail.com <mailto:> >, Wikimedia
>developers
>        <wikitech-l at wikimedia.org <mailto:> >
>Message-ID: <1115697895.5779.25.camel at shuttle.piclab.com <mailto:> >
>Content-Type: text/plain; charset=utf-8
>
>On Tue, 2005-05-10 at 03:44 +0000, Cvar ArnfjC6rC0 Bjarmason wrote:
>
>
>>>like Reiserfs would probably speed it up, and also speed up access to
>>>the rest of the metadata in the database which would become much
>>>smaller.
>>>
>>>
>>How about something like a version control system, subversion for
>>example, I don't know how it would do speed wise for something like
>>this but with that you'd get
>>
>>
>
>Waaaay too slow (have you ever used Subversion?)  But it might not be
>a bad idea to put a WebDAV/DeltaV front end on whatever we create to
>make it possible for third-party tools to access it.
>
>
>


from_name: David A. Desrosiers

body:
> Which would give us an answer to "who put this sentence in?" Which
> would be very helpful for collaborative editing, requesting
> references, etc.

	"...identifying plagarism..." ;)

David A. Desrosiers
desrod at gnu-designs.com
http://gnu-designs.com


from_name: Neil Harris

subject: [Foundation-l] Wikiversity=>Wikisophia

body:
Erik Moeller wrote:

>
>
> Finally, consistency is important. We have an established naming
> scheme -- our projects have unique names that usually begin with
> "Wiki-" -- and in order to maintain our corporate identity, new
> projects should follow the same pattern. (Incidentally, I named the
> "Wikimedia Commons" using a different pattern because it is not a
> regular Wikimedia project, but an umbrella project used by all others.
> Regardless, it is often abbreviated as "Wikicommons", indicating a
> strong desire of users to have names following this pattern..)
>
> FWIW, the "cute" part of "Wikisophia" is not "-sophia", but "Wiki",
> and we're pretty much stuck with that no matter what we do.
>
> Erik
>
Wikiversity is great, but quite rightly suggests tertiary education. How
about Wikademy ("academy"), and Wikitutor for the intermediate and
junior versions of the educational resource, respectively?

-- Neil


subject: [Foundation-l] Wikiversity=>Wikisophia


body:
> I've had some good experience with Python+Psyco. Psyco can deliver
> excellent performance boosts for tight loops doing simple operations
> on numbers and strings, and does a good job with function calls.

	Don't forget Pyrex[1] also.. (I abhor Python, but I will give
it a fair shake in arguments such as this).

[1] http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/

David A. Desrosiers
desrod at gnu-designs.com
http://gnu-designs.com


subject: [Wikitech-l] Re: DNS weirdness today?

body:
John Fader wrote:
> Twice today I've requested a page (in one case the search page) on
> en.wikipedia and ended up at some page in what I think is
> ja.wikipedia.

Whoa. So I'm not the only one. I had a lot of tabs with Wikipedia pages
open, and as I flipped through them I suddenly noticed that one of them
was in Polish... it was http://pl.wikipedia.org/wiki/Drzewo (the Polish
article on [[tree]]s), and I have absolutely no idea how I got there! :)

Timwi