body:
On Mon, 2003-04-28 at 16:46, Erik Moeller wrote:
> Brion, you're missing my point. I agree with you entirely that things need
> to "get done". My suggestion to have a public discussion was to find out
> which things we can get done reasonably quickly (because, realistically,
> we all have other things to do) with substantial impact; to figure out the
> server situation, which features should be disabled, who might contribute
> which piece of code etc. If we can sort these things out in the next few
> days via mail, fine. I'm no IRC junkie. But we need to implement at least
> some reasonable emergency fixes, and think about a mid term strategy.

Well maybe, but my experience with using online chats like this is:
* Everyone sits around for several hours babbling, waiting for the other
folks to show up and complaining about the problems they're having
logging in.
* By the end, someone has scribbled up a page with a work plan, which
everyone ignores in the future.
* During all this time, they _could_ have been doing something
productive instead...

> As for code, this is one thing I'd like to talk about: If we have the
> Nupedia Foundation set up...

The status of the non-profit is indeed another thing Jimbo could shed
some light on...

> > * Page viewing is still kinda inefficient. Rendering everything on every
> > view is not so good...
>
> Why? It's just PHP stuff.

Obviously not, since that PHP stuff needs data from the database to
work. :) Buying milk at the grocery store is more convenient than
keeping and milking cows at home not because the milking process is more
time consuming than grabbing a bottle from the fridge, but because
maintaining the cow is a huge effort and milk is only available from the
cow under certain conditions. Or, um, something like that.

>  Our bottleneck is the database server. Fetching
> stuff from CUR and converting it into HTML is not an issue. 20 pass
> parser? Add another zero. Until I see evidence that this has any impact on
> performance, I don't care. Turn off link checking and all pages are
> rendered lightning fast.

And that would be a pretty piss-poor wiki, wouldn't it? :)

> What would be useful is to maintain a persistent (over several sessions)
> index of all existing and non existing pages in memory for the link
> checking. A file on a ramdisk maybe? I think it would be worth giving it a
> try at least, and not a lot of work.

Sure, it _might_ help. Code it up and see!

> > * The page saving code is rather inefficient, particularly with how it
> > deals with the link tables...
>
> I doubt that a *relatively* rare activity like that makes much of an
> impact, but I'll be happy to be proven wrong.

Slow saving impacts everyone who tries to edit articles; four edits per
minute may be _relatively_ rare compared to page views, but we're still
running thousands of edits per day and it's a fundamental part of what a
wiki is. It's absolutely vital that editing be both swift and bug-free,
and if we can reduce the opportunities for saving to get hung up, so
much the better.

> Caching special pages seems like a reasonable approach.

Unfortunately that doesn't really solve the problem any more than
replacing the search with a link to Google solves the search problem.

If updating these cached pages is so slow and db-intensive that it takes
the 'pedia offline for fifteen-twenty minutes (which it does), then
nobody's going to want to update the caches (last updated April 9...)
and they become outdated and useless.

It works as a temporary crutch in place of "blank page -- this feature
has been disabled, please find a less popular web site to play on", but
it's not a solution.

>  Watchlists could
> definitely be improved, haven't seen a good way to do this yet, though. It
> could be done on page save, but with a much-watched page, this again would
> add severe drain, with possibly no overall benefit. Improve the SQL and
> indexes? Maybe, but I'm no SQL guru.

Which Himalayan mountain do we have to climb to find one? :)

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20030428/a34b5779/attachment.pgp>


list: wikitech-l

source: pipermail-mbox

from_name: Thomas Corell

subject: [Wikitech-l] SQL Request

body:
Guillaume Blanchard wrote:
>>>Tu penses vraiment que c'est une requete lourde ?
>>>Si tu as peur que j'abime la base avec une erreure de requete, je prefere
>>>que tu me le dise franchement ;o)
>>non non c'est pas ca!
>>mais j'ai l'impression que le serveur rame pour un rien en ce moment.
>>Mais je ne suis pas developeur, c'est juste ce que j'entends sur cette
>>liste.
>>Les plages peu encombrees: quand les Americains dorment!!!
> Ils dorment a quel heure UTC ? :oP
> Aoineko

Sorry, but can you stop writing french in Wikitech-l? Try an appropriate
wikifr-l. I don't have any time to learn french only for this international
mailinglist. Thanks.

Smurf
--
-------------------------  Anthill inside! ---------------------------


subject: [Wikitech-l] error during installation

body:
Lee Daniel Crocker wrote:
  > Thanks for the report--yes, that's a bug. The original encryptPassword()
> function was static, but the recent change to add a salt to the
> encryption used a non-static function. I'm not sure why the install
> script has been working for me, but at any rate I'll fix it right away.

I tried to install too (trying to get checktrans running).

Following problems occoured:

$wgLanguageCode = "de" (example) in Localsettings php is unhappy.

After testing the existens of LocalSettings and AdminSettings in
install.php, it would help to include them. The next test ($wgUseTex) won't
  work without loading them.

If another language is defined, install should copy the appropriate
language files.

That's just as far as I got, it's of course still not working ;)

Smurf
--
-------------------------  Anthill inside! ---------------------------


from_name: Tomasz Wegrzanowski

subject: [Wikitech-l] SQL Request

body:
On Mon, Apr 28, 2003 at 06:28:06PM +0200, Thomas Corell wrote:
> Guillaume Blanchard wrote:
> >>>Tu penses vraiment que c'est une requete lourde ?
> >>>Si tu as peur que j'abime la base avec une erreure de requete, je prefere
> >>>que tu me le dise franchement ;o)
> >>non non c'est pas ca!
> >>mais j'ai l'impression que le serveur rame pour un rien en ce moment.
> >>Mais je ne suis pas developeur, c'est juste ce que j'entends sur cette
> >>liste.
> >>Les plages peu encombrees: quand les Americains dorment!!!
> >Ils dorment a quel heure UTC ? :oP
> >Aoineko
>
> Sorry, but can you stop writing french in Wikitech-l? Try an appropriate
> wikifr-l. I don't have any time to learn french only for this international
> mailinglist. Thanks.

No, it's ok to write in any language.
Especially apropriate are multilingual posts.


subject: [Wikitech-l] Patch for LanguageDe.php

body:
Lee Daniel Crocker wrote:
> I just applied Thomas's patch to LanguageDe.php. It may be a while
> before it shows up in the live wiki, but it's in CVS now.
>

Thank you. I will try to accumulate some typos before I send the next one :)

Smurf
--
-------------------------  Anthill inside! ---------------------------


from_name: Brion Vibber

subject: [Wikitech-l] Chat about Wikipedia performance?

body:
On Mon, 2003-04-28 at 19:21, Erik Moeller wrote:
[on a persistent link-existence table]
> I might. I'll have to see if it makes any difference on the relatively
> small de database which I'm currently using locally. It would have to be
> optional -- setting up the software is already difficult enough.

I don't know whether you've already looked into this, but PHP does seem
to have some support for shared memory:

http://www.php.net/manual/en/ref.sem.php
  or
http://www.php.net/manual/en/ref.shmop.php

These seem to require enabling compile-time options for PHP.

It's also possible to create an in-memory-only table in MySQL
(type=HEAP), which may be able to bypass other MySQL slownesses (but it
may not, I haven't tested it).

> > Slow saving impacts everyone who tries to edit articles; four edits per
> > minute may be _relatively_ rare compared to page views, but we're still
> > running thousands of edits per day and it's a fundamental part of what a
> > wiki is. It's absolutely vital that editing be both swift and bug-free,
> > and if we can reduce the opportunities for saving to get hung up, so
> > much the better.
>
> Yeah yeah yeah. I still think we should care more about the real
> showstoppers. But hey, you can always _code it_. (Finally an opportunity
> to strike back ;-)

Touch?. :) My point is just that we need to keep that critical path
clean and smooth -- and working. (I would consider not differentiating
live from broken links, or getting frequent failures on page save to be
fatal flaws, whereas not having a working search or orphans function is
just danged annoying.)

> If this downtime is unacceptable, we might indeed have to think about a
> query only server with somewhat delayed data availability. This could be a
> replacement for the sysops, too. Mirroring the Wikipedia database files
> (raw) should be no issue with a SCSI system, or a low priority copy
> process.

Sure, MySQL's database replication can provide for keeping a synched db
on another server. (Which, too, could provide for some emergency
fail-over in case the main machine croaks.)

The wiki would just need a config option to query the replicated server
for certain slow/nonessential operations (search, various special pages,
sysop queries) and leave the main db server free to take care of the
business of showing and saving pages and logging in users.

However this is all academic until we have reason to believe that a
second server will be available to us in the near future.

> Maybe we should stop looking in the Himalayan mountains and start
> searching the lowlands ..  In other words: Don't search those who will do
> it for society or for the glory. Just hand over the cash and be done with
> it.

A lovely idea, but there _isn't_ any cash as of yet, nor a non-profit
foundation to formally solicit donations with which to fund programmers.
Until this gets done, or unless someone wants to fund people more
directly, all we've got is volunteer developers, who are only rarely
unemployed database gurus who can spend all day working on Wikipedia. :)

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20030428/02a7e77e/attachment.pgp>


list: wikien-l

from_name: daniwo59 at aol.com

subject: [WikiEN-l] Zog

body:
I pointed out Zog last night to the list ("two annoying vandals"). I vote to
get rid of him. With his racist rants, there really isn't much reason for
debate.

Danny
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/wikien-l/attachments/20030428/eba1fbac/attachment.htm


from_name: K Forstner

subject: [WikiEN-l] I've almost had enough

body:
I just wrote a longer message to Zog about his Johnny Rebel article on the
talk page -- something which I normally don't save before sending it -- and
abracadabra, and the whole thing was gone when I wanted to save it, and I
was shown the empty talk page. What the hell is going on here?

KF


from_name: Tony Wilson

subject: [WikiEN-l] Questions about honesty and fair play

body:
>From time to time, it is said that deliberately hiding substantive
changes to an entry is a bannable offence.  Is this true? If so, where
is it written down? Is there a mechanisim in place to warn offenders
that their behaviour is unacceptable?  Or, if it is not true, should a
policy change be instituted to make it true?

Tony Wilson
(Tannin)


subject: [Wikitech-l] error during installation

body:
Lee Daniel Crocker wrote:
>>(Thomas Corell <T.Corell at t-online.de>):
>>
>>Following problems occoured:
>>$wgLanguageCode = "de" (example) in Localsettings php is unhappy.
> You'll have to tell me what this means in plain English before I
> can help you.  Yes, you'll have to add that line to LocalSettings.php,
> with the semicolon, to get a German wiki.  This LocalSettings.php file
> must be in the top level directory of the phase3 tree (along with a
> localized AdminSettings.php as well) when you run the install script,
> and you must run it from that directory.

Sorry, this was written too fast, and with too few brain activated. I
should have written it this way:

It would be nice if in the LocalSettings.sample a line like:

# If you use a language different from english (en) uncomment the
# following line and edit the Language Code appropriate:
# $wgLanguageCode ="En";

could be added, for an easy installation of non english wikis.

PS: I will think about to change to german language for my mails here, to
avoid problems(for me), as proposed by Tomasz Wegrzanowski.

Smurf
--
-------------------------  Anthill inside! ---------------------------


from_name: Guillaume Blanchard

subject: [Wikitech-l] SQL Request

body:
> since we're having server problems at the moment, I'd advise against it.
>
> etant donn? que le serveur rame comme c'est pas possible en ce moment,
> je dirais qu'il vaudrait mieux attendre.
> Ou faire les articles un par un - tu les copies-colles dans un editeur
> de texte qui fait les expression regulieres. (je te conseile TextPad)

Il y en a vraiment beaucoup d'article a faire...
N'y a t'il pas des plages horraires pas trop encombre ?
Je suis pres a faire tourner la requete a n'importe quel heure de ton choix.
Si je fait une recherche avec l'outil de recherche sur "Egypte antique" puis
"egypte antique" et enfin "?gypte antique", ca me parait plus lourd que une
seul requete qui cherche les trois et les remplacent par "?gypte antique".
Je me trompe ?
Tu penses vraiment que c'est une requete lourde ?
Si tu as peur que j'abime la base avec une erreure de requete, je prefere
que tu me le dise franchement ;o)
Merci pour la reponse.

Aoineko


list: wikipedia-l

subject: [Wikipedia-l] Server overload?

body:
Stan Shebs wrote:
> Brion Vibber wrote:
> It seems more frequently bogged down than two months ago, which
> I guess is good if it means more random people are coming by.
> Does anybody have numbers comparing load due to logged-in users
> vs random anons?

I only see increased complains from logged-in users in the german
wikipedia. Mostly it is worried if the permanently impossibility to edit
articles (getting no answers from the server over 10 min is unreachable in
the Web) drives away new authors and of course some of the old one's.

I only see myself that in the timespan normal people would write articles
for wikipedia the best you can do is writing mails here.

I hope you get the new server running as fast as possible.

Smurf
--
-------------------------  Anthill inside! ---------------------------


from_name: Zoe

subject: [WikiEN-l] Lir is back

body:
He isn't even trying to hide now.  He's calling
himself Ril.

Zoe

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com


from_name: Erik Moeller

subject: [Wikitech-l] Search feature not working

body:
> On Mon, 2003-04-28 at 03:30, Erik Moeller wrote:
>> As I wrote you by private mail, I think people would be much more willing
>> to accept a temporary deactivation of the sysop SQL feature, which, if at
>> all, should be run on a separate machine (possibly a mirror, could really
>> be 1-2 days out of date).

> I have only very rarely seen slowdowns caused by sysops making SQL
> queries, whereas the special pages and the search cause much more
> continuous states of slowness as they are constantly pounded on.

Yes, you are correct, they are very rare. But if they occur, they will
effectively slow down all other queries for the time they run. Because of
the risk that such an out of control query can slow us down for hours, I
think there's really no argument that can be made for keeping them active,
unless we implement a timeout (which would mean you or Lee, because I
frankly never cared much about the SQL feature). That doesn't mean that we
should not consider other options for speedings things up as well
(although I agree that the server situation needs to improve in any case).

Regards,

Erik


subject: [WikiEN-l] Zog again

body:
Please do something about this new user. He is turning Wikipedia into a forum
for his racist diatribes.

Danny
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/wikien-l/attachments/20030428/267a4c6b/attachment.htm


body:
How about organizing a chat this week about the ongoing Wikipedia
performance crisis and how to solve it? Talking to people can provide
additional motivation for getting things done, and help us organize our
priorities. It might also reduce some frustration. If we do this, all the
relevant people should be present:

- Jimbo
- Jason
- Brion
- Lee
- Magnus
- ...

It might be best to meet on the weekend, so that work does not interfere.
My suggestion would be Saturday, 20:00 UTC.

What do you think?

Regards,

Erik


from_name: Youandme

subject: [Wikitech-l] clock out of sync

body:
I watch a constant degradation of Wikipedia's clock.
Before the weekend it was 5 minutes (acceptable for me)
but today I see the difference is about 10 minutes.
Some time ago there was a small thread started
about synchronization through the Net.
Has been the idea dropped or the possible solutions
are to complicated?

Youandme (worried, as Wikipedia slips into the Past)


subject: [Wikitech-l] SQL Request

body:
> >Tu penses vraiment que c'est une requete lourde ?
> >Si tu as peur que j'abime la base avec une erreure de requete, je prefere
> >que tu me le dise franchement ;o)
> >
> non non c'est pas ca!
> mais j'ai l'impression que le serveur rame pour un rien en ce moment.
> Mais je ne suis pas developeur, c'est juste ce que j'entends sur cette
> liste.
> Les plages peu encombrees: quand les Americains dorment!!!

Ils dorment a quel heure UTC ? :oP

Aoineko


body:
> I'll have some more performance numbers by then, and I'm happy
> to participate in whatever the group wants to do. But personally,
> I've never been big on online chats, and I don't see that this
> one could accomplish anything that wouldn't be better accomplished
> here on wikitech-l.

Cool. The problem with mailing list discussion is that they can die
quickly, for many reasons, which can delay things unnecessarily. I've seen
many situations where a mailing list was used to report a serious problem,
but the post (in spite of hundreds of members) was ignored.

We all know that the performance issue is one of our most pressing
problems right now -- many people can't use the site anymore, and the
international Wikipedians are getting a bit irritated. So I think the best
way to address this *on time* is to sit down (virtually) and go through an
agenda.

Regards,

Erik


from_name: John Knouse

subject: [WikiEN-l] Johnny Rebel and ZOG

body:
Good god, this ZOG person just won't let it rest.  He wants
to list every song that the racist Johnny Rebel wrote and
wants to promote him in general.  There's a limit.  I mean,
do we have to exhaustively provide details on someone so
utterly revolting?  It took repeated efforts to even make
ZOG admit that there is more than one Johnny Rebel.  Can we
do something about this ZOG guy?

--
John Knouse
jaknouse at frognet.net


subject: [Wikitech-l] SQL Request

body:
Hi,
Again Aoineko from the french Wikipedia.
Here is a good way of updating all articles that contain ' foo1 ' to ' foo2
' ?
Don't panic, I will not run this request just now ;o)
It's just to know.
There are many spelling errors we can solve with it (especially for french
accent accent).
What is the best time to run SQL request (not especially this one) ?

UPDATE LOW_PRIORITY cur
SET cur_text = REPLACE( cur_text, '% foo1 %', '% foo2 %')
WHERE cur_text
LIKE '% foo1 %';

Aoineko


from_name: Krzysztof Kowalczyk

subject: [Wikitech-l] Bug in install.php script

body:
Hi,

There's a typo in install.php script. It has:
  copydirectory( "./stylesheets", $wgStyleSheetsDirectory );
while it should have:
  copydirectory( "./stylesheets", $wgStyleSheetDirectory );

The end result is that *.css files are copied to "/" instead of
"${IP}/style"

Regards,

Krzysztof Kowalczyk

Index: install.php
===================================================================
RCS file: /cvsroot/wikipedia/phase3/install.php,v
retrieving revision 1.4
diff -u -r1.4 install.php
--- install.php 28 Apr 2003 18:14:48 -0000      1.4
+++ install.php 29 Apr 2003 07:03:51 -0000
@@ -53,7 +53,7 @@
 copyfile( ".", "texvc.phtml", $IP );

 copydirectory( "./includes", $IP );
-copydirectory( "./stylesheets", $wgStyleSheetsDirectory );
+copydirectory( "./stylesheets", $wgStyleSheetDirectory );

 copyfile( "./images", "wiki.png", $wgUploadDirectory );
 copyfile( "./languages", "Language.php", $IP );


subject: [Wikitech-l] SQL Request

body:
> UPDATE LOW_PRIORITY cur
> SET cur_text = REPLACE( cur_text, '% foo1 %', '% foo2 %')
> WHERE cur_text
> LIKE '% foo1 %';

oups, copy&past error :o)
here is the right version:

UPDATE LOW_PRIORITY cur
SET cur_text = REPLACE( cur_text, ' foo1 ', ' foo2 ')
WHERE cur_text
LIKE '% foo1 %';

Aoineko


from_name: Daniel Mayer

subject: [WikiEN-l] Johnny Rebel and ZOG

body:
John Knouse wrote:
>Good god, this ZOG person just won't let it rest.
>He wants to list every song that the racist Johnny
>Rebel wrote

We have discographies for many other musicians - Why not Johnny Rebel? I would
say this even if "nigger" were replaced by "faggot" for each song title (I'm
gay BTW).

>and wants to promote him in general.

As far as I can tell this JR person isn't the most well-known or famous of
musicians so his article should only be linked from a few other highly
relevant entries. Excessive linking to this article should not be allowed.

>There's a limit.  I mean, do we have to exhaustively
>provide details on someone so utterly revolting?

Yes we do - otherwise we would not have articles on Adolf Hitler, Richard
Ramirez or Pol Pot. Whenever a person is famous enough for inclusion and
facts about the person can be confirmed, we should include their biography in
Wikipedia. /Why/ a particular person is famous is not an important thing for
us to consider when allowing an article on them to be included in Wikipedia.
But we should, of course, state that information in the article and do so in
an NPOV way.

>It took repeated efforts to even make ZOG admit
>that there is more than one Johnny Rebel.  Can
>we do something about this ZOG guy?

I protected the page to stop the edit war and several people have been looking
over his edits. He may calm down and embrace NPOV or he may continue to be a
POV zealot and need to be banned. But he is new and we can't expect miracles
to occur over night.

-- Daniel Mayer (aka mav)


from_name: Ray Saintonge

subject: [WikiEN-l] Classification of China?

body:
Jimmy Wales wrote:

>Is China authoritarian?  Sure it is.  But it's not necessary for us
>to actualy make the claim ourselves.
>
Those who make the specific statement "China is authoritarian", and get
flamed for it, cannot validly defend themselves by saying "I didn't know
the word was loaded." :-)

Ec


subject: [WikiEN-l] Classification of China?

body:
Jimmy Wales wrote:

>Fred Bauder wrote:
>
>>We get all over those who would deny the holocaust; no reason not to
>>not adopt the same sort of disrespect and impose the same restraints
>>upon those who would deny the sins of Marxism-Leninism.
>>
>
>I agree completely, but neither case requires us to deviate from NPOV.
>
My complaint about Fred's argument in this is that he compared an event
with a philosophy.  The sinfulness that may be associated with
Marxism-Leninism is not a sin of the philosophy but of those who sought
to impose it.

Ec


body:
Brion, you're missing my point. I agree with you entirely that things need
to "get done". My suggestion to have a public discussion was to find out
which things we can get done reasonably quickly (because, realistically,
we all have other things to do) with substantial impact; to figure out the
server situation, which features should be disabled, who might contribute
which piece of code etc. If we can sort these things out in the next few
days via mail, fine. I'm no IRC junkie. But we need to implement at least
some reasonable emergency fixes, and think about a mid term strategy.

As for code, this is one thing I'd like to talk about: If we have the
Nupedia Foundation set up, we can collect donations. It would be stupid
not to use some of that money for funding development. I don't care who is
funded, but I think this could greatly speed things up. If we can't get
the NF set up reasonably quickly, we should collect donations regardless,
tax-deductible or not.

> * Page viewing is still kinda inefficient. Rendering everything on every
> view is not so good...

Why? It's just PHP stuff. Our bottleneck is the database server. Fetching
stuff from CUR and converting it into HTML is not an issue. 20 pass
parser? Add another zero. Until I see evidence that this has any impact on
performance, I don't care. Turn off link checking and all pages are
rendered lightning fast.

What would be useful is to maintain a persistent (over several sessions)
index of all existing and non existing pages in memory for the link
checking. A file on a ramdisk maybe? I think it would be worth giving it a
try at least, and not a lot of work.

> We need to tell which pages are or aren't cacheable (not a diff, not a
> special page, not a history revision, not a user with really weird
> display options -- or on the other hand, maybe we _could_ cache those,
> if only we can distinguish them), we need to be able to generate and
> save the cached material appropriately, we need to make sure it's
> invalidated properly, and we need to be able to do mass invalidation
> when, for instance, the software is upgraded. Cached pages may be kept
> in files, rather than the database.

Wasted effort, IMHO. Cache improvements have added little measurable
performance benefits, and there are many, many different situations to
test here (different browsers, different browser cache settings etc.).
Meanwhile, our real bottlenecks (search, special pages, out of control
queries) remain in place.

> * The page saving code is rather inefficient, particularly with how it
> deals with the link tables (and potentially buggy -- sometimes pages end
> up with their link table entries missing, possibly due to the system
> timing out between the main save chunk and the link table update). If
> someone would like to work on this, it would be very welcome. Nothing
> that needs to be _discussed_, it just needs to be _done_ and changes
> checked in.

I doubt that a *relatively* rare activity like that makes much of an
impact, but I'll be happy to be proven wrong. Bugs are annoying, but I'm
writing this for one reason: we need to make Wikipedia usable again on a
regular basis. There are countless small problems that need to be fixed.
This is not the issue here.

> * Various special pages are so slow they've been disabled. Most of them
> could be made much more efficient with better queries and/or by
> maintaining summary tables. Some remaining ones are also pretty
> inefficient, like the Watchlist. Someone needs to look into these and
> make the necessary adjustments to the code.

Caching special pages seems like a reasonable approach. Watchlists could
definitely be improved, haven't seen a good way to do this yet, though. It
could be done on page save, but with a much-watched page, this again would
add severe drain, with possibly no overall benefit. Improve the SQL and
indexes? Maybe, but I'm no SQL guru.

> * Can MySQL 4 handle fulltext searches better under load? Is boolean
> mode faster or slower? Someone needs to test this (Lee has a test rig
> with mysql4 already, but as far as I know hasn't tested the fulltext
> search with boolean mode yet), and if it's good news, we need to make an
> upgrade a high priority.

Sounds good to me. If safe enough, we should update in any case; it is my
understanding that MySQL4 has support for subqueries which could, if we
know what we're doing, potentially be used to write significantly more
effective queries.

Regards,

Erik


from_name: Kurt Jansson

body:
Brion Vibber schrieb:

> * Various special pages are so slow they've been disabled. Most of them
> could be made much more efficient with better queries and/or by
> maintaining summary tables. Some remaining ones are also pretty
> inefficient, like the Watchlist. Someone needs to look into these and
> make the necessary adjustments to the code.

Could we set the set the length of the watchlist to 50 or something like
that per default, and not making it dependent of the length you choose
in the preferences for the RecentChanges? (I for example have set it to
150, because otherwise the "show changes since ..." is cut off too
early. On the English Wp I'd have to set it even higher.)

> Jimbo, we really need some news on this front. If parts and/or a whole
> machine really *is* on order and can be set up in the near future, we
> need to know that. If it's *not*, then it may be time to pass around the
> plate and have interested parties make sure one does get ordered, as had
> begun to be discussed prior to the March 19 announcement.

Even with a second server, and the software and database being faster,
how long will it take until this again isn't enough because articles,
editors and visitors should be growing exponentially in theory. There
will be the foundation, and maybe we'll get some money through it and
can buy new hardware, but will it be sufficient? And for how long? And
aren't there other important things we could spend the money on, if an
other free project or a university would host us for free? But maybe I
cherish an illusion here.

Kurt


from_name: Jimmy Wales

subject: [WikiEN-l] Zog

in_reply_to: <200304290002.h3T02to06913@www.wikipedia.org>; from list@redhill.net.au on Tue, Apr 29, 2003 at 10:11:15AM +1000

body:
I'm satisfied, and this user should be banned.  Normally Ed does these
things for me, but he's taking a rest right now, so perhaps someone
else could handle it for me?

Tony Wilson wrote:

> We need to do something about Zog.
>
> Consider his contributions to History of Cuba: "Although Mexico is
> spanish-speaking, like Cuba, the filth level is considerably higher."
> Or: "This guerilla campaign would not have been able to be carried out
> without the gorillas which were a commonplace throughout Cuba. African
> Americans' participation in the revolution is very unique for this very
> reason."
>
> Or the entry "Thats how a nigger goes" which, aside from its racist
> content (which can be guessed without too much trouble)  consisted of
> little but a complete copy of the song lyrics, which I see no need to
> repeat here.
>
> This user previously added offensive anti-Semetic material to several
> articles as an anon, until blocked.
>
> Tony Wilson
> (Tannin)
>
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikien-l


references: <8khdmaoCpVB@erik_moeller>	 <1051569281.29985.331.camel@frank.vibber.org>  <3EADCFAC.7070300@gmx.net> <1051577690.30555.33.camel@frank.vibber.org>

body:
Brion Vibber schrieb:
> On Mon, 2003-04-28 at 18:04, Kurt Jansson wrote:
>
>>Could we set the set the length of the watchlist to 50 or something like
>>that per default, and not making it dependent of the length you choose
>>in the preferences for the RecentChanges?
>
> Well, that wouldn't help for performance as the data goes through a
> temporary table. Basically the DB's grabbing your *entire* watchlist,
> then only sending the most recent X items to the wiki for formatting in
> a list.

I see. I hadn't thought this through completely.
(Maybe a link in the Watchlist for easy removal of articles would help
people to keep their watchlist small and tidy.)

>>Even with a second server, and the software and database being faster,
>>how long will it take until this again isn't enough because articles,
>>editors and visitors should be growing exponentially in theory. There
>>will be the foundation, and maybe we'll get some money through it and
>>can buy new hardware, but will it be sufficient? And for how long?
>
> How long will the internet be able to deal with all those new users?
> Won't we run out of IP addresses if IPv6 never rolls out? When will the
> sun burn out, leaving the earth a lifeless ball of coal?? :) Hopefully,
> we'll be able to keep up.

Okay, I'll remind you in a year or two :-)

> Do feel free to ask other free projects and universities if they'd be
> interested in supporting the project...

I'll do. Could you describe what our requirements are? (Sorry, I'm not
very experienced with this server stuff. I'm just trying to install
Debian with a friends help.) - I'll go tromping (right word?) in the
three universities in Berlin then.

Kurt


body:
Brion Vibber schrieb:
> On Mon, 2003-04-28 at 16:46, Erik Moeller wrote:
>> Improve the SQL and indexes? Maybe, but I'm no SQL guru.
>
> Which Himalayan mountain do we have to climb to find one? :)

Wasn't there someone on this list a while ago who has written a book
about MySQL? Or am I fantasising about this?

Kurt


body:
> Well maybe, but my experience with using online chats like this is:
> * Everyone sits around for several hours babbling, waiting for the other
> folks to show up and complaining about the problems they're having
> logging in.
> * By the end, someone has scribbled up a page with a work plan, which
> everyone ignores in the future.
> * During all this time, they _could_ have been doing something
> productive instead...

Depends on the moderator. No moderation=unpredictable, bad moderator=bad
result, good moderator=possibly good result. Just like in real life
meetings.

> Obviously not, since that PHP stuff needs data from the database to
> work. :)

Duh. But if our database is so slow that it can't even answer simple
SELECTs, we can't do anything useful, cache or no cache. And if it isn't,
then we should concentrate on the queries which aren't simple. The
linkcache might still be one of those bottlenecks (simply because of the
sheer number of queries involved), I haven't checked your latest changes
to that code.

>>  Our bottleneck is the database server. Fetching
>> stuff from CUR and converting it into HTML is not an issue. 20 pass
>> parser? Add another zero. Until I see evidence that this has any impact on
>> performance, I don't care. Turn off link checking and all pages are
>> rendered lightning fast.

> And that would be a pretty piss-poor wiki, wouldn't it? :)

Yes, but this is really one of the more expensive wiki features that also
limits all caching options severely. Impossible to work without it, but
apparently hard to implement in a scalable fashion.

>> What would be useful is to maintain a persistent (over several sessions)
>> index of all existing and non existing pages in memory for the link
>> checking. A file on a ramdisk maybe? I think it would be worth giving it a
>> try at least, and not a lot of work.

> Sure, it _might_ help. Code it up and see!

I might. I'll have to see if it makes any difference on the relatively
small de database which I'm currently using locally. It would have to be
optional -- setting up the software is already difficult enough.

> Slow saving impacts everyone who tries to edit articles; four edits per
> minute may be _relatively_ rare compared to page views, but we're still
> running thousands of edits per day and it's a fundamental part of what a
> wiki is. It's absolutely vital that editing be both swift and bug-free,
> and if we can reduce the opportunities for saving to get hung up, so
> much the better.

Yeah yeah yeah. I still think we should care more about the real
showstoppers. But hey, you can always _code it_. (Finally an opportunity
to strike back ;-)

> If updating these cached pages is so slow and db-intensive that it takes
> the 'pedia offline for fifteen-twenty minutes (which it does), then
> nobody's going to want to update the caches (last updated April 9...)
> and they become outdated and useless.

If this downtime is unacceptable, we might indeed have to think about a
query only server with somewhat delayed data availability. This could be a
replacement for the sysops, too. Mirroring the Wikipedia database files
(raw) should be no issue with a SCSI system, or a low priority copy
process.

>>  Watchlists could
>> definitely be improved, haven't seen a good way to do this yet, though. It
>> could be done on page save, but with a much-watched page, this again would
>> add severe drain, with possibly no overall benefit. Improve the SQL and
>> indexes? Maybe, but I'm no SQL guru.

> Which Himalayan mountain do we have to climb to find one? :)

Maybe we should stop looking in the Himalayan mountains and start
searching the lowlands ..  In other words: Don't search those who will do
it for society or for the glory. Just hand over the cash and be done with
it.

Regards,

Erik


from_name: Karl Eichwalder

subject: [Wikipedia-l] Re: Server overload?

body:
Brion Vibber <brion at pobox.com> writes:

> Yes, the server is overloaded because there are too many people trying
> to use the site.

If the server cannot answer within a acceptable time frame (10 s?), it
should say so.  Letting editors wait for a longer period of time, is
userunfriendly.

If the database is too busy, return this info to the editor
immediately, please.

--
                                                         |      ,__o
http://www.gnu.franken.de/ke/                            |    _-\_<,
ke at suse.de (work) / keichwa at gmx.net (home)               |   (*)/'(*)


from_name: Fred Bauder

subject: [WikiEN-l] Classification of China?

body:
Its the subject that is loaded. But it is and has been a part of our world
and is thus a part of wikipedia (in appropriate contexts).

Fred

> From: Ray Saintonge <saintonge at telus.net>
> Reply-To: wikien-l at wikipedia.org
> Date: Tue, 29 Apr 2003 00:35:22 -0700
> To: wikien-l at wikipedia.org
> Subject: Re: [WikiEN-l] Classification of China?
>
> Jimmy Wales wrote:
>
>> Is China authoritarian?  Sure it is.  But it's not necessary for us
>> to actualy make the claim ourselves.
>>
> Those who make the specific statement "China is authoritarian", and get
> flamed for it, cannot validly defend themselves by saying "I didn't know
> the word was loaded." :-)
>
> Ec
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikien-l


subject: [WikiEN-l] Classification of China?

body:
Heh, it is a philosophy of imposition.

Fred

> From: Ray Saintonge <saintonge at telus.net>
> Reply-To: wikien-l at wikipedia.org
> Date: Tue, 29 Apr 2003 00:47:22 -0700
> To: wikien-l at wikipedia.org
> Subject: Re: [WikiEN-l] Classification of China?
>
> Jimmy Wales wrote:
>
>> Fred Bauder wrote:
>>
>>> We get all over those who would deny the holocaust; no reason not to
>>> not adopt the same sort of disrespect and impose the same restraints
>>> upon those who would deny the sins of Marxism-Leninism.
>>>
>>
>> I agree completely, but neither case requires us to deviate from NPOV.
>>
> My complaint about Fred's argument in this is that he compared an event
> with a philosophy.  The sinfulness that may be associated with
> Marxism-Leninism is not a sin of the philosophy but of those who sought
> to impose it.
>
> Ec
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikien-l


from_name: Hunter Peress

subject: [Wikipedia-l] Re: Server overload?

body:
Its amazingly simple what needs to be done:

either

software wise:
make wikipedia less database reliant, namely, most of wikiepedia's CPU time is being wasted on
relational operations for ultimately simple queries. EG, when you go to a page, it hits the
Database, this is clearly unnaccetable for a site of this scale. Simple solution is to dump out
current versions of articles to file for orders of performance increase. And you could even have
pre-prepared edit pages as well.  And im even told that the database is still being hit for
"recent-changes", whereas it should be dumped to file every 1-5 minutes.

hardware wise:
put up a donation sign and run a completely transparent donation campaign. eg: say "we need $4000
for a new server, we have 2000 so far".

The hardware donation issue will eventually become necessary, but it would be an emabarrassing
course of action given what should have been done software wise ages ago.

I'll do it if you give me an admin account.

--- Karl Eichwalder <ke at gnu.franken.de> wrote:
> Brion Vibber <brion at pobox.com> writes:
>
> > Yes, the server is overloaded because there are too many people trying
> > to use the site.
>
> If the server cannot answer within a acceptable time frame (10 s?), it
> should say so.  Letting editors wait for a longer period of time, is
> userunfriendly.
>
> If the database is too busy, return this info to the editor
> immediately, please.
>
> --
>                                                          |      ,__o
> http://www.gnu.franken.de/ke/                            |    _-\_<,
> ke at suse.de (work) / keichwa at gmx.net (home)               |   (*)/'(*)
> _______________________________________________
> Wikipedia-l mailing list
> Wikipedia-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikipedia-l

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com


subject: [WikiEN-l] More about Zog

body:
In a post to the Johnny Rebel article, Zog wrote:

"The main reason why nothing was determined through "talking" is because
Danny refused to talk and defend his arguments. Yet, its his version that
wins out in the end? Very un-diplomatic"

I will reiterate my position here. Zog is a racist and a bigot and an
anti-Semite. He has written about the "gorillas" in Cuba who were behind the
revolution, the level of filth in various Latin American countries, and given
a list of racist epithets under the heading "song titles." His contributions,
particularly the earliest versions, are even more hateful. For example, he
write: "In JR's songs, are powerful messages about unity, equality, freedom,
and creating a better America. Although a bit silly at times, like with "KKK
man", which talks about killing Bill Cosby, bludgeoning Urkel, and extreme
hatred for Dr. Zaius, JR's Political messages have been poignant, political
and innovating for those in his hometown of Lousiana, and beyond."

Zog's very name is anti-Semitic. ZOG is an acronym for "Zionist Occupation
Government," a term used by neo-Nazis and White Power advocates to promote
the conspiracy theory that a cabal of Jews is secretly running the government
of the United States. Zog denied even knowing this, however, in his original
draft of the Johnny Rebel article he quoted David Duke as saying: "I have
been in politics for several decades and I will be in for several more, but I
always will never be able to articulate the way ZOG controls the media, the
coloreds want their hand-outs and how chimps DO, in fact, want their 'ugly
stick' as well as JR has." In other words, he is fully aware of the racist
connotations of his name.

I, for one, will not engage Zog in dialogue. To debate the content of his
articles is to grant the subject-matter legitimacy.  To debate whether people
of African descent are "gorillas" is dehumanizing to them. Personally, I
believe it is dehumanizing to anyone involved in the debate. To discuss which
songs of his to list (why "Some Niggers Never Die" or "No Niggers in Our
Schools" but not "Coon Shootin' Boogie"--for non-Americans, "coon" is another
derogatory term for African-Americans) is similarly offensive. To discuss
whether he is a member of the Ku Klux Klan or simply has views that run
parallel to theirs (though he did record the "KKK Song") is an attempt to
evade the point.

My position is that entering into dialogue with Zog grants him and his views
legitimacy. Sifting through his material so that his additions to the [[Chaco
War]] can stay, but comments like those he put in the [[History of Jews in
the Soviet Union]] should be removed is also an exercise in futilty. Do we
have to go through each of his edits to determine whether they are blatantly
racist or not.

For this reason, I will not enter into dialogue with him. I will remove his
changes as I see them, and hope that he will be banned. I hope people support
me in this. If you have any doubts, just look at the history of his user
contributions. This is certainly not the kind of person we want in Wikipedia.

Danny

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/wikien-l/attachments/20030429/6b796cf3/attachment.htm


subject: [Wikitech-l] SuperMiserMode

body:
I forced the site into SuperMiserMode to deal with the overloading
this morning.  Maybe it was in SuperMiserMode already, though.

# TEMPORARILY ADDED BY JIMBO 4/29/03
$wgSuperMiserMode = true;
$wgMiserMode = true;

In LocalSettings.php

This is just an emergency thing, so I just edited the live file,
I have nothing useful for CVS!


from_name: sannse

subject: [WikiEN-l] Lir is back

body:
Zoe said:

> He isn't even trying to hide now.  He's calling
> himself Ril.

Much as I hate troll-spotting...

Ril has actually said that he has been banned - and I don't think he is
Michael

http://www.wikipedia.org/w/wiki.phtml?title=User_talk:John_Stewart&diff=8727
27&oldid=872711

sannse


subject: [Wikitech-l] Every little bit helps theory

body:
On the "every little bit helps" theory, I've offloaded wiki.png to a
different machine for now.  At least that's going to free up some
httpd's to do something else.


subject: [Wikitech-l] [Sales@Computers4SURE.com: We've received your order #C030450733, Jimmy. Thank you.]

body:
Motherboard for the server we're going to use as Wikipedia front end.
I haven't heard from Jason yet as to when he can make the trip to
install it, but possibly Friday or Monday.

----- Forwarded message from Sales at Computers4SURE.com -----

From: <Sales at Computers4SURE.com>
Date: Tue, 29 Apr 2003 11:29:37 -0400
To: <jwales at bomis.com>
Subject: We've received your order #C030450733, Jimmy. Thank you.

Thank you for shopping with Computers4SURE.com. We would like to confirm
that your order has been received.

On 29-Apr-2003, you ordered:

Item		         Quantity	           Price
----------------------------------------------------------
SERVERWORKS LE-T DUAL PGA370	1	$485.95

-----------------SNIP-----------------------


subject: [WikiEN-l] Classification of China?

body:
Fred Bauder wrote:

>Heh, it is a philosophy of imposition.
>
>Fred
>
What quotes do you have from Marx or Lenin to back that up.  Marx was
never in power, and Lenin never went half way around the world to impose
regime change.  Marx, in particular, was probably na?ve enough to
believe that capitalism would collapse of its own accord.

Ec

>>Jimmy Wales wrote:
>>
>>>Fred Bauder wrote:
>>>
>>>>We get all over those who would deny the holocaust; no reason not to
>>>>not adopt the same sort of disrespect and impose the same restraints
>>>>upon those who would deny the sins of Marxism-Leninism.
>>>>
>>>I agree completely, but neither case requires us to deviate from NPOV.
>>>
>>My complaint about Fred's argument in this is that he compared an event
>>with a philosophy.  The sinfulness that may be associated with
>>Marxism-Leninism is not a sin of the philosophy but of those who sought
>>to impose it.
>>


from_name: Brion Vibber

subject: [Wikitech-l] SuperMiserMode

body:
On Tue, 2003-04-29 at 07:27, Jimmy Wales wrote:
> I forced the site into SuperMiserMode to deal with the overloading
> this morning.  Maybe it was in SuperMiserMode already, though.

Yes, it was already in SuperMiserMode. (That's the mode that pops up the
google search box instead of actually searching.)

Any news on hardware, or should we start passing the plate?

-- brion vibber (brion @ pobox.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20030429/29dc4f46/attachment.pgp>


subject: [WikiEN-l] Zog

body:
We need to do something about Zog.

Consider his contributions to History of Cuba: "Although Mexico is
spanish-speaking, like Cuba, the filth level is considerably higher."
Or: "This guerilla campaign would not have been able to be carried out
without the gorillas which were a commonplace throughout Cuba. African
Americans' participation in the revolution is very unique for this very
reason."

Or the entry "Thats how a nigger goes" which, aside from its racist
content (which can be guessed without too much trouble)  consisted of
little but a complete copy of the song lyrics, which I see no need to
repeat here.

This user previously added offensive anti-Semetic material to several
articles as an anon, until blocked.

Tony Wilson
(Tannin)


subject: [WikiEN-l] Classification of China?

in_reply_to: <3EAEAEF0.1050008@telus.net>; from saintonge@telus.net on Tue, Apr 29, 2003 at 09:57:20AM -0700

body:
Please let's not discuss politics on the list.

Ray Saintonge wrote:

> Fred Bauder wrote:
>
> >Heh, it is a philosophy of imposition.
> >
> >Fred
> >
> What quotes do you have from Marx or Lenin to back that up.  Marx was
> never in power, and Lenin never went half way around the world to impose
> regime change.  Marx, in particular, was probably na?ve enough to
> believe that capitalism would collapse of its own accord.
>
> Ec
>
> >>Jimmy Wales wrote:
> >>
> >>>Fred Bauder wrote:
> >>>
> >>>>We get all over those who would deny the holocaust; no reason not to
> >>>>not adopt the same sort of disrespect and impose the same restraints
> >>>>upon those who would deny the sins of Marxism-Leninism.
> >>>>
> >>>I agree completely, but neither case requires us to deviate from NPOV.
> >>>
> >>My complaint about Fred's argument in this is that he compared an event
> >>with a philosophy.  The sinfulness that may be associated with
> >>Marxism-Leninism is not a sin of the philosophy but of those who sought
> >>to impose it.
> >>
>
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikien-l


subject: [Wikitech-l] SQL Request

body:
to Thomas Corell :

> Sorry, but can you stop writing french in Wikitech-l? Try an appropriate
> wikifr-l. I don't have any time to learn french only for this
international
> mailinglist. Thanks.

Sorry but I just answered Tarquin who has the kindness to answer my english
question in my native language.

to Tomasz Wegrzanowski :

> No, it's ok to write in any language.
> Especially apropriate are multilingual posts.

Thanks, I will try to post in english (or both).

to Jason Richey :

> Just a thought...  Wouldn't it be less load on the server if someone
> were to download the fr database dump, run a query on their local
> machine to figure out what entries need updated, then run several
> specific queries on the live database to update the appropriate
> entries?
>
> This way, we could avoid the full text matching on "%whatever%" to
> discover which entries need updating.

If we download the "current" french database each time we want to run a
query (to get the last updated entry) it is heavy, isn't it ?

> Please forgive me if my SQL knowledge is lacking, or if a solution has
> already been found during discussion in French...  I'd love to read
> (in English) what was said in the French discussion of this thread, if
> anyone cares to translate...

We only discuss the better time to run a query according to the americans
sleeping time ;o)
By the way I didn't get the answer yet, what is the best time to run queries
(not especially this one) ?
It perhaps not the right time to speak about query since you have so much
performence problems !
To be continued... (later)

Aoineko


subject: [WikiEN-l] Zog

body:
 Danny says: "I vote to get rid of him. With his racist rants, there
really isn't much reason for debate."

Seconded.

Tony Wilson
(Tannin)


subject: [WikiEN-l] Classification of China?

body:
I don't think that Fred appreciates the power of the word.  "In the
beginning was the word...."  To the Irish the word has been their most
powerful defensive weapon in their battles against the beligerent
warlords on the island to the east.

The choice of words which one makes can easily colour the POV of any
article more effectively than blatant lies.  China is what China is.
 If, as Fred proposes, China is authoritarian the facts will speak for
themselves without the need for it to be spelled out.

Ec

Fred Bauder wrote:

>Its the subject that is loaded. But it is and has been a part of our world
>and is thus a part of wikipedia (in appropriate contexts).
>
>Fred
>
>>
>>Jimmy Wales wrote:
>>
>>>Is China authoritarian?  Sure it is.  But it's not necessary for us
>>>to actualy make the claim ourselves.
>>>
>>Those who make the specific statement "China is authoritarian", and get
>>flamed for it, cannot validly defend themselves by saying "I didn't know
>>the word was loaded." :-)
>>
>>Ec
>>


from_name: Jason Richey

body:
I'll be able to make the trip to San Diego on Friday.  The server
should be ready to use (ssh-able for all people with accounts on the
current server) on Saturday.

Should I install Apache?  Anything sepcial besides mod-php?  Any
special compile-time options for PHP itself?

Jason

Jimmy Wales wrote:

> Motherboard for the server we're going to use as Wikipedia front end.
> I haven't heard from Jason yet as to when he can make the trip to
> install it, but possibly Friday or Monday.
>
> ----- Forwarded message from Sales at Computers4SURE.com -----
>
> From: <Sales at Computers4SURE.com>
> Date: Tue, 29 Apr 2003 11:29:37 -0400
> To: <jwales at bomis.com>
> Subject: We've received your order #C030450733, Jimmy. Thank you.
>
> Thank you for shopping with Computers4SURE.com. We would like to confirm
> that your order has been received.
>
> On 29-Apr-2003, you ordered:
>
> Item		         Quantity	           Price
> ----------------------------------------------------------
> SERVERWORKS LE-T DUAL PGA370	1	$485.95
>
>
> -----------------SNIP-----------------------
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikitech-l

--
"Jason C. Richey" <jasonr at bomis.com>


subject: [Wikitech-l] kernel on wiki server

body:
I know there was some discussion about kernel upgrade of the current
wiki server.  I'll be in front of the servers Friday evening, so I
could easily make such an upgrade.

I am familiar with the kernel and compiling and installing (I use
LILO, if that's okay with everyone) and such.  So, if we can reach a
consensus about the kernel version, I can manage.  Alternatively, if
there is already one compiled and ready, I can just install that.

--
"Jason C. Richey" <jasonr at bomis.com>


subject: [Wikitech-l] two servers.

body:
If I recall correctly, both of the servers have serial ports
available.  I don't know much (or anything) about serial terminals,
but I can install a null modem cable between the two.  I'll look into
what is involved in setting up a serial terminal...

--
"Jason C. Richey" <jasonr at bomis.com>


subject: [Wikitech-l] server names

body:
I can barely stand to refer to the current wiki server as "the current
wiki server", and I don't even know how I'll refer to the new server.
I'd like to give the servers official names, just so I can refer to
them in a single word.

Any suggestions?

--
"Jason C. Richey" <jasonr at bomis.com>


body:
Lee Daniel Crocker wrote:

> I left instructions for how I configured and installed everything
> on the old server, but it's rather tricky, so you might just want to
> let me install this one too

Be my guest...  I'll send you a mail as soon as ssh is working on the
new server.

> (you can check out the process yourself
> at /usr/wikidev/docs/INSTALL, skipping the MySQL parts). I've also
> updated the procedure a bit based on some testing I've done recently.
> As before I'll document everything step-by-step as I go.
> We should also figure out a procedure for installing a testing all
> the wikis, then scheduling a DNS switchover.
>

I am in favor of a quick IP swap instead of a DNS switch.
Register.com's DNS management is clumsy at best, and DNS is slow at
propagating anyway.  By swapping IP's, we can avoid the pain of
waiting for DNS to change (and hoping pages are served correctly in
the interim).

So, I think the new server should take the current server's IP (when
we go live), and the current server should get a new IP.  This implies
that the mailing list software should either be moved to the new
server, or an MX record should be added to DNS (simple in relation to
changing dozens of subdomains) to point to the new ip of the current
server.

> Brion, now's the time to clean up all the ad-hoc changes made to
> the live wikis and get them into LocalSettings. I'm sure there are
> a few I've missed--can you get me a complete list?
>
> --
> Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
> "All inventions or works of authorship original to me, herein and past,
> are placed irrevocably in the public domain, and may be used or modified
> for any purpose, without permission, attribution, or notification."--LDC
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikitech-l

--
"Jason C. Richey" <jasonr at bomis.com>


body:
Nick Reinking wrote:
> I have a much better idea to handle these files, I'll detail it shortly.
> But with HTTP/1.1, moving those files will pages load _slower_ for most
> people (since they have to do an additional DNS lookup and connect()),
> rather than piggybacking off an already existing HTTP/1.1 connection.

Under normal circumstances, I would agree.  But the machine was just
about *this far* from falling over completely, and pages were not
really getting served *at all*.  At least the site works for now. :-)

But I'd love to hear your suggestions, too!

Anyhow, since a new server is going to be installed this weekend,
hopefully hardware will save the day... for now.  :-)


subject: [Wikitech-l] kernel on wiki server

body:
I can understand the ease of installation of an RPM kernel, but I
think we would do well to use a kernel that was compiled with our
hardware in mind.  Surely, we don't need sound support or other such
things boating our kernel, and we might enjoy a little extra
performance from a kernel that doesn't have to use modules to access
it's primary IO devices.

Jason

Nick Reinking wrote:

> On Tue, Apr 29, 2003 at 11:12:18AM -0700, Jason Richey wrote:
> > I know there was some discussion about kernel upgrade of the current
> > wiki server.  I'll be in front of the servers Friday evening, so I
> > could easily make such an upgrade.
> >
> > I am familiar with the kernel and compiling and installing (I use
> > LILO, if that's okay with everyone) and such.  So, if we can reach a
> > consensus about the kernel version, I can manage.  Alternatively, if
> > there is already one compiled and ready, I can just install that.
>
> I think you should use the Redhat kernel.  :)  I believe the current
> version for us (after a cursory look) appears to be:
>
> kernel-smp-2.4.18-27.7.x.athlon.rpm (for Redhat 7.2).
>
> --
> Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikitech-l

--
"Jason C. Richey" <jasonr at bomis.com>


subject: [Wikitech-l] server names

body:
Jason Richey wrote:
> I can barely stand to refer to the current wiki server as "the current
> wiki server", and I don't even know how I'll refer to the new server.
> I'd like to give the servers official names, just so I can refer to
> them in a single word.

I propose db.wikipedia.org for the backend and some system like
www1.wikipedia.org for the frontend (web) server.  These are internal
names, not for the public, because of course the public doesn't care
about that.

In this way, when we add more frontend machines (inevitable) and a
loadbalancing system, we will have a good naming scheme, www2, www3
and so on.


subject: [WikiEN-l] Lir is back

body:
No, Michael and Lir are different people.

Zoe

--- sannse <sannse at delphiforums.com> wrote:
> Zoe said:
>
> > He isn't even trying to hide now.  He's calling
> > himself Ril.
>
> Much as I hate troll-spotting...
>
> Ril has actually said that he has been banned - and
> I don't think he is
> Michael
>
>
http://www.wikipedia.org/w/wiki.phtml?title=User_talk:John_Stewart&diff=8727
> 27&oldid=872711
>
> sannse
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikien-l

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com


from_name: Nick Reinking

body:
On Mon, Apr 28, 2003 at 05:45:18PM -0500, Lee Daniel Crocker wrote:
> > We don't need to sit and chat. We need *code* and we need a
> > second server to divide the "must-do-fast" web work the and
> > "chug-chug-chug" database labor.
>
> The code issue is mostly a matter of focus: one or two developers
> is probably sufficient to keep the codebase up to date, but neither
> Brion nor I are focused on that right now.
>
> So while Brion has issued a call for coders, that could be answered
> in other ways: for example, if a good admin stepped up to take some
> of admin tasks Brion is currently swamped with, he might be more free
> to code (assuming he's interested, which is not a given either).
> I've chosen to focus more on long-term goals because like Brion I was
> expecting hardware to bail us out in the short term. If that's going
> to be delayed, then I can put off things like testing file systems
> and focus on caching and tuning.

I'm certainly willing to help out here.  I'm not in SoCal, but I should
be able to help out with most administrivial tasks.  I'm going to be
able to help out a much more with tuning at a file system/OS/Apache
level than I will be at a PHP/SQL level.

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


subject: [Wikitech-l] kernel on wiki server

body:
No offense, Nick, but we've been doing this for a long time, with some
very high traffic websites, and Jason knows what he's doing.


body:
Lee Daniel Crocker wrote:
> This has been mentioned before, but how about moving the mailings
> lists off entirely?  That way, users can still communicate when the
> server is overloaded.

I think this is a reasonable idea, and I'll think about what would be
best on this.  We're running a variety of mailing lists for a variety
of domains on various machines right now.  Possibly it would make sense
to unify everything.

Jason and I will chat about it and get back to the mailing list on
this question.


from_name: cprompt

subject: [WikiEN-l] Re: More about Zog

body:
Mirko Thiessen wrote:

> Danny wrote:
>
>>
>> For this reason, I will not enter into dialogue with him. I will
>> remove his changes as I see them, and hope that he will be banned. I
>> hope people support me in this. If you have any doubts, just look at
>> the history of his user contributions. This is certainly not the kind
>> of person we want in Wikipedia.
>>
>
>
> Well stated, Danny. I agree with any single sentence. If anyone
> thinks, that we can convince a racist like Zog of writing NPOV, he/she
> is in error. I would feel *very* uncomfortable to know such a person
> as a tolerated Wikipedia contributor. I hope, that we can get rid of
> him and never hear about him again.
>
>
> Mirko (Cordyph)
>
    As a matter of procedure, for any new user who violates the rules,
    we should give him a chance to learn the rules and stick around. Zog
    doesn't appear to be interested in following the rules.


from_name: Lee Daniel Crocker

body:
> (Jimmy Wales <jwales at bomis.com>):
> On the "every little bit helps" theory, I've offloaded wiki.png to a
> different machine for now.  At least that's going to free up some
> httpd's to do something else.

You might move the stylesheets over there as well. The software
is also quite happy to serve all images from a different site, but
that's more work because you have to set up an NFS share for uploads.

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


subject: [Wikitech-l] SuperMiserMode

body:
> (Brion Vibber <brion at pobox.com>):
>
> Yes, it was already in SuperMiserMode. (That's the mode that pops up the
> google search box instead of actually searching.)

Just FYI, please note that the code in CVS changes that variable
to "$wgSearchDisabled"; I prefer variable names that are descriptive
of what they actually do, not reasons for doing it.

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


subject: [Wikitech-l] server names

body:
Tomasz Wegrzanowski wrote:
> But these names are sooo boring ;-)
> And they don't follow RFC 1178.

You're right, and I retract my suggestion completely if someone comes
up with something more fun and standards-compliant!  :-)

Famous encyclopedists?

Pliny, Vincent of Beavais, Denis Diderot, Jimbo? (ha ha)

Or, how about, for an infinite supply of names, we use various
incarnations of Lir

Lir, Adam, Bridget, Vera Cruz, Ril, etc., etc., etc., etc.!!!

--Jimbo


from_name: Leonard Tulipan

subject: [Wikitech-l] Performance Problem - Caching solutions

body:
Hi!

With all the talk going on, I wanted to give you my little thoughts.
I am more on the wiki user side, so keep that in mind (I do know php, so I
am not a complete dork)

1) how about a query cache (for edits and searches).
http://www.vbulletin.com/ (not OpenSource), does something like that for
search queries:
http://forum.doom9.org/search.php
Basically you instantly see a html page, which tells you that the search is
in progress. This looks good for the user. The page reload can probably be
configured in intervals. That way the best server load (e.g: 4 searches /
second) can be obtained and it will never be used more that what you specify
I think http://www.phpbb.com/downloads.php (GPL'ed) can do the same, but I
am not sure.
The thing is, when users see, that the server is doing something, they will
not try to hit refresh every few seconds, because they think it helps (which
it doesn't). User feedback is and always should be top priority.

2) What is really slowing down wikipedia
As I see it, not everyone agrees on that. I am also guessing it's the db,
and not the php-code that does the rendering. Maybe someone can somehow put
this together, like searches (7/second avg. - serverload: 30%),
edit(1/second: - serverload: 5%), recentchanges ( 5/second, 25% load), etc.
maybe some real numbers will help focus on the problem.
So If I am guessing right, and the sql-query for the searches takes the most
time, building up a queue for searches like above might satisfy users (at
least they know know something is happening) and it will reduce serverload.

I hope this helps the discussion somewhat.

Cheers
Leo


from_name: Guillaume Blanchard

subject: [Wikitech-l] Accent in mispeelings page

body:
Hi,
The "Pages with misspellings" don't care the accent common misspellings we add in "Wikip?dia:List of common misspellings".
"e", "?", "?" and "?" are interpret all as same. It's useful in search page, but very bad for misspellings detection.
Do "search" & "misspellings" using the same algorithm ?
Can we solve it easly ?

Aoineko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20030429/2ea36ef7/attachment.html>


subject: [Wikitech-l] Mail server box - since Jason is going to San Diego anyway...

body:
Jason wrote:
>I'll be able to make the trip to San Diego on Friday.

>The server should be ready to use (ssh-able for all
>people with accounts on the current server) on
Saturday.

It might be more trouble than it is worth, but do you
want me to send you the AMD 500 Linux box I have it
serve as the new mail server? If you can use it I'm
more than happy to send it but if another solution is
easier/makes more sense (like having Lee's ISP host
WikiTech-L) then I'm cool with that too.

I don't want to force hardware on anybody - all I want
is for the developers to be able to talk to each other
via a mailing list when the Wikipedia server is down.

--Daniel Mayer (aka mav)

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com


body:
On Tue, Apr 29, 2003 at 12:10:53PM -0500, Lee Daniel Crocker wrote:
> > (Jimmy Wales <jwales at bomis.com>):
> > On the "every little bit helps" theory, I've offloaded wiki.png to a
> > different machine for now.  At least that's going to free up some
> > httpd's to do something else.

I'd like to point out that I think this is a bad thing(tm) to do.  Not
only will some users not receive the image (because they have their
browser set up to not accept images that don't originate from the
original server), but it will also cause a lot of browsers to fail to
load the page if jimmywales.com goes down (or at least they'll hang and
hang and hang...).

I have a much better idea to handle these files, I'll detail it shortly.
But with HTTP/1.1, moving those files will pages load _slower_ for most
people (since they have to do an additional DNS lookup and connect()),
rather than piggybacking off an already existing HTTP/1.1 connection.

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


body:
> (Jason Richey <jasonr at bomis.com>):
>
> I'll be able to make the trip to San Diego on Friday.  The server
> should be ready to use (ssh-able for all people with accounts on the
> current server) on Saturday.
>
> Should I install Apache?  Anything sepcial besides mod-php?  Any
> special compile-time options for PHP itself?

I left instructions for how I configured and installed everything
on the old server, but it's rather tricky, so you might just want to
let me install this one too (you can check out the process yourself
at /usr/wikidev/docs/INSTALL, skipping the MySQL parts). I've also
updated the procedure a bit based on some testing I've done recently.
As before I'll document everything step-by-step as I go.

We should also figure out a procedure for installing a testing all
the wikis, then scheduling a DNS switchover.

Brion, now's the time to clean up all the ad-hoc changes made to
the live wikis and get them into LocalSettings. I'm sure there are
a few I've missed--can you get me a complete list?

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


subject: [Wikitech-l] kernel on wiki server

body:
On Tue, Apr 29, 2003 at 11:12:18AM -0700, Jason Richey wrote:
> I know there was some discussion about kernel upgrade of the current
> wiki server.  I'll be in front of the servers Friday evening, so I
> could easily make such an upgrade.
>
> I am familiar with the kernel and compiling and installing (I use
> LILO, if that's okay with everyone) and such.  So, if we can reach a
> consensus about the kernel version, I can manage.  Alternatively, if
> there is already one compiled and ready, I can just install that.

I think you should use the Redhat kernel.  :)  I believe the current
version for us (after a cursory look) appears to be:

kernel-smp-2.4.18-27.7.x.athlon.rpm (for Redhat 7.2).

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


subject: [Wikitech-l] server names

body:
On Tue, Apr 29, 2003 at 11:19:10AM -0700, Jason Richey wrote:
> I can barely stand to refer to the current wiki server as "the current
> wiki server", and I don't even know how I'll refer to the new server.
> I'd like to give the servers official names, just so I can refer to
> them in a single word.
>
> Any suggestions?

How about "sweat" and "tears", or "pain" and "agony"?  ;)

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


from_name: Nicolas Delahaye

subject: [Wikitech-l] error during installation

body:
it worked for me too, after the bug fix.
got the same problem as Krzysztof for the styles, but overall, that works.
Thanks!

Nicolas

----- Original Message -----
From: "Lee Daniel Crocker" <lee at piclab.com>
To: <wikitech-l at wikipedia.org>
Sent: Monday, April 28, 2003 10:10 AM
Subject: Re: [Wikitech-l] error during installation

> > (Krzysztof Kowalczyk <krzysztofk at pobox.com>):
> > Apologies if this is something obvious.
> >
> > I tried to install the latest CVS phase3 code on RH Linux 7.2, MySQL
> > 3.23.54, PHP 4.3.1. I got the following error when running "php
> > install.php":
>
> Thanks for the report--yes, that's a bug. The original encryptPassword()
> function was static, but the recent change to add a salt to the
> encryption used a non-static function. I'm not sure why the install
> script has been working for me, but at any rate I'll fix it right away.
>
> --
> Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
> "All inventions or works of authorship original to me, herein and past,
> are placed irrevocably in the public domain, and may be used or modified
> for any purpose, without permission, attribution, or notification."--LDC
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikitech-l
>


body:
> (Jason Richey <jasonr at bomis.com>):
>
> So, I think the new server should take the current server's IP (when
> we go live), and the current server should get a new IP.

For testing, then, the current server should get that new IP now,
so that we can install Apache on the new one, configure it to talk
to mysql at that IP, and then switch it over without having to
reconfigure. Can you switch IPs remotely?

> This implies that the mailing list software should either be moved
> to the new server, or an MX record should be added to DNS (simple in
> relation to changing dozens of subdomains) to point to the new ip of
> the current server.

This has been mentioned before, but how about moving the mailings
lists off entirely?  That way, users can still communicate when the
server is overloaded.

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


subject: [Wikitech-l] kernel on wiki server

body:
On Tue, Apr 29, 2003 at 11:36:10AM -0700, Jason Richey wrote:
> I can understand the ease of installation of an RPM kernel, but I
> think we would do well to use a kernel that was compiled with our
> hardware in mind.  Surely, we don't need sound support or other such
> things boating our kernel, and we might enjoy a little extra
> performance from a kernel that doesn't have to use modules to access
> it's primary IO devices.
>
> Jason

I don't think that we'll derive much benefit from Linux, being the
modular monolithic kernel that it is.  The only thing building our own
kernel would really save us is boot time (since it won't be probing a
bunch of modules).  I don't think we want to mess with the kernel -
there is a good chance that putting in something foreign could cause
Redhats bootup scripts to have weird errors.  (can't say for sure, I
haven't installed a custom kernel on Redhat since 4.x...)  Still, if we
are aiming for stability over a couple percent speed improvement, I
think that using the tested Redhat kernel would definitely be the way to
go.

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


subject:
[Wikitech-l] [Sales@Computers4SURE.com: We've received your order
 #C030450733, Jimmy. Thank you.]

body:
On Tue, 29 Apr 2003, Jimmy Wales wrote:
> Motherboard for the server we're going to use as Wikipedia front end.

Sa-weeeet!

Thanks, Jim! Wikipedia will live to serve another day. :)

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] Ad-hoc changes

body:
On Tue, 29 Apr 2003, Lee Daniel Crocker wrote:
> Brion, now's the time to clean up all the ad-hoc changes made to
> the live wikis and get them into LocalSettings. I'm sure there are
> a few I've missed--can you get me a complete list?

Off the top of my head:

Code changes:
* Disabling updating and display of page counters on the English wiki

Style sheet changes:
* Paragraph indentation tweak for Chinese
* Background color for meta

And there are a number of things for Wiktionary that need to get into the
code; mainly a zillion instances of "Wikipedia" changed to "Wiktionary",
that still needs to get generalized. It's also behind on database schema
updates, I suspect.

-- brion vibber (brion @ pobox.com)


body:
On Tue, Apr 29, 2003 at 11:34:59AM -0700, Jimmy Wales wrote:
> Nick Reinking wrote:
> > I have a much better idea to handle these files, I'll detail it shortly.
> > But with HTTP/1.1, moving those files will pages load _slower_ for most
> > people (since they have to do an additional DNS lookup and connect()),
> > rather than piggybacking off an already existing HTTP/1.1 connection.
>
> Under normal circumstances, I would agree.  But the machine was just
> about *this far* from falling over completely, and pages were not
> really getting served *at all*.  At least the site works for now. :-)
>
> But I'd love to hear your suggestions, too!
>
> Anyhow, since a new server is going to be installed this weekend,
> hopefully hardware will save the day... for now.  :-)

Actually, I wanted to use the mod_mmap_static module that comes with
Apache.  This way, we can avoid stat()s or read()s to the hard disk for
these files that don't change.

Unfortunately, the Apache we have in /usr/local/apache doesn't have it
built in, nor does it have mod_so.  Does anybody know why this is?

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


subject: [Wikitech-l] kernel on wiki server

body:
On Tue, Apr 29, 2003 at 11:50:18AM -0700, Jimmy Wales wrote:
> No offense, Nick, but we've been doing this for a long time, with some
> very high traffic websites, and Jason knows what he's doing.

To be fair, so have I.  :)  But, if you think the extra risk and work is
justified, go right ahead.

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


body:
> (Nick Reinking <nick at twoevils.org>):
>
> Unfortunately, the Apache we have in /usr/local/apache doesn't have it
> built in, nor does it have mod_so.  Does anybody know why this is?

Because I compiled it, and didn't include anyting I wasn't going to
use. I'll be happy to recompile it to include whatever you think would
be useful.  Tell me what modules you need, I'll compile and test your
suggested setup on Piclab, then move it over to the wiki server.

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


subject: [Wikitech-l] kernel on wiki server

body:
> (Nick Reinking <nick at twoevils.org>):
> On Tue, Apr 29, 2003 at 11:50:18AM -0700, Jimmy Wales wrote:
> > No offense, Nick, but we've been doing this for a long time, with some
> > very high traffic websites, and Jason knows what he's doing.
>
> To be fair, so have I.  :)  But, if you think the extra risk and work is
> justified, go right ahead.

I tend to tread the middle ground myself--I have a kernel built from
RedHat source RPMs using a RedHat-supplied alternate config, with one
or two minor tweaks.

The big win for me at home was compiling for the Athlon.

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


body:
On Tue, Apr 29, 2003 at 02:19:25PM -0500, Lee Daniel Crocker wrote:
> > (Nick Reinking <nick at twoevils.org>):
> >
> > Unfortunately, the Apache we have in /usr/local/apache doesn't have it
> > built in, nor does it have mod_so.  Does anybody know why this is?
>
> Because I compiled it, and didn't include anyting I wasn't going to
> use. I'll be happy to recompile it to include whatever you think would
> be useful.  Tell me what modules you need, I'll compile and test your
> suggested setup on Piclab, then move it over to the wiki server.

Everything it has now plus mod_mime_static would be great.  :)

Right now it has:
  http_core.c
  mod_env.c
  mod_log_config.c
  mod_mime.c
  mod_negotiation.c
  mod_status.c
  mod_include.c
  mod_autoindex.c
  mod_dir.c
  mod_cgi.c
  mod_asis.c
  mod_imap.c
  mod_actions.c
  mod_userdir.c
  mod_alias.c
  mod_rewrite.c
  mod_access.c
  mod_auth.c
  mod_setenvif.c
  mod_php4.c

Thanks!

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


subject: [Wikitech-l] kernel on wiki server

body:
On Tue, Apr 29, 2003 at 02:21:55PM -0500, Lee Daniel Crocker wrote:
> > (Nick Reinking <nick at twoevils.org>):
> > On Tue, Apr 29, 2003 at 11:50:18AM -0700, Jimmy Wales wrote:
> > > No offense, Nick, but we've been doing this for a long time, with some
> > > very high traffic websites, and Jason knows what he's doing.
> >
> > To be fair, so have I.  :)  But, if you think the extra risk and work is
> > justified, go right ahead.
>
> I tend to tread the middle ground myself--I have a kernel built from
> RedHat source RPMs using a RedHat-supplied alternate config, with one
> or two minor tweaks.
>
> The big win for me at home was compiling for the Athlon.

Yeah, I believe the rpm I quoted earlier was compiled for the Athlon,
something I doubt about the current kernel (which was compiled for
i686).

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


from_name: Mirko Thiessen

subject: [WikiEN-l] Re: More about Zog

body:
Danny wrote:
>
> For this reason, I will not enter into dialogue with him. I will remove
> his changes as I see them, and hope that he will be banned. I hope people
> support me in this. If you have any doubts, just look at the history of
> his user contributions. This is certainly not the kind of person we want
> in Wikipedia.
>

Well stated, Danny. I agree with any single sentence. If anyone thinks,
that we can convince a racist like Zog of writing NPOV, he/she is in error.
I would feel *very* uncomfortable to know such a person as a tolerated
Wikipedia contributor. I hope, that we can get rid of him and never hear
about him again.

Mirko (Cordyph)

--
Mirko Thiessen
http://www.mirko-thiessen.de


subject: [Wikitech-l] server names

body:
> (Jimmy Wales <jwales at bomis.com>):
>
> You're right, and I retract my suggestion completely if someone comes
> up with something more fun and standards-compliant!  :-)
> Famous encyclopedists?
>
> Pliny, Vincent of Beavais, Denis Diderot, Jimbo? (ha ha)

...don't forget Mortimer Adler. Besides, having a server named
mortimer is about as nerdy as it's possible to get.

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


body:
I am confident that we can find a place for the mailing lists on one
of our current servers...

A tower style case wouldn't do well in our cage, since we don't have
much room left.  I haven't spoken to Jimbo yet, but I think we
probably have an idea in mind that will work out okay.

We do appreciate the offer, though.

Jason

Daniel Mayer wrote:

> Jason wrote:
> >I'll be able to make the trip to San Diego on Friday.
>
> >The server should be ready to use (ssh-able for all
> >people with accounts on the current server) on
> Saturday.
>
> It might be more trouble than it is worth, but do you
> want me to send you the AMD 500 Linux box I have it
> serve as the new mail server? If you can use it I'm
> more than happy to send it but if another solution is
> easier/makes more sense (like having Lee's ISP host
> WikiTech-L) then I'm cool with that too.
>
> I don't want to force hardware on anybody - all I want
> is for the developers to be able to talk to each other
> via a mailing list when the Wikipedia server is down.
>
> --Daniel Mayer (aka mav)
>
> __________________________________
> Do you Yahoo!?
> The New Yahoo! Search - Faster. Easier. Bingo.
> http://search.yahoo.com
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikitech-l

--
"Jason C. Richey" <jasonr at bomis.com>


from_name: Christopher Mahan

subject: [Wikipedia-l] [Wikitech-l] server names

body:
What about naming the servers after famous vandals? Like Lir, and
SusanMason, so on and so forth?

/me holds head in hands and sobs uncontrollably.

--- The Cunctator <cunctator at kband.com> wrote:
> There's a second server joining the first, and Jason would like a
> naming
> scheme for the servers. This is less a technical than an asthetic
> issue,
> so I'm bringing it to the attention of the general list. I like the
> idea
> of naming the servers after famous encyclopedists.
>
> -----Forwarded Message-----
>
> From: Lee Daniel Crocker <lee at piclab.com>
> > (Jimmy Wales <jwales at bomis.com>):
> >
> > You're right, and I retract my suggestion completely if someone
> comes
> > up with something more fun and standards-compliant!  :-)
> > Famous encyclopedists?
> >
> > Pliny, Vincent of Beavais, Denis Diderot, Jimbo? (ha ha)
>
> ...don't forget Mortimer Adler. Besides, having a server named
> mortimer is about as nerdy as it's possible to get.
>
>
>
>
> _______________________________________________
> Wikipedia-l mailing list
> Wikipedia-l at wikipedia.org
> http://www.wikipedia.org/mailman/listinfo/wikipedia-l

=====
Christopher Mahan
chris_mahan at yahoo.com
818.943.1850 cell
http://www.christophermahan.com/

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com


from_name: John R. Owens

subject: [Wikitech-l] kernel on wiki server

body:
On Tue, 29 Apr 2003, Nick Reinking wrote:

> Date: Tue, 29 Apr 2003 14:37:41 -0500
> From: Nick Reinking <nick at twoevils.org>
> Subject: Re: [Wikitech-l] kernel on wiki server
>
> On Tue, Apr 29, 2003 at 02:21:55PM -0500, Lee Daniel Crocker wrote:
> > > (Nick Reinking <nick at twoevils.org>):
> > > On Tue, Apr 29, 2003 at 11:50:18AM -0700, Jimmy Wales wrote:
> > > > No offense, Nick, but we've been doing this for a long time, with some
> > > > very high traffic websites, and Jason knows what he's doing.
> > >
> > > To be fair, so have I.  :)  But, if you think the extra risk and work is
> > > justified, go right ahead.
> >
> > I tend to tread the middle ground myself--I have a kernel built from
> > RedHat source RPMs using a RedHat-supplied alternate config, with one
> > or two minor tweaks.
> >
> > The big win for me at home was compiling for the Athlon.
>
> Yeah, I believe the rpm I quoted earlier was compiled for the Athlon,
> something I doubt about the current kernel (which was compiled for
> i686).

If you're talking specifically for RH 7.2 or 7.3, that's what
kernel-2.4.18-27.7.x.athlon.rpm is for. If it's 8.0, you've got your
kernel-2.4.18-27.8.0.athlon.rpm. But I've always used a kernel customized
and tweaked, ever since I was compiling with Cyrix 686 optimization,
possibly even earlier. I do it with Athlon now myself; although my system
is mostly RH 8.0, I use the raw kernel.org kernel, configured to have just
the goodies I need/want, and do it well. I very much recommend a
customized kernel, even given the availability of Athlon-optimized RPMs.

--
John R. Owens http://www.ghiapet.homeip.net/
The Babylon Project was our last, best hope for peace.  It failed.  But
in the year of the Shadow War, it became something greater: our last,
best hope for victory.  The year is 2260.  The place--Babylon 5.
				--Commander Susan Ivanova


subject: [Wikitech-l] Ad-hoc changes

body:
> (Brion Vibber <vibber at aludra.usc.edu>):
> [Configuration info]

Here's a thought: when we get the new server up, let's install the
English wiki cleanly from the distribution, make sure it's all happy,
then do the switch, _leaving the foreign wikis on the old server_ for
a while, then do the same thing for each of the foreign ones in turn,
updating them to the newest software, and making everything in sync,
before we shut down apache on the old server.

Any problems with that?

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


subject: [Wikitech-l] Possible performance issue?

body:
I notice that /usr (/dev/sda2) is at 96%.  ext2 has some pretty bad
problems with fragmentation once it gets above a certain percentage.
This can cause some pretty bad performance problems.  Once it has
fragmented, it is difficult to get it back to a contiguous state.

There are defrag programs, but they are fairly scary.  The only other
way to get it back to normal is to back everything up, mkfs, and restore
it.

Perhaps somebody can remove a bunch of the packages that are installed
that we don't use?

--
Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN


body:
> (Nick Reinking <nick at twoevils.org>):
> I notice that /usr (/dev/sda2) is at 96%...
> Perhaps somebody can remove a bunch of the packages that
> are installed that we don't use?

The server is very clean in terms of software. The big culprits
for disk usage are MySQL's Innodb transaction data (currently a
single 10Gb file!), and logfiles from MySQL and Apache.

I don't know enough about MySQL to know how to limit that data
and what the impact might be. I do think we could lighten the
load on Apache log files considerably now, to save both disk
space and gain some performance. For instance, we logged user
agents and referrers to get some stats, but I don't think we
really need that anymore.

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


from_name: Mark Christensen

subject: [Wikitech-l] Ad-hoc changes

body:
The only issue I see would be if database threads crowd out apache on
the old server, and all the foreign wiki's go down, while the English
wiki stays up.  This could create new tensions with the foreign wiki's.

-----Original Message-----
From: Lee Daniel Crocker [mailto:lee at piclab.com]
Sent: Tuesday, April 29, 2003 5:13 PM
To: wikitech-l at wikipedia.org
Subject: Re: [Wikitech-l] Ad-hoc changes

> (Brion Vibber <vibber at aludra.usc.edu>):
> [Configuration info]

Here's a thought: when we get the new server up, let's install the
English wiki cleanly from the distribution, make sure it's all happy,
then do the switch, _leaving the foreign wikis on the old server_ for a
while, then do the same thing for each of the foreign ones in turn,
updating them to the newest software, and making everything in sync,
before we shut down apache on the old server.

Any problems with that?

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/> "All
inventions or works of authorship original to me, herein and past, are
placed irrevocably in the public domain, and may be used or modified for
any purpose, without permission, attribution, or notification."--LDC
_______________________________________________
Wikitech-l mailing list
Wikitech-l at wikipedia.org
http://www.wikipedia.org/mailman/listinfo/wikitech-l


from_name: The Cunctator

subject: [Wikipedia-l] [Wikitech-l] server names

body:
There's a second server joining the first, and Jason would like a naming
scheme for the servers. This is less a technical than an asthetic issue,
so I'm bringing it to the attention of the general list. I like the idea
of naming the servers after famous encyclopedists.

-----Forwarded Message-----

From: Lee Daniel Crocker <lee at piclab.com>
> (Jimmy Wales <jwales at bomis.com>):
>
> You're right, and I retract my suggestion completely if someone comes
> up with something more fun and standards-compliant!  :-)
> Famous encyclopedists?
>
> Pliny, Vincent of Beavais, Denis Diderot, Jimbo? (ha ha)

...don't forget Mortimer Adler. Besides, having a server named
mortimer is about as nerdy as it's possible to get.


body:
> (Jason Richey <jasonr at bomis.com>):
>
> I am confident that we can find a place for the mailing lists on one
> of our current servers...
>
> A tower style case wouldn't do well in our cage, since we don't have
> much room left.  I haven't spoken to Jimbo yet, but I think we
> probably have an idea in mind that will work out okay.
>
> We do appreciate the offer, though.

I'd just like to make it clear that Mav's offer of my server is
not unilateral on his part--my rackspace is yours, too, such as it
is.  But if you're well-supplied, that's fine. I don't think the risk
of all your servers being down at once is significant. When San Diego
falls into the ocean after the big one, we'll have bigger problems
than Wikipedia. :-)

--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC


body:
On Tue, 29 Apr 2003, Lee Daniel Crocker wrote:
> > (Nick Reinking <nick at twoevils.org>):
> > I notice that /usr (/dev/sda2) is at 96%...
> > Perhaps somebody can remove a bunch of the packages that
> > are installed that we don't use?
>
> The server is very clean in terms of software. The big culprits
> for disk usage are MySQL's Innodb transaction data (currently a
> single 10Gb file!), and logfiles from MySQL and Apache.

Innodb keeps most of its goodies in that one big file, which can expand
but cannot contract. Certain operations (like altering the table
structure) involve making a complete duplicate of the database, altering
it, then replacing the old one; so it's taking up nearly twice the space
it actually _needs_ on a regular basis. On the plus side, it gives us room
to grow. :)

There's also the www-bin.### files, which are the binary log. These track
changes made to the database, and are rotated at 1 gigabyte or when the
server is restarted. These are mainly useful for database replication,
which we don't do _yet_ but will do in the future. For now, I just
periodically delete the old ones. It can be disabled somehow, but we'll
likely want them in the future so I've not bothered.

Now, here's the space used by the actual wiki files under
/usr/local/apache:

2017344 htdocs
387056  logs
302384  htdocs-fr
183700  htdocs-sv
172480  htdocs-de
133512  htdocs-meta
129772  htdocs-eo
103748  htdocs-pl
98588   htdocs-es
97832   htdocs-ja
88128   htdocs-nl
71080   htdocs-da
31276   htdocs-zh
30896   htdocs-test
17036   htdocs-wiktionary
10868   htdocs-ko
7788    htdocs-ru
6784    htdocs-cs
4056    htdocs-bs
3256    htdocs-ms
3020    htdocs-el
2960    htdocs-tr
2788    htdocs-sh
2788    htdocs-ml
2772    htdocs-sr
2772    htdocs-hr
2740    htdocs-sep11

These include the php files, uploaded images, backup tarballs, webalizer
stuff, and TeX-generated images. I've deleted saved log files from prior
to one week ago (and those that are retained are gzipped).

Further breakdown on the English wiki:
1306996 tarballs
440728  upload
120176  stats
92972   tmp
28472   math
16796   images
4444    w
... some other small smidgens of files...

> I do think we could lighten the load on Apache log files considerably
> now, to save both disk space and gain some performance. For instance,
> we logged user agents and referrers to get some stats, but I don't
> think we really need that anymore.

Oh, I think it's quite useful to get that information, otherwise I
wouldn't know about *($%@^&*$%@# Grub.

Anyway, I cleaned out a few things and moved some of the older tarballs
over to the archives in the home partition, and we're down to 85% usage on
/usr.

-- brion vibber (brion @ pobox.com)