from_name: Neil Harris

subject: [Wikipedia-l] A data point -- page parsing on beta.wikipedia.com

body:
I've been running two programmes recently, the stressbot and a new
program, 'postit' that creates random articles

Each article contains a block of random alphabet soup, just to stress
the system.
The random characters are chosen from a range of characters, with a
weighting on special chars such as '&', '[', '], '<', '>' etc. to stress
the parser.

I've just tweaked the program to use characters in the range 32-254,
rather than the previous ASCII-only random characters.
I've noticed that this seems to speed article posting up a fair bit. The
only reason I can think of is that the parser is taking significant CPU
time for these pages, and the non-ASCII chars are causing some regexps
to terminate early, saving time.

I'll do a couple more tests to try to confirm this.

Neil


list: wikitech-l

source: pipermail-mbox

from_name: lcrocker at nupedia.com

subject: [Wikitech-l] Re: testing new server

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020711/d2db9276/attachment.ksh>


list: wikipedia-l

subject: [Wikipedia-l] "Email this user" function

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020711/a309f886/attachment.diff


subject: [Wikipedia-l] Re: [Intlwiki-l] uber lingua wiki

body:
On Wed, Jul 10, 2002 at 09:06:47PM -0800, Brion VIBBER wrote:
>
> The current software is intended to be able to be run from the same
> source files, with the exception of a unique wikiLocalSettings.php for
> each wiki, which specifies among other things which localization file to
> use. (Note however that it expects everything to be in one directory, so
> symbolic links for everything but the settings file pointing to a master
> directory is the simplest way to achieve this.)

On a more practical note. I assume that future changes will often involve
adding new files, so Jimbo would have to add new links for every separate
wikipedia every time. I would propose to divide the files in a 'local' and
'global' directory such that every wikipedia only needs a single link to the
common scripts that doesn't change.

> [...] Personally I prefer a separate database table; people should be able
> to edit the link list from their own language environment without delving
> into the English wiki. In either case, though, the programs will all need
> access to the same shared link database, so we would need to keep things
> on the same server.

I fully agree with all of the above. Another reason for keeping everything on
the same DBMS server is that this is the only way we can build a page for
checking if an uploaded file is used anywhere.

-- Jan Hidders


from_name: wojtek pobratyn

body:
----- Original Message -----
From: <lcrocker at nupedia.com>
To: <wikipedia-l at nupedia.com>
Sent: Thursday, July 11, 2002 11:11 AM
Subject: [Wikipedia-l] "Email this user" function

| One of the post-live features I'd like to implement is a link on User
| pages for "email this user", which will produce a form that can be
| used to send email to the user without actually revealing the email
| address, for those times when you want to talk to someone directly who
| may not be active on the list or follow the talk pages.  This will
| also help reduce the clutter on talk pages.

I think it's a great idea. I could take my address of my user page
and stop it from being harvested my malicious bots.

| I'd like to get a sense from the community what privacy issues or
| other issues there are in implementing such a feature.  We may want to
| have a user setting for "don't allow email" or something, though I
| personally would rather not have that feature, and I definitely want
| it turned off for sysops (a working, responsive email address is the
| single most important qualification for a sysop as far as I'm
| concerned, although I suppose active participation on the mailing list
| might be an acceptable alternative).

Since the code does not verify email addresses on sign-up anyway
I don't think this is such a big issue. The traffic generated would be
minimal anyway and should anyone feel annoyed they could just change
their address in their user prefs to eg. foo at whatever.ac

| I'd also like to brainstorm a bit on how to raise the visibility of
| the mailing list.

Well we should somehow bring [[Wikipedia_Announcements]] back
to life. The last entry is from May 17!!

regards,
[[user:WojPob]]


subject: [Wikitech-l] Stress testing of beta.wikipedia.com

body:
Here is another data point regarding the performance of beta.wikipedia.com

I'm now running
7 stressbots (page readers)
2 postits (page writers)
concurrently accessing beta.wikipedia.com via a 512K DSL connection.

I'm getting the following statistics:
average page read time 2.9 seconds
average page write time 4.9 seconds

corresponding to an
average page read rate of 7/2.9 = 2.4 pages/sec
average page write rate of 2 / 4.9 = 0.41 pages/sec

making a total sustained transaction rate of around 2.8 hits/sec, or
around 240,000 hits/day or over 7 million hits a month.

However, my inbound traffic is around 61 kbytes/sec doing this, so my
512k DSL link is currently the bottleneck, not the server.

Dropping the concurrency to
3 stressbots
1 postit

gives:
average page read time 1.9  seconds
average page write time 3.1  seconds

corresponding to an
average page read rate of 3/1.9 = 1.57 pages/sec
average page write rate of 1 / 3.1 = 0.32 pages/sec

total transaction rate: 1.9 hits/sec

for an inbound traffic rate of about 26 kbytes/s, where my DSL link is
no longer the bottleneck, but the system is under less load.

To really stress test the server, we will need several clients to run at
once on several different links. I'm going to stop the test now.

It would be useful for testing if we could have a page that gave current
Linux operating system stats, perhaps in a sysop-only page?

Neil


body:
I've noticed that beta.wikipedia.com has not got a robots.txt file.
Yes, I know that most recent robots will read the metadata in the page,
but I'm willing to bet that some of the dimmer or older ones don't.

Should we have one?

Also, there's no favicon.ico. I enclose a file Walone2.ico which should
work if renamed to favicon.ico and placed at
http://beta.wikipedia.com/favicon.ico

Neil

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Walone2.ico
Type: image/x-icon
Size: 318 bytes
Desc: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020711/ac412fd2/attachment.ico>


from_name: Tim Chambers

body:
I'm fully in favor of it. I don't care about keeping my e-mail a
secret, though. I still trace my spam like I describe at [1], and have
never noticed e-mail getting sent to timc+wikipedia at divide.net. If
they try to remove the stuff after the plus sign, it goes in the bit
bucket, though, so I wouldn't know about that.

[1] http://makeashorterlink.com/?O3F532E31

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com


from_name: Gareth Owen

in_reply_to: Stephen Gilbert's message of "Thu, 11 Jul 2002 05:23:10 -0700 (PDT)"

body:
Stephen Gilbert <canuck_in_korea2002 at yahoo.com> writes:

> It depends on what program you're using to write your articles. MS Word
> inserts quite a few non-standard characters; I don't recommend it. Try using
> a pure text editor instead of a word processor.

Most (all?) of these can be eliminated in Word by turning off SmartQuotes(tm)

Its under Format -> Autoformat... -> Options... -> Replace
--
Gareth Owen
"Wikipedia does rock.  By the count on the "brilliant prose" page, there
 are 14 not-bad articles so far" -- Larry Sanger (12 Jan 2001)


subject: [Wikipedia-l] what's being done about performance?

body:
Maybe I just haven't noticed the appropriate messages sent to this
list, but it seems to me that progress reports have been spotty. Is
there discussion going on somewhere else?

I'm very bummed. I try to pepper my postings at K5 with links to the
'pedia, but today I couldn't even get to the main page. I had to link
to archives.org instead [1]. I would have preferred to put that
high-quality off-site link [2] in the 'pedia Titanic article and
linked to it instead of [2]. :-( (Breaking News: While composing this,
I was finally able to get to wikipedia.com and make the edit.)

I'm deeply saddened that Wikipedia is a victim of its own success, but
I also want to praise Jimbo, Magnus, Lee, and all the others who've
supported the project technically. I'd offer to help right now, but I
have other commitments for the next couple of weeks... I'd just like a
way to stay abreast of developments.

<>< Tim

[1] see the Titanic link at
http://www.kuro5hin.org/comments/2002/7/9/04416/40645/222#222
[2]
http://www.archives.gov/exhibit_hall/american_originals/titanic.html

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com


from_name: Karen AKA Kajikit

subject: [Wikipedia-l] Yay it worked!

body:
They say a picture is worth a thousand words... I didn't have a lot to
add to the text of the brief banksia article, but I hope my photographs
speak for themselves :)

http://www.wikipedia.com/wiki/Banksia

Now to see what else I've got in my digital file cabinet :)

--

Karen AKA Kajikit

You can take the dragon out of Alfandra, but you can never take Alfandra
out of the dragon (or the Kitty)...

Come and visit my part of the web:
Kajikit's Corner: http://Kajikit.netfirms.com/
Aussie Support Mailing List: http://groups.yahoo.com/group/AussieSupport
Allergyfree Eating Recipe Swap:
http://groups.yahoo.com/group/Allergyfree_Eating

Love and huggles to all!


body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020711/96739b21/attachment.diff


subject: [Wikitech-l] Estimating parser time

body:
I'm testing the postit script with and without non-ASCII chars, to see
if the apparent speed difference is real.

I'm not running any other load generators. Each test is an average over
roughly 50 pages. I'll alternate between the with and without states.

With: 2.39 secs average
Without: 2.91 secs average
With: 2.84 secs average
Without: 2.98 secs average
With: 2.64
Without: 3.18

Well, there does seem to be a difference, but it still might be noise.

All with = 7.87 / 3
All without = 9.07 / 3
Difference = 1.2 / 3 = 0.4 seconds

I then did two long runs:

With: 2.74 (averaged over 151 pages)
Without: 3.21 (averaged over 153 pages)

Difference: 0.47 seconds

Another two long runs:
With: 2.78 (averaged over 151 pages)
Without: 3.21 (averaged over 151 pages)

It looks like the difference is probably real, I guess due to the
greater likelihood of things that will make the regexp engine have to
lookahead and backtrack to check if things are links.

This is _not_ a big performance issue at the moment, as the Wikipedia
load is read-mostly.  However, it's worth considering as a place to look
at in the future.

By then, we'll probably be running a three-tier architecture, with the
DB running on a separate machine from the PHP scripts, and so even then
this may be a low-priority issue.

Neil

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Walone2.ico
Type: image/x-icon
Size: 318 bytes
Desc: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020711/7be7104c/attachment.ico>


from_name: Daniel Mayer

subject: [Wikipedia-l] RE: "Email this user" function and @wikipedia.com email accounts

body:
LDC wrote:
>The questions, then, are (1) when a new user signs
up, giving an e-
>mail address, should we further require that he check
a box
>saying "allow other wikipedians to mail me" or should
we let him
>check a box saying "don't allow other wikipedians to
mail me".  The e-
>mail address is never published in any case, and the
only mails
>received are one-to-ones, not lists, but I suppose
opt-in is still
>the more responsible thing to do.  (2) Perhaps the
double opt-in
>confirmation message could be sent whenever this
option is selected.
> (3) The mail messages themselves might have a
section something
>like "This mail was sent from Wikipedia's e-mail
function.  If you
>don't want to receive mails like this, go to..."

If we do decide to have this function, I think that in
order for it to be at all useful the user account
set-up should allow the user to opt-out if they click
the box next to "don't allow other wikipedians to mail
me". It has been my experience that most people simply
accept the defaults on webforms and if the default is
set /not/ to enable this function, then there won?t be
enough users with the function enabled to make it
commonplace (if the great majority of users don?t have
a link to ?Email this user? then other users visiting
user pages won?t look to the link).

I also agree with your previous statement about
requiring sysops to have this function enabled
(anybody taking on sysop status needs to be
contactable to the community and other sysops). I?m
not sure if it would be worth the effect to hard-code
this requirement into the software ? visiting sysop
pages will reveal who is and who is not adhering to
policy (if this does become one).

It would also be nice if sysops could be able to email
all other sysops when needed (or even to each and
every user signed up if absolutely needed ? like if
somebody hacked the server and stole user passwords).
For example, I had to manually post a message to each
and every sysop a while back warning them not to use
the administrative move feature because of a serious
bug that truncated page histories. It would have been
nice to draft a single email for that.

BTW, would it be possible for users to eventually have
@wikipedia.com email addresses (on a pay basis open
only to users with user accounts ? but that
requirement isn?t essential to me)? I would happily
give Bomis or a ?Wikipedia Foundation? 30 bucks a year
for a maveric149 at wikipedia.com
(maveric149 at wikipedia.org would be even better) than
have to pay the same amount for maveric149 at yahoo.com.

--maveric149

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com


body:
>> I'd also like to brainstorm a bit on how to raise
>> the visibility of
>> the mailing list.

>Hmmm. How about announcing its existence on the front
>page? Or is that too easy a suggestion? :)
>
>--Stephen Gilbert

I would suggest putting such an announcement on
RecentChanges instead --- 99% of the people accessing
the main page couldn't care less about the list ? that
is, except for spammers.

--maveric149

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com


from_name: Tomasz Wegrzanowski

subject: [Wikitech-l] question about performance

body:
Wikipedia is often extremely slow.
What's the bottleneck ?
* network i/o ?
* database performance ?
* wikipedia script performance ?
* something else ?


subject: [Wikitech-l] question about performance

body:
Tomasz Wegrzanowski wrote:

>Wikipedia is often extremely slow.
>What's the bottleneck ?
>* network i/o ?
>* database performance ?
>* wikipedia script performance ?
>* something else ?
>
>
>
No-one really knows. Another possibility would be
* dodgy hardware
Perhaps a disk, or a network switch, is intermittently hanging.

One interesting observation is that the even when the English-language
Wikipedia is jammed up,
the international ones, which I believe run on the same server, are
often working OK. This suggests a software, not a hardware, problem.

The new software is currently under test, and it's being filled up with
test articles, and exercised using test scripts.

Network I/O is not a big bottleneck: the new server has been clocked
doing 2.8 hits/sec sustained, and that was limited by the 512k bandwidth
of the ADSL link to the testing machine. Further tests are being carried
out, and I hope to have some results soon.

Assuming Bomis has 10 Mbit/s bandwidth available, such a link could
support 20 x 2.8 = 56 hits/sec, or 4.8 million hits/day assuming evenly
distributed load.

Database performance is likely to be contstrained by two things:
* locking
* disk I/O

Locking is a problem because it serializes accesses, reducing
opportunities for parallel processing, and creating bottlenecks on the
locked resources.
Locking can be made better by:
* locking for as short a time as possible
* locking with the finest grain possible
* using a database which supports concurrent transactions with reduced
locking

Disk I/O can be made faster by
* using disks which spin fast (rotational latency is reduced), and
* putting them in a big RAID with lots of spindles and a high-speed
attachment
* using an operating system which multi-threads I/O properly

Wikipedia script performance is unlikely to be the bottleneck. We now
have the opportunity to
load the test system heavily and measure CPU load, to be able to
estimate this factor accurately.

Something else could be:
* Memory hogging

This is a little-known nasty factor in server programming. Here, the
problem is worker threads being tied up by slow or malfunctioning
clients, such as those on modems, or with high packet loss, or both.
Say a worker thread consumes W Mbytes of store, and an access transfers
50k bytes (400 kbits) of data.
Then a really slow link at say 20 kbps will take 20 seconds to download
this page. In doing so, it locks W Mbytes in store for that entire time.
If we have X megabytes of store, and slow clients are the dominating
factor, then we can only accomodate X/W concurrent workers, serving
(1/20)*X/W pages per second.

For X = 256, W=2, that's 6.4 hits per second. Therefore, a server needs
to have lots of RAM to prevent slow clients from blocking it. Hmm...
increasing the OS socket buffer size to > 50k might be a win here.
Fortunately, the new server has lots of RAM.

* Swapping
Once you are doing VM swapping on a webserver or database, performance
plummets. Memory leaks somewhere could be bloating processes, causing
the server to swap.

* Congestion collapse
Whatever the mechanism for going slow, once a system is overloaded it
can enter a state known as 'congestion collapse', where it's slow
because it's overloaded, and overloaded because it's slow. This is made
worse if users keep on retrying their requests. A system may take some
time to recover from congestion collapse, and after recovery it will be
back to normal as if the collapse had never happened. This resembles the
current state of affairs.

* Cracking
Someone may have cracked the server, and be using it for malicious
purposes. Yow! I've just checked, and wikipedia.com is still running
Apache/1.3.23 (Unix) PHP/4.0.6 mod_fastcgi/2.2.12, which may well be
vulnerable to the chunked-transfer-encoding bug unless it's been patched.

Beta.wikipedia.com is running Apache/1.3.26 (Unix) PHP/4.2.1, so it's
not vulnerable to this bug.

* Denial-of-service attacks
SYN floods, that sort of thing.

Well, that's a list of the first few things off the top of my head.

Neil


subject: [Wikitech-l] Security alert - old version of Apache runnung on main server

body:
I've just checked, and wikipedia.com is still running
Apache/1.3.23 (Unix) PHP/4.0.6 mod_fastcgi/2.2.12, which may well be
vulnerable to the chunked-transfer-encoding bug unless it's been patched.

Beta.wikipedia.com is running Apache/1.3.26 (Unix) PHP/4.2.1, so it's
not vulnerable to this bug.


from_name: Sven Burkhart

subject: [Wikitech-l] question about performance

body:
Hi all,

after watching various wikipedia mailing lists silently for a while, it
might be time for my first two cents.

Neil Harris wrote:
> One interesting observation is that the even when the English-language
Wikipedia is jammed up,
> the international ones, which I believe run on the same server, are often
working OK

I have noticed that, too, but also the other way round, that e.g. the German
Wikipedia is jamming while the English version is still working. Since I
have often been looking at pages from both the German version (de), the
German test version (test-de) and the English version, I think I have seen
all possible combinations of 3 working or jamming wikis. So I don't think
the English version is something special regarding this problem. But usually
when one is on strike, the others are following soon within a few minutes.
Maybe they're all in the same labor union ... ;-)

Sven (Ben-Zin)

Btw ... has anyone noticed yet that the automatically added address
Wikitech-l at ross.bomis.com at the bottom doesn't seem to work? Mailserver
told me something about "User unknown". Maybe someone can update this
address.


subject: [Wikitech-l] question about performance

body:
Tomasz Wegrzanowski wrote:

>On Thu, Jul 11, 2002 at 05:21:40PM +0100, Neil Harris wrote:
>
[snip]

>>
>>Locking is a problem because it serializes accesses, reducing
>>opportunities for parallel processing, and creating bottlenecks on the
>>locked resources.
>>Locking can be made better by:
>>* locking for as short a time as possible
>>* locking with the finest grain possible
>>* using a database which supports concurrent transactions with reduced
>>locking
>>
>>
>
>What about switching to Postgres ?
>It is said to have better locking.
>
>
Indeed so. I was going to get to that later. ;-)

>
>
>>Disk I/O can be made faster by
>>* using disks which spin fast (rotational latency is reduced), and
>>* putting them in a big RAID with lots of spindles and a high-speed
>>attachment
>>* using an operating system which multi-threads I/O properly
>>
>>Wikipedia script performance is unlikely to be the bottleneck. We now
>>have the opportunity to
>>load the test system heavily and measure CPU load, to be able to
>>estimate this factor accurately.
>>
>>
>
>Even if is far from consuming 100% CPU, if it's slow, it occupies memory
>for longer time. Or it may simply be using too much memory per thread.
>
>
>
>>Something else could be:
>>* Memory hogging
>>
>>This is a little-known nasty factor in server programming. Here, the
>>problem is worker threads being tied up by slow or malfunctioning
>>clients, such as those on modems, or with high packet loss, or both.
>>Say a worker thread consumes W Mbytes of store, and an access transfers
>>50k bytes (400 kbits) of data.
>>Then a really slow link at say 20 kbps will take 20 seconds to download
>>this page. In doing so, it locks W Mbytes in store for that entire time.
>>If we have X megabytes of store, and slow clients are the dominating
>>factor, then we can only accomodate X/W concurrent workers, serving
>>(1/20)*X/W pages per second.
>>
>>For X = 256, W=2, that's 6.4 hits per second. Therefore, a server needs
>>to have lots of RAM to prevent slow clients from blocking it. Hmm...
>>increasing the OS socket buffer size to > 50k might be a win here.
>>Fortunately, the new server has lots of RAM.
>>
>>
>
>2 megabytes of non-shared memory per thread ?
>That would be enormous.
>
>What's the real value like ?
>
>Also if the thread is up it may be unnecessarily holding database
>connection. But it's not likely to be major problem.
>
>
Think in terms of a C stack, and a complete set of kernel data
structures, sockets etc.
OK, maybe more like 1 Mbyte+

Of course, a lightweight thread takes significantly less resources.

>
>
>>* Swapping
>>Once you are doing VM swapping on a webserver or database, performance
>>plummets. Memory leaks somewhere could be bloating processes, causing
>>the server to swap.
>>
>>
>
>Swaping isn't a problem, it's a symptom.
>
>Heavy apache or mysql bloat is unlikely and Wikipedia threads live
>too short to have chance of bloating too much.
>
>
True. But something weird's got to be happening. Maybe the different
national 'pedias are too big to all fit in the working set at once.

>
>


subject: [Wikitech-l] question about performance

body:
On Thu, Jul 11, 2002 at 05:21:40PM +0100, Neil Harris wrote:
> One interesting observation is that the even when the English-language
> Wikipedia is jammed up,
> the international ones, which I believe run on the same server, are
> often working OK. This suggests a software, not a hardware, problem.

I've seen both English-down-others-working, and all-down.

> Database performance is likely to be contstrained by two things:
> * locking
> * disk I/O
>
> Locking is a problem because it serializes accesses, reducing
> opportunities for parallel processing, and creating bottlenecks on the
> locked resources.
> Locking can be made better by:
> * locking for as short a time as possible
> * locking with the finest grain possible
> * using a database which supports concurrent transactions with reduced
> locking

What about switching to Postgres ?
It is said to have better locking.

> Disk I/O can be made faster by
> * using disks which spin fast (rotational latency is reduced), and
> * putting them in a big RAID with lots of spindles and a high-speed
> attachment
> * using an operating system which multi-threads I/O properly
>
> Wikipedia script performance is unlikely to be the bottleneck. We now
> have the opportunity to
> load the test system heavily and measure CPU load, to be able to
> estimate this factor accurately.

Even if is far from consuming 100% CPU, if it's slow, it occupies memory
for longer time. Or it may simply be using too much memory per thread.

> Something else could be:
> * Memory hogging
>
> This is a little-known nasty factor in server programming. Here, the
> problem is worker threads being tied up by slow or malfunctioning
> clients, such as those on modems, or with high packet loss, or both.
> Say a worker thread consumes W Mbytes of store, and an access transfers
> 50k bytes (400 kbits) of data.
> Then a really slow link at say 20 kbps will take 20 seconds to download
> this page. In doing so, it locks W Mbytes in store for that entire time.
> If we have X megabytes of store, and slow clients are the dominating
> factor, then we can only accomodate X/W concurrent workers, serving
> (1/20)*X/W pages per second.
>
> For X = 256, W=2, that's 6.4 hits per second. Therefore, a server needs
> to have lots of RAM to prevent slow clients from blocking it. Hmm...
> increasing the OS socket buffer size to > 50k might be a win here.
> Fortunately, the new server has lots of RAM.

2 megabytes of non-shared memory per thread ?
That would be enormous.

What's the real value like ?

Also if the thread is up it may be unnecessarily holding database
connection. But it's not likely to be major problem.

> * Swapping
> Once you are doing VM swapping on a webserver or database, performance
> plummets. Memory leaks somewhere could be bloating processes, causing
> the server to swap.

Swaping isn't a problem, it's a symptom.

Heavy apache or mysql bloat is unlikely and Wikipedia threads live
too short to have chance of bloating too much.


from_name: Magnus Manske

body:
Thanks for the praise - I guess... :)

To answer the question (the subject line), we'll switch to new hardware and
software soon, and I am pretty sure the performance problems will decrease
or go away completely.

Try it out at http://beta.wikipedia.com (both new hard- and software).

Magnus

> -----Original Message-----
> From: wikipedia-l-admin at nupedia.com
> [mailto:wikipedia-l-admin at nupedia.com]On Behalf Of Tim Chambers
> Sent: Thursday, July 11, 2002 10:28 PM
> To: wikipedia-l at nupedia.com
> Subject: [Wikipedia-l] what's being done about performance?
>
>
> Maybe I just haven't noticed the appropriate messages sent to this
> list, but it seems to me that progress reports have been spotty. Is
> there discussion going on somewhere else?
>
> I'm very bummed. I try to pepper my postings at K5 with links to the
> 'pedia, but today I couldn't even get to the main page. I had to link
> to archives.org instead [1]. I would have preferred to put that
> high-quality off-site link [2] in the 'pedia Titanic article and
> linked to it instead of [2]. :-( (Breaking News: While composing this,
> I was finally able to get to wikipedia.com and make the edit.)
>
> I'm deeply saddened that Wikipedia is a victim of its own success, but
> I also want to praise Jimbo, Magnus, Lee, and all the others who've
> supported the project technically. I'd offer to help right now, but I
> have other commitments for the next couple of weeks... I'd just like a
> way to stay abreast of developments.
>
> <>< Tim
>
> [1] see the Titanic link at
> http://www.kuro5hin.org/comments/2002/7/9/04416/40645/222#222
> [2]
> http://www.archives.gov/exhibit_hall/american_originals/titanic.html
>
>
> __________________________________________________
> Do You Yahoo!?
> Sign up for SBC Yahoo! Dial - First Month Free
> http://sbc.yahoo.com
> [Wikipedia-l]
> To manage your subscription to this list, please go here:
> http://www.nupedia.com/mailman/listinfo/wikipedia-l


subject: [Wikitech-l] Stress testing

body:
Axel Boldt wrote:

>Neil,
>
>maybe you could pepper your stress testing with some calls to special
>functions; I would imagine that searching and RecentChanges are the
>most important ones, but the more the better.
>
>Axel
>_______________________________________________
>Wikitech-l mailing list
>Wikitech-l at ross.bomis.com
>http://ross.bomis.com/mailman/listinfo/wikitech-l
>
>
>
>
>
Recent changes and all the special pages are already being loaded,
including the big queries: the spider also completely ignores any robots
metadata (but won't walk out of the site).

I'm not doing any searches, though, or hitting any other forms. I'm
probably not going to be able to do so for a couple of days -- I have
other commitments for the next day or two.

I'm concentrating at the moment on filling up the database with
interlinked dummy articles in a Zipf distribution, to make stress
testing more representative of the situation as if in a year or so's
time. Otherwise we risk saying that performance is fine with the current
database, and then everything falling down later when the system fails
to scale.

I'm only stress testing in bursts, as I think an hour's testing every so
often is fine as a representative probe.

I've also sent the scripts to Jimbo, and I think he's going to do
LAN-local stress testing with essentially unlimited bandwidth.

Can someone please let me have access to the logs?

Neil


from_name: Axel Boldt

subject: [Wikitech-l] Stress testing

body:
Neil,

maybe you could pepper your stress testing with some calls to special
functions; I would imagine that searching and RecentChanges are the
most important ones, but the more the better.

Axel


from_name: Lars Aronsson

subject: [Wikipedia-l] Re: City, state convention for Australian cities

body:
On Sat, 22 Jun 2002, Karen AKA Kajikit wrote:
> I would think that the best policy is consistency... no, Australian
> placenames do not repeat from state to state (much). Not to the extent
> that it would interfere with entries... but at the same time I think
> that if the policy for other countries is city,state then we should
> stick to that.

The policy to stick to is "make it easy to link to".  The only country
that I know uses a "city, state" notation on regular basis is the USA.
Phrases like "Chicago, Illinois" and "St Petersburg, Florida" are
familiar and can easily be used in other articles that need to link to
them, but "Sydney, Australia" and "Stockholm, Sweden" just sound silly
in any text.  It would be as strange as "floppy, computer device" or
"cheese, food item".  Writing [[Sydney, Australia|Sydney]] is not a good
solution.

--
  Lars Aronsson (lars at aronsson.se)
  tel +46-70-7891609
  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/


subject: [Wikitech-l] Stress testing: odd observation

body:
The alphabet-soup filling of the dummy articles has added lots of junk
to the search indices, which is good.

Try doing a search for the three-letter string 'vyf'. It brings up lots
of articles containing 'vyf' in various combinations of upper and
lower-case.

But only one of them has a keyword-in-context display, which seems strange.

Neil


from_name: Pierre Abbat

subject: [Wikipedia-l] ATTN: Caltrop: GDBT1920

body:
I asked two weeks ago what this picture is about and have gotten no answer.
What is it for? If I don't hear back soon, I shall delete it.

phma


from_name: Fred Bauder

body:
I agree, but still, with new software almost ready to go, serious digging
into what's wrong with the old softwear seems senseless. (Although one
might stumble on something).

Fred Bauder

At 03:14 PM 7/12/02 +0200, you wrote:
>On Thu, 11 Jul 2002, Tim Chambers wrote:
>> I'm deeply saddened that Wikipedia is a victim of its own success,
>
>No, it is victim to poor architecture.  It's a software problem in the
>Wikipedia PHP source code.
>
>The developers seem totally clueless about performance issues. There is a
>constant focus on new functions, and none on performance and response
>times. They cannot even say what the average response time has been in the
>last week or 24 hours.  (Hey, prove me wrong: Tell me the average.)
>
>
>--
>  Lars Aronsson (lars at aronsson.se)
>  tel +46-70-7891609
>  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/
>
>[Wikipedia-l]
>To manage your subscription to this list, please go here:
>http://www.nupedia.com/mailman/listinfo/wikipedia-l
>


body:
Lars Aronsson wrote:
>
> On Sat, 22 Jun 2002, Karen AKA Kajikit wrote:

> The policy to stick to is "make it easy to link to".  The only country
> that I know uses a "city, state" notation on regular basis is the USA.
> Phrases like "Chicago, Illinois" and "St Petersburg, Florida" are
> familiar and can easily be used in other articles that need to link to
> them, but "Sydney, Australia" and "Stockholm, Sweden" just sound silly
> in any text.  It would be as strange as "floppy, computer device" or
> "cheese, food item".  Writing [[Sydney, Australia|Sydney]] is not a good
> solution.

I don't think it sounds silly... sure within the context of a single
country it's wierd and wouldn't be used, but when you're talking
worldwide then it's quite normal to use the country name as well as the
city just as an identifier. You want to make it obvious to people who
are ignorant just where Sydney is, and if they don't know where
AUSTRALIA is then the situation is hopeless and you can just give up
:)Lars Aronsson wrote:

--

Karen AKA Kajikit

You can take the dragon out of Alfandra, but you can never take Alfandra
out of the dragon (or the Kitty)...

Come and visit my part of the web:
Kajikit's Corner: http://Kajikit.netfirms.com/
Aussie Support Mailing List: http://groups.yahoo.com/group/AussieSupport
Allergyfree Eating Recipe Swap:
http://groups.yahoo.com/group/Allergyfree_Eating

Love and huggles to all!


body:
On Thu, 11 Jul 2002, Tim Chambers wrote:
> I'm deeply saddened that Wikipedia is a victim of its own success,

No, it is victim to poor architecture.  It's a software problem in the
Wikipedia PHP source code.

The developers seem totally clueless about performance issues. There is a
constant focus on new functions, and none on performance and response
times. They cannot even say what the average response time has been in the
last week or 24 hours.  (Hey, prove me wrong: Tell me the average.)

--
  Lars Aronsson (lars at aronsson.se)
  tel +46-70-7891609
  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/


subject: [Wikipedia-l] Re: ATTN: Caltrop: GDBT1920

body:
On Friday 12 July 2002 21:54, Axel Boldt wrote:
> I don't think it is appropriate to delete a picture just because you
> don't know what it is for and the uploader didn't respond to your
> mailing list request. The upload area is used by all Wikipedias, and
> unless we are dealing with a clear case of warez or music swapping,
> the uploaders should get the benefit of the doubt.

It looks like Caltrop came back after an extended absence. Hopefully he'll
see my note.

phma


subject: [Wikitech-l] Performance

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020713/b591a9a2/attachment.ksh>


subject:
[Wikitech-l] Suggested mechanism for math formulas and other
 on-the-fly graphics

body:
On Thu, 20 Jun 2002, Axel Boldt wrote:
> I thought I'd bring up this idea again since it might be easy to
> implement with the new codebase.
>
> If you put text such as [$\int_{x=0}^\infty x^2 dx$] in Wiki, upon
> saving the article, TeX will be called and translate the formula into

Excellent proposal. Write the implementation. Then wrap it in code that
monitors how often it gets called and how long each call takes, and
allows the entire thing to be switched off if it would become a
performance bottleneck.

--
  Lars Aronsson (lars at aronsson.se)
  tel +46-70-7891609
  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/


body:
On Friday 12 July 2002 12:01 pm, : Karen AKA Kajikit  wrote:
> I don't think it sounds silly... sure within the context of a single
> country it's wierd and wouldn't be used, but when you're talking
> worldwide then it's quite normal to use the country name as well as the
> city just as an identifier. You want to make it obvious to people who
> are ignorant just where Sydney is, and if they don't know where
> AUSTRALIA is then the situation is hopeless and you can just give up

I agree with Karen here. Since [[Sydney]] redirects to [[Sydney, Australia]]
you still will be able to simply type [[Sydney]] and end up in the most
famous Sydney.

However, since [[Sydney]] redirects to the Australian city it is necessary to
have the following statement at the top of that [[Sydney, Australia]]:
"[[Sydney]] redirects here. There is also another article named [[Sydney,
Nova Scotia]]."

And if there were a long list of other, even less well know uses of the word
"Sydney", then they would be in [[Sydney (disambiguation)]] and linked to
after the above mention of the Nova Scotia city.

This is called a disambiguation block and this form of disambiguation is
useful in cases like this where a famous thing has the same name less famous
things.  Paris is an even more obvious example.

All this has already been worked out to a great deal of detail here on the
list and in about a dozen talk pages.

More documentation can be found at:
[[wikipedia talk:naming conventions (city names)]] and
[[wikipedia:naming conventions]]

BTW, it looks like Canada has systemic internal naming conflict issues with
the [[City, Nation]] format and I am considering whether it would be a good
idea to advocate moving Canada to the [[City, Province]] name format (similar
to the USA) because of this (most of the Canadian cities are already in this
format anyway). For example, there appears to be several significantly
important places named Richmond in Canada.

As for every country, whatever is decided should be internally consistant for
that country (per the naming convention).

--maveric149


body:
On Friday 12 July 2002 12:01 pm, you wrote:
> The developers seem totally clueless about performance issues. There is a
> constant focus on new functions, and none on performance and response
> times.

Much is now being done to remedy performance problems -- so I do believe what
you said is needlessly rude (even if there is a grain of truth to it).  This
is an issue that has crept up upon the developers as new features were added
-- many of which were asked for by the users.

Software development seems to often work a lot like article development  --
many new things are added by different people, but after a point the software
or article needs to be heavily edited and reorganized to resore efficiency
and flow (although, admittedly, the php software never was /very/ efficient
to my recollection -- it still rocks though).

--maveric149


body:
> I asked two weeks ago what this picture is about and have gotten no
> answer. What is it for? If I don't hear back soon, I shall delete
> it.

I don't think it is appropriate to delete a picture just because you
don't know what it is for and the uploader didn't respond to your
mailing list request. The upload area is used by all Wikipedias, and
unless we are dealing with a clear case of warez or music swapping,
the uploaders should get the benefit of the doubt.

Axel


body:
At 02:29 AM 7/13/02 -0700, maveric149 wrote:
>Much is now being done to remedy performance problems -- so I do believe
what
>you said is needlessly rude (even if there is a grain of truth to it).

My experience with coders has been in MUDs. Coders like to make new areas,
tweak races and guilds, etc. Performance issues are not nearly as
attractive nor is success in dealing with them easily attainable or
recognizable to others. Like a MUD we are relying on voluntary coders so,
sure, courtesy is necessary.

But so is focus. Right now Wikipedia is like some Ionesco play with a dead
body in the room that everyone inexplicably ignores.

Fred Bauder


from_name: Vicki Rosenzweig

subject: [Wikipedia-l] Re: City, state convention

body:
At 10:29 AM 7/13/02 +0100, Tarquin wrote:

>I agree with Lars:
> > "Sydney, Australia" and "Stockholm, Sweden" just sound silly in any
> text.  It would be as strange as "floppy, computer device" or cheese,
> food item". Writing [[Sydney, Australia|Sydney]] is not a good solution.
>
>I've suggested before that for countries which do not have an existing
>disambiguation nomenclature, we use standard Wikipedia format: "Paris (France)"
>
>Someone just created "Shikoku, Japan" -- AFAIK there is no other
>"shikoku". There may be a need for "Shikoku, Japan" to exist, should
>another writer link to it, but the article should be on "Shikoku".
>
>In general, like Lars said, phrases such as "Paris, France" are poor
>style. If the context is not already clear from the article -- "French
>composer, born in Paris" for instance -- it is better to write "Paris, in
>France" or even "Paris (France)".

I agree that "Shikoku" is sufficient, assuming it's the only place of that
name. But by
what standard is "Paris (France)" better style in an English sentence than
"Paris, France"?
If there's need to specify what city called Paris you mean, "Paris, France"
is normal
English usage. "Paris (France)" is not common English usage, and in fact,
taken as a
model, risks other confusion: someone might reasonably write that
"So-and-so was born
in Christiana ([[Oslo]])", meaning not that Christiana is in Oslo, but that
the two names
refer to the same place.

Vicki, who lives in New York, New York, not in New York (New York).
--
Vicki Rosenzweig
vr at redbird.org
http://www.redbird.org


from_name: tarquin

body:
Daniel Mayer wrote:

>On Friday 12 July 2002 12:01 pm, : Karen AKA Kajikit  wrote:
>
>
>>I don't think it sounds silly... sure within the context of a single
>>country it's wierd and wouldn't be used, but when you're talking
>>worldwide then it's quite normal to use the country name as well as the
>>city just as an identifier. You want to make it obvious to people who
>>are ignorant just where Sydney is, and if they don't know where
>>AUSTRALIA is then the situation is hopeless and you can just give up
>>
>>
>This is called a disambiguation block and this form of disambiguation is
>useful in cases like this where a famous thing has the same name less famous
>things.  Paris is an even more obvious example.
>
>
The point of my "disambiguation block" suggestion was that at least one
city would get the plain, unquantified name.

I agree with Lars:
 > "Sydney, Australia" and "Stockholm, Sweden" just sound silly in any
text.  It would be as strange as "floppy, computer device" or cheese,
food item". Writing [[Sydney, Australia|Sydney]] is not a good solution.

I've suggested before that for countries which do not have an existing
disambiguation nomenclature, we use standard Wikipedia format: "Paris
(France)"

Someone just created "Shikoku, Japan" -- AFAIK there is no other
"shikoku". There may be a need for "Shikoku, Japan" to exist, should
another writer link to it, but the article should be on "Shikoku".

In general, like Lars said, phrases such as "Paris, France" are poor
style. If the context is not already clear from the article -- "French
composer, born in Paris" for instance -- it is better to write "Paris,
in France" or even "Paris (France)".


body:
Fred Bauder wrote:

>At 02:29 AM 7/13/02 -0700, maveric149 wrote:
>
>
>>Much is now being done to remedy performance problems -- so I do believe
>>
>>
>what
>
>
>>you said is needlessly rude (even if there is a grain of truth to it).
>>
>>
>
>My experience with coders has been in MUDs. Coders like to make new areas,
>tweak races and guilds, etc. Performance issues are not nearly as
>attractive nor is success in dealing with them easily attainable or
>recognizable to others. Like a MUD we are relying on voluntary coders so,
>sure, courtesy is necessary.
>
>But so is focus. Right now Wikipedia is like some Ionesco play with a dead
>body in the room that everyone inexplicably ignores.
>
>Fred Bauder
>
>[Wikipedia-l]
>To manage your subscription to this list, please go here:
>http://www.nupedia.com/mailman/listinfo/wikipedia-l
>
>
>
>
>
Hi Fred,

I'm not a Wikipedia coder, I mostly contribute articles. Wearing my
contributor hat, I've been really irritated by the disastrous
performance in the last couple of months. My style of editing is
pointillistic, with lots of tiny edits, even breaking down large
articles into a series of smaller commits. I can understand that the
poor performance will have discouraged many people.

Having said that, I am helping test the new system at the moment, and I
can say that the coders are putting a lot of effort into making the
system fast and smooth. I think they've taken the right decision, to
give up on maintaining the old system, and work instead on getting the
new system up and running really well before putting it into service
ASAP. That means testing it really hard before going live.

Here's a progress report.

The new server is a dual-Athlon multiprocessor SMP machine with 2G of
RAM, and an ext3 journalling filesystem.

At the moment,  the new server is running a hugely expanded database
containing over 80,000 'articles', making a total of over 100,000
entries in the database. This consists of a snapshot of the entire
Wikipedia, together with about 50,000 machine-generated articles. It's
serving around 1.2 articles per second under as realistic a traffic load
as we can generate using lots of bots running on external servers - the
traffic is nice and bursty, and the system is regualrly peaking at 12
articles/second or more.

77% of pages are served in less than a second: what I'd consider to be
'instant'
89% of pages are served in less than 2 seconds: what I'd consider to be
'quick'
98% of pages are served in less than 5 seconds, about the start of the
threshold of irritation
99.3% of pages are served in less than 10 seconds, a significant delay

Only 0.1% of articles take longer than 20 seconds to serve: what I would
regard as failures.
About 30% of these are very big special pages. I'm trying to do a finer
analysis of the rest, and find the exact conditions that seem to trigger
the hiccups.

What this means, I hope, is that when the new server comes into service
it will perform really well under normal loads, and editing will be
smooth and lovely again.

Neil


body:
Have a look at the new software which (thanks to Lee) will go online on the
new server (thanks to Jimbo) today in a week:
	http://beta.wikipedia.com

For example,
	http://beta.wikipedia.com/wiki/Performance_tuning
is generated by the script in 0.08 seconds (check the bottom of the HTML
source). You were saying? ;)


body:
On Saturday 13 July 2002 12:01 pm,  tarquin wrote:
> Someone just created "Shikoku, Japan" -- AFAIK there is no other
> "shikoku". There may be a need for "Shikoku, Japan" to exist, should
> another writer link to it, but the article should be on "Shikoku".

This person also made [[Shikoku]] into a redirect to  [[Shikoku, Japan]].
Although this may need to be changed because Shikoku is also the name of a
breed of Japanese dog. That's one of the reasons why the city naming
conventions were developed -- you can't expect a user to know all uses of a
non-disambiguated term so having a system for naming similar things is
useful. So long as it is consistant within the country in question.

> In general, like Lars said, phrases such as "Paris, France" are poor
> style. If the context is not already clear from the article -- "French
> composer, born in Paris" for instance -- it is better to write "Paris,
> in France" or even "Paris (France)".

I've never seen "Paris in France" used a noun before. And the chances of
[[Paris (France)]] being linked to without using pipes is about nil.  This
perhaps isn't a good example because [[Paris]] redirects to [[Paris, France]]
since the city in France is far and away the most widely used meaning of
simply [[Paris]].

Also, when a person visits [[Paris, France]] the disambiguation block at the
top of the article makes it very clear that [[Paris]] redirects to that
article (a link to other uses is provided there too). I don't expect
contributors to start using pipes to link directly to the most famous city by
the name "Paris". Thus the redirect priority the French city has over
[[Paris]]. The software handles this fine in "pages the link here"

City disambiguation wasn't really made for the most famous examples anyway --
it was made for the thousands of other less well known cities that have
naming conflicts. We shouldn't expect a contributor to know that there are at
least 20 places called Richmond in North America alone. All they need to know
is that the one they are writing about is in California, so following the
city naming convention for the United States, that article should be titled
[[Richmond, California]].

The trouble is, that new contributors tend to first follow famous examples
before looking into [[wikipedia:naming conventions]] (usually looking at and
contributing for cities in their own country first). So the most famous
examples have to be disambiguated too. W can't have naming conventions
without at least some kind of logical consistency for naming similar types of
things. Redirects can take care of the oddities like Paris so as not to break
any links.

--maveric149


from_name: Kurt Jansson

subject: [Wikitech-l] Performance

body:
> Many thanks to Neil Harris for the bots;

Do we have an emergency tool to redo any edits from a certain IP on a
certain time? Just in case the bot falls into the wrong hands, or a
script kiddy makes one himself.

I know bots can also be useful. Ben-Zin has made one to automatically
generate the year pages. It could also be helpful in other cases, so it
would be a good idea to share it with others, but I am afraid someone
might misuse it.

Kurt


from_name: Neil Harris

subject: [Wikitech-l] Re: [Wikipedia-l] what's being done about performance?

body:
Eliminating 'unsuccessful search' and 'special' pages from the count
gives the following stats:

Analysing 100,000 lines from the raw log with this filtering gives:

bin in seconds, total pages, cumulative percentage

0 57360 83.443651%
1 6929 93.523516%
2 2028 96.473720%
3 1034 97.977917%
4 640 98.908948%
5 314 99.365735%
6 157 99.594129%
7 81 99.711962%
8 61 99.800701%
9 46 99.867619%
10 18 99.893804%
11 12 99.911261%
12 16 99.934537%
13 13 99.953448%
14 6 99.962177%
15 6 99.970905%
16 6 99.979634%
17 2 99.982543%
18 0 99.982543%
19 3 99.986907%
20 2 99.989817%

summary 68741 hits in  41366.343 secs, avg = 0.601771039118
only 9 non-special pages took over 20 seconds: here they are:

20020713011714 28.783 /wiki/Historical_anniversaries
20020713012523 20.301 /wiki/Sport
20020713014205 23.161 /wiki/Federal_Standard_1037C
20020713014723 25.357
/w/wiki.phtml?title=Free_On-line_Dictionary_of_Computing/O
_-_Q&redirect=no
20020713015936 21.513
/w/wiki.phtml?title=Wikipedia:Bug_reports&action=history
20020713022203 25.252 /wiki/Free_On-line_Dictionary_of_Computing/L_-_N
20020713025105 29.975
/w/wiki.phtml?title=Free_On-line_Dictionary_of_Computing/E_-_H&redirect=no
20020713033140 20.802 /wiki/Feature_requests
20020713043401 41.392
/w/wiki.phtml?title=Complete_list_of_encyclopedia_topics/R&diff=78830&oldid=71983

It's interesting to note that random spidering hits 'special' pages
about 30% of the time.

Where the page accesses have been binned by the integer part of their
service time as recorded in the logs.

This is looking really good.

-------------------------------------------------

SUGGESTION #1:

Looking at the logs suggests that many of the worst results are
generated on the special page options with large counts -- particularly
the versions with count=5000.

Here's my proposal: we should not list the options with count > 500 for
users *that are not logged in*.

So, at the bottom of the orphans page, a logged in user would see

  View (previous 50) (next 50) (20 | 50 | 100 | 250 | 500 | 1000 | 2500
| 5000).

and an casual browser (and any busy bots or spiders) would see

  View (previous 50) (next 50) (20 | 50 | 100 | 250 | 500 ).

Random selection from the first list will search on average
50+50+20+50+100+250+500+1000+2500+5000 / 10 = 952 pages

Random selection from the second list will search on average
50+50+20+50+100+250+500 / 10 = 102 pages

a reduction in load of almost an order of magnitude.

Removing these big outlier loads may well take some of the strain off
ordinary page loads that happen to occur at the same time.

------------------------------------------------

SUGGESTION #2:

The 'Unsuccessful search' pages can be enormous. They accumulate all the
bad searches in a whole month. As Wikipedia becomes more popular, they
have become huge, and they now take a long time to load. We should make
these weekly or daily instead of monthly, and perhaps split up the old
ones using a script.

This will also have the effect of improving the 'most wanted' rating of
frequently missed searches, as currently only one instance a month counts.

Or perhaps they should be generated as a special page from the database?

---------------------------------------------------

Neil


from_name: Mark Christensen

subject:
[Wikitech-l] Re: [Wikipedia-l] what's being done about perfor
	mance?

body:
> In the last week, my script made 410 attempts at 20 minute
> intervals to reach the page http://www.wikipedia.com/wiki/Chemistry
> Out of these, only 86% were served in less than 5 seconds.
> Five percent of the calls timed out (my limit is 60 seconds).
> Now, this is far better than the worst problems that Wikipedia
> saw in April or May, but it is still pretty miserable.

I believe the performance in the new code is much improved.

Even under all the load I can put on it with a T1 line, the beta software is
producing average load times of less than 1.5 seconds.

A number of people are now working on stress testing this software before it
is put into production.  And I think there is a general commitment to
solving the performance problem, and I see lots of movement in the right
direction.


body:
On Sat, 13 Jul 2002, Daniel Mayer wrote:
> Much is now being done to remedy performance problems -- so I do believe what
> you said is needlessly rude (even if there is a grain of truth to it).  This
> is an issue that has crept up upon the developers as new features were added
> -- many of which were asked for by the users.

If performance issues creep upon you, then you have not designed your
system for performance measurement and monitoring.  This means you are
clueless.  It is like driving a car without a speedometer, and being all
surprised when you are caught for speeding.

In the last week, my script made 410 attempts at 20 minute intervals to
reach the page http://www.wikipedia.com/wiki/Chemistry
Out of these, only 86% were served in less than 5 seconds.  Five percent
of the calls timed out (my limit is 60 seconds).  Now, this is far better
than the worst problems that Wikipedia saw in April or May, but it is
still pretty miserable.  The non-English Wikipedias feature very similar
numbers.

The Sevilla project (http://enciclopedia.us.es/) serves 96% of all my
attempts in under 2 seconds, and 99% in under five seconds.  This should
probably be attributed to luck rather than skill, but it helps move people
from the Spanish Wikipedia over to the breakout project.

> Software development seems to often work a lot like article development  --

That's OK, but just like the basic Wiki software defines the concept of an
article (it can be written, reviewed, its history tracked, modified,
removed), the software should define a framework for new functionality
that can measure its impact on performance, and turn it on or off.  Think
modules.

--
  Lars Aronsson (lars at aronsson.se)
  tel +46-70-7891609
  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/


body:
Having read Lars' post, to avoid confusion, perhaps I should point out
that my results in the previous mail were for beta.wikipedia.com.

Neil


subject: [Wikitech-l] Better apache server logs

body:
The apache server allows for customized server log messsages
(http://httpd.apache.org/docs/logs.html#accesslog). I think we should
include the directive %T, which reports the time it took to serve a
request. That way, we could process the server logs to pinpoint the
precise conditions which cause requests to take a long time.

Adding the line
  LogFormat "%h %l %u %t \"%r\" %>s %b %T" custom
to httpd.conf and changing
  CustomLog /usr/local/apache/logs/access_log common
to
  CustomLog /usr/local/apache/logs/access_log custom
should do the job.

Axel


subject: [Wikitech-l] Saturating the server

body:
I used the nice hammerhead tool (http://hammerhead.sourceforge.net) to
stress test the beta.wikipedia.com server. The tool lets you simulate
several simultaneous users repeatedly requesting pages from (or
posting to) the server.

I have my users request RecentChanges (33% of the time) and issue
searches (66% of the time). Here's the average response time of the
server:

  1 user:    2 sec
  5 users:   4 sec
 10 users:   8 sec
 20 users:  11 sec
100 users:  42 sec

These are only the times it took the server to respond; the actual
total time to complete a request is not as useful in my case because
of my limited bandwidth.

Axel


subject: [Wikitech-l] Re: performance

body:
> Some special pages are still moderately slow (particularly "wanted"
> and "random page"), but the real time hogs now are very long pages
> with lots of links.

I looked at the random page code, and right now it fetches a complete
list of all article IDs, in order to pick one out randomly. There must
be a better way to do this. This should be an O(log n) operation, not O(n).

The "wanted" special page will get a lot faster if we implement
Jan's idea of a table recording the number of broken links to every
unwritten article.

If long pages with lots of links cause trouble, maybe we should revive
the caching idea of the current code: the cur table gets another
column cur_cache where we store the rendered HTML. When displaying an
article, we simply pump out the contents of cur_cache, or, if cur_cache
is empty, we render, display and store in cur_cache. If a newly saved
article necessitates the updating of links, we junk the cur_cache of
all affected articles.

Axel


body:
On Sat, 13 Jul 2002, Neil Harris wrote:
> Eliminating 'unsuccessful search' and 'special' pages from the count
> gives the following stats:
> Analysing 100,000 lines from the raw log with this filtering gives:

Where does this "raw log" come from?  I don't think I've heard it
mentioned before.  Is it from the real use of www.wikipedia.com or from
one of the test servers?

--
  Lars Aronsson (lars at aronsson.se)
  tel +46-70-7891609
  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/


subject:
[Wikitech-l] Re: [Wikipedia-l] what's being done about perfor
 mance?

body:
On Sat, 13 Jul 2002, Mark Christensen wrote:
> A number of people are now working on stress testing this software before it
> is put into production.  And I think there is a general commitment to
> solving the performance problem, and I see lots of movement in the right
> direction.

This is fine, but you are trying a new car, and I want you to mount a
speedometer so we can monitor the speed while driving, continuously
keeping an eye on the problem instead of making believe we can fix it
once and for all.  The timer that is included in an HTML comment could be
appended to a log file, and then there would have to be a way to review
that log file.  We did discuss this before, and someone was guessing that
appending a line to a file would take too much time, but I don't think
anybody tested this.  The append to file could be conditioned so only
accesses that took more than 2 (or 5) seconds were logged.

And, needless to say, this needs to be done on all web pages, including
the edits, diffs, and specials, and for all languages.

--
  Lars Aronsson (lars at aronsson.se)
  tel +46-70-7891609
  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/


from_name: Stephen Gilbert

body:
I don't see how anyone can say that performance issues
are being ignored. Jimbo bought a new server
specifically for Wikipedia. Mr. Crocker re-designed
the PHP script from the ground up, and Neil has been
running different kinds of bots on the news
server/software to test it under stressful conditions.

Don't you guys read the mailing list? :)

-- Stephen Gilbert

--- Fred Bauder <fredbaud at ctelco.net> wrote:
> At 02:29 AM 7/13/02 -0700, maveric149 wrote:
> >Much is now being done to remedy performance
> problems -- so I do believe
> what
> >you said is needlessly rude (even if there is a
> grain of truth to it).
>
> My experience with coders has been in MUDs. Coders
> like to make new areas,
> tweak races and guilds, etc. Performance issues are
> not nearly as
> attractive nor is success in dealing with them
> easily attainable or
> recognizable to others. Like a MUD we are relying on
> voluntary coders so,
> sure, courtesy is necessary.
>
> But so is focus. Right now Wikipedia is like some
> Ionesco play with a dead
> body in the room that everyone inexplicably ignores.
>
> Fred Bauder
>
> [Wikipedia-l]
> To manage your subscription to this list, please go
> here:
> http://www.nupedia.com/mailman/listinfo/wikipedia-l

__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com


body:
--- Lars Aronsson <lars at aronsson.se> wrote:
> If performance issues creep upon you, then you have
> not designed your
> system for performance measurement and monitoring.
> This means you are
> clueless.  It is like driving a car without a
> speedometer, and being all
> surprised when you are caught for speeding.

Correct. That's why the old script has been abandoned
and re-written.

Here's a quick Wikipedia software history for the
curious (if there are any errors, please correct
them...). When we were still using UseModWiki, there
was a general agreement to move to a SQL-based system,
but there wasn't a satisfactory option available.
Magnus was the only one willing to give it a go, and
so he put together the PHP/MySQL combo we're currently
using. I believe Magnus is a biologist who does some
coding, not a software engineer; his code worked but
was not "engineered", so to speak. After the script
was deployed, various developers stepped in to help
develop the software. However, there were too many
performance problems, and so Lee Danial Crocker took
up the challenge and redesigned the whole thing. The
new server running the new code will replace our
current setup as soon as it is ready.

-- Stephen Gilbert

__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com


subject: [Wikitech-l] Dummy page 95200

body:
I've just looked at this page: can you see how the italic markup seems
to have propagated into the quickbar?

And the W3C validator doesn't like it either: see
http://validator.w3.org/check?uri=http%3A%2F%2Fbeta.wikipedia.com%2Fwiki%2FDummy_page_95200&charset=%28detect+automatically%29&doctype=Inline
for details.

The cause seems to be unescaped '&'s, at first sight.

Neil

http://beta.wikipedia.com/wiki/Dummy_page_95200

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020714/2b056af4/attachment.html>


body:
Daniel Mayer wrote:

> The trouble is, that new contributors tend to first follow famous examples
> before looking into [[wikipedia:naming conventions]] (usually looking at and
> contributing for cities in their own country first). So the most famous
> examples have to be disambiguated too. W can't have naming conventions
> without at least some kind of logical consistency for naming similar types of
> things. Redirects can take care of the oddities like Paris so as not to break
> any links.

Consistency of formatting is the number one most important thing to
maintain. If some of the links are [[Fred Nerk City]] and some of them
are [[Fred Nerk City, Swaziland]] and some of them are [[Fred Nerk City
(Swaziland]] then it's just plain confusing. Sure there may be only one
Fred Nerk City in the world but like Mav said we can't expect everyone
to know that... It's like the Years templates - there isn't really any
likelihood that there'll need to be specific articles for 536BC AND
537BC but the articles were made anyway because the standard year
template allowed for links to them.

Now that we have the format in place for cities whenever we see a city
entry that's just under the name we need to move it to [[City, Country]]
to keep the consistency going. The redirect will take care of anyone who
is just looking for the city name if there is only one place by that
name.

--

Karen AKA Kajikit

You can take the dragon out of Alfandra, but you can never take Alfandra
out of the dragon (or the Kitty)...

Come and visit my part of the web:
Kajikit's Corner: http://Kajikit.netfirms.com/
Aussie Support Mailing List: http://groups.yahoo.com/group/AussieSupport
Allergyfree Eating Recipe Swap:
http://groups.yahoo.com/group/Allergyfree_Eating

Love and huggles to all!


from_name: Jimmy Wales

body:
Lars Aronsson wrote:
> The developers seem totally clueless about performance issues. There is a
> constant focus on new functions, and none on performance and response
> times. They cannot even say what the average response time has been in the
> last week or 24 hours.  (Hey, prove me wrong: Tell me the average.)

I don't think this is accurate.  The blame for the performance problem
rests squarely on my shoulders, not on the shoulders of the
developers.

On the current machine, Jason and I have login accounts on the
machine, but none of the developers do.  This makes it virtually
impossible for them to do serious study of performance issues.  They
have a lot of clues, and a lot of suggestions, but no means to
implement them -- I have been a bottleneck to the process.

This is all about to change, this upcoming Saturday.  The new machine
is wikipedia-only, and developers already have access to the machine.
Root access is available for a small handful, even.

This means that the developers can sit on the "live machine" and study
and tweak for performance, without having to guess what is wrong, and
with instant feedback possible, without waiting for me to get around
to fixing things.

It would be unfair for me to let the developers take the blame for a problem
caused by a single-point-of-failure that is me!  :-)

--Jimbo


subject: [Wikitech-l] Re: [Wikipedia-l] what's being done about perfor mance?

body:
Lars Aronsson wrote:
>We did discuss this before, and someone was guessing that
> appending a line to a file would take too much time, but I don't think
> anybody tested this.  The append to file could be conditioned so only
> accesses that took more than 2 (or 5) seconds were logged.

In my opinion, this would work just fine.  At bomis, which is run on
fastcgi/perl scripts cruder than you can possibly imagine, I
frequently hand-tune things by writing logfiles in this fashion.  At
least in perl/fastcgi, appending loglines to a file is fast enough
that it plays only the very tiniest role in the overall performance of
the site.

It's a very useful tool for getting a mental picture of what's going on.

--Jimbo


body:
Hum, I just read the [[Shikoku, Japan]] article. I seemed to have forgoten
from sixth grade geography that Shikoku is one of the islands of Japan. It
therefore does not at all pertain to the city naming conventions.

I would personally prefer such a large geographic entity to be not
disambiguated, esp. in a way that makes it appear as if it were a city (other
uses could be covered in a disambiguation block). Trouble is, which is the
most widely used use of the word -- the dog breed, the island or something
else I'm missing? Since there is no specific naming convention covering this,
the standard give and take of general disambiguation will have to be
followed.

I just did a google search. <Shikoku Japan -dog -breed> got over 60,000
results and <Shikoku dog breed> got only 697 results (<Shikoku dog> got less
than 10,000). Clearly [[Shikoku]] should be where the article on the island
should be (as I hoped). A link to [[Shikoku (dog breed)]] can be in a
disambiguation block at [[Shikoku]] (is this breed of dog know by anything
other than simply "Shikoku"? It would be a shame to have to needlessly use
parenthetical disambiguation).

--maveric149


from_name: Sean Barrett

body:
On a similar note, "chihuahua mexico -dog" yields 183,000 results, while
"chihuahua -mexico dog" yields 65,200.  We might dare to generalize: if
in doubt, place names take priority over things named for places -- if
the names are identical.  If the names vary even slightly, do the right
thing: [[Dalmatia]] is a region of [[Croatia]], but [[Dalmatian]] is a
[[dog]].

-----Original Message-----
From: wikipedia-l-admin at nupedia.com
[mailto:wikipedia-l-admin at nupedia.com]On Behalf Of Daniel Mayer
Sent: Sunday, July 14, 2002 12:31
To: wikipedia-l at nupedia.com
Subject: Re: [Wikipedia-l] Re: City, state convention

Hum, I just read the [[Shikoku, Japan]] article. I seemed to have
forgoten
from sixth grade geography that Shikoku is one of the islands of Japan.
It
therefore does not at all pertain to the city naming conventions.

I would personally prefer such a large geographic entity to be not
disambiguated, esp. in a way that makes it appear as if it were a city
(other
uses could be covered in a disambiguation block). Trouble is, which is
the
most widely used use of the word -- the dog breed, the island or
something
else I'm missing? Since there is no specific naming convention covering
this,
the standard give and take of general disambiguation will have to be
followed.

I just did a google search. <Shikoku Japan -dog -breed> got over 60,000
results and <Shikoku dog breed> got only 697 results (<Shikoku dog> got
less
than 10,000). Clearly [[Shikoku]] should be where the article on the
island
should be (as I hoped). A link to [[Shikoku (dog breed)]] can be in a
disambiguation block at [[Shikoku]] (is this breed of dog know by
anything
other than simply "Shikoku"? It would be a shame to have to needlessly
use
parenthetical disambiguation).

--maveric149
[Wikipedia-l]
To manage your subscription to this list, please go here:
http://www.nupedia.com/mailman/listinfo/wikipedia-l


subject: [Wikitech-l] Re: what's being done about performance?

body:
> This is fine, but you are trying a new car, and I want you to mount a
> speedometer so we can monitor the speed while driving, continuously
> keeping an eye on the problem instead of making believe we can fix it
> once and for all.

I agree that we need to have more clear data about performance numbers, and
that we need to monitor that data over time.  My main point was to say that
1) we've got a faster and more adaptable code base in the wings, and 2) this
will give us time to do performance monitoring right, as well as to think
clearly about which things to optimize.  As I said before, I think everybody
is trying to solve this problem, and that the developers are clearly moving
in the right direction.  I'm no performance expert, but  as a jack of all
trades and master of none, I'd like to toss out a few thoughts.

I'm concerned that we don't try to gather that data in a way which does
creates performance problems.

My experience is that it is a bad idea to make every function append
information to log file. I've never seen someone append to a log file on
every function call without a noticeable effect on performance, at least for
well designed ASP or JSP sites which have already minimized disk I/O.
I assume the same would be true with PHP.  However, I having just read
Jimbo's e-mail, and it could be  the I/O performance of the underling system
we were using -- mostly WinNT on slightly older machines -- that was to
blame.

That said, I'm aware of three common ways to get around this performance
bottleneck.

Most commonly, people just turn up the detain on their web server reporting
and parse those logs for the performance information they can get from
there.  The problem with this is that it is sometime fails to provide the
fine grained information we'd need to see exactly which pages are causing
the slowdown.

Second, I've seen a lot of people storing log information in a database.
This is probably what we should do if we need more precise information than
we can get from the server logs.

The third solution is the most flexible, which is to set up a thread which
asynchronously processes a queue of log data and occasionally writes the
information to disk.  This is clearly the most complex to implement, and I
wouldn't even consider it unless we decide we need realtime performance
monitoring, or there's something built into PHP to do this.

Regardless of how we choose to get this data, we can and should still be
thinking about how to optimize those functions which the server spends a LOT
of time doing, or those things which take a LONG time, as wikipedia usage
could very well continue to scale up dramatically, and we need to be ready
for this ahead of time. On the other hand, there is a cost to optimizations,
if only in the added complexity of the code.  Complex code is more difficult
to update and maintain, and it is less likely to attract new developers over
time, so we need to make considered choices about what functions to 1) leave
unutilized, 2) optimize, or 3) remove altogether.


from_name: Hr. Daniel Mikkelsen

body:
On Sun, 14 Jul 2002, Stephen Gilbert wrote:

> I don't see how anyone can say that performance issues
> are being ignored. Jimbo bought a new server
> specifically for Wikipedia. Mr. Crocker re-designed
> the PHP script from the ground up, and Neil has been
> running different kinds of bots on the news
> server/software to test it under stressful conditions.

The performance problem is a very serious issue, an emergency. I don't
know if it's been done already, but the front page should have a
prominent notice about what the problem is, and what's being done
about it.

Has this been done already? The performance problem prevents me from
actually checking... :)

Anyway, such a notice would put a lot of contributers at ease I
expect. That _something is being done_ is a very good feeling, almost
as good as having it work.

-- Daniel


body:
> Here's a quick Wikipedia software history for the
> curious (if there are any errors, please correct
> them...). When we were still using UseModWiki, there
> was a general agreement to move to a SQL-based system,
> but there wasn't a satisfactory option available.
> Magnus was the only one willing to give it a go, and
> so he put together the PHP/MySQL combo we're currently
> using. I believe Magnus is a biologist who does some
> coding, not a software engineer; his code worked but
> was not "engineered", so to speak. After the script
> was deployed, various developers stepped in to help
> develop the software. However, there were too many
> performance problems, and so Lee Danial Crocker took
> up the challenge and redesigned the whole thing. The
> new server running the new code will replace our
> current setup as soon as it is ready.

Besides me being a mere biologist:) IMHO part of the problem of "my"
software is this:
When I started, the aim was to replace the UseModWiki, and add new
functions. But it was unclear which functions, and how to implement them.
Some of the UseModWiki functions I implemented were removed (mainly the
subpages). Some ideas for new functions were implemented rather early (e.g.,
namespaces), some later (e.g., orphans). Some became popular (Most Wanted),
some were dropped (category functions, AutoWikification).
So, the whole script was constantly under development, trying new things,
removing others, altering, rewriting etc.

Lee's rewrite does not add many new features on the outside (except the
image namespace). The idea was that, now we have a better idea of what the
software should actually do, it can be coded more structured and technically
sound (I confess, Lee's code looks *a lot* better than mine;)
For example, Lee's software uses the database with a special field for each
article that marks redirect pages. That is very useful to sort out redirect
pages, as it is very fast. When I started my software, I didn't realize how
important this distinction would become, and thus my script always makes
MySQL compare the article text for something like "#REDIRECT %", which is
quite inefficient when used a lot.

This is one of the reasons why Lee's software ("Phase III") currently
handles >130.000 pages (>100.000 articles) on the test server, and still
generates the Main Page in less than 0.9 seconds.

Magnus


body:
Jimmy Wales wrote:

>Lars Aronsson wrote:
>
>
>>We did discuss this before, and someone was guessing that
>>appending a line to a file would take too much time, but I don't think
>>anybody tested this.  The append to file could be conditioned so only
>>accesses that took more than 2 (or 5) seconds were logged.
>>
>>
>
>In my opinion, this would work just fine.  At bomis, which is run on
>fastcgi/perl scripts cruder than you can possibly imagine, I
>frequently hand-tune things by writing logfiles in this fashion.  At
>least in perl/fastcgi, appending loglines to a file is fast enough
>that it plays only the very tiniest role in the overall performance of
>the site.
>
>It's a very useful tool for getting a mental picture of what's going on.
>
>--Jimbo
>_______________________________________________
>Wikitech-l mailing list
>Wikitech-l at ross.bomis.com
>http://ross.bomis.com/mailman/listinfo/wikitech-l
>
>
>
>
>
I agree: logfile writes will normally be buffered in disk cache RAM by
the OS unless a file is explicitly sync'd, so the overhead is perhaps
one disk hit per 5 seconds, at an average cost of a seek, half a spin,
and a single block write: perhaps 15 ms in 5s, or 0.3% of total disk
performance.

Using Apache's log rotation system is good too: having, say, 14 logs and
rotating them daily will keep two weeks of logs. At 100,000 hits/day and
80 chars/log entry, each logfile is 8 Mbytes, and the entire set of logs
will only take 112 Mbytes on disk: hardly noticeable on modern systems
with tens of gigabytes of spare disk space.

Neil


from_name: The Cunctator

subject: [Wikitech-l] Performance

body:
On 7/13/02 4:19 AM, "lcrocker at nupedia.com" <lcrocker at nupedia.com> wrote:
> "Current events" strikes me as a particularly big, yet solvable,
> problem.  We should come up with a way of breaking it into manageable
> pieces.  Chinese sovereign can clearly be broken up as well, and we
> can do things like replace the Scrabble diagrams with images.
>
We really shouldn't replace the diagrams with images. That would be a major
step backward. It's bearable that the page is slow--it should be a test case
for improvement.

However, I agree that super-high-traffic pages like "Current events" should
handled expediently.


from_name: Chuck Smith

subject: [Wikipedia-l] Brazil tomorrow, Europe in August

body:
Hey everybody,

As most of you know, I haven't been very active with
Wikipedia lately, so I'll really not be active soon.
Tomorrow, I'm going off to Brazil to go to the two
largest annual Esperanto conventions in the world.  I
got a nice new digital camera, so maybe I can take
some pictures for Wikipedia.  In a month, I'll be
heading to Europe to travel for a year or so.  My
travel plans are available at
http://eo.wikipedia.com/wiki/Chuck_SMITH.

I want to thank Brion Vibber publicly for his work on
the Vikipedio brochure that I'm going to pass out at
my lectures in Brazil, France and Germany.  It can be
found at
http://eo.wikipedia.com/wiki/Vikipedia_Proklamo.  I
just hope the Vikipedio is going at a reasonable speed
when I demo it...

I'm now unsubscribing from Wikipedia-L and Wikitech-L,
but I'll stay on Intlwiki-L.

Best wishes!
Chuck

=====
Guns don't kill people, cars kill people!
=========================================
Travel Plans: http://eo.wikipedia.com/wiki/Chuck_SMITH
Mia Taglibro: http://amuzulo.livejournal.com/
Enciklopedio: http://eo.wikipedia.com/

__________________________________________________________________

Gesendet von Yahoo! Mail - http://mail.yahoo.de
M?chten Sie mit einem Gru? antworten? http://grusskarten.yahoo.de


subject: [Wikitech-l] Re: [Intlwiki-l] How to multiply?

body:
On Mon, 24 Jun 2002, Tomasz Wegrzanowski wrote:
> Count of all contributors would be over 30, but number of current
> active is certainly lower. How much lower depends on how you define
> "current active".

I guess the definition could be discussed, but here is my suggestion.
At the end of each month, I run statistics on the rclog file as produced
by the old UseModWiki software used in my Wiki site.  This file contains
one line for every time anybody pressed "Save" (minor or major changes).


from_name: lcrocker at nupedia.com

subject: [Wikitech-l] Last-minute changes before announcement.

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.wikimedia.org/pipermail/wikitech-l/attachments/20020715/14af364f/attachment.ksh>


subject: [Wikipedia-l] New server, software coming Saturday.

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/ec84af68/attachment.diff


body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/9928abf1/attachment.diff


subject: [Wikitech-l] Page layout problem in mozilla

body:
If this doesn't happen to Windows/IE and Netscape customers, it is probably not
particularly important.

http://beta.wikipedia.com/wiki/Special%3ARecentchanges

In mozilla, this page renders with the logo on top of the menu dropdown box and
the "Go" button.  (Also, above that, the "Main Page" and "Recent Changes" links
have the logo on top of them.)

The features "under" the logo are all functional, it just looks ugly.


subject: [Wikitech-l] Oh, and random pages

body:
Random page works fine for me, too, in GNU/Linux (Redhat), KDE 2.1 (or something like that),
Mozilla.

--Jimbo


from_name: koyaanisqatsi at nupedia.com

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/4e3b8a46/attachment.diff


body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/277d40ec/attachment.diff


subject: [Wikipedia-l] what's being done about performance?

body:
> Besides me being a mere biologist:)...

Never let it be said that I called you "mere", good
sir.

--Stephen Gilbert

__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com


in_reply_to: <MLEPJFEEOFMFHGIHGJPIKEAICCAA.magnus.manske@epost.de>; from magnus.manske@epost.de on Mon, Jul 15, 2002 at 06:52:09PM +0200

body:
Magnus Manske wrote:
> Why would anyone read wikipedia in a window 400 pixels wide when a single
> click can make it 800? You can "uglify" most web pages that way.

Sure!  But, for me, that isn't the issue.  I'm at 1024x768, a pretty normal setting.  I
am not putting my window to any strange size, either.

I have to say, though, that if this is a Mozilla-only problem, we don't need to worry
a lot about it.

Some will say, and I have sympathy for them, that they like to run a 400 pixel wide
browser so that they can have two things going at the same time.  I don't operate that
way, but some people do.  So maybe we should care a little about them, someday.

> As for the "Random Page" issue, I tried from work and from home, with and
> without proxy, on beta.wikipedia.com and on Lee's piclab server, but I
> couldn't reproduce the effect. I get another random page every time. And I
> don't have any "fancy settings" in my browser.

Same here.

Perhaps the person who had the problem can try again today?


from_name: Daniel Mayer

subject: [Wikipedia-l] Naming conventions page

body:
>perhaps it would help if the naming conventions
weren't such a mess. I
>spent ages there looking for information when I first
started writing,
>as I was anxious to follow the established
conventions -- hence my
>recent suggestions to refactor that area -- which I'm
still mulling
>over.
>
>tarquin

I agree, the naming conventions page needs to be
streamlined ? it has reached an unwieldy length and
organization. If nobody objects I will summarize each
convention and place the rationale and details in
their own separate pages (without changing any
conventions, of course). The ship naming convention
has already been organized this way.

--maveric149

__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com


subject: [Wikipedia-l] RE: "Email this user" function and @wikipedia.com email accounts

body:
This may be a duplicate, so please disregard if this
if the case.

LDC wrote:
>The questions, then, are (1) when a new user signs
up, giving an e-
>mail address, should we further require that he check
a box
>saying "allow other wikipedians to mail me" or should
we let him
>check a box saying "don't allow other wikipedians to
mail me".  The e-
>mail address is never published in any case, and the
only mails
>received are one-to-ones, not lists, but I suppose
opt-in is still
>the more responsible thing to do.  (2) Perhaps the
double opt-in
>confirmation message could be sent whenever this
option is selected.
> (3) The mail messages themselves might have a
section something
>like "This mail was sent from Wikipedia's e-mail
function.  If you
>don't want to receive mails like this, go to..."

If we do decide to have this function, I think that in
order for it to be at all useful the user account
set-up should allow the user to opt-out if they click
the box next to "don't allow other wikipedians to mail
me". It has been my experience that most people simply
accept the defaults on webforms and if the default is
set /not/ to enable this function, then there won?t be
enough users with the function enabled to make it
commonplace (if the great majority of users don?t have
a link to ?Email this user? then other users visiting
user pages won?t look to the link).

I also agree with your previous statement about
requiring sysops to have this function enabled
(anybody taking on sysop status needs to be
contactable to the community and other sysops). I?m
not sure if it would be worth the effect to hard-code
this requirement into the software ? visiting sysop
pages will reveal who is and who is not adhering to
policy (if this does become one).

It would also be nice if sysops could be able to email
all other sysops when needed (or even to each and
every user signed up if absolutely needed ? like if
somebody hacked the server and stole user passwords).
For example, I had to manually post a message to each
and every sysop a while back warning them not to use
the administrative move feature because of a serious
bug that truncated page histories. It would have been
nice to draft a single email for that.

BTW, would it be possible for users to eventually have
@wikipedia.com email addresses (on a pay basis open
only to users with user accounts ? but that
requirement isn?t essential to me)? I would happily
give Bomis or a ?Wikipedia Foundation? 30 bucks a year
for a maveric149 at wikipedia.com
(maveric149 at wikipedia.org would be even better) than
have to pay the same amount for maveric149 at yahoo.com.

--maveric149

__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com


in_reply_to: <200207151748.g6FHmdV17481@ross.bomis.com>; from lcrocker@nupedia.com on Mon, Jul 15, 2002 at 10:48:39AM -0700

body:
lcrocker at nupedia.com wrote:

>
> > http://beta.wikipedia.com/wiki/Special%3ARecentchanges
> >
> > In mozilla, this page renders with the logo on top of the menu
> > dropdown box and the "Go" button.  (Also, above that, the "Main
> > Page" and "Recent Changes" links have the logo on top of them.)
>
> I'm not sure what you mean here; I'll need to know at least
> which Skin you're using and what your quickbar setting is.  A
> screenshot would be nice.

I'm not logged in, so it's the default.

> I'd really like to see a screenshot.

Can you recommend a screenshot program?

--Jimbo


subject: [Wikipedia-l] Caching proxies

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/39be9329/attachment.diff


from_name: Magnus Manske

subject: [Wikipedia-l] AW: Re: [Wikipedia-l] New server, software (Random page problem)

body:
Must be your proxy settings or something; it works great for me, in IE and
Mozilla. You obviously get a cached page, either from your computer or a
proxy.

Magnus

________________________________________
Online Fotoalben - jetzt kostenlos bei http://www.ePost.de


subject: [Wikipedia-l] Are we an educational purpose?

in_reply_to: <MLEPJFEEOFMFHGIHGJPIKEAKCCAA.magnus.manske@epost.de>; from magnus.manske@epost.de on Mon, Jul 15, 2002 at 08:31:54PM +0200

body:
This would be satisfactory *for our site*, I'm sure.  However, it would not
be consistent with the GNU FDL.  That is, we could use the images for our site,
which is for educational purposes, but we couldn't authorize other people to have
as much freedom as the GNU FDL gives them.

--Jimbo

Magnus Manske wrote:

> Lost of web pages make their content available without charge for
> "educational purposes". IANAL, but I guess an encyclopedia is clearly
> educational, so does that apply to wikipedia as well? If yes, it would mean
> that we could copy texts and images (all while giving proper credit to the
> original source).
>
> Magnus
>
> [Wikipedia-l]
> To manage your subscription to this list, please go here:
> http://www.nupedia.com/mailman/listinfo/wikipedia-l


in_reply_to: <200207151900.g6FJ07i26110@ross.bomis.com>; from lcrocker@nupedia.com on Mon, Jul 15, 2002 at 12:00:07PM -0700

body:
I think everything should be GNU FDL or GPL or similar.  I'm not really comfortable
with deviating from that.

lcrocker at nupedia.com wrote:

>
> > Lost of web pages make their content available without charge
> > for "educational purposes". IANAL, but I guess an encyclopedia
> > is clearly educational, so does that apply to wikipedia as well?
> > If yes, it would mean that we could copy texts and images (all
> > while giving proper credit to the original source).
>
> The trouble is that we could not re-license it under the GFDL.
> That's probably OK for images.  That's one of the main uses I
> envisioned for the image description pages--to list credits and
> external sources.  But I think it's important for our text to be
> GFDL'd, and we can't do that with borrowed copyrighted texts. We can,
> however, summarize them in our own words and link to them--that's
> really not that much work.
> 0


body:
lcrocker at nupedia.com wrote:

>As for the links to large queries like the "5000" recent changes, it
>occurs to me that the user can get any number he wants by composing
>the URL himself, and anyone who actually wants a list of 5000 things
>will probably know how to do that, so I removed all the big-number
>links entirely, logged in and otherwise.
>
>Also, Magnus showed me a SQL trick I'd never seen before ("order by
>rand()"...wouldn't have thought of that) that enabled a much faster
>implementation of RandomPage, and I thought that was worth changing
>before the announcement and isolated enough that it's unlikely to
>affect anything else in the code.
>
>Those changes are installed and test well under the bots, so I'm going
>to go ahead and announce the transition on the main list and the front
>page to try to get some realistic user testing.
>
>0
>

I've been extracting data from the server logs, and graphing it in Excel.

The removal of the big count options seems to have occured after server
timestamp 20020714152016.
This _seems_ to have made things better, but I can't be sure, as the
system has been fairly lightly loaded since them, and it's difficult to
find periods of matching load.

Could some more people please load the beta test site some more, as I
can't generate enough load by myself on my 512k link.

Thanks.

 Neil


body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/4b1c97ec/attachment.diff


subject: [Wikipedia-l] Cologne Blue

body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/206caa90/attachment.diff


from_name: Karen AKA Kajikit

body:
Darn, I was just going to say that the wikipedia performance seems to
have improved again... last night I was able to keep on working on
entries until I went to bed, when usually it starts timing out at around
7pm. However, it's just become non-responsive again at midday :(

I can't wait for the new version to be installed... hope that it works a
LOT better!

--

Karen AKA Kajikit

You can take the dragon out of Alfandra, but you can never take Alfandra
out of the dragon (or the Kitty)...

Come and visit my part of the web:
Kajikit's Corner: http://Kajikit.netfirms.com/
Aussie Support Mailing List: http://groups.yahoo.com/group/AussieSupport
Allergyfree Eating Recipe Swap:
http://groups.yahoo.com/group/Allergyfree_Eating

Love and huggles to all!


from_name: Zoe Comnena

subject: [Wikipedia-l] Geographic naming conventions

body:
I asked this question in the naming conventions area, but nobody replied, so I'm hoping somebody can come up with a response here.

When we're referring to places that begin with Saint, Mount, Fort, etc., do we spell the whole word out?  We have instances of [[Saint Louis, Missouri]] and [[St. Paul, Minnesota]], if I remember correctly.

Zoe

---------------------------------
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/cca00f0a/attachment.htm


body:
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://lists.wikimedia.org/pipermail/wikipedia-l/attachments/20020715/b21623d9/attachment.diff


from_name: Lars Aronsson

subject:
[Wikipedia-l] AW: Re: [Wikipedia-l] New server, software (Random
  page problem)

body:
On Mon, 15 Jul 2002, Karen AKA Kajikit wrote:
> Magnus Manske wrote:
> > Must be your proxy settings or something; it works great for me, in IE and
> > Mozilla. You obviously get a cached page, either from your computer or a
> > proxy.
> Well I shouldn't. It's not up to me whether or not my isp uses a proxy
> server to service the network faster. Labyrinth does as a matter of
> fact. But it's hardly the only server that uses proxies and I think it's

The URL http://www.wikipedia.com/wiki/special%3ARandomPage
does not (according to Mozilla) specify the "Expires" HTTP header.  There
is no HTML tag that specifies it either. This might explain why it sticks
in the proxy cache.  Try to set Expires to Jan 1, 1970 and see if that
helps:

<META HTTP-EQUIV="expires" CONTENT="Thu, 1 Jan 1970 00:00:00 GMT">

(Do not waste time to find out the current time, because the proxy's
internal clock might be out of sync.)

--
  Lars Aronsson (lars at aronsson.se)
  tel +46-70-7891609
  http://aronsson.se/  http://elektrosmog.nu/  http://susning.nu/