from_name: Jimmy Wales

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
Timwi wrote:
> >  $sql = "SELECT cur_title as title from cur where cur_namespace=0";
>
> This query sucks big time.
>
> Do you know what this does? This retrieves the titles of ALL ARTICLES in
> Wikipedia. Do you know how many there are? ...

Right.  The purpose here is to make a friendly giant XML file to
enable Yahoo (and presumably other likeminded whoevers) to grab a
single giant document to study rather than having to crawl over the
whole site.  The purpose (from our point of view) is to have search
engines more up to date, since this file can be downloaded by them
once per day or hour rather than a crawl taking weeks.

> I'm afraid I don't understand the purpose of the script. It seems to me
> that it is generating one ridiculously huge file that contains all of
> Wikipedia. What use would such a file be to anyone, even Yahoo?

*nod* It's so they can do the same thing they would do with a crawl of
the site, but virtually instantaneously.

> I stress I don't really understand the purpose of the script, nor do I
> know exactly what Yahoo!'s (or anyone else's) requirements are, but it
> would seem way more sensible to me to have several smaller files, each
> of which containing maybe at most 100 articles or perhaps at most 1 MB
> of data or something. Each file should then contain a list of cur_ids,
> and then you can easily check for each file if any of the articles
> therein have changed since the last update.

It does seem that rather than feeding them One Big File, we could feed
them files of diffs or whatever.  But that'd be more complex and
require greater co-ordination.  This at least has the virtue of
simplicity.

It shouldn't run more than once per day at first.  I'm not sure what
their goals are with respect to how often they would *like* to receive
it, but daily is a fine start.

--Jimbo


list: wikitech-l

source: pipermail-mbox

from_name: Brion Vibber

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
On Apr 2, 2004, at 00:35, Timwi wrote:
>>   $sql = "SELECT cur_title as title from cur where cur_namespace=0";
>
> This query sucks big time.
>
> Do you know what this does? This retrieves the titles of ALL ARTICLES
> in Wikipedia.

That's kinda the point, yeah. It might be better to skip redirects,
though; otherwise they should be handled in some distinct way.

> It seems that getPageData() retrieves the text of a page. In other
> words, it performs yet another database query. And you're calling that
> FOR EVERY ARTICLE in Wikipedia!

That's obviously a bit inefficient, but yes. Incremental updates of
only changed pages could hypothetically lead to faster output
generation after the first run, though this would require some
intermediate storage (since we don't yet have a running parser cache).

> I'm afraid I don't understand the purpose of the script. It seems to
> me that it is generating one ridiculously huge file that contains all
> of Wikipedia. What use would such a file be to anyone, even Yahoo?

(It would produce a series of files up to about 12.5 megabytes in
length, not one big file.)

A text base without the unnecessary UI elements could improve search
results, and I suppose can be kept more complete more easily than
constant spidering of a 200k+ page site. *shrug* If that's the data
format they want, hey fine, though having to download the entire set of
a couple hundred megabytes for every update doesn't sound ideal.

Jason, would each output need to be self-contained, or can they accept
incremental updates in IDIF? How often would they pull updates?

-- brion vibber (brion @ pobox.com)


list: wikien-l

subject: [WikiEN-l] TDC is starting to piss me off

body:
steven l. rubenstein wrote:
> I just found this message from TDC on my talk page
> http://en.wikipedia.org/wiki/User:Slrubenstein
>
> >A question for you:
> >Now, I know you Marxist types dont believe in an afterlife and the like,
> >but if I were to hold a seance and channel Marx's spirit through you, do
> >you think you would try and suck your own dick. TDC 04:15, 1 Apr 2004
> >(UTC)

Clearly, comments of that sort have no place in the wikipedia
community.  You're right, Steven, that it's generally best to try
kindness and education on people like this -- at first.  We need to
always project the attitude that, hey, we aren't uptight ninnies or
anything, but after all there is such a thing as being anti-social and
unhelpful, and when it comes right down to it, the community can't
function _in our capacity as encyclopedists_ if behavior like that
flourishes.

There's a certain amount of rough-and-tumble in the give-and-take to
build great articles, of course.  But then there's crossing the line,
which comments like this clearly do.

--Jimbo


subject: [WikiEN-l] Re: Why am I blocked ?

body:
Tim Starling wrote:
> It's possible for a malicious person to trick your browser into
> requesting the Special:Blockme address, e.g. with an image with its
> source set to Special:Blockme

Hmmm, that doesn't sound sensible, it's just too easy for someone to
screw around with.

Is this right?  All I need to do is have a cgi script on a web page
that dynamically generates a link like this:

<img src=http://en.wikipedia.org/w/wiki.phtml?title=Special:Blockme&ip=xxx.xxx.xxx.xxx>

where I substitute xxx.xxx.xxx.xxx with the victim's ip number?

So, on my User:EvilUser homepage I just write: "Sysops and
wikipedians!  Before you ban me or get upset with my actions, please
read my explanation of my behavior at
http://www.eviluser.com/wikipedia.cgi !  Thanks!"

Heh.  But, not good.

This seems easy enough to fix.  The link above should do nothing.  If
we're testing a proxy, we should try to get the client to request
...?title=Special:Blockme&validation=xxxxxxxxxxxxxxxx

where 'xxxxxxxxxxxxxx' is something that we can generate easily but
that's difficult for User:EvilUser to duplicate.

--Jimbo


list: foundation-l

from_name: Erik Moeller

subject: [Foundation-l] Test for foundation-l

body: Test offensive


list: wikimedia-l

subject: [Foundation-l] Test for foundation-l

body: Test offensive


subject: [Wikitech-l] Re: Yahoo! XML feed

body:
On Apr 2, 2004, at 00:53, Jimmy Wales wrote:
> It shouldn't run more than once per day at first.  I'm not sure what
> their goals are with respect to how often they would *like* to receive
> it, but daily is a fine start.

It would take hours just to run a complete dump, which would be the
equivalent of a sizeable fraction of our total daily page views. (Best
case might be 100ms per page for 240,000 pages =~ 6 hours 40 minutes)

If we're going to run something like this daily, some sort of
incremental updates are a must, though we can probably get away with
stuffing the saved data per page in a database or such and slurping it
back out fairly quickly.

-- brion vibber (brion @ pobox.com)


from_name: Jason Richey

subject: [Wikitech-l] Yahoo! XML feed

body:
I figured as much.  The way the spec is worded, I'm not sure it needs
to be valid XML.  They basically say that anything aside from
"</CONTENT>" can be between the CONTENT tags.

Jason

Brion Vibber wrote:

> Another thing; our HTML output is not yet guaranteed to be valid XML.
> This could cause a problem for literal inclusion of the HTML content
> into the XML output.
>
> -- brion vibber (brion @ pobox.com)
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikitech-l

--
"Jason C. Richey" <jasonr at bomis.com>


subject: [Wikitech-l] Re: Yahoo! XML feed

body:
Brion Vibber wrote:
> It would take hours just to run a complete dump, which would be the
> equivalent of a sizeable fraction of our total daily page views. (Best
> case might be 100ms per page for 240,000 pages =~ 6 hours 40 minutes)
>
> If we're going to run something like this daily, some sort of
> incremental updates are a must, though we can probably get away with
> stuffing the saved data per page in a database or such and slurping it
> back out fairly quickly.

*nod* That's very interesting.

As of this weekend, assuming things go well, we will have geoffrin and
suda as database servers, and there's no problem in the short run with
having gunther (still semi-on-loan from bomis) continue to run a 3rd
replicator.  So perhaps gunther could be tasked with generating this
daily, so as to make sure it isn't interfering with anything else.

Am I right in assuming that the load on the main db of having a
replicator attached is quite small?

Although we are not a business, so that pageviews don't equal revenue,
I still like to make decisions as if that were true.  For the
organization as a whole, with my grand longterm vision of the
Wikimedia Foundation being an organization similar in size and scope
to say the National Geographic Society or Consumers Union, that's not
a bad approximation.  More pageviews means more fame which will
ultimately mean more donations, more book sales, whatever.

Therefore, feeding data in a helpful way to major search engines is a
very inexpensive form of advertising.  It's worth doing, I think.

--Jimbo


list: wikipedia-l

subject: [Wikipedia-l] foundation-l mailing list created

body:
There is now a new mailing list for all issues related to the Wikimedia
foundation as a whole:

http://mail.wikipedia.org/mailman/listinfo/foundation-l

Please subscribe if you are interested in
- starting new Wikimedia projects
- finding clever ways to raise funds
- setting up a local chapter of the Wikimedia Foundation
- helping with the Wikimedia Board elections
etc.

Please also copy this information where appropriate (local goings-on
etc.).

Regards,

Erik


subject: [Wikitech-l] Re: Yahoo! XML feed

body:
On Apr 2, 2004, at 01:39, Timwi wrote:
> If the problem is that each article has navigational elements on the
> top, bottom, and left, which multiplies the time required to crawl
> things, then how about allowing something like &raw=1 which would
> output only the parsed page, without a skin or anything, with all the
> links in it also pointing to &raw=1, and then arranging with Yahoo to
> have them spider these pages, but still send people to the URL without
> &raw=1?

Then they'll still have to make 240,000+ HTTP connections to check
every individual page for updates, which can take days or weeks
depending on the crawl delay.

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] MediaWiki 1.2.3 release

body:
Changes since 1.2.2:
* Fixed an in-place install bug with non-root MySQL user
* Fixed history diff checkboxes bug on titles with ampersands
* Fixed printable link bug on special pages with parameters
* Fixed bug that broke IP blocking w/o memcached
* Turns off E_NOTICE warnings if PHP settings have them on
   (you can grope in and turn this off if you like to debug)

There are no database format changes. The proxy blocker hasn't yet been
merged, I'll try to get that in sometime...

Release notes:
https://sourceforge.net/project/shownotes.php?release_id=228178

Download:
http://prdownloads.sourceforge.net/wikipedia/mediawiki-1.2.3.tar.gz?
download

If you encounter a problem in the installation process, please include
*all* of the output under the 'Checking environment' heading with your
report!

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] Merge status on SQL old

body:
On Apr 2, 2004, at 02:03, Carlos Baquero Moreno wrote:
> Is there any flag on table "old" that indicates that a given update
> was done
> after merging with the previous version ? This kind of information
> would be usefull for my extraction of patterns, since I am interested
> on the
> conflict rate.

Sorry, this info is not recorded.

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] Title-template associations and default subpages

body:
It would be neat to be able to associate certain regular expression
matches in a title, e.g.

"^Talk.*\/Deletion$"
(matches Talk:<anything>/Deletion)

with certain templates, e.g. [[Template:Deletion]].

When I create a new page that matches the regex, say,
[[Talk:Pompeii/Deletion]], the template would automatically be loaded into
the usually empty edit box. It could contain things like

  == Deletion discussion ==

  == Alternatives ==

  [[Category:Deletion]]

  Please read the [[Wikipedia:Deletion policy]] before participating
  in this discussion.

[[Category:Deletion]] could then be used with Magnus' scheme to get a list
of all pages with ongoing deletion discussions. Similarly

"^Talk.*\/Peer_review$"

could be associated with [[Template:Peer review]], which could look like
this:

  == Are the facts in the article correct? ==

  [[Category:Fact review (yellow)]]

  == Is the article well written? ==

  [[Category:Writing review (yellow)]]

  == Is the copyright situation clear? ==

  [[Category:Copyright review (yellow)]]

  == Is the article sufficiently illustrated? ==

  [[Category:Illustrations review (yellow)]]

  == Overall result ==

  [[Category:Overall review (yellow)]]

Now, when I create [[Talk:Pompeii/Peer review]], I could add comments in
each of these sections: The facts are OK, the copyright is unclear etc.
The category tags in each section would reflect the current review level:

  yellow: status unclear
  red:    status negative
  green:  status positive

Using the category feature I would then be able to obtain lists of all
articles with factual problems, unclear copyright situation etc. Articles
in [[Category:Overall review (green)]] could be moved to
[[Wikipedia:Featured articles]].

Instead of the category system we could also use my template<->metadata
idea:

  {{
    Peer review
    Facts => yellow
    Writing => yellow
    Copyright => yellow
    Illustrations => yellow
    Overall => yellow
  }}

To avoid having to manually create /Deletion and /Peer review pages, all
we need is a little extra code that optionally adds a few links to useful
subpages to each talk page:

  Talk:Pompeii
  From Wikipedia, the free encyclopedia

  (some CSS)
  * '''[[Talk:Pompeii/Deletion|Vote for deleting this page]]'''
  * '''[[Talk:Pompeii/Peer review|Peer review process]]'''
  (end CSS)

Bam, you have
- a solution to the VfD nightmare
- a flexible, consensus-based quality control mechanism.

For the quality control part, being able to easily link to a flagged
revision of an article would be a bonus, e.g. [[Pompeii+]] links to the
version of Pompeii which has been previously marked "stable" by a sysop.

This does not address the details of the peer review process (e.g. what to
do when an article changes substantially), but I believe these can all be
worked out within the described framework (e.g. set overall review status
back to yellow).

If we want to pursue this course, I suggest getting the category system in
working order first (it seems to break every couple of days). The
template<->regex association is fairly trivial, making it easy to
internationalize without requiring changes to all 50 language files may be
more difficult. The automatic subpage links are also trivial.

Thoughts?

Regards,

Erik


from_name: Sean Barrett

subject: *****SPAM***** Re: [WikiEN-l] Re: Why am I blocked ?

body:
> I have to say that I'm not a Unix guru at all.
>
> Well, I've inspected my computer and found a typo in the IPtables that left
> the port 3128 open.
> I think it should be OK now but I'm not certain. If someone can verify and
> unblock me I'll be happy.
>
> I would suggest to improve the proxy blocker :
> - By reporting what kind of security hole was discovered,
> - By adding a button to allow the blocked user to for a recheck of his
> computer and unblock himself when the hole is fixed.
>
> Eric Demolli

I don't know if this will help or not, but opm.blitzed.org also thinks
you have an open proxy.  My SpamAssassin flagged your last message
thusly:

 pts rule name              description
---- ---------------------- --------------------------------------------------
 4.3 RCVD_IN_OPM            RBL: Received via a relay in opm.blitzed.org
                            [62.212.103.37 listed in opm.blitzed.org]
 4.3 RCVD_IN_OPM_HTTP_POST  RBL: OPM: sender is open HTTP POST proxy
                            [62.212.103.37 listed in opm.blitzed.org]

--
 Sean Barrett     | Gee, Mr. Wizard! Aren't nuclear reactors dangerous?
 sean at epoptic.com |


from_name: Jens Frank

subject: [Wikitech-l] proxy blocker mistakes

body:
On Thu, Apr 01, 2004 at 06:51:34PM +0000, Tomos at Wikipedia wrote:
> Hello.
>
> It seems that one of our trusted users was blocked by proxy blocker even
> though his is IPs are not open proxies. IPs I was informed of by the user
> were as follows:
>
> 220.146.24.126
> 220.146.22.87
> 220.146.22.10
>
> I will unblock these addresses, but is it really effective if I do that? I
> am afraid that the blocker will re-block those addresses as soon as he
> start editing. Can I do anything? Or is there anything the user should do?
> I would appreciate any suggestion.

Those are dynamic dial up IPs I guess, hostnames are like
ntkyto039126.kyto.nt.ftth.ppp.infoweb.ne.jp.

Does it make sense to block these permanently? The next person using this
IP will probably not have an open proxy.

Best regards,

JeLuF


from_name: Wesley Sheldahl

subject: [Wikipedia-l] Re: Patent idea

body:
On Friday, April 2, 2004, at 08:15  AM, Andre Engels wrote:

> "Wesley Sheldahl" <wsheldahl at iglou.com> schrieb:
>> True. However, the historical stated purpose of patents is to allow an
>> inventor to profit from his or her invention for a period of time
>> before everyone else starts profiting from the invention, and thereby
>> provide incentive for further innovation.  What other benefit would
>> there be to patenting something?  There needs to be some benefit to
>> justify the expense involved.
>
> Well, there is the technique of 'defensive patents'. The idea is to use
> the patents as a protection against paying royalties on other people's
> patents, by threatening to counter-claim when someone tries to do so.
> Problem is that to make it work, one will have to have a portfolio of
> very many patents, which is nothing that Wikipedia (or any .org, I
> think)
> can hope to get.
>
> Andre Engels
>

True, I had forgotten about those, which is why I asked. Patents don't
seem worth the bother to me for a non-profit, but if someone really
wants to go for it just for the bragging rights or whatever, more power
to them.

Wesley Sheldahl
wsheldahl at iglou.com


from_name: Fred Bauder

subject: [WikiEN-l] Arbitration Policy Ratification Vote

body:
As the discussion as wound down on its talk page I have activated this vote
and added the following to goings-on:

* '''Arbitration policy ratification vote''' Vote commences now at
[[Wikipedia:Arbitration policy ratification vote]] to continue for one week.
2/3 majority required for approval, voters must have minimum of 500 edits.

Fred


from_name: Walter Vermeir

subject: [Wikitech-l] Re: proxy blocker mistakes

references:
<BAY10-F46IskOuN9JVx00029cce@hotmail.com>
	<c4hvnm$o8n$1@sea.gmane.org> <c4i5rm$gp$5@sea.gmane.org>

body:
Timwi <timwi at ...> writes:

> > A white list function can be usefull to have. A proxy blocker override.
>
> That would mean knowingly allowing specific open proxies (which, by
> definition, anyone can then use to troll/vandalise/etc.).

Yes, off course. That is the idea of a override. It is not because a user is
using a open proxy the will vandalise anything. I would like to have the option
to allow write acces to a user who has been blocked by the proxy blocker if I
feel it is necessary.

Do you know what the effect is when the user "proxy blocker" is blocked? Will
that stop the proxy blocker? And does it harm the working the proxy blocker on
the other wikipedias?

Label;
Now the proxy blocker is using the notice;

/Your IP address has been blocked because it is an open proxy. Please contact
your Internet service provider or tech support and inform them of this serious
security problem/

For the Dutch Wikipedia I would like to see this notice used;

/Dit is een automatische preventieve blokkering omdat U gebruik maakt van een
open proxyserver. Zie [[gebruiker:Proxy blocker|Proxy blocker]] voor meer
informatie/

---
Walter Vermeir


from_name: Vicki Rosenzweig

subject: [WikiEN-l] Re: Uncle Ed reads Irismeister the riot act

body:
At 10:56 AM 4/2/04 -0500, Viajero wrote:

>Ed's behaviour simply confirms my suspicion that there is an authoritarian
>lurking in the soul of every rightwinger.

This is also unhelpful.

I strongly suspect that there is an authoritarian lurking somewhere in
*every* human soul, one of the unfortunate impulses we need to guard
against in ourselves.

Vicki Rosenzweig


from_name: Timwi

subject: [Wikitech-l] Re: proxy blocker mistakes

body:
Jens Frank wrote:

> Does it make sense to block these permanently?

At least on en, Proxy Blocker no longer blocks anyone permanently. They
are only blocked for 7 days. After that period, the IP is re-assessed
when it tries to make its next edit.

This also means that someone who has a dynamic dial-up IP and is blocked
by Proxy Blocker, can simply dial in again after securing (or disabling)
their proxy, and they will have a different IP that won't be blocked.

Personally, I think this should be reduced to 24 hours, as dynamic IPs
can be recycled pretty soon, and Wikipedia is growing enormously in
popularity.

Timwi


from_name: Andre Engels

subject: [Wikipedia-l] Re: Patent idea

body:
"Wesley Sheldahl" <wsheldahl at iglou.com> schrieb:
> True. However, the historical stated purpose of patents is to allow an
> inventor to profit from his or her invention for a period of time
> before everyone else starts profiting from the invention, and thereby
> provide incentive for further innovation.  What other benefit would
> there be to patenting something?  There needs to be some benefit to
> justify the expense involved.

Well, there is the technique of 'defensive patents'. The idea is to use
the patents as a protection against paying royalties on other people's
patents, by threatening to counter-claim when someone tries to do so.
Problem is that to make it work, one will have to have a portfolio of
very many patents, which is nothing that Wikipedia (or any .org, I think)
can hope to get.

Andre Engels


subject: [WikiEN-l] Re: Why am I blocked ?

body:
Tim Starling wrote:

> The proxy blocker works by attempting to send a proxied request for
> Special:Blockme via the target computer. Special:Blockme will block the
> address if the originating IP matches the IP in the query string.

Isn't this really quite insecure? You've already mentioned the problem
that someone can thus block someone else by tricking them into viewing a
page with an embedded image. Another insecurity is that anyone can block
a shared computer.

My suggestion would be to have the Proxy Prober (I'll call it that for
lack of an established term) send not just the to-be-blocked IP address,
but also the entire ban-reason string through the proxy. This means that
someone would have to fake the reason string, or else sysops will be
able to unblock them easily.

Another thing I would like to suggest, and I've actually suggested that
before, is that the reason string should include the port number, so
that any sysop can check if the IP is still an open proxy.

Thanks,
Timwi


subject: [Wikitech-l] Re: Yahoo! XML feed

body:
Jason Richey wrote:
>
> * Someone would look at it (I attached it) and say "this sucks
>   because..."

OK, since you asked for it... :)

>   $sql = "SELECT cur_title as title from cur where cur_namespace=0";

This query sucks big time.

Do you know what this does? This retrieves the titles of ALL ARTICLES in
Wikipedia. Do you know how many there are? ...

The Main Page states 239180, and that's only articles that meet certain
criteria...

>     $data = getPageData($s->title);

It seems that getPageData() retrieves the text of a page. In other
words, it performs yet another database query. And you're calling that
FOR EVERY ARTICLE in Wikipedia!

I'm afraid I don't understand the purpose of the script. It seems to me
that it is generating one ridiculously huge file that contains all of
Wikipedia. What use would such a file be to anyone, even Yahoo?

I stress I don't really understand the purpose of the script, nor do I
know exactly what Yahoo!'s (or anyone else's) requirements are, but it
would seem way more sensible to me to have several smaller files, each
of which containing maybe at most 100 articles or perhaps at most 1 MB
of data or something. Each file should then contain a list of cur_ids,
and then you can easily check for each file if any of the articles
therein have changed since the last update.

Of course, that's just a suggestion.

Greetings,
Timwi


subject: [Wikitech-l] Re: Title-template associations and default subpages

body:
Erik Moeller wrote a lot.

[... wow ...]

> Thoughts?

Yes, very positive ones!


subject: [Wikipedia-l] Re: foundation-l mailing list created

body:
Timwi <timwi at ...> writes:

> How long until gmane.org will mirror it?

I have send the request to gmane.org for the creation
of "gmane.org.wikimedia.foundation"

The are usually fairley quick to act.

Walter


subject: [Wikipedia-l] Re: Patent idea

body:
Wesley Sheldahl wrote:

> On Thursday, April 1, 2004, at 05:34  PM, Timwi wrote:
>
>> Hr. Daniel Mikkelsen wrote:
>>
>>> On Thu, 1 Apr 2004, Timwi wrote:
>>>
>>>> We were looking for something we came up with that we could patent,
>>>> right?
>>>
>>> Are we? I think it's an awful idea to have the Wikimedia Foundation
>>> patent
>>> something.
>>
>> Notice that patenting something doesn't mean asking for royalties.
>
> True. However, the historical stated purpose of patents is [...]
> There needs to be some benefit to justify the expense involved.

Sometimes I don't understand why the people who originally posted an
idea (Jimbo in this case:
http://mail.wikipedia.org/pipermail/wikitech-l/2004-March/009034.html)
remain entirely silent when it is brought up again.


body:
Erik Moeller wrote:

> There is now a new mailing list for all issues related to the Wikimedia
> foundation as a whole:
>
> http://mail.wikipedia.org/mailman/listinfo/foundation-l

How long until gmane.org will mirror it?


from_name: Ravindra Seetharam

subject:
[WikiEN-l] Request to post the names of all the winners of Padma
	Shri Awards.

body:
Hi,
  Can u post all the winners of the Padama awards in the hierarchy of Padma
awards.

Thanks
Ravindra


from_name: Ray Saintonge

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
Brion Vibber wrote:

> On Apr 2, 2004, at 00:53, Jimmy Wales wrote:
>
>> It shouldn't run more than once per day at first.  I'm not sure what
>> their goals are with respect to how often they would *like* to receive
>> it, but daily is a fine start.
>
> It would take hours just to run a complete dump, which would be the
> equivalent of a sizeable fraction of our total daily page views. (Best
> case might be 100ms per page for 240,000 pages =~ 6 hours 40 minutes)
>
> If we're going to run something like this daily, some sort of
> incremental updates are a must, though we can probably get away with
> stuffing the saved data per page in a database or such and slurping it
> back out fairly quickly.

I can't see where the proposal that some have made to have all the
projects on one database would help this situation.

Ec


body:
Vicki Rosenzweig wrote:
> At 10:56 AM 4/2/04 -0500, Viajero wrote:
> >Ed's behaviour simply confirms my suspicion that there is an authoritarian
> >lurking in the soul of every rightwinger.
>
> This is also unhelpful.
>
> I strongly suspect that there is an authoritarian lurking somewhere in
> *every* human soul, one of the unfortunate impulses we need to guard
> against in ourselves.

Vicki is very wise.

--Jimbo


from_name: Constans, Camille (C.C.)

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
>
>
>On Apr 2, 2004, at 00:53, Jimmy Wales wrote:
>> It shouldn't run more than once per day at first.  I'm not sure what
>> their goals are with respect to how often they would *like*
>to receive
>> it, but daily is a fine start.
>
>It would take hours just to run a complete dump, which would be the
>equivalent of a sizeable fraction of our total daily page views. (Best
>case might be 100ms per page for 240,000 pages =~ 6 hours 40 minutes)
>
>If we're going to run something like this daily, some sort of
>incremental updates are a must, though we can probably get away with
>stuffing the saved data per page in a database or such and slurping it
>back out fairly quickly.
>
>-- brion vibber (brion @ pobox.com)

Incremental need to check in the database each time if this is the last version, no ?

Why not using a slave db server on less loaded server ? (like zwinger for example (need to add another ide drive))

Shaihulud


subject: [Wikitech-l] Re: proxy blocker mistakes

references:
<BAY10-F46IskOuN9JVx00029cce@hotmail.com>	<c4hvnm$o8n$1@sea.gmane.org>
	<c4i5rm$gp$5@sea.gmane.org>
	<loom.20040402T093629-942@post.gmane.org>

body:
Walter Vermeir wrote:

> Timwi <timwi at ...> writes:
>
>>>A white list function can be usefull to have. A proxy blocker override.
>>
>>That would mean knowingly allowing specific open proxies (which, by
>>definition, anyone can then use to troll/vandalise/etc.).
>
> Yes, off course. That is the idea of a override. It is not because a user is
> using a open proxy the will vandalise anything.  I would like to have the option
> to allow write acces to a user who has been blocked by the proxy blocker if I
> feel it is necessary.

You are forgetting that an OPEN proxy means that ANYONE can use it to
troll and vandalise. People are free to use some sort of proxy to edit
Wikipedia, but it MUST be a secure proxy. If even just one open proxy is
explicitly allowed on Wikipedia, then the whole Proxy Blocker business
is entirely pointless.

Timwi


subject: [Wikitech-l] Re: proxy blocker mistakes

body:
Walter Vermeir wrote:

> Do you know what the effect is when the user "proxy blocker" is blocked? Will
> that stop the proxy blocker? And does it harm the working the proxy blocker on
> the other wikipedias?

No, of course not. The blocking of open proxies happens through an
automatic process (Special:Blockme) rather than through an action by a
user who is logged in as "Proxy Blocker". In that sense, the page
[[User:Proxy blocker]] is just for informational purposes.

Timwi


subject: [Wikitech-l] Re: Yahoo! XML feed

body:
Brion Vibber wrote:

> A text base without the unnecessary UI elements could improve search
> results, and I suppose can be kept more complete more easily than
> constant spidering of a 200k+ page site. *shrug* If that's the data
> format they want, hey fine, though having to download the entire set of
> a couple hundred megabytes for every update doesn't sound ideal.

If the problem is that each article has navigational elements on the
top, bottom, and left, which multiplies the time required to crawl
things, then how about allowing something like &raw=1 which would output
only the parsed page, without a skin or anything, with all the links in
it also pointing to &raw=1, and then arranging with Yahoo to have them
spider these pages, but still send people to the URL without &raw=1?


subject: [WikiEN-l] Re: Why am I blocked ?

body:
Jimmy Wales wrote:

> This seems easy enough to fix.  The link above should do nothing.  If
> we're testing a proxy, we should try to get the client to request
> ...?title=Special:Blockme&validation=xxxxxxxxxxxxxxxx
> where 'xxxxxxxxxxxxxx' is something that we can generate easily but
> that's difficult for User:EvilUser to duplicate.

Additionally, it should probably be a POST request, so it can't be
embedded in an IMG tag anyway.


body:
Vicki Rosenzweig wrote:

> At 10:56 AM 4/2/04 -0500, Viajero wrote:
>
>> Ed's behaviour simply confirms my suspicion that there is an
>> authoritarian
>> lurking in the soul of every rightwinger.
>
> This is also unhelpful.
>
> I strongly suspect that there is an authoritarian lurking somewhere in
> *every* human soul, one of the unfortunate impulses we need to guard
> against in ourselves.

I agree.  It often takes a great deal of intrapersonal intelligence to
overcome this.  I regret that I often see some otherwise highly
respected members of the Wikipedia community become so taken by the
correctness of their own views that they leave very little room for
alternative ones.

Yes, I do see Ed as having a strong tendency to authoritarian
opinionation, but unlike most of the crew in that boat he doesn't insist
on riding the boat to the bottom when it has been torpedoed.

Ec


from_name: Viajero

body:
On 04/01/04  at 08:25 AM, The Cunctator <cunctator at kband.com> said:

> The problem isn't that you overdid it--the problem is that you
> explicitly abused the power entrusted in you to threaten another user.

> The language you use harms not only yourself but also Wikipedia. If
> anyone needs evidence that Wikipedia is run by a cabal of arbitrary,
> puerile autocrats, you've supplied it.

Ed's behaviour simply confirms my suspicion that there is an authoritarian
lurking in the soul of every rightwinger.

V.


from_name: Carlos Baquero Moreno

subject: [Wikitech-l] Merge status on SQL old

body:
I am doing research on concurrent editing of shared objects and I am
thinking on using Wikipedia as a source for update patterns. After
downloading and importing the database to my mysql, I can now check
the info I need: Object, User and Timestamp.

However, I noticed that unlike other Wikis there is no longer a lock when
multiple users concurrently try to edit the same object. As you know, currently
in Wikipedia the user that tries to commit a concurrent version will get asked
to merge its contents with the previously commited state.

My questions is:

Is there any flag on table "old" that indicates that a given update was done
after merging with the previous version ? This kind of information
would be usefull for my extraction of patterns, since I am interested on the
conflict rate.

Thanks in advance for any help that you might be able to give me.

Regards,
Carlos

Carlos Baquero
Distributed Systems Group            Fax +351 25360 4471
University of Minho, Portugal        Voice +351 25360 4449
cbm at di.uminho.pt                     http://gsd.di.uminho.pt/cbm/


from_name: Alfio Puglisi

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
On Fri, 2 Apr 2004, Brion Vibber wrote:

>On Apr 2, 2004, at 00:35, Timwi wrote:
>>>   $sql = "SELECT cur_title as title from cur where cur_namespace=0";
>>
>> This query sucks big time.
>>
>> Do you know what this does? This retrieves the titles of ALL ARTICLES
>> in Wikipedia.
>
>That's kinda the point, yeah. It might be better to skip redirects,
>though; otherwise they should be handled in some distinct way.
>
>> It seems that getPageData() retrieves the text of a page. In other
>> words, it performs yet another database query. And you're calling that
>> FOR EVERY ARTICLE in Wikipedia!

Is it possible to leverage the already existing periodic database
dump, for example importing it into some machine not live on the web, and
generating there the necessary XML dumps and diffs? If it's only working
on the cur table, it's not even so heavy on memory usage

Alfio


from_name: Brion Vibber

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
On Apr 2, 2004, at 10:14, Ray Saintonge wrote:
> Brion Vibber wrote:
>> On Apr 2, 2004, at 00:53, Jimmy Wales wrote:
>>
>>> It shouldn't run more than once per day at first.  I'm not sure what
>>> their goals are with respect to how often they would *like* to
>>> receive
>>> it, but daily is a fine start.
>>
>> It would take hours just to run a complete dump, which would be the
>> equivalent of a sizeable fraction of our total daily page views.
>> (Best case might be 100ms per page for 240,000 pages =~ 6 hours 40
>> minutes)
>>
>> If we're going to run something like this daily, some sort of
>> incremental updates are a must, though we can probably get away with
>> stuffing the saved data per page in a database or such and slurping
>> it back out fairly quickly.
>
> I can't see where the proposal that some have made to have all the
> projects on one database would help this situation.

I can't see how it would hurt, either. In fact it doesn't seem to have
any bearing on this at all.

-- brion vibber (brion @ pobox.com)


from_name: Carl Witty

subject: [Wikitech-l] Re: proxy blocker mistakes

references:
<BAY10-F46IskOuN9JVx00029cce@hotmail.com>
	<20040402050416.GX6641@jeluf.mine.nu>  <c4j73r$138$1@sea.gmane.org>

body:
On Fri, 2004-04-02 at 00:09, Timwi wrote:
> At least on en, Proxy Blocker no longer blocks anyone permanently. They
> are only blocked for 7 days. After that period, the IP is re-assessed
> when it tries to make its next edit.
>
> This also means that someone who has a dynamic dial-up IP and is blocked
> by Proxy Blocker, can simply dial in again after securing (or disabling)
> their proxy, and they will have a different IP that won't be blocked.
>
> Personally, I think this should be reduced to 24 hours, as dynamic IPs
> can be recycled pretty soon, and Wikipedia is growing enormously in
> popularity.

I think it should be reduced to 5 minutes -- is there any reason to have
it be longer?

Carl Witty


subject: [Wikitech-l] Re: proxy blocker mistakes

body:
On Apr 2, 2004, at 11:36, Carl Witty wrote:
> On Fri, 2004-04-02 at 00:09, Timwi wrote:
>> Personally, I think this should be reduced to 24 hours, as dynamic IPs
>> can be recycled pretty soon, and Wikipedia is growing enormously in
>> popularity.
>
> I think it should be reduced to 5 minutes -- is there any reason to
> have
> it be longer?

To avoid having to recheck the same static proxies *every five minutes*.

-- brion vibber (brion @ pobox.com)


from_name: Tim Starling

subject: [WikiEN-l] Re: Why am I blocked ?

body:
Eric Demolli wrote:
> Thanks Jimmy.
> You know I'm blocked again. As of today I considered my proxy as secure.  I
> just like to know what kind of security hole was discovered. If a proxy
> blocker as been implemented I think it would be fair to say exactly why an
> address is blocked.
> Eric Demolli

Your computer has two ports open which are on the proxy checker's port
list: 80 and 3128. Both seem to be correctly configured. I manually
triggered the proxy blocker to attempt to block those two ports, and
nothing happened. You have Apache running on 80, and it didn't
understand the proxy request. You have squid on port 3128, and it gives
an access denied error.

Nonetheless, the server logs show your computer asking for itself to be
blocked, at April 1, 16:15.

62.212.103.37 - - [01/Apr/2004:16:15:35 +0000] "GET
http://en.wikipedia.org/w/wiki.phtml?title=Special:Blockme&ip=62.212.103.37
HTTP/1.0" 200 4143 "-" "-"

The proxy blocker works by attempting to send a proxied request for
Special:Blockme via the target computer. Special:Blockme will block the
address if the originating IP matches the IP in the query string.

The logs also show a matching request for the edit page, which triggered
the scan:

62.212.103.37 - - [01/Apr/2004:16:15:33 +0000] "GET
http://en.wikipedia.org/w/wiki.phtml?title=Image_talk:Hindenburg.jpg&action=edit
HTTP/1.1" 200 3550 "http://en.wikipedia.org/wiki/Image:Hindenburg.jpg"
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"

Was that you clicking on that edit link? Is MSIE 6.0 your browser?

It's possible for a malicious person to trick your browser into
requesting the Special:Blockme address, e.g. with an image with its
source set to Special:Blockme, a misleading link or a java applet. The
fact that there is also a request for an edit page makes this seem
pretty unlikely, although not impossible.

Possible explanations range from the mundane to the extraordinary. You
could have reconfigured your proxy after 16:15 and forgotten to tell us.
There may have been an elaborate script embedded in Wikipedia or another
web page you were surfing at the time. Your computer might have been
hacked.

If this happens again, can you please contact me privately, immediately
after the event? By IRC, user talk page, or email (t.starling at
ph.unimelb.edu.au).

Tomos at Wikipedia wrote:
 > Hello.
 >
 > It seems that one of our trusted users was blocked by proxy blocker even
 > though his is IPs are not open proxies. IPs I was informed of by the
 > user were as follows:
 >
 > 220.146.24.126
 > 220.146.22.87
 > 220.146.22.10
 >
 > I will unblock these addresses, but is it really effective if I do that?
 > I am afraid that the blocker will re-block those addresses as soon as he
 > start editing. Can I do anything? Or is there anything the user should
 > do? I would appreciate any suggestion.

This user appears to be on a dynamic IP address, so it's a bit hard for
me to scan it and check for security. Can you have this person contact
me when s/he is online? Perhaps by IRC? I found one relevant log entry:

220.146.22.87 - - [01/Apr/2004:00:56:55 +0000] "GET
http://meta.wikipedia.org/w/wiki.phtml?title=Special:Blockme&ip=220.146.22.87
HTTP/1.0" 200 4017 "-" "-"

And a matching edit request:

220.146.22.87 - - [01/Apr/2004:00:56:55 +0000] "GET
http://meta.wikipedia.org/w/wiki.phtml?title=MediaWiki_feature_request_and_bug_report_discussion&action=edit
HTTP/1.1" 200 89899 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
5.1; .NET CLR 1.1.4322)"

This user may have an open proxy on his/her computer without knowing it.

The thing about the proxy blocker is that it's not particularly prone to
false positives. If you get blocked, it means either you have an open
proxy, or something fishy is going on. If people are being blocked by a
malicious user, we will need to enhance the security in Special:Blockme,
adding some sort of authentication to ensure the requests are genuine.

-- Tim Starling


from_name: Jimmy Wales

subject: [Wikitech-l] geoffrin coming up

body:
geoffrin will be coming up very soon on *.237.

226-234 are the 'real' numbers for suda,bart-zwinger
235 and 236 are being used as aliases, right?

237 looks empty to me.


from_name: Pablo Saratxaga

subject: [Wikitech-l] Re: improving search engine hits

body:
Kaixo!
Li Thu, 18 Mar 2004 18:13:01 +0100,
 Ashar Voultoiz <thoane at altern.org> scrijheut:

 AV> Although the english wikipedia comes to top when you search google for
 AV> "free encyclopedia" [1], the french wikipedia doesn't appear in the top 10
 AV> list when searching for "encyclop?die gratuite" cause other pages got this

Maybe because wikip?dia is *NOT* "gratuite" but "libre"...

 AV> http://fr.wikipedia.org/
 AV> L'encyclop?die gratuite et libre.

Non, wikip?dia c'est "l'encyclop?die libre", c'est tout.
Que l'un puisse permettre l'autre n'est pas suffisant pour mettre la
libert? et l'avarice sur le m?me pied.

--
? bient?t,
Pablo Saratxaga           PGP Key available, key ID: 0xD9B85466

http://chanae.walon.org/


from_name: Eric Demolli

subject: [WikiEN-l] Re: Why am I blocked ?

body:
I have to say that I'm not a Unix guru at all.

Well, I've inspected my computer and found a typo in the IPtables that left
the port 3128 open.
I think it should be OK now but I'm not certain. If someone can verify and
unblock me I'll be happy.

I would suggest to improve the proxy blocker :
- By reporting what kind of security hole was discovered,
- By adding a button to allow the blocked user to for a recheck of his
computer and unblock himself when the hole is fixed.

Eric Demolli

----- Original Message -----
From: "Timwi" <timwi at gmx.net>
To: <wikien-l at Wikipedia.org>
Sent: Friday, April 02, 2004 11:41 AM
Subject: [WikiEN-l] Re: Why am I blocked ?

> Jimmy Wales wrote:
>
> > This seems easy enough to fix.  The link above should do nothing.  If
> > we're testing a proxy, we should try to get the client to request
> > ...?title=Special:Blockme&validation=xxxxxxxxxxxxxxxx
> > where 'xxxxxxxxxxxxxx' is something that we can generate easily but
> > that's difficult for User:EvilUser to duplicate.
>
> Additionally, it should probably be a POST request, so it can't be
> embedded in an IMG tag anyway.
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l


subject: [Wikitech-l] geoffrin coming up

body:
On Apr 2, 2004, at 12:21, Jimmy Wales wrote:
> geoffrin will be coming up very soon on *.237.
>
> 226-234 are the 'real' numbers for suda,bart-zwinger
> 235 and 236 are being used as aliases, right?

Right.

> 237 looks empty to me.

As far as I know.

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] Categories

body:
On Apr 2, 2004, at 13:15, Magnus Manske wrote:
> I just tried the category feaature, and found it broken - again. This
> must be*at least* the third time I left intact code and found it
> broken. Please, be a little more careful next time, OK?

As far as I know it *never* worked completely. Could you please test it
thoroughly and document what it's supposed to do so that people can
avoid breaking it?

> Also, I found that "brokenlink" seems to amass multiple identical
> links each time one saves a page with broken links. I have DISTINCT
> the queries for now, but we'll have to do someting about that.

Would you mind tracking that down?

-- brion vibber (brion @ pobox.com)


from_name: Gabriel Wicke

subject: [Wikitech-l] Re: Re: Yahoo! XML feed

body:
On Fri, 02 Apr 2004 01:55:31 -0800, Brion Vibber wrote:

> Then they'll still have to make 240,000+ HTTP connections to check
> every individual page for updates, which can take days or weeks
> depending on the crawl delay.

What about adding a squid running at yahoo to our cache purge list? They
could constantly crawl that one, and only purged pages would be fetched
(from the wp caches configured as parent to the yahoo box, no extra db
access).

That's zero extra db usage and no scripting on our part required. And the
wp html should be pretty easy to parse for them by cutting out the content
of <div id='content'>.
--
Gabriel Wicke


subject: [Wikitech-l] Re: proxy blocker mistakes

body:
"Timwi" <timwi at gmx.net> schrieb:
> Jens Frank wrote:
>
> > Does it make sense to block these permanently?
>
> At least on en, Proxy Blocker no longer blocks anyone permanently. They
> are only blocked for 7 days. After that period, the IP is re-assessed
> when it tries to make its next edit.
>
> This also means that someone who has a dynamic dial-up IP and is blocked
> by Proxy Blocker, can simply dial in again after securing (or disabling)
> their proxy, and they will have a different IP that won't be blocked.
>
> Personally, I think this should be reduced to 24 hours, as dynamic IPs
> can be recycled pretty soon, and Wikipedia is growing enormously in
> popularity.

I think it would be good to automatically unblock all those 'idefinite'
blocks by proxy blocker. I checked, and on de: there are now over 100
indefinitely blocked IPs blocked by Proxy blocker (plus 66 one week blocks
by Proxy blocker plus 1 one-day block by a normal sysop, which might
well be considered out of balance, by the way)

Andre Engels


from_name: John Pozniak

subject: [WikiEN-l] PC Magazine article

body:
I just read the April 20 edition of PC Magazine, and
they've got an article entitled "2004 100 Top Websites

You Didn't Know You Couldn't Live Without"
(http://www.pcmag.com/article2/0,1759,1558375,00.asp).

Listed under "Information" is a nice little entry on
Wikipedia
(http://www.pcmag.com/article2/0,1759,1558744,00.asp)

Just felt like sharing.

John Pozniak (Gentgeen)


subject: [Wikitech-l] More proxy blocker problems

body:
I noticed another problem with Proxy blocker: It seems that every time
it blocks an IP, a new userid is created - just check
http://de.wikipedia.org/w/wiki.phtml?title=Spezial:Listusers&limit=250&offset=15360.
Apart from that it would be good to have proxy blocker unblock all addresses
it has blocked 'indefinitely'. Those are now still in existence on many
Wikipedias. Proxy blocker is doing the great majority of blockings at the
moment, by the way. Again looking at de:, I find:

* over 100 indefinite blocks by Proxy blocker
* 66 one-week blocks by Proxy blocker
* 1 normal sysop block (for 24 hours).

Andre Engels


from_name: Magnus Manske

body:
Erik Moeller wrote:

>If we want to pursue this course, I suggest getting the category system in
>working order first (it seems to break every couple of days).
>
Could someone put the latest CVS on the test site and enable categories?
My ssh access was lost somewhere during the Server Swap Orgy (TM) some
time ago, and hasn't been restored :-(

Last time I checked on my Linux box, categories were basically fine.

Magnus


from_name: A

from_email: A

subject: [WikiEN-l] how to keep me from edit warring as much

body:
One really good way to keep me from so much reverting
(note, my reverts are always reverts of reverts -- I
don't delete other people's test (I NPOV it instead);
however, when people delete my text I do tend to
revert their revert)

But, a good way to slow my reverting down, is to give
me a second watchlist.

I use one watchlist so I can keep an idea of whats
going on with articles I have edited (so that I can
revert people who revert me); if I had a second
watchlist, then I could deliberately watch articles at
which someone was reverting me -- this would allow me
to not have to worry about forgetting that so-and-so
deleted my text.

If I didn't have to worry about the deleting being
forgotten, then I might be more willing to wait a day
or two and see if the reverter gets around to
explaining themselves.

I also think it should become a rule that whenever
someone deletes a chunk of text, that they include
some sort of notification in the edit summary.

__________________________________
Do you Yahoo!?
Yahoo! Small Business $15K Web Design Giveaway
http://promotions.yahoo.com/design_giveaway/


from_name: Matt M.

subject: [WikiEN-l] authoritarian

body:
> > I strongly suspect that there is an authoritarian lurking somewhere in
> > *every* human soul, one of the unfortunate impulses we need to guard
> > against in ourselves.
>
> Vicki is very wise.

Yup. I'm sure we all know the feeling that Gore Vidal summed up: "I am, at
heart, a tiresome nag complacently positive that there is no human problem
that could not be solved if people would simply do as I advise."

Matt (Montrealais)


subject: [Wikitech-l] Re: Re: Yahoo! XML feed

body:
On Fri, 02 Apr 2004 14:29:19 +0200, Gabriel Wicke wrote:

Sorry for responding to myself, but..

What about just recording the updated (purged) url's in a text file? Could
be on the wp servers, could be done at yahoo if they want real-time. No
need for a squid on their part in that case, a simple http daemon logging
the requests would be enough to build the list at yahoo. Or a five-line or
so function to appends the url to an hourly-rotated log file at
wikipedia that can be fetched from yahoo or others.
--
Gabriel Wicke


subject: [Wikitech-l] Re: proxy blocker mistakes

body:
On Fri, 2004-04-02 at 11:41, Brion Vibber wrote:
> On Apr 2, 2004, at 11:36, Carl Witty wrote:
> > On Fri, 2004-04-02 at 00:09, Timwi wrote:
> >> Personally, I think this should be reduced to 24 hours, as dynamic IPs
> >> can be recycled pretty soon, and Wikipedia is growing enormously in
> >> popularity.
> >
> > I think it should be reduced to 5 minutes -- is there any reason to
> > have
> > it be longer?
>
> To avoid having to recheck the same static proxies *every five minutes*.
>
> -- brion vibber (brion @ pobox.com)

How about this: if we detect an open proxy, keep that information for 5
minutes; if we don't, keep that for 24 hours.

It just seems really bad to have an auto-block potentially affect a
different user with the same dynamic IP address.

Carl Witty


from_name: Andre Engels

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
"Brion Vibber" <brion at pobox.com> schrieb:
> On Apr 2, 2004, at 00:53, Jimmy Wales wrote:
> > It shouldn't run more than once per day at first.  I'm not sure what
> > their goals are with respect to how often they would *like* to receive
> > it, but daily is a fine start.
>
> It would take hours just to run a complete dump, which would be the
> equivalent of a sizeable fraction of our total daily page views. (Best
> case might be 100ms per page for 240,000 pages =~ 6 hours 40 minutes)
>
> If we're going to run something like this daily, some sort of
> incremental updates are a must, though we can probably get away with
> stuffing the saved data per page in a database or such and slurping it
> back out fairly quickly.

What about having a table daily of all pages that are changed, removed
or new? In that case, we could read that table when the new version is
made, and only those pages need to be in the XML dump (for slower search
engines, we should then keep them for a while so they can get a number of
days if they download them not every day). A search engine would then have
to do a complete spidering (either by itself or through XML feed) once,
but after that the XML feed can be much smaller.

As another issue, what do we do with the international aspect? My proposal
would be to have XML feeds for the larger Wikipedias, and a single one for
the whole of the smaller ones; the cut-off being determined by the size
of the files in the feed.

Andre Engels


subject: [Wikipedia-l] Re: Re: stats - referrer spam

body:
On Thu, 01 Apr 2004 15:15:10 +0200, Peter Gervai wrote:

> On Thu, Apr 01, 2004 at 11:45:50AM +0000, Walter Vermeir wrote:
>> Peter Gervai <grin at ...> writes:
>> > Hi,
>> >
>> > Could yo filter out these referrer spammers from the stats?
>> >
>> > *lmltechnologies.com
>> > *cyberxscape.com

Done.

HideReferrer lmltechnologies.com/
HideReferrer cyberxscape.com/
HideReferrer partypeeps2000.com/

--
Gabriel Wicke


body:
This has nothing to do with the proxy this has to do with the smtp port.
OK I will block smtp.

Eric Demolli.

----- Original Message -----
From: "Sean Barrett" <sean at epoptic.org>
To: <demolli at unice.fr>
Cc: <wikien-l at Wikipedia.org>
Sent: Friday, April 02, 2004 4:19 PM
Subject: Re: *****SPAM***** Re: [WikiEN-l] Re: Why am I blocked ?

> > I have to say that I'm not a Unix guru at all.
> >
> > Well, I've inspected my computer and found a typo in the IPtables that
left
> > the port 3128 open.
> > I think it should be OK now but I'm not certain. If someone can verify
and
> > unblock me I'll be happy.
> >
> > I would suggest to improve the proxy blocker :
> > - By reporting what kind of security hole was discovered,
> > - By adding a button to allow the blocked user to for a recheck of his
> > computer and unblock himself when the hole is fixed.
> >
> > Eric Demolli
>
> I don't know if this will help or not, but opm.blitzed.org also thinks
> you have an open proxy.  My SpamAssassin flagged your last message
> thusly:
>
>  pts rule name              description
> ---- ---------------------- ----------------------------------------------
----
>  4.3 RCVD_IN_OPM            RBL: Received via a relay in opm.blitzed.org
>                             [62.212.103.37 listed in opm.blitzed.org]
>  4.3 RCVD_IN_OPM_HTTP_POST  RBL: OPM: sender is open HTTP POST proxy
>                             [62.212.103.37 listed in opm.blitzed.org]
>
> --
>  Sean Barrett     | Gee, Mr. Wizard! Aren't nuclear reactors dangerous?
>  sean at epoptic.com |
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l


body:
Ooops I'm wrong !

----- Original Message -----
From: "Sean Barrett" <sean at epoptic.org>
To: <demolli at unice.fr>
Cc: <wikien-l at Wikipedia.org>
Sent: Friday, April 02, 2004 4:19 PM
Subject: Re: *****SPAM***** Re: [WikiEN-l] Re: Why am I blocked ?

> > I have to say that I'm not a Unix guru at all.
> >
> > Well, I've inspected my computer and found a typo in the IPtables that
left
> > the port 3128 open.
> > I think it should be OK now but I'm not certain. If someone can verify
and
> > unblock me I'll be happy.
> >
> > I would suggest to improve the proxy blocker :
> > - By reporting what kind of security hole was discovered,
> > - By adding a button to allow the blocked user to for a recheck of his
> > computer and unblock himself when the hole is fixed.
> >
> > Eric Demolli
>
> I don't know if this will help or not, but opm.blitzed.org also thinks
> you have an open proxy.  My SpamAssassin flagged your last message
> thusly:
>
>  pts rule name              description
> ---- ---------------------- ----------------------------------------------
----
>  4.3 RCVD_IN_OPM            RBL: Received via a relay in opm.blitzed.org
>                             [62.212.103.37 listed in opm.blitzed.org]
>  4.3 RCVD_IN_OPM_HTTP_POST  RBL: OPM: sender is open HTTP POST proxy
>                             [62.212.103.37 listed in opm.blitzed.org]
>
> --
>  Sean Barrett     | Gee, Mr. Wizard! Aren't nuclear reactors dangerous?
>  sean at epoptic.com |
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l


subject: [Wikitech-l] Re: Re: Yahoo! XML feed

body:
On Fri, 02 Apr 2004 15:18:00 +0000, Andre Engels wrote:

> What about having a table daily of all pages that are changed, removed
> or new?

A list of urls that are changed is easy to generate and could even be
distributed in realtime by using the purge messages we send out anyway.
All it needs is another ip added in the squid array.

> As another issue, what do we do with the international aspect? My proposal
> would be to have XML feeds for the larger Wikipedias, and a single one for
> the whole of the smaller ones; the cut-off being determined by the size
> of the files in the feed.

The purges are for all languages and can be filtered by language (i wrote
a small python script that does this already daily for the stats).

The main tasks i see for an xml feed are
* improving the parser to produce validating xhtml and

* either writing a small wrapper that includes the same rendered content
area as used by regular page views (-> esi fragment in squid3, on the
todo, relies on the parser being fixed)
* or fetches the (often cached) (x)html from the squid, runs it through
tidy --asxml, wraps it in a small xml file and returns the result. If this
script was accessed through squid as well subsequent requests from other
search engines would use the cached version until the page changes again.
An additional url would need to be added to the purge call in Article.php
to purge the feed version.

This would ensure that there are no additional db requests and no
additional content rendering involved, and if the feed was mainly fetched
at night it would also pre-fill the squids with up-to-date anon pages.
--
Gabriel Wicke


from_name: Michael Snow

subject: [Wikipedia-l] Re: Patent idea

body:
Ray Saintonge wrote:

> Wesley Sheldahl wrote:
>
>> On Thursday, April 1, 2004, at 05:34  PM, Timwi wrote:
>>
>>> Hr. Daniel Mikkelsen wrote:
>>>
>>>> On Thu, 1 Apr 2004, Timwi wrote:
>>>>
>>>>> We were looking for something we came up with that we could patent,
>>>>> right?
>>>>
>>>> Are we? I think it's an awful idea to have the Wikimedia Foundation
>>>> patent
>>>> something.
>>>
>>> Notice that patenting something doesn't mean asking for royalties.
>>
>> True. However, the historical stated purpose of patents is to allow
>> an inventor to profit from his or her invention for a period of time
>> before everyone else starts profiting from the invention, and thereby
>> provide incentive for further innovation.  What other benefit would
>> there be to patenting something?  There needs to be some benefit to
>> justify the expense involved.
>
> For some the bragging rights are the only benefit that they need. :-)

And then, there are even people out there who would consider patenting
something, just to deprive the rest of the world the use of it.
Remember, a patent is the right to prevent use by others; the
patentholder is not obligated to make the patented thing available to
the public.

--Michael Snow


from_name: Walter Vermeir

subject: [Wikipedia-l] Re: Patent idea

body:
Imran Ghory <imran at ...> writes:

> Actually given that Ward Cunningham now works for Microsoft and Microsoft
> is developing their own wiki software, it might be a good idea to get a
> patent on some of MediaWiki's unique features.
>
> Imran

... and maybe recieve a donation form Microsoft to WikiMedia for use of those
unique features.

Walter


from_name: Theresa Robinson

subject: [WikiEN-l] From Tannin, blocked a signed-in user

body:
Tannin says he has difficulty posting to the mailing list and I offered to
do so on his behalf.

He blocked a signed-in-user, Lord Kenneth, on his own initiative.  There
is currently a quickpoll in order to validate the block.  Lord Kenneth is
alleged to have vandalized of a number of pages, including user pages,
cleanup, and pages in the main namespace.

moink


from_name: Imran Ghory

subject: [Wikipedia-l] Re: Patent idea

body:
On Fri, 2 Apr 2004, Andre Engels wrote:
> Well, there is the technique of 'defensive patents'. The idea is to use
> the patents as a protection against paying royalties on other people's
> patents, by threatening to counter-claim when someone tries to do so.

Actually given that Ward Cunningham now works for Microsoft and Microsoft
is developing their own wiki software, it might be a good idea to get a
patent on some of MediaWiki's unique features.

Imran
--
http://bits.bris.ac.uk/imran


subject: [Wikitech-l] Categories

body:
I just tried the category feaature, and found it broken - again. This
must be*at least* the third time I left intact code and found it broken.
Please, be a little more careful next time, OK?

I fixed it and put it back in the CVS - working, but probably ugly.

Also, I found that "brokenlink" seems to amass multiple identical links
each time one saves a page with broken links. I have DISTINCT the
queries for now, but we'll have to do someting about that.

Magnus


subject: [Wikitech-l] logo update

body:
There are a lot of Wikipedias who still use the old English logo even when there
is a nice translated new logo available.

Can somebody change them?

Walter

List;
http://meta.wikipedia.org/wiki/Wikipedia_subtitle_translations

http://af.wikipedia.org/
http://meta.wikipedia.org/upload/a/aa/Nohat-logo-X-af.png

http://als.wikipedia.org/
http://meta.wikipedia.org/upload/8/8f/Nohat-logo-X-als.png

http://co.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/a/ac/Nohat-logo-X-co.png

http://cs.wikipedia.org  (is using the English new-style logo)
http://meta.wikipedia.org/upload/d/d3/Nohat-logo-X-cs.png

http://el.wikipedia.org/
http://meta.wikipedia.org/upload/e/e0/Nohat-logo-X-el.png

http://et.wikipedia.org/   (is using the English new-style logo)
http://meta.wikipedia.org/upload/6/63/Nohat-logo-X-et.png

http://fa.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/5/5c/Nohat-logo-X-fa.png

http://fy.wikipedia.org/
http://meta.wikipedia.org/upload/d/d5/Nohat-logo-X-fy.png

http://ga.wikipedia.org/
http://meta.wikipedia.org/upload/3/37/Nohat-logo-X-ga.png

http://gl.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/b/bb/Nohat-logo-X-gl.png

http://is.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/b/b4/Nohat-logo-X-is.png

http://it.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/b/b3/Nohat-logo-X-it.png

http://ko.wikipedia.org/
http://meta.wikipedia.org/upload/5/57/Nohat-logo-X-ko.png

http://lt.wikipedia.org/
http://meta.wikipedia.org/upload/5/57/Nohat-logo-X-lt.png

http://lv.wikipedia.org/  Old style English logo
http://meta.wikipedia.org/upload/4/41/Nohat-logo-X-lv.png
! this logo is missing the "the free encyclopedia" label

http://mk.wikipedia.org/ (is using the English new-style logo)
http://meta.wikipedia.org/upload/f/ff/Nohat-logo-X-mk.png
! this logo is missing the "the free encyclopedia" label

http://ml.wikipedia.org/
http://meta.wikipedia.org/upload/f/f0/Nohat-logo-X-ml.png

http://nds.wikipedia.org/ (is using the English new-style logo)
http://meta.wikipedia.org/upload/6/62/Nohat-logo-X-nds.png

http://oc.wikipedia.org/ !is using a localized new-style logo but whit a
non-standard font
http://meta.wikipedia.org/upload/c/cb/Nohat-logo-X-oc.png

http://om.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/0/08/Nohat-logo-X-om.png

http://pt.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/2/2e/Nohat-logo-X-pt.png

http://ru.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/f/f9/Nohat-logo-X-ru.png

http://sk.wikipedia.org/
http://meta.wikipedia.org/upload/6/6e/Nohat-logo-X-sk.png

http://sl.wikipedia.org/  !is using a localized new-style logo but whit a
non-standard font
http://meta.wikipedia.org/upload/6/65/Nohat-logo-X-sl.png

http://sr.wikipedia.org/
http://meta.wikipedia.org/upload/e/e3/Nohat-logo-X-sr.png

http://th.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/3/32/Nohat-logo-X-th.png

http://tl.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/9/9e/Nohat-logo-X-tl.png

http://tr.wikipedia.org/
http://meta.wikipedia.org/upload/c/c2/Nohat-logo-X-tr.png

http://tt.wikipedia.org/  (is using the English new-style logo)
http://meta.wikipedia.org/upload/d/da/Nohat-logo-X-tt.png

http://vo.wikipedia.org/   (is using the English new-style logo)
http://meta.wikipedia.org/upload/6/60/Nohat-logo-X-vo.png

http://wikipedia.walon.org/ !not hosted by Wikimedia


from_name: David Gerard

subject:
[WikiEN-l] IPA pronounciation Unicode symbols,
	old browsers and Wikipedia

body:
Please read [[Fucking, Austria]] and [[Talk:Fucking, Austria]].
There's considerable discussion concerning IPA pronounciation
symbols being unreadable in a stock install of Internet Explorer.

Is there any policy or relevant past discussion on the matter?
We make many concessions to aging browsers on Wikipedia, but
these are perfectly legitimate Unicode characters we're talking
about here ...

- d.


from_name: Ray Kiddy

subject: [Wikitech-l] Re: Yahoo! XML feed

body:
MySQL has a facility for distributing databases. It sounds as though
you all are trying to solve a problem at the application level, when it
is a data distribution, caching and integrity problem. And this is
exactly the kind of problem that databases can solve.

If the wiki database is synchronized out, clients of that
synchronization can do stuff for Yahoo or do whatever they want.

MySQL handles all this "changed since" checking and such. It also does
it more efficiently than a PHP script. So, what problem are you all
trying to solve?

- ray

On Apr 2, 2004, at 1:19 AM, Alfio Puglisi wrote:

> On Fri, 2 Apr 2004, Brion Vibber wrote:
>
>> On Apr 2, 2004, at 00:35, Timwi wrote:
>>>>   $sql = "SELECT cur_title as title from cur where cur_namespace=0";
>>>
>>> This query sucks big time.
>>>
>>> Do you know what this does? This retrieves the titles of ALL ARTICLES
>>> in Wikipedia.
>>
>> That's kinda the point, yeah. It might be better to skip redirects,
>> though; otherwise they should be handled in some distinct way.
>>
>>> It seems that getPageData() retrieves the text of a page. In other
>>> words, it performs yet another database query. And you're calling
>>> that
>>> FOR EVERY ARTICLE in Wikipedia!
>
> Is it possible to leverage the already existing periodic database
> dump, for example importing it into some machine not live on the web,
> and
> generating there the necessary XML dumps and diffs? If it's only
> working
> on the cur table, it's not even so heavy on memory usage
>
> Alfio
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikitech-l
>


from_name: Nikola Smolenski

subject: [Wikitech-l] logo update

body:
On Saturday 03 April 2004 01:46, Walter Vermeir wrote:
> http://sr.wikipedia.org/
> http://meta.wikipedia.org/upload/e/e3/Nohat-logo-X-sr.png

The Serbian logo at least is wrong.


from_name: fabiform

subject:
[Wikitech-l] Re: [WikiEN-l] Re: IPA pronounciation Unicode symbols,
	old browsers and Wikipedia

body:
I'm replying as requested to wikitech-l, but I haven't received the original
message through this list yet, so you might need to resend it.

--- David Friedland <david at nohat.net> wrote:

> My recommended solution is:
> * Implement some kind of wiki markup for indicating IPA text in the wiki
> source, ideally as part of the WikiTeX system.
> * Mark up all IPA symbols on Wikipedia using the markup!
> * Logged-in users can set their preference to Unicode IPA, WikiTeX TIPA
> images, and/or (X-)SAMPA.
> * This system would put <span style="font-family:..."> tags around
> Unicode IPA in the HTML output so Windows IE users who have set their
> preference to Unicode will be able to see the symbols regardless of
> their browser's font setting.

This sounds like a nice solution (as long as their are user preferences as you
suggest).

> * What do do for anonymous users: should we do a browser detect and
> serve a page with unicode IPA or WikiTeX-ed image of the IPA, depending
> on the browser, or just send WikiTeX images to all anonymous users? It
> is hard to guarantee via a browser detect that the user has the proper
> unicode fonts, although certainly for all Safari users, the Unicode will
> render correctly, because Mac OS X has fonts with unicode characters
> installed by default. We need to do a browser/platform survey to
> determine what combinations are likely to support Unicode IPA.

Well, the simplest would be to deliver X-SAMPA by default as this would require
no guessing/scanning on our parts.

> * Inline TeX renderings won't be in the same font and may not be
> properly aligned with the surrounding text, and so may not be attractive.

This is particularly relevant to wikipedia (rather than, say, wiktionary) as
most pronunciation guides will appear not only inline, but in the very first
sentence.  That's not the place we want to be messing up the formatting/line
height.  Also, TeX outputs as a png, right?  We should consider those who surf
with images switched off.  Since linguists have gone to the bother of
translating the IPA into ascii X-SAMPA, we should consider this as a default
for anons as it has maximum cross-browser compatability.

I'm comfortable with using the IPA, but many people do find all the symbols
off-putting.  In fact, very few dictionaries use it, presumably for that
reason.  Perhaps SAMPA is a bit friendlier?

If we're going to be encouraging the use of pronunciation guides, perhaps it
would be best to create seperate guides to the IPA, SAMPA etc, which are
designed to be linked from articles and help the reader decode the guide as
quickly as possible.  We might also need to do the same for editors, to help
them write in SAMPA/the IPA in order to produce these guides in the first
place.  And I've been wondering if the string of symbols itself should be the
link: (/[[SAMPA chart for English|"hE.l at U w3:ld]]/) for the sake of tidiness?

Fabi.

__________________________________
Do you Yahoo!?
Yahoo! Small Business $15K Web Design Giveaway
http://promotions.yahoo.com/design_giveaway/


from_name: Geoff Burling

body:
On Fri, 2 Apr 2004, Vicki Rosenzweig wrote:

> At 10:56 AM 4/2/04 -0500, Viajero wrote:
>
> >Ed's behaviour simply confirms my suspicion that there is an authoritarian
> >lurking in the soul of every rightwinger.
>
> This is also unhelpful.
>
> I strongly suspect that there is an authoritarian lurking somewhere in
> *every* human soul, one of the unfortunate impulses we need to guard
> against in ourselves.
>
[Response delayed from yesterday because it was too nice outside to sit
in front of my computer any longer.]

Somehow, I can't help but substitute for " an authoritarian" the words
"a four year old about to throw a tantrum" in the above sentence.

Maybe this is due to  some fragment of Transactional Analysis stuck in
my memory somewhere. Maybe it's because when I get frustrated, I want to
throw a tantrum. Or maybe because I can't help but remember that one of
the items the dictator of the moment, Saddam Hussein, had in his last
hideout was a supply of sugary American junkfood.

And I expect next week someone will furnish definitive proof that Adolf
Hitler was fond of Southern Fried Chicken.[*]

Geoff

[*] Yes, I'm aware of a few things regarding Mr Hitler.


from_name: Daniel P.B.Smith

subject: [WikiEN-l] Re: WikiEN-l Digest, Vol 9, Issue 2

body:
> From: Vicki Rosenzweig <vr at redbird.org>
> At 10:56 AM 4/2/04 -0500, Viajero wrote:
>
>> Ed's behaviour simply confirms my suspicion that there is an
>> authoritarian
>> lurking in the soul of every rightwinger.
>
> This is also unhelpful.
>
> I strongly suspect that there is an authoritarian lurking somewhere in
> *every* human soul, one of the unfortunate impulses we need to guard
> against in ourselves.
>
> Vicki Rosenzweig

My thoughts exactly.

"There is so much good in the worst of us
And so much bad in the best of us
That it ill-behooves any of us
To talk about the rest of us." -- Anon.

--
Daniel P. B. Smith, dpbsmith at verizon.net alternate:
dpbsmith at alum.mit.edu
"Elinor Goulding Smith's Great Big Messy Book" is now back in print!
Sample chapter at http://world.std.com/~dpbsmith/messy.html
Buy it at http://www.amazon.com/exec/obidos/ASIN/1403314063/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 955 bytes
Desc: not available
Url : http://lists.wikimedia.org/pipermail/wikien-l/attachments/20040403/ab8df077/attachment.bin


subject: [Wikitech-l] Re: logo update

body:
Nikola Smolenski wrote:
> On Saturday 03 April 2004 01:46, Walter Vermeir wrote:
>
>>http://sr.wikipedia.org/
>>http://meta.wikipedia.org/upload/e/e3/Nohat-logo-X-sr.png
>
>
> The Serbian logo at least is wrong.

Then it must be fixt. To be fixt the error must be reported.

http://meta.wikipedia.org/wiki/Talk:Wikipedia_in_other_languages

Walter


from_name: David Friedland

subject:
[WikiEN-l] Re: IPA pronounciation Unicode symbols,
	old browsers and Wikipedia

body:
(I'm cross-posting this to wikitech-l, please reply there)

David Gerard wrote:
> Please read [[Fucking, Austria]] and [[Talk:Fucking, Austria]].
> There's considerable discussion concerning IPA pronounciation
> symbols being unreadable in a stock install of Internet Explorer.
>
> Is there any policy or relevant past discussion on the matter?
> We make many concessions to aging browsers on Wikipedia, but
> these are perfectly legitimate Unicode characters we're talking
> about here ...

I've brought this up before (at least a couple times), and this is the
situation as I see it:

* Windows users that have the font "Arial Unicode" installed can select
that as their default font in IE to see the IPA symbols. I've tested
this and it works. This also works for other fonts that have IPA symbols
like Gentium and Code2000, although those fonts have to be downloaded
and installed by the user. On pages that include IPA symbols, we might
suggest that the users select "Arial Unicode" as their default font, or
link to a page that suggests this.
* If IPA symbols were surrounded by  tags, more people would be able
to read them, as it will work on any recent Windows IE if they have one
of those fonts installed, regardless of what their browser's default
font setting is. However, despite my previous requests, the current
MediaWiki software does not allow  tags. It is possible to use
 tags to achieve the same results, but  tags have been
deprecated from the HTML standard for nearly 8 years, and so should be
avoided.
* The LaTeX package TIPA supports IPA and could render the symbols much
like the current <math> tags do, if such support were installed. Such
renderings would be visible in any browser.
* The WikiTeX system supports TIPA
[http://wikisophia.org/wiki/Wikitex#Tipa], but isn't yet completed or
supported on Wikipedia, although there is hope to do so.

My recommended solution is:
* Implement some kind of wiki markup for indicating IPA text in the wiki
source, ideally as part of the WikiTeX system.
* Mark up all IPA symbols on Wikipedia using the markup!
* Logged-in users can set their preference to Unicode IPA, WikiTeX TIPA
images, and/or (X-)SAMPA.
* This system would put  tags around
Unicode IPA in the HTML output so Windows IE users who have set their
preference to Unicode will be able to see the symbols regardless of
their browser's font setting.

The following items also have to be considered
* What do do for anonymous users: should we do a browser detect and
serve a page with unicode IPA or WikiTeX-ed image of the IPA, depending
on the browser, or just send WikiTeX images to all anonymous users? It
is hard to guarantee via a browser detect that the user has the proper
unicode fonts, although certainly for all Safari users, the Unicode will
render correctly, because Mac OS X has fonts with unicode characters
installed by default. We need to do a browser/platform survey to
determine what combinations are likely to support Unicode IPA.
* What format to use for the input?
**Unicode is a standard method of representing IPA symbols, but can be
tricky to edit, and UTF-8 isn't supported on the English Wikipedia so it
would have to be entered as entities.
**The TIPA input method isn't standard, and for some rare characters is
(IMHO unnecessarily) verbose, but it supports a wider range of
characters than Unicode, and is ASCII compatible.
**SAMPA or X-SAMPA is more of a standard than TIPA, and is also ASCII
compatible, so it might be the best solution.
**The International Phonetic Association has specified a set of 3-digit
numbers for every symbol in the International Phonetic Alphabet, which
is specified in Appendix 2 of the Handbook of the International Phonetic
Association, so this is a standard endorsed by the creators of the IPA,
but it's not particularly accessible, as I haven't seen a listing of IPA
numbers on the web anywhere.

One might be tempted to just use TIPA TeX rendering for IPA symbols,
thereby reducing the complexity and the amount of development required,
but there are several disadvantages to this method:
* TeX renderings don't look good when printed, as they are rendered at
72 dpi.
* TeX renderings can't be enlarged by changing the font size, something
people with poor vision or low quality displays often do, and would be
especially helpful for unfamiliar symbols like IPA symbols.
* Inline TeX renderings won't be in the same font and may not be
properly aligned with the surrounding text, and so may not be attractive.

Therefore, where it is possible to use Unicode IPA, we should.

Whatever method is chosen, at least one translator will have to be
created to convert from one input method to another. I've already
created such a system using lex to convert SAMPA input to HTML entities
of Unicode symbols, so I can attest that it's not particularly
difficult, although it's a bit tedious.

Last, although pronunciations are somewhat rare on Wikipedia, there are
some pages that rely on them, like
[http://en.wikipedia.org/wiki/List_of_words_of_disputed_pronunciation].
Nevertheless, good support for IPA symbols would be a valuable component
of MediaWiki, as it obviously would be widely used on Wiktionary.

- David


from_name: Jimmy Wales

body:
Why don't you just take notes for yourself on a subpage of your
user: namespace?

A [name omitted for privacy reasons] wrote:

> One really good way to keep me from so much reverting
> (note, my reverts are always reverts of reverts -- I
> don't delete other people's test (I NPOV it instead);
> however, when people delete my text I do tend to
> revert their revert)
>
> But, a good way to slow my reverting down, is to give
> me a second watchlist.
>
> I use one watchlist so I can keep an idea of whats
> going on with articles I have edited (so that I can
> revert people who revert me); if I had a second
> watchlist, then I could deliberately watch articles at
> which someone was reverting me -- this would allow me
> to not have to worry about forgetting that so-and-so
> deleted my text.
>
> If I didn't have to worry about the deleting being
> forgotten, then I might be more willing to wait a day
> or two and see if the reverter gets around to
> explaining themselves.
>
> I also think it should become a rule that whenever
> someone deletes a chunk of text, that they include
> some sort of notification in the edit summary.
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Small Business $15K Web Design Giveaway
> http://promotions.yahoo.com/design_giveaway/
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l
>


subject: [Wikitech-l] Re: Yahoo! XML feed

body:
Ray Kiddy wrote:
> MySQL handles all this "changed since" checking and such. It also does
> it more efficiently than a PHP script. So, what problem are you all
> trying to solve?

We have another organization (Yahoo in this instance, but to make the
problem more general, let's assume that other cases will arise) which
we would like to accomodate by supplying for them a feed that is ready
made for their existing software.  They have a specification for input
which we can easily meet.  We want to do so in the most efficient way.

Yahoo wants an XML file of a particular kind.  It's not complex for us
to generate it.  We may wonder why they want this file, and of course
based on feedback from here, either Jason or I will talk to them to
propose something else if it seems helpful to do so.  But, this is the
format that they want, and we want them to have it.

--Jimbo


subject: [WikiEN-l] Deliberate chain yanking

body:
[Note: At the bottom of this is an important "blessing" by me of the
quickpoll methodology.  I put this note up here after I was done
because I realized that part was pretty important and I don't want
people to miss it.]

An edit by 'Michael33':
http://en.wikipedia.org/w/wiki.phtml?title=User_talk:Jimbo_Wales&dummy=1&diff=3015477&oldid=3015334

>In this case we chose the name of a hard-banned user
>([[User:Michael|Michael]]) hoping to make you believe
>[[User:Michael3|Michael3]] was his sock puppet.

An edit by 'Bird':
http://en.wikipedia.org/w/wiki.phtml?title=User_talk:Bird&dummy=1&diff=0&oldid=2831400

>Your distrust for this username is a product of efforts by the
>creator of this username to undermine your trust. I have manipulated
>other user names to demonstrate continued niave trust among this
>juvenile group.

But in both cases, it may be instructive to read the whole thing.

I just wanted to express my firm public opinion, Speaking Officially
and all that, that both of these admissions are sufficient in and of
themselves to justify a hard ban.  In both cases, the confession is to
actual trolling, i.e. misrepresenting oneself in order to get a
reaction from people.

The mental pathology of some people is astounding.  Wikipedia works
because such people are rare.  But they do exist.  I sit here in
amazement at a person who comes to my user_talk page to ask for action
against a sysop for banning someone while *at the same time* admitting
to having been trolling for the ban in the first place.

--

I fully approve of and authorize the use of QuickPolls to authorize a ban,
as per the policy on http://en.wikipedia.org/wiki/Wikipedia:Quickpolls_policy.

This policy gives teeth to the "three revert guideline" but also
provides for a good mechanism for community feedback and control over
the process.

I added "a signed in user confesses to deliberate trolling" but if for
some reason that's pushing it too far, I didn't mean for that to be
Policy From On High, just something that I think is a good idea.

The biggest problem I see with the mediation/arbitration paradigm is
that while it is fair and extremely respectful of due process, it's
just very very slow, and some people who ought to be banned end up
driving us bonkers for awhile.

Temporary banning, with community oversight, is a good interim
measure.

--Jimbo


subject: [Wikitech-l] Shopping?

body:
Geoffrin is physically installed and presumably will go into service
when the developers (Brion, esp.) get the time.  Brion had mentioned
possibly this weekend, but of course *I* think he should take his
time.  :-)

So now we have a pretty sweet setup, but let me know: what are our
next needs?  When should I start thinking about shopping?  What will
we want to get?

Random guesses are discouraged -- I think the most productive
recommendations will be based on specific information of bottlenecks
or points-of-failure that have formed or will form soon based on
actual empirical evidence.

I know coronelli has a stability problem, and we have money in the
bank.  Perhaps we could take coronelli out of rotation?

--Jimbo


subject: [WikiEN-l] Re: how to keep me from edit warring as much

body:
Timwi is right, Adam.  A good way to keep yourself from reverting so
much is just to relax and don't do it.  The world won't end.  Just
let things go sometimes.

Timwi wrote:

> A [name omitted for privacy reasons] wrote:
>
> >One really good way to keep me from so much reverting
> >(note, my reverts are always reverts of reverts -- I
> >don't delete other people's test (I NPOV it instead);
> >however, when people delete my text I do tend to
> >revert their revert)
>
> Then why do you do that? If you are aware that your reverts are likely
> to cause wars, why do you do them? Why not instead leave a note on a
> talk page, or perhaps the User's talk page, or perhaps just accept their
> revert (maybe it was the right thing to do after all)? How about hoping
> for others to reinstate your edit if it was good rather than assuming
> that if you don't fight for the Good And True, nobody will? Sorry if
> that sounded a bit polemic, but I'm trying to make a point.
>
> Timwi
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> http://mail.wikipedia.org/mailman/listinfo/wikien-l
>


subject: [WikiEN-l] edit wars

body:
It may be easy to criticize me, because I communicate
about the edit wars I am in. However, I think your
criticism would be far more useful if directed at
those who refuse to communicate.

Never in my history here at the wikipedia, never has
someone replaced text of mine that was deleted --
thats either because my text isn't any good or because
its really not that easy to tell that an edit conflict
occurred (unless, of course, you revert and revert
until it becomes very apparent that an edit war is
occurring).

In any case, if you want to decrease the number of
edit wars -- allow people to have two watchlists, so
they can use one watchlist solely to watch articles at
which they are involved in an "edit crisis". This will
allow people the luxury of not having to worry that
they will forget about the incident.

__________________________________
Do you Yahoo!?
Yahoo! Small Business $15K Web Design Giveaway
http://promotions.yahoo.com/design_giveaway/


from_name: Brion Vibber

subject: [Wikitech-l] Shopping?

body:
On Apr 3, 2004, at 14:29, Jimmy Wales wrote:
> So now we have a pretty sweet setup, but let me know: what are our
> next needs?  When should I start thinking about shopping?  What will
> we want to get?
>
> Random guesses are discouraged -- I think the most productive
> recommendations will be based on specific information of bottlenecks
> or points-of-failure that have formed or will form soon based on
> actual empirical evidence.

Distribution of disk space is a bit of a sore point at present. The
apache boxes don't really need much, as most of the space-sucking data
is coming off the wire over the database or NFS, however the space
requirements for the database and a couple generations of backups are
pretty pushy on zwinger and suda. If we could get expanded disk
capacity on those two, that would be a help. (Geoffrin IIRC should have
a goodly amount of space itself.)

As far as points of failure; zwinger has been pretty reliable, but it's
a big single point of failure: if NFS goes down, the apache's can't do
squat. We currently keep some of the common files on the local disks
for performance's sake and rsync them when things are updated, but
image uploads, thumbnails, and math rasterizations don't fit that model
well, having the requirement for files to be available from a
randomly-selected mirror less than a second after being created. We
might consider adding an explicit mirroring setup into the code, or...?

> I know coronelli has a stability problem, and we have money in the
> bank.  Perhaps we could take coronelli out of rotation?

Coronelli and browne both crash occasionally, though coronelli seems to
go a little more often. They're also both running experimental kernel
packages, which may reduce their stability (though greatly improving
performance compared with the RH9 stock 2.4.x kernels).

It probably wouldn't hurt to take coronelli down for another round of
stress testing, though.

-- brion vibber (brion @pobox.com)


from_name: vlad at isys-software.de

subject: [Wikipedia-l] Localisation (LanguageUk.php)

body:
Hi All,

I have translated Language.php into ukrainian. Now i would like to test it.
There are several questions.
1. How can i create new file (LanguageUk.php) in Wikimedia CVS?
2. How can it be installed an tested for uk.wikipedia.org?

Regards,
Gutsul


subject: [Wikipedia-l] New wikipedias: tpi, tp, io

body:
tpi.wikipedia.org - Tok Pisin
One of the official languages of Papua New Guinea. Some pages should be
moved from temporary work at bi.wikipedia.org over here.

io.wikipedia.org - Ido
Ido is a cousin of Esperanto, rarely seen but of some historical
interest.

tp.wikipedia.org - Toki Pona
Toki Pona is a more recent conlang, but with a relatively active and
enthusiastic user base. ('tp' is not assigned as a 2-letter code, and
ISO isn't assigning any more 2-letter codes. Rather than fight about
what to call it, I figured 'tp' will do.)

-- brion vibber (brion @ pobox.com)


subject: [Wikitech-l] XHTML

body:
Things aren't 100% clean yet, but some tweaks are starting to go in to
make output pass as XHTML.

If you'd like to test it, set these in your LocalSettings.php:

	$wgMimeType = "text/xml";
	$wgDocType = "-//W3C//DTD XHTML 1.0 Transitional//EN";
	$wgDTD = "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";

Mozilla will let you know immediately when it finds output that's not
well-formed XML by freaking out and refusing to show the page. :)

UI markup cleanup is tedious but easy. The parser still may need some
tweaking here and there, and particularly clean-up for user-written
HTML in wiki pages is an area that needs to be carefully checked.

Someday it might be neat to conform to XHTML 1.0 Strict, but that's a
freakier land where things like 'width' and 'height' attributes are
forbidden.

-- brion vibber (brion @ pobox.com)


from_name: Till Westermayer

subject: [Wikipedia-l] Patent idea

body:
. . . . . . . . . . . . . . . . . . . . . . . . . . . till we *) . . .

>On Thu, 1 Apr 2004, Timwi wrote:

>> We were looking for something we came up with that we could
>> patent, right?

>Are we? I think it's an awful idea to have the Wikimedia
>Foundation patent something.

Even if it's April1: indymedia.org has the same language-code scheme,
e.g. de.indymedia.org.
 __ .
 / / / /                            ... Till Westermayer - till we *)
. .
  . mailto:till at tillwe.de . www.westermayer.de/till/ . icq 320393072
  . Habsburgerstr. 82 . 79104 Freiburg . 0761 55697152 . 0160 96619179
  . . . . .


subject:
[Wikitech-l] Re: [MediaWiki-CVS] phase3/includes Parser.php, 1.45,
	1.46

body:
On Apr 3, 2004, at 17:45, Gabriel Wicke wrote:
> * invalid anchor names- attention! this might break links! Replacing
> [^a-z0-9] -> '_' now

That's going to be pretty bad for languages that don't write in Latin
script, isn't it?

-- brion vibber (brion @ pobox.com)


subject: [Wikipedia-l] Re: Localisation (LanguageUk.php)

body:
On Apr 3, 2004, at 15:21, Timwi wrote:
> vlad at isys-software.de wrote:
>> I have translated Language.php into ukrainian.
>
> I assume that http://meta.wikipedia.org/wiki/LanguageUk.php is your
> version? I have committed that version into CVS; hopefully someone
> will look it over and make corrections as necessary. Please try to get
> as many proofreaders as possible.
>
>> 1. How can i create new file (LanguageUk.php) in Wikimedia CVS?
>
> I've done it for you.

I made a couple small fixes so the file would run.

>> 2. How can it be installed an tested for uk.wikipedia.org?
>
> If you're lucky, someone will reply and do that for you. I can't.

Done. Give a shout if there are any problems.

-- brion vibber (brion @ pobox.com)


subject:
[Wikitech-l] Re: [WikiEN-l] Re: IPA pronounciation Unicode
	symbols, old browsers and Wikipedia

body:
On Apr 3, 2004, at 18:55, Karl Eichwalder wrote:
> fabiform <fabiform_wikipedian at yahoo.com> writes:
>> I'm comfortable with using the IPA, but many people do find all the
>> symbols
>> off-putting.  In fact, very few dictionaries use it, presumably for
>> that
>> reason.  Perhaps SAMPA is a bit friendlier?
>
> All printed dictionary I am own use the IPA.  Maybe, online ditionaries
> are different?  If yes, they might be different because editor don't
> know how to enter IPA symbols.

IPA seems to be fairly uncommon in monolingual English-language
dictionaries, which tend to use their own idiosyncratic systems. My
bilingual dictionaries are pretty much all IPA, though.

> This SAMPA looks very ugly to me and I have never learnt it.

SAMPA is just an ASCIIfied IPA. It's a crutch for the ancient days of
crappy uninternationalized computers which are coming to an end.

I can't say SAMPA is any friendlier to an unfamiliar reader than IPA,
it's just easier to type.

-- brion vibber (brion @ pobox.com)


from_name: Pedro Fortuny

body:
My ISP's filter does not allow me to visit "[[Fucking, Austria]]".
Just to make you smile, though :)

Pedro.

* David Gerard <fun at thingy.apana.org.au> [2004-04-02]:
> Please read [[Fucking, Austria]] and [[Talk:Fucking, Austria]].
> (...)

--
Pedro Fortuny Ayuso: http://pfortuny.sdf-eu.org
Colegio Mayor Pe?afiel, Universidad de Valladolid
C/ Estudios 6, 47005 Valladolid, Spain -->        www.cmpenafiel.org
pfortuny at sdf-eu.org                         Tfn. Nr. 34  983  298277


from_name: Joao Mário Miranda

subject: [Wikipedia-l] portuguese mailing list:administration password

body:
Hello,

It seems I' m the administrator of the portuguese mailing list:

http://mail.wikipedia.org/mailman/listinfo/wikipt

But I don't have the administration password. Could someone
send it to me?

Thanks.

Joao Miranda


subject: [Wikipedia-l] Re: portuguese mailing list:administration password

body:
Joao M?rio Miranda <jmiranda at ...> writes:

> But I don't have the administration password. Could someone
> send it to me?

Done.

Walter


from_name: Florian Caspari

subject: [Wikitech-l] help needed

body:
Hi,

since a few days I am trying to install the wikipedia software on my server
and finally capitulated. Next I looked for a forum where I could ask for
help - and found nothing.

I do not know if this mailinglist ist the right place for my question: "Is
there anybody who is willing to set up the wikipedia-software on my server?"
or who could give me a hint about persons I could ask?

Best regards

-------------------
Florian Caspari
info at fcaspari.de


from_name: Anthere

subject: [Wikipedia-l] Meta main page

body:
Before :

http://meta.wikipedia.org/w/wiki.phtml?title=Main_Page&oldid=29076

Now : http://meta.wikipedia.org/wiki/Main_Page

What do you think ?

-----

It is not entirely done.

*Some links should probably be added on the main page
(please do add them, if you think these are very
useful ones, I in particular think of all technical
pages I never look at, so can't imagine how important
they are to some people)

*Please, someone find a way to convince text to fit at
the top of boxes, this is beyond me

*Some boxes need improvement, they are just draft
right now (collaboration in particular)

*Of course, many second level links are a bit messy
right now. Some are desperatly poor...look at
[[copyright]] for example...such a major matter...how
shameful :-)

*I tried to put the boxes according to my perception
of usefulness. That is...most people come here first
for technical issues (such as languages files)...or
hopefully, information...so, I put them at the top.
Whilst , most people probably do not access wikimedia
projects from meta itself, so I put them at the
bottom.

*I put some pink (yes !) on a couple of links, either
because they are most asked at some point, or to make
advertisement. Of course, I suggest that changes. We
could put pink on "press release" the next time we
make a general press release.

I hope I made meta a little bit more friendly
accessible. Still a lot of work to do :-)

__________________________________
Do you Yahoo!?
Yahoo! Small Business $15K Web Design Giveaway
http://promotions.yahoo.com/design_giveaway/


from_name: Timwi

subject: [Wikitech-l] Re: Categories

body:
Magnus Manske wrote:

> I just tried the category feaature, and found it broken - again. This
> must be *at least* the third time I left intact code and found it broken.
> Please, be a little more careful next time, OK?

I bet this has to do with the fact that the category stuff is not in
live use on Wikipedia. That makes people forget about it much more easily.

> Also, I found that "brokenlink" seems to amass multiple identical links
> each time one saves a page with broken links. I have DISTINCT the
> queries for now, but we'll have to do someting about that.

The solution to this is called a UNIQUE index, and using REPLACE INTO
instead of INSERT INTO.

The same should apply to the 'links' table.

Timwi


subject: [Wikitech-l] Re: proxy blocker mistakes

body:
Andre Engels wrote:

> I checked, and on de: there are now over 100
> indefinitely blocked IPs blocked by Proxy blocker

Then you should probably unblock them. I had unblocked all the
indefinitely-blocked on en after it was switched to banning them only
for 7 days. I would have thought the other Wikipedias would do the same.

Timwi


subject: [Wikitech-l] Re: More proxy blocker problems

body:
Andre Engels wrote:

> I noticed another problem with Proxy blocker: It seems that every time
> it blocks an IP, a new userid is created - just check
> http://de.wikipedia.org/w/wiki.phtml?title=Spezial:Listusers&limit=250&offset=15360.

Geez! Why does nothing in the DB have a UNIQUE index where one would
make sense?!

> Apart from that it would be good to have proxy blocker unblock all addresses
> it has blocked 'indefinitely'.

Sysops can do that, since it only needs to be done once. Even if you
have only 10 sysops, each one would need to unblock only 10 addresses.

Timwi