[00:42:17] Reedy it's !ask | david____ [00:42:18] heh [00:42:35] Not necessary to ping people [00:45:06] ok [03:01:20] funny that there is no bulk email extension for MediaWiki, you'd think that would be quite a simple thing [03:01:41] is there some simple way to do bulk mailing that everyone uses? [03:03:32] sendmail bash script? [03:06:55] Depends how bulk. Mailing list? BCC field in any standard e-mail client? [03:07:20] For a MediaWiki wiki, you can post on the talk pages of the users and that generally triggers an e-mail notification. [03:07:29] We did that in the past for the strategy.wikimedia.org. [03:08:47] 40,000 email addresses [03:09:22] I think thunderbird would go OOM if I pasted that into the BCC field [03:09:52] It's one of those features that seems obvious but is also basically encouraging spam. [03:10:01] So I can see why places like Gmail would not support it too easily. [03:10:14] for the SecurePoll mailout, there was a PHP script that just generates a list of email addresses, it didn't actually do the sending [03:10:40] I think the fundraising folks switched to some third-party service. [03:10:42] so I assume there was some other script or utility for that [03:11:11] There's definitely at least one script somewhere for bulk-sending e-mail. Werdna did one at some point. [03:11:21] I remember because it led to the creation of an opt-out list on Meta-Wiki. [03:11:37] It'd probably be more work to find the script than to just write one, tho. [03:13:38] it's for http://rationalwiki.blogspot.com.au/2017/06/server-upgrade-data-breach-on-old.html in case you're wondering [03:14:05] A mailing list isn't totally crazy. [03:14:13] Though people would probably not appreciate being subscribed to one. [03:14:28] But I think mailman has a bulk-add feature and it's a well-tested mail server. [03:14:35] so I (hopefully) only need to do this once, unlike fundraising which needs to handle opt-out etc. [03:14:41] Sure. [03:23:16] I think I'd use `mail` with a for loop in a shell script, assuming I had a decent mail server. [03:26:09] That blog post is a bit of a tease, no details about the breach. Was there an SQL file left around? Incriminating .bash_history? Server access logs? [03:31:42] I've been thinking about writing up a proper post-mortem for wikitech-l [03:32:10] TimStarling: ping Reedy maybe, i think hes done a few securepoll mailouts in the past [03:32:24] it was an RW-specific extension with an SQL injection vulnerability [03:33:17] the attacker found it by fuzzing, and then used SELECT INTO OUTFILE [03:33:31] MW was running with mysql root equivalent permissions, so could do that [03:33:57] and there was a directory which was world-writable in the document root, which had PHP enabled [03:34:01] Does MySQL separate SELECT (or OUTFILE) permissions? [03:34:07] Ahh. [03:34:30] so the server was fully compromised with user www-data [03:35:03] they then installed Adminer, a single-script MySQL administration tool, and used it to download the user table [03:35:50] Hmmm. I wonder if the user table could/should not store e-mail addresses in plaintext. [03:36:28] I guess it doesn't really protect you much in any case. [03:37:09] RW had a squid server in front, which was not compromised, all the access logs were preserved [03:37:47] It kept months of logs? [03:37:51] years [03:38:27] Wild. [03:39:49] Seems many others have beaten me to the "IrrationalWiki" joke. [03:39:53] Such that it is. [03:40:09] https://www.urbandictionary.com/define.php?term=RationalWiki [03:42:00] http://rationalwiki.org/wiki/Essay:I_thought_this_was_supposed_to_be_RATIONALWiki [03:53:27] Esther: you wrote T100918, you're practically the expert on this [03:53:28] T100918: Standardize and document process for mass electronic mailings (Board elections, fundraising, etc.) - https://phabricator.wikimedia.org/T100918 [03:54:03] SecurePoll actually used sendMails.php, which is a loop of UserMailer::send() [03:54:55] with sleep(0.1) [03:55:15] interestingly the parameter to sleep() is an integer, it is rounded down, so that is the same as sleep(0) [04:32:07] some spambot made 905 accounts with unique email addresses that match a particular pattern [04:34:02] Oh, only two years ago. [04:34:44] Ah, right: https://phabricator.wikimedia.org/T100454#1323620 [04:35:39] When I was like "how is the Wikimedia Foundation 12 years old and still so amateur hour?" [06:08:33] TimStarling: was the SQLi because the extension wasn't using MW's database abstraction, or something else? [06:09:52] it was using it for most things, but the author didn't know how to make a LIKE clause, so just did that by string concatenation with unvalidated user input [06:10:19] you could say it comes down to our choice of structured data versus prepared statements [06:10:43] easy enough to figure out "WHERE field LIKE ?" [06:11:02] not so easy to figure out WHERE field LIKE ' . $dbr->addQuotes($x) [06:11:12] let alone buildLike [06:12:42] https://github.com/RationalWiki/mediawiki-extensions-RWEditcount/commit/b75a2212aed9a8092641b342716b62649ecd4b6b [06:14:32] https://phabricator.wikimedia.org/T56847 <-- I don't remember which script we used to e-mail about this. [06:15:33] I wrote a new script: https://github.com/RationalWiki/mediawiki-extensions-RationalWiki/blob/master/cli/breachMailout.php [06:17:13] no HTTPS on rationalwiki? [06:17:44] no, dgerard is licking the cookie on that one [06:17:59] Needing a sanitizeLike function in Editcount.body.php has a nasty code smell. [06:18:14] simultaneously while finishing writing a book [06:18:24] no kidding Esther [06:18:28] Like you'd think there would be a common function for that in PHP or in MediaWiki. [06:18:43] Glad we're all agreed! [06:18:46] or change the input format [06:19:19] we welcome pull requests [06:20:00] I had some from JackPhoenix, which was great to see [06:20:27] ashley is great. [06:20:43] that's the one [06:21:44] was looking through the channel list trying to spot them [06:22:37] can squid handle SSL? getting a cert with let's encrypt takes all of 30 seconds these days [06:23:06] we use varnish now, David set it up [06:23:13] he wants nginx in front of varnish, like WMF [06:23:16] I noticed today that Tim is using a LE cert, so I guess it must be safe enough. [06:24:42] yep [06:24:54] even WMF is using LE for some minor services [06:24:57] Domas too, look at that. [06:25:15] they're going to start offering wildcard certs next year [06:25:24] Esther: =) [06:25:56] :-) [06:27:07] brionv.com and legoktm.com too. All the cool kids have adopted it. [06:28:31] It's your turn now! [06:30:03] I was talking to legoktm earlier about RW's tech budget [06:30:46] we've gotten it down to $290/month now [06:32:34] the hard part about the remainder is the disk usage, which is expensive on linode [06:32:47] until recently they had a >100GB mysql database [06:33:02] Oh wow [06:33:06] down to 36GB now after I cleaned up some old data and ran compressOld.php [06:33:39] plus 26GB images, 23GB dumps, 6GB elasticsearch [06:34:36] I think DO has pretty decent storage prices, $10 for 100GB/month: https://www.digitalocean.com/products/storage/ [06:35:04] linode has a block storage feature in beta, which (probably deliberately) matches that price exactly [06:35:32] I figure we can wait a few months for that to be rolled out properly [06:35:57] http://rationalwiki.org/wiki/File:Oprah_gotohell.jpeg [06:36:18] most of the images are actually conservapedia screenshots [06:36:54] We're in the process of moving the Wikimedia DC farm from linode to digitalocean. Our bill is going from $60/mo to about $10/mo [06:36:55] Just sampling a couple of these images, I'm not sure why they're being hosted on this site. [06:36:59] they have some truly obsessive conservapedia watchers, and an extension which makes file red links for the purposes of screenshot uploads [06:37:11] Like http://rationalwiki.org/wiki/File:TheSerpentTemptsGod.gif isn't used on any wiki pages. [06:37:35] But has four versions. [15:54:35] Hi, I want to add the extra bullet point to mediawiki API page but would like to discuss with page maintainers the change. Will Discuss tab on the page be appropriate channel? ( https://www.mediawiki.org/wiki/API:Tutorial#You_may_actually_want ) [16:28:12] How do I stop new pages from showing up in recentchanges after being deleted [16:35:53] c: you mean, if a page gets deleted, it should not be mentioned in the RC? (except the deletion log entry)? [16:36:41] IIRC you can't change that. edits at pages who where deleted are not shown in the RC. If there are shown, the jobrunner is not doing his work properly [16:38:21] Sagan: well the deletion log entry is ok, it's the new page entry that is the problem, more specifically when it's a spam page and the content is shown in the edit summary. as a workaround i have been blanking the page, hiding the edit summary, then deleting [16:39:20] c: is that a wikimedia wiki, or another one? We had the same problem once at the beta-cluster, we fixed it by fixing the jobrunners [16:39:39] it's another one [16:39:56] the new page entry get hidden if the page gets deleted normally, but in case the jobrunners are not working properly, they are not sorting these entrys out, I'd say [16:40:16] the new page entry is added to the job queue you mean? [16:40:57] c: no, if a page gets deleted, the jobrunners will normally take care, that the page does not occur at Special:RecentChanges, Special:NewPages etc [16:41:18] ok where would I remedy it [16:41:30] in general. but I don't know much about the exakt way how the jobrunners are doing that, I did not take a look at it [16:44:19] hm, sorry, I don't know much about jobrunners to tell you the right answer. Maybe someone other does? I did not find something at mw.org [16:44:47] it must be due to a more recent release because this was not an issue before we upgraded to 1.28.x [16:49:20] c: do you have shell there? maybe you can take a look at the queue with mwscript showJobs.php (https://www.mediawiki.org/wiki/Manual:ShowJobs.php) [16:49:41] if that's the problem, https://www.mediawiki.org/wiki/Manual:RunJobs.php can help maybe [16:50:50] https://www.mediawiki.org/wiki/Manual:Job_queue has more infos about the queue [16:59:48] Sagan: I have full access, but you just said the job runner was different from the job queue (which due to the complexity of our site, runs daily) [17:57:16] Hello everyone [17:58:16] I would like to know, is it possible to extract a list of articles from a wiki's category? I would like to copy all the names from this - https://www.nordicnames.de/wiki/Category:Old_Swedish_Male_Names - but doing so manually would be VERY tedious. [17:58:34] I would merely like to extract the list of articles within said category, in plain text form [18:18:39] c: I'm not sure if there is a difference, I'd say not. I'm not a mediawiki dev, I remebered that the last time we had that issue at beta, it was the job queue. So maybe this is a question for -tech, or -dev, or if somebody with more konwledge then me is online? I'd ask there, or try, if you run the jobs once manually, if the problem still exists then [18:21:40] Okay, I kind of found a way to do it, but it's not very effective [19:29:15] metalhead33: Sounds like a job for the API. See https://www.mediawiki.org/wiki/API:Categorymembers [19:29:25] Yes, that is what I am using [19:29:33] Not the most efficient thing, but still better than nothing [23:43:53] is there no way to require registered users to have a confirmed email AND have anonymous editing at the same time?