[15:58:56] hi y'all! [15:59:17] sup ejegg, where are you now? [15:59:39] in the Bogotá airport about to hop on a short flight to Cartagena [15:59:53] jealous [15:59:56] I'll arrive a little ways into tech talk [15:59:58] :) [16:00:21] Bogotá airport wifi is way better than Boston Logan [16:01:18] ha! [16:02:44] yesterday I met up with the country's sole source of Yubikeys in a coffee shop in a mall [16:02:57] slipped him 150,000 Colombian pesos for a Yubikey 4 [16:03:14] he promised to email me a receipt... [16:03:18] :) [16:04:52] whaaaaat [16:04:54] that's awesome [16:05:06] hehe [16:05:55] i was working with OIT on friday and tried both my yubikeys but have still been unable to auth [16:06:16] oh dang, maybe the key version wasn't my problem after all! [16:08:02] yeah i am not sure what is up [16:08:06] i will be following up today [16:59:52] fundraising-tech-ops: rename backup4001 to frbackup4001 and jessie reimage - https://phabricator.wikimedia.org/T158220#3043528 (Jgreen) a:Jgreen>RobH reassigning to Rob since we're stuck at a hardware issue [17:00:37] fundraising-tech-ops: rename backup4001 to frbackup4001 and jessie reimage - https://phabricator.wikimedia.org/T158220#3043533 (Jgreen) a:RobH>Jgreen [17:01:49] fundraising-tech-ops, Operations, ops-ulsfo: upgrade backup4001 hard disk array - https://phabricator.wikimedia.org/T157473#3043535 (Jgreen) a:Jgreen>RobH Reassigning to Rob because we're stuck at a hardware problem (new HDDs appear to be incompatible with the controller/BIOS/firmware?) [17:04:13] fundraising-tech-ops, Operations, ops-ulsfo: upgrade backup4001 hard disk array - https://phabricator.wikimedia.org/T157473#3043539 (RobH) So now the system is in a bad state where I cannot login to the webGUI to upgrade firmware, and its not coming back from it by racreset. in detail: trying to log... [17:25:43] bienvenidos awight [17:29:19] o/ [17:29:44] It's funny being "back" yet everything is still virtual [17:30:08] I was glad to find that the office hadn't been moved [17:30:25] yet... [17:31:31] awight: you are into internet radio right? could you hook me up with some recommendations? i find that i can stream low kbps stuff just fine on my connection [17:32:18] Moving the office: http://eurekahouse.info/wp-content/gallery/hedger-photos/11.Moving-building-by-horse-and-wagon.-Ballarat-Golden-City-A-Pictorial-History-John-Reid-John-Chisholm-1989.jpg [17:32:22] cwd: yah rad [17:32:36] I enjoy KALX, KDVS, WFMU, KPOO [17:33:13] KZSC although a bit stony, StartFM though I haven't figured out if they're 50% loop tapes or what [17:33:30] WSUM and WRUV for typical college fare [17:33:40] CKUT a new favorite [17:42:07] fr-tech: I'll review T128692, for lack of other thing to do [17:42:08] T128692: Store and update list of currenly working IDEAL banks - https://phabricator.wikimedia.org/T128692 [18:00:21] fr-tech: They Tore Out My Heart and Stomped That Sucker Flat [18:00:21] -- Book title by Lewis Grizzard [18:00:21] -- discuss. [18:11:00] * awight twirls hair in fr-tech-talk [18:11:41] ah, be there in 5 [18:11:44] finishing breakfast [18:11:57] ejegg said he'll be back during it [18:12:34] although he's flying around south america so that could get complicated [18:13:34] heyo! sitting on te tarmac at my destination. will try to call in from the airport. [18:13:37] I'm bluffing anyway, seems like I can't reach the audio jack with anything I have here [18:13:44] p'aps we do it in IRC? [18:15:01] shore [18:15:12] (CR) Awight: Look up iDEAL banks, provide PSR6 to SmashPig (4 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/337420 (https://phabricator.wikimedia.org/T128692) (owner: Ejegg) [18:15:42] I'm Adam. I have been back on duty for one hour. [18:15:50] ever since verizon announced unlimited the t-mobile network at my house has been a reliable 2mb [18:15:55] ooh [18:15:59] i don't know what the exact interplay is but it's something [18:16:17] so i suddenly have 3 fairly reliable connections [18:16:26] I'm sure it will last until they control enough of one another's stock portfolio [18:16:52] they're all criminals [18:17:00] I could keep busy reviewing things in the Review column, I think. [18:17:57] i need to get the vpn working so i can test https://gerrit.wikimedia.org/r/#/c/338410/ [18:21:55] oh wow, there's not much airport to go through after you land [18:23:38] fundraising-tech-ops, Operations, Traffic, HTTPS: update SSL certificate for benefactorevents.wikimedia.org by 2017-03-02 - https://phabricator.wikimedia.org/T158684#3043892 (Jgreen) [18:24:15] fundraising-tech-ops, Operations, Traffic, HTTPS: update SSL certificate for benefactorevents.wikimedia.org by 2017-03-02 - https://phabricator.wikimedia.org/T158684#3043908 (Jgreen) @EWilfong_WMF are you the right point of contact for Trilogy for this? [18:26:59] fundraising-tech-ops, Operations, Traffic, HTTPS: update SSL certificate for benefactorevents.wikimedia.org by 2017-03-02 - https://phabricator.wikimedia.org/T158684#3043925 (RobH) Please note that some potential details for this are also on private task T156849. However, relevant info has been... [18:34:50] tmob / sprint merger is back in play [18:36:06] sigh [18:37:27] could explain the usable network [18:37:44] but what we need is not another behemoth wireless carrier [19:00:23] fundraising-tech-ops, DBA: fundraising database tuning - https://phabricator.wikimedia.org/T158446#3044038 (Jgreen) innodb_buffer_pool_size yep, I've increased that to 75-80% of system RAM. The tables are all innodb but the config dates back before we were fully converted. Looks like we should reduce ke... [19:03:22] (PS6) Ejegg: Look up iDEAL banks, provide PSR6 to SmashPig [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/337420 (https://phabricator.wikimedia.org/T128692) [19:13:42] found a likely work spot in town - escaping the airport! [19:22:11] awight: boo! u back? [19:22:36] hi! Yep, at least physically here. [19:22:46] I was just daydreaming about playing with kids actually [19:28:54] oops, checking with the stats team [19:28:59] *check-in [19:30:13] ejegg: I'll be there in 2 min... Dunno if there's much to discuss [19:30:24] I sure can't think of anything [19:30:40] unless they've started digging into the internal dash code [19:30:58] in that case I've got a lot to answer for :P [19:33:09] One sec! [19:36:23] (CR) Awight: [C: 2] Add $wgCentralSelectedMobileBannerDispatcher global for mobile [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/338160 (https://phabricator.wikimedia.org/T154954) (owner: AndyRussG) [19:45:00] (CR) Awight: "Ooh iterating over every URL param makes me sad. I'll dig around for regex purge possibilities... I know it's supported by Varnish, I ca" (2 comments) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/336237 (https://phabricator.wikimedia.org/T154954) (owner: AndyRussG) [19:48:19] (Merged) jenkins-bot: Add $wgCentralSelectedMobileBannerDispatcher global for mobile [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/338160 (https://phabricator.wikimedia.org/T154954) (owner: AndyRussG) [20:23:47] fundraising-tech-ops, Operations, Traffic, HTTPS: update SSL certificate for benefactorevents.wikimedia.org by 2017-03-02 - https://phabricator.wikimedia.org/T158684#3044424 (EWilfong_WMF) @Jgreen Yes, I will be the point of contact for this update. This domain is hosted using Azure's App Servic... [20:36:10] ejegg|afk: do u recall how to instrument temporary patches on the betaclust / link to doc thereabout? [20:36:27] awight|lunch: ^ [20:40:49] AndyRussG: https://www.mediawiki.org/wiki/Beta_Cluster#Testing_changes_on_Beta_Cluster ? [20:49:02] AndyRussG: not exactly! did cwd's link help? [20:51:05] ejegg: cwd: yeah that's a start... I remember a lot more details, tho, something like some re-deploying code from master every minute or two on a schedule? [20:53:34] I seem to remember 10 minute windows [20:54:33] https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/How_code_is_updated [20:55:15] AndyRussG: section 2.1 in that page is relevant ^^^ [20:57:57] cool! [20:58:23] ejegg: cwd. thx much! [20:58:43] :) [20:58:45] yw [21:13:30] eileen1: and awight|lunch if you have a second, can you put estimates on your tasks from December? This link is filtered for you: https://phabricator.wikimedia.org/project/board/2364/query/D9fCC9tSeQq4/ [21:15:47] I feel like I already did - what ones did I miss? [21:24:01] coastal accent is rough [21:26:49] Adonde fuiste? [21:31:37] eileen1: I see one: https://phabricator.wikimedia.org/T152507 [21:31:47] estoy en Cartagena [21:32:24] AndyRussG: I'm alarmed at finding $wgSquidServers in CdnCacheUpdate.php [21:34:05] dstrine: thx [21:34:32] awight: ? [21:34:45] ejegg|afk: dang, looks saucy! [21:35:00] AndyRussG: I'm reviewing https://gerrit.wikimedia.org/r/#/c/336237/4 [21:35:18] , but haven't convinced myself that CdnCacheUpdate is what we want [21:35:36] Have you gotten confirmation that it's the current best practice for purging? [21:36:06] awight: hmmm, k... No, but it's what's used, as far as I could see, when mw wants to purge a page [21:36:07] e.g. I thought we had gotten rid of squid everywhere, yet squid features prominently in that class [21:36:24] awight: I think it's just called squid for nostalgia purposes [21:36:29] but it's really Varnish [21:36:38] Whenever you see Squid, think Varnish [21:36:56] Updated like, https://enlighted.com/img/products/frontx600-10.jpg [21:37:21] That's the class mw uses when a pages is changes or is moved, to purge frontend, which is parallel to this use case [21:37:46] Yeah! That's the jacket we want! [21:37:50] I know that Varnish supports purge by regex [21:37:55] We need to talk to ops about this [21:38:09] cos AFAIK, each purge is maintained as a separate rule for some... amount of time [21:38:20] awight: I did think about that option but didn't dig in... Ah hmm that's interesting [21:38:22] therefore, a single regex might be way better than thousands of URL permutations [21:38:35] Yeah I was going to ask about the impact of purging so many all at once [21:38:42] Also, we can't guarantee the order of URL params, so I'm not this this even works [21:38:51] s/this/sure/ [21:39:23] I did talk to bblack, but he's a full-on Varnish guy, and doesn't want to comment much on mw/php code [21:39:33] awight: ahhhh hrrmmmm yeah good catch re: param order... [21:39:36] hehe that's one way to stay sane [21:39:59] or to keep from switching to a new form of insanity [21:43:38] Reading https://www.varnish-cache.org/docs/3.0/tutorial/purging.html ... [21:43:43] I think we would want "ban" [21:44:09] I don't see any ban endpoints in operations/puppet though [21:44:19] Other than the mysterious ./modules/scap/files/purge-varnish:dsh -cM -g bits -o -lroot "varnishadm -T :6082 -S /etc/varnish/secret ban.url $1; echo done" [21:45:06] Fundraising-Backlog: Upate roles on wiki - https://phabricator.wikimedia.org/T158710#3044743 (ggellerman) [21:45:21] ./modules/varnish/templates/vcl/wikimedia-common.inc.vcl.erb -> wm_common_recv_purge [21:45:49] awight: why ban? [21:46:37] I thought I remember it being less expensive. Also, cos purge seems to only get one page at a time [21:46:46] Hmmm [21:46:55] Ban is more intensive at the outset [21:47:03] I guess it only uses CPU time if you don't replace the object? [21:47:24] Dunno, I'm inclined to think that whatever MW does for pages is what we want [21:47:48] But yea, let's ask bblack or ema (the other traffic person) for more input! [21:50:05] " If you have a lot of objects with long TTL, that are seldom accessed you might accumulate a lot of bans. This might impact CPU usage and thereby performance." [21:50:07] = [21:50:09] ? [21:50:31] I think our TTL is pretty long [21:50:40] but we're only going to be adding a few dozen bans [21:50:48] one per banner, I think [21:51:50] Sounds like you already discussed forcing a cache miss--like never caching banners from the office IP, for example [21:53:01] AndyRussG / awight : the order should be deterministic [21:53:16] ejegg: But it's created from a hash in Javascript [21:53:33] hmm [21:53:39] * ejegg looks at mwscript [21:53:43] awight: no didn't look into that... The main use case is for online creating a banner on desktop, but then testing on the fones [21:53:53] aah [21:54:10] 'cause logged-ins are already not cached [21:54:11] > It should be noted that a Map that is a map of an object, especially a dictionary of dictionaries, will only map to the object's insertion order -- which is random and not ordered. [21:54:15] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map [21:54:19] Sounds kind of deterministic, actually [21:55:13] oho, url.extend is standard! [21:55:34] (or also could be interesting just to have banners update fast if there's a need to get rid of an incriminating typo) [21:55:50] I do like the idea [21:56:01] Basically just like articles [21:56:41] awight: not deterministic: "which is random and not ordered." [21:57:05] I think when it says insertion order, it refers to the internal implementation's insertion order? [21:57:20] * AndyRussG imagines a zillion more permutations [21:57:48] hurm, I was thinking it was the order in which we add params [21:58:02] I guess this can actually be checked, though [21:58:05] Yeah could be, doesn't seem too clear [21:58:17] see you in hangouts! [21:58:21] I think in most practical cases it's _pretty _ deterministic [22:05:57] sorry fr-tech, having trouble starting hangout [22:08:41] Fundraising-Backlog, FR-Smashpig, Epic, Patch-For-Review: [EPIC] Reduce use of SmashPig\Core\Configuration where possible - https://phabricator.wikimedia.org/T158374#3044799 (ggellerman) p:Triage>Normal [22:09:32] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Currency conversion for non-USD matching gift wires - https://phabricator.wikimedia.org/T158267#3044803 (ggellerman) p:Triage>Normal [22:11:07] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Occasional unexplained missing banner messages - https://phabricator.wikimedia.org/T158261#3044817 (ggellerman) p:Triage>Low [22:15:11] Fundraising-Backlog: Upate roles on wiki - https://phabricator.wikimedia.org/T158710#3044835 (ggellerman) p:Triage>Normal [22:15:19] Fundraising-Backlog: Update fr-tech job roles on wiki - https://phabricator.wikimedia.org/T158710#3044836 (awight) p:Normal>Triage [22:15:55] Fundraising Sprint Deferential Equations, Fundraising-Backlog: Update fr-tech job roles on wiki - https://phabricator.wikimedia.org/T158710#3044839 (ggellerman) [22:23:38] Fundraising Sprint Deferential Equations, Fundraising-Backlog: Update fr-tech job roles on wiki - https://phabricator.wikimedia.org/T158710#3044852 (ggellerman) p:Triage>Normal [22:58:18] ack, phone got kicked off the coffeeshopnet [23:20:35] (CR) Awight: "Following up on my earlier comment, (and from a great distance), it does look consistent to purge the URLs one at a time." [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/336237 (https://phabricator.wikimedia.org/T154954) (owner: AndyRussG) [23:29:35] (PS7) Ejegg: Look up iDEAL banks, provide PSR6 to SmashPig [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/337420 (https://phabricator.wikimedia.org/T128692) [23:33:15] (CR) Ejegg: "Thanks for the CR awight! Answers inline" (3 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/337420 (https://phabricator.wikimedia.org/T128692) (owner: Ejegg) [23:41:05] * ejegg needs actual food [23:41:13] be back after I get some dinner in me! [23:41:33] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Upgrade our server php version to 5.5+ (php 7 would be fine) - https://phabricator.wikimedia.org/T158717#3045056 (Eileenmcnaughton) [23:43:46] (CR) Awight: [C: 2] "That one looks nice!" (3 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/337420 (https://phabricator.wikimedia.org/T128692) (owner: Ejegg) [23:45:36] dstrine: oh hey. Remind me, you'll be in the office WTF? [23:45:42] (Merged) jenkins-bot: Look up iDEAL banks, provide PSR6 to SmashPig [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/337420 (https://phabricator.wikimedia.org/T128692) (owner: Ejegg) [23:45:44] ahem: wed, thurs, fri [23:45:57] lololol [23:46:05] I'm saving that one [23:46:17] yeah so tomorrow [23:46:53] not T/F tho? [23:47:08] yes that too [23:47:16] ftw? [23:47:52] AndyRussG: yeah but I hope to experience it in the usual linear way [23:47:58] cool thanks [23:48:05] I'll see you W at least