[00:00:35] TabbyCat: https://github.com/wikimedia/mediawiki-extensions-CentralAuth/blob/master/db_patches/patch-requestrename.sql [00:00:40] TabbyCat: https://phabricator.wikimedia.org/diffusion/ECAU/browse/master/central-auth.sql;ed0ca511f3eb24a3374adf7274e53d47c49003bd$194-233 [00:02:14] lol, both the same :) [00:02:21] you're awesome, thanks both [00:02:34] are the multiple mounts on labstore1001 on purpose? e.g. /dev/md0 /exp/backup-maps is repeated four times [00:10:43] godog: sounds like a bug from some NFS messing about, but madhuvishy and chasemp might know more. I think both of them are AFK for a few days though [00:14:00] bd808: *nod* thanks, I'll open a task on labs [00:39:30] (03PS3) 10Legoktm: Subsume grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328147 [00:39:58] (03CR) 10jenkins-bot: [V: 04-1] Subsume grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328147 (owner: 10Legoktm) [00:43:04] (03PS4) 10Legoktm: Subsume grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328147 [00:47:51] 06Labs, 10Labs-Infrastructure: Duplicate /proc/mounts entries on labstore1001 - https://phabricator.wikimedia.org/T153728#2888830 (10fgiunchedi) [01:00:08] Bot isnt commenting on tasks? [01:00:32] (03CR) 10Legoktm: [] Subsume grrrit-wm (034 comments) [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328147 (owner: 10Legoktm) [01:00:44] Zppix: ? [01:01:29] I uploaded a patch for T152495 and the bot didnt say anything bout it [01:01:30] T152495: E-Mail notification on failures of Wikidata-builds - https://phabricator.wikimedia.org/T152495 [01:04:11] hmm, not sure [01:06:48] (03CR) 10Legoktm: [C: 032] "Let's do it." [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328147 (owner: 10Legoktm) [01:07:13] (03Merged) 10jenkins-bot: Subsume grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328147 (owner: 10Legoktm) [01:14:08] (03PS1) 10Legoktm: grrrrit: Use absolute path to ssh key [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328317 [01:14:53] (03CR) 10Legoktm: [C: 032] grrrrit: Use absolute path to ssh key [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328317 (owner: 10Legoktm) [01:15:14] (03Merged) 10jenkins-bot: grrrrit: Use absolute path to ssh key [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328317 (owner: 10Legoktm) [01:19:27] every once in a while it misses one update [01:19:35] not a lot but i have seen it [01:22:29] (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328317 (owner: 10Legoktm) [01:23:28] (03PS1) 10Legoktm: grrrrit: Catch all exceptions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328319 [01:23:47] (03CR) 10Legoktm: [C: 032] grrrrit: Catch all exceptions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328319 (owner: 10Legoktm) [01:24:23] (03Merged) 10jenkins-bot: grrrrit: Catch all exceptions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328319 (owner: 10Legoktm) [01:28:18] (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328319 (owner: 10Legoktm) [01:30:34] (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328319 (owner: 10Legoktm) [01:39:41] PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [01:52:43] #mediawiki-feed, #wikimedia-labs: (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328319 (owner: 10Legoktm) [01:52:43] (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328319 (owner: 10Legoktm) [01:52:50] oh whoops [01:53:36] #mediawiki-feed, #wikimedia-labs: (03PS1) 10Legoktm: grrrrit: Remove debug stuff [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328320 [01:53:36] (03PS1) 10Legoktm: grrrrit: Remove debug stuff [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328320 [01:53:49] #mediawiki-feed, #wikimedia-labs: (03CR) 10Legoktm: [C: 032] grrrrit: Remove debug stuff [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328320 (owner: 10Legoktm) [01:53:49] (03CR) 10Legoktm: [C: 032] grrrrit: Remove debug stuff [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328320 (owner: 10Legoktm) [01:54:00] I'll shut off the other bot in a few minutes [01:54:13] #mediawiki-feed, #wikimedia-labs: (03Merged) 10jenkins-bot: grrrrit: Remove debug stuff [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328320 (owner: 10Legoktm) [01:54:13] (03Merged) 10jenkins-bot: grrrrit: Remove debug stuff [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328320 (owner: 10Legoktm) [01:54:29] legoktm ^^ thats confussing having wikibugs doing phabricator and gerrit under one username [01:55:06] could we have wikibugs have duplicate usersnames one for wikibugs and one that sends gerrit results to irc [01:56:05] maybe [01:56:20] + it also looks wierd it saying the project in the patch like #mediawiki-feed, #wikimedia-labs: [01:56:26] yes, that part was a bug [01:56:28] 06Labs, 07Puppet: Retire and remove module labs_debrepo - https://phabricator.wikimedia.org/T153612#2888907 (10scfc) @Multichill: Sorry for being too terse. `labs_debrepo` is used inter alia by WDQ in the form of `operations/puppet`'s [[https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/module... [01:56:29] Oh [01:56:41] legoktm i can switch grrrit-wm of for you? [01:56:44] of = off [01:56:49] I'll do that in a few minutes [01:57:10] Ok [01:57:42] probaly want to do the username first (just to prevent confusion) [01:58:01] mostly fighting with grid engine right now [01:58:14] oh [01:58:26] should we switch wikibugs over to kubenetes? [01:58:41] Kubernetes [01:59:21] probably, yuvi said he'd help me do that later [02:00:31] oh [02:00:32] :) [02:01:25] (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328320 (owner: 10Legoktm) [02:01:25] (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328320 (owner: 10Legoktm) [02:01:43] hm [02:01:45] dash [02:01:58] lol [02:02:17] legoktm try a merge too + some c+1, c+2 v+1 v+2 [02:03:10] (03CR) 10Paladox: [C: 031] Send nick serv password when you change the nick back to original [labs/tools/grrrit] 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:11] (03CR) 10Paladox: [C: 031] Send nick serv password when you change the nick back to original [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:23] (03CR) 10Paladox: [V: 031 C: 031] Send nick serv password when you change the nick back to original [labs/tools/grrrit] 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:25] (03CR) 10Paladox: [V: 031 C: 031] Send nick serv password when you change the nick back to original [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:29] (03CR) 10Paladox: [C: 031] Send nick serv password when you change the nick back to original [labs/tools/grrrit] 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:31] (03CR) 10Paladox: [C: 031] Send nick serv password when you change the nick back to original [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:39] (03CR) 10Paladox: [C: 04-1] Send nick serv password when you change the nick back to original [labs/tools/grrrit] 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:40] (03CR) 10Paladox: [C: 04-1] Send nick serv password when you change the nick back to original [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:43] (03CR) 10Paladox: [V: 04-1 C: 04-1] Send nick serv password when you change the nick back to original [labs/tools/grrrit] 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:46] (03CR) 10Paladox: [V: 04-1 C: 04-1] Send nick serv password when you change the nick back to original [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [02:03:56] (03PS1) 10Legoktm: grrrrit: Add a missing - [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328321 [02:03:56] (03PS1) 10Legoktm: grrrrit: Add a missing - [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328321 [02:04:25] legoktm looks like you fixed most of our issues [02:04:27] :_ [02:04:28] (03CR) 10Legoktm: [C: 032] grrrrit: Add a missing - [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328321 (owner: 10Legoktm) [02:04:29] :) [02:04:30] (03CR) 10Legoktm: [C: 032] grrrrit: Add a missing - [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328321 (owner: 10Legoktm) [02:04:33] :) [02:04:55] (03Merged) 10jenkins-bot: grrrrit: Add a missing - [labs/tools/wikibugs2] 10https://gerrit.wikimedia.org/r/328321 (owner: 10Legoktm) [02:04:56] (03Merged) 10jenkins-bot: grrrrit: Add a missing - [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328321 (owner: 10Legoktm) [02:05:35] but wikibugs needs a split username before grrrit-wm be taken offline (after you managed to get grid thingy working) [02:05:36] :) [02:08:02] legoktm run kubectl scale --replicas=0 deployment/grrrit [02:10:38] (03CR) 10Legoktm: "test" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328320 (owner: 10Legoktm) [02:10:57] I wonder how do we go about giving wikibugs two username like grrrrit and wikibugs please? [02:11:00] legoktm ^^ [02:11:53] I'm not going to do that right now [02:12:30] if other people find it confusing, I'll reconsider merging the two [02:12:45] ok [02:14:58] 10Wikibugs: Seperate wikibugs username - https://phabricator.wikimedia.org/T153732#2888932 (10Paladox) [02:15:04] 10Wikibugs: Seperate wikibugs username, into grrrrit and wikibugs - https://phabricator.wikimedia.org/T153732#2888947 (10Paladox) [02:15:13] just created ^^ for later. [02:15:19] Please don't [02:15:27] Oh sorry, why not [02:15:31] are you finding it confusing? [02:15:40] or do you think other people will find it confusing? [02:16:45] Well i find it confussing, manly doint like looking at seeing gerrit + phabricator changes under the same username. [02:17:14] I doint know what people would think, currently there is no complaints so it seems that no one minds. [02:17:19] ok, could you clarify that on the task? [02:17:22] Ok [02:17:47] 10Wikibugs: Seperate wikibugs username, into grrrrit and wikibugs - https://phabricator.wikimedia.org/T153732#2888950 (10Paladox) I manly doint like looking at seeing gerrit + phabricator changes under the same username. [02:17:56] but keeping them together makes things a lot easier, so unless other people also do complain I'm going to keep it as is [02:18:15] legoktm probaly also want to update the grrrit-wm repo to show the new wikibugs location [02:18:20] to update that yaml file. [02:19:56] 10Wikibugs: Seperate wikibugs username, into grrrrit and wikibugs - https://phabricator.wikimedia.org/T153732#2888955 (10Paladox) what is confussing is if wikibugs sends a couple of messages to irc and some are gerrit, it is now harder to pick out which is what. [02:27:52] (03PS1) 10Legoktm: Avoid having a comment of "(# comments)" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328325 [02:28:07] (03CR) 10Legoktm: [C: 032] Avoid having a comment of "(# comments)" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328325 (owner: 10Legoktm) [02:28:31] (03Merged) 10jenkins-bot: Avoid having a comment of "(# comments)" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328325 (owner: 10Legoktm) [02:31:25] hey, so there is a security upgrade for tomcat [02:31:36] i randomly grepped and [02:31:44] modules/role/manifests/toollabs/node/web/generic.pp: package { [ 'tomcat7-user', 'xmlstarlet' ]: [02:32:08] but that is just that -user package [02:32:14] hmm [02:39:43] RECOVERY - Puppet run on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0] [03:37:27] 10Tool-Labs-tools-meetbot: meetbot: when in #wikimedia-office, always add "this channel is logged" to topic - https://phabricator.wikimedia.org/T64271#2889042 (10scfc) a:05scfc>03None [03:38:14] 10Tool-Labs-tools-meetbot: Package MeetBot - https://phabricator.wikimedia.org/T63043#2889044 (10scfc) 05Open>03declined No longer interested, and it appears to be working as is. [03:46:31] 06Labs, 10wikitech.wikimedia.org: Job queue has 119663 entries - https://phabricator.wikimedia.org/T153618#2889046 (10scfc) Question of understanding: ``` 2016-12-19 07:26:41 refreshLinksPrioritized Nova_Resource:Captcha-apiproxy-03.privpol-captcha.eqiad.wmflabs rootJobTimestamp=20161219050057 useRecursiveLin... [03:53:46] 10Tool-Labs-tools-meetbot: meetbot: when in #wikimedia-office, always add "this channel is logged" to topic - https://phabricator.wikimedia.org/T64271#668054 (10bd808) I looked into this once. We would pretty much need to rewrite the bot to be able to customize the topic management. [03:54:35] 10Tool-Labs-tools-meetbot: meetbot: when in #wikimedia-office, always add "this channel is logged" to topic - https://phabricator.wikimedia.org/T64271#668054 (10yuvipanda) Maybe stashbot could take over meetbot duties :) [04:19:28] 06Labs, 10wikitech.wikimedia.org: Job queue has 119663 entries - https://phabricator.wikimedia.org/T153618#2889063 (10bd808) I would guess sometime around the introduction of https://gerrit.wikimedia.org/r/#/c/305453/ . The exact error message has changed since then, but that was when the protective check was... [05:38:43] 10Wikibugs: (grrrrit) Note (or maybe ignore) when a change is a rebase - https://phabricator.wikimedia.org/T59116#2889084 (10Legoktm) [05:40:55] 10Wikibugs: Don't echo on IRC when title change only - https://phabricator.wikimedia.org/T135363#2889087 (10Legoktm) 05Open>03declined No, I think those changes should be visible on IRC. [05:42:14] 10Wikibugs: Bot should use notices in-channel - https://phabricator.wikimedia.org/T72881#2889093 (10Legoktm) [05:45:03] 10Wikibugs: wikibugs is having issues joining channels on startup - https://phabricator.wikimedia.org/T92301#2889094 (10Legoktm) 05Open>03Resolved a:03Legoktm Fixed by {f09815aee08458b7fb283db7c7e0aed49e3b149d}. [05:46:08] 10Wikibugs, 13Patch-For-Review: Provide more / more detailed CTCP responses. - https://phabricator.wikimedia.org/T88070#2889098 (10Legoktm) 05Open>03Resolved a:03valhallasw [05:59:27] 10Wikibugs: wikibugs is having issues joining channels on startup - https://phabricator.wikimedia.org/T92301#1105754 (10MZMcBride) >>! In T92301#2889094, @Legoktm wrote: > Fixed by {f09815aee08458b7fb283db7c7e0aed49e3b149d}. A `FIXME HACK` resolves this task? Hmmm. [06:01:31] 10Wikibugs: wikibugs is having issues joining channels on startup - https://phabricator.wikimedia.org/T92301#2889118 (10Legoktm) I guess at this point the hack has lived on long enough that this bug is fixed, but eventually it could use some tech debt cleanup? [07:04:48] PROBLEM - Puppet run on tools-bastion-05 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [07:06:58] PROBLEM - Puppet run on tools-exec-1403 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:42:00] RECOVERY - Puppet run on tools-exec-1403 is OK: OK: Less than 1.00% above the threshold [0.0] [07:44:49] RECOVERY - Puppet run on tools-bastion-05 is OK: OK: Less than 1.00% above the threshold [0.0] [10:02:32] 06Labs, 10Labs-Infrastructure, 10DBA: Create a cronjob/check to run check_private_data data script and report back - https://phabricator.wikimedia.org/T153680#2889408 (10jcrespo) [10:02:40] 06Labs, 10DBA: Make watchlist table available as curated foo_p.watchlist_count on labsdb - https://phabricator.wikimedia.org/T59617#2889410 (10jcrespo) [10:09:38] 06Labs, 10DBA: Add and sanitize s2, s4, s5, s6 and s7 to sanitarium2 and new labsdb hosts - https://phabricator.wikimedia.org/T153743#2889412 (10Marostegui) [10:10:40] 06Labs, 10DBA: Add and sanitize s2, s4, s5, s6 and s7 to sanitarium2 and new labsdb hosts - https://phabricator.wikimedia.org/T153743#2889412 (10Marostegui) [10:59:08] (03PS1) 10Jean-Frédéric: Filter out blacklisted categories during categorisation [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/328348 (https://phabricator.wikimedia.org/T153744) [11:00:51] (03CR) 10jerkins-bot: [V: 04-1] Filter out blacklisted categories during categorisation [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/328348 (https://phabricator.wikimedia.org/T153744) (owner: 10Jean-Frédéric) [11:11:21] (03PS2) 10Jean-Frédéric: Filter out blacklisted categories during categorisation [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/328348 (https://phabricator.wikimedia.org/T153744) [12:02:10] 06Labs, 10Wikispeech: Request creation of Wikispeech labs project - https://phabricator.wikimedia.org/T153430#2889643 (10Lokal_Profil) >>! In T153430#2887641, @Andrew wrote: > by the way, your username is actually lokal-profil :) My bad. I read wikitech as meaning the wikitech wiki username not the shell acco... [13:02:01] (03PS1) 10Ema: add new fake entries for digicert-2016-{ecdsa,rsa}-unified.key [labs/private] - 10https://gerrit.wikimedia.org/r/328359 [13:04:42] (03CR) 10Ema: [V: 032 C: 032] add new fake entries for digicert-2016-{ecdsa,rsa}-unified.key [labs/private] - 10https://gerrit.wikimedia.org/r/328359 (owner: 10Ema) [14:47:09] (03CR) 10Jean-Frédéric: "@Gergő Does this make sense to you? Can I go ahead and merge this?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/320363 (https://phabricator.wikimedia.org/T137882) (owner: 10Jean-Frédéric) [14:52:14] (03PS3) 10Jean-Frédéric: Set-up Raven client for error reporting via Sentry [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/320363 (https://phabricator.wikimedia.org/T137882) [14:52:29] (03PS4) 10Jean-Frédéric: Set-up Raven client for error reporting via Sentry [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/320363 (https://phabricator.wikimedia.org/T137882) [15:05:27] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Naraht was created, changed by Naraht link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Naraht edit summary: Created page with "{{Tools Access Request |Justification=Extension of Database Reports (My first effort will be to duplicate the Polluted Categories script to view draft rather than user) |Compl..." [15:11:03] (03PS1) 10Lokal Profil: [WIP] Dissalow adding hidden categories [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/328378 (https://phabricator.wikimedia.org/T153746) [15:16:21] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Dissalow adding hidden categories [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/328378 (https://phabricator.wikimedia.org/T153746) (owner: 10Lokal Profil) [15:26:10] 06Labs, 10Labs-Infrastructure, 06Operations, 10netops, and 3 others: Provide read-only access to OpenStack APIs from WMF IP space - https://phabricator.wikimedia.org/T150092#2890030 (10Andrew) 05Open>03Resolved I can now do 'openstack project list' on a labs Jessie machine with addition of openstack::c... [15:26:44] 06Labs, 07Tracking: Existing Labs project quota increase requests (Tracking) - https://phabricator.wikimedia.org/T140904#2890036 (10Andrew) [15:26:46] 06Labs, 10Tool-Labs: Freenode sometimes throttles bot connections from tools - https://phabricator.wikimedia.org/T151704#2890033 (10Andrew) 05Open>03stalled a:05Andrew>03None No response from freenode [15:27:44] 06Labs: Announce sunsetting of Precise on Labs as of March 31st - https://phabricator.wikimedia.org/T153686#2890038 (10Andrew) 05Open>03Resolved Email sent, subject "Ubuntu instances on Labs will be shut down at the end of March" and then another email with subject "Correction: Ubuntu PRECISE instances on L... [15:28:39] 06Labs, 07LDAP: Clean up ldap host entries and references - https://phabricator.wikimedia.org/T148781#2890040 (10Andrew) p:05Triage>03Low [15:29:11] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Support project creation without OpenStackManager - https://phabricator.wikimedia.org/T150091#2890041 (10Andrew) p:05Triage>03Normal [15:58:47] 06Labs: Request increased quota for services-test labs project - https://phabricator.wikimedia.org/T153711#2890153 (10Eevans) This request is similar to {T148869}; When finished with the analysis, we can revert the quota. [16:31:36] 10Labs-project-Wikistats: all kinds of mixed issues with miraheze table (was: allthetropes is not updating on wikistats) - https://phabricator.wikimedia.org/T146712#2890226 (10Dzahn) >>! In T146712#2875124, @NDKilla wrote: > Please purge the statistics for all wikis mentioned in T146712#2856064 Done now. for m... [16:32:10] 10Labs-project-Wikistats: all kinds of mixed issues with miraheze table (was: allthetropes is not updating on wikistats) - https://phabricator.wikimedia.org/T146712#2890228 (10Dzahn) p:05Triage>03Normal [16:35:25] 10Labs-project-Wikistats: wikistats - make table columns sortable in display.php - https://phabricator.wikimedia.org/T153762#2890243 (10Dzahn) [16:36:05] 10Labs-project-Wikistats: wikistats - make table columns sortable in display.php - https://phabricator.wikimedia.org/T153762#2890259 (10Dzahn) p:05Triage>03Low [17:32:18] yuvipanda: your twitter avatar is very fierce [17:32:55] urandom: :D [17:33:08] urandom: ty! I was I think 16 in that [17:33:10] like a crouched panther or something [17:33:34] I think I was channelling 'awww fuck, I'm stuck in this city aaagh!' [18:30:11] (03PS3) 10BryanDavis: Display existing SSH keys [labs/striker] - 10https://gerrit.wikimedia.org/r/328058 (https://phabricator.wikimedia.org/T144711) [18:30:13] (03PS3) 10BryanDavis: Allow deleting SSH keys [labs/striker] - 10https://gerrit.wikimedia.org/r/328117 (https://phabricator.wikimedia.org/T144711) [18:30:15] (03PS1) 10BryanDavis: Add SSH key importing [labs/striker] - 10https://gerrit.wikimedia.org/r/328403 (https://phabricator.wikimedia.org/T144711) [18:31:47] (03CR) 10jerkins-bot: [V: 04-1] Add SSH key importing [labs/striker] - 10https://gerrit.wikimedia.org/r/328403 (https://phabricator.wikimedia.org/T144711) (owner: 10BryanDavis) [18:32:46] bd808_: legoktm I think the working 'jerkins-bot' change is my favorite feature of the rewrite [18:33:32] oh man, I hadn't even noticed [18:34:02] (03CR) 10BryanDavis: "(2 comments)" (032 comments) [labs/striker] - 10https://gerrit.wikimedia.org/r/328058 (https://phabricator.wikimedia.org/T144711) (owner: 10BryanDavis) [18:35:57] lol [18:38:02] what change? [18:38:53] "jenkins-bot" -> "jerkins-bot" :) [18:39:38] :O [18:43:01] yuvipanda: hahah yeah I snuck that in there :P [18:43:29] legoktm: what it was just a port of my code! I claim credit for the idea ;) [18:43:35] I just wrote shitty cod [18:43:48] legoktm hi, there's also a bug where it does (2 comments)" (2 comments) twice [18:44:03] hi paladox, it's still doing that? Hmm. [18:44:16] oh [18:44:21] https://github.com/wikimedia/labs-tools-wikibugs2/commit/8539c4bb54950aa70a9b3631e4e248b9fcccea07 <-- I'm really dumb [18:44:22] Yep [18:44:53] I found the bug [18:45:15] oh thanks [18:45:16] :) [18:49:08] (03PS4) 10BryanDavis: Allow deleting SSH keys [labs/striker] - 10https://gerrit.wikimedia.org/r/328117 (https://phabricator.wikimedia.org/T144711) [18:49:10] (03PS2) 10BryanDavis: Add SSH key importing [labs/striker] - 10https://gerrit.wikimedia.org/r/328403 (https://phabricator.wikimedia.org/T144711) [18:49:41] (03CR) 10BryanDavis: "(2 comments)" (032 comments) [labs/striker] - 10https://gerrit.wikimedia.org/r/328117 (https://phabricator.wikimedia.org/T144711) (owner: 10BryanDavis) [18:50:07] paladox: re: https://gerrit.wikimedia.org/r/#/c/326177/11 in PS11 i just added the variable in Hiera, so show what i mean [18:50:20] Ok [18:50:22] needs the actual lookup and be used in template [18:50:24] yeh, that looks fine [18:50:26] oh [18:50:28] but you know what i mean [18:50:32] yeh [18:52:12] mutante is lookup resolve? [18:52:35] @resolve(${prometheus_ferm_nodes}) [18:52:37] something like [18:52:38] tht [18:52:40] that [18:52:52] no, @resolve is a DNS lookup [18:52:57] you want hiera lookup [18:53:05] Oh [18:53:12] you want "how to looup from hiera and use in erb template" [18:53:26] it's like we did with other puppet stuff for phab , right [18:53:32] except you use a value in a template [18:54:05] hold on, i'll amend [18:54:09] Yep [18:54:30] hiera_lookup? [18:54:37] https://github.com/wikimedia/operations-puppet/blob/06fe8f125c6cf64efa89f095fd7554e33dda2a08/utils/hiera_lookup [18:56:11] (03PS1) 10Legoktm: Fix 8539c4bb54950 [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328406 [18:56:12] see this random example [18:56:12] modules/torrus/templates/varnish.xml.erb: role_nodes = scope.function_hiera(["cache::#{role}::nodes"]) [18:56:19] see how that is an .erb as well [18:56:22] and it gets stuff from hiera [18:56:27] there are different ways to do this [18:56:47] https://gerrit.wikimedia.org/r/#/c/324832/ [18:56:53] yep [18:56:54] oh [18:56:58] you can also do it in puppet, remember for phabricator.. $phab_servername = hiera('phabricator_servername', .. [18:57:34] yep [18:57:36] https://gerrit.wikimedia.org/r/#/c/324796/36/modules/role/manifests/phabricator/rsync.pp [18:57:40] and once you have it in .pp you can use it in .erb, or you can directly do it in .erb [18:57:51] oh [18:58:17] so we do it in jetty.pp (gerrit) and we can then use it in erb [19:03:43] yea, 2 ways or more to Rome [19:03:53] (03CR) 10Legoktm: [C: 032] Fix 8539c4bb54950 [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328406 (owner: 10Legoktm) [19:05:40] PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [19:06:00] (03Merged) 10jenkins-bot: Fix 8539c4bb54950 [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328406 (owner: 10Legoktm) [19:19:13] andrewbogott, maps-warper is also running 12.04 but was not on the list... I imagine because its listed as unknown? in the instance list. Will coordinate in the new year to migrate to an instance. it suffers from disk space issues currently so a new instance would be welcome. [19:19:55] hm… let me check my report query, maybe I'm missing a lot like those [19:21:15] yeah it has (missing) as the image ID on wikitech [19:23:23] oh, also in wikitech it's showing (deprecated 2014-06-10) for 14.04 Trusty. (instead of 2019) [19:23:38] bd808: it was 'renameuser_queue' the table we queried yesterday, right? [19:23:41] (deprecated) is when the image was deprecated, not the distro [19:23:47] chippy: but, that makes it sound like it's trusty [19:24:10] TabbyCat: yes [19:24:21] ahh I see [19:24:57] andrewbogott, sorry, i mean for other instances it's showing.... [19:25:06] chippy: ok [19:25:11] other instances in maps project. for maps-warper it's (missing) [19:25:23] hm, puppet is broken on maps-warper as well [19:25:48] yeah it seems like I have to do it manually from time to time [19:26:31] chippy: it includes a class that doesn't exist [19:26:39] * chippy boggles [19:28:13] doing puppet agent -t gives a permission error for an nfs directory too [19:28:53] there was an issue with nfs a few months ago, when I believe some work was being done on it, which someone fixed for it [20:15:41] RECOVERY - Puppet run on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0] [20:29:13] !log wikimania-support Deleted instance wikimania-scholarships (age: 2yrs, 9mo) [20:29:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikimania-support/SAL [20:30:38] !log wikimania-support Pointed wikimania-scholarships.wmflabs.org proxy at scholarships-01 [20:30:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikimania-support/SAL [20:35:47] !log wikimania-support enabled role::lockdown and cahnged admin password on scholarships-01 [20:35:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikimania-support/SAL [21:03:39] (03PS1) 10Legoktm: Archive [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328423 [21:04:56] (03CR) 10Legoktm: [V: 032 C: 032] Archive [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328423 (owner: 10Legoktm) [21:15:21] RECOVERY - Puppet run on tools-docker-builder-03 is OK: OK: Less than 1.00% above the threshold [0.0] [21:46:45] (03PS2) 10Lokal Profil: Disallow adding hidden categories [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/328378 (https://phabricator.wikimedia.org/T153746) [21:47:58] (03CR) 10jerkins-bot: [V: 04-1] Disallow adding hidden categories [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/328378 (https://phabricator.wikimedia.org/T153746) (owner: 10Lokal Profil) [22:18:27] 06Labs, 10Tool-Labs, 06Operations, 10Traffic, 07HTTPS: Migrate tools.wmflabs.org to https only (and set HSTS) - https://phabricator.wikimedia.org/T102367#2891325 (10abian) I've been using HTTPS Everywhere for a long time. This extension has [[ https://github.com/EFForg/https-everywhere/blob/master/src/ch... [22:21:22] 06Labs, 10Tool-Labs, 06Operations, 10Traffic, 07HTTPS: Migrate tools.wmflabs.org to https only (and set HSTS) - https://phabricator.wikimedia.org/T102367#2891340 (10bd808) I really think we could just flip the switch at the ingress proxy and then deal with the fallout. Mixed content warnings/errors are r... [22:24:52] 06Labs, 10Tool-Labs, 06Operations, 10Traffic, 07HTTPS: Migrate tools.wmflabs.org to https only (and set HSTS) - https://phabricator.wikimedia.org/T102367#2891349 (10bd808) >>! In T102367#2891340, @bd808 wrote: > I really think we could just flip the switch at the ingress proxy and then deal with the fall... [22:59:20] 06Labs, 10Tool-Labs, 06Operations, 10Traffic, 07HTTPS: Migrate tools.wmflabs.org to https only (and set HSTS) - https://phabricator.wikimedia.org/T102367#1363321 (10Legoktm) We could do the same thing as what we did in prod. Allow HTTP POST for a while, then make a percentage of requests fail, and then f... [23:00:11] (03Abandoned) 10Legoktm: Send nick serv password when you change the nick back to original [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328139 (owner: 10Paladox) [23:00:28] (03Abandoned) 10Legoktm: Remove grrrit-wm: force-restart [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/328025 (owner: 10Paladox) [23:00:54] (03Abandoned) 10Legoktm: [WIP] Attempting to send a quit message + add logging [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/323709 (owner: 10Paladox) [23:01:09] (03Abandoned) 10Legoktm: Adding quit message when issued command 'force-restart' [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/323342 (owner: 10Zppix) [23:01:23] (03Abandoned) 10Legoktm: grrrit-wm: Improve logging [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/320214 (owner: 10Paladox) [23:01:39] (03Abandoned) 10Legoktm: Update npm to npm 2 [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/309307 (owner: 10Paladox) [23:01:49] (03Abandoned) 10Legoktm: Add mediawiki/skins/Metrolook to #wikimedia-editing [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/292555 (owner: 10Paladox) [23:02:26] (03Abandoned) 10Legoktm: Don't duplicate things from #wikimedia-collaboration into -dev [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/230945 (owner: 10Jforrester) [23:02:31] (03Abandoned) 10Legoktm: Don't duplicate #wikimedia-collaboration notifications in #wikimedia-dev [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/241788 (owner: 10Catrope) [23:36:49] 06Labs: Cleanup leftover lucid instances on Labs - https://phabricator.wikimedia.org/T113199#2891577 (10scfc) 05Open>03Resolved The four instances mentioned in this task have been deleted: * https://wikitech.wikimedia.org/wiki/Nova_Resource:conventionextension-trial.conventionextension.eqiad.wmflabs * https... [23:47:33] 06Labs, 07Puppet: Migrate references from $instance.eqiad.wmflabs to $instance.$project.eqiad.wmflabs - https://phabricator.wikimedia.org/T153608#2891604 (10scfc)