[00:03:34] YuviPanda, are you around? [00:03:43] and on duty? [00:12:54] RECOVERY - Puppet failure on tools-exec-24 is OK: OK: Less than 1.00% above the threshold [0.0] [01:43:52] (03PS1) 10Alex Monk: Update regex for Wikipedia apps [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/201883 [02:08:36] 6Labs: Renaming scheme for labs servers - https://phabricator.wikimedia.org/T95042#1179702 (10RobH) Wasn't the plan to rename labsdb to labdb to match the standard of the other hostnames? Also the plan was to rename all virt to lab, right? [03:02:14] (03PS4) 10Tim Landscheidt: Tool Labs: Add .jsubrc support to jsub/jstart [labs/toollabs] - 10https://gerrit.wikimedia.org/r/156608 (https://phabricator.wikimedia.org/T56054) (owner: 10coren) [03:12:07] (03CR) 10Tim Landscheidt: [C: 032 V: 032] "Tested with the testsuite that the processing works and tested in Tools to see that it indeed reads ~/.jsubrc. I removed the macro from t" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/156608 (https://phabricator.wikimedia.org/T56054) (owner: 10coren) [03:12:56] (03CR) 10Tim Landscheidt: Tool Labs: Add .jsubrc support to jsub/jstart (031 comment) [labs/toollabs] - 10https://gerrit.wikimedia.org/r/156608 (https://phabricator.wikimedia.org/T56054) (owner: 10coren) [03:13:24] (03CR) 10Tim Landscheidt: "(Hadn't published the comment; that was one error I fixed.)" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/156608 (https://phabricator.wikimedia.org/T56054) (owner: 10coren) [03:32:41] (03CR) 10Tim Landscheidt: "And one thing only Trusty's lintian pointed out to me: "Thu, 17 Mar 2015 10:50:00 -0400" was a Tuesday :-)." [labs/toollabs] - 10https://gerrit.wikimedia.org/r/156608 (https://phabricator.wikimedia.org/T56054) (owner: 10coren) [03:53:57] 10Tool-Labs, 5Patch-For-Review: grid default output location - https://phabricator.wikimedia.org/T56054#1179758 (10scfc) 5Open>3Resolved [06:40:37] PROBLEM - Puppet failure on tools-exec-01 is CRITICAL: CRITICAL: 12.50% of data above the critical threshold [0.0] [06:41:21] PROBLEM - Puppet failure on tools-exec-15 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [07:01:11] RECOVERY - Puppet failure on tools-exec-15 is OK: OK: Less than 1.00% above the threshold [0.0] [07:10:39] RECOVERY - Puppet failure on tools-exec-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:37] 6Labs, 10Tool-Labs, 7Tracking: Make toollabs reliable enough (Tracking) - https://phabricator.wikimedia.org/T90534#1179830 (10Pine) @yuvipanda I was told by a Wikimetrics user that the reliability of Wikimetrics depends on Labs, so you may want to check with someone in Wikimetrics to verify whether or not th... [08:10:39] (03PS1) 10Tim Landscheidt: Fix undefined variable and similar warnings [labs/toollabs] - 10https://gerrit.wikimedia.org/r/201888 [08:12:58] (03CR) 10Tim Landscheidt: [C: 032 V: 032] "Tested live on Tools." [labs/toollabs] - 10https://gerrit.wikimedia.org/r/201888 (owner: 10Tim Landscheidt) [08:37:30] 6Labs, 10Tool-Labs, 7Tracking: Make toollabs reliable enough (Tracking) - https://phabricator.wikimedia.org/T90534#1179848 (10scfc) JFC. @yuvipanda wrote (probably based on some deeper knowledge) "Nope, wikimetrics isn't dependent on toollabs", you report hearsay by an unnamed user that Wikimetrics depends... [08:42:22] 6Labs, 10Tool-Labs, 7Tracking: Make toollabs reliable enough (Tracking) - https://phabricator.wikimedia.org/T90534#1179850 (10Pine) My intention was to ask for confirmation. I was not making an accusation. [08:44:51] PROBLEM - ToolLabs Home Page on toollabs is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Temporarily Unavailable - string 'Magnus' not found on 'http://tools.wmflabs.org:80/' - 401 bytes in 0.002 second response time [08:49:54] RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 760703 bytes in 3.066 second response time [09:03:18] 6Labs, 10Tool-Labs, 7Tracking: Make toollabs reliable enough (Tracking) - https://phabricator.wikimedia.org/T90534#1179854 (10scfc) Yes, that's the problem. No facts at all, just spreading fear and doubt. Why don't //you// ask "someone in Wikimetrics" if their service depends on Tool Labs? [10:09:46] 6Labs, 10Tool-Labs, 7Tracking: Make toollabs reliable enough (Tracking) - https://phabricator.wikimedia.org/T90534#1179855 (10Steinsplitter) Agree with @scfc. You need to ask "someone in Wikimetrics". Maybe it is just wikimetric's scripts... [11:47:06] 6Labs, 6operations, 5Patch-For-Review, 7Puppet, 7Regression: Puppet: "Package[gdb] is already declared in file modules/java/manifests/tools.pp" - https://phabricator.wikimedia.org/T94917#1179901 (10hashar) 5Open>3Resolved >>! In T94917#1179105, @Dzahn wrote: > merged. should be fixed now. wanna confi... [16:48:45] 10Tool-Labs: Configure web services in such a way that users don't have to (re)start it ever - https://phabricator.wikimedia.org/T94883#1180011 (10Ricordisamoa) FYI, some of my uwsgi tools were sometimes returning "Bad Request" errors even though their index pages seemed to work fine. Restarted webservices, prob... [17:56:46] Hey, anyone from admins around? [19:18:07] (03PS1) 10Tim Landscheidt: Remove debugging script [labs/toollabs] - 10https://gerrit.wikimedia.org/r/201904 [19:58:16] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/DZ was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=152216 edit summary: [20:42:03] Krinkle|detached: do you have a clue? https://translatewiki.net/wiki/Thread:Support/Translations_not_moving_to_Intuition [21:23:14] (03CR) 10Yuvipanda: [C: 032 V: 032] Remove debugging script [labs/toollabs] - 10https://gerrit.wikimedia.org/r/201904 (owner: 10Tim Landscheidt) [22:36:45] YuviPanda: what's the difference between the labs-l mailing list and wikitech-l? [22:38:09] Amir1: I’m around now. what’s up? [22:39:16] Negative24: one is for the labs infrastructure while the other is used for general development talk, tech announcements and production infrastructure talk [22:39:57] Subtle differences. [22:41:33] no [22:48:57] Krenair: Subtle differences on the page where they are described. Maybe not in practice. [23:11:34] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Liang860908 was modified, changed by Liang860908 link https://wikitech.wikimedia.org/w/index.php?diff=152223 edit summary: [23:16:31] 6Labs, 5Patch-For-Review, 3ToolLabs-Q4-Sprint-1: Comprehensive monitoring / alerting for labstore* instances - https://phabricator.wikimedia.org/T94606#1180244 (10yuvipanda) 5Open>3Resolved a:3yuvipanda Alright, so I'm going to consider this 'done' for now. More checks as warrented. [23:31:46] 6Labs, 10Tool-Labs, 7Tracking: Make toollabs reliable enough (Tracking) - https://phabricator.wikimedia.org/T90534#1180253 (10yuvipanda) @Pine @scfc since this seems to be a common enough confusion, I've created a page at https://wikitech.wikimedia.org/wiki/Labs_labs_labs with some rudimentary information ab... [23:40:03] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Liang860908 was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=152227 edit summary: [23:58:01] 10Tool-Labs: Some grid jobs are in odd state - https://phabricator.wikimedia.org/T95094#1180258 (10scfc) 3NEW