[00:08:47] 6Labs, 10Wikimedia-Site-requests, 10wikitech.wikimedia.org: Fix another broken module import - https://phabricator.wikimedia.org/T107726#1505127 (10scfc) (I had to null-edit [[https://wikitech.wikimedia.org/wiki/Help:Tool_Labs|Help:Tool Labs]] to get rid of "Script error: No such module 'Section link'".) [00:24:15] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Xavier Combelle was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=172947 edit summary: [01:43:20] I use a crontab to manage the execution of several scripts. I invoke these scripts via jsub and so it writes error logs to .err files in my home directory. These log entries have no timestamps and I have no idea when they happen. Is there any way to get an email every time an error happens? [01:45:48] My scripts have been broken for weeks and I have had no idea. [01:47:06] And in reading these .err files, because they lack timestamps I don't know if they are old errors resolved long ago or errors that are still extant. [03:31:02] PROBLEM - Puppet failure on tools-exec-1204 is CRITICAL 50.00% of data above the critical threshold [0.0] [04:11:01] RECOVERY - Puppet failure on tools-exec-1204 is OK Less than 1.00% above the threshold [0.0] [04:37:14] /var is full in language-cx instance. Any quick solution? [04:37:39] (it is just mediawiki + some extensions to run ContentTranslation) [05:12:41] kart_: probably log files in /var/log. check for /var/log/atop.log.?* [05:13:13] kart_: you can delete those without hurting anything at all. we do it on deployment-prep hosts pretty regularly [05:13:48] newer instances no longer have the tiny 2G /var partition. it was a bad idea [05:20:19] bd808: thanks! [05:31:07] RECOVERY - Host tools-static-01 is UPING OK - Packet loss = 0%, RTA = 0.91 ms [06:49:39] PROBLEM - Puppet failure on tools-exec-gift is CRITICAL 60.00% of data above the critical threshold [0.0] [07:24:38] RECOVERY - Puppet failure on tools-exec-gift is OK Less than 1.00% above the threshold [0.0] [09:00:21] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Kelson was created, changed by Kelson link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Kelson edit summary: Created page with "{{Tools Access Request |Justification=Further dev and admin of the "enwp10" tool. |Completed=false |User Name=Kelson }}" [09:59:56] hi! thanks for the welcome [10:21:10] 6Labs, 10Labs-Infrastructure: MySQL spectacularly slow on Labs - https://phabricator.wikimedia.org/T106372#1505922 (10Magnus) [10:21:11] 6Labs: wikidatawiki.labsdb user database is impressively slow - https://phabricator.wikimedia.org/T95276#1505923 (10Magnus) [11:48:44] 6Labs, 10Beta-Cluster: Have a public IP & DNS for Beta's HTTP proxy - https://phabricator.wikimedia.org/T107866#1506039 (10mobrovac) 3NEW [13:21:43] 6Labs: Labs Docker Project - https://phabricator.wikimedia.org/T106969#1506158 (10Negative24) [13:22:23] 6Labs: Labs Docker Project - https://phabricator.wikimedia.org/T106969#1483024 (10Negative24) @yuvipanda Do you have someone specific in mind that can be CCed on this task? [13:44:29] 6Labs, 10Beta-Cluster: Have a public IP & DNS for Beta's HTTP proxy - https://phabricator.wikimedia.org/T107866#1506208 (10akosiaris) Couple of questions: * Why would applications what to check if they are about to do an internal or external request ? Got a use case in mind ? * Even if there is a legitimate u... [13:58:31] 6Labs, 10Tool-Labs: webservicemonitor has no throttling - https://phabricator.wikimedia.org/T107876#1506255 (10scfc) 3NEW [14:00:18] 6Labs, 10Beta-Cluster: Have a public IP & DNS for Beta's HTTP proxy - https://phabricator.wikimedia.org/T107866#1506272 (10mobrovac) 5Open>3Invalid a:3mobrovac >>! In T107866#1506208, @akosiaris wrote: > Couple of questions: > > * Why would applications what to check if they are about to do an internal... [14:02:31] Coren: any maintenance on labstore going on? [14:02:50] mutante: Yeah, it's in the SAL. I'm doing a reinstall and test. [14:03:17] Coren: ok, cool [14:03:21] On 1001. Shouldn't affect anything else though. [14:03:51] yep. i just ask because Icinga [14:08:27] Hm. I should probably put it in maintenance so it stfu. [14:08:29] * Coren does that now. [14:10:16] thanks, i did it already, it will shut up until the next state change [14:11:00] Yeah, I'm putting a fixed period on it because I'll probably have to reboot a couple more times before I'm done. [14:12:00] ok, that's perfect then, thanks [14:13:24] Is there anyone about who could take a look at https://phabricator.wikimedia.org/P1814 for me please? [14:19:29] 6Labs, 10Tool-Labs: Limit webservice manifest restarts - https://phabricator.wikimedia.org/T107878#1506295 (10valhallasw) 3NEW [15:44:49] 6Labs, 10Tool-Labs: webservicemonitor has no throttling - https://phabricator.wikimedia.org/T107876#1506543 (10scfc) [15:44:51] 6Labs, 10Tool-Labs: Limit webservice manifest restarts - https://phabricator.wikimedia.org/T107878#1506545 (10scfc) [15:45:03] (03PS1) 10Southparkfan: web: require_once instead of include [labs/tools/WMT] - 10https://gerrit.wikimedia.org/r/229142 [15:50:36] 10Wikibugs: Add wikibugs to #wikimedia-rename - https://phabricator.wikimedia.org/T107418#1506559 (10Glaisher) a:3Glaisher Asked there and no one said no. [15:54:24] (03PS1) 10Glaisher: Send GlobalRename changes to #wikimedia-rename [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/229144 (https://phabricator.wikimedia.org/T107418) [15:59:26] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Kelson was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=173006 edit summary: [16:14:20] (03CR) 10Southparkfan: [C: 032 V: 032] web: require_once instead of include [labs/tools/WMT] - 10https://gerrit.wikimedia.org/r/229142 (owner: 10Southparkfan) [16:30:29] 6Labs, 10Labs-Infrastructure: wikidatawiki.labsdb user database is impressively slow - https://phabricator.wikimedia.org/T95276#1506728 (10Nemo_bis) [16:45:30] 6Labs, 10Analytics, 10Labs-Infrastructure, 3Labs-Sprint-108, 5Patch-For-Review: Set up cron job on labstore to rsync data from stat* boxes into labs. - https://phabricator.wikimedia.org/T107576#1506752 (10Ottomata) Oof, am looking at all that. I really am not excited about setting up another cron job +... [16:45:34] hello YuviPanda [16:50:28] 6Labs, 10Tool-Labs, 6Design Research Backlog, 6Learning-and-Evaluation, 6Research-and-Data: Organize a (annual?) toollabs survey - https://phabricator.wikimedia.org/T95155#1506762 (10leila) [16:56:06] leila: yo [16:56:55] so, can you give me the list of 5-6 programming languages you want to see in the Tool Labs survey when we ask people what programming languages they prefer to program in? [16:57:03] YuviPanda, ^ [16:57:39] oh, so tempting to only mention languages that I /want/ them to program in [16:57:52] :-) [17:01:14] I propose: python, php, bash, c#, ruby, perl, C [17:01:18] But let’s see what yuvi thinks [17:01:41] thanks andrewbogott. I put these and update if YuviPanda adds/drops. [17:02:02] big caveats on c# tho with mono afaik [17:02:19] probably better to put it last...with a link to a giant chasm of time suck [17:02:35] chasemp: I only mention because I’m pretty sure we already have someone (petr) using it. [17:21:28] leila: python2 php python3 nodejs Ruby perl other [17:22:27] ooki YuviPanda. thanks! [17:22:47] should have it ready for review in couple of hours. [17:35:56] leila: awesome! [17:47:27] (03CR) 10Legoktm: [C: 032] "Will continue to go to -dev since it is always used with CentralAuth." [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/229144 (https://phabricator.wikimedia.org/T107418) (owner: 10Glaisher) [17:47:48] (03Merged) 10jenkins-bot: Send GlobalRename changes to #wikimedia-rename [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/229144 (https://phabricator.wikimedia.org/T107418) (owner: 10Glaisher) [17:48:34] !log tools.wikibugs Updated channels.yaml to: 5b039d4ec16094a553be431e4d94f0bf880cfa47 Send GlobalRename changes to #wikimedia-rename [17:48:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL, Master [17:52:50] 10Wikibugs, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 5Patch-For-Review: Add wikibugs to #wikimedia-rename - https://phabricator.wikimedia.org/T107418#1506997 (10Legoktm) 5Open>3Resolved [17:53:20] 10Wikibugs, 10GlobalRename, 10MediaWiki-extensions-CentralAuth, 5Patch-For-Review: Add wikibugs to #wikimedia-rename - https://phabricator.wikimedia.org/T107418#1507006 (10Legoktm) ``` [10:52:53] --> wikibugs (tools.wiki@wikimedia/bot/pywikibugs) has joined #wikimedia-rename [10:52:54] GlobalRen... [19:11:55] 6Labs, 10Tool-Labs, 6Design Research Backlog, 6Learning-and-Evaluation, 6Research-and-Data: Organize a (annual?) toollabs survey - https://phabricator.wikimedia.org/T95155#1507483 (10leila) A first set of questions collected from Yuvi last week. A first draft is ready for Yuvi's review now (Yuvi, check y... [19:16:33] 6Labs, 10Tool-Labs, 6Design Research Backlog, 6Learning-and-Evaluation, 6Research-and-Data: Organize a (annual?) toollabs survey - https://phabricator.wikimedia.org/T95155#1507515 (10egalvezwmf) We have a few surveys going out this month, one for Grants consultation and another for Community Tech, which... [19:17:13] 6Labs, 10Tool-Labs, 6Design Research Backlog, 6Learning-and-Evaluation, 6Research-and-Data: Organize a (annual?) toollabs survey - https://phabricator.wikimedia.org/T95155#1507516 (10leila) [19:32:43] 6Labs, 10Tool-Labs, 6Design Research Backlog, 6Learning-and-Evaluation, 6Research-and-Data: Organize a (annual?) toollabs survey - https://phabricator.wikimedia.org/T95155#1507596 (10leila) @egalvezwmf Thanks for sharing the form. I did not know about it. It seems I should have asked you guys for helping... [19:46:42] 6Labs, 10Tool-Labs, 6Design Research Backlog, 6Learning-and-Evaluation, 6Research-and-Data: Organize a (annual?) toollabs survey - https://phabricator.wikimedia.org/T95155#1507636 (10egalvezwmf) @leila, @Capt_Swing, I just created the form yesterday, and will be a way for me to support WMF teams in desig... [20:08:09] 6Labs, 6operations: Investigate whether to use Debian's jessie-backports - https://phabricator.wikimedia.org/T107507#1507688 (10coren) @scfc Right now, we have one clearly used package from it but we've used backports (in general, not necessarily from a repo) before in production and in Labs so I think the que... [20:19:20] 6Labs, 10Tool-Labs, 6Design Research Backlog, 6Learning-and-Evaluation, 6Research-and-Data: Organize a (annual?) toollabs survey - https://phabricator.wikimedia.org/T95155#1507745 (10leila) [20:47:49] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1409 is CRITICAL 37.50% of data above the critical threshold [0.0] [21:12:42] 6Labs, 10Analytics, 10Labs-Infrastructure, 3Labs-Sprint-108, 5Patch-For-Review: Set up cron job on labstore to rsync data from stat* boxes into labs. - https://phabricator.wikimedia.org/T107576#1507951 (10yuvipanda) @Ottomata it isn't super hard to do, I can write it up if you'd like :) I think the 'one... [21:22:49] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1409 is OK Less than 1.00% above the threshold [0.0] [21:50:48] (03PS1) 10BearND: Switched to new server and new folder structure [labs/tools/wikipedia-android-builds] - 10https://gerrit.wikimedia.org/r/229263 [21:56:09] (03CR) 10BearND: Switched to new server and new folder structure (033 comments) [labs/tools/wikipedia-android-builds] - 10https://gerrit.wikimedia.org/r/229263 (owner: 10BearND) [22:03:38] (03CR) 10Niedzielski: [C: 031] "lg2m" [labs/tools/wikipedia-android-builds] - 10https://gerrit.wikimedia.org/r/229263 (owner: 10BearND) [22:15:26] niedzielski: https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/Wikipedia_Android_app_hacking is updated for new folder structure [22:15:47] bearND: thanks! [22:18:40] dbrant|bbl: niedzielski : ready to send the new production apk to TSG? [22:20:10] bearND dbrant|bbl: i don't have any objections. i think we got pretty good coverage of the beta yesterday. [22:44:12] 6Labs, 3Labs-Sprint-107, 3Labs-Sprint-108: Evaluate a 'cluster solution' for use on Tool Labs - https://phabricator.wikimedia.org/T106475#1508318 (10yuvipanda) >>! In T106475#1499162, @valhallasw wrote: > # **Have a healthy development community** > # **Have substantial adoption** > # **Easy configurability/... [22:56:05] 6Labs, 3Labs-Sprint-107, 3Labs-Sprint-108: Evaluate a 'cluster solution' for use on Tool Labs - https://phabricator.wikimedia.org/T106475#1508352 (10yuvipanda) [22:57:00] 6Labs, 3Labs-Sprint-107, 3Labs-Sprint-108: Evaluate a 'cluster solution' for use on Tool Labs - https://phabricator.wikimedia.org/T106475#1469900 (10yuvipanda) [22:57:14] 6Labs, 3Labs-Sprint-107, 3Labs-Sprint-108: Evaluate a 'cluster solution' for use on Tool Labs - https://phabricator.wikimedia.org/T106475#1469900 (10yuvipanda) Updated based on feedback.