[00:31:41] 3Tool-Labs: Packages to be added to toollabs puppet - https://phabricator.wikimedia.org/T55704#986125 (10yuvipanda) [00:31:43] 3Tool-Labs: Install rake for Tools Labs - https://phabricator.wikimedia.org/T70208#986124 (10yuvipanda) 5Open>3stalled [00:40:33] 3Labs-Team: virt1002/labstore1001 network exhaustion - https://phabricator.wikimedia.org/T84003#986140 (10faidon) [01:02:10] 3Tool-Labs: Install valgrind on bastions - https://phabricator.wikimedia.org/T87117#986173 (10scfc) [01:02:13] 3Tool-Labs: Packages to be added to toollabs puppet - https://phabricator.wikimedia.org/T55704#986172 (10scfc) [01:06:36] 3Tool-Labs: Packages to be added to toollabs puppet - https://phabricator.wikimedia.org/T55704#986184 (10yuvipanda) [01:06:38] 3Tool-Labs: Install valgrind on bastions - https://phabricator.wikimedia.org/T87117#986182 (10yuvipanda) 5Open>3Resolved [01:17:36] 3Labs-Team: New disk partition scheme for labs instances - https://phabricator.wikimedia.org/T87003#986222 (10yuvipanda) [01:22:48] 3operations, Labs-Team: stray files created in /etc/ssh/userkeys - https://phabricator.wikimedia.org/T85814#986231 (10faidon) a:3faidon [01:25:33] 3Labs-Team: New disk partition scheme for labs instances - https://phabricator.wikimedia.org/T87003#986234 (10yuvipanda) p:5Normal>3High We should do this for Trusty and Precise too. I've to re-image deployment-jobrunner01 to be HAT... [02:30:21] !log deployment-prep created deployment-mediawiki04 to test roles [02:30:27] Logged the message, Master [03:46:03] 3Wikimedia-Labs-General: Replace role::labs::lvm::biglogs with increasing the default /var partition - https://phabricator.wikimedia.org/T69926#986339 (10scfc) [03:46:04] 3Labs-Team: New disk partition scheme for labs instances - https://phabricator.wikimedia.org/T87003#986340 (10scfc) [06:33:57] PROBLEM - Puppet failure on tools-exec-13 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [07:03:57] RECOVERY - Puppet failure on tools-exec-13 is OK: OK: Less than 1.00% above the threshold [0.0] [08:02:59] 3Labs-Team: Allow everyone in ops group in LDAP to login to all Labs instances - https://phabricator.wikimedia.org/T87094#986419 (10yuvipanda) @andrew thoughts? [08:30:55] 3Wikimedia-Labs-Infrastructure: Weird state of /data/project for dumps (semi-missing files) - https://phabricator.wikimedia.org/T87224#986466 (10Nemo_bis) 3NEW [11:44:30] Is something screwy with "crontab" on labs? [11:44:44] I type it, and it line returns, and then does nothing [12:07:58] clearly no one here, hopefully coren or yuvipanda or andrewbogott_afk can have a look at why crontab is hanging when they have time [16:38:01] 3Analytics-Engineering, Tool-Labs: Copy paid MaxMind geolocation dbs to tool labs - https://phabricator.wikimedia.org/T87151#986719 (10LuisV_WMF) I think I'm missing some context here. Few questions: 1. What's the general purpose of putting it on Labs? Just enabling volunteers or are there specific use cases? 2... [16:39:47] 3Analytics-Engineering, Tool-Labs: Copy paid MaxMind geolocation dbs to tool labs - https://phabricator.wikimedia.org/T87151#986721 (10yuvipanda) As for (2) the answer is 'any registered labs user', which is basically anyone (since anyone can register) [16:41:10] 3Analytics-Engineering, Tool-Labs: Copy paid MaxMind geolocation dbs to tool labs - https://phabricator.wikimedia.org/T87151#986731 (10Ironholds) 1. Yes! Volunteers are already building tools like https://tools.wmflabs.org/wikipedia-contributor-locations/#!/main using the (limited release) CC-BY-SA dataset. 2. A... [16:45:45] 3Tool-Labs-tools-Commons-Delinker: CommonsDelinker should not replace/remove images if a different version of the target exists locally - https://phabricator.wikimedia.org/T63656#986735 (10Steinsplitter) 5Open>3Resolved a:3Steinsplitter https://bitbucket.org/magnusmanske/commons-delinquent/src/75a8d483e795... [16:46:13] 3Tool-Labs-tools-Commons-Delinker: CommonsDelinker should not replace/remove images if a different version of the target exists locally - https://phabricator.wikimedia.org/T63656#986738 (10Steinsplitter) a:5Steinsplitter>3None [16:48:56] 3Tool-Labs-tools-Commons-Delinker: new version - https://phabricator.wikimedia.org/T70274#986742 (10Steinsplitter) 5Open>3Invalid a:3Steinsplitter commons delinker has been rewritten. [16:49:31] 3Tool-Labs-tools-Commons-Delinker: Live hacks to git - https://phabricator.wikimedia.org/T70273#986747 (10Steinsplitter) 5Open>3Invalid a:3Steinsplitter commons delinker has been rewritten by magnus. [16:49:32] 3Tool-Labs-tools-Commons-Delinker: new version - https://phabricator.wikimedia.org/T70274#986750 (10Steinsplitter) [16:50:01] 3Tool-Labs-tools-Commons-Delinker: when file deleted CommonsDelinker not always remove it from the wiki articles - https://phabricator.wikimedia.org/T85447#986753 (10Steinsplitter) 5Open>3Resolved a:3Steinsplitter commons delinker has been rewritten by magnus. should be resolved therfore. [16:51:40] 3Tool-Labs-tools-Commons-Delinker: COMMONSDELINKER-9 Never released lock in skipped pages handling - https://phabricator.wikimedia.org/T63679#986759 (10Steinsplitter) 5Open>3Invalid a:3Steinsplitter commons delinker has been rewritten by magnus. [20:17:30] YuviPanda, hello [20:18:37] i tried to websocket connect to a gridengine host, "but i get connection refused" is there a firewall issue there? [20:20:08] and if i nmap that machine from which i believe to be websocket broadacsting, i don't see the port i thought i opened to be open [20:42:24] hello all, can anyone add things to the crontab on the tool labs server? I tried editing the crontab for myself with "crontab -u musikanimal -e" and it says I am not privileged to use -u [21:18:11] 3Wikimedia-Labs-Other: [tracking] OSM on Labs - https://phabricator.wikimedia.org/T60797#987447 (10akosiaris) [21:18:12] 3Tool-Labs: Set up a tileserver for OSM in Labs - https://phabricator.wikimedia.org/T62819#987445 (10akosiaris) 5Open>3Resolved This is now actually done. The maps project exists in labs with multiple VMs named maps-tiles and maps-wma which serve the tiles under tiles.wmflabs.org and wma.wmflabs.org and resp... [21:30:35] MusikAnimal: are you using your tool account? [21:30:47] Betacommand: yes [21:31:42] or actually, when I've "become" another user [21:32:05] just use crontab -e [21:32:22] I'm safe to edit that? [21:32:49] it automatically uses the tool user account [21:33:04] Wonderful, all I need to know. Thank you! [21:37:45] successfully written, but it was automatically modified, adding "/usr/bin/jsub -N cron-tools.xtools-4 -once -quiet" [21:55:26] MusikAnimal: correct cron jobs are not allowed to be run on the login servers [21:56:16] 3Wikimedia-Labs-General: Fix virt1000 OAI errors - https://phabricator.wikimedia.org/T87079#987542 (10Aklapper) [@aaron: Please associate a project when creating tasks. Thanks! I assume this about Wikimedia-Labs-General, but no idea where OAI resides.] [22:01:42] Betacommand: So I am not going to be able to do this? I just want to have it automatically start the websever [22:01:46] *restart [22:04:15] MusikAnimal: you can just use https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Grid#Bigbrother to keep your server restarted [22:04:31] notconfusing: hey! I’m stuck in meetings all day :( but shouldn’t be firewall issues. [22:04:43] notawaffle: well, where are you nmapping from? [22:06:14] YuviPanda ok. So if I open up a port on a grid machine I should be able to see it? [22:07:36] notconfusing: not from the general internet, no. [22:08:12] notconfusing: were you nmapping from tools or from the internet? [22:08:31] YuviPanda from tools [22:08:33] YuviPanda: the problem here is the service does not stop, it just becomes unresponsive. I have "webservice" in my .bigbrotherrc [22:08:53] oh, hmm [22:09:43] YuviPanda so if we bind to 0.0.0.0 from a grid submitted process should that be visible to other Labs processes? [22:09:49] notconfusing: should be, yeah [22:10:02] notconfusing: I’m still in meetings all day :( [22:10:07] Thanks [22:10:09] it’s the ops offsite. [22:13:53] PROBLEM - Puppet failure on tools-webgrid-tomcat is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [22:43:56] RECOVERY - Puppet failure on tools-webgrid-tomcat is OK: OK: Less than 1.00% above the threshold [0.0] [22:54:25] edsu, if i want to pipe in different recentchanges dicts to wikistream then i think i just have to modify the "changes" function on line 95 of app.js in wikistream, right? [23:21:38] Coren or YuviPanda can one of you please start the webservice for https://tools.wmflabs.org/imagemapedit/ ? [23:22:04] oh, andrewbogott is here too ^ :) [23:22:36] matanya: we're in a meeting, all three arguing. But we'll pay attention in a bit [23:22:51] thanks andrewbogott :) [23:39:49] 3Labs-Team: Investigate replacing our custom DNS code with Designate - https://phabricator.wikimedia.org/T87280#987813 (10Andrew) 3NEW [23:40:18] 3Labs-Team: Install OpenStack Horizon for production labs - https://phabricator.wikimedia.org/T87279#987820 (10Andrew) [23:40:54] 3Labs-Team: Investigate replacing our custom DNS code with Designate - https://phabricator.wikimedia.org/T87280#987822 (10Andrew) We're hobbled in various ways by our custom OpenStackManager-embedded DNS code. Designate looks like a good candidate for a replacement. https://wiki.openstack.org/wiki/Designate [23:59:15] matanya: {{done}}