[00:03:16] RECOVERY - Puppet run on tools-logs-01 is OK: OK: Less than 1.00% above the threshold [0.0] [00:03:25] 06Labs, 10Tool-Labs, 10WLX-Jury, 13Patch-For-Review, 10Wiki-Loves-Monuments (2016): Figure out a way to support java 1.8 on tool labs (For WLX Jury) - https://phabricator.wikimedia.org/T124903#2500725 (10yuvipanda) Hi @intracer! User @Sn1per has spent some time stting up and documenting how to run a Play... [00:08:55] (03PS1) 10BryanDavis: Change scap git_repo to deploy server repo path [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/301524 [00:09:22] (03CR) 10BryanDavis: [C: 032 V: 032] Change scap git_repo to deploy server repo path [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/301524 (owner: 10BryanDavis) [00:19:27] how does the role list on e.g. https://wikitech.wikimedia.org/w/index.php?title=Special:NovaInstance&action=configure&instanceid=7ebb2447-7c74-4fd1-aa5e-19c48011e39d&project=deployment-prep®ion=eqiad work? [00:19:49] him TimStarling [00:20:20] *hi [00:20:21] where does it come from, how do I get my role to be on it, how is it applied to the host? [00:20:22] you can manage it from https://wikitech.wikimedia.org/wiki/Main_Page [00:20:38] bah, wrong link :| [00:20:50] so you can edit them from https://wikitech.wikimedia.org/wiki/Special:NovaPuppetGroup [00:21:04] you can get your role to be on on that special page [00:21:13] and then it'll show up in the link you provided earlier. [00:21:22] ok, thanks [00:21:27] yw [00:23:00] and this will be applied next time puppet runs on the host? [00:23:46] looks like it is [00:23:50] TimStarling yup. so 1. add it in Special:NovaPuppetGroup, 2. Select it in Special:NovaInstance / Configure 3. wait for puppet run [00:24:18] I just ran puppet manually, and it failed because of an error in the class [00:24:32] right [00:24:34] yeah, that works too [00:24:35] which is fine, I am mostly doing this to test the puppet manifest [00:24:53] the node name -> list of roles is maintained via LDAP just now [00:25:24] so the puppetmaster queries LDAP every time? [00:26:19] yep [00:26:52] we're in the process of moving it to a mysql backed python service, mostly for the flexibility (Puppet's LDAP backend requires a very specific format) [00:29:51] I saw that icinga has been removed from labs in favour of shinken [00:30:25] labs never really had a working icinga setup that wasn't hand-built / worked only in a small subset of cases. [00:30:41] it has a shinken install, but it's setup pretty much completely different from prod [00:30:59] and is pretty much useless for testing actual monitoring changes [00:31:06] well, actual monitoring changes meant for prod [00:31:14] ok [00:31:26] (is useful for other things, like alerting us about puppet failures / disk space issues / ssh / etc) [00:49:15] PROBLEM - Puppet run on tools-logs-01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [01:14:13] RECOVERY - Puppet run on tools-logs-01 is OK: OK: Less than 1.00% above the threshold [0.0] [01:42:37] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Aggregate system logs from kubernetes nodes - https://phabricator.wikimedia.org/T141270#2500815 (10yuvipanda) 05Open>03Resolved Done and documented in https://wikitech.wikimedia.org/wiki/Tools_Kubernetes#Infrastructure_centralized_Logging [01:48:30] 06Labs, 10Tool-Labs, 10Wikimedia-Logstash: Setup ELK based logging for tool labs infrastructure components - https://phabricator.wikimedia.org/T141500#2500822 (10yuvipanda) [01:48:37] 06Labs, 10Tool-Labs, 10Wikimedia-Logstash: Setup ELK based logging for tool labs infrastructure components - https://phabricator.wikimedia.org/T141500#2500837 (10yuvipanda) p:05Triage>03Normal [02:33:48] 10Tool-Labs, 07Tracking: [tracking] Block spider / web crawler on tool labs - https://phabricator.wikimedia.org/T70300#2500875 (10Phabricator_maintenance) [02:34:07] 06Labs, 10Labs-Infrastructure, 07Tracking: Migrate Labs from pmtpa to eqiad data-center (tracking) - https://phabricator.wikimedia.org/T64042#2500888 (10Phabricator_maintenance) [02:35:33] 06Labs, 10Wikimedia-Labs-General, 10DBA, 06Operations, 07Tracking: Database replication services (tracking) - https://phabricator.wikimedia.org/T50930#2500925 (10Phabricator_maintenance) [02:41:17] !log wikibugs killed the irc part for a bit to suppress the spam from batch edit [02:41:18] Did you mean tools.wikibugs instead of wikibugs? [02:41:18] wikibugs is not a valid project. [02:41:28] !log tools.wikibugs killed the irc part for a bit to suppress the spam from batch edit [02:41:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL, Master [02:44:46] !log tools.wikibugs started the irc part [02:44:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL, Master [04:18:21] !log tools.xtools Restarted webservice due to 503 errors [04:18:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.xtools/SAL, Master [04:39:14] 10Tool-Labs-tools-Xtools: Unable to login to X!'s Tools - https://phabricator.wikimedia.org/T132855#2501110 (10Matthewrbowker) 05Open>03Resolved I can't reproduce the error. If you see it again, feel free to re-open this task with more information.` [13:05:37] 06Labs, 10Labs-Kubernetes: Tools failing with 500 - https://phabricator.wikimedia.org/T141459#2501697 (10Bugreporter) autolist also affected. [13:09:50] bd808: On a whim I tried the kubernetes webservice thing. Apparently it doesn't have the same packages installed, failed with "Can't locate JSON.pm in @INC". [13:15:28] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Billcountry was created, changed by Billcountry link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Billcountry edit summary: Created page with "{{Tools Access Request |Justification=Am a developer exploring |Completed=false |User Name=Billcountry }}" [14:28:41] 10PAWS, 10Revision-Scoring-As-A-Service-Backlog: Install revscoring inside PAWS - https://phabricator.wikimedia.org/T120317#2501872 (10Halfak) We don't need to do this because we can use magic commands to install things through pip inside of a PAWS instance. [14:28:47] 10PAWS, 10Revision-Scoring-As-A-Service-Backlog: Install revscoring inside PAWS - https://phabricator.wikimedia.org/T120317#2501873 (10Halfak) 05Open>03declined [15:04:35] anomie: Yeah, there isn't a Perl container yet. The default one is really stripped down for PHP. Yuvi has been working on some other languages. [15:05:17] We need to make a perl image in https://gerrit.wikimedia.org/r/#/admin/projects/operations/docker-images/toollabs-images for tools like yours [15:05:18] bd808: My case is weird: it uses PHP that executes a command-line perl script and dumps the output. [15:05:42] heh. The "fat" images are coming at some point too I think [15:06:02] Does Tool Labs or Labs work with Docker? [15:06:07] the plan last I heard was to make an image that matched the grid setup [15:06:32] Ohhhh, Watching :) [15:06:53] d3r1ck: sort of, but not generally yet. The new kubernetes system runs docker images. The is no facility for a user running an arbitrary image yet though [15:08:06] bd808: Ohhhh, cool. [15:08:13] d3r1ck: the platform as a service future will probably run docker or some similar containers. See T136265 and T136264 [15:08:14] T136265: Develop evaluation criteria for comparing Platform as a Service (PaaS) solutions - https://phabricator.wikimedia.org/T136265 [15:08:14] T136264: Evaluate Kubernetes based workflow replacement options for SGE - https://phabricator.wikimedia.org/T136264 [15:09:06] and if you have ideas about PaaS from things you read/used/dreamed about please add criteria to the T136265 ticket [15:09:06] T136265: Develop evaluation criteria for comparing Platform as a Service (PaaS) solutions - https://phabricator.wikimedia.org/T136265 [15:09:45] playing around on the free tier of heroku or google compute engine may give you ideas [15:10:27] bd808: Ok, thanks to k8s :) [15:10:38] and Yuvi :) [15:22:28] 06Labs, 06Operations: labnet100[12].eqiad.wmnet need to be reimaged with RAID - https://phabricator.wikimedia.org/T136718#2502029 (10Andrew) labnet1001 is reimaged now. The steps are a bit weird due to unusual networking setup: The 10g nic can't support pxe-booting. So to install you have to go into the bio... [15:34:26] 06Labs: confirm that new base labs base image is adequate for kubernetes &c. - https://phabricator.wikimedia.org/T134944#2502053 (10Andrew) @yuvipanda, is there a quick cmdline test to check whether cgroups are or aren't enabled? [15:34:49] 06Labs, 10Horizon, 07Tracking: Make OpenStack Horizon useful for production labs - https://phabricator.wikimedia.org/T87279#2502054 (10Andrew) a:05Andrew>03None [15:41:46] 06Labs, 10Horizon, 13Patch-For-Review: Create puppet backend with REST api for labs instance configuration - https://phabricator.wikimedia.org/T133412#2231107 (10AlexMonk-WMF) @Andrew: Please see Yuvi's question above [15:41:55] PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [16:15:57] anomie yeah, I don't have a clear solution for that use case [16:16:15] there'll definitely be a 'fat' container for running trusty images at some point, but that won't get all the new versions from jessie [16:19:59] I suppose I could try to figure out how whatever-perl-does-for-webpages works. I've just been too lazy to get to it, since this is just one page to dump some bot status information. [16:21:54] RECOVERY - Puppet run on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:23:13] anomie right. [16:24:19] bd808 do you want to use this as an opportunity to build an image? :) but I guess since you've hacked on build.py you already know... [16:25:51] madhuvishy do you? ^ [16:25:53] I at least know how the Dockerfiles get created. I should make an image but probably not today. Trying to untangle uwsgi puppet [16:26:41] bd808 right, funky. [16:26:54] YuviPanda: sure. Can I do it tomorrow? [16:27:04] YuviPanda: my uwsgi patch :( [16:27:26] madhuvishy sure. [16:27:31] madhuvishy uh oh, I know... [16:27:42] let's try to have you merge and babysit it? [16:27:49] then you can go through the babysitting a patch process :D [16:27:57] Yeah, need to manually rebase I think [16:28:11] right [16:30:49] YuviPanda: when office? I'll probably come in an hour or two [16:31:56] madhuvishy probably in a couple of hours. I'll be there before lunch, for real this time [16:39:53] did someone just kill a process of mine? [16:40:11] YuviPanda: ^ [16:40:12] or is it possible that it gets killed by other means? [16:40:24] YuviPanda: there's metrics today at 11 [16:40:27] gifti I've killed your tcl process as well, was taking up 100% CPU and seems to have been running for a while. [16:40:32] sorry, I told you in the wrong channel [16:40:32] hm [16:40:51] that I'm not in, yeah [16:41:13] I'll try it on the grid [16:41:20] I see taxonbot's allcat.tcl has been running for a long time as well, so I'm going to kill that too [16:41:29] doctaxon: ^ [16:47:50] 06Labs, 10Horizon, 13Patch-For-Review: Create puppet backend with REST api for labs instance configuration - https://phabricator.wikimedia.org/T133412#2502313 (10Andrew) The gui prototype is on labtesthorizon.wikimedia.org and the code is on labtestweb2001.wikimedia.org in /usr/lib/python2.7/dist-packages/wi... [16:50:40] 06Labs, 10Horizon, 13Patch-For-Review: Create puppet backend with REST api for labs instance configuration - https://phabricator.wikimedia.org/T133412#2502355 (10Andrew) Scratch that -- that 'too many connections' error with mysql is a thing I've been seeing with other services as well. So I'm pretty sure i... [16:51:23] 06Labs, 10Graphite: Install WmfPageview datasource plugin on Labs Grafana install - https://phabricator.wikimedia.org/T120298#2502358 (10Addshore) 05Open>03declined Not going to do. https://tools.wmflabs.org/pageviews has developed well and is great! :) [16:51:31] 06Labs, 10Horizon, 13Patch-For-Review: Create puppet backend with REST api for labs instance configuration - https://phabricator.wikimedia.org/T133412#2502361 (10yuvipanda) I suspect grants have a restricted connection count? [16:52:07] andrewbogott madhuvishy chasemp since someone asked for a naming convention setup for tools/k8s, I started one at https://wikitech.wikimedia.org/wiki/Tools_Kubernetes#Node_naming_conventions [16:52:16] I'll probably move it out to its own page soon since it has gridengine stuff there too now [16:54:06] edits / additions welcome [16:59:30] my current highest priority task is: Investigate docker storage drivers other than our current ones (to deal with hung kernels), and find ways to deploy node-problem detector [16:59:40] both dealing with kernel hangs [17:13:54] 06Labs, 10DBA, 06Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2502450 (10jcrespo) 05Open>03Resolved labsdb1002 itself has been decomissioned; the tracking of the new server setup (and labs fix in general) will be done at T140452 [17:55:00] (03PS1) 10BBlack: add fake ecc-star.planet key [labs/private] - 10https://gerrit.wikimedia.org/r/301654 [17:55:12] (03CR) 10BBlack: [C: 032 V: 032] add fake ecc-star.planet key [labs/private] - 10https://gerrit.wikimedia.org/r/301654 (owner: 10BBlack) [18:05:41] anomie looks like there's http://plackperl.org/ for perl, I'll try to build a simple thing that supports it in a nice and modern way :) [18:17:40] 10Tool-Labs-tools-Pageviews: Add option to view all redirects of a page to Massviews - https://phabricator.wikimedia.org/T141552#2502775 (10MusikAnimal) [18:27:30] 06Labs, 10Labs-Infrastructure, 10DBA: Users can't run EXPLAIN queries to check the theoretical efficiency of their SQL - https://phabricator.wikimedia.org/T141095#2502814 (10zhuyifei1999) [18:27:33] 06Labs, 10Tool-Labs, 07Upstream: Unable to explain queries on replicated databases - https://phabricator.wikimedia.org/T50875#2502817 (10zhuyifei1999) [18:27:44] 06Labs, 10Wikimedia-Labs-General, 10DBA, 06Operations, 07Tracking: Database replication services (tracking) - https://phabricator.wikimedia.org/T50930#2502820 (10zhuyifei1999) [18:27:46] 06Labs, 10Tool-Labs, 07Upstream: Unable to explain queries on replicated databases - https://phabricator.wikimedia.org/T50875#522161 (10zhuyifei1999) 05Resolved>03Open [18:33:40] 06Labs, 10Labs-Infrastructure, 10DBA: Users can't run EXPLAIN queries to check the theoretical efficiency of their SQL - https://phabricator.wikimedia.org/T141095#2502834 (10zhuyifei1999) [18:33:42] 06Labs, 10Labs-Infrastructure, 10DBA, 07Epic, 07Tracking: Labs databases rearchitecture (tracking) - https://phabricator.wikimedia.org/T140788#2502833 (10zhuyifei1999) [18:33:44] 06Labs, 10Tool-Labs, 07Upstream: Unable to explain queries on replicated databases - https://phabricator.wikimedia.org/T50875#2502835 (10zhuyifei1999) [19:03:20] (03PS1) 10BryanDavis: Add scap3 check to create/update virtualenv [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/301662 [19:19:58] (03CR) 10Thcipriani: [C: 031] "Should work fine. This is very similar to what ORES is currently doing (as noted in the commit). The checks system will become stronger in" [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/301662 (owner: 10BryanDavis) [19:33:36] !log gerrit http://gerrit-test.wmflabs.org/gerrit/ is down [19:33:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Gerrit/SAL, Master [19:53:47] can we have a bot to restart other bots :) [19:54:26] !gerrit-wm [19:54:26] ircecho: http://svn.wikimedia.org/viewvc/mediawiki/trunk/debs/ircecho/ [19:54:33] mutante ^^ [19:54:36] should be updated [19:54:42] lol [19:56:07] !gerrit-wm del [19:56:08] Successfully removed gerrit-wm [19:56:21] !log git starting gerrit-test [19:56:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master [19:56:52] mutante: chatops for tools! [19:57:05] !grrrit-wm [19:57:05] The dark knight of all bots, with no fear, brave and written in... um... javascript [19:57:09] lol [19:57:15] LOL [19:57:25] bd808: ? [19:57:39] oh, yes [19:57:53] a slackbot to run your whole network [19:58:38] mutante why not create a !grrrit-wm restart [19:58:43] !grrrit-wm restart [19:58:44] The dark knight of all bots, with no fear, brave and written in... um... javascript [19:58:44] it's so buzz worthy that there are even conferences about it [19:58:51] :) i especially like it when one bot talks to another bot [19:58:52] hehe [19:59:15] !log git gerrit-test started with bin/gerrit.sh run [19:59:17] !grrrit-wm del [19:59:18] Successfully removed grrrit-wm [19:59:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master [19:59:45] !grrrit-wm The dark knight of all bots, with no fear, brave and written in... um... javascript | how to restart https://wikitech.wikimedia.org/wiki/Grrrit-wm#Deploying_or_restarting [20:00:00] !grrrit-wm is The dark knight of all bots, with no fear, brave and written in... um... javascript | how to restart https://wikitech.wikimedia.org/wiki/Grrrit-wm#Deploying_or_restarting [20:00:01] Key was added [20:00:08] !log git taking gerrit-test down again so i can fix the regix of its-phabricator [20:00:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master [20:00:32] !gerrit-wm is did you mean grrrit-wm? [20:00:33] Key was added [20:01:10] paladox: ^ on wikitech we redirect the page [20:01:16] Oh [20:01:20] is there another gerrit-wm ? [20:01:28] the reviewer bot from mediawiki? [20:01:32] Maybe not sure though [20:04:48] !log git started gerrit-test again with another fix [20:04:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master [20:05:14] (03PS1) 10BryanDavis: checks/virtualenv: Don't fail when virtualenv exists [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/301669 [20:05:30] (03CR) 10BryanDavis: [C: 032 V: 032] checks/virtualenv: Don't fail when virtualenv exists [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/301669 (owner: 10BryanDavis) [20:59:50] !log git taking gerrit-test down again to test its-phabricator bot [20:59:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master [21:00:22] 10Wikibugs: Don't notify actions by @Phabricator_maintenance - https://phabricator.wikimedia.org/T141570#2503358 (10Danny_B) [21:08:19] !log git starting gerrit-test seeing if it will log to phabricator (phab-01) [21:08:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master [21:34:48] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Add a nice and simple perl5 webservice type based on modern perl5 principles - https://phabricator.wikimedia.org/T141572#2503479 (10yuvipanda) [21:35:15] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Add a nice and simple perl5 webservice type based on modern perl5 principles - https://phabricator.wikimedia.org/T141572#2503492 (10yuvipanda) [21:35:31] anomie ^ for perl5, I'll probably have something next week [21:36:34] not sure if too early for perl6 or what [21:50:28] 10Tool-Labs-tools-Xtools: Unable to login to X!'s Tools - https://phabricator.wikimedia.org/T132855#2503537 (10He7d3r) 05Resolved>03Open It is still happening, but I needed to click on the link a second time (because at first nothing changed after clicking allow: i.e. I was not logged in apparently). [21:50:52] YuviPanda: perl6 is pretty much a whole new language. It might be neat to support but it won't see a lot of use IMO [21:51:36] bd808 yeah, I think it's mostly a 'yuvi likes it', but I'll probably not actually work on it until someone else asks [21:52:07] * bd808 asks for a brainfuck container [21:52:48] 10Tool-Labs-tools-Xtools: Unable to login to X!'s Tools - https://phabricator.wikimedia.org/T132855#2503543 (10Matthewrbowker) Ah. I can explain that. When we authorize oauth, we authorize it for "http://tools.wmflabs.org/xtools" This is due to the way that OAuth is configured. Therefore, the authorization d... [21:52:50] bd808 if you write a webservice on it sure i'll make you one ;) [21:53:18] I'd have to transpile so .. cheating [21:54:10] bd808 but even if you transpile, you'll need some form of sockets... [21:54:15] I don't know if the transpiler lets you syscall [21:54:22] I guess depending on what you're transpiling *to* [21:54:25] or from [21:56:04] YuviPanda: lol. https://esolangs.org/wiki/Brainfuck%2B%2B [21:56:12] adds sockets and file handles [21:56:36] O_O [21:57:42] bd808 no turing complete metatemplate language? doesn't deserve to be ++ [21:59:04] YuviPanda: I think brainfucker is uring complete… [21:59:47] Platonides sure but BrainFuck++ doesn't add an additional layer of turing complete metaprogramming that can happen in the compiler [22:00:06] (http://stackoverflow.com/questions/189172/c-templates-turing-complete) [22:00:25] see also: wikitext [22:03:05] "Is this fact useful in practice? [22:03:11] 10Tool-Labs-tools-Xtools: Unable to login to X!'s Tools - https://phabricator.wikimedia.org/T132855#2212237 (10MusikAnimal) Do we even need OAuth anymore? I believe it was introduced for XEcho, and cross-wiki notifications are native now. [22:03:11] "only if you are a template daemon" xD [22:04:47] :D [22:17:13] 06Labs, 10Incident-20151216-Labs-NFS, 06Operations, 05Wikimedia-Incident: Investigate need and candidate for labstore100(1|2) kernel upgrade - https://phabricator.wikimedia.org/T121903#2503651 (10greg) [22:17:17] 06Labs, 10Incident-20151216-Labs-NFS, 06Operations, 05Wikimedia-Incident: Investigate better way of deferring activation of Labs LVM volumes (and corresponding snapshots) until after system boot - https://phabricator.wikimedia.org/T121629#2503654 (10greg) [22:17:26] 06Labs, 10Incident-20150617-LabsNFSOutage, 05Wikimedia-Incident: Labs: investigate alternatives to maps' storage requirements - https://phabricator.wikimedia.org/T103264#2503657 (10greg) [22:17:30] 06Labs, 10Labs-Sprint-102, 10Labs-Sprint-103, 10Labs-Sprint-104, and 4 others: Audit projects' use of NFS, and remove it where not necessary - https://phabricator.wikimedia.org/T102240#2503659 (10greg) [22:17:33] 06Labs, 10Labs-Infrastructure, 06DC-Ops, 10Incident-20151216-Labs-NFS, and 3 others: labstore1002 issues while trying to reboot - https://phabricator.wikimedia.org/T98183#2503660 (10greg) [22:17:35] 06Labs, 10Labs-Q4-Sprint-1, 10Incident-20150331-LabsNFS-Filesystem-Switch, 05Wikimedia-Incident: Create a simple checklist to follow for announcing / doing planned maintenance (on labs) - https://phabricator.wikimedia.org/T94608#2503665 (10greg) [22:17:37] 06Labs, 10Incident-20150331-LabsNFS-Filesystem-Switch, 05Wikimedia-Incident: Create scripts to help stagger restarts of labs VMs by different criteria - https://phabricator.wikimedia.org/T94613#2503664 (10greg) [22:34:58] 10Tool-Labs-tools-Pageviews: Add option to view all redirects of a page to Massviews - https://phabricator.wikimedia.org/T141552#2503770 (10MusikAnimal) a:03MusikAnimal [22:56:04] 06Labs: Request creation of readmore labs project - https://phabricator.wikimedia.org/T139863#2444900 (10yuvipanda) I've helped ellery set this up on toollabs, but the webservice seems to need at least 6G of RAM to work fine. [22:57:17] 06Labs, 13Patch-For-Review: Kill ldapsupportlib.py - https://phabricator.wikimedia.org/T114063#1683588 (10Dzahn) I just used ldaplist on terbium and saw the warning: If you are still relying on ldaplist and not using ldapsearch, please comment on https://phabricator.wikimedia.org/T114063 before 30 August 2016... [22:57:27] 06Labs, 07Tracking: New Labs project requests (tracking) - https://phabricator.wikimedia.org/T76375#2503835 (10yuvipanda) [22:57:29] 06Labs: Request creation of detox labs project - https://phabricator.wikimedia.org/T139864#2503833 (10yuvipanda) 05Open>03declined I talked to @ellery, we're going to host this on tools instead. [22:57:59] 06Labs, 13Patch-For-Review: Kill ldapsupportlib.py - https://phabricator.wikimedia.org/T114063#2503836 (10yuvipanda) @Dzahn ah, can you tell me what you were trying to do with ldaplist? [23:01:01] 06Labs, 13Patch-For-Review: Kill ldapsupportlib.py - https://phabricator.wikimedia.org/T114063#2503839 (10yuvipanda) ah, I see you included it, nvm. [23:01:15] 06Labs, 13Patch-For-Review: Kill ldapsupportlib.py - https://phabricator.wikimedia.org/T114063#2503851 (10yuvipanda) @Krenair do you think you can help? [23:03:26] 06Labs, 13Patch-For-Review: Kill ldapsupportlib.py - https://phabricator.wikimedia.org/T114063#1683588 (10AlexMonk-WMF) Yes, you need `ldapsearch -x` followed by your LDAP query. The -x is important, it specifies basic authentication instead of SASL. [23:21:00] 06Labs, 13Patch-For-Review: Kill ldapsupportlib.py - https://phabricator.wikimedia.org/T114063#2503951 (10Dzahn) Could you paste the entire command that replaces "ldaplist -l passwd? [23:23:09] 06Labs, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: madhuvishy is moving to operations on 7/18/16 - https://phabricator.wikimedia.org/T140422#2503966 (10Dzahn) a:03yuvipanda Hey Yuvi, could you please do the checkbox about labs lists? Otherwise i'd have to reset the password for all you admi... [23:24:18] 06Labs, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: madhuvishy is moving to operations on 7/18/16 - https://phabricator.wikimedia.org/T140422#2503969 (10yuvipanda) @Dzahn I meant 'admin'. I don't have the password at all, I've just been using the mailman master password, which is probably pret... [23:28:06] 06Labs, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: madhuvishy is moving to operations on 7/18/16 - https://phabricator.wikimedia.org/T140422#2503993 (10Dzahn) I would prefer it if the list admins could handle administration of their lists. The password needs to be shared with the other admins. [23:29:05] 06Labs, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: madhuvishy is moving to operations on 7/18/16 - https://phabricator.wikimedia.org/T140422#2503994 (10yuvipanda) Sure, I'll do that later then. There's no hurry for us to finish that box off. [23:38:10] PROBLEM - SSH on tools-docker-builder-03 is CRITICAL: Server answer [23:49:37] 06Labs, 10Tool-Labs, 07Tracking: Useful graphite metrics to be tracked for Tool labs (tracking) - https://phabricator.wikimedia.org/T69879#2504052 (10yuvipanda) [23:49:39] 06Labs, 10Tool-Labs, 13Patch-For-Review: Track web request stats for each tool on tool labs - https://phabricator.wikimedia.org/T69880#2504051 (10yuvipanda) 05Open>03Resolved [23:50:17] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Improvements to cold-migrate script - https://phabricator.wikimedia.org/T139272#2504053 (10yuvipanda) 05Open>03Resolved Yup! Thanks for the fixes! [23:50:33] 06Labs: tools-worker-1011 has no working user accounts?! - https://phabricator.wikimedia.org/T138447#2504055 (10yuvipanda) 05Open>03declined [23:51:12] 06Labs, 10Horizon, 13Patch-For-Review: Create puppet backend with REST api for labs instance configuration - https://phabricator.wikimedia.org/T133412#2504056 (10yuvipanda) a:05yuvipanda>03Andrew ok :D [23:52:35] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Investigate moving docker to use direct-lvm devicemapper storage driver - https://phabricator.wikimedia.org/T141126#2504061 (10yuvipanda) On further digging, here's what needs to happen: 1. Delete /dev/vda4 which we create to be an lvm partition 2. Recreate /dev/vda4... [23:57:59] 06Labs, 13Patch-For-Review: Kill ldapsupportlib.py - https://phabricator.wikimedia.org/T114063#2504069 (10AlexMonk-WMF) `ldapsearch -x objectClass=posixaccount` should give you the same thing as `ldaplist -l passwd` `ldapsearch -x uid=krenair` should give you the same thing as `ldaplist -l passwd krenair`