[00:25:00] 06Labs, 10DBA, 13Patch-For-Review: Move labs pdns database off of m5-master - https://phabricator.wikimedia.org/T128737#2250376 (10chasemp) Is it possible apparmor is killing mysql on holmium? ```Apr 29 00:20:09 holmium kernel: [ 8382.403156] type=1400 audit(1461889209.191:68): apparmor="STATUS" operation="... [00:52:31] is there a way to start lighttpd with more than 4 GB of virt mem ? [00:58:24] is it a 32bit system? [00:59:10] on labs, it's x64 [00:59:39] was that a joke? [01:03:17] no, because 32bit has a 4GB limit [01:03:54] unless PAE mode etc [01:04:05] but it's not relevant to your question then [01:10:08] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T85913#957178 (10lfschenone) Hi! I'd like to request a rename too. My global username has recently changed from "Luis Felipe Schenone" to "Felipe Schenone" so... [01:15:08] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T85913#957178 (10Peachey88) >>! In T85913#2250474, @lfschenone wrote: > Hi! I'd like to request a rename too. My global username has recently changed from "Lui... [01:20:45] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T133968#2250492 (10lfschenone) [01:22:22] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T133968#2250511 (10lfschenone) [01:22:25] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T85913#957178 (10lfschenone) [01:22:28] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T133968#2250492 (10lfschenone) [01:23:08] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T133968#2250492 (10lfschenone) [01:23:53] 06Labs, 06Operations, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T133968#2250515 (10lfschenone) [01:48:33] 06Labs: Cleanup proxies that point to nonexistent instances - https://phabricator.wikimedia.org/T132231#2250536 (10AlexMonk-WMF) This script will list details of: * Backends not pointing to *.wmflabs* * Backends with a route associated not pointing to *.wmflabs* * Backends pointing to a host that does not resolv... [01:51:58] 06Labs, 06Operations, 06Release-Engineering-Team, 10wikitech.wikimedia.org: Rename specific account in LDAP, Wikitech, Gerrit and Phabricator - https://phabricator.wikimedia.org/T133968#2250550 (10Dzahn) [02:13:19] 10Tool-Labs-tools-Other, 06Community-Tech, 07I18n: [[Wikimedia:Pageviews-num-languages/en]] needs PLURAL - https://phabricator.wikimedia.org/T133766#2250563 (10Niharika) CC @MusikAnimal [04:37:08] 06Labs, 10DBA, 13Patch-For-Review: Move labs pdns database off of m5-master - https://phabricator.wikimedia.org/T128737#2250619 (10MoritzMuehlenhoff) @chasemp: Yes, that is caused by apparmor: holmium used to have the mysql-5.5 package from Ubuntu installed and it was removed with "dpkg --rc", which left /et... [05:54:55] 10Tool-Labs-tools-Other, 06Community-Tech, 07I18n: [[Wikimedia:Pageviews-num-languages/en]] needs PLURAL - https://phabricator.wikimedia.org/T133766#2250650 (10MusikAnimal) This one is tricky because the value to be passed into Intuition is dynamically generated on the clientside. We're currently interpolat... [09:30:24] !ping [09:30:24] !pong [09:59:02] 06Labs, 10Tool-Labs, 13Patch-For-Review: support python3 uwsgi apps - https://phabricator.wikimedia.org/T104374#2250898 (10Ricordisamoa) Maybe @yuvipanda is able to unstall this based on {T98440} [10:15:53] (03CR) 10Lokal Profil: "After a crash course in unittest.mock I have a few questions but overall this looks good." (033 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/283108 (owner: 10Jean-Frédéric) [10:22:12] 06Labs, 06WMF-Legal, 07Epic, 07Privacy: [EPIC] Protect end-user privacy by restricting non-consentual third-party browser interactions - https://phabricator.wikimedia.org/T133919#2250972 (10Danny_B) [10:24:55] 06Labs, 10Labs-Infrastructure, 05Continuous-Integration-Scaling, 13Patch-For-Review: Bump quota of Nodepool instances (contintcloud tenant) - https://phabricator.wikimedia.org/T133911#2250978 (10hashar) [10:36:06] 10Tool-Labs-tools-Other, 06Community-Tech, 07I18n: [[Wikimedia:Pageviews-num-languages/en]] needs PLURAL - https://phabricator.wikimedia.org/T133766#2242145 (10Ricordisamoa) >>! In T133766#2250650, @MusikAnimal wrote: > I'm guessing we'll just have to have two messages: > ``` > "num-languages": "$1 language"... [10:39:08] 06Labs, 10Labs-Infrastructure, 05Continuous-Integration-Scaling, 13Patch-For-Review: Bump quota of Nodepool instances (contintcloud tenant) - https://phabricator.wikimedia.org/T133911#2248624 (10hashar) [10:54:57] Change on 12www.mediawiki.org a page OAuth/Owner-only consumers was modified, changed by A33aa69 link https://www.mediawiki.org/w/index.php?diff=2109949 edit summary: a [11:42:28] Change on 12www.mediawiki.org a page OAuth/Owner-only consumers was modified, changed by Syum90 link https://www.mediawiki.org/w/index.php?diff=2109962 edit summary: Reverted edits by [[Special:Contributions/A33aa69|A33aa69]] ([[User talk:A33aa69|talk]]) to last revision by [[User:Anomie|Anomie]] [11:49:12] 06Labs, 10Labs-Infrastructure, 06Operations: investigate slapd memory leak - https://phabricator.wikimedia.org/T130593#2251099 (10MoritzMuehlenhoff) I'll build a backport of 2.4.41. [12:33:25] (03CR) 10Lokal Profil: "An update on this. Once 283108 is merged I'll build some tests for the functions being refactored here." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/281930 (https://phabricator.wikimedia.org/T39422) (owner: 10Lokal Profil) [12:37:40] (03PS1) 10Lokal Profil: [NOT TESTED] Remove monument_random from database and api. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/286149 (https://phabricator.wikimedia.org/T39974) [12:40:07] (03PS1) 10Hashar: Portability for argparse_whence [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286150 [12:40:09] (03PS1) 10Hashar: Allow 'importing' of jstub script [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286151 [12:40:38] bd808: ^^^^ [12:42:44] (03CR) 10Hashar: "Bryan and I talked about this change and neither of us were quite happy with "changedir" and "--doctest" hack. I eventually hacked a chan" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/285435 (https://phabricator.wikimedia.org/T132475) (owner: 10BryanDavis) [12:43:14] (03CR) 10Hashar: "Should probably be squashed into https://gerrit.wikimedia.org/r/#/c/285435/" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286150 (owner: 10Hashar) [12:43:26] (03CR) 10Hashar: "Should probably be squashed into https://gerrit.wikimedia.org/r/#/c/285435/" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286151 (owner: 10Hashar) [12:43:50] (03CR) 10Hashar: "check experimental" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286150 (owner: 10Hashar) [12:43:54] (03CR) 10Hashar: "check experimental" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286151 (owner: 10Hashar) [13:44:21] (03PS1) 10Lokal Profil: [NOT TESTED] Better support for sister projects [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/286156 [13:53:07] (03PS2) 10Lokal Profil: [NOT TESTED] Better support for sister projects [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/286156 (https://phabricator.wikimedia.org/T132647) [13:54:41] Can someone explain to me why this task seems to be difficult to do? https://phabricator.wikimedia.org/T128926 [13:55:00] Danny_B: ^ [14:03:57] Josve05a: asking in #wikimedia-commtech might be more useful. [14:04:11] oh, that's a channel? :D [14:05:57] yeah [14:07:14] (03CR) 10Lokal Profil: "The "NOT TESTED" bit means I haven't figured out how to test this other than deploying and then checking that https://tools.wmflabs.org/he" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/286156 (https://phabricator.wikimedia.org/T132647) (owner: 10Lokal Profil) [14:09:59] (03CR) 10Lokal Profil: "The second link should be https://tools.wmflabs.org/heritage/api/api.php?action=search&format=gpx&limit=10&srcountry=ru&srlang=ru&srid=000" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/286156 (https://phabricator.wikimedia.org/T132647) (owner: 10Lokal Profil) [14:20:20] (03CR) 10Lokal Profil: ""NOT TESTED" here means I haven't figured out a good way of doing this which doesn't require deploying first." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/286149 (https://phabricator.wikimedia.org/T39974) (owner: 10Lokal Profil) [14:55:45] It's probably obvious, but how do I assign a puppet class to a lab instance? Can't find it in wikiteck/nova nor in Horizon... [14:56:06] gehel: 'configure' on the manage instances page [14:56:13] in wikitech [14:56:28] Horizon doesn't know about puppet yet… we're working on that now [14:57:02] wikitech provides a limited number of pre-selected classes. If you want to add a special class that's 'manage puppet groups' [14:57:10] Ok, then I have another issue. That page is empty for me. I thought that would be expected now that we are migrating to Horizon... [14:58:22] 'that page' being the configure page? [14:58:31] Probably that's a log-out-and-in thing, if not I don't know what's happening [14:58:35] are you projectadmin? [14:58:51] yep, I am project admin... Let me try to logout / in ...$ [14:59:33] https://www.youtube.com/watch?v=nn2FB1P_Mn8 [15:00:41] andrewbogott: amazing, that worked! [15:26:32] 06Labs, 06Discovery, 06Discovery-Search-Backlog, 06Operations, 10hardware-requests: eqiad: (2) Relevance forge servers - https://phabricator.wikimedia.org/T131184#2251477 (10chasemp) Does this usurp nobelium so we can decomission it in its labs support role? I assumed yes but want to make sure. Just a... [15:28:32] bd808: meanwhile I have sent a few patches for whatever labs python utility you were working on yesterday :d [15:29:07] hashar: *nod* looking at them now [15:29:25] bd808: i replaced your hack with a different hack :D but at least that does not involve changedir= [15:29:38] I really hate __name__ == '__main__' but that is probably the right thing to do here [15:29:38] the trick was to make jqub "importable", i.e. have it not execute any code [15:29:57] yeah I find that ugly [15:30:13] but then that is the de facto standard way of handling it in python [15:30:26] why do you hate teh __main__ convention? (just curious) [15:30:27] the pythonist way being to make it a proper module with a class jqub [15:30:41] chasemp: mostly because it is seldom needed [15:30:54] then have another .py file that can be used as the CLI entry point and have it installed by setup.py [15:31:06] ah, I got in the habit of it at my last gig as a lot of debugging was done w/ python -i [15:31:18] and i saw the value in being able to load vs execute [15:31:33] my argument against was always, I don't want this to be imported as a lib because I'm not maintaining it as one but [15:31:41] that isn't the dominant case etc [15:31:48] but ah gotcha [15:32:24] something like http://stackoverflow.com/questions/27784271/how-can-i-use-setuptools-to-generate-a-console-scripts-entry-point-which-calls :D [15:32:56] hashar: that's a bigger change than I want to bite off right now [15:33:42] in the "long term" the toollabs repo should get split into 2-3 repos and use a proper module for cli scripts though [15:33:46] * hashar hints at http://docs.openstack.org/developer/pbr/ [15:44:11] andrewbogott: good morning! once you get breakfast complete, I could use some feedback regarding raising the pool of Nodepool instances ( https://phabricator.wikimedia.org/T133911 ) . I am about to migrate a bunch of PHP jobs to it ;} [15:44:34] (03PS12) 10BryanDavis: Rewrite jsub in python [labs/toollabs] - 10https://gerrit.wikimedia.org/r/285435 (https://phabricator.wikimedia.org/T132475) [15:45:02] hashar: ^ squashed with a couple of minor changes [15:45:21] (03CR) 10BryanDavis: "check experimental" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/285435 (https://phabricator.wikimedia.org/T132475) (owner: 10BryanDavis) [15:46:41] bd808: neat! [15:46:45] note you can run tox locally as well [15:46:48] (03CR) 10BryanDavis: "Squashed into https://gerrit.wikimedia.org/r/#/c/285435/12" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286151 (owner: 10Hashar) [15:46:48] tox -e py27 [15:46:58] *nod* I'm doing that [15:47:11] hashar: is this moving capacity from historical CI to new CI or extending capacity solely on new CI? [15:47:18] (03CR) 10BryanDavis: "Squashed into https://gerrit.wikimedia.org/r/#/c/285435/12" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286150 (owner: 10Hashar) [15:47:48] bd808: I havent looked at the rest of the patch. I really been curious about that use cases yesterday and took some time during lunch break to figure out a fix ;D I am happy you like it! [15:48:29] chasemp: that is a short term extension but once I start migrating to the new hosts I will be able to delete the historical instances [15:49:05] the path processing portability you added is unnecessary, but doesn't hurt anything. This script will only ever run on trusty [15:49:10] for Precise we are down to just three instances ( https://integration.wikimedia.org/ci/label/UbuntuPrecise/ ) for php53 [15:49:14] ok I'll try to remind andrew but fyi he is at the openstack conf and then traveling through tuesday hashar [15:49:21] so it's a bad few days here to coordinate [15:49:38] and a bunch of historical Trusty ones https://integration.wikimedia.org/ci/label/UbuntuTrusty/ . Will be able to shred a bunch of them [15:49:56] chasemp: no problem :-) [15:52:19] 06Labs, 10Labs-Infrastructure, 05Continuous-Integration-Scaling, 13Patch-For-Review: Bump quota of Nodepool instances (contintcloud tenant) - https://phabricator.wikimedia.org/T133911#2251564 (10hashar) We have 14 Trusty instances on the historical CI https://integration.wikimedia.org/ci/label/UbuntuTrusty... [15:53:03] chasemp: I have captured the above on the task. Will wait for andrew to shine in whenever he has time / he is back. Thank you for the update! [16:50:00] 10Tool-Labs-tools-Other, 06Community-Tech, 07I18n: [[Wikimedia:Pageviews-num-languages/en]] needs PLURAL - https://phabricator.wikimedia.org/T133766#2251641 (10Nemo_bis) a:05Nemo_bis>03None [17:20:54] PROBLEM - Host tools-worker-1011 is DOWN: PING CRITICAL - Packet loss = 100% [17:32:48] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Upgrade to Kubernetes 1.2.3 - https://phabricator.wikimedia.org/T133458#2251767 (10chasemp) 05Open>03Resolved done [18:15:04] (03Abandoned) 10Hashar: Portability for argparse_whence [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286150 (owner: 10Hashar) [18:15:08] (03Abandoned) 10Hashar: Allow 'importing' of jstub script [labs/toollabs] - 10https://gerrit.wikimedia.org/r/286151 (owner: 10Hashar) [19:09:26] 06Labs, 10Tool-Labs: Goal: Allow using k8s instead of GridEngine as a backend for webservices (Tracking) - https://phabricator.wikimedia.org/T129309#2251970 (10yuvipanda) [19:10:35] 06Labs, 10Tool-Labs: Goal: Allow using k8s instead of GridEngine as a backend for webservices (Tracking) - https://phabricator.wikimedia.org/T129309#2101957 (10yuvipanda) Ok, so the new webservice code has been fully rolled out! Next steps: - Whitelist only our registry - Provide k8s credentials for all tool... [19:12:55] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Provision a .kube/config file for all tools - https://phabricator.wikimedia.org/T133999#2251975 (10yuvipanda) [19:13:12] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Write a k8s admission controller to enforce that all containers running come from our private repository - https://phabricator.wikimedia.org/T133515#2251989 (10yuvipanda) [19:13:13] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Provision a .kube/config file for all tools - https://phabricator.wikimedia.org/T133999#2251975 (10yuvipanda) [19:36:03] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 10Diffusion, 15User-bd808: Create application to manage Diffusion repositories for a Tool Labs project - https://phabricator.wikimedia.org/T133252#2252000 (10mmodell) >>! In T133252#2235001, @chasemp wrote: > A small bit of pile on due to the subject matte... [19:53:34] 10Tool-Labs-tools-Other, 06Community-Tech, 07I18n: [[Wikimedia:Pageviews-num-languages/en]] needs PLURAL - https://phabricator.wikimedia.org/T133766#2252031 (10Purodha) >>! In T133766#2251006, @Ricordisamoa wrote: >>>! In T133766#2250650, @MusikAnimal wrote: >> Far from ideal, but it'd work. No. It would w... [19:58:27] Hi... there's presently a {{help me}} request pending that has to do with a couple tools on labs that list pages created: https://en.wikipedia.org/wiki/User_talk:Vaselineeeeeeee#Help_-_page_curation_issue [19:58:42] someone here might be able to answer it [20:03:39] NickW557: it's shown normally on https://en.wikipedia.org/wiki/Special:Contributions/Vaselineeeeeeee [20:03:50] so it's not a wiki issue, just an xtools issue it looks [20:04:05] and my wild guess is .. it has a redis backend or something [20:04:14] and that was down for a moment [20:04:53] but to get real answers please file a bug and add the tag "xTools-on-Labs" [20:05:20] that will notify the tool maintainers [20:05:34] The other tool doesn't show it either though. [20:06:27] They mention this: https://tools.wmflabs.org/sigma/created.py?name=Vaselineeeeeeee&server=enwiki&ns=,,&redirects=none and xtools: https://tools.wmflabs.org/xtools/pages/index.php?name=Vaselineeeeeeee&lang=en&wiki=wikipedia&namespace=0&redirects=noredirects [20:06:33] that's the same "tool" though [20:06:37] in terms of toollabs [20:06:46] "xtools" consists of multiple sub-tools i guess [20:06:59] ah, sigma [20:07:16] Neither tool shows it. [20:07:21] 10Tool-Labs-tools-Other, 06Community-Tech, 07I18n: [[Wikimedia:Pageviews-num-languages/en]] needs PLURAL - https://phabricator.wikimedia.org/T133766#2252071 (10MusikAnimal) Ah, looking at [[ https://www.mediawiki.org/wiki/Localisation#Be_aware_of_PLURAL_use_on_all_numbers | this doc ]] I see what you mean. I... [20:07:34] I can imagine the one tool maybe not showing it, but not 2. [20:07:51] http://tools.wmflabs.org/sigma/ uhmm [20:08:07] tom29739: but if they use the same database server? [20:08:17] i dont know where they store the data [20:08:27] You mean that it might not have replicated? [20:08:44] can you send him to https://en.wikipedia.org/wiki/User:%CE%A3 [20:09:32] This user? https://en.wikipedia.org/wiki/User_talk:Vaselineeeeeeee [20:09:55] yea [20:10:22] Is there maybe a bug reporting system we can send them to that covers both tools? Since both tools are missing the article? [20:10:27] Yes. [20:10:30] Phab. [20:10:34] yes [20:10:46] i assumed that was known, sorry [20:10:49] there was a ticket on there about missing data in the db replica. [20:10:51] yea, please use phabricator.wikimedia.org [20:11:09] and there is a tag for xtools [20:11:14] and tool-labs [20:11:24] I'll point the user there, then. Thanks. [20:11:47] yw [20:14:07] https://phabricator.wikimedia.org/T133715 [20:14:13] That ticket. [20:20:56] 10Tool-Labs-tools-Other, 06Community-Tech, 07I18n: [[Wikimedia:Pageviews-num-languages/en]] needs PLURAL - https://phabricator.wikimedia.org/T133766#2252090 (10Purodha) >>! In T133766#2252071, @MusikAnimal wrote: > Ah, looking at [[ https://www.mediawiki.org/wiki/Localisation#Be_aware_of_PLURAL_use_on_all_nu... [20:39:41] 06Labs, 06Discovery, 06Discovery-Search-Backlog, 06Operations, 10hardware-requests: eqiad: (2) Relevance forge servers - https://phabricator.wikimedia.org/T131184#2252155 (10Gehel) Damn, this labs thing is confusing... @chasemp thanks for the precisions. I'm not entirely sure I understand what you mean b... [20:58:33] 06Labs, 07Tracking: New Labs project requests (tracking) - https://phabricator.wikimedia.org/T76375#2252248 (10tom29739) [20:58:35] 06Labs: Create new labs project for ircredirector - https://phabricator.wikimedia.org/T133941#2252245 (10tom29739) 05Open>03Resolved a:03tom29739 @bd808 We have discussed this on #wikipedia-en-helpers, and have come to the conclusion that a ban tracker could be completely IRC based, and would not need the... [22:23:35] 06Labs, 13Patch-For-Review: Periodic internal labs dns outages - https://phabricator.wikimedia.org/T124680#2252726 (10chasemp) A few notes. These failures tend to happen in waves lasting 20-30 minutes, they resolve on their own, they are also not always total outages. We have periods of "flakiness" where som...