[01:12:18] !log phabricator - gerrit:355572 and manually editing root crontab to stop cron spam from exim4::ganglia caused by mariadb upgrade [01:12:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Phabricator/SAL [01:17:53] !log phabricator - disabling puppet on phabricator instance - it's still adding the cron job back - see mail to ops list [01:17:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Phabricator/SAL [02:30:57] PROBLEM - Puppet errors on tools-exec-1439 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [03:06:01] RECOVERY - Puppet errors on tools-exec-1439 is OK: OK: Less than 1.00% above the threshold [0.0] [06:02:34] 06Labs, 10DBA: Labs database replica drift - https://phabricator.wikimedia.org/T138967#2415416 (10Marostegui) Maybe what we can try is to stop sanitarium2 and sanitarium at the same replication position and import the page table (it is around 7.3G). [06:54:53] 06Labs, 10Tool-Labs-tools-Database-Queries, 10Analytics, 10DBA, 10MediaWiki-Page-deletion: Database replication issues with deleted pages (affecting Tool Labs and Analytics Store) - https://phabricator.wikimedia.org/T166194#3290992 (10jcrespo) > what's the relation between dbstore1001 and and analytics-s... [07:01:36] 06Labs, 10DBA: Labs database replica drift - https://phabricator.wikimedia.org/T138967#3291032 (10jcrespo) @Marostegui - you can do that if you want, but a) it will cause more complains of missing pages and the lag b) it will break in only few days (all tables were reimported for enwiki in a long process that... [08:59:54] How do you now request access to the tools project? [09:00:05] The link from https://tools.wmflabs.org/ for "request access" is now a 404 [09:10:11] I found it on toolsadmin but I think the links from tools.wmflabs.org need to be fixed. I'll make a ticket [09:12:16] 06Labs, 10Tool-Labs-tools-Database-Queries, 10Analytics, 10DBA, 10MediaWiki-Page-deletion: Database replication issues with deleted pages (affecting Tool Labs and Analytics Store) - https://phabricator.wikimedia.org/T166194#3291248 (10jcrespo) To clarify my previous statement, analytics hosts are conside... [09:16:05] hi, i am new to tools. i am at wikicite and would like to hack on the wikifactmine pipeline today. i just sent a membership request, would be great if you could approve it. [09:16:07] :D [09:18:31] petan: ^^ [09:18:48] Reedy: ^^ :D [09:21:29] 06Labs, 10Tool-Labs: tools.wmflabs.org needs a documentation update - https://phabricator.wikimedia.org/T166295#3291278 (10Tarrow) [09:57:26] Anyone looking for a project? [09:57:42] Should be relatively simple... [09:59:17] It’s more the explanation of why, lol. [10:05:24] https://phabricator.wikimedia.org/T133821 [10:06:50] ^ Specificlly, “Currently, the majority of the real pragmatic problems this is causing are on upload.wikimedia.org links for Commons deletions, as seen in e.g. T119038, T109331, T133819, and probably several other duplicates of the same basic thing. A lot of the urgency from requestors on these is driven by a rise in Commons abuse from mobile networks to upload copyvio material (especially through labs-based proxy tools), which Commons ad [10:06:50] T133819: upload-lb.ulsfo.wikimedia.org still allow access to some deleted files - https://phabricator.wikimedia.org/T133819 [10:06:51] are having to deal with an alarming rate of. At the rate at which they're deleting copyvio content, and the degree to which they care this content isn't visible from our servers anymore, they're falling into a bucket where they are affected by the general purging issues to a much greater and more-noticeable degree than most.” [10:06:51] T109331: Deleted files sometimes remain visible to non-privileged users if permanently linked - https://phabricator.wikimedia.org/T109331 [10:06:51] T119038: Image cache issue when 'over-writing' an image on commons - https://phabricator.wikimedia.org/T119038 [10:07:05] * Revent pets the bot. [10:07:05] 06Labs, 10Tool-Labs, 07Documentation: tools.wmflabs.org needs a documentation update - https://phabricator.wikimedia.org/T166295#3291364 (10Framawiki) [10:07:57] It would be useful if there was a tool on Labs to poke the various mirrors of upload to see if a deleted copyvio was still visible. [10:09:48] This would allow nagging operations to make them go away on some basis other than ‘happening to notice’ because some abuser shared an upload link on Facebook. [11:55:28] Does striker show labs projects along with tools? [12:12:06] Zppix: no, only tools. https://tools.wmflabs.org/openstack-browser shows all projects [12:12:37] madhuvishy: i knew that existed i was just curious bout striker [12:12:48] ah okay [12:12:50] But thanks [12:13:04] pkraker: did your request get approved? [12:14:16] mutante: is there a phab task on phab on wmflabs for puppet being disabled? [12:14:24] pkraker: I just approved you [12:19:24] mahuvishy: awesome thanks! [12:19:59] madhuvishy* [12:20:07] pkraker: yw :) [12:22:05] pkraker: welcome to the developer world of labs/tools labs [12:24:08] Zppix: thanks, great to be on board :) [12:24:35] pkraker: feel free to ask me or anyone in here if you have questions [12:24:56] thx, will do! [13:01:10] 06Labs, 10DBA: Labs database corruption - https://phabricator.wikimedia.org/T166091#3291733 (10Marostegui) @Legoktm would you mind adding the above comment to: T138967 as per: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Database/Replica_drift It is easier for us to track in a single place and we try to... [14:13:14] 06Labs, 10Tool-Labs, 07Documentation: tools.wmflabs.org needs a documentation update - https://phabricator.wikimedia.org/T166295#3291278 (10chasemp) Thanks for logging this task! We had had a bunch of different scattered instructions, where (what page(s)) were you working from? [14:13:21] 06Labs, 10Tool-Labs, 07Documentation: tools.wmflabs.org needs a documentation update - https://phabricator.wikimedia.org/T166295#3291972 (10chasemp) p:05Triage>03Normal [14:15:34] 10MediaWiki-extensions-OpenStackManager, 13Patch-For-Review: MW OpenStackManager: add support for ED25519 SSH keys - https://phabricator.wikimedia.org/T159070#3291983 (10Krinkle) [15:23:55] 06Labs, 10Tool-Labs, 07Documentation: tools.wmflabs.org needs a documentation update - https://phabricator.wikimedia.org/T166295#3292277 (10Tarrow) Just from the homepage. http://tools.wmflabs.org/ This is where you'll probably land if you Google wikimedia tool labs. [15:44:28] yo andrewbogott, yt? [15:44:39] what's up? [15:44:54] how can I include a profile class on a standalone puppetmaster? [15:45:04] the class isn't in horizon interface, cause its new [15:45:12] adding it to Other Classes [15:45:16] There's an 'other classes' text widget at the bottom [15:45:20] yeah, that should do it [15:45:26] hmm, tried, that, will try again [15:46:37] ya andrewbogott i get Could not find class profile::kafka::simple::broker [15:47:04] but its def there [15:47:04] root@kafkatest02:/var/lib/git/operations/puppet# head modules/profile/manifests/kafka/simple/broker.pp | grep class [15:47:04] class profile::kafka::simple::broker { [15:47:13] So that means that the 'Other Classes' bit is working and the puppetmaster doesn't have it [15:47:25] So maybe your instance isn't using the right puppetmaster? What's in /etc/puppet.conf? [15:47:47] HMM [15:47:48] server = labs-puppetmaster-eqiad.wikimedia.org [15:47:49] you are right [15:48:06] 10Tool-Labs-tools-Xtools, 03Community-Tech-Sprint: If revisions are revdel'd, articleinfo compares the surrounding edits as if it were one edit - https://phabricator.wikimedia.org/T148857#3292367 (10MusikAnimal) Thanks! I think I've fixed xtools-dev, should work now. [15:48:08] but [15:48:10] the docs about the standalone class explain how to set that up, I think it's just a single hiera setting [15:48:17] role::puppetmaster::standalone i sincluded, [15:48:22] and default server_name is $::fqdn [15:48:33] that makes your instance a puppetmaster but doesn't change the puppetmaster for the instance itself [15:48:42] they're unrelated [15:48:54] 'am I a puppetmaster' vs. 'what is my puppetmaster' are two different things [15:48:56] oh i have to set it upas a client [15:48:58] ahhh ok [15:49:00] oook tahnks [15:49:16] I don't remember what the setting is but the docs are pretty good. [15:49:29] 10Tool-Labs-tools-Xtools, 06Community-Tech: [Epic] Rewrite XTools: Articleinfo - https://phabricator.wikimedia.org/T157602#3292373 (10MusikAnimal) 05Open>03Resolved Looks like we're all done here! [15:49:31] 10Tool-Labs-tools-Xtools, 06Community-Tech: Epic: Rewriting XTools - https://phabricator.wikimedia.org/T153112#3292376 (10MusikAnimal) [15:49:45] ya, on it, previously selfhosted was always configured as a client of itself [15:49:50] didn't htink i'd have to do that part [15:49:53] thank you [16:02:39] Zppix: i made one just now, so you guys can read https://phabricator.wikimedia.org/T166322 [16:02:43] paladox: ^ [16:02:57] Thanks mutante [16:03:00] i've removed the package [16:03:06] and replaced it with the lighter one [16:03:09] which dosen't need mysql [16:03:50] In the matter of fact does phabricator even need the exim4-daemon-heavy [16:05:53] 06Labs, 07Tracking: Existing Labs project quota increase requests (Tracking) - https://phabricator.wikimedia.org/T140904#3292483 (10Freddy2001) [16:05:55] 06Labs: Request increased quota (floating ip) for getstated labs project - https://phabricator.wikimedia.org/T166324#3292471 (10Freddy2001) [16:08:17] 06Labs, 10Tool-Labs, 07Documentation, 15User-bd808: tools.wmflabs.org needs a documentation update - https://phabricator.wikimedia.org/T166295#3292487 (10bd808) a:03bd808 [16:26:54] (03PS1) 10BryanDavis: Update getting started and error documentation [labs/toollabs] - 10https://gerrit.wikimedia.org/r/355631 (https://phabricator.wikimedia.org/T166295) [16:27:12] paladox: we need something that makes it NOT pull the 'heavy' config if in labs [16:27:20] while we also still avoid $realm checks [16:27:31] Yep. [16:27:37] so i think more like the Hiera override "has_default_mail_relay: [16:27:39] or similar to it [16:27:51] yep [16:28:06] if you had "has_default_mail_relay: false [16:28:11] in Hiera: page for it [16:28:14] mutante: guard with a hiera setting that defaults to false and set to true in prod's roles hiera? [16:28:39] I've done it in horizion [16:28:43] tandard::has_default_mail_relay: false [16:28:48] standard::has_default_mail_relay: false [16:30:44] default false sounds good too, yes [16:30:52] so what this does is: [16:31:00] avoids " include ::standard::mail::sender [16:31:43] 2 class { 'exim4': [16:31:43] 3 queuerunner => 'queueonly', [16:31:44] 4 config => template("standard/mail/exim4.minimal.${::realm}.erb"), [16:31:51] so it does NOT do that ^ now [16:31:53] yep [16:31:58] but that is already "minimal" [16:32:02] and not "heavy" [16:32:40] you are getting mail config from 2 places, from "include ::standard" , that's the part above [16:32:47] which can be fixed with the Hiera setting [16:32:56] but in addition you are getting it from the phabricator role itself [16:33:03] yep [16:33:11] it sets the alias [16:33:12] for root [16:33:13] so that needs a second change [16:33:44] that and it pulls in exim4 "heavy" right [16:33:51] Yep [16:34:00] Which i doint think we need for phabricator or am i wrong? [16:34:02] both things need to be flexible with Hiera seting [16:34:20] !log tools.admin Cherry-picked https://gerrit.wikimedia.org/r/#/c/355631/ [16:34:22] We could just change it to light without needing to set a config if it does not need heavy for prod phab. [16:34:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.admin/SAL [16:34:31] i dont know. but either way to test if it does, you need the same thing [16:34:36] a setting to disable.enable it [16:34:45] (03CR) 10BryanDavis: [V: 032] "Cherry-picked to the admin tool" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/355631 (https://phabricator.wikimedia.org/T166295) (owner: 10BryanDavis) [16:35:07] Yep [16:35:51] mutante what about doing apt-get install exim4-daemon-light [16:35:54] that will remove the heavy one [16:36:05] wont need a config for that. Then puppet will re add it. [16:36:24] phabricator uses some exim mail rules to make a few special things happen [16:36:34] paladox: we don't want anything that involves "apt-get install", we want puppet. [16:36:37] chasemp set that up so I'm not at all familiar with how it works [16:36:39] Oh [16:36:55] anyway changing the exim package could break things [16:36:58] i don't want to change the prod setup for this [16:37:04] i only want to fix the labs setup [16:37:08] ahh ok [16:37:46] we were talking about a setting that makes it not pull in the "heavy" exim4 config if not in prod [16:38:44] paladox: here's 2 things i DON'T want. changing the prod setup. manually installing and removing packages. [16:38:53] ok [16:39:54] I see this https://github.com/wikimedia/puppet/blob/production/modules/role/templates/exim/exim4.conf.phab.erb [16:40:08] which needs to also be updated on the side to use a wmflabs domain. [16:40:29] sigh [16:40:37] yes, there is lots there that is needed [16:40:59] yep [16:41:23] start with the low-hanging fruit. the easy things [16:41:29] Yep [16:41:33] like that alias to root@ [16:41:51] Yeh [16:42:01] i need to get some food. we just have to do this step by step [16:42:12] good that we have the ticket for it [16:42:26] if hiera('standard::has_default_mail_relay') { [16:42:34] ok [16:49:44] twentyafterfour: this is still correct (even though it gets overridden), right https://gerrit.wikimedia.org/r/#/c/355455/ [16:50:00] * twentyafterfour looks [16:50:15] yeah [16:50:29] k, thx [17:51:07] Hey guys just wanted to let labs users know that andrewbogott has posted on phabricator about an experimental Debian os. https://phabricator.wikimedia.org/J13 [18:09:16] (03CR) 10BryanDavis: [V: 032 C: 032] Update getting started and error documentation [labs/toollabs] - 10https://gerrit.wikimedia.org/r/355631 (https://phabricator.wikimedia.org/T166295) (owner: 10BryanDavis) [18:10:26] (03Merged) 10jenkins-bot: Update getting started and error documentation [labs/toollabs] - 10https://gerrit.wikimedia.org/r/355631 (https://phabricator.wikimedia.org/T166295) (owner: 10BryanDavis) [18:15:22] Hi all, how can I customize web server configuration in ToolLabs instance if I'm running Python app via UWSGI on kubernetes? (I need to allow cross origin requests) [18:25:52] PROBLEM - Puppet errors on tools-exec-1407 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [18:29:12] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3292883 (10kaldari) [18:29:49] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3292883 (10kaldari) [18:30:14] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3292883 (10kaldari) p:05Triage>03Normal [18:36:57] ok, if someone ever googles logs for this channel, you can enable CORS on uWSGI on Kubernetes here, by adding "route = .* addheader:Access-Control-Allow-Origin: * " to your uwsgi.ini [18:38:00] Xelgen: could you add it to https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web#Python_.28uWSGI.29 ? Thanks :-) [18:38:19] actually, I can just do that myself :D [18:41:16] valhallasw`cloud: Yes, please if you can. Frankly I'm slighlty confused on what we put where in docs [18:41:39] web things go under web [18:41:44] and that's as much structure as there is [18:42:08] so for uwsgi you mostly need to refer to the uwsgi docs [18:42:11] ...which are impenetrable [18:44:14] and addheader seems to literally not be documented [18:58:39] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3292950 (10Matthewrbowker) a:03Matthewrbowker I'll take a look [18:58:59] 10Tool-Labs-tools-Xtools, 06Community-Tech: Epic: Rewriting XTools - https://phabricator.wikimedia.org/T153112#3292954 (10Matthewrbowker) [18:59:01] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3292953 (10Matthewrbowker) [19:05:53] RECOVERY - Puppet errors on tools-exec-1407 is OK: OK: Less than 1.00% above the threshold [0.0] [19:12:39] PROBLEM - Free space - all mounts on tools-webgrid-lighttpd-1422 is CRITICAL: CRITICAL: tools.tools-webgrid-lighttpd-1422.diskspace._tmp.byte_percentfree (<30.00%) [19:28:26] hm… is anyone around who knows about wsexports? [19:33:17] Tpt[m], phe, that alert (on 1422) is because of a million wsexport files in /tmp. Does your tool generally clean those up automatically or have they been leaking for ages? i see files dating back to April 30th. [19:33:40] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3292883 (10Superyetkin) I do not think it is a good idea to throw an exception when a project does not exist. //databasePrepare()// in //LabsHelper.php// must change. Having so... [19:36:42] 06Labs, 10Tool-Labs: wsexport tool leaking files in /tmp - https://phabricator.wikimedia.org/T166337#3293050 (10Andrew) [19:37:31] andrewbogott: they are difficult to get in touch with usually, takes a bit [19:37:36] I would cleanup as you need to and restart the tool? [19:37:41] ok, I made a bug and will just delete some things [19:37:53] the tool is probably working fine, making file after file :) [19:38:11] 06Labs, 10DBA: Labs database replica drift - https://phabricator.wikimedia.org/T138967#3293067 (10russblau) @jcrespo: Is there (or will there be) a way to access and create user databases on the new servers (as per https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Database#User_databases)? [19:47:42] RECOVERY - Free space - all mounts on tools-webgrid-lighttpd-1422 is OK: OK: All targets OK [20:11:29] andrewbogott: sorry for the troubles. It seems that the call to unlink from the tool to remove the files does not actually remove them. I've tried to fix this issue multiple time but I've not manage to get it work. I usually do a hand cleanup once a month but it's definitely not a good thing. We should at least have a cron task to cleanup things once a day (or even every hour) [20:12:11] Tpt[m]: huh — is it something that works in unit tests but fails when the tool is really running? [20:12:21] Deleting a file shouldn't be a hard problem :) [20:12:22] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3293156 (10kaldari) It looks like this happens in the Edit Counter interface as well. Put in a bogus project and you get a 500 error: http://tools.wmflabs.org/xtools-dev/ec/dfg... [20:12:30] Yes, it seems [20:12:43] Definitely, should not be hard [20:13:27] This is maybe related to permissions as the .pdf are not created by the same process [20:14:08] Is it possible that you're leaking file handles, so the files are still open when they're getting unlinked? [20:14:19] But yeah, permissions seem like the most likely culprit [20:14:35] But I have seen nothing in logs [20:32:58] PROBLEM - Puppet errors on tools-exec-1434 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [20:45:03] 06Labs, 10Tool-Labs, 07Documentation, 13Patch-For-Review, 15User-bd808: tools.wmflabs.org needs a documentation update - https://phabricator.wikimedia.org/T166295#3293275 (10bd808) 05Open>03Resolved I fixed the instructions and a few other small issues. Feel free to reopen if you find more problems. [20:45:57] 06Labs, 10Tool-Labs, 15User-bd808, 06cloud-services-team (Kanban): Modernize the admin tool's codebase - https://phabricator.wikimedia.org/T140254#3293282 (10bd808) See {T166295} for some content updates that will be needed. [20:50:33] good evening, can someone help me with "chown" on tool labs? I have some problems. :/ [20:51:47] FNDE: which tool? [20:52:01] "request" [20:52:45] what file do you need fixed up? [20:54:44] 10Tool-Labs-tools-Xtools, 06Community-Tech: Admin stats fatals if project isn't recognized - https://phabricator.wikimedia.org/T166332#3293303 (10Matthewrbowker) Yes, I've had a todo list item for a while. The expected behavior is to add a "flash notice" then redirect back to the form. @Superyetkin Indeed.... [20:55:01] bd808: some directories with subdirectories. some of them are owned by "tools.request", and I don't know how to own it by "fnde". the problem is, I can't edit it with WinSCP [20:55:51] 10Tool-Labs-tools-Xtools, 06Community-Tech: Unknown project should not cause an Internal Server Error - https://phabricator.wikimedia.org/T166332#3293317 (10Matthewrbowker) [20:56:42] hmmm.. ok. chowning to your user isn't the best thing since that will make it harder for you to share maintainership with other users. But I know that we have difficulties with our current setup and using remote editing tools. [20:58:16] bd808: okay, I understand. well, do you know how I can own the files temporarily? [20:59:25] FNDE: I can chown things for you and then you can use `take $FILENAME` later as the tool to change the ownership back [21:00:37] okay, so I can't own it by myself? [21:03:02] bd808: okay, so I can't own it by myself? [21:05:56] 06Labs, 06Operations, 10Phabricator, 13Patch-For-Review: spam from phabricator in labs - https://phabricator.wikimedia.org/T166322#3293330 (10Zppix) [21:07:59] RECOVERY - Puppet errors on tools-exec-1434 is OK: OK: Less than 1.00% above the threshold [0.0] [21:10:00] hi, where does root@wmflabs.org redirect it's emails? [21:10:33] We set it on the phabricator instance in labs to not send spam to root@wikimedia.org but didn't think that it may do it for labs or even redirect back to the same place [21:14:31] paladox: doesnt labs have an equilvirnt to tools labs emails such as tools.toolname@wikimedia.org [21:14:41] not sure [21:15:32] FNDE: I think you need help from someone like me who has sudo rights to change from the tool to your user. I'll fix them up for you if you can write a phab task listing the files/directories you want changed [21:16:00] the `become` script can go the other direction (from your user to the tool user) [21:16:25] err... `take` not become [21:20:56] paladox: Zppix: i think ideal would be an email address like $project-admins@wmflabs.org where you can automatically reach the people who are project admins and then use that for root mail [21:21:12] ok [21:21:22] mutante: does the even exist [21:21:29] bd808: okay thank you very much. I solved the problem with download, delete the direction, and re-upload. [21:21:34] Zppix: no :) [21:21:38] i am not sure what exists already [21:21:45] but, I think it's the hard way :)) [21:21:45] mutante: could it be set up? [21:23:07] Zppix: "could" - sure. but that kind of thing needs discussion with cloud team [21:23:32] mutante: i mean its just like the tool email thing [21:23:41] it was just a thought, the ideal recipient of root mail would be the project admins of that project [21:23:42] I mean just steal the code and stuff from that [21:24:21] Zppix: you know more than me then about tool email :) [21:24:35] was just thinking about "VPS root mail" [21:24:52] mutante: considering i run 2 tools i hope i would know :) [21:25:00] Ill see if i can find the docs on it for you [21:33:15] mutante: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Mail_to_a_Tool [21:35:32] FNDE: glad you hacked around the problem. "some day"™ we will find an easier way for remote editing [21:35:38] Zppix: ah:) yea, that seems similar. like projectname.root@vps.wmfcloud.org or something, heh [21:35:57] bd808: I hope so :) [21:36:02] thank you. [21:45:47] 06Labs, 10Labs-project-Phabricator, 06Operations, 13Patch-For-Review: spam from phabricator in labs - https://phabricator.wikimedia.org/T166322#3293439 (10Aklapper) [ wmflabs → not #Phabricator ] [21:46:05] Yes [21:47:38] 06Labs, 10Labs-project-Phabricator, 06Operations, 13Patch-For-Review: spam from phabricator in labs - https://phabricator.wikimedia.org/T166322#3293445 (10Dzahn) It's also an issue of the Phabricator Puppet module in general. [21:47:56] Zppix, paladox, mutante: the $project-admins@wmflabs.org sounds like something worth writing up as a phab task. I can't promise we will get to it soon, but I have a vague idea of how we could make it possible. [21:48:42] bd808: if i dont need access to servers im willing to attempt to do it myself [21:49:09] Thanks [21:49:29] basically we would need to hook exim up with the nova api that can list project members. That might jsut be a periodic dump into a static alias file or it could be a fancier integration [21:49:48] 06Labs: Support per project email address @wmflabs.org - https://phabricator.wikimedia.org/T166349#3293447 (10Paladox) [21:50:25] 06Labs: Support per project user email address $project-admins@wmflabs.org - https://phabricator.wikimedia.org/T166349#3293459 (10Paladox) [21:50:52] heh, ok, cool. thanks all [21:51:11] the current tools implementation is ldap based, using https://github.com/wikimedia/puppet/blob/production/modules/toollabs/files/maintainers and https://github.com/wikimedia/puppet/blob/production/modules/toollabs/templates/mail-relay.exim4.conf.erb [21:52:15] 06Labs: Support per project user email address $project-admins@wmflabs.org - https://phabricator.wikimedia.org/T166349#3293465 (10bd808) The rough idea would be to figure out how to route root@instance emails to the list of project maintainers tracked in keystone. [21:52:34] bd808 : We already support per project emailaddress, right? [21:52:46] We use tools.pywikibot@tools.wmflabs.org for several things [21:52:57] 06Labs: Support per project user email address $project-admins@wmflabs.org - https://phabricator.wikimedia.org/T166349#3293468 (10bd808) ``` lang=irc [21:51] the current tools implementation is ldap based, using https://github.com/wikimedia/puppet/blob/production/modules/toollabs/files/mainta... [21:53:15] multichill: per tool yes, per project (e.g. deployment-prep) no AFAIK [21:54:57] bd808: will i need special access to do any of this? [21:55:45] Weird, I thought we also had it for general labs [21:56:27] 06Labs: Support per project user email address $project-admins@wmflabs.org - https://phabricator.wikimedia.org/T166349#3293471 (10Zppix) I will try to do this but dont assign incase i am unable [21:57:15] Zppix: I would imagine that the proper solution will have to be made in ops/puppet.git, but anyone could try to figure out what is needed on the exim side. There is also public read-only access to the keystone data that would be needed. https://tools.wmflabs.org/openstack-browser/ shows how to find the list of maintainers for a project. [21:57:57] I can do the puppet im unfamlar with keystone [22:00:25] bd808: ---^ [22:01:27] Zppix: learn or wait for someone else to pick up the task ;) [22:02:00] bd808: i can wait til keystone is started then i can collab [22:02:34] Does phab support html tags [22:03:01] no [22:03:06] if you mean in a comment etc [22:03:14] Zppix: https://secure.phabricator.com/book/phabricator/article/remarkup/ [22:03:26] that's what you can put in phab things [22:06:53] bd808: strikethrough didnt work :/ [22:24:58] 06Labs: Support per project user email address $project-admins@wmflabs.org - https://phabricator.wikimedia.org/T166349#3293518 (10Dzahn) [22:25:09] 06Labs: Support per project user email address $project-admins@wmflabs.org - https://phabricator.wikimedia.org/T166349#3293447 (10Dzahn) [23:13:15] 10Tool-Labs-tools-Other, 10DBA, 13Patch-For-Review: Tired of APIError: readonly - https://phabricator.wikimedia.org/T164191#3293594 (10Multichill)