[00:18:58] bd808: WIll this take lnger o shorter time than using the p-prop table? https://quarry.wmflabs.org/query/9103 [00:19:02] longer or* [00:21:28] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 10Diffusion, 15User-bd808: Create application to manage Diffusion repositories for a Tool Labs project - https://phabricator.wikimedia.org/T133252#2283630 (10bd808) [00:27:11] Josve05a: I'm not sure. I'm not the greatest query optimizer [00:29:45] well, I'm even worse xD [00:49:06] 06Labs, 10DBA: Querying the logging table on labs is slow - https://phabricator.wikimedia.org/T131266#2283648 (10Sigma) Any updates? [01:01:09] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 10Diffusion, 15User-bd808: Create application to manage Diffusion repositories for a Tool Labs project - https://phabricator.wikimedia.org/T133252#2283658 (10bd808) [01:03:36] Josve05a: "This query took longer than 30 minutes to execute and was killed." -- mine didn't do any better [01:03:46] :/ [01:04:14] perhaps limit the result? "LIMIT 100; LIMIT 1000 etc. to see where it gets kiled... [01:05:46] I think what you'd need to do is iterate through the page table in reasonable sized chunks and check those batches of pages against the pagelinks and templatelinks tables [01:06:11] * Josve05a grunts at the internet [01:07:10] it would be a good excuse to learn a little bit about programming :) [01:07:55] if you can do sql queries you can learn just enough python to do something like that I think [01:09:40] I can't do sql queries...I just copy+paste [01:10:08] Or, someone could fix that special:LonelyPages to work again... [01:10:31] which works serverside and not via toollabs... [01:20:13] bd808: I have no idea what I'm doing xD I'm just removing random lines in that query from before xD [01:21:23] I think the reason that special page isn't getting updates anymore is related to your attempts to recreate the data [01:22:09] hmm....but there got to be a way... [01:22:44] I'm gettting obsessed, but not as obsessed as to acually learn how to code... [01:24:46] so the problem is basically one of scale. If pages only had 100K rows it would be fine [01:25:14] but pages has 5M+ rows and I have no idea how big pagelinks and templatelinks are [01:26:32] to answer the question you are asking you have to multiply rows in pages by rows in pagelinks by rows in templatelinks, hold that in ram and look for places where pagelinks and templatelinks have holes in them [01:27:13] tehre are some shortcuts that the db server can take but not many unfortunately [01:27:33] :/ [01:28:06] so sad :( [03:22:23] 06Labs, 10Tool-Labs, 10Tool-Labs-tools-Other: Provide a list of the videos available on video2commons servers - https://phabricator.wikimedia.org/T134914#2283756 (10zhuyifei1999) The backends are currently [[https://wikitech.wikimedia.org/wiki/Nova_Resource:Video/Help/video2commons#Nginx|nginx-powered withou... [03:22:39] 06Labs, 10Tool-Labs, 10Tool-Labs-tools-Other: Provide a list of the videos available on video2commons servers - https://phabricator.wikimedia.org/T134914#2283757 (10zhuyifei1999) a:03zhuyifei1999 [04:38:16] PROBLEM - SSH on tools-worker-1004 is CRITICAL: Connection refused [05:54:36] abartov: btw, newer, much nicer (and stabler) way of running generic webservices on tools now: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web#Other_.2F_generic_web_servers [05:55:39] YuviPanda: thanks! [05:56:41] abartov: np. [06:47:30] PROBLEM - Puppet run on tools-exec-cyberbot is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0] [10:09:37] hi, I just asked this question in #wikimedia-mobile, repeating here. is there some place people can point me toward that documents mediawiki, GPS/location based experiments people are doing? [10:42:47] jgay: perhaps the discovery / maps team? #wikimedia-discovery / yurik [10:42:59] ? [10:43:23] YuviPanda, #wikimedia-interactive ;) [10:43:54] ah or that :) [10:45:43] YuviPanda, yurik, sweet, thanks. [11:34:48] !ping [11:34:48] !pong [11:56:09] 10Tool-Labs-tools-Other, 10Analytics, 06Community-Tech, 10Pageviews-API, and 2 others: Pageview Stats tool - https://phabricator.wikimedia.org/T120497#2284462 (10Izno) [12:26:39] Hi, is it possible to get some more disk space for a wmflab vm? If so, who should I contact? [12:27:23] I think so [12:29:34] Though I can't see documentation for it in our OpenStack docs [12:30:30] https://wikitech.wikimedia.org/wiki/Help:Shared_storage [12:30:46] gehaxelt, maybe file a ticket in Maniphest against #Labs? [12:36:16] Krenair, okay, the shared storage wiki helps a bit. I'd like to setup a local hadoop (hdfs) instance on the vm, but I don't think that putting the hdfs storage folder onto nfs is a good idea (it should fall into the don'ts-list). [13:04:58] 06Labs: More local storage on a wmflabs vm? - https://phabricator.wikimedia.org/T134986#2284914 (10Gehaxelt) [13:08:30] 06Labs: More local storage on a wmflabs vm? - https://phabricator.wikimedia.org/T134986#2284914 (10Andrew) It's not possible to resize an existing VM. You can certainly create a new larger one and migrate your work over to it; let me know if you need a change in your disk space quotas to do so. [13:31:03] 06Labs: Put a firewall on labtestcontrol2001 - https://phabricator.wikimedia.org/T132598#2285037 (10MoritzMuehlenhoff) 05Open>03Resolved labcontrol2001 now uses base::firewall [14:03:48] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 06Developer-Relations, 07Documentation: Run a documentation sprint for Labs - https://phabricator.wikimedia.org/T101659#1344508 (10chasemp) >>! In T101659#2238481, @Qgil wrote: > Would this task or any of the subtasks be a good candidate for #wikimania-hac... [14:31:27] gehaxelt: yup, definitely don't put hdfs on NFS... [14:32:02] I won't, promise :) [14:36:14] PROBLEM - SSH on tools-webgrid-lighttpd-1408 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:40:17] gehaxelt: :D [14:50:18] 06Labs: More local storage on a wmflabs vm? - https://phabricator.wikimedia.org/T134986#2284914 (10Physikerwelt) @Andrew Thank you for looking into this issue. @Gehaxelt is working in the math cluster on the mlp instance. The 160GB was the largest disk I was able to create. (I'm not sure if that's due to quotas... [15:22:14] labsadmin here? [15:24:40] what's up? [15:26:01] andrewbogott: Can you take a look at tools.merlbot? It stopped working at the 9th may, and we are now not sure, if that was expected by merl, or not [15:26:13] The fact is, that currently no jobs from this tool are running [15:26:24] and merl is currently offline, since the first february [15:27:04] so by 'take a look' you mean restart it? [15:27:42] andrewbogott: it's starting jobs by crontab, IIRC, so maybe take a look at first, if the stop is expected is at first enough ;) [15:28:00] (it's not a webservice, but the most active bot @dewiki [15:28:01] ) [15:32:11] bd808: ^ I wonder if this is related to jsub rewrite? [15:32:14] bd808: I suspect that ^ is related to the crontab failures we had on the 9th. Are you around to help investigate? [15:32:18] :) [15:32:20] merl was one of the ol' school qsub people [15:32:29] I'm also taking a look, but am a few thousand feet in the air [15:32:49] in fact, I'm *not* taking a look until after I land... [15:33:04] Probably is my fault. I'll poke around [15:33:20] There was a phab task about merl bot and jdk 8 a couple days ago... [15:33:49] woah. that's a lot of *.err files [15:33:55] so many jobs [15:34:29] bd808: Yeah, merl said that all of his jobs together are about running more than 60 hours a day [15:34:57] bd808: Maybe that helps, the last succeful edit of the bot was at the 9th may at 17:17:50 UTC [15:35:50] A lot of the *.err logs end with several occurrences of "there is a job named '*' already active" [15:36:08] bd808: yeah, but actually there is not active job [15:37:50] I found a heavy user of config in script comments... :/ [15:38:29] and he is using qsub args that I haven't supported in the new jsub [15:38:46] so yeah... my fault in some way [15:40:59] Luke081515: can you open a phab ticket for this please? [15:41:08] ok [15:41:49] YuviPanda: there is some serious qsub wizardry going on in this account. I'm not sure you will like it much. [15:42:21] very low level things like `-l hostname=!tools-exec-12*` and `-l user_slot=31` [15:45:16] teh hostname thing looks like a backwards way to set trusty online [15:45:22] only I mean [15:45:41] bd808: are all these in cron? [15:45:41] 06Labs, 10Tool-Labs: tools.merlbot stopped working - https://phabricator.wikimedia.org/T135006#2285495 (10Luke081515) [15:45:49] yeah [15:45:54] bd808: if so I feel ok with hacking it until we get it to work and documenting it for merl to look at [15:45:58] yeah the -l looks like that [15:46:25] 06Labs, 10Tool-Labs: tools.merlbot stopped working - https://phabricator.wikimedia.org/T135006#2285512 (10Luke081515) MerlBot is the most active bot at dewiki, so a quick solution would be very great for the community there. [15:47:13] there are 78 *.qsub files in the tool $HOME. A subset of them are run via cron [15:50:58] bd808: was it using qsub directly? [15:51:36] YuviPanda: no, but using shell scripts that use the #$ -flag comments to set lots of qsub settings [15:51:46] so jstart foo.qsub [15:52:39] sigh [15:52:59] * YuviPanda isn't sure what to do [15:53:01] it's not too bad. I have a list of the flags the scripts are using now [15:53:10] short term I'll add support for them [15:53:41] https://www.mediawiki.org/wiki/Toolserver:Job_scheduling [15:53:44] never saw that before [15:53:55] user_slot=1 [15:53:55] This resource is limit to 10 slots for each user. It has no specific meaning and can be used for limiting the number of job that are executed in parallel by a single user. [15:53:59] weirdly documented? [15:55:09] chasemp: SGE has actually hundreds of config settings [15:55:54] yeah, thought this was curious as it's from original tool server and seems to say that setting is invalid at 31 [15:56:00] along w/ "has no specific meaning..." [16:04:30] 06Labs, 10Tool-Labs, 15User-bd808: tools.merlbot stopped working - https://phabricator.wikimedia.org/T135006#2285549 (10bd808) a:03bd808 This looks to be fallout from the rewrite of `jsub`. The cronjobs run by tools.merlbot use several qsub options that are not currently supported by the jsub runner script. [16:06:42] thanks bd808 [16:07:02] I broke it so I should have to fix it :) [16:08:19] bd808: that's a praiseworthy working attitude ;) [16:08:58] @seen volons [16:08:58] Cyberpower678: I have never seen volons [16:09:03] wut [16:09:15] @seen Volons [16:09:15] Cyberpower678: I have never seen Volons [16:09:20] @seen Volon [16:09:20] Cyberpower678: I have never seen Volon [16:09:24] @seen volon [16:09:24] Cyberpower678: I have never seen volon [16:09:27] WTH? [16:09:43] @seen Cyberpower678 [16:09:44] Cyberpower678: are you really looking for yourself? [16:09:50] @seen petan [16:09:50] Cyberpower678: petan is in here, right now [16:09:52] Cyberpower678: volans [16:09:57] ^ [16:09:58] oh [16:10:07] @seen volans [16:10:07] Cyberpower678: Last time I saw volans they were talking in the channel, they are still in the channel #wikimedia-databases at 5/11/2016 3:42:22 PM (27m44s ago) [16:10:37] Cyberpower678: he's at #wikimedia-operations IIRC [16:10:54] found him in -databases [16:35:03] btw, is there a way to connect to bastion-02 too? [16:38:12] Luke081515, yes [16:38:39] Last time I checked, most of the bastion project's old domains in ldap-based dns didn't get copied [16:38:55] but you should be able to ssh to the IP - 208.80.155.153 [16:39:11] (listed at https://wikitech.wikimedia.org/wiki/Nova_Resource:Bastion ) [16:39:46] Krenair: hm, ok, but is there a domain I can connect to, such as primary.bastion.wmflabs for bastion-01? [16:42:29] I'm not an admin of the bastion project so can't pull that information through horizon right now [16:42:35] Pretty sure there's a bug about that. . . [16:43:20] Oh, other problem is lack of up-to-date python-designateclient on silver [16:43:56] Luke081515: secondary.bastion? ;-) [16:44:39] (03PS1) 10BryanDavis: jsub: Add support for qsub args used by tools.merlbot [labs/toollabs] - 10https://gerrit.wikimedia.org/r/288226 (https://phabricator.wikimedia.org/T135006) [16:44:56] ah, that works :D [16:45:14] maybe I should check the obvious way first, next time :D [16:45:37] valhallasw`cloud: can you give https://gerrit.wikimedia.org/r/#/c/288226 a quick look? [16:46:00] bd808: alternative option: migrate merlbot to use qsub rather than jsub? [16:46:30] well... that *could* be done. it's just kicking the can a bit farther down the road [16:46:45] I am admin of wmflabsdotorg and can look at those domains directly under there [16:46:46] or allow a -qsubparams "-t 3 -q 5 etc" parameter [16:46:57] but I'd prefer the qsub option [16:47:08] because otherwise we need to somehow migrate the behavior to k8s [16:47:33] seems to just have bastion-restricted.wmflabs.org., bastion-eqiad.wmflabs.org. [16:47:40] merlbot is a heavy user of array jobs [16:47:52] which is going to cause a problem either now or later for sure [16:48:00] right [16:48:25] but the problem is, that the maintainer is not reachable at the moment [16:48:25] switching the cron to use qsub directly would be pretty easy [16:49:18] and the is no license and there are no co-maintainers I bet :/ [16:50:25] oh the maintainer list is scary [16:50:39] it has other tools as co-maintainers [16:50:46] okay, managed to get the list via designateclient.v1 [16:50:54] [u'restricted.bastion.wmflabs.org.', u'primary.bastion.wmflabs.org.', u'secondary.bastion.wmflabs.org.', u'bastion.wmflabs.org.'] [16:53:44] bd808: The point is that merl is an java expert, so it's hard to find a maintainer, who can understand the code.... [16:54:17] really? java devs are hard to find these days? [16:54:38] kids these days :) [16:56:03] valhallasw`cloud: ther `-once` flag for jsub is used on pretty much every cron line. Is there a direct qsub replacement for that? [16:56:29] oof. [16:56:36] I'm not sure [16:57:00] no, I don't think so [16:57:22] How about we patch jsub now to get the jobs running again and leave a task open to find a way to migrate later [16:57:46] The breakage now was mostly unintended [16:57:55] bd808: I'd prefer a way to pass any qsub commands through, but in a clear 'this will not be supported forever' way [16:58:21] hmmm [16:58:22] so e.g. the -qsubparams "-t 3 -q 5 etc" option. Would that be doable? [16:59:24] it would take rewriting 78 *.qsub scripts [16:59:44] on the jsub side, yeah it's not that hard to figure out how to do [16:59:51] * valhallasw`cloud nods [17:00:03] alternatively... we can re-deploy the old jsub as jsub_unsupported [17:01:45] 06Labs: Backup files request - https://phabricator.wikimedia.org/T135014#2285835 (10Mjbmr) [17:01:45] Or copy it into ~/bin for merlbot? [17:03:01] guess the jsub_unsupported would a be a good temporary solution, till we found a good alternative, what do you thimk? [17:03:04] *think [17:14:11] valhallasw`cloud: putting the perl version into ~tools.merlbot would be the quickest punt [17:20:56] PROBLEM - Host tools-worker-1011 is DOWN: PING CRITICAL - Packet loss = 100% [17:22:48] valhallasw`cloud: did there used to be a runner script called "jlocal" ? [17:24:52] * bd808 finds reference to it in https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Grid and the merlbot crontab [17:33:47] bd808: yes, on the cron hosts [17:34:01] it runs whatever is passed as argument [17:34:36] *nod* [17:35:28] bd808: valhallasw`cloud related: https://phabricator.wikimedia.org/T134846 [17:35:34] I'm writing up the hack fix. I have setup ~/bin for tools.merlbot and prepended that to the $PATH both for interactive shells and in the crontab [17:35:42] bd808: it still exists unfortunately (jlocal) despite my attempts to fix it many times. [17:35:46] it's in puppet [17:35:54] valhallasw`cloud: also growing body of actual docs in https://wikitech.wikimedia.org/wiki/Tools_Kubernetes [17:36:00] YuviPanda: <3 [17:37:32] valhallasw`cloud: needs more docs on what our current container image strategy is, and our LDAP workarounds as well as how our proxy works [17:37:44] valhallasw`cloud: once that's all done it's just then practical commands and 'HOW DO I?' stuff I think [17:38:52] bd808: if you manage to test it let me know and I can merge, although maybe you want to try doing the deployment yourself [17:39:15] YuviPanda: I'm going with a more merlbot specific hack [17:39:22] writing it up now [17:39:24] bd808: aaah, cool! [17:44:15] 06Labs, 10Tool-Labs, 13Patch-For-Review, 15User-bd808: tools.merlbot stopped working - https://phabricator.wikimedia.org/T135006#2286069 (10bd808) After discussion with @valhallasw on irc, we are not really excited about adding support in the new version of `jsub` for the `qsub` array job settings that too... [17:45:12] (03CR) 10BryanDavis: [C: 04-1] "See T135006 for documentation of local fix for tools.merlbot" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/288226 (https://phabricator.wikimedia.org/T135006) (owner: 10BryanDavis) [17:45:16] bd808: the cron perl hack looks for the word 'jsub' in the cron entries, so you might be ok [17:46:50] I'm out for a while for lunch with a friend. I'll try to check on merlbot stuff when I get back [17:47:32] I fear we may have the same problem for Tools.merlbot2, Tools.merlbot-web, Tools.merlbot-read [17:48:28] * YuviPanda nods [18:00:50] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 06Developer-Relations, 07Documentation: Run a documentation sprint for Labs - https://phabricator.wikimedia.org/T101659#2286151 (10Aklapper) >>! In T101659#2285264, @chasemp wrote: > Is there a hackathon as part of wikimania again? [[ https://wikimania201... [18:14:14] 06Labs, 10DBA: No replica for adywiki - https://phabricator.wikimedia.org/T135029#2286195 (10MaxSem) [18:18:23] 06Labs, 10Labs-Infrastructure: Install python-requests-oauthlib on labs - https://phabricator.wikimedia.org/T130529#2286214 (10Steinsplitter) >>! In T130529#2138614, @Billinghurst wrote: > To note that it was also suggested by @anomie that I run as trusty, I had tried that and it still seems to fail > Lo... [18:44:45] !log ocg create project, add dzahn, cscott as admins [18:44:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ocg/SAL, Master [18:59:32] what's with the failed exim on puppet run on each fresh instance [18:59:46] before it has a specific class configured [19:00:08] always the Error: Could not start Service[exim4]: Execution of '/usr/sbin/service exim4 start' from "standard" i guess [19:00:14] but it wasnt always like that? [19:03:35] May 11 18:59:09 ocg-jessie-01 exim4[15415]: main option "add_environment" unknown [19:03:40] creates ticket [19:04:09] mutante: no it wasn't and andrewbogott I believe already knows and is building a new image [19:04:14] not sure an issue exists or not [19:05:19] chasemp, mutante, it's some mismatch with new exim config and old exim version [19:05:33] I have a new base image which should fix it… meanwhile if you apt-get install exim it should be better [19:06:05] i found this: [19:06:07] Warning! Invalid configuration file for exim4. Exiting....failed. [19:06:14] from the main option "add_environment" unknown [19:06:24] thanks chasemp and andrewbogott [19:06:30] https://phabricator.wikimedia.org/T135033 [19:06:34] had already saved it [19:07:12] apt-get install exim = package exim doesnt exist anymore.. but "exim4-base" [19:08:17] !log ocg ocg-jessie-01 apt-get install exim4-base fixed issue reported in T135033 [19:08:17] T135033: failed exim service on labs instances - https://phabricator.wikimedia.org/T135033 [19:08:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ocg/SAL, Master [19:08:38] andrewbogott: yes, it fixed it, but have to use that package name [19:08:53] :) [19:09:01] good for now [19:11:43] 06Labs, 10Mail: failed exim service on labs instances - https://phabricator.wikimedia.org/T135033#2286378 (10Aklapper) [19:44:19] 06Labs, 10DBA, 06Operations, 07Blocked-on-Operations: No replica for adywiki - https://phabricator.wikimedia.org/T135029#2286518 (10Krenair) Am guessing this needs ops to run maintain-replicas.pl [20:07:36] 06Labs, 10Mail: failed exim service on labs instances - https://phabricator.wikimedia.org/T135033#2286585 (10Dzahn) 12:57 actually solving the exim thing is just new images AFAIK, but that's blocked by a different ticket about the software building it creating broken images, if I understand correctly [20:09:07] 06Labs, 10Mail: failed exim service on labs instances - https://phabricator.wikimedia.org/T135033#2286589 (10Dzahn) if somebody knows that ticket that blocks it, please link [20:11:40] 06Labs, 10Tool-Labs, 13Patch-For-Review, 15User-bd808: tools.merlbot stopped working - https://phabricator.wikimedia.org/T135006#2286592 (10bd808) I set `PATH=/data/project/merlbot/bin:$PATH` for interactive and cron managed jobs in the tools.merlbot2 project as well. That tool account is a member of the t... [20:14:54] 06Labs, 10Labs-Infrastructure, 10Horizon: Horizon web interface lost CSS/JS assets - https://phabricator.wikimedia.org/T135041#2286619 (10hashar) [20:28:41] 06Labs, 10Labs-Infrastructure, 10Horizon: Horizon web interface lost CSS/JS assets - https://phabricator.wikimedia.org/T135041#2286619 (10Krenair) happened to me earlier too [20:40:25] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 13Patch-For-Review, 15User-bd808: tools.merlbot stopped working - https://phabricator.wikimedia.org/T135006#2286764 (10bd808) [20:45:38] 06Labs, 10Labs-Infrastructure, 07Privacy: Whitelist labs instances that need XFF header passed through the web proxy - https://phabricator.wikimedia.org/T135046#2286772 (10csteipp) [20:47:14] 06Labs, 10Labs-Infrastructure, 07Privacy: Whitelist labs instances that need XFF header passed through the web proxy - https://phabricator.wikimedia.org/T135046#2286787 (10yuvipanda) (the one instance is UTRS). We can do this by maintaining a whitelist of IPs / domains in puppet, and adding an if in nginx f... [20:55:58] YuviPanda: I've heard that the postgresql puppet module is used on labs, but I can't find the reference greping through our puppet code. I'm probably blind, but any chance you could point me in the right direction? [20:57:09] gehel: there's a shared postgres server; I think it's probably using modules/postgresql/* ? [20:57:52] https://github.com/wikimedia/operations-puppet/tree/production/modules/postgresql [20:58:39] valhallasw`cloud: probably... but no reference in site.pp I guess we use another way to do categorization in labs? [20:59:33] I'm actually trying to do some refactoring on the postgresql module, but I'd like to see how it is used to not break too many things... [21:00:38] gehel: labs uses wikitech to configure hosts rather than site.pp [21:00:49] yuvi built a tool to see which hosts use a given manifest... /me checks [21:01:05] valhallasw`cloud: Right, I now remember. Easier to use, but harder to grep... [21:02:18] ah, watroles [21:02:20] but it's broken [21:02:21] ugh. [21:02:46] an nginx 404? what... [21:02:46] bd808: thanks for bringing the MerlBot back. The first jobs started already, and IIRC the main part will start at about 4-5 UTC ;) [21:03:04] Luke081515: awesome. [21:03:32] Matthew_: xtools-ec is slow [21:03:43] very slow, the page won't load here [21:04:18] 10Tool-Labs-tools-Other: watroles 404's - https://phabricator.wikimedia.org/T135053#2286902 (10valhallasw) [21:11:28] Luke081515: looking in a minute. [21:12:14] 10Tool-Labs-tools-Other: watroles 404's - https://phabricator.wikimedia.org/T135053#2286902 (10bd808) https://tools.wmflabs.org/watroles/role/role::puppet::self works. As I recall the 404 for https://tools.wmflabs.org/watroles/ is because the app has no route handling other than the role lookup. [21:13:12] 06Labs, 10Labs-Infrastructure, 07Privacy: Whitelist labs instances that need XFF header passed through the web proxy - https://phabricator.wikimedia.org/T135046#2286772 (10Krenair) Labs instances can get public IPs though [21:13:40] 10MediaWiki-extensions-OpenStackManager, 10MediaWiki-Authentication-and-authorization, 06Reading-Infrastructure-Team: Update OpenStackManager to use AuthManager - https://phabricator.wikimedia.org/T110288#1574317 (10Anomie) The hard part is done with the merge of {ca5602fbc4}. The one bit remaining is the us... [21:15:00] 10MediaWiki-extensions-OpenStackManager, 10MediaWiki-Authentication-and-authorization, 06Reading-Infrastructure-Team: Update OpenStackManager to use AuthManager - https://phabricator.wikimedia.org/T110288#2286970 (10Anomie) Note that "do the PreAuthenticationProvider" probably also wants it converted to use... [21:16:26] 06Labs, 10Labs-Infrastructure, 06WMF-Legal, 07Privacy: Whitelist labs instances that need XFF header passed through the web proxy - https://phabricator.wikimedia.org/T135046#2287017 (10ZhouZ) [21:19:15] Luke081515: Try now. [21:19:34] Matthew_: thx, works now :) [21:19:40] Good :) [21:19:45] Just had to reboot the webservice. [21:20:02] !log tools.xtools Restarted webservice for xtools-ec, as it was running slow. [21:20:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.xtools/SAL, Master [21:33:36] Matthew_: An addtional issue for you: If I click at the wikidatalink at -ec, to view wikidata analysis, I get: wt::getUserInfo is not a valid wiki [21:33:47] wikidata works normal, but not at this way [21:34:15] Huh. OK. Do me a favor and file a Phab task, assign it to me. I'll look into it in a bit. [21:34:40] ok [21:34:51] Because that appears to actually be an issue. [21:38:04] Matthew_: T135067 [21:38:05] T135067: Can't use the link to wikidata at xtools-ec - https://phabricator.wikimedia.org/T135067 [21:38:26] Luke081515: Thank you :) [21:38:44] np ;) [21:46:51] !log tools.xtools Changed cron job that restarts the xtools-ec webservice to run every three hours instead of four [21:46:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.xtools/SAL, Master [22:47:34] valhallasw`cloud: it's because I am lazy and wrote no home page, http://tools.wmflabs.org/watroles/role/role::puppet::self [23:01:34] 06Labs, 07Diamond: diamond: no NFS mount points were found - https://phabricator.wikimedia.org/T135078#2287630 (10Dzahn) [23:03:10] 06Labs, 07Diamond: diamond: no NFS mount points were found - https://phabricator.wikimedia.org/T135078#2287619 (10Dzahn) [23:03:44] 06Labs, 07Diamond: labs: diamond: no NFS mount points were found - https://phabricator.wikimedia.org/T135078#2287636 (10Dzahn) [23:04:23] 06Labs, 07Diamond: labs: diamond: no NFS mount points were found - https://phabricator.wikimedia.org/T135078#2287619 (10Dzahn) p:05Triage>03Low [23:07:14] 06Labs, 10DBA, 06Operations, 07Blocked-on-Operations: No replica for adywiki - https://phabricator.wikimedia.org/T135029#2286195 (10Dzahn) it's a "labsdb" thing afaict. one hit in entire wikitech " springle: ran operations/software maintain-replicas.pl and fedtables.pl on labsdbs" [23:13:59] 06Labs, 10DBA, 06Operations, 07Blocked-on-Operations: No replica for adywiki - https://phabricator.wikimedia.org/T135029#2287649 (10Dzahn) 05Open>03stalled [23:19:01] 06Labs, 10WM-Bot, 07Privacy: http://wm-bot.wmflabs.org/browser/ is loading assets from multiple 3rd party domains - https://phabricator.wikimedia.org/T133644#2287652 (10Andrew) We've just now discussed this some more. Here's my understanding: 1) Everyone agrees that leaking user IPs to these third party do... [23:20:06] 06Labs, 10WM-Bot, 07Privacy: http://wm-bot.wmflabs.org/browser/ is loading assets from multiple 3rd party domains - https://phabricator.wikimedia.org/T133644#2287654 (10yuvipanda) You could also just use tools.wmflabs.org/cdnjs to load the assets from instead. [23:20:37] 06Labs, 10WM-Bot, 07Privacy: http://wm-bot.wmflabs.org/browser/ is loading assets from multiple 3rd party domains - https://phabricator.wikimedia.org/T133644#2287660 (10yuvipanda) Note that the ambiguity exists only for Labs projects - not for tools projects, since projects on tools abide explicitly by the W... [23:24:32] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 13Patch-For-Review, 15User-bd808: tools.merlbot stopped working - https://phabricator.wikimedia.org/T135006#2285495 (10scfc) I don't think this work-around (or T134846 as the pièce de résistance) is an adequate solution, as there is no "problem" to be solv... [23:37:08] 10Tool-Labs-tools-Other: Fix tool kmlexport - https://phabricator.wikimedia.org/T92963#2287709 (10bd808) @Para please take a minute to respond to T92963#2265474 by stating an OSI-approved license that you find acceptable. Until you do this, no one can actually step in to help solve the issues raised by @Thgoiter.