[00:03:57] (03CR) 10Yuvipanda: [C: 032] Broadcast mw-ext-ORES changes to #wikimedia-ai [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/247183 (owner: 10Awight) [00:04:06] (03PS3) 10Yuvipanda: Update js-yaml and replace require()s with read file and yaml.safeLoad [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/241871 (https://phabricator.wikimedia.org/T114024) (owner: 10Polybuildr) [00:04:16] (03CR) 10Yuvipanda: [C: 032] Update js-yaml and replace require()s with read file and yaml.safeLoad [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/241871 (https://phabricator.wikimedia.org/T114024) (owner: 10Polybuildr) [00:04:22] (03Merged) 10jenkins-bot: Broadcast mw-ext-ORES changes to #wikimedia-ai [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/247183 (owner: 10Awight) [00:04:32] (03PS3) 10Yuvipanda: Add a proper .jshintrc and pass JSHint [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/241904 (owner: 10Polybuildr) [00:04:44] (03Merged) 10jenkins-bot: Update js-yaml and replace require()s with read file and yaml.safeLoad [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/241871 (https://phabricator.wikimedia.org/T114024) (owner: 10Polybuildr) [00:05:02] (03CR) 10Yuvipanda: [C: 032] Add a proper .jshintrc and pass JSHint [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/241904 (owner: 10Polybuildr) [00:05:25] (03Merged) 10jenkins-bot: Add a proper .jshintrc and pass JSHint [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/241904 (owner: 10Polybuildr) [00:09:39] Hey guys sorry for going on like a broken record but if someone could take a look at my labs tool that'd be wonderful, because I have no idea what's going on over there: https://phabricator.wikimedia.org/T115920 [00:13:04] 6Labs: Need help getting a uWSGI Python-Flask Labs tool to work - https://phabricator.wikimedia.org/T115920#1737097 (10yuvipanda) I see: ```@app.route('../wm_blog_importer', methods=['GET', 'POST']) ``` in the flask code. I'm not sure what the '..' represents. Have you tried making those be ``` @app.route('/... [00:13:22] ResMar: hey [00:13:27] ResMar: I responded... [00:13:32] actually it should probably be @app.route('/wm_blog_importer', methods=['GET', 'POST']) [00:14:05] that could be, too [00:14:06] unless... I'm not understanding what you're doing [00:14:22] I'm not really sure what ../wm_blog_importer does [00:14:38] but if it's supposed to serve tools.wmflabs.org/signpostlib/wm_blog_importer from the 'signpostlib ' tool then you shouldn't be including the /signpostlib/ [00:14:43] Earwig: Oh hey I just added you on LinkedIn. Turns out we went to the same high school... [00:14:47] oh shit no way [00:14:48] One moment I need to flip over to my development computer [00:15:16] when did you graduate? [00:15:20] oh [00:15:40] I swear if I lost all those hours of my life because of a simple Flask application error... [00:16:39] our documentation sucks as well [00:16:41] :( [00:16:57] * YuviPanda feels fairly overwhelmed atm - too many things to do, not enough time / people [00:17:01] I mean...I'm not going to lie... [00:17:23] That is also true. [00:18:39] Servers are just inherently user unfriendly, though. [00:19:18] well, hopefully we can end up with something like https://devcenter.heroku.com/articles/getting-started-with-python#introduction [00:19:42] (in a glorious future!) [00:21:21] Ah so the two dots [00:21:29] That was one of my many attempts to fix it by iterative incrementation. [00:21:44] "It works locally, maybe it'll work on the web too?" [00:22:41] the @app.routes are web URL paths, no relation to the server-side filesystem [00:24:15] So what changes should I try? Just removing the two dots? [00:24:59] (03PS2) 10Yuvipanda: Add simple Dockerfile [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/238622 [00:25:55] depends what you're trying to do [00:26:43] The render_template() method expects to find those .html pages in a subfolder called /templates/ [00:26:58] That's all there... [00:27:42] > if it's supposed to serve tools.wmflabs.org/signpostlib/wm_blog_importer from the 'signpostlib ' tool then you shouldn't be including the /signpostlib/ [00:28:42] right [00:28:51] Ah, so what you're saying is that that path is incorrect. [00:28:53] Right, because .. means one directory up. [00:28:56] er, signpostlab* [00:29:33] so for example [00:29:34] https://github.com/earwig/copyvios/blob/master/app.py#L108 [00:29:40] Alright so that fixes something, but it doesn't change the base issue I don't think. [00:29:54] @app.route("/settings", methods=["GET", "POST"]) means it serves "tools.wmflabs.org//settings" [00:29:55] Which is that it doesn't serve @app.route('/') at all either. [00:30:07] that's unrelated to the template it uses for that [00:30:10] which is named by render_template("settings.mako", **kwargs) [00:30:44] Yeah. So I've removed those dots, and so it should point to the right subpage now. [00:31:09] Once I get the webservice started the code will work better! :) Webservice still won't work though. [00:32:49] The wierd thing is that it worked for a while. But then I deleted an app.py that was located in the wrong folder and it stopped working. [00:34:26] ResMar: contents of uwsgi.ini is "[uwsgi] wsgi-file=app.py" [00:37:48] Servers though \o/ [00:38:40] the wsgi-path is probably wrong... maybe it needs to be relative to a different directory? [00:39:03] I know that much, but I don't know how to get it to point somewhere else. [00:39:18] StackOverflow says: http://stackoverflow.com/questions/33205274/changing-the-application-file-of-interest-for-uwsgi [00:40:06] I don't know anything about uwsgi so I'm flying blind here. Does that seem accurate? [00:40:13] gerrit... [00:41:16] erm [00:41:18] why not do [00:41:27] wsgi-file=signpostlab/app.py [00:41:37] the default doesn't need you to have a uwsg.ini file at all [00:41:47] it just looked for app.py with a flask app named app inside ~/www/python/src [00:41:50] *looks [00:42:01] YuviPanda: he has it nested one level deeper [00:42:17] but I'm not sure why you thought you needed to go up a level? [00:42:46] Up or down? [00:42:49] I was following the instructions at https://merlijn.vandeen.nl/2015/flask-mwoauth-on-tools.html [00:43:13] valhallasw`cloud: ^ around [00:43:15] ? [00:43:22] okay so [00:43:23] the thing starts in ~/www/python/src [00:43:25] valhallasw`cloud: do you know why you setup a custom uwsgi.ini? [00:43:28] I emailed him earlier, he said he'd took at it at some point when he has time [00:43:34] originally it looks for ~/www/python/src/app.py [00:43:43] you want it to look for ~/www/python/src/signpostlib/app.py [00:43:51] ah [00:43:53] therefore you should use 'wsgi-file=signpostlab/app.py' [00:43:54] right? [00:44:25] I have a custom wsgi.ini but it's only to clean up after the awful uwsgi.log spew [00:45:08] I thought so [00:45:22] I'll try it again [00:49:44] Progress, it's now displaying the splash page (https://tools.wmflabs.org/signpostlab/) [00:49:54] The links are broken, however. [00:50:27] you have them relative to / [00:51:12] 'request.script_root' in your templates should get the right prefix [00:51:26] although there may be another variable you can use as well... there are a few... [00:51:29] I didn't even notice [00:51:47] oh wow [00:51:55] Where were you guys two hours ago? :) [00:52:26] Would request.script_root/tn_importer work? [00:52:40] Or is it request.script_root + "/tn_importer" [00:52:50] I've never worked with decorators before. [00:55:14] are you using jinja2? [00:55:27] oh, I meant from within the template [00:55:31] not in the app file [00:55:45] the URL that needs to be changed is the one you are linking is [00:55:58] *to [00:56:14] It points to https://tools.wmflabs.org/tn_importer whereas I want it to point to https://tools.wmflabs.org/signpostlab/tn_importer [00:56:23] Doesn't that mean changing the decorator? [00:56:32] no, the decorator controls where the server serves the file from [00:56:46] the template controls where the links in the splash page go to [00:57:07] you're already serving the file in the right place, as you can see from https://tools.wmflabs.org/signpostlab/tn_importer [00:58:12] Ah, right. [00:58:14] Because the link itself is pointing the wrong place. [00:58:16] D'oh. [00:58:43] So request.script_root + '/tn_importer' right? [01:04:10] uhh, yeah, but that's not what a template looks like [01:04:22] I'm confused [01:04:55] oh [01:04:58] kay [01:04:59] well [01:04:59] https://github.com/ResidentMario/signpostlab/blob/master/templates/frame.html [01:05:15] "../fc_importer" -> "fc_importer" [01:05:19] probably good enough [01:10:29] Is it just me or is Labs under heavy load right now? [01:10:50] Huh. Never mind, it seems to have gone away. [01:12:11] Great, it works! [01:12:29] Ironically I took a day off from classes today to get work done. [01:36:28] 6Labs: Need help getting a uWSGI Python-Flask Labs tool to work - https://phabricator.wikimedia.org/T115920#1737236 (10ResMar) Thanks to some wonderful sleuthing this problem has now be resolved! It turned out to be a very simple fix, via Murphy's law: just changing the uwsgi.ini name to ``` wsgi=signpostlab/a... [01:36:47] Hey that's me! [01:36:53] Next up, why pywikibot doesn't want to update. [01:51:46] Well, shit. [01:52:36] The code I get out of my script is formatted correctly. [01:52:46] But then boxing it and returning it in HTML removes all the spaces. [01:53:30] Well then. [02:12:22] Huh. [02:12:38] pywikibot is complaining that "1.27.0-wmf.3" is an invalid version number. [02:12:51] But that's the current version... https://www.mediawiki.org/wiki/MediaWiki_1.27/wmf.3 [02:21:54] Argh. Once again I made changes locally that worked locally, but broke the webservice when deployed. [02:22:01] How the **** even [02:30:12] you need to update pywikibot [02:30:16] the version scheme changed [02:34:37] Yeah. I have some problems with pywikibot on my local machine. [02:35:00] Also it won't git pull. [02:41:11] There are some complex issues involving the installation of pywikibot on my computer and on Labs that I don't have the patience to work through at the moment. [02:41:22] Man, I hate servers. [03:00:38] 6Labs, 10Tool-Labs: Unable to boot Ruby app on tool labs - https://phabricator.wikimedia.org/T109322#1737277 (10MusikAnimal) 5Open>3Resolved a:3MusikAnimal I think I've got this figured out. I used the full path via the tilde to link to the unicorn config file, like `exec portgrabber musikanimal unicorn... [05:19:09] 6Labs, 10Tool-Labs: /public/dumps/public/ is not updating on Tool Labs - https://phabricator.wikimedia.org/T115969#1737324 (10MZMcBride) 3NEW [07:37:21] 6Labs, 10Tool-Labs, 5Patch-For-Review, 3labs-sprint-116, 3labs-sprint-117: Allow direct ssh access to tools - https://phabricator.wikimedia.org/T113979#1737416 (10yuvipanda) The new package is on copper:/var/cache/pbuilder/result/trusty-amd64/*6.9p1*, I'll start testing it tomorrow. [08:35:05] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Database-Queries: FULLTEXT not working - https://phabricator.wikimedia.org/T115940#1737526 (10jcrespo) 5Open>3Invalid a:3jcrespo @Dispenser: FULLTEXT indexes only work on strings of text, not on byte strings. MySQL replicas have the same character set than the prod... [08:35:28] Dispenser ^ [08:50:11] 6Labs, 10Labs-Infrastructure, 10Wikimedia-Apache-configuration, 6operations, and 2 others: wikitech-static sync broken - https://phabricator.wikimedia.org/T101803#1737542 (10jcrespo) @Dzahn I do not see anything in a critical state - I suppose you meant the "lag" between the servers. Did you or andrew did... [09:40:32] 6Labs, 10Beta-Cluster-Infrastructure, 10Labs-Infrastructure, 6operations: beta: Get SSL certificates for *.{projects}.beta.wmflabs.org - https://phabricator.wikimedia.org/T50501#1737694 (10Chmarkine) >>! In T50501#1669896, @Chmarkine wrote: > [[ https://letsencrypt.org/ | Let's Encrypt ]] provides free tru... [09:46:39] PROBLEM - Puppet staleness on tools-k8s-bastion-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [10:48:40] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Gordelwig was created, changed by Gordelwig link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Gordelwig edit summary: Created page with "{{Tools Access Request |Justification=provide a self-updating instance of Vizidata (http://wwwpub.zih.tu-dresden.de/~s5219191/vizidata/) |Completed=false |User Name=Gordelwig }}" [11:04:44] 6Labs, 6Discovery, 6WMDE-Analytics-Engineering, 10Wikidata, 10Wikidata-Query-Service: Wikidata Metrics - https://phabricator.wikimedia.org/T115120#1737838 (10Christopher) [11:06:28] 6Labs, 7Database: Lots of rows are missing from enwiki_p.`revision` - https://phabricator.wikimedia.org/T115207#1737840 (10Krenair) I left the query running on each labsdb overnight, and got this result: labsdb100[12]: 1107 rows labsdb1003: 1929 rows [11:55:20] 6Labs: Two small instances: for WikiToLearn development - https://phabricator.wikimedia.org/T115282#1737956 (10CristianCantoro) Hi, any updates about this? Cristian [13:38:57] (03CR) 10ArthurPSmith: "and see response from Magnus - doesn't sound like it will be fixed soon, but eventually it should work. In the meantime is the speed conce" [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/245591 (owner: 10ArthurPSmith) [14:26:36] 6Labs, 6Discovery, 6WMDE-Analytics-Engineering, 10Wikidata, 10Wikidata-Query-Service: Wikidata Metrics Labs project - https://phabricator.wikimedia.org/T115120#1738253 (10Addshore) [15:07:36] 6Labs, 10Labs-Infrastructure, 6operations, 10ops-eqiad: Rack/Setpup labvirt1010 and 1011 - https://phabricator.wikimedia.org/T116019#1738448 (10Cmjohnson) 3NEW a:3Cmjohnson [15:11:57] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Gordelwig was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=195472 edit summary: [15:18:34] 6Labs: Neutron networking, with IPv6 at eqiad - https://phabricator.wikimedia.org/T85611#1738498 (10hashar) [16:10:05] hi all - does anyone have more knowledge of the lets-encrypt certificate authority ? [16:11:12] I saw in an article yesterday that a certain milestone was passed, cross-signatures from IdenTrust [16:11:50] .. wondering if this is *only* for browsers, or is it intended to be a general purpose certificate for signing (e.g. code) [16:18:14] Well I think first aim is to provide certificates for TLS for everyone with easy automatical setup for common webservers [16:28:33] yes [16:29:29] although I am not sure I would accept "automatical" in a game of Scrabble ;-) [16:39:07] sorry I am not a native speaker [16:40:05] sometimes there might be some German in my posts ;) [16:43:02] schönen dank [16:58:40] 6Labs, 10Tool-Labs: /public/dumps/public/ is not updating on Tool Labs - https://phabricator.wikimedia.org/T115969#1738979 (10ArielGlenn) when the dblists were moved to a subdirectory, this changeset https://gerrit.wikimedia.org/r/#/c/244743/ did not catch the monitor conf, which affects generation of the lis... [17:39:07] 6Labs: Map Warper - No space left on device - https://phabricator.wikimedia.org/T102414#1739188 (10Chippyy) [17:39:09] 6Labs, 10Maps, 5Patch-For-Review: maps-warper /mnt vbd partition errored, turned read only and went missing after reboot - https://phabricator.wikimedia.org/T112641#1739185 (10Chippyy) 5Open>3Resolved a:3Chippyy Okay, I'm assuming that the /mnt filesystem is gone, not actually needed and not important... [17:51:52] 6Labs, 6Discovery, 10Maps: Request to enable critical alert notifications for maps-warper instance on shinken.wmflabs.org - https://phabricator.wikimedia.org/T116053#1739286 (10Chippyy) 3NEW a:3yuvipanda [17:53:18] chippy I think the map warper is suffocating :-( [17:53:40] dbb, it was, but it's getting better [17:53:56] what are the specs on its VM ? [17:56:37] dbb, it's a labs instance, 4gb ram, 2cpu etc [17:56:56] but the disk space was filling up so it caused issues [17:57:03] that's sorted now [17:57:36] A VM we use for a lot of heavy work in San Diego is .. Xeon X7460 @ 2.6 GHz, 8 cores, 32GB RAM, /boot 100M, /home 500GB, some TB additional mounts [17:58:26] ok, so you have more like a 2 core, 4GB RAM sort of setup [17:59:06] not sure what you are warping, but geoTIFF can get really really big these days [18:00:12] if its rendered vectors of Openstreetmap only, thats different [18:00:50] dbb, it basically shells out to gdal utilities, it warps map images from commons [18:01:12] map images ? [18:01:22] so generally they aren't that big. historical map scans [18:01:34] geoTIFF comes from satellites and drones [18:02:06] you can also make your own via the warper [18:08:52] do you have an example URL for a "map image from commons" ? [18:18:16] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/ErrantX was created, changed by ErrantX link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/ErrantX edit summary: Created page with "{{Tools Access Request |Justification=stats.grok.se is down and the maintainer looks to be AFK. Would like to put together a replacement. |Completed=false |User Name=ErrantX }}" [18:31:50] 10Quarry, 6Analytics-Backlog: Build an internal Quarry instance to share data & sample queries between researchers (and other analytics users?) - https://phabricator.wikimedia.org/T75142#1739514 (10madhuvishy) [19:07:10] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/ErrantX was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=195579 edit summary: [21:50:53] 10Tool-Labs-tools-Other, 6Community-Tech: Fix broken Edit Count tool on labs - https://phabricator.wikimedia.org/T111512#1740243 (10kaldari) [21:53:53] 10Tool-Labs-tools-Other, 6Community-Tech: Edit Count tool on labs shows text in wrong language - https://phabricator.wikimedia.org/T111512#1740249 (10kaldari) [21:54:42] 10Tool-Labs-tools-Other, 6Community-Tech: Edit Count tool on labs shows text in wrong language - https://phabricator.wikimedia.org/T111512#1606176 (10kaldari) [21:58:51] 10Tool-Labs-tools-Other, 6Community-Tech: Edit Count tool on labs shows text in wrong language - https://phabricator.wikimedia.org/T111512#1740293 (10kaldari) Assessment for prioritization: Support: Medium+ Feasibility: Medium+ (looks like it just needs to fragment the cache on user language) Impact: Low Risk... [23:34:04] 6Labs, 10Labs-Infrastructure, 6operations, 10ops-eqiad: Rack/Setpup labvirt1010 and 1011 - https://phabricator.wikimedia.org/T116019#1740677 (10Cmjohnson) Racked, cabled and ILO setup. DNS completed ge-3/0/15 labvirt1010 wmf4713 10.65.3.236 ge-3/0/16 labvirt1011 wmf4714 10.65.3.237