[00:05:44] questions about hadoop, if anyone has a minute. I guess i've forgotten too much since working on this before the holidays... [00:06:01] i can use `oozie job -info 0016105-160113150205991-oozie-oozi-W@aggregate` to get info about a workflow oozie kicked off for me, but that reports using job job_1441303822549_412383 [00:06:28] trying to `yarn logs -applicationId job_1441303822549_412383` says invalid ApplicationId specified. Additionally hue is just erroring out when i try and view the workflow [00:07:34] https://hue.wikimedia.org/oozie/list_oozie_workflow/0016105-160113150205991-oozie-oozi-W/ specifically [00:08:48] i can just look at /mnt/hdfs/var/log/hadoop-yarn/apps/ebernhardson/logs/application_1441303822549_412383/analytics1035.eqiad.wmnet_8041 directly, but being a binary file it's a pain [00:09:31] which answers my question about `yarn logs` ... i was using the wrong type of application ID. nevermind :) [00:10:09] Analytics: Add wm:BE-Wikimedia-Belgium to Wikimetrics tags {dove} - https://phabricator.wikimedia.org/T124492#1958184 (kevinator) thread on mailing list: https://lists.wikimedia.org/pipermail/wikimetrics/2016-January/000376.html [04:12:43] so odd...spark decided to use teh entire storage memory on 1 executor out of 10. The rest are completely empty :S http://analytics1001.eqiad.wmnet:8088/proxy/application_1441303822549_412742/storage/rdd?id=17 [04:13:03] (and the stage is failing..guessing because that one runs out of memory? dunno...) [05:17:45] Quarry: Allow users to choose free licenses other than CC0 - https://phabricator.wikimedia.org/T124528#1958791 (Dalba) NEW [05:21:59] Quarry, Labs: 502 Bad Gateway on HTTP-requests to quarry.wmflabs.org - https://phabricator.wikimedia.org/T121502#1958799 (Krenair) Is it still doing this to you? Can't reproduce the issue... [05:26:17] Quarry: Allow users to choose free licenses other than CC0 - https://phabricator.wikimedia.org/T124528#1958801 (yuvipanda) Thank you for filing this bug! I considered this when initially setting it up and decided that CC0 is the one that makes most sense, since it is the easiest to remix and re-use. This is... [10:06:22] Quarry: Allow users to choose free licenses other than CC0 - https://phabricator.wikimedia.org/T124528#1958882 (Dalba) I know, It's obviously easier to work with CC0 code, but [[https://www.fsf.org/blogs/rms/public-domain-manifesto|not everyone]] is willing to use that license. Though I still think that it's... [12:24:27] Analytics, Wikimedia-Developer-Summit-2016: Developer summit session: Pageview API from the Event Bus perspective - https://phabricator.wikimedia.org/T112956#1958944 (Milimetric) Open>Resolved This task took place, and it now serves two purposes. First is as a place where we gathered a *lot* of... [13:16:53] Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1958974 (jcrespo) The maintenance for db1046 has finished. We have saved over 600GB of space. The application can start a... [14:57:37] using pageviews api with python 3, if the url has spaces the API woun't return the expected resoult, any idea on how to overpass it? [15:34:18] Quarry: Allow users to choose free licenses other than CC0 - https://phabricator.wikimedia.org/T124528#1959021 (Aklapper) p:Triage>Lowest [20:34:05] ebernhardson: when you use applicationId the prefix should be application_ [20:34:34] s/yarn logs -applicationId job_1441303822549_412383/yarn logs -applicationId application_1441303822549_412383 [20:35:08] also oozie job -info 0016105-160113150205991-oozie-oozi-W without @aggregate may be? [20:35:41] ah you found out :) okay [20:42:36] yea eventually figured it out :) sometimes just needs some rubber duck debugging [20:43:23] * madhuvishy hands over her surfer duck [21:50:01] Quarry, Labs: 502 Bad Gateway on HTTP-requests to quarry.wmflabs.org - https://phabricator.wikimedia.org/T121502#1959316 (Stigmj) No. The wally period expired long ago and the issue has gone away... [21:50:43] Quarry, Labs: 502 Bad Gateway on HTTP-requests to quarry.wmflabs.org - https://phabricator.wikimedia.org/T121502#1959318 (Krenair) Open>Resolved a:Krenair [22:37:13] Quarry, Internet-Archive: quarry.wmflabs.org/robots.txt gives 502 - https://phabricator.wikimedia.org/T105662#1959397 (Krenair) According to Yuvi, this is uwsgi swallowing these exceptions: ```File "/srv/venv/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__ return self.wsgi_app(environ,... [23:32:09] (PS1) Alex Monk: Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) [23:32:26] Quarry, Internet-Archive, Patch-For-Review: quarry.wmflabs.org/robots.txt gives 502 - https://phabricator.wikimedia.org/T105662#1959438 (Krenair) a:Krenair [23:32:36] (CR) jenkins-bot: [V: -1] Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) (owner: Alex Monk) [23:33:21] (PS2) Alex Monk: Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) [23:33:45] (CR) jenkins-bot: [V: -1] Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) (owner: Alex Monk) [23:34:04] (PS3) Alex Monk: Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) [23:34:28] (CR) jenkins-bot: [V: -1] Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) (owner: Alex Monk) [23:37:16] (CR) Alex Monk: "recheck" [analytics/quarry/web] - https://gerrit.wikimedia.org/r/218131 (owner: Yuvipanda) [23:37:32] (CR) jenkins-bot: [V: -1] Update compiled javascript template for previous patch [analytics/quarry/web] - https://gerrit.wikimedia.org/r/218131 (owner: Yuvipanda) [23:37:54] (CR) Alex Monk: "recheck" [analytics/quarry/web] - https://gerrit.wikimedia.org/r/197076 (owner: Milimetric) [23:38:06] (CR) Alex Monk: "recheck" [analytics/quarry/web] - https://gerrit.wikimedia.org/r/176506 (https://phabricator.wikimedia.org/T76084) (owner: Rtnpro) [23:38:09] (CR) jenkins-bot: [V: -1] Add VIM mode [analytics/quarry/web] - https://gerrit.wikimedia.org/r/197076 (owner: Milimetric) [23:38:30] (CR) jenkins-bot: [V: -1] Remember recent queries filter last used by a user. [analytics/quarry/web] - https://gerrit.wikimedia.org/r/176506 (https://phabricator.wikimedia.org/T76084) (owner: Rtnpro) [23:42:45] (PS1) Alex Monk: Try to make Jenkins happy [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266022 [23:45:14] (CR) Yuvipanda: [C: 2] Try to make Jenkins happy [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266022 (owner: Alex Monk) [23:48:05] (PS2) Alex Monk: Remember recent queries filter last used by a user. [analytics/quarry/web] - https://gerrit.wikimedia.org/r/176506 (https://phabricator.wikimedia.org/T76084) (owner: Rtnpro) [23:48:15] (PS4) Alex Monk: Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) [23:48:29] (CR) jenkins-bot: [V: -1] Remember recent queries filter last used by a user. [analytics/quarry/web] - https://gerrit.wikimedia.org/r/176506 (https://phabricator.wikimedia.org/T76084) (owner: Rtnpro) [23:50:53] (CR) Yuvipanda: [C: 2] "<3" [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) (owner: Alex Monk) [23:51:19] (Merged) jenkins-bot: Don't die on a non-existent user page [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266021 (https://phabricator.wikimedia.org/T105662) (owner: Alex Monk) [23:54:23] Quarry, Internet-Archive, Patch-For-Review: quarry.wmflabs.org/robots.txt gives 502 - https://phabricator.wikimedia.org/T105662#1959475 (Krenair) Open>Resolved [23:58:13] Quarry: The last run time of all queries reports 5 months ago - https://phabricator.wikimedia.org/T87200#1959477 (Krenair) Open>Invalid a:Krenair Looks fine to me now. Only 4 of EpochFail's queries are listed as 5 months ago.