[07:19:20] Analytics / Refinery: Story: Admin has CDH5 running on the cluster - https://bugzilla.wikimedia.org/67130#c1 (Kevin Leduc) NEW>RESO/FIX Andrew has finished installing CDH5 on the cluster and it runs. [07:25:04] Analytics / Refinery: Story: Admin has kafkatee running in lieu of varnishcsa and udp2log - https://bugzilla.wikimedia.org/68139 (Kevin Leduc) p:Unprio>Normal [07:34:50] Analytics / EventLogging: Story: Product groups have EventLogging validation - https://bugzilla.wikimedia.org/67126 (Kevin Leduc) [07:42:06] Analytics / Visualization: Story: EEVSUser loads dashboard - https://bugzilla.wikimedia.org/68140 (Kevin Leduc) NEW p:Unprio s:enhanc a:None default metrics are loaded: Rolling Monthly Active Editors enwiki, jawiki, eswiki, ruwiki, dewiki, dewiki, frwiki, zhwiki 2007 to now [07:42:34] Analytics / Visualization: Story: EEVSUser loads dashboard - https://bugzilla.wikimedia.org/68140 (Kevin Leduc) p:Unprio>High [07:43:04] Analytics / Visualization: Story: EEVSUser loads dashboard - https://bugzilla.wikimedia.org/68140 (Kevin Leduc) [07:45:39] Analytics / Visualization: Story: EEVSUser adds/removes a metric/project - https://bugzilla.wikimedia.org/68142 (Kevin Leduc) NEW p:Unprio s:enhanc a:None Graph displays/disapears when user add/removes a metric and a project [07:46:06] Analytics / Visualization: Story: EEVSUser adds/removes a metric/project - https://bugzilla.wikimedia.org/68142 (Kevin Leduc) p:Unprio>High [09:55:26] Analytics / General/Unknown: Turn off default vhost on stat1001.wikimedia.org - https://bugzilla.wikimedia.org/68150 (christian) NEW p:Unprio s:normal a:None Currently, we're using the default vhost on stat1001.wikimedia.org for various public datasets. We should migrate away from the defau... [09:58:35] Analytics / General/Unknown: Turn off default vhost on stat1001.wikimedia.org - https://bugzilla.wikimedia.org/68150#c1 (christian) The corresponding RT ticket is RT 7858. [11:48:07] Analytics / General/Unknown: Current puppet does not allow to bring up a cluster in labs - https://bugzilla.wikimedia.org/68161 (christian) NEW p:Unprio s:normal a:None When trying to bring up a namenode in labs, puppet fails with Error: Cannot create /var/lib/hadoop/name; parent directo... [11:52:35] Analytics / General/Unknown: Current puppet does not allow to bring up a cluster in labs - https://bugzilla.wikimedia.org/68161#c1 (christian) Btw. bringing up hadoop workers with current puppet also fails with a (different) directory in /var/lib/hadoop not existing. (Again, when using puppet at ebcbe... [12:07:50] Analytics / General/Unknown: Current puppet does not allow to bring up a cluster in labs - https://bugzilla.wikimedia.org/68161#c2 (christian) It seems the part that creates /var/lib/hadoop [1] has been lost in translation for commit 87bd718e678d290b80b0916d255f1bae8666e7d7. [1] Search for "unlikely"... [13:04:50] (CR) QChris: [C: -1] "Only Nits." (5 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/146878 (owner: Ottomata) [13:51:11] nuria: are you around ? [13:53:11] ottomata is i guess [13:55:08] milimetric: can you help me please ? :) [13:55:33] hi matanya, sure! [13:55:39] (CR) QChris: [C: -1] Avoid pickle max recursion while serializing chain (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/146297 (https://bugzilla.wikimedia.org/67823) (owner: Milimetric) [13:55:39] I've got 5 min before standup though [13:55:51] but matanya - I missed anything you typed above, I just rebooted [13:56:46] hello milimetric. is a simple question. wmil would like to know the impact of the war in israel, and see if the rumors that people forced to stay home make more people sign up to wikipedia and edit [13:57:18] who can i gather the information and see how many people in the last 10 days registered and did more than 10 edits to he.wiki ? [13:57:29] hmm, wikimetrics? [13:57:31] and compare that to e.g. May? [13:57:46] matanya: yeah, wikimetrics would be the answer but right now that functionality is not user facing [13:57:48] I don't have any cohorot [13:58:02] well, wikimetrics understands "he.wiki" as a cohort [13:58:09] i tried it, but came short on usern names [13:58:21] *user names. [13:58:24] right, no, that is new stuff that's not available for everyone yet [13:58:42] now, wikimetrics doesn't have "and made 10 edits" [13:58:48] we're building that metric right now too [13:59:00] so, right right now, the easiest way is a custom query in labsdb [13:59:30] you can use the newly registered metric query as a starting point and combine it with the threshold query: [13:59:39] https://metrics.wmflabs.org/metrics/#NewlyRegistered [13:59:39] which is done by asking one of the labs people with sql knowledge ? [13:59:43] https://metrics.wmflabs.org/metrics/#Threshold [13:59:44] back online matanya [13:59:46] the queries are listed there [13:59:56] nono, labsdb is a public database [14:00:20] and you can access it from tools-login.eqiad.wmflabs [14:00:44] thanks a lot milimetric. i'll see how far i get [14:00:56] i'll run standup and help after if you're stufck [14:01:37] thank you [14:03:24] thanks nuria dan helped out [14:41:35] (CR) Ottomata: Add webrequest table schema (5 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/146878 (owner: Ottomata) [14:41:48] (PS2) Ottomata: Add webrequest table schema [analytics/refinery] - https://gerrit.wikimedia.org/r/146878 [14:43:44] (PS3) Terrrydactyl: [WIP] Add ability to global query a user's wikis [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/129858 [14:43:50] (CR) jenkins-bot: [V: -1] [WIP] Add ability to global query a user's wikis [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/129858 (owner: Terrrydactyl) [15:21:01] ottomata: do you have a bit of time [15:21:08] to come help us with tasking [15:21:18] of kafkatee stuff [15:29:01] sure [15:29:05] lemme get headphones [15:29:56] ottomata: WE're taking a break until 15:34 UTC (Just in case the hangout is empty) [17:06:27] [travis-ci] wikimedia/mediawiki-extensions-EventLogging#229 (wmf/1.24wmf14 - 6ec3cf0 : Reedy): The build passed. [17:06:27] [travis-ci] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/commit/6ec3cf0f8296 [17:06:27] [travis-ci] Build details : http://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/30187074 [17:17:59] leaving written record of this for posterity : nose issues with setup appear if you put a test inside a folder that does not have [17:18:06] a test prefix [17:18:08] that is [17:18:23] ./manual/test_blah.py will not run a setup [17:20:53] but ./test_manual/test_blah will [17:21:30] (all things being setup right on the test folder) [17:24:08] Ah , no scrtach that ./manual/test_blah.py [17:24:17] will work if you have an init file [17:30:39] (PS3) Nuria: Avoid pickle max recursion while serializing chain [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/146297 (https://bugzilla.wikimedia.org/67823) (owner: Milimetric) [17:34:29] (CR) Nuria: Avoid pickle max recursion while serializing chain (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/146297 (https://bugzilla.wikimedia.org/67823) (owner: Milimetric) [18:45:25] (PS1) Nemo bis: Link tables from edit/reverts reports [analytics/wikistats] - https://gerrit.wikimedia.org/r/147175 (https://bugzilla.wikimedia.org/65992) [20:22:45] qchris: would you have time to try the tests on this patch and confirm that they work? [20:22:46] https://gerrit.wikimedia.org/r/#/c/146297/2 [20:23:02] I tried before, and they failed for me [20:23:10] They are now running in backgroud with capture. [20:23:11] I will continue working on the patch cause it still needs a small bugfix [20:23:30] But did they fail with a db error? [20:24:05] no. an assertion error ... let me dpaste. [20:24:12] sorry qchrisit's https://gerrit.wikimedia.org/r/#/c/146297/3 [20:25:23] http://dpaste.com/09ZNBDH [20:25:45] Yup. I tested Patch Set 3. [20:25:56] The one with the empty __init__.py [20:26:11] ya, i only fixed the db error but patch has other issues [20:26:31] as CELEREY_ALWAYS_EAGER (sync queue) does not work with celery chain [20:26:44] thus tests are undetermenistic [20:27:31] I had not looked at the error in detail ... since the test takes ~10 minutes for me, it is just running in the background. [20:27:38] but the error you are getting has to do with the "clean db" step being missing from tear down too. [20:28:01] So you seem to have a better understanding of the issue than me ... could you comment on the change [20:28:15] and then mili.metric can improve the change? [20:28:27] I will submit another patch that fixes the teardown [20:28:31] Ah. ok. [20:28:57] I prefer we not steal the change from milimetric. [20:29:17] He said to be fine with fixing the db stuff. [20:29:21] but i am not sure the undeterminism can be fixed though... will research [20:29:22] I am not sure about other things. [20:30:27] we don't need to make this test deterministic, it's just a proof of concept that we could reproduce the recursion problem [20:30:49] ok [20:31:19] will add note to teh chain code that it does not work with teh "always eager" stuff and submit [20:39:06] (PS1) QChris: Make test script not complain if no *.pyc files exist [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/147225 [20:40:33] (CR) QChris: "For example output like the one at" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/147225 (owner: QChris) [21:07:36] (CR) QChris: Add webrequest table schema (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/146878 (owner: Ottomata) [21:07:50] (CR) QChris: [C: -1] "(Note also the comments on Patch Set 1 around LOCATION)" (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/146878 (owner: Ottomata) [21:24:57] (PS4) Nuria: Avoid pickle max recursion while serializing chain [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/146297 (https://bugzilla.wikimedia.org/67823) (owner: Milimetric) [21:26:12] that is the best sentence I've ever seen [21:26:20] "avoid pickle max recursion while serializing chain" [21:26:51] (PS1) Milimetric: Add monthly active editor [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/147312 (https://bugzilla.wikimedia.org/67458) [21:26:53] (CR) jenkins-bot: [V: -1] Add monthly active editor [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/147312 (https://bugzilla.wikimedia.org/67458) (owner: Milimetric) [21:27:49] (PS2) Milimetric: Add monthly active editor [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/147312 (https://bugzilla.wikimedia.org/67458) [21:29:17] (CR) Nuria: [C: 2] Make coalesced report format more efficient [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/146521 (https://bugzilla.wikimedia.org/67822) (owner: Milimetric) [21:29:18] :) lots of metaphore [21:39:50] (CR) Nuria: Add autcomplete to tags on tag insertion (3 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/145039 (owner: Terrrydactyl) [21:51:38] Analytics / Wikimetrics: Story: EEVSuser has agregate metrics - https://bugzilla.wikimedia.org/68193 (Kevin Leduc) NEW p:Unprio s:enhanc a:None Story: EEVSuser has agregate metrics - across all projects - deduplicated - for Newly Registered User, Rolling Monthly Active Editor, Rolling New A... [21:52:19] Analytics / Wikimetrics: Story: EEVSuser has agregate metrics - https://bugzilla.wikimedia.org/68193 (Kevin Leduc) p:Unprio>Normal [23:01:07] Analytics / General/Unknown: ULSFO post-move verification - https://bugzilla.wikimedia.org/68199 (Kevin Leduc) NEW p:Unprio s:normal a:None (happened on wed 9th of July) * check with gage whether this work already took place * check that during the switchover, hosts were correctly reportin... [23:08:02] (PS5) Milimetric: Make coalesced report format more efficient [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/146521 (https://bugzilla.wikimedia.org/67822) [23:09:09] (PS3) Milimetric: Add monthly active editor [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/147312 (https://bugzilla.wikimedia.org/67458) [23:43:45] (PS4) Milimetric: Add monthly active editor [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/147312 (https://bugzilla.wikimedia.org/67458) [23:47:05] Analytics / General/Unknown: ULSFO post-move verification - https://bugzilla.wikimedia.org/68199 (Dan Andreescu)