[04:38:23] (03PS3) 10Sahil505: Hide "Load more rows..." once max data is visible in Table Chart [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/427774 (https://phabricator.wikimedia.org/T192407) [04:42:39] (03CR) 10Sahil505: "Moved valuesShown to computed. Separated it our in 3 different function, totalItems, valuesShown, loadMoreRows. This keeps the code more c" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/427774 (https://phabricator.wikimedia.org/T192407) (owner: 10Sahil505) [08:12:35] heya teaaaam :] [08:17:09] o/ [09:33:04] joal: o/ - FYI I am working on upgrading the Hadoop labs workers to stretch, so if you want to test druid keep this in mind, something might be flaky [09:33:07] :) [09:36:24] * elukey early lunch + errand! [10:40:14] (03CR) 10Mforns: [V: 032] "LGTM! I tested and works for me! The code change also looks great. I left a couple comments: one small nit I think could be improved, and " (033 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/427774 (https://phabricator.wikimedia.org/T192407) (owner: 10Sahil505) [10:54:06] (03PS4) 10Sahil505: Hide "Load more rows..." once max data is visible in Table Chart [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/427774 (https://phabricator.wikimedia.org/T192407) [10:56:20] (03CR) 10Sahil505: Hide "Load more rows..." once max data is visible in Table Chart (033 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/427774 (https://phabricator.wikimedia.org/T192407) (owner: 10Sahil505) [11:11:57] (03CR) 10Mforns: [V: 032 C: 032] "LGTM!" (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/427774 (https://phabricator.wikimedia.org/T192407) (owner: 10Sahil505) [11:56:44] mforns: cron deployed! [11:57:25] elukey, thanks! it's still failing though... but because of the code, the cron is right now, thx [13:23:48] ottomata, hello! [13:25:00] I've been troubleshooting EL sanitization and found a weird problem, that I think is related to retrieving schemas from parquet... [13:27:30] mforns: hello! i'm not really working today, i'm mostly online to see if i can do some data stuff before I have to go [13:27:35] SO! i want to help but I can't right now :/ [13:28:00] ottomata, no worries, I also have to go in short for the interview and presentation [13:29:03] ottomata, I will try to join stand'up from there [13:29:09] see ya [13:29:12] seeyaa [13:30:56] (03PS1) 10Sahil505: Added a friendly metric description to the tooltip of metric link [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433566 (https://phabricator.wikimedia.org/T182109) [13:34:01] (03CR) 10Sahil505: "> Uploaded patch set 1." [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433566 (https://phabricator.wikimedia.org/T182109) (owner: 10Sahil505) [13:41:26] (03PS1) 10Mforns: Fix bug when merging partition fields in WhitelistSanitization.scala [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/433569 (https://phabricator.wikimedia.org/T193176) [13:42:23] (03CR) 10Mforns: [C: 04-1] "This should fix the problem. But please do not merge, until I figure out if the other problem is actually code-related as well." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/433569 (https://phabricator.wikimedia.org/T193176) (owner: 10Mforns) [13:47:13] fdans: need your opinion on wikistats [13:47:20] oh [13:47:24] wikistats is pretty cool! [13:47:25] ok, so persistent state is file for fullscreen [13:47:35] * milimetric shakes fist [13:47:41] :) [13:47:43] ok, focus! [13:47:52] tell me [13:48:05] so fullscreen is file - it keeps it until you toggle it off [13:48:09] but chartType is interesting [13:48:29] because if it remembers the last selected chart type and you're switching from two components that have bar/table and line/table that's fine [13:48:38] if you're on table it stays there, otherwise it uses the default [13:49:09] but what if you're on bar chartType, change from a bar/table metric to like top articles which is just table, and back to bar/table [13:49:22] then it'll go to table instead of the default [13:49:49] does that make sense? You see the problem? [13:50:12] hmmm I'm trying to replicate [13:50:19] fdans: nono, this is only on my local [13:50:24] aaaahhh [13:50:28] the bookmarks code I'm working on [13:50:32] yes yes [13:50:32] just needed to brainbounce [13:50:33] sorry [13:50:40] no, that's my bad, should've specified [13:50:42] sorry [13:51:33] I don't think the interface should keep the chart from one metric to the other [13:51:57] ok, so it just keeps it in the URL [13:51:58] that works [13:52:32] thanks! [13:52:58] yeah it's cool to keep the state in the url, but the moment the user changes metric, it all should invalidate [13:53:17] anytiiiiime [13:58:40] * mforns leaves for presentation [13:58:41] byeeeeee [14:21:24] joal: fyi, hadoop labs workers on stretch! Just finished [14:22:50] * elukey brb [14:43:39] fdans: I like keeping around the fullscreen and time range options though, it feels more solid. The filter/split/chartType though are not kept [14:44:19] milimetric: that sounds good to me, yeah [14:55:33] (03CR) 10Nuria: Added a friendly metric description to the tooltip of metric link (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433566 (https://phabricator.wikimedia.org/T182109) (owner: 10Sahil505) [15:00:53] (03PS1) 10Milimetric: Ignore search preferences [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433584 [15:00:55] (03PS1) 10Milimetric: Expand setState to be more explicit [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433585 [15:00:57] (03PS1) 10Milimetric: Move detail state into store [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433586 [15:01:59] fdans: you were right, making those ui buttons loop over time ranges was a little weird, but it worked in the end: https://gerrit.wikimedia.org/r/#/c/433586/1/src/components/TimeRangeSelector.vue [15:03:38] milimetric: so what made it work was just adding classes to each button? [15:03:48] that was so not in the docs [15:04:04] fdans: you had the classes on there, but I think ... hang on [15:05:18] fdans: hm, actually now I can't break it again [15:05:24] but my first go at it it was definitely broken... [15:06:34] oh I was just using the wrong property in the button text, no it works fine like this [15:06:43] on your original buttons you had classes on each one as well [15:07:41] and that's in the docs: https://semantic-ui.com/elements/button.html#Groups [15:08:20] it did seem really broken when I tried it first, but maybe it's just tricky to get the iteration right or something [15:08:57] anyway, I got all the state except split into the store so far, in a separate module, feel free to critique the approach [15:09:13] I'm gonna do split now and then update the Router to mirror these things to the URL and back [15:16:52] (03PS8) 10Nuria: UA parser specification changes [analytics/ua-parser/uap-java] (wmf) - 10https://gerrit.wikimedia.org/r/429527 (https://phabricator.wikimedia.org/T189230) [15:18:17] (03CR) 10Nuria: UA parser specification changes (036 comments) [analytics/ua-parser/uap-java] (wmf) - 10https://gerrit.wikimedia.org/r/429527 (https://phabricator.wikimedia.org/T189230) (owner: 10Nuria) [15:22:24] fdans: hola. updated ua parser [15:22:58] nuria_: nice, will take a look now [15:23:29] fdans: i hope my battery last to standup, brough my charger but it NO work cause we are not in kansas anymore [15:23:37] fdans: and the good charger is at home [15:26:06] (03PS2) 10Sahil505: Added a friendly metric description to the tooltip of metric link [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433566 (https://phabricator.wikimedia.org/T182109) [15:26:49] (03CR) 10Sahil505: Added a friendly metric description to the tooltip of metric link (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433566 (https://phabricator.wikimedia.org/T182109) (owner: 10Sahil505) [15:27:52] nuria_: quickly testing locally, and I'll merge [15:28:21] fdans: did you tested udf building with this code ? [15:28:31] fdans: or where your tests to date python? [15:29:34] yeah I tested the udf the other day, not sure what you mean with python [15:29:37] nu [15:29:46] nuria_ [15:30:23] fdans: tested the udf building from the wmf branch of ua-parser(or your local checkout), correct? [15:32:08] nuria_: yea, should I test it in beta at this point or should we wait for the refinery-source patch? [15:32:45] fdans: did you build the java udf and run it through webrequest data? [15:33:59] nuria_: I ran it with some ua strings that I checked were being parsed differently between versions [15:34:23] fdans: the udf ? [15:44:14] nuria_: GetUAPropertiesUDF [15:44:54] fdans: ok, then we should be ready for update, after merge we need to push to archiva [15:44:58] tested with windows strings and the weird CFNetwork ones [15:45:04] fdans: and after bump up the pom version [15:45:08] yep [15:46:53] fdans: not sure how are we releasing this though, did you talked to ottomata about it? [15:47:47] nuria_: we talked a few weeks ago. I got the credentials to push to archiva, in refinery source it should be just changing the name of the dependency in the pom.xml [15:48:12] fdans: also ua-aprser package needs to bump up version [15:48:34] although I haven't tested that... right now I'm adding ua-parser in IDEA through a direct classpath source add [15:49:03] fdans: ya, that would not work to make refinery use package [15:49:18] refinery specifies a explicit dependency [15:49:24] fdans: it is in the POm [15:49:32] yeayea [15:49:34] fdans: we need to push a new package [15:49:44] nuria_: one question, which wasn't fully clear with andrew... which number do we bump? [15:50:09] fdans: there are two, one is the version of this package [15:50:20] (03PS1) 10Elukey: [WIP] Index some fields from isp_data and geocoded_data in Druid's webrequest [analytics/refinery] - 10https://gerrit.wikimedia.org/r/433597 (https://phabricator.wikimedia.org/T194055) [15:50:23] fdans: the other is the version of the package that refinery uses [15:50:59] nuria_: the one specified in pom.xml in refinery hive is v1.3.0-wmf3 [15:51:15] should we publish as 1.3.0-wmf4? [15:52:16] nuria_: also: worth considering that there was an attempt to release 1.3.0-wmf4 already, as you can see here https://github.com/wikimedia/analytics-ua-parser-uap-java/commits/wmf [15:52:41] fdans: is https://archiva.wikimedia.org/ the right link for archiva? [15:53:10] nuria_: yeah, afaik https://archiva.wikimedia.org/repository/mirrored/ua_parser/ua-parser/ [15:53:52] 10Analytics, 10Patch-For-Review, 10User-Elukey: Add maxmind ip info to webrequest dataset on druid - https://phabricator.wikimedia.org/T194055#4212098 (10elukey) a:03elukey [15:54:01] fdans: i do not get the "1.3.0-wmf" naming scheme [15:54:23] fdans: shouldn't this be 1.4? [15:54:41] nuria_: i think 1.3.0 was the last release in the upstream library [15:54:58] nuria_: and then we forked it and started tagging it as blablabla-wmf3 [15:55:11] nuria_: BUT [15:55:44] since we're actually updating and maintaining the code in this upcoming release, I say we just tag it as 1.4.0 [15:56:34] fdans: look: someone did just recently our same changes: https://github.com/ua-parser/uap-java/commit/f65c6a79a32b4e09487b14e545b9ac1e9a2f1f73#diff-1613930d2a98ffce105cfa29feac98da [15:56:57] 10Analytics, 10Patch-For-Review, 10User-Elukey: Add maxmind ip info to webrequest dataset on druid - https://phabricator.wikimedia.org/T194055#4212108 (10elukey) From what I can see in the webrequest's format, these are the things that we could add: * **geocoded_data ** map Geocoded... [15:57:10] fdans: ah sorry, no, they just fixed javsdoc: https://github.com/ua-parser/uap-java/commit/4b6acf74167f6eefad1abb5710cde49cf5a93959#diff-1613930d2a98ffce105cfa29feac98da [15:57:38] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Add maxmind ip info to webrequest dataset on druid - https://phabricator.wikimedia.org/T194055#4212112 (10elukey) [15:58:21] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Add maxmind ip info to webrequest dataset on druid - https://phabricator.wikimedia.org/T194055#4212116 (10Nuria) For webrequest data even city does not seem of much help (given that it is sampled to 1/100) but up to ops. [16:02:03] ping fdans milimetric [16:02:09] a-team - StandUPPPPP :) [16:09:46] (03CR) 10Elukey: "@Dan: it is still super WIP, so be patient with what you'll read :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/433597 (https://phabricator.wikimedia.org/T194055) (owner: 10Elukey) [16:22:54] I have to crank it all the way to Laurel to hear it: https://www.nytimes.com/interactive/2018/05/16/upshot/audio-clip-yanny-laurel-debate.html?smid=tw-share [16:27:01] nuria_: let me know when you want to deploy on archiva [16:27:54] 10Analytics, 10MediaWiki-Database, 10MediaWiki-Page-deletion: Store data about reasons for deletion in a way that is easier to analyze - https://phabricator.wikimedia.org/T194891#4212185 (10Amire80) [16:30:02] (03CR) 10Milimetric: [V: 032 C: 032] Added a friendly metric description to the tooltip of metric link [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/433566 (https://phabricator.wikimedia.org/T182109) (owner: 10Sahil505) [16:30:09] fdans: did you merged code? [17:11:36] joal: let me know when you are back and we can deploy [17:19:16] milimetric: tooltip code looks fine but tooltips take about 2 secs to display which i think is too long. [17:19:25] milimetric: are we using semantic for those? [17:19:56] nuria_: no, just browser tooltips [17:20:07] milimetric: old fashion ones? [17:20:12] yeah [17:20:15] title tag [17:20:19] milimetric: jaja that thought did not even occur to me [17:20:19] unless semantic is triggering something I'm not aware of [17:20:24] juas [17:20:27] :) [17:20:59] yeah, nothing fancy yet, I still have to find a decent tooltip library [17:41:06] Hi nuria_ - Back! [17:41:14] joal: hola [17:41:40] fdans: the patch was reday to merge right? [17:41:55] joal: so we just have to merge this: https://gerrit.wikimedia.org/r/#/c/429527/ [17:42:06] * elukey off! [17:42:06] joal: but also we need to decide how to version [17:42:11] ciao elukey [17:42:25] Bye elukey [17:42:35] joal: current version is 1.3.0-wmf3 [17:43:27] nuria_: Sorry had to leave, I'm back in 10min, yes it's ready, I can talk to joal for release [17:43:43] fdans: no worries we can do it now [17:43:48] fdans: i will just merge [17:43:49] cool! [17:44:08] nuria_: currently checking in archiva to look how versioning was done up to now [17:44:09] (03CR) 10Nuria: [V: 032 C: 032] UA parser specification changes [analytics/ua-parser/uap-java] (wmf) - 10https://gerrit.wikimedia.org/r/429527 (https://phabricator.wikimedia.org/T189230) (owner: 10Nuria) [17:44:37] joal: https://archiva.wikimedia.org/repository/mirrored/ua_parser/ua-parser/1.3.0/ [17:45:21] nuria_: I was after that one: https://archiva.wikimedia.org/#artifact~releases/ua_parser/ua-parser [17:46:06] nuria_: if the ua-parser release version has not changed, we should go for 1.3.0-wmf4, else 1.4.0-wmf1 - right? [17:46:36] joal: it has not changed, actually i think my hackathon project is going to be sending them a pull request [17:46:54] nuria_: ok! [17:46:56] joal: do we update ua-parser pom then? [17:47:06] let's go for 1.3.0-wmf4 [17:47:49] joal: i think that one was rolled back .mm one sec [17:48:11] joal: ya [17:48:15] https://www.irccloud.com/pastebin/9VZZQ0FD/ [17:49:21] joal: so next one? [17:49:41] nuria_: hm - Given wmf4 was not dpeloyed, why shouldn't we go for it now? [17:50:11] joal: no totally, i just thought archiva might get confused if that was uploaded and deleted [17:50:32] nuria_: I have no trace of it - I'm trying to check for tags in git [17:51:22] nuria_: tag for wmf3 doesn't even exist in git [17:51:27] joal: [17:51:30] https://www.irccloud.com/pastebin/NPHkTIXO/ [17:51:45] Ok - Let's be good citizens :) Let's deploy, push a tag for wmf4, and deploy it to archiva [17:52:03] joal: wait deploy how? [17:52:31] Interesting nuria_ - github doesn't know about the 1.3.0-wmf3 [17:52:34] anyway [17:52:45] s/deploy/merge [17:53:15] joal: it is merged [17:53:25] joal: but no version changed on pom [17:53:33] nuria_: We should do it I think [17:55:18] nuria_: The way we did it was to modify the pom before building, upload hat modified version to archiva, but not change the default one, to still match upstream [17:56:09] joal: but wait ... why does the version of this package need to match upstream? [17:57:14] nuria_: doesn't have to - just easier to merge upstream changes done Ithink [17:58:50] joal: we just override version and that's it, our code hardly resembles the mainstream anymore, it hasn't been maintained for 2 years [17:59:04] nuria_: goodfor me [17:59:11] joal: does that make sense? [17:59:14] joal: ah ok [18:00:25] (03PS1) 10Nuria: Bumping up version for release [analytics/ua-parser/uap-java] (wmf) - 10https://gerrit.wikimedia.org/r/433617 (https://phabricator.wikimedia.org/T189230) [18:00:52] joal: https://gerrit.wikimedia.org/r/#/c/433617/ [18:01:05] (03CR) 10Joal: [V: 032 C: 032] "LGTM - Merging" [analytics/ua-parser/uap-java] (wmf) - 10https://gerrit.wikimedia.org/r/433617 (https://phabricator.wikimedia.org/T189230) (owner: 10Nuria) [18:01:13] merged nuria_ [18:01:29] joal: there is also [18:01:33] https://www.irccloud.com/pastebin/bBvp5F2g/ [18:01:38] this code in pom [18:02:33] nuria_: means we should be able o autmoagically release using maven - But version numbers are not setup as of so [18:03:05] joal: ok are we pushing tags [18:03:10] joal: yeahah? [18:03:17] nuria_: Yes we should [18:04:29] joal: mmm.. i wonder if i pushed tag to given branch [18:04:42] joal: as we are building from wmf branch not master [18:05:32] nuria_: you're absolutely right! [18:05:47] nuria_: tag 1.3.0-wmf4 actually exist on branch wmf! [18:06:21] nuria_: Ah - You've just created it :) [18:06:32] joal [18:06:37] https://www.irccloud.com/pastebin/Kk2Ur0Tw/ [18:06:43] joal: ya, just did [18:06:47] indeed :) [18:06:50] ok great [18:06:57] joal: now build locally [18:06:57] now lets deploy on archiva [18:07:02] yes madam [18:07:02] k [18:07:07] reday sir [18:07:12] *ready sir [18:26:04] 10Analytics, 10Analytics-EventLogging, 10Readers-Web-Backlog: Explore an API for logging events sampled by session - https://phabricator.wikimedia.org/T168380#4212321 (10Nuria) I think this might be of interest and I am not sure if when this ticket was created this code existed on EL but you can sample per s... [18:28:39] fdans, joal: artifact in archiva: https://archiva.wikimedia.org/#artifact~releases/ua_parser/ua-parser [18:29:11] fdans: try to use it on udf end (changing pom locally) and let me know if it works [18:29:19] yep! doing it now [18:30:09] nuria_: I had to remove ~/.m2 a bit ago so maven is currently downloading all the internet, will report soon [18:30:23] fdans: ya, more like tomorrow [18:31:12] I think right now it's fetching russian wikivoyage and afterwards it'll get reddit [18:33:03] fdans: juas, i am logging off but will be online tomorrow [18:33:05] nuria_: you should log off since it's pretty late in spain and you've had a long trip, I'm going to test the remote jar on refinery-core and will send a patch. We can deploy refinery source on monday joal [18:33:15] fdans: my 1st hackathon project is to send this guys a pull request [18:33:25] ack fdans :) [18:33:33] fdans: they might not even look at it but at least we tried [18:33:34] +1000 nuria :) [18:33:39] hell yea nuria_ [18:33:49] fdans, joal : ok will do PR tomorrow [18:33:53] ciaooo [18:33:54] baaai [18:33:56] Bye nuria_ [19:05:39] joal: nuria_ tests failing all over the place because of Windows 7 => Windows [19:05:42] so success!! [19:31:37] (03PS1) 10Fdans: Update refinery user agent parser version [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/433633 (https://phabricator.wikimedia.org/T192463) [19:31:56] joal nuria_ ^ :) [19:33:25] (03CR) 10jerkins-bot: [V: 04-1] Update refinery user agent parser version [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/433633 (https://phabricator.wikimedia.org/T192463) (owner: 10Fdans) [20:45:52] o/ How up to date is the prod db in hive? [20:46:52] Also, any plans on adding the pagelinks (or other) table to hive? Is there a procedure that I can follow to request these additions? [20:50:12] bmansurov: the data lake? [20:50:21] bmansurov: it is about 1 month old [20:51:03] nuria_: yes, the data lake. [20:51:20] bmansurov: it is always 1 month behind or more [20:51:32] bmansurov: now data is from 2018-04-01 [20:51:49] nuria_: I see. I think that's good for my purposes. [22:05:25] 10Analytics-Legal, 10WMF-Legal, 10Wikidata: Solve legal uncertainty of Wikidata - https://phabricator.wikimedia.org/T193728#4212631 (10Denny) @Rspeer Copyright has to be about some concrete expression. Are you claiming that the interwiki links that used to be in Wikipedia articles until five years ago sho... [22:13:10] 10Analytics-Legal, 10WMF-Legal, 10Wikidata: Solve legal uncertainty of Wikidata - https://phabricator.wikimedia.org/T193728#4212632 (10Aschmidt) Am 18.05.18 um 00:05 Uhr schrieb Denny: > Copyright has to be about some concrete expression. I'm afraid this is not the case. According to German law copyright i... [22:52:41] 10Analytics-Legal, 10WMF-Legal, 10Wikidata: Solve legal uncertainty of Wikidata - https://phabricator.wikimedia.org/T193728#4212685 (10Denny) I was reading the article you linked to - https://de.wikipedia.org/wiki/Sch%C3%B6pfungsh%C3%B6he#Sch%C3%B6pfungsh%C3%B6he_seit_2013 - and nothing there lets me believe... [22:59:10] 10Analytics-Legal, 10WMF-Legal, 10Wikidata: Solve legal uncertainty of Wikidata - https://phabricator.wikimedia.org/T193728#4212699 (10Rspeer) @Denny Nobody's copyright is going to be invalidated by your personal beliefs. And what do bots have to do with anything? Wiki bots are simple scripts operated by hu... [23:04:47] 10Analytics-Legal, 10WMF-Legal, 10Wikidata: Solve legal uncertainty of Wikidata - https://phabricator.wikimedia.org/T193728#4212702 (10Rspeer) > [...] without any reference to the originating author. So if that is the case, Wikipedia has already never been compliant with that license. Wikipedia's interpreta...