[00:04:50] chelsyx: i am not sure I understand , i see 201801 and 201811 in your query ... [00:05:38] nuria: it's not in my query: https://www.irccloud.com/pastebin/X80NtSiD/ [00:05:50] nuria: And I've figure out why [00:06:57] nuria: the `log_comment` field is deprecated, and log_comment_id and the comment table should be used instead [02:56:39] 10Analytics, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10MNeisler) Hi @mforns! Thanks for the update re the timeline. A meeting would be great - I’ll set up a time for us and @Neil_P._... [07:07:57] 10Analytics, 10Analytics-Kanban, 10User-Marostegui: Migrate users to dbstore100[3-5] - https://phabricator.wikimedia.org/T215589 (10elukey) [08:08:58] (03CR) 10Joal: "> Patch Set 1:" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/495734 (owner: 10Joal) [08:16:38] Morning team - I've been kicked out of the chat last night ... [08:16:45] I hope I didn't miss to much of the party [08:37:53] joal: bonjour! There is nothing super pressing since you left, I can see only Nuria chatting with Chelsy for a query [08:41:40] okey :) [08:41:40] joal: whenever you have time, can you quickly check the new superset and tell me if anything looks weird? [08:41:55] elukey: can do (I'm no superset expert though :) [08:42:13] ssh -L 9080:analytics-tool1004.eqiad.wmnet:80 analytics-tool1004.eqiad.wmnet [08:42:24] ah yes I am looking for things that break visually [08:42:33] the only one I found was the World Map [08:42:51] that breaks when you add the bubbles [08:42:53] /o\ [09:02:22] elukey: the thing I found is that many charts break (including world-map one) when using the same metric twice for different display dimensions [09:02:52] joal: I've published 2 patches for T217967, one which renames the shaded jar, once which does not [09:02:52] T217967: Publish both shaded and unshaded artifacts from analytics refinery - https://phabricator.wikimedia.org/T217967 [09:03:20] joal: can you give me an example about a non-world-map use case? [09:03:26] sure [09:03:35] my preference goes to the renaming, but I don't have enough context about the peripheral use cases, so I'll leave it to you (or your team) [09:04:13] and once one of those patches is merged, I might be interested in doing some cleanup to your maven configuration :) [09:04:48] ping me if you need help / more discussion, atm I'm assuming that the ticket is in your very capable hands [09:05:15] Hi gehel - Thanks for that - We have not yet made decision on renaming or not, but I think Andrew prefers renaming as well (works for me, not too much opinionated on this) [09:06:12] elukey: example (query makes no sense in term of data, but succeeds in term of UI) [09:06:15] http://localhost:9080/superset/explore/?form_data=%7B%22datasource%22%3A%22352__druid%22%2C%22viz_type%22%3A%22bubble%22%2C%22url_params%22%3A%7B%7D%2C%22granularity%22%3Anull%2C%22druid_time_origin%22%3Anull%2C%22time_range%22%3A%22Last+quarter%22%2C%22series%22%3A%22access_method%22%2C%22entity%22%3A%22agent_type%22%2C%22x%22%3A%22sum__view_count%22%2C%22y%22%3A%22min__view_count%22%2C%22adhoc_ [09:06:21] filters%22%3A%5B%5D%2C%22size%22%3A%22max__view_count%22%2C%22max_bubble_size%22%3A%2225%22%2C%22color_scheme%22%3A%22bnbColors%22%2C%22show_legend%22%3Atrue%2C%22x_axis_label%22%3A%22%22%2C%22left_margin%22%3A%22auto%22%2C%22x_axis_format%22%3A%22.3s%22%2C%22x_ticks_layout%22%3A%22auto%22%2C%22x_log_scale%22%3Afalse%2C%22x_axis_showminmax%22%3Afalse%2C%22y_axis_label%22%3A%22%22%2C%22bottom_marg [09:06:27] in%22%3A%22auto%22%2C%22y_axis_format%22%3A%22.3s%22%2C%22y_log_scale%22%3Afalse%2C%22y_axis_showminmax%22%3Afalse%7D [09:06:30] Aouch [09:06:31] ouch :D [09:06:33] sorry - Will do on gist [09:06:48] elukey: https://gist.github.com/jobar/80cacffdeff98c85c724a978cf2cd037 [09:07:45] elukey: this query works - Now try to change one of the X-Axis, Y-Axis or Bubble-size to the same value as another one - for instance Y-Axis to Sum(view_count) [09:08:09] This means having 2 UI components relying on the same metric - it fails [09:08:49] Something else I noticed is that the UI sometimes require a hard-refresh (F5) to actually update the query [09:09:50] the error is "Wrong number of items passed 2, placement implies 1" ? [09:10:16] correct [09:10:31] different error message, but same issue: reusing the same metric [09:11:18] elukey: I added a comment to the gist above with an example of world-map with bubble [09:11:22] +s [09:12:05] joal: for the second one I already opened https://github.com/apache/incubator-superset/issues/7006 [09:12:23] I think it is due to Pandas upgrading to a new version that behaves differently [09:12:37] but I hoped that the blast radious was limited [09:12:37] sigh [09:12:43] elukey: I actually think it's related to 2 times the same metric [09:13:04] since I can make a world-map with bubbles working when using different ones [09:16:13] joal: sure, but it seems to be a pandas issue, see my comments in the issue [09:16:17] no? [09:16:27] elukey: maybe? [09:16:33] I can add the bit about the same metric, could be useful to them to debug further [09:16:36] nono I am pretty sure [09:17:08] because I live debugged 0.26 and for the same bit of code, pandas returns different types [09:17:13] Dataframe vs series [09:17:24] I'll add your examples thanks! [09:18:42] elukey: There are 2 other I don't manage to have working: calendar-heatmap and force-directed-graph [09:19:47] sigh [09:19:50] elukey: unrelated to errors - Super interesting visualization as 2nd comment on gist :) [09:20:32] joal: nice! A new one? [09:20:48] heuuu, I don't get the question :) [09:21:13] Ah - new way of visualizing data we know - yes [09:22:41] This diagram shows us visualy that user-traffic (by opposition to spider) is about half-half coming from desktop and mobile-web, and that desktop traffic is 2/3 1/3 to user/spider [09:23:04] I like it :) [09:23:16] Will add to the pageview dashboard [09:23:45] hehe - something else I notice on this new version of superset: slices have been renamed charts [09:25:58] ok elukey - pageview dashboard is back on track :) [09:26:44] joal: on 0.29? [09:26:59] yessir [09:27:16] ah yes you removed the bubbles [09:27:37] elukey: not only - reaasigned a different metric to bubbles, even if not showing them [09:30:42] elukey: I love this pageview dashboard - could spend hour exploring :) [09:30:54] joal: confirmed from the stacktrace that the error you found is relatd to pands [09:30:57] *pandas [09:30:59] ufff [09:31:47] elukey: IMO everything data-related in superset will be related to pandas as it is the data-wrangling piece [09:35:47] elukey: ok for you if stop testing? [09:39:04] joal: sure thanks! I agree that everything in superset leverages pandas, but there are traces of comments like "I assume that in here pandas returns XYZ and hence I do bla" that doesn't hold anymore when pandas upgrades [09:39:22] makes sense elukey [09:39:29] better: when they bump pandas versions! [09:39:44] elukey: this is a hint toward not enough unit-testing :S [09:39:45] I'll retry to build the current one and see [09:39:56] but numpy doesn't like python 3.7 afaics (at least, that version) [09:40:09] joal: did you say none testing? :D [09:40:20] elukey: Naaaaaaah [09:40:38] elukey: I don't envy you, dealing with python versions :S [09:48:20] (03PS7) 10Joal: Update delete/restore in mediawiki-history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/485710 (https://phabricator.wikimedia.org/T213603) [10:03:05] (03PS8) 10Joal: Refactor mediawiki-history core data gathering [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T216603) [10:07:53] (03PS4) 10Joal: Add revision_hidden_parts to mediawiki-history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492304 (https://phabricator.wikimedia.org/T178587) [10:10:51] (03PS6) 10Joal: Add change_tags to mediawiki_history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492320 [10:16:57] (03PS5) 10Joal: Update mediawiki-reconstruction with log info [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493012 [10:32:29] (03PS7) 10Joal: Refactor mediawiki-page-history computation [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493390 (https://phabricator.wikimedia.org/T190434) [10:43:52] 10Analytics-Kanban, 10Patch-For-Review: Coordinate work on minor changes for Edit Data Quality - https://phabricator.wikimedia.org/T213603 (10JAllemandou) [10:55:10] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Staging environment for upgrades of superset - https://phabricator.wikimedia.org/T212243 (10elukey) I tried to force pandas to 0.22 but this requires numpy 1.13.2, that is not compatible (apparently) with Python 3.7, leading to compilation... [10:57:19] * elukey cries in a corner [10:58:05] * joal pads elukey on the back [10:59:21] elukey: question for you - I have quite some errors when trying sqooping with high parallelization (22 jobs at the same time) [10:59:42] elukey: can you confirm our number of connections limitation is 100? [11:02:06] * joal plays with parallelization knobs to find better setttings [11:06:53] joal: yep it is, I can raise it if you need [11:07:22] elukey: first, trying to see if errors can be handled with lower parallelization [11:07:45] ah I didn't see the message above [11:08:15] I thought you used more in the previous attempts [11:08:40] what errors do you get? [11:09:09] elukey: some sqooping jobs are failing (no more details because of python -java) [11:09:41] elukey: Mysql dashboards show connection issue (https://grafana.wikimedia.org/d/000000273/mysql?panelId=10&fullscreen&orgId=1&var-dc=eqiad%20prometheus%2Fops&var-server=labsdb1012&var-port=9104&from=now-3h&to=now) [11:10:04] mostly seems aborted conns [11:11:05] hm [11:14:41] elukey: trying with more logs [11:14:48] yeah I was about to ask [11:20:49] (03PS10) 10Elukey: Add artifacts for Debian Buster and upgrade to 0.31 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/495182 (https://phabricator.wikimedia.org/T212243) [11:26:42] elukey: it's what I had expected: User 's53272' has exceeded the 'max_user_connections' resource (current value: 100) [11:26:56] elukey: however I don't understand how it;s possible :( [11:28:20] elukey: trying again with smaller conn-pool [11:31:17] elukey: as if connections used by sqoop were not closed successfully [11:35:33] joal: ah! [11:36:15] maybe each sqoop job opens more than one TCP conn? [11:36:24] the 100 limit is related to that I believe [11:36:32] we can raise it to say 1000 [11:36:48] elukey: would be awesome :) [11:37:16] elukey: I'm monitoring the db health while sqooping, and you're around - so we can stop if any issue arise [11:39:25] joal: raised to 1000 [11:39:43] \o/ :) Thanks [11:39:49] LAUNCHING :) [11:41:30] (03PS11) 10Elukey: Add artifacts for Debian Buster and upgrade to 0.31 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/495182 (https://phabricator.wikimedia.org/T212243) [11:47:07] elukey: found my bug - was my fault, sorry [11:49:29] good! [11:49:47] I tried superset 0.31 but it seems that the migration script is super broken [11:52:03] :S [11:53:46] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+2] "Did the gate-and-submit job failed, or does this codebase require manual merges?" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494190 (owner: 10Thiemo Kreuz (WMDE)) [12:40:16] I found a way to start the db migration to 0.31 [12:40:19] let's see how it goes [12:40:22] boy what a pain [12:40:32] it is like playing Jenga [12:40:59] I feel a bit the same with sqoop :) [12:41:16] with an advantage though: it only impacts machines :) [12:42:17] also joal, for the fun https://github.com/apache/incubator-superset/pull/6765 [12:42:25] the last versions are also panda broken [12:42:30] but way worse :D [12:43:04] YAY :) what an awesome tool to maintain :) [12:43:19] 10Analytics, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10mforns) @MNeisler Great, I already accepted the invite. Thank you! [12:53:41] * elukey lunch! [13:28:06] 10Analytics, 10Product-Analytics, 10Patch-For-Review: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10Ottomata) @JAllemandou, my understanding is that if we get a newer version of Hive, we will be able to use Hive timestamp types with ISO-8601 string formats.... [13:35:45] 10Analytics, 10Product-Analytics, 10Patch-For-Review: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10JAllemandou) @Ottomata : I don't think Hive timestamp-functions will parse ISO by default. All functions doc gives examples without the 'T'. Also, the above t... [13:38:59] 10Analytics, 10Discovery-Search (Current work), 10Patch-For-Review: Publish both shaded and unshaded artifacts from analytics refinery - https://phabricator.wikimedia.org/T217967 (10Ottomata) Cool thank you!!! I think adding '-shaded' classifier to the shaded jar makes the most sense, but it will require ch... [13:46:20] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Enable encryption and authentication for TLS-based Hadoop services - https://phabricator.wikimedia.org/T217412 (10Ottomata) > use a self signed CA Another option would be to use the Puppet CA to sign cergen created certificates. The trus... [13:46:24] 10Analytics, 10ChangeProp, 10EventBus, 10WMF-JobQueue, and 3 others: Consider disabling automatic topic creation in main-kafka - https://phabricator.wikimedia.org/T199432 (10Ottomata) I think we should close. Maybe we'll do this one day if we have a really solid '[[ https://phabricator.wikimedia.org/T2053... [13:46:27] 10Analytics, 10ChangeProp, 10EventBus, 10WMF-JobQueue, and 3 others: Consider disabling automatic topic creation in main-kafka - https://phabricator.wikimedia.org/T199432 (10Ottomata) 05Open→03Declined [13:52:28] hey yall [13:52:46] 10Analytics, 10Product-Analytics, 10Patch-For-Review: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10Ottomata) Hm, no I mean once we get Hive 1.2.0+, we can do SET SERDEPROPERTIES ("timestamp.formats"="yyyy-MM-dd'T'HH:mm:ssXXX"); https://issues.apache.or... [13:52:48] Hi milimetric [13:53:05] oh man, I was sick all weekend, still sick [13:53:07] weirdest thing ever [13:53:10] :( [13:53:17] I thought I got better on Friday and then it came back with a vengance [13:57:00] 10Analytics, 10Product-Analytics, 10Patch-For-Review: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10JAllemandou) Nice! I don't see flaws in this approach @ottomata, but before testing I don't want to say it'll work for all cases (timestamps work for us in hi... [13:57:32] milimetric: I don't know for New York but in here weather is very bizzare and tempreature strongly unstable [14:23:26] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Enable encryption and authentication for TLS-based Hadoop services - https://phabricator.wikimedia.org/T217412 (10elukey) >>! In T217412#5017402, @Ottomata wrote: >> use a self signed CA > > Another option would be to use the Puppet CA to... [14:24:40] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Enable encryption and authentication for TLS-based Hadoop services - https://phabricator.wikimedia.org/T217412 (10Ottomata) Ah I see! Is that a problem? Can a CA not create multiple certificates with the same CN? [14:26:28] 10Analytics, 10MediaWiki-Vagrant, 10Core Platform Team Backlog (Watching / External), 10Services (watching): Vagrant's /var/log/daemon.log filling up with kafka errors - https://phabricator.wikimedia.org/T187102 (10DLynch) @Elukey No, nothing to fix it. Though it had, looking at it just now, switched into... [14:39:20] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Discovery, and 4 others: Rewrite Avro schemas (ApiAction, CirrusSearchRequestSet) as JSONSchema and produce to EventGate - https://phabricator.wikimedia.org/T214080 (10Ottomata) [14:39:23] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Core Platform Team Kanban (Doing), and 2 others: Add monolog adapters for Eventbus - https://phabricator.wikimedia.org/T216163 (10Ottomata) 05Open→03Resolved [14:45:27] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Core Platform Team Kanban (Done with CPT), and 2 others: Add monolog adapters for Eventbus - https://phabricator.wikimedia.org/T216163 (10CCicalese_WMF) [14:45:29] 10Analytics, 10CirrusSearch, 10EventBus, 10WMF-JobQueue, and 6 others: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10CCicalese_WMF) [14:47:19] ok I managed to deploy 0.31 [14:47:21] (superset) [14:47:42] the UI's CSS seems to have been through a blender [14:50:13] https://github.com/apache/incubator-superset/blob/master/UPDATING.md#superset-0310 - they are saying that they use yarn from 0.31 onwards [14:50:16] mmmm [14:52:50] nope of course I forgot that we have our special setup.py in our fork [14:57:24] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Enable encryption and authentication for TLS-based Hadoop services - https://phabricator.wikimedia.org/T217412 (10elukey) >>! In T217412#5017592, @Ottomata wrote: > Ah I see! Is that a problem? Can a CA not create multiple certificates w... [14:59:00] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Update wikimedia-history revision data with deleted field (and find it a new name?) - https://phabricator.wikimedia.org/T178587 (10Milimetric) Ok, here's what we think Neil, two options: * `page_is_deleted` for page-level, every revision with this page_id... [15:03:00] (03CR) 10Mforns: [C: 03+1] "LRGTM! I added a couple comments, mostly positive, and a couple minor things like typos on comments." (0312 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T216603) (owner: 10Joal) [15:05:02] heya team! looking into alarm [15:05:16] (03CR) 10Nuria: "I see, so you already run an alter on tables for this field , right?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/495734 (owner: 10Joal) [15:06:18] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Enable encryption and authentication for TLS-based Hadoop services - https://phabricator.wikimedia.org/T217412 (10MoritzMuehlenhoff) >>! In T217412#5015285, @elukey wrote: > use a self signed CA, and generate one certificate for each hostn... [15:16:33] mforns: as FYI I was working with Manuel on an-coord1001, some mariadb transactions were stalling waiting for a lock [15:16:45] no idea why, I have only worked on superset in there today [15:17:04] we cleared everything but since the db is used by oozie/hive/etc.. it might have affected some jobs [15:17:20] elukey, I see [15:18:07] elukey, is that mariadb instance the one that stores the hive meta-store? [15:23:11] yep [15:26:06] joal: https://apachecon.com/aceu19/index.html [15:27:54] k, thx [15:30:26] so if you don't find a specific issue let's re-run the coordinator [15:36:22] elukey, already rerun, but I think there are already 2 jobs that are stuck [15:36:24] and with errors [15:36:36] was looking at the grafana dashboard for Hive [15:36:46] and there's some GC spikes [15:37:26] what jobs are stuck? [15:37:33] in theory now they should go fine [15:42:38] elukey, https://hue.wikimedia.org/oozie/list_oozie_workflow/0145638-181112144035577-oozie-oozi-W/?coordinator_job_id=0070354-181112144035577-oozie-oozi-C&bundle_job_id=0070353-181112144035577-oozie-oozi-B [15:42:44] this one has errors [15:42:59] it's hour 12 [15:43:25] same for hour 13 [15:44:25] they are marked as running, but stuck for 35 minutes now [15:44:30] gonna look at logs [15:50:21] ack [15:54:47] yea elukey: 2019-03-12 13:15:43,744 [main] WARN org.apache.hadoop.hive.metastore.RetryingMetaStoreClient - MetaStoreClient lost connection. Attempting to reconnect. [15:54:54] that was the alarmed failure [15:55:31] the logs for the following hourly jobs are not available yet [15:56:31] elukey, if I kill them, we'll get yarn logs? [15:56:57] (03CR) 10Joal: "I did it yesterday evening, to fix Chelsy's problem" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/495734 (owner: 10Joal) [15:59:26] mforns: at some point yes IIRC [15:59:32] I'd kill them and re-run [16:00:30] not super impressed by our resiliency [16:01:00] k [16:06:13] !log Rerun webrequest-load-wf-text-2019-3-12-11 after error [16:06:18] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:02:04] * elukey brb for 10/15 mins! [17:14:08] 10Analytics: Update mediawiki-history subgraph-partitioner so that it uses page_id for pages - https://phabricator.wikimedia.org/T218130 (10JAllemandou) [17:14:36] 10Analytics-Kanban, 10Patch-For-Review: Coordinate work on minor changes for Edit Data Quality - https://phabricator.wikimedia.org/T213603 (10JAllemandou) [17:54:13] * elukey off! [18:02:38] o/ is there a way to determine internal traffic in the webrequest table? or should I just do it based on IP? [18:13:13] addshore: only way I can think of is by IP [18:13:30] ack, IP will do, everything interla is 10. i think :) [18:13:33] simple enough [18:18:29] 10Analytics, 10Product-Analytics, 10Patch-For-Review: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10Ottomata) @Neil_P._Quinn_WMF, I discussed this with Joseph a bit more today and realized that even though what I said above is true, it might not be exactly w... [18:29:52] (03CR) 10Nuria: [C: 03+2] Correct mediawiki_logging log_user field [analytics/refinery] - 10https://gerrit.wikimedia.org/r/495734 (owner: 10Joal) [18:30:31] addshore: there is handy udf you can use [18:31:30] addshore: see https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/wikimedia/analytics/refinery/core/IpUtil.java [18:32:34] addshore: https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-hive/src/main/java/org/wikimedia/analytics/refinery/hive/NetworkOriginUDF.java [18:33:02] oooooh [18:42:21] (03CR) 10Joal: [V: 03+2] Correct mediawiki_logging log_user field [analytics/refinery] - 10https://gerrit.wikimedia.org/r/495734 (owner: 10Joal) [19:33:02] (03CR) 10Joal: "Thanks for the review mforns :) One comment that can lead to new changes, except from that, seems ready." (036 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T216603) (owner: 10Joal) [19:33:15] git st [19:33:17] oops [19:33:47] (03PS9) 10Joal: Refactor mediawiki-history core data gathering [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T216603) [19:34:05] mforns: if you have a minunte, let's close this --^ (one comment to comment ;) [19:34:21] joal, reading [19:35:23] joal, btw is the Phab task correct? [19:35:31] euhhh [19:35:33] looking :) [19:35:51] Wow - It's not !!! incredibly not :) [19:38:05] (03PS10) 10Joal: Refactor mediawiki-history core data gathering [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T206883) [19:38:20] should be better - Thanks mforns [19:38:35] hehe [19:39:05] joal, changes look good! want me to merge? [19:39:23] mforns: dependent on a parent change, we need to wait for that one first [19:39:29] ok will +2 [19:39:35] mforns: Thanks :) [19:39:44] (03CR) 10Mforns: [C: 03+2] "LGTM!" (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T206883) (owner: 10Joal) [19:45:12] 10Analytics, 10Knowledge-Integrity, 10Research, 10Epic, 10Patch-For-Review: Citation Usage: run third round of data collection - https://phabricator.wikimedia.org/T213969 (10leila) @RyanSteinberg Is this something you can address at the analysis time? This is increase in the scope of fixing the schema an... [20:17:18] (03PS5) 10Joal: Add revision_hidden_parts to mediawiki-history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492304 (https://phabricator.wikimedia.org/T178587) [20:19:07] (03PS7) 10Joal: Add change_tags to mediawiki_history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492320 [20:23:17] (03PS6) 10Joal: Update mediawiki-reconstruction with log info [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493012 [20:34:00] (03PS8) 10Joal: Refactor mediawiki-page-history computation [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493390 (https://phabricator.wikimedia.org/T190434) [20:35:28] Gone for tonight team - see you tomorrow [20:56:19] (03PS3) 10Milimetric: Add concept of metric groups, rotate in dashboard [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/494241 (https://phabricator.wikimedia.org/T187806) (owner: 10Fdans) [21:08:14] (03CR) 10Milimetric: [C: 03+2] Bump up the referenced node version [analytics/aqs] - 10https://gerrit.wikimedia.org/r/495255 (https://phabricator.wikimedia.org/T210706) (owner: 10Milimetric) [21:08:57] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Move AQS to nodejs 10 - https://phabricator.wikimedia.org/T210706 (10Milimetric) ok, agreed, will build and deploy to staging now [22:05:04] 10Analytics, 10Analytics-Kanban, 10Analytics-SWAP, 10Product-Analytics, 10Patch-For-Review: Publishing html files generated on notebook hosts - https://phabricator.wikimedia.org/T217619 (10chelsyx) Hi @Ottomata , `published-datasets-sync` works well when I (user chelsyx) execute it. But when I execute it... [22:05:54] 10Analytics, 10Analytics-Kanban, 10Analytics-SWAP, 10Product-Analytics, 10Patch-For-Review: Publishing html files generated on notebook hosts - https://phabricator.wikimedia.org/T217619 (10Ottomata) It is in /usr/local/bin, perhaps your PATH doesn't contain it! But! You shouldn't need to execute it in a... [22:07:03] 10Analytics, 10Analytics-Kanban, 10Analytics-SWAP, 10Product-Analytics, 10Patch-For-Review: Publishing html files generated on notebook hosts - https://phabricator.wikimedia.org/T217619 (10chelsyx) Got it. Thanks @Ottomata ! [22:12:17] 10Analytics, 10Analytics-Cluster, 10Discovery-Search (Current work): Impleentation and testing of glent m0 integration with wmf infra - https://phabricator.wikimedia.org/T218164 (10Jdforrester-WMF) [22:22:45] 10Analytics, 10Analytics-Cluster, 10Discovery-Search (Current work): Implementation and testing of glent m0 integration with wmf infra - https://phabricator.wikimedia.org/T218164 (10EBernhardson) [23:38:17] 10Analytics, 10Research: Check home leftovers of ISI researchers - https://phabricator.wikimedia.org/T215775 (10leila) @MoritzMuehlenhoff do I recall correctly that you were working with 1+ of the researchers to figure out how to keep their data? @elukey in the meantime I'll pick this up again.