[06:12:16] 10Quarry, 10DBA, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10Marostegui) >>! In T246970#5952464, @Mike_Peel wrote: > I'm now getting the normal 'killed' message for going over 30 minutes, rather than the MySQL error. So perhaps things a... [07:07:11] good morning! [07:07:48] I am starting maintenance for notebooks [07:08:07] upgrade to nodejs 10 + deploy of new version [07:12:03] (03CR) 10Elukey: [V: 03+2 C: 03+2] Upgrade nodejs and jupyterhub-systemdspawner [analytics/jupyterhub/deploy] - 10https://gerrit.wikimedia.org/r/577477 (https://phabricator.wikimedia.org/T247055) (owner: 10Elukey) [07:26:30] !log upgrade nodejs from 6->10 on stat1* and notebook1* [07:26:31] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:27:28] !log deploy jupyterhub on notebook100[3,4] (manual venv re-creation) to allow the use of the user.slice - T247055 [07:27:30] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:27:31] T247055: Upgrade jupyterhub-systemdspawner from 0.9.9 to 0.13 to allow the use of systemd custom slices - https://phabricator.wikimedia.org/T247055 [07:36:06] ok deployed and tested briefly, looks working [07:36:31] the caveat is that people will need to stop/start their notebooks to get into the user.slice [07:40:14] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Upgrade jupyterhub-systemdspawner from 0.9.9 to 0.13 to allow the use of systemd custom slices - https://phabricator.wikimedia.org/T247055 (10elukey) From my tests the new environment seems working fine. Caveat: users need to stop/start their notebooks to... [07:51:49] next step for buster is https://gerrit.wikimedia.org/r/#/c/analytics/jupyterhub/deploy/+/577761/1/frozen-requirements.txt [07:54:11] very good news is also https://phabricator.wikimedia.org/T246472#5951271 [07:54:18] stat1008's GPU! [08:03:02] Good morning :) [08:03:44] elukey: the above about GPU seems to say that the machine has enough space for 2 GPUs? That'd be great! [08:08:56] joal: bonjour! yes it seems so from that John says, I kinda worry about the thermal heat thogh [08:08:59] *though [08:09:08] very reasonnable doubt elukey :) [08:09:25] This machine will be good to keep caffe warm :) [08:10:27] ahhahaha yes [08:40:43] joal: if you are ok I'd deploy jupyterhub on stat1004 and stat1005 [08:40:50] +1 elukey :) [08:46:40] (03Abandoned) 10Elukey: Add Debian Buster artifacts [analytics/jupyterhub/deploy] - 10https://gerrit.wikimedia.org/r/577761 (https://phabricator.wikimedia.org/T245179) (owner: 10Elukey) [08:58:15] I am sending a new patch --^ [08:58:33] basically I also upgraded jupyterhub to latest when releasing to buster [08:58:48] so in my mind on stat1005 we should have the latest and greatest to test [08:58:51] (03PS1) 10Elukey: Upgrade dependencies for Debian Buster [analytics/jupyterhub/deploy] - 10https://gerrit.wikimedia.org/r/578290 (https://phabricator.wikimedia.org/T245179) [08:58:55] and on stretch the current version [09:03:42] (03CR) 10Elukey: [V: 03+2 C: 03+2] Upgrade dependencies for Debian Buster [analytics/jupyterhub/deploy] - 10https://gerrit.wikimedia.org/r/578290 (https://phabricator.wikimedia.org/T245179) (owner: 10Elukey) [10:29:39] (03CR) 10Fdans: "@Nuria if you're referring to the dashboard, yes it is still there. This change only removes the one under the detail page graphs." (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577531 (https://phabricator.wikimedia.org/T212032) (owner: 10Fdans) [11:12:16] 10Analytics, 10Analytics-Kanban, 10Release Pipeline, 10Patch-For-Review, and 2 others: Migrate EventStreams to k8s deployment pipeline - https://phabricator.wikimedia.org/T238658 (10akosiaris) >>! In T238658#5948568, @Ottomata wrote: >> merged and pooled 1 kubernetes host with a very low weight (1) in the... [11:47:35] all right took a bit but stat1004 is ready [11:47:53] stat1005 shows some errors but i expected those, brand new stuff [11:49:44] 10Analytics, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10fgiunchedi) >>! In T226986#5949928, @Ottomata wrote: > @fgiunchedi very few error events are flowing in... [11:56:01] 10Analytics, 10Inuka-Team, 10Product-Analytics: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10hueitan) >>! In T244547#5913262, @nshahquinn-wmf wrote: >>>! In T244547#5912212, @Nuria wrote: >>>Regarding the base url en.wikipedia.org/beacon/event, do we >> >> Those beac... [12:04:17] 10Analytics, 10Research, 10Patch-For-Review, 10User-Elukey: Add SWAP profile to stat1005 - https://phabricator.wikimedia.org/T245179 (10elukey) Deployed on stat1004, everything seems working fine. On stat1005 (running buster) I can see: ` Mar 09 11:56:23 stat1005 jupyterhub[38874]: [C 2020-03-09 11:56:23... [12:06:12] * elukey lunch! [12:27:17] 10Analytics, 10Analytics-Kanban, 10Release Pipeline, 10Patch-For-Review, and 2 others: Migrate EventStreams to k8s deployment pipeline - https://phabricator.wikimedia.org/T238658 (10akosiaris) > Converting it to a histogram is easy enough, but in combination with the path variable will generate even more m... [13:28:03] joal: thanks a lot for your analysis! It was really interesteing! [13:30:20] gehel: Happy to help - It's fun for me as well :) [13:30:31] gehel: I'll get some more done based on what we have discussed [13:30:37] cool! [13:31:10] There is quite a bit to digest on my side. It isn't entirely clear to me yet how this will influence what we do and how we do it. [13:31:28] We already had some action items during the discussion, but I did not take notes :/ [13:39:36] o/ [13:44:33] 10Analytics, 10Analytics-Kanban: Upgrade jupyterhub-systemdspawner from 0.9.9 to 0.13 to allow the use of systemd custom slices - https://phabricator.wikimedia.org/T247055 (10Ottomata) > Caveat: users need to stop/start their notebooks to pick up the new user.slice change :) You can restart them manually if y... [14:04:21] 10Analytics, 10Inuka-Team, 10Product-Analytics: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10Nuria) In order to count pageviews the UA of the app needs to be "wikipediaApp/" [14:05:51] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Virtual pageviews should set access_type to mobile if webhost is a mobile one - https://phabricator.wikimedia.org/T246309 (10Nuria) @hueitan Hello, Please be so kind to modify the VirtualPageview schema adding access method, that will ensure access method... [14:30:13] 10Analytics, 10ContentTranslation: Test Performance of marian NT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10Nuria) [14:31:05] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-January-March): Test Performance of marian NT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10Pginer-WMF) [14:31:46] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-January-March): Test Performance of marian NT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10Pginer-WMF) p:05Triage→03High [14:32:48] 10Analytics, 10ContentTranslation, 10Operations, 10SRE-Access-Requests, 10Language-Team (Language-2020-January-March): Request for access for stats machines for Santhosh - https://phabricator.wikimedia.org/T247246 (10Nuria) [14:47:11] 10Analytics, 10Pageviews-API: Add wikimania.wikimedia.org to pageview definition - https://phabricator.wikimedia.org/T216525 (10Milimetric) I think we can include it now, I will move this to our inbox and re-triage [14:48:34] 10Analytics, 10Analytics-Kanban: Problem with Matomo page overlay - https://phabricator.wikimedia.org/T246046 (10Milimetric) Yes, I think VIP needs to check your headers to make sure they're right and the matomo settings align as well. I'm not sure how they've set them, but the linked article describes one wa... [14:50:36] already time for standup? [14:51:10] ah daylight hour change in US! [14:55:01] indeed elukey [14:57:16] 10Analytics, 10ContentTranslation, 10Operations, 10SRE-Access-Requests, 10Language-Team (Language-2020-January-March): Request for access for stats machines for Santhosh - https://phabricator.wikimedia.org/T247246 (10Nuria) @santhosh What is your LDAP user? [15:07:17] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-January-March): Test Performance of marian NT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10Nuria) cc @JAllemandou so he can help @santhosh if he needs help. We recommend you use stat1006.eqiad.wmnet Some docs abo... [15:25:32] 10Analytics, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10fgiunchedi) And we're on: https://logstash.wikimedia.org/goto/edf04ab8ff11b50a69ecf9988337b7e1 [15:27:41] hey all am missing meetings today, kinda half working :) [15:27:43] but check this out! [15:27:44] https://logstash.wikimedia.org/app/kibana#/dashboard/default?_g=h@44136fa&_a=h@98157c9 [15:27:46] nuria: ^^^ [15:31:20] 10Analytics, 10Analytics-Kanban, 10Release Pipeline, 10Patch-For-Review, and 2 others: Migrate EventStreams to k8s deployment pipeline - https://phabricator.wikimedia.org/T238658 (10fgiunchedi) >>! In T238658#5953276, @akosiaris wrote: > It does have it's usefulness. As I pointed out, gauges have the probl... [15:39:45] 10Analytics: Question box in wikistats, instrument usage with piwik - https://phabricator.wikimedia.org/T247106 (10fdans) p:05Triage→03High [15:41:52] 10Analytics: clear bot spam-scraping [[en:United States Senate]] not being detected as a bot - https://phabricator.wikimedia.org/T247085 (10fdans) a:03JAllemandou [15:43:59] 10Analytics, 10Scoring-platform-team, 10draftquality-modeling, 10artificial-intelligence: Productionize monthly article quality prediction datasets - https://phabricator.wikimedia.org/T194741 (10fdans) ping us if anything is needed from us [15:45:47] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: [Wikistats v2] Default selection for (active) editors is confusing for inexperienced users - https://phabricator.wikimedia.org/T213800 (10fdans) let's do it! [15:46:01] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: [Wikistats v2] Default selection for (active) editors is confusing for inexperienced users - https://phabricator.wikimedia.org/T213800 (10fdans) p:05Medium→03High [15:50:14] 10Analytics, 10Pageviews-API: Add wikimania.wikimedia.org to pageview definition - https://phabricator.wikimedia.org/T216525 (10fdans) This is a minor change, but can we have a reason why this is needed? We're wondering who's interested in this data. [15:52:10] 10Analytics, 10Operations, 10Wikidata, 10Wikidata-Query-Service: Deployment strategy and hardware requirement for new Flink based WDQS updater - https://phabricator.wikimedia.org/T247058 (10Nuria) I think it will be very helpful to have a design document for this service so we are all in the same page of w... [15:57:16] lexnasser: o/ - ok if I stop you notebook on notebook1003? [15:57:31] elukey: yep, thats fine [15:57:41] thanks! [15:59:11] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Tech Tunning Session metrics - https://phabricator.wikimedia.org/T247100 (10Milimetric) a:03Nuria [16:01:14] 10Analytics: Kerberos credential cache expiry time on notebook is different than the OS one - https://phabricator.wikimedia.org/T247084 (10Milimetric) p:05Triage→03High [16:02:00] 10Analytics: Upgrade AMD ROCm to latest upstream - https://phabricator.wikimedia.org/T247082 (10Milimetric) p:05Triage→03High [16:05:03] 10Analytics: Metric widget won't stretch when content overflows - https://phabricator.wikimedia.org/T246913 (10Milimetric) p:05Triage→03Medium [16:05:36] 10Analytics: Wiki selector: "All wikis" is not translated - https://phabricator.wikimedia.org/T246911 (10Milimetric) p:05Triage→03Medium [16:06:14] 10Analytics: Requesting Kerberos authentication principal for Michael Holloway - https://phabricator.wikimedia.org/T246834 (10Milimetric) p:05Triage→03High a:03elukey [16:07:59] 10Analytics, 10Analytics-Kanban: Drastically reduce build time for languages - https://phabricator.wikimedia.org/T246778 (10Milimetric) p:05Triage→03High a:03Milimetric [16:08:18] 10Analytics: Requesting Kerberos authentication principal for Michael Holloway - https://phabricator.wikimedia.org/T246834 (10elukey) ` elukey@krb1001:~$ sudo manage_principals.py create mholloway-shell --email_address=mholloway@wikimedia.org Principal successfully created. Make sure to update data.yaml in Puppe... [16:09:15] 10Analytics: Use version of Lato that renders non-roman alphabets - https://phabricator.wikimedia.org/T246777 (10Milimetric) p:05Triage→03Medium [16:10:28] 10Analytics, 10User-Elukey: Investigate Hadoop HDFS ACLs - https://phabricator.wikimedia.org/T246755 (10Milimetric) p:05Triage→03Medium [16:12:29] 10Analytics, 10Analytics-Kanban: geoeditors-yearly job times out - https://phabricator.wikimedia.org/T246753 (10Milimetric) p:05Triage→03High a:03Milimetric [16:13:22] 10Analytics: Make spark-refine resilient to incorrectly formatted _REFINED files - https://phabricator.wikimedia.org/T246706 (10Milimetric) p:05Triage→03High a:03mforns [16:13:44] 10Analytics, 10Patch-For-Review: Requesting Kerberos authentication principal for Michael Holloway - https://phabricator.wikimedia.org/T246834 (10elukey) 05Open→03Resolved [16:14:36] 10Analytics: Add historical page protection status to MediaWiki history - https://phabricator.wikimedia.org/T246723 (10Milimetric) p:05Triage→03Medium [16:16:42] 10Analytics, 10Analytics-Cluster: Hue doesn't show executor details - https://phabricator.wikimedia.org/T246724 (10Milimetric) p:05Triage→03Medium [16:17:29] 10Analytics: Remove Topic Explorer from Wikistats - https://phabricator.wikimedia.org/T247064 (10Milimetric) [16:17:31] 10Analytics: Question box in wikistats, instrument usage with piwik - https://phabricator.wikimedia.org/T247106 (10Milimetric) [16:17:43] 10Analytics, 10Analytics-Cluster: Hue doesn't show executor details - https://phabricator.wikimedia.org/T246724 (10elukey) @awight hi! Can you clarify a bit what do you mean with "executors" and how to repro? Is it Spark related or Hive related? [16:18:02] 10Analytics: Remove Topic Explorer from Wikistats - https://phabricator.wikimedia.org/T247064 (10Milimetric) p:05Triage→03Medium [16:20:19] 10Analytics, 10Analytics-Wikistats: Wikistats. How about historic data? - https://phabricator.wikimedia.org/T186124 (10Milimetric) 05Open→03Declined We're not likely to migrate directly from Wikistats 1, we will probably move the metrics and provide data all the way back. Keeping Wikistats 1 for archival... [16:21:29] 10Analytics: Consider changing Phab projects to sub-projects - https://phabricator.wikimedia.org/T208798 (10Milimetric) 05Open→03Declined assuming this is no longer an urgent issue [16:23:53] 10Analytics: Design Work - Wikistats2 - https://phabricator.wikimedia.org/T198415 (10Milimetric) 05Open→03Declined will create a fresh version of this based on @fdans' design goals doc [16:25:59] 10Analytics, 10Android-app-Bugs, 10Wikipedia-Android-App-Backlog: Count link previews on the Android app - https://phabricator.wikimedia.org/T194961 (10Milimetric) 05Open→03Declined in favor of a general approach as part of Better Use of Data [16:26:55] 10Analytics, 10Analytics-Wikistats, 10Product-Analytics: Contribution inequality graphs for Wikistats - https://phabricator.wikimedia.org/T195033 (10Milimetric) p:05Triage→03Low [16:27:53] 10Analytics: Implement digest-only mediawiki_history_reduced dataset in spark - https://phabricator.wikimedia.org/T181703 (10Milimetric) When we triage this next, let's define/task our solution from Offsite Mallorca and assign/prioritize. [16:30:34] 10Analytics: Enhance mediawiki-history page reconstruction with best historical information possible - https://phabricator.wikimedia.org/T179692 (10Milimetric) p:05Triage→03Low [16:31:45] 10Analytics, 10Cloud-VPS, 10Event-Platform, 10Core Platform Team Legacy (Watching / External), 10Services (watching): Set up a Cloud VPS Kafka Cluster with replicated eventbus production data - https://phabricator.wikimedia.org/T187225 (10Milimetric) 05Open→03Declined we can help with this but declin... [16:32:16] 10Analytics, 10Analytics-Wikistats: Contextualize wikistats metrics - https://phabricator.wikimedia.org/T187212 (10Milimetric) p:05Triage→03Medium [16:33:41] 10Analytics, 10Analytics-Wikistats: Trends for editor types, and new editors in particular (in Wikistats 2.0) - https://phabricator.wikimedia.org/T186791 (10Milimetric) p:05Triage→03Medium [16:37:04] 10Analytics: Provide data dumps in the Analytics Data Lake - https://phabricator.wikimedia.org/T186559 (10Milimetric) 05Open→03Resolved a:03Milimetric all child tasks resolved, will tag the remaining ones mentioned in the description as #analytics [16:37:17] 10Analytics, 10Data-release, 10Research-Backlog: Create a Historical Link Graph for Wikipedia - https://phabricator.wikimedia.org/T186558 (10Milimetric) [16:37:27] 10Analytics, 10Growth-Team, 10Product-Analytics: Measurement for AfC improvement (April 2018) - https://phabricator.wikimedia.org/T192515 (10Milimetric) [16:41:52] 10Analytics, 10Data-release, 10Research-Backlog: Create a Historical Link Graph for Wikipedia - https://phabricator.wikimedia.org/T186558 (10Milimetric) Let's collaborate on infrastructure for this [16:41:59] 10Analytics, 10Analytics-Kanban: Data Quality Alarms - https://phabricator.wikimedia.org/T198986 (10Milimetric) [16:42:01] 10Analytics, 10Analytics-EventLogging: Alarm on errors on /var/log/upstart/eventlogging* files - https://phabricator.wikimedia.org/T170620 (10Milimetric) 05Open→03Invalid no longer relevant [16:45:25] 10Analytics, 10Analytics-Cluster, 10Analytics-Kanban: Sanitize pageview_hourly - https://phabricator.wikimedia.org/T114675 (10Milimetric) Rework this to be more generic and allow a differentially private approach instead of k-anonimizing. [16:46:01] 10Analytics: Deploy pageview sanitization and start ongoing process {hawk} - https://phabricator.wikimedia.org/T118841 (10Milimetric) [16:46:03] 10Analytics: Productionize Pageview_sanitization hive code with Oozie job and refinery inclusion {hawk} - https://phabricator.wikimedia.org/T118839 (10Milimetric) 05Open→03Declined leaving parent task open, considering diff privacy approach [16:46:05] 10Analytics, 10Analytics-Cluster, 10Analytics-Kanban: Sanitize pageview_hourly - https://phabricator.wikimedia.org/T114675 (10Milimetric) [16:46:45] 10Analytics, 10Product-Analytics: Get "edits hourly" on a daily basis - https://phabricator.wikimedia.org/T231938 (10Milimetric) p:05Medium→03High [16:49:39] 10Analytics: Wikistats2 and SEO - https://phabricator.wikimedia.org/T192172 (10Milimetric) p:05Medium→03Low [16:52:51] 10Analytics: Remove user_agent_map from pageview_hourly long term - https://phabricator.wikimedia.org/T156965 (10Milimetric) 05Open→03Declined subset of sanitizing, etc. [16:55:04] 10Analytics: Add templating support to reportupdater scripts - https://phabricator.wikimedia.org/T163252 (10Milimetric) 05Open→03Invalid Dan doesn't understand how Reportupdater works :) [16:55:06] 10Analytics: reportupdater TLC - https://phabricator.wikimedia.org/T193167 (10Milimetric) [16:55:53] 10Analytics: marker for backlog (later) - https://phabricator.wikimedia.org/T247253 (10Nuria) [16:56:39] 10Analytics: marker for backlog (later) - https://phabricator.wikimedia.org/T247253 (10Milimetric) p:05Triage→03Medium [16:57:39] 10Analytics: (Marker) Everything Above is for Deprioritized column - https://phabricator.wikimedia.org/T247253 (10Milimetric) [18:05:14] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): EventLogging MEP Upgrade - https://phabricator.wikimedia.org/T238544 (10Krinkle) @Ottomata If I understand correctly, this proposal would require that the configuration data: * … for... [18:06:58] ottomata: that logstash url did not work, can you send it again? [18:07:08] ottomata: there is some share button you need to use [18:07:43] ottomata: *i think*, cause it says "cannot restore url" [18:08:13] * elukey off! [18:09:25] ottomata: never mind filipo's url on ticket works [18:14:38] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-January-March): Test Performance of Marian NMT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10ssastry) [18:17:03] 10Analytics, 10Operations, 10Traffic, 10Performance-Team (Radar): Send X-Analytics information from Varnish to Hadoop with VCL_Log - https://phabricator.wikimedia.org/T196558 (10Krinkle) 05Open→03Resolved [18:34:12] sorry team, I didn't notice an internet hiccup broke my IRC connection [18:39:57] 10Analytics, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10Nuria) Almost there, so we have errors on test wiki but we have not yet enabled haw.wikipedia.org from w... [18:45:02] 10Analytics, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10Nuria) ping @Tgr where should the client side error logging be documented? (cc @phuedx for appropriate m... [19:04:49] nuria: heya - would be there by any chance? [19:15:57] (03PS1) 10Joal: Add automated agent-type to pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/578373 (https://phabricator.wikimedia.org/T238363) [19:31:10] (03CR) 10Nuria: Add automated agent-type to pageview_hourly (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/578373 (https://phabricator.wikimedia.org/T238363) (owner: 10Joal) [19:33:32] (03CR) 10Joal: Add automated agent-type to pageview_hourly (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/578373 (https://phabricator.wikimedia.org/T238363) (owner: 10Joal) [19:47:37] (03PS2) 10Joal: Add automated agent-type to pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/578373 (https://phabricator.wikimedia.org/T238363) [19:49:44] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): EventLogging MEP Upgrade - https://phabricator.wikimedia.org/T238544 (10Ottomata) @Krinkle, my most recent suggestion was based on @jlinehan's conversation with and the summary he made... [19:58:06] nuria: shadow-mode started - will review tomorrow on tonights-computed data (https://hue.wikimedia.org/oozie/list_oozie_coordinator/0017297-200303081945184-oozie-oozi-C/) [20:10:49] (03PS3) 10Joal: Add automated agent-type to pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/578373 (https://phabricator.wikimedia.org/T238363) [20:26:33] Gone for tonight [22:47:39] 10Analytics, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10Krinkle) >>! In T226986#5954010, @fgiunchedi wrote: > And we're on: https://logstash.wikimedia.org/goto/... [23:34:44] 10Analytics, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10Tgr) Shouldn't the stack trace be structured? These look like just the native browser stack trace. >>!...