[00:58:44] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Desktop Improvements, and 8 others: Enable client side error logging in prod for small wiki - https://phabricator.wikimedia.org/T246030 (10Nuria) @Tgr Super thanks for the help, gergo
[01:30:15] <wikibugs>	 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Jan-Mar 2020): Further improvements to the WMCS edits dashboard - https://phabricator.wikimedia.org/T240040 (10srishakatux)
[01:31:17] <wikibugs>	 10Analytics, 10Cloud-Services, 10Developer-Advocacy, 10Documentation: Set up a "WMCS Edits Dashboard" page on Meta - https://phabricator.wikimedia.org/T246932 (10srishakatux) (added link to the meta-page from the dashboard; [[ https://github.com/srish/analytics-dashiki/commit/362780babb1e739d9683b14be18de1...
[01:37:12] <icinga-wm>	 PROBLEM - Hadoop NodeManager on analytics1074 is CRITICAL: PROCS CRITICAL: 0 processes with command name java, args org.apache.hadoop.yarn.server.nodemanager.NodeManager https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Administration
[02:05:19] <wikibugs>	 10Analytics: Modify ReportUpdater to support `YYYY-MM` dates for monthly reports - https://phabricator.wikimedia.org/T245096 (10srishakatux)
[02:06:55] <wikibugs>	 10Analytics, 10Tools: Pie chart is missing on the WMCS Edits dashboard - https://phabricator.wikimedia.org/T246963 (10srishakatux)
[02:07:33] <wikibugs>	 10Analytics, 10Tools: Pie chart is missing on the WMCS Edits dashboard - https://phabricator.wikimedia.org/T246963 (10srishakatux) @mforns related to our conversation this morning. Realized there is more to it :)
[06:23:37] <wikibugs>	 10Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10Jdx)
[06:48:53] <elukey>	 !log restart yarn on analytics1074 (GC overhead, traces of network errors with datanodes)
[06:48:54] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[06:54:02] <icinga-wm>	 RECOVERY - Hadoop NodeManager on analytics1074 is OK: PROCS OK: 1 process with command name java, args org.apache.hadoop.yarn.server.nodemanager.NodeManager https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Administration
[06:54:02] <icinga-wm>	 RECOVERY - Check if the Hadoop HDFS Fuse mountpoint is readable on notebook1003 is OK: CRITICAL https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Administration%23Fixing_HDFS_mount_at_/mnt/hdfs
[06:54:35] <elukey>	 gooood
[06:57:24] <wikibugs>	 10Analytics, 10Product-Analytics (Kanban): Update wmfdata to support multiple SQL engines for Hive databases - https://phabricator.wikimedia.org/T246060 (10nshahquinn-wmf) a:05mpopov→03nshahquinn-wmf Mikhail reviewed and merged the pull request! I'm going to do a little extra cleanup while we're at it, and...
[07:15:22] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Add multilanguage ability to Wikistats - https://phabricator.wikimedia.org/T238752 (10fdans) There are now a number of languages available in the wikistats website. We'll be adding more languages as they are pushed from TranslateWiki.  There are other i...
[07:37:41] <wikibugs>	 10Analytics: Errors on wikistats UI on console - https://phabricator.wikimedia.org/T246789 (10fdans) The annotations meta page for this metric was malformed. A user tried to delete his own annotation but instead deleted the template in the page.  https://meta.wikimedia.org/w/index.php?title=Config%3ADashiki%3AAn...
[07:38:24] <wikibugs>	 10Analytics: Errors on wikistats UI on console - https://phabricator.wikimedia.org/T246789 (10fdans) p:05Triage→03High
[07:38:37] <wikibugs>	 10Analytics, 10Analytics-Kanban: Errors on wikistats UI on console - https://phabricator.wikimedia.org/T246789 (10fdans)
[07:38:45] <wikibugs>	 10Analytics: Specify in build command which languages to bundle - https://phabricator.wikimedia.org/T246745 (10fdans) p:05Triage→03High
[07:39:03] <wikibugs>	 10Analytics, 10Analytics-Kanban: Specify in build command which languages to bundle - https://phabricator.wikimedia.org/T246745 (10fdans) a:03fdans
[07:50:47] <joal>	 Good morning
[08:29:30] <wikibugs>	 10Analytics, 10Analytics-Kanban: Language selector is not pressable in mobile site - https://phabricator.wikimedia.org/T246971 (10fdans)
[08:29:39] <wikibugs>	 10Analytics, 10Analytics-Kanban: Language selector is not pressable in mobile site - https://phabricator.wikimedia.org/T246971 (10fdans) p:05Triage→03High
[08:30:40] <joal>	 fdans: hello :) I checked wikistats in french - it's great :)
[08:31:08] <fdans>	 joal: awwww thank you joseph :D
[08:31:14] <joal>	 fdans: question around usability - Wouldn't the language-selector be better close to the top-page (visible straight)?
[08:31:42] <fdans>	 joal: that's a good question - I'm not sure
[08:32:06] <fdans>	 right now dan and I are reassessing a lot of the ui elements in the main page and that might get changed
[08:32:07] <joal>	 fdans: with thew selector hidden, people might not even think it is feasible to change language
[08:32:16] <fdans>	 yeah it's not very obvious
[08:32:30] <joal>	 fdans: just saying - I'm by no mean UI advisor :)
[08:32:54] <joal>	 Thanks a lot nonetheless fdans :)
[08:33:09] <fdans>	 joal: no, it makes sense - at the same time the language is auto selected if your browser is in a non-english language
[08:33:26] <joal>	 Ah - makes sense as well
[08:33:35] * joal is in deeper wonder :)
[08:34:44] <fdans>	 joal: my thought is... the main reason why you would want to change the language to one other than your browser is if you want to switch to english/you're unhappy with the translation to your language
[08:34:50] <fdans>	 like it happens in wikipedia
[08:35:06] <fdans>	 so
[08:35:33] <joal>	 fdans: or like me, you use english as default lang in your computer and would rather have your main language in statsfor instance
[08:35:46] <joal>	 this is probably a lot less frequent --^
[08:36:08] <fdans>	 the fact that you're seeing wikistats in a non-english language should be an implicit indication that there is a language selector somewhere, I think
[08:36:16] <fdans>	 but yeah, there's also your usecase joal 
[08:37:28] <joal>	 fdans: me even talking about that subject is because on most websites I have visited, language selector is in the top-bar - it's almost a convention in my mind - But this opinion is very personal :)
[08:45:02] <wikibugs>	 10Analytics, 10Data-Services, 10cloud-services-team (Kanban): Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511 (10Marostegui)
[09:46:00] <wikibugs>	 (03PS1) 10Addshore: Track wikibase repo table auto increment usage above 25% [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577202 (https://phabricator.wikimedia.org/T68025)
[09:58:38] <wikibugs>	 (03PS2) 10Addshore: Track wikibase repo table auto increment usage above 25% [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577202 (https://phabricator.wikimedia.org/T68025)
[10:03:20] <wikibugs>	 (03CR) 10Joal: [C: 04-1] "1 naming discussion and 1 missing file :)" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/576618 (https://phabricator.wikimedia.org/T244597) (owner: 10Lex Nasser)
[10:15:41] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down for a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Mainframe98)
[10:15:44] <wikibugs>	 (03PS3) 10Addshore: Track wikibase repo table auto increment usage above 25% [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577202 (https://phabricator.wikimedia.org/T68025)
[10:16:22] <wikibugs>	 (03CR) 10Addshore: [V: 04-1 C: 04-1] "As far as I can tell the researcher user that this query would run under here doesn't have access to run this query" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577202 (https://phabricator.wikimedia.org/T68025) (owner: 10Addshore)
[10:48:21] <wikibugs>	 (03PS1) 10WMDE-Fisch: [DNM] Count disables of the TwoColumnConflict interface [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577213 (https://phabricator.wikimedia.org/T246104)
[10:59:33] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down for a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Aklapper) Hi, what does "down" mean exactly? Is there an error message in the browser? Which HTTP error code does the network tab in the browser dev...
[11:05:59] <wikibugs>	 (03PS1) 10Fdans: Add available languages variable in dev mode too [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577215
[11:07:19] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down for a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Ciell) {F31666596}  {F31666594}  "Down" is this case means "completely blanc", doesn't load, no error message. (see attached) Pinging I tried, but I...
[11:52:35] <wikibugs>	 (03PS1) 10Fdans: Set access method to 'mobile' in virtualpageview when detected [analytics/refinery] - 10https://gerrit.wikimedia.org/r/577220 (https://phabricator.wikimedia.org/T246309)
[11:56:02] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down for a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Ciell) Fout tijdens het parsen van waarde voor ‘-webkit-text-size-adjust’.  Declaratie genegeerd. main.bundle.2.7.1.css:12:150 Onbekende eigenschap...
[12:19:48] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Ciell)
[12:23:11] <joal>	 fdans: heya - I htink this might be related to the change on languages https://phabricator.wikimedia.org/T246976
[12:24:09] <joal>	 fdans: see error in last comment, it mentions ‘https://stats.wikimedia.org/assets-v2/main.bundle.2.7.1.nl.js’
[12:25:31] <fdans>	 joal yep, i’m on it, was waiting for him to answer
[12:25:45] <joal>	 ack fdans - thanks a lot :)
[12:32:31] <wikibugs>	 (03CR) 10Jcrespo: "I am not convinced this is the right approach- we should talk more. I_S querying could have some performance implications, plus I am not s" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577202 (https://phabricator.wikimedia.org/T68025) (owner: 10Addshore)
[12:41:44] <wikibugs>	 (03PS1) 10Fdans: Don't enable locales if they haven't been built [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577230
[12:44:52] <fdans>	 joal: mind reading this and giving me a +1 for a speedy deploy? 
[12:45:01] <joal>	 reading fdans !
[12:45:14] <fdans>	 sorry 1 sec joal havent updated commit message
[12:45:32] <wikibugs>	 (03PS2) 10Fdans: Don't enable locales if they haven't been built [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577230 (https://phabricator.wikimedia.org/T246976)
[12:45:36] <fdans>	 now joal :)
[12:45:39] <fdans>	 sorry
[12:48:14] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10fdans) Thank you for filing @Ciell, I just uploaded a patch to fix this. It seems we were accepting "nl" as an available locale, but it hasn't bee...
[12:48:32] <joal>	 fdans: just to be sure I understand - what will happen with nl local thne?
[12:49:27] <joal>	 fdans: I assume there is a default one, but can't understand from code
[12:52:41] <joal>	 fdans: I think I could do with an explanation of why moving the HtmlWebpackPlugin from config to build fixes :)
[12:53:53] <fdans>	 joal: the “avalableLocales” variable is the key
[12:54:35] <fdans>	 with that move that property is set to “locales” which is filtered to those that we’ve actually built, which doesn’t include nl, for example
[12:55:19] <joal>	 fdans: triple checking my understanding - the i18ndir contains more than teh locales we build 
[12:58:32] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Ciell) Thank you very much @fdans!   I'll put your request on my to do list, I don't mind doing translations. But first let me get through this we...
[12:59:41] <fdans>	 joal: that's right, some of them have way too few strings translated for them to be useful
[12:59:51] <joal>	 fdans: now another wonder - where do we define languages to be built since there is more than the ones in the folder
[12:59:54] <joal>	 ?
[13:00:00] <joal>	 fdans: sorry for asking so many, trying to et an understanding :)
[13:00:10] <fdans>	 joal: with the deploy command
[13:00:15] <joal>	 Ahhh :)
[13:00:22] <joal>	 ok
[13:01:33] <joal>	 fdans: like the 'npm run build' one
[13:01:53] <fdans>	 joal: yes!
[13:01:56] <fdans>	 right now it's npm run build languages zh,fr,lb,pt,tr,uk,en
[13:01:57] <joal>	 cool :)
[13:02:37] <joal>	 fdans: shall I mergE?
[13:02:50] <fdans>	 joal: please :)
[13:02:51] <joal>	 fdans: you have tested I assume :)
[13:02:58] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "LGTM - assumed tested." [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577230 (https://phabricator.wikimedia.org/T246976) (owner: 10Fdans)
[13:03:19] <wikibugs>	 (03CR) 10Joal: [C: 03+2] Add available languages variable in dev mode too [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577215 (owner: 10Fdans)
[13:03:40] <joal>	 ok, merged
[13:03:48] <fdans>	 joal: thank you!
[13:03:59] <joal>	 one request fdans: can you please update deployment docs once finished the prod-fix?
[13:04:17] <fdans>	 joal: sure
[13:04:24] <joal>	 Thanks mate
[13:08:24] <wikibugs>	 (03PS2) 10WMDE-Fisch: Count disables of the TwoColumnConflict interface [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577213 (https://phabricator.wikimedia.org/T246104)
[13:17:13] <wikibugs>	 (03PS1) 10Fdans: Release 2.7.2 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577233
[13:17:38] <wikibugs>	 (03CR) 10Fdans: [V: 03+2 C: 03+2] Release 2.7.2 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577233 (owner: 10Fdans)
[13:18:56] <wikibugs>	 (03PS2) 10Fdans: Release 2.7.2 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577233
[13:30:07] <elukey>	 back!
[13:36:38] <elukey>	 Amir1: <3 for moving stuff to stat1005!
[13:37:52] <Amir1>	 elukey: oh don't worry. I should have done it waaaay soooner
[13:41:05] <Amir1>	 sorry about that
[13:41:37] <elukey>	 nono please, thanks for doing it :)
[13:41:44] <elukey>	 I am trying to load balance people across stat boxes
[13:41:59] <elukey>	 eventually all stat hosts will be configured the same, so it will be easier to move
[13:42:12] <wikibugs>	 10Analytics, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10fdans) Just checked and this issue is now fixed in production. Thank you again for filing!
[13:43:13] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10fdans) p:05Triage→03High
[13:45:44] <Amir1>	 https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&refresh=5m&var-server=stat1007&var-datasource=eqiad%20prometheus%2Fops&var-cluster=analytics&fullscreen&panelId=12 
[13:45:54] <Amir1>	 basically 2% more free 
[13:46:21] <elukey>	 :)
[13:53:08] <wikibugs>	 (03CR) 10Thiemo Kreuz (WMDE): [C: 04-1] Count disables of the TwoColumnConflict interface (032 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577213 (https://phabricator.wikimedia.org/T246104) (owner: 10WMDE-Fisch)
[14:14:30] <elukey>	 I am going to roll restart hdfs/yarn masters to pick up the proxy changes that I made for Superset
[14:14:38] <elukey>	 joal: green light? 
[14:14:49] <joal>	 ack elukey!
[14:16:58] <elukey>	 !log restart hdfs/yarn master daemons to pick up new core-site changes for Superset
[14:17:00] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[15:01:12] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Desktop Improvements, and 8 others: Enable client side error logging in prod for small wiki - https://phabricator.wikimedia.org/T246030 (10Jdforrester-WMF)
[15:07:21] <elukey>	 so, I have a surprise
[15:07:43] <elukey>	 namely I am now checking Marcel's dashboard in superset staging in all its glory
[15:18:27] <elukey>	 I just used the wmf.netflow table for a count
[15:19:01] <elukey>	 there are some things to sort out, like how to log user and queries, etc..
[15:23:18] <elukey>	 and impersonation seems to work 
[15:23:32] <elukey>	 I am checking the hdfs-audit.log, and I see me proxied via presto
[15:23:42] <elukey>	 if I turn off impersonation, I see superset
[15:28:58] <ottomata>	 ohhhh boy
[15:29:20] <elukey>	 yep it looks very good :)
[15:29:34] <elukey>	 I had to find a way to pass to sqlalchemy the self signed CA cert
[15:29:49] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 6 others: Public EventGate instance and endpoint for analytics event intake: eventgate-analytics-external - https://phabricator.wikimedia.org/T233629 (10Ottomata)
[15:33:19] <elukey>	 be back in a bit
[15:55:18] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Add new dimensions to  virtual_pageview_hourly and pageview_hourly - https://phabricator.wikimedia.org/T243090 (10Nuria) There was not so I guess it is not needed.
[16:01:06] <wikibugs>	 (03CR) 10Nuria: [C: 04-1] "We need to look at this a bit more" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/577220 (https://phabricator.wikimedia.org/T246309) (owner: 10Fdans)
[16:02:47] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Virtual pageviews should set access_type to mobile if webhost is a mobile one - https://phabricator.wikimedia.org/T246309 (10Nuria) Thinking about this (ping @nshahquinn-wmf ) to do this best i think the access method should be set by the client. Can we ad...
[16:03:54] <wikibugs>	 10Analytics, 10Analytics-Kanban: Errors on wikistats UI on console - https://phabricator.wikimedia.org/T246789 (10Nuria) Excellent, thanks for the fast response
[16:08:42] <elukey>	 nuria: hola!
[16:08:45] <elukey>	 ssh an-tool1005.eqiad.wmnet -L 9080:an-tool1005.eqiad.wmnet:80
[16:08:52] <elukey>	 http://localhost:9080/superset/dashboard/73/
[16:09:32] <nuria>	 elukey: OMG
[16:09:37] <nuria>	 elukey: I SEE the future
[16:09:46] <elukey>	 ta daaaan
[16:10:54] <wikibugs>	 10Analytics, 10Core Platform Team, 10Event-Platform, 10Product-Analytics, 10CPT Initiatives (Modern Event Platform (TEC2)): Eventbus revisions are duplicated in event.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T218246 (10mpopov) >>! In T218246#5941239, @Ottomata wrote: > Hm yeah i...
[16:11:10] <elukey>	 the new version of PyHive (the one currently manually installed on an-tool1005) should be cut soon, and after that it seems that it is just a matter of re-building vanilla superset with updated packages
[16:11:44] <elukey>	 I think that we should concentrate on Presto metrics + good logging (like users + queries, etc..)
[16:12:27] <wikibugs>	 (03PS5) 10Mforns: Add dimensions to druid's pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090)
[16:12:59] <wikibugs>	 (03PS6) 10Mforns: Add dimensions to druid's pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090)
[16:13:37] <wikibugs>	 (03CR) 10Mforns: [V: 03+2] "I tested this and loaded a test datasource in druid using both the hourly job and the daily one." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns)
[16:15:29] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Add new dimensions to druid's pageview_hourly datasource - https://phabricator.wikimedia.org/T243090 (10mforns) a:03mforns
[16:17:28] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Add new dimensions to druid's pageview_hourly datasource - https://phabricator.wikimedia.org/T243090 (10mforns) @cchen @Nuria  OK then, I modified the change to just include additions to pageview_houly datasource in druid. I tested b...
[16:20:51] <ottomata>	 elukey:  yeah we neeed that asap
[16:29:26] <wikibugs>	 10Analytics, 10Core Platform Team, 10Event-Platform, 10Product-Analytics, 10CPT Initiatives (Modern Event Platform (TEC2)): Eventbus revisions are duplicated in event.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T218246 (10Ottomata) EventBus is the MW extension that registers the ho...
[16:34:34] <mforns>	 !log restart turnilo to refresh deleted datasources
[16:34:36] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[16:38:44] <elukey>	 mforns: holaaaa
[16:38:49] <elukey>	 did you see the backlog?
[16:39:00] <elukey>	 (of the chan)
[16:44:31] <mforns>	 elukey, no...
[16:45:01] <mforns>	 I was logged off during night by internet cuts, reading the channel logs
[16:47:34] <elukey>	 mforns: ssh an-tool1005.eqiad.wmnet -L 9080:an-tool1005.eqiad.wmnet:80
[16:47:39] <elukey>	 http://localhost:9080/superset/dashboard/73/
[16:48:41] <mforns>	 elukey, \\\\\\o///////
[16:48:48] <mforns>	 <3
[16:48:58] <elukey>	 :D
[16:49:29] <mforns>	 btw, bots.wmflabs.org seems down...
[16:50:02] <RhinosF1>	 mforns: its an old link
[16:50:26] <RhinosF1>	 You want http://wm-bot.wmflabs.org/browser/index.php?display=%23wikimedia-analytics
[16:50:33] <mforns>	 oh RhinosF1 thx, it's the one I found on wiki and in the channel description
[16:50:45] <mforns>	 thanks a lot RhinosF1!
[16:50:57] <wikibugs>	 10Analytics, 10Product-Analytics, 10Wikipedia-Android-App-Backlog, 10Wikipedia-iOS-App-Backlog, 10Epic: [EPIC] Count unique iOS & Android users precisely and in a privacy conscious manner that does not require opt in to send data - https://phabricator.wikimedia.org/T202664 (10SNowick_WMF) @Nuria Is this...
[16:51:09] <RhinosF1>	 I changed one of them links earlier. Someone should probably track down any broken links to that and fix
[16:51:33] <mforns>	 OK, will change the one I saw
[16:51:42] <RhinosF1>	 thx
[16:55:17] <wikibugs>	 10Analytics, 10Product-Analytics, 10Wikipedia-Android-App-Backlog, 10Wikipedia-iOS-App-Backlog, 10Epic: [EPIC] Count unique iOS & Android users precisely and in a privacy conscious manner that does not require opt in to send data - https://phabricator.wikimedia.org/T202664 (10Nuria) This ticket was never...
[16:57:23] <wikibugs>	 10Analytics, 10Growth-Team, 10Product-Analytics: Hash edit session ID in EditAttemptStep and VisualEditorFeatureUse whitelisting - https://phabricator.wikimedia.org/T244931 (10nshahquinn-wmf) @nettrom_WMF I haven't worked with EditAttemptStep in a while, but I don't think this will break anything I used to d...
[16:59:24] <RhinosF1>	 mforns: I’ve left a message in #wm-bot asking someone to look into them links
[16:59:41] <mforns>	 thanks RhinosF1 :]
[17:00:39] <RhinosF1>	 np
[17:02:07] <nuria>	 ping ottomata milimetric standduppp
[17:03:10] <nuria>	 hola ottomata  STANDUP
[17:06:07] <wikibugs>	 10Analytics, 10Tools: Pie chart is missing on the WMCS Edits dashboard - https://phabricator.wikimedia.org/T246963 (10mforns) Hi @srishakatux, I'm sorry we're getting so many issues (even if small) with this dashboard. I changed to 64 the `showLastDays` offset of the sunburst chart, so it can always show some...
[17:16:09] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Tools: Pie chart is missing on the WMCS Edits dashboard - https://phabricator.wikimedia.org/T246963 (10Nuria) a:03mforns
[17:46:51] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Event-Platform, 10serviceops: Create production and canary releases for existent eventgate helmfile services - https://phabricator.wikimedia.org/T245203 (10Ottomata)
[17:47:40] <wikibugs>	 10Analytics, 10Operations, 10User-Elukey: Refactor Analytics POSIX groups in puppet to improve maintainability - https://phabricator.wikimedia.org/T246578 (10elukey)
[17:47:42] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Unify puppet roles for stat and notebook hosts - https://phabricator.wikimedia.org/T243934 (10elukey)
[17:51:56] <J-Mo>	 elukey, fdans, @here: I tried to log into SWAP via stat1003 yesterday, and got kicked out. I saw on this channel that this was a common issue. Now, I can log in, but when I try to authenticate with Jupyter Hub via my LDAP credentials, it just hangs forever...
[17:52:02] <J-Mo>	 what am I missing?
[17:52:27] <J-Mo>	 "Now I can log in" = I can SSH directly into stat1003
[17:53:08] <elukey>	 J-Mo: hi! you mean notebook1003 right?
[17:53:18] <J-Mo>	 yes!
[17:54:04] <elukey>	 so we currently have some problems with those hosts, from the home directory space point of view (limited space) and from the resource point of view (too many people running heavy jobs)
[17:54:34] <elukey>	 we are trying to fix the space part, moving notebooks to the stat hosts (will be available soon hopefully)
[17:54:55] <elukey>	 J-Mo: what is your ldap username?
[17:55:08] <J-Mo>	 elukey jmorgan
[17:55:18] <elukey>	 J-Mo: ahhhh hello! 
[17:55:25] <J-Mo>	 ciao Luca ;)
[17:55:25] <elukey>	 now I get the nick, sorry :D
[18:00:43] <lexnasser>	 joal: Thanks for the code review! One quick question: Why would I need an hql script for creating the table in Oozie -- wouldn't I just create the table before deployment manually because it doesn't need to be created multiple times?
[18:01:50] <J-Mo>	 elukey is the host transition the reason why I can't access my Jupyter hub? Is there a stat host that I should use instead (maybe stat1007)?
[18:03:28] <ottomata>	 lexnasser: yes, but we try to keep the creaete table files as schemas to refer to in code
[18:03:39] <ottomata>	 so that they can be recreateed later, or in your own db for development, etc.
[18:03:50] <ottomata>	 and we can tell if there is some mismatch between what we want and what  is actually there
[18:03:56] <ottomata>	 (sometimes things happen)
[18:04:07] <elukey>	 J-Mo: not yet sorry :( can you retry now?
[18:04:31] <J-Mo>	 elukey it works!
[18:04:39] <joal>	 lexnasser: Heya - indeed oozie doesn't create tables, its done manually - We keep the scripts for table creation in the refinery/hive folder - We add table creation scsripts with jobs initial patches to make sure we have the necessary code in place to deploy 
[18:04:40] <J-Mo>	 anything I need to watch out for, or keep in mind?
[18:05:27] <lexnasser>	 ottomata: Got it, thanks!
[18:05:52] <joal>	 np lexnasser - thank you!
[18:28:50] <elukey>	 J-Mo: super! I just restarted your notebook, it might have been borked :)
[18:29:07] <J-Mo>	 thank you so much elukey
[18:29:48] <elukey>	 lexnasser: o/ - do you have 5 mins by any chance?
[18:30:06] <lexnasser>	 elukey: Yeah, what's up
[18:30:30] <elukey>	 lexnasser: good morning :) so I am trying to load balance people to the various stat boxes, since stat1007 is a bit crowded
[18:30:46] <elukey>	 would it be ok to move your ~100G home dir to say stat1005 ?
[18:31:07] <elukey>	 (it can be done with rsync, I'll help)
[18:32:01] <lexnasser>	 elukey: yeah, actually that's just the raw data from the caching data. was keeping it around for a bit in case there were any issues, but I think I could just delete it at this point. I'm assuming that would be sufficient?
[18:35:42] <elukey>	 lexnasser: oh yes sure! it is also good to use other stat boxes, if you want to try stat1005 I'd be glad (it runs Debian Buster so you could be a good tester for hive etc..)
[18:35:59] <lexnasser>	 elukey: just deleted the data, I should be taking up very little space now
[18:36:21] <elukey>	 thanks!
[18:36:34] <lexnasser>	 np
[19:03:53] <elukey>	 mforns: do you want to check an-launcher?
[19:05:56] <elukey>	 ottomata: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/577320/ might solve all our problems with notebooks and alarming
[19:06:28] <elukey>	 not sure if it is the right move, but if it works we are done with icinga spamming when people run a heavy notebook
[19:06:53] <ottomata>	 NICE!
[19:07:01] <ottomata>	 NICE SLICE!
[19:07:26] <elukey>	 also, systemd-cgls and systemd-cgtop are great
[19:08:08] <elukey>	 it will also work just fine when we'll move jupyterhub on stat boxes
[19:08:30] <elukey>	 any notebook running heavy will trigger the user.slice oom
[19:08:40] <elukey>	 and the system daemons will not be affected
[19:08:48] <elukey>	 since they run under the system.slice
[19:09:26] <elukey>	 probably we'll have to ask people to restart their notebooks
[19:17:39] <wikibugs>	 (03PS2) 10Lex Nasser: Configure geoeditors monthly public Oozie job to work with geoeditors public monthly table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/576618 (https://phabricator.wikimedia.org/T244597)
[19:21:46] <elukey>	 mforns: what RU jobs data is missing from datasets.w.o?
[19:21:51] <elukey>	 so I can check on an-launcher
[19:22:15] <mforns>	 heya elukey its wmcs
[19:22:46] <mforns>	 could it be rsync is still working from stat1007?
[19:22:51] <mforns>	 or not working?
[19:29:12] <elukey>	 mmmm
[19:29:27] <elukey>	 in theory it should merge stuff as they come from different places
[19:30:42] <elukey>	 so published on an-launcher has recent files
[19:30:49] <elukey>	 /srv/published/datasets/periodic/reports/metrics/wmcs
[19:31:29] <elukey>	 mforns: what is the link to check published files?
[19:31:54] <mforns>	 elukey, https://analytics.wikimedia.org/published/datasets/periodic/reports/metrics/wmcs/
[19:32:37] <mforns>	 yes, I checked that reports in an-launcher are up to date
[19:32:42] <elukey>	 /srv/published-rsynced/an-launcher1001/datasets/periodic/reports/metrics/wmcs on thorium is correct too
[19:32:47] <elukey>	 last modified is today
[19:32:52] <mforns>	 ???
[19:34:50] <elukey>	 mforns: ok so last modified by match the ones on thorium but in the stat1007 dir
[19:35:09] <elukey>	 I guess that we'd need to move the reportupdater dir in there to reportupdater-backup
[19:35:12] <elukey>	 and wait 
[19:35:49] <mforns>	 ok
[19:39:09] <elukey>	 !log mv /srv/reportupdater to /srv/reportupdater-backup05032020 on stat100[6,7]
[19:39:10] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:41:05] <elukey>	 mmmmm but what about the published dirs on the same hosts
[19:42:27] <elukey>	 ah no reports -> /srv/reportupdater/output
[19:44:57] <mforns>	 yea
[19:45:44] <elukey>	 !log deleted dangling 'reports' symlink on stat100[6,7] in /srv/published
[19:45:46] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:45:58] <elukey>	 mforns: looks better now
[19:46:03] <elukey>	 https://analytics.wikimedia.org/published/datasets/periodic/reports/metrics/wmcs/
[19:47:03] <elukey>	 IIUC the problem should be fixed
[19:47:06] <elukey>	 can you confirm?
[19:48:53] * elukey sings Marcelll Marcelll where are you Marceeeellll
[19:51:28] <mforns>	 elukey, hehehe
[19:51:55] <mforns>	 I still see old files in the endpoint
[19:52:01] <mforns>	 probably cached
[19:52:35] <mforns>	 elukey, yes, works now!
[19:52:38] <mforns>	 thanks a lot
[19:52:55] <elukey>	 niiceeee
[19:52:59] <elukey>	 \o/
[19:53:15] <elukey>	 all right going to dinner, o/
[19:53:19] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Tools: Pie chart is missing on the WMCS Edits dashboard - https://phabricator.wikimedia.org/T246963 (10mforns) OK, the problem is fixed now!
[19:54:46] <leila>	 milimetric: do you have time for a 5-min chat in video? :)
[19:54:46] <mforns>	 byeeeeee
[19:55:04] <leila>	 mforns: byeee
[19:55:56] <mforns>	 leila, :) I was saying bye to Luca
[19:56:19] <leila>	 mforns: right. I realized only after I typed. :D
[19:56:20] <leila>	 hi
[19:56:43] <mforns>	 hehe
[20:07:15] <milimetric>	 yes leila, let's do it
[20:07:40] <leila>	 milimetric: I'll send you a link privately
[20:24:14] <joal>	 nuria: where do you wish me to document sizing for historical pagecounts in cassandra?
[20:24:47] <nuria>	 joal: wikitech ?
[20:24:54] <nuria>	 joal: as in capacity planning for pageviews?
[20:26:18] <joal>	 nuria: I suggest Analytics/Systems/AQS/Historical_pagecounts_capacity_planning ?
[20:26:43] <nuria>	 Analytics/Systems/AQS/capacityplanning (and inside page we talk about historical?)
[20:26:59] <joal>	 actually I didn't notice, there is already a pattern
[20:27:27] <joal>	 actually I didn't notice, there is already a pattern: Analytics/Systems/AQS/Scaling/2020/Cluster Expansion
[20:27:30] <joal>	 ?
[20:27:31] <joal>	 nuria: --^
[20:34:15] <nuria>	 joal: perfect
[20:34:21] <joal>	 ack nuria :)
[20:34:23] <nuria>	 joal: see OUTSELVES FROM THE PAST!
[20:34:26] <joal>	 :)
[20:37:30] <joal>	 Gone for diner
[20:46:20] <wikibugs>	 10Quarry, 10Cloud-Services: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10Mike_Peel) Also under discussion at https://www.mediawiki.org/w/index.php?title=Topic:Vhw07swro9jqy4w0
[20:52:48] <wikibugs>	 (03CR) 10Nuria: "Can @ladsgroup be so kind to test this patch?" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/576449 (https://phabricator.wikimedia.org/T236895) (owner: 10Joal)
[20:57:01] <wikibugs>	 10Analytics, 10Analytics-Kanban: Import siteinfo dumps onto HDFS - https://phabricator.wikimedia.org/T234333 (10Nuria) 05Open→03Resolved
[20:57:29] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Add multilanguage ability to Wikistats - https://phabricator.wikimedia.org/T238752 (10Nuria)
[20:59:17] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Add multilanguage ability to Wikistats - https://phabricator.wikimedia.org/T238752 (10Nuria) 05Open→03Resolved
[20:59:31] <wikibugs>	 10Analytics, 10Analytics-Kanban: Specify in build command which languages to bundle - https://phabricator.wikimedia.org/T246745 (10Nuria) 05Open→03Resolved
[20:59:49] <wikibugs>	 10Analytics, 10Analytics-Kanban: Errors on wikistats UI on console - https://phabricator.wikimedia.org/T246789 (10Nuria) 05Open→03Resolved
[21:08:29] <nuria>	 joal: I think data on /wmf/data/event/wdqs_internal_sparql_query/ needs to be also deleted on schedule (or moved to sanitized if needed be)
[21:08:38] <nuria>	 joal: will make ticket
[21:12:09] <wikibugs>	 10Analytics, 10Discovery: Data for events from wdqs needs to be deleted after 90 days and/or sanitized - https://phabricator.wikimedia.org/T247034 (10Nuria)
[21:13:18] <wikibugs>	 10Analytics, 10Discovery: Data for events from wdqs needs to be deleted after 90 days and/or sanitized - https://phabricator.wikimedia.org/T247034 (10Nuria)
[21:13:34] <wikibugs>	 10Analytics, 10Discovery: Data for events from wdqs needs to be deleted after 90 days and/or sanitized - https://phabricator.wikimedia.org/T247034 (10Nuria) pinging @Gehel and @dcausse
[21:14:15] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10serviceops: Clarify multi-service instance concepts in helm charts and enable canary releases - https://phabricator.wikimedia.org/T242861 (10Nuria)
[21:14:19] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Event-Platform, 10serviceops: Create production and canary releases for existent eventgate helmfile services - https://phabricator.wikimedia.org/T245203 (10Nuria) 05Open→03Resolved
[21:15:10] <wikibugs>	 10Analytics, 10Analytics-Kanban: Prevent detail sidebar from getting too wide - https://phabricator.wikimedia.org/T246744 (10Nuria) 05Open→03Resolved
[21:18:10] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Nuria) Thanks for the fast turnaround on this fix
[21:18:26] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Nuria) 05Open→03Resolved
[21:18:43] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: https://stats.wikimedia.org/ is down since a couple of hours from the Netherlands - https://phabricator.wikimedia.org/T246976 (10Nuria) Thanks to @Ciell for the fast ping too
[21:19:11] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Better Use Of Data, and 9 others: Modern Event Platform: Stream Configuration: Implementation - https://phabricator.wikimedia.org/T233634 (10Nuria) 05Open→03Resolved
[21:19:13] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Event-Platform, 10CPT Initiatives (Modern Event Platform (TEC2)), and 2 others: Modern Event Platform: Stream Configuration - https://phabricator.wikimedia.org/T205319 (10Nuria)
[21:19:27] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10MW-1.35-notes (1.35.0-wmf.20; 2020-02-18): Change link in wikis footer so that they point to stats.wikimedia.org - https://phabricator.wikimedia.org/T244961 (10Nuria) 05Open→03Resolved
[21:19:54] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Multimedia, 10Tool-Pageviews: Make job to backfill data from mediacounts into mediarequests tables in cassandra so as to have historical mediarequest data - https://phabricator.wikimedia.org/T234591 (10Nuria) 05Open→03Resolved
[21:19:58] <wikibugs>	 10Analytics, 10Multimedia, 10Tool-Pageviews: Statistics for views of individual Wikimedia images - https://phabricator.wikimedia.org/T210313 (10Nuria)
[21:20:08] <wikibugs>	 10Analytics, 10Analytics-Kanban: Fix webrequest host normalization - https://phabricator.wikimedia.org/T245453 (10Nuria) 05Open→03Resolved
[21:20:25] <wikibugs>	 10Analytics, 10Analytics-Kanban: Hourly Feature extraction for bot detection from webrequest - https://phabricator.wikimedia.org/T238360 (10Nuria) 05Open→03Resolved
[21:20:27] <wikibugs>	 10Analytics: Deploy high volume bot spike detector to hungarian wikipedia - https://phabricator.wikimedia.org/T238358 (10Nuria)
[21:20:44] <wikibugs>	 10Analytics, 10Analytics-Kanban: Remove stats gathering for mediawiki_history production job - https://phabricator.wikimedia.org/T246748 (10Nuria) 05Open→03Resolved
[21:21:43] <wikibugs>	 10Analytics, 10Analytics-Data-Quality, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Bot field in edits_hourly dataset ignores username - https://phabricator.wikimedia.org/T244632 (10Nuria) Can @nshahquinn-wmf confirm issue is fixed?
[21:22:39] <wikibugs>	 10Analytics, 10Analytics-Kanban: Problem with Matomo page overlay - https://phabricator.wikimedia.org/T246046 (10Nuria) Closing as it is not a matomo problem
[21:23:31] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+1] "I tested the query and it works fine, you can do this change instead but it's not a big deal, it can happen later or not happen at all." (033 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/576449 (https://phabricator.wikimedia.org/T236895) (owner: 10Joal)
[21:24:42] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 6 others: Public EventGate instance and endpoint for analytics event intake: eventgate-analytics-external - https://phabricator.wikimedia.org/T233629 (10Nuria) 05Open→03Resolved
[21:24:44] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Event-Platform, 10Core Platform Team Legacy (Watching / External), 10Services (watching): Modern Event Platform: Stream Intake Service - https://phabricator.wikimedia.org/T201068 (10Nuria)
[21:25:05] <wikibugs>	 10Analytics-Kanban: Add Presto to Analytics' stack - https://phabricator.wikimedia.org/T243309 (10Nuria)
[21:25:07] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Enable shell access to presto from jupyter/stats machines - https://phabricator.wikimedia.org/T243312 (10Nuria) 05Open→03Resolved
[21:26:02] <wikibugs>	 10Analytics, 10Analytics-Kanban: Make mediawiki-history spark jobs single-attempt - https://phabricator.wikimedia.org/T246747 (10Nuria) 05Open→03Resolved
[21:26:23] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Create English strings json for vue-i18n to use - https://phabricator.wikimedia.org/T240617 (10Nuria)
[21:26:32] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Create English strings json for vue-i18n to use - https://phabricator.wikimedia.org/T240617 (10Nuria) 05Open→03Resolved
[21:26:34] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Add multilanguage ability to Wikistats - https://phabricator.wikimedia.org/T238752 (10Nuria)
[21:26:51] <wikibugs>	 (03PS2) 10Joal: Fix wikidata article-placeholder job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/576449 (https://phabricator.wikimedia.org/T236895)
[21:27:03] <wikibugs>	 (03CR) 10Joal: Fix wikidata article-placeholder job (033 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/576449 (https://phabricator.wikimedia.org/T236895) (owner: 10Joal)
[21:30:03] <wikibugs>	 10Analytics, 10Analytics-Kanban: Productionize item_page_link table - https://phabricator.wikimedia.org/T244707 (10Nuria) I think we need docs that point to all info that is available from wikidata on cluster, let's at least create the ones for this table, cc @JAllemandou
[21:32:58] <wikibugs>	 10Analytics, 10Analytics-Kanban: Productionize item_page_link table - https://phabricator.wikimedia.org/T244707 (10JAllemandou) Already done (not properly linked): https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Wikidata_item_page_link
[21:43:23] <joal>	 Gone for tonight
[22:08:21] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Release Pipeline, 10Patch-For-Review, and 2 others: Migrate EventStreams to k8s deployment pipeline - https://phabricator.wikimedia.org/T238658 (10Ottomata) FYI grafana dash here:   https://grafana.wikimedia.org/d/znIuUcsWz/eventstreams-k8s  Not totally sure what is up wi...
[22:10:45] <wikibugs>	 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): EventLogging MEP Upgrade - https://phabricator.wikimedia.org/T238544 (10Ottomata)
[22:21:02] <wikibugs>	 10Analytics, 10Analytics-Kanban: Productionize item_page_link table - https://phabricator.wikimedia.org/T244707 (10Nuria) 05Open→03Resolved
[22:24:10] <wikibugs>	 (03CR) 10Ladsgroup: Fix wikidata article-placeholder job (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/576449 (https://phabricator.wikimedia.org/T236895) (owner: 10Joal)
[22:24:34] <wikibugs>	 (03CR) 10Ladsgroup: "> Patch Set 2:" (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/576449 (https://phabricator.wikimedia.org/T236895) (owner: 10Joal)
[22:54:19] <wikibugs>	 10Analytics, 10Analytics-Wikistats: Include locale string jsons as webpack chunks so that only the required language is bundled - https://phabricator.wikimedia.org/T240618 (10Piramidion) Why is the "month" word capitalized in the translation? E. g.: the current translation says "Минулого **М**ісяця" instead of...
[23:31:26] <wikibugs>	 10Quarry, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10bd808)
[23:32:43] <wikibugs>	 10Analytics, 10Analytics-Kanban: Problem with Matomo page overlay - https://phabricator.wikimedia.org/T246046 (10Nuria) >This would make sense with the 401 (unauthorized) error on the console I think the 401 is because we are requesting  https://piwik.wikimedia.org/plugins/CoreHome/javascripts/manifest.json wh...
[23:34:44] <wikibugs>	 10Quarry, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10bd808) @zhuyifei1999 I see that we are wheel warring on the tags here. The datasource may be Wiki Replicas, but the runtime is Quarry. The error is either related to the query killer o...
[23:48:35] <wikibugs>	 10Quarry, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10zhuyifei1999) This isn't just a quarry issue: ([[https://wm-bot.wmflabs.org/browser/index.php?start=03%2F05%2F2020&end=03%2F05%2F2020&display=%23wikimedia-cloud|logs]])  ` 2020-03-05 1...