[00:17:08] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Grafana, icinga, prometheus in cloud-analytics project - https://phabricator.wikimedia.org/T211640 (10Nuria) Looks like this ticket can be closed, from our meeting after reviewing the state of monitoring in labs we think it would be best to run presto in p... [00:17:39] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Presto cluster online and usable with test data pushed from analytics prod infrastructure accessible by Cloud (labs) users - https://phabricator.wikimedia.org/T204951 (10Nuria) [00:17:45] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Grafana, icinga, prometheus in cloud-analytics project - https://phabricator.wikimedia.org/T211640 (10Nuria) 05Open→03Declined [00:59:41] 10Analytics, 10EventBus, 10WMF-JobQueue, 10Core Platform Team (Security, stability, performance and scalability (TEC1)), and 3 others: EventBus error "Unable to deliver all events: (curl error: 28) Timeout was reached" - https://phabricator.wikimedia.org/T204183 (10Krinkle) The wmf.14 branch [has gone out]... [00:59:45] 10Analytics, 10EventBus, 10WMF-JobQueue, 10Core Platform Team (Security, stability, performance and scalability (TEC1)), and 2 others: EventBus error "Unable to deliver all events: (curl error: 28) Timeout was reached" - https://phabricator.wikimedia.org/T204183 (10Krinkle) [03:08:00] 10Analytics, 10Operations, 10ops-eqiad: Rack A2's hosts alarm for PSU broken - https://phabricator.wikimedia.org/T212861 (10Marostegui) [03:08:08] 10Analytics, 10DBA, 10Operations, 10ops-eqiad, 10Patch-For-Review: swap a2-eqiad PDU with on-site spare - https://phabricator.wikimedia.org/T213748 (10Marostegui) 05Open→03Resolved I believe there is nothing else pending here, and this was re-opened just to get an answer from Chris, which was done. G... [03:24:04] 10Analytics, 10Analytics-Kanban, 10Operations, 10Product-Analytics: dbstore1002 Mysql errors - https://phabricator.wikimedia.org/T213670 (10Marostegui) s3 thread broke with: ` Last_SQL_Error: Error 'Got error 22 "Invalid argument" from storage engine TokuDB' on query. Default database: 'incu... [04:00:02] 10Analytics, 10Analytics-Kanban, 10Operations, 10Product-Analytics, 10Patch-For-Review: dbstore1002 Mysql errors - https://phabricator.wikimedia.org/T213670 (10Marostegui) All the tables on `incubatorwiki` are now InnoDB and replication is catching up. [06:30:01] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Core Platform Team (Modern Event Platform (TEC2)), and 3 others: Modern Event Platform (TEC2) - https://phabricator.wikimedia.org/T185233 (10Krinkle) [06:30:05] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Operations, and 3 others: RFC: Modern Event Platform - Choose Schema Tech - https://phabricator.wikimedia.org/T198256 (10Krinkle) 05Open→03Resolved a:03Krinkle [08:04:16] 10Analytics, 10Research, 10WMDE-Analytics-Engineering, 10User-Addshore, 10User-Elukey: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10elukey) >>! In T212386#4910158, @Nuria wrote: > Now, the sql running will be don... [08:06:52] 10Analytics, 10Operations, 10User-Elukey: notebook/stat server(s) running out of memory - https://phabricator.wikimedia.org/T212824 (10elukey) Thanks a lot for all the work :) I am leaning towards having a cgroup that limits CPU/memory usage of all processes to something like 80/85%, so hopefully this will... [08:07:05] 10Analytics, 10Operations, 10User-Elukey: notebook/stat server(s) running out of memory - https://phabricator.wikimedia.org/T212824 (10elukey) p:05Normal→03High a:03elukey [08:30:13] 10Analytics: Clean up home dirs for users jamesur and nithum - https://phabricator.wikimedia.org/T212127 (10elukey) We got an answer back, green light to free nithum's data. [16:17:01] 10Analytics, 10Core Platform Team, 10EventBus, 10Parsoid, and 5 others: How to surface link changes as a stream? - https://phabricator.wikimedia.org/T214706 (10Lydia_Pintscher) >>! In T214706#4909780, @bmansurov wrote: > @Pchelolo yes, replacing "properties" with "links" should do it. I don't think we need... [17:55:47] 10Analytics, 10Analytics-Wikistats, 10I18n, 10Patch-For-Review, 10good first bug: WikiReportsLocalizations.pm still fetches language names from SVN - https://phabricator.wikimedia.org/T64570 (10Krinkle) Has this been deployed and/or disabled? [20:33:50] (03PS5) 10Framawiki: vagrant: fix hostname config and enable services [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/486390 [20:34:27] (03CR) 10Framawiki: vagrant: fix hostname config and enable services (031 comment) [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/486390 (owner: 10Framawiki) [22:01:38] (03CR) 10Krinkle: "check experimental" [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/486390 (owner: 10Framawiki)