[00:47:18] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10SDC General, 10Wikidata: Create reportupdater reports that execute SDC requests - https://phabricator.wikimedia.org/T239565 (10Nuria) Let's pause this work as it turns out as there is a parallel effort happening , @Abit to create a ticket for ongoi... [02:38:25] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10SDC General, 10Wikidata: Create reportupdater reports that execute SDC requests - https://phabricator.wikimedia.org/T239565 (10Nuria) It looks like we are going to have to report this number on the tunning session so taking back my comment above, le... [04:30:01] (03CR) 10Milimetric: "Oh yeah, this is to prevent errors in maintenance going forward. It's handy to have the correct usage example in the comments (one of the" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/554915 (owner: 10Milimetric) [04:42:19] 10Analytics, 10Wikimedia Design Style Guide: Analytics: Some pages/page requests are not reflected in statistics - https://phabricator.wikimedia.org/T239685 (10Milimetric) {F31460116} I'm not exactly sure what's going on but I only see the piwik script on the root of the site. Looking at the piwik report for... [05:00:45] 10Analytics, 10Analytics-Kanban, 10Cloud-VPS (Debian Jessie Deprecation): "dashiki" Cloud VPS project jessie deprecation - https://phabricator.wikimedia.org/T236586 (10Milimetric) [05:16:04] 10Analytics, 10Analytics-Kanban, 10Cloud-VPS (Debian Jessie Deprecation): "dashiki" Cloud VPS project jessie deprecation - https://phabricator.wikimedia.org/T236586 (10Milimetric) I'll take care of this either tomorrow or early next week. I was going to do it tonight but it's rejecting my ssh connect, proba... [05:38:10] 10Analytics, 10Wikimedia Design Style Guide: Analytics: Some pages/page requests are not reflected in statistics - https://phabricator.wikimedia.org/T239685 (10Nuria) Given that other websites have no issue tracking subpages this is probably something about the way piwik is setup in your site, you are doing so... [07:12:34] 10Analytics, 10User-Elukey: Port IRCRecentChanges to Kafka - https://phabricator.wikimedia.org/T232483 (10elukey) [07:13:07] 10Analytics, 10Code-Stewardship-Reviews, 10Operations, 10Tools, 10Wikimedia-IRC-RC-Server: IRC RecentChanges feed: code stewardship request - https://phabricator.wikimedia.org/T185319 (10elukey) >>! In T185319#5716701, @Dzahn wrote: > Is this really replacing the IRCd from T134271 ? Yep! Closed it as du... [10:33:57] elukey: Hey! Do you know if we have a good way to transfer large files from production to the analytics cluster? [10:34:07] elukey: context: https://phabricator.wikimedia.org/T239898#5717656 [10:35:49] gehel: hey! do you mean to HDFS or to a stat machine? [10:36:12] dcausse: ^ [10:36:28] running the process that reads the 500Gb file it'll have to be a stat machine [10:36:32] My guess is we want the the initial file on a stat machine [10:36:48] so stat1004 is your best target [10:36:52] the output will be then copied in hdfs [10:37:07] /dev/mapper/stat1004--vg-data 7.2T 1.8T 5.1T 26% /srv [10:37:13] nice [10:37:19] please don't use all the 5T :D [10:37:24] nah :) [10:37:46] elukey: and how do you transfer to stat1004? It's all firewalled? [10:38:36] gehel: it is, you can open a temp hole in one high port and use something like netcat in my opinion [10:39:22] the only FW is ferm? I thought we had some more isolation around the analytics network [10:39:46] gehel: ah no no you mean the firewall on the routers, that is for traffic going from analytics to production [10:39:58] ferm is the one filtering for traffic the other way around [10:40:12] Oh, then that's probably easy. [10:40:37] yep should be easy, if you guys need rsync support it should be easy to add some rules on stat1004 [10:40:48] otherwise if netcat etc.. is fine please go ahead [10:41:05] nope, it's a one off, some pigz / netcat should be fine [10:41:24] we already have a similar process in place to copy those files between wdqs nodes [10:41:30] super [10:41:35] elukey: thanks! [10:41:36] there are no quota that could kill the tranfert at 99%? :) [10:41:49] we'll see in a few hours :) [10:41:58] dcausse: none yet :) [10:42:08] ok [10:42:14] for some reason we have problems on stat1007 because people cluster only on that host [10:42:23] problems with home space [12:04:24] * elukey lunch + errand! [14:19:43] 10Analytics, 10Research: Taxonomy of new user reading patterns - https://phabricator.wikimedia.org/T234188 (10MGerlach) 05Open→03Resolved [14:56:32] 10Analytics, 10Core Platform Team: Update mediawiki-history to use new Multi-Content-Revision tables - https://phabricator.wikimedia.org/T239591 (10Anomie) >>! In T239591#5704568, @JAllemandou wrote: > The text-related fields (`text_id`, `text_len`) are planned to be deleted from the `revision`table, and acces... [14:57:08] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10mpopov) [15:01:28] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10mpopov) [15:04:54] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10mpopov) [15:05:39] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10mpopov) @Nuria: would love to get your thoughts on this [15:07:24] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10mpopov) [15:30:46] 10Analytics, 10Discovery-Search, 10Product-Analytics: Reportupdater cant run on stat1007 because "No module named pymysql" - https://phabricator.wikimedia.org/T240002 (10mpopov) p:05Triage→03High [15:46:05] (03PS1) 10Elukey: update_reports.py: move shabang to python3 [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/555518 (https://phabricator.wikimedia.org/T240002) [15:47:15] (03CR) 10Bearloga: [C: 03+1] update_reports.py: move shabang to python3 [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/555518 (https://phabricator.wikimedia.org/T240002) (owner: 10Elukey) [15:47:21] (03CR) 10Elukey: [C: 03+2] update_reports.py: move shabang to python3 [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/555518 (https://phabricator.wikimedia.org/T240002) (owner: 10Elukey) [15:52:53] 10Analytics: Check home leftovers of maxsem - https://phabricator.wikimedia.org/T239047 (10MaxSem) Everything can be safely deleted. [16:07:56] 10Analytics, 10Discovery-Search, 10Product-Analytics, 10Patch-For-Review: Reportupdater cant run on stat1007 because "No module named pymysql" - https://phabricator.wikimedia.org/T240002 (10mpopov) 05Open→03Resolved And the backfill started! ` bearloga@stat1007:/srv/discovery/golden$ sudo -u analytics... [16:08:33] 10Analytics, 10Discovery-Search, 10Product-Analytics: Reportupdater cant run on stat1007 because "No module named pymysql" - https://phabricator.wikimedia.org/T240002 (10mpopov) [16:54:06] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10Nuria) @mpopov there are two things here: - send events in batches for better use of radio in mobile, that already... [17:07:41] 10Analytics, 10Better Use Of Data, 10Performance-Team, 10Product-Infrastructure-Team-Backlog, 10Product-Analytics (Kanban): Switch mw.user.sessionId back to session-cookie persistence - https://phabricator.wikimedia.org/T223931 (10jlinehan) Hey @Nuria, @Krinkle, I think it will be simple to patch this, m... [17:17:22] 10Analytics, 10Better Use Of Data, 10Performance-Team, 10Product-Infrastructure-Team-Backlog, 10Product-Analytics (Kanban): Switch mw.user.sessionId back to session-cookie persistence - https://phabricator.wikimedia.org/T223931 (10Nuria) @jlinehan We have too many balls up in the air now that need code r... [18:17:29] 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: Superset Updates - https://phabricator.wikimedia.org/T211706 (10kzimmerman) [18:17:33] 10Analytics, 10Better Use Of Data, 10Core Platform Team, 10MediaWiki-API, and 2 others: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10mpopov) [18:52:37] 10Analytics, 10Analytics-Kanban: Create kerberos principals for users - https://phabricator.wikimedia.org/T237605 (10jkumalah) Hi! I would also like to request Kerberos credentials for stat100x and notebook100x machines. My username is jkumalah. [19:00:35] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Further improvements to the WMCS edits dashboard - https://phabricator.wikimedia.org/T240040 (10srishakatux) [19:02:08] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Further improvements to the WMCS edits dashboard - https://phabricator.wikimedia.org/T240040 (10srishakatux) p:05Triage→03High [19:05:29] (03PS1) 10Conniecc1: Add two columns and update one column to edit_hourly table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) [19:05:31] (03CR) 10Welcome, new contributor!: "Thank you for making your first contribution to Wikimedia! :) To learn how to get your code changes reviewed faster and more likely to get" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) (owner: 10Conniecc1) [19:06:44] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Create a WMCS edits dashboard via Dashiki - https://phabricator.wikimedia.org/T226663 (10srishakatux) [19:06:46] 10Analytics, 10Analytics-Kanban, 10Research: Improve quality of external referer data - https://phabricator.wikimedia.org/T239625 (10Nuria) a:03lexnasser [19:43:56] 10Analytics, 10Analytics-Kanban: Create kerberos principals for users - https://phabricator.wikimedia.org/T237605 (10Mstyles) I too am requesting Kerberos credentials for the stat and notebook machines. My username is mstyles [19:54:09] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Sort tabular view data by wmcs_percent - https://phabricator.wikimedia.org/T240044 (10srishakatux) [19:54:17] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Sort tabular view data by wmcs_percent - https://phabricator.wikimedia.org/T240044 (10srishakatux) p:05Triage→03High [20:30:16] (03PS1) 10Srishakatux: Sort tabular view data by wmcs percent [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/555604 (https://phabricator.wikimedia.org/T240044) [20:37:43] 10Analytics, 10Analytics-Dashiki: Allow sorting data in the Tabular View by multiple columns - https://phabricator.wikimedia.org/T240049 (10srishakatux) [20:39:37] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019), 10Patch-For-Review: Sort tabular view data by wmcs_percent - https://phabricator.wikimedia.org/T240044 (10srishakatux) [21:01:56] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10SDC General, 10Wikidata: Create reportupdater reports that execute SDC requests - https://phabricator.wikimedia.org/T239565 (10Abit) @Nuria where will this number be reported in the tuning session? [21:16:08] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10SDC General, 10Wikidata: Create reportupdater reports that execute SDC requests - https://phabricator.wikimedia.org/T239565 (10Nuria) @abit: Numbers about SDC will be reported in the platform evolution slides. [21:36:52] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10mpopov) @Nuria: By batching I meant an array of events sent in a single request as one payload – as one //batch// o... [22:11:28] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10Nuria) >In some situations, there might be a benefit of POSTing 1 big request containing an array (batch) of, say,...