[00:16:12] 10Analytics-Radar, 10Dumps-Generation, 10Okapi, 10Platform Engineering: HTML Dumps - June/2020 - https://phabricator.wikimedia.org/T254275 (10Legoktm) >>! In T254275#6355528, @ArielGlenn wrote: > Note that the code is now available on line at https://github.com/wikimedia/OKAPI I can't find any license sta... [00:49:52] PROBLEM - Check the last execution of monitor_refine_eventlogging_legacy_failure_flags on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_eventlogging_legacy_failure_flags https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [07:35:22] RECOVERY - Check the last execution of monitor_refine_eventlogging_legacy_failure_flags on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_eventlogging_legacy_failure_flags https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [08:27:19] fdans: o/ [08:27:27] if you are around we can merge the puppet jar bump [08:27:28] hellooo [08:27:35] elukey: yea sounds good! [08:27:44] do you want to file the code changE? [08:30:45] elukey: oh i thought i had already [08:31:06] but I'm getting this, do you know what's it about? [08:31:18] https://usercontent.irccloud-cdn.com/file/0fZxky42/%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%BC%E3%83%B3%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88%202020-08-14%20%E5%8D%88%E5%89%8D10.30.55.png [08:35:32] 10Analytics-Clusters: Upgrade the Hadoop Analytics cluster to BigTop - https://phabricator.wikimedia.org/T255142 (10elukey) p:05Triage→03Medium a:03elukey [08:36:11] fdans: weird, never happened to me [08:36:26] are you sure that the diff is ok? [08:36:33] also, what command are you using? [08:36:54] git push origin HEAD:refs/for/production [08:45:38] I'm re-cloning puppet [08:48:39] very strange [08:54:28] 10Analytics: Establish what data must be backed up before the HDFS upgrade - https://phabricator.wikimedia.org/T260409 (10elukey) [08:57:16] 10Analytics-Clusters: Create a temporary hadoop backup cluster - https://phabricator.wikimedia.org/T260411 (10elukey) [09:00:52] 10Analytics-Clusters, 10Research: Can we have more RAM on stat machines - https://phabricator.wikimedia.org/T255716 (10elukey) 05Open→03Resolved a:03elukey Closing this task since we reached a consensus about what to buy in T256017. [09:05:49] elukey: it seems now it went fine :) [09:06:06] elukey: https://gerrit.wikimedia.org/r/c/operations/puppet/+/620293 [09:08:47] elukey: how dare you modify my precious patch [09:09:32] fdans: I am making the commit msg more explicit for other sres, merging in a sec :) [09:10:10] elukey: I know I know, I always mess up the commit msgs in puppet [09:10:21] thank you for doing that, sorry for the extra work [09:12:20] np! I am also not 100% sure all the times what is the preferred format [09:12:50] fdans: can you !log in there (with the task id) that we are changing the refine jar ? [09:12:59] so we'll be able to track it if needed [09:13:06] I am running puppet now [09:13:36] !log restarting refine to apply T257860 [09:13:39] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:13:39] T257860: Update PageviewDefinition to only include /api/rest_v1/page/mobile-html requests with X-Analytics: pageview=1 in pageviews - https://phabricator.wikimedia.org/T257860 [09:13:46] elukey: 👍🏻 [09:16:54] 10Analytics-Radar, 10Cite, 10Reference Previews, 10Patch-For-Review, 10User-awight: Instrument Cite to record the nubmer of footnote marks and references list entries rendered in each article - https://phabricator.wikimedia.org/T241833 (10awight) 05Open→03Declined As the author, I think this task is... [09:17:08] done! [09:40:54] 10Analytics-Clusters: Create the new Hadoop test cluster - https://phabricator.wikimedia.org/T255139 (10elukey) [09:43:40] 10Analytics-Clusters: Create the new Hadoop test cluster - https://phabricator.wikimedia.org/T255139 (10elukey) [09:43:42] 10Analytics-Clusters, 10Operations, 10decommission-hardware, 10ops-eqiad: Decommission analytics10[28-31,33-41] - https://phabricator.wikimedia.org/T227485 (10elukey) [09:48:01] 10Analytics-Clusters: Create the new Hadoop test cluster - https://phabricator.wikimedia.org/T255139 (10elukey) p:05Triage→03Medium [10:02:32] fdans: when you have a moment, can you review https://phabricator.wikimedia.org/T259438#6375608 ? [10:02:46] Manuel pinged me about the task but I am not sure if it is important or not for us [10:05:28] elukey: I'm not sure if we would have to change the labsdb host in our config for sqooping? [10:05:42] not familiar enough with this [10:06:57] elukey: doesn't seem like we would be affected judging by looking at puppet [10:19:38] fdans: thanks! [10:19:52] in theory we read the mapping between sections and dbs from mwconfig [10:20:13] so anything that they change should be safe for us, as long as we have a fresh copy of mwconfig [10:27:38] * elukey lunch! [11:14:49] (03PS1) 10Awight: Filter out two different bugs [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620313 (https://phabricator.wikimedia.org/T246440) [11:14:52] (03PS1) 10Awight: [WIP] Report on the conflictless conflict [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620314 (https://phabricator.wikimedia.org/T246440) [11:22:52] 10Analytics-Radar, 10Dumps-Generation, 10Okapi, 10Platform Engineering: HTML Dumps - June/2020 - https://phabricator.wikimedia.org/T254275 (10Reedy) >>! In T254275#6384557, @Legoktm wrote: >>>! In T254275#6355528, @ArielGlenn wrote: >> Note that the code is now available on line at https://github.com/wikim... [11:35:39] (03PS2) 10Awight: [WIP] Refresh "new article" conflicts for 2020 [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620048 (https://phabricator.wikimedia.org/T246439) [11:35:41] (03PS2) 10Awight: Filter out two different bugs [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620313 (https://phabricator.wikimedia.org/T246440) [11:35:44] (03PS2) 10Awight: [WIP] Report on the conflictless conflict [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620314 (https://phabricator.wikimedia.org/T246440) [12:25:28] (03PS3) 10Awight: [WIP] Report on the conflictless conflict [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620314 (https://phabricator.wikimedia.org/T246440) [13:17:13] (03PS2) 10Awight: Sample more of 2020 [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620000 (https://phabricator.wikimedia.org/T246439) [13:17:15] (03PS3) 10Awight: [WIP] Refresh conflict bug reports [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620048 (https://phabricator.wikimedia.org/T246439) [13:17:42] (03Abandoned) 10Awight: [WIP] Refresh conflict bug reports [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620048 (https://phabricator.wikimedia.org/T246439) (owner: 10Awight) [13:17:54] (03Abandoned) 10Awight: [WIP] Report on the conflictless conflict [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/620314 (https://phabricator.wikimedia.org/T246440) (owner: 10Awight) [14:33:52] ottomata: that was a fast merge! [14:34:42] let me know if there is anything unclear or surprising in that new parent pom. There should be a reasonably good documentation at https://github.com/wikimedia/wikimedia-discovery-discovery-parent-pom [15:10:57] a-team: I am going into vacation mode soon, lemme know if there is anything that you need from me :) [15:12:24] Enjoy elukey (not a team) [15:12:57] thanks :) [15:14:35] elukey: no problem, I think I'll be needing one if I hear "the build is still failing" in response to a github commit once more today [15:50:46] 10Analytics-Clusters: AMD ROCm kernel drivers on stat1005/stat1008 don't support some features - https://phabricator.wikimedia.org/T260442 (10elukey) [15:51:29] 10Analytics-Clusters: AMD ROCm kernel drivers on stat1005/stat1008 don't support some features - https://phabricator.wikimedia.org/T260442 (10elukey) [15:52:44] the joy of the GPUs --% [15:52:46] --^ [15:54:49] logging off, see you in a week! [15:54:50] o/ [16:12:31] elukey: BYEEE <3 have a good vaca! [16:14:02] bye elukey !!! [16:27:22] 10Analytics, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-40] - https://phabricator.wikimedia.org/T260445 (10RobH) [16:27:44] 10Analytics, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-40] - https://phabricator.wikimedia.org/T260445 (10RobH) [16:40:23] 10Analytics, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: 2020-10-31) upgrade/replace memory in stat100[58] - https://phabricator.wikimedia.org/T260448 (10RobH) [16:40:33] 10Analytics, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: 2020-10-31) upgrade/replace memory in stat100[58] - https://phabricator.wikimedia.org/T260448 (10RobH) [18:38:42] (03CR) 10Mforns: [V: 03+2] "OK, this seems to work!" [analytics/aqs] - 10https://gerrit.wikimedia.org/r/593660 (https://phabricator.wikimedia.org/T238365) (owner: 10Lex Nasser) [18:46:52] 10Analytics, 10Analytics-Kanban: Configure Oozie job for loading geoeditors data into Cassandra - https://phabricator.wikimedia.org/T248289 (10mforns) When working on T238365 I discovered an error in this change: The JSON blob in the results should use 'editors-ceil' instead of 'editors_ceil'. Will change in a... [18:48:48] (03PS1) 10Mforns: Fix typo in geoeditors cassandra loader [analytics/refinery] - 10https://gerrit.wikimedia.org/r/620385 (https://phabricator.wikimedia.org/T248289) [18:49:29] (03CR) 10Mforns: [V: 03+2] Fix typo in geoeditors cassandra loader [analytics/refinery] - 10https://gerrit.wikimedia.org/r/620385 (https://phabricator.wikimedia.org/T248289) (owner: 10Mforns) [19:27:57] 10Analytics, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: 2020-09-15) upgrade/replace memory in stat100[58] - https://phabricator.wikimedia.org/T260448 (10elukey) [20:48:03] (03CR) 10Nuria: [C: 03+1] Fix typo in geoeditors cassandra loader [analytics/refinery] - 10https://gerrit.wikimedia.org/r/620385 (https://phabricator.wikimedia.org/T248289) (owner: 10Mforns) [20:48:09] (03CR) 10Nuria: [C: 03+2] Fix typo in geoeditors cassandra loader [analytics/refinery] - 10https://gerrit.wikimedia.org/r/620385 (https://phabricator.wikimedia.org/T248289) (owner: 10Mforns)