[00:17:29] Looking at the article, it looks like the President let out a fart and called it a government policy. Probably won't mean anything for now. [01:16:24] Hmm something looks wrong with the itwiki goodfaith model. It's almost like true and false are reversed [01:17:53] The goodfaith=true graph looks like what I'd expect the "false" graph to look like, and vice versa https://usercontent.irccloud-cdn.com/file/UC5eFyRX/itwiki%20goodfaith%20graphs [01:18:48] The vast majority of edits are good faith, so you'd expect the precision line (light blue) in the "true" graph to start at a high percentage and rise slowly, and for it to clib from near 0 in the false graph. That's how other wikis behave, but this one is reversed [01:32:59] lol whoops that is the wrong screenshot, it's the nlwiki graph [01:33:11] I'll write about this in more detail on the itwiki task [01:36:00] 10Scoring-platform-team, 10Edit-Review-Improvements-Integrated-Filters, 10editquality-modeling, 10Growth-Team (Current Sprint), and 2 others: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Catrope) I've scheduled the above patch for deployment on Thursday Feb... [12:22:41] 10Scoring-platform-team, 10Wikilabels, 10editquality-modeling, 10User-revi, 10artificial-intelligence: Complete editquality campaign for Korean Wikipedia - https://phabricator.wikimedia.org/T161627 (10revi) @ladsgroup and @halfak: it's complete. 50 remainings are 'revision not found' when I retrieved the... [14:43:27] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Re-enable eslint test for ORES CI - https://phabricator.wikimedia.org/T215906 (10Halfak) [14:45:18] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Halfak) Is this specific to ORES or is it general to anyone who is loading this type of data into hadoop via event bus? Ping @Ottomata.... [14:47:50] 10Scoring-platform-team, 10Wikilabels, 10editquality-modeling, 10User-revi, 10artificial-intelligence: Complete editquality campaign for Korean Wikipedia - https://phabricator.wikimedia.org/T161627 (10Halfak) Great news! Thanks for your work on this. I'll get you in the queue for a new set of models. :) [14:48:15] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10User-revi, 10artificial-intelligence: Complete editquality campaign for Korean Wikipedia - https://phabricator.wikimedia.org/T161627 (10Halfak) [14:48:39] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train/test damaging and goodfaith models for kowiki - https://phabricator.wikimedia.org/T161629 (10Halfak) [14:49:16] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train/test damaging and goodfaith models for kowiki - https://phabricator.wikimedia.org/T161629 (10Halfak) @Ladsgroup feel free to pick this up if you like, but let me know here. Otherwise, I'll pick it up at my first oppo... [14:49:23] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train/test damaging and goodfaith models for kowiki - https://phabricator.wikimedia.org/T161629 (10Halfak) p:05Low→03High [15:06:30] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Ottomata) Ya not specific to ORES. This would affect anyone trying to schedule Oozie jobs on data imported periodically from Kafka. @P... [15:10:51] 10ORES, 10Scoring-platform-team, 10Scap: Investigate: why are we getting overload errors during ORES deployments? - https://phabricator.wikimedia.org/T213116 (10akosiaris) >>! In T213116#4943317, @Ladsgroup wrote: > Thanks for the blogpost @akosiaris I had a complete misunderstanding of what blue/green deplo... [15:48:04] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Pchelolo) > ORES change propagation is configured to mostly go through one datacenter Which of the ORES-related CP rules are you referrin... [15:56:03] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review: Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10Pchelolo) There's one more issue that's not being addressed here. The precaching config does not have all the models available for the wiki, s... [15:58:28] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review, 10Services (watching): Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10Pchelolo) [16:52:51] 10Scoring-platform-team, 10Diffusion, 10GitHub-Mirrors, 10Release-Engineering-Team: LFS objects are not mirroring from Github through Phab to Gerrit consistently - https://phabricator.wikimedia.org/T212818 (10Halfak) 05Resolved→03Open I ran into this issue again. I needed to do @Ladsgroup's work aroun... [18:00:58] Heading out to lunch [18:01:10] (and shoveling) [18:17:09] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Ottomata) It isn't different from any other event. The issue is that Adam is building an Oozie job that gets launched based on full exist... [18:20:33] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Pchelolo) So in this case we have all the events in eqiad, and some rogue events in codfw. Is it possible to consume from 2 topics in the... [18:20:36] Can’t tell if you got a new round of snow or if this is the same accumulation from last time [18:27:58] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Ottomata) Ah, no sorry, let me try to be clearer. The problem here is that there are NO events in codfw, but the job needs to handle the... [18:31:00] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Pchelolo) > If one of the topics in a stream has no data for a given hour, we can't tell if that is because of an upstream (producer relat... [18:36:53] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review, 10Services (watching): Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10Pchelolo) I'm trying to get alternative solutions because I would honestly hate emitting a heartbeat event. [18:44:52] halfak|Lunch: harej do we have topics for this ORES meeting or should we cancel for the week? Has anyone seen awight? [19:00:12] srrodlund, I think awight is still OoO today. [19:00:15] No topics from me [19:00:20] Sorry to be AFK until now [19:00:24] Okay I will cancel [19:03:09] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Ottomata) > < previous hour Maybe, we'd have to find some way to do that in Oozie, but it might be possible. Even if it is, this would be... [19:10:08] 10ORES, 10Scoring-platform-team, 10Scap: Investigate: why are we getting overload errors during ORES deployments? - https://phabricator.wikimedia.org/T213116 (10Halfak) I think the goal is to make sure that we don't end up restarting the cluster all at once. We want to retain some amount of capacity during... [19:10:31] harej, just saw your comment. This is a new round. In fact, it's currently falling. [19:11:26] I didn't make it outside to shovel after all. I used some of my lunch time to work on a workshop paper about ORES instead. [19:21:59] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Pchelolo) Hmm.... Just brainstorming here... This doesn't seem solvable even with the artificial event too. If we emit the artificial ev... [19:30:09] 10Scoring-platform-team, 10Edit-Review-Improvements-Integrated-Filters, 10editquality-modeling, 10Growth-Team (Current Sprint), and 2 others: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Halfak) **I did some analysis to figure out that it is configuration i... [19:37:48] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Ottomata) > If we emit the artificial event via the same machinery and we emit normal events (which we should do) it will get delayed the... [19:40:24] wikimedia/editquality#479 (fix_itwiki_population - 0157c61 : halfak): The build passed. https://travis-ci.org/wikimedia/editquality/builds/492337753 [20:09:53] 10Scoring-platform-team, 10Edit-Review-Improvements-Integrated-Filters, 10editquality-modeling, 10Growth-Team (Current Sprint), and 2 others: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Halfak) https://github.com/wikimedia/editquality/pull/187 [20:10:33] 10Scoring-platform-team, 10Edit-Review-Improvements-Integrated-Filters, 10editquality-modeling, 10Growth-Team (Current Sprint), and 2 others: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Halfak) @Catrope, we can probably get this out in the next deployment... [20:10:49] 10Scoring-platform-team (Current), 10Edit-Review-Improvements-Integrated-Filters, 10editquality-modeling, 10Growth-Team (Current Sprint), and 2 others: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Halfak) [20:43:49] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review, 10Services (watching): Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10Nuria) >I'm trying to get alternative solutions because I would honestly hate emitting a heartbeat event. I gotta say... [20:44:50] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review, 10Services (watching): Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10Ottomata) Generally agree with both of you. [20:45:13] 10ORES, 10Scoring-platform-team (Current), 10Analytics: Emit synthetic mediawiki.revision-score events for both datacenters - https://phabricator.wikimedia.org/T214545 (10Nuria) Let's spend a bit of time trying to see if we can fix this issue in the consumer end which where the problem is, rather than tryin...