[05:12:03] PROBLEM - check disk on ORES-web02.Experimental is WARNING: DISK WARNING - free space: / 1080 MB (5% inode=92%); [06:26:03] RECOVERY - check disk on ORES-web02.Experimental is OK: DISK OK [09:48:30] o/ [10:08:09] 10ORES, 10Scoring-platform-team (Current), 10Performance, 10User-Ladsgroup: ores sends duplicate page text over the wire - https://phabricator.wikimedia.org/T214841 (10Ladsgroup) Actually the problem is `_extract_many()` returns only errors in case of error and nothing otherwise: https://github.com/wikimed... [10:52:35] 10ORES, 10Scoring-platform-team (Current), 10Performance, 10User-Ladsgroup: ores sends duplicate page text over the wire - https://phabricator.wikimedia.org/T214841 (10Ladsgroup) https://github.com/wikimedia/ores/pull/319 [10:56:38] wikimedia/ores#1295 (T214841 - dcf2117 : Amir Sarabadani): The build failed. https://travis-ci.org/wikimedia/ores/builds/498097693 [11:59:39] wikimedia/ores#1299 (T214841 - 5a85bf3 : Amir Sarabadani): The build was fixed. https://travis-ci.org/wikimedia/ores/builds/498124447 [13:12:25] 10ORES, 10Scoring-platform-team: Implement sentinel for ORES production Redis - https://phabricator.wikimedia.org/T122676 (10Ladsgroup) >>! In T122676#4976469, @Halfak wrote: > Fair questions. I'd been thinking about this in terms of //avoiding a SPOF//, but your availability % question is reasonable. If our... [13:25:44] 10ORES, 10Scoring-platform-team: Implement sentinel for ORES production Redis - https://phabricator.wikimedia.org/T122676 (10Ladsgroup) >>! In T122676#4974717, @Joe wrote: > Personally speaking, I think the real blocker is proper resourcing for doing this work, which isn't something we can do as a side job. Us... [13:29:45] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Create new editquality labeling campaign for hewiki - https://phabricator.wikimedia.org/T215363 (10Ladsgroup) a:03Ladsgroup I'm doing this. [14:08:41] 10ORES, 10Scoring-platform-team: Implement sentinel for ORES production Redis - https://phabricator.wikimedia.org/T122676 (10akosiaris) >> Also, I'd like someone to answer the questions I asked in https://phabricator.wikimedia.org/T122676#4796953 >>> What it means, from your point of view, that redis is "avail... [14:09:15] o/ [14:25:43] 10ORES, 10Scoring-platform-team: Implement sentinel for ORES production Redis - https://phabricator.wikimedia.org/T122676 (10akosiaris) >>! In T122676#4980750, @Ladsgroup wrote: >>>! In T122676#4976469, @Halfak wrote: >> Fair questions. I'd been thinking about this in terms of //avoiding a SPOF//, but your av... [14:35:10] o/ [14:35:17] hows it going halfak / [14:35:48] Not too bad. Just getting to gear. How are you, Zppix [14:35:50] ? [14:36:24] halfak: good trying to figure out why my IRCbot hates me and can't *seem* to keep an connnection for weeks on end like it used to [14:54:48] Maybe just needs some fault tolerance. Can it auto-reconnect? [14:56:43] halfak: it does but it shouldnt be disconnecting in the first place [15:05:50] halfak: I'm coming up with come way to handle several review samples in the automated make file [15:06:06] it's not super complex [15:06:21] Amir1, is this a fork from the current template or the modular one? [15:07:01] it's from the current version [15:07:11] the keep things small as possible [15:08:38] my longer term plan is to get rid of some bits there but not for now [15:17:25] Sorry I guess I'm confused about your plan. But I'm interested in moving toward a more modular makefile strategy long term. It's a bit easier to work with than our current template for simple stuff and it's flexible for how I see models changing in the future. [15:17:51] Does that make sense to you too? [15:18:05] What do you mean by "keep things small"? [15:18:24] The modular version makes both the template and the code we need to maintain substantially less complex. [15:24:01] by things I meant keeping changes smaller [15:38:44] Gotcha. I'm interested in applying the modular template since that's going to make future work a lot easier. Restructuring configs should be algorithmicly do-able. [15:38:58] This is not the first time our lack of modularity has bit us. [15:39:27] Anyway, it looks like the only note you have on that PR is related to getting that one little map in a config file. [15:39:41] Do you intend to block the PR on me designing a global config reading strategy? [15:39:47] Any other notes? [15:40:51] If you show how new local configs would look like (for example one wiki is done) that would be great [15:41:01] That's in the PR and in the task. [15:41:19] Strike that. It looks like it is only in the task. [15:41:42] Huh. I'll put it in the PR too. [15:42:40] my changes are going in that direction. I'm introducing "review_samples" [15:42:47] and removing those bits [15:43:46] I'm about to put up the PR for you to see [15:43:56] Why are you duplicating my work? [15:44:17] I guess I have my answer above (to make smaller changes) [15:44:20] Just seems strange. [15:44:21] it's not duplication, it's doing the first bit so I can unblock myself for hewiki new labeling campaign [15:46:13] I gotcha. I was hoping to get this modularization done for that. [15:46:27] It's a smaller change that renaming all of our classes on ORES/revscoring :P [15:48:20] well, I haven't done that either :D [15:48:53] * halfak tries to find where to even put "globals.yaml" [15:49:01] https://github.com/wikimedia/editquality/pull/189/files [15:49:05] Right now, it doesn't make much sense to put it anywhere really. [15:49:25] This PR doesn't make any change to the auto-generated Makefile. [15:50:22] I want to get rid of default_sample too (what the hell is that) [15:50:50] wikimedia/editquality#499 (hewiki_new_campaign - ff39f0c : Amir Sarabadani): The build failed. https://travis-ci.org/wikimedia/editquality/builds/498233155 [15:50:57] No idea. This stuff all looked like a mess to me -- hence why I essentially started from scratch. I'd really like to get the work I did merged rather than doing it again in parallel. [16:06:30] Amir1, some configs specify balanced: true. Others have unbalanced: true. Some have nothing specified. Anything I'm missing as to why? [16:06:44] yes [16:06:53] it's fawiki and other old cases [16:07:08] where the campaign was labeled automatically inside wikilabels [16:07:22] So it's unbalanced, but we're not saying so? [16:09:29] yes because if we say, the gen treat it as one of the new unbalanced cases [16:09:48] (where we merge things afterwards) [16:12:07] the biggest problem with the gen is all of these edge cases that need to be addressed somehow. [16:12:08] Hmm. This is not really explicit. Maybe we could adopt some of the modularization I'm doing instead. In that case, it'd be very explicit. [16:12:34] Right. Hence why making an explicit, modular pattern can help. [16:17:12] Amir1, maybe it would help if I showed an example config for an edge case wiki? [16:17:18] Take your pick and I'll give oyu one. [16:17:42] fawiki would be great [16:46:41] wikimedia/editquality#503 (hewiki_new_campaign - ccea520 : Amir Sarabadani): The build was fixed. https://travis-ci.org/wikimedia/editquality/builds/498244515 [16:46:42] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Create new editquality labeling campaign for hewiki - https://phabricator.wikimedia.org/T215363 (10Ladsgroup) https://github.com/wikimedia/editquality/pull/189 [16:58:35] Amir1, https://github.com/wikimedia/editquality/pull/181/files#diff-75fab677af5b2064a06c6a6e7f4e8f62 [16:59:42] Amir1, IMO this hardly looks like an edge case at all. The only difference between this and other wikis is we can just download the labeled dataset from wikilabels without needing to merge. [17:04:01] harej, coming to the meeting this morning? [17:04:11] yes, hold on [17:18:29] 10Jade, 10Scoring-platform-team (Current): Create simple pitch deck for Jade - https://phabricator.wikimedia.org/T217060 (10Harej) [17:47:50] 10ORES, 10Scoring-platform-team, 10Operations, 10Release Pipeline (Blubber): Build blubber file for ORES - https://phabricator.wikimedia.org/T210268 (10thcipriani) [17:51:19] halfak: is there anything particularly interesting this morning for the sync with research? [18:30:44] afk for lunch [18:36:43] * halfak is doing meetings for a while. [18:36:53] harej, yes. [18:37:00] But you're too late. [18:44:21] I can see that [18:44:51] For reference, see 2/25 on https://etherpad.wikimedia.org/p/scoring_research_hangout [18:45:01] harej, ^ let me know if you have questions. [18:51:18] Amir1, looks like we might need to pick up our discussion on Makefile for tomorrow [18:52:24] I'm just about to head to the U and then I'll have meetings for most of the rest of the day. If you want to either leave notes on my PR or update your PR to address the issues we discussed, I'll use my downtime to review. [19:04:34] OK I'm heading out for lunch/travel to UMN. [19:28:32] back now [20:17:28] 10Scoring-platform-team (Current), 10Edit-Review-Improvements-Integrated-Filters, 10editquality-modeling, 10Growth-Team (Current Sprint), and 2 others: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Ladsgroup) @Catrope Hey, the model is live. Please check if... [23:09:20] Meetings complete! [23:09:24] * halfak looks at review column [23:12:26] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/JADE] - 10https://gerrit.wikimedia.org/r/492849 (owner: 10L10n-bot) [23:27:21] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/492870 (owner: 10L10n-bot) [23:29:44] 10ORES, 10Scoring-platform-team, 10Operations: [Discuss] ORES without celery - https://phabricator.wikimedia.org/T216838 (10Halfak) The reason to separate IO from CPU parallelization is for batching efficiency. One process performs IO in batch. Many processes process subsets of that data in parallel. [23:31:04] 10ORES, 10Scoring-platform-team, 10Operations: [Discuss] ORES without celery - https://phabricator.wikimedia.org/T216838 (10Halfak) Oh! I misread. I agree with your point about not needing to drop it. I think the idea of dropping it comes from @Joe's proposal that we make ORES look like other services....