[06:00:05] updated that notebook with a better plot and added precision and recall of the different strategies [06:00:09] https://paws-public.wmflabs.org/paws/user/Groceryheist/notebooks/Illustrating%20the%20tradeoff%20between%20balance%20and%20calibration.ipynb [10:03:22] 10ORES, 10Scoring-platform-team, 10serviceops: Ores hosts: mwparserfromhell tokenizer random segfault - https://phabricator.wikimedia.org/T222866 (10Volans) [13:43:03] o/ [13:43:35] halfak: \o [13:43:56] * halfak digs through groceryheist's notebook [13:43:58] Hey Zppix [13:53:06] halfak: anything juicy in that notebook xD [13:53:43] Yeah. The performance of using threshold tuning for fairness is quite a lot better than the method employed by prior work. [13:54:10] halfak: not exactly what i was looking for but that works lol [13:55:30] Juicy to me :) [13:56:29] o/ akosiaris. Have you seen https://phabricator.wikimedia.org/T222866 ? [13:59:59] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10Spanish-Sites, and 2 others: Create editquality campaign for Spanish Wikiversity - https://phabricator.wikimedia.org/T209670 (10Halfak) @lsanabria, I'm still waiting on your response to my last questions. No rush. Just want to mak... [14:09:15] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10Spanish-Sites, and 2 others: Create editquality campaign for Spanish Wikiversity - https://phabricator.wikimedia.org/T209670 (10Lsanabria) Sorry for the delay. I have been somewhat disconnected these days. I just got admin rights in... [14:14:50] halfak: yup [14:15:06] halfak: already +1ed one of the changes in the task [14:26:24] Great! Thanks. [14:29:09] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10Spanish-Sites, and 2 others: Create editquality campaign for Spanish Wikiversity - https://phabricator.wikimedia.org/T209670 (10Halfak) OK let me re-run. We were already considering admins "trusted" but I'll see how much of a diffe... [15:31:40] o/ hare & groceryheist: running late to standyup. Sorry for the delay. [15:33:17] OK I'm in [16:08:29] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10Spanish-Sites, and 2 others: Create editquality campaign for Spanish Wikiversity - https://phabricator.wikimedia.org/T209670 (10Halfak) Turns out that cuts out only 500 edits -- from 10727 to 10214. That's a lot of edits to label.... [19:59:43] PROBLEM - puppet on ORES-worker01.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [20:29:29] RECOVERY - puppet on ORES-worker01.experimental is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [20:58:31] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10Spanish-Sites, and 2 others: Create editquality campaign for Spanish Wikiversity - https://phabricator.wikimedia.org/T209670 (10Halfak) Of the ~17k edits we get, here's the breakdown by our "autolabeler": | edits | type | "needs re... [21:15:08] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10Spanish-Sites, and 2 others: Create editquality campaign for Spanish Wikiversity - https://phabricator.wikimedia.org/T209670 (10Lsanabria) If they have not been reverted, I think, they are very likely good edits. Some bad ones might... [22:36:19] https://teblunthuis.cc/outgoing/time_to_revert_median.pdf [22:36:32] any idea what spikes in median time to revert are driven by? [22:36:39] is this like cluebot going down? [22:43:50] doesn't seem related to the number of reverts [22:43:52] https://teblunthuis.cc/outgoing/time_to_revert_geom_mean.pdf [23:19:17] So I think with these spikes in the data that the ITS model says that ORES increases TTR