[00:21:02] 10Scoring-platform-team (Current), 10Global-Collaboration, 10MediaWiki-extensions-ORES: [Epic] Audit of pending ORES GUI deployments - https://phabricator.wikimedia.org/T187836#4012460 (10Catrope) [00:23:32] 10Scoring-platform-team (Current): Scoring platform team FY18 Q2 - https://phabricator.wikimedia.org/T176324#4012461 (10Catrope) [00:23:32] 10Scoring-platform-team (Current), 10editquality-modeling, 10Epic, 10artificial-intelligence: [Epic] Edit quality models (damaging/goodfaith) - https://phabricator.wikimedia.org/T130213#4012462 (10Catrope) [00:23:32] 10Scoring-platform-team (Current), 10Collaboration-Team-Triage, 10Edit-Review-Improvements-Integrated-Filters, 10editquality-modeling, and 5 others: Deploy ORES filters to Spanish Wikipedia - https://phabricator.wikimedia.org/T130279#4012458 (10Catrope) 05Open>03Resolved a:03Catrope [00:29:41] 10Scoring-platform-team (Current), 10Global-Collaboration, 10MediaWiki-extensions-ORES: [Epic] Audit of pending ORES GUI deployments - https://phabricator.wikimedia.org/T187836#4012482 (10Catrope) [06:01:46] PROBLEM - puppet on ORES-worker04.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:02:04] PROBLEM - puppet on ORES-worker03.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:11:26] PROBLEM - puppet on ORES-worker05.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:12:54] PROBLEM - puppet on ORES-redis02.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:14:45] PROBLEM - puppet on ORES-worker02.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:21:52] PROBLEM - puppet on ORES-worker06.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:41:50] PROBLEM - puppet on ORES-worker04.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:42:05] PROBLEM - puppet on ORES-worker03.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:44:15] RECOVERY - puppet on ORES-worker02.experimental is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [06:51:22] RECOVERY - puppet on ORES-worker06.experimental is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [06:51:30] PROBLEM - puppet on ORES-worker05.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:52:55] PROBLEM - puppet on ORES-redis02.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [07:01:16] RECOVERY - puppet on ORES-worker04.experimental is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [07:01:34] RECOVERY - puppet on ORES-worker03.experimental is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures [07:11:56] RECOVERY - puppet on ORES-worker05.experimental is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [13:50:30] 10Scoring-platform-team (Current), 10editquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: JSON error in building models - https://phabricator.wikimedia.org/T188535#4013848 (10Ladsgroup) p:05Triage>03High a:03Ladsgroup [13:51:04] 10Scoring-platform-team (Current), 10editquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: JSON error in building models - https://phabricator.wikimedia.org/T188535#4013852 (10Ladsgroup) https://github.com/wiki-ai/editquality/pull/137 [13:56:05] wiki-ai/editquality#147 (fix_quote - 589a595 : Amir Sarabadani): The build passed. https://travis-ci.org/wiki-ai/editquality/builds/347770448 [13:58:34] halfak: for when you have some time: https://github.com/wiki-ai/editquality/pull/137 [13:59:12] This helps me move forward with wrapping up the Makefil [14:04:47] Amir1, merged [14:32:47] PROBLEM - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is CRITICAL: CRITICAL: https://grafana.wikimedia.org/dashboard/db/ores is alerting: 5xx rate (Change prop) alert. [14:45:57] RECOVERY - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is OK: OK: https://grafana.wikimedia.org/dashboard/db/ores is not alerting. [14:52:11] you rule halfak [14:52:20] :D [14:55:00] Amir1, can you tell what failed here? https://travis-ci.org/wiki-ai/revscoring/jobs/347784588 [14:55:08] I'm super confused about the warning at the bottom. [14:55:22] I can't reproduce it locally [14:56:25] halfak: me too, I will take deeper look but it seems enchant is not happy about something [14:56:45] It's weird that the test still exited with zero [14:56:54] So it might be a red herring [14:57:41] Oh! It's a flake8 issue way earlier [14:57:43] I'll get that [14:57:44] :) [14:58:18] :D [15:00:11] Sorry to bug you [15:02:35] akosiaris, I just checked out our last icinga ping. Looks like we had a brief period of 500s from codfw. Is that something we should worry about? [15:03:38] 10Scoring-platform-team (Current), 10Edit-Review-Improvements-Integrated-Filters, 10ORES, 10editquality-modeling, and 8 others: Enable ORES filters for svwiki - https://phabricator.wikimedia.org/T174560#4014027 (10Trizek-WMF) >>! In T174560#4006298, @awight wrote: > Here's a link to the untranslated string... [15:21:04] halfak the one on 10:37 UTC ? [15:21:21] ah no, the one on 14:32 UTC [15:22:02] https://grafana.wikimedia.org/dashboard/db/ores?refresh=1m&panelId=23&fullscreen&orgId=1&from=1519914416653&to=1519915046360 [15:22:04] Right [15:26:50] halfak: tbh I am more interested in https://grafana.wikimedia.org/dashboard/db/ores?orgId=1&from=1519908185902&to=1519910954447 than the alert but I am looking at the alert related anyway [15:27:45] what's wrong there? [15:27:53] Looks like there was a sudden burst of external requests. [15:28:12] The overall errors don't look concerning to me. maybe we should turn that plot into an error-rate. [15:28:17] yes, should we have errored out ? [15:28:40] it is an error rate already, isn't it ? [15:28:49] with a peak of 15 errors per minute [15:29:02] if I read that graph correctly (and the graph is correct as well) [15:29:08] We're seeing 10x external request rate and 10x the error rate == same error/request rate [15:30:54] ok but why ? it's not due to an overload, why were we not able to service those requests ? [15:31:39] the "it's not due to an overload" assumption comes from the empty "overload errors" graph for that period of time [15:32:03] could very well have been malformed requests btw and it would be perfectly valid to error out to them [15:32:11] it's just that I don't offhand know that [15:32:33] It's normal to error on many types of revisions. [15:32:41] We should not see this from malformed requests. [15:32:58] This is likely due to requesting scores for pages that have been deleted or revisions that have been suppressed. [15:33:27] It tends to always happen because there are always deleted pages and suppressed revisions. [15:33:35] So when request rates go up, error rates do too. [15:33:41] errors/request shouldn't [15:34:29] aha ok that sheds some light into it. thanks for the info [15:40:22] PROBLEM - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is CRITICAL: CRITICAL: https://grafana.wikimedia.org/dashboard/db/ores is alerting: 5xx rate (Change prop) alert. [15:40:32] eeek. [15:40:54] Now it's eqiad [15:41:01] Something maintenance going on? [15:41:09] akosiaris, ^ [15:41:29] Seems strange given that codfw was only serving precaching and eqiad is serving traffic. We're not seeing overloads. [15:45:18] Identical two peaks with the second slightly lower than the first. https://grafana.wikimedia.org/dashboard/db/ores?orgId=1&from=1519914033344&to=1519919021486&refresh=1m&panelId=23&fullscreen [15:45:37] akosiaris, ^ [15:45:58] Woops. Sorry for multiple pings. Forgot that I'd just pinged a bit ago [15:48:56] codfw is changeprop btw [15:49:09] changeprop async is only there active [15:49:22] RECOVERY - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is OK: OK: https://grafana.wikimedia.org/dashboard/db/ores is not alerting. [15:49:26] changeprop async == jobqueue for all intents and purposes in this conversation [15:50:11] oh yeah. When I say "precaching" that's just what we call the endpoint that changeprop hits from on ORES. [15:50:19] So precaching == changeprop for ORES [15:51:26] but to answer the questions yes there was a new deploy for jobqueueevent bus just a few mins ago [15:51:41] (05:26:49 μμ) logmsgbot: !log mobrovac@tin Synchronized php-1.31.0-wmf.22/extensions/EventBus/includes/JobQueueEventBus.php: EventBus: Specify that EventBus queue supports delayed jobs (wmf/1.31.0-wmf.22) - T188540 (duration: 01m 13s) [15:51:42] T188540: Switch cdnPurge to Kafka - https://phabricator.wikimedia.org/T188540 [15:51:46] I am guessing it's related [15:52:35] akosiaris, oh interesting. We're getting the statsd from ChangeProp for those graphs/alerts -- not ORES. [15:52:55] yes that alert is changeprop generated [15:53:46] I guess we should just keep in mind that this happened in case it happens again some time soon, but maybe there's nothing else to investigate right now -- if we think maybe the EventBus deploy has something to do with it. [15:53:57] I think so too [15:54:04] in fact there were 2 event bus deploys [15:54:12] and the events in the graph are 2 as well [15:54:19] and with the same time apart [15:54:29] it's a bit too much to be a coincidence [16:00:15] 10Scoring-platform-team (Current), 10Edit-Review-Improvements-Integrated-Filters, 10ORES, 10editquality-modeling, and 8 others: Enable ORES filters for svwiki - https://phabricator.wikimedia.org/T174560#4014188 (10Lokal_Profil) >>! In T174560#4014027, @Trizek-WMF wrote: >>>! In T174560#4006298, @awight wro... [16:04:10] 10Scoring-platform-team (Current), 10Global-Collaboration, 10MediaWiki-extensions-ORES: [Epic] Audit of pending ORES GUI deployments - https://phabricator.wikimedia.org/T187836#4014198 (10Trizek-WMF) [16:04:22] 10Scoring-platform-team (Current), 10Edit-Review-Improvements-Integrated-Filters, 10ORES, 10editquality-modeling, and 8 others: Enable ORES filters for svwiki - https://phabricator.wikimedia.org/T174560#4014196 (10Trizek-WMF) 05Open>03Resolved >>! In T174560#4014188, @Lokal_Profil wrote: > Posted it as... [16:05:08] ohey [16:08:39] tgr|away: Sorry I’m late for the Outreachy meeting—still happening? [16:40:17] o/ awight [16:40:40] mornoon [16:40:44] :) [16:40:52] * halfak cranks through persian article quality. [16:54:28] 10Scoring-platform-team (Current), 10articlequality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Article quality campaign for Persian Wikipedia - https://phabricator.wikimedia.org/T174684#4014464 (10Halfak) https://github.com/wiki-ai/wikilabels-wmflabs-deploy/pull/44 [16:55:22] awight, one note from the conversation with prateek -- [16:55:41] He's really interested in the judgment/endorsement pattern [16:56:04] And he suggested that we consider recording it in the JSON on the content pages for JADE. [16:56:15] ah we don’t have that in our sights any more. [16:56:16] I hadn't considered that and I think it's an interesting idea. [16:56:23] This would be a quantitative endorsement, then? [16:56:29] Rather than just a talk page +1? [16:56:32] Right [16:57:08] So someone querying JADE could identify disagreements in a structured way. [16:57:16] Still not sure if we even want something like that. [16:57:24] But it was interesting to consider capturing it in the page content. [16:58:28] That will wreak havoc with our isomorphism where JADE content is one judgment, authored by one person. It’s okay, we just need to figure out a thing to do. [17:03:12] right. But it will solve a bunch of other problems [17:03:22] e.g. comments can now be related to specific thing. [17:09:19] Anyway, I thought it was an interesting idea not sure we need to pursue it. [17:12:14] I like the idea. It really begs the question about how we’ll migrate from the current strategy to the next iterations, though. [17:12:23] We’re tightly coupled to this page model... [17:19:51] Oh dear. I’m just reading staff etherpad notes. I missed registration for the Data Justice conf?? [17:25:51] 10Scoring-platform-team, 10MediaWiki-Vagrant, 10MediaWiki-extensions-ORES: Can't enable ores role in vagrant - https://phabricator.wikimedia.org/T177555#4014601 (10Sau226) [17:42:28] net difficulties. [17:42:29] halfak: Can we chat about conferences? [17:42:42] staff meeting [17:43:04] But I promise to read and respond if you dump some thoughts here ^_^ [17:43:11] * awight jiggles Internet [17:44:12] halfak: Just what I had pasted into the staff etherpad so far. [17:45:38] How is June 1 not this fiscal? I thought FY started July 1? [17:50:40] June 30: Fiscal year ends [17:50:43] https://office.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Calendar [17:52:39] halfak: Also, will you have a minute to look at my JADE blog outline today? [17:53:06] can do [17:53:13] :D [17:53:20] Might have misread the dates [17:55:44] halfak: I see why you’re glued to the meeting. Wish I hadn’t missed it. [17:55:48] https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Staff_and_Contractors [17:57:42] That is… 0.55 fewer FTEs for us. [18:03:08] James_F pointed out this this was for *last* year vs current year. [18:03:12] darn FYs [18:05:56] This might miss the mark for me, but halfak: http://www.cs.rpi.edu/~xial/COMSOC18/ [18:09:19] awight: sorry, had some last minute work that lasted until 3AM so I had to throw that meeting :( [18:09:40] tgr: Argh! Sorry to hear about the late night [18:10:06] we'll do it Friday 1 hour later if that works for you [18:10:32] so 17h UTC I think [18:11:15] Fridays are usually hard, but I think it’ll work this week. [18:20:13] lunching. [18:42:00] I'm off to lunch too. [18:42:11] Will get back to conference stuff and jade blog when I'm back [18:50:11] 10Scoring-platform-team (Current), 10User-Ladsgroup: Train/test damaging/goodfaith model for Catalan Wikipedia - https://phabricator.wikimedia.org/T187732#4015138 (10Ladsgroup) https://github.com/wiki-ai/editquality/pull/138 [19:13:28] hey halfak|Lunch what's the syntax for injecting new values for features in ORES v3? I can't find an example in the docs [19:14:58] Amir1, Nettrom: if you happen to know the answer ^^ :) [19:16:09] J-Mo: features= should work [19:16:09] https://ores.wikimedia.org/v3/scores/testwiki/?models=damaging|goodfaith&revids=2&precache=1&format=json&features= [19:18:07] Amir1 I'm looking to see how changing the value of one of the features affects predictions. So for example https://ores.wmflabs.org/v3/scores/enwiki/?models=wp10&revids=793978592&features=true I want to see how changing the value of feature.wikitext.revision.headings_by_level(2) to 6 would change the class prediction. [19:18:29] it looks like theres [19:18:45] there's an inject= arg, but not sure how to format the injection key/val [19:19:58] let me take a look at the codebase [19:22:15] J-Mo: okay, the code that does it is not dependent on the version. it should be a valid json it seems [19:24:35] hmmm. just tried https://ores.wmflabs.org/v3/scores/enwiki/?models=wp10&revids=793978592&features=true&inject={%27feature.wikitext.revision.headings_by_level(2)%27:6} and it doesn't seem to work [19:27:54] J-Mo: This works [19:27:54] https://ores.wmflabs.org/v3/scores/enwiki/?models=wp10&revids=793978592&inject={%22793978592%22:%20{%22feature.english.stemmed.revision.stems_length%22:%20456}}&features= [19:28:06] you need to give rev_id mapping [19:28:16] (I found it from the codes) [19:28:40] perfect :) thanks Amir1! [19:29:48] Thank you for using it :) [20:16:18] J-Mo, also an option is to directly specify the features as a query parameter. E.g. "?feature.wikitext.revision.headings_by_level(2)=6" [20:16:45] That way you don't have to have a bunch of JSON stuff. [20:16:55] ah, that's easier. Thanks halfak [20:17:02] no problem :) [20:54:54] oummkay [20:54:59] o/ [20:56:06] awight, I don't recognize any of the committee for http://www.cs.rpi.edu/~xial/COMSOC18/ [20:56:10] Which could be good or bad. [20:56:23] Could be a blind spot. Could be not a high quality event. [20:59:11] I dug into "Random Sample Voting" -- apparently "invented" by the lead keynote speaker. [20:59:20] AFAICT, this guy re-invented polling [21:01:40] 10Scoring-platform-team (Current), 10Edit-Review-Improvements-Integrated-Filters, 10ORES, 10editquality-modeling, and 8 others: Enable ORES filters for svwiki - https://phabricator.wikimedia.org/T174560#4015629 (10Lokal_Profil) >>! In T174560#4014196, @Trizek-WMF wrote: >>>! In T174560#4014188, @Lokal_Prof... [21:05:22] o/ awight, did you see my notes re COMSOC? [21:05:55] Internet has been flaking, I’m just reading https://wm-bot.wmflabs.org/logs/%23wikimedia-ai now [21:06:52] COMSOC notes sound horrifying. I’d already decided it wasn’t quite my wheelhouse, so that finishes that line of inquiry. [21:07:50] Any thoughts about the other three which I’d pasted into the staff etherpad? [21:07:59] Those all seem genuinely exciting [21:08:20] I’m reading about a few others (see “current work” if you’re interested) [21:08:46] This one sounds good from my superficial understanding, http://ecis2018.eu [21:14:27] Oh great, https://etherpad.wikimedia.org/p/sp_design has stuff that’s relevant to the blog post [21:31:25] This is a nice sentiment, but I’m put off by the fact that they reuse the same conference description every year: http://stpis2018.blogs.dsv.su.se [21:38:39] halfak: I would love your opinion on the Data Justice conf in particular, i.e. am I authorized to attend? [21:39:00] awight, it looks like that's a workshop of another conference. [21:39:07] yes I think so [21:39:14] but seems that you can register for it directly [21:39:31] indeed it does. [21:40:32] I don’t know how to interpret that relationship. They explain it as, > ‘Data Justice’ will take place shortly before the ICA 2018 conference in Prague [21:41:14] OK I'd like to see us represented at this event. [21:41:34] Can you make a cost estimate using the template that travel has and email me that? [21:43:16] ok, cool. I made a random cost estimate of $130 flight, $140 registration, and $300 room and board, but I’ll dig around for the travel template. [21:44:48] Holy moley that's super cheap. [21:44:54] I'll almost certainly approve :) [21:44:58] https://phabricator.wikimedia.org/T183200 [21:45:07] yeah. part of why I’m psyched to linger in the EU after the hackathon [21:45:13] awight, can you add a link to your blog outline here to that task [21:45:16] +1 [21:45:18] \o/ [21:45:49] 10Scoring-platform-team (Current), 10JADE, 10WMF-Communications: Blog about JADE - https://phabricator.wikimedia.org/T183200#4015806 (10awight) Current outline draft, https://docs.google.com/document/d/1g2_mHPg92CgK1iY3s1TYPVAI-ioiAc7AzudhMD_eLhw/edit [21:46:19] Thanks [21:46:51] oops, I left out a more expensive plane ticket ;-). here comes actual estimate via email. [21:47:35] lol accidentally worked as my personal google account for a second. [21:48:02] Oh. I can't edit [21:48:04] awight, ^ [21:48:11] kk [21:48:38] You can edit, but feel free to do so in Suggesting mode [21:55:58] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/JADE] - 10https://gerrit.wikimedia.org/r/415715 (owner: 10L10n-bot) [21:56:42] :-) someone is excited [21:59:36] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/415722 (owner: 10L10n-bot) [22:00:03] \o/ [22:00:31] Looks like it's Hauskatze [22:00:41] Thanks for working on the translated strings! [22:01:01] two different editors. rad! [22:01:25] 10Scoring-platform-team (Current), 10JADE, 10WMF-Communications: Blog about JADE - https://phabricator.wikimedia.org/T183200#4015849 (10Halfak) Moved to https://www.mediawiki.org/wiki/JADE/Intro_blog so we can track changes easier. [22:01:38] Oh good point. "Red Winged Duck" hmmm [22:01:46] Oh and on JADE :D [22:04:07] Makes me wish we had more to translate... [22:04:14] awight, I'm hacking here: https://www.mediawiki.org/wiki/JADE/Intro_blog [22:04:19] ty [22:04:38] I'm working on https://github.com/wiki-ai/wikilabels-wmflabs-deploy/pull/44 [22:04:44] That will add some new strings to translate :) [22:04:52] But we need Amir1 to help me with the persian. [22:05:18] I will be checking it really soon [22:05:22] Thanks :) [22:06:04] * Hauskatze was called? [22:06:35] I fixed something on twn.net for ORES. Thanks to you for deploying the filters to the eswikis [22:06:46] I'm using them now and love them [22:07:01] Great! [22:07:10] I'm happy they are working for you. :))) [22:07:48] :D Thanks 100x for contributing back to the project, of course! [22:07:53] for now only few-ish edits that were actually bad were not highlighted, the same with good edits [22:08:35] Hauskatze: feel free to flag those bad predictions, we’re actually knee-deep in a system to make this sort of reporting easier: https://www.mediawiki.org/wiki/JADE [22:08:49] For now, just an ad-hoc talk page would be awesome. [22:08:57] awight: didn't knew about that system [22:09:11] It… doesn’t exist yet :) but we’re pushing hard on the implementation. [22:10:14] do we have ad-hoc talk page for eswiki reporting? [22:11:00] Hauskatze: Here’s a stub if you’d like, https://www.mediawiki.org/wiki/ORES/Issues/Edit_quality [22:11:22] Hauskatze, fun story, we're working on a better way to do this right now. Thanks for dealing with the messy wiki-page for the time being. [22:11:32] It also links to the metawiki page where some ad-hoc reports were made. [22:11:58] It'll be very helpful for us if you can help us track issues with ORES in eswiki. Any trends in the types of mistakes that it makes could point to a bug. [22:12:14] awight, just finished updating the lead here: https://www.mediawiki.org/wiki/JADE/Intro_blog [22:12:15] ok, so I'll add issues there on mediawiki [22:12:20] I'll be back in a bit. [22:12:26] for now I've detected few [22:12:35] 2 to be exact [22:12:46] !! awesome. [22:12:49] out of hundreds of thousands of edits, that ain't bad [22:13:17] * halfak sees his own typos and stops reading [22:13:19] * Hauskatze continues reading a book [22:13:19] * halfak runs away [22:14:40] heck yeah. If you want to dig into the weeds, we have statistics on what we *think* the accuracy is, e.g. https://ores.wikimedia.org/v3/scores/eswiki/?models=damaging&model_info=statistics [22:25:56] awight: how can I get the ores score for a diff? [22:26:03] api? [22:26:15] If you want the API location... [22:26:20] First of all, here are the docs: [22:26:42] https://ores.wikimedia.org/v3/ [22:26:54] Here’s an example API result for all the scores on a diff: [22:27:07] https://ores.wikimedia.org/v3/scores/eswiki/123456 [22:28:16] so for https://es.wikipedia.org/w/index.php?title=Pino_Ojeda&diff=prev&oldid=105911378 [22:28:33] as you can see on https://es.wikipedia.org/wiki/Especial:Contribuciones/81.47.161.203 [22:28:34] ah yeah—the last number is the revision ID, so you’d have to get that first [22:28:47] the second edit was not highlighted [22:29:22] ooh that’s a nasty edit [22:29:56] weird, I’m baffled by the MediaWiki URL. I guess oldid is the revision of the change, if diff=prev [22:30:17] so yeah, https://ores.wikimedia.org/v3/scores/eswiki/105911378 [22:30:30] https://es.wikipedia.org/w/index.php?title=Pino_Ojeda&diff=105911378 [22:30:41] That was predicted to be bad faith and damaging, which seems right. [22:30:58] yep, that's right [22:31:01] 0.8192 [22:31:05] of damaging [22:31:19] ah so it didn’t reach the highlighting threshold? [22:31:24] not sure why it wasn't highlighted though [22:31:33] maybe it's on my side [22:31:45] my custom configuration on Special:Preferences [22:31:49] We just enabled ORES on eswiki, so it’s possible that we just need to adjust the thresholds… It’s still helpful to report the problem, though [22:31:51] I'll check that [22:35:18] trying to dig up the current thresholds... [22:37:41] This is how the current threshold is defined for eswiki, 'verylikelybad' => [ 'min' => 0, 'max' => 'recall_at_precision(min_precision=0.98)' ], [22:37:52] that formula resolves to... [22:38:57] that edit is at 0.81 < 0.98 [22:40:02] "threshold": 0.995 [22:40:09] https://ores.wikimedia.org/v3/scores/eswiki/?models=damaging&model_info=statistics.thresholds.true.%22maximum%20recall%20@%20precision%20%3E=%200.98%22 [22:40:20] thanks [22:40:25] Looks like that threshold is damn near impossible. [22:41:03] * halfak would like them to be using this threshold: https://ores.wikimedia.org/v3/scores/eswiki/?models=damaging&model_info=statistics.thresholds.true.%22maximum%20precision%20@%20recall%20%3E=%200.75%22 [22:41:08] Which would catch it. [22:41:13] halfak: btw, I just noticed that the MediaWiki config is still using old-style threshold formulas [22:41:27] awight, I think they get converted somewhere. [22:41:33] they do. [22:42:40] but I ran into a silly glitch, where config is munged, but requests are not. So someone with the config will have a PITA trying to make a request by hand. [22:42:52] IMO we should migrate all the config and drop the old style. [22:47:22] +1 [22:47:27] Think it will be much trouble? [22:50:05] no, but I think Amir1 ran into a mysterious issue converting wikidata ORES config. [22:50:12] I’m curious what that was. [22:50:24] awight: yup, it's in gist [22:50:29] let me get it for you [22:50:36] Thanks! [22:50:53] https://gist.github.com/Ladsgroup/3efbe704382b3299c00bb7be03dbfb2f [22:51:13] Amir1: ah sorry, we’re looking at something slightly different. [22:51:28] the mediawiki-config thresholds are still using the old syntax [22:51:35] ohhh [22:51:38] I remember you had converted wikidata, but something went wrong? [22:52:21] yeah, I think they had issues [22:52:29] started to 500 [22:52:49] That might have been unrelated, though? [22:54:05] https://phabricator.wikimedia.org/T181159 [22:57:56] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES: Migrate ORES extension threshold config from old to new syntax - https://phabricator.wikimedia.org/T181159#4015972 (10awight) [23:03:22] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES: Migrate ORES extension threshold config from old to new syntax - https://phabricator.wikimedia.org/T181159#4015981 (10awight) [23:25:09] This is ready for review: https://github.com/wiki-ai/editquality/pull/138 [23:25:24] This is a little bit weird: https://github.com/wiki-ai/editquality/pull/136 [23:29:21] Amir1, just looking at that makefile and it doesn't look like "goodfaith_weight" is defined. [23:29:28] Looks like it's an old bug. [23:29:36] This should be able to work. [23:29:59] See https://github.com/wiki-ai/editquality/blob/master/Makefile.manual#L32 [23:35:04] halfak: on which PR? [23:37:16] 138 [23:37:22] https://github.com/wiki-ai/editquality/pull/138 [23:38:25] By including the manula file we are actually defining it [23:38:46] but you're right, we should move them to automated file [23:38:58] that's another PR IMO [23:44:07] Cool. That makes sense. [23:44:11] I need to relocate, will try to get a bit more time in after hours. [23:45:49] I'm out of here too. It's getting late-ish and I have some evening chores. [23:46:25] Amir1, left a comment on PR 138 and on PR 136. [23:46:48] Thanks. Will get to them soon [23:47:21] halfak: let me update the cawiki PR [23:47:25] that would be fast [23:47:35] the enwiktionary PR seems more complex [23:47:46] agreed. [23:50:12] halfak: double checking, we are excluding reverted models, tuning reports and even Makefile rules [23:50:22] is there anything else needs to be done? [23:50:36] I don't think so. Everything else looked OK to me. [23:53:51] OK actually leaving now. Will likely merge in the AM. [23:53:52] o/