[00:15:15] 06Revision-Scoring-As-A-Service, 10wikilabels: Remove expired tasks from worksets - https://phabricator.wikimedia.org/T134619#2271455 (10Ladsgroup) [00:40:55] 10Revision-Scoring-As-A-Service-Backlog, 10bwds, 10revscoring: Language assets for Norwegian - https://phabricator.wikimedia.org/T131855#2271497 (10Ladsgroup) @Galar71. Hey, Is it finished? Please keep me posted :) Thanks [00:48:01] 06Revision-Scoring-As-A-Service, 10ORES: [Investigate] ORES spike of errored requests every hour - https://phabricator.wikimedia.org/T134109#2271503 (10Ladsgroup) Almost all of errors registered in logs are happening because of unsupported wikis. Most of all, Commonswiki and nowiki. It would be nice to support... [01:27:53] 06Revision-Scoring-As-A-Service, 10ORES: [Investigate] ORES spike of errored requests every hour - https://phabricator.wikimedia.org/T134109#2271564 (10Ladsgroup) On a second thought, these errors should not get to out grafana dashboard. So I want to turn up error logging a little bit: https://github.com/wiki-... [01:28:07] 06Revision-Scoring-As-A-Service, 10ORES: [Investigate] ORES spike of errored requests every hour - https://phabricator.wikimedia.org/T134109#2271565 (10Ladsgroup) [03:08:00] 06Revision-Scoring-As-A-Service, 10wikilabels: u_wikilabels_test database should be a copy of u_wikilabels - https://phabricator.wikimedia.org/T134627#2271670 (10Ladsgroup) [03:21:28] 10Revision-Scoring-As-A-Service-Backlog, 10Wikimania-Hackathon-2016-Organization, 06Developer-Relations (Apr-Jun-2016): A training session on supporting ores in more languages in Wikimania Hackathon - https://phabricator.wikimedia.org/T134628#2271692 (10Ladsgroup) [03:31:06] 06Revision-Scoring-As-A-Service, 10Wikimania-Hackathon-2016-Organization, 10bwds, 06Developer-Relations (Apr-Jun-2016): Generate bad words for all languages more than 100K articles - https://phabricator.wikimedia.org/T134629#2271709 (10Ladsgroup) [04:52:55] 06Revision-Scoring-As-A-Service, 10Wikimania-Hackathon-2016-Organization, 10bwds, 06Developer-Relations (Apr-Jun-2016): Generate bad words for all languages more than 100K articles - https://phabricator.wikimedia.org/T134629#2271779 (10Ladsgroup) With [[https://github.com/wiki-ai/Bad-Words-Detection-System... [04:53:19] 06Revision-Scoring-As-A-Service, 10Wikimania-Hackathon-2016-Organization, 10bwds, 06Developer-Relations (Apr-Jun-2016): Generate bad words for all languages more than 100K articles - https://phabricator.wikimedia.org/T134629#2271780 (10Ladsgroup) [06:46:56] 10Revision-Scoring-As-A-Service-Backlog, 10ORES, 10revscoring: Score multiple models with the same cached dependencies - https://phabricator.wikimedia.org/T134606#2271181 (10Ladsgroup) https://github.com/wiki-ai/ores/pull/142 [06:47:12] 06Revision-Scoring-As-A-Service, 10ORES, 10revscoring: Score multiple models with the same cached dependencies - https://phabricator.wikimedia.org/T134606#2271811 (10Ladsgroup) a:03Ladsgroup [06:47:28] 06Revision-Scoring-As-A-Service, 10ORES, 10revscoring: Score multiple models with the same cached dependencies - https://phabricator.wikimedia.org/T134606#2271181 (10Ladsgroup) [14:19:10] o/ [14:27:33] o/ [14:36:20] Hey GhassanMas. [14:36:29] Did amir help you get access to logs the other day? [14:38:29] yeah, he added me as a member in wikilabels project [14:38:37] Great :) [14:38:48] Happy you're working on this stuff with us. [14:39:45] halfak, me as well :) [14:42:05] by the way, halfak I am concerned about labeling difference of the talk pages and the articles [14:42:19] do you treat those samples the same [14:43:22] GhassanMas, roughly, yes. The model has features for knowing the difference between the different places. [14:43:36] But I think it is important that we can detect talk page damage/vandalism too. [14:44:47] yeah I definitely agree on that as well. I am just concerned about the tune. [14:49:25] Seems to work pretty well right now. The only thing I am really concerned about is whether we get enough signal about talk page edits to even catch vandalism there. [14:53:23] you meant enough samples of talk pages ? [14:53:30] Yeah. [14:53:35] Vandalism definitely happens there. [14:53:48] There's a few talk pages I have on my watchlist just because they get vandalized so often. [14:56:46] we could load campaigns for only talk pages edits, right? [14:59:48] Yeah. The hard part is then training the model. [14:59:59] We want to give the model a representative sample to learn on. [15:00:14] So that the probabilities will be *meaningful* to a consumer of the service. [15:00:29] So that means, we should have a representative proportion of talk page edits in our training set. [15:01:03] Amir and I have been experimenting with ways to produce representative training sets from biased samples. [15:01:14] So it's not impossible, but I'm not sure how it is going to work yet. [15:01:19] Actually, arwiki is one of those already :) [15:02:29] Right now, the 5k edits that need labeling are 2.5k "edits that are certainly not vandalism" and 2.5k "edits that need review". When the labeling is done, we'll have to resample to restore the usual proportion of these types of edits because they are not 50/50 :) [15:02:41] Time will tell if this strategy works. [15:02:58] But if it does, we can target other types of edits for labeling. [15:05:27] yeah as i understand that you wan't to avoid over-fitting the training samples since mostly they are biased [15:06:20] GhassanMas, I'm not so worried about overfitting so much as distorting the prediction probability. [15:07:10] The percentage should roughly represent the likelihood of a True Positive on a pure random sample of real data. [15:07:23] Right now, the wikidatawiki model doesn't do that [15:07:37] So we end up setting the threshold far higher than for the rest of the editquality models. [15:07:53] This is because the wikidatawiki model is trained on a dataset of 50% reverted edits and 50% not-reverted edits. [15:08:16] Far fewer than 50% of edits in wikidata are reverted. [15:08:24] It's more like 0.02% :D [15:09:34] yeah I can tell about that when I am labeling good-faith/damaging only around 1 sample of the 50 workset is damaged [15:11:47] so the problem is like in real life only around 0.02 is reverted, but to train the model we need 50/50 samples while make the prediction distorted [15:11:53] GhassanMas, yeah. Our efforts of mixing "edits needing review" with "edits that are probably fine" is to boost that likelihood of finding a damaging edit. [15:16:31] halfak: o/ [15:16:33] o/ Amir1 [15:16:40] Been working through your list. [15:16:41] got my email? [15:16:46] I have 45 minutes before I'll have to leave. [15:16:46] awesome [15:16:51] I saw merges [15:16:56] I need to go to [15:17:04] I'll be back in one hour [15:17:12] I just wanted to thank you :) [15:17:16] :) [15:17:21] One quick update for you Amir1: https://meta.wikimedia.org/wiki/User_talk:He7d3r#Update_to_ScoredRevisions.js_to_deal_with_regression [15:17:46] halfak: when do you have time? [15:17:56] Tomorrow, most likely [15:17:57] :( [15:18:37] :((((( [15:18:54] Sorry dude. Gotta do a lot of things AFK today [15:19:16] don't worry [15:19:22] please assign some stuff [15:19:33] Will do. I'll respond to your mail with ideas. [15:19:35] so it keep me busy for a while [15:20:16] (in 12 hours, I will pick up my passport and it'll be determined if I get the visa or not) [15:21:39] * halfak crosses fingers [15:21:46] halfak: http://stackoverflow.com/questions/7206499/match-letter-in-any-language [15:22:01] https://www.irccloud.com/pastebin/iw6G4WmR/ [15:22:12] regex library is great [15:22:16] we should use it more often [15:22:45] Yeah. I'm down for a switch. I had to do a lot of work to handle non-latin. [15:23:03] See the tokenizer code here: https://github.com/halfak/deltas/blob/master/deltas/tokenizers/wikitext_split.py [15:24:07] That's another place that we can look at doing performance optimizations. :) [15:24:26] one thing is about bwds: We use the dump extractor all the time [15:24:35] we don't use API extractor at all [15:24:38] it's useless [15:25:02] +1 makes sense. I think solve() is the right pattern. [15:26:29] yeah [15:26:42] I'll switch to that very soon [15:27:00] :) [15:28:04] I've got to go [15:28:07] see you soon [15:28:20] Take care! [15:28:21] o/ [15:28:25] you too [15:28:27] o/ [16:26:24] okay I'm back! [18:10:06] got to go [21:21:09] 10Revision-Scoring-As-A-Service-Backlog, 10revscoring: Simplify and optimize persian regular expressions - https://phabricator.wikimedia.org/T134268#2272676 (10Halfak) @ladsgroup, interested in taking a look at this? [21:24:08] 10Revision-Scoring-As-A-Service-Backlog, 10bwds, 10rsaas-editquality: migrate bad words detection to editquality repo - https://phabricator.wikimedia.org/T131861#2180913 (10Halfak) @ladsgroup, this one should be easy if you want to take a look. [21:27:18] 10Revision-Scoring-As-A-Service-Backlog, 10revscoring, 10rsaas-editquality: [Spike] Explore using PR-AUC to score when tuning - https://phabricator.wikimedia.org/T133698#2272679 (10Halfak) @ladsgroup, this one should be easy to pick up and the results will likely be very interesting :)