[00:01:21] 10Revision-Scoring-As-A-Service-Backlog, 10MediaWiki-extensions-Wikilabels: Copy JS into extension and create special page - https://phabricator.wikimedia.org/T146405#3027794 (10Tgr) The steps are described [[https://www.mediawiki.org/wiki/Gerrit/Inactive_projects|here]]. Given that this extension was never pu... [00:01:21] 10[2] 04https://meta.wikimedia.org/wiki/https://www.mediawiki.org/wiki/Gerrit/Inactive_projects [04:28:00] 06Revision-Scoring-As-A-Service, 10rsaas-articlequality : [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3028336 (10yuvipanda) Here's my plan of action: 1. We don't need to update this db, it'll be a one time operation 2. I'll co-ordinate with DBAs... [05:24:47] 10Revision-Scoring-As-A-Service-Backlog, 10AbuseFilter, 10Bad-Words-Detection-System, 07Community-Wishlist-Survey-2015: Suggesting AbuseFilter by machine learning - https://phabricator.wikimedia.org/T120741#3028375 (10Tgr) Part of the problem statement was that regexes are hard to use for non-technical use... [15:29:03] 10Revision-Scoring-As-A-Service-Backlog, 10AbuseFilter, 10Bad-Words-Detection-System, 07Community-Wishlist-Survey-2015: Suggesting AbuseFilter by machine learning - https://phabricator.wikimedia.org/T120741#3029546 (10Halfak) Oh! That's a good point. Seems like we're looking at two separate problems here... [15:30:20] 06Revision-Scoring-As-A-Service, 06Research-and-Data, 07Epic: Estimate ORES capex for FY2018 - https://phabricator.wikimedia.org/T157222#3029547 (10Halfak) [17:08:58] 10Revision-Scoring-As-A-Service-Backlog, 10Mobile-Content-Service, 10ORES, 06Operations, 06Services (watching): Limit resources used by ORES - https://phabricator.wikimedia.org/T146664#3029929 (10mobrovac) >>! In T146664#3027004, @Halfak wrote: > @mobrovac, let me try again. Who from #operations did you... [17:27:06] 10Revision-Scoring-As-A-Service-Backlog, 10Mobile-Content-Service, 10ORES, 06Operations, 06Services (watching): Limit resources used by ORES - https://phabricator.wikimedia.org/T146664#3029987 (10Halfak) Great! But note that ORES is stateless unless you consider our cache to be "state". Surely "consens... [18:40:41] Amir1_: o/ [18:42:00] so I was asking a good way to track what properties used in what items. I have tried using ldf interface to do this. But obviously it did not work for properties like P31, which are used by many items [18:43:03] glorian_wd: what exactly do you need? [18:43:03] Hence, I guess we cannot use SPARQL in this case, and we should find another way to do this. I was thinking if we can use SQL query to query the database table [18:43:18] number of items used by P31? [18:44:28] for instance, items that use P31 [18:58:55] Amir1_: [18:59:01] let me find an example [18:59:01] for instance items that use P31 [18:59:47] okay [19:00:44] glorian_wd: https://www.wikidata.org/w/index.php?title=Special%3AWhatLinksHere&target=Property%3AP31&namespace=0 [19:00:54] there is an API version of it [19:01:35] how to access the API version? [19:04:14] 10Revision-Scoring-As-A-Service-Backlog, 10ORES, 06Community-Liaisons (Jan-Mar 2017): Tell wikis that use ORES being turned as a by-default feature on wikis that have it as a Beta feature - https://phabricator.wikimedia.org/T158225#3030378 (10Trizek-WMF) [19:05:23] 10Revision-Scoring-As-A-Service-Backlog, 10ORES, 06Community-Liaisons (Jan-Mar 2017): Tell wikis that use ORES being turned as a by-default feature on wikis that have it as a Beta feature - https://phabricator.wikimedia.org/T158225#3030396 (10Trizek-WMF) I've cc-ed some people who can help outreaching that c... [19:12:07] glorian_wd: Let me find that [19:12:25] Amir1_: okay ping me once you find it :) [19:13:23] 10Revision-Scoring-As-A-Service-Backlog, 10ORES, 06Community-Liaisons (Jan-Mar 2017): Tell wikis that use ORES being turned as a by-default feature on wikis that have it as a Beta feature - https://phabricator.wikimedia.org/T158225#3030428 (10Trizek-WMF) [19:15:38] 10Revision-Scoring-As-A-Service-Backlog, 10ORES, 06Community-Liaisons (Jan-Mar 2017): Tell wikis that use ORES being turned as a by-default feature on wikis that have it as a Beta feature - https://phabricator.wikimedia.org/T158225#3030378 (10Trizek-WMF) [19:21:43] glorian_wd: https://www.wikidata.org/w/api.php?action=query&prop=linkshere&titles=Property:P31&lhnamespace=0 [19:22:44] Amir1_: *looking at the link* [19:25:10] Amir1_: but I guess the number of items generated from the API on that link was limited for some reason [19:26:12] glorian_wd: yes, depends on what property you need [19:26:45] if you need p31, you really need to reconsider your idea because it's too big to be handled properly [19:26:51] yeah right [19:27:10] and database queries are expensive (and reading them is also resource consuming) [19:27:13] do you know where I can set up this constraint? so I can see the full list of items [19:27:20] actually I am looking for it right now [19:27:26] we can use other properties than P31 [19:27:40] hmm [19:36:55] Amir1_: I guess apparently using API also won't work, since the max items that can be shown is 500 (5000 for bots) [19:40:11] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Implement selective purging of model scores in varnish - https://phabricator.wikimedia.org/T148999#3030583 (10Halfak) Hm... that would more than duplicate the work the servers need to do, but yeah, it could simplify caching a lot. We'd need a lot more capacity... [19:49:17] 06Revision-Scoring-As-A-Service, 10rsaas-articlequality : [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3030674 (10Halfak) This all sounds great to me :) [19:52:02] glorian_wd: if you need more than 500 cases, you need to reconsider using it. [19:55:10] Amir1_: okay. Is it possible to get most used properties? [20:00:29] glorian_wd: There should be some sparql queries [20:10:20] Amir1_: I thought there is some statistics for it? [20:11:15] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Implement selective purging of model scores in varnish - https://phabricator.wikimedia.org/T148999#3030753 (10Halfak) One more note -- if we do that, we can ditch the celery workers and rely solely on uwsgi workers for parallelization. Currently we use a redis... [20:11:56] Yeah [20:39:24] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Implement selective purging of model scores in varnish - https://phabricator.wikimedia.org/T148999#3030803 (10Tgr) Deduplication is IMO vital to a high-scalability website to avoid cache stampedes, but there are be other strategies to do it. If all the requests... [20:43:54] 10Revision-Scoring-As-A-Service-Backlog, 10ORES, 06Community-Liaisons (Jan-Mar 2017): Tell wikis that use ORES being turned as a by-default feature on wikis that have it as a Beta feature - https://phabricator.wikimedia.org/T158225#3030821 (10Ladsgroup) One quick note, ORES review tool is not enabled in cswi... [21:45:10] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Implement selective purging of model scores in varnish - https://phabricator.wikimedia.org/T148999#3030994 (10Halfak) "Varnish request coalescing" or something like that sounds great. That would reduce our capacity concerns substantially out of the box. Honest... [22:27:25] 06Revision-Scoring-As-A-Service, 10Wikilabels, 07Spanish-Sites: Edit quality campaign for Spanish Wikipedia - https://phabricator.wikimedia.org/T114507#3031274 (10MarcoAurelio) [22:28:13] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Implement selective purging of model scores in varnish - https://phabricator.wikimedia.org/T148999#3031306 (10Tgr) Varnish coalescing is per-URL though so some thought would have to be put into the URL design - if two different URLs require the same backend work...