[00:13:33] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, 10Patch-For-Review: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3935879 (10mmodell) [00:20:04] ooh :) [14:26:23] 10Scoring-platform-team (Current): Host Google-News-word2vec.bin publicly - https://phabricator.wikimedia.org/T185147#3907561 (10awight) I found a few existing mirrors: * https://github.com/mmihaltz/word2vec-GoogleNews-vectors (requires git-lfs) * https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-ne... [14:49:07] 10Scoring-platform-team (Current), 10Beta-Cluster-Infrastructure, 10Recommendation-API, 10Release-Engineering-Team, 10Patch-For-Review: What to do with deployment-sca03? - https://phabricator.wikimedia.org/T184501#3885749 (10awight) @mobrovac @Ottomata Great, thanks for the confirmation! I searched oper... [14:54:06] 10Scoring-platform-team, 10JADE, 10Design: Design conceptual prototype of JADE integration with MediaWiki - https://phabricator.wikimedia.org/T182829#3937670 (10awight) Still getting ahead of myself here, but I've asked for a Gerrit clone for Extension:JADE. [14:54:29] 10Scoring-platform-team (Current), 10JADE, 10MediaWiki-Vagrant: Vagrant role for JADE - https://phabricator.wikimedia.org/T182055#3937671 (10awight) [15:04:10] o/ [15:04:43] I'm doing some annual planning stuff this morning. I'm writing a short essay on "human tech" so that I can communicate better with vcoleman about it. [15:29:13] halfak: ^ don’t watch… I’m sneaking JADE things into the workboard [15:29:24] awight, good deal. Keep going :) [15:29:28] ;-) [15:29:39] I want to get our ORES cluster running before I join you though. [15:30:00] +1. I’m starting with a vagrant role so we can all try the eventbus stuff [15:40:25] halfak: Have you poked at Python’s new coroutine support? [15:40:45] Nope. Haven't looked at it at all. [15:40:57] It might be useful for API extraction, donno. [15:41:06] True parallelization? [15:41:09] https://docs.python.org/3/library/asyncio.html [15:41:20] Well, sorta better. [15:41:42] I thought async IO wasn't parallel [15:41:55] "threads" [15:41:59] I don’t think it overcomes the GIL, but allows us to do IO stuff like crazy, with just syntactic sugar and no Celery fooling around. [15:42:15] Hmm.. Not sure we're limited on IO [15:42:18] IO happens outside of celery [15:43:10] Oh, I thought it was one of the stages in a scoring pipeline? [15:43:57] https://docs.python.org/3/library/asyncio-subprocess.html apparently this lib does include heavy process support, but we might as well stick with Celery for that, unless message-passing starts to wear on us [15:45:05] Anyway, something to keep an eye on IMO. We could obfuscate our code as unreadably as Javascript ^_^ [15:47:46] lol @ obfuscatey JS [15:51:05] It’s madness. I like that the JS world has spent the last decade building a seemingly intentional, impenetrable hell of callbacks, and then inventing yet more glue so they don’t have to see the callbacks. [15:54:14] halfak: Anything I can pick up to help with migrating to the new cluster? [15:57:52] awight, if you want to start the deploys to the new cluster, that'd be great. I just finished my "human tech" essay. I'll post it to Meta shortly. :) [15:58:26] I've got a sequence of meetings today about planning stuff out for next year but I think I'll join you on cluster stuff tomorrow. [15:59:25] Can do! [16:10:14] 10Scoring-platform-team (Current), 10Beta-Cluster-Infrastructure, 10Recommendation-API, 10Release-Engineering-Team, 10Patch-For-Review: What to do with deployment-sca03? - https://phabricator.wikimedia.org/T184501#3937894 (10Ottomata) > I couldn't find the configuration for the Kafka "jumbo" cluster in b... [16:26:08] halfak: Do you already have an entry point for running the JADE service? [16:26:25] wat [16:26:52] I see a wsgi server, but nothing pointing at it [16:27:02] Have you run it from the commandline, somehow? [16:27:19] Oh yeah. I see what you mean. That's not set up yet [16:27:27] K cool [16:27:28] You should add something like applications/wsgi.py in ORES [16:27:31] :) [16:27:32] +1 [16:27:39] I’m mostly copy-pasting [17:05:31] 10Scoring-platform-team, 10ORES, 10Epic: Migrate Scoring services to Debian Stretch - https://phabricator.wikimedia.org/T186238#3938116 (10awight) [17:06:32] 10Scoring-platform-team, 10ORES, 10Epic: Migrate Scoring services to Debian Stretch - https://phabricator.wikimedia.org/T186238#3938126 (10awight) [17:06:34] 10Scoring-platform-team (Current), 10ORES, 10Patch-For-Review: Make sure ORES is compatible with stretch - https://phabricator.wikimedia.org/T182799#3938127 (10awight) [17:07:41] 10Scoring-platform-team, 10ORES, 10Continuous-Integration-Config: Migrate ORES CI to Stretch - https://phabricator.wikimedia.org/T186239#3938129 (10awight) [19:08:00] halfak: I have the vagrant role working, and there’s a PR for JADE with enough infrastructure to launch the server (and crash due to unimplemented stuff, of course) [19:08:38] Amir1: Is there some trick with the IRC user for GitHub notifications? Looks like it’s “ores” with a password? [19:11:17] merged [19:11:25] Heading out for lunch [19:11:27] Back in a bit. [19:15:04] nice! [19:44:41] 10Scoring-platform-team, 10AbuseFilter, 10ORES: Investigate training ORES with AbuseFilter conditions - https://phabricator.wikimedia.org/T186256#3938852 (10dbarratt) [19:49:21] 10Scoring-platform-team, 10AbuseFilter, 10ORES: Investigate training ORES with AbuseFilter conditions - https://phabricator.wikimedia.org/T186256#3938897 (10TBolliger) [21:39:43] does the ORES API return an article quality score? the support table here https://www.mediawiki.org/wiki/ORES indicates yes, but actually hitting the api does not return `wp10` [21:40:44] norch: Hi! What language are you working with? wp10 only exists for enwiki, so far. [21:42:34] Oh really? is the support table just preemptively optimistic? it says there is wp10 coverage for enwiki, frwiki, ruwiki, trwiki, and wikidatawiki [21:43:22] awight, we support all those other languages :P [21:43:36] oh dear. [21:43:46] halfak: Thanks for catching me before I misinform again [21:45:58] for example, https://ores.wmflabs.org/v3/scores/wikidatawiki/622250285/ only returns damaging, goodfaith, and itemquality. those are all scores for the edit. is there a way to get wp10? [21:46:13] norch, itemquality ~~ wp10 [21:47:16] oh, great [21:48:16] thanks - why itemquality property name, not wp10? [21:48:37] https://www.wikidata.org/wiki/Wikidata:Item_quality [21:48:52] https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_1.0 [21:49:02] https://fr.wikipedia.org/wiki/Projet:Wikip%C3%A9dia_1.0 [21:51:18] etc. :) [21:51:36] ha, cool. thanks for the background. [21:53:49] halfak: fwiw, those checkmarks point somewhere cool, which doesn’t exist yet. [21:54:06] Also, I changed the wikidata / wp10 cell to “itemquality” for future mass confusion [21:57:26] nice [21:59:49] fmc is great [22:03:36] * awight dials the chaos to x_k+1 [23:39:46] OK done for the day. Will hopefully be working on cluster stuff with awight tomorrow.