[03:16:17] Hm.. I suspect the 'reverted' model of nlwiki may be broken in some way [03:16:29] RTRC currently only uses the 'damaging' model when available [03:16:43] but since nlwiki doesn't have that, I considered trying 'reverted' if damaging is unavailable [03:17:04] but it seems to mark virtually all edits by IPs as reverted probality higher than 0.80 [03:38:56] Even simple edits like https://nl.wikipedia.org/w/index.php?diff=46265157&oldid=46254925&rcid=79206728 has reverted probability of 0.89 [03:55:30] Filed as https://phabricator.wikimedia.org/T129624 for now [14:31:33] o/ halfak [14:31:39] o/ Amir1 [14:31:47] Got those i18n changes merged. [14:32:04] yeah, I saw that [14:32:05] amazing [14:32:08] thanks [14:32:24] I'm almost done with getting the old linear_svc models updated [14:32:35] awesome [14:32:42] Once that's ready, we can update ORES to revscoring 1.0.x and deploy new models. [14:32:43] halfak: https://git.io/vassJ [14:33:31] yeas, [14:33:33] *yes [14:33:43] {{merged}} :) [14:33:43] 10[1] 04https://meta.wikimedia.org/wiki/Template:merged [14:33:44] Once you're done, I start to update wheels [14:33:48] thanks [14:34:00] We're just about at the point where we *could* update wheels. [14:34:15] I'll see what I can do for that today regardless of the linear_svc models. [14:34:54] awesome [14:35:22] is there any chance of making progress regarding deployment to prod? [14:53:07] Amir1, it seems like, on our side, we can unblock prod by getting this repo of wheels to a good state. [14:53:18] And then updating our deploy pattern to use that repo instead of pip. [14:55:35] yeah [14:55:39] I'm one it [15:00:49] Arg! The Linear SVC models take a really long time. For some reason, the fawiki model is the fastest of them all. [15:02:01] :D [15:22:51] OK. Now I'm using every core and most of the memory to build these stupid linear SVC models. [15:23:07] So hopefully they will finish soon [15:23:17] BTW, ores-compute-01 is going to be under heavy load for a while. [15:24:31] halfak: can I use ores-compute-02? [15:25:33] Sure! Want to start it up? [15:25:47] Or do we already have it? [15:26:19] I'm not sure [15:26:22] let me check [15:26:53] halfak: I can't connect, I think we need to start it [15:27:11] Amir1, you should be able to do that from the wikitech wiki. [15:27:13] Want a hand? [15:27:22] دo [15:27:24] *no [15:27:31] I do it, thanks :) [15:27:35] kk :) [15:28:02] You know... just thinking about what we do and do not let people do with permission. [15:28:18] E.g. we can start VMs, but we needed to be granted a project. [15:29:04] In Wikipedia, you only really need permission to do advanced curation actions. [15:29:28] In PAWS & Quarry, you need to register an account. [15:29:34] But everything else is open. [15:29:55] In Wiki labels, you have to do an awkward dance with me to get a campaign set up because we haven't built the infra yet. [15:30:06] But I really want that to be a no-permission-necessary thing. [15:30:17] that's why I'm working on translatewiki [15:30:21] to make more open [15:30:26] Good point. [15:35:47] halfak: Is "ores-experimental-01.revscoring.eqiad.wmflabs" as the same as "ores-compue-01.revscoring.eqiad.wmflabs"? [15:35:55] because I can't find them in revscoring project [15:36:05] https://wikitech.wikimedia.org/wiki/Special:NovaInstance [15:36:21] or should I add another instance? [15:36:28] Amir1, the compute server is in the ORES project [15:36:29] another project? [15:36:36] oh I see [15:36:40] :) [15:36:51] I didn't know I have access to ores project [15:36:52] cooool [15:37:03] But I set up ores-experimental in revscoring so that we could set up a mini-cluster for sabya's work if necessary [15:37:14] Amir1, with great power comes great responsibility ;) [15:37:33] Don't delete the load balancer ;) [15:37:41] Or the redis server. [15:37:54] But you can make a mistake with the web nodes or workers and that'll be OK [15:38:48] :)))))) [15:39:24] I try not to delete them but the temptation is overwhelming :D [15:40:12] halfak: I want to make the ores-compute-02 xlarge. is it okay? [15:40:22] Yes. [15:40:26] (it should be as the same as ores-compute-01) [15:40:32] sweat [15:40:56] eww. [15:40:59] lol [15:41:04] sweet != sweat [15:41:12] :))))) [15:41:16] sweet [15:41:18] neat [15:41:18] https://en.wikipedia.org/wiki/Perspiration [15:41:33] Oh! I should have used AsimovBot [15:41:38] I've got these two confused [15:41:40] [[:en:Sweat]] [15:41:40] 10[2] 04https://meta.wikimedia.org/wiki/:en:Sweat [15:42:13] oh, done [15:42:29] aaand I logged it [15:42:30] \o/ [15:42:44] Nice [15:46:47] yuvipanda: in order to install packages like scipy I need to install some system-wide applications like blas, etc. http://www.scipy.org/install.html Is it okay? [15:47:00] or should we make wheel for them too? and if yes how? [15:47:51] same goes for enchant dictionaries [16:16:25] building wheels for numpy and scipy is a huge PITA [16:18:12] + scikit-learn [16:41:10] o/ halfak [16:41:43] is there a syncup meeting tomorrow? [16:46:43] Hey sabya [16:46:54] There is. Would you like to join us? [16:47:19] would like to, what time it is? [16:47:19] Invite sent :) [16:47:26] 1500 UTC [16:48:25] Google Hangout, right? [17:01:32] sabya, yes. There's one attached to the calendar event. [17:01:55] We might end up using skype though if something goes wrong. [17:05:48] ok. it'll be Saturday evening, if I am not going out, I'll join in :-) [17:18:48] OK great! No worries if you can't make it. I'll be on IRC around the same time on Sunday too if that time works better to catch up. [17:18:53] sabya, ^ [17:33:23] halfak: can you release 0.2.3 for wikiclass? [17:33:34] Amir1, sure! [17:33:41] building wheels errored [17:33:53] thanks [17:38:40] tell me when it's released, thanks [17:38:53] halfak: ^ [17:42:18] Amir1, {{done}} [17:42:18] How cool, halfak! [17:54:07] thanks :) [17:56:17] gotta run to lunch [18:29:13] halAFK: wheels are working \o/ [18:29:34] python ores_wsgi.py works with wheels [18:40:12] Amir1, \o/ [18:40:16] Just setting up for talk [18:40:18] my AFK continues [18:42:05] awesome [18:42:08] break a leg [20:31:43] o/ [20:37:18] halfak: finally I finished this [20:38:05] the only thing I need to solve is the issue that even with wheels we need c dependencies installed [20:38:10] and that's strange [20:44:14] yuvipanda: hey, if you check this, I would really appreciate: https://phabricator.wikimedia.org/T129112#2113031? [20:50:35] sorry Amir1, going on vacation, I don't really have the time :( [20:50:51] Amir1: my understanding with halfak was that you /schana will take this up for the most part [20:51:18] yuvipanda: Sure, I won't bother you. I have one question and that's all. I promise asking anything for a very long time :D [20:51:43] *I promise not asking anything [20:52:01] Amir1, we should be able to get any dependency that comes via APT without issue. [20:52:17] oh, great [20:52:22] so it's good enough [20:52:37] I'm not sure how we're doing to download the NLTK data. [20:52:46] We might have to zip that up and put it into the wheel repo [20:52:58] I tested them and it's working [20:53:53] Even with stopwords and stuff? [20:54:04] Those aren't in the package. [20:54:08] They go to another directory. [20:54:11] https://github.com/wiki-ai/ores-wikimedia-config/blob/master/fabfile.py#L102 [20:54:35] Ahh yes. In Prod, we probably can't use that downloader though. [20:54:54] So we'll probably want to download, zip and add to wheel repo. We can re-extract when deploying [20:55:02] yeah, because it can't talk to the server [20:55:25] * halfak wishes yuvipanda an awesome vacation [20:55:30] :) [20:56:06] me too [20:56:09] :) [20:56:22] let me find a way for that [20:59:20] I'm imagining that we can put together a Makefile in the ores-wikimedia-config repo [20:59:31] That will have rules for "make wheels" [20:59:41] and "make nltk_data.bz2" [21:00:14] Oh. no. I suppose this would be in the wheel repo [21:00:26] yeah [21:01:09] Really getting that CPU utilization up on ores-compute-01 [21:01:14] halfak: it's okay to merge this: https://github.com/wiki-ai/ores-wikimedia-config/pull/43 [21:01:21] These SVC models are an awful pain. [21:01:27] * halfak checks [21:01:52] we should switch to something better soon (wrt svc) [21:02:42] ++ done [21:02:49] Amir1, yeah. We need a plan for that switch [21:02:51] awesome [21:02:56] It'll affect the extension and everything else. [21:02:59] thanks [21:06:05] wrt the extension, it would be a simple config change [21:06:15] it's flexible enough for this :) [21:07:36] halfak: can you see this change? https://gerrit.wikimedia.org/r/#/c/276310/ [21:07:53] it returns error to me [21:11:47] Yeah. I can't save a comment [21:11:58] Lots of errors [21:13:41] I can't even see it [21:13:52] and that's my commit for the repo [21:13:55] wheels [21:18:27] halfak: the repo is not accessible at all [21:18:39] it's the place we need to put our wheels [21:18:43] * halfak waits for gerrit to burn down [21:35:57] halfak: please +2 this before it becomes inaccessible again :D [21:36:02] https://gerrit.wikimedia.org/r/#/c/276310/ [21:40:22] {{done}} [21:40:22] How efficient, halfak! [21:40:27] Thanks AsimovBot [21:40:51] thanks [21:40:57] I'll add nltk files later [21:41:10] during fixing the fabfile.py [21:41:21] +1 sounds good [21:41:38] getting late there, right? [21:41:44] yeah [21:41:47] exactly [21:42:02] very productive day [21:42:28] i18n merged, ores-config merged, wheels merged and fixed [21:42:35] awesome [21:42:50] o/ [21:46:29] o/ [21:46:39] Agreed. Super productive day. [21:46:45] Hard to keep up with you these days ;) [21:46:49] halfak: since we don't have jenkins in this gerrit repo, you need to +2 verify and submit it [21:47:11] kk [21:47:12] Will do [21:47:16] thanks [21:47:43] gerrit is sometimes ... difficult [21:48:00] wrt. keeping up: I'm flattered :) [21:50:10] halfak: it's not merged yet, can you send me the screenshot [21:50:22] if it's okay [21:50:47] Amir1, what do you want a screenshot of? [21:50:59] this page: https://gerrit.wikimedia.org/r/#/c/276310/ [21:51:20] you need to click on submit instead of the default button [21:52:12] also it would be good if I can get +2 right on research/ores project [21:52:37] legoktm: hey, can you give me +2 right on research/ores in gerrit? [21:52:46] thanks [21:54:34] I realy need to go [21:54:37] *really [21:54:41] o/ [21:55:27] o/