[04:18:47] halfak: ^ [04:19:51] yuvipanda: re. https://phabricator.wikimedia.org/T129112. I built all wheels. Can you make a gerrit repo and then I commit? [08:24:33] yuvipanda: hey, around? [15:12:23] o/ halfak [15:12:25] ^ [15:12:32] I finally fixed the i18n [15:12:39] Amir1, been looking at it. [15:12:53] I don't like the nasty hack but I tried everything else [15:13:02] Which nasty hack? [15:16:35] Amir1, leaving some thoughts on the forms i18n. These thoughts are loose and you should feel free to push back. [15:16:51] [https://github.com/wiki-ai/wikilabels/commit/7241772613a4815b92db465812c23ce66cf527cf#diff-09ad181be69d94256ccddafe411b7bd7R42 [15:17:01] cool [15:17:06] Also check PR 98 [15:19:02] Nice to have it wrapped up in a function. [15:19:13] I like the use of json.dumps to make sure we get valid JS too. [15:21:30] Oh wait. I see. You didn't fix the js file yet [15:21:54] halfak: most importantly please check https://phabricator.wikimedia.org/T129112#2102183 [15:22:04] Can't check it all at once. [15:22:08] Slow down :P [15:22:15] :D [15:22:16] sure [15:22:32] I wonder if the old `ores` package included `revscoring` as a submodule. [15:22:37] That was an issue we had before. [15:23:02] several external packages have this issue so I guess it's not the problem [15:23:17] it's not just ores and revscoring [15:23:47] Amir1, I'm getting close to giving up on the old SVC models. [15:23:51] They are finicky. [15:24:06] :D [15:24:16] But FWIW, we got roughly the same AUC out of them as we get out of the GradientBoosting classifier. [15:24:37] So, we're not really making a meaningful improvement :/ [15:24:49] :( [15:25:14] If I get these things done [15:25:20] I will come and help you [15:25:44] I think that, in order to get the SVC models working as expected, I'll need to do some feature tuning [15:25:48] Which is slow and tedious. [15:42:26] Amir1, I think that those missing model errors are too early in the loading process to know if all dependencies were met. [15:43:10] You need to do "git submodule update --init --recursive" [15:43:15] In ores-wikimedia-config [15:43:19] To pull down the submodules. [15:44:08] I get it [15:48:33] halfak: re. PR 96. it's entangled with another PR in wikilabels-wikimedia-config. they should be merged and deployed at the same time [15:48:55] link to that PR? [15:50:22] sure [15:50:58] https://github.com/wiki-ai/wikilabels-wikimedia-config/pull/12 [15:51:03] halfak: ^ [15:51:49] OK. I left some notes on the wikilabels PR [15:51:52] Thoughts on that? [15:52:13] Oh I see your notes. [16:00:06] Fixed that part [16:01:50] ^ [16:48:03] halfak: https://usercontent.irccloud-cdn.com/file/PhjdqIrC/ [16:48:36] Amir1, yes. The bug is that we're trying to open() that URL [16:49:52] We don't actually use FormBuilder.js, so we could just discard the route entirely. [16:50:17] because for strange reasons "//" doesn't work with the code [16:50:26] just changing it to https:// [16:50:42] make the system aware that it should read URL not open it [16:50:47] Amir1, so, I'm amazed that we can open() a URL at all! [16:50:59] Amir1, IMO, dropping the route is the best option. [16:51:07] Why maintain something we don't use? [16:51:36] okay [16:51:43] It's totally okay for me [17:01:24] halfak: https://github.com/wiki-ai/wikilabels/commit/28a012023467a989adf257201ee480b3fa8fda48#commitcomment-16594753 [17:02:06] Changed your mind from what to what? [17:02:22] Also, the keys are still form_name, not file_name. [17:05:49] IMO, we should drop the form_name field and use the file_name minus the extension. [17:06:01] That way, we can have the file names match the i18n folder [17:07:05] it is [17:07:29] because d['name'] is as the same as file name [17:07:50] aand we attach these i18n to form_map keys [17:08:05] Well, the file_name has an extension [17:08:24] do you mean .yaml? [17:08:44] Right now, we could have a firm config named "foo.yaml" with "name: bar" [17:09:10] It's only convention that keeps the name the same as the filename - extension. [17:09:16] I think we should just *use* the filename [17:10:16] I know, it would be good but the problem is that it would be hard for us to amend these data to form_map [17:10:22] see inside the for loop [17:10:36] form_map[file_name]['i18n'] = ( [17:11:00] that's why I'm doing it [17:11:30] But the variable you are calling "file_name" is actually the form name [17:11:36] yeah [17:11:38] The file_name includes an extension [17:11:44] I changed it to "campaing_name" [17:11:54] But campaigns are different things entirely [17:12:31] form_map keys are "edit_quality", "edit_type" and "notability_draft" [17:12:58] okay [17:13:02] I change it to form_name [17:14:07] I'm happy I can just amend patches instead of making new commits every time [17:14:21] OK. So we still have the problem that a form could be called "foo.py" with "name: bar" and then it's i18n would be in "bar/" [17:14:38] *foo.yaml [17:15:12] Sorry to be picky, but either we address these issues now, or they'll be confusing as hell later. [17:15:22] halfak: I changed it to form_name [17:15:38] form would be called foo.py [17:15:46] *foo.yaml [17:15:48] Yeah. Saw that. The last problem I noted still remains. [17:15:59] halfak: I see that probelm [17:16:02] kk [17:16:15] but the issue is if we want to solve these, it would be really hard [17:16:40] because we are adding these data to form_map [17:16:56] and form_map is based on name, not file_name [17:17:08] so form_map is {'bar': etc.} [17:18:20] if we want to know which file contains which name, it would PITA [17:18:21] *would be [17:18:34] progromatically [17:18:51] Yes. So let's just use the file name, strip off the extension and then call *that* the form_name. [17:19:13] And remove "name: whatever" from the YAML. [17:19:21] Or just ignore it. [17:19:39] hmm, [17:19:41] okay on it [17:21:53] Amir1, https://gist.github.com/halfak/5d3265bd1b55accc5e2d [17:22:00] ^ a nice way to remove the extension [17:22:22] hmm [17:22:24] awesome [17:33:26] I'm heading out for lunch. [17:33:31] Back in 30-45 mins [17:39:24] wiki-ai/revscoring#567 (fix_repeated_char_bug - 9e75ab9 : halfak): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/114849879 [18:01:20] :D [18:20:28] back [18:26:15] wiki-ai/revscoring#570 (balanced_sample - 1cb600f : halfak): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/114861138 [18:30:00] o/ Amir1|ZzzZ [18:30:16] I still see sklearn 0.17.1 in https://github.com/wiki-ai/ores-wikimedia-config/pull/43/files [18:30:41] We can use the more recent version, but that means we need to wait to test this until I can get ORES updated for new revscoring. [18:34:44] o/ yuvipanda [18:35:07] * halfak takes his Q to -labs [19:09:56] couldn't sleep [19:11:14] halfak: I think we are good to go for i18n :D [19:11:29] Amir1|ZzzZ, OK. [19:11:51] Will try to get to that by EOD. I think that we're far enough along that I can fix any minor issues I find when merging. [19:12:28] "When you have insomnia, you're never really asleep... and you're never really awake." [19:12:33] https://en.wikiquote.org/wiki/Fight_Club_(film) [19:12:54] halfak: awesomme [19:12:56] *awesome [19:13:10] Amir1|ZzzZ, Wondering about your thoughts re. sklearn 0.17.1 [19:13:30] The version of packages you built wheels with needs 0.15.2 [19:13:38] *but* new revscoring will need 0.17.1 [19:13:57] we can bump everything to newer version [19:14:03] revscoring to 1.0.1 [19:14:11] skrean to 0.17.1 [19:14:17] I'm trying my damnedest to get revscoring 1.0 out the door today. [19:14:28] awesome [19:14:38] I really want the models to be in good shape for a deploy -- hence my backtracking to SVC models. [19:14:42] I would help on merging things, etc. when I wake up [19:14:58] I merged the balancing thing [19:15:05] No worries. Just thinking that, if we're aiming for revscoring 1.0.x, it will delay the merge of the wheels PR. [19:15:14] *requirements.txt for wheels PR [19:16:31] if it was blocking other things and everyone else was waiting for that, I would say. let's merge it then we update the wheels [19:16:53] but I asked yuvipanda to make a repo to put the wheels, no response :( [19:17:14] been running around and rushing :( [19:17:38] also how did you create the wheels? should be done on a labs debian machine [19:17:45] it's okay. but please tell me you're busy and I step away as long as you want [19:17:55] yuvipanda: yes, I did that [19:17:56] I only saw your messages this morning :) [19:18:16] yuvipanda: check this picture: https://phabricator.wikimedia.org/T129112 [19:18:22] * yuvipanda has gotten better about not checking IRC later at night [19:18:36] thank you :) [19:19:34] I let a comment [19:19:36] *left [19:19:56] thanks [19:20:01] yuvipanda, options are (1) run with the versions we have deployed now or (2) wait 24-48 to start building the full set of wheels and experimenting with deploying them. [19:20:17] Either way, I won't be able to experiment with deploying them for 24-48 hours. [19:20:37] yuvipanda: I did with pip freeze [19:20:39] Gotta get these versions in order. It's become a really heavy plate to keep spinning. [19:20:47] fun [19:20:49] but I had delete two or three of them [19:20:50] you guys decide :D [19:21:08] Amir1|ZzzZ: right, so need to figure out why and fix that :) where did you run pip freeze? [19:21:12] I think that focusing on aligning the versions is a good next step before wheels. [19:21:13] because of a probable bug [19:21:30] in upstream [19:21:50] that can't handle conflicts properly [19:21:51] yuvipanda, is there anything that can be done about other python in prod concerns? [19:21:57] ^ in the meantime [19:22:00] It's merged but not released yet [19:22:01] I think wheels are the biggest ones [19:22:06] kk [19:22:10] Amir1|ZzzZ: we'll have to use whatever version of pip exists in debian anyway [19:22:16] Oh! Do we have a repo to put wheels in yet? [19:22:22] no [19:22:24] let me create one [19:22:27] Thanks [19:22:36] I think we decided on research/ores, right? [19:22:49] Maybe research/ores-packages [19:23:53] I created research/ores/wheels [19:23:58] Nice [19:23:59] k [19:24:16] Can we access these repos thought phab yet? [19:24:20] Amir1|ZzzZ: ^ [19:24:21] halfak: no idea [19:24:37] awesome thanks [19:24:57] yuvipanda: that's the point re. using wheels [19:25:06] https://phabricator.wikimedia.org/diffusion/ [19:25:22] I made an entire new virtualenv [19:25:26] Seems like the right place, but I'm not sure how to find the repo. [19:25:29] mostly nobody wants to maintain the debs :) [19:25:39] and also that in prod you can't reach out to pypi [19:25:47] or the internet, in general [19:25:48] https://phabricator.wikimedia.org/diffusion/EORS/ [19:25:50] and tried to use it [19:26:03] I see the extension repo [19:26:07] yuvipanda: one thing: I put all wheels needed there [19:26:21] try to install it [19:26:23] am going to check out of this for a bit now, sorry! [19:26:31] juggling too many things [19:26:35] no worries. Thansk for your help yuvipanda [19:26:36] o/ [19:26:43] thanks [19:26:45] :) [19:27:26] np :) and thanks for your work, Amir1|ZzzZ [19:27:50] I hope to do something useful :) [19:28:02] not duplicating the work, reducingit [19:43:17] halfak: I'm testing on wheels [19:43:32] Great [19:43:32] the dependencies situation is too complicated [20:50:19] https://github.com/wiki-ai/ores/issues/128 [21:30:55] yuvipanda: I want to make a patch but it gives me error, saying there is no .gitreview file [21:31:14] Amir1|ZzzZ, there's some docs on this [21:31:15] * halfak looks [21:31:32] thanks [21:31:34] :) [21:31:48] https://www.mediawiki.org/wiki/Gerrit/git-review [21:31:54] git-review -s ? [21:31:59] That might be what you need. [21:32:23] didn't work [21:32:34] What version of "didn't work"? [21:34:13] the same error [21:34:22] But git-review -s worked? [21:34:35] I think it's time for a paste :) [21:35:08] https://www.irccloud.com/pastebin/tgJKnu1y/ [21:35:12] none of them worked [21:35:18] the same error [21:35:55] Oh! Manually create the gerrit remote? [21:36:16] git config --global gitreview.remote origin [21:36:36] https://www.mediawiki.org/wiki/Gerrit/Tutorial#Configuring_git-review [21:36:41] back in 15 [21:38:39] it should fixed it but it works and git review -s returns the same erro [21:38:42] *error [21:40:37] ok, using virtualenv it's working [21:43:28] https://gerrit.wikimedia.org/r/276310 [21:43:37] yuvipanda: halfak please test them [21:43:55] I might take a look at it [21:43:59] tomorrow [21:44:05] but I now go to sleep [21:44:12] o/ [21:44:47] o/ [23:33:06] IT WORKS! [23:33:18] You just need to let the darn SVC go through a billion iterations! [23:34:09] We get way better PR-AUC with the GradientBoosting (for the few models I've looked at) [23:34:12] So that's good :)