[09:06:51] 10Jade, 10FileAnnotations, 10Growth-Team, 10MassMessage, and 23 others: Eventually deprecate SkinTemplateNavigation::SpecialPage and SkinTemplateNavigation hooks in favor of SkinTemplateNavigation::Universal - https://phabricator.wikimedia.org/T255319 (10ashley) [10:06:19] https://github.com/kubeflow/kfserving/blob/master/python/kfserving/kfserving/storage.py is interesting! [10:07:20] the main question that we had about where to store models is probably answered with Swift, hopefully it works fine (swift supports in theory the s3 api) [10:07:49] how to make some models available to the public is also interesting [10:15:56] "A tarball on an FTP server" probably isn't the right answer for that :D [10:42:13] ahahahah [10:42:52] something like commons may be really nice to use, but not sure if it is the right use case [11:14:59] Hi everyone! I am Leijie Wang, an undergraduate from Tsinghua University, Beijing, and currently collaborating with professors at Carnegie Mellon University for fairness-related research projects. [11:16:02] I am currently exploring how the flagging effect of ORES filters vary across communities as well as different kinds of articles. The key point of this is to obtain the ORES scores in the past when the revert decision was made. [11:16:03] I have asked Aaron Halfaker about this, and he pointed out to me the dataset of past ORES scores in english Wikipedia (see this link https://analytics.wikimedia.org/published/datasets/one-off/ores/scores_dumps/damaging_goodfaith_enwiki/) but now I am just confused about how to obtain thoses in other Wikipedia, russian for example. [11:16:32] I would really appreciate it if you could give me some advice about this problem. [12:26:49] LeijieWang: hi! the dataset IIRC was created as one off as part of https://phabricator.wikimedia.org/T277609 [12:28:55] Thanks for your reply, elukey! I’ve looked into the dump you shared but it seems to only include english wikipedia. Are there similar datasets for other wikis? [12:35:53] LeijieWang: for the moment no, it was done as part of a one-off request for that particular task [12:36:45] if you need a few other similar dumps it may be good to contact the Analytics team, opening a similar task is fine [12:37:31] (it requires a phabricator account etc..) [12:38:41] otherwise you can try to reach out to the team via https://lists.wikimedia.org/mailman/listinfo/analytics [12:38:44] LeijieWang: --^ [12:48:52] I see. That is really helpful. Thanks a lot for your advice! [13:46:33] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks): Find a way to store models for Kubeflow - https://phabricator.wikimedia.org/T280025 (10elukey) [13:47:29] created --^ as follow up from the earlier conversation about models, lemme know your thoughts :) [13:47:45] (it seems something to think about even for the MVP use case) [16:42:54] elukey: thanks for making that task! i agree swift should work in theory. It will be really nice to get away from git-lfs [16:43:26] accraze: morning! I know that you secretly love git-lfs [16:43:32] LOL [16:44:10] having docker image pipelines + model storage pipelines etc.. may become a little weird in the beginning [16:44:36] I think the flow should be 1) push the new model to swift 2) build the new docker image that uses it 3) push it to k8s [16:45:00] hopefully this will not be as complicated as the current deployment flow :) [16:46:19] yeah i think it will become more clear once we have Train Wing to push a newly trained model to swift and then have that trigger a new image build & deploy [16:47:15] and there is also the bit of making the public models available to the community [16:47:21] (maybe automatic push to commons?) [16:47:30] ^ this would be really cool [16:50:20] im not very familiar with swift, but is some sort of routing mechanism that we can make things private or public there? [16:50:38] good qs, this part is not clear to me either [16:50:57] I think that we could investigate this in parallel with the istio/knative/etc.. madness [16:54:10] I mean not ending up with everything completely polished etc.. for the MVP, but at least some initial scaffolding [16:56:13] haha yeah, i imagine this will be iterative, but it's smart to start thinking about what a good model registry would look like for us [16:59:37] 10ORES, 10artificial-intelligence, 10articlequality-modeling, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): ORES deployment - Spring 2021 - https://phabricator.wikimedia.org/T278723 (10Halfak) Deploy failed with the following error: ` halfak@deployment-deploy01:/srv/deployment/ores/dep... [17:01:54] accraze can you take a look? It is late for Kevin [17:03:07] yeah i can dig around [17:03:48] accraze, I'm in -releng [17:04:02] If you could take over that would be great. I need ot go back to my day job [17:23:40] I feel that this is an issue that happens from time to time, I have seen in countless time on phab for deployment-prep [17:26:16] accraze: /etc/ssh/ssh_known_hosts seems to have the correct key [17:26:19] Interesting! elukey, does it tend to work if you just retry? [17:26:29] on deploy01 I mean [17:26:41] halfak: I am looking on phab to see if there are known recipes [17:28:48] OK really leaving now. Thanks for looking elukey. [17:29:09] accraze: mind if I try to deploy? [17:29:24] elukey: go for it [17:30:08] ack doing so [17:32:11] accraze: worked :) [17:33:09] \o/ [17:33:13] did you do something? I retried at some point and it didn't work [17:33:28] 10ORES, 10artificial-intelligence, 10articlequality-modeling, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): ORES deployment - Spring 2021 - https://phabricator.wikimedia.org/T278723 (10elukey) After checking the ssh known hosts on deployment-ores01 without finding anything weird, I tried... [17:35:24] Majavah: not really, just tried to ssh with my user to ores01 to manually accept the fingerprint [17:35:43] I checked on hal*AFK's ssh known hosts and there was the correct one as well [17:35:57] and the main /etc/ssh/ssh_known_hosts had the right one as well [17:36:06] so.. no idea :) [17:36:36] weird [17:37:08] if you look for the same error on phab there are a ton of occurrences, but IIRC in the past it was some misconfigured known_hosts somewhere causing this [17:37:29] no idea what happened this time (misteries of deployment-prep :D) [17:37:59] anyway, going afk! ttl :) [17:38:21] cya elukey [17:57:14] 10Machine-Learning-Team, 10artificial-intelligence, 10editquality-modeling, 10Turkish-Sites: Update Turkish Wikipedia's labeling campaign for 2020 - https://phabricator.wikimedia.org/T257359 (10Halfak) This change will look a lot like this work for ptwiki: https://github.com/wikimedia/editquality/pull/225/... [18:04:30] 10ORES, 10artificial-intelligence, 10articlequality-modeling, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): ORES deployment - Spring 2021 - https://phabricator.wikimedia.org/T278723 (10Andrew) > deployment-ores01.deployment-prep.eqiad.wmflabs this may or may not be the issue but I'd enc... [18:57:50] I remember how good this article is http://karpathy.github.io/2019/04/25/recipe/ [21:27:45] > 1. Become one with the data [21:28:27] so true, gotta fully understand the data inside-out first [21:28:37] great article chrisalbon [21:36:37] I think there is alot we can do around making our datasets more transparent and easier to understand for non-experts [21:39:12] i've been seeing stuff about datacards/datasheets alot lately, similar idea to model cards but with a focus on datasets. [21:39:32] datacards playbook looks interesting: https://pair-code.github.io/datacardsplaybook/