[14:29:19] * halfak tries fixing the ores deployment. Looks like we'll need to execute the command in docker. [14:32:46] Huh. Still. "Could not restore untracked files from stash entry" [14:45:20] wikimedia/ores#1409 (master - dc363e4 : halfak): The build has errored. https://travis-ci.org/wikimedia/ores/builds/655822726 [14:56:01] * halfak kicks travis. [14:56:05] C'mon. Go again. [14:56:16] I think I might be onto something. [15:24:43] wikimedia/ores#1409 (master - dc363e4 : halfak): The build has errored. https://travis-ci.org/wikimedia/ores/builds/655822726 [15:28:38] aha! Figured out how to trigger a build. [15:28:48] (on the new commit) [15:52:19] * halfak learns how to pass envvars via docker-compose. [15:56:48] Aha. So travis is using an old version that doesn't support passing envvars in exec. I must need to get them in another way. [16:02:05] Woops. Na. I got the order wrong. [16:06:32] "Setting environment for exec is not supported in API < 1.25 (1.24)" [16:06:33] lol [16:06:34] wat [16:06:41] :'( [16:17:55] Luckily this archaic version is a known issue and there's a decent process for updating it. [16:39:15] Stuck on "job received" -- they might be throttling me :\ [16:51:22] akosiaris, do you have any updates for https://phabricator.wikimedia.org/T242705 based on tests with gunicorn? [16:51:47] halfak: I don't think I have the time to do such tests unfortunately. [16:52:23] with the OKR reprioritization in Service Ops regarding the outages we had I have a huge backlog to serve now [16:52:28] Gotcha. Is the time crunch a short term issue or a long term one? [16:52:33] and it's only growing [16:52:34] Oh gotcha. So maybe medium term? [16:52:39] Oh then long term :( [16:52:44] at the very least medium term. [16:52:57] it does look like it's getting better, but the entire team is backlogged now [16:53:04] Godspeed. I'll try to see what I can work out on my side. [17:21:25] accraze, https://github.com/wikimedia/ores/commit/791fd3ef9fd3cf425055f25827946dd346ae1d17 [17:27:33] hmm weird it looks like the docker-compose version isn't getting updated [17:34:45] you might be able to just do `docker run` instead of docker-compose for the deploy stuff [17:35:40] but that is really weird that travis shows docker-compose v1.25.4 in the before_install section, but during the deploy section it says v1.24 [17:38:16] accraze, any thoughts on better ways to get those envvars into the right place? I was thinking that we could set up basic python on the main VM in order to do the deploy. [17:38:26] But that feels like DRY [17:38:52] I also considered adding the envvars when building the images, but that won't work outside of master so we'd have to somehow set that conditionally. [17:41:59] taking a look at the repo to understand whats going on real quick [17:47:57] halfak -- one thing you could try is adding the environment variables to the ores-worker container in the docker-compose.yml file [17:48:13] like this: https://github.com/wikimedia/ores/blob/master/docker-compose.yml#L24 [17:48:23] accraze, it'll get added on non-master branches if we do. [17:48:44] And that could result in weirdness. Or maybe they'll just be blank? [17:49:01] yeah those env vars only should be present on master [17:49:23] so they would most likely be blank on dev branches [17:50:22] oh wait [17:50:32] deploy only gets called on the master branch [17:50:40] https://github.com/wikimedia/ores/blob/master/.travis.yml#L43 [17:51:20] Right but if I put it in docker-compose, it'll try adding it every time. Blank could be fine. [17:53:19] yeah that should be fine [17:55:44] OK giving it a try. [18:01:21] oh i meant removing the `-e PYPI_USER=$PYPI_USER` stuff in the docker-compose exec command and then adding `- PYPI_USER=$PYPI_USER` in the yaml config [18:01:46] Yeah. I missed the -e args :| [18:01:56] But I did add it in the docker-compose.yaml [18:04:55] wikimedia/ores#1422 (master - 8d0af6d : halfak): The build passed. https://travis-ci.org/wikimedia/ores/builds/655901360 [18:04:58] YES [18:05:09] OK now to actually release the new version :) [18:05:09] wikimedia/ores#1423 (master - 8d0af6d : halfak): The build passed. https://travis-ci.org/wikimedia/ores/builds/655901440 [18:11:14] nice one halfak [18:14:24] just wired up libraries.io to have scoring-internal report new releases of ores to us [18:14:35] Nice! Thanks. [18:34:47] halfak: i've setup irc bouncer so i'll not miss any messages here even when away :) [18:35:29] Nice! Say, would you want to get on a call some time to chat about updates we have made to your work on the topic model? [18:35:37] (only if it's useful for you) [18:36:24] i'd be happy to do that! i'm away next week, i can find sometime on your calendar next to next week for the same, that works? [18:39:57] Sure! [18:40:10] I think it'd take us about 30 minutes for an overview and a chat. [18:40:28] I think some of the stuff we learned will be useful for your new project :) [18:45:12] AFK for food [18:59:23] groceryheist: i was asked about data on ORES scores for anonymous editors. i was able to find your work on bias against editor types (https://meta.wikimedia.org/wiki/Research:Exploring_systematic_bias_in_ORES/Calibration_and_balance_for_newcomers_and_anons) but was wondering if you had descriptive stats for the ORES outputs anywhere -- i.e. percentage of edits deemed bad faith or damaging by ORES for anons, new editors, long-term editors [19:03:57] okay thats nice! i'll setup a 30min slot in 2nd week of march [19:04:15] hmnmm...maybe i answered my own question based on mako et al's excellent summary (https://meta.wikimedia.org/wiki/User:Benjamin_Mako_Hill/Research_on_the_value_of_IP_Editing): https://mako.cc/academic/tran_etal-tor_users_wikipedia-DRAFT.pdf [19:04:28] but if you know of more data, i'd love to hear about it! [19:09:16] 10Scoring-platform-team, 10Discovery-Search, 10NewcomerTasks 1.1, 10Growth-Team (Current Sprint): Once the ORES drafttopic - ElasticSearch pipeline is set up, update data about all articles - https://phabricator.wikimedia.org/T243357 (10EBernhardson) A manual all-pages export has been run against arwiki, c... [19:19:44] issacj: I haven't got those exact numbers handy. But historical data on ores scores for revisions is available in the data lake at event.mediawiki_revision_score [19:43:44] groceryheist: gotcha -- thanks for the pointer! i'll pass along [19:49:09] isaacj: i understand that there was a project to build and release a public version of the dataset. I'm interested in seeing that accomplished! [19:50:50] i think that project was at least semi-abandoned [19:54:53] 10ORES, 10Scoring-platform-team, 10Graphite, 10good first task: Look at additional uWSGI metrics for potential use in the ORES dashboard - https://phabricator.wikimedia.org/T182915 (10Chtnnh) Hello @awight, I am a new contributor to wikimedia. I would love to help out with this issue. Could you point me in... [20:22:24] groceryheist: yeah, in general i would greatly support that too (dumps of ORES scores) as it enables a lot of fascinating questions about the effect/evolution of algorithmic tools on Wikipedia/Wikidata. i'll do my best to push for it where i see the opportunity though don't have any insight into what the blocker is [20:24:34] isaacj: thank you [20:33:09] 10Scoring-platform-team, 10drafttopic-modeling, 10Documentation, 10good first task: Add topic information to the ores-support-checklist - https://phabricator.wikimedia.org/T245068 (10Chtnnh) @srishakatux @Halfak Would you like to collaborate on fixing this issue? I am a new contributor to wikimedia and wou... [20:39:23] 10Scoring-platform-team, 10artificial-intelligence, 10good first task: Text complexity scoring - https://phabricator.wikimedia.org/T155843 (10Chtnnh) Hello @Halfak. I am a new contributor to wikimedia and would love to help out with this issue. What are the restrictions on the model to be used to detect comp... [20:41:47] 10ORES, 10Scoring-platform-team, 10good first task: Request raw features/datasources from ORES - https://phabricator.wikimedia.org/T133585 (10Chtnnh) Hello @Halfak, I seem to be a little lost without context here. Could you point me in the right direction? The link given seems to be broken. Thanks, Chaitanya [20:49:58] 10Jade, 10Scoring-platform-team (Current), 10MW-1.35-notes (1.35.0-wmf.22; 2020-03-03): Address Jade UI issues. - https://phabricator.wikimedia.org/T245311 (10ACraze) [20:51:44] Hello halAFK [20:52:16] From our previous discussion, you said "For weird reasons, lambdas don't work as process functions." [20:52:48] 10Jade, 10Scoring-platform-team (Current), 10MW-1.35-notes (1.35.0-wmf.22; 2020-03-03): Address Jade UI issues. - https://phabricator.wikimedia.org/T245311 (10ACraze) Ok, I believe these issues have now been addressed. All patchsets have been merged to master and should be live on beta. [20:53:09] Does this mean at some point the lambdas won't work? because at the moment they are able to detect the patterns as I want. [23:58:26] 10Scoring-platform-team, 10Discovery-Search, 10Growth Design, 10Growth-Team (Current Sprint), 10Patch-For-Review: Newcomer tasks: UX changes for ORES topics - https://phabricator.wikimedia.org/T244421 (10MMiller_WMF)