[00:29:20] <wikibugs>	 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#4122111 (10awight) Still finding strangeness...  Reading the virtualenv so...
[00:45:06] <wikibugs>	 10Scoring-platform-team, 10ORES: mwparserfromhell ParserError on Premier League - https://phabricator.wikimedia.org/T191861#4122119 (10Platonides) [[https://ores.wikimedia.org/v3/scores/eswiki/?models=damaging&revids=106943027|Another instance]]: https://es.wikipedia.org/w/index.php?diff=106943027
[00:45:07] <AsimovBot>	 10[1] 04https://meta.wikimedia.org/wiki/https://ores.wikimedia.org/v3/scores/eswiki/%3Fmodels%3Ddamaging%26revids%3D106943027
[00:50:48] <wikibugs>	 10Scoring-platform-team, 10ORES: mwparserfromhell ParserError on Premier League - https://phabricator.wikimedia.org/T191861#4118999 (10Legoktm) ```lang=python import requests, mwparserfromhell as mwp  print('mwph: %s' % mwp.__version__) r = requests.get('https://es.wikipedia.org/wiki/?oldid=106888140&action=ra...
[01:51:06] <wikibugs>	 10Scoring-platform-team, 10ORES: mwparserfromhell ParserError on Premier League - https://phabricator.wikimedia.org/T191861#4122162 (10Earwig) This was fixed in mwparserfromhell v0.5 (latest stable is 0.5.1, this bug existed in versions 0.4.4 and earlier). Please upgrade.
[06:44:57] <wikibugs>	 10Scoring-platform-team, 10MediaWiki-extensions-ORES, 10WMSE-Bug-Reporting-and-Translation-2018, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): The filters menu is closed when a filter colour is selected - https://phabricator.wikimedia.org/T188903#4122391 (10Sebastian_Berlin-WMSE) You're right, th...
[07:17:10] <icinga2-wm>	 PROBLEM - puppet on ORES-web02.Experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues
[07:57:14] <icinga2-wm>	 PROBLEM - puppet on ORES-web02.Experimental is CRITICAL: CRITICAL: Puppet has 17 failures. Last run 11 minutes ago with 17 failures. Failed resources (up to 3 shown): Package[ntp],Service[systemd-timesyncd],Service[exim4],Package[update-manager-core]
[08:16:40] <icinga2-wm>	 RECOVERY - puppet on ORES-web02.Experimental is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures
[12:06:09] <travis-ci>	 wiki-ai/articlequality#49 (fawiki_wp10 - 5c8fcfc : Amir Sarabadani): The build passed. https://travis-ci.org/wiki-ai/articlequality/builds/365082508
[12:22:20] <wikibugs>	 10Scoring-platform-team (Current), 10ORES, 10articlequality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Rename "wikiclass" to "articlequality" - https://phabricator.wikimedia.org/T181820#4122963 (10Ladsgroup) a:03Ladsgroup https://github.com/wiki-ai/articlequality/pull/64
[12:24:30] <travis-ci>	 wiki-ai/articlequality#51 (rename - 2a0d478 : Amir Sarabadani): The build has errored. https://travis-ci.org/wiki-ai/articlequality/builds/365088998
[14:07:52] * halfak pokes at Amir1's WIP PR
[14:15:10] <wikibugs>	 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#4123214 (10Halfak) This sounds surprising and strange.  Please ping me on...
[14:37:50] <Amir1>	 Thank!
[14:37:55] <Amir1>	 *Thanks
[15:16:02] <travis-ci>	 wiki-ai/articlequality#55 (rename - 507064c : Amir Sarabadani): The build was fixed. https://travis-ci.org/wiki-ai/articlequality/builds/365171391
[15:44:09] <Amir1>	 halfak: I'm sorry but this rename will conflict with some of my changes I'm going to make in fetch_label utility now so if it's possible can this be merged now to avoid that conflict (so I rebase it on top of master before moving forward?)
[16:00:43] <Amir1>	 I will be there in two minutes
[16:49:33] <halfak>	 Amir1, https://github.com/wiki-ai/articlequality/pull/64#discussion_r180757840
[16:50:06] <Amir1>	 halfak: I fixed that and pushed the change, isn't it live?
[16:51:54] <halfak>	 Ahh  github confused me 
[16:54:51] <Amir1>	 thanks!
[17:09:30] <awight>	 halfak: re the virtualenv woes…  in hindsight, I think I misinterpreted the initial problem, but the current solution is still correct / better than my earlier plan.
[17:09:41] <awight>	 At this point, I’m just trying to rebuild 2 virtualenv dirs
[17:09:47] <awight>	 one in the old location and one in the new location.
[17:10:09] <awight>	 What I’m running into seems to be a new Stretch thing, where we can’t build *any* virtualenv without network connectivity.
[17:40:32] <wikibugs>	 10Scoring-platform-team, 10MediaWiki-extensions-ORES, 10WMSE-Bug-Reporting-and-Translation-2018, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): The filters menu is closed when a filter colour is selected - https://phabricator.wikimedia.org/T188903#4124178 (10Etonkovidova) Thanks, @Sebastian_Berlin...
[17:43:28] <miriam_>	 \msg dsaez
[17:43:32] <miriam_>	 ups
[17:43:33] <miriam_>	 sorry
[17:43:49] <miriam_>	 :)
[17:44:17] <halfak>	 o/ awight just got out of meeting
[17:44:43] <halfak>	 Oh crud re. network connectivity. 
[17:44:48] <halfak>	 Can we downgrade pip?
[17:44:57] <halfak>	 Or maybe that isn't the problem. 
[17:45:19] <awight>	 I’m in SoS for a few minutes, but I think something can be done.
[17:45:31] <halfak>	 cool
[17:45:42] <awight>	 Weirdly, the virtualenv package requires a bundle of wheels—however, it isn’t using this directory by default!
[17:46:14] <awight>	 When I point (newer) virtualenv to the wheels, it uses 2 out of 3 cached wheels, but still downloads the third.
[17:46:22] <awight>	 That’s how far I got last night.
[17:56:38] <halfak>	 Strange.  So pip install --no-deps *.whl is not idempotent?
[17:56:52] <halfak>	 Can we just not run the pip install on rollback?
[17:57:06] <halfak>	 Because if the wheels are installed in the venv, we should just be able to *use* it. 
[17:58:08] <awight>	 ah it’s worse
[17:58:16] <awight>	 This is actually the virtualenv initialization step.
[17:58:24] <awight>	 So there’s very little control over what happens
[17:58:25] <halfak>	 Oh.  
[17:58:28] <halfak>	 Hmm
[17:58:33] <awight>	 Just a —extra-search-dir argument
[18:02:37] <halfak>	 Lunch!
[18:02:47] <awight>	 Another annoying detail is that virtualenv *will* use pip’s wheel cache, but for some reason this isn’t already populated, and mangled file paths make it too horrible to create the cached wheel directly.
[18:02:49] <awight>	 o/
[18:10:48] <wikibugs>	 (03PS1) 10Awight: Prevent virtualenv from being silly [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425580 (https://phabricator.wikimedia.org/T181071)
[18:11:04] <awight>	 halfak: Amir1: ^
[18:12:52] <halfak>	 What do you mean about virtualenv using a wheel cache? 
[18:13:01] <halfak>	 Does virtualenv need something in order to operate? 
[18:13:07] <halfak>	 Like maybe a local pip install or something?
[18:13:59] <wikibugs>	 (03CR) 10Halfak: [C: 032] Prevent virtualenv from being silly [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425580 (https://phabricator.wikimedia.org/T181071) (owner: 10Awight)
[18:14:12] <halfak>	 "--never-download" seems right to me
[18:14:19] <halfak|Lunch>	 :) 
[18:14:43] <awight>	 ty!  yeah documentation about that param was crappy, but reading the code it’s what we want.
[18:14:52] <wikibugs>	 (03CR) 10Awight: [V: 032] Prevent virtualenv from being silly [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425580 (https://phabricator.wikimedia.org/T181071) (owner: 10Awight)
[19:43:43] <halfak>	 o/ Amir1 
[19:44:06] <halfak>	 Wondering how the analysis of reverts in fawiki/eswiki/wikidata/etc. is going
[19:44:14] <Amir1>	 halfak: hey sup
[19:44:18] <Amir1>	 On it
[19:44:47] <halfak>	 Cool :) I'm jumping between papers and remembered that was one of the things we want to get into the CSCW ORES paper. 
[19:46:54] <Amir1>	 that's for sure. When is the deadline?
[19:49:00] <halfak>	 April 19th
[19:49:03] <halfak>	 Next Thursday
[19:49:06] <halfak>	 SO about a week 
[19:49:09] <Amir1>	 That's too close
[19:49:20] <halfak>	 No rush.  We'll still be able to submit if we don't have this. 
[19:49:33] <Amir1>	 I will get you the raw data for Spanish by tomorrow 
[19:54:08] <codezee>	 halfak: when you mentioned you were not quite there with the drafttopic writeup, did you mean in terms of style or missing some point?
[19:56:03] <halfak>	 codezee, not quite the story I wanted to tell.  But now that I have had time to think about it, I want to skip the discussion of newcomers. 
[19:56:34] <halfak>	 I plan to make a graphic for the WP Directory today to help us talk about it. 
[19:57:39] <codezee>	 halfak: oh, btw, I just now saw that CSCW 2018 states to use ACM small template, so I'm porting all the existing stuff
[19:57:57] <halfak>	 Right.  Cool. :) 
[19:58:04] <halfak>	 That's single column, right? 
[19:58:07] <codezee>	 yes
[19:58:11] <halfak>	 goood 
[19:58:12] <halfak>	 :) 
[19:58:22] <codezee>	 graphics will have more space \o/
[20:05:21] <awight>	 halfak: virtualenvs on ores1001 both look good, I’ll go ahead and finish this deployment.
[20:05:52] <halfak>	 awesome!
[20:23:20] <wikibugs>	 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#4124827 (10awight)
[20:26:34] <wikibugs>	 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#4124865 (10awight) @akosiaris We're finally ready to deploy the puppet cha...
[20:27:08] <wikibugs>	 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#3851398 (10awight) p:05Triage>03High
[20:27:33] <halfak>	 GJ awight !
[20:27:36] <wikibugs>	 (03PS1) 10Ladsgroup: Use constants for rc_patrolled values [extensions/ORES] - 10https://gerrit.wikimedia.org/r/425628
[20:28:05] * awight fetches kibble from the office supply :p
[20:28:48] <awight>	 Not quite there yet, but happily this was a mostly safe migration.
[20:28:57] <awight>	 halfak: Important note, we can’t deploy any changes to wheels until the task is complete.
[20:29:14] <halfak>	 Gotcha.  
[20:30:33] <awight>	 halfak: you know… I think I’m wrong about that.  We’re still deploying to the old location.
[20:30:46] <awight>	 Maybe I just wanted to sound important :-)
[20:31:09] <halfak>	 :P 
[20:46:07] <wikibugs>	 (03CR) 10Hoo man: [C: 032] Use constants for rc_patrolled values [extensions/ORES] - 10https://gerrit.wikimedia.org/r/425628 (owner: 10Ladsgroup)
[20:52:07] <wikibugs>	 (03Merged) 10jenkins-bot: Use constants for rc_patrolled values [extensions/ORES] - 10https://gerrit.wikimedia.org/r/425628 (owner: 10Ladsgroup)
[21:01:27] <wikibugs>	 (03PS3) 10Awight: [DNM] Experimental git-lfs submodule [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/419613 (https://phabricator.wikimedia.org/T180627)
[21:01:29] <wikibugs>	 (03PS2) 10Awight: [DNM] Update to assets with an LFS file [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/419642 (https://phabricator.wikimedia.org/T180627)
[21:01:31] <wikibugs>	 (03PS2) 10Awight: [DNM] Configure scap to do git-lfs [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/419759 (https://phabricator.wikimedia.org/T180627)
[21:01:33] <wikibugs>	 (03PS2) 10Awight: Use gerrit for assets submodule [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/419878
[21:01:35] <wikibugs>	 (03PS1) 10Awight: Enable git_upstream_submodules [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425710 (https://phabricator.wikimedia.org/T180627)
[21:19:53] <wikibugs>	 (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/425667 (owner: 10L10n-bot)
[21:36:15] <wikibugs>	 (03CR) 10jenkins-bot: Use constants for rc_patrolled values [extensions/ORES] - 10https://gerrit.wikimedia.org/r/425628 (owner: 10Ladsgroup)
[21:36:27] <wikibugs>	 10Scoring-platform-team, 10Scap, 10Patch-For-Review, 10Release-Engineering-Team (Kanban): [Blocked] Support git-lfs - https://phabricator.wikimedia.org/T180627#4125077 (10awight) Errors thrown by my experiment with `git_upstream_submodules: True`: ``` scap deploy-log -v -f scap/log/scap-sync-2018-04-11-000...
[21:57:36] <wikibugs>	 (03PS1) 10Awight: [DNM] remove some submodules to simplify tests [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425716
[21:57:38] <wikibugs>	 (03PS1) 10Awight: point remaining submodule at gerrit for fun [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425717
[22:13:37] <halfak>	 https://www.mediawiki.org/wiki/Topic:Ub3g57qa9gflrlrc
[22:13:43] <halfak>	 OK with that, I'm out of here. 
[22:13:46] <halfak>	 Gotta pedal! 
[22:13:57] <halfak>	 Stupid exercise.  Why doesn't it just happen on its own. 
[22:14:10] <halfak>	 Have a good evening/rest of your day/morning, folks!
[22:15:42] <awight>	 kbye!
[22:16:08] <wikibugs>	 10Scoring-platform-team (Current), 10Research, 10Research-outreach, 10Epic, 10Paper: [Epic] Write paper about ORES as a socio-technical probe - https://phabricator.wikimedia.org/T121719#4125177 (10Capt_Swing)
[22:40:29] <wikibugs>	 (03PS2) 10Awight: Remove transitional virtualenv copy [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425115 (https://phabricator.wikimedia.org/T181071)
[22:58:50] <wikibugs>	 (03PS3) 10Awight: Remove transitional virtualenv copy [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/425115 (https://phabricator.wikimedia.org/T181071)
[23:07:04] <wikibugs>	 10Scoring-platform-team, 10ORES: mwparserfromhell ParserError on Premier League - https://phabricator.wikimedia.org/T191861#4118999 (10awight) We're pinned to 0.4.4; I'll get us upgraded, thanks for the investigation!
[23:08:07] <wikibugs>	 10Scoring-platform-team, 10ORES: Upgrade ORES mwparserfromhell dependency to 0.5.1 - https://phabricator.wikimedia.org/T192038#4125326 (10awight)
[23:13:36] <travis-ci>	 wiki-ai/revscoring#1464 (mwph_upgrade - 2278730 : Adam Wight): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/365368782
[23:14:36] <wikibugs>	 10Scoring-platform-team, 10ORES: Upgrade ORES mwparserfromhell dependency to 0.5.1 - https://phabricator.wikimedia.org/T192038#4125344 (10awight) https://github.com/wiki-ai/revscoring/pull/396
[23:14:50] <wikibugs>	 10Scoring-platform-team (Current), 10ORES, 10revscoring, 10artificial-intelligence: Upgrade ORES mwparserfromhell dependency to 0.5.1 - https://phabricator.wikimedia.org/T192038#4125346 (10awight)
[23:21:00] <wikibugs>	 10Scoring-platform-team (Current), 10ORES, 10revscoring, 10artificial-intelligence: Upgrade ORES mwparserfromhell dependency to 0.5.1 - https://phabricator.wikimedia.org/T192038#4125361 (10awight) a:03awight
[23:32:20] <travis-ci>	 wiki-ai/editquality#293 (huwiki-damaging-goodfaith - 5d11a50 : Gergő Tisza): The build passed. https://travis-ci.org/wiki-ai/editquality/builds/365374288
[23:33:34] <wikibugs>	 10Scoring-platform-team (Current), 10ChangeProp, 10ORES, 10Patch-For-Review: Drop "non bot" condition from ORES changeprop rules - https://phabricator.wikimedia.org/T187927#4125400 (10awight) Everything discussed so far still holds, roughly, but I wanted to mention that I re-read the deployed ORES config a...