[03:19:30] <wikibugs>	 10Scoring-platform-team (Current), 10ORES, 10Patch-For-Review: Wheels built on ores-misc-01 are incompatible with ores* and scb* - https://phabricator.wikimedia.org/T179095#3717302 (10awight) Not quite complete: my pip hack left ores-beta in a broken state, the issue seems to be that upgrading pip doesn't co...
[03:19:32] <wikibugs>	 10Scoring-platform-team (Current), 10ORES, 10Patch-For-Review: Wheels built on ores-misc-01 are incompatible with ores* and scb* - https://phabricator.wikimedia.org/T179095#3713113 (10awight) a:03awight
[04:23:09] <travis-ci>	 wiki-ai/revscoring#1273 (nltk_corpora - 1d5c6c7 : Adam Roses Wight): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/293998317
[16:07:13] <wikibugs>	 10Scoring-platform-team, 10ORES, 10Operations, 10Traffic, and 4 others: 503 spikes and resulting API slowness starting 18:45 October 26 - https://phabricator.wikimedia.org/T179156#3717711 (10hoo) Happening again, this time on `cp1055`.  Example from `mw1180`:  ``` $ ss --tcp -r | grep -oP 'cp\d+' | sort |...
[16:14:11] <wikibugs>	 10Scoring-platform-team, 10ORES, 10Operations, 10Traffic, and 4 others: 503 spikes and resulting API slowness starting 18:45 October 26 - https://phabricator.wikimedia.org/T179156#3717712 (10hoo) Also on `mw1180`:  ``` $ sudo -u www-data ss --tcp -r -p > ss $ cat ss | grep -c FIN-WAIT-2 16 $ cat ss | grep...
[16:22:59] <wikibugs>	 10Scoring-platform-team, 10ORES, 10Operations, 10Traffic, and 4 others: 503 spikes and resulting API slowness starting 18:45 October 26 - https://phabricator.wikimedia.org/T179156#3717713 (10Paladox) p:05Triage>03Unbreak! Spoke to hoo on irc, who agreed it's an UBN now.
[16:39:14] <halfak>	 o/
[16:39:16] <halfak>	 Sorry I'm late! 
[16:39:33] <halfak>	 I'm hacking for a few hours.  Ping me if you need anything
[16:39:45] * halfak considers looking at JADE for fiwiki
[16:39:48] <halfak>	 *or
[17:07:25] <Amir1>	 halfak: hey there
[17:07:34] <halfak>	 Hi Amir1!  How's conference?
[17:07:36] <Amir1>	 I'm still in the conference, 
[17:07:46] <Amir1>	 lots of talks and familiar faces
[17:07:48] <Amir1>	 \o/
[17:07:55] <halfak>	 \o/  :) 
[17:07:55] <halfak>	 Sorry I'm not there
[17:08:03] <Amir1>	 no worries, I wish you could 
[17:08:13] <Amir1>	 we'll meet soon
[17:08:43] <Amir1>	 halfak: I have two PRs, one is pytest in ores (last repo), the other one is info_url in wikilabels
[17:08:48] <Amir1>	 I tested it and works just fine
[17:09:17] <halfak>	 OK cool.  I'll have a look :) 
[17:11:57] <halfak>	 Amir1, the wikilabels one has conflicts
[17:12:04] <halfak>	 Meh.  I'll get 'em
[17:12:05] <halfak>	  :) 
[17:12:41] <halfak>	 {{done}}
[17:12:41] <AsimovBot>	 How cool, halfak!
[18:34:51] <Amir1>	 halfak: thanks
[20:02:17] <wikibugs>	 10Scoring-platform-team, 10Patch-For-Review: ORES deployment submodules should point to phabricator HTTPS repos. - https://phabricator.wikimedia.org/T179009#3717826 (10mmodell) Now I'm wondering if Phabricator has the same bug with it's own git-lfs support (file size limit due to large HTTP transactions buffer...
[20:21:40] <wikibugs>	 10Scoring-platform-team, 10ORES, 10Operations, 10Traffic, and 4 others: 503 spikes and resulting API slowness starting 18:45 October 26 - https://phabricator.wikimedia.org/T179156#3717847 (10BBlack) Updates from the Varnish side of things today (since I've been bad about getting commits/logs tagged onto th...
[22:05:42] <wikibugs>	 10Scoring-platform-team, 10ORES, 10Operations, 10Traffic, and 4 others: 503 spikes and resulting API slowness starting 18:45 October 26 - https://phabricator.wikimedia.org/T179156#3717895 (10BBlack) A while after the above, @hoo started focusing on a different aspect of this we've been somewhat ignoring as...