[06:02:54] <_joe_>	 mutante: 32 GB of memory needed => dedicated hardware
[06:03:03] <_joe_>	 end of story :)
[06:03:19] <_joe_>	 and 16 GB too, generally speaking
[06:05:04] <_joe_>	 we're relatively short on ganeti resources
[07:38:51] <kormat>	 new ganeti policy: 640KB should be enough for everybody
[08:05:49] <XioNoX>	 I'm going to upgrade cr3-knams shortly, no esams depool planned
[08:40:16] <XioNoX>	 I'm going to depool codfw to upgrade the routers over there
[08:40:37] <XioNoX>	 no impact expected, this is a "just in case" depool
[08:48:52] <_joe_>	 so only the edge traffic?
[08:49:02] <_joe_>	 can internal traffic be impacted too?
[08:50:29] <XioNoX>	 _joe_: if something goes wrong, yes, but it's not supposed to
[08:50:39] <_joe_>	 ok
[08:50:47] <XioNoX>	 if you have some easy depool to do feel free to
[08:58:12] <_joe_>	 no, no need
[11:03:04] <XioNoX>	 codfw routers upgraded smoothly
[11:05:04] <mark>	 it's never the upgrade itself that is the problem, it's what comes after it... ;-)
[11:07:04] <XioNoX>	 sometimes the upgrade can have its share of surprises
[11:07:22] <XioNoX>	 it's a very minor version upgrade, so hopefully it will stay quiet
[14:31:35] <kormat>	 aaanyone know how to tell postgres to exit recovery mode?
[14:31:58] <cdanis>	 oh this is a classic, kormat, first press <Esc> and then :wq
[14:32:05] <kormat>	 https://phabricator.wikimedia.org/P12306
[14:33:27] <kormat>	 cdanis: so close
[15:05:29] <kormat>	 `pg_ctlcluster 11 main promote` might have been the answer
[15:06:24] <_joe_>	 I love that "11" in the middle
[15:06:42] <kormat>	 that only took 1.5h to figure out. thanks postgres docs.
[15:18:22] <jynus>	 anyone up for a sanity check on a php patch?
[15:19:04] <jynus>	 non-mw
[15:19:33] <jynus>	 apparently kormat can learn postgress in 1.5 hours, but not php in 5 minutes!
[15:19:50] <kormat>	 👍
[15:19:55] <jynus>	 :-D
[15:20:06] <jynus>	 https://jynus.com/gif/dissappointed.gifv
[15:21:49] <andrewbogott>	 hm, there's a wikidough patch in the queue for puppetmasters from Ssingh; what's their IRC handle?
[15:22:22] <cdanis>	 sukhe: ^
[15:22:32] <sukhe>	 andrewbogott: please merge, thanks
[15:22:49] <andrewbogott>	 done!
[15:22:53] <sukhe>	 thank you!
[15:58:51] <mutante>	 _joe_: ack, 16GB is grey area and 32GB means dedicated hardware. that sounds about right to me, thanks
[15:59:46] <mutante>	 we have wildfires here in the mountains. the sky and sun this morning looks like i'm on another planet, it was raining ashes.. the doomsday feeling is now complete and multiple reasons for masks
[16:00:24] <cdanis>	 mutante: it looks really bad :(
[16:01:33] <sukhe>	 mutante: stay safe :(
[16:02:46] <mutante>	 also, there could be power outages and the hotels are starting to be full of people evacuated from the mountains
[16:02:49] <mutante>	 thank you
[16:03:59] <sukhe>	 this is not easy during regular days, can't imagine during times of a pandemic
[16:04:14] <mutante>	 i am in the "flood plains" in downtown.. won't be directly affected by the fires but the smoke sucks
[16:04:39] <mutante>	 don't exactly have aircon or hepa filters
[16:05:38] <cdanis>	 I'm guessing that air filters are basically impossible to find right now, too
[16:09:03] <mutante>	  40,000 acres with zero containment ...uhm
[16:12:32] <mutante>	 they can't use aircraft to fight it or even map it.. due to the heavy smoke. what a crappy cycle
[16:22:16] <lmata>	 :-(
[18:15:16] <hauskatze>	 hi
[18:15:21] <hauskatze>	 what's hue.wikimedia?
[18:15:32] <mutante>	 hauskatze: an analytics tool
[18:15:45] <hauskatze>	 there's a Task about it but not sure which Tag it belongs to
[18:15:58] <hauskatze>	 mutante: I'll tag analytics
[18:15:59] <mutante>	 a web UI for hadoop
[18:16:00] <mutante>	 https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hue
[18:16:15] <mutante>	 hauskatze: that sounds right, yes
[18:16:50] <mutante>	 analytics-clusters
[18:17:29] <hauskatze>	 thanks mutante - very helpful as always
[18:19:29] <mutante>	 de nada
[18:19:32] <hauskatze>	 task was T260929 just in case
[18:19:32] <stashbot>	 T260929: hue.wikimedia.org throws an exception when trying to log in with a non-ASCII username - https://phabricator.wikimedia.org/T260929
[18:20:55] <hauskatze>	 diner, etc. - see you
[18:21:29] <mutante>	 sounds like demon tried to login as 🙂
[21:13:46] <ryankemper>	 I want to see what the result of running a `spicerack.prometheus.Prometheus.query(query_string, datacenter)` in production looks like (to verify I'm mocking the response format correctly for a spicerack PR I'm working on)
[21:13:54] <ryankemper>	 What would be the best way to do that?
[21:14:08] <ryankemper>	 for a spicerack patch*
[21:15:15] <volans|off>	 ryankemper: see volans-tmp etherad (first block)
[21:18:13] <ryankemper>	 volans|off: Thanks!
[21:23:29] <ryankemper>	 that was easy :)