[07:48:18] <marostegui>	 I am failing over phabricator dbproxy, it should be transparent but just saying it here in case someone notices something strange
[08:58:50] <godog>	 moritzm: merging your patch too
[09:15:36] <moritzm>	 godog: ack, thx
[09:37:22] <effie>	 slyngs: marostegui I will be switching a couple of things that may cause a mediawiki kerfuffle, though I am very optimistic 
[09:37:34] <marostegui>	 effie: Thanks for the heads up
[11:51:49] <taavi>	 I am disabling puppet on C:bird to roll out https://gerrit.wikimedia.org/r/c/operations/puppet/+/1289919
[12:32:37] <klausman>	 anybody else having trouble reaching google services? Quad-8 is not routable for me, but neither is quad-9 which kinda points to my ISP having a problem
[12:33:33] <klausman>	 ...and of course the moment I ask, it starts working again
[12:35:07] <taavi>	 glad to be of assistance
[12:39:09] <A_smart_kitten>	 https://bash.toolforge.org/quip/gA8gHZ0Bvg159pQrJzGF
[15:18:14] <brouberol>	 sukhe: FWIW the issue I had yesterday with the kafka-jumbo host was due to the OS SSD being mounted as /dev/sdb and the data HDD raid array as /dev/sda, instead of the opposite. I assumed the vconsole would stream the output of the debian installer, but it didn't and thus kept showing some iPXE-related output
[15:19:13] <sukhe>	 brouberol: ah ok. well glad to know it resolved, could be worse when it comes to this stuff :P 
[15:19:36] <brouberol>	 sure can
[15:19:40] <brouberol>	 thanks again for the help!
[15:20:02] <sukhe>	 <3  (not that my advice was useful :))
[16:45:43] <jhathaway>	 One of our users received over 2k emails of the form from=<wiki-enwiki-suezo-tf5byi-eZK6PA+WQ4pKson/@wikimedia.org>. Their email volume was a factor in Yahoo blocking all mail from wikipedia. Does anyone know if this is watchlist email?
[16:46:32] <jhathaway>	 I think we need to look ratelimiting per recipient, but I'm not sure where the email is comfing from, nor which team would be tasked with the work
[16:52:16] <jynus>	 that's not normal user emailing, their from is From: "Wikipedia" <wiki@wikimedia.org>
[16:54:37] <taavi>	 there is https://www.mediawiki.org/wiki/Extension:BounceHandler which will rewrite the envelope-level from for all mediawiki mail to something of that form
[16:59:26] <jynus>	 ah, it may be smtp mailfrom, not content from
[16:59:40] <jhathaway>	 yeah, that is correct I believe
[16:59:55] <jhathaway>	 so probably watchlist email?
[17:00:07] <taavi>	 well, something from mediawiki
[17:00:10] <jynus>	 both user mail and notifications use that format
[17:00:25] <jynus>	 what t says
[17:02:13] <jhathaway>	 okay thanks, any idea who could help track down the cause
[17:02:50] <jynus>	 I don't think it would be easy without the content or a few extra headers, the details would be there
[17:04:26] <jhathaway>	 yeah, that is tough, as we don't log either of those
[17:05:23] <jhathaway>	 I have an ip address, but I don't think that tells me much beyond its a wikimedia pod