[09:22:01] <_joe_>	 moritzm / jbond42 is it expected that my sso token lasts less than 3 days?
[09:24:14] <moritzm>	 if you changed your IP within the three days, then yes. We're planning to change this within CAS (so that it doesn't require the same IP for the session), but the 6.3 update is currently blocked by a regression in their memcached support
[09:29:00] <_joe_>	 moritzm: heh yes very possible I did change IP
[14:37:27] <dcaro>	 akosiaris: ping, you have some pending puppet changes, should I merge them?
[14:37:43] <akosiaris>	 dcaro: yes please
[14:37:46] <dcaro>	 ack
[14:38:07] <dcaro>	 done :)
[19:52:58] <Krinkle>	 legoktm: around for https://gerrit.wikimedia.org/r/c/operations/puppet/+/664903/ whenver
[19:53:04] <Krinkle>	 will do some status quo checking in beta first
[19:53:08] <Krinkle>	 I forgot about fatal-error.php
[19:58:14] <legoktm>	 Krinkle: ack, I was thinking I'd disable puppet on one of the mwdebug hosts, manually copy the file in, we'd test it, and then merge it and let puppet roll it out everywhere
[20:01:24] <legoktm>	 just let me know when you're done with beta
[20:05:16] <Krinkle>	 legoktm: I'm unable to reproduce the issue in beta
[20:05:35] <Krinkle>	 it seems all fatal-error invocations, except action=oom, don't involve php-wmerror there
[20:05:53] <Krinkle>	 maybe message loss, or maybe something diverged from prod
[20:06:12] <Krinkle>	 and the oom one seems to actually behave the way the puppet script expects so my change would remove information that is currently present
[20:06:32] <legoktm>	 hm, I remember logstash-beta having issues a few weeks ago
[20:07:16] <Krinkle>	 https://logstash-beta.wmflabs.org/goto/1abc24dd22840c083f63bccad765452a
[20:07:21] <Krinkle>	 messages are generally arriving though
[20:07:27] <Krinkle>	 just reported directly by MW
[20:10:41] <Krinkle>	 compare to https://logstash.wikimedia.org/goto/085c92315c68b69e8f5129883759111e
[20:10:57] <Krinkle>	 (oh my, the number of clicks and decisions required ot generate this link in Kibana 8 are insane)
[20:11:32] <Krinkle>	 seems most of htem are actually fine
[20:11:44] <Krinkle>	 only one was weirdly formatted:
[20:11:45] <Krinkle>	 >  Uncaught Wikimedia\Rdbms\DBUnexpectedError: DB connection was already closed in /srv/mediawiki/php-1.36.0-wmf.31/includes/libs/rdbms/database/Database.php:982 in /srv/mediawiki/php-1.36.0-wmf.31/includes/libs/rdbms/database/Database.php on line 982
[20:15:22] <legoktm>	 I'd rather err on the side of having duplicated info rather than losing it
[20:15:49] <Krinkle>	 yeah
[20:15:56] <Krinkle>	 I'll update the examples and docs still
[20:16:05] <Krinkle>	 but no special deploy needed
[20:16:58] <legoktm>	 ok :)
[21:02:17] <Krinkle>	 legoktm: hm.. what do I do for a change in /etc/php/php7-fatal-error.php to take effect?
[21:02:38] <Krinkle>	 I've done restart apache2 and restart php72-fpm and 'touch', but to no avail
[21:02:45] <Krinkle>	 on deployment-mediawiki-07
[21:03:16] <Krinkle>	 http header tells me I'm definitely missing varnish and going to `server: deployment-mediawiki-07`
[21:03:37] <legoktm>	 does php -i have that file configured as wmerrors.error_script_file ?
[21:03:51] <Krinkle>	 yet despite having remove 'postsend' parameter in code..
[21:03:51] <Krinkle>	 oh
[21:03:57] <Krinkle>	 I'm an idiot
[21:04:11] <Krinkle>	 I've pasted updated contents for fatal-error.php in /etc/ instead of /w/
[21:04:23] <Krinkle>	 these file names are too similar lol
[21:04:24] <Krinkle>	 nvm
[21:06:01] <legoktm>	 heh
[21:06:03] <legoktm>	 :)))
[22:12:10] <Krinkle>	 legoktm: ok, ready
[22:12:17] <Krinkle>	 still a minor change to the string format
[22:12:29] <Krinkle>	 I'll roll out the added w/fatal-error.php testability first
[22:12:38] <Krinkle>	 https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/666217
[22:46:39] <legoktm>	 Krinkle: ok, give me a minute, just got back from lunch