[07:21:05] <wikibugs>	 10Traffic, 10netops, 10Operations, 10Patch-For-Review: eqiad row D switch upgrade - https://phabricator.wikimedia.org/T172459#3517929 (10Joe) I don't think we're safe to do this maintenance until we do rack all the new mediawiki machines. We have almost half of our capacity for MediaWiki in row D. We have...
[15:31:15] <wikibugs>	 10netops, 10Operations, 10fundraising-tech-ops, 10Patch-For-Review: Move codfw frack to new infra - https://phabricator.wikimedia.org/T171970#3518627 (10ayounsi) >>! In T171970#3510082, @ayounsi wrote: > https://librenms.wikimedia.org/device/device=153/tab=port/port=13330/ (possibly MTU related) > Possible...
[18:35:14] <wikibugs>	 10Traffic, 10netops, 10Operations, 10Patch-For-Review: eqiad row D switch upgrade - https://phabricator.wikimedia.org/T172459#3519246 (10ayounsi)
[18:37:11] <wikibugs>	 10Traffic, 10netops, 10Operations, 10Patch-For-Review: eqiad row D switch upgrade - https://phabricator.wikimedia.org/T172459#3499267 (10ayounsi) 05Open>03stalled Sounds fair :) Marking this task as a dependency of T165519. Any idea of the time-line for T165519?
[18:37:41] <wikibugs>	 10Traffic, 10netops, 10Operations, 10Patch-For-Review: eqiad row D switch upgrade - https://phabricator.wikimedia.org/T172459#3519249 (10ayounsi)
[20:19:25] <wikibugs>	 10Traffic, 10Operations, 10Pybal: lvs servers report 'Memory allocation problem' on bootup - https://phabricator.wikimedia.org/T82849#3519518 (10ema) >>! In T82849#3503596, @ema wrote: > A more general patch has been submitted by Julian Anastasov http://archive.linuxvirtualserver.org/html/lvs-devel/2017-08/m...
[20:41:07] <mark>	 reactor stats:
[20:41:08] <mark>	 # HELP pybal_reactor_iteration_duration Reactor iteration latency
[20:41:08] <mark>	 # TYPE pybal_reactor_iteration_duration summary
[20:41:08] <mark>	 pybal_reactor_iteration_duration_count 513.0
[20:41:08] <mark>	 pybal_reactor_iteration_duration_sum 94.78530859947205
[20:41:24] <mark>	 pybal_reactor_do_read_or_write_duration_count{method="doRead",selectable="ProcessReader"} 9.0
[20:41:24] <mark>	 pybal_reactor_do_read_or_write_duration_sum{method="doRead",selectable="ProcessReader"} 0.0027060508728027344
[20:41:24] <mark>	 pybal_reactor_do_read_or_write_duration_count{method="doRead",selectable="ProcessWriter"} 3.0
[20:41:24] <mark>	 pybal_reactor_do_read_or_write_duration_sum{method="doRead",selectable="ProcessWriter"} 0.000308990478515625
[20:41:26] <mark>	 pybal_reactor_do_read_or_write_duration_count{method="doWrite",selectable="Client"} 92.0
[20:41:28] <mark>	 pybal_reactor_do_read_or_write_duration_sum{method="doWrite",selectable="Client"} 0.14849519729614258
[20:41:30] <mark>	 pybal_reactor_do_read_or_write_duration_count{method="doRead",selectable="Server"} 0.0
[20:41:33] <mark>	 pybal_reactor_do_read_or_write_duration_sum{method="doRead",selectable="Server"} 0.0
[20:41:35] <mark>	 pybal_reactor_do_read_or_write_duration_count{method="doRead",selectable="Port"} 18.0
[20:41:37] <mark>	 pybal_reactor_do_read_or_write_duration_sum{method="doRead",selectable="Port"} 0.016401290893554688
[20:41:39] <mark>	 pybal_reactor_do_read_or_write_duration_count{method="doRead",selectable="_SIGCHLDWaker"} 4.0
[20:41:41] <mark>	 pybal_reactor_do_read_or_write_duration_sum{method="doRead",selectable="_SIGCHLDWaker"} 0.0007390975952148438
[20:41:43] <mark>	 pybal_reactor_do_read_or_write_duration_count{method="doRead",selectable="Client"} 54.0
[20:41:46] <mark>	 pybal_reactor_do_read_or_write_duration_sum{method="doRead",selectable="Client"} 0.019361495971679688
[20:41:50] <mark>	 etc
[21:43:52] <wikibugs>	 10Traffic, 10Operations, 10Wikimedia-Incident: Deploy redundant unified certs - https://phabricator.wikimedia.org/T148131#3519793 (10RobH) >>! In T148131#2920097, @BBlack wrote: > These are now deployed (digicert in esams, globalsign elsewhere).  Pending closing this until we document switching off either of...
[22:13:02] <wikibugs>	 10Traffic, 10Operations: setup/install cp4022 - https://phabricator.wikimedia.org/T171967#3519829 (10RobH)