[07:50:19] 10Certcentral: Take into account LE rate limits on sensitive operations - https://phabricator.wikimedia.org/T207927 (10Vgutierrez) [08:09:05] 10Certcentral: Take into account LE rate limits on sensitive operations - https://phabricator.wikimedia.org/T207927 (10Vgutierrez) p:05Triage>03High a:03Vgutierrez [10:26:47] 10Certcentral, 10Patch-For-Review: Take into account LE rate limits on sensitive operations - https://phabricator.wikimedia.org/T207927 (10Vgutierrez) [13:07:15] 10Traffic, 10Operations, 10Wikimedia-Incident: Power incident in eqsin - https://phabricator.wikimedia.org/T206861 (10faidon) [13:07:17] 10Traffic, 10Operations, 10Wikimedia-Incident: Add maint-announce@ to Equinix's recipient list for eqsin incidents - https://phabricator.wikimedia.org/T207140 (10faidon) 05Resolved>03Open I see emails for SG3 that (as far as I can tell) haven't made it to maint-announce, e.g. ``` Date: Thu, 25 Oct 2018 1... [15:00:04] 10netops, 10Operations, 10ops-eqiad, 10Patch-For-Review: Rack/setup cr2-eqord - https://phabricator.wikimedia.org/T204170 (10Papaul) Router has been unracked and dropped at shipping for ship out. shipping information below {F26792797} [15:16:04] 10Traffic, 10Operations, 10ops-eqiad: cp1076 hardware failure - https://phabricator.wikimedia.org/T206394 (10Cmjohnson) @BBlack the idrac h/w log does not show any failures [16:45:09] bblack, ema: you opinion on https://gerrit.wikimedia.org/r/c/operations/puppet/+/469649 would be welcomed [16:45:46] for the context, wdqs1003 is part of the public wdqs cluster, and is having trouble keeping up with updates (the other servers in that cluster are mostly fine) [16:46:32] we're not exactly sure what is wrong with that specific server, so we'd like to exchange one of the wdqs-internal with wdqs1003, see what the different load of the internal cluster does [16:48:19] I'm not an expert but I think it could be safer to split it in 2 commits (removal + addition) [16:48:23] at the traffic layer [16:48:30] and the question is: how much will I break pybal config with a change like that [16:48:34] and that will also allow to do the puppet changes to both hosts [16:48:52] remove both first and re-add both after? [16:49:32] I could even split it in 4 commits, remove / add for each server [16:49:41] I think so [16:50:17] yeah, seems at least safer (if not easier) [17:10:50] 10netops, 10Analytics, 10Analytics-Kanban, 10Operations: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster - https://phabricator.wikimedia.org/T207321 (10Ottomata) hey hey heyyy, the nodes are in! https://phabricator.wikimedia.org/T204177#4695147 How can we move this forwa... [18:55:51] 10netops, 10Analytics, 10Analytics-Kanban, 10Operations: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster - https://phabricator.wikimedia.org/T207321 (10faidon) So, this is quite the can of worms :) There are several pieces to this, and honestly, I feel like VLANs is kind o... [19:58:35] volans|off: if still around https://gerrit.wikimedia.org/r/q/topic:%22wdqs-cluster-switch%22 [19:58:48] Oh damn, your off, sorry :/ [20:00:54] bblack, ema: I'm looking for someone reviewing the second try on moving wdqs1003 [20:01:02] ^^^ [21:21:22] 10netops, 10Cloud-Services, 10Operations, 10Patch-For-Review: Renumber cloud-instance-transport1-b-eqiad to public IPs - https://phabricator.wikimedia.org/T207663 (10ayounsi) Thanks for investigating it! See https://gerrit.wikimedia.org/r/c/operations/dns/+/469771 for the IPs, I took the same model as the...