[07:45:48] bblack: yeah, it must be because of base::service_unit. I've deployed a cache node in labs after the role/profile refactoring and dependencies were alright [08:13:14] 10Traffic, 10Operations, 10Wikidata, 10wikiba.se, and 2 others: [Task] move wikiba.se webhosting to wikimedia misc-cluster - https://phabricator.wikimedia.org/T99531#3698826 (10Ladsgroup) >>! In T99531#3692384, @Dzahn wrote: > You might hate me for this question but have you considered using an org domain... [09:39:11] 10netops, 10Operations, 10Patch-For-Review: Find a new PIM RP IP - https://phabricator.wikimedia.org/T167842#3699063 (10Gehel) Jolokia is only listening on localhost, and we are not using discovery. The patch above will disable discovery. [13:16:22] 10Traffic, 10Operations, 10Patch-For-Review: Allocate address space for Singapore (APNIC) - https://phabricator.wikimedia.org/T156256#3699583 (10faidon) OK, so APNIC fixed the "57 duplicate objects" situation, so I proceeded with the rest and specifically: - Updated our objects for the new office address - U... [13:20:35] 10netops, 10Operations, 10Patch-For-Review, 10Performance-Team (Radar), 10Performance-Team-notice: Merge AS14907 with AS43821 - https://phabricator.wikimedia.org/T167840#3699587 (10faidon) Sounds fine to me. Before we resolve this task, let's not forget that we'll need to cleanup our RIPE objects by remo... [14:34:33] bblack: thoughts on https://gerrit.wikimedia.org/r/#/c/385378/? It seems cleaner to set varnish runtime parameters in hiera. This way we can also enable temporary experiments on a single host and such :) [15:32:12] ema: yeah. arguable the same could be done with vcl_config, too :) [15:32:59] (and simply re-state the duplicated ones between fe+be, rather than trying to do complex hacks to DRY the "common" ones) [15:45:12] yup! [16:30:49] bblack: I'm going to remove the old router configuration that adds MED to the secondary LVS (before we had BGP-MED in pybal), starting in ulsfo [16:32:34] XioNoX: ok :) [16:55:43] 10Traffic, 10netops, 10Operations, 10Pybal, 10Patch-For-Review: Deploy pybal with BGP MED support (for primary/backup) in production - https://phabricator.wikimedia.org/T165584#3700045 (10ayounsi) 05Open>03Resolved a:03ema Done! [17:15:15] ipv6 PTR are such a pain [17:42:31] 10Traffic, 10Operations, 10PAWS, 10Pywikibot-Commons: Server error (500) while trying to download files from Commons from PAWS - https://phabricator.wikimedia.org/T178567#3700199 (10Chicocvenancio) [17:46:27] 10Traffic, 10Operations, 10PAWS, 10Pywikibot-Commons: Server error (500) while trying to download files from Commons from PAWS - https://phabricator.wikimedia.org/T178567#3696395 (10BBlack) Most likely the error is just inconsistent over the time domain at the backend (swift -> (MW || Thumbor)). If Varnis... [17:51:12] 10Traffic, 10Operations, 10PAWS, 10Pywikibot-Commons: Server error (500) while trying to download files from Commons from PAWS - https://phabricator.wikimedia.org/T178567#3700231 (10zhuyifei1999) I don't think thumbor should be involved here. The script fetch original versions of the files, not the thumbna... [18:28:36] I'm sure bblack used http://www.ipv6buddy.com/ to write those DNS records [18:30:09] lol [18:31:04] do they actually sell that thing? I figured at first it was just a joke.... [18:33:18] I think it's both [19:25:02] 10netops, 10Operations, 10Patch-For-Review: Find a new PIM RP IP - https://phabricator.wikimedia.org/T167842#3700551 (10hashar) [19:25:06] 10netops, 10Continuous-Integration-Infrastructure, 10Operations, 10Jenkins, and 2 others: Disable Jenkins autodiscovery system - https://phabricator.wikimedia.org/T178608#3700549 (10hashar) 05Open>03Resolved `netstat -gn` seems to indicate there is no more any multicast. [19:27:02] 10netops, 10Operations, 10Patch-For-Review: Find a new PIM RP IP - https://phabricator.wikimedia.org/T167842#3346542 (10hashar) >>! In T167842#3352923, @BBlack wrote: > ... these were the surprises: > ``` > ===== NODE GROUP =====... [19:27:42] XioNoX: the Jenkins multicast listener is gone from contint1001 ^^ [20:04:57] 10Traffic, 10Operations, 10PAWS, 10Pywikibot-Commons: Server error (500) while trying to download files from Commons from PAWS - https://phabricator.wikimedia.org/T178567#3700598 (10BBlack) So, I did some varnishlog tracing on the frontend @zhuyifei1999 was hitting with a reproduction of this. I caught on... [20:06:12] 10Traffic, 10Operations, 10PAWS, 10Pywikibot-Commons, 10media-storage: Server error (500) while trying to download files from Commons from PAWS - https://phabricator.wikimedia.org/T178567#3700601 (10BBlack) [23:31:06] 10Traffic, 10netops, 10Operations: Japanese hotel resolving to esams and going the wrong way round - https://phabricator.wikimedia.org/T178726#3700842 (10Reedy) [23:31:29] 10Traffic, 10netops, 10Operations: Japanese hotel resolving to esams and going the long way round - https://phabricator.wikimedia.org/T178726#3700855 (10Reedy)