[09:11:35] if anybody has time for some mesh-config-upgrade-related code changes https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1144455 (onward) :) [09:17:55] +1ed various ones [09:32:35] <3 [09:32:39] going to merge some [09:32:51] I have only 20 lined up, total of 45 :( [09:33:35] me and Filippo already checked the airflow metrics, all perfect, so the changes can be probably picked up by others when deploying [09:33:40] I'll send an email to ops@ [13:50:13] Hmm, has anybody seen some deployment-charts CI failure caused by `rake aborted! NoMethodError: undefined method `filter!' for true:TrueClass` [13:50:20] example in https://integration.wikimedia.org/ci/job/helm-lint/24718/console [13:51:18] note that the associated patch only touches on the airflow chart, and not our rakefile/ci setup [13:51:59] I've rebuilt the jenkins job, to no avail [13:52:27] really weird, does it render locally? [13:52:30] I mean rake [13:53:49] brouberol: we (serviceops) are working on it ATM [13:54:16] heisenbug introduced a few commits ago that went unnoticed because heisenbug [13:54:48] but we're going to either work around it or revert it soon [13:55:12] Raine: thanks! [13:56:29] can't wait to see the show played by Brian Cranston [13:58:08] ah nice! [14:05:42] Walter White, a chemistry teacher, discovers that he has cancer and decides to get into the helm CI business to repay his medical debts. [14:05:58] what could possibly go wrong :D [14:15:21] I'd pay for that [14:21:41] the commit msg of https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1145195 is simply great [14:32:14] second batch of mesh config changes! https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1145214 [14:58:53] ... ah I see I just ran into the same CI failure as b.rouberol earlier [14:59:46] is there a task open? [15:12:03] cdanis: I don't think so...but the workaround should be merged [15:12:27] I've run a recheck about 10-15min ago and still bumped into the same failue FWIW [15:12:38] seems like it does not solve the issue for you - oh [15:13:01] was merges ~14.30Z [15:13:07] *merged [15:17:11] yeah I have https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1145242 pending, it is up to date with master, and a recheck also fails [15:18:07] hm...shit [15:23:09] akosiaris is gonna be so mad his v. involved and smart fix didn't work :( [15:23:14] s/mad/sad/ [15:24:24] I'll revert my patch that triggered this for now [15:55:13] didn't work? Oh dammit [15:55:23] I've submitted the revert [15:55:39] https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1145248 [15:55:43] will take a bit [15:58:16] ok [16:12:13] cdanis: should be good [16:15:05] I 'll also revert my ugly ugly ugly sleep patch [16:17:00] akosiaris: did so already [16:17:56] thanks [16:18:19] why am I not seeing it in the git pull [16:18:51] akosiaris: I squashed it with my revert to not run rake all 4 times [16:18:52] ah, got abandoned? [16:18:57] ah cool, makes sense [16:18:58] thanks