[07:41:42] does someone remember one ticket about ipv6 support on production? [07:42:40] probably https://phabricator.wikimedia.org/T102099? [07:57:14] jynus: did you meant T253173 ? [07:57:15] T253173: Some clusters do not have DNS for IPv6 addresses (TRACKING TASK) - https://phabricator.wikimedia.org/T253173 [08:04:05] thanks, yeah, I found also T102099 [08:04:05] T102099: Fix IPv6 autoconf issues once and for all, across the fleet. - https://phabricator.wikimedia.org/T102099 [08:04:41] maybe those shoulr be linked in some way, even if different tasks [08:04:49] that was what mo.ritz pasted above, depends what you're looking for [08:04:53] are quite orthogonal things [08:05:10] oh, sorry, I only read you, not moritzm [08:08:33] going to reimage netmon1002 with Buster shortly, and flip back later today FYI [08:10:27] cool :-) [08:12:41] we could probably do with an `ipv6` tag on phab, if we don't have it already [08:13:39] kormat: https://phabricator.wikimedia.org/project/view/184/ [08:16:45] Majavah: ah hah :) [08:27:27] ema: the solution in https://unix.stackexchange.com/a/445793/358110 worked \o/ thanks <3 [08:28:17] kormat: excellent! Have you solved the confctl build mystery? [08:28:26] nope, not a fucking clue on that one :) [08:28:58] the path to enlightenment is rarely straight [08:29:00] ohh. just noticed something. confctl does not define `PYBUILD_NAME`. i wonder if that is significant [08:29:31] and defines PYBUILD_SYSTEM=distutils [08:30:17] but yeah i don't recall how we did the dbctl separate package stuff, I/we might have tried something and justworked [08:30:17] volans: i _think_ that's ignored, as `--buildsystem=pybuild` exists [08:31:08] yeaaaah [08:31:17] so, commenting out PYBUILD_SYSTEM has no effect, [08:31:29] but adding PYBUILD_NAME=confctl breaks everything [08:31:35] ahahahahah [08:31:54] ahhh maybe it's because without that it accepts blissfully the multipackage [08:32:05] while with that you're forcing a name? [08:32:40] kormat: You usually won't need debian/python-foo.install or debian/python3-foo.install files even if you have multiple binary packages, because again, pybuild does the magic for you (if PYBUILD_NAME is correctly set). You might need these if there are some non-standard installation tricks you need to implement. [08:32:45] `if more than one binary package is build: add debian/python-foo.install files, or export PYBUILD_NAME=modulename` - i'm not sure how to interpret that [08:32:45] from https://wiki.debian.org/Python/Pybuild [08:33:23] I think it means export from the shell that builds [08:33:29] and not setting it in debian/ [08:34:19] that bit from the wiki page is the usual level of 'helpful' in this area. it sounds great, but it gives no goddamn clue as to how things are supposed to work, which is impossible to debug when they don't. [08:34:40] indeed [08:34:43] it's magic [08:34:59] "it's magic, you don't need to know the details. if it doesn't work for you, you're not worthy" [08:35:34] lol [08:35:40] sorry for the trouble [08:35:49] I might have make it worse pointing you to the confctl one [08:36:13] volans: i'm happy to both thank you for your support, and blame you for your support :) [08:39:26] it's called Schrödinger's support ;) [08:43:41] kormat: ok so I think I found out something more. Try building both wmfmariadbpy and conftool with `fakeroot debian/rules binary` [08:43:45] then look under debian/ [08:43:58] you'll see that the latter does not have a "tmp" directory, while the latter does [08:44:02] hem [08:44:13] the former (wmfmariadbpy) does not [08:46:02] I think but cannot prove that the files listed under debian/python3-conftool.install are found under `conftool/` and under `debian/tmp/` [08:47:24] and indeed there's: debian/tmp/usr/bin/confctl debian/tmp/usr/bin/conftool-sync debian/tmp/usr/lib/python3.7 [08:47:55] From debhelper compatibility level 7 on, dh_install will fall back to looking in debian/tmp for [08:47:59] files, if it does not find them in the current directory (or wherever you've told it to look [08:47:59] ahh. so maybe everything goes under debian/tmp, and then the `*.install` files copy that stuff over to the per-package dirs [08:48:02] using --sourcedir). [08:48:04] that's from dh_install(1) [08:50:38] bingo. ok, that's how it works [08:50:48] i'm even more impressed at just how bad the docs are [08:51:50] dh_install(1) is alright [08:51:53] look at gbp-buildpackage(1) [08:52:42] ema: i'm not complaining about dh_install. i'm complaining about dh_python3/pybuild [08:54:27] kormat: the great news is that you're now in the best position to improve them :) [08:54:42] :doubt: [12:09:49] do we have maxmind geoip data available on all mw application servers atm? [12:19:08] mark: yes should be avalible in /usr/share/GeoIP [12:20:08] included via profile::mediawiki::common -> geoip -> geoip::puppet [12:49:13] yeah, thanks [13:54:57] <_joe_> mark: at the same time, php-geoip is broken [13:55:07] <_joe_> since it only supported maxmind v1 [14:08:59] _joe_: so what's up with the cloudelastic conftool service names anyway [14:09:12] <_joe_> cdanis: I have no idea [14:09:28] I see [17:54:24] kubecon spam -- "Register by August 3, 2020 at 23:59 PDT and you will be entered into a drawing* to win one of the below gift boxes." -- talk to your manager to get the magic $0 USD registration cost code. If they don't have it, send them to me to get it. :) [21:49:14] Not sure whether here or #wikimedia-cloud is the right place, but anyone see anything obviously wrong with what I'm doing here? : [21:49:21] I'm trying to connect to `wdqspuppet.wikidata-query.eqiad.wmflabs`. Per the table `How should you proxy?` from https://wikitech.wikimedia.org/wiki/Help:Accessing_Cloud_VPS_instances#Accessing_Cloud_VPS_instances I should be proxying through `restricted.bastion.wmflabs.org`. Also my shell user is `ryankemper`. My development (gerrit/wikitech) aka cloud ssh key is my `id_ed25519`. [21:50:31] As a manual command that gives me `ssh -vvv -J ryankemper@restricted.bastion.wmflabs.org ryankemper@wdqspuppet.wikidata-query.eqiad.wmflabs`, which isn't working despite `-vvv` showing `debug1: Offering public key: /Users/user/.ssh/id_ed25519 ED25519 SHA256:C2RercsX1RqQlAsNEsEXt0jjcIzwLISHgtCsdVgbe1s explicit` [21:51:31] To start I figure I should ssh into `ryankemper@restricted.bastion.wmflabs.org`, which fails in the same way, it proceeds through the default keys including my cloud key and then goes into the keyboard-interactive mode once those have failed [21:51:47] So I guess the current question is, what could I be doing wrong in connecting to `restricted.bastion.wmflabs.org`? [21:52:21] ryankemper: the one that works for me is bastion-restricted.wmflabs.org fwiw [21:53:12] -cloud will be better to make sure what the differences are [21:53:28] Yeah I saw that in the ssh config (don't remember where I found that config originally tho) and tried `ssh ryankemper@bastion-restricted.wmflabs.org -i ~/.ssh/id_ed25519 -vvv` which fails in the same way [21:53:54] (the `-i` is unnecessary but I included it out of paranoia) [21:56:02] ryankemper: /var/log/auth.log on bastion-restricted-eqiad1-01 says "failed publickey" [21:56:15] the user is correct and exists [21:56:30] it is also a member of 52354(project-wikidata-query) [21:56:43] And this publickey should match the one in my gerrit settings? Or is there somewhere else I need to stick my key [21:56:52] the gerrit key is unrelated [21:56:53] have you put it in wikitech? [21:56:58] /horizon? [21:56:58] it can be the same or a different one [21:57:08] what Reedy said [21:57:39] https://wikitech.wikimedia.org/wiki/Special:Preferences#mw-prefsection-openstack [21:57:40] or [21:58:06] ah, yup that is my problem, no key in wikitech [21:59:23] there are 3 keys, prod shell key (in puppet repo), wmcs key (in wikitech), gerrit key (in gerrit). the first 2 need to be separate, the third can be either or yet another one [21:59:54] Thanks, that's helpful [21:59:56] I'm in now [22:00:04] (after adding my key) [22:00:18] great