[08:40:55] volans: hi! how do we manage setup.py dependencies on other wmf python packages? [08:41:05] do they need to be published to pypi? [08:41:34] kormat: usually yes, we don't have a private or custom pypi repo [08:41:54] :sad_face: [08:42:04] if something very specific to us I've used a wmf prefix on pypy even on things that don't have that internally [08:42:19] like https://pypi.org/project/wikimedia-spicerack/ (s/wmf/wikimedia/) [08:42:27] even though the library and debian packages are just spicerack [08:44:46] if you need a hand lmk, I have it all laid out for the other packages I maintain [09:14:18] <_joe_> what is needed to maintain a local pypi repo, btw? I have no idea [09:14:54] separate with just our packages or a mirror + our pacakages? [09:14:57] I don't know either [09:15:11] you run https://pypi.org/project/devpi/ [09:15:50] but then you have to add it everywhere you build venvs [09:16:01] so should be publicly reachable (for our local envs) [09:16:06] etc.. [09:16:11] i think you have to have it mirror pypi.org [09:16:23] as i don't think pip allows you to say which repo to fetch which package from [09:16:57] (it does the mirroring on-demand - if a package is requested it doesn't have locally, it'll fetch it from upstream and cache it) [09:17:08] it's kinda pants though [09:19:40] kormat: congratulations, you have volunteered to setup a private pypi repo :-] [09:20:16] more seriously, yes folks are publishing to the upstream package managers (packagist.org for php, npmjs.com for nodejs, pypi for python) [09:20:59] we once had the idea of setting up our own PHP package archive, but eventually had no incentive to do so when packagist just work [09:22:39] an alternative is to use git in the requirement: git+https://gerrit.wikimedia.org/r/operations/software/transferpy.git@master#egg=transferpy [09:24:28] kormat: and one can point to a private pypi setup using --index-url: pip install --index-url http://127.0.0.1/pypi/simple [09:26:07] huuh, interesting re: git requirement [09:26:20] volans: that would save a whole bunch of pain, what do you think? [09:26:25] for devpi I have some notes at https://phabricator.wikimedia.org/T114871#1708391 [09:26:31] and something saying: [09:26:32] export PIP_INDEX_URL=http://pmcache.integration.eqiad.wmflabs:3141/root/pypi/ [09:26:32] export PIP_TRUSTED_HOST=pmcache.integration.eqiad.wmflabs [09:26:57] (that instance no more exists, but that hints at how one can point pip to a different repo [09:27:02] kormat: not sure if you can specify tags, and how that would go our of hand quickly [09:27:06] but something we could investigate [09:27:13] you can specify a tag: [09:27:21] transferpy.git@1.0.0#egg=transferpy [09:27:37] I don't think it gpg validates it though [09:28:00] hashar: devpi requires a Lot of work to make it work from my workstation as well as from CI i'm guessing [09:28:06] yeah [09:28:19] so the git approach is very attractive, as it requires making git tags, aand that's it [09:28:21] theorically we could have our own wikimedia package store [09:28:34] but it seems easier to just delegate that maintenance to 3rd parties and publishes there [09:29:38] in an ideal world, adding a tag to a Gerrit repository would be restricted to the repo owner/releaser. Then CI could react on the tag creation to run whatever commands are needed to build the package and ultimately publish it to pypi [09:30:23] which is probably straightforward given CI already knows how to run something when a tag is created, and we can inject credentials to run whatever pip command is used for publication [09:31:05] the placeholder task https://phabricator.wikimedia.org/T182657 [10:52:33] hmm, I seem to not be authorised to ack alerts in icinga any more [10:53:19] hnowlan: jynus had the same issue yesterday, but I didn't follow up closely what was the fix for that [10:53:20] capitalization problem I would guess [10:53:46] hnowlan: I was able to ack on another browser session [10:54:03] but couldn't do it on my normal browser [10:54:09] yeah, it was capitalisation :) [10:54:53] didn't set it properly in the new login pane but had it in the old http auth [10:55:11] that would get it