[01:31:24] 10DBA, 10MediaWiki-General, 10Patch-For-Review, 10Schema-change, 10Technical-Debt: Normalise MW Core database language field length - https://phabricator.wikimedia.org/T253276 (10Krinkle) [01:32:22] 10DBA, 10MediaWiki-General, 10Patch-For-Review, 10Schema-change, 10Technical-Debt: Normalise MW Core database language fields length - https://phabricator.wikimedia.org/T253276 (10Krinkle) [04:49:33] 10DBA, 10MediaWiki-General, 10Patch-For-Review, 10Schema-change, 10Technical-Debt: Normalise MW Core database language fields length - https://phabricator.wikimedia.org/T253276 (10Marostegui) p:05Triage→03Medium I hace checked the gerrit patch - should be an easy schema change I think. We receive qui... [04:51:55] 10DBA, 10cloud-services-team (Kanban): Reimage labsdb1011 to Buster and MariaDB 10.4 - https://phabricator.wikimedia.org/T249188 (10Marostegui) Labsdb1011 has been working fine since Thursday, the lag though keeps growing now less fast, but still there. I am going to go ahead and do the above test: ` Depool... [04:58:40] 10DBA: tendril_purge_global_status_log_5m and global_status_log needs more frequent purging - https://phabricator.wikimedia.org/T252331 (10Marostegui) With the latest changes looks like we have `global_status_log_5m` under control. It doesn't grow over 35-40M rows. I am going to spend time now with the other big... [05:13:15] 10Blocked-on-schema-change, 10DBA: Apply Babel schema change expanding babel_lang in Wikimedia production - https://phabricator.wikimedia.org/T253342 (10Marostegui) a:03Marostegui [05:52:57] 10DBA, 10Patch-For-Review: Add more information to --help option of transfer.py - https://phabricator.wikimedia.org/T253219 (10Privacybatm) Just have a look at our documentation: https://transferpydoc.imfast.io/index.html :-) [07:10:05] 10DBA: kill_job function in remote execution module of transfer framework does not close the ports instantly - https://phabricator.wikimedia.org/T252950 (10Privacybatm) what we do now is, we start a nc-listen command in the target machine with the `start_job` which makes a new process in the framework running ma... [09:04:36] 10DBA: kill_job function in remote execution module of transfer framework does not close the ports instantly - https://phabricator.wikimedia.org/T252950 (10jcrespo) I see- our command does some piping- this means that it generates a few subprocesses with a single command. I think the right way would be to use ne... [09:16:32] 10DBA: kill_job function in remote execution module of transfer framework does not close the ports instantly - https://phabricator.wikimedia.org/T252950 (10jcrespo) Starting to work a bit on PIDs and ports for our library of methods would not be a waste of time, as we may want to reuse it later for concurrency h... [10:01:16] sorry, got invested in a review, will enter late to the meeting [11:44:31] 10DBA: tendril_purge_global_status_log_5m and global_status_log needs more frequent purging - https://phabricator.wikimedia.org/T252331 (10Marostegui) This is approximately what `global_status_log` has accumulated since 10th May (15 days): ` mysql:root@localhost [(none)]> show explain for 102846891; +------+---... [12:30:51] 10Blocked-on-schema-change, 10DBA: Apply Babel schema change expanding babel_lang in Wikimedia production - https://phabricator.wikimedia.org/T253342 (10Marostegui) [13:03:15] 10DBA: kill_job function in remote execution module of transfer framework does not close the ports instantly - https://phabricator.wikimedia.org/T252950 (10Privacybatm) Thank you for the suggestion, I will try with netcat. [13:03:27] 10DBA, 10Operations: In-place conversion from LVM to normal partition - https://phabricator.wikimedia.org/T252195 (10Kormat) a:03Kormat Here's a ~finished version of the PoC: {P11298} It has error handling, will silently exit if there is no LVM on the machine, and uses low-level lvm commands instead of need... [13:16:04] 10DBA: kill_job function in remote execution module of transfer framework does not close the ports instantly - https://phabricator.wikimedia.org/T252950 (10jcrespo) Sorry, when I said netcat before, I meant netstat. =:-D [13:19:45] 10DBA: kill_job function in remote execution module of transfer framework does not close the ports instantly - https://phabricator.wikimedia.org/T252950 (10jcrespo) fuser will actually be more elegant than netstat: ` fuser 4040/tcp ` and can kill processes too: ` fuser -k 4040/tcp ` We could also use this b... [13:22:09] 10DBA, 10Operations: In-place conversion from LVM to normal partition - https://phabricator.wikimedia.org/T252195 (10Marostegui) Thanks for working on this. This is probably not an issue, but worth checking that this works as expected with both HP and Dell controllers. Again, shouldn't be an issue, but worth... [13:44:54] 10DBA: Exception raised when setting trivial, but incorrect parameters to transfer.py - https://phabricator.wikimedia.org/T253560 (10jcrespo) [13:45:30] I am tagging GSoC tickets with DBA, sorry if there is spam, but I cannot find a better alternative [13:45:47] no worries :) [13:45:55] 10DBA: Exception raised when setting trivial, but incorrect parameters to transfer.py - https://phabricator.wikimedia.org/T253560 (10jcrespo) [13:46:05] technically we want to know that is going on there [13:46:12] make create a column for gsoc on the dashboard? [13:46:33] I can do so [13:46:39] doing [13:46:43] thank you! [13:49:18] 10DBA, 10Google-Summer-of-Code (2020): GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [13:53:18] 10DBA: Exception raised when setting trivial, but incorrect parameters to transfer.py - https://phabricator.wikimedia.org/T253560 (10Privacybatm) We are happy to help :D [13:55:20] 10DBA: kill_job function in remote execution module of transfer framework does not close the ports instantly - https://phabricator.wikimedia.org/T252950 (10Privacybatm) I will try with fuser then! Thank you! [13:56:26] 10DBA, 10Google-Summer-of-Code (2020): GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) Hey, I have created a specific column for tasks related to the GSOC, on the DBA project- I think we should use that to classify it under the D... [14:17:15] 10DBA, 10Google-Summer-of-Code (2020): GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10Privacybatm) Okay, I will use GSOC column for the tickets. Thank you! [14:37:48] 10DBA: Upgrade and restart s1 (enwiki) primary database master: Thu 21th May - https://phabricator.wikimedia.org/T251982 (10Trizek-WMF) [14:56:11] jynus: you have almost aced the addition of doc publishing for wmfmariadbpy :) [14:58:25] you mean aced to make it all wrong? [14:59:13] :-( [15:01:07] na [15:01:15] jynus: I replied on https://gerrit.wikimedia.org/r/#/c/integration/config/+/598492/1/jjb/operations-misc.yaml [15:01:25] thanks for the help! [15:01:30] it fails because zuul config refers to a job named transferpy-tox-publish [15:01:34] which does not exist [15:01:41] so one question [15:01:47] albeit the error message is hidden in 65MBytes of console log bah [15:01:59] should we create one per doc or one per repo? [15:02:13] (one job) [15:02:15] so there is a single rpeo [15:02:18] yes [15:02:22] but it has two modules [15:02:25] yes [15:02:28] the doc is just for the transferpy repo [15:02:33] correct [15:02:55] wmmariadbpy repo, transfer.py module [15:03:00] what the job is doing is roughly: tox -e doc && rsync [15:03:19] so that would run tox from the root of the repository [15:03:25] ok [15:04:02] and that would end up being published at https://doc.wikimedia.org/ [15:04:11] so https://doc.wikimedia.org/transferpy/doc/build/ [15:04:20] ok, that I think clarifies it [15:04:22] we might want to change docdest to just "transferpy" [15:04:25] it was the internal name [15:04:29] yeah :/ [15:04:48] that got mislead as I wasn't fully comprehending the autofileld variables [15:04:48] or maybe we can have the doc for the whole repository? and have a single doc config for all the modules the repository has [15:05:08] I think for now we can have just one for trasnferpy [15:05:25] and we will walk when other modules has good documentation [15:05:38] at that time it may make sense to just split the repo into several other repos [15:05:50] possibly yeah [15:05:53] this is the only thing with active development for now [15:06:01] so I didn't want to make things more complicated [15:06:20] so I will just alias the repo transferpy for the docs [15:06:31] and that should work good enough [15:07:37] I also have to work with the student on settining the sphyinx build correctly [15:11:28] jynus: yeah should not be too hard [15:11:59] the job right now would 'tox -e py37-sphinx' from the root of the repo [15:12:00] cumin helped both of us, as it was setup similarly and and same stack [15:12:18] I have to check which python version we are supposed to be using [15:14:04] marostegui, jynus : lvm support added. https://phabricator.wikimedia.org/T252027#6163226 [15:14:33] hashar: yeah, top hierarchy does nothing at the moment with docs, but that will be up to us, don't worry [15:14:44] will try to send a good ci patch [15:14:54] but will delay it until the repo is setup [15:15:04] one can possibly add a transferpy/tox.ini file with the proper env. Then for CI the /tox.ini would need a testenv that chains to that sub transferpy/tox.ini . I have done that for operations/software to chain into sub dir ./clouseau : https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/+/master/tox.ini#8 [15:15:08] it is a bit of a hack though [15:15:24] don't worry, as I said, we will be handling that [15:15:38] cool ;]]] [15:15:44] I think it is ok to just build the transfer docs on root [15:15:51] because there are no others [15:15:53] :-D [15:16:00] ;D [15:16:11] we will review the strategy [15:16:19] when we add more docs to the same repo [15:16:26] I have subscribed to the student gerrit change to get the mail spam [15:16:27] (probably by spliting the repo itself) [15:16:41] yeah, that is still unreviewed [15:16:54] but I wanted to research this first [15:17:04] so I could advice basicaly what you told me [15:17:16] which might be a bad advice haha [15:17:18] but of course he will be happy to get more feedback! [15:17:30] oh, no, for CI I can tell you much better than mine [15:17:36] I have a good gap there [15:17:46] too many yaml files [15:17:48] well CI is really dumb [15:17:50] and templates that I get lost [15:18:10] it just run bunch of shell commands. But the three levels of yaml inception makes it a bit hard to gasp :/ [15:18:10] you may want to update the docs, however to mention doc1001 [15:21:47] we have also not decided what is the release strategy and doc update, etc. [15:22:21] let me know how that is, CI-infra wise, and if it is correct [15:22:29] I will not work on the repo patch [15:22:35] *now [15:22:48] one last question, hashar [15:23:01] I saw "experimental debian package build" [15:23:11] on CD jobs [15:23:21] how experimental is that? [15:25:06] (I don't need it, but I would guess that for packaging 7 py files maybe it would be a nice experiment for the student? [15:28:08] jynus: hmm [15:28:17] it is not really experimental indeed [15:28:25] feel free to tell me no if that is not a good idea :-D [15:28:28] but the result debian packages are not published anywhere [15:28:34] ah! [15:28:38] they are just stored in Jenkins as artifats and get deleted after X days [15:28:48] 7 < X < 30 (I cant remember) [15:29:10] I would love to have a system to ultimately push them to apt.wm.o :] [15:29:11] yeah, so maybe we should just focus on creating a proper debian source package [15:29:33] in CI jobs are named debian-glue-something [15:29:47] that uses the same setup that SRE is using for building packages, but on WMCS instance [15:29:55] so there are the hooks from puppet.git modules/package_builder [15:30:10] I should refersh whatever doc we have about debian packaging in ci [15:30:21] don't worry [15:30:29] it was just an idea to understand the status of that [15:31:05] this was mostly for me to learn so next time I can help rather than being a burden [15:31:22] oh [15:31:30] the burden is me not writing enough doc and tutorials :/ [15:31:38] no, the docs where ther [15:31:44] now that I understand them I can see [15:31:59] but you have the same issue than I have for literally this piece of software [15:32:07] I have done some update to https://www.mediawiki.org/wiki/Continuous_integration/Documentation_generation [15:32:25] the debian packaging page is hmm [15:32:26] sparse [15:32:27] https://www.mediawiki.org/wiki/Continuous_integration/Tutorials/Debian_packaging [15:32:30] the docs are clear for me, but without a bit of understanding they are very difficult to comprehend [15:33:00] ah https://wikitech.wikimedia.org/wiki/Debian_Glue [15:33:15] I am going to proofread / improve that page [15:33:43] so let me give you an example [15:33:56] * hashar listens [15:34:03] I can understand very quickly that I can select lots of options to customize my jobs [15:34:19] but my first question is: where can I see what those options are? [15:34:28] docker images, chaintools, etc.? [15:35:11] it is a rethorical question [15:35:16] don't need an answer [15:35:20] in short: there is none [15:35:32] but I hope you see where I am lost [15:35:36] one has to read the source, or namely read the various definitions in the ./jjb yaml files [15:35:54] which has projects , jobs, jobs groups, macro [15:36:07] I have the same issue, everthing for transfer.py is documented, from my point of view [15:36:14] each supporting variables that are passed from somewhere else in the chain [15:36:41] but people in my own team doesn't even know the context on how to do simple commands [15:36:55] so I need to do some UI testing and polish a lot those docs [15:36:56] luckily a lot of jobs are generic enough and can be applied as is to most repositories. So in lot of case one just have to trigger one of those generic jobs by referencing to them in zuul/layout.yaml [15:38:14] so just to be clear, I am just reflecting on myself what I am doing wrong [15:38:43] not critizising your work, which helped me a lot [15:44:58] jynus: I am always listening to feedback / critics. It is a good way to improve :] [15:45:22] no, no, what I meant is I am thinking about what I do, it doesn't apply to you [16:01:53] at least I have updated the deiban glue page ;) https://wikitech.wikimedia.org/wiki/Debian_Glue [16:02:13] I will be more than happy to pair on adding it to a repo and make it work ;] [16:02:24] I am going out for jynus . Ping me any time ;]