[09:51:05] hashar: is there a way to use docker-compose in our CI? [09:53:46] kormat: nop [09:54:29] kormat: we only run images we have build ourselves [09:54:44] most jobs only run a single image that has everything and includes an entry point / test script to run [09:55:11] there was a plan to eventually adopt Kubernetes as the running system [09:55:22] mmph [09:55:35] in which a job would have a k8s job in a pod which could have several containers [09:55:42] as to how to define them, we never went that far [09:55:54] the k8s idea got deprioritized in favor of something else [09:56:20] interestingly, last week I had a similar conversation for testin gmediawiki [09:56:40] the test runner (Quibble) spawns mysql/php builtin webserver itself [09:56:53] but it might be interesting to have a container for mysql and another one for php/apache [09:57:11] so our test runner would just be given the ip of the mysql db and webserver instead of having to spawn them [09:57:22] which has lead us to mention docker-compose as well [09:57:36] i'll write up a task for what i have in mind [09:57:59] there are a few other use cases that could use running docker containers as well, such as the puppet beaker suite [09:58:16] which can provision a docker container with puppet and is used for end to end tests of puppet [09:58:23] jbond42 knows about beaker ;) [09:58:47] tentatively, we could have a VM that comes with Docker and isolated from the rest of the infra [09:58:54] and lets jobs run whatever containers in them [10:01:34] hashar: https://phabricator.wikimedia.org/T265266 is what i'm looking for [10:13:20] kormat: can it be done by the testsuite itself? [10:13:40] I mean, have the integration test suite written in python to spawns the mariadb instances it needs to accomplish the test [10:14:44] hashar: maybe, but it gets a bit clumsy for development [10:15:13] I do that with Quibble, using mysql_install_db to craft an empty database at some directory [10:15:19] then start it with some magic invocation ( https://gerrit.wikimedia.org/r/plugins/gitiles/integration/quibble/+/refs/heads/master/quibble/backend.py#260 ) [10:15:55] so potentially, you could even have each test to start with an entirely fresh set of database [10:16:51] hashar: i'd probably have a standalone script to manage the database cluster, and have the test suite use it [10:16:52] and if ones want to point the test suite to existing servers, that could be done by setting some environment variable which would prevent the testsuite from creating/spawning the databases and use the info given via the environment [10:17:12] so locally on development you could use a docker composer that spawns X mysql server, set the env variable and the testsuite will happilly use those [10:17:22] while on CI or without docker composer, the testsuite itself would take care of that [10:17:43] or yeah a script to spin them up [10:18:30] then potentially that could fit in the current CI given the job is using releng/tox-mysql image which does have the mariadb binary [10:20:19] can the version of mariadb be selected in CI? [10:20:36] because being able to qualify the code against different versions would be really useful [10:21:01] scratch that - essential. [10:24:30] the image is based on stretch and installs mysql-server / perconal-toolkit [10:24:34] that is all it has right now [10:25:56] that's what i feared. docker would definitely be a much better fit [11:19:09] <_joe_> please everyone: take a look at T257118 and claim VMs you know are needed. [11:19:09] T257118: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118