[05:55:42] 10DBA, 10Patch-For-Review: transferpy 1.0 release - https://phabricator.wikimedia.org/T257601 (10Privacybatm) >>! In T257601#6349705, @jcrespo wrote: > ` > 16:28:13 Warning, treated as error: > 16:28:13 /src/transferpy/transfer.py:docstring of transferpy.transfer.to_bool:5:Field list ends without a blank line;... [06:01:10] 10DBA: transferpy: Multiprocess the transfers - https://phabricator.wikimedia.org/T259327 (10Privacybatm) [06:24:00] 10DBA: Make checksum parallel to the data transfer in transferpy package - https://phabricator.wikimedia.org/T254979 (10Privacybatm) I think we can close this! What do you think? [06:24:36] 10DBA: Solve transferpy concurrency issue with auto port detection and checksum file names - https://phabricator.wikimedia.org/T256450 (10Privacybatm) I think we can close this! What do you think? [07:26:04] 10DBA: Make checksum parallel to the data transfer in transferpy package - https://phabricator.wikimedia.org/T254979 (10jcrespo) 05Open→03Resolved [07:26:06] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [07:26:30] 10DBA: Use logging package instead of print statements in transferpy package - https://phabricator.wikimedia.org/T255999 (10jcrespo) 05Open→03Resolved [07:26:34] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [07:26:48] 10DBA: Solve transferpy concurrency issue with auto port detection and checksum file names - https://phabricator.wikimedia.org/T256450 (10jcrespo) 05Open→03Resolved [07:26:51] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [07:27:17] 10DBA: transferpy --checksum wrongly output `checksums do not match` message - https://phabricator.wikimedia.org/T256755 (10jcrespo) [07:27:20] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [07:27:44] 10DBA, 10Patch-For-Review: Create more tests for transferpy package - https://phabricator.wikimedia.org/T257600 (10jcrespo) 05Open→03Resolved [07:27:47] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [07:28:33] 10DBA: Create temp and config directories at the installation time of transferpy deb package - https://phabricator.wikimedia.org/T257599 (10jcrespo) 05Open→03Resolved We agreed temp cannot be created at install time as the temp directories are created on the remote hosts, not the place where the package is i... [07:28:36] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [07:29:00] 10DBA: Make transferpy configurable using a configuration file - https://phabricator.wikimedia.org/T257602 (10jcrespo) 05Open→03Resolved [07:29:03] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [07:29:43] 10DBA, 10Patch-For-Review: transferpy: Multiprocess the transfers - https://phabricator.wikimedia.org/T259327 (10jcrespo) [07:29:46] 10DBA, 10Google-Summer-of-Code (2020), 10Patch-For-Review: GSoC 2020 Proposal: Improve the framework to transfer files over the LAN - https://phabricator.wikimedia.org/T248256 (10jcrespo) [08:59:04] kormat: I am working with luca on pms, we can reconsider the patch I sent with manuel tomorrow [08:59:29] but I think it was good enough to just send it- I won't even reload config for now [09:00:21] ack [09:00:39] except on the intnded hosts- the analytics one [09:00:54] I think we set it to 7 long ago because space restrictions [09:01:08] we can put it low back again when we have centralized backups for binlogs [09:04:48] * kormat nods [09:06:14] we push it to the back down of the pile low priority because we have one copy on every one of the 20 dbs per section [09:06:34] but will be needed/handy for the automatic point in time recovery [09:22:08] jynus: re: port 3351/3352, i think that needs to be updated in wmfmariadbpy too [09:24:39] oh, true [09:25:15] and that is why we need a single library for that stuff [09:25:19] yep [09:28:12] I am not doing but the minimum effort, as I told you I would like to see that on a puppet yaml [09:32:06] https://gerrit.wikimedia.org/r/c/operations/software/wmfmariadbpy/+/617661 [09:37:06] jynus: oops, my review crossed your update [09:37:49] np [09:38:07] you can re-review on new version [09:38:30] and/or close comments [09:38:46] done [09:39:13] and yes, this is calling for an array- we will get there when we have a plave to read that [09:39:30] were you actually working on that or did you end up distracted? [09:40:21] because of your work on the switchover script, I mean [09:40:48] I am guessing it paused because now you will be querying zarcillo? [09:43:28] it's on the todo list, but somewhat lower priority for the moment [09:44:53] more than understandable [10:25:47] 10DBA, 10Goal: Expand database provisioning/backup service to accomodate for growing capacity and high availability needs - https://phabricator.wikimedia.org/T257551 (10jcrespo) [11:15:55] we have some oom ongoing on dbprov1004 [11:18:54] I don't see any ongoing query [11:19:02] I am going to restart s4 at dbstore1004 [12:18:26] 10DBA, 10MediaWiki-extensions-OAuthRateLimiter, 10Patch-For-Review, 10Platform Team Initiatives (API Gateway), and 3 others: Review request for a new database table for OAuthRateLimiter - https://phabricator.wikimedia.org/T258711 (10WDoranWMF) [15:23:44] 10DBA, 10MediaWiki-Page-derived-data, 10Platform Engineering, 10TechCom-RFC, and 2 others: RFC: Normalize MediaWiki link tables - https://phabricator.wikimedia.org/T222224 (10Krinkle) @TK-999 Hi, I wonder if Fandom have perhaps done something in this area? [15:25:50] 10DBA, 10MediaWiki-Page-derived-data, 10Platform Engineering, 10TechCom-RFC, and 2 others: RFC: Normalize MediaWiki link tables - https://phabricator.wikimedia.org/T222224 (10TK-999) >>! In T222224#6351967, @Krinkle wrote: > @TK-999 Hi, I wonder if Fandom have perhaps done something in this area? @Krinkle... [17:06:07] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for lijwikisource - https://phabricator.wikimedia.org/T258389 (10Bstorm) Ok, DNS script run. It's not very //targeted// so it will not need to be run for: `lang=shell-session $ wmcs-wikireplica-dns --aliases --shard s3 W... [17:12:29] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for lijwikisource - https://phabricator.wikimedia.org/T258389 (10Bstorm) Works fine 🙂 `lang=shell-session bstorm@tools-sgebastion-08:~$ sql lijwikisource Reading table information for completion of table and column nam... [17:12:35] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for lijwikisource - https://phabricator.wikimedia.org/T258389 (10Bstorm) 05Open→03Resolved [17:32:55] 10DBA, 10Data-Services, 10User-Ladsgroup, 10cloud-services-team (Kanban): Prepare and check storage layer for shnwiktionary - https://phabricator.wikimedia.org/T256010 (10Bstorm) 05Open→03Resolved a:03Bstorm All scripts run and tested auth. This is good to go. [17:58:18] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for arywiki - https://phabricator.wikimedia.org/T257725 (10Bstorm) 05Open→03Resolved a:03Bstorm All set up from the cloud side and tested connection and auth from Toolforge. [18:56:26] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for avkwiki - https://phabricator.wikimedia.org/T258077 (10nskaggs) I created the views and confirmed it's existence on toolforge. Should be all set! ` sql avkwiki show tables; select * from revision limit 10; ` [18:56:35] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for avkwiki - https://phabricator.wikimedia.org/T258077 (10nskaggs) 05Open→03Resolved [22:34:29] 10DBA, 10Patch-For-Review, 10User-Urbanecm: Move muswiki and mhwiktionary (closed wikis) from s3 to s5 - https://phabricator.wikimedia.org/T259004 (10Urbanecm) @Marostegui Hello, please note my availability will be limited during August 4-6. Since your vacation ends on Aug 3, I guess we should do this in the...