[00:09:20] paladox: now would be good to start the replication one more time [00:09:43] wants to stop and restart it though [00:15:42] !log ssh -p 29418 paladox@gerrit.wikimedia.org replication start --url gerrit2001.wikimedia.org --all [00:15:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:15:48] mutante {{done}} [00:16:46] paladox: thanks, i see replication.log is active [00:16:51] :) [00:17:06] * paladox sees tasks going down [00:17:41] mutante you can start gerrit up :) [00:18:02] paladox: actually it's supposed to be up :( [00:18:06] It dosen't use the index [00:18:09] oh? [00:18:54] paladox: fails to start ..on 2001 [00:18:58] oh [00:19:03] mutante what's in the error log? [00:19:46] Cannot connect to SQL database [00:19:47] heh [00:19:55] huh [00:20:00] oh [00:20:08] mutante did you symnlink the driver in lib/? [00:20:25] /var/lib/gerrit2/review_site/lib [00:20:40] no.. i only did the scp .. [00:20:48] ok [00:20:49] but trying to stricly follow our own docs [00:21:01] mutante we should add symnlink temporarily to the doc [00:21:49] paladox: missing on https://wikitech.wikimedia.org/wiki/Gerrit#Migrating ? [00:21:57] yeh [00:22:41] unsure about the step for /var/lib/gerrit/review_site/ [00:22:44] since it's a replica [00:22:55] we dont want to copy the ./etc/ for sure [00:22:58] you need to symnlink the mysql driver into lib/ [00:23:08] what we did for gerrit1001 [00:23:10] yea, doing that first..it was a separate question [00:23:16] oh [00:27:20] mutante https://gerrit-replica.wikimedia.org/r/ works [00:28:08] paladox: yes:) just logged in operations and started it [00:28:11] hmm https://gerrit-replica.wikimedia.org/r/monitoring/ [00:28:17] is 404 for me [00:28:26] yea .. [00:29:13] apparently it's all 404's [00:29:15] https://gerrit-replica.wikimedia.org/g/ [00:29:18] should be working [00:29:24] mutante any errors in the log? [00:29:56] git clone works [00:29:57] though [00:30:06] so it's the plugins not working, perms issue? [00:30:13] paladox: i just tried that but the repo i used was not found [00:30:20] oh? [00:30:29] yes, perms issues because the UIDs keep changing [00:30:33] i just did: [00:30:33] git clone https://gerrit-replica.wikimedia.org/r/mediawiki/core testtttttt [00:30:40] which didn't 404 [00:31:01] confirmed. i tried parsoid/deploy though [00:31:19] mutante mediawiki/services/parsoid/deploy [00:31:36] oh, right :) [00:31:46] yea, let me fix permissions then [00:32:04] ok :) [00:34:16] hmm https://gerrit-replica.wikimedia.org/g/ still throws [00:34:22] well not throws but 404 [00:34:22] indeed [00:34:35] anything in gerrit.log? [00:37:17] that it cannot load some plugins .. and then that it's ready ..as a slave [00:37:34] ok [00:37:40] checking perms on plugins dir [00:37:42] is there a way to re deploy the plugins? [00:37:53] what does the log specifically say? [00:38:59] Cannot load Jar plugin /var/lib/gerrit2/review_site/plugins/zuul.jar [00:39:14] Cannot load plugin webhooks [00:39:18] ok, that'll be because it cannot read it [00:39:22] maybe wrong chmod [00:39:28] Ignoring missing gc schedule configuration [00:39:29] i've had that before too [00:39:39] i just redploy [00:39:46] *reploy the plugins [00:40:07] (03PS1) 10Jforrester: phpunit: Drop --debug-tests command, killed off in PHPUnit 8 [integration/quibble] - 10https://gerrit.wikimedia.org/r/547843 (https://phabricator.wikimedia.org/T192167) [00:40:22] mutante is /var/lib/gerrit2/review_site/plugins/zuul.jar that symnlinked to scap? [00:40:29] ls -la /var/lib/gerrit2/review_site/plugins/zuul.jar [00:41:32] paladox: ^ that isn't a link on either of the servers [00:41:34] zuul.jar: Zip archive data, at least v1.0 to extract [00:41:38] same on both [00:41:52] and both owned gerrit2:gerrit2 [00:42:35] ls -lah /var/lib/gerrit2/review_site/plugins [00:42:36] lrwxrwxrwx 1 root root 37 Apr 18 2019 /var/lib/gerrit2/review_site/plugins -> /srv/deployment/gerrit/gerrit/plugins [00:42:58] mutante: [00:42:59] -rwxr-xr-x 1 paladox wikidev 328K Apr 18 2019 zuul.jar [00:43:05] does zuul.jar have that as chmod? [00:43:26] yes, plugins -> /srv/deployment/gerrit/gerrit/plugins but root owned! [00:43:37] did not do that manually [00:43:51] yeh, i doin't think this is a gerrit2 problem, rather chmod problem [00:49:02] Caused by: java.nio.file.NoSuchFileException: /var/lib/gerrit2/review_site/tmp/plugin_zuul_191102_0048_2154321057013518806.jar [00:49:22] the perms are the same (now).. even the link itself [00:50:16] feels like what you said about reinstalling the plugins .. after syncing plugin data [00:50:28] it is trying to find that in the tmp dir [00:50:53] oh look, /var/lib/gerrit2/review_site/tmp/ is not even a thing that exists [00:51:12] oh! [00:51:16] so either the "init" creates it .. [00:51:19] mutante create that path [00:51:23] yeh init creates it [00:51:40] well, i'm trying hard to avoid adding any manual things again [00:51:49] our docs also miss this [00:52:56] created the tmp dir and restarting [00:53:59] it created plugin_*.jar files in the tmp dir [00:54:17] ok [00:54:19] but does not mean it's fixed [00:54:37] mutante works! [00:54:49] ooh.. https://gerrit-replica.wikimedia.org/g/ :) [00:54:52] indeed, very nice [00:55:02] ok, that's great [00:55:08] because they are about to kick me out of this place [00:55:13] and i really did not want to leave that broken [00:55:17] thanks for helping [00:55:29] heh [00:55:32] i think the rest can be done on Monday [00:55:33] your welcome :) [00:55:37] thanks for fixing it! :) [00:55:38] as long as replication and the UI works [00:55:46] and the upgrade :) [00:55:49] yup [00:55:50] :) yw [01:12:51] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations: Reimage gerrit1001 and gerrit2001 as buster - https://phabricator.wikimedia.org/T176774 (10Dzahn) [01:14:29] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations: Reimage gerrit1001 and gerrit2001 as buster - https://phabricator.wikimedia.org/T176774 (10Dzahn) cobalt is mostly decom'ed. gerrit1001 is on buster and replace it. and now gerrit2001 is also on bu... [01:51:53] (03PS1) 10Ammarpad: Update RESTBase link [integration/docroot] - 10https://gerrit.wikimedia.org/r/547846 (https://phabricator.wikimedia.org/T227766) [01:52:54] 10Continuous-Integration-Config, 10RESTBase, 10Documentation, 10Patch-For-Review, 10good first bug: Remove link to defunct https://rest.wikimedia.org/ on https://doc.wikimedia.org/ - https://phabricator.wikimedia.org/T227766 (10Ammarpad) a:03Ammarpad [06:37:03] (03CR) 10MaxSem: "I don't remember seeing tests that just compare two literals, as the matter of fact I don't see anything like the examples in your tests i" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/547721 (owner: 10Thiemo Kreuz (WMDE)) [06:40:14] (03CR) 10MaxSem: [C: 03+2] "The reason was that we've grepped though our repos and whitelisted every non-bogus annotation found. If they weren't used, they aren't whi" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/546476 (owner: 10simetrical) [06:40:40] (03Merged) 10jenkins-bot: Allow @testWith and @doesNotPerformAssertions [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/546476 (owner: 10simetrical) [09:22:08] (03PS1) 10Umherirrender: [GettingStarted] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/547861 [09:40:21] (03CR) 10Hashar: "It does not come from PHPUnit 8 but from our wrapper tests/phpunit/phpunit.php" [integration/quibble] - 10https://gerrit.wikimedia.org/r/547843 (https://phabricator.wikimedia.org/T192167) (owner: 10Jforrester) [09:58:02] (03CR) 10Thiemo Kreuz (WMDE): "Uh, sorry. I should have explained this somewhere. The sniff cares only about the first (expected) value in the assertEquals() call. The s" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/547721 (owner: 10Thiemo Kreuz (WMDE)) [10:10:52] (03CR) 10Thiemo Kreuz (WMDE): Create PHPUnitAssertEquals sniff to find problematic assertEquals() (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/547721 (owner: 10Thiemo Kreuz (WMDE)) [11:46:42] (03PS1) 10Umherirrender: [UniversalLanguageSelector] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/547870 [13:04:23] (03CR) 10Daimona Eaytoy: [C: 03+1] [UniversalLanguageSelector] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/547870 (owner: 10Umherirrender) [13:44:45] 10Phabricator, 10Commons, 10WorkType-Maintenance: Create a herald script for adding #commons - https://phabricator.wikimedia.org/T118234 (10Masumrezarock100) This Herald rule is now adding #Commons tag to tasks even when Commons has nothing to do with the tasks in question. I think we should make the criter... [16:06:23] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, 10Google-Code-in-2019, 10MW-1.34-notes (1.34.0-wmf.5; 2019-05-14): Stop using jsonlint (as it's abandonware) and instead use eslint-plugin-json fo... - https://phabricator.wikimedia.org/T220036 [16:16:01] !log Created cswiki (T237077) [16:16:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:16:05] T237077: Set up beta cswiki - https://phabricator.wikimedia.org/T237077 [16:16:36] 10Beta-Cluster-Infrastructure, 10Growth-Team, 10Patch-For-Review: Set up beta cswiki - https://phabricator.wikimedia.org/T237077 (10Ladsgroup) I created the wiki, it works https://cs.wikipedia.beta.wmflabs.org/wiki/Hlavn%C3%AD_strana [16:19:58] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO (201908), 10Code-Stewardship-Reviews, 10Graphoid, and 3 others: graphoid: Code stewardship request - https://phabricator.wikimedia.org/T211881 (10Arkas) Someone must have fixed it? I just noticed that it works at the moment! [16:26:10] !log mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki=cswiki (T237077) [16:26:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:26:13] T237077: Set up beta cswiki - https://phabricator.wikimedia.org/T237077 [16:33:48] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO (201908), 10Code-Stewardship-Reviews, 10Graphoid, and 3 others: graphoid: Code stewardship request - https://phabricator.wikimedia.org/T211881 (10Arkas) Hm, i see that it's not working on every article. Hope someone is working on it... [16:36:57] 10Beta-Cluster-Infrastructure, 10Growth-Team, 10Patch-For-Review, 10User-Ladsgroup: Set up beta cswiki - https://phabricator.wikimedia.org/T237077 (10Urbanecm) 05Open→03Resolved a:03Ladsgroup Let's say it works. [16:44:16] !log deployment-prep: mwscript createAndPromote.php --wiki=cswiki --force --bureaucrat --sysop 'Martin Urbanec' [16:44:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:52:12] !log deployment-prep: mwscript importDump.php --wiki=cswiki /tmp/Wikipedie-20191031004003.xml (T236823) [17:52:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:52:17] T236823: Newcomer tasks: create test pages on beta enwiki - https://phabricator.wikimedia.org/T236823 [17:52:56] !log deployment-prep: mwscript rebuildrecentchanges.php --wiki=cswiki (T236823) [17:52:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:55:04] !log deployment-prep: mwscript initSiteStats.php --wiki=cswiki --update (T236823) [17:55:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:55:59] 10Beta-Cluster-Infrastructure, 10Growth-Team, 10User-Ladsgroup: Set up beta cswiki - https://phabricator.wikimedia.org/T237077 (10Urbanecm) @tgr maybe we should remove enwiki GE overrides from config? [18:52:11] (03CR) 10Daimona Eaytoy: [C: 03+1] phpunit: Drop --debug-tests command, killed off in PHPUnit 8 [integration/quibble] - 10https://gerrit.wikimedia.org/r/547843 (https://phabricator.wikimedia.org/T192167) (owner: 10Jforrester) [19:02:48] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, 10Google-Code-in-2019, 10MW-1.34-notes (1.34.0-wmf.5; 2019-05-14): Stop using jsonlint (as it's abandonware) and instead use eslint-plugin-json fo... - https://phabricator.wikimedia.org/T220036 [19:03:28] (03CR) 10Jforrester: [C: 03+2] [UniversalLanguageSelector] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/547870 (owner: 10Umherirrender) [19:03:38] (03CR) 10Jforrester: [C: 03+2] [GettingStarted] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/547861 (owner: 10Umherirrender) [19:05:25] (03Merged) 10jenkins-bot: [UniversalLanguageSelector] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/547870 (owner: 10Umherirrender) [19:05:29] (03Merged) 10jenkins-bot: [GettingStarted] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/547861 (owner: 10Umherirrender) [19:07:07] !log Zuul: Adding phan to ULS and GettingStarted [19:07:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:12:42] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, 10Google-Code-in-2019, 10MW-1.34-notes (1.34.0-wmf.5; 2019-05-14): Stop using jsonlint (as it's abandonware) and instead use eslint-plugin-json fo... - https://phabricator.wikimedia.org/T220036 [19:22:00] 10Gerrit: Rename Gerrit repository "LdapGroups" to "LDAPGroups" - https://phabricator.wikimedia.org/T200736 (10MarcoAurelio) @MarkAHershberger @Osnard We now have two extensions named similarly: [[ https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/LDAPGroups/ | `LDAPGroups` ]] and [[ https://ge... [20:51:34] (03PS1) 10MarcoAurelio: Allow jenkins-bot to merge changes in this repo [react.i18n] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/547894 [20:52:07] (03PS2) 10MarcoAurelio: Allow jenkins-bot to merge changes in this repo [react.i18n] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/547894 [20:52:46] (03CR) 10MarcoAurelio: "This is to fix issues such as https://gerrit.wikimedia.org/r/#/c/react.i18n/+/530244/ not being merged by the bot when the test passes." [react.i18n] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/547894 (owner: 10MarcoAurelio) [23:35:20] 10Gerrit, 10Developer-Advocacy, 10Gerrit-Privilege-Requests, 10TechCom: Requesting access for SemanticACL extension - https://phabricator.wikimedia.org/T234124 (10Tinss) Thanks a bunch everyone!