[06:56:49] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Phlogiston (Reporting): Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#4167241 (10ArielGlenn) 05Open>03Resolved a:03ArielGlenn New dumps are copied over properly, closing t... [07:53:16] 10Project-Admins: Create phabricator project for the Wiki Loves Love contest - https://phabricator.wikimedia.org/T193377#4167301 (10Psychoslave) [08:05:47] (03CR) 10Hashar: [C: 032] [NoCat] Add test jobs [integration/config] - 10https://gerrit.wikimedia.org/r/429643 (owner: 10Umherirrender) [08:05:50] (03CR) 10Hashar: [C: 032] [NamespaceSidebar] Add test jobs [integration/config] - 10https://gerrit.wikimedia.org/r/429645 (owner: 10Umherirrender) [08:05:53] (03CR) 10Hashar: [C: 032] [AutoCategoriseUploads] Add test jobs [integration/config] - 10https://gerrit.wikimedia.org/r/429644 (owner: 10Umherirrender) [08:07:07] (03Merged) 10jenkins-bot: [NoCat] Add test jobs [integration/config] - 10https://gerrit.wikimedia.org/r/429643 (owner: 10Umherirrender) [08:07:26] (03Merged) 10jenkins-bot: [NamespaceSidebar] Add test jobs [integration/config] - 10https://gerrit.wikimedia.org/r/429645 (owner: 10Umherirrender) [08:07:28] (03Merged) 10jenkins-bot: [AutoCategoriseUploads] Add test jobs [integration/config] - 10https://gerrit.wikimedia.org/r/429644 (owner: 10Umherirrender) [08:10:56] (03CR) 10Hashar: [C: 032] "Seems NamespaceSidebar is an empty repository for now. I guess it will one day be populated with some code that will have all the CI requi" [integration/config] - 10https://gerrit.wikimedia.org/r/429645 (owner: 10Umherirrender) [08:14:42] (03CR) 10Hashar: [C: 032] "PHP CodeSniffer has some failures on AutoCategoriseUploads (tried on https://gerrit.wikimedia.org/r/#/c/429734/ ). That seems easy to fix " [integration/config] - 10https://gerrit.wikimedia.org/r/429644 (owner: 10Umherirrender) [08:29:04] (03PS2) 10Legoktm: Run non-voting phan-taint-check plugin for newly bundled extensions too [integration/config] - 10https://gerrit.wikimedia.org/r/429638 [08:29:06] (03PS1) 10Legoktm: Enable phan for AbuseFilter [integration/config] - 10https://gerrit.wikimedia.org/r/429740 [08:29:29] (03CR) 10Legoktm: [C: 032] Run non-voting phan-taint-check plugin for newly bundled extensions too [integration/config] - 10https://gerrit.wikimedia.org/r/429638 (owner: 10Legoktm) [08:29:35] (03CR) 10Legoktm: [C: 032] Enable phan for AbuseFilter [integration/config] - 10https://gerrit.wikimedia.org/r/429740 (owner: 10Legoktm) [08:31:37] (03Merged) 10jenkins-bot: Run non-voting phan-taint-check plugin for newly bundled extensions too [integration/config] - 10https://gerrit.wikimedia.org/r/429638 (owner: 10Legoktm) [08:31:40] (03Merged) 10jenkins-bot: Enable phan for AbuseFilter [integration/config] - 10https://gerrit.wikimedia.org/r/429740 (owner: 10Legoktm) [08:32:45] !log deployed https://gerrit.wikimedia.org/r/429638 https://gerrit.wikimedia.org/r/429740 [08:32:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:44:06] Project mwext-phpunit-coverage-publish build #3906: 04FAILURE in 3.4 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3906/ [08:47:07] Yippee, build fixed! [08:47:07] Project mwext-phpunit-coverage-publish build #3907: 09FIXED in 1 min 1 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3907/ [09:09:19] !log Quibble announce on Phame blog thing https://phabricator.wikimedia.org/phame/post/view/99/introducing_quibble/ (thanks zeljkof ) [09:09:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:54:32] (03PS4) 10Hashar: Convert doc to Sphinx [integration/quibble] - 10https://gerrit.wikimedia.org/r/428628 [10:02:54] (03PS5) 10Hashar: Convert doc to Sphinx [integration/quibble] - 10https://gerrit.wikimedia.org/r/428628 (https://phabricator.wikimedia.org/T193164) [10:11:10] 10Release-Engineering-Team (Kanban), 10Quibble, 10Documentation, 10Patch-For-Review: document quibble cache / src reuse for local installation - https://phabricator.wikimedia.org/T193164#4167519 (10hashar) a:03hashar Finally paired that one with @zeljkofilipin this morning. Doc is polished up https://ge... [10:16:02] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4167528 (10hashar) Result from running quibble-vendor-mysql-php70-docker which uses mediawik... [10:23:52] 10Release-Engineering-Team (Kanban), 10Quibble, 10Documentation, 10Patch-For-Review, 10User-zeljkofilipin: document quibble cache / src reuse for local installation - https://phabricator.wikimedia.org/T193164#4167535 (10zeljkofilipin) [10:30:03] Newbie tester here, learn about theory, wanna apply in testing testing [10:43:02] 10Project-Admins: Create phabricator project for the Wiki Loves Love contest - https://phabricator.wikimedia.org/T193377#4167301 (10MarcoAurelio) What's the specific use case for a Phabricator project? I mean, what do plan to use Phabricator for? Ping @Aklapper [11:04:12] 10Release-Engineering-Team (Kanban), 10MediaWiki-SWAT-deployments, 10User-greg, 10User-zeljkofilipin: Proposal: Effective immediately, disallow multi-sync patch deployment - https://phabricator.wikimedia.org/T187761#3984680 (10MarcoAurelio) For those not very tech-savy and whose language is not English, co... [11:12:49] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Services, 10MW-1.32-release-notes (WMF-deploy-2018-04-24 (1.32.0-wmf.1)), and 2 others: deployment-prep has jobqueue issues - https://phabricator.wikimedia.org/T192473#4167675 (10MarcoAurelio) I think @mobrovac knows about Kafka as well, and help... [11:33:19] 10Project-Admins: Create phabricator project for the Wiki Loves Love contest - https://phabricator.wikimedia.org/T193377#4167709 (10Wikilover90) The point is to plan the entire project of Wiki Loves Love. Specific Use : Start by documenting what is required for making Wiki Loves Love happen and possibly divide... [12:08:05] (03PS6) 10Hashar: Convert doc to Sphinx [integration/quibble] - 10https://gerrit.wikimedia.org/r/428628 (https://phabricator.wikimedia.org/T193164) [12:38:28] (03PS3) 10Hashar: docker: enable MariaDB strict mode for Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429406 (https://phabricator.wikimedia.org/T118371) [12:38:30] (03PS2) 10Hashar: docker: MariaDB on Jessie to utf8mbr for Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/429434 (https://phabricator.wikimedia.org/T193222) [12:41:21] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [extensions/ImapAuthorization] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/429788 [12:41:23] (03CR) 10QChris: [V: 031 C: 032] Allow “Gerrit Managers” to import history [extensions/ImapAuthorization] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/429788 (owner: 10QChris) [12:44:15] (03PS2) 10QChris: Allow "Gerrit Managers" to import history [extensions/ImapAuthorization] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/429788 [12:44:17] (03CR) 10QChris: [V: 031 C: 032] Allow "Gerrit Managers" to import history [extensions/ImapAuthorization] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/429788 (owner: 10QChris) [12:44:52] (03PS1) 10QChris: Import done. Revoke import grants [extensions/ImapAuthorization] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/429789 [12:44:54] (03CR) 10QChris: [V: 031 C: 032] Import done. Revoke import grants [extensions/ImapAuthorization] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/429789 (owner: 10QChris) [13:16:05] 10Project-Admins: Create phabricator project for the Wiki Loves Love contest - https://phabricator.wikimedia.org/T193377#4167867 (10Psychoslave) So far, discussion mainly happened in Telegram. It's fine to discuss around, but not very handy for project management. We possibly will divide with sub-contest if some... [13:18:18] (03PS1) 10Hashar: Migrate skins/Amethyst to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429797 [13:21:46] 10Project-Admins: Create phabricator project for the Wiki Loves Love contest - https://phabricator.wikimedia.org/T193377#4167897 (10Psychoslave) Please highlight anything you think relevant for our WLL project in - https://meta.wikimedia.org/wiki/Learning_patterns - https://meta.wikimedia.org/wiki/Wikimedia_Reso... [13:28:13] (03PS2) 10Hashar: Migrate most mediawiki skins to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429797 (https://phabricator.wikimedia.org/T183512) [13:49:49] (03CR) 10Hashar: [C: 032] Migrate most mediawiki skins to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429797 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [13:50:29] 10Project-Admins: Create phabricator project for the Wiki Loves Love contest - https://phabricator.wikimedia.org/T193377#4167972 (10Aklapper) See https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects for required data [13:51:14] (03Merged) 10jenkins-bot: Migrate most mediawiki skins to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429797 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [14:06:32] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4168008 (10hashar) [14:14:51] 10Release-Engineering-Team (Kanban), 10Quibble, 10Documentation, 10Patch-For-Review, 10User-zeljkofilipin: document quibble cache / src reuse for local installation - https://phabricator.wikimedia.org/T193164#4168025 (10zeljkofilipin) p:05Triage>03Normal [14:29:46] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4168069 (10hashar) [14:32:21] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4168076 (10hashar) [14:35:52] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4168078 (10hashar) [14:37:09] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#3855599 (10hashar) [14:50:06] (03PS1) 10Hashar: Migrate mediawiki/extensions/A* to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429817 (https://phabricator.wikimedia.org/T183512) [14:53:31] (03PS1) 10Hashar: Migrate mediawiki/extensions/B* to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429819 (https://phabricator.wikimedia.org/T183512) [14:54:31] (03CR) 10Hashar: [C: 032] Migrate mediawiki/extensions/A* to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429817 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [14:54:34] (03CR) 10Hashar: [C: 032] Migrate mediawiki/extensions/B* to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429819 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [14:55:57] (03Merged) 10jenkins-bot: Migrate mediawiki/extensions/A* to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429817 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [14:56:05] (03Merged) 10jenkins-bot: Migrate mediawiki/extensions/B* to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/429819 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [14:58:11] Heads-up, I'm doing a brief test deployment to our canary server, to verify that the latest scap supports LFS in production. [15:14:52] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4168196 (10hashar) [15:15:56] 10Continuous-Integration-Config, 10Patch-For-Review: mwext-testextension-* jobs does not remove skins from previous runs - https://phabricator.wikimedia.org/T113841#4168202 (10hashar) [15:22:08] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4168244 (10hashar) [15:47:45] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Phlogiston (Reporting): Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#4168338 (10JAufrecht) Confirming: automated Phlogiston run from last night has fresh data. [16:13:00] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4168423 (10hashar) [16:14:39] 10Release-Engineering-Team, 10DNS, 10Operations, 10Traffic, and 2 others: Move Foundation Wiki to new URL when new Wikimedia Foundation website launches - https://phabricator.wikimedia.org/T188776#4168429 (10demon) a:03demon I'll handle this. Should just be a domain swap--no need to bother doing renames... [16:44:37] 10Release-Engineering-Team, 10Epic: Migrate all CI jobs from Nodepool, deprecate its use - https://phabricator.wikimedia.org/T190097#4168527 (10Ebe123) [16:53:53] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.31-release, 10MW-1.31-release-notes (WMF-deploy-2018-04-10 (1.31.0-wmf.29)): Bundle AbuseFilter extension with MW 1.31 - https://phabricator.wikimedia.org/T191740#4168590 (10Legoktm) [16:54:14] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.31-release, 10MW-1.31-release-notes (WMF-deploy-2018-04-10 (1.31.0-wmf.29)): Bundle AbuseFilter extension with MW 1.31 - https://phabricator.wikimedia.org/T191740#4115148 (10Legoktm) [16:59:14] 10Release-Engineering-Team, 10Operations, 10SRE-Access-Requests, 10User-Urbanecm: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830#4168627 (10RobH) [17:00:17] 10Release-Engineering-Team (Kanban), 10Quibble, 10Patch-For-Review: Error: 1071 Specified key was too long; max key length is 767 bytes - https://phabricator.wikimedia.org/T193222#4168633 (10hashar) There are too many extensions failing with `utf8mb4` so I will make the Stretch MySQL to use latin1 like on Je... [17:00:42] (03CR) 10Hashar: [C: 04-2] "Too many extensions are failing for us to switch to utf8mb4. Instead the MySQL configuration on Stretch should be made to use 'latin1' / '" [integration/config] - 10https://gerrit.wikimedia.org/r/429434 (https://phabricator.wikimedia.org/T193222) (owner: 10Hashar) [17:26:55] 10Release-Engineering-Team, 10Patch-For-Review, 10Scoring-platform-team (Current): ORES deployment submodules should point to phabricator HTTPS repos. - https://phabricator.wikimedia.org/T179009#3710070 (10awight) Yep, this became an issue again. I can't make fresh checkouts of the ores-deploy repo because... [17:26:57] 10Release-Engineering-Team, 10Patch-For-Review, 10Scoring-platform-team (Current): ORES deployment submodules should point to phabricator HTTPS repos. - https://phabricator.wikimedia.org/T179009#4168759 (10awight) [17:30:57] Project mwext-phpunit-coverage-publish build #3922: 04FAILURE in 3.4 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3922/ [17:31:03] Project mwext-phpunit-coverage-publish build #3923: 04STILL FAILING in 3.4 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3923/ [17:33:21] Yippee, build fixed! [17:33:21] Project mwext-phpunit-coverage-publish build #3924: 09FIXED in 1 min 40 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3924/ [17:44:23] I think I'm doing something stupid. Trying to deploy an ORES WIP patch to the production canary, and getting "no common commits" errors from git. This is something about trying to push things that aren't on the master branch?? [17:48:07] Looks like I can resolve this myself, just need to merge in the submodule patches. [18:07:19] 10Release-Engineering-Team, 10Operations, 10SRE-Access-Requests, 10User-Urbanecm: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830#4150980 (10RobH) @Urbanecm: Please note we'll also need you to review and agree/sign the L3 document on phabricator for... [18:07:49] 10Release-Engineering-Team, 10Operations, 10SRE-Access-Requests, 10User-Urbanecm: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830#4168937 (10RobH) [18:08:42] twentyafterfour: FYI, I just attempted a pilot LFS deployment on our canary server (ores1001) and the LFS-enabled directory is empty. No errors! [18:15:25] 10Gerrit, 10ORES, 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current): Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#4168948 (10awight) Pilot deployment to the canary server failed, with no errors: {P7055} Looks good, but the large file was never ch... [18:20:14] 10Beta-Cluster-Infrastructure, 10DBA, 10Operations: Possible to run writes (e.g. UPDATE) on Beta Cluster replica - https://phabricator.wikimedia.org/T110115#1569333 (10EddieGP) Just judging from the task title, this and {T183245} look like being duplicates? [18:20:41] 10Gerrit, 10ORES, 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current): Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#4168958 (10awight) Maybe we need `git lfs pull --recursive`? No clue why this would have worked on beta without the `--recursive`, ho... [18:27:18] 10Release-Engineering-Team, 10Patch-For-Review, 10Scoring-platform-team (Current): ORES deployment submodules should point to phabricator HTTPS repos. - https://phabricator.wikimedia.org/T179009#4168994 (10awight) Just noticed that we've already rolled this back for git-lfs work, we're now using gerrit again... [18:28:39] 10Release-Engineering-Team (Watching / External), 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#4168997 (10awight) \o/ just did a rollback of one machine in 24 seconds, t... [18:30:01] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169008 (10awight) p:05Triage>03High [18:30:18] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169021 (10awight) p:05High>03Normal [18:51:46] 10Gerrit, 10ORES, 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current): Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#4169095 (10awight) Apparently I'm bad at git, and I failed to commit the right submodule pointers... trying again. [18:53:25] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169101 (10Dzahn) a:05awight>03Dzahn [19:03:07] awight: weird [19:07:26] awight: I'd take a look but I guess I don't have access to that server [19:07:32] (I mean, I assume I don't) [19:10:01] twentyafterfour: sorry, it was a false alarm... I botched the submodule pointer! [19:10:18] oh, ok cool. So it works? :) [19:10:29] Still testing! [19:10:34] :) [19:16:26] 10Phabricator, 10Discourse, 10Developer-Relations (Apr-Jun-2018): Enable Wikimedia Phabricator login in discourse-mediawiki.wmflabs.org - https://phabricator.wikimedia.org/T184987#4169146 (10Qgil) I enabled the updated version, I logged out, I logged in using the "with Phabricator" button and... **It works!*... [19:16:52] (03PS1) 10Umherirrender: Check if the default of null is in the type list of @param [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/429860 [19:17:29] (03CR) 10Umherirrender: "See I8ab28ae3d86befa9ebf4d84f58580d9c0915f3d9 for an example change" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/429860 (owner: 10Umherirrender) [19:28:01] hmm, how can I authenticate to beta services [19:28:02] e.g. [19:28:07] https://logstash-beta.wmflabs.org/ [19:28:08] ? [19:28:16] none of my usual ldapy stuff seems to work [19:30:18] ottomata: you probably need to create a separate account for that [19:30:24] oh? [19:30:25] where/how? [19:30:34] accounts in beta can't share passwords with your prod accounts because beta is very insecure [19:30:42] but its...wikitech/ldap? [19:30:58] oh we don't want someone slurping them up or something ok [19:30:59] I'm not sure... [19:31:02] hm [19:31:02] ok [19:31:10] do you know who/how runs logstash there? [19:31:14] I don't know what logstash-beta is using for auth [19:31:23] 10MediaWiki-Codesniffer: codesniffer should fix wrong @throw to @throws on private functions - https://phabricator.wikimedia.org/T193430#4169230 (10Umherirrender) [19:31:35] ottomata: no I'm not sure but I'll try to figure it out hang on [19:32:05] i have found an htpasswd file with encrypted pw on that host... :) [19:32:24] ottomata: that's likely to be the thing [19:32:26] ya must be it [19:32:29] so i ahve username, but not pw :) [19:32:37] oh! [19:32:40] add a user to it? [19:32:40] twentyafterfour: doc in code [19:32:48] it is in a file on deployment tin! [19:32:54] ah [19:35:01] next up: do you know where i can find some logstash docs? i want to log from python logger to logstash [19:35:06] not much on wikitech that i can find [19:35:17] just a page that says 'we use logstash for this!', but not how to set it up for new stuff [19:35:59] ottomata: logstash is sort of straightforward, I think you just post to a rest endpoint [19:36:13] but I'm not sure we have docs on it, maybe just upstream docs [19:36:30] yeah, but i don't knwo how to troubleshoot for our stuff, i think i got somethign working [19:36:35] but there are no docs about where to post, etc. [19:36:42] ottomata: you're browser is supressing the tip: The site says: “Logstash (ssh deployment-tin.eqiad.wmflabs sudo cat /root/secrets.txt)” [19:36:44] i've got a python-logstash logging handler that seems like it would work [19:36:52] THE SITE SAYS THAT?! [19:36:56] yup [19:36:58] well i'm in now haha [19:37:08] sheesh, i'm good reader aren't i [19:37:09] thanks greg [19:37:19] ottomata: well, chrome supresses it or something [19:37:25] oh it does!? [19:37:31] ok phew not my fault phew i'm not a dummy [19:38:31] I assume you've seen https://wikitech.wikimedia.org/wiki/Logstash#Logstash which isn't terribly helpful [19:40:23] yeah [19:40:49] bd808: yt :)? [19:41:00] ottomata: so you are trying to find your log entries in beta-logstash to confirm that it's working? [19:41:49] yes [19:42:11] OH actually, i found them, twentyafterfour i wasn't searching well [19:42:19] great. [19:42:22] :) [19:42:41] well if I can help with elasticsearch somehow I'll try to be of assistance. I know it fairly well [19:42:42] thanks for the help releng room! :) [19:42:47] you're welcome [19:42:48] (bd 808 please ignore) [19:55:57] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169396 (10Dzahn) @awight I think the correct path is /srv/deployment/ores/venv (vs. /srv/... [19:57:14] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Epic: Use Beta cluster as a true canary for code deployments (epic) - https://phabricator.wikimedia.org/T53494#4169402 (10Jdforrester-WMF) [19:57:47] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Watching / External), 10Performance-Team, 10Availability (MediaWiki-MultiDC): Performance Q2 2017/18 goal: Install and use mcrouter in deployment-prep - https://phabricator.wikimedia.org/T151466#4169405 (10aaron) 05Open>03Resolved This has now... [20:03:10] hashar: was reading https://lists.wikimedia.org/pipermail/qa/2018-April/002699.html, very cool man [20:09:56] twentyafterfour: I finally have a result to share. Check ores1001.eqiad.wmnet:/srv/deployment/ores/deploy-cache/revs/4601497c4f4363fcea639dc3e13d4f178c421a1b/submodules/assets/word2vec/ [20:10:10] the pointer file is checked out, but not the large file. [20:10:38] awight: I probably don't have access to ores1001.eqiad.wmnet [20:10:45] ah hrm [20:11:30] I can get any info that might be helpful, but don't have root and can't run sudo deploy-service [20:13:37] the deploy-log might be helpful [20:13:44] I can probably find that myself [20:14:57] I see that it ran git lfs pull, that should have been good [20:16:51] hmm the most recent deploy log looks good [20:17:37] chasemp: yup and i finally started switching mediawiki* jobs to Docker\o/ [20:18:11] if I am not too bad, I will post some kind of status update at the end of the week [20:18:25] twentyafterfour: +1 I was just reading through the log and "git lfs pull" took 0 seconds. So I think we might be running the wrong command... [20:18:35] i.e. it might not be recursive. [20:20:50] I'll play with a test repo and see if I can reproduce this [20:26:06] argh, it's hard to do cos many git actions pull the lfs files implicitly [20:29:00] twentyafterfour: I'm seeing different behavior locally. "git submodule update -i" pulls all the LFS files, so we shouldn't even need git lfs pull. This is not the case, though. [20:38:16] 10Gerrit, 10ORES, 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current): Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#4169582 (10awight) Gave it another try, with commit 4601497c4f4363fcea639dc3e13d4f178c421a1b, and got strange results. The LFS data s... [20:44:08] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169598 (10awight) @Dzahn Thanks, I think you're right on both counts. We must have deploye... [20:46:18] hrm [20:46:32] CI's failing on a PdfHandler patch on REL1_28: 20:41:06 Fatal error: Class undefined: Wikibase\DataModel\Entity\ItemId in /home/jenkins/workspace/mediawiki-extensions-hhvm-jessie/src/extensions/Wikibase/lib/WikibaseLib.entitytypes.php on line 37 [20:47:47] or is 1.28 out of support or sth and i shouldn't bother [20:49:05] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169633 (10awight) [20:51:24] brion: that job is broken, we should drop it [20:51:41] brion: that would be the 'mediawiki-extensions-...' kind of jobs that test several extensions together [20:51:50] ah [20:51:51] it makes sense on master/wmf branches, but not much for REL branches [20:52:44] and 1.28 is no more maintained [20:53:50] brion: I would say to force merge it (remove jenkins-bot vote from gerrit, vote Vreified+2 and Code-Review+2, then a blue submit button should show up) [20:53:54] heh ok [20:54:32] hashar: thx, doing thsat [20:54:34] *that [20:54:49] once I am done refactoring migrating jobs, I will look at enhancing the jobs being run for release branches [20:55:06] but that will be in a few months :/ [20:58:06] cool [20:58:14] i'll use that remove jenkins trick if i need it again ;) [21:02:48] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Watching / External), 10Performance-Team, 10Availability (MediaWiki-MultiDC): Performance Q2 2017/18 goal: Install and use mcrouter in deployment-prep - https://phabricator.wikimedia.org/T151466#4169664 (10EddieGP) >>! In T151466#4169405, @aaron wr... [21:22:18] 10Gerrit, 10RelEng-Archive-FY201718-Q2, 10Scap (Tech Debt Sprint FY201718-Q2), 10ORES, and 3 others: Support git-lfs files in gerrit - https://phabricator.wikimedia.org/T171758#4169696 (10awight) [21:22:22] 10Release-Engineering-Team, 10Patch-For-Review, 10Scoring-platform-team (Current): ORES deployment submodules should point to phabricator HTTPS repos. - https://phabricator.wikimedia.org/T179009#4169695 (10awight) 05Open>03declined [21:54:07] (03CR) 10MarcoAurelio: "@hashar: I am not the owner of the repo, I just created it based on a request on MediaWiki for User:Tosfos which wanted to develop a new e" [integration/config] - 10https://gerrit.wikimedia.org/r/429645 (owner: 10Umherirrender) [22:03:21] 10Release-Engineering-Team, 10Operations, 10SRE-Access-Requests, 10User-Urbanecm: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830#4169796 (10Urbanecm) I signed L3. There are some upcoming issues with the NDA, I hope they'll be resolved soon. [22:10:32] 10Gerrit, 10translatewiki.net, 10User-MarcoAurelio: Authorize L10n-bot to give +2 for SecureAuth extension - https://phabricator.wikimedia.org/T193227#4169813 (10MarcoAurelio) Just to mention that the fix indeed seems to have worked as expected as you can see at https://gerrit.wikimedia.org/r/#/c/429956/ --... [23:07:17] 10Release-Engineering-Team, 10MediaWiki-Parser, 10MediaWiki-Platform-Team, 10Parsing-Team, 10Tidy: Different versions of Tidy installed on different production servers? - https://phabricator.wikimedia.org/T193414#4169904 (10Jdforrester-WMF) Tagging in Release Engineering in case this is something they ne... [23:08:10] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169909 (10Dzahn) @awight confirmed! it only existed on ores1001 and i deleted it. confirm... [23:08:41] 10Release-Engineering-Team (Watching / External), 10Operations, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Clean up deprecated shared virtualenv directories - https://phabricator.wikimedia.org/T193422#4169912 (10Dzahn) 05Open>03Resolved [23:08:44] 10Release-Engineering-Team (Watching / External), 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#4169913 (10Dzahn) [23:09:11] 10Release-Engineering-Team, 10MediaWiki-Parser, 10MediaWiki-Platform-Team, 10Parsing-Team, 10Tidy: Different versions of Tidy installed on different production servers? - https://phabricator.wikimedia.org/T193414#4169917 (10Catrope) The following hosts have the wrong config: ``` mw1262.eqiad.wmnet: bool(... [23:17:01] 10Release-Engineering-Team, 10MediaWiki-Parser, 10MediaWiki-Platform-Team, 10Parsing-Team, 10Tidy: Different versions of Tidy installed on different production servers? - https://phabricator.wikimedia.org/T193414#4169929 (10Catrope) My sync appears to have fixed that, the config var is now true everywhere. [23:19:24] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Collaboration-Team-Triage, 10Notifications, and 2 others: mwext-ruby-jessie fails on mediawiki/extensions/Echo - https://phabricator.wikimedia.org/T193074#4169931 (10Jdforrester-WMF) 05Open>03Resolved a:03Catrope Fixed in htt... [23:38:50] Project mwext-phpunit-coverage-publish build #3963: 04FAILURE in 3.6 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3963/ [23:38:52] Project mwext-phpunit-coverage-publish build #3964: 04STILL FAILING in 2.2 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3964/ [23:38:55] Project mwext-phpunit-coverage-publish build #3965: 04STILL FAILING in 2.3 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3965/ [23:43:03] Yippee, build fixed! [23:43:03] Project mwext-phpunit-coverage-publish build #3966: 09FIXED in 4 min 6 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/3966/ [23:51:54] 10Release-Engineering-Team, 10Patch-For-Review, 10Scoring-platform-team (Current): ORES deployment submodules should point to phabricator HTTPS repos. - https://phabricator.wikimedia.org/T179009#4169982 (10demon) So, we're using Gerrit's git-lfs now?