[00:08:34] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations: contint1001: DISK WARNING - free space: /srv 88397 MB (10% inode=94%): - https://phabricator.wikimedia.org/T219850 (10Dzahn) The long term fix should still be T178663 [06:25:57] PROBLEM - Puppet errors on integration-slave-jessie-1002 is CRITICAL: (Service Check Timed Out) [08:33:00] https://integration.wikimedia.org/ci/job/mediawiki-fresnel-patch-docker/4457/consoleFull lies. dump scripts do not impact page read/load time :-/ [09:40:29] (03CR) 10Awight: "Note: We'll have to update the `tests/selenium/README.md` for many tests once this is merged, to document that you can no longer run exten" [integration/quibble] - 10https://gerrit.wikimedia.org/r/510709 (https://phabricator.wikimedia.org/T199116) (owner: 10Awight) [09:49:05] (03CR) 10Awight: Run node browser tests in each repo (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/510709 (https://phabricator.wikimedia.org/T199116) (owner: 10Awight) [09:53:11] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T220732 (10zeljkofilipin) 05Open→03Resolved [10:21:48] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10akosiaris) One thing that I just met is that kask stops accepting HTTP connections if kask cert/key pair is configure... [10:29:32] (03CR) 10Zfilipin: "> Patch Set 3:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/510709 (https://phabricator.wikimedia.org/T199116) (owner: 10Awight) [10:35:48] (03CR) 10Awight: "> > Patch Set 3:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/510709 (https://phabricator.wikimedia.org/T199116) (owner: 10Awight) [10:48:27] (03CR) 10Zfilipin: "> Patch Set 3:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/510709 (https://phabricator.wikimedia.org/T199116) (owner: 10Awight) [10:58:17] (03CR) 10Ladsgroup: [C: 03+1] "It looks way cleaner now." [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [12:31:49] 10Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2019, 10User-zeljkofilipin: Quality Assurance Special Interest Group Meetup - https://phabricator.wikimedia.org/T223623 (10zeljkofilipin) 05Open→03Resolved [12:32:37] 10Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2019, 10User-zeljkofilipin: Investigate Postman for API testing - https://phabricator.wikimedia.org/T223627 (10zeljkofilipin) 05Open→03Resolved [12:33:58] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Testing, 10Wikimedia-Hackathon-2019, 10User-zeljkofilipin: Write the first Selenium test for a repository - https://phabricator.wikimedia.org/T223624 (10zeljkofilipin) 05Open→03Resolved [12:34:50] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Testing, 10Wikimedia-Hackathon-2019, 10User-zeljkofilipin: Patterns and anti-patterns in Selenium tests - https://phabricator.wikimedia.org/T223625 (10zeljkofilipin) 05Open→03Resolved [13:17:40] (03PS1) 10Umherirrender: [CleanChanges] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513591 [13:23:45] PROBLEM - Puppet errors on integration-slave-jessie-1004 is CRITICAL: CRITICAL: 3.33% of data above the critical threshold [3.0] [13:28:51] 10Continuous-Integration-Config, 10MediaWiki-extensions-General: Add phan to MediaWiki extensions and skins for static analysis [cloneable] - https://phabricator.wikimedia.org/T179554 (10Umherirrender) [13:59:01] 10Continuous-Integration-Config, 10I18n: recheck for L10n-bot patch set should start the i18n jobs to check - https://phabricator.wikimedia.org/T224731 (10Umherirrender) [14:23:59] (03PS1) 10Umherirrender: [VisualEditor] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513611 [14:45:04] 10Continuous-Integration-Config, 10Pywikibot, 10Documentation, 10Pywikibot-Documentation, 10good first bug: Pywikibot documentation build throws many warnings to console - https://phabricator.wikimedia.org/T187009 (10Dvorapa) [14:55:47] 10Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2019, 10User-zeljkofilipin: Quality Assurance Special Interest Group Meetup - https://phabricator.wikimedia.org/T223623 (10zeljkofilipin) Photo from the session: https://commons.wikimedia.org/wiki/File:Quality_Assurance_Special_Interest_Group_Meetup.jpg [14:58:31] (03PS1) 10Umherirrender: [cldr] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513618 [15:09:15] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10Eevans) >>! In T220401#5225728, @akosiaris wrote: > One thing that I just met is that kask stops accepting HTTP conne... [15:18:02] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10fgiunchedi) >>! In T220401#5226287, @Eevans wrote: >> 1. We use an Exec probe that executes something like `curl http... [15:36:22] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10Eevans) >>! In T220401#5226287, @Eevans wrote: >>>! In T220401#5225728, @akosiaris wrote: >> One thing that I just me... [15:37:28] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10akosiaris) >>! In T220401#5226287, @Eevans wrote: >>>! In T220401#5225728, @akosiaris wrote: >> One thing that I just... [15:47:52] (03PS1) 10Umherirrender: [GlobalPreferences] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513630 [15:57:03] 10Diffusion: Cannot connect to vcs@git-ssh.wikimedia.org (since move from phab1001 to phab1003) - https://phabricator.wikimedia.org/T224677 (10mmodell) `sudo -u vcs /srv/phab/phabricator/bin/ssh-auth` works as expected on phab1003, however, from my machine, `ssh -T vcs@git-ssh.wikimedia.org` results in a long pa... [16:25:21] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10akosiaris) > And this uncovered now that prometheus can't talk to it (cause it expects HTTP I guess?). /me looking in... [16:40:04] 10Release-Engineering-Team, 10Librarization, 10Quibble: Install extension require-dev dependencies in wmf-quibble-vendor-mysql-hhvm-docker - https://phabricator.wikimedia.org/T220723 (10EBernhardson) [16:50:20] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10akosiaris) One minor question. Given per T220401#5128786 1 kask instance is able to handle ~300req/s, how many instan... [16:56:19] 10Gerrit, 10Release-Engineering-Team (Backlog), 10Striker, 10Tools: Add Gerrit support to Striker (toolsadmin) - https://phabricator.wikimedia.org/T224676 (10Paladox) p:05Triage→03Normal [17:21:54] (03PS1) 10Umherirrender: [ContentTranslation] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513644 [17:25:10] (03PS1) 10Paladox: Fix puppet-compiler parsing the Hosts header [integration/config] - 10https://gerrit.wikimedia.org/r/513647 [17:25:40] (03PS2) 10Paladox: Fix puppet-compiler parsing the Hosts header [integration/config] - 10https://gerrit.wikimedia.org/r/513647 [17:26:16] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10Eevans) >>! In T220401#5226531, @akosiaris wrote: > One minor question. Given per T220401#5128786 1 kask instance is... [17:27:38] (03PS1) 10Umherirrender: [Collection] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513648 [17:32:03] Hello all! I'm trying to get new hire jeh access to the jenkins UI for the puppet catalog compiler. Can anyone advise about what permissions he needs? [17:47:21] (03PS1) 10Umherirrender: [MobileFrontend] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/513654 [17:47:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations: contint1001 store docker images on separate partition or disk - https://phabricator.wikimedia.org/T207707 (10greg) >>! In T207707#5223912, @Cmjohnson wrote: > @greg @robh I am just plugging these disks into the server... [17:56:01] !log Reloading Zuul to deploy I9c115b1f6 and I0d98f01d [17:56:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:56:07] (03PS1) 10Umherirrender: [CentralNotice] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513657 [17:57:07] (03PS2) 10Jforrester: [VisualEditor] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/512972 (owner: 10Umherirrender) [17:57:09] (03PS2) 10Jforrester: [VisualEditor] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513611 (owner: 10Umherirrender) [17:57:18] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/512972 (owner: 10Umherirrender) [17:57:25] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/513611 (owner: 10Umherirrender) [17:57:53] andrewbogott: not sure what access you need, but your Wikimedia Developer Account is what logs into the Jenkins webui, nothing is documented about this on https://wikitech.wikimedia.org/wiki/Nova_Resource:Puppet-diffs (where it should be, probably) [17:58:52] (03Merged) 10jenkins-bot: [VisualEditor] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/512972 (owner: 10Umherirrender) [17:58:59] (03Merged) 10jenkins-bot: [VisualEditor] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513611 (owner: 10Umherirrender) [17:59:09] So anyone with an account can launch a job? [17:59:24] i doin't think so [17:59:32] it says permissions denied to me when i tryied. [18:01:38] (03PS2) 10Jforrester: [CleanChanges] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513591 (owner: 10Umherirrender) [18:01:54] (03PS1) 10Umherirrender: [MobileFrontend] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513659 [18:02:19] !log Reloading Zuul to deploy I0ee1c8166 [18:02:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:03:11] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/513591 (owner: 10Umherirrender) [18:04:58] (03Merged) 10jenkins-bot: [CleanChanges] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513591 (owner: 10Umherirrender) [18:07:59] (03CR) 10Jforrester: [C: 03+2] [cldr] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513618 (owner: 10Umherirrender) [18:08:01] (03CR) 10Jforrester: [C: 03+2] [GlobalPreferences] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513630 (owner: 10Umherirrender) [18:09:10] (03CR) 10Umherirrender: "Only XAnalytics is new, the others are already implicit provided as dependency of dependencies" [integration/config] - 10https://gerrit.wikimedia.org/r/513654 (owner: 10Umherirrender) [18:10:03] (03Merged) 10jenkins-bot: [cldr] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513618 (owner: 10Umherirrender) [18:10:09] (03Merged) 10jenkins-bot: [GlobalPreferences] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513630 (owner: 10Umherirrender) [18:10:40] !log Reloading Zuul to deploy phan for cldr and GlobalPreferences [18:10:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:14:19] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10User-greg: contint1001 store docker images on separate partition or disk - https://phabricator.wikimedia.org/T207707 (10Cmjohnson) a:03greg @greg the disks have been added and assigned to you [18:15:57] (03CR) 10Jforrester: [C: 03+2] [MobileFrontend] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/513654 (owner: 10Umherirrender) [18:16:02] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations: contint1001 store docker images on separate partition or disk - https://phabricator.wikimedia.org/T207707 (10greg) a:05greg→03None [18:16:46] (03CR) 10Jforrester: "Is this wanted? Happy to deploy…" [integration/config] - 10https://gerrit.wikimedia.org/r/511066 (https://phabricator.wikimedia.org/T216630) (owner: 10Michael Große) [18:17:15] (03Merged) 10jenkins-bot: [MobileFrontend] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/513654 (owner: 10Umherirrender) [18:17:21] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10Release Pipeline: Switch CI Docker Storage Driver to its own partition and to use devicemapper - https://phabricator.wikimedia.org/T178663 (10greg) [18:17:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations: contint1001 store docker images on separate partition or disk - https://phabricator.wikimedia.org/T207707 (10greg) 05Stalled→03Open [18:17:38] !log Reloading Zuul to deploy extra dependencies for MobileFrontend [18:17:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:20:09] andrewbogott: I'm not sure which group. I'd ask someone who manages the puppet-compiler :) [18:21:33] (03PS3) 10Jforrester: GrowthExperiments: Add MobileFrontend and VisualEditor dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/496559 (https://phabricator.wikimedia.org/T218345) (owner: 10Kosta Harlan) [18:22:05] (03PS1) 10Umherirrender: [EventBus] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513663 [18:23:03] (03CR) 10jerkins-bot: [V: 04-1] GrowthExperiments: Add MobileFrontend and VisualEditor dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/496559 (https://phabricator.wikimedia.org/T218345) (owner: 10Kosta Harlan) [18:23:42] (03CR) 10Kosta Harlan: [C: 04-1] "I don’t think we (Growth) need this for now." [integration/config] - 10https://gerrit.wikimedia.org/r/496559 (https://phabricator.wikimedia.org/T218345) (owner: 10Kosta Harlan) [18:25:11] (03Abandoned) 10Jforrester: GrowthExperiments: Add MobileFrontend and VisualEditor dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/496559 (https://phabricator.wikimedia.org/T218345) (owner: 10Kosta Harlan) [18:27:05] (03PS1) 10Umherirrender: [EventBus] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513665 [18:29:04] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T220733 (10Jdforrester-WMF) [18:33:02] (03CR) 10Jforrester: [C: 03+2] [EventBus] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513663 (owner: 10Umherirrender) [18:34:35] (03Merged) 10jenkins-bot: [EventBus] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513663 (owner: 10Umherirrender) [18:35:31] !log Reloading Zuul to deploy CentralNotice dependency to EventBus [18:35:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:37:31] (03PS1) 10Umherirrender: [Math] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513669 [18:46:04] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T220733 (10Daimona) [19:01:13] 10Continuous-Integration-Infrastructure, 10phan-taint-check-plugin: mwext-php70-phan-seccheck-docker times out for CommonsMetadata extension - https://phabricator.wikimedia.org/T224351 (10Umherirrender) The failure not for the whole repository, because another simple whitespace change passed the job - https://... [19:03:51] marxarelli: did you already leave a comment on a task? wasn't sure which one you meant [19:06:43] (03PS1) 10Umherirrender: [Sentry] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513673 [19:12:19] legoktm: yeah, i was highly caffeinated this morning so it turned into a long proposal for something on top of what you introduced in extension.json :) [19:12:36] haha [19:13:24] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Core Platform Team (Extension Management (TEC13)), 10Core Platform Team Kanban (Doing), and 2 others: Determine a standard way of installing MediaWiki lib/extension dependencies within containers - https://phabricator.wikimedia.org/T193824 (10dduval... [19:13:40] (03PS1) 10Umherirrender: [WikimediaIncubator] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513675 [19:14:14] legoktm: ^ [19:14:28] hopefully it's semi-coherent [19:15:32] I think so [19:16:08] what I was thinking aobut in my head was definitely overly focused on CI [19:16:58] (03CR) 10Jforrester: [C: 03+2] [Sentry] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513673 (owner: 10Umherirrender) [19:19:04] (03Merged) 10jenkins-bot: [Sentry] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513673 (owner: 10Umherirrender) [19:19:30] legoktm: well and i think that what hashar was thinking about too when he elaborated on the task [19:19:57] which is great, really. it's important to improve what we current do, because i don't see mediawiki going through the pipeline for a while [19:20:35] !log Reloading Zuul to add phan for Sentry [19:20:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:20:43] i'm also trying to solve for the pipeline work (and dev tooling work that brennen and longma are working on) [19:27:47] 10Continuous-Integration-Config, 10User-MarcoAurelio: Archive ApiSandbox on CI - https://phabricator.wikimedia.org/T127012 (10MarcoAurelio) [19:29:52] (03CR) 10Jforrester: [C: 03+2] [MobileFrontend] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513659 (owner: 10Umherirrender) [19:31:31] (03Merged) 10jenkins-bot: [MobileFrontend] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513659 (owner: 10Umherirrender) [19:31:52] !log Reloading Zuul to add phan for MobileFrontend [19:31:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:36:28] (03PS1) 10Umherirrender: [LdapAuthentication] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513680 [19:36:47] 10Phabricator: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10MarcoAurelio) [19:40:45] 10Diffusion, 10Phabricator: Set new Diffusion repos URIs to No I/O by default - https://phabricator.wikimedia.org/T224753 (10MarcoAurelio) [19:41:51] (03CR) 10Jforrester: [C: 03+2] [Math] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513669 (owner: 10Umherirrender) [19:43:23] (03Merged) 10jenkins-bot: [Math] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513669 (owner: 10Umherirrender) [19:43:48] !log Reloading Zuul to add phan for Math(s) [19:43:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:46:08] 10Deployments, 10Release-Engineering-Team (Kanban), 10MediaWiki-Internationalization, 10Wikimedia-Incident, 10Wikimedia-production-error: Post-mortem "MWException: No localisation cache found for English." - https://phabricator.wikimedia.org/T217719 (10Krinkle) 05Open→03Resolved [19:50:42] (03PS3) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [19:50:47] (03PS4) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [19:50:50] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [19:52:45] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [20:01:36] 10Gerrit, 10Release-Engineering-Team, 10serviceops: Gerrit http threads stuck behind sendemail thread - https://phabricator.wikimedia.org/T224448 (10Paladox) I think https://groups.google.com/forum/#!topic/repo-discuss/0yej8sQDcPo may be related. Also im leaning towards (based on the user reply in that threa... [20:08:34] ohhh [20:08:37] i can reproduce ^^ [20:09:32] (03PS5) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [20:10:21] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [20:15:17] (03PS1) 10Umherirrender: [ReadingLists] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513689 [20:17:21] 10Gerrit, 10Release-Engineering-Team, 10serviceops: Gerrit http threads stuck behind sendemail thread - https://phabricator.wikimedia.org/T224448 (10Paladox) I can reproduce this locally now (after adding the web cache to my config locally and set memory to 10, the threads sky rocketed to over 140) [20:17:22] (03PS6) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [20:18:24] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [20:29:20] (03PS7) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [20:31:12] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [20:36:31] (03CR) 10Dvorapa: "Roadmap:" [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [20:38:31] (03PS1) 10Umherirrender: [Jade] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513695 [20:45:01] 10Phabricator, 10Mail: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10Aklapper) p:05Triage→03Unbreak! Thanks for catching that. Confirming. I replied via email to this very task and I don't see my email reply showing up here. Wondering if that's maybe related to movin... [20:46:09] (03PS1) 10Umherirrender: [TimedMediaHandler] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513696 [20:51:32] (03PS1) 10Umherirrender: [JsonConfig] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513698 [20:51:54] (03PS2) 10Umherirrender: [TimedMediaHandler] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513696 [20:52:27] (03PS2) 10Umherirrender: [ReadingLists] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513689 [20:56:18] (03PS8) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [20:56:59] (03PS1) 10Umherirrender: [FlaggedRevs] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/513699 [20:57:11] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [20:57:56] (03PS9) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [20:58:41] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [20:58:55] 10Scap (Scap3-MediaWiki-MVP), 10MediaWiki-Configuration: Scap should not rely on extension-list, instead pass --extension-dir to mergeMessageFileList.php - https://phabricator.wikimedia.org/T125678 (10Krinkle) [21:02:22] (03PS10) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 [21:03:21] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (owner: 10Dvorapa) [21:07:52] hmm, i looked at the wrong graph :( [21:09:24] 10Phabricator, 10Mail: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10greg) Hopefully no data loss as people's sent email shouldn't just disappear from their client/server. BUT, this isn't good. @Dzahn anything from the move/reimage that could have caused this? @mmodell h... [21:10:02] (03PS11) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) [21:11:09] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [21:11:24] (03CR) 10Jforrester: [C: 03+2] [Collection] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513648 (owner: 10Umherirrender) [21:11:26] (03CR) 10Jforrester: [C: 03+2] [ContentTranslation] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513644 (owner: 10Umherirrender) [21:11:28] (03CR) 10Jforrester: [C: 03+2] [Jade] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513695 (owner: 10Umherirrender) [21:13:02] (03Merged) 10jenkins-bot: [Collection] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513648 (owner: 10Umherirrender) [21:13:05] (03Merged) 10jenkins-bot: [ContentTranslation] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513644 (owner: 10Umherirrender) [21:13:08] (03Merged) 10jenkins-bot: [Jade] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513695 (owner: 10Umherirrender) [21:13:31] !log Reloading Zuul to add phan for Collection, ContentTranslation, and Jade [21:13:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:14:42] (03CR) 10Jforrester: [C: 03+2] [TimedMediaHandler] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513696 (owner: 10Umherirrender) [21:14:47] (03CR) 10Jforrester: [C: 03+2] [ReadingLists] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513689 (owner: 10Umherirrender) [21:14:55] (03CR) 10Jforrester: [C: 03+2] [JsonConfig] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513698 (owner: 10Umherirrender) [21:15:04] (03CR) 10Jforrester: [C: 03+2] [FlaggedRevs] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/513699 (owner: 10Umherirrender) [21:17:45] (03Merged) 10jenkins-bot: [TimedMediaHandler] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513696 (owner: 10Umherirrender) [21:17:50] (03Merged) 10jenkins-bot: [ReadingLists] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513689 (owner: 10Umherirrender) [21:17:53] (03PS12) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) [21:18:35] (03Merged) 10jenkins-bot: [JsonConfig] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/513698 (owner: 10Umherirrender) [21:18:37] (03Merged) 10jenkins-bot: [FlaggedRevs] Add dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/513699 (owner: 10Umherirrender) [21:18:53] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [21:18:53] !log Reloading Zuul to add dependencies for TimedMediaHandler, ReadingLists, JsonConfig, and FlaggedRevs. [21:18:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:19:54] (03PS13) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) [21:20:53] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [21:23:36] 10Phabricator, 10Mail: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10Paladox) @greg this is not related to the reimage :) (i know why it's happening, it's due to upstream removal of some old configs) We never moved the "reply to email" config to the new syntax. I only mov... [21:29:25] (03PS14) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) [21:31:58] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Test Pywikibot in Wikimedia CI (2/2) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [21:34:12] (03CR) 10Jforrester: [WIP] Test Pywikibot in Wikimedia CI (2/2) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [21:40:45] 10Continuous-Integration-Config, 10MediaWiki-extensions-General: Add phan to MediaWiki extensions and skins for static analysis [cloneable] - https://phabricator.wikimedia.org/T179554 (10Umherirrender) [21:43:32] (03CR) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [21:58:09] (03CR) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [22:01:09] (03CR) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [22:07:52] (03CR) 10Jforrester: [WIP] Test Pywikibot in Wikimedia CI (2/2) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [22:09:58] (03CR) 10Jforrester: [C: 03+2] [LdapAuthentication] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513680 (owner: 10Umherirrender) [22:11:42] (03Merged) 10jenkins-bot: [LdapAuthentication] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/513680 (owner: 10Umherirrender) [22:12:03] !log Reloading Zuul to add phan for LdapAuthentication [22:12:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:12:29] 10Continuous-Integration-Config, 10Wikibase-Containers, 10Wikidata, 10Maps (Kartographer): CachingKartographerEmbeddingHandlerTest phpunit test from Wikibase is failing - https://phabricator.wikimedia.org/T224785 (10Umherirrender) [22:12:34] (03CR) 10Umherirrender: "Maybe related T224785" [integration/config] - 10https://gerrit.wikimedia.org/r/513698 (owner: 10Umherirrender) [22:14:16] 10Phabricator, 10Mail: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10mmodell) >>! In T224752#5227524, @Paladox wrote: > @greg this is not related to the reimage :) (i know why it's happening, it's due to upstream removal of some old configs) We never moved the "reply to e... [22:14:24] 10Continuous-Integration-Config, 10Wikibase-Containers, 10Wikidata, 10Maps (Kartographer): CachingKartographerEmbeddingHandlerTest phpunit test from Wikibase is failing - https://phabricator.wikimedia.org/T224785 (10Umherirrender) [22:19:59] 10Continuous-Integration-Infrastructure, 10phan-taint-check-plugin: mwext-php70-phan-seccheck-docker times out for CommonsMetadata extension - https://phabricator.wikimedia.org/T224351 (10Umherirrender) p:05Triage→03Lowest For phan I have created stubs and there I have extends a class from itself, which se... [22:21:10] 10Phabricator, 10Mail: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10mmodell) I can't seem to find where mx1001 / mx2001 are configured to forward mail? [22:21:48] 10Continuous-Integration-Config, 10Wikibase-Containers, 10Wikidata, 10Maps (Kartographer): CachingKartographerEmbeddingHandlerTest phpunit test from Wikibase is failing - https://phabricator.wikimedia.org/T224785 (10Umherirrender) p:05Triage→03Unbreak! [22:24:22] 10Phabricator, 10Mail, 10Operations: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10greg) [22:31:37] 10Phabricator, 10Mail, 10Operations: Phabricator email comments not posted - https://phabricator.wikimedia.org/T224752 (10mmodell) from /var/log/exim4/mainlog: `2019-05-31 22:30:20 Warning: No server certificate defined; will use a selfsigned one.` [22:44:03] (03CR) 10Dvorapa: [WIP] Test Pywikibot in Wikimedia CI (2/2) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510927 (https://phabricator.wikimedia.org/T132138) (owner: 10Dvorapa) [23:32:15] (03PS1) 10Jforrester: Revert "[JsonConfig] Add dependency" [integration/config] - 10https://gerrit.wikimedia.org/r/513707 (https://phabricator.wikimedia.org/T224785) [23:32:22] (03CR) 10Jforrester: [C: 03+2] Revert "[JsonConfig] Add dependency" [integration/config] - 10https://gerrit.wikimedia.org/r/513707 (https://phabricator.wikimedia.org/T224785) (owner: 10Jforrester) [23:33:23] 10Continuous-Integration-Config, 10Wikibase-Containers, 10Wikidata, 10Maps (Kartographer), and 2 others: CachingKartographerEmbeddingHandlerTest phpunit test from Wikibase is failing - https://phabricator.wikimedia.org/T224785 (10Jdforrester-WMF) a:03Jdforrester-WMF [23:33:54] (03Merged) 10jenkins-bot: Revert "[JsonConfig] Add dependency" [integration/config] - 10https://gerrit.wikimedia.org/r/513707 (https://phabricator.wikimedia.org/T224785) (owner: 10Jforrester) [23:34:09] !log Reloading Zuul to remove Kartographer dependency for JsonConfig T224785 [23:34:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:34:11] T224785: CachingKartographerEmbeddingHandlerTest phpunit test from Wikibase is failing - https://phabricator.wikimedia.org/T224785 [23:48:54] 10MediaWiki-Releasing, 10Release-Engineering-Team (Kanban), 10MW-1.33-release: Branch REL1_33 for MediaWiki and deployed extensions - https://phabricator.wikimedia.org/T220653 (10Jdforrester-WMF) a:0320after4 [23:53:37] 10MediaWiki-Releasing, 10Release-Engineering-Team (Kanban), 10MW-1.33-release: Branch REL1_33 for MediaWiki and deployed extensions - https://phabricator.wikimedia.org/T220653 (10Paladox) a:0520after4→03mmodell [23:55:31] 10MediaWiki-Releasing, 10Release-Engineering-Team (Kanban), 10MW-1.33-release: Branch REL1_33 for MediaWiki and deployed extensions - https://phabricator.wikimedia.org/T220653 (10mmodell) thanks @Paladox