[00:04:24] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10User-Elukey: EventLogging broken in beta - https://phabricator.wikimedia.org/T185952#3935858 (10Nuria) I see vents being inserted: MariaDB [log]> select timestamp from MobileWikiAppShareAFact_12588711 order by timestamp desc limit 10;... [00:04:27] (03CR) 10Hashar: [C: 032] Register extreg-wos for tox-docker tests [integration/config] - 10https://gerrit.wikimedia.org/r/406999 (owner: 10MarcoAurelio) [00:05:22] (03CR) 10Hashar: [C: 032] Make ORES extension selenium tests mandatory [integration/config] - 10https://gerrit.wikimedia.org/r/406989 (https://phabricator.wikimedia.org/T184451) (owner: 10Ladsgroup) [00:05:41] (03Merged) 10jenkins-bot: Register extreg-wos for tox-docker tests [integration/config] - 10https://gerrit.wikimedia.org/r/406999 (owner: 10MarcoAurelio) [00:06:31] (03Merged) 10jenkins-bot: Make ORES extension selenium tests mandatory [integration/config] - 10https://gerrit.wikimedia.org/r/406989 (https://phabricator.wikimedia.org/T184451) (owner: 10Ladsgroup) [00:08:46] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832#3935863 (10Paladox) @dzahn did you get a strace? :) [00:08:49] 10Phabricator, 10Scap: GPG Sign git tags - https://phabricator.wikimedia.org/T150696#3935864 (10mmodell) 05Open>03Resolved a:03mmodell I think signing with our individual keys is good enough. [00:10:57] Hm.. which team tag should I add to issues with Special:Version? [00:12:48] (03CR) 10Hashar: [C: 032] "Flake8 fails but https://gerrit.wikimedia.org/r/#/c/407180/ address that." [integration/config] - 10https://gerrit.wikimedia.org/r/406999 (owner: 10MarcoAurelio) [00:13:25] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10User-Elukey: EventLogging broken in beta - https://phabricator.wikimedia.org/T185952#3935878 (10Nuria) InputDeviceDynamics_17661826 has couple records from today, but it does not look there were many events sent from all-events.log [00:13:28] 10Deployments, 10Release-Engineering-Team (Kanban): Add jobrunners to Scap canary process - https://phabricator.wikimedia.org/T172480#3935880 (10mmodell) [00:13:32] 10Gerrit, 10ORES, 10Operations, 10Scoring-platform-team, 10Patch-For-Review: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3935879 (10mmodell) [00:13:36] 10Release-Engineering-Team (Next): When "scap pull" does a (slow) CDB rebuild, it should tell me that that's what it's doing - https://phabricator.wikimedia.org/T162207#3935882 (10mmodell) [00:13:38] 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Use git as transport mechanism for MediaWiki scap deploys - https://phabricator.wikimedia.org/T147938#3935883 (10mmodell) [00:13:40] 10Release-Engineering-Team (Kanban), 10scap2, 10Patch-For-Review: Eliminate symlinks in mediawiki-config (as much as possible) - https://phabricator.wikimedia.org/T126306#3935884 (10mmodell) [00:14:54] (03CR) 10Hashar: "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/406989 (https://phabricator.wikimedia.org/T184451) (owner: 10Ladsgroup) [00:16:29] 10Scap: Don't continue scap if sync to all proxies failed - https://phabricator.wikimedia.org/T110791#3935887 (10mmodell) p:05Triage>03Normal [00:16:56] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832#3935889 (10Dzahn) Unfortunately not, i'll try to catch it next time. I have the Grafana link f... [00:17:02] 10Scap: scap shouldn't log completion (it should log fail!) - https://phabricator.wikimedia.org/T110793#3935890 (10mmodell) p:05Triage>03Normal [00:17:33] 10Scap: Improve scap canary check messages - https://phabricator.wikimedia.org/T142342#3935892 (10mmodell) p:05Triage>03Normal [00:17:39] (03CR) 10Ladsgroup: "Thank you :)" [integration/config] - 10https://gerrit.wikimedia.org/r/406989 (https://phabricator.wikimedia.org/T184451) (owner: 10Ladsgroup) [00:19:17] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10User-Elukey: EventLogging broken in beta - https://phabricator.wikimedia.org/T185952#3935908 (10Tgr) ``` tgr@deployment-eventlog02:~$ date Thu Feb 1 00:18:32 UTC 2018 tgr@deployment-eventlog02:~$ ack-grep InputDeviceDynamics /srv/log... [00:19:31] 10Release-Engineering-Team (Someday), 10Scap: Support shallow clones - https://phabricator.wikimedia.org/T157149#3935909 (10mmodell) 05Open>03Resolved a:03mmodell [00:20:20] 10Release-Engineering-Team (Someday), 10Scap: Support shallow clones - https://phabricator.wikimedia.org/T157149#2996963 (10mmodell) This is done automagically now, at least for submodules. [00:24:43] 10Scap: Support git-lfs - https://phabricator.wikimedia.org/T180627#3764014 (10mmodell) p:05Triage>03Normal [00:25:02] 10Scap, 10Global-Collaboration, 10MediaWiki-extensions-LocalisationUpdate, 10User-Nikerabbit: Alert when l10update fails - https://phabricator.wikimedia.org/T171925#3480394 (10mmodell) p:05Triage>03Normal [00:25:46] 10Scap, 10WorkType-NewFunctionality: Play elevator music while scap is running - https://phabricator.wikimedia.org/T170484#3935937 (10mmodell) 05Open>03declined :( [00:26:44] 10Scap: Include timestamp in `Failed to acquire lock` message - https://phabricator.wikimedia.org/T174466#3935940 (10mmodell) p:05Triage>03Normal [00:27:55] 10Scap: Replace scap.args with docopt - https://phabricator.wikimedia.org/T186110#3935944 (10mmodell) p:05Triage>03Lowest scap is heavily dependent on argparse. This would be a large amount of work and it would break many things. [00:28:23] 10Scap: Replace scap.args with docopt - https://phabricator.wikimedia.org/T186110#3935946 (10mmodell) docopt was evaluated at the time when we built scap3. I can't remember the history now but there was a reason we went with argparse. [00:29:38] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Scap sync-file: report the file on IRC/SAL on canary error rate failure - https://phabricator.wikimedia.org/T186064#3935950 (10mmodell) p:05Triage>03Normal [00:30:21] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Scap sync-file: allow to sync multiple files in different directories - https://phabricator.wikimedia.org/T186067#3932756 (10mmodell) Isn't this already the case? [00:31:16] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [00:31:44] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10User-Elukey: EventLogging broken in beta - https://phabricator.wikimedia.org/T185952#3935956 (10Nuria) >So yes, it is working somewhat, but some of the events seem to get lost (delayed?). They are delayed yes (always, EL insertion in M... [00:39:59] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10User-Elukey: EventLogging broken in beta - https://phabricator.wikimedia.org/T185952#3935958 (10Tgr) Seems like all the beta eventlogging tables use InnoDB. [00:40:52] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10User-zeljkofilipin: Continuous integration for mediawiki-vagrant - https://phabricator.wikimedia.org/T183456#3935959 (10zeljkofilipin) There is also [[ https://wikitech.wikimedia.org/wiki/Help:MediaWiki-Vag... [00:44:16] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10User-Elukey: EventLogging broken in beta - https://phabricator.wikimedia.org/T185952#3935967 (10Tgr) ``` MariaDB [log]> show global variables like 'default_storage_engine'; +------------------------+--------+ | Variable_name |... [00:59:27] zeljkof: https://gerrit.wikimedia.org/r/407185 [01:26:21] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10User-zeljkofilipin: Continuous integration for mediawiki-vagrant - https://phabricator.wikimedia.org/T183456#3936003 (10hashar) vagrant up --provider=libvirt ends up falling with: Error while activating ne... [03:07:40] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<22.22%) [04:23:16] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10User-zeljkofilipin: Continuous integration for mediawiki-vagrant - https://phabricator.wikimedia.org/T183456#3936111 (10bd808) Cloud VPS problems possibly related to {T180377}? I did test basic provisioning... [06:04:33] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:05:25] PROBLEM - Free space - all mounts on integration-slave-jessie-1001 is CRITICAL: CRITICAL: integration.integration-slave-jessie-1001.diskspace._mnt.byte_percentfree (No valid datapoints found)integration.integration-slave-jessie-1001.diskspace._srv.byte_percentfree (<100.00%) [06:09:32] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [06:15:31] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:20:30] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [06:52:39] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:08:17] I'm going to trip the CI alarm soon [07:09:54] !log legoktm@integration-slave-jessie-1001:/srv/jenkins-workspace/workspace$ sudo rm -rf * [07:10:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:16:39] PROBLEM - Work requests waiting in Zuul Gearman server on contint1001 is CRITICAL: CRITICAL: 42.86% of data above the critical threshold [140.0] https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [07:20:26] RECOVERY - Free space - all mounts on integration-slave-jessie-1001 is OK: OK: integration.integration-slave-jessie-1001.diskspace._mnt.byte_percentfree (No valid datapoints found) [07:24:23] legoktm: do I understand right that the thing you did should fix the problem of test queue being stuck? [07:25:07] SMalyshev: no, there's just a ton of changes in the queue, it'll take an hour or two to catch up and handle the load [07:25:32] sorry about that, I normally try and do it at US night before Europeans wake up [07:25:37] oh, so it's not some failure, it's just me being unlucky? ok, I'll wait [07:25:59] no big deal, I just wanted to be sure it's not some problem there... [07:28:56] 10Continuous-Integration-Config, 10BlueSpice, 10Patch-For-Review: Enable unit tests on BlueSpice* repos - https://phabricator.wikimedia.org/T130811#3936456 (10Osnard) @Umherirrender, thanks for the explanation. We will switch to `extension.json/AutoloadNamespaces` step by step. @Paladox, another question ab... [07:34:16] 10Phabricator, 10WMSE-Bug-Reporting-and-Translation-2018: Trying to import workboard for some projects give error 400 - https://phabricator.wikimedia.org/T186189#3936461 (10Sebastian_Berlin-WMSE) [07:38:35] 10Phabricator: Tag URL for milestone without board causes weird 404 redirect - https://phabricator.wikimedia.org/T186173#3935733 (10Peachey88) Hmm i was suspecting it may have been we already had a project called Maps (https://phabricator.wikimedia.org/project/edit/1127/) so I renamed it, but that didn't seem to... [08:20:40] RECOVERY - Work requests waiting in Zuul Gearman server on contint1001 is OK: OK: Less than 30.00% above the threshold [90.0] https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [09:11:14] 10Scap: scap should always announce when it starts changing the cluster state - https://phabricator.wikimedia.org/T164980#3936625 (10Aklapper) [09:14:36] Project mwext-phpunit-coverage-publish build #444: 04FAILURE in 37 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/444/ [10:23:03] Yippee, build fixed! [10:23:04] Project mwext-phpunit-coverage-publish build #445: 09FIXED in 1 min 0 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/445/ [10:29:34] 10Project-Admins: Create a project for Tool-Hashtags - https://phabricator.wikimedia.org/T186103#3936792 (10Aklapper) Where to find that tool? Is this supposed to be a subproject of https://phabricator.wikimedia.org/tag/tools/ ? If so, no "Tool-" name prefix needed [10:33:37] 10Project-Admins: Create a project for Tool-Hashtags - https://phabricator.wikimedia.org/T186103#3936811 (10Samwalton9) At http://tools.wmflabs.org/hashtags - I wanted to distinguish the tool from any (possible) future efforts to integrate hashtags into Mediawiki directly, and wasn't sure how I should name the t... [10:59:08] 10Release-Engineering-Team (Kanban), 10Mediawiki-extensions-PropertySuggester, 10Repository-Admins, 10Wikidata: Move PropertySuggester-Python to gerrit - https://phabricator.wikimedia.org/T166672#3936883 (10Lydia_Pintscher) [11:31:00] 10Phabricator: Tag URL for milestone without board causes weird 404 redirect to https://phabricator.wikimedia.org/tag// - https://phabricator.wikimedia.org/T186173#3937072 (10Aklapper) [12:03:37] Project mwext-phpunit-coverage-publish build #458: 04FAILURE in 1 min 27 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/458/ [12:33:30] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [12:42:53] PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [12:43:32] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [13:17:53] RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [13:26:50] 10Phabricator: My username shows wrong - https://phabricator.wikimedia.org/T185998#3937372 (10Aklapper) Whatever you entered as username is now the username: https://phabricator.wikimedia.org/p/CodeCat/ Usernames can be changed by admins, so yes it could be fixed. [13:31:15] 10Deployments, 10MediaWiki-extensions-LocalisationUpdate: Localization update not reflected on arwiki - https://phabricator.wikimedia.org/T186038#3937384 (10Aklapper) 05Open>03Invalid See https://www.mediawiki.org/wiki/MediaWiki_1.31/Roadmap [13:44:29] 10Phabricator: Phabricator account deletion request - https://phabricator.wikimedia.org/T185703#3937421 (10Aklapper) I have no idea what "subs" are. [13:53:20] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Proton, 10Readers-Web-Backlog, and 2 others: Set up Jenkins for chromium-render and chromium-render-deploy repositories - https://phabricator.wikimedia.org/T179552#3728431 (10pmiazga) a:05pmiazga>03None [14:18:37] Yippee, build fixed! [14:18:37] Project mwext-phpunit-coverage-publish build #459: 09FIXED in 53 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/459/ [14:49:07] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Recommendation-API, 10Patch-For-Review, 10Scoring-platform-team (Current): What to do with deployment-sca03? - https://phabricator.wikimedia.org/T184501#3885749 (10awight) @mobrovac @Ottomata Great, thanks for the confirmation! I searched oper... [14:53:14] PROBLEM - Host deployment-puppetdb01 is DOWN: CRITICAL - Host Unreachable (10.68.23.76) [15:02:15] 10Release-Engineering-Team (Someday), 10Scap: Support shallow clones - https://phabricator.wikimedia.org/T157149#3937681 (10Halfak) Thank you! This'll make a big difference for ORES deployments :) [15:26:05] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate operations-mw-config-composer-hhvm-jessie to Docker - https://phabricator.wikimedia.org/T186145#3935015 (10Addshore) +999 for this and mediawiki-config no longer having to wait for nodepool! > - doctrine/instantia... [15:28:48] 10Release-Engineering-Team (Someday), 10Scap: Support shallow clones - https://phabricator.wikimedia.org/T157149#2996963 (10awight) O_O Don't tell releng, but this is most of what we wanted git-fat for... [15:46:18] 10Release-Engineering-Team (Someday), 10Scap: Support shallow clones - https://phabricator.wikimedia.org/T157149#3937820 (10Halfak) Na. Development of ORES is terrible without git-lfs. I challenge you to try to experiment with a modeling strategy on a slow connection in Asia. :P [15:47:11] halfak: ^ lol, a hippie ISP in California was a good-enough approximation. I would go into the office to git. [15:47:53] :D [15:59:42] 10Release-Engineering-Team (Kanban), 10Operations, 10Release Pipeline: Package/upload service-checker for Debian stretch - https://phabricator.wikimedia.org/T184224#3937871 (10akosiaris) 05Open>03Resolved a:03akosiaris Package uploaded! Since it's a native package I 've had to bump the version number t... [15:59:44] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Patch-For-Review: Build service-checker image for use with helm test - https://phabricator.wikimedia.org/T184220#3937874 (10akosiaris) [16:07:24] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10User-Elukey: EventLogging broken in beta - https://phabricator.wikimedia.org/T185952#3937881 (10Nuria) I am going to restart eventlogging from master (it was on a different changeset than what we have now inprod). Let's take a look ag... [16:10:13] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Recommendation-API, 10Patch-For-Review, 10Scoring-platform-team (Current): What to do with deployment-sca03? - https://phabricator.wikimedia.org/T184501#3937894 (10Ottomata) > I couldn't find the configuration for the Kafka "jumbo" cluster in b... [16:13:08] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Proton, 10Readers-Web-Backlog, and 2 others: Set up Jenkins for chromium-render and chromium-render-deploy repositories - https://phabricator.wikimedia.org/T179552#3937900 (10Niedzielski) a:03Niedzielski [16:14:48] !log deleting deployment-sca03 (T184501) [16:14:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:14:55] T184501: What to do with deployment-sca03? - https://phabricator.wikimedia.org/T184501 [16:17:23] (03CR) 10Niedzielski: [C: 04-1] "Should this patch be abandoned since I0565bdf22188c3990d521f996a0f94ce0a958a77 is merged?" [integration/config] - 10https://gerrit.wikimedia.org/r/394058 (https://phabricator.wikimedia.org/T179552) (owner: 10Phuedx) [16:17:56] PROBLEM - Host deployment-sca03 is DOWN: CRITICAL - Host Unreachable (10.68.21.183) [16:29:18] (03Abandoned) 10Phuedx: Add npm job for the Chromium render service [integration/config] - 10https://gerrit.wikimedia.org/r/394058 (https://phabricator.wikimedia.org/T179552) (owner: 10Phuedx) [16:32:16] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Proton, 10Readers-Web-Backlog, and 2 others: Set up Jenkins for chromium-render and chromium-render-deploy repositories - https://phabricator.wikimedia.org/T179552#3937989 (10phuedx) >>! In T179552#3907227, @mmodell wrote: > So what re... [16:33:18] PROBLEM - Puppet errors on deployment-logstash2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:07:41] 10Continuous-Integration-Config, 10ORES, 10Scoring-platform-team: Migrate ORES CI to Stretch - https://phabricator.wikimedia.org/T186239#3938129 (10awight) [17:25:12] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832#3835986 (10elukey) a gdb `thread apply all bt` would probably be more useful to get where http... [17:28:47] PROBLEM - Puppet errors on deployment-snapshot01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:32:39] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832#3938199 (10elukey) The other useful thing to do, without waiting for a complete leak, is to ch... [17:32:53] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations, 10User-Elukey: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832#3938200 (10Paladox) Also it seems that restarting it every sunday would not r... [17:32:56] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations, 10User-Elukey: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832#3938201 (10elukey) [17:38:54] 10Continuous-Integration-Config, 10BlueSpice, 10Patch-For-Review: Enable unit tests on BlueSpice* repos - https://phabricator.wikimedia.org/T130811#3938227 (10Umherirrender) The extension itself is loaded first and than the dependencies. That is also a problem for BlueSpice, because some callback using Defin... [17:53:40] Project mwext-phpunit-coverage-publish build #471: 04FAILURE in 2 min 42 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/471/ [17:58:41] Yippee, build fixed! [17:58:42] Project mwext-phpunit-coverage-publish build #472: 09FIXED in 1 min 6 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/472/ [18:20:27] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-Addshore: Provide php-ast 0.1.5 or later as a Debian package for CI - https://phabricator.wikimedia.org/T174338#3938342 (10hashar) a:03hashar Both Debian bugs got fixed: * [[ https://bugs.debian.org/... [18:30:58] (03PS4) 10Zoranzoki21: Add few extensions to zuul/layout.yaml in so Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [18:46:44] 10Phabricator (2017-06-01), 10RelEng-Archive-FY201718-Q1: Enable embedding of media from Wikimedia Commons - https://phabricator.wikimedia.org/T116515#1751376 (10Tgr) Upstream task is [[https://secure.phabricator.com/T4190|T4190]]. [18:50:27] 10Phabricator: Enable image hotlinking - https://phabricator.wikimedia.org/T186246#3938548 (10Tgr) [18:52:58] 10Phabricator: Enable image hotlinking - https://phabricator.wikimedia.org/T186246#3938559 (10Tgr) [18:54:29] PROBLEM - Puppet errors on deployment-secureredirexperiment is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:06:45] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate operations-mw-config-composer-hhvm-jessie to Docker - https://phabricator.wikimedia.org/T186145#3938606 (10hashar) It is an HHVM container :-) I am not sure yet why it works on a Nodepool Jessie instance though. S... [19:07:30] James_F: So I'm on train duty next week...I s'pose we could do some quick dry runs of our new favorite script <3 [19:13:32] 10Continuous-Integration-Infrastructure, 10Operations, 10Traffic, 10Patch-For-Review: Lower varnish caching length on doc.wikimedia.org - https://phabricator.wikimedia.org/T184255#3877424 (10hashar) The patch got merged and deployed. So tentatively that is fixed? [19:13:41] 10Continuous-Integration-Infrastructure, 10Operations, 10Traffic: Lower varnish caching length on doc.wikimedia.org - https://phabricator.wikimedia.org/T184255#3938632 (10hashar) [19:21:29] 10MediaWiki-Codesniffer: Add phpcs codesniffer check to verify presence of @covers annotation in phpcs - https://phabricator.wikimedia.org/T186251#3938706 (10pmiazga) [19:24:02] 10MediaWiki-Codesniffer: Add phpcs codesniffer check to verify presence of @covers annotation in phpcs - https://phabricator.wikimedia.org/T186251#3938717 (10pmiazga) After creating a task I found there is something similar - like it's a duplicate of T179094. [19:25:25] 10MediaWiki-Codesniffer: Add rule to require use of @covers in PHPUnit tests - https://phabricator.wikimedia.org/T179094#3713087 (10pmiazga) There is also a `@coversNothing` annotation which may be useful in some high level/integration tests. [19:26:57] 10MediaWiki-Codesniffer: Add phpcs codesniffer check to verify presence of @covers annotation in phpcs - https://phabricator.wikimedia.org/T186251#3938744 (10pmiazga) 05Open>03declined [19:27:06] 10MediaWiki-Codesniffer: Add phpcs codesniffer check to verify presence of @covers annotation in phpcs - https://phabricator.wikimedia.org/T186251#3938661 (10pmiazga) Declined as duplicate [19:31:31] brion: Re: making scap faster on ops list....I think the next big gain we'll have is when we drop cdb files for l10n [19:31:49] Validating-and-possibly-rebuilding those is the *bulk* of our time [19:32:16] Bulk of the time on a long-running scap, that is [19:32:40] funnnn [19:33:11] it's been a while since i looked at scap's innards -- is the actual data pushing part based on rsync still or something more like a git pull? [19:33:31] sounds like that's not the slow part though anymore :D [19:34:09] * brion really just wanted an excuse to post that far side cartoon [19:35:06] oh look! cumulative update for windows 10. /me goes to update all his darn windows test boxen [19:36:49] hah this version disables some of the meltdown/spectre mitigations that broke amd processors [19:36:59] * brion is Intel Inside [19:37:34] Yeah, it's rsync [19:37:39] There's some git fun-stuff [19:37:46] But the core of "move files" is rsync [19:38:39] *nod* sensible enough, that's what it does :D [19:39:04] could be slightly faster to pull updates from a git tree and then only apply the latest changes as necessary, but that could be more error prone [19:39:11] and if it's not the key slowdown, i wouldn't push on it [19:39:14] 10Gerrit, 10RfC: [RFC] Allow 100 characters per line - https://phabricator.wikimedia.org/T186255#3938812 (10Zoranzoki21) [19:39:31] 10Gerrit, 10RfC: [RFC] Allow 100 characters per line - https://phabricator.wikimedia.org/T186255#3938823 (10Zoranzoki21) @Dvorapa Please fix me. [19:39:58] no_justification ^^ that requires us changing the gerrit config right? [19:41:54] 10MediaWiki-Codesniffer, 10Readers-Web-Backlog (Tracking): Add rule to require use of @covers in PHPUnit tests - https://phabricator.wikimedia.org/T179094#3938831 (10pmiazga) >>! In T179094#3713477, @Tgr wrote: > A lot of extensions do not follow the `tests/phpunit` directory structure. IMHO checking that fun... [19:43:57] greg-g: can I grab a Monday deploy window for T186244? [19:43:58] T186244: Deploy AICaptcha data collection - https://phabricator.wikimedia.org/T186244 [19:44:36] I think it would fit into SWAT but it involves a bunch of backports and I imagine Monday will be extra busy so I don't want to stress people out :) [19:47:20] brion: That's the long-term plan [19:47:28] (merging scap2 and scap3 behaviors) [19:47:30] nice [19:50:21] paladox: Yes, I'm about to decline it [19:50:30] I think the reporter is confused ;-) [19:51:02] ok [19:51:03] heh [19:51:47] tgr: yeah, that sounds reasonable. I was just doing my due diligence on it :) [19:53:51] 10Gerrit: [RFC] Allow 100 characters per line - https://phabricator.wikimedia.org/T186255#3938916 (10demon) What do you mean by problems? Just click the gear icon and change your diff preferences to 100 or whatever number works for you (mine is set to 72). I see no reason to change this. Also: this is not an RF... [19:54:01] 10Gerrit: [RFC] Allow 100 characters per line - https://phabricator.wikimedia.org/T186255#3938918 (10demon) 05Open>03declined [20:07:44] 10Release-Engineering-Team (Someday), 10Scap: Support shallow clones - https://phabricator.wikimedia.org/T157149#3938997 (10demon) >>! In T157149#3937766, @awight wrote: > O_O Don't tell releng, but this is most of what we wanted git-fat for... There's plenty of other reasons for (and users of) git-fat suppo... [20:22:39] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate operations-mw-config-composer-hhvm-jessie to Docker - https://phabricator.wikimedia.org/T186145#3939064 (10Addshore) >>! In T186145#3938606, @hashar wrote: > It is an HHVM container :-) I am not sure yet why it wo... [21:08:22] PROBLEM - Puppet errors on deployment-mx02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:36:45] Project mwext-phpunit-coverage-publish build #480: 04FAILURE in 2 min 32 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/480/ [21:37:08] Yippee, build fixed! [21:37:08] Project mwext-phpunit-coverage-publish build #481: 09FIXED in 22 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/481/ [21:39:53] (03CR) 10Legoktm: "Why not in gate-and-submit too?" [integration/config] - 10https://gerrit.wikimedia.org/r/406989 (https://phabricator.wikimedia.org/T184451) (owner: 10Ladsgroup) [21:42:53] (03PS1) 10Legoktm: Flow depends upon Echo [integration/config] - 10https://gerrit.wikimedia.org/r/407560 [22:36:31] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: npm-node-6-docker tests failing for Android project. - https://phabricator.wikimedia.org/T185931#3939477 (10hashar) With the all-hands and offsite I could not look into it. I am travelling currently until Friday evening. Will investigate/fix... [22:38:01] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: npm-node-6-docker tests failing for Android project. - https://phabricator.wikimedia.org/T185931#3928972 (10Paladox) We should probably publish a npm package with npm 3 in it. It will fix alot of issues. [22:45:57] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: npm-node-6-docker tests failing for Android project (test fallback to broken npm 1.4.21 instead of 3.8.3) - https://phabricator.wikimedia.org/T185931#3939513 (10hashar) [23:44:53] 10Beta-Cluster-Infrastructure, 10Operations, 10Wikimedia-General-or-Unknown: Beta English Wikipedia: History of the page 'Bird' generates a 500 or 503 error - https://phabricator.wikimedia.org/T185969#3939659 (10Dzahn) [23:47:17] 10Beta-Cluster-Infrastructure, 10Operations, 10Wikimedia-General-or-Unknown: Beta English Wikipedia: History of the page 'Bird' generates a 500 or 503 error - https://phabricator.wikimedia.org/T185969#3939673 (10matmarex) It is still broken for me in the same way. Looks like the error only happens when I'm l...