[04:06:11] Yippee, build fixed! [04:06:11] Project selenium-MultimediaViewer » safari,beta,OS X 10.9,contintLabsSlave && UbuntuTrusty build #285: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=safari,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=contintLabsSlave%20&&%20UbuntuTrusty/285/ [06:38:39] PROBLEM - Puppet run on integration-slave-trusty-1011 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:06:14] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T155525#2995668 (10aaron) [07:13:40] RECOVERY - Puppet run on integration-slave-trusty-1011 is OK: OK: Less than 1.00% above the threshold [0.0] [08:24:47] 10Gerrit: ldap/wmf group should not have +2 in Gerrit - https://phabricator.wikimedia.org/T62412#667269 (10Bawolff) >This would be much more transparent. There is also a transparency benefit of keeping separate groups for people who have +2 rights due to community process vs due to WMF employment. [09:35:07] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 07HHVM: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923#2995857 (10MoritzMuehlenhoff) 05Open>03Resolved a:03MoritzMuehlenhoff A new HHVM package has been uploaded... [09:48:27] 10Gerrit: ldap/wmf group should not have +2 in Gerrit - https://phabricator.wikimedia.org/T62412#2995888 (10Legoktm) >>! In T62412#667322, @MZMcBride wrote: > Is there a way to see the members of the ldap/wmf group? I created to make the list more easily accessible. [09:54:21] !log Regenerate Nodepool Jessie snapshot. Would get a new HHVM version T156923 [09:54:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:54:25] T156923: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923 [10:12:04] !log Image ci-jessie-wikimedia-1486115643 in wmflabs-eqiad is ready T156923 [10:12:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:12:07] T156923: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923 [10:17:14] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: Make it possible to run the mediawiki testsuite against a staging repo of apt.wikimedia.org - https://phabricator.wikimedia.org/T157038#2995994 (10MoritzMuehlenhoff) The tests should also cover the extensions (e.g. Scribunto)... [10:25:49] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 07HHVM: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923#2996018 (10hashar) 05Resolved>03Open Reopening since it might still fails on some CI jobs and/or on the beta... [10:29:11] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 07HHVM: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923#2996021 (10MoritzMuehlenhoff) p:05Unbreak!>03High [10:31:31] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: Make it possible to run the mediawiki testsuite against a staging repo of apt.wikimedia.org - https://phabricator.wikimedia.org/T157038#2993373 (10hashar) We can get a copy of the mediawiki-extensions-* job. That clones mediaw... [10:34:26] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: Make it possible to run the mediawiki testsuite against a staging repo of apt.wikimedia.org - https://phabricator.wikimedia.org/T157038#2996046 (10MoritzMuehlenhoff) The mechanism should not be specific to HHVM, but apply to t... [10:42:26] !log Image ci-jessie-wikimedia-1486115643 in wmflabs-eqiad is ready T156923 [10:42:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:42:30] T156923: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923 [10:58:16] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 07HHVM: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923#2996078 (10hashar) 05Open>03Resolved There were mentions of Scribunto errors. I did a `check experimental`... [11:00:42] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 07HHVM: New HHVM 3.12.11 segfault at end of MediaWiki PHPUnit tests - https://phabricator.wikimedia.org/T156923#2996082 (10hashar) The Scribunto error is on production canaries and got reported at T157110 [11:05:47] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: Make it possible to run the mediawiki testsuite against a staging repo of apt.wikimedia.org - https://phabricator.wikimedia.org/T157038#2996102 (10hashar) Ok that makes sense. For Nodepool we would need a new image, that woul... [11:09:41] !log beta: removed old kernels from deployment-redis02 to free up disk space [11:09:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:13:31] 10Gerrit: ldap/wmf group should not have +2 in Gerrit - https://phabricator.wikimedia.org/T62412#2996117 (10Nemo_bis) >>! In T62412#2995731, @Bawolff wrote: >>This would be much more transparent. > > There is also a transparency benefit of keeping separate groups for people who have +2 rights due to community p... [11:14:39] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996118 (10hashar) Can it be that sometime we exhaust the number of concurrent sessions? Though we run the tests serially, so unless w... [11:22:11] (03Abandoned) 10Hashar: Whitelist Alexia [integration/config] - 10https://gerrit.wikimedia.org/r/332002 (owner: 10Paladox) [11:49:47] (03PS1) 10Hashar: non voting testextension jobs to nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/335788 [11:52:56] (03PS2) 10Hashar: non voting testextension jobs to nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/335788 [11:53:11] (03CR) 10Hashar: [C: 032] non voting testextension jobs to nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/335788 (owner: 10Hashar) [11:54:34] (03Merged) 10jenkins-bot: non voting testextension jobs to nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/335788 (owner: 10Hashar) [12:02:51] Project selenium-RelatedArticles » chrome,beta-mobile,Linux,contintLabsSlave && UbuntuTrusty build #296: 04FAILURE in 1 min 50 sec: https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-mobile,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/296/ [12:11:58] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996224 (10zeljkofilipin) @hashar Good idea about checking the limit, but we never reach it. {F5464191} Do you have a Wikimedia Sauce... [12:40:25] (03PS1) 10Hashar: Shift HHVM jobs from Trusty to Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/335797 [12:45:13] (03CR) 10Hashar: [C: 032] Shift HHVM jobs from Trusty to Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/335797 (owner: 10Hashar) [12:47:01] (03Merged) 10jenkins-bot: Shift HHVM jobs from Trusty to Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/335797 (owner: 10Hashar) [13:04:21] Project selenium-Math » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #290: 04FAILURE in 21 sec: https://integration.wikimedia.org/ci/job/selenium-Math/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/290/ [13:24:27] 06Release-Engineering-Team, 06Operations, 10Phabricator, 10hardware-requests, 10ops-eqiad: replacement hardware for iridium (phabricator) - https://phabricator.wikimedia.org/T156970#2996318 (10Paladox) For testing phabricator on Jessie, I've setup phabricator-01.wmflabs.org. All looks ok at the moment. [13:46:24] Project selenium-VisualEditor » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #294: 04FAILURE in 2 min 23 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/294/ [14:01:55] (03PS1) 10Hashar: Unconfigure mediawiki/skins/Slate [integration/config] - 10https://gerrit.wikimedia.org/r/335799 [14:01:57] (03PS1) 10Hashar: Clean out usage of jshint/jsonlint/phplint [integration/config] - 10https://gerrit.wikimedia.org/r/335800 (https://phabricator.wikimedia.org/T119973) [14:07:31] (03PS1) 10Hashar: Remove check jobs from wikimedia/fundraising [integration/config] - 10https://gerrit.wikimedia.org/r/335803 (https://phabricator.wikimedia.org/T119973) [14:09:42] twentyafterfour: I'm surprised this hasn't gotten more ire yet https://phabricator.wikimedia.org/T156996#2995090 https://phabricator.wikimedia.org/T156996#2995680 https://phabricator.wikimedia.org/T157106 [14:09:52] weird things start to happen when that feed is down [14:11:26] (03CR) 10Hashar: [C: 032] Unconfigure mediawiki/skins/Slate [integration/config] - 10https://gerrit.wikimedia.org/r/335799 (owner: 10Hashar) [14:11:28] (03CR) 10Hashar: [C: 032] Clean out usage of jshint/jsonlint/phplint [integration/config] - 10https://gerrit.wikimedia.org/r/335800 (https://phabricator.wikimedia.org/T119973) (owner: 10Hashar) [14:11:30] (03CR) 10Hashar: [C: 032] Remove check jobs from wikimedia/fundraising [integration/config] - 10https://gerrit.wikimedia.org/r/335803 (https://phabricator.wikimedia.org/T119973) (owner: 10Hashar) [14:12:11] Krinkle ^ fyi [14:12:29] (03Merged) 10jenkins-bot: Unconfigure mediawiki/skins/Slate [integration/config] - 10https://gerrit.wikimedia.org/r/335799 (owner: 10Hashar) [14:12:31] (03Merged) 10jenkins-bot: Clean out usage of jshint/jsonlint/phplint [integration/config] - 10https://gerrit.wikimedia.org/r/335800 (https://phabricator.wikimedia.org/T119973) (owner: 10Hashar) [14:13:01] (03Merged) 10jenkins-bot: Remove check jobs from wikimedia/fundraising [integration/config] - 10https://gerrit.wikimedia.org/r/335803 (https://phabricator.wikimedia.org/T119973) (owner: 10Hashar) [14:13:17] ERRRKAKZ [14:13:24] chasemp: that is irc.wikimedia.org being broken right? [14:13:32] yeah [14:13:38] ... [14:13:41] afaiu [14:13:44] if it was only me [14:13:47] I would leave it broken [14:13:53] so folks STOP using irc.wikimedia.org entirely [14:14:10] that's a terrible plan considering not notice and the tools that rely on it [14:14:35] that's like breaking your car so you can lose weight walking an hour before you need to go to work [14:14:36] yeah :( [14:15:03] or you know, stop fixing that decades old car that consumes lot of oil and use a bicycle instead :-} [14:15:22] so [14:15:42] chasemp: looks like the issue comes from translatewiki they use the tip of master branches [14:15:50] timo has a few patches [14:15:57] and I guess it does not affect prod yet [14:18:58] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.11 deployment blockers - https://phabricator.wikimedia.org/T155526#2996351 (10hashar) [14:19:05] ok, I didn't tinker with any priorities as I can verify at least the en.wikipedia feed is still working [14:19:24] so I figured something like that but wanted to highlight since it was noticed overnight [14:19:26] chasemp: so yeah that is "just" translatewiki.net [14:19:33] which runs master branches on prod ahead of us [14:19:34] kind of a beta cluster [14:19:47] I have made both the tasks you mentioned blockers to the next week train ( https://phabricator.wikimedia.org/T155526 ) [14:19:53] and Timo is aware/fixing them [14:20:05] sounds good [14:20:10] given Timo also provides supports to volunteers gadgets and tools using irc.wikimedia.org / vandalism [14:20:21] I am sure he will make a point to not have that pushed to Wikimedia prod :} [14:20:30] yep all faith in timo [14:20:34] yeah [14:20:46] last time we broke IRC feed (a few years ago), I reviewed his patches [14:20:56] that was like 3 or 4 years ago and it was a major sisue [14:20:57] issue [14:21:17] not so much for prod itself but that prevented several tools from the community. Ideally folks should move to RCStream [14:21:27] I mined the old non-puppet server for package setup and puppetization as my first project here [14:21:33] and it uh, was eye opening [14:21:34] but lot of those tools are abandonware, and I believe at least one there are no more source code for it [14:21:47] hehe [14:22:03] yeah the migration to puppet has been a long trip for ops [14:22:06] s/trip/journey/ [14:25:13] chasemp: wanna do the nodepool images renaming today? :) [14:25:31] sure, what do you need from us to do that? [14:25:42] some time for me to figure out the nodepool config [14:25:53] a review/merge of the yaml crap + puppet run [14:26:09] probably gotta stop nodepool and rename the images manually (I can do via the openstack CLI) [14:26:40] do you mean nodepool cli? but sure ping me when you have things ready but I can't provide much contextual review [14:27:08] the openstack cli [14:27:11] to rename the base images [14:27:46] I push to glance some base images that are named ci-trusty-wikimedia and ci-jessie-wikimedia [14:27:48] gotta rename those [14:27:49] ah, there is an actual 'openstack' cli tool so I was trying to clarify, I assume you are using the 'nodepool' cli tool [14:27:53] and rename the snapshots [14:28:17] 'nodepool' only interacts with the nodepool mysql database / the state known to nodepool [14:28:27] so eg: nodepool delete 1234 [14:28:39] makes nodepool to flag instance id=1234 to state=delete [14:28:44] got it, you do mean the openstack cli [14:28:53] and eventually at some point nodepool daemon will do the actual openstack API request to delete the server [14:29:03] yup [14:49:33] chasemp: https://gerrit.wikimedia.org/r/#/c/335809/ would do it I believe [14:49:41] but gotta think about it a bit for a few minutes :-} [14:55:35] hashar: ok, gotta detour to pickup a kid from school last minute so take your time :) [15:12:16] (03PS1) 10Hashar: Make some puppet submodule rake jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/335815 (https://phabricator.wikimedia.org/T154894) [15:19:40] Hey folks. I'm struggling with an error that I get from scap3. " xrange() arg 3 must not be zero" in targets.py. [15:19:54] Is this a common enough error that someone can tell me offhand what the likely problem is? [15:20:01] * halfak makes a proper paste [15:21:03] http://pastebin.ca/3763274 [15:24:15] (03PS2) 10Hashar: Make some puppet submodule rake jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/335815 (https://phabricator.wikimedia.org/T154894) [15:25:01] (03CR) 10Hashar: [C: 032] Make some puppet submodule rake jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/335815 (https://phabricator.wikimedia.org/T154894) (owner: 10Hashar) [15:25:57] (03Merged) 10jenkins-bot: Make some puppet submodule rake jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/335815 (https://phabricator.wikimedia.org/T154894) (owner: 10Hashar) [15:27:19] It looks like the error actuall happens after "20:30:20 Finished deploy [ores/deploy@9fd75a1] (duration: 00m 07s)" [15:36:57] 10Continuous-Integration-Config, 13Patch-For-Review: Phase out jobs "pplint-HEAD" and "erblint-HEAD" - https://phabricator.wikimedia.org/T154894#2996591 (10hashar) In Gerrit I have made the operations/puppet/.* sub repos to inherit rights from a new repository operations/puppet-modules. Then I have granted ri... [15:37:17] dapatrick, I see that you've been resolving repo requests recently. Can you help me with https://phabricator.wikimedia.org/T157049 ? [15:43:25] halfak: please please fill it as a task! [15:43:37] hashar, the error? Sure [15:44:03] halfak: yup and you can land the paste on http://phabricator.wikimedia.org/paste/ [15:44:08] then inline it in the task with something like: {P12345} :-} [15:44:17] hashar, which tags should I apply? [15:44:33] I would add : #scap #deployment-systems #releng [15:44:46] that should send mail notifications to every single person on the plante [15:44:50] plante :-} [15:44:51] planet [15:44:58] damn spell checker [15:45:10] we have updated scap recently i think [15:46:27] 10Deployment-Systems, 06Release-Engineering-Team, 03Scap3, 10scap: Error after "Finished deploy": xrange() arg 3 must not be zero - https://phabricator.wikimedia.org/T157136#2996635 (10Halfak) [15:46:40] Cool. thanks hashar [15:46:53] 10Deployment-Systems, 06Release-Engineering-Team, 03Scap3, 10scap, and 2 others: Error after "Finished deploy": xrange() arg 3 must not be zero - https://phabricator.wikimedia.org/T157136#2996649 (10Halfak) [15:46:55] halfak: I guess something is off in the scap/scap.cfg [15:47:06] or scap no more properly process the group/subgroups [15:47:58] Ahh. I'll get a link to our scap config for the task. [15:49:04] 10Deployment-Systems, 06Release-Engineering-Team, 03Scap3, 10scap, and 2 others: Error after "Finished deploy": xrange() arg 3 must not be zero - https://phabricator.wikimedia.org/T157136#2996658 (10Halfak) Relevant scap config: https://phabricator.wikimedia.org/diffusion/1880/browse/master/scap/ [15:50:40] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996671 (10zeljkofilipin) Looks like the tests are failing randomly :( | Build | Test | Error | Sauce Labs | |--|--|--|--| | 258 | Edi... [15:50:41] halfak: seems it deploys first to the "worker" group [15:50:53] ores-worker-labs: deployment-sca03.deployment-prep.eqiad.wmflabs [15:51:03] then try to deploy on the 'default' group [15:51:11] which is the same file [15:51:25] so maybe we did an optimization that filter out host that already got deployed [15:51:44] so when deploying the "worker" group, deployment-sca03 is done [15:51:52] then to deploy the rest, scap check the default group [15:52:03] get a list of host that is only [ 'deployment-sca03' ] [15:52:11] that host is filtered out because it got deployed [15:52:16] we end up with an empty list to deploy to [15:52:20] and xrange() bork [15:52:40] hashar, aha! So we could just do a quick check for len() and not deploy [15:52:49] yeh [15:52:51] something like: [15:52:54] Project selenium-MobileFrontend » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #313: 04FAILURE in 30 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/313/ [15:52:58] xrange should be able to just not iterate with a range of zero :\ [15:53:07] "all hosts from this group have all been deployed by previous groups" or something like that [15:53:14] and in your scap.cfg [15:53:18] if you have a single group [15:53:23] I guess you can drop part of the config [15:53:24] oh. no it's the step size [15:53:51] I haven't read the code, just guessing :-} feel free to copy paste some part of above on the task [15:53:56] and the scap guru will show up soon [15:54:15] * hashar grab some fresh water / take a short break etc [15:55:46] Thanks hashar [15:55:58] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996691 (10zeljkofilipin) Sauce Labs error is: - Internal Server Error - The Sauce VMs failed to start the browser or device. For mor... [15:58:27] 10Gerrit, 06Release-Engineering-Team, 06Operations: setup/install gerrit2001/WMF6408 - https://phabricator.wikimedia.org/T152525#2996697 (10demon) >>! In T152525#2994909, @RobH wrote: > Assigning this task to Chad. Once he is aware that this system is all theirs, he can resolve. Confirmed. >>! In T152525#... [16:02:09] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996700 (10zeljkofilipin) [16:02:47] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2864860 (10zeljkofilipin) [16:03:54] 10Deployment-Systems, 06Release-Engineering-Team, 03Scap3, 10scap, and 2 others: Error after "Finished deploy": xrange() arg 3 must not be zero - https://phabricator.wikimedia.org/T157136#2996635 (10hashar) (Time is UTC+1) ``` lang=irc [16:50:41] <+hashar> halfak: seems it deploys first to th... [16:04:39] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996707 (10zeljkofilipin) [16:04:52] halfak, hashar: Ouch. Thanks for filing that one [16:13:19] :) [16:16:36] (03CR) 10Alexia: "> Abandoned" [integration/config] - 10https://gerrit.wikimedia.org/r/332002 (owner: 10Paladox) [16:18:47] ostriches, I'm looking to use git-fat in wmflabs and prod. Is the archiva instance something that ya'll maintain? [16:20:26] Archiva? No, we don't touch/maintain that at all [16:20:29] git-fat we do [16:21:12] Oh. What repo should I use for git-fat and how do I set it up? [16:21:23] Been reading https://wikitech.wikimedia.org/wiki/Archiva [16:21:28] But I guess that's not right? [16:27:52] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996764 (10zeljkofilipin) [16:28:45] halfak: Well you configure it in your scap config (git_fat: true) [16:28:56] Beyond that...not 100% sure.... I know we've got some repos using it already [16:30:19] I imagine that I'll need to put some config lines in my git fat config. Any docs you know about? [16:30:19] Ah, yes... .gitattributes [16:30:21] https://wikitech.wikimedia.org/wiki/Archiva#Setting_up_git-fat_for_your_project [16:30:30] ^ specifies using archiva [16:30:38] remote = archiva.wikimedia.org::archiva/git-fat [16:30:59] I think it'll work with basically any rsync target that your deploy targets can hit... [16:31:09] But I haven't used it in awhile [16:31:41] Gotcha. Would this be accessible from WMFLabs? I'm guessing not. [16:31:59] rsync to a production service like archiva? Probably not [16:32:18] Yeah. So basically then we need to find some other way to deploy to beta labs [16:32:23] and our experimental labs instance [16:32:41] Or just keep putting our massive files in a repo. [16:32:56] Massive binaries in git makes me a sad panda :( [16:33:03] So let's find a way to make sure git-fat will work from labs :D [16:33:11] Two options I see.... [16:33:13] ostriches, indeed. me too. I wish we had better options. [16:33:27] Eh, 3 options actually... [16:33:42] git is starting to complain about our massive file sizes https://phabricator.wikimedia.org/T157141 [16:33:44] 1) Make archiva accessible over anon rsync from labs (less likely, would need ops involvement) [16:34:05] 2) Setup an archiva mirror in labs for testing things like this (not a terrible idea) [16:34:14] 3) Find somewhere other than archiva to rsync from [16:34:34] 3.5) Differ the target when testing in beta? That'd be lame for maintenance.... [16:34:34] ostriches, can't have a repo point to multiple archivas, so (1) would be better than (2). [16:35:03] I'd like our volunteer devs to be able to submit PRs that affect large files. [16:35:38] Labs isn't just our testing environment. We run a service with an SLA in labs too. [16:35:58] * ostriches nods [16:37:36] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996812 (10zeljkofilipin) [16:41:35] ostriches, seems like the best bet in the short term is to just get diffusion to stop 'sploding whenever we try to clone/pull from it. [16:41:47] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996822 (10zeljkofilipin) [16:41:58] halfak: Not an easy lift either... [16:42:08] I think we can get git-fat working for you easy enough [16:46:26] (03PS1) 10Umherirrender: Make some unit test voting [integration/config] - 10https://gerrit.wikimedia.org/r/335827 [16:47:58] 10Browser-Tests-Infrastructure, 10Wikidata, 15User-Tobi_WMDE_SW, 15User-zeljkofilipin: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#2996886 (10zeljkofilipin) I have reported the problem to Sauce Labs support. [16:52:24] (03CR) 10Hashar: [C: 032] "It is a kind of magic!! ™" [integration/config] - 10https://gerrit.wikimedia.org/r/335827 (owner: 10Umherirrender) [16:54:50] (03Merged) 10jenkins-bot: Make some unit test voting [integration/config] - 10https://gerrit.wikimedia.org/r/335827 (owner: 10Umherirrender) [16:55:37] ostriches, I figured that diffusion is just choking on a large http transfer. [16:55:49] (03CR) 10Hashar: [C: 032] "Quite great thank you very much :-}" [integration/config] - 10https://gerrit.wikimedia.org/r/335827 (owner: 10Umherirrender) [16:55:58] Maybe I can just switch to ssh-style clone/pull [17:02:45] halfak: That could work [17:02:49] Another option is a shallow clone, then un-shallow it [17:03:28] `git clone --depth 1 ` [17:03:31] Then [17:03:32] Hmm... I wonder if I could do a shallow `git submodule update` [17:03:33] `git fetch --unshallow` [17:04:00] Hmmm, dunno if --depth propgates to submodules. [17:04:02] I doubt it [17:04:22] * halfak starts working on ssh-based solution [17:04:45] Looks like you can do shallow stuff on submodules.... in git 2.10+ [17:05:30] Actually, --depth looks like it works for submodules going way back [17:05:45] Ooh. Interesting. That could be a great option for this. [17:06:04] `git config -f .gitmodules submodule..shallow true` is what's new in 2.10+ [17:06:49] 2.1.4 on deployment tin [17:07:01] Yep, --depth exists for both submodule add and submodule update on our deployed git version [17:07:01] Looks like I'll be manually editing the configs [17:07:03] Which is Yeah, 2.1.x [17:07:17] Maybe there's something in scap that I can use for this. [17:07:19] So you'd want to clone the repo, then `git submodule update --depth=1` [17:07:35] You don't need the gitconfig bit, you can do it just with --depth :D [17:08:20] (Sidebar: I'm working with ops to get git 2.11.x available in apt, 2.1.x is *ancient*) [17:09:08] ostriches, will try. [17:09:27] ostriches, doesn't scap use git though? Wouldn't scap also know to do a shallow pull/update? [17:10:34] It doesn't right now. But it could be taught to do so :) [17:11:35] Support for shallow clones is actually a really good idea in scap...i should file a task for that [17:15:24] 06Release-Engineering-Team, 03Scap3: Support shallow clones - https://phabricator.wikimedia.org/T157149#2996963 (10demon) [17:15:33] 06Release-Engineering-Team, 03Scap3: Support shallow clones - https://phabricator.wikimedia.org/T157149#2996976 (10demon) p:05Triage>03Normal [18:18:33] 06Release-Engineering-Team, 06Discovery, 06Discovery-Search, 10Elasticsearch, 10Phabricator (Search): phab+elasticsearch: support multiple elasticsearch clusters / datacenters - https://phabricator.wikimedia.org/T157156#2997117 (10demon) [18:19:40] 06Release-Engineering-Team, 03Scap3: Support shallow clones - https://phabricator.wikimedia.org/T157149#2997121 (10demon) The motivation behind this was {T157141} [18:22:28] 06Release-Engineering-Team, 06Discovery, 06Discovery-Search, 10Elasticsearch, 10Phabricator (Search): phab+elasticsearch: support multiple elasticsearch clusters / datacenters - https://phabricator.wikimedia.org/T157156#2997124 (10mmodell) [19:34:49] halfak, I just did the one for OIT as part of a security review I'm working on this week. I don't generally do stuff with Diffusion repos. [19:34:55] Unless it's an emergency, I suppose. [19:35:04] I'm still just getting up to speed on diffusion myself. [19:36:23] Who would I ask for repo admin rights? It's a pain to wait for these repos to be created. Usually when I'm in a position to complain I try to ask for a mop instead. (See my admin rights and a few wikis and my status as an OAuth approver) [19:36:25] :) [19:36:48] Is there a phab repo master? [19:37:08] halfak qchris [19:38:45] gerrit repos are requested on wiki, but phab repos are requested in phab itself.. is that right? [19:39:59] eh.. gerrit project / diffusion repo [19:40:20] mutante not really, all is done on the wiki [19:40:36] * halfak tries to learn who qchris is and where I could find them [19:40:37] there is no actual process yet to do it as a phab task yet. [19:40:46] https://wikitech.wikimedia.org/wiki/User:QChris "Nothing to see here" [19:41:12] halfak manly you can find him in #wikimedia-devtools [19:41:20] https://meta.wikimedia.org/wiki/User:QChris_(WMF) [19:41:21] his nick will probaly be qchris_ [19:41:31] Aha! I know Christian. [19:41:45] yep [19:42:17] paladox: ok, gotcha. i just thought that because i saw a ticket requesting one in diffusion [19:42:42] Yep, you can do it both ways, but current offical way is the wiki. [19:43:00] https://meta.wikimedia.org/wiki/User:QChrisNonWMF [19:43:34] paladox: alright [19:43:39] ;) [19:49:23] * halfak runs to lunch. [19:49:26] Thanks for the help [19:49:30] fyi: https://phabricator.wikimedia.org/T157163 [19:49:36] your welcome [19:56:04] halfak: What backlog? ;-) [19:58:39] halfak: https://phabricator.wikimedia.org/T157163#2997390 :D [20:01:18] lol [20:24:56] ostriches there's now a git called GVFS [20:24:57] https://github.com/Microsoft/gvfs [20:25:02] Already saw it [20:25:06] Windows only *shrug* [20:25:12] So useless! [20:25:41] lol, you could always convert it into *nix compatible [20:27:28] Yeah, with all the copious free time I have :P [20:30:59] Woo! [20:31:24] ostriches, any guidelines you'd like me to review? [20:31:45] ostriches, they should just update the git source code to enable this, now that's a function i would use. Save downloading everything. [20:32:10] halfak: https://www.mediawiki.org/wiki/Phabricator/Diffusion covers some high-level stuff, but is woefully empty [20:32:17] If you have questions, please ask :) [20:34:04] * halfak reads [20:34:13] Looks pretty straightforward. Thanks :) [20:37:02] somebody making a git-related software should call it "igitt". it'd be funny to Germans https://en.wiktionary.org/wiki/igitt [20:43:56] https://phabricator.wikimedia.org/H221 [20:54:56] 06Release-Engineering-Team, 03Scap3: Support shallow clones - https://phabricator.wikimedia.org/T157149#2996963 (10hashar) Still take a while though: ``` $ time git clone --depth 1 --recurse --shallow-submodules https://phabricator.wikimedia.org/diffusion/1913/research-ores-editquality.git Cloning into 'resear... [20:55:59] 06Release-Engineering-Team, 03Scap3: Support shallow clones - https://phabricator.wikimedia.org/T157149#2997523 (10demon) Yes, using git-fat is a good idea and we're going to do it in this specific case (see the task I mentioned), but generally speaking it's not a bad idea. [21:48:58] Yippee, build fixed! [21:48:59] Project selenium-RelatedArticles » chrome,beta-mobile,Linux,contintLabsSlave && UbuntuTrusty build #298: 09FIXED in 42 sec: https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-mobile,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/298/ [22:12:31] twentyafterfour: fyi when i try to upload a meme I'm getting Unhandled Exception ("PhutilInvalidStateException") which is causing me much distress [22:12:51] hmm [22:13:08] jdlrobson: no stack trace? [22:13:22] Just Call setViewer() before calling getViewer()! [22:13:27] not very helpful [22:13:33] not sure if it's the image or something more sinister [22:36:51] jdlrobson: cannot reproduce, currently it seems to work for me? [22:37:16] should i try logging in and out again? [22:37:40] that might work... or maybe the macro it's based on is corrupt? [22:37:50] not sure [22:37:57] memes, the most important part of work [22:37:58] still getting the same.. [22:38:00] :) [22:38:02] :) [22:38:15] https://usercontent.irccloud-cdn.com/file/Gwkvkchj/i3lbNZhnB1Jle.gif [22:38:17] This is the gif i'm trying to use fyi [22:38:51] if you're trying to use the url to upload it then that's probably the problem [22:39:00] outbound connections are firewalled [22:39:18] yea, try downloading it and then drag/drop to upload [22:43:04] security, always hampering our fun [22:43:11] * greg-g helps [22:45:30] but.. if we'd set the webproxy... [22:45:34] then it would work [22:45:37] twentyafterfour: nope uploading it.. [22:45:56] am using https://phabricator.wikimedia.org/macro/create/ [22:46:11] let me try to do the same thing [22:46:20] i just made one not that long ago [22:46:34] mutante: greg-g twentyafterfour it just doesnt like me :/ [22:47:02] https://phabricator.wikimedia.org/macro/view/48/ [22:47:05] works for me [22:47:23] maybe just in one browser. i used FF [22:48:33] using chrome [22:48:39] trying FF now [22:50:10] Nope doesnt work there either [22:50:17] account specific? Do I need permissions or something? [22:51:14] i am not a phab admin or anything [22:51:34] unless it's tied to operations, wouldnt think so [22:51:53] although its not behaving like a gif.. [22:53:28] it plays once, just does not loop [22:54:06] what did you enter as "name" [22:54:16] maybe something illegal there? [22:54:24] i just put "hulk" [22:54:42] and URL field empty [22:56:30] yeh i dunno.. i can upload files normally.. just that form [22:56:35] not a big deal [23:09:53] twentyafterfour i get the same error [23:10:01] Unhandled Exception ("PhutilInvalidStateException") [23:10:01] Call setViewer() before calling getViewer()! [23:10:35] twentyafterfour https://phabricator.wikimedia.org/F5470256 [23:11:32] i have a trace back [23:11:37] reproducable on phab-01 [23:11:55] so it is this specific image? [23:11:59] twentyafterfour https://phabricator.wikimedia.org/P4883 [23:12:18] twentyafterfour, nope, i just uploaded a screenshot of a white spefic part of my screen. [23:12:46] Thanks paladox for replicating!! :) [23:13:01] Your welcome, i also have logs :) [23:15:03] twentyafterfour the change was https://github.com/wikimedia/phabricator/commit/18a9fe85c8359710b619c8b8de47ea559ffafafc but that's over two years old, so maybe an update somwhere in phab? [23:16:03] well something else is happening to prevent viewer from being populated [23:16:12] i uploaded the same image [23:20:21] twentyafterfour, i think a setViewer needs to be added here https://secure.phabricator.com/source/phabricator/browse/master/src/applications/macro/controller/PhabricatorMacroEditController.php;ab923e0a752b9dc03a850be74d198832a5620373$187 [23:20:34] see https://github.com/wikimedia/phabricator/blob/3d4b39a62200e3c148eb144fefd48399f62f315d/src/applications/files/markup/PhabricatorEmbedFileRemarkupRule.php#L287 [23:22:38] yay [23:22:40] fixed it [23:22:47] twentyafterfour jdlrobson ^^ [23:22:51] patch incomming [23:22:55] https://phab-01.wmflabs.org/macro/view/1/ [23:23:08] mutante ^^ [23:24:30] paladox: cool! any idea why it just happens to some ? [23:24:59] looks like that branch only happens in certain circumstances [23:25:01] twentyafterfour mutante https://phabricator.wikimedia.org/D553 [23:25:05] * twentyafterfour hasn't figured out what condition causes it [23:25:07] jdlrobson ^^ [23:25:27] that fixes it on phab-01 anyways :) [23:27:04] lol ^^ [23:27:43] 10Gerrit, 10Developer-Wishlist (2017), 07Documentation: Enable and document "WIP" workflow status in Gerrit - https://phabricator.wikimedia.org/T135245#2292962 (10srishakatux) This proposal is selected for the developer wishlist voting round and will be added to a MediaWiki page very soon. To the subscribers... [23:27:51] 10Gerrit, 06Developer-Relations, 10Developer-Wishlist (2017): Add a welcome bot to Gerrit for first time contributors - https://phabricator.wikimedia.org/T73357#745463 (10srishakatux) This proposal is selected for the developer wishlist voting round and will be added to a MediaWiki page very soon. To the sub... [23:27:54] 10Gerrit, 06Developer-Relations, 10Developer-Wishlist (2017): Implement a way to bring GitHub pull requests into gerrit - https://phabricator.wikimedia.org/T37497#398624 (10srishakatux) This proposal is selected for the developer wishlist voting round and will be added to a MediaWiki page very soon. To the s... [23:32:18] 10Gerrit, 06Developer-Relations, 10Developer-Wishlist (2017): Add a welcome bot to Gerrit for first time contributors - https://phabricator.wikimedia.org/T73357#2997879 (10demon) >>! In T73357#2627012, @Aklapper wrote: > @demon: Any technical input? Yes! >>! In T73357#2586429, @mmodell wrote: > I don't see... [23:32:27] 10Deployment-Systems, 06Release-Engineering-Team, 06MediaWiki-Stakeholders-Group, 10MediaWiki-extensions-General-or-Unknown, 10Developer-Wishlist (2017): Improve LTS support of extensions - https://phabricator.wikimedia.org/T156640#2982107 (10srishakatux) This proposal is selected for the #developer-wish... [23:33:55] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Differential, 10Developer-Wishlist (2017), 07Jenkins: Add support for a wmf-ci.yaml type file for wikimedia jenkins - https://phabricator.wikimedia.org/T145669#2637873 (10srishakatux) This proposal is selected for the #developer-... [23:33:58] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Developer-Wishlist (2017), 13Patch-For-Review: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885483 (10srishakatux) This proposal is selected for the #developer-wishlist voting rou... [23:35:23] 10Continuous-Integration-Infrastructure, 10Developer-Wishlist (2017), 07Technical-Debt: Relocate CI generated docs and coverage reports - https://phabricator.wikimedia.org/T137890#2382711 (10srishakatux) This project is selected for the #developer-wishlist voting round and will be added to a MediaWiki page v... [23:35:40] 10Gerrit, 10Developer-Wishlist (2017): Enable Gerrit reviewers-by-blame plugin - https://phabricator.wikimedia.org/T101131#1330591 (10srishakatux) This project is selected for the #developer-wishlist voting round and will be added to a MediaWiki page very soon. To the subscribers, or proposer of this task: ple... [23:35:51] 10Gerrit, 10Developer-Wishlist (2017): [MediaWiki-commits] Reverts are not notified by gerrit - https://phabricator.wikimedia.org/T49252#528983 (10srishakatux) This project is selected for the #developer-wishlist voting round and will be added to a MediaWiki page very soon. To the subscribers, or proposer of t... [23:35:53] 10Gerrit, 10Developer-Wishlist (2017), 07Upstream: Announce all creations, deletions and renaming of gerrit repos (for e.g. translatewiki.net workflow) - https://phabricator.wikimedia.org/T48982#492851 (10srishakatux) This project is selected for the #developer-wishlist voting round and will be added to a Me... [23:35:55] 10Gerrit, 10Developer-Wishlist (2017), 07Upstream: Free-form tagging in gerrit - https://phabricator.wikimedia.org/T37534#403845 (10srishakatux) This project is selected for the #developer-wishlist voting round and will be added to a MediaWiki page very soon. To the subscribers, or proposer of this task: ple... [23:54:51] 10Gerrit, 05Security, 15User-greg: Gerrit spam from GuerellaNuke23 - https://phabricator.wikimedia.org/T157083#2997986 (10greg) [23:54:55] 10Gerrit, 05Security, 15User-greg: Gerrit spam from GuerellaNuke23 - https://phabricator.wikimedia.org/T157083#2997988 (10greg)