[10:19:58] akosiaris: hey, around? [10:20:38] Amir1: yup [10:21:44] akosiaris: awesome, this my progress wrt scap3 https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service/Work_log/2016-03-25#Scap3_puppet_settings_for_ORES [10:22:27] ok, that looks promising [10:23:15] akosiaris: I need you to decide on user [10:23:33] which user we should choose and what would it be its credentials [10:23:38] hmm, that one I need to figure out [10:24:29] it's my first software to deploy with scap3 as well [10:26:35] akosiaris: in ores in labs a user name is used called www-data [10:27:25] and also the new scap3 is in https://github.com/wiki-ai/ores-wikimedia-config/tree/scap_again/scap [10:28:50] one of the things releng told me that they need two directories in tin for example we need to go to /srv/deployment create a directory called ores and clone ores-wikimedia-config in it [10:29:10] so /srv/deployment/ores/ores-wikimedia-config/... [10:29:17] otherwise it doesn't work [10:32:45] you mean the 2 level deep directory for the deployment ? yes that is true for both scap3 and trebuchet [10:33:29] akosiaris: yes [10:33:48] and since we will need the repo anyway to be in phabricator, it might also be good to name it something with "deploy" in the name [10:34:10] like ores/deploy (that is the defacto standard up to now) [10:34:19] +1 [10:34:24] cool [10:37:17] akosiaris: do you want to make a mirror? [10:37:41] yup, I 'll ping chad about that [10:39:43] awesome [11:55:08] Im looking to Build friendships and share knowledge. [11:55:16] Are there any active members? [12:00:07] (03CR) 10Ladsgroup: [C: 032] Let user make hidenondamaging default [extensions/ORES] - 10https://gerrit.wikimedia.org/r/278839 (https://phabricator.wikimedia.org/T130560) (owner: 10Ladsgroup) [12:01:04] (03Merged) 10jenkins-bot: Let user make hidenondamaging default [extensions/ORES] - 10https://gerrit.wikimedia.org/r/278839 (https://phabricator.wikimedia.org/T130560) (owner: 10Ladsgroup) [12:58:22] akosiaris: I wanted to make this patch but I realized you did it sooner: https://gerrit.wikimedia.org/r/#/c/278950/1 [12:58:37] do you want to merge it? [13:01:02] o/ [13:01:15] I'm actually just about to pack up my laptop and head to the University. [13:01:22] But I thought I'd wave hello anyway :) [13:02:18] o/ [13:02:26] halfak: hey [14:15:24] back again [14:15:28] Hacking on article quality models. [14:15:44] Got a new idea for label extraction from nettrom and am testing now. [14:30:11] And I messed up somewhere https://meta.wikimedia.org/wiki/Research_talk:Automated_classification_of_article_quality/Work_log/2016-03-25 [14:47:52] halfak: hey, what do you think of the work log [14:48:19] Oh yeah! Let me look again. I didn't have the mental capacity to consider it last time. [14:49:15] we need some patches merged for puppet, akosiaris made them but not merged yet [14:49:36] specifically this one: https://gerrit.wikimedia.org/r/#/c/278950/1 [14:49:39] and https://gerrit.wikimedia.org/r/#/c/278951/ [14:50:26] Amir1, seems like we should make a "ores-web-prod" role [14:50:33] Or maybe just have a "prod" role [14:50:38] That will contain this scap stuff./ [14:51:02] halfak: like this ? https://gerrit.wikimedia.org/r/#/c/278989/ ? [14:51:03] +1 [14:51:46] so, I 'll merge my other 2 puppet patches now. There is one more coming to clean up ores::base from the git::clone resource [14:52:23] akosiaris, I note that these roles are *not* in a labs folder -- that implies they are for prod setup, right? [14:52:48] So we can have the roles define which contexts we'll use scap in, right [14:52:49] ? [14:53:23] yup [14:53:58] kk +1 [14:54:57] akosiaris, how will we manage the difference between beta and production? [14:55:20] E.g. Amir1 is setting the "deployment_server" variable in puppet. [14:55:20] beta and production should have pretty much 0 difference [14:55:35] the few configuration variables will be set per environment in hiera [14:55:43] Gotcha. We should expect deployment-tin.deployment-prep.eqiad.wmflabs to be available everywhere? [14:55:56] look under hieradata/labs/deployment-prep [14:56:26] it has configuration values for that specific labs project (also known as beta) [14:56:35] akosiaris: the problem I ran into yesterday wast that even it loads the hiera, it couldn't set th edeployment_server properly [14:56:55] Amir1: link to change ? [14:57:20] FYI: https://github.com/wikimedia/operations-puppet/blob/production/hieradata/labs/deployment-prep/common.yaml#L141 [14:57:26] The variable in question, I assume [14:57:49] akosiaris: https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service/Work_log/2016-03-25 [14:58:26] but I must say I copied https://wikitech.wikimedia.org/wiki/Hiera:Deployment-prep into /etc/puppet/hiera.yaml [14:58:42] which is why it did not work [14:59:00] that file, despite the .yaml extension is actually configuration [14:59:10] it had scap::deploymenet_server set [14:59:11] it tells hiera where to look for to find data [14:59:24] it is not used to actually *find* data [14:59:58] data == configuration variables in this context btw, if that wasn't already clear [15:00:16] okay [15:00:21] It's your call [15:00:24] :) [15:00:24] also, the lookup key matters. I am not sure what exactly was not working [15:00:53] this one ? class {'scap': deployment_server => 'deployment-tin.deployment-prep.eqiad.wmflabs', }? [15:01:23] for that to work you need a scap::deployment_server key, not a "role::deployment::salt_masters::deployment_server" key [15:02:44] but I am happy we are getting closer to deploying this with scap. Those issues are rather minor AFAIC [15:05:34] awesome [15:07:03] It works with the key set [15:07:03] but without it ("include scap") it doesn't set the key properly [15:07:03] because it couldn't load the hiera data [15:07:03] I'm super new to puppet, so forgive me if my questions are dumb [15:14:38] so, when you do not define a parameter's value, it means it will be looked up implicitly in hiera [15:15:00] the key that will be looked up is "::" [15:16:14] and guess what... there is no scap::deployment_server key in deployment-prep right now [15:16:23] which is why it does not work [15:16:34] https://wikitech.wikimedia.org/wiki/Hiera:Deployment-prep [15:17:05] akosiaris: it has "scap::deployment_server" at the bottom [15:17:48] I checked that [15:18:30] ah, the onwiki hiera, hmm it should work then [15:19:34] and it didn't [15:19:48] I used debug and saw it loaded my hiera [15:20:05] maybe it was caching, or things like that [15:42:27] Amir1, https://meta.wikimedia.org/wiki/Grants:IdeaLab/Bot_to_search_for_and_recommend_infobox_and_wikidata_citations [15:42:51] Your thoughts and experience with Kian might be relevant here [15:43:19] sure [15:43:21] :) [16:53:20] akosiaris, halfak: I was able to deploy in web nodes using this puppet [17:02:30] https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service/Work_log/2016-03-25#Friday.2C_March_25.2C_2016 [19:38:18] halfak: o/ [19:38:24] o/ Amir1 [19:38:25] have you seen my notes? [19:40:46] Haven't had a chance to look yet [19:41:59] I was able to deploy with puppet \o/ [19:47:43] \o/. [19:47:58] So scap3 config gets onto the machines and then you can scap? [19:48:02] Amir1, ^ [19:48:35] when you apply to role scap (run the puppet) it deploys too [19:48:54] and you can deploy it again in tin [19:48:59] great :) [19:49:08] So that's basically done then, right? [19:49:12] Do you have a new scap PR? [19:49:45] I don't have PR but I have a branch ready [19:49:49] let me check [19:50:37] kk [19:50:49] Will review and merge when you are ready :) [19:52:45] sure [19:53:32] akosiaris: what would be final name of the repo? "ores/deploy"?