[15:30:13] ejegg: any idea what these fail mails about /srv/dashdata are? [15:30:45] i know jeff tweaked a puppet thing yesterday re: samarium [15:30:51] cwd shoot, no [15:31:46] ejegg: they are coming from civi1001, shouldn't this be frdev only? [15:31:54] if it is dash related [15:33:26] right, it's not frdev if it's coning from a p-c job [15:33:42] since those are all civi [15:33:58] ah so does civi just push the files around? [15:35:28] ah i think i see the problem [15:39:14] ejegg: we have a new server to replace samarium, that is the one complaining [15:39:17] frdata1001 [15:42:57] hmmm [15:44:22] i oh [15:44:27] s/i // [15:45:09] there is a little problem [15:45:41] Jeff_Green: there is a p-c job that is going to bomb as long as there is nothing in /srv/dashdata on frdata1001 [15:47:19] wat [15:47:26] looking [15:48:56] Jeff_Green: if you look at the p-c job: fundraiser_public_data_mover [15:49:16] scp fails if there is nothing to move [15:49:21] ya, I'm just perplexed about where the /srv/dashdata came from in the first place [15:49:38] oh, yeah i'm not sure [15:49:53] ah ha [15:49:54] i think it gets deployed to by fr-tech [15:50:11] this is the wackiest scheme to sync data [15:50:19] :) [15:51:26] ok i see what needs the happening, thanks for catching [15:53:32] great [15:55:21] the thing I don't understand is why /srv/dashdata exists at all on samarium [15:56:06] looking [15:56:12] i noticed it in the frdeploy module [15:56:41] it looks like it's a project that can be deployed to [15:57:10] maybe if we do an f_c_u --all [15:57:21] pseudocommand [15:57:39] err and add frdata1001 to the project [16:00:57] if we deploy it instead to civi1001, there's no need to do that first rsync to fetch it from a box in the DMZ [16:01:26] yeah i don't understand why that is the destination [16:01:29] pub reporting hosts [16:01:54] i'm gonna adjust that [16:01:58] ejegg might be able to shine a light [16:03:39] i'm kinda puzzled by that dashdata name too [16:04:04] civi seems like a more sensible place for it [16:04:10] also tools... [16:04:35] maybe that's for the IRC bot [16:05:42] that sounds right [16:36:31] huh. the bot doesn't appear to use the tools project at all, I don't see any reason we need to deploy it to the public reporting box [16:37:12] this box should be reduced to a raspberry pi :-) [16:44:16] I'm going to remove samarium (and its replacement frdata1001) from deployment altogether, it's only doing flat HTML and the bot from a deb package [16:55:41] Jeff_Green: excellent [16:55:57] nice way to knock some rust loose [16:58:45] where did you notice it complaining before? [17:14:50] ah HA. I think found where it used to need the tools library [17:15:08] Jeff_Green: fail mails [17:15:25] from p-c? I must have filtered those totally away :-( [17:15:50] Jeff_Green: you must have edited the job in localsettings repo? [17:15:57] yeah I did [17:16:08] cool [17:16:28] yeah that one was sending a mail [17:16:32] ok [17:17:21] yeah that fail mail is usually not relevant to us [17:17:23] usually [17:18:42] there used to be a banner snapshot job that ran on samarium, trying to remember the history [17:20:01] we removed it 3/2017 [17:24:10] and that's where that data came from? [17:24:14] dashdata? [17:24:31] it's why we were ever deploying /srv/tools [17:26:44] ooh right [18:50:30] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi: enable Force Merge Selected Duplicates for new DS-Admin permissions level - https://phabricator.wikimedia.org/T193674#4267889 (MBeat33) @E... [18:58:07] fundraising-tech-ops, Operations, netops: adjust NAT mapping for frdata.wikimedia.org - https://phabricator.wikimedia.org/T196656#4267897 (Jgreen) [20:42:27] PROBLEM - check_procs on frdb1001 is CRITICAL: PROCS CRITICAL: 1283 processes [20:44:46] looks like dedupe [20:47:27] RECOVERY - check_procs on frdb1001 is OK: PROCS OK: 309 processes [20:57:50] is namely super slow for anyone else? [20:57:57] Or is it just the connection here? [21:13:27] oh hey, reviews were almost two months later last year. spose i should make sure none of my examples are from last june or july [21:30:42] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi: enable Force Merge Selected Duplicates for new DS-Admin permissions level - https://phabricator.wikimedia.org/T193674#4268287 (Eileenmcnaug... [21:33:01] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi: enable Force Merge Selected Duplicates for new DS-Admin permissions level - https://phabricator.wikimedia.org/T193674#4268288 (MBeat33) Th... [21:33:59] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi: enable Force Merge Selected Duplicates for new DS-Admin permissions level - https://phabricator.wikimedia.org/T193674#4268289 (Eileenmcnaug... [21:41:12] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi: enable Force Merge Selected Duplicates for new DS-Admin permissions level - https://phabricator.wikimedia.org/T193674#4268310 (MBeat33) I... [21:47:27] PROBLEM - check_procs on frdb1001 is CRITICAL: PROCS CRITICAL: 1345 processes [21:48:42] sup eileen [21:49:08] logging in on the weekend eh >:| [22:07:27] RECOVERY - check_procs on frdb1001 is OK: PROCS OK: 347 processes [23:36:54] have a great weekend, all! [23:47:23] PROBLEM - check_procs on frdb1001 is CRITICAL: PROCS CRITICAL: 1196 processes [23:57:23] PROBLEM - check_procs on frdb1001 is CRITICAL: PROCS CRITICAL: 1050 processes