[02:31:49] Fundraising Sprint Value Subtracting, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-Logging, and 2 others: Some wmf_civicrm exceptions should be logged at info and not error - https://phabricator.wikimedia.org/T146063#2649382 (cwdent) How about that MISSING PREDECESSOR one? [16:41:44] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/322285 [16:41:57] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/322285 (owner: Ejegg) [16:42:09] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/322285 (owner: Ejegg) [16:46:33] !log disabled job 'Dedupe CiviCRM contacts' [16:46:53] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [16:48:23] !log disabled job 'Dedupe CiviCRM contacts (name-match)' [16:48:38] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [16:58:07] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Donation import falling behind - https://phabricator.wikimedia.org/T151068#2806499 (Ejegg) [16:58:43] Fundraising Sprint Value Subtracting, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Donation import falling behind - https://phabricator.wikimedia.org/T151068#2806511 (Ejegg) p:Triage>Unbreak! [17:40:52] (PS1) Ejegg: Clean up a couple unused things [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322291 [17:42:45] (PS2) Ejegg: Clean up a couple unused things [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322291 [17:53:15] (PS1) Ejegg: Expire old pending messages [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322294 [17:54:32] (CR) jenkins-bot: [V: -1] Expire old pending messages [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322294 (owner: Ejegg) [17:55:03] (PS2) Ejegg: Expire old pending messages [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322294 [17:56:12] cwd ok, that should clear out old pending messages. smoke tested OK locally [17:56:49] errr, one sec, lemme make sure that's not zapping the whole table [17:57:44] yah, ok, it's doing the right thing [18:00:04] (PS3) Ejegg: Expire old pending messages [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322294 [18:00:38] fr-tech: Cops never say good-bye. They're always hoping to see you again in the [18:00:39] line-up. [18:00:39] -- Raymond Chandler [18:00:39] -- discuss. [18:17:53] (CR) XenoRyet: [C: 2] Expire old pending messages [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322294 (owner: Ejegg) [18:19:23] (Merged) jenkins-bot: Expire old pending messages [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322294 (owner: Ejegg) [18:25:11] (CR) XenoRyet: [C: 2] Clean up a couple unused things [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322291 (owner: Ejegg) [18:29:00] (Merged) jenkins-bot: Clean up a couple unused things [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/322291 (owner: Ejegg) [19:45:56] (PS1) Ejegg: Cache some more custom field lookups [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/322304 (https://phabricator.wikimedia.org/T151068) [19:46:38] cwd ^^^ should make those lookups only happen once per field (per process) [19:47:22] which will save around 8000 queries over a 90 second donation queue import run [19:48:22] seems like it should be a win, barring any unintended consequences [19:48:45] ejegg: but see privchan, i think we only actually have 14 custom fields by their definition [19:49:33] yeah, maybe they meant custom groups if they're talking about numbers of joins [19:50:39] i should think so, it would be weird to have only 20 options for N fields available [19:51:23] so i guess i'd really like to see if that caching helps performance or not before we add another level of potential confusion [19:51:32] caching problems are the hardest [19:51:46] maybe we coudl do some metrics on staging? [19:52:15] might be a good way to track down slowness in general [19:52:42] also try it with dedupe on and off [19:52:50] might be useful [19:54:20] sure, that could be cool [19:54:46] trying to find where that's being called from [19:54:53] looks like the constructor for CustomQuery [19:55:12] but it's being called with a list of custom fields that are only for contributions [19:55:25] and the only place I see that class used is in the contact custom query [19:55:56] from the queries around it, the same process calls getLineItems a few queries earlier [19:56:05] but a ton of things call that [19:59:39] so custom fields can reference other tables which generates a bunch of joins? [20:10:50] yeah, each custom group gets its own table [20:38:06] ejegg: hah I can't believe all those begin_ columns on the CN banner log table are just null thru and thru [20:38:28] hehe, oh well [20:42:51] ah and that spurious DB_MASTER param is also beautiful [20:43:39] Maybe if we just add a DB_MASTER param wherever, say to a method that doesn't take any params, it'll work better [20:43:55] Also, if the network is slow, try jiggling your mouse [20:44:19] * AndyRussG censures snooty comments [21:29:37] (PS1) Ejegg: Speed up ty date update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/322352 (https://phabricator.wikimedia.org/T151068)