[00:21:31] <wikibugs>	 Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for dstrine - https://phabricator.wikimedia.org/T273134 (Dwisehaupt)
[04:53:09] <wikibugs>	 Fundraising-Backlog, DBA, Blocked-on-schema-change: CentralNotice: Update DB schema on Meta for campaign types feature - https://phabricator.wikimedia.org/T272953 (AndyRussG) >>! In T272953#6777633, @Marostegui wrote: > testwiki is done:  Ahh cool thanks so much!! Yeah looks good to me! :D
[06:51:21] <wikibugs>	 Fundraising-Backlog, DBA, Blocked-on-schema-change: CentralNotice: Update DB schema on Meta for campaign types feature - https://phabricator.wikimedia.org/T272953 (Marostegui) a:Marostegui
[07:39:55] <wikibugs>	 (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/659171 (owner: L10n-bot)
[07:52:41] <wikibugs>	 (CR) Raimond Spekking: [C: +2] "false positive" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/659171 (owner: L10n-bot)
[13:12:14] <wikibugs>	 Fundraising Sprint B 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Matching Gifts: Add fields to all_wikimedia from SSI database - https://phabricator.wikimedia.org/T269294 (jgleeson) @KHaggard I'm also around between those times.   Does it make sense to do the call before o...
[15:09:02] <wikibugs>	 (PS1) Jgleeson: Add matchings gifts prefix to mg fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/659262
[15:12:33] <jgleeson>	 fr-tech, anyone around to +2 a small update to some sql field names? ^^^ that patch just renames two new fields to add a "matching_gifts" prefix to make them less ambiguous in the main export. thanks much!
[15:12:54] <jgleeson>	 back later
[15:54:24] <wikibugs>	 Fundraising-Backlog, Analytics, Event-Platform: CentralNoticeBannerHistory and CentralNoticeImpression Event Platform Migration - https://phabricator.wikimedia.org/T271168 (Ottomata) @AndyRussG if ok with you, I'll make some patches to CentralNotice to make it always use `mw.eventLog.logEvent`.  This...
[16:00:52] <wikibugs>	 (CR) Cstone: [C: +2] Add matchings gifts prefix to mg fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/659262 (owner: Jgleeson)
[16:03:07] <wikibugs>	 (Merged) jenkins-bot: Add matchings gifts prefix to mg fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/659262 (owner: Jgleeson)
[16:16:51] <wikibugs>	 Fundraising-Backlog, Analytics, Event-Platform: CentralNoticeBannerHistory and CentralNoticeImpression Event Platform Migration - https://phabricator.wikimedia.org/T271168 (Ottomata) Actually, @AndyRussG is anything actually needed to be done?  I see that `mw.eventLog.logEvent` is called as long as t...
[16:29:55] <wikibugs>	 Fundraising-Backlog, Analytics, Event-Platform: CentralNoticeBannerHistory and CentralNoticeImpression Event Platform Migration - https://phabricator.wikimedia.org/T271168 (AndyRussG) Hi @Ottomata! Thanks so much for your help here!!  > we'll be removing support for browsers without sendBeacon  Ah ok...
[16:50:56] <wikibugs>	 Fundraising Sprint B 2021, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, fr-email-preference-center: Create new subpage for Special:EmailPreferences - https://phabricator.wikimedia.org/T268510 (AndyRussG) I would like to suggest maybe that instead of using DonationInterface for thi...
[17:00:25] <jgleeson>	 thanks cstone !
[17:39:35] <wikibugs>	 (PS1) Jgleeson: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/659340
[17:46:18] <wikibugs>	 (CR) Jgleeson: [C: +2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/659340 (owner: Jgleeson)
[17:46:53] <wikibugs>	 (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/659340 (owner: Jgleeson)
[17:49:06] <jgleeson>	 !log fundraising-tools tools updated from 41cab089da to d64b2f8cee
[17:49:08] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[17:51:29] <jgleeson>	 fr-tech I'm about to run the silverpop_daily job to generate and transfer over a new export which includes the new matching gifts fields so that katers can set up the mapping at her end. just a heads up!
[17:52:07] <cstone>	 nice jgleeson !
[17:56:00] <XenoRyet>	 Good times
[18:15:38] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1203
[18:20:36] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1502
[18:23:12] <icinga-wm>	 ACKNOWLEDGEMENT - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1502 Dwisehaupt silverpop daily job chewing away.
[18:24:36] <jgleeson>	 ah dwisehaupt
[18:24:44] <jgleeson>	 I was wondering why it was taking so long
[18:24:57] <wikibugs>	 Fundraising-Backlog: Ingenico SSL certificate renewal - https://phabricator.wikimedia.org/T273214 (EMartin)
[18:24:58] <jgleeson>	 hmmm
[18:25:04] <jgleeson>	 maybe we should kill that for now
[18:26:03] <jgleeson>	 it shouldn't be taking this long
[18:30:32] <jgleeson>	 ok I just killed the query
[18:30:59] <jgleeson>	 looks like the job got stuck
[18:32:42] <dwisehaupt>	 jgleeson: oh. there wasn't a need to kill it on our end. if it was stuck, sure. but otherwise i just wanted to ack it so it wouldn't keep hitting our phones.
[18:33:08] <jgleeson>	 yeah I guess I'm a little unsure why it was hanging
[18:33:19] <jgleeson>	 my session timed out also
[18:33:27] <jgleeson>	 I forgot to run it via tmux
[18:34:12] <dwisehaupt>	 yeah, feel free to run it again. not a bother to me.
[18:34:31] <jgleeson>	 hmm it looks like the process-control job is stuck also
[18:34:41] <jgleeson>	 it hasn't realised the query has been killed
[18:35:38] <icinga-wm>	 RECOVERY - check_mysql on frdev1001 is OK: Uptime: 525771 Threads: 17 Questions: 22893832 Slow queries: 321905 Opens: 18590180 Flush tables: 1 Open tables: 200 Queries per second avg: 43.543 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0
[18:41:22] <jgleeson>	 hmm
[18:41:38] <jgleeson>	 ok so it looks like the job just went away after my ssh session was d/c
[18:41:48] <jgleeson>	 and left the logs in a half running state
[18:41:59] <jgleeson>	 half-complete*
[18:43:44] <jgleeson>	 dwisehaupt: Jeff_Green can you see anything for these processes on civi1001?
[18:43:49] <jgleeson>	 silverpop_daily - Silverpop build and export    {pid: 11748, status: running}
[18:43:51] <jgleeson>	 silverpop_emails_build_export_files - Silverpop emails - Build export files     {pid: 11750, status: running}   Y
[18:44:07] <jgleeson>	 I can't see them but process-control is saying they're still running
[18:44:11] <Jeff_Green>	 looking
[18:44:14] <jgleeson>	 thanks
[18:45:14] <Jeff_Green>	 neither are running, nor is run-job
[18:46:29] <jgleeson>	 oh weird
[18:47:02] <jgleeson>	 this cmd shows them as running `run-job --list-jobs --only-running`
[18:47:26] <jgleeson>	 but I'm suspecting that command is looking at lock files to make it's decision on the output
[18:47:33] <Jeff_Green>	 the test is not clever, it just looks for the existence of the pidfile, but there are cases where that can be left behind as cruft
[18:47:39] <Jeff_Green>	 yes
[18:48:16] <Jeff_Green>	 there's a bit of a fix for the test awaiting review in gerrit
[18:48:38] <jgleeson>	 ok well that's somewhat good news
[18:48:51] <Jeff_Green>	 which reminds me, I gotta see if Adam reviewed it again after I fixed some the stuff he pointed out in the first rev
[18:48:52] <jgleeson>	 process-control clears out old lockfiles I think when run
[18:49:09] <jgleeson>	 so I should be able to fix the issue and rerun
[18:49:15] <Jeff_Green>	 I think you're right, yeah
[18:55:29] <jgleeson>	 ok fr-tech I think I know what went wrong
[18:55:34] <jgleeson>	 I need to rebuild the silverpop tables
[18:57:20] <jgleeson>	 because the table intended to store the new matching gift fields has the following create statement
[18:57:22] <jgleeson>	 CREATE TABLE IF NOT EXISTS silverpop_export
[18:57:36] <jgleeson>	 and as it already exists, it will never add the new fields.
[18:57:56] <jgleeson>	 so I'm gonna run the rebuild-schema.sql file on frdev1001 as the silverpop export user
[18:58:26] <jgleeson>	 which should update the tables
[18:58:47] <jgleeson>	 hmm
[18:58:55] <jgleeson>	 I guess i could just update that table in isolation
[18:59:05] <jgleeson>	 to save rebuilding them all
[19:05:40] <jgleeson>	 ok it doesn't look that straight forward
[19:06:05] <jgleeson>	 there are 23mn records in the table that needs replacing
[19:06:23] <jgleeson>	 which i wasn't expecting based on our incremental update process
[19:06:40] <jgleeson>	 so to preserve that data, I'm gonna back it up in the event we need to restore that
[19:07:26] <jgleeson>	 hmmm I guess I could also just update the existing table adding the new fields
[19:08:32] <jgleeson>	 although depending on the table that could take some time
[19:23:09] <wikibugs>	 Fundraising-Backlog, Analytics, Event-Platform: CentralNoticeBannerHistory and CentralNoticeImpression Event Platform Migration - https://phabricator.wikimedia.org/T271168 (Ottomata) >  since it looks like we currently do still support IE 11, which doesn't support sendBeacon. Do you know the timeline...
[20:14:07] <wikibugs>	 Fundraising Sprint B 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Matching Gifts: Add fields to all_wikimedia from SSI database - https://phabricator.wikimedia.org/T269294 (KHaggard) Update:  @jgleeson and I met and discussed a game plan for the Matching Gifts fields. We de...
[20:25:38] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1500
[20:26:33] <icinga-wm>	 ACKNOWLEDGEMENT - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1500 Dwisehaupt silverpop export running.
[20:38:24] <eileen>	 I'm back...
[20:42:25] <cstone>	 hellllllo eileen
[20:42:32] <eileen>	 how goes?
[20:43:14] <eileen>	 I'm just looking to see if there is something I can work on today & tomorrow
[20:51:02] <eileen>	 I'm going to take a look at updating civi on staging - there are some changes that need merging upstream - but happy to look at something else if someone wants me to
[20:55:36] <icinga-wm>	 RECOVERY - check_mysql on frdev1001 is OK: Uptime: 534171 Threads: 17 Questions: 23282516 Slow queries: 327020 Opens: 18888395 Flush tables: 1 Open tables: 200 Queries per second avg: 43.586 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0
[21:29:12] <jgleeson>	 welcome back eileen !
[21:31:07] <wikibugs>	 (CR) Jforrester: [C: +2] build: Updating eslint-config-wikimedia to 0.18.0 [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/659049 (owner: Libraryupgrader)
[21:31:37] <eileen>	 thanks jgleeson
[21:31:44] <eileen>	 I hope you are doing ok?
[21:34:17] <jgleeson>	 good thanks. just been doin some work on the silverpop export today add a few of the matching gift fields to the main export
[21:34:26] <jgleeson>	 to*
[21:34:30] <jgleeson>	 how was your brennen ?
[21:34:32] <jgleeson>	 oops
[21:34:37] <jgleeson>	 how was your break*
[21:34:39] <jgleeson>	 :)
[21:34:41] <brennen>	 haha
[21:34:50] <jgleeson>	 sry! :)
[21:35:11] <brennen>	 having been summoned, i will take a moment to echo the welcome back eileen, before disappearing. :)
[21:35:29] <eileen>	 :-)
[21:35:39] <eileen>	 the break was great - we did all the touristy things
[21:38:09] <jgleeson>	 :)
[22:10:27] <wikibugs>	 Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for lnavarro - https://phabricator.wikimedia.org/T272646 (Dwisehaupt) Open→Resolved
[22:10:38] <wikibugs>	 Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for cvirtue - https://phabricator.wikimedia.org/T272894 (Dwisehaupt) Open→Resolved
[22:11:39] <wikibugs>	 Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for bbarnes - https://phabricator.wikimedia.org/T273131 (Dwisehaupt) Certificate renewed and delivered via email. Password sent via SMS.  CN name change on the cert to reflect a name change. Cert name adjusted to match email for...
[22:15:31] <wikibugs>	 Fundraising Sprint B 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Matching Gifts: Add fields to all_wikimedia from SSI database - https://phabricator.wikimedia.org/T269294 (jgleeson) hey, @KHaggard just a heads up that the updates are live as discussed and a test run of exp...
[23:15:34] <wikibugs>	 Fundraising-Backlog, Analytics, FR-Tech-Analytics: Whitelist Portal and WikipediaApp event data for (sanitized) long-term storage - https://phabricator.wikimedia.org/T273246 (EYener)
[23:29:06] <wikibugs>	 Fundraising Sprint B 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Matching Gifts: Add fields to all_wikimedia from SSI database - https://phabricator.wikimedia.org/T269294 (KHaggard) Hi @jgleeson ! Thanks for this update. I saw it before logging off today, so I went forward...