[20:14:00] < icinga-wm> PROBLEM - Mobileapps LVS eqiad on mobileapps.svc.eqiad.wmnet is CRITICAL: /{domain}/v1/page/mobile-summary/{title} (retrieve page [20:14:03] preview of Dog page) is CRITICAL: Test retrieve page preview of Dog page responds with malformed body: NoneType object has no attribute [20:14:06] get [20:14:13] sorry, that paste was a fail [20:14:31] but see the error about the preview of "Dog page" [20:14:45] it just started popping up a minute ago [20:15:08] mutante: thanks for the heads-up. i'm looking into it. /cc bearND [20:15:14] mutante: interesting. [20:16:24] honestly i'm not sure why we're keeping that particular endpoint running in prod [20:16:38] bmansurov: there are no ordering guarantees for parser functions either [20:17:19] i see [20:18:12] strictly speaking this is all implementation defined behavior, and it does differ between PHP parser & Parsoid [20:18:46] no order guarantees allow for parallel execution and caching, which is good for performance [20:20:47] mdholloway: yes, we can get rid of this endpoint since the infobox stuff is now handled better in the new experimental endpoints [20:22:19] bearND: cool. i'll create a phab ticket. probably better announce it on the lists first [20:22:30] mdholloway: already writing one up [20:22:40] bearND: cool :) [20:28:30] mdholloway: feel free to update T152135 if needed [20:28:31] T152135: Remove obsolete mobile-summary endpoint in MCS - https://phabricator.wikimedia.org/T152135 [20:28:51] bearND: will do, thx [20:40:10] mdholloway: you think 1 point is good for this? [20:40:33] bearND: i think so [20:40:41] k [21:02:02] niedzielski: created T152141 to follow up on our convo on the description edit success activity RTL issue [21:02:03] T152141: DescriptionEditSuccessView / AppTextViewWithImages doesn't support RTL languages - https://phabricator.wikimedia.org/T152141 [21:02:23] mdholloway: very cool, thanks! [21:02:53] mdholloway: i think that test patch is good to go. i just had one question about the dark mode that i wasn't sure if it was ok or not [21:03:34] mdholloway: Looks like now other patches fail tests due to the mobile-summary endpoint, too. I guess once that is merged we'll rebase your template patch, then I can do a deploy [21:04:18] hmm... i *think* there shouldn't be any difference b/w light and dark mode for this activity. i'll dig into that after i have a quick look at bearND's patch (hi priority) [21:11:48] bearND: do you know the command to run the service checks manually on the prod machines? [21:12:02] i thought scap3 had a handy command for that but can't find it documented anywhere [21:12:28] mdholloway: check-mobileapps [21:12:45] it' s on the MCS deployment instruction page [21:12:47] aha, thanks! [21:13:58] bearND: i usually run them locally before deploying and then i know scap runs them while deploying, so i don't usually do it manually again afterward [21:27:08] niedzielski: ok, i see what you mean on the font coloring there. i interpreted the mock as specifying the same dark translucent background should be used regardless of whether the app was on light or dark mode, so i think the font color can stay the same. reets, is that correct? [21:27:25] sorry, this is on the description edit success activity in the android app [21:27:35] ("description saved!") [21:29:25] mdholloway: yes, that's correct – it's the same #fff color on dark or light mode since it's always showing under the translucent overlay. [21:29:43] ^ per https://zpl.io/RGP19 [21:31:59] reets: excellent! thanks. [21:35:05] reets mdholloway: cool! ima merge this then [21:35:37] niedzielski: \o/ [21:42:17] mdholloway: I just noticed the wikidata capitalization has been removed from the article - was this intentional? [21:42:37] reets: no, recently introduced regression, i just pushed a patch to fix it actually [21:42:58] mdholloway – phewh ok cool, thanks! [22:03:33] mdholloway: hmm, I keep getting failures when running the tests in Docker, inside an Ubuntu VM, in a similar fashion that you saw with Docker for Mac. Would you mind trying on your setup? [22:03:45] sure [22:03:55] mdh I hope we don't have to revert the revert of the revert, lol [22:04:07] bearND: ha! i hope not [22:04:42] bearND: i tested it so many times... [22:15:32] bearND: tests passed for me [22:15:40] mdholloway: cool. Thanks! [22:15:49] it's really strange the failures are happening for you now. [22:16:09] mdholloway: still need to work on improving my Ubuntu VM, I guess [22:17:25] mdholloway: I'll do the deploy once it is merged, unless you want to [22:17:55] bearND: go ahead, i was actually just about to head out the door and back home [22:18:04] bearND: i'll be back in ~15 minutes [22:18:28] mdholloway: np. I'm happy to do it. Thanks for testing it [22:38:21] bearND: the failing tests are in the experimental endpoint and likely caused by an endpoint. any objection to me force-submitting so we can deploy? [22:38:25] *caused by an edit [22:40:58] mdholloway: yes, sounds good to me. We really need to get more unit tests instead of integration test [22:44:05] hmm, submit button is gone, i thought i could just force it but guess not if there's a failing test [22:46:45] mdholloway: mine is gone, too [23:07:27] bearND: just out of meeting and have to run and pick up my wife, i'll look at the deploy patch again shortly if the recheck doesn't fix it [23:08:19] mdholloway|afk: I'm going to submit a patch to skip the two tests [23:09:04] https://gerrit.wikimedia.org/r/324825 [23:12:20] you fixed it! thanks [23:13:42] FYI: https://phabricator.wikimedia.org/T152155 [23:13:45] mutante: ? We haven't deployed it yet [23:13:48] Seems like something's pretty fucked in PageImages [23:13:53] bmansurov ^ [23:14:46] bearND: well, all the monitoring checks that made me report it earlier, have now recovered [23:14:49] RECOVERY - mobileapps endpoints health on scb2002 is OK: All endpoints are healthy [23:14:53] a whole page full of that [23:14:58] on all of them [23:15:14] looking [23:15:37] mutante: Yeah, saw that on ops channel. Was wondering what that was all about. The alerts had been ack'd earlier [23:16:04] bearND: ack'd means "dont talk about it until the next state change" but CRIT -> OK is a state change [23:16:35] mutante: I see. Had an incling about that after I wrote it. Godd to have confirmation [23:17:05] as opposed to "disable notifications" which turns it all off until we manually turn it back on [23:17:10] but we try to avoid that [23:31:30] Deskana: what's EBernhardson's irc nick? [23:31:46] looks like his patch fixes the problem for now, I'll need to swat it today [23:34:05] mdholloway|afk: cool. Thanks! Would you do the deploy prep again? [23:34:25] bearND: sure. [23:34:39] thanks! [23:35:07] we should probably see about adding a flag or something for experimental endpoint tests so we're not blocking deployments on them [23:35:25] np, coming up shortly [23:38:23] tgr: running the maintenance script in the page images extension should take care of https://phabricator.wikimedia.org/T152155 right? [23:55:45] mdholloway: tests passed this time. I'm getting ready to do the deploy [23:56:38] bearND: sounds good [23:58:16] come on zuul. Only two minutes till SWAT deploy window [23:58:31] there it is [23:58:50] bmansurov: ebernhardson [23:59:02] bmansurov: Looks like you found him. :-) [23:59:13] Deskana: ok thanks, yeah