[11:08:30] we're now serving Argentina, Chile and Uruguay from magru [11:08:33] https://usercontent.irccloud-cdn.com/file/nzswivpy/image.png [11:08:39] \o/ [11:08:56] 🎉 [11:11:45] \o/ [11:11:47] Do we have an idea how different the site feels to people in those countries? [11:12:03] yes [11:13:56] I'm not sure if there are new ones with more recent data but you can see for example https://phabricator.wikimedia.org/F53633438 [11:22:23] 🥳 [14:19:23] {◕ ◡ ◕} [15:12:30] klausman: volans: we can do better than that in a few weeks, we'll be able to look at the `navtiming` metrics which have real-user latencies reported by browsers like `firstContentfulPaint` and `mediaWikiLoadEnd` https://wikitech.wikimedia.org/wiki/Performance/Metrics#navtiming [16:26:17] Hello! Am seeking review for https://gerrit.wikimedia.org/r/c/operations/puppet/+/1063224 - Rewrite mediawiki.org/beacon/event to /beacon/event/index.php [16:26:17] As is this will not have any effect on live traffic, because /beacon/event is intercepted by varnish. We'll have to remove the vcl handling to do the final deploy. [16:26:17] _joe_ ? look okay to you? [16:26:36] <_joe_> in a meeting [16:33:52] ottomata: o/ how was the rewrite tested/ [16:33:53] ? [16:37:14] ok I see https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1063222 [16:37:25] I was also wondering if there was a rollout plan for the VCL change :) (if not, I suggest asking Traffic to do it, rolling forward to just one cp node at first, to test everything in prod without affecting all user traffic) [16:39:25] the rewrite rule looks ok, but I'll defer to Joe's final +1 since there is a ton of discussion in the task [16:39:56] ottomata: I'd add a bit more context to the commit msg so it is easier without context to know what you are doing, from the task it is difficult :) [16:40:43] particularly if this is actually a noop for now, pending rollout of the VCL changes, which will happen at a later step [16:42:01] it changes the httpd config so not 100% no-op in any case, but.. :D [16:45:15] yes, semantically "functional noop for requests that would match" not "literal noop" [17:45:22] anyone have admin rights on the test phab instance, phabricator.wmcloud.org, or want to grant them to me? [17:45:38] I would like to create an "In Progress" status as exists in prod [17:58:14] jhathaway: that's probably a releng question, although, you could also use your (implicit) WMCS global root as SRE to do that perhaps [17:58:57] cdanis: thanks, I'll ask in releng, before asking for forgiveness [18:57:52] jhathaway: do you have an existing account and just need more rights? [18:59:05] dzahn@phabricator-bullseye:/srv/phab/phabricator/bin$ sudo ./user empower --user jhathaway DONE Empowered user account "jhathaway". [18:59:08] here you go [18:59:28] docs are basically https://phabricator.wikimedia.org/T377236 [19:16:26] mutante: thanks! [19:23:34] yw:) [19:26:05] Hi SRE team, [19:26:05] it seems like the contribution is not visible on Gerrit. See https://gerrit.wikimedia.org/r/q/owner:0freerunning@gmail.com. I have 500+ patches. [19:31:26] mutante: do you happen to have admin on prod phab? If, so any chance you could export the maniphest status config, so I don't fat finger the test one? [19:33:10] Jayprakash12345: I see "1-25 of many" ... do you perhaps have Javascript disabled? [19:35:10] Or logged out? [19:35:38] I see none actually [19:38:05] cdanis: It is showing "No changes" on different browser with logged-in and log out state. See https://paste.pics/2f61da323f2871a6c506c3dbd8dca1a0 [19:38:59] owner:"Jayprakash12345" seems to consistently work as expected as long as it does not autocomplete to email [19:39:15] it is consistent for me [19:39:20] There's 3 seemingly different accounts showing with that email [19:39:28] if you put a space after that in the search bar [19:40:27] author: shows 1-25 of many, but I also see nothing for owner: [19:43:37] https://gerrit.wikimedia.org/r/q/owner:%22Jayprakash12345%22 and https://gerrit.wikimedia.org/r/q/author:0freerunning@gmail.com is working for me. Thanks cdanis and AntiComposite. This is strange because I used to share https://gerrit.wikimedia.org/r/q/owner:0freerunning@gmail.com to others. It was working well from last couple of years. [19:44:32] jhathaway: https://phabricator.wikimedia.org/P71701 [19:44:55] perfect thanks mutante [19:45:18] Jayprakash12345, if it worked before and doesn't now, you should file a phab task and tag Gerrit [19:55:34] Done https://phabricator.wikimedia.org/T382028 [19:57:54] Jayprakash12345: I see a bunch of changes when I click that link [19:58:11] mutante: yes, it's inconsistent between users, interestingly [19:58:18] I also consistently saw a bunch of changes [19:58:43] I only see them when I am logged in. not when logged out. that's odd. ok [19:59:03] btw folks, I got nerdsniped this afternoon by something that's long irritated me about Prometheus historical views, and turned it into a sort of interactive blog post: https://grafana.wikimedia.org/d/e16fec87-72a6-405e-b931-d04a0bc26e48/cdanis-temporal-downsampling-considered-harmful-max-over-time-rate-interval-trick [20:00:30] this is likely a side-effect of what Reedy also noted above, that there appear to be three different gerrit accounts associated with that address (leading to some kind nondeterminism as to which you end up seeing when searching by email) [20:00:50] cdanis: nice! [20:01:28] yes, agreeed. 3 different accounts on one email. I think Tyler will be able to dig into this further. [20:01:46] good that a ticket was created, thanks [20:02:24] Jayprakash12345: try this: https://gerrit.wikimedia.org/r/q/author:0freerunning@gmail.com [20:02:30] author vs owner [20:02:35] per comment from James. wfm [20:03:09] Just a note that the other two accounts were also from 2018. [20:03:42] one thing that changed since 2018 is that gerrit does not use a mysql db anymore and instead all users and things like that are in git itself [20:04:40] renaming users is hard [20:07:15] Jayprakash12345: see a new comment on the ticket. I'm sure there will be more soon-ish, just some patience please because some key people are currently away [20:09:04] mutante: No problem. It is not a critical or high-priority thing. If it easy to fix then do. Otherwise, I have change the links from lot of places. [20:09:30] I have to* [20:09:33] Jayprakash12345: ok, sounds good:) until then, maybe consider using the author: URL [20:44:28] A new key was added to https://www.mediawiki.org/keys/keys.txt today — it appears on the keys.html page, but not yet on there. Busting the cache with a query string shows the key. How would one purge the cache of https://www.mediawiki.org/keys/keys.txt ? [20:45:41] TheresNoTime: if you're a deployer, the same way you can purge any other URL https://wikitech.wikimedia.org/wiki/Kafka_HTTP_purging#One-off_purge [20:46:59] I thought that was just for things in `/static/`, but I'll give that a go in a sec, thanks! I should have done it when deploying https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1101577 earlier I think [20:47:37] if that doesn't work, poke me [20:54:23] (yup that worked, thanks cdanis :D) [20:59:10] 💙 [21:04:03] updated for kubernetes: https://wikitech.wikimedia.org/w/index.php?title=Kafka_HTTP_purging&diff=prev&oldid=2253865 [21:05:21] ty ty! [21:05:48] (you don't need to redo anything, that's just the preferred way for next time) [21:08:17] ack :) [21:38:28] elukey: i manually edited that index.php file on mwdebug1002, then I hit it via the test-event symlink (bypassing vcl interception) see https://phabricator.wikimedia.org/T353817#10381650 [21:38:56] but for the rewrite rule? heh. i think i tested locally in mw docker a while ago, but it has been a while and it wasn't...easy? [21:45:15] gerrit doesn't allow multiple accounts to have the same email I think mutante ? When I click on https://gerrit.wikimedia.org/r/q/owner:0freerunning@gmail.com, in search I just enter a space and it pops down three people.