[11:08:05] a-team: brb in one hour more or less, if you need me please write and I'll get back to you asap :) [11:58:37] hi a-team! [12:27:13] mforns: o/ [12:27:20] hey elukey :] [12:57:04] Analytics, Research-and-Data: Percentage of users with DNT on - https://phabricator.wikimedia.org/T127571#2050879 (mforns) > Not quite my focus area, but I happen to know that @mforns et al. are already generating internal datasets for this that should be available on stat1002, see https://wikitech.wikime... [12:57:10] Analytics-Kanban: Add wm:BE-Wikimedia-Belgium to Wikimetrics tags {dove} [1 pts] - https://phabricator.wikimedia.org/T124492#1958065 (Romaine) Thanks! [14:22:55] Analytics-Tech-community-metrics, Possible-Tech-Projects: Misc. improvements to MediaWikiAnalysis (which is part of the MetricsGrimoire toolset) - https://phabricator.wikimedia.org/T89135#1027974 (AbdealiJK) Hey there, I'm interested in doing this as a GSoC project. @jgbarah @Dicortazar are you still in... [14:54:31] hey everyone [14:55:36] ottomata: I'm ready to deploy whenever you are [14:56:09] milimetric: hello1 [14:56:11] !! [14:56:23] hi :) [14:58:45] hallo! [14:58:46] oh yea [14:58:47] i'm ready [15:00:00] looking for that task... [15:00:19] oh milimetric just found this one [15:00:19] https://phabricator.wikimedia.org/T114999 [15:00:23] i'm going to merge them? [15:00:29] https://phabricator.wikimedia.org/T126294 [15:00:38] yeah, I think that's what we were about to do, but disable puppet first [15:00:56] Analytics, Operations, Services, scap, Scap3: Deploy AQS with scap3 - https://phabricator.wikimedia.org/T114999#2051034 (Ottomata) [15:00:59] Analytics, HyperSwitch, RESTBase, Services, Patch-For-Review: Separate AQS off of RESTBase - https://phabricator.wikimedia.org/T126294#2051035 (Ottomata) [15:02:44] ok milimetric disabling puppet on aqs* and depooling aqs1001 [15:02:49] k [15:03:48] hmm, how to depool.. [15:04:04] !log deploying analytics/aqs to aqs100[1-3] and replacing the old restbase deploy there [15:04:29] you said it was LVS and that was easy to depool [15:04:47] ottomata: onfctl --find --action set/pooled=no aqs1001.eqiad.wmnet should do it [15:05:00] or rather confctl --find --action set/pooled=no aqs1001.eqiad.wmnet should do it [15:05:38] thank you! [15:06:09] moritzm: does that take some time? [15:06:10] http://config-master.wikimedia.org/pybal/eqiad/aqs [15:06:12] aqs1001.eqiad.wmnet: pooled changed yes => no [15:06:19] or maybe aqs is just pybal? [15:06:44] ah [15:06:45] no [15:06:45] http://config-master.wikimedia.org/conftool/eqiad/aqs [15:08:30] milimetric: do you know if restbase requests logs go somewhere on disk? [15:08:33] or just to logstash? [15:08:43] I think I asked that and it's just logstash [15:09:10] it's usually effective immediately [15:09:45] for a stateless server like restbase I'd expect it to be effective within a few seconds, once it finishes all current requests [15:10:03] ottomata: o/ [15:10:22] hi elukey! [15:10:36] aye, i just want to see that reqs aren't going throuhg [15:10:42] and...we should remove this config master pybal stuff if it isn't used [15:10:44] reading https://phabricator.wikimedia.org/T126294 [15:10:46] confusing to have two location [15:10:47] s [15:11:00] yep definitely, same thing for eventbus [15:11:03] milimetric: i am very bad at logstash [15:11:09] aye, i actually looked at your page elukey :) [15:11:11] thank you! [15:11:20] how can I filter for aqs1001 restbase access logs? [15:12:31] I'm also bad at it and rely on pre-created dashboards [15:12:57] its kinda annoying that there are no logs on disk [15:13:03] there is a /var/log/restbase [15:13:06] but nothing is in there! [15:15:06] ok i got some process logs out of journalctl [15:15:17] but just proces slogs [15:15:19] no access logs [15:15:20] oh well [15:15:26] i guess we'll just have to assume that the depool worked [15:15:37] ottomata: quick question about hive/oozie when you have time.. can we restart them on analytics1027 for glibc updates safely or better to wait for tomorrow's CDH upgrade? [15:15:53] ottomata: confctl allows you to check whether it worked: [15:16:08] elukey: should wait, since we are just going to restart them tomorrow, and probalby move them to analytics1015 [15:16:23] yep I figured, thanks :) [15:16:26] oh moritzm? i believe that confctl value changed properly [15:16:32] http://config-master.wikimedia.org/conftool/eqiad/aqs [15:16:38] moritzm: do you know why we also have http://config-master.wikimedia.org/pybal/eqiad/aqs [15:16:38] ? [15:17:12] if you run confctl --find --action --get aqs1001.eqiad.wmnet if will show you the status as seen by conftool/etcd [15:17:42] not sure about http://config-master.wikimedia.org/pybal/eqiad/aqs [15:17:56] best ask Giuseppe about that [15:17:57] how can pybal and confctl have two different readings of the same LVS setup? [15:18:26] it might be the config data which we previously managed by editing the text files, dunnp [15:18:29] yeah [15:18:33] i would assume to [15:18:34] too [15:18:38] if so, we should remove those [15:18:48] moritzm: that's why i'm worried, i believe conftool/etcd is all cool [15:18:56] but how can I be sure that pybal is not routing stuff anyway? [15:19:04] i see no way to view restbase access logs [15:20:07] i think I am using kibana correctly [15:20:13] but can find no logs for any aqs hosts there [15:21:09] not sure, but I recently depooled aqs via confctl for the glibc security update and it worked fine, I think the http://config-master.wikimedia.org/pybal/eqiad/aqs is a red herring [15:22:01] ok [15:22:18] milimetric: do you have a pageview api url I can use for simple testing of the service? [15:22:32] locally, right? [15:22:36] one sec [15:23:08] sure [15:23:09] well [15:23:11] i guess [15:23:13] maybe not locally [15:23:41] ok i think a see less traffic on aqs1001 restbase, so i'm going to guess the traffic i see there is just lvs monitoring [15:24:51] curl localhost:7232/analytics.wikimedia.org/v1/pageviews/per-article/en.wikipedia/all-access/all-agents/Selfie/daily/2016020100/2016030100 [15:25:01] not local? [15:25:11] why? remote it would just go through LVS, no? [15:25:16] Analytics, Research-and-Data: Percentage of users with DNT on - https://phabricator.wikimedia.org/T127571#2051136 (Tbayer) >>! In T127571#2050879, @mforns wrote: >> Not quite my focus area, but I happen to know that @mforns et al. are already generating internal datasets for this that should be available... [15:25:20] yes, but i know its going to work local [15:25:23] both are good [15:25:28] so I can test it after we change the setup [15:25:28] https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/all-agents/Selfie/daily/2015100100/2015103000 [15:25:30] with local [15:25:37] I'll do the local test [15:25:50] no worries, I'm on all the machines and I'm making a little script to test some other article titles [15:26:02] ok cool [15:26:08] ok, welp, proceeding, NOW stopping puppet :) [15:27:05] merging change [15:27:31] running puppet on tin [15:29:42] hmm, aqs repo not on tin [15:29:43] hmmm [15:30:45] how does that work, don't you clone analytics/aqs/deploy on tin and then from tin it's pushed to the cluster? [15:31:54] oh ja i have to run puppet on salt master first [15:31:58] yes [15:32:03] puppet run on tin should now do that clone [15:32:08] but, it uses salt pillars [15:32:15] and that has to be in place on salt master first.. [15:32:24] (kinda glad scap does not use salt...) [15:32:39] this is the bootstrap process that is not yet ironed out with scap [15:32:49] got it [15:32:50] currently it relies on the old salt/trebuchet bootstrap [15:32:55] salt pillars sound to me like some biblical thing [15:32:56] to get tin all set up [15:32:59] hah [15:33:24] puppet run did not put aqs [pillar in place? [15:33:25] hmm [15:33:42] oh maybe palladiumm is not salt master anymore [15:34:05] https://wikitech.wikimedia.org/wiki/Salt#Salt_master_locations [15:34:06] :) [15:38:15] ok! repo present on tin [15:39:30] i'm going to stop restbase on aqs1001, and then scap deploy [15:39:52] !log stopping restbase on aqs1001 [15:40:07] milimetric: please keep checking to make sure things are ocol [15:40:27] i'm checking the other two, yea [15:40:46] deploying via scap to just aqs1001 [15:41:04] cool, my script should give me one extra response when the new version's in place with the fixes [15:41:15] oh, ned to run puppet on aq1001 first [15:41:24] oh right hmmm [15:41:30] hm, no this should be fine [15:41:57] the scap ssh keys need setup by puppet first.. [15:42:00] I see aqs1001 down, public hits to the cluster still good [15:46:37] hmm, having public key problems during deploy [15:46:38] not sure why [15:48:42] milimetric: moving to rel eng for help [15:49:45] weird, k [16:00:05] milimetric: got eventbus meeting...:/ [16:00:14] aiy :( [16:00:24] uh... well, I'll keep pinging that public URL, ottomata [16:00:35] and if I see it getting slow or something, I'll let you know [16:18:29] milimetric: do I need to keep all the actions below ? [16:18:35] for the page view dumps workflow [16:19:00] * milimetric looks to answer elukey [16:19:00] (whenever you have time) [16:19:13] I am building https://wikitech.wikimedia.org/wiki/User:Elukey/Analytics/PageViewDumps [16:19:58] mmm maybe it is related to the wikis whitelisted? [16:21:51] * elukey is probably stating obvious things [16:21:58] * elukey is very ignorant about oozie [16:22:56] nuria thanks for https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Oozie#Running_a_real_oozie_example, really helpful :) [16:23:12] elukey: so I think you should keep the "transform" step as the root of the workflow: https://github.com/wikimedia/analytics-refinery/blob/master/oozie/pageview/hourly/workflow.xml#L184 [16:23:27] and from there, any step that's used by the path from there [16:24:14] all right [16:24:18] elukey: so the whitelist check is part of that... but it's not super relevant [16:24:36] keeping it would mean we get warnings that aren't particularly actionable at this point [16:25:19] on the other hand, removing it is harder and the white-list is almost guaranteed to have all the domains in it because this data has already been processed just not archived into files [16:25:38] makes sense [16:25:41] or ... mmm, that's not exactly right [16:25:52] because we didn't add that whitelist thing in May [16:26:16] so it's your call - if you'd find it interesting to know what we're archiving that's not in the whitelist, then leave it [16:27:06] milimetric: no no it is fine :) so I cut off aggregate and start with transform [16:31:07] Joal, madhuvishy standddduppp [16:35:07] Analytics, Beta-Cluster-Infrastructure, Services, scap, and 3 others: Set up AQS in Beta - https://phabricator.wikimedia.org/T116206#2051522 (greg) [16:39:55] Analytics, Operations, Ops-Access-Requests: Add Analytics engineers to `deploy-service` group - https://phabricator.wikimedia.org/T127720#2051551 (Ottomata) [16:40:34] Analytics, Operations, Ops-Access-Requests: Add Analytics engineers to `deploy-service` group - https://phabricator.wikimedia.org/T127720#2051582 (Ottomata) [16:40:58] Analytics, Operations, Ops-Access-Requests: Add Analytics engineers to deploy-service group - https://phabricator.wikimedia.org/T127720#2051551 (Ottomata) [16:59:59] (CR) Nuria: [C: 2 V: 2] "Merging, thanks for doing the refactor." (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/271253 (https://phabricator.wikimedia.org/T126282) (owner: Mforns) [17:04:38] (CR) Nuria: "I think the "insource" still appears twice in "regex" and case:" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/254461 (https://phabricator.wikimedia.org/T118218) (owner: Bearloga) [17:04:46] Analytics, Operations, Ops-Access-Requests: Add Analytics engineers to deploy-service group - https://phabricator.wikimedia.org/T127720#2051551 (Krenair) Which users count as 'analytics engineers'? everyone in analytics-admins? [17:11:39] hello nuria. when you have 15 min, can we chat about mobile users and IP? [17:12:04] leila: sure, after 11am [17:12:09] sounds good, nuria. [17:12:17] I'll ping you then, nuria. [17:12:21] k [17:16:24] ottomata2: I had to push an update to the aqs submodule to aqs/deploy, maybe now it'll work [17:17:02] but ... hm... this looks like a hyperswitch error and the update didn't change the version of hyperswitch [17:18:32] Analytics, Operations, Ops-Access-Requests: Add Analytics engineers to deploy-service group - https://phabricator.wikimedia.org/T127720#2051753 (Ottomata) Hm, on second look, we do already have an aqs-admins group. Perhaps we can reuse that group for deployment. Looking into how this would work. [17:27:54] hi milimetric, I’d like to draft a note for the research teams I mentioned the other day, I’ll run it past you before I send it out [17:28:29] cool, DarTar, happy to chat too if you want [17:28:39] * milimetric is thinking catching up with DarTar would be fun anyway [17:28:53] likewise :) [17:34:23] milimetric: taskingggg holaaa [17:34:29] elukey: taskingggg [17:35:25] arg, brt [17:36:26] nuria: ops meeting, will join later on :) [17:39:53] Analytics-Kanban, Patch-For-Review: Pageviews's title parameter is impossible to generate from wiki markup {melc} [3 pts] - https://phabricator.wikimedia.org/T127034#2051883 (Milimetric) [17:42:38] I AM EATING SOYLANT FOR LUNCH [17:43:00] half done with this bottle, the remainder is getting harder to swallow [17:44:19] Analytics-Kanban, Patch-For-Review: Update AQS yaml format to match new convention {melc} [3 pts] - https://phabricator.wikimedia.org/T127323#2051906 (Milimetric) [17:45:02] ottomata, how horrible is it? My friend paultag likes cooking with it. Soylent-based dishes. Paul is nuts, mind. [17:45:21] Analytics-Kanban, Patch-For-Review: Improve the data format of the browser report {lama} [13 pts] - https://phabricator.wikimedia.org/T126282#2051912 (Milimetric) [17:46:20] Analytics-Kanban, Patch-For-Review: Update AQS yaml format to match new convention {melc} - https://phabricator.wikimedia.org/T127323#2039784 (Milimetric) [17:49:03] Analytics-Kanban: Create reportupdater browser reports that query hive's browser_general table {lama} - https://phabricator.wikimedia.org/T127326#2051946 (Milimetric) a:mforns [17:53:45] Analytics-Kanban: Puppetize reportupdater to be executed in stat1002 and run the browser reports {lama} - https://phabricator.wikimedia.org/T127327#2051974 (Milimetric) p:Triage>Normal [17:53:57] Analytics-Kanban: Create reportupdater browser reports that query hive's browser_general table {lama} - https://phabricator.wikimedia.org/T127326#2051977 (Milimetric) p:Triage>Normal [17:54:40] ottomata, elukey can you join tasking at 10 am PST? (in 5 mins) [17:55:17] nuria: sure! [18:02:45] ottomata: can you join tasking too? [18:02:59] ottomata: that way we can task the datacenter failover for EL [18:03:08] ottomata: if not possible we can do it on thu [18:04:09] ottomata, elukey never mind let's move tasking to thu [18:04:43] leila: can talk now if you have time [18:04:57] leila: we finished tasking early [18:05:05] leila: otherwise 11 am is fine [18:05:07] Ironholds: yt? [18:05:24] nuria: give me 1 min, please. I'll call you from Hangout. [18:05:30] leila: k [18:08:00] calling you nuria. [18:08:39] leila: me no see call [18:08:44] leila: i can callyou [18:08:48] mm, adding you again [18:16:21] hey sorry, was eating lunch! [18:16:31] nuria: tasking is where? [18:16:44] ottomata: changed to thu [18:17:20] oh ok [18:17:28] nuria: i don't think we need to task the datacenter failover [18:17:45] ottomata: on meeting , we can talk in a bit [18:18:14] k [18:20:41] Ironholds: yt? [18:22:17] nuria, yup [18:22:22] sorry, in retrospective [18:22:30] but it's a dull retrospective and I can't speak because YAY BLUEJEANS ;p [18:22:40] Ironholds: let's talk later [18:23:43] okay! [18:38:24] Ironholds: let me know when you are free [18:38:53] nuria, should be out in 15? [18:38:58] k [18:39:01] cool! [18:41:02] milimetric: am hanging up, let's get back into batcave when i get the deployment group stuff figured out [18:43:01] a-team: logging off, tomorrow I'll be online probably only later on in the evening (sorry ottomata to skip the CDH upgrade :( [18:43:17] elukey, see you tomorrow! [18:43:21] latesr! [18:54:39] hmmmmm! [18:54:40] actually [18:54:41] milimetric: [18:54:44] can you try the scap deploy? [18:54:49] i'm not in aqs-admins :( [18:54:51] so [18:54:55] ahh, i can just put myself in there [18:55:52] i can try it though, ottomata, sure [18:55:54] from tin? [18:56:00] ya you try [18:56:00] yup [18:56:01] tin [18:56:06] /srv/deployment/analytics/aqs/deploy [18:56:07] then [18:56:16] deploy --limit aqs1001.eqiad.wmnet [18:56:27] that's it? [18:56:29] oh [18:56:31] do git pull firs [18:56:32] t [18:56:34] since you want cahnges [18:56:35] changes [18:56:39] but ja [18:56:54] and submodule update --init right? [18:56:57] sure [18:57:01] yeah, hm, yeah i think you do have to do that [18:57:03] with scap [18:57:03] yeah [18:57:52] sudo? [18:57:53] Sorry, user milimetric is not allowed to execute '/bin/mkdir -p /srv/deployment/analytics/aqs/deploy' as milimetric on tin.eqiad.wmnet [18:58:04] no, it won't let me sudo [18:58:10] should I -u to something ottomata? [18:58:32] nuria, around [18:58:54] Ironholds: ok, about IPS and uniqness [18:59:08] Ironholds: remember, remember? [18:59:18] yes! [18:59:20] hmmm [18:59:26] no sudo [19:00:17] milimetric: moving to releng, that is a q for tyler [19:00:23] k [19:00:29] actually , gimme a sec and i want to try too [19:00:50] Ironholds: did you finish your project on that regard? [19:01:33] Ironholds: I remember you were going to establish uniqness and decide when that uniqnes expired [19:01:59] nuria, ah, did Dario poke you? :D [19:02:22] Ironholds: yes, and i remember we talked about NAT-ing so i wanted to make sure not to duplicate work [19:02:38] Ironholds: I did, Sarah will try and find out a time that works for a check in with a bunch of people [19:02:44] so, I never quite finished it because we didn't have the technical support to get a nuanced answer. But Yuvi and I have just started working on a new project to establish precisely this. Basically we're going to draw up IP/UA pseudo-IDs compared to actual IDs but also be able to split by platform and geographic metadata [19:03:06] (the "technical support" is not a criticism of AnEng - this was back when basically everything was bailing wire, spit and hope :D) [19:04:28] Ironholds: ay ay [19:04:47] Ironholds: ok, so this is basically what leila is going to do [19:04:51] okay! [19:04:55] Ironholds: it is actually a real short project [19:05:14] * leila is reading nuria and Ironholds's discussion. [19:05:23] Ironholds: you can graph uniqness on a desktop dataset and compare it to a mobile dataset with a histogram very fast [19:05:43] Ironholds: shouldn't take more than couple days max [19:06:01] yup, agreed! It's pretty simple for a ballpark figure once you have the desktop collection [19:06:29] Ironholds: for desktop data will have a level of uniqness [19:06:43] Our full project is a bit longer because we plan to split it by more variables and write it up (this problem is interesting to >Wikimedia) but, same basic principle [19:06:47] nuria: I'm reading Ironholds' proposal and they want to use a token to compare numbers. [19:06:47] Ironholds: for mobile not so much but it will depend on countries i bet [19:06:56] yeah, countries and connection types are my guess [19:07:13] and, what leila said. So it sounds like your approach is to use desktop as an unknown baseline and compare to that, which sounds totally sensible. [19:07:22] leila: you can add a token for a true compration but for a 1st stab at it is uneeded [19:07:23] Ironholds: we are not going to use tokens, so it may still make sense for you to do your project in parallel, and we share results frequently with each other so we don't duplicate work. [19:07:30] nuria, agreed! [19:07:50] like I said, the complexity and precision here is because of the writeup goal, not because it's necessary for an approximation [19:08:02] ow, got you Ironholds. [19:08:06] leila, absolutely down! I'd like that :). My end goal is for us to end up with a good idea inside the org and maybe an actual tool. [19:08:25] what idea and tool Ironholds? [19:08:57] Ironholds, leila I would do a 1st stab w/o tokens cause if all buckets are big there is no need to proceed further [19:09:01] like, a good idea of the entropy rate, and the tool: like I would love if there was a tool where you could say "I want to use pseudo-UUIDs lasting N hours" and it would go "okay based on the existing data, you'll have N% unreliable on desktop, N% on mobile..." [19:09:15] and also, just so I don't become a blocker for you, Ironholds: our first priority right now is to get the survey out of the door, and then I'll be in a conference for almost the whole week. So I won't get to this until next week for sure. [19:09:17] the tool is mostly spitballing and may never show up, of course :D. Depends if we find something useful and the science lines up or not [19:09:37] leila, yeah, absolutely. I don't think you're a blocker on the idea; DarTar was more concerned we made sure we weren't duplicating efforts [19:09:47] got you, Ironholds, re the tool. [19:09:52] and it sounds like our conclusion here is "we're not duplicating things but we would totally benefit from each others' work so let's check in as we go"? [19:09:59] which sounds awesome [19:10:16] yup, that's my understanding Ironholds. [19:10:20] :-) [19:10:32] awesome! Okay, I will report as such on the email thread with Dario/Michelle :). Thanks! [19:10:38] Good luck with the survey and I hope CSCW is fun! [19:10:52] ooo, cscw is for Dario. I'm WSDM. [19:11:06] and thanks for the good luck. I really need that today, Ironholds. [19:11:22] leila: hitting the casinos after work again? [19:11:53] mmm :D [19:12:06] leila, good luck at WSDM too! :D [19:12:13] WSDM? [19:12:20] and I'm glad I could help, but I would also say you don't need luck :D [19:12:30] oo yeah, ori. you should totally come to Ashwin's talk on Thursday [19:12:34] let me pull out the schedle [19:12:37] schedule* [19:12:46] Ashwin is awesome [19:12:58] jajaj [19:13:03] okay, this is wsdm: http://www.wsdm-conference.org/2016/ [19:13:04] Ironholds: btw, re: T103017 -- sorry, you're right [19:13:34] ori, no need to apologise! I've totally done that :D [19:13:40] phabricator doesn't make as clear "btw this thing is closed" [19:13:46] everything looks like "a ticket" in UI terms [19:15:07] ori: Mission Bay, third talk in the session that starts at 1:50pm: http://www.wsdm-conference.org/2016/schedule.html [19:15:28] leila: are you presenting? [19:15:46] Ashwin is presenting a joint paper, ori. I'll be a quiet listener. [19:15:59] it's the paper on link recommendation. [19:16:05] ahhhh awesome! [19:16:08] congrats! [19:16:13] sorry for my stupid joke re: casinos! [19:17:07] haha, np, I'm a bit slow these days, ori. :D [19:17:23] good luck, for real! [19:21:49] I mean, link tracing would probably be applicable to card-counting. So if you DO hit the casinos we could handle our annual budget nicely :D [19:28:19] fun fact of the day: after years of debate on the exact spelling of referer vs referrer, the first implementation of the meta referrer policy included a typo in the value, deviating from the W3C recommendations, which got Google and Mozilla to discuss whether or not to support both spellings [19:28:35] Ironholds: I thought you’d love this [19:28:48] DarTar, yep! One of my favourite pieces of trivia :D [19:29:22] People are like "why are you so grumpy all the time". Because my entire job is premised on a typo. You'd be grumpy too. [19:29:23] what I didn’t know is that both Chrome and Firefox implemented an origin-when-crossorigin value and forgot a hyphen [19:29:28] lol [19:29:36] DarTar, and did you know that many of the original unix commands are typos? [19:29:55] the person taking the minutes at the first team meeting had terrible handwriting. No joke. Let me find the article on it. [19:30:12] there’s a sad UN joke somewhere in that line [19:30:55] milimetric: I’ll have something ready for you in 5 [19:31:10] cool [19:31:23] hahah [19:49:30] that looks good DarTar, ship it! [19:49:54] checking what legal and nuria think of it first :) [19:50:24] DarTar: offf? [19:50:33] nuria: mail for you [19:50:37] DarTar: ah k [20:47:15] Analytics, Operations, Ops-Access-Requests, Patch-For-Review: Allow aqs-admins to deploy via scap using deploy-service ssh eky - https://phabricator.wikimedia.org/T127720#2052793 (Ottomata) [20:47:20] Analytics, Operations, Ops-Access-Requests, Patch-For-Review: Allow aqs-admins to deploy via scap using deploy-service ssh key - https://phabricator.wikimedia.org/T127720#2051551 (Ottomata) [20:47:28] Analytics, HyperSwitch, RESTBase, Services, Patch-For-Review: Separate AQS off of RESTBase - https://phabricator.wikimedia.org/T126294#2052798 (Ottomata) [20:47:31] Analytics, Operations, Ops-Access-Requests, Patch-For-Review: Allow aqs-admins to deploy via scap using deploy-service ssh key - https://phabricator.wikimedia.org/T127720#2051551 (Ottomata) Open>Resolved [20:47:51] ottomata: maybe wait on cleanup? [20:48:04] that way we can fall back on something and we don't have to revert puppet etc. [20:48:29] like until it's public and we can confirm that side of it too [20:49:27] Analytics, HyperSwitch, RESTBase, Services, Patch-For-Review: Separate AQS off of RESTBase - https://phabricator.wikimedia.org/T126294#2010298 (Ottomata) FYI Step 8. is sudo rm -rf /etc/restbase/ sudo rm -rf /var/log/restbase/ sudo rm -f /lib/systemd/system/restbase.service sudo updat... [20:49:35] oops [20:49:38] milimetric: too late [20:50:00] milimetric: we'd have to revert anyway, because some configs are managed by puppet [20:50:01] like ferm [20:50:08] ok, cool [20:50:12] hmm, well, its the same port for aqs so maybe it woulda worked [20:50:13] hm [20:50:14] oh well [20:50:24] you can leave it on the other two if you want [20:50:25] ok milimetric repooling [20:50:43] (we don't have to fall back on all 3, it's been responding with just 2 all day so far) [20:51:31] k [20:51:54] I should've checked the dashboard earlier, but it looks fine despite working with just 2 servers: https://grafana.wikimedia.org/dashboard/db/pageviews [20:52:38] Analytics-Tech-community-metrics, Phabricator, User-greg: Closed tickets in Bugzilla migrated without closing event? - https://phabricator.wikimedia.org/T107254#2052816 (greg) a:greg>None [20:52:40] ok [20:52:50] ok depooling aqs1002 [20:52:53] Analytics-Tech-community-metrics, Phabricator: Closed tickets in Bugzilla migrated without closing event? - https://phabricator.wikimedia.org/T107254#1490798 (greg) [20:53:52] running puppet on aqs1002 [20:54:48] Analytics-Kanban, Patch-For-Review: Use a new approach to compute monthly top 1000 articles (brute force probably works) {slug} - https://phabricator.wikimedia.org/T120113#2052826 (Milimetric) [20:55:04] Analytics-Kanban, Patch-For-Review: Pageviews's title parameter is impossible to generate from wiki markup {melc} - https://phabricator.wikimedia.org/T127034#2030266 (Milimetric) [20:58:07] Analytics-EventLogging, Analytics-Kanban, DBA, Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2052839 (Milimetric) [20:58:20] Analytics-Kanban: Correct bug in mobile_apps session metrics job {hawk} - https://phabricator.wikimedia.org/T125960#2052841 (Milimetric) [20:58:36] Analytics-Kanban, Patch-For-Review: Caching on pageview API should be for 1 day - https://phabricator.wikimedia.org/T127214#2035871 (Milimetric) [20:58:44] Analytics-Kanban, Patch-For-Review: Pageview API not dealing with url quoting very well {melc} - https://phabricator.wikimedia.org/T126669#2052849 (Milimetric) [20:58:53] Analytics-Kanban, Patch-For-Review: Improve the data format of the browser report {lama} - https://phabricator.wikimedia.org/T126282#2052853 (Milimetric) [20:58:59] Analytics-Kanban: Communicate the WikimediaBot convention {hawk} - https://phabricator.wikimedia.org/T108599#2052855 (Milimetric) [20:59:08] Analytics-Kanban, Patch-For-Review: Dashiki visualization that shows a hierarchy {lama} - https://phabricator.wikimedia.org/T124296#2052859 (Milimetric) [20:59:18] Analytics-Kanban: Research and validate assumptions for pageview sanitization with Research Team {mole} - https://phabricator.wikimedia.org/T120640#2052863 (Milimetric) [20:59:39] milimetric: yt? [20:59:59]