[00:49:11] Analytics-Wikistats, HTTPS: Fix the mixed content issue on Wikimedia Statistics - https://phabricator.wikimedia.org/T93702#1573838 (Krenair) a:ezachte [04:19:56] Analytics, MediaWiki-Authentication-and-authorization, Reading-Infrastructure-Team: Kill MediaWiki.authmanager.login.*.failure. statsd buckets - https://phabricator.wikimedia.org/T108386#1574064 (Tgr) @ori, could you do this? [04:37:17] Analytics, MediaWiki-Authentication-and-authorization, Reading-Infrastructure-Team: Kill MediaWiki.authmanager.login.*.failure. statsd buckets - https://phabricator.wikimedia.org/T108386#1574082 (ori) Open>Resolved Sure. Done. [04:37:20] Analytics, MediaWiki-Authentication-and-authorization, Reading-Infrastructure-Team, MW-1.26-release, Patch-For-Review: Create dashboard to track key authentication metrics before, during and after AuthManager rollout - https://phabricator.wikimedia.org/T91701#1574084 (ori) [08:04:21] Analytics-Backlog, Analytics-Dashiki, Browser-Support-Firefox: vital-signs doesn't display pageviews graph in Firefox 41, 42 - https://phabricator.wikimedia.org/T109693#1574507 (Nemo_bis) Firefox 39 here (firefox-39.0-8.fc21.x86_64 for the sake of precision). The only error I see in the console is "Ty... [09:39:23] Analytics-Tech-community-metrics, ECT-August-2015, Patch-For-Review: Fine tune "Code Review overview" metrics page in Korma - https://phabricator.wikimedia.org/T97118#1574655 (Nemo_bis) I already gave a better proposal above. > For example, when a code review process starts, by submission of a propose... [10:22:33] Analytics-Tech-community-metrics, ECT-August-2015, Patch-For-Review: Fine tune "Code Review overview" metrics page in Korma - https://phabricator.wikimedia.org/T97118#1574733 (Aklapper) @Nemo_bis: It requires a better word than "Code Review". See and join the conversation in the upstream ticket. Thanks. [11:28:00] halfak: Hi ! [11:28:16] halfak: job finished succesfully [11:30:07] halfak: I'll submit a CR as planned [11:30:35] halfak: Let me know if you wish me to start a job on enwiki (you can also do it if you wish :) [11:42:21] Analytics-Wikistats, HTTPS: Fix the mixed content issue on Wikimedia Statistics - https://phabricator.wikimedia.org/T93702#1574936 (QChris) Open>Resolved Some days ago, I've been told that the issue got fixed in the relevant repo clone. Re-checking the HTML page confirms this: ``` ________________... [11:50:17] (PS1) Joal: Add json revisions sorted per page job [analytics/wikihadoop] - https://gerrit.wikimedia.org/r/233937 (https://phabricator.wikimedia.org/T108684) [13:13:05] joal, cool! Thanks. [13:13:15] halfak: No problem :) [13:13:19] I'll be moving the data to do an enwiki run today. [13:13:24] awesome [13:13:42] halfak: Don't forget to tweak the number of reducers to increase the number of output fiels ! [13:13:56] I ran simplewiki with 10k reducers, took longer but worked fine [13:15:06] halfak: results on altiscale in /user/joal/simplewiki_sortedjson [13:15:41] Yeah, I kinda like the 2.5k we have now. [13:15:48] 10k wouldn't be bad though [13:16:13] I let you decide :) [13:48:54] joal: https://www.flickr.com/photos/ottomatona/20699465268/in/dateposted-public/ [13:49:00] mforns: https://www.flickr.com/photos/ottomatona/20264840314/in/dateposted-public/ [13:49:34] ottomata: Heya ! [13:49:37] Thx :) [13:50:10] joal: fyi, upload finished moving last night, and i started text [13:50:29] I have guessed that, watching rainbows :) [13:50:33] ottomata: --^ [13:50:50] :) [14:01:31] Analytics-Wikistats, HTTPS: Fix the mixed content issue on Wikimedia Statistics - https://phabricator.wikimedia.org/T93702#1575319 (ezachte) @QChris, thanks for your help :-) [14:37:10] Analytics-Backlog: Stats for en.wikinews.org not working - https://phabricator.wikimedia.org/T109146#1575454 (ezachte) automated monitoring: agree, but not a trivial task. I experimented with MoM comparisons and alerts based on thresholds. But we would need to have flexible threshold as smaller wiki have more... [14:49:27] milimetric, yt? [14:56:47] ottomata, thanks for the picture :] [14:57:52] Analytics-Tech-community-metrics, Research consulting, Research-and-Data: Data for audit report - https://phabricator.wikimedia.org/T110067#1575580 (ezachte) Thanks @Qgil > I wonder whether http://korma.wmflabs.org/browser/data_sources.html would serve you better. I took a quick glance, and can se... [15:31:00] sorry mforns, was gone all morning [15:31:08] milimetric, np :] [15:31:10] wanna talk after standup? [15:33:48] Analytics-Kanban: back up data used to crunch numbers - https://phabricator.wikimedia.org/T110255#1575737 (ggellerman) a:Milimetric [15:37:55] Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1575766 (Dicortazar) Thanks for the change! We're updating a bit the process. I'm creating some "blacklists" that will be the projects t... [15:39:10] Analytics-Kanban, Patch-For-Review: Foundation-only Geowiki stopped updating - https://phabricator.wikimedia.org/T106229#1575778 (Milimetric) @Ijon, I fixed this script sometime last week, and it's been catching up. Could you please let us know if the data looks right from your perspective so we can close... [15:39:37] Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1575783 (Dicortazar) btw, change available at https://github.com/Bitergia/mediawiki-repositories/commit/a0f98ec129587f04c94d0a1aa909f73f1... [16:02:10] milimetric: I was asking about permissions, and apparently, https://gerrit.wikimedia.org/r/#/c/180249/ is why we can't move them [16:03:17] Analytics-Backlog: Puppetize dashiki dashboard deployments - https://phabricator.wikimedia.org/T110351#1575870 (Milimetric) NEW [16:38:39] ah, milimetric, no -r needed at all :) [16:38:56] confusing ;) [16:40:00] Analytics-Wikistats, HTTPS: Fix the mixed content issue on Wikimedia Statistics - https://phabricator.wikimedia.org/T93702#1575971 (Chmarkine) [16:45:29] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1575983 (Milimetric) I really like the thinking here. I'd like to propose tracking three numbers on a weekly basis, which I think eliminate the need for lead time. An exampl... [16:46:42] Analytics-Backlog, Analytics-Dashiki, Browser-Support-Firefox: vital-signs doesn't display pageviews graph in Firefox 41, 42 - https://phabricator.wikimedia.org/T109693#1575985 (Milimetric) Thanks Nemo, that's very useful, will try to track down after some other urgent matters. [16:47:31] Analytics-Dashiki, Analytics-Kanban, Browser-Support-Firefox: vital-signs doesn't display pageviews graph in Firefox 41, 42 {crow} [3 pts] - https://phabricator.wikimedia.org/T109693#1575986 (Milimetric) [16:49:51] Analytics-Backlog: Stats for en.wikinews.org not working - https://phabricator.wikimedia.org/T109146#1575991 (Milimetric) I'm thinking lower level like making sure all the data is where we think it should be, all the computation that we expect to happen happens, and so on. I agree that trying to automaticall... [16:52:52] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1575993 (ksmith) As an outside observer, I'll express minor discomfort with the concept of getting "partial credit" just for advancing a task to the next column. My standard m... [17:02:28] Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1576016 (Dicortazar) Hi again, The process keeps going. With the new automatic way to remove repositories, I've detected the following o... [17:05:17] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1576017 (ggellerman) @ksmith Thanks for your input! +1 to the idea that a task is done or it's not done @mforns @milimetric We limit the number of tasks that we allow to... [17:13:49] ottomata: Hey, quick question [17:14:03] stat1002 doesn't have access to the internet ? [17:19:06] joal, it does. You need to use an http proxy though [17:19:34] https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Access#HTTP_Access [17:20:01] Wait... this isn't what I'm looking for [17:20:11] https://wikitech.wikimedia.org/wiki/Http_proxy [17:20:12] There [17:20:17] joal, ^ [17:22:46] thx halfak [17:23:02] I thought I had that configured already :( [17:23:28] cd [17:27:40] joal: in meeting, with you rhostly [17:27:56] milimetric: yt? real quick, do we have a page about how analytics team can take open source contributions? [17:27:58] ottomata: halfak answered, not to worrie [17:28:20] * halfak :D [17:28:22] ottomata: nothing analytics specific, what project for? [17:31:46] joal: can I say at Scrum of Scrums "the cluster is fully recovered after the kafka upgrade? It seems all good, right, no more loss since last week [17:31:49] d'ana is asking, examples are good. how folks could get involved, etc. [17:32:05] milimetric: I think you can say that [17:32:17] joal: thx [17:32:20] ottomata: Do you have recent runs of your hourly stats [17:32:34] ottomata: hm, yeah, we should put up a page [17:32:38] milimetric: I stll ask ottomata in case, but I think it's ok [17:35:08] ottomata: forget my dumb question please #facepalm #tired [17:46:38] milimetric: joal, yes need to make a report about stats [17:46:48] but, joal, hourly stats should all be in webrequest_sequence_stats_hourly now [17:46:49] no? [17:47:05] indeed, that's why #facepalm [17:47:08] ah, heh :) [17:47:22] milimetric: uhm, the migration is on going, but everything is normal since we fixed the snappy bug. [17:47:31] so i don't expect anything else bad to happen [17:47:34] last friday, a bit of loss (mostly on upload topic) --> analytics1021 disk failure ? [17:47:54] ottomata: --^ [17:48:15] mforns: this is sean's last week [17:48:17] possibly, although i wouldn't expect a lot a lot. maybe some if latency went up while leaders rebalanced. [17:48:22] we need to talk to jaime instead [17:48:27] And every now and then, some non-zero values, but with very small percent amounts [17:48:30] we still have issues with vk sometimes. joal on a todo today for me is to look int othat. [17:48:31] yeah [17:48:38] i want reporting on >1% i think, something like that [17:48:43] k [17:49:07] Only for upload on 2015-08-21 19:00 then [17:49:16] I checked for days after 20th [17:51:43] Analytics-Kanban, Patch-For-Review: Foundation-only Geowiki stopped updating - https://phabricator.wikimedia.org/T106229#1576228 (HaithamS) Looks great to me, thank you Dan! [17:57:51] joal: are you gonna work on https://phabricator.wikimedia.org/T109547 soon? :) [17:58:11] Not as soon as I'd like [17:58:19] madhuvishy: You can claim it if you want :) [17:58:36] thanks! I've been feeling a bit jobless [17:58:50] :) [17:59:47] joal: have you seen the last comments on it? [17:59:52] nope [17:59:56] I don't think [18:00:18] Do you have 5 minutes to chat about it? I wanna make sure I approach it right [18:01:22] madhuvishy: sure [18:01:24] cave ? [18:01:31] joal: omw [18:18:05] Analytics-Kanban, RESTBase: Create a metric for overall RESTBase request rates from Varnish logs {hawk} [13 pts] - https://phabricator.wikimedia.org/T109547#1576375 (madhuvishy) a:madhuvishy [18:37:49] ottomata: Weird, I don't see my spark job in the scheduler :( [18:40:21] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1576466 (mforns) @ksmith @ggellerman I understand your concerns. You're right, the "it's X% complete", most of the time, turns out to be wrong :] However, considering we're u... [18:44:17] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1576501 (ksmith) @mforms: Actually I would argue that # tasks completed is exactly appropriate in Kanban. Showing fewer tasks completed in a week could help point to problems... [18:46:35] joal: ehhh? [18:49:00] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1576528 (ggellerman) @mforns Thanks for feedback! Let's talk about this more in tomorrow's retro [18:58:42] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1576567 (mforns) @ksmith @ggellerman > Actually I would argue that # tasks completed is exactly appropriate in Kanban. Showing fewer tasks completed in a week could help point... [19:32:23] joal: stumbled over your lightning talk slides on pageviews the other day - very useful https://docs.google.com/presentation/d/1ucbBhkWa6uigUq7NJH2LZBM5jklTAA-TciaUvyhWfJM/edit ... [19:32:49] ... how about uploading them to commons so that they can e.g be embeded in the documentaiton pages on wikitech? [19:33:27] i'd volunteer to transfer it myself this time if that works for you [19:38:30] Hi HaeB [19:38:57] Please upload those slides to commmons :) [19:39:36] It's a great idea, thank you for offering that HaeB ! [19:46:20] Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1576749 (Aklapper) p:Normal>High [19:48:08] Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1347475 (Aklapper) [19:50:57] ottomata: I don't get it [19:52:22] ottomata: In fact I get it :) [19:52:28] ottomata: nevermind [19:52:48] Will go to bed, too many mistakes tonight [19:52:58] have a good end of day a-team ! [19:53:16] ciao joal, see you tmrw! [19:53:31] ha, goodnight! [19:53:51] ottomata: I forgot to run the stuff in yarn mode #facepalm-bis [19:54:58] so long, nite! [19:56:03] joal|night: thanks! done. this abotu right? https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive [19:56:21] I imagined Joseph walking out the door as he throws an overcoat on his shoulders and flips on a really classy fedora, we're all smoking cigars and typing on typewriters as the camera focuses on "The Office of Analytics" [19:57:40] milimetric: :D [20:00:30] HaeB: That seems right :) [20:00:50] HaeB --> Not sure what you wanted me to double check, but the page looks good [20:06:59] joal|night: just the caption ("within the Wikimedia Foundation's pageview data pipeline") [20:07:02] good night! [20:10:10] HaeB: yes indeed, looks good :) [20:10:46] Hi, any ideas about the following eventLogging schema validation errors for this schema: https://meta.wikimedia.org/wiki/Schema:CentralNoticeBannerHistory [20:11:10] Analytics-Kanban: Write scripts to track cycle time of tasked tickets and velocity [8 pts] - https://phabricator.wikimedia.org/T108209#1576827 (ggellerman) @ksmith Well done! @mforns There is no partial credit- tasks are done or not done, but I do see value in measuring context switches that occur during t... [20:11:17] 2015-08-26 19:19:43,248 (MainThread) Unable to validate: ?%7B%22event%22%3A%7B%22version%22%3A1%2C%22action%22%3A%22saveIntent%22%2C%22editor%22%3A%22visualeditor%22%2C%22platform%22%3A%22desktop%22%2C%22integration%22%3A%22page%22%2C%22page.id%22%3A12362%2C%22page.title%22%3A%22User%3ARYasmeen_(WMF)%22%2C%22page.ns%22%3A2%2C%22page.revid%22%3A257516%2C%22page.length%22%3A-1%2C%22editingSessionId%22%3A% [20:11:17] 22463554047e770a1d%22%2C%22user.id%22%3A1348%2C%22user.editCount%22%3A7187%2C%22mediawiki.version%22%3A%221.26alpha%22%2C%22action.saveIntent.timing%22%3Anull%7D%2C%22revision%22%3A11448630%2C%22schema%22%3A%22Edit%22%2C%22webHost%22%3A%22en.m.wikipedia.beta.wmflabs.org%22%2C%22wiki%22%3A%22enwiki%22%7D; deployment-cache-text04.deployment-prep.eqiad.wmflabs 18805 2015-08-26T19:19:42 198.73.209.5 Mozilla/5.0 ( [20:11:17] Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0 [20:11:17] (None is not of type u'integer') [20:11:17] 2015-08-26 19:20:19,867 (MainThread) Unable to validate: ?%7B%22event%22%3A%7B%22version%22%3A1%2C%22action%22%3A%22saveIntent%22%2C%22editor%22%3A%22visualeditor%22%2C%22platform%22%3A%22desktop%22%2C%22integration%22%3A%22page%22%2C%22page.id%22%3A12362%2C%22page.title%22%3A%22User%3ARYasmeen_(WMF)%22%2C%22page.ns%22%3A2%2C%22page.revid%22%3A257517%2C%22page.length%22%3A-1%2C%22editingSessionId%22%3A% [20:11:17] 22a6306920e93c26a5%22%2C%22user.id%22%3A1348%2C%22user.editCount%22%3A7188%2C%22mediawiki.version%22%3A%221.26alpha%22%2C%22action.saveIntent.timing%22%3Anull%7D%2C%22revision%22%3A11448630%2C%22schema%22%3A%22Edit%22%2C%22webHost%22%3A%22en.m.wikipedia.beta.wmflabs.org%22%2C%22wiki%22%3A%22enwiki%22%7D; deployment-cache-text04.deployment-prep.eqiad.wmflabs 18810 2015-08-26T19:20:18 198.73.209.5 Mozilla/5.0 ( [20:11:19] Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0 [20:11:21] (None is not of type u'integer') [20:11:23] 2015-08-26 19:48:23,159 (MainThread) Unable to process: ?%7B%22event%22%3A%7B%22r%22%3A1%2C%22n%22%3A2%2C%22l%22%3A%5B%7B%22t%22%3A1440607735%2C%22s%22%3A%226%22%2C%22b%22%3A%22CN_banner_history_test_banner%22%7D%2C%7B%22t%22%3A1440618506%2C%22s%22%3A%226%22%2C%22b%22%3A%22CN_banner_history_test_banner%22%7D%5D%7D%2C%22revision%22%3A13172419%2C%22schema%22%3A%22CentralNoticeBannerHistory%22%2C%22webHost%22%3A%22en [20:11:25] .wikipedia.beta.wmflabs.org%22%2C%22wiki%22%3A%22enwiki%22%7D; deployment-cache-text04.deployment-prep.eqiad.wmflabs 18828 2015-08-26T19:48:12 184.161.137.172 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36 [20:11:28] (u'' is not of type u'boolean') [20:11:30] 2015-08-26 19:50:13,672 (MainThread) Unable to process: ?%7B%22event%22%3A%7B%22r%22%3A1%2C%22n%22%3A3%2C%22l%22%3A%5B%7B%22t%22%3A1440607735%2C%22s%22%3A%226%22%2C%22b%22%3A%22CN_banner_history_test_banner%22%7D%2C%7B%22t%22%3A1440618506%2C%22s%22%3A%226%22%2C%22b%22%3A%22CN_banner_history_test_banner%22%7D%2C%7B%22t%22%3A1440618609%2C%22s%22%3A%226%22%2C%22b%22%3A%22CN_banner_history_test_banner%22%7D%5D%7D%2C% [20:11:32] 22revision%22%3A13172419%2C%22schema%22%3A%22CentralNoticeBannerHistory%22%2C%22webHost%22%3A%22en.wikipedia.beta.wmflabs.org%22%2C%22wiki%22%3A%22enwiki%22%7D; deployment-cache-text04.deployment-prep.eqiad.wmflabs 18840 2015-08-26T19:50:07 184.161.137.172 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36 [20:11:34] (u'' is not of type u'boolean') [20:11:41] oops, lemme do pastebin, sorry!! [20:12:29] https://tools.wmflabs.org/paste/view/11e5c65c [20:13:41] The last one decodes to this: {"event":{"r":1,"n":3,"l":[{"t":1440607735,"s":"6","b":"CN_banner_history_test_banner"},{"t":1440618506,"s":"6","b":"CN_banner_history_test_banner"},{"t":1440618609,"s":"6","b":"CN_banner_history_test_banner"}]},"revision":13172419,"schema":"CentralNoticeBannerHistory","webHost":"en.wikipedia.beta.wmflabs.org","wiki":"enwiki"} [20:14:11] Again the schema: https://meta.wikimedia.org/wiki/Schema:CentralNoticeBannerHistory [20:14:54] milimetric: ottomata: hi! ^ [20:15:52] hiya! [20:16:21] hi AndyRussG [20:17:53] (looking) [20:18:01] thx! [20:22:57] Analytics-EventLogging, Analytics-Kanban: Update Schema Talk pages {tick} [8 pts] - https://phabricator.wikimedia.org/T103133#1576882 (mforns) @madhuvishy I went through the schema talk pages and they are awesome. I saw you transformed all schema owner's names into links and also normalized the "purging"... [20:23:45] AndyRussG: I don't see a problem with that event right away [20:24:29] the error message it gives you makes no sense [20:24:40] because there aren't even booleans in your schema [20:25:36] milimetric: yeah hence my confusion... [20:26:35] AndyRussG: but the first couple you pasted are not your events, those are from the Edit schema [20:27:22] ottomata, do you know where the code is for https://phabricator.wikimedia.org/T108684 's code review? [20:27:58] AndyRussG: sadly I'm a little rushed to do something high priority at the moment. If you want to look further into it you could run the event logging validation code on it by pip-installing the eventlogging server and running the eventlogging_processor with stdin:// as input and stdout:// as output [20:28:09] then you can just paste it your event and see more of what's wrong [20:28:25] sorry that's all I can offer now, but if you're stuck file a bug with us (tag it Analytics-Backlog) [20:28:32] milimetric: np, many thanks 4 looking and 4 the tip! :) [20:28:46] tsk tsk joal|night and his missing Bug: T** [20:28:56] oh, no it sthere. [20:29:03] huh slow jenkins? [20:29:04] iunno :) [20:29:07] mforns: https://gerrit.wikimedia.org/r/#/c/233937/ [20:29:55] ottomata, thanks! there were two of them and I wasn't sure [20:30:59] didn't see the link to the task :/ [20:32:31] AndyRussG: do you have other events that validate with this schema? [20:32:59] ottomata: apparently none so far. It's only on the beta cluster for now! [20:33:24] aye [20:33:52] ottomata: you should be able to trigger them like this: http://en.wikipedia.beta.wmflabs.org/wiki/Main_Page?country=AQ&bannerLoggerRate=1 [20:34:26] (We're rolling out to Antarctica first, 'cause cold log entries are stay fresh longer.) [20:35:21] ottomata: maybe I did something wrong in the nested schema defs? [20:37:41] No errors in debug mode or anything [20:38:10] hm, yeah, AndyRussG it seems to be a schema problem, not a validatino problem, pasting stacktrace [20:38:39] https://tools.wmflabs.org/paste/view/f87f8795 [20:39:43] not sure why though [20:40:28] ottomata: hmmmmm! OK that's a start... Yeah since it validates on-wiki, I assumed it was all good [20:40:53] yeah that is funky [20:40:54] P.S. Where did u get that backtrace? [20:41:03] i loaded up eventlogging code in ipython [20:41:06] and tried to parse your event [20:41:27] cd EventLogging/server [20:41:32] export PYTHONPATH=. [20:41:35] ipython [20:41:41] from eventlogging import (capsule_uuid, create_event_error, LogParser, [20:41:41] ...: get_reader, get_writer, validate, setup_logging, [20:41:41] ...: uri_force_raw, uri_append_query_items) [20:41:49] parser = LogParser('%q %{recvFrom}s %{seqId}d %t %h %{userAgent}i') [20:41:54] event = parser.parse(e) [20:42:02] validate(event) [20:42:12] (e is your raw event string) [20:43:39] ottomata: cool! [20:50:55] ottomata: from the stacktrace, maybe it was complaining about the lack of the "required" property for a subschema? I just added it, now on version 13332080 [20:51:01] https://meta.wikimedia.org/wiki/Schema:CentralNoticeBannerHistory [20:53:42] hm, just tried it by manually changing revision in raw string [20:53:44] samw thing [20:53:46] same thing [20:54:11] but maybe I did that wrong [20:55:54] ja, it is just the schema smoehow AndyRussGweird [20:56:10] get_schema(('CentralNoticeBannerHistory', 13332080), True) [20:56:15] causes the same stacktrace [20:56:24] huh [20:56:47] Failed validating u'type' in schema[u'properties'][u'properties'][u'additionalProperties'][u'properties'][u'items'][u'items'][u'properties'][u'properties'][u'additionalProperties'][u'properties'][u'required']: [20:56:47] {u'default': False, u'type': u'boolean'} [20:56:47] On instance[u'properties'][u'l'][u'items'][0][u'properties'][u's'][u'required']: [20:56:47] u'' [20:57:12] yeah it looks like it is saying that the value it has for 'required': is '' [20:57:18] and not boolean [20:59:02] huh [20:59:02] yeah [20:59:33] i get an empty string when i get the json from that url [20:59:57] AndyRussG: [20:59:58] https://meta.wikimedia.org/w/api.php?action=jsonschema&title=CentralNoticeBannerHistory&revid=13332080 [21:00:11] "s":{"type":"string","required":"" [21:00:11] ... [21:00:35] daaas weird [21:00:56] oooh [21:01:47] yeah very weird [21:04:53] same thing for the t property [21:05:07] Something weird about the parsing maybe? [21:09:36] ottomata: since I do have El on my local wiki, I am setup to debug that at least! [21:13:16] nice! :) [21:16:07] ottomata: yeah! I am getting the same weirdness ("required": "") locally, BTW [21:18:29] sounds like abuuuug [21:30:23] wut? no way, EL doesn't get those! [21:30:42] Just the way sharks never get cancer! [21:38:42] ottomata: I think I fixed it in revision 13332718 [21:39:42] seemed to work! [21:40:31] ottomata: fantastic! thx so much! I'm gonna push the new revid to master, see if it works now on betacluster [21:59:58] AndyRussG / ottomata: this sounds like a reversion of the fix the mediawiki folks put in place for the API change [22:00:26] we had this same problem before, I had forgotten [22:00:47] but it's nothing to do with us, and the schema shouldn't have to be changed to accomodate, this means that everyone's events will start being invalid if this change is pushed [22:00:52] (from beta to prod) [22:00:59] so we must file an urgent blocker [22:02:04] oooooooh!!! I know what's wrong :) [22:02:17] the fix they did doesn't fix the nested "require" properties [22:02:20] only the top level ones [22:02:39] phew, all good but AndyRussG: it's a bug with the mediawiki api essentially [22:02:52] I gotta run out for a bit but if you guys don't know what I mean, cc me on the bug and I'll explain [22:03:17] this is the key link, that shows how the require property is fetched properly at the top level but improperly when nested: https://meta.wikimedia.org/w/api.php?action=jsonschema&title=CentralNoticeBannerHistory&revid=13332080 [22:04:32] milimetric: heh interesting... I'm just starting an hour-long meeting, back in a bit :) [22:05:03] AndyRussG: it's ok bryan davis is adding it to phab [22:05:10] (check #mediawiki-core) [22:05:15] bbl [22:05:16] milimetric: ah great! thx!!! :) [22:05:22] ca [22:05:23] cya [22:09:30] Analytics-Backlog, Analytics-Dashiki, Editing-Analysis, VisualEditor: Start generating a visual editor adoption metric - https://phabricator.wikimedia.org/T109158#1577289 (Jdforrester-WMF) [22:10:04] Analytics-EventLogging, MediaWiki-API: Mediawiki API is returning empty strings for 'required' boolean fields - https://phabricator.wikimedia.org/T97487#1577300 (bd808) Resolved>Open Reopening following reports from @milimetric that the fix doesn't work for nested elements as shown by https://meta.... [22:38:45] Analytics-EventLogging, Analytics-Kanban: Update Schema Talk pages {tick} [8 pts] - https://phabricator.wikimedia.org/T103133#1383127 (madhuvishy) Thanks @mforns :) [23:26:47] good night team! see ya [23:36:12] Analytics-Tech-community-metrics, ECT-September-2015: Patches with Verified -1 should not be counted as open in our code review metrics - https://phabricator.wikimedia.org/T108507#1577735 (Aklapper) > Just wondering, is there a Verified -2 value? Depends on each repo's configuration but [[ https://git.wi... [23:46:05] ottomata: still around? do you know if we can directly send metrics to graphite without statsd?