[07:17:36] Never4ever [12:44:24] Lydia_WMDE: hey, do you have a min? [12:44:39] Amir1: hey [12:44:42] wasup? :) [12:45:00] okay :) [12:45:02] http://mw-revscoring.wmflabs.org/wiki/Special:RecentChanges [12:45:04] see this [12:45:08] looking [12:45:28] there are several "r"s [12:45:36] which means these edits need review [12:45:49] wooooooooooooooooooo [12:45:50] :D [12:45:54] extension? [12:45:59] yeah [12:46:03] and and now if possible make an account [12:46:03] sweeeeet! [12:46:09] ok [12:46:35] go to your preferences, tab "Recent changes" [12:46:35] done [12:46:39] ok [12:46:52] you can see there is a "ores" section [12:46:59] yes [12:47:07] you can change the threshold to "soft" [12:47:16] so more edits will be marked as "r" [12:47:17] well it shows message keys still but that's ok [12:47:18]

[12:47:21] for example [12:47:22] yeah [12:47:24] ok [12:47:32] I'm adding the messages [12:47:38] I just wanted to show you [12:47:46] it's not merged, the patch is WIP [12:47:53] :) [12:47:59] this is awesome [12:48:05] and if you patrol an edit, it won't be marked as "r" [12:48:23] two comments: maybe make the r a bit more visible by giving it a more in-your-face color? [12:48:50] that should be easy to do [12:49:03] * Amir1 noted [12:49:04] and i would have expected soft to give me less revisions marked as potential vandalism. so maybe need to find another word for the preference [12:49:28] for this one [12:49:40] everything is configurable in LocalSettings [12:49:46] even number of options [12:49:50] :) [12:49:57] what any one option mean, etc. [12:49:58] this is very nice [12:50:02] ok [12:50:10] we are requesting security review [12:50:19] it's scheduled [12:50:24] \o/ [12:50:29] some more patches to come and we are ready to go [12:50:56] yay [12:51:05] I've done some other stuff too [12:51:23] like this one: https://tools.wmflabs.org/dexbot/scorer.html [12:51:39] (it'll be in the ores server any time soon) [12:52:15] looking [12:57:17] oh I forgot to add, the same "r" works in watchlist too [12:57:38] our plan is to add it to user contribs [13:03:08] Lydia_WMDE, what's the status of commons file--wikidata connection? [13:04:17] yurik: working on it but big project - similar to wikidata itself really [13:04:22] so taking time [13:05:15] Lydia_WMDE, are you setting up a separate wikibase instance for files? Because it would be good to simply have a wikidata item for each file in commons, and store some info about it in wikidata [13:05:22] any ETA? [13:05:44] hey, yurik: three things about graph extension: 1- it's awesome, i've been using it in fa.wp 2- it seems it has problem with permanent links (when you read an article in permanent link it doesn't work) 3- legend in pie charts in RTL langs are LTR (I can show it to you) [13:05:58] we won't do that as that will cause all kinds of other technical and social issues [13:06:34] Lydia_WMDE, "do that" ? separate or joined? [13:06:55] putting image data into wikidata [13:07:19] Amir1, thx, would love to work with you on it - I spoke with Jeff (main Vega guy) about RTL, and send him Moriell's site [13:08:35] Lydia_WMDE, oh? Wouldn't something like GLAM metadata be stored there? Like if its picture of a famous painting, store the author of that painting there? [13:09:25] there is a difference between the painting as an entity and the scan of it on commons [13:09:29] they have different data [13:11:18] Lydia_WMDE, that's fine, so the scan could have a separate item, right? [13:11:24] linking to the original [13:11:46] yes [13:12:05] but as i said that is not how we're going to do it because of technical and social issues [13:12:15] sorry eating atm s won't go into the nitty gritty details [13:20:12] hi :) [13:21:25] so, we're trying to investigate these jumps in the rate of htmlCacheUpdate jobs: [13:21:28] http://graphite.wikimedia.org/render/?width=586&height=308&_salt=1453466175.547&target=MediaWiki.jobqueue.inserts_actual.htmlCacheUpdate.rate&from=-80days [13:21:53] apparently when those jumps starts happening in December, it started causing backlogging and queueing in job runners [13:22:20] recently someone fixed the jobrunner backlogging issue, which then just pushed the problem down to varnish (as these all general html page purges as well) [13:22:50] which then creates this gigantic rate spike we're seeing in PURGE traffic here: [13:22:53] https://grafana.wikimedia.org/dashboard/db/varnish-aggregate-client-status-codes?from=1453148402001&to=1453463631092&var-site=All&var-cache_type=All&var-status_type=1&var-status_type=2&var-status_type=3&var-status_type=4&var-status_type=5&theme=dark [13:23:32] we're backtracking through "what's causing the big htmlCacheUpdate rate increases in that first graph", and it looks like some kind of wikidata updates driving it [13:23:45] which then of course invalidates pages across all kinds of other wikis that reference wikidata... [13:24:40] as in, likely htmlCacheUpdates being spawned by https://github.com/wikimedia/mediawiki-extensions-Wikidata/blob/master/extensions/Wikibase/repo/maintenance/dispatchChanges.php#L127 [13:24:48] cronjobs that run on terbium, etc... [13:24:53] aude: ^ thoughts? [13:25:16] Lydia_WMDE: DanielK_WMDE__ ^^ too [13:26:11] bblack: is this related to a particular client (wikipedia)? [13:26:49] tbh I'm not even sure what "client" means in this context [13:26:59] Wikidata is the repo [13:27:00] bblack: in general we need the cron jobs to do page purging and notifying people on wikipedia about changes happening related to their articles on wikidata [13:27:03] Other wikis are a client [13:27:20] they show up in their watchlists etc [13:27:28] right [13:28:05] 12:49 < ori> bblack: (wrapping up the purge recap) the increase seems to have happened in stages: on 12/04 we went from ~5/s to ~7.5s (assuming i am using the right units); on 12/11 it went to ~20/s, and on the 20th we went to ~27/s [13:28:05] for more investigation hoo, DanielK_WMDE__, aude and maybe addshore can be of more help [13:28:22] basically I'm wondering if those dates sound familiar [13:28:45] not particularly but who knows :D [13:28:47] something happened on those days that resulted in a lot more net purges through this mechanism with htmlCacheUpdate, and likely it's wikidata related [13:28:54] *nod* [13:29:14] let me check one thing [13:29:17] sec [13:29:51] (I should add: this is to the point where we're actually invalidating pages at a rate/second higher than they're being viewed, globally) [13:31:22] Oo [13:31:27] well, that's a tricky thing to quantify I guess, since they get multiplied across clusters. it looks that way in graphs, but isn't really true heh [13:31:41] in any case, it's something like 5x more purge traffic than we had before [13:31:49] hmm ok no the graph i was thinking of doesn't go back far enough [13:32:10] so daniel, katie, marius or adam need to have a look [13:32:18] ok [13:38:16] Interesting... [13:39:34] seems the populateChanges triggers too many cdnUpdate and htmlCacheUpdates changes :/ [13:42:31] it could be that this isn't really a code bug, but more like the fallout of reality: that it's natural for lots of articles to ref lots of wikidata, and it's natural for lots of wikidata to change often, and recently all of that's starting to happen a lot more [13:42:50] I don't really understand most of the details though [13:44:10] so it started happening around the 8th of dec? [13:45:30] well there's 3 bumps up in the graph, one somewhere around Dec 4, one somewhere around Dec 11, and then another somewhere around Jan 20 [13:46:05] which takes us from a rate (in some unknown units) of "5" before the first bump to ~27 after the last [13:46:50] we're still not 100% sure it's even wikidata that's driving this, but circumstantial evidence seems to lean that way [13:47:33] mostly I was hoping someone in here would go "oh yeah those dates sound familiar, we started doing this new thing then" or something :) [13:51:33] it might be some wikis migrating templates used on a lot of pages [13:51:36] but pure guess tbh [13:51:46] that would likely trigger more edits [13:52:33] we've seen spikes from a template change before though, usually those die down after everything related gets purged [13:52:43] these are persistent rate increases that never really go away [13:52:58] *nod* [13:53:13] well [13:53:23] if one of those often used templates gets migrated to wikidata [13:53:38] then we'd purge those pages more often when edits are made to the items relating to them on wikidata [13:53:53] so you would likely see more purges also continuously [13:53:59] unlike a one-time change to a template [13:55:02] hmmmm [14:53:30] Lydia_WMDE, could you point me to the relevant discussion about commons? I would love to read up on the social & technical issues you are referring to [15:19:04] hi Lydia_WMDE, do you know what's the status on https://phabricator.wikimedia.org/T67397 ... I had not time to take care of that after returning from SFO [16:02:14] is something up with test.wikidata.org? [16:04:04] Anyone know if ListeriaBot will be up again soon? Throws LoginError at the momement. I reported it to Magnus this morning. [16:05:39] tarrow: Such as? [16:05:51] Reedy: Exception encountered, of type "BadMethodCallException" [16:05:56] is all I get [16:06:00] but perhaps it's me [16:06:04] Doing what? [16:06:12] Ooh [16:06:12] I see [16:06:12] going to test.wikidata.org in chrome [16:07:13] :/ [16:07:19] interesting [16:07:31] I see loads of those in the exception logs [16:07:32] but not for testwikidatawiki [16:07:34] * Reedy files some bugs [16:07:35] there's some ongoing issues with login/centralauth in general with the .11 release [16:07:37] :/ [16:07:47] it's been discussed in -operations since yesterday [16:08:17] trying to find a good overall link for that [16:08:20] Isn't this something different? [16:08:30] oddly it seemed to be working for me fine 10m ago [16:09:12] no idea [16:10:10] https://phabricator.wikimedia.org/T124252 is the NeedToken bit [16:10:23] I think there are other tickets related to login + .11, maybe [16:10:24] Just filed bugs for 3 other BadMethodCallExceptions [16:13:10] looks like they started happenign at 15:50ish [16:13:10] Is it served only by mw1017? [16:13:24] If so, could be Brad debugging [16:13:25] also Call to a member function getPriority() on a non-object (NULL) Reedy [16:14:11] addshore: where? [16:14:22] reedy@fluorine:/a/mw-log$ grep getPriority exception.log [16:14:22] reedy@fluorine:/a/mw-log$ grep getPriority fatal.log [16:14:22] reedy@fluorine:/a/mw-log$ [16:14:24] testwikidatawiki, thats the one I see the most infact [16:14:32] ahh, im looking in logstash [16:15:07] well, dump the stack trace on a bug somewhere :P [16:15:14] xD [16:16:34] Oh, wait [16:16:37] 2016-01-22 14:32:39 mw1017 testwikidatawiki fatal ERROR: [d29ecba7] PHP Fatal Error: Object does not implement ArrayAccess {"exception_id":"d29ecba7"} [16:16:47] 2016-01-22 14:32:40 mw1017 testwikidatawiki fatal ERROR: [9aca16dc] PHP Fatal Error: File not found: /srv/mediawiki/php-1.27.0-wmf.11/extensions/CentralAuth/includes/session/CentralAuthSessionProvider.php {"exception_id":"9aca16dc"} [16:17:14] 2016-01-22 14:31:26 mw1017 testwikidatawiki fatal ERROR: [d54abaed] PHP Fatal Error: Nesting level too deep - recursive dependency? {"exception_id":"d54abaed"} [16:19:21] Reedy: https://phabricator.wikimedia.org/T124433 was the one I saw [16:19:57] What's the latest timestamp? [16:20:39] well I had 2016-01-22T16:16:34.000Z [16:20:43] *refereshesss* [16:21:09] 2016-01-22T16:20:45.000Z [16:21:30] https://usercontent.irccloud-cdn.com/file/UoyxkqP2/ [16:21:42] Yup [16:21:59] Jenkins just V-1'd anomies patch for the same reason :) [16:22:25] tarrow: cheers for the heads up :) [16:22:53] It's fixed now [16:23:04] Reedy: no problem. It was entirely self motivated: I wanted to test a script on it and couldn't :P [16:23:10] awesome! thanks! [17:24:33] Amir1: ping [17:25:13] regarding https://phabricator.wikimedia.org/T123795 [17:25:52] hey [17:25:54] :) [17:26:24] hoo: your input would be great [17:27:44] Amir1: I'm happy to help [17:27:54] but I need to know what exactly the script is supposed to do [17:28:02] Populate the `ores_classification` table? [17:28:30] hoo: yes, from rc table [17:29:36] So you want to have one row in that table for revisions that are in the rc tables [17:29:59] It should run Cache function in includes/Cache.php (in ORES extension) [17:30:20] and it probably make several row per a row in rc [17:30:39] (because there are different models, and different values for each of them per edit) [17:30:44] makes sense [17:31:08] Already wondered why ores_rev isn't a primary key [17:31:14] The biggest issue ATM is Cache does this one by one but when populating we should get scores in batches [17:31:53] That should be easy to do [17:32:05] just collect the arrays there up to maybe 500 [17:32:09] and then insert them at once [17:33:00] Did you have a performance database scheme review, yet? [17:33:09] performance or database scheme [17:35:02] Amir1: ^ [17:35:12] I see a few things that could probably be improved [17:35:22] Doing that post deploy is very painful and performance matters here [17:36:04] that would be great [17:36:26] I don't know if it's have been done [17:36:28] I'll create a task with suggestion in a bit [17:36:36] and make it a blocker for the maint. one [17:36:37] awesome :) [17:36:39] thanks [17:36:46] hi! this weird thing appened: I edited and created entities through the API (wbeditentity, wbcreateclaim) passing tokens obtained from my credentials as usually, but this time my edits where considered edits from an anonymous user https://www.wikidata.org/wiki/Special:Contributions/89.12.97.161 which is super weird as I don't see how I could be logged out while doing edits requireing a token oO any idea what happen? [17:37:31] Amir1: If you have anything else regarding binding against MediaWiki, performance or other "fun stuff", feel free to ping me [17:37:50] sure :) [17:38:03] maxlath: I guess this is due to the current authentication problems [17:38:10] hoo: some code review would be awesome here [17:38:12] all of Wikimedia is affected and it is being worked on [17:38:23] if you work withmw hooks [17:38:24] add me to patches :) [17:38:26] *with mw [17:38:49] hoo: ok, thanks :) [17:39:11] Amir1: I do :) [17:39:25] awesome :) [17:42:27] maxlath in case you really don't want your ip to show up, you can always add the "assert=user" param to your wb* edit api request :) [17:50:24] Alphos: I don't really mind the Freifunk IP to appear, it's more about the ability to go back through the edits I do, so yes, I will try assert=user, thanks! [17:52:04] maxlath and if you expect to have a bot flag, obviously assert=bot ;) [17:58:29] I'm looking for an admin [17:58:59] yes? [17:59:15] Alphos: I'm not a bot, I'm a well tooled human ^^ [18:01:23] maxlath uuuuh, i'm gonna assume "TMI" :D [18:03:29] Amir1: Ticket opened (screwed a bit with phabricator because I opened it as subtask of the wrong task... doh). [18:03:31] Will fix that [18:03:59] thanks :) [18:10:12] Alphos: damned, it didn't sounded like that in my mind, how do you say anything in this perverced world :p [18:10:52] maxlath with me you don't, i'll always find the pervy way of understanding what you say [18:18:55] hey Amir1 [18:19:40] hey, multichill [18:19:51] :) [18:19:55] how are you? [18:20:10] I hope you're well :) [18:24:19] Sure, just anoyed by the nice session mess [18:26:13] Amir1: Did you see that? Someone implemented new session handling and suddenly I was editing under someone else his/her account [18:27:07] wow [18:27:21] that's interesting :) [18:30:25] hmm, keep getting logged out again :( [18:33:05] aude: FYI: I scheduled a branch for Monday in the calendar [18:33:09] guess that's ok with you [18:33:20] I'll probably be able to be around next week [18:33:27] ok [18:34:11] hopefully have power again by monday [18:34:21] oO blackout? [18:34:22] but then not around on tuesday and not sure about wednesday [18:34:26] likely [18:35:51] Ok, good to know [18:36:18] I'm leaving for today... cu [18:36:22] * aude workign on https://www.wikidata.org/wiki/Q22222608 :) [18:36:38] hehe :) [18:36:58] So close to Q22222222 [18:37:20] and i guess not really working so much today [18:38:54] the storm is named after jonas :) [18:39:12] aude: Related to that I filed https://phabricator.wikimedia.org/T124451 [18:40:08] thanks multichill [18:40:12] agree, privacy issue [18:40:14] multichill: Wanted to do that for quite some time... but a little hard to get right re dependency injection [18:40:36] But might be easier now that we use a factory method for api objects [18:40:55] anyway... bye [18:40:59] multichill: not just wikidata, but any edits [18:41:06] hoo: bye! [18:41:24] aude: Yeah, but regular edits don't use fancy javascript+api [18:41:34] And I would get a token error [18:41:40] multichill: well, there's visual editor etc. [18:41:45] but see your point [18:41:53] Yup, we have another bug that. I'll link it :-) [18:42:25] * aude got logged out yesterday when going from say, en.wiktionary to cs.wiktionary [18:42:55] because i didn't have an account on cswiki and handling of autocreating/attaching accounts was broken [18:43:32] Haha, this is a shitty editing week for me. Every night I run into another bug [18:44:09] The hmm-why-am-I-editing-under-someone-else-his-account bug was kind of funny [18:44:11] yeah :( [18:44:35] i know they are trying to make the authentication stuff better and in the end it should be [18:44:42] but frustrating, yeah [18:45:21] I know, but this gives the idea that the work is sloppy [18:48:25] :/ [18:48:35] aude: https://phabricator.wikimedia.org/T124440 is a cute [18:48:47] How the hell do I add the security tag to this bugs? legoktm ? [18:49:01] i saw that [18:49:14] multichill: i think set the security field [18:49:19] it's a separate field [18:49:21] multichill: edit task -> Security -> Software security bug [18:49:25] Bug also a tag [18:49:49] is there something new? :S [18:53:29] harej: have you seen https://phabricator.wikimedia.org/T120451? #3 in daniel's comment and the last comment reminded me of how you (if I got the right person :)) were talking a couple of months ago about having items for categories to add semantic information about them [18:53:30] multichill: ? [18:53:43] I don't know [18:53:48] That's phab magic I don't understand [18:55:20] link to the bug? [18:56:50] Ok. Screw this shit, tired of having to log in, I'm going to do something else