[02:40:20] 06Multimedia, 10Analytics, 06Commons, 10Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2226202 (10Yurik) Juts wrote in-depth implementation details at https://www.mediawiki.org/wiki/Extension:J... [02:53:24] 06Multimedia, 10Analytics, 06Commons, 10Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2226211 (10Yurik) @brion, could you think of use cases for partial data reads? I think it will be mostly "... [03:07:26] 06Multimedia, 10Analytics, 06Commons, 10Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2226245 (10brion) Pulling individual data items out of large lists; pulling relevant columns in order to s... [04:53:39] (03CR) 10Brion VIBBER: [C: 032] Correct typo in variable naming [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/284268 (owner: 10TheDJ) [05:01:43] (03Merged) 10jenkins-bot: Correct typo in variable naming [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/284268 (owner: 10TheDJ) [05:19:50] (03CR) 10Brion VIBBER: "Looks functional, but it may do db writes during read operations I think. Do we need to be avoiding that for multi-master?" [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/284368 (https://phabricator.wikimedia.org/T123849) (owner: 10TheDJ) [05:49:13] (03CR) 10Brion VIBBER: [C: 032] "Looks good lets do it" [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/284255 (https://phabricator.wikimedia.org/T132616) (owner: 10TheDJ) [05:55:21] (03Merged) 10jenkins-bot: Don't list transcodes that were never initiated as failures. [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/284255 (https://phabricator.wikimedia.org/T132616) (owner: 10TheDJ) [10:07:21] (03PS8) 10TheDJ: [WIP] TimedText should use ContentHandler [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) [10:19:27] (03CR) 10Brion VIBBER: "Do we foresee any compat problems with this initial move? Just wondering if we should prep to merge this bit close to as is or if it need " [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [10:37:14] (03CR) 10TheDJ: "i was just looking at this patch again, checking what else it needs. It definitely wasn't finished in my mind when i stored it, not sure i" [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [11:34:02] MarkTraceur: I am going to try integrating https://github.com/fengyuanchen/cropper [11:34:07] It looks alive :) [12:57:26] I remember now that we were considering this instead of Caman. We wont be using it (if it all) for all its functionality, just the cropping part. [13:23:53] 06Multimedia, 10MediaWiki-API, 10MediaWiki-Uploading: Allow continuing the partially-complete upload in stashed mode after warning - https://phabricator.wikimedia.org/T112543#2227172 (10Xqt) [13:34:06] (03PS9) 10TheDJ: [WIP] TimedText should use ContentHandler [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) [13:38:04] (03CR) 10jenkins-bot: [V: 04-1] [WIP] TimedText should use ContentHandler [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [13:40:37] 06Multimedia, 10Analytics, 06Commons, 10Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2227240 (10matmarex) As I understand these are stored as regular MediaWiki pages now, so they have a maxim... [13:42:39] (03CR) 10TheDJ: "Right, I have mostly adapted the system to use a new content model (the storage format is still wikitext however). I'm somewhat confused h" [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [14:11:37] thedj: I guess you'll want some backwards compatibility code that supports the old wikitext model and converts it on load if it's the wrong format. [14:19:22] Progress! http://i.imgur.com/9eZxb54.png [14:19:32] And possible copyvio :S [14:23:25] 06Multimedia, 10MediaWiki-Uploading: 'finishStash(...) is undefined' when uploading with upload dialog - https://phabricator.wikimedia.org/T133295#2227347 (10matmarex) [14:31:40] James_F: ideally yes [14:32:40] James_F: that would be better than changing all existing revisions [14:32:49] thedj: Not really familiar with TimedText though. [14:33:11] its just wikitext for now with wikitext content model [14:33:20] We can do a final batch conversion in a year's time for the remaining ones when removing the b/c code. [14:33:24] Or whenever. [14:33:38] and i want to now convert it to it's own contentmodel (same format) [14:33:46] Bulk conversion on release sucks though. [14:33:56] That's a good idea. [14:33:58] yu [14:34:35] once it has it's own contentmodel, i might introduce a new contentformat for serialization, but that is for a future endeavor [14:36:27] i thought we did this for JS/CSS as well at some point.. but i can't find what piece of code enabled that transition. [14:36:53] perhaps legoktm or Krinkle knows. [14:45:14] thedj: we didn't change anything in the database for JS/CSS, i think [14:45:38] thedj: we just declared that from this point on, the same data we had is going to be treated as a different content model, for pages ending with .css/.js in some namespaces [14:46:07] we also didn't have the contentmodel stored in the database until late last year. perhaps that helps [14:50:39] this seems related: https://phabricator.wikimedia.org/T104033 [14:54:34] 06Multimedia, 10MediaWiki-Uploading: 'finishStash(...) is undefined' when uploading with upload dialog - https://phabricator.wikimedia.org/T133295#2227392 (10RobLa-WMF) [15:10:40] 06Multimedia, 10Analytics, 06Commons, 10Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2227437 (10brion) >>! In T120452#2227240, @matmarex wrote: > As I understand these are stored as regular M... [15:12:03] 06Multimedia, 10Analytics, 06Commons, 10Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2227438 (10brion) Side note: headers are rejected if they contain spaces. That seems odd? [15:36:45] (03CR) 10Brion VIBBER: "I think old revisions would stay immutably as wikitext; you'd want to upgrade-on-edit or something (and/or via a batch job maintenance scr" [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [15:37:25] 06Multimedia, 10MediaWiki-Uploading: 'finishStash(...) is undefined' when uploading with upload dialog - https://phabricator.wikimedia.org/T133295#2227494 (10matmarex) p:05Triage>03Unbreak! [15:37:38] MarkTraceur: https://gerrit.wikimedia.org/r/284710 [15:39:07] MarkTraceur: apparently, the upload dialog has been broken for a whole week and no one reported it until today [15:39:11] MatmaRex: Sorry [15:39:21] Yeah, funny that [15:39:26] hey, i merged that, my fault just as well [15:39:35] I thought I tested it for sure [15:39:43] Maybe I only tested the new use case. [15:40:02] Too late for morning SWAT [15:40:13] but we really need to instrument that thing [15:40:21] to realize when it's throwing more errors [15:40:27] and to realze when it stops throing any at all [15:40:46] I doubt errors would have told us about this [15:40:46] (the upload dialog, i mean.) [15:40:55] MarkTraceur: they'd have stopped coming, so i think they would [15:40:55] But I agree, of course [15:41:13] I thought the upload went through as expected, but the dialog just didn't close? [15:41:28] Whatever happened to tgr's experimentation with Sentry? [15:41:32] yes [15:41:42] but the function was returning nothing rather than a Promise [15:41:55] so everything that relied on the Promise's result/failure would not run [15:42:01] Hm [15:42:08] presumably that would include the instrumentation that we don't have [15:42:09] Fair enough [15:42:52] so… wanna do it? :P [15:43:15] i'm still not very familiar with that part of EventLogging. the schemas and stuff [15:43:20] Instrumentation we don't have, for software apparently nobody uses, so we can fix it with time we don't have, instead of other software that we can't build anyway [15:43:22] Sure whatever [15:43:27] Sentry is hybernating until AuthManager is done [15:43:47] tgr: Cool, thanks for the update [15:44:19] MatmaRex: Probably not today because I want to roll up my sleeves and delve into some Commons gadgets [15:44:29] MarkTraceur: tons of people use it. but they're all newbies who wouldn't know where or how to report the failure [15:44:48] i mean, https://commons.wikimedia.org/w/index.php?namespace=&tagfilter=cross-wiki-upload&translations=filter&title=Special%3ARecentChanges [15:45:01] Good point. [15:45:04] that's an upload every few minutes [15:45:08] and they're all be redlink usernames [15:45:12] by* [15:46:54] *nod* [15:52:25] 06Multimedia, 10MediaWiki-Uploading, 13Patch-For-Review: 'finishStash(...) is undefined' when uploading with upload dialog - https://phabricator.wikimedia.org/T133295#2227544 (10matmarex) The uploads go through, but the upload dialog hangs and doesn't show the successful upload :/ This has been broken for th... [15:57:30] 06Multimedia, 10MediaWiki-Uploading, 05WMF-deploy-2016-04-12_(1.27.0-wmf.21), 05WMF-deploy-2016-04-26_(1.27.0-wmf.22): 'finishStash(...) is undefined' when uploading with upload dialog - https://phabricator.wikimedia.org/T133295#2227579 (10matmarex) [16:15:21] 06Multimedia, 10MediaWiki-Uploading, 05MW-1.27-release-notes, 05WMF-deploy-2016-04-12_(1.27.0-wmf.21), 05WMF-deploy-2016-04-26_(1.27.0-wmf.22): 'finishStash(...) is undefined' when uploading with upload dialog - https://phabricator.wikimedia.org/T133295#2227646 (10matmarex) 05Open>03Resolved Fixed an... [16:16:00] Might not be attending tomorrow's (today's?) stand-up. I am making progress on the crop tool, that is my only update. [16:16:09] MarkTraceur James_F ^ [16:17:55] prtksxna: Okay, cool [16:24:09] Does it take a while for the File usage section to update? [16:24:47] If a file on commons is used on en.wiki for example [16:26:50] 06Multimedia, 10Wikimedia-Site-requests: Configure testwiki and test2wiki to perform cross-wiki uploads to each other - https://phabricator.wikimedia.org/T133305#2227706 (10matmarex) [16:27:15] no idea [16:31:36] 06Multimedia, 10MediaWiki-Uploading: Add EventLogging instrumentation to the upload dialog - https://phabricator.wikimedia.org/T133306#2227729 (10matmarex) [16:36:32] (moving from -operations) ooh, I stand corrected, https://en.wikipedia.org/wiki/McNamara_Alumni_Center has no picture and includes the phrase "most architecturally striking" [16:36:44] Sorry, "more" [16:36:57] ha [16:37:24] i filed some more bugs about this. https://phabricator.wikimedia.org/T133305 https://phabricator.wikimedia.org/T133306 https://phabricator.wikimedia.org/T133307 [16:37:46] James_F: can you help make the last one happen? https://phabricator.wikimedia.org/T133307 "Create a dashboard or alerts or something for the rate of errors in various uploading tools, and rate of completed uploads using various uploading tools" [16:37:57] i have no idea where to start, and you probably have something like this for VE [16:38:39] 06Multimedia, 10Wikimedia-Site-requests: Configure testwiki and test2wiki to perform cross-wiki uploads to each other - https://phabricator.wikimedia.org/T133305#2227706 (10Jdforrester-WMF) This is what the Beta Cluster is for… [16:39:15] MatmaRex: More of a dashboard than https://edit-analysis.wmflabs.org/multimedia-health/#projects=enwiki,dewiki,commonswiki/metrics=Uploads ? [16:39:27] MatmaRex: MarkTraceur did all the hard work on that, don't give me credit. [16:39:42] 06Multimedia, 10Wikimedia-Site-requests: Configure testwiki and test2wiki to perform cross-wiki uploads to each other - https://phabricator.wikimedia.org/T133305#2227797 (10matmarex) No, beta cluster is for pre-deployment testing. This is for post-deployment verification. [16:39:48] Some tools aren't represented there [16:39:53] (Yet.) [16:39:57] And I think it's only monthly [16:40:08] James_F: that's a good start, but i am hoping for more [16:40:15] So it's not good for catching errors, unless the error happens...no, never mind, that still wouldn't help anything [16:40:37] We could add a daily data stream to it though? [16:40:44] We could, yes [16:40:50] James_F: i dunno, maybe we should have icinga alerts or something. "there were no uploads in the last 5 minutes, something is horribly broken" [16:40:51] Dashiki takes whatever data you throw at it, right? [16:40:54] It's all the same data, just with a different granularity [16:41:04] MatmaRex: Well, icinga is Opsen magic. [16:41:28] 5 minutes is probably longer than we want [16:41:43] right. just an example [16:42:05] Icigna is a "OMG the DB is on fire wake everyone up with expensive text messages". [16:42:22] It is entirely possible for users to not upload a file to Commons for a few minutes. [16:43:04] (E.g. when we set the entire damn data centre to read-only for thirty minutes. ;-)) [16:50:39] (Also, for VE/WT we don't have live monitoring; we have QA testing, and "people will shout at us".) [16:56:26] well, we have no QA testing for multimedia. [16:57:15] Actually, that's not entirely true; there's also https://grafana.wikimedia.org/dashboard/db/edit-count [16:57:36] Yeah. I could ask Rummana to spend some time QAing, but… [17:14:16] thedj: probably need something like https://github.com/wikimedia/mediawiki/blob/master/maintenance/fixDefaultJsonContentPages.php [17:14:20] (I only skimmed the scrollback) [17:34:56] legoktm: right, so flip the contentmodel of the old revisions to forced wikitext. [17:36:51] yeah [17:36:56] and set the default content model to the new thing [17:37:48] right. [17:38:06] will try that. [17:38:33] how would that work with editing an old version and saving it as new then ? would it then save as 'new' ? [17:43:53] thedj: it wouldn't allow you to edit the old revision, just view source [17:44:08] But an existing one that was the top revision but the old format. [17:44:15] You'd have to convert that for the user? [17:44:45] then they'd just continue editing it in the old format [17:44:50] until it got converted [17:44:58] it's the same format [17:45:06] only different content model [17:46:33] uh [17:46:45] when you say format, do you mean the contenthandler concept of "content format"? [17:47:30] yes [17:49:37] so you're going to continue using CONTENT_MODEL_WIKITEXT with the new content model? [17:49:42] er, CONTENT_FORMAT_WIKITEXT* [17:50:06] correct [17:51:09] uhh, why? [17:52:01] cause it's still wikitext for now. [17:52:06] I actually don't know your usecase yet [17:52:08] Hrmm. [17:52:47] because our current subtitles contain wikitext formatting, and i'll need to convert it to something else at some point, but i don't want to open to tar pits at once [17:52:56] s/to/two [17:53:15] got it [17:53:42] so off the top of my head, I don't know, because I've never tested that scenario [17:53:55] (03CR) 10TheDJ: "note for after dinner: https://github.com/wikimedia/mediawiki/blob/master/maintenance/fixDefaultJsonContentPages.php" [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [17:54:09] right, i'll just toy around then :) [17:54:38] i'm already closer than before, so that will be helpful [17:56:26] and as a fair warning, the content format feature has never really been used before, and I don't think anyone has tried to switch formats without changing content models before. and the last time I looked, there were a few gotchas [17:56:31] (03CR) 10Paladox: [C: 04-1] "Ok, I'm going to use this patch for testing something that is unrelated." [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 (owner: 10Paladox) [17:56:37] (03PS2) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [18:03:25] (03PS3) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [18:03:31] James_F: But what? [18:03:38] (03CR) 10Paladox: [C: 04-1] Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 (owner: 10Paladox) [18:04:43] ? [18:05:19] (03PS4) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [18:05:29] James_F: You'd ask Rummana to do QA, but what? [18:07:27] (03PS5) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [18:09:42] (03PS6) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [18:12:44] MarkTraceur: But (a) that means less testing for wikitext editing, OOUI, and VE, and (b) that's what Beta Cluster is for and MatmaRex just said he didn't want to use the testing infrastructure we have. [18:12:52] Sorry, thought it was obvious. [18:22:05] James_F: Hm, OK [18:26:32] 06Multimedia, 06Commons, 10Tool-Labs-tools-Other: Commons API fails (413 error) to upload file within 100MB threshold - https://phabricator.wikimedia.org/T86436#2228157 (10Nemo_bis) [18:26:40] 06Multimedia, 06Commons, 10Tool-Labs-tools-Other: Commons API fails (413 error) to upload file within 100MB threshold - https://phabricator.wikimedia.org/T86436#968269 (10Nemo_bis) a:05BBlack>03None [18:56:53] (03PS7) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [19:10:14] (03PS8) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [19:15:50] (03PS9) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [19:21:20] (03PS10) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [19:29:54] (03PS11) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [20:40:44] (03PS12) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 [21:28:14] Hey guys, I'm making a record by backing out of three of four standups this week [21:28:27] I have to run early today but I might be back on later tonight [21:29:44] :D [21:41:46] (03CR) 10Krinkle: Remove transcodes that are no longer enabled (031 comment) [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/284368 (https://phabricator.wikimedia.org/T123849) (owner: 10TheDJ) [21:44:37] (03Abandoned) 10Paladox: Enable 720p for ogv [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/283868 (owner: 10Paladox) [21:47:17] (03CR) 10Legoktm: [C: 04-1] "Quick review for now" (034 comments) [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [22:59:51] (03PS1) 10Bartosz Dziewoński: Log all failing API responses when uploading [extensions/UploadWizard] - 10https://gerrit.wikimedia.org/r/284832 (https://phabricator.wikimedia.org/T130485) [23:00:51] allll byyyy myyy selllllllf [23:01:24] * lindseyanne looks at the clock [23:01:31] * lindseyanne sings more lonely songs [23:02:36] James_F prtksxna ? [23:02:47] Oh, whoops. [23:02:49] Hi there. [23:02:54] heyyo [23:03:34] MarkTraceur: ping me when youre around later? have small news to catch up on... [23:04:35] lindseyanne: Sorry, yeah, can do [23:04:56] lindseyanne: If you like you can PM me here or on Hangouts. [23:42:51] (03PS10) 10TheDJ: [WIP] TimedText should use ContentHandler [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) [23:44:01] my god, every single day i'm surprised by how much overdue work there is in that extension. It's UW 2.0 [23:46:44] (03CR) 10jenkins-bot: [V: 04-1] [WIP] TimedText should use ContentHandler [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/236486 (https://phabricator.wikimedia.org/T111651) (owner: 10TheDJ) [23:57:03] 06Multimedia, 10Analytics, 06Commons, 10Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2229568 (10Jdforrester-WMF) >>! In T120452#2226245, @brion wrote: > Population of every US census place fo...