[00:00:13] bawolff: No, enwiki is not until next month, but we want to start the discussion early, since I’m sure people will have a lot to say about it. Here’s the latest release plan, which I summarized in today’s email: https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan [00:00:43] Always good to give enwiki a little warning [00:01:11] Exactly. The more advance notice we give, the less stress for everyone :) [00:01:21] I'll link to that page in my comment [00:01:48] That sounds great, thank you! [00:02:09] OK, folks, time for me to start the long voyage home across the great waters … I will turn this IRC off so I don’t drive you all crazy with connections going on and off, but you can still reach me via email. [00:02:15] Over and out for today. [00:03:56] btw, props for actually talking to the communities instead of just shoving it at them. I think that goes a long way to making sure everyone gets along [00:51:46] (03PS1) 10Gergő Tisza: Quick fix for black screen of death [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124783 [01:05:23] marktraceur: what's the easiest way to get DB read access? do we have an instance in the multimeda labs project where it is set up already? [01:05:31] Uhhh [01:05:48] tgr: Production, 100% up to date DB? Or like...research databases? [01:06:07] tool labs gives you read access to most of it [01:06:13] either one is fine [01:06:34] bawolff: i know, i was just hoping i dont have to set up a new instance for it [01:07:06] tgr: stat1 is good for the latter [01:09:39] (03CR) 10MarkTraceur: [C: 032] "Suuuuure" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124665 (owner: 10Gergő Tisza) [01:09:55] * marktraceur inspires all manner of confidence with his merge messages [01:10:08] (03Merged) 10jenkins-bot: Make categories behave [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124665 (owner: 10Gergő Tisza) [01:10:27] > bootstrapping is an unmaintainable mess which should be cleaned up. [01:10:33] itym "MultimediaViewer"? [01:10:48] But yeah sure, I'll look at that next next [01:12:27] the most messy part is probably how hash handling is split between mmv and mmv.boostrap [01:12:39] Yeah [01:12:43] with mmv.bootstrap also doing some of the UI part [01:18:39] * marktraceur vows to split the repo stuff into its own UI class soon [01:25:44] TIL: We have code comments like "// XXX: don't spam exception log" in the upload api module [01:26:06] Apparently uploads fail so often, we need to have code to prevent the exceptions from being logged (?) [01:31:05] hmm, to be fair, half of those are user did something wrong exceptions [01:36:41] * marktraceur points at https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/417 [01:36:47] Interesting problem to think about [01:37:30] (03CR) 10Aarcos: "The test below doesn't work? Plus the comments below that haven't been addressed. Thanx !" (033 comments) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [01:37:41] (03CR) 10Aarcos: [C: 04-1] Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [01:38:54] (03PS11) 10Aarcos: Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [01:39:26] (03CR) 10jenkins-bot: [V: 04-1] Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [01:50:03] marktraceur: (I can't remember) is the extmetadata api data the same across all wikis? or does it matter where you access it from [01:51:21] marktraceur: Because if it was the same, it could all be cached in one place, and then purged when someone edits the page at commons [08:11:37] (03PS1) 10Gergő Tisza: [WIP] Guess thumbnail URLs without extra API call [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 [08:12:10] (03CR) 10jenkins-bot: [V: 04-1] [WIP] Guess thumbnail URLs without extra API call [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 (owner: 10Gergő Tisza) [08:38:39] (03PS6) 10Gilles: Test to compare the performance of MMV and the Commons File: page [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/119917 [09:40:23] (03CR) 10Gilles: [C: 032] Show tooltip when all sorts of conditions are met [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124032 (owner: 10Gergő Tisza) [09:40:54] (03Merged) 10jenkins-bot: Show tooltip when all sorts of conditions are met [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124032 (owner: 10Gergő Tisza) [09:50:50] (03CR) 10Gilles: [C: 04-1] "Lacks test coverage." (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [10:04:38] (03PS4) 10Pginer: Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [10:20:19] (03PS5) 10Pginer: Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [10:45:24] (03CR) 10Siebrand: [C: 04-1] "i18n/L10n reviewed." (032 comments) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [11:53:10] (03PS7) 10Gilles: Test to compare the performance of MMV and the Commons File: page [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/119917 [12:05:43] (03CR) 10Siebrand: Make Commons link more prominent (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 (owner: 10Gergő Tisza) [12:15:20] (03PS1) 10Pginer: Adjust text style for size indicaions [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124828 [12:15:53] (03PS2) 10Pginer: Adjust text style for size indications [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124828 [12:58:38] (03PS1) 10Pginer: Adjustments to the Commons icon in Media Viewer To achieve consistency with the icons in the Media Viewer panel, the following has been adjusted: - Colors onf the "share" icon to match the rest of the icons (updated to the new gray values of Agora). - Siz [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124831 [15:21:36] (03CR) 10Gilles: "I'm skipping this review for now, as this will probably be a non-issue or will need to be revisited if my proposal to get rid of the boots" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124783 (owner: 10Gergő Tisza) [15:22:28] pginer: I'm usually reactive to your changesets and calls for help but I'm really swamped today, I'll try to get around to it tomorrow if nobody has by then [15:23:04] ok, don’t worry [15:25:34] I’m only afraid of changes getting old because git tends to start complaining about rebases and other technical jargon. [15:26:00] and my git/gerrit processes are very limited. [15:29:38] (03CR) 10Gilles: "Would be awesome if we can get that finished before this week's deploy! It would make the image show up 500ms sooner on average, according" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 (owner: 10Gergő Tisza) [15:40:59] (03Abandoned) 10Nischayn22: GroupProgressBar: Only show status "finished" once [extensions/UploadWizard] - 10https://gerrit.wikimedia.org/r/54127 (owner: 10Nischayn22) [15:44:05] (03CR) 10Gilles: [C: 04-1] Make Commons link more prominent (032 comments) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 (owner: 10Gergő Tisza) [15:44:21] (03CR) 10Gilles: [C: 032] Adjust text style for size indications [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124828 (owner: 10Pginer) [15:44:53] (03Merged) 10jenkins-bot: Adjust text style for size indications [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124828 (owner: 10Pginer) [15:46:02] (03CR) 10Gilles: [C: 032] Adjustments to the Commons icon in Media Viewer To achieve consistency with the icons in the Media Viewer panel, the following has been adju [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124831 (owner: 10Pginer) [15:46:32] (03Merged) 10jenkins-bot: Adjustments to the Commons icon in Media Viewer To achieve consistency with the icons in the Media Viewer panel, the following has been adjusted: - Colors onf the "share" icon to match the rest of the icons (updated to the new gray values of Agora). - Siz [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124831 (owner: 10Pginer) [17:05:01] (03CR) 10MarkTraceur: "I'm squeamish about this. See comment." (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 (owner: 10Gergő Tisza) [17:09:58] gi11es marktraceur : Hi guys, how are we coming along for tomorrow’s release? need anything special tested today? I see cards are moving to the right on Mingle, which is a good thing, though a lot are still awaiting code review :) [17:10:17] I think I'll be wallowing in that code review for most of today [17:10:27] Nothing special needed for now I think [17:12:30] (03CR) 10Brian Wolff: "Note, there is the in-between approach of doing https://commons.wikimedia.org/wiki/Special:Redirect/file/Example.png?width=40 (But guessin" (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 (owner: 10Gergő Tisza) [17:14:52] marktraceur: considering we might not have time to implement and review my suggestion on the mailing list to change most of bootstrap, the top priority changeset to review is the black-screen-of-death bugfix from gergo [17:15:57] *nod* [17:16:07] I'm halfway through another one, but will come to that next [17:16:10] Probably after SoS [17:17:05] I'll do the analytics ones too [17:17:08] Ugh so much CR [17:20:13] marktraceur: Thanks for this good overview. For now, I’ll poke around on beta, to get a sense of where we are overall. Unless you would prefer that I use alpha or another site. [17:23:24] (03CR) 10Gilles: "The redirect just took 650ms for me, which is pretty much in the same ballpark as the current thumbnail API call, so no improvement if we " [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 (owner: 10Gergő Tisza) [17:26:06] marktraceur: is limn0/limn1 where multimedia-metrics is hosted? [17:27:22] Alpha hasn't been updated in a long time [17:27:31] marktraceur gi11es : One thing I noticed on current production sites is that when you download a file on Chrome, the file downloads with name 'download.jpeg' instead of actual file name, whereas it works fine on Firefox. But when I test this on beta, I don’t get that problem. Does this mean you guys fixed the problem but didn’t release the code yet? [17:27:33] Still running old version of share/embed AFAIK [17:27:37] gi11es: Yeah [17:27:43] Oh, right, it's limn1 now [17:27:48] Silly eqiad, trix are for kids [17:28:00] fabriceflorin: which production site? [17:28:21] Yeah, thought we fixed that [17:28:54] Commons. Here’s an example: https://commons.wikimedia.org/wiki/Category:Featured_pictures_of_Estonia#mediaviewer/File:Allium oleraceum - rohulauk Keilas2.jpg [17:30:26] fabriceflorin: this is running old code, that issue has been fixed afaik [17:30:52] if I recall correctly the fix was for the download attribute to be empty, instead of being equal to "download" [17:31:11] commons still have it equal to "download", which affects the downloaded file name [17:31:38] gi11es : Thanks for confirming. I’m doing some testing on Commons now, because I am about to post an update to let folks know about the upcoming release. So I want to make sure the site is in reasonable shape before publicizing this … [17:43:37] gi11es: i can finish the thumbnail guessing today, but then I will probably have to drop the ball on the out of focus bugfix thing [17:43:40] anyone willing to pick that up? [17:43:40] gi11es : I will do the rest of my testing on beta now, unless you guys want me to do it elsewhere. I just filed a minor issue reported by Bawolff yesterday: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/419 [17:43:52] (if you see this twice, i blame irssi) [17:43:52] gi11es: Is there anything that concerns you more than anything else for tomorrow’s launch, besides Gergo’s black screen of death? [17:45:14] tgr: do you have a link for the out of focus? I'm not sure I know which one you're referring to [17:45:40] gi11es: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/403 [17:46:18] oh, that... [17:46:23] * gi11es looks at aaron_arcos [17:46:59] honestly, while this is something we should do during normal weeks, I would be comfortable with making an exception on our launch week, considering the list of last minutes things we need to review and merge [17:47:43] I'd rather spend tomorrow rewriting most of bootstrap out of existence to improve our cold cache experience, which will be most people's first impression [17:51:32] We are talking here, I agreed to help with card/403. [17:51:41] bd808: does it look like we're going to be able to deploy '22 on schedule tomorrow? [17:51:59] gi11es: I think so, yes [17:52:05] awesome [17:52:29] I got wmf21 back on track yesterday (with a complete site outage thrown in for fun) [17:52:38] (03CR) 10MarkTraceur: [C: 04-1] "Few small ideas; I think I will wind up fixing these and merging if tgr isn't working on it already" (035 comments) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 (owner: 10Gergő Tisza) [17:52:49] tgr: ^^ are you busy with other things or should I let you work on that? [17:53:40] no, i can fix that [17:55:03] 'kay [18:06:17] (03CR) 10Gergő Tisza: [WIP] Guess thumbnail URLs without extra API call (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 (owner: 10Gergő Tisza) [18:07:27] fabriceflorin: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/419 [18:07:36] ...should be fixed on master already [18:07:44] on mw.org too. i think [18:11:47] tgr: Thanks. I tried to confirm that it has been fixed on mw.org, but cannot verify it because the Embed wikitext still shows a ‘Loading …’ message on both mw.org and beta. When do we expect that to be fixed? That seems like a showstopper for tomorrow’s launch. https://www.mediawiki.org/wiki/Lightbox_demo#mediaviewer/File:Gori military base.jpg [18:12:26] i think that goes away once you use the size selector [18:16:14] gi11es: I agree with you that we should prioritize today’s tasks to focus on the most critical issues, and postpone others to later. If you think you can rewrite bootstrap quickly to improve performance, that would seem like a worthwhile improvement, assuming it’s doable in a short time-frame. I will let you take the lead on prioritizing technical tasks for today, but do let me know if there’s anything I can do to help. Thanks [18:16:14] again for your excellent report on image load performance, which was invaluable :) [18:18:30] tgr: You are correct that the ‘Loading’ goes away when you select a size in the Embed panel, but the average user would not know to do that. Is there a way for us to automatically fill the Wikitext with the wikitext for thumbnails? That seems important to have for launch, IMHO. Do we need a ticket for that? [18:19:25] fabriceflorin: i am just saying, it it still possible to test issues with the embed text [18:20:36] the Loading... issue is a remnant of the OOJS UI problems we've been going through, worth a ticket, I don't see one specifically for that [18:21:09] tgr: aaron_arcos: wasn't one of you working on that particular problem? I haven't followed the latest news of the OOJS UI issues [18:25:14] (03CR) 10MarkTraceur: "We changed our minds in the past three hours; reviewing now" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124783 (owner: 10Gergő Tisza) [18:25:23] gi11es: James_F and the VE team thank you for your patch on the OOJS code, which they just merged and updated two minutes ago. tgr will update our code so we can see if that solves the problem. [18:26:26] aaron_arcos: Thanks for helping solve the ‘Loading’ issue in the Embed panel with tgr — I really think we want a fix for that so we don’t confuse users tomorrow. [18:28:39] gi11es: aaron_arcos has a patch [18:29:17] oh yeah I remember now, the one with a metric ton of code :) [18:29:32] I am taking care of all OOJS issues and I have a patch that I will verify with the latest core, it works fine with core from yesterday but there were some recent changes. [18:29:44] the ticket is #408 i think [18:30:04] this is patch that already works but that I am improving: https://gerrit.wikimedia.org/r/#/c/124772 [18:30:24] I am using card/407 [18:30:28] (03CR) 10MarkTraceur: [C: 032] "Thanks, tgr." [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124783 (owner: 10Gergő Tisza) [18:30:42] but that includes all the other issues. [18:31:04] (03Merged) 10jenkins-bot: Quick fix for black screen of death [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124783 (owner: 10Gergő Tisza) [18:32:11] tgr: I just tested the embed code as you suggested and can confirm that the white space issue has been solved. I accepted the Mingle ticket, so that issue is closed now. http://en.wikipedia.beta.wmflabs.org/wiki/Lightbox_demo#mediaviewer/File:Gori military base.jpg [18:33:57] (03CR) 10MarkTraceur: "Patch incoming" (034 comments) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [18:34:05] (03PS12) 10MarkTraceur: Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 [18:34:36] (03CR) 10jenkins-bot: [V: 04-1] Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [18:37:31] fabriceflorin: you can preview the graph improvements I've sent to Mark for review here: http://superdevoluy.dubuc.fr:5000/dashboards/mmv (might be a bit slow, this is served from my laptop) [18:47:10] (03PS2) 10Aarcos: Fix issues with size menus after oojs-ui update [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124772 [18:48:42] AHA [18:48:49] aaron_arcos: Annoying failure to clone the text node [18:49:03] Now it is closer but not perfect I guess [18:49:58] (03CR) 10Aarcos: "This should fix all the issues we are having in share/embed, PTAL." [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124772 (owner: 10Aarcos) [18:50:05] (03CR) 10Aarcos: Fix issues with size menus after oojs-ui update (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124772 (owner: 10Aarcos) [18:53:45] (03CR) 10Gilles: [C: 032] Fix issues with size menus after oojs-ui update [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124772 (owner: 10Aarcos) [18:54:16] (03Merged) 10jenkins-bot: Fix issues with size menus after oojs-ui update [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124772 (owner: 10Aarcos) [18:55:15] (03PS13) 10MarkTraceur: Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 [18:55:23] aaron_arcos: ^^ works now, your test was also wrong but unrelatedly so [18:55:51] fabriceflorin: marktraceur: https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Template_compatibility [18:56:00] ;-), ok, should I take another look? [18:56:01] gi11es: Thanks for sharing your graph improvements, which are very helpful. Particularly the geographical network performance, which lets us anticipate mean image loads in our upcoming pilots (e.g.: 1.34 secs. in Hungary vs. 2.38 in Korea, 3.06 in Spain, 1.27 in US). Are these based on Media Viewer image loads or file page loads? [18:56:16] aaron_arcos: Yesplz [18:56:27] fabriceflorin: media viewer, with API and image treated separately (hence the two maps) [18:57:02] fabriceflorin: remember to check the sample size, might be very low in some countries, which makes the figures less reliable [18:57:44] gi11es: Thanks. sI also have questions about http://superdevoluy.dubuc.fr:5000/dashboards/mmv#media_viewer_vs_file_page-graphs-tab [18:58:04] fabriceflorin: that one is still experimental, the figures you're seeing came from my laptop [18:58:15] Agh, metrics. [18:58:20] fabriceflorin: I'm still waiting on QA to approve so that I can run this in a more controlled environment (cloudbees) [18:58:21] * marktraceur flails [18:58:57] gi11es: Sorry about typing too fast and not completing my sentence about the Media Viewe vs. File page graph. I don’t understand why the dates are based on March data, vs. April. Maybe I should wait a bit more, until you’ve fleshed it out some more … [18:59:37] fabriceflorin: there's very little data on that graph because there are only points on days when I ran the test on my laptop [19:00:16] fabriceflorin: this one will start being really useful when we're running the test every hour automatically [19:00:43] fabriceflorin: and I plan to have one graph like that for each pilot site, right now it's only been measured against mediawiki.org [19:02:33] gi11es: Thanks for clarifying the difference between API and image load times in the geographic graphs. So if Hungary has a 620 millisecond API performance, that means the time from the time the API receives the request to the time it serves the image file? So the extra 720 milliseconds is all client-side activity on the user’s browser and in-between operations, for a total of 1.34 secs? [19:03:43] fabriceflorin: the status quo is that in order to display the image we need one API call followed by the image itself. gergo is working on a patch so that we can display the image directly, without any preliminary API call [19:04:15] fabriceflorin: so at the moment, you can add the average API call time to the average image load time and the total is a good estimate of how long on average it takes to see the sharp image after clicking [19:04:49] gi11es: Cool. Sounds like shaving off that extra API call will help. [19:05:03] (03PS1) 10Aarcos: Clean up, get rid of leftover comment [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124910 [19:07:10] gi11es: Thanks for the clarification that we need to add the API time to the image time to get the total time. So am I correct in thinking that Hungary’s mean load time would be 0.62 api + 1.34 image = 1.96 total load? Or am I misunderstanding? If I am right, you may want to consider a third overall graph at the top showing the total, or people may get the wrong impression. [19:08:46] gi11es: ERROR 1054 (42S22) at line 1: Unknown column 'datestring' in 'field list' [19:08:56] Because datestring is something we construct in the query [19:09:00] gi11es: It will be great to see the Media Viewer vs. File page graph on an hourly basis, as it may help us understand any differences throughout the day. Is the File page mean a combination of cold and warm cache? Are they not letting you differentiate between the two for comparison purposes? [19:09:04] Not an actual field [19:11:39] (03CR) 10Gergő Tisza: Make Commons link more prominent (033 comments) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 (owner: 10Gergő Tisza) [19:18:10] (03CR) 10Siebrand: [C: 04-1] Make Commons link more prominent (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 (owner: 10Gergő Tisza) [19:21:08] fabriceflorin: File page is only warm cache because the cold cache comparison isn't fair: when you open the file page on a cold cache, you have to load all the mediawiki JS and CSS, but when you open media viewer it's already been loaded with the article [19:21:58] fabriceflorin: and the cold cache for the File: page is much less likely to happen to users than Media Viewer's, which we're introducing [19:23:04] fabriceflorin: your calculation is correct, but I think a third graph isn't that useful, considering that gergo is working on a fix as we speak which will mean there won't be a meaningful total anymore [19:23:24] marktraceur: which merge request is that? [19:23:37] marktraceur: and which sql file [19:49:14] marktraceur: ah I see it now, I'll fix it and send a new merge request pronto [19:58:07] marktraceur: fixed https://gitorious.org/analytics/multimedia/merge_requests/5 [20:00:19] (03CR) 10Gergő Tisza: [C: 032] Clean up, get rid of leftover comment [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124910 (owner: 10Aarcos) [20:01:05] (03Merged) 10jenkins-bot: Clean up, get rid of leftover comment [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124910 (owner: 10Aarcos) [20:01:22] All the SQL files [20:01:35] There we go [20:02:51] Much more better [20:03:06] Soon we shall have new datums [20:03:56] gi11es: Thanks for your helpful explanations about the graphs. I think I have a pretty good understanding of them now. It’s great to have more clarity on these image load questions. Really appreciate this important work. [20:11:53] marktraceur: have you run the update jobs manually? I don't see the new data yet [20:12:54] I did [20:12:59] It takes time to copy them [20:13:03] Probably in about 2-3 minutes [20:26:04] tgr: so, regarding caching API request, a suggestion that came from Aaron Schulz is to use a long caching period, and to include a timestamp in the URL parameters which would represent the last time something about the file was modified (file itself, metadata, etc.). for it to work for all our API calls, we'd need to add that timestamp as an attribute to the thumbnail DOM [20:26:51] tgr: now, what I wonder is how feasible it is to have that universal "last modification that should invalidate the API" timestamp, and how cheap it could be to fetch it and apply it to thumb DOM [20:27:42] that depends on how generic you want it to be [20:27:53] the weakness of that approach, of course is that new code introduced that updates file-related data would have to remember to update that timestamp somehow, if it's not automatic [20:28:22] we have the image itself and basic metadata, that should be invalidated when a new file version is uploaded [20:28:38] that's pretty good already [20:28:46] we can include the file version as a url paramter on those calls [20:28:47] there is the extended file metadata, which depends on the last edit timestamp of the image description page [20:28:57] same thing [20:29:09] (03PS6) 10MarkTraceur: Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 [20:29:28] (03CR) 10MarkTraceur: "Addressed the specific case concerns, now more general" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [20:29:29] it doesn't have to be generic, each API call can have its own "version/timestamp" paramter [20:29:44] which might be on the same wiki, or in a different database on the same database server (WMF-Commons) or a remote site with an api (instantcommons) [20:30:13] ah, so in that case it's tough, because we aren't notified of the change [20:30:16] (03PS7) 10MarkTraceur: Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 [20:30:25] (03CR) 10MarkTraceur: "Addressed i18n concerns" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [20:30:42] there is the local/global file usage, which can basically be invalidated by any article edit anywhere, and probably has its own caching mechanism (just guessing here) [20:31:03] plus the userinfo, which changes when the user updates his preferences [20:31:20] well we can start with the easy ones, even if we can't cache all calls, it's already a win for the ones we can [20:31:54] in practice they run in parallel, but some metadata will show up sooner with this optimization [20:31:59] or we can cache everything, and not invalidate the hard ones [20:32:13] (03PS8) 10MarkTraceur: Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 [20:32:19] i doubt anyone would cry if the user gender changes take a few weeks to pick up [20:32:27] the ones we can't invalidate would need to have a much shorter TTL [20:32:29] like a day or something [20:32:46] and i would expect the largest performance win if globalusage + userinfo [20:32:48] the ones that can be invalidated can be much more aggressive with the TTL [20:33:17] (03CR) 10MarkTraceur: "Further removed file-page-specific stuff." [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [20:33:24] since userinfo is a CORS request to a different server, and globalusage is some weird cross-DB thing [20:33:31] I agree that gender update is minor [20:33:53] I think it's as simple as using the functions brian wolff mentioned, which set http headers [20:34:26] globalusage is already a pretty ghetto feature as we know, I'm happy with caching that one for a while [20:35:06] even when one day it uses elastic search or something, it will make sense to cache it for a while [20:36:23] definitely [20:36:34] (03PS9) 10MarkTraceur: Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 [20:36:44] ideally all MediaViewer would do is contact the nearest varnish cluster [20:37:10] (03CR) 10jenkins-bot: [V: 04-1] Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [20:37:29] so then the question is, how easy it is to add an image + filepage version timestamp to the HTML? [20:38:19] it's very easy to add, but how expensive is it to fetch for 100 thumbs on the page? [20:38:29] and then invalidate varnish html (and maybe parser cache? not familiar with that) when either is updated [20:38:53] oh yeah I totally forgot about that... I think we can't [20:39:19] james told me on wikitech-l that it's either on resave or 31 days later that the article cached copy is updated [20:39:29] so we're screwed by that part of the equation [20:40:25] i'm not sure about that [20:40:36] unless there is a cheap way to get the extensive list of articles that have that thumb [20:40:57] and then a mechanism to re-save/uncache the parsed article [20:41:01] when you modify a template, i don't think it takes 30 days for the articles to reflect it, even if you are logged out [20:41:57] true, so whatever saves templates must be reparsing all articles that use that template [20:42:32] i think a job is put in the queue which does just that [20:42:38] what about commons images included on other sites? [20:43:16] plus, an UDP request goes to varnish to drop that page (that might be a side effect of the reparsing) [20:43:52] as for which articles use an image, that's imageusage/localusage api [20:44:31] both are based on a database table which is updated on article edits, i think [20:45:21] marktraceur: still no luck with http://stat1001.wikimedia.org/public-datasets/all/multimedia/media-viewer-perf-mediawiki.tsv [20:45:56] so for local images at least, it could be done effectively in theory: the image is changed, you get the list of articles from the imagelinks table, and invalidate them [20:46:18] how easy is that to do in the current codebase, i have absolutely no idea [20:47:14] for commons, the situation is probably worse: you can still get the list of articles, but they are not on the same wiki so you would need some sort of cross-wiki queue [20:47:20] Hm. [20:47:50] Maybe I need to be doing this on stat1003 [20:47:58] this is not done, even in extensions where it would be a really core feature, like Echo, so it is probably not possible [20:49:19] and given that the majority of the images are on Commons, if something cannot be done for remote images, it is probably not worth doing at all [20:50:02] this is my impression of the whole thing, but then, i am probably the least knowledgeable person about such issues here :) [20:51:13] marktraceur: last time I looked at stat1003 the datasets folder wasn't accessible from the web [20:51:25] http://stat1003.wikimedia.org/public-datasets/ [20:51:31] It's not supposed to be [20:51:34] Nor is it on stat1 [20:51:44] where is it exposed, then? [20:52:00] the people who worked on Echo have probably given a lot of thought to cross-wiki change tracking, might be worth asking them [20:52:01] ah, you think it's piping from stat1003 to stat1001 now? [20:52:12] instead of reading it from stat1? [20:53:01] On stat1001 [20:53:03] Yeah [20:53:04] That's the idea [20:53:08] See -analytics [20:57:03] OK, sorted it out [20:57:13] My incompetence at SSH is now resolved [21:02:05] gi11es: Updated [21:03:03] yay, data [21:04:08] Datum sounds like a good idea [21:05:12] (03PS14) 10Aarcos: Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [21:05:36] the maps stopped working :( [21:05:52] Oh? [21:06:03] Wait, we had maps? Wat [21:06:24] last tab, part of what I added today, and what dan's update was for [21:06:49] Yeah [21:06:51] no map at all is generally an indication of a limn configuration problem, not an issue with the data itself [21:06:54] Uhm...dunno [21:07:03] gi11es: It was working earlier? [21:07:21] I'm pretty sure it was, let me check if it's still working locally [21:07:25] (03CR) 10Aarcos: [C: 04-1] "Added some things and have a comment, please take a look." (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [21:08:10] yep, still works: http://superdevoluy.dubuc.fr:5000/dashboards/mmv#geographical_network_performance-graphs-tab [21:08:20] are you sure the limn update was applied to that server? [21:09:24] No. [21:09:26] Not sure. [21:11:38] fabriceflorin: any thoughts on siebrand's comment here? https://gerrit.wikimedia.org/r/#/c/124769/1/i18n/en.json [21:12:59] * marktraceur thinks "collection" but may not be helpful [21:15:52] My current instincts would be to stick to the current ‘repository’ wording, because it was a community decision to use that term for Commons. I don’t love that term either, but the alternatives are not much better. Of the thesaurus entries that Siebrand proposed, only ‘archive’ seems possibly friendlier. Perhaps we could leave a note for translators to use ‘archive’ as a guideline when looking for a good solution? [21:15:53] http://thesaurus.com/browse/repository [21:17:17] (03CR) 10Fabriceflorin: Make Commons link more prominent (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 (owner: 10Gergő Tisza) [21:23:02] (03PS2) 10Gergő Tisza: Make Commons link more prominent [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 [21:27:26] marktraceur: i recall you opening a bug about how debugging LESS errors sucks [21:27:36] Truth [21:27:40] is there a less sucky way since then? [21:27:43] I doubt it [21:27:48] https://bugzilla.wikimedia.org/show_bug.cgi?id=62081 [21:45:51] (03PS3) 10Gergő Tisza: Make Commons link more prominent [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124769 [21:54:38] (03CR) 10Aarcos: [C: 031] Workaround for OOJS ES3 incompatibility [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124365 (owner: 10Gergő Tisza) [21:58:42] (03PS1) 10Gergő Tisza: Notify user about early errors via mw.notify [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/125012 [22:04:57] (03PS10) 10MarkTraceur: Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 [22:05:31] (03CR) 10jenkins-bot: [V: 04-1] Open MMV on hash change on file pages, add link [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124750 (owner: 10MarkTraceur) [22:07:21] (03PS15) 10MarkTraceur: Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 [22:07:41] (03CR) 10MarkTraceur: "Guess I didn't need that after all. :3" (031 comment) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [22:09:11] tgr: Is mw.notify a dependency we need to load? [22:09:54] (03CR) 10MarkTraceur: [C: 032] "Quick hack? Viable? One line? SIGN ME UP!" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/125012 (owner: 10Gergő Tisza) [22:09:57] unless i am misreading things, mw.notify is a core stub which loads the actual dependency [22:10:00] 'kay [22:10:04] mw.notification or something [22:10:24] (03Merged) 10jenkins-bot: Notify user about early errors via mw.notify [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/125012 (owner: 10Gergő Tisza) [22:11:50] yup, dependency of mw.util, and that is top-loaded [22:53:51] (03PS16) 10Aarcos: Add truncatable text field, use for some fields [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [23:20:38] (03CR) 10Jforrester: "Just to note that this won't achieve ES3 compatibility; OOjs is riddled with ES5-only tools like Object.create, Object.keys, Array.isArray" [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124365 (owner: 10Gergő Tisza) [23:39:17] (03CR) 10Aarcos: [C: 031] "This looks good to me but it is relatively big change so it wouldn't hurt to have another pair of eyes review it." [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/121309 (owner: 10MarkTraceur) [23:39:47] (03PS2) 10Gergő Tisza: [WIP] Guess thumbnail URLs without extra API call [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 [23:40:51] (03CR) 10jenkins-bot: [V: 04-1] [WIP] Guess thumbnail URLs without extra API call [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/124796 (owner: 10Gergő Tisza) [23:51:54] Added our config patch to [[Deployments]] [23:51:55] https://wikitech.wikimedia.org/w/index.php?title=Deployments&diff=109207&oldid=109197 [23:52:06] We're in it now