[00:17:39] TimStarling: do you know why Zero implemented qlow for JPGs but not for PNGs? [00:18:07] PNGs are lossless, there's no quality setting [00:21:58] well, that's only sort of true [00:22:09] you can remove colors from the palette [00:24:32] right... [00:24:42] and some encoders can will try harder to compress if you tolerate increased runtime / resource utilisation [00:25:10] we already use the maximum compression that imagemagick allows [00:25:27] ah [00:25:59] OK, so probably no big gains to be had there [00:26:57] we should stop rasterizing trivial SVGs to PNGs on browsers with native SVG rendering capabilities, that'd be a big win [00:27:04] I don't think we produce indexed colour PNGs under any circumstances [00:27:47] http://caniuse.com/#feat=svg says native SVG support is at 96.43% globally [00:28:05] but note that downscaled images typically have more colours than the source image, due to antialiasing [00:28:20] good point [00:28:55] in fact when we first started downscaling PNGs and GIFs we noticed that the file size of the thumbnail is often about the same size for the thumbnail and the source image [00:29:10] heh [00:29:21] you are basically moving information from space to colour, not losing it [00:29:50] yeah. that's a counterintuitive result, but it makes sense on reflection. [00:31:11] as for SVGs, note that the rendered PNG is sometimes smaller than the source SVG [00:32:49] I don't know how common that is, we should probably do some statistics [00:32:50] yes, but SVGs are more desirable, because vector and scriptable. There are some terribly bloated SVGs, but we'd do better running those through something like https://github.com/svg/svgo than rasterizing [00:34:32] we've been doing that manually, for user chrome SVGs that we ship with the software, and getting good results [00:37:01] here's a typical example: https://upload.wikimedia.org/wikipedia/commons/thumb/f/f8/2006Australian_exports.svg/300px-2006Australian_exports.svg.png [00:37:21] that is a thumbnail shown on [[Australia]], it is 26KB [00:37:36] the SVG source is 1.4MB [00:37:52] That map is useless at 300px. [00:37:55] But sure. [00:38:22] let's try compressing... [00:43:23] compressed with gzip it is 454KB [00:43:41] with svgo it is 897KB uncompressed, 355KB compressed [00:44:20] so still 13.7 times bigger than the PNG [00:44:59] argh. yeah. [00:44:59] try scaling it down if you're planning to render it in small size. svgo will remove invisible details from paths, i think. [00:45:13] (scaling it down before optimizing) [00:45:45] seems like cheating, but ok [00:47:04] it's not useless at 300px; the PNG thumbnail is doing exactly what it should, which is give you some sense of what you'd see if you were to download the full image [00:47:12] I'm using https://jakearchibald.github.io/svgomg/ which doesn't have an option for scaling [00:47:16] so yeah, we should probably do some statistics [00:47:29] maybe they are not so common [00:47:40] it is probably pretty common with maps [00:47:42] but moving the precision slider to 1 reduces the size to 181KB [00:48:43] maps on commons are often unreasonably detailed. some also contain lots of hidden markup, in vain hope that someone will use a text editor to unhide it and enable some effects. [00:48:46] yeah, but at that point you're losing fidelity, so SVG loses the high ground [00:50:15] here's a non-maps example [00:50:49] featured articles have a little star in the top left corner, a 20px thumbnail [00:50:55] 987 bytes of PNG [00:51:09] the source SVG is 67KB: https://en.wikipedia.org/wiki/File%3ACscr-featured.svg [00:51:35] top right corner, rather [00:53:47] all icons have the same problem [00:55:27] But OTOH you have things like https://upload.wikimedia.org/wikipedia/commons/6/6e/Buckling_curvature_2.gif [00:56:07] I don't know if it exists in vector format, but I imagine it would be a lot smaller [00:57:41] obviously you won't get me to say anything nice about animated GIF [00:58:08] they've been great for cats [00:58:25] :P cats in 256 colours [00:58:45] brion said on facebook that he was looking into SVG animation [00:59:46] allowing client-side scripting is difficult [00:59:59] I mean, user-authored client-side scripts [01:01:05] preventing DoS attacks against the browser would be almost impossible [01:01:47] but he's right that there is big payoff to solving that [01:01:48] (pity smil's dead) [01:02:02] https://upload.wikimedia.org/wikipedia/commons/4/4f/Soccer_ball_animated.svg [01:02:03] try clicking on https://upload.wikimedia.org/wikipedia/commons/6/6c/Trajans-Column-lower-animated.svg [01:03:51] we could use SMIL, I don't mind using dead things [01:04:48] or some other domain-specific language [01:06:09] chrome gives ominous console warnings about it, though. "SVG's SMIL animations (, , etc.) are deprecated and will be removed. Please use CSS animations or Web animations instead." [01:07:32] (CSS animations are pretty meh - although you could do that rotating ball fairly easily at least - and "Web animations" i think mean just "JavaScript" here) [01:07:33] a DSL that compiles to javascript sounds feasible [01:08:40] https://leunen.me/fakesmile/status.html huh. [01:10:01] (i think i side-tracked this conversation, sorry) [01:10:51] no, that's interesting :) [01:11:55] it is a different problem -- we're talking about supporting new kinds of content (or improving support for existing content), rather than the more prosaic and less interesting problem of keeping asset sizes small [01:12:28] but i don't know that there's much left to say on the asset size question without some additional data [01:12:39] we can always check the thumbnail size after we downscale and then send the original if it is smaller [01:12:53] for the purposes of keeping page assets size small I suspect that a crude heuristic of the form "don't rasterize SVGs that are smaller than X kilobytes" would do [01:13:18] you can presumably estimate PNG/JPEG sizes based on area [01:13:40] that's what I'd use for a heuristic [01:14:04] could do [01:14:05] send SVGs that have less than some number of bytes per pixel area [01:14:42] there is another aspect to the SVG vs PNG question which is cost to render (vs. network costs) [01:16:29] http://frozeman.de/blog/2013/08/why-is-svg-so-slow/ [01:47:53] "< ori> could do" -- I see what you did there [01:48:34] * ori doesn't get it [02:04:38] ori: I guess I see "could do" as a TimStarling catch phrase and you used it in a conversation with him [02:05:04] heh. wasn't conscious. maybe https://en.wikipedia.org/wiki/Mirroring_(psychology) . [02:06:52] i have two other instances of a bare "could do" in my irc logs, one is Tim, the other is you [02:08:12] * bd808 remembers Tim unmuting briefly during many team meetings to say "could do" when prodded by RobLa to weigh in on a proposal [02:09:09] for values of many that may or may not be greater than 3 [02:12:52] TimStarling: does the plan I wrote up in the ticket description of https://phabricator.wikimedia.org/T119797 make sense to you? [02:14:33] anomie: is there a way to mock whether the user is logged in? [02:15:14] if they're near esams you could make fun of them for destroying their latency [02:16:18] "I hope you like seeing your name in the top right corner, because you just went to Virginia and back for that." [02:33:33] ori: when you say geolocation, do you mean https://www.maxmind.com/en/geoip2-connection-type-database ? [02:34:03] if you didn't then you probably mean it now, right? [03:02:27] ori: I'm going to Virginia in a few weeks. Maybe I can bring a list of names back so we can cache them at ulsfo? [03:19:51] * robla starts reading the backlog [03:28:28] MatmaRex: I know way more about SMIL Animation than I should admit, since I'm partly to blame for it being murder-worthy [03:30:17] :o [03:32:12] SMIL 2.0 was the product of a standards political battle between RealNetworks, Microsoft, and CWI. no one won :-( [03:33:38] a camel was born with humps in all of the wrong places [03:40:51] TimStarling: do you think CSS animations or Web animations are secure enough to replace SMIL, or is there still a need for SMIL as an animation DSL? [03:41:51] I'm not really an expert in this [03:42:13] my understanding is that CSS animations are very limited, and SMIL is not so limited [03:42:38] I don't know what a web animation is [03:43:39] I'm not looking to bring SMIL back from the dead it truly has been replaced by something simpler/better, but I think I still understand enough about SMIL to be useful in helping out as necessary/appropriate [03:44:35] those of us on the SMIL working group were probably all a little too naive about the security implications of what we were doing [03:44:58] the current best practice is client-side scripting [03:45:20] which is fine, it's nice to have a small set of client-side technologies [03:46:15] obviously we can't emit user input HTML without validation, so we shouldn't be surprised that the same is true for SVG [03:47:00] but scripting doesn't seem like a great UI for artists and designers to me [03:47:19] I think we (W3C SYMM working group) accidentally made something more secure than open-ended scripting. The Chrome folks of today are probably more savvy about security than the W3C members of yesteryear [03:48:12] and yeah, one of the things we talked about a lot was making something that could be generated/edited by tools [03:50:06] if there is an authoring tool that produces a safe intermediate file format output, then that would be a good situation to be in [03:50:26] then we can convert it from that format to something which can be interpreted on the client side [03:50:49] Patrick Schmitz was the guy who really understood multimedia authoring tools. Sadly, he was at Microsoft at the time, so the politics of working with him was rather difficult [03:51:00] I don't think he [03:51:22] I don't think he's there anymore, and it'd be interesting to see what he's up to... [03:54:26] interesting, he's at UC Berkeley now: https://www.linkedin.com/in/schmitzpatrick [03:59:42] this naming isn't confusing at all: https://en.wikipedia.org/wiki/Microsoft_Chrome [04:00:29] I believe Patrick was part of Microsoft Chrome [04:07:45] just reading some designer-oriented articles [04:07:56] e.g. http://www.webdesignerdepot.com/2013/10/html5-app-smackdown-which-tool-is-best/ [04:08:46] * robla follows the link and starts reading [04:09:42] well, that article is not exactly written for us [04:10:15] but it gives you some idea of what designers need [04:15:09] it depresses me how grim it still is for animation authors [04:16:22] a lot of my full time work at Real was spent thinking about multimedia authoring, and how we thought we were defining the future, so that article actually depresses me a little [04:16:33] we _still_ don't have jetpacks :-/ [04:17:03] about GWD: "However, it is only free if you do not value your time, as you will be spending a lot banging your head off the desk." [04:18:36] MatmaRex: … and "Web animations" i think mean just "JavaScript" here <-- I think it means http://www.w3.org/TR/web-animations/ [04:19:00] GWD: "half-hearted shovelware" [04:19:45] * robla reads the web-animations link [04:23:11] from the web-animations draft: "It is intended that by defining SVG’s animation features in terms of the Web Animations model, the dependency between SVG and SMIL Animation can be removed." [04:24:18] my quick skim of that draft though makes me think you're right, prtksxna. it looks very DOM-o-riffic to me [04:27:19] dom-o-riffic, heh [04:29:22] yeah, that means one person will be able to author something using one web-animations authoring tool, and then someone else will be able to edit it with another web-animations tool. dom-o-riffic! [04:30:25] of course, the tools will be great "if you do not value your time, as you will be spending a lot banging your head off the desk." [04:37:38] on the inkscape wiki there are some ideas and UI mockups for SVG+SMIL, mostly dating to around 2009 [04:37:41] I wonder what prototyping tools like Hype (http://tumult.com/hype/) use for animation. Maybe pginer knows. [04:38:37] Or https://www.google.com/webdesigner/ [04:38:51] that's what we've just been reading about [04:38:55] hype and webdesigner [04:39:10] they output HTML+JS directly [04:39:42] Oh, I missed that in the backlog :\ [04:40:13] there may be a lot that can be done with CSS that wasn't possible back when we were writing SMIL Animation [04:41:14] that comparison article mentioned that adobe edge uses CSS for simple things and JS for complex things [04:41:54] how about http://wiki.synfig.org/Sif2svg [04:41:54] I'm just wondering what is meant by "simple" in modern terms [04:43:20] http://www.synfig.org/cms/en/features/ [04:44:26] it is in ubuntu, package name synfigstudio [04:45:02] The Synfig stuff looks interesting [04:45:13] robla: I'd say anything that's possible with CSS transitions (maybe even CSS animation) [04:50:46] TimStarling: (re: MaxMind) that was my first thought too -- see https://phabricator.wikimedia.org/T119801 . But since MaxMind's netspeed database does not differentiate between different types of mobile connections, it is not very useful. [04:50:59] http://alternativeto.net/tag/animation/?license=opensource looks like a good list to investigate further. [04:51:08] better than lumping a whole country in together [04:53:25] we log country and region with NavigationTiming data, so we could make it slightly more granular. And we could also have JavaScript code that upgrades the netspeed designation based on client-side timing data [04:55:01] Personally speaking, I'm not sure the low-res variant would be all that punishing. I'm in SF with LTE and I think I'd prefer whatever loads quickest. [04:56:03] I don't want to open up THAT can of worms, though. [04:56:56] ori: don't worry, we're having a big worm party here tonight! we have all the worms celebrating their can liberation! [04:57:30] heh [04:58:09] * robla tries to backtrack to the low-res part of the conversation [04:58:17] I don't know if ori's capacity for patience is a good metric in quality vs speed discussions ;) [04:58:24] heh [05:00:03] ori: what is your current thinking on the right near-ish term approach to dealing with SVG thumbnailing? [05:00:55] I don't know. My intuition was pretty wide of the mark. [05:02:08] I have no clue myself. I've got some ideas for some longer term SVG animation strategy work that we could explore, but I've got no idea on a short term thing [05:02:15] * robla wants to be spoon fed [05:04:27] I tried naively and indiscriminately changing all the rasterized svg pngs to the original svg for [[Barack Obama]] and it increased the payload size from 756kb to to 1,250kb -- http://www.webpagetest.org/result/151201_41_4C9/ [05:06:11] CPU utilisation is dramatically different too [05:07:12] well, like you say, there are already some SMIL animations [05:07:30] ori: I'm not sure how to read the results page you linked to, but I'll take your word for it. that's a bummer [05:07:33] we could have an image link parameter which forces client-side rendering, for the purposes of animation [05:08:42] * ori shares http://dn.ht/picklecat/ [05:08:50] and also we need client-side SMIL emulation for clients that don't have it [05:10:08] ori: directly scripted [05:10:22] would make for excellent goatse vandalism wouldn't it? [05:10:56] haha [05:13:24] pickle cat wants to live on Wikipedia [05:13:47] who are we to deny pickle cat? [05:19:40] his pickleness [05:19:57] or her, I'm not sure [05:21:17] robla: here is just the CPU utilization graphs for the two page loads -- the top one for Obama with SVGs, the bottom with rasterized PNGs: http://i.imgur.com/Y5lhIq8.png [05:23:04] ok, I can read that. ouch. [05:23:17] anyways, I think (again) that these are two separate problems. In the cases where SVG provides a palpable value added, then the performance penalty is moot, since the PNG is not even a close approximation [05:25:21] in the case of static SVG thumbnails it looks like rasterizing is the right call more often than not [05:25:56] I really didn't expect that; I thought it was just a case of the browser support for native SVGs not being there when this was implemented [05:26:49] even if we decided it was a palpable value add and wanted to make a switch, we would need to figure out how to ease the site into it rather than flipping a switch. [05:27:44] there are probably a lot of cases where page authors would make a different choice given their own page viewing experience [05:28:45] probably best to start with a specific example [05:29:16] where's one of the more expensive graphics on the Barack Obama page? [05:30:34] I mean start by identifying a particular SVG that would substantially improve an existing article if rendered natively within it [05:31:13] I don't think there are any on Barack Obama, it's mostly flags and things like https://upload.wikimedia.org/wikipedia/commons/1/11/Barack_Obama_signature.svg [05:32:35] (the B looks suspiciously like the head of Mr. Linea https://upload.wikimedia.org/wikipedia/en/a/ac/La_linea_.jpg ) [05:34:35] ori: I suppose the map on this page might be a good example: https://en.wikipedia.org/wiki/United_States_presidential_election,_2012 [05:35:45] looks OK to me as a PNG [05:36:55] someone had to create https://en.wikipedia.org/wiki/Template:United_States_presidential_election,_2012_imagemap [05:38:18] creating a vector overlay on an PNG thumb of an SVG. ugh [14:13:33] tgr|away: Try setting a logged-in User object into $wgUser and/or passing it to RequestContext::getMain()->setUser() and/or ->setUser() on whatever DerivativeContext object gets passed into the thing you're wanting to mock a user on. To get an actual logged-in User object, you'll likely need to declare your test class as @group Database, then you can either use UTSysop or create your own test user with the usual User methods. [18:21:54] anomie: can you point me to any examples of editors doing careful profiling work to improve the performance of Scribunto modules? I have seen it before and am looking for it again now. Figured you might know off the top of your head. [18:23:22] ori: https://en.wikipedia.org/wiki/User_talk:BJorsch_%28WMF%29#Performance_tests is one example. [18:24:29] anomie: ok, that's excellent, exactly what i was looking for. (thanks!) [18:24:31] got any others? [18:25:23] oh here we go: https://en.wikipedia.org/wiki/User:Dragons_flight/Lua_performance [18:26:33] ori: That's the other one I can thing of, yeah. You could also look into the history of pages like https://en.wikipedia.org/wiki/Wikipedia_talk:Lua and https://en.wikipedia.org/wiki/Wikipedia:Lua/Requests [18:26:40] s/thing/think/ [18:27:53] fantastic, thanks [18:50:23] * anomie can't make any sense of what T119978 is trying to talk about [18:56:48] anomie: I think he wants to be able to invoke a module as it was on a given date, with the name that it had then. [18:57:36] so changing and renaming modules does not break callers [20:49:36] * ori needs review for https://gerrit.wikimedia.org/r/#/c/256051/ , which he's hoping to get out soon [22:02:06] bd808, tgr: could either of you possibly take a look at https://gerrit.wikimedia.org/r/#/c/256051/ ? [22:09:00] ori: where do the schemas live? On meta somewhere? [22:09:31] https://meta.wikimedia.org/wiki/Schema:NavigationTiming?oldid=14899847 [22:12:24] bd808: \o/ thanks very much. [22:12:27] yw [22:46:42] bd808: will anything blow up if I install a few debs manually on a beta cluster server? [22:46:45] wrt https://phabricator.wikimedia.org/T112421 [22:47:47] should be ok unless the packages are marked with a specific version or "ensure => latest" in the puppet manifests [22:48:25] worst case is you'll have puppet fighting you and need to do `sudo puppet agent --disable "tgr testing new debs"` to stop that [22:50:44] and `sudo puppet agent --enable` when you are done testing and ready got Puppet to take over again [22:50:56] s/got/for/ [22:51:31] how often does puppet run? 5 mins? [22:51:37] 20 I think