[00:02:06] that's true, stackoverflow has far more of those as well than questions about any other extension [00:12:24] Yeah. :-( [00:12:49] I really really want us to switch to VM image distribution for MW to reduce the difficulties for users. [00:22:16] James_F, omg think of the children who only have hared hosts with FTP access [00:23:26] Quite. [00:29:36] well, no one prevents the distribution of VM images; seems like more of a resourcing problem [00:29:40] Think of how our own Ops doesn't want to use other peoples' VM image distribution. [00:30:36] do we have any 2FA-related logs? [00:30:55] hmm, wrong channel I guess [00:31:13] asked again in the right channel [00:31:55] tgr: By covering the whole of mediawiki.org in great big pointers to .tgz files, *we* prevent that. Crowding out alternatives is non-zero cost. [00:34:31] are there any usable alternatives though? [00:34:46] promoting them in mw.org is the easy part [00:35:31] gwicke's experimental project is the only attempt I know of [00:35:52] ...experimental docker project... [00:37:04] also, whenever we ask the response is overwhelmingly that VMs are either too expensive or too hard to use [00:37:52] of course, people not needing VisualEditor also tends to be an overwhelming response which suggests that it's not exactly representative [00:38:20] would be nice if someone put more effort into polling potential/actual users [00:47:44] * James_F nods. [01:03:22] Seems I still can't actually login to my dev wiki [01:04:57] And when I can... [01:04:57] ( ! ) Warning: preg_match() expects parameter 2 to be string, array given in /var/www/wiki/mediawiki/core/includes/libs/time/ConvertibleTimestamp.php on line 85 [01:48:01] I keep seeing FOUC on Special:UserLogin locally [01:48:32] * Reedy moves his dev wiki sessions to redis [01:48:46] That seems to have fixed login... Something skewy with memcached still though it seems [03:33:59] bd808: Sorry, had to leave earlier (RE: slow-parse on Kibana) [03:35:24] earlier ~= 23 hours ago :) [03:35:52] bd808: https://logstash.wikimedia.org/goto/6af238e19e551385f575dbdb822c07ce [03:36:03] bd808: It's still sorting the numbers oddly though. 99, then 9 [03:36:12] presumably 100+ might be somewhere [03:36:15] but not shown [03:36:27] which is kind of the point :) [03:36:48] ugh. that's happening because we are indexing the values as strings rather than numbers [03:37:27] Yeah [03:37:45] bd808: I was looking into the DSL syntax and looking for a way to cast it as a number [03:37:48] just for one query [03:37:53] but couldn't find anything [03:38:01] but then again, I don't really know what I'm doing. [03:38:17] wouldn't be surprised if there is some syntax that would make it work without config changes [03:38:27] so... when we moved to es 2.x we added some rules to cast everything to strings on storage [03:39:08] I'm not sure if "time" is used in any other log events as a string [03:39:26] if it's not we could add a static mapping to make it a number [03:39:45] but that seems a bit scary with the other crap we've seen in the logs recently [03:40:45] I'd feel better about making the slow-parse meta data a bit less generic in the naming before adding the cast [03:41:16] but you're right that we may be able to add some stuff to the es query itself to cast for the sort [03:42:13] bd808: Yeah, either a way to cast ad-hoc to number, or a way to override sort to be nat sort. [03:42:20] Either would work I imagine [03:51:09] tgr, ostriches: https://www.mediawiki.org/wiki/Requests_for_comment/Moving_database_abstractions_out_of_MediaWiki_core [03:54:51] Krinkle: boo. "Visualize: scripts of type [inline], operation [aggs] and lang [groovy] are disabled" [03:55:52] I tried using -- {"script" : "Float.parseFloat(doc['time'].value)"} -- to cast the values as part of the aggregation [03:56:47] we could probably turn this on for the ELK cluster. Its locked down for the cirrus cluster because scripts can do scary stuff [03:57:20] legoktm: nice [03:57:24] Either way, this question probably warrants a phab task [03:57:28] is that something you are still planning to do? [04:00:16] Krinkle: erik and I have been talking about a fix for another problem that might actually make this all easier. We are having issues with naming collisions from multiple log sources and are considering a custom logstash plugin to add some hungarian notation to the field names at parse time to keep "time" as an int and "time" as an object from colliding [04:01:15] bd808: Interesting. Do we also want to apply that convention to MediaWiki by extend? [04:01:24] Or would we do it in MediaWiki's layers instead? [04:01:35] I'd rather not have to write 'intTime' in MW php code. [04:01:43] But maybe our abstraction layer could add it [04:01:46] we'd do it in logstash [04:02:02] for everything we process. [04:02:06] Interesting [04:02:29] this is a complication from elasticsearch 2.x internal changes [04:03:41] our hack fix would likely be to do something like leave string value names alone, and append/prepend a type hint for int, float, bool, and object value names [04:04:09] right now we cast everything except a short whitelist to string values [04:04:58] but that blows up still when one log event has a string named "args" and another one has an array of objects named "args" [04:05:30] and it makes problems like this sorting thing happen too [04:08:43] the bug for that problem is T150106 [04:08:43] T150106: Type collisions in log events causing indexing failures in ELK Elasticsearch - https://phabricator.wikimedia.org/T150106 [08:17:43] tgr: not right now though [11:33:24] jackmcbarn: pong [15:08:55] Krinkle: re https://phabricator.wikimedia.org/T116354 , nice chrome bug you've noticed, eh… [15:09:23] Krinkle: try toggling 'display: inline-table;' in the inspector for the – the blank space disappears, even when you toggle it back [15:09:34] the legend is still not displayed though… [16:41:22] bd808: Meeting? [16:50:30] anomie: doh [16:51:24] anomie: want to talk now or ..? I didn't load my calendar page this morning apparently [18:10:00] anomie: do you have an estimated timeline for when you want the API error warning stuff reviewed/merged? [18:20:12] legoktm: Not really. I did it because I finished the ORES quarterly goal early and there was an increase in people commenting on the error i18n task in Phab.