[05:29:45] is there a limit on how much text can be in a GET query parameter? [06:07:58] legoktm: I think the answer to that is "it depends" and is based on the web server stack you are using [06:08:30] in Wikimedia production? [06:10:20] there would definately be some total query string length limit [06:10:29] probably around 4k [06:11:30] looks like maybe 8k -- https://httpd.apache.org/docs/2.4/mod/core.html#limitrequestline [06:43:23] bd808: thanks, https://gerrit.wikimedia.org/r/393523 is why I was asking. 4k or 8k should be fine for 99% of uses so I'm just going to ignore it for now [14:09:14] legoktm: Because just using the same topic doesn't make Jenkins not merge if someone +2s early. [17:39:29] anomie: so i just had an idea. the wltoken thing we have for watchlist RSS sucks. could we use a BotPassword for that instead? [17:41:06] MatmaRex: You mean like setting wlowner to the BotPassword username and wltoken to the BotPassword password? That could work, although it'd be more setup for people wanting to use it. [17:41:33] i.e. they'd have to go through the trouble of setting up a BotPassword, with the correct permissions. [17:41:39] s/permissions/grants/ [17:43:07] anomie: i'm thinking using a BotPassword for authentication there, instead of the wltoken, which is a weird special exception. ideally the BotPassword would be set up when clicking the RSS link on your watchlist somehow magically. [17:43:24] anomie: i'm asking if this idea makes enough sense to be worth filing a task about [17:45:04] MatmaRex: If you're meaning actual action=login, generic RSS readers probably don't know how to do anything more complicated than handling HTTP 401. Or do they? [17:46:51] I guess having the API module have a parameter to tell it to do HTTP 401 Basic to get the reader to send the BotPassword credentials might work. [17:47:00] anomie: yeah, no, probably not actual action=login. i have not thought this through, it's just a very vague idea [17:47:25] Or a generic RSS reader might support OAuth. In which case, they could use Extension:OAuth. [17:59:57] Is there a way to tag Action API requests so you can see the requests in logstash (or some other log)? [18:00:19] (these are requests being made directly from the client) [18:01:31] i see requestid, but I'm not sure if that will show up in the log(s) and if that can be non-unique. [18:01:56] All action API requests are logged to mwlog1001:/srv/mw-log/api.log. They don't go to logstash because the volume of log entries from that channel would be too much. [18:03:04] requestid isn't specifically logged, but will show up anywhere parameters in general are logged. It's allowed to be non-unique; MediaWiki doesn't care what's in the value, it just copies it to the response. [18:06:58] ... Actually, it looks like my information was outdated. Logstash does have channel:api. [18:08:24] * anomie wonders when that happened [18:11:34] anomie ah! but if there's not a way to filter by requestid (or tag it some other way) then that isn't all that helpful... but maybe all that would need to be done to support that is add the requestid to logstash? [18:14:51] davidwbarratt: You can search for requestid like https://logstash.wikimedia.org/goto/3f9fcd59075961970f5b6ef15550a56d. Adding it as an actual field in the context could probably be done, if needed. [18:15:55] anomie oh great! thanks! [18:16:22] Oh, duh [18:16:54] davidwbarratt: Bad news. That search isn't finding the "api" channel, it's finding channel:api-feature-usage. [18:17:55] logstash doesn't have the hardware to keep up with the full action api usage feed. That firehose is just too much data [18:18:03] So api.log on mwlog1001, or data in Analytics's stuff (https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/ApiAction I think documents it?) [18:20:59] doh! https://phabricator.wikimedia.org/T180088#3789645 [18:21:38] so uhh... no way to do that right? [18:22:31] You could get access to the Analytics host to run a query through beeline, or you could do a big grep over api.log. [18:23:31] davidwbarratt: eventlogging is another way you could track that sort of thing possibly [18:24:04] EventLogging is probably the more normal way to do it. [18:24:32] https://www.mediawiki.org/wiki/Extension:EventLogging && https://www.mediawiki.org/wiki/Extension:EventLogging [18:24:37] legoktm: max URL length is 2083 or something like that if you care about old IE compatibility [18:24:44] err -- https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging [18:26:06] bd808 anomie thanks! [18:28:07] would be really nice if there was a way to "tag" requests and have them (the request params and the headers) be in logstash.. but I supposed that still could be too many requests? [18:28:15] davidwbarratt: in practice you'd probably want to measure timing on the client since everything else is an estimate at best, and EventLogging is the only thing that works there [18:28:39] errr... it would only put them in logstash if you have added a tag [18:29:15] Analytics is looking into tagging requests in Kafka, that might be another option in the not-so-near future [18:29:16] it could be done with some fancy monolog configuration, but it would be tricky in lots of ways [18:29:45] tgr ah! yeah that's what I was really thinking about [18:29:58] tgr analytics more than time it takes [18:30:36] we have some kind of tagging for logstash, adding an X-Wikimedia-Debug header sends things to a separate log channel [18:30:44] but that's meant for a different use case [18:30:52] ah true [19:41:19] Curious if this is still relevant, or whether logging has improved enough since. Old rdbms patch. https://gerrit.wikimedia.org/r/#/c/248340/1 [19:52:55] TimStarling: tgr: Curious about status on https://gerrit.wikimedia.org/r/#/c/338901/ - https://phabricator.wikimedia.org/T45086. Not sure I understand what's next and/or whether the patch is still needed. [19:56:54] ooh custom gerrit logo [19:56:58] nifty [20:08:32] tgr: thcipriani whipped that one up for us so we could ditch the ugly b&w one :) [20:08:37] Krinkle: the goal of the patch is to be able to inspect suppressed errors [20:09:30] tgr: It looks like we already get those though? Or are we using 'error' in logstash now instead of error-json? [20:09:39] I thought we transformed error-json into error in logstash ingest config [20:09:50] I've definitely seen various entries with "supressed: false" [20:10:38] I think we were planning to get rid of error-json, let me try to find the task [20:11:16] but anyway, still useful on a vagrant box / other non-logstash environment [20:12:14] no_justification: cool. wanna also update the phab avator for gerritbot? [20:21:12] tgr: OK. That makes more sense. [20:21:21] I've updated the commit slightly to reflect this [20:21:29] Krinkle: in any case, it doesn't seem like we are sending *-json to logstash anymore [20:21:31] As a result, if we do use error-json still that means the patch won't break it [20:21:42] And we have the option to migrate it smoothly [20:21:55] To the non-json variant [20:22:43] wmgMonologChannels doesn't mention it at least, maybe there is some custom config for it somewhere else? [20:46:04] bd808: https://gerrit.wikimedia.org/r/#/c/393630/