[03:28:00] Anyone know if [[Special:GlobalBlock]] accepts URL parameters the same way [[Special:Block]] does? [03:41:37] NVM, yes it does :P [06:18:27] anyone alive to answer a probably dumb question? [06:29:44] maybe [06:29:46] snotty: sure [06:30:01] hehe [06:30:35] i wrote some text files using getText.php [06:30:47] as in --> php getText.php "Apropos" > "Medien Apropos.txt" [06:31:26] the files exist, i can see them in two ftp programs, but i can't download them [06:32:05] What error? [06:32:54] ls -la gives me [06:32:56] -rw-rw-r-- 1 1472 Sep 3 07:43 Medien Apropos.txt? [06:33:14] i have NO clue what the ? means [06:33:58] possibly a permission issue with the directory it's in? [06:34:06] 644 [06:34:21] and i can get all OTHER files in that folder [06:34:35] just not the ones written by the script [06:34:57] and only those have a question mark after the filename in ls -la [06:36:41] using fireftp and totalcommander sftp plugin, no dice downloading, fireftp writes a 0 byte file, tv just gives an error [06:37:38] What if you move the file to 'test.txt' and try to download it? [06:38:44] mv "Medien Apropos.txt" ""Medien Apropos Test.txt" ? [06:39:24] aaaaaaaahhh [06:39:36] Yeah, something like that [06:39:39] autocomplete just gave me [06:39:48] "Medien Apropos.txt^M" [06:39:48] non-printable character in the filename? [06:39:51] yep [06:39:52] yep [06:39:58] I think ls -lb might detect that too [06:40:07] where does THAT comes form? my script, presumably [06:40:42] You're piping it out. Make sure there isn't any space at the end of that line. [06:40:52] i have a CR LF at each line's end [06:40:58] can taht cause it? [06:41:08] just LF, just CR? [06:41:19] <-- clueless windows user : ) [06:41:39] Windows text editors often add CR [06:41:46] notepad++ [06:41:56] just LF then? [06:42:17] There should be an option to change to unix line endings in notepad++ [06:42:46] loooking for that : ) [06:43:24] Unix systems use only a LF. Windows based use CR LF [06:43:36] and apples CR [06:43:40] thanks, world [06:44:09] Old apple is... new apple is unix AFAIK [06:44:15] found it [06:44:35] yeah...its all unnessarily annoying and confusing [06:47:44] \o/ [06:47:52] it works [06:47:55] whew [06:48:58] thanks guys : ) [06:51:33] np [06:52:08] bye! [14:40:00] https://www.irccloud.com/pastebin/djBsphBh/ [14:41:42] legoktm: ^ here you have a full profile. This server runs MW 1.31 w/ PHP 7.2.9 and the following configuration: https://github.com/miraheze/mw-config [14:43:12] database and redis servers are external but all with < 0.600 ms ping. [14:50:37] SPF|Cloud: do you have caching setup for that? It seems like its taking an excessively long time to load extension [14:51:29] I'm guessing, it could be something else in setup.php [14:51:44] that just seems like the main non-trivial thing in Setup.php [14:53:20] bawolff_: are you talking about a specific extension for loading? [14:54:09] SPF|Cloud: Actually nevermind, I think I just misinterpreted things [14:56:42] I'm not sure what's going on there, but your install is spending much much more time in Setup.php then it should [14:57:01] this profile is from our development server, which is somewhat specced worse than our production servers, but those production servers also heavily fluctuate between 150-500ms for the same page(!) [14:58:36] oh, wait, I forgot that the units are miliseconds not seconds. So its not as extreme as i first thought [15:00:26] spending 400ms in Setup.php stil seems pretty high, but no where as rediculous as 400 seconds [15:00:36] haha :P [15:00:53] fluctuation probably depends on other load [15:03:45] SPF|Cloud: Nothing super obvious pops out at me (But I'm not an expert at looking at these lists). I do notice it does make quite a few db queries, and message fetching is a big portion [15:03:57] agreed. though, shouldn't MediaWiki::* be much above 90%? [15:04:09] ah, okay [15:04:20] SPF|Cloud: So that would lead me to ask what type of caching do you have in place (What is $wgMainCacheType and friends) [15:04:53] https://github.com/miraheze/mw-config/blob/master/Redis.php [15:05:15] As in a normal request, with full caching setup, I would not expect 59 database requests in a single page view (unless it was specificly db heavy page) [15:06:07] SPF|Cloud: Do you have uAPC (or whatever its called now) installed? [15:06:18] php opcache? is installed and enabled [15:06:22] Also, $wgMessageCacheType = CACHE_NONE; pops out in that file [15:07:15] SPF|Cloud: Not the opcache aspect (although that's important), the local caching (On newer versions of MW, it can use multilevel caching, where it will go to apc for things that don't have to be shared between servers, and use the main cache type for the things that have to be shared amongst all the servers) [15:08:35] SPF|Cloud: I think you should change $wgMessageCacheType back to redis. Or if you don't want that for some reason, to CACHE_DB [15:08:46] fetching messages was a signficant portion of your profile [15:09:02] oh that. in June we tried memcached for local caching (so basically what you'd use APC for) and redis for the other stuff (we heavily intend to keep redis for jobqueue and sessions only) [15:09:59] Somehow despite $wgSessionCacheType = 'redis'; logins (CentralAuth, 95% similar to WMF setup) kept failing so we had to revert back everything to redis.. [15:10:54] I could set wgMessageCacheType to CACHE_DB, redis isn't an option due to limited RAM on the redis server. [15:12:52] SPF|Cloud: MessageCache is supposed to be small I think [15:13:10] Certainly much smaller than $wgMainCacheType [15:13:17] we do have 3000 wikis though :-) [15:13:27] are most of the keys shared in that case? [15:13:48] Umm, I'm not sure what's in wgMessageCacheType vs the other Localisation cache [15:14:04] In Wikimedia's case, wikimedia does special stuff with the localisation cache so its shared amongst all wikis [15:14:48] by the uAPC thing, I meant was ObjectCache::getLocalServerInstance() gets you, not $wgMainCacheType (or anything else you can set to memcache) [15:16:21] Hmm, I can't even figure out what $wgMessageCacheType is used for [15:16:47] CACHE_DB doesn't improve it, although for some reason I'm not seeing any entries in the l10n_cache table so that's why [15:17:35] localisation cache is separate from message cache [15:17:48] oh right. sorry! [15:17:51] honestly, I'm not even sure $wgMessageCacheType is even used anymore [15:17:55] all those things are confusing me [15:17:58] I can't find where its used in grep [15:18:09] If in doubt, there are 5 more layers of caching then you thought there were [15:19:20] AFAICT, the only thing $wgMessageCacheType is used for, is as a fallback for CACHE_ANYTHING values [15:19:38] * SPF|Cloud I can only find https://github.com/miraheze/mediawiki/blob/REL1_31/includes/GlobalFunctions.php#L3145 and https://github.com/miraheze/mediawiki/blob/REL1_31/includes/objectcache/ObjectCache.php#L245 [15:20:09] oh, there I found it [15:20:49] Its MessageCache::clusterCache [15:22:03] Ok, so message cache just stores a blob of all the messages for a wiki, I think [15:22:47] CACHE_DB actually seems to degrade performance [15:23:21] How significantly (out of curiosity)? [15:23:43] I think it is the sort of thing you probably want in redis (or memcache/whatever) [15:24:58] actually, nevermind [15:25:15] seems to be the heavy fluctuation I talked about [15:25:26] (The current mediawiki default is to put it in the database if no other caches are available. If putting it in db drops performance significantly, we probably want to change MW default) [15:26:01] For reference, the $wgMessageCacheType thing will show up in db in objectcache table, the stuff that has a kename starting with 'messages' [15:26:16] s/kename/keyname [15:27:13] MariaDB [metawiki]> select count(*) from objectcache where keyname like '%messages%'\G [15:27:14] count(*): 2 [15:28:08] contains :en and :nl so I guess it was actually working, just no improvement [15:30:06] SPF|Cloud: Based on the profile you gave, I think it would only make about a 5% improvement at most [15:30:57] SPF|Cloud: but you might want to retry getting a profile and see if Message::fetchMessage makes up less of the percentage now [15:31:04] nope, it didn't [15:31:22] sad :( [15:31:24] It's 18.54% [15:32:19] Why is MediaWiki::run only 70% though? [15:33:41] SPF|Cloud: Next thing I would try would be try setting up $wgLocalisationCacheConf . to use cdb (or LCStoreStaticArray which is supposed to be faster at least in hhvm) [15:33:48] I have a profile back from 2015 (iirc when we ran HHVM and MediaWiki... 1.26?) and in that profile MediaWiki::run is responsible for 95% [15:34:09] https://github.com/miraheze/mw-config/blob/master/LocalSettings.php#L411 already done.. [15:39:53] SPF|Cloud: so you have it set to manual recache. I assume you have all the other scripts in place to make that work (in that mode you have to run a script anytime you upgrade the wiki or install a new extension) [15:40:10] we always did/do that yes [15:42:32] * bawolff_ should point out I'm mostly guessing here, and don't look at MW profiles all that often, and may be saying stupid things [15:44:18] SPF|Cloud: for profiling purposes, it might be best to profile Special:BlankPage, to separate out stuff happening to every page, as opposed to some specific page [15:44:25] unless you have a specific page you are worried about [15:45:05] even on that page it's fluctuating much :P [15:45:36] Well flucuation is normal on a busy web server depending on what else is hitting the server [15:45:47] well, I know a main page on a wiki that takes 20 seconds to parse. I know its parser cache is disabled (randomising functions) but 20 seconds is a bit high ;) [15:46:26] I guess I'm overlooking something but I don't know what [15:47:17] On the whole though, nothing sticks out at me as taking an abnormal amount of type, and the things near the top of the list are things that aren't high-compute things [15:47:48] well Setup.php was excessive I think? don't know [15:48:40] My guess (And I'm a bit out of my knowledge area here, so take with salt) would be, that if you're getting slowness, its not a problem with MW code itself, but maybe something else on the system [15:49:15] SPF|Cloud: Don't let your users disable parser cache. Problem solved :P [15:49:41] heh, if they would listen to me... :) [15:50:56] SPF|Cloud: Take the wikimedia approach of not caring what they say :P [15:51:02] hehe [15:53:54] SPF|Cloud: with big pages that are uncached like that, you run the risk of that one page monopolizing your sites resources (It gets linked somewhere popular, suddenly you have 100 requests are piled up being rendered at the same time) [15:54:41] that's true.. varnish won't help me then either [15:55:15] Well disabling parser cache doesn't disable varnish cache (at least not by default) [15:55:21] but it won't help if the user's are logged in [15:55:30] For that sort of thing, Wikimedia uses pool counter [15:55:38] but its not that effective if you have cache disabled [15:55:42] disabling parser cache prevents varnish from caching it [15:56:33] anyways, for that specific page I found CurlHttpRequest::execute to be executed 104 times, so I think the culprit is not hard to find [15:58:41] Umm, that probably means your hitting somebody else's webserver repetitively [15:59:05] yes, seems to be those commons images [15:59:10] but I'd imagine we cache them? [15:59:15] Oh well if its Wikimedia, who cares :) [15:59:35] Its probably the image metadata [15:59:37] plenty of resources left for your cache proxies ;-) [15:59:48] Which indeed should be cached [15:59:57] InstantCommons hotlinks the thumbs now a days [16:01:22] SPF|Cloud: anyways, you should also try to keep track of server resources (How much ram is being used. How much time does php spend doing CPU vs waiting on IO, that sort of things). It may reveal insight into performance [16:02:28] we definitely try to do that, just remember that they are VPS' (yes, would love dedicated servers). that does explain a lot of things of course [16:03:53] I still don't get why it's hitting wikimedia's servers [16:10:47] SPF|Cloud: regarding performance, maybe look at other layers, like how fastcgi is configured and what not [18:12:23] I've been trying to enable the wikidata role of vagrant following the instructions of https://www.mediawiki.org/wiki/MediaWiki-Vagrant#wikidata but after following all the steps I get an uncaught error with the following message: Fatal error: Uncaught Error: Class 'Wikibase\DataModel\Entity\ItemId' not found in /vagrant/mediawiki/extensions/Wikibas [18:12:23] e/lib/WikibaseLib.entitytypes.php on line 35 [18:13:13] Are the instructions outdated and I need to do something else or is there a problem with the wikidata role? [18:28:46] Agabi10: try maybe running composer update in the extensions/wikibase directory [18:29:01] im not a wikibase person, so thats just a guess [18:34:47] Or maybe `composer install` if composer has never run before in that repo [18:35:26] vagrant provision should run that [18:35:29] bawolff with either of them I get "Nothing to install or update" in the console [18:36:35] Agabi10: Does the directory have a composer.json file in it? [18:37:09] yes [18:43:20] I suspect the dependency is missing (wikibase/data-model) but beyond that i dont know :S [18:45:18] all the files in my wiki are broken. i can see their names in Special:ListFiles, but they don't lead anywhere, and the images/ directory is empty. i can't import them because they are "Skipped" because they "exist" already. [18:46:58] if i use --overwrite with importImages.php i get the error "foo.png exists, overwriting...failed. (at recordUpload stage)" [18:47:23] khalella: so it sounds like your images directory got deleted/removed but mw still has all the image records in db [18:47:48] fine. can i reimport all the images via importImages.php [18:50:56] Khalella: what i meant was-am i understanding the situation correctly? [18:51:55] its something like that, i think [18:52:20] i think i ran importImages.php in a container which did not write to the correct location on disk... so the database got the writes, but the images didn't stick around [18:52:56] So its probably best just to copy the images directly to the place in the filesystem. ImportImages is more about the db records [18:54:04] The image file layout is take the md5 of the image filename (not the file but its name). And that forms the first two subdirectories [18:54:40] alternatively you could try killing the relavent records in the image db, so that importImages will work [18:54:59] cool. i think simply moving the images/ directory like you descrbed solved it [18:58:28] bawolff: I checked and the wikibase/data-model dependency is included in the composer.json file, so no idea what can be the problem... [20:28:57] I have legacy code in Common.js that references addPortletLink. I gather I need to use util.addPortletLink instead, but am unsure how to declare a dependency on mediawiki.util outside of an extension context. [20:30:50] Abulafia: https://www.mediawiki.org/wiki/ResourceLoader/Developing_with_ResourceLoader#Client-side_(dynamically) [20:31:10] mw.loader.using(['mediawiki.util'])... [20:31:32] Thanks Trela! [23:07:22] I have the "hideuser" right on my wiki but the "Hide username from edits and lists" option does not appear on the block page. [23:10:37] bonnedav: Are you set expiry as "indef"? [23:13:34] that worked, thank you. [23:14:03] np :)