[00:19:09] !log copypatrol copypatrol-backend-prod-01 deploy 60a68f3..8995949 - security update [00:19:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Copypatrol/SAL [01:40:32] Is there any efdicient way to download the thumbnail of 85k images from wikimedia server? [01:40:57] On-demand or by batch basis or a combination of both [03:47:35] you can see if it was cached or not in response headers and use that to guide your request rate. (re @nokibsarkar: Is there any efficient way to download the thumbnail of 85k images from wikimedia server?) [03:48:08] why are you getting so many? analyzing them or process somehow? (re @nokibsarkar: Is there any efficient way to download the thumbnail of 85k images from wikimedia server?) [03:49:47] Yep (re @jeremy_b: why are you getting so many? analyzing them or process somehow?) [05:08:13] does `webservice logs` currently show stdout? because otherwise there's no way my bot shouldn't have logged [05:16:33] actually nevermind, I just saw logs [08:01:16] nokibsarkar: the temp dir issue is probably because you are missing this option on your nginx config https://gitlab.wikimedia.org/toolforge-repos/wm-lol/-/blob/fakelb/nginx.cfg?ref_type=heads#L11 [08:09:09] Hi guys [08:09:47] hi [08:11:20] I'm just here to see if there is anybody who is interested in Ali Cloud services, or having any issues with Ali Could, I can help. [08:11:41] no thanks [08:11:53] Thank you for your time~ [08:32:28] Yep, now seems like it's working [08:33:54] thanks, again @dcaro [08:41:50] np :) [09:33:54] Regarding this, which headers should be interesting for me? I see a lot of headers but no suggestion of rate limit like how much I should wait before downloading the next image (re @jeremy_b: you can see if it was cached or not in response headers and use that to guide your request rate.) [12:22:59] !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed cae8c3c341 (upgrade dependencies, including toolforge 6.1.0; use toolforge.load_private_yaml() from T333728) [12:23:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [13:11:38] https://wikitech.wikimedia.org/wiki/Robot_policy#Media_API_rules (re @nokibsarkar: Regarding this, which headers should be interesting for me? I see a lot of headers but no suggestion of rate limit like how much...) [16:50:03] "pregenerated" is not accurate AFAIK. still using a 404 handler? (re @bd808: https://wikitech.wikimedia.org/wiki/Robot_policy#Media_API_rules) [16:54:59] @jeremy_b: there are both thumbs that we create and store on upload and 404 handler thumbs that are created in real-time when a cache lookup fails. [16:56:03] I always thought that was just the file description page is loaded after upload, uses a thumb, so now we have one generated. [16:56:25] by the 404 handler (re @jeremy_b: I always thought that was just the file description page is loaded after upload, uses a thumb, so now we have one generated.) [16:58:20] does that also apply when using the `imageinfo` API with `iiprop=url`? (re @bd808: https://wikitech.wikimedia.org/wiki/Robot_policy#Media_API_rules) [16:58:46] I just noticed that a test in my PagePile Visual Filter tool got broken, presumably related to these changes, but I’m struggling to understand the difference [17:01:03] (MediaWiki seems to be returning different responsiveUrls than it used to) [17:04:05] my understanding is that the api will indeed only now return urls to pre-generated thumbnail sizes [17:05:07] I’m seeing URLs with 250px, 330px and 500px in them, none of which are mentioned in https://noc.wikimedia.org/wiki.php?wiki=commonswiki&format=json&var=wgUploadThumbnailRenderMap [17:07:26] https://gerrit.wikimedia.org/g/mediawiki/core/+/02a71b18f5ec7d68074fab8532221bb51e66fd1c/includes/jobqueue/jobs/ThumbnailRenderJob.php is the thing that is fired to create "default" thumbnails in response to an upload. [17:07:38] Enqueued by https://gerrit.wikimedia.org/g/mediawiki/core/+/02a71b18f5ec7d68074fab8532221bb51e66fd1c/includes/filerepo/file/LocalFile.php#1567 [17:09:23] The sizes we set for WMF hosted wikis -- https://gerrit.wikimedia.org/g/operations/mediawiki-config/+/6fc702312c043c279d855cc83d4f4217a81fb2e6/wmf-config/InitialiseSettings.php#11230 [17:19:42] !log lucaswerkmeister@tools-bastion-13 tools.pagepile-visual-filter deployed 86ffb4c5c0 (upgrade dependencies, including toolforge 6.1.0; use toolforge.load_private_yaml() from T333728) [17:19:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.pagepile-visual-filter/SAL [17:20:11] fixed in https://gitlab.wikimedia.org/toolforge-repos/pagepile-visual-filter/-/commit/0cead0a5c36fc8451c9cad2f996c461de40b68f8 (re @lucaswerkmeister: I just noticed that a test in my PagePile Visual Filter tool got broken, presumably related to these changes, but I’m struggling...) [17:24:45] !log lucaswerkmeister@tools-bastion-13 tools.pagepile-visual-filter deployed 975e4e6800 (remove unneeded dependencies missed in previous commit) [17:24:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.pagepile-visual-filter/SAL [17:32:27] !log lucaswerkmeister@tools-bastion-13 tools.quickcategories deployed 56f47f15b7 (upgrade dependencies, including toolforge 6.1.0; use toolforge.load_private_yaml() from T333728) [17:32:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [17:41:00] !log lucaswerkmeister@tools-bastion-13 tools.ranker deployed a167fc8e71 (upgrade dependencies, including toolforge 6.1.0; use toolforge.load_private_yaml() from T333728) [17:41:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.ranker/SAL [17:43:56] thanks to whoever got gitlab.w.o/toolforge-repos/ included in codesearch btw, it’s very useful ^^ [17:44:12] lets me identify the remaining tools where I still need to replace `read_private()` with `toolforge.load_private_yaml()` https://codesearch.wmcloud.org/search/?q=def+read_private%5C%28&files=&excludeFiles=&repos= [19:50:41] @lucaswerkmeister: you're welcome. :) We added it after the mwclient breakage. T371992 [19:50:42] T371992: Index https://gitlab.wikimedia.org/toolforge-repos/ repos - https://phabricator.wikimedia.org/T371992 [20:26:09] * Krinkle looking at an apparent codesearch outage [20:29:37] OK, well, I don't lknow what's up but grafana-cloud shows the instance down, and http/ssh don't respond. [20:29:47] * Krinkle is going to restart from horizon [20:31:35] it's funny how everything grinds to a halt. I've got three windows open, each with about a dozen tabs, where the last tab I tried to open there is codesearch. After 3 things I tried to work on grinded to a halt, because codesearch timing out, decided I better fix that instead of starting a 4th task [21:56:06] !log lucaswerkmeister@tools-bastion-13 tools.speedpatrolling deployed 9d8920826c (upgrade dependencies, including toolforge 6.1.0; use toolforge.load_private_yaml() from T333728) [21:56:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.speedpatrolling/SAL [21:58:25] !log admin created new keystone role in eqiad1 and codfw1dev, 'object_storage' T396594 [21:58:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [21:58:30] T396594: Create OpenStack role that allows object storage access only - https://phabricator.wikimedia.org/T396594 [22:00:33] !log lucaswerkmeister@tools-bastion-13 tools.wd-image-positions deployed 9d8920826c (upgrade dependencies, including toolforge 6.1.0; use toolforge.load_private_yaml() from T333728) [22:00:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wd-image-positions/SAL