[03:54:48] can an op here restart all the bots that should be on IRC? [03:54:51] op/dev* [03:59:26] wm-bot works :) [04:44:30] labs-morebots: how's it going? [04:44:31] I am a logbot running on tools-exec-1208. [04:44:31] Messages are logged to wikitech.wikimedia.org/wiki/Server_Admin_Log. [04:44:31] To log a message, type !log . [06:50:16] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Dinusha was created, changed by Dinusha link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Dinusha edit summary: Created page with "{{Tools Access Request |Justification=Offer application |Completed=false |User Name=Dinusha }}" [08:58:14] 06Labs, 10Tool-Labs, 10DBA: labsdb1001 and labsdb1003 short on available space - https://phabricator.wikimedia.org/T132431#2384740 (10jcrespo) [08:58:49] 06Labs, 10Tool-Labs, 10DBA: labsdb1001 and labsdb1003 short on available space - https://phabricator.wikimedia.org/T132431#2198300 (10jcrespo) labsdb1003 is now also at 10% free space. [09:04:34] 10Tool-Labs-tools-Other, 07I18n: [[Intuition:Monumentsapi-title/en]] i18n issue - https://phabricator.wikimedia.org/T137951#2384748 (10Beta16) [10:03:27] #wikapedia [10:37:36] 10Tool-Labs-tools-Other, 07I18n: [[Intuition:Monumentsapi-title/en]] i18n issue - https://phabricator.wikimedia.org/T137951#2384748 (10Liuxinyu970226) #wiki-loves-monuments ? [11:33:45] 06Labs, 10Phabricator: Upgrade phab-01.wmflabs.org - https://phabricator.wikimedia.org/T127617#2384974 (10Pokefan95) [11:33:49] 06Labs, 10Phabricator: https://phab-01.wmflabs.org (and other installs, like phab-02) returns a 502 error - https://phabricator.wikimedia.org/T137270#2384972 (10Pokefan95) 05Resolved>03Open phab-02 to phab-04 are working, but not phab-01. The error message was: >[Core Exception/PhutilBootloaderException]... [11:37:34] 06Labs, 10labs-sprint-116, 10DBA, 13Patch-For-Review: Make watchlist table available on labs - https://phabricator.wikimedia.org/T59617#2384976 (10jcrespo) I have now the watchlist count generator running on all public wikis. I will measure how much it takes for a second run (after indexes are hot) on all... [11:37:53] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations, 07Blocked-on-Operations: No replica for adywiki - https://phabricator.wikimedia.org/T135029#2286195 (10jcrespo) [11:37:56] 06Labs, 10labs-sprint-116, 10DBA, 13Patch-For-Review: Make watchlist table available on labs - https://phabricator.wikimedia.org/T59617#2384978 (10jcrespo) [11:38:11] 06Labs, 10labs-sprint-116, 10DBA, 13Patch-For-Review: Make watchlist table available on labs - https://phabricator.wikimedia.org/T59617#609458 (10jcrespo) [11:47:04] 06Labs, 10Phabricator: https://phab-01.wmflabs.org (and other installs, like phab-02) returns a 502 error - https://phabricator.wikimedia.org/T137270#2385000 (10Paladox) @mmodell I think the phabricator wikimedia extension needs updating in phab-01 please. [14:38:30] 06Labs, 10Tool-Labs, 10DBA: p50380g50816__pop_stats (popularpages) using 53G on labsdb1001 (enwiki) - https://phabricator.wikimedia.org/T133326#2228235 (10Stevietheman) What are the current development needs of this tool? I'm really missing its output in projects I'm associated with. (note that I'm not com... [15:23:49] 06Labs, 10Tool-Labs, 13Patch-For-Review: Figure out a way to keep MerlBot running when the HTTP POST loophole is closed - https://phabricator.wikimedia.org/T121279#2385561 (10Luke081515) @bd808 Concerning the http libary etc MerlBot is using, I found T105794#1486712. [15:28:44] I'm no longer seeing tools-prometheus instance under tools, anyone knows what happened to it? [15:45:51] godog: I have the vague recollection that it was giving me trouble and yuvipanda told me to 'just delete it'. [15:45:53] But, let me check... [15:47:11] godog: yeah, definitely not there anymore. I don't know what it was/what it was for/what it was doing, though, so can't help much beyond that [15:54:35] andrewbogott: np, yuvipanda was right no harm done, I'll create it back [15:57:48] 06Labs, 10Tool-Labs, 10DBA: p50380g50816__pop_stats (popularpages) using 53G on labsdb1001 (enwiki) - https://phabricator.wikimedia.org/T133326#2385640 (10kaldari) > did you get a chance at further deletion @kaldari? @chasemp: I've deleted everything older than 2011 now. Can you see how much disk space is b... [16:00:44] 06Labs, 10Tool-Labs, 10DBA: p50380g50816__pop_stats (popularpages) using 53G on labsdb1001 (enwiki) - https://phabricator.wikimedia.org/T133326#2385658 (10jcrespo) > Can you see how much disk space is being used currently? ``` du -hs p50380g50816__pop_stats 50G p50380g50816__pop_stats ``` [16:27:38] 06Labs, 10Labs-Infrastructure: Copy labmon data to new SSDs - https://phabricator.wikimedia.org/T137924#2383843 (10RobH) [16:58:08] labs gurus: anybody encountered puppet getting stuck on labs? I have a number of labs machines where puppet just gets stuck in update and sits there basically forever [17:43:24] SMalyshev can you 'ls /public/dumps' and see if that returns? [17:43:34] if it does, then let me know and I'll take a deeper look [17:43:51] if it Doesn't then it's fall out from an NFS change from almost a month ago, and basically needs an instance restart :( [17:44:37] (I'll brb) [17:45:18] yuvipanda: looks like reboot fixed it... [17:45:27] ok! [17:45:29] sorry about that [19:21:28] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Dinusha was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=659948 edit summary: [19:33:39] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Fabian24d97 was created, changed by Fabian24d97 link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Fabian24d97 edit summary: Created page with "{{Tools Access Request |Justification=I am planning to use the database access to gain data for my new project WikiGraph (https://github.com/fdamken/wikigraph) which is about..." [20:42:48] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure, 10MediaWiki-Unit-tests: CI jobs failing with DNS resolution errors such as "Could not resolve host: gerrit.wikimedia.org" - https://phabricator.wikimedia.org/T137460#2386808 (10hashar) And something I thought about: maybe the base Jess... [22:04:49] 06Labs, 10Tool-Labs, 10DBA, 06Operations, 10Traffic: Antigng-bot improper non-api http requests - https://phabricator.wikimedia.org/T137707#2376097 (10Legoktm) >>! In T137707#2382924, @Antigng_ wrote: > Lack of hard and fast limit on read requests can be a problem, since your definition of request limit... [22:09:47] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Fabian24d97 was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=660787 edit summary: [22:35:57] 06Labs, 10Tool-Labs: Missing replicy.my.conf on Tools Labs - https://phabricator.wikimedia.org/T138037#2387374 (10Fabian24d97) [22:37:02] 06Labs, 10Tool-Labs: Missing replicy.my.conf on Tools Labs - https://phabricator.wikimedia.org/T138037#2387388 (10Fabian24d97) p:05Triage>03Normal