[00:53:29] 6Labs, 10Math, 10MathSearch, 10Mathoid: Fix 503 on Help:Forumla purge - https://phabricator.wikimedia.org/T104549#1984515 (10Danny_B) [04:42:27] 6Labs, 10Tool-Labs: Clean up cruft in urlproxy - https://phabricator.wikimedia.org/T95442#1984574 (10scfc) Default `package.path` is `/etc/nginx/lua/?.lua;./?.lua;/usr/share/luajit-2.0.3/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/share/lua/5.1/?.lua;/usr/share/lua/5.1/?/init.... [04:45:25] 6Labs, 10Tool-Labs: Clean up cruft in urlproxy - https://phabricator.wikimedia.org/T95442#1984576 (10yuvipanda) Yup, we can just remove the packages line and it should 'just work' by default. [05:32:06] 6Labs, 10Tool-Labs: Fix htmlpurifier deployment - https://phabricator.wikimedia.org/T95815#1984594 (10scfc) 5Open>3Invalid No longer current with 1f8c07a9fd675b3c97fdf634acab189c81236e73 merged. [08:48:49] 10Tool-Labs-tools-Other, 6Community-Tech, 7Community-Wishlist-Survey, 7Milestone: Pageview Stats tool - https://phabricator.wikimedia.org/T120497#1984663 (10Nemo_bis) [09:30:32] hey guys, would you please have a look at you spare time maybe https://gerrit.wikimedia.org/r/#/c/267513/ [09:32:10] 6Labs, 10Labs-Infrastructure, 5Patch-For-Review: Install hunspell on tools - https://phabricator.wikimedia.org/T125193#1984681 (10Ebraminio) a:3Ebraminio [10:50:49] Can somebody help with restarting a bot, it is still used for archiving pages on the Dutch Wikipedia but the user is inactive... [10:59:14] You ask here because the bot run from toollabs? [11:01:40] Apparently. Not a big fan of all this legaly stuff but our village pump is becoming long... [11:24:55] Nemo_bis: do we have a doc somewhere explaining how to avoid toolabs account to be "polluted" by diverse log files? [11:25:00] ls [11:25:23] Nemo_bis: I have logs from the web servers and from qsub which should IMO not be there [11:33:43] sjoerddebruin: legaly? [11:34:01] Nemo_bis: sorry, legacy. [11:34:09] Kelson: which logs would you move where? [11:34:12] ah [11:34:35] Nemo_bis: I have for example a "Logs" directory where I put this kind of stuff [11:34:46] Nemo_bis: any solution to configure this? [11:38:17] for which of thelogs [11:38:46] Nemo_bis: ideally all of them [11:38:58] Nemo_bis: ~ is not the place for log files [11:44:00] Nemo_bis: for the web logs, seems not possible to change that https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web#Web_logs [11:48:11] Kelson: use jsub instead of qsub [11:48:25] because qsub defaults to *.eXXXXX and *.oXXXXX files [11:49:01] Kelson: and no, webservice logs are not easily moved [11:49:25] valhallasw`cloud: man jsub tells nothing about the log files. How should I make the call to specify log path? [11:49:52] Kelson: you can override it the same way as for qsub, but jsub uses a sane default (jobname.out and jobname.err) [11:50:51] valhallasw`cloud: indeed, that would be better, so I just replace "qsub" by "jsub" in the crontab? [11:50:57] Kelson: yes [12:00:35] !log tools.nlwikibots account voor archivering omgezet van erwint85bot naar nlwikibots [12:00:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.nlwikibots/SAL, Master [13:32:25] !log morebots restarted qamorebot [13:32:26] morebots is not a valid project. [13:32:32] !log tools restarted qamorebot [13:32:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master [13:39:46] valhallasw`cloud: thx, works fine [13:44:09] Somebody here who knows the reason for the rollback to wmf 10? [14:00:16] PROBLEM - ToolLabs Home Page on toollabs is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:05:08] RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 778140 bytes in 3.383 second response time [14:40:17] 10Labs-Other-Projects: Add OAuth 2.0 support to MediaWiki - https://phabricator.wikimedia.org/T125337#1984869 (10AdHuikeshoven) 3NEW [15:56:34] 6Labs, 7Tracking: Labs project for MW fuzzing - https://phabricator.wikimedia.org/T125340#1984961 (10Bawolff) 3NEW [15:57:28] 10Wikibugs: wikibugs - throttle output, don't get kicked for flooding - https://phabricator.wikimedia.org/T112032#1984967 (10Samtar) @Luke081515 I think it's currently working the best it's going to - the current redis delay prevents all but absolutely massive jobs from causing fn to kick wikibugs. Perhaps the i... [16:38:18] Hey folks. I'm having trouble sshing into an instance. snuggle-en.eqiad.wmflabs [16:38:27] Suddently, I'm getting a key failure. [16:38:33] All of my other instances work as expected. [16:38:41] This is an old 12.04 instance. [16:38:44] FWIW. [16:43:12] 6Labs, 7Tracking: Labs project for MW fuzzing - https://phabricator.wikimedia.org/T125340#1984985 (10bd808) [16:43:29] 6Labs: Labs project for MW fuzzing - https://phabricator.wikimedia.org/T125340#1984961 (10bd808) [16:47:57] 10Labs-Other-Projects: Can't SSH to snuggle-en.eqiad.wmflabs - https://phabricator.wikimedia.org/T125342#1984988 (10Halfak) 3NEW [17:31:17] 10Labs-Other-Projects: Can't SSH to snuggle-en.eqiad.wmflabs - https://phabricator.wikimedia.org/T125342#1985013 (10valhallasw) `snuggle.snuggle-en.puppetagent.time_since_last_run` ( http://graphite.wmflabs.org/render/?width=586&height=308&_salt=1454261326.489&target=snuggle.snuggle-en.puppetagent.time_since_las... [18:01:22] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Other: `tools.anon` causing excessive I/O - https://phabricator.wikimedia.org/T125349#1985086 (10valhallasw) 3NEW [18:05:13] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Other: templatetiger runs `sort` on huge NFS files - https://phabricator.wikimedia.org/T124822#1985101 (10valhallasw) Yes, of course. The removal of access was mostly because we have no easy way to make sure a user reads a message, so the approach is heavy-handed. Sorry... [18:08:47] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Other: `tools.anon` causing excessive I/O - https://phabricator.wikimedia.org/T125349#1985108 (10edsu) Sorry about that. I really had no idea it was such a large file when I started to grep it :-) I have since deleted it, which probably wasn't so nice to NFS either...oops! [18:32:12] 10Tool-Labs-tools-Other, 10Possible-Tech-Projects: Fix TreeViews to provide pageviews statistics for all articles of any wikiproject etc. - https://phabricator.wikimedia.org/T56184#1985132 (10Kelson) Having this would be really helpful for the WP1/Kiwix selections. Now that we have more or less achieved to mak... [18:49:51] !log tools tools-elastic-01.tools.eqiad.wmflabs not responsive to ssh or Elasticsearch requests; rebooting via wikitech interface [18:49:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master [18:51:28] !log tools tools-elastic-01.tools.eqiad.wmflabs console shows blocked tasks, possible kernel bug? [18:51:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master [18:52:36] sounds fun :( [18:58:41] ebernhardson: yeah :( web reboot isn't working and I can't ssh in. I think I need YuviPanda or andrewbogott or chasemp to forcibly reboot tools-elastic-01.tools.eqiad.wmflabs for me [18:59:28] bd808: have you... tried logging out and back in (to wikitech?) [18:59:35] bd808: also try rebooting from horizon.wikimedia.org/ [18:59:58] I can click the reboot button, it just seems to have no effect [19:00:09] yeah, that feels like sessionfucku [19:00:11] p [19:00:38] kk. gotte shower and find food. will look later I suppose [19:00:50] ok [19:00:52] thanks :D [19:26:59] YuviPanda: do you know who else were on the templatetiger tool? [19:27:09] I re-added kolossos and nosy, but I think there were two others [19:28:43] i was, but don't add me :p [19:29:19] i thought i could help but i couldn't [19:31:02] * bd808 got his instance rebooted [19:42:56] valhallasw`cloud: re T125332, did you understand that i cannot edit the file? [19:43:14] gifti: yes, but you can check in your part of the code, right? [19:43:26] my part of the code, what do you mean? [19:56:20] 10Labs-Other-Projects: Can't SSH to snuggle-en.eqiad.wmflabs - https://phabricator.wikimedia.org/T125342#1985183 (10Halfak) Maybe we could somehow get puppet to run again? I had root in the project 6 months ago, but I'm guessing you are talking about a Labs root like @YuviPanda or @Andrew. I'm happy to reb... [19:59:35] 6Labs: Labs project for MW fuzzing - https://phabricator.wikimedia.org/T125340#1985185 (10Bawolff) [20:28:35] 10Wikibugs: wikibugs - throttle output, don't get kicked for flooding - https://phabricator.wikimedia.org/T112032#1985202 (10Luke081515) I think we can solve the actual issue: Figure out, which numbers of messages are allowed in 5 minutes, and we can solve this, if we count how many messages were send in the las... [21:01:15] PROBLEM - ToolLabs Home Page on toollabs is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:06:10] RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 778313 bytes in 4.075 second response time [21:51:00] Hi all [21:51:09] is there a text table on wikidata? [21:56:35] ok it says here : "Unfortunately there is no way to access the wikitext directly via MySQL replica databases. You have to use the the mw:API instead." https://wikitech.wikimedia.org/wiki/Help:MySQL_queries#Wiki_text [22:37:02] 6Labs, 10Labs-Infrastructure, 10Tool-Labs: toolsbeta-puppetmaster3 instance has "BUG: soft lockup - CPU#0 stuck for 23s! [git:1366]" - https://phabricator.wikimedia.org/T125360#1985347 (10scfc) 3NEW [23:27:04] YuviPanda: I'm curious, have you tried out hyperkube for k8s?