[00:54:38] 6Labs, 10MediaWiki-extensions-OpenStackManager, 5Patch-For-Review: Permissions error on creating Hiera pages on wikitech - https://phabricator.wikimedia.org/T121602#1889574 (10Krenair) 5Open>3Resolved That should take effect when wikitech updates to a version including the above patch. [03:22:39] (03PS11) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [03:29:56] (03CR) 10Ricordisamoa: "PS11 registers the formatter for the 'url' data type. It already existed but wasn't being used." [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [07:32:49] Haha, wow! What a fun time I had tonight. Turns out Kylo Ren is Han and Leia's son, Ben but was seduced to the dark side. He even kills his own dad at the end.. It was really tense. Oh and Rea finds out she has jedi powers and does a mind trick on a storm trooper to escape captivity. She kicks Kylo Ren's ass with a lightsaber too! Luke Skywalker only shows up for 20 seconds at the end though whi [07:32:56] ch is kind of lame. Oh well. [07:34:39] PROBLEM - Host tools-worker-04 is DOWN: CRITICAL - Host Unreachable (10.68.16.122) [08:55:08] Hi there [10:18:18] (03PS12) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [10:21:03] (03CR) 10Ricordisamoa: "PS12 fixes language detection" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [11:40:12] 10Tool-Labs-tools-Other: Migrate https://toolserver.org/~dcoetzee/duplicationdetector/ to Tool Labs - https://phabricator.wikimedia.org/T60868#1890230 (10Liuxinyu970226) [13:49:55] hi, is there a labs admin here ? [14:01:25] Linedwell: just ask your question? [14:02:48] one of our bots (StewardBot) is down atleast 5hrs, and we can't restart it because it's job cannot be terminated (it doesn't care of qdel) [14:02:59] in case of need, I have it's job ID [14:03:53] Yes, that would be useful. Also, what's the tool name? [14:04:07] 381185? [14:04:08] 381185 [14:04:09] , stewardsbot [14:04:20] ok, taking a look. [14:04:20] stewardbots* [14:04:27] ty [14:06:34] Linedwell: just to confirm -- the bot is currently dead? [14:06:54] 6Labs, 10Tool-Labs: Cannot login to tools-exec-1409 (host dead?) - https://phabricator.wikimedia.org/T121860#1890352 (10valhallasw) 3NEW [14:07:21] 6Labs, 10Tool-Labs: Cannot login to tools-exec-1409 (host dead?) - https://phabricator.wikimedia.org/T121860#1890360 (10valhallasw) I also cannot login with my root key... [14:07:22] half dead [14:07:36] it doesn"t do his reporting job, that's why we have to reboot it [14:09:49] 6Labs, 10Tool-Labs: Cannot login to tools-exec-1409 (host dead?) - https://phabricator.wikimedia.org/T121860#1890366 (10valhallasw) Jobs currently running there: ``` 26262 0.33878 nop-arin.p tools.jimmy r 12/04/2015 10:42:03 task@tools-exec-1409.eqiad.wmf 1 228538 0.62006 gerrit-to- tools.gerrit R... [14:10:13] andrewbogott / chasemp: around? ^ best option is probably to just reboot the whole box? [14:10:28] although it does respond to pings :/ [14:11:23] I don't get why I can't login as root though, as that shouldn't hit NFS :/ [14:30:59] valhallasw`cloud: "INFO: task java:18818 blocked for more than 120" [14:31:28] "INFO: task python:6909 blocked for more than 120 seconds." [14:31:30] etc. [14:32:17] This is the third one that has happened to [14:44:31] Linedwell: I have one more thing I can try to do to kill that job, would you like me to try? [14:45:21] yes andrewbogott, you can [14:45:30] beat me to it :) [14:45:34] (the bot is useless in its current state) [14:46:56] qdel -f? [14:47:41] runs the risk of having two running bots, though :/ [14:49:11] Linedwell: better? [14:51:37] andrewbogott, that's odd, the bot is still logged in on IRC [14:52:00] will launch its restard cmd [14:52:22] No job 'stewardbot' is currently queued or running [14:52:22] Your job 85451 ("stewardbot") has been submitted [14:52:40] that's ok [14:52:47] thanks andrewbogott and valhallasw`cloud [14:54:05] it seems to work now [14:55:39] cool [15:16:29] !log tools rebooting locked up host tools-exec-1409 [15:16:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, dummy [15:39:03] 6Labs, 10Tool-Labs, 5Patch-For-Review: Web nodes complain "su: Permission denied" - https://phabricator.wikimedia.org/T121765#1890483 (10scfc) a:3faidon There were no more errors, so this seems to have fixed it indeed. Thanks! [15:39:08] 6Labs, 10Tool-Labs, 5Patch-For-Review: Web nodes complain "su: Permission denied" - https://phabricator.wikimedia.org/T121765#1890485 (10scfc) 5Open>3Resolved [16:08:16] 6Labs, 6operations: labswiki cannot connect to x1-slave (db1031), and soon, x1-master, either [Error connecting to 10.64.16.20: :real_connect(): (HY000/2003): Can't connect to MySQL server on '10.64.16.20' (4)] - https://phabricator.wikimedia.org/T121866#1890567 (10jcrespo) 3NEW [16:37:40] !log wikiviajesve deleting all instances and project; abandoned. [16:37:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikiviajesve/SAL, dummy [17:26:40] 6Labs, 10Labs-Infrastructure, 6operations, 10ops-eqiad, 10procurement: labstore1002 issues while trying to reboot - https://phabricator.wikimedia.org/T98183#1890897 (10mark) @cmjohnson: so where is the original controller in labstore1001 now, and what was wrong with it? If it's not usable, let's order a... [17:30:06] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/XZise was created, changed by XZise link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/XZise edit summary: Created page with "{{Tools Access Request |Justification=I am working on WikiProject X and need the access to test changes. |Completed=false |User Name=XZise }}" [17:58:41] 6Labs, 10Labs-Infrastructure, 6operations, 10ops-eqiad, 10procurement: labstore1002 issues while trying to reboot - https://phabricator.wikimedia.org/T98183#1890973 (10Cmjohnson) I do not think there were any real problems with it but @coren wanted to replace it jic there was a h/w issue. [18:03:20] 6Labs, 10Labs-Infrastructure, 6operations, 10ops-eqiad, 10procurement: labstore1002 issues while trying to reboot - https://phabricator.wikimedia.org/T98183#1890991 (10mark) @cmjohnson, do we have any ticket or other communication on that? I'm wondering what the issue was, i'm only aware of issues with l... [18:14:06] 6Labs, 10Labs-Infrastructure, 6operations, 10ops-eqiad: labstore1002 issues while trying to reboot - https://phabricator.wikimedia.org/T98183#1261827 (10RobH) [18:14:41] 6Labs, 10Labs-Infrastructure, 6operations, 10ops-eqiad: labstore1002 issues while trying to reboot - https://phabricator.wikimedia.org/T98183#1261827 (10RobH) I've created T121893 for the pricining/quoting/ordering of the controllers. Keeping that order to a sub-task will allow this troubleshooting task t... [18:15:57] (03PS13) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [18:17:41] (03CR) 10Ricordisamoa: "PS13 fixes the conflict checker for entities without descriptions" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [18:26:17] (03PS14) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [18:27:03] (03CR) 10Ricordisamoa: "PS14 fixes the conflict checker for entities without labels" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [18:34:24] (03PS15) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [18:35:48] (03CR) 10Ricordisamoa: "PS15 doesn't add titles to empty sections" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [19:24:45] 6Labs, 10Incident-Labs-NFS-20151216, 10Labs-Infrastructure, 6operations, 10ops-eqiad: labstore1002 issues while trying to reboot - https://phabricator.wikimedia.org/T98183#1891304 (10chasemp) [19:24:59] 6Labs, 10Incident-Labs-NFS-20151216, 6operations: Investigate better way of deferring activation of Labs LVM volumes (and corresponding snapshots) until after system boot - https://phabricator.wikimedia.org/T121629#1891305 (10chasemp) [19:29:59] Hi [19:30:19] I can't reset my developper password [19:30:44] https://wikitech.wikimedia.org/wiki/Special:PasswordReset [19:36:18] 6Labs, 10Incident-Labs-NFS-20151216, 6operations: Investigate better way of deferring activation of Labs LVM volumes (and corresponding snapshots) until after system boot - https://phabricator.wikimedia.org/T121629#1891348 (10chasemp) [19:49:42] The_Photographer: what's wrong? [19:51:24] I can't login in wikitech [19:51:49] Yes, I understand that. That form should allow you to reset your password. Does that not work? [19:52:03] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/XZise was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=235382 edit summary: [19:57:47] valhallasw`cloud: the email is not comming [19:58:43] The_Photographer: odd, I got my e-mail pretty quickly. Are you using the right username? "The Photographer", not "wilfredor" [19:59:15] I am using my email to recover the password [19:59:25] I used wilfredor and The Photographer [19:59:35] spam folder? [19:59:48] valhallasw`cloud: :) no [19:59:58] I can try submitting the form for you -- I'm not sure why that would matter, but let's try anyway... [20:00:09] A password reset email has already been sent, within the last 24 hours. To prevent abuse, only one password reset email will be sent per 24 hours. [20:00:11] right [20:00:50] I'm not sure why you would not get the e-mail -- I got mine within a minute (gmail) [20:00:50] The mail just came while I talk to you, what happend? [20:01:13] dunno, maybe the mail was held up somewhere [20:01:40] maybe [20:01:42] grey listing, spam filters, .. mail is complex [20:01:52] annoying to debug too [20:03:14] I dont underestand, my mail is gmail like you valhallasw`cloud [20:03:16] anyway [20:03:19] thanks guys [20:04:34] even commercial mail delivery services only advertise deliverability in the mid to high 90's, its indeed a hard thing to guarantee [20:05:55] I wonder how they even measure that [20:08:32] my shell could be disabled, How I could check it? [20:09:02] ebernhardson: comercial = non free servers? [20:12:28] The_Photographer: commercial are the kinds of companies you would pay if you own a store and you want to sent receipts to your customers [20:12:50] ebernhardson: for example [20:13:22] yea [20:16:45] The_Photographer: you should have access to tool labs. [20:17:05] valhallasw`cloud: Yes, I am making a new key [20:17:13] The_Photographer: ssh wilfredor@tools-login.wmflabs.org [20:24:08] (03PS16) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [20:26:57] valhallasw`cloud: working, thanks for your help [20:27:45] (03CR) 10Ricordisamoa: "PS16 fixes the fixed header (no pun intended)" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [20:28:09] valhallasw`cloud: where are my deployement folder?, I need make a web application [20:31:17] valhallasw`cloud: I need access to https://tools.wmflabs.org/wikiradio/ [20:32:16] valhallasw`cloud: I am helping with this project how you can see in https://github.com/emijrp/wikiradio [20:32:39] valhallasw`cloud: however, the project creator, I dont know what happend with him [20:32:41] https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web [20:32:44] The_Photographer: ^ [20:32:57] eeh [20:33:37] so you'll have to ask emijrp for access [20:34:01] Yes, however, I dont know where are he [20:34:13] he seems active on enwiki: https://en.wikipedia.org/wiki/Special:Contributions/Emijrp [20:34:26] I am writing he here https://meta.wikimedia.org/wiki/Talk:Wikiradio_(tool) [20:34:32] and I am making commits [20:34:39] however, He is not answering [20:34:46] any email or ping [20:35:03] The_Photographer: try his talk page on enwiki? [20:35:18] valhallasw`cloud: he is a spanish user [20:35:46] ...or his talk page on any other wiki where he is active [20:36:26] and he's not active on eswiki [20:36:46] I already did that in this moment [20:37:44] The_Photographer: as long as at least one of the owners of a tool is still active, the admins will not add users to the project [20:37:52] it's up to the current maintainers to make decisions on that [20:38:38] well, if he is not answering, I could make a git clone, however, I dont want do that [20:40:41] valhallasw`cloud: where I could ask for service group? [20:41:17] ? [20:42:08] to perform a management of the tool [20:43:31] The_Photographer: again, emijrp has to add you. [20:45:59] (03PS17) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [20:46:23] wilfredor@tools-bastion-01:~/wikiradio$ webservice start Only tool accounts have web services. [20:49:09] (03CR) 10Ricordisamoa: "PS17 makes the fixed header work correctly for new entities" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [21:02:30] Only tool accounts have web services. [21:02:35] What is a tool account? [21:02:54] This message is when I write "webservice start@ [21:03:39] The_Photographer: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Joining_and_creating_a_Tool_account [21:04:10] The_Photographer: that page has a lot of information on how things work on tool labs [21:04:23] valhallasw`cloud: Yes, I am reading, thanks [21:08:26] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/The Photographer was created, changed by The Photographer link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/The_Photographer edit summary: Created page with "{{Tools Access Request |Justification=https://meta.wikimedia.org/wiki/Wikiradio_(tool) |Completed=false |User Name=The Photographer }}" [21:11:07] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/The Photographer was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=235465 edit summary: [21:22:02] (03PS18) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [21:22:42] (03CR) 10Ricordisamoa: "PS18 moves the form out of the container" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [21:44:40] (03PS19) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [21:45:46] (03CR) 10Ricordisamoa: "PS19 updates grunt-jscs from ~2.4.0 to ~2.5.0" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [21:50:09] Krenair: Since it messes with your code quite a bit, I would love a review of https://gerrit.wikimedia.org/r/#/c/260037/ if you can fit it in before the weekend starts [21:58:20] andrewbogott, hey [21:58:31] why are you sorting lua_hooks? [22:00:58] Krenair: just trying to avoid puppet from reordering on every run [22:01:05] ok [22:01:09] it’s /probably/ not needed but I’ve had bad experiences [22:01:46] where does /etc/powerdns/labs-ip-alias.lua actually come from? [22:02:28] modules/role/manifests/labs/dnsrecursor.pp: $alias_file = '/etc/powerdns/labs-ip-alias.lua' [22:02:46] Oh I see [22:02:56] then it's written by the script, yep [22:03:21] Yeah, it’s from https://gerrit.wikimedia.org/r/#/c/243357/ [22:05:10] it looks good andrewbogott [22:05:13] but I did not test it [22:05:31] ok, good enough for me. I’ll run it through the compiler and such... [22:05:38] just hard to do now because there are so many precursors [22:05:57] thanks for reading [22:08:39] Earwig: I'm working on copyvios, and am getting a SiteNotFoundError when I change the defaultsite from testwiki (originally set up) to enwiki [22:09:06] the exception is coming up through app.update_sites() [22:09:07] fhocutt: um, traceback? [22:09:23] I'll paste it, one sec [22:10:22] https://phabricator.wikimedia.org/P2440 [22:11:04] looking at sitesdb.py, it seems to expect a sites table? This isn't present in my local sites.db [22:12:59] yeah, I got it [22:13:11] standby, some people want me to deal with something first [22:14:02] sure thing [22:14:06] thanks. [22:15:07] fhocutt: this should probably be tweaked internally, but you should be able to (as an immediate fix) import earwigbot; earwigbot.bot.Bot("").wiki.add_site(lang="en", project="wikipedia")... I think [22:15:30] ok, will try it [22:23:09] Earwig: well, I'm getting different errors now! Thanks for the help. [22:23:21] I hope that's a good thing [22:24:18] I think so! [22:29:46] Earwig: That seems ok, but it's using the defaultsite to check for page existence instead of the lang I've selected [22:30:02] umm, are you sure> [22:30:08] that doesn't sound right [22:31:15] hm--I search for en.wikipedia and User:EarwigBot/Copyvios/Tests/10, and it gives me "The given page doesn't seem to exist: https://test.wikipedia.org/wiki/User:EarwigBot/Copyvios/Tests/10" [22:33:10] though I've changed the defaultsite in .earwigbot/ to enwiki and it's still giving me testwiki links. [22:33:41] this may be good enough to test my turnitin integration, though, I can just add test pages to testwiki. [22:33:46] fhocutt: did you change the site in the query box as well? [22:33:54] I'm a bit perplexed [22:34:35] yes, I did, it shows "en" [22:34:43] did you restart the server? [22:34:50] yes. [22:34:54] that's very strange [22:35:05] the current tool doesn't have that problem, at the very least [22:35:20] and it shows lang=en in the query string [22:35:27] * fhocutt shrugs [22:35:42] I also made the local changes so it doesn't check for a search engine [22:35:56] so maybe that could affect things? Definitely strange. [22:36:12] wouldn't think so [22:36:32] hm. [22:37:11] then I'll just make test pages with the same names on testwiki so it thinks they exist--I don't think that should affect testing the turnitin stuff [22:38:11] yeah, that's fine I thin [22:38:12] k [22:38:37] great! expect a pull request today or Tuesday [22:38:46] oh, exciting