[06:30:03] Hi all [06:30:13] I have a Q about crontab [06:54:15] Elph: Hello! When you're on IRC it's usually good practice to jump right in and ask a question, as it saves time for everyone involved :) how can we help you? [06:54:59] :) [06:55:11] Why i can not use crontab? [06:55:24] You mean on toollabs? [06:55:29] yes [06:57:36] Elph: Do you get any errors? [06:57:52] I have some commands (bot's codes) and i need to run frequently [06:58:02] no error message [06:59:58] but commands not be run on the time [07:00:15] Elph: Can you check any logs or your mail account on toollabs for errors? [07:00:39] yes i can [07:00:51] K thanks, that would be helpful. [07:01:25] but no email in my inbox [07:01:43] Hm, K [07:02:56] Can you try running something really simple, with logging to a file, in your crontab, so we can verify that something runs accurately [07:03:15] Elph: When do you expect the scripts to run? [07:03:39] In ar.wiki [07:03:45] When. [07:04:09] Arabic Wikpedia [07:04:16] Not *where*, *when*. [07:04:19] At what time. [07:04:32] oh sorry [07:04:49] Every 00:00 [07:04:56] Hm [07:05:03] Your time, or server time? [07:05:12] yes [07:05:16] No [07:05:20] server time [07:05:24] OK [07:05:31] And what time is it on the server now? [07:05:46] IDK [07:06:05] Can you check? [07:06:07] 18:32 [07:06:14] OK, and when did the scripts last run? [07:06:48] i run it directly some minutes ago [07:06:59] When did it last run via crontab? [07:07:35] not run in crontab yet [07:07:40] because i can set my crontab [07:07:57] i can't [07:08:11] Elph: Why can't you? [07:08:20] IDK [07:08:41] i set it but it's not be run [07:08:52] [bz] (8NEW - created by: 2Tilman Bayer, priority: 4Unprioritized - 6normal) [Bug 50556] Total number of Wikipedia articles on details (but not summary) table is ca. 11 million too high - https://bugzilla.wikimedia.org/show_bug.cgi?id=50556 [07:08:56] Elph: When did you set it up? [07:09:10] some days ago [07:09:21] Elph: And how would you have known it had run? [07:10:02] i check the contributions of my bot [07:10:26] and i see nothing done [07:10:29] Hrm. [07:13:17] I'm out of ideas [07:13:45] The people who run labs are probably asleep now, but feel free to wait around, they'll likely get back to you in the next day - or email the labs-l list [07:13:54] do email labs-l if not [07:14:07] also, pinging Coren and petan to see if they can help when they get around to it [07:14:25] Ok [07:14:29] thanks a lot [07:24:06] [bz] (8NEW - created by: 2Nemo, priority: 4Low - 6normal) [Bug 44144] [Errors 792] Wikis displaying bogus Special:Version URLs - https://bugzilla.wikimedia.org/show_bug.cgi?id=44144 [07:24:18] [bz] (8NEW - created by: 2Nemo, priority: 4Low - 6normal) [Bug 44143] [Errors 792] Wikis with old_statsurl in wrong column - https://bugzilla.wikimedia.org/show_bug.cgi?id=44143 [07:28:22] Good morning [07:30:17] Moooorning [07:42:12] [bz] (8NEW - created by: 2Nemo, priority: 4Unprioritized - 6normal) [Bug 54161] [Errors 500] Identify wikis with $wgEnableAPI=false - https://bugzilla.wikimedia.org/show_bug.cgi?id=54161 [07:42:13] [bz] (8NEW - created by: 2Nemo, priority: 4Low - 6minor) [Bug 44144] [Errors 792/991/997] Wikis displaying bogus Special:Version URLs - https://bugzilla.wikimedia.org/show_bug.cgi?id=44144 [07:51:39] Good Morning!! Does anyone know how i can have access to English WIkipedia database. Because when i enter the command " mysql --defaults-file="${HOME}"/replica.my.cnf -h enwiki.labsdb enwiki_p " it says that The program 'mysql' is currently not installe. Thank you in advance [08:43:23] panos_: type sql en [08:46:27] panos_: on which server you are [08:51:07] panos_: you need to login to tools-login or tools-dev [08:51:14] on bastion replicated db's are not available [08:54:15] petan: when i try to connect to tools-login with this command "ssh yourshellaccountname@tools-login.wmflabs.org" it delivers me this message "ssh: connect to host tools-login.wmflabs.org port 22: Connection timed out" [08:56:02] let me check [08:59:15] what happens when you do ping tools-login.wmflabs.org [09:01:02] it shows this "PING tools-login.wmflabs.org (208.80.153.224) 56(84) bytes of data." without terminating the command [09:02:00] try same w tools-dev.wmflabs.org [09:03:43] with the previous i had 100% packet loss [09:05:54] it happens the same [09:10:12] ok [09:10:21] ssh to bastion then and from there ssh to tools-login [09:14:52] so i will type "ssh myname@bastion.wmflabs.org" [09:18:35] yes [09:18:42] get back to server where you were [09:19:03] after this command (ssh -A panos@bastion.wmflabs.org) which connects me to the server i try this command (ssh panos@tools-dev.wmflabs.org) which is terminated [09:39:38] panos_: just ssh tools-login [09:39:44] nothing else [09:44:45] i type this command but it doesn t connect me it says "If you are having access problems, please see: https://wikitech.wikimedia.org/wiki/Access#Accessing_public_and_private_instances" so as i am already connected [09:45:47] to the server after that i type "ssh panos.pmtpa.wmflabs" but it says "ssh: Could not resolve hostname panos.pmtpa.wmflabs: Name or service not known" [09:47:17] panos_: on bastion type ssh tools-login [09:48:05] ok that works now [09:51:15] and from here how i can have access to the wikipedia db... o thanks a lot for your help [10:05:40] [bz] (8NEW - created by: 2Liangent, priority: 4Unprioritized - 6normal) [Bug 54164] Expose revision.rev_content_format on replicated wikidatawiki - https://bugzilla.wikimedia.org/show_bug.cgi?id=54164 [10:31:19] !tunnel [10:31:19] ssh -f user@bastion.wmflabs.org -L :server: -N Example for sftp "ssh chewbacca@bastion.wmflabs.org -L 6000:bots-1:22 -N" will open bots-1:22 as localhost:6000 [14:13:40] Steinsplitter: Ping [14:13:49] ? [14:13:52] sorry for misclick [14:13:56] np [14:13:57] wrong revert [14:14:10] I even didn't know that was reverted... [15:19:05] addshore, ping [15:20:41] [bz] (8NEW - created by: 2Antoine "hashar" Musso, priority: 4Highest - 6major) [Bug 48501] [OPS] beta: get SSL certificates - https://bugzilla.wikimedia.org/show_bug.cgi?id=48501 [15:20:42] [bz] (8UNCONFIRMED - created by: 2Nicolas Raoul, priority: 4Unprioritized - 6normal) [Bug 54065] Enable HTTPS on a Labs instance - https://bugzilla.wikimedia.org/show_bug.cgi?id=54065 [15:39:26] why hashar is not online? [15:39:28] @seen hashar [15:39:28] Steinsplitter: Last time I saw hashar they were quitting the network with reason: Quit: I am a manual virus, please copy me to your quit message. N/A at 9/13/2013 12:04:54 AM (3d15h34m33s ago) [15:39:37] 3d15h34m33s ago o-O [15:46:31] <^d> Steinsplitter: Might be traveling or jetlagged. [16:42:58] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Tips "but doing\ntouch ~/.suppresstips" should be "by doing\ntouch ~/.suppresstips" I assume [16:54:14] [bz] (8NEW - created by: 2Betacommand, priority: 4High - 6major) [Bug 54052] tools.wmflabs.org inaccessible via labs instances - https://bugzilla.wikimedia.org/show_bug.cgi?id=54052 [16:55:03] Toto_Azero: That seems like a reasonable assumption. :-) [16:55:22] :) [17:04:09] [bz] (8PATCH_TO_REVIEW - created by: 2DrTrigon, priority: 4Unprioritized - 6normal) [Bug 54135] Install python tool fab from package 'fabric' - https://bugzilla.wikimedia.org/show_bug.cgi?id=54135 [17:07:28] [bz] (8PATCH_TO_REVIEW - created by: 2DrTrigon, priority: 4Unprioritized - 6normal) [Bug 53870] Install binary ffprobe from package 'libav-tools' - https://bugzilla.wikimedia.org/show_bug.cgi?id=53870 [17:09:23] [bz] (8PATCH_TO_REVIEW - created by: 2DrTrigon, priority: 4Unprioritized - 6normal) [Bug 53869] Install binaries pdftotext and pdfimages from package 'poppler-utils' - https://bugzilla.wikimedia.org/show_bug.cgi?id=53869 [17:11:26] [bz] (8PATCH_TO_REVIEW - created by: 2DrTrigon, priority: 4Unprioritized - 6normal) [Bug 53868] Install binary exiftool from package 'libimage-exiftool-perl' - https://bugzilla.wikimedia.org/show_bug.cgi?id=53868 [17:14:30] addshore, ping [17:17:56] hey all, what labs instance is ganglia.wmflabs.org on? [17:18:37] [bz] (8PATCH_TO_REVIEW - created by: 2DrTrigon, priority: 4Unprioritized - 6normal) [Bug 53867] Install C header libdmtx - https://bugzilla.wikimedia.org/show_bug.cgi?id=53867 [17:19:41] ah found it! aggregator1, thanks! [17:28:32] I've been trying to set up an SSH tunnel with MySQLWorkbench to connect to my tool labs database, but I haven't had any luck. I've heard this is supposed to be possible though. Could anyone help me with the configuration? [17:30:04] …or point me to any documentation on how to do so. I looked, but didn't see any :( [17:36:12] kaldari, try [17:37:02] ssh -N bastion1.pmtpa.wmflabs -L 3306:.pmtpa.wmflabs:3306 [17:37:08] then you can connect mysql to localhost:3306 [17:37:35] thanks, I'll try that... [17:37:38] if you need different ports [17:37:45] the first port is the locally bound port [17:37:48] so that is arbitrary [17:38:14] ssh -N bastion1.pmtpa.wmflabs -L :.pmtpa.wmflabs: [17:38:42] oh, i don't know much about toollabs though, that's how it would work in labs [17:38:47] dunno if toollabs has a different bastion or somethign [17:41:45] yeah, I don't think tool labs uses bastion1.pmtpa.wmflabs [17:52:12] Ryan_Lane: hello! [17:52:19] Ryan_Lane: need you to do your domain name removal magic once again [17:52:27] howdy [17:52:29] Ryan_Lane: can you remove metrics.wmflabs.org from its current assignments? [17:52:40] sure. one sec [17:52:45] Ryan_Lane: ty! [17:53:24] it should no longer be 208.80.153.185 ? [17:54:05] Ryan_Lane: yeah [17:54:14] Ryan_Lane: should be 208.80.153.190 [17:54:22] done [17:54:43] Ryan_Lane: swweeet [17:54:44] thank you! [17:55:06] yw [17:55:23] Coren, ping [17:55:36] Ryan_Lane: also, started work on adding URL based routing to the proxy. should be done before end of week [17:55:40] Cyberpower678: Pong [17:55:58] Coren, have you had a chance to look at the cyberbot node? [17:56:01] YuviPanda: sweet [17:56:11] Ryan_Lane: but we'll lose load balancing capabilities, but I think that's ok [17:57:05] why would we lose load balancing? [17:57:08] Cyberpower678: I'm still in SF; I'll *try* to look at it before I leave but chances are I'll only be able to do this once I'm back at my home office. [17:58:00] Coren, ok. Memory usage can rise all the way up to 1.6GB and CPU is constantly exceeding 100% usage. [17:58:57] Ryan_Lane: because I can't figure a way out to store both 'list of instances' + 'url prefixes to instance/port combos' in an effecient way in redis [17:59:22] hm [18:00:14] Ryan_Lane: so now it is a set, with effecient 'random member' operation. [18:00:53] Ryan_Lane: i'm going to change that into a hash, with effecient 'get me all routes for this domain' operation, and then do matching in lua [18:06:42] YuviPanda: that'll give us back load balancing? [18:06:48] I think load balancing is a good idea [18:06:58] Ryan_Lane: so right now we get load balancing, but no URL routing [18:07:09] and if people want URL routing - they need to put their own reverse proxy there [18:07:31] Ryan_Lane: I can't think of a way to do both effeciently. [18:12:09] Ryan_Lane: you know much about labs ganglia setup? [18:12:24] I'm trying to send metrics to ganglia and now having much luck so far [18:13:42] The labs ganglia setup is like middle earth [18:14:19] Damianz: That's not fair; the history and geography of middle earth is /very/ well documented. [18:14:53] Well yes, at least most people know how to avoid the axe weilding half people in middle earth [18:14:54] haha [18:15:14] uh oh, who wields the labs ganglia axes? [18:15:22] no one anymore [18:15:35] it use to be sara until she ran away [18:19:12] [bz] (8PATCH_TO_REVIEW - created by: 2Daniel Kinzler, priority: 4Unprioritized - 6enhancement) [Bug 52693] Allow login using mosh as an alternative to plain ssh on bastion - https://bugzilla.wikimedia.org/show_bug.cgi?id=52693 [18:28:20] I added documentation on how to configure MySQLWorkbench to the Tool Labs documentation: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Configuring_MySQL_Workbench [18:28:38] since we can't use PhpMyAdmin [18:29:55] [bz] (8ASSIGNED - created by: 2Robin Krahl, priority: 4Normal - 6normal) [Bug 54107] *_userindex tables need to be documented - https://bugzilla.wikimedia.org/show_bug.cgi?id=54107 [18:30:28] ottomata: it's set up similarly to production [18:30:56] is there a security group issue I need to fix? i guess not since most of the nodes default ganglia stats work [18:32:17] kaldari: Thank you; this is going to be especially useful for the users that are not used to command line. [18:32:20] Ryan_Lane: should I be configuring ganglia to talk to localhost:8649, 10.4.0.79:8649, or aggregator1.pmtpa.wmflabs:50090 (analytics cluster data_source) [18:32:53] i honestly have no clue [18:33:08] probably the 50090 port [18:33:11] Coren: true, or for those of us who are just to lazy to type everything :) [18:33:17] since it's the aggregator [18:33:38] but if it's reporting stats for the individual instance, maybe the local port? I really don't know [18:33:42] I just do basics with ganglia [18:42:34] ha, nobody knows how ganglia works here! hehe :) [20:26:43] [bz] (8RESOLVED - created by: 2Tilman Bayer, priority: 4Unprioritized - 6normal) [Bug 50556] Total number of Wikipedia articles on details (but not summary) table is ca. 11 million too high - https://bugzilla.wikimedia.org/show_bug.cgi?id=50556 [21:18:54] seems like the 'namespace' table/view on the replicated servers isn't working [21:18:58] ERROR 1356 (HY000): View 'enwiki_p.namespaces' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them [21:23:23] hmkay, I'll file a bug report [22:49:48] [bz] (8NEW - created by: 2Liangent, priority: 4Unprioritized - 6normal) [Bug 54164] Expose revision.rev_content_format on replicated wikidatawiki - https://bugzilla.wikimedia.org/show_bug.cgi?id=54164 [22:53:00] [bz] (8NEW - created by: 2Peter Bena, priority: 4Normal - 6normal) [Bug 48930] (Tracking) Database replication services - https://bugzilla.wikimedia.org/show_bug.cgi?id=48930 [23:06:05] [bz] (8NEW - created by: 2Legoktm, priority: 4Unprioritized - 6normal) [Bug 54192] HTTPS for parsoid.wmflabs.org - https://bugzilla.wikimedia.org/show_bug.cgi?id=54192 [23:14:42] Ryan_Lane: ping! [23:14:48] Ryan_Lane: remove DNS for parsoid.wmflabs.org? :) [23:16:31] YuviPanda: one sec [23:17:06] Ryan_Lane: ok [23:17:42] YuviPanda: done [23:17:47] Ryan_Lane: sweeet [23:22:50] Ryan_Lane: bah, those people still want their public IP. Can you clear teh DNS again? [23:23:58] Ryan_Lane: actually hold it. don't do anything yet. [23:36:31] YuviPanda: heh [23:36:37] YuviPanda: they still have the public ip [23:36:41] Ryan_Lane: yeah they do. [23:36:43] just no hostname associated with it :) [23:36:46] Ryan_Lane: but we can't make any new associations [23:36:49] because we can't remove the old ones ;) [23:37:00] Ryan_Lane: the URL based routing stuff should really help. [23:37:09] you can totally add a different hostname to that old IP [23:37:22] it just can't be metrics, since it's associated with yours [23:37:48] I think we should stop adding things until dns is fixed [23:37:51] Ryan_Lane: sure, but can't 'change' parsoid.wmflabs.org, which is now pointing to proxy-dammit [23:37:54] Ryan_Lane: true. [23:38:01] do I need to do anything? [23:38:34] Ryan_Lane: yeah. can you clear parsoid.wmflabs.org DNS again? [23:38:38] just revert it back for now [23:38:52] they can add the hostname back to their IP [23:38:57] and everything is back to how it was [23:40:12] I removed it [23:40:15] they can add it back [23:45:34] Ryan_Lane: thanks [23:47:46] yw [23:48:23] YuviPanda: you marked the bug as fixed, but it isn't now, right? [23:48:29] Ryan_Lane: yeah, need to re-open [23:48:32] * Ryan_Lane nods [23:48:36] Ryan_Lane: how's the DNS fixing coming along? [23:48:41] no clue :) [23:48:47] we're having meetings all week [23:49:08] Ryan_Lane: sounds productive