[00:56:35] !bzqs Image null edit template [02:08:11] I was trying to get to a website from a nonbrowser program and the program give the error that my default browser was currently disabled. I have Windows 7 and have already checked that my default browser was one of the browser I have on my computer and it is. Is there a way to check if it is disabled? [05:17:13] These spikes in the load are very annoying. [05:17:45] I'm constantly using labs and constantly keep hitting them when I want to log in. [05:20:35] I was trying to get to a website from a nonbrowser program and the program give the error that my default browser was currently disabled. I have Windows 7 and have already checked that my default browser was one of the browser I have on my computer and it is. Is there a way to check if it is disabled? [07:42:59] !ping [07:42:59] pong [07:54:43] I was trying to get to a website from a nonbrowser program and the program give the error that my default browser was currently disabled. I have Windows 7 and have already checked that my default browser was one of the browser I have on my computer and it is. Is there a way to check if it is disabled? [07:55:01] LLckfan this isn't windows help channel [07:56:06] !log deployment-prep deleting deployment-puptest , unused, no class applied [07:56:10] Logged the message, Master [07:57:41] !log deployment-prep deleting deployment-varnish-t3 , used as a mobile cache, now replaced by deployment-cache-mobile01 [07:57:44] Logged the message, Master [09:00:26] !log test [09:00:27] Message missing. Nothing logged. [09:08:52] !log bots petrb: test [09:08:55] Logged the message, Master [09:15:01] [bz] (8NEW - created by: 2Antoine "hashar" Musso, priority: 4Unprioritized - 6normal) [Bug 51700] https://login.wikimedia.beta.wmflabs.org/ trapped in an infinte self redirect - https://bugzilla.wikimedia.org/show_bug.cgi?id=51700 [09:17:01] YuviPanda: how would a redis queue work? i couldn't figure it out [09:19:21] oh my, is it not labs documentation overhaul day today? :O [09:20:45] it is [09:32:29] [bz] (8NEW - created by: 2Antoine "hashar" Musso, priority: 4Unprioritized - 6normal) [Bug 51700] https://login.wikimedia.beta.wmflabs.org/ trapped in an infinite self-redirect - https://bugzilla.wikimedia.org/show_bug.cgi?id=51700 [09:32:30] [bz] (8NEW - created by: 2Chris Steipp, priority: 4Unprioritized - 6normal) [Bug 51622] Add loginwiki to beta - https://bugzilla.wikimedia.org/show_bug.cgi?id=51622 [09:50:15] [bz] (8NEW - created by: 2This, that and the other, priority: 4Unprioritized - 6enhancement) [Bug 51745] Implement a custom HTTP 500 error page on labs - https://bugzilla.wikimedia.org/show_bug.cgi?id=51745 [10:01:05] [bz] (8RESOLVED - created by: 2Michelle Grover, priority: 4Unprioritized - 6major) [Bug 51694] Beta labs is down - https://bugzilla.wikimedia.org/show_bug.cgi?id=51694 [12:04:01] [bz] (8NEW - created by: 2Chris McMahon, priority: 4Unprioritized - 6major) [Bug 50622] Special:NewPagesFeed intermittently fails on beta cluster; causes test failure - https://bugzilla.wikimedia.org/show_bug.cgi?id=50622 [12:31:00] Hi all [12:31:45] I was trying to get to a website from CyberLink PowerDVD and it is giving an error that my default browser was currently disabled. I have Windows 7 and have already checked that my default browser was one of the browser I have on my computer and it is. And it opens when I click a link in a chat channel. Is there a way to check if it is disabled?4 [12:31:58] Is there someone to help me? [12:33:04] When i want to create account i face "There was either an authentication database error or you are not allowed to update your external account." [12:33:15] what's the problem? [12:34:38] No one there? [12:43:37] Hi petan: ^^Can you help Elph? [12:43:54] Hi all btw, I'm back from vacation. :) [12:44:42] I would be happy to help but I am afraid I have no permissions to fix this [12:44:51] he needs to talk to Ryan or Coren [12:45:07] aha, ok. [12:46:23] Elph: Ryan -> that would be Ryan_Lane. Can you try and ask again later, please? [12:46:56] thanks [12:47:02] i will call him [12:52:14] I was trying to get to a website from CyberLink PowerDVD and it is giving an error that my default browser was currently disabled. I have Windows 7 and have already checked that my default browser was one of the browser I have on my computer and it is. And it opens when I click a link in a chat channel. Is there a way to check if it is disabled? [13:05:09] Hi sumanah! How are you doing? :) I'm back. What do you know about volunteers taking part in the doc sprint? How many people are in? [13:05:26] hi Silke_WMDE! I don't know, sorry; I hadn't asked people to sign up anywhere in particular [13:05:32] Silke_WMDE: Kirsten is a WMF contractor [13:05:49] cool! [13:06:11] I'm just looking at the pad she sent around. [13:06:20] oh, today's 23 already? [13:06:26] hmm, apparently not [13:06:31] nono [13:06:59] YuviPanda: I'm only catching up with what happened during my absence. [13:07:08] Silke_WMDE: you or I could reply to the Labs-l thread and ask for people to put their names on the pad? [13:07:20] I'll do that. [13:07:37] :) Silke_WMDE. I am just bad at dates, so was just making sure. [13:08:46] sumanah: I will available partly only during those days. [13:10:18] @kick LLckfan this is not a windows help channel, come back when you have questions about labs [14:47:34] Damianz: poke [15:10:11] Cyberpower678: you here? [15:10:21] T13|college, yes [15:10:31] You're in college now? :) [15:10:32] :O [15:11:07] Been in college for 2 years... I have a Q. About X's tool [15:11:23] go ahead [15:11:37] also T13|college - check out https://www.mediawiki.org/wiki/Summer_of_Code_2013 and think about working with us next summer :) [15:11:47] Is there a way to do a global opt-in instead of having to do it on each wiki? [15:12:00] T13|college, yes. [15:12:20] Do tell... :) [15:12:28] Just follow the instructions it gives you when you are currently nor opted-in :p [15:12:33] *not [15:12:58] I'm opted-in on en right now I think.. [15:13:44] hi [15:13:54] hi Elph [15:14:00] is an admin there? [15:14:14] an admin for Labs? yes [15:14:33] T13|college, User has not yet opted in. If you want to see graphs, please create User:User/EditCounterOptIn.js with any content. Alternatively, you can create meta:User:user/EditCounterGlobalOptIn.js to opt-in across all Wikimedia wikis. [15:14:33] https://wikitech.wikimedia.org/wiki/Shell_Request/Elph [15:14:35] Elph: you don't need admin you need directly Ryan Lane or andrewbogott_afk, or Coren [15:14:48] regular admins can't fix this [15:15:09] pentan i created my account [15:15:09] Will do Cyberpower678 [15:15:51] now in need someone to confirm my shell request [15:16:08] ok [15:16:11] petan, you can't grant shell access? [15:16:16] I can... [15:16:25] I thought they need to fix that mediawiki error [15:16:31] Is it done now? [15:16:32] Then why did you just say an admin can't do it? [15:16:43] Oh [15:16:47] Cyberpower678 was talking about the mediawiki bug [15:17:02] petan, what MediaWiki bug? [15:17:14] I wasn't talking about a MediaWiki bug. [15:17:59] Elphi I suppose it's that ElphiBot user account? [15:18:45] my main account is "Elph" and my bot is "ElphiBot" [15:19:11] eh [15:19:24] you don't need to create account for your bot, you need to create a service group for it [15:19:39] your main account is only wiki-account you need to create [15:19:41] T13|college, enwiki will no longer require you to opt-in soon. [15:19:43] but i want get shell for "Elph" [15:19:47] ok [15:20:11] Elph: Don't you have already? [15:20:12] done [15:20:21] Thanks a lot [15:20:35] scfc_de: no there was some bug - they were listed in requests but user account didn't exist [15:20:52] petan: So like Joe123 the other day? [15:21:11] yes [15:21:24] @ petan : how i create service group? [15:21:46] Elph you probably want to be a member of tools project now [15:22:54] I want to be member of Bot projects [15:23:21] Elph not really, bots project is deprecated, now we use tools for bots [15:23:35] oh [15:24:00] you'r there :P [15:24:02] i need to read guidelines again :) [15:24:44] !toolsdocs [15:24:45] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help [15:36:15] Cyberpower678: Did you count up how many people wanting to keep opt-in were confusing X's edit counter with wikichecker.com? For example, number 257 taking about someone "snoop[ing] around [his] circadian rhythm[s]". Then I wonder how many others had the same confusion without making it obvious. [15:40:25] anomie, the principle still applies. If they want data aggregation privacy which is what both wikichecker and the edit couunter do, then let them have it. [15:41:00] Cyberpower678: I'm just curious how many people voted without actually paying attention to what they were voting about. [15:41:12] And I'm too lazy to count it up myself ;) [15:42:15] It was honestly not even 20 people. I was actively monitoring that page and even if I removed those confused and careless votes, the opt-in supports are still in the lead. [15:45:11] Hi. I'm using my Tool Labs account to view the replica of the MediawikiWiki database. In the list of existing tables (via "show tables;"), some tables seem to be missing. E.g. there's no table "archive", no table "text", etc. . What's the reason for those tables missing? [15:45:24] hi Martin_____ [15:45:29] that sounds weird to me [15:45:41] * sumanah looks at https://www.mediawiki.org/wiki/Schema  [15:45:46] !wm-bot2 [15:45:46] wm-bot2 is me :) read http://lists.wikimedia.org/pipermail/wikitech-l/2013-July/070375.html [15:45:53] @ petan [15:46:08] when i want to login via puty [15:46:19] face an error message [15:47:12] "Server un expectedly closed network connection" [15:47:18] login where [15:47:20] Martin_____: either "show tables;" is broken, or some aspect of the replica is broken, in my opinion; it seems very unlikely that we have deliberately blocked or removed those from the replica or from Tool Labs users to access [15:47:27] !access | Elph [15:47:35] o.O [15:47:38] where is this key [15:47:45] http://bots.wmflabs.org/~wm-bot/dump/%23wikimedia-labs.htm [15:47:50] private key? [15:47:53] anomie, but look forward to the opt-in being removed on enwiki at least. [15:47:53] hold on [15:47:54] ;) [15:49:07] !access is https://wikitech.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [15:49:08] Key was added [15:49:12] Elph see that [15:49:13] Martin_____: archives isn't replicated (yet); cf. https://bugzilla.wikimedia.org/49088 [15:52:06] scfc_de: thanks for the info. do you have any idea why there are more tables missing? [15:52:11] Martin_____, sumanah: Actually, tables such as "archive" specifically are removed because they are non-public data (there are plans to allow some limited access to archive, which scfc_de just linked to the bug about). As for 'text', I'm not sure if it's the same deal or if it's just that the actual text is stored elsewhere (as in other servers entirely) and 'text' just contains pointers to it and so is basically useless. [15:53:15] I was wrong! [15:53:53] thank you anomie [15:55:35] sumanah: You're welcome [15:56:22] anomie: the tables of the "CodeReview" extension are also missing, but at least the comments and flags of the revisions are accesible via the Special-Pages of the wiki, so these tables should be available? [15:57:58] Martin_____: It's likely that, since that extension is enabled on only one wiki, it hasn't been reviewed for non-public data yet so the default is "not available". You'd need to ask Coren about it, unless maybe petan or someone else knows too. [15:58:14] hmm, Coren seems away [16:00:45] anomie, Martin_____: The replication happens outside Tools, so I think only Coren and binasher (and perhaps some Labs ops in emergency) have access to that. [16:03:19] Is there any other way to access the data of codereview, e.g. through the available db-dumps of the wiki? (except downlaoding the web-pages, of course) [16:17:03] $ scp foo jkroll@tools.wmflabs.org:~ [16:17:04] If you are having access problems, please see: https://wikitech.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [16:17:04] Connection closed by 208.80.153.201 [16:17:04] lost connection [16:17:21] i can ssh to tools-login though, and write stuff there [16:17:26] what am i doing wrong? [16:17:27] Coren? [16:19:05] JohannesK_WMDE: he seems to be away at the moment [16:19:14] labs-l might be better :/ or Damianz or petan ? [16:22:08] JohannesK_WMDE: tools.wmflabs.org is the webserver; you can't access it. [16:22:59] !started [16:23:13] https://wikitech.wikimedia.org/wiki/Help:Getting_Started the bastion is what we log into, yes? [16:23:47] scfc_de: omg. must be the heat. *crawls away in shame* [16:23:59] it's ok :) [16:24:06] JohannesK_WMDE: my brain has also been melting [16:24:13] it's tempered, like chocolate [16:24:13] :-] [16:25:22] thx @ everyone for the nice help :) [16:25:47] sumanah: In general, for Labs instances, yes, but Tools' tools-login.wmflabs.org and tools-dev.wmflabs.org are directly accessible without going through bastions. [16:26:28] duh [16:26:32] sorry [16:28:56] regarding the documentation sprint: i would like to put some documentation about catgraph there but i don't quite know where in the etherpad it fits [16:30:06] JohannesK_WMDE: https://wikitech.wikimedia.org/wiki/Nova_Resource:Catgraph do you see the "Add documentation" redlink? [16:30:30] yes :p [16:30:54] JohannesK_WMDE: I'm sorry, I don't mean to be a jerk [16:31:03] JohannesK_WMDE: I just meant to point that out to you [16:31:05] no, you're right. adding some now. [16:31:14] it seems like the best place to put the documentation about a specific tool [16:31:37] also on http://tools.wmflabs.org/ Catgraph is not a link - I presume because there's no index.html - style page for it? [16:31:39] that would be nice [16:33:02] JohannesK_WMDE: in the labs-l thread you saw me suggest "a curated directory of popular tools/bots that are already settled in, and some ideas for case studies to feature on blog.wikimedia.org"? you could write a blog entry to post at blog.wikimedia.org about CatGraph! [16:33:41] it is a service for tool authors. kind of like a cache in a sense. i could create an entry like the redis one, what do you think sumanah? [16:33:53] a blog entry would be nice... if i find the time [16:34:15] JohannesK_WMDE: you could work on the blog entry during the doc sprint! [16:34:42] JohannesK_WMDE: an entry like what YuviPanda wrote about Redis? yes, that would be good [16:35:05] sumanah: yes, but i have lots of stuff to do in that time... i have this page: https://de.wikipedia.org/wiki/Benutzer:Johannes_Kroll_%28WMDE%29/CatGraph-en [16:35:10] i could just link to that ;) [16:36:23] or move it to wikitech.wikimedia.org, yeah [16:57:52] done, sumanah. i'm finally "being bold" and copy-pasted the stuff to the documentation page. ;) [16:58:20] nice :) [16:58:34] you could just move it to avoid divergences! but copypasting is good! [16:59:37] !tools-help [16:59:37] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help [17:00:01] JohannesK_WMDE: where did you paste it to? [17:00:28] thank you JohannesK_WMDE - I appreciate your work [17:00:33] YuviPanda: https://wikitech.wikimedia.org/wiki/Nova_Resource:Catgraph [17:00:39] ah nice [17:00:43] let me look [17:00:46] YuviPanda: or do you think it would fit better on a separate page? dunno. [17:01:10] well, since it's a labs project by itself that is just accessible from other labs projects, I think that's the right namespace [17:01:56] YuviPanda: do you know how your suggestion of a redis queue in bug 51310 would work? [17:02:05] !b 51310 [17:02:05] https://bugzilla.wikimedia.org/51310 [17:02:19] giftpflanze: ah, yes. [17:02:29] giftpflanze: you have 1 instance that just finds URLs to check, puts them in the queue [17:02:33] btw, if you notice any strange wordings let me know. my english isn't perfect. [17:02:49] giftpflanze: then you have any number of continuous jobs that pop from the queue and do the checking [17:03:06] giftpflanze: this way you can have 3-4 jobs running across multiple grid engines and it'll be alright [17:03:21] i thought more like 100 (or 1000) [17:04:04] in general i think i get the mechanism, but i can't figure out how to do it in redis specifically [17:04:23] giftpflanze: how many links are you going to be checking? [17:05:13] all in all dewiki articles (i think there were like 10 million last year) [17:05:42] giftpflanze: you are just going to be blocked on IO here, so number of instaneces isn't really going to help [17:05:52] clever ways of doing multiple IO instances might help [17:06:04] hm [17:06:34] what do you mean with multiple io instances? [17:06:47] giftpflanze: multipel jsubs, all continuous? [17:07:08] as opposed to what? [17:08:43] oh [17:08:46] multiple 'io' instances? [17:08:55] giftpflanze: have you heard of the concept of event driven io? [17:09:13] yes [17:09:52] giftpflanze: that's what I was talkinga bout, which will let you do more IO work with less CPU work [17:10:10] how? [17:10:52] giftpflanze: http://stackoverflow.com/questions/3231018/how-does-event-driven-i-o-allow-multiprocessing [17:11:10] instead of 'request comes in', think 'response comes in, after i had sent a request to the external url' [17:25:22] [bz] (8NEW - created by: 2m.p.roppelt, priority: 4Unprioritized - 6normal) [Bug 51310] create dedicated instance for exturl checking - https://bugzilla.wikimedia.org/show_bug.cgi?id=51310 [17:27:00] eh? [17:27:16] giftpflanze: I think toollabs can handle it just fine. Don't worry too much about it [17:27:32] i'm confused [17:28:55] giftpflanze: What about? [17:29:24] how i should program it [17:32:57] giftpflanze: Isn't it already written? [17:33:27] yes, with threading [17:33:41] not for jobs [17:35:54] giftpflanze: Well, I would start by running it as one job and see if it works in general on Tools. Afterwards, I would like at the architecture whether the threads can be rewritten as jobs. [17:36:26] in general it works, checking weblinks is simple [17:37:02] So everything's fine? [18:02:52] JohannesK_WMDE: Probably nfs playing up again :( [18:08:06] Damianz: you still around? [18:15:44] @techs: In the last days a lot of problems with SUL: Always logged out... :( [18:16:03] ls: cannot access project: Transport endpoint is not connected [18:16:11] wrong channel [18:16:26] Ryan_Lane: Paging Dr. Lane, Dr. Lane to the GlusterFS clinic [18:16:38] On orgchart.pmtpa.wmflabs [18:16:45] orgchart project? [18:16:47] Yup [18:17:04] Causing amusing but unhelpful messages for our nice HR folk [18:18:06] marktraceur: project or home? [18:18:14] project [18:18:23] Though maybe both, but I don't think I need home [18:18:33] Actually, that may be necessary for the backup scripts. [18:18:54] marktraceur: try now [18:19:18] Seems happier, danke [18:19:21] yw [18:19:36] Glad the database didn't crash [18:19:44] I *hate* the DB setup on that machine [18:20:05] Mongo is always crashing and leaving behind garbage lock files and corrupted data [18:24:32] Ryan_Lane: I feel like I should learn how to do that so I don't have to bother you [18:24:43] Since it happens pretty frequently [18:25:23] heh [18:25:36] marktraceur: well, it may be better to put it on /mnt [18:26:06] Noted [18:26:55] Last occurrence on 2013-06-03 [18:27:05] Maybe it's not *that* frequent [18:27:12] Though I think I'd rather know than be safe [18:27:27] Since the backups would also fail if /data/project were down [18:34:56] hmmmm, can't login [18:35:16] aude: via ssh or http to wikitech? [18:35:20] ssh [18:35:23] to labs [18:35:28] bastion? [18:35:31] or tools-login? [18:35:32] yep [18:35:35] bastion [18:36:51] hm. seems the keys mount isn't working [18:36:52] gets stuck at offering my public key [18:36:54] yeah [18:39:25] alright, going home... can try later [18:40:56] uuuggghhh gluster [18:41:25] hm. I think we're running the keys script on the nfs server [18:41:25] maybe I should just switch that [18:46:21] Ryan_Lane, demon and me (dr0ptp4kt) having problems on bastion.wmflabs.org, too. key denied. ditto for me trying to ssh dr0ptp4kt@zero-test.wmflabs.org [18:46:27] yeah [18:46:34] the nfs mount is broken [18:46:38] and stupid gluster won't start it [18:46:56] Ryan_Lane, thx. time to disavow computers. [18:47:24] <^demon> Ryan_Lane: Can I log in if we do some in-person auth protocol? I'll write a note telling you it's me, and you can let me login ;-) [18:47:33] :) [18:48:06] if I can't get this working, I'm switching the keys to the new nfs server [18:48:15] s/working/working soon/ [18:48:55] fixed [18:49:07] stupid stupid gluster [18:49:22] it doesn't actually check to see if services are running [18:49:41] if they fail, they just fail till you kill the managers and all the nfs services, then restart the manager [18:49:49] and all the nfs stuff has to be dead everywhere [18:57:55] ... apparently petan is taking a long lunch [19:19:33] T13 no lunch a dinner :D [19:19:43] I was preparing it several hours [19:20:39] What you make? [19:20:51] YuviPanda: you got two credit points in today's ops meeting ;) [19:20:59] Ryan_Lane: wooo! for what? [19:21:10] if you get enough of those you can cash them in for some whisky [19:21:11] :) [19:21:31] YuviPanda: grrrit-wm and redis with secret key prefixes [19:21:37] aah, nice :) [19:21:47] I blogged about the secret key prefixes thing too. [19:22:01] yep. I linked to it in the etherpad [19:22:24] i'm writing up an RFC of sorts about an icinga for tools, will post to labs-l in a while [19:22:57] Ryan_Lane: I'll keep count of the points :) [19:23:01] heh [19:23:07] YuviPanda: sounds good [19:23:13] in production we use exported resources in puppet [19:23:16] we can't do that in labs [19:31:17] Ryan_Lane, thx for restoring ssh access... [19:31:22] yw [19:31:24] sorry it broke [19:45:54] Ryan_Lane: https://wikitech.wikimedia.org/wiki/User:Yuvipanda/Icinga_for_tools [19:45:55] email sent [19:45:57] YuviPanda: did you take this into account when you did grrrit-wm? https://gerrit.wikimedia.org/r/#/c/64012/ [19:46:16] I'm going to abandon it, but I want their changes to apply :) [19:46:30] Betacommand: Yes [19:46:35] Ryan_Lane: no, don't think so. I just looked at the revert commit from ^demon and wrote that. [19:46:35] Well, sorta, cooking [19:46:41] ok [19:46:42] let me add it [19:46:45] great. thanks [19:46:54] it's trivial, just a YAML config change [19:46:54] so [19:46:55] Damianz: cluebot needs restarted [19:47:05] 3? [19:47:18] yeah the archive bot [19:47:37] Let me give it a poke, might see if I can stop it dying as well... before I get stuck into this thrift work [19:49:20] YuviPanda: ok. I'm going to abandon saying it's being added to grrrit [19:49:33] Ryan_Lane: yeah, I finished the commit. should be there in about 5 minutes [19:49:38] sweet. thanks. [19:49:44] <^demon> YuviPanda: Also, I made you your repo for it on gerrit, so we can get it off github :) [19:50:02] ^demon: yeah, can you delete that? and create it as 'grrrit' instead? [19:50:08] people don't seem to like lolrrit [19:50:25] <^demon> Why would they care? Doesn't affect anything :) [19:50:36] ^demon: I don't want it to be named different things :) [19:50:38] can you? [19:50:50] <^demon> *sigh loudly* [19:50:55] I can't hear you! [19:51:03] <^demon> *SIGHS AS LOUD AS POSSIBLE OVER IRC* [19:51:07] :D [19:52:08] my review queue is only a little longer than my change queue :) [19:52:45] * T13 covers his ears.. why all the shouting? [19:52:46] <^demon> `ssh -p 29418 gerrit.wikimedia.org delete-project delete --yes-really-delete labs/tools/lolrrit` [19:53:00] <^demon> --YESIMSUREJUSTDELETEALREADYOMGYOUSUCK [19:53:02] YuviPanda: you know we have icinga for labs, right? [19:53:11] YuviPanda: http://icinga.wmflabs.org/icinga/ [19:53:16] Ryan_Lane: yeah, but do you really want to have that check individual tool statuses? [19:53:21] it spams -operations, doesn't it? [19:53:24] no [19:53:27] !smb [19:53:27] YuviPanda: What's SMB? Some Magic Bean? [19:53:31] it spams some labs channel [19:53:32] !icinga [19:53:36] -_- [19:53:39] @search nagios [19:53:39] Results (Found 3): monitor, alert, nagios, [19:53:42] !nagios [19:53:43] http://208.80.153.210/nagios3 http://nagios.wmflabs.org/nagios3 [19:53:47] !monitor [19:53:48] http://nagios.wmflabs.org/cgi-bin/nagios3/status.cgi?host=$1 [19:53:52] * Ryan_Lane groans [19:53:54] Can't believe Cyberpower678 saved that... [19:53:55] !alert [19:53:55] http://nagios.wmflabs.org/cgi-bin/nagios3/history.cgi?host=$1 [19:54:02] where's the damn bot? [19:54:04] :D [19:54:28] T13, huh? [19:54:30] !smb [19:54:30] YuviPanda: What's SMB? Some Magic Bean? [19:54:55] Ryan_Lane: I think I was talkign to Coren about this. Not sure if we want to couple those two together, since this is specifically a lot more lightweight and 'open'. [19:55:13] <^demon> YuviPanda: labs/tools/grrrit created [19:55:16] <^demon> Old repo nuked [19:55:26] ^demon: can you do an import from https://github.com/yuvipanda/lolrrit-wm [19:55:27] ? [19:55:34] :o coren died [19:55:38] T13, what do you want. What did I save? [19:55:48] petan: Any reason why mysql randomly times out? [19:55:56] <^demon> YuviPanda: You can faster than I can :p [19:56:04] YuviPanda: meh, I have no strong feelings [19:56:05] ^demon: I can direct push? [19:56:14] <^demon> It's a new repo, fresh branch [19:56:18] Damianz: because you do not behave [19:56:18] <^demon> So yes [19:56:28] petan: It's no fun behaving [19:56:40] you should have already move to tools-db [19:56:50] That db is fine [19:56:54] replicas keep timing out [19:56:55] which? [19:56:57] ah [19:57:03] in that case, you do behave [19:57:03] T13? [19:57:06] ost connection to MySQL server at 'reading authorization packet' < randomly thoughout the day [19:57:06] ^demon: ah, sweet. let me do that [19:57:21] Damianz: I think that Coren set up too strict timeout limits [19:57:24] I'll move to tools-db when I have like a week to move the bot and can hassle people for packages [19:57:28] I thought it was you that assigned my quote to !smb [19:57:42] Damianz you can hassle me anytime you know XD [19:58:03] T13, no. [19:58:11] !cyberpower678 [19:58:17] !Cyberpower678 [19:58:17] addshore, how do you rollback? with the rollback button? :D [19:58:21] It's too hot - I want rain back :( [19:58:29] was done by addshore. [19:58:42] too hot? [19:58:47] it is never too hot in here [19:58:50] it is cold all time [19:58:57] I really want to move somewhere where it is hot [19:58:59] Damianz, I'm in Germany. We're breaking 40 Celcius. [19:59:01] like africa [19:59:02] It's 28C here atm, was 34 over the weekend [19:59:04] @infobot-detail smb [19:59:04] Info for smb: this key was created at 7/18/2013 1:54:21 PM by petan, this key was displayed 2 time(s), last time at 7/22/2013 7:54:30 PM (00:04:33.3635550 ago) this key is normal [19:59:09] fuck you all [19:59:16] er. I love you my friends [19:59:17] ^^ [19:59:22] Lol [19:59:30] I forgot sumana is watching me :P [19:59:30] Cyberpower678: You getting thunder storms this week? [19:59:37] Apparently we are, I look forward to them :D [19:59:37] NO [19:59:40] I WISH [19:59:42] petan: summana watches everyone [19:59:56] petan: I know for a fact it has been displayed more than 2 times... [19:59:59] Bug.. [20:00:01] that was just a common typo s/fuck you/I love you/ [20:00:14] meh [20:00:35] * Cyberpower678 hands petan LAVA 3000 heater. [20:00:47] * petan sinks in all the lava [20:01:24] !smb [20:01:25] YuviPanda: What's SMB? Some Magic Bean? [20:01:31] @infobot-detail smb [20:01:31] Info for smb: this key was created at 7/18/2013 1:54:21 PM by petan, this key was displayed 3 time(s), last time at 7/22/2013 8:01:25 PM (00:00:06.5354660 ago) this key is normal [20:01:34] petan, are you nice toasty now? :D [20:01:37] counter works [20:01:44] Cyberpower678 no, but lava is frozen solid [20:02:03] hmm.. [20:02:04] Damianz: whats the status of the archive bot? [20:02:26] * Cyberpower678 hands petan a magmanizer. [20:02:33] Betacommand archive bot was archived :> [20:02:35] See if that helps. [20:02:45] Betacommand: Should be running, I've put a cron in place to ensure the job is running for now [20:02:53] Should make sure it's not down for more than 15min or so [20:03:55] There are over 45000 external links that are listed on the spam blacklist. :/ [20:04:05] Only 450k [20:04:26] Damianz, it's not done scanning yet. [20:04:35] nfs llaaaagggg [20:04:39] ^demon: ! [remote rejected] master -> master (prohibited by Gerrit) [20:04:44] do i need to force push? [20:04:47] <^demon> dammit gerrit [20:04:50] <^demon> I'll do it [20:05:07] Btw does anyone use crypt_fs with a ssd? (Randomly) my laptop seems to suuuuccckkkk performance wise on disk more than I'd expect :( [20:05:29] Damianz, not me. [20:05:47] ^demon: :D [20:06:51] <^demon> Done'd. [20:06:55] \o/ [20:12:00] YuviPanda: Icinga for Tools would be very cool; there's Ganglia as well (https://bugzilla.wikimedia.org/show_bug.cgi?id=48737). If you happen to drive-by https://bugzilla.wikimedia.org/show_bug.cgi?id=50585 on your quest, very much appreciated :-). [20:12:56] scfc_de: :D you should respond on the thread too [20:13:40] Ryan_Lane: can you remind me of the parts of the multi instance self hosted puppetmaster docs you found confusing [20:13:42] ? [20:14:09] also: https://gerrit.wikimedia.org/r/#/c/75158 [20:14:13] ottomata: it's necessary to run puppet first on all of them [20:14:24] (i decided to support both hostname and fqdn anyway, so it is still backwards compatible) [20:14:25] then to add the class and config, which needs to be the same on all nodes [20:14:28] then run it again [20:14:31] ok [20:14:33] but you need to run it on the master before the clients [20:14:40] k riht [20:14:41] right [20:14:42] will fix [20:14:47] and it's not very clear that the master should also list the hostname [20:14:51] YuviPanda: Oh, the mailing list IRC announcer still not working :-). Re requirement #5, I think that's already in Icinga. [20:14:56] also the FQDN/short name issue [20:15:02] thanks :) [20:15:30] scfc_de: yeah, I assumed so. But authentication, etc? [20:15:32] scfc_de: I guess there should be a link in the email [20:15:33] YuviPanda: No idea. [20:15:42] scfc_de: 6 might be the hardest [20:15:58] YuviPanda: Yes, probably. [20:16:21] not sure how to do that, though :) [20:17:07] But on the other hand, we didn't shield tool authors from the effect of NFS outages either. So I wouldn't spend too much effort on that. [20:50:30] Ryan_Lane: ergh, my changes to a global in instance configuration isnt' sticking [20:50:54] scfc_de: sure, but NFS outage isn't actionable by them [20:51:10] scfc_de: I remember reading something about 'good practices for monitoring' someplace. Stressed on only having actionable alerts [20:52:44] ottomata: global instance config? [20:52:49] oh [20:52:51] to a global [20:52:54] heh [20:54:35] yes [20:54:47] i'm testing the puppetmaster hostname vs fqdn thing [20:54:56] i can't seem to change the var to be 'kraken-namenode.pmpta.wmflabs' [20:55:06] every time I come back to that page it is still set at 'kraken-namenode' [20:59:20] hashar: do you know anything about Varnish not working on beta labs? http://en.m.wikipedia.beta.wmflabs.org/wiki/San_Francisco [20:59:39] unrelated ? [20:59:45] we don't use varnish there [20:59:47] [bz] (8REOPENED - created by: 2Michelle Grover, priority: 4Unprioritized - 6major) [Bug 51694] Beta labs is down - https://bugzilla.wikimedia.org/show_bug.cgi?id=51694 [21:00:48] hashar: that link shows a fatal that looks like it's from varnish Request: 1 via deployment-cache-mobile01 deployment-cache-mobile01 ([127.0.0.1]:3128), Varnish XID 912598010 [21:00:48] Error: 500, Internal Server Error at Mon, 22 Jul 2013 20:59:41 GMT [21:01:14] hashar: this would be MobileFrontend on beta [21:01:51] seems to have been like this since Friday I hear [21:02:49] chrismcmahon: sorry got confused :) [21:03:24] yeah it is not working properly for me either hehe [21:06:03] chrismcmahon: got the stack trace :) [21:06:25] [22-Jul-2013 21:05:53] Fatal error: Call to a member function removeChild() on a non-object at /data/project/apache/common-local/php-master/extensions/MobileFrontend/includes/formatters/HtmlFormatter.php on line 256 [21:06:26] :( [21:06:38] nice [21:07:33] I think michelle wrote me an email about it [21:07:41] must be lost in the thousand mail I received last week hehe [21:10:09] ergh, yeah, Ryan_Lane, I can't tell if I broke role::puppet::self or if something is just weird with the puppet db (or wherever these global vars are stored) [21:10:44] chrismcmahon: I am filling a bug against MobileFrontend [21:11:17] ottomata: it's stored in ldap [21:11:29] ottomata: which instance and which variable are you trying to modify? [21:12:03] for simplicity, lets just look at limn0 [21:12:09] i'm trying to change puppetmaster [21:12:12] hashar: great, we were just talking about it in #wikimedia-mobile. we have many many IRC channels [21:12:17] to limn0.pmtpa.wmflabs [21:12:31] YuviPanda: How is that solves on production? [21:12:37] ok. which variable? [21:12:43] scfc_de: icinga selectiveness? [21:12:46] AH [21:12:51] Ryan_Lane: , i got it [21:12:54] ottomata: it's defined twice ;) [21:12:57] that global is available yeah [21:12:57] puppetmaster? :) [21:12:57] yeah [21:12:59] yeah [21:13:08] that was from before it was available in main pupet groups [21:13:11] its still in my dev one [21:13:11] fixin [21:13:14] ok thanks! [21:13:14] haha [21:13:18] scfc_de: i think it isn't, since the people who are responsible for the underlying infrastructure are also generally responsible for the services :) [21:13:26] it applies them in order, so it likely was overridding it [21:13:29] scfc_de: here we have two sets of people - labs admins who care about infra, and tool users who don't [21:13:48] aye [21:14:18] one of these days I'm going to make that interface not suck [21:14:26] maybe I'll just replace it with yaml input [21:15:09] yaml input + docs or something [21:15:18] Not sucking would be good. [21:15:33] yaml wouldn't not suck [21:15:49] because most people wouldn't know what to put there [21:15:51] yaml is teh suck. [21:15:59] YuviPanda: Yep, I meant that. I believe (very far leap) that on airplanes and similar machines some warnings are only displayed when there are no warnings of other sorts. But I don't know how easy that is in Icinga. [21:17:54] k i gotta run, laters [21:17:55] YuviPanda: Also because if tool A is broken due to infrastructure problems and the maintainer/user looks at Icinga for tool A and sees "everything's fine", that may be misleading. Perhaps only condition the alert triggers on the infrastructure working fine, but the display of the alarms themselves not? [21:18:41] scfc_de: indeed, I think we should display status of infrastructure too in the alert emails [21:18:57] scfc_de: 1. NFS 2. SGE 3. apaches 4. Redis 5. uwsgi (when it comes up) [21:19:04] that should be fine, I think? [21:20:15] YuviPanda: No, I meant alert per mail/IRC/etc. of failures in tool A *only* if infrastructure claims to be passing tests. [21:20:49] scfc_de: wait I think i misread your original statement. let me re-read [21:21:15] ah, right. I meant we should be 'hierarchical' of sorts - if the infra checks are failing, then don't bother with the local ones [21:21:20] display state, but do not spam them [21:21:24] we both agree, yes [21:21:38] so if nfs is down, and someone looks up their tool status, it'll say 'down' and also say 'NFS is down' [21:22:22] YuviPanda: Yes, so that they know a) that their tool is down and b) that they should nag an admin/ops/etc. [21:22:27] yup [21:26:23] [bz] (8REOPENED - created by: 2Michelle Grover, priority: 4Unprioritized - 6major) [Bug 51694] Beta labs is down for MobileFrontend - https://bugzilla.wikimedia.org/show_bug.cgi?id=51694 [21:28:34] scfc_de: https://wikitech.wikimedia.org/wiki/User:Yuvipanda/Icinga_for_tools [22:01:33] tools lab seems to be sluggish for me when I try to log in [22:02:07] bawolff: must be the NFS [23:40:30] [bz] (8NEW - created by: 2Alex Monk, priority: 4Low - 6minor) [Bug 48786] Labs should not be able to trigger Newprojects announcements - https://bugzilla.wikimedia.org/show_bug.cgi?id=48786