[00:08:33] Coren, ping [07:54:01] weee! I just created a database! [08:02:22] but when trying to set up mediawiki, I am redirected to http://tools-webserver-03/lifeweb/core/mw-config/index.php?localsettings=1 at the end [08:02:32] when I should download LocalSettings.php [08:06:51] ok, worked after I manually replaced the server name by tools.wmflabs.org [10:41:16] anybody around who can help me with creating a bot please? i'm form Wikivoyage [12:04:14] Coren, ping [12:04:40] petan, ping [12:04:49] kjih [12:05:12] petan, can you have a look at qstat for cyberbot and tell me what's wrong? [12:05:32] which one [12:05:47] Just look at the task list. [12:05:57] I guess you have too many :/ [12:06:02] Er what? [12:06:06] Coren did set some limits, though I have no idea why [12:06:20] On how many tasks can be running? [12:06:33] I think about 15 or 10 [12:06:42] or 16 [12:06:50] If I had known this, I would've just kept firing crontab every minute. [12:06:52] given that you have 16 running [12:07:02] I know it's a stupid limit [12:07:15] it makes people solve stuff in a silly way which usually is worse [12:07:45] It's easily bypassed by creating another Cyberbot project, but that's stupid. [12:07:48] you can create multiple service accounts if you want to bypass it [12:07:51] yes [12:07:57] I would do that :P [12:08:16] or you can ask Coren to increase it, or see why you need more than 16 jobs running at the same time [12:08:18] or maybe you can try to convince Coren to disable this limit, it has no use other than it causes troubles [12:08:33] Coren, can you up the limit to, say, 30. That's more reasonable for bots doing a lot of stuff like Cyberbot. [12:08:39] Yuvi|NoPower: maybe because he needs to execute many tasks in paralel? [12:08:49] and as a proper programmer, he issue 1 process per task [12:08:56] It's blocking adminstats now. [12:09:03] And Snotbot [12:10:42] petan, can't you up those limits? [12:11:24] I would happily do that, but Coren was kind of like "these limits will never need to be increased and if they ever needed to be, they need to ask me" or something like that... I asked him to document this a while ago, but he never did [12:11:39] there is an irc log of that [12:11:43] !logsearch [12:11:43] http://bots.wmflabs.org/~wm-bot/searchlog [12:12:32] !toolsadmin [12:12:32] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Documentation/Admin [12:12:34] !limitations is Labs has too many limits. [12:12:35] Key was added [12:12:40] !limitations [12:12:40] Labs has too many limits. [12:13:17] since the implementation of this limit I was notifying him it's gonna cause nothing but problems [12:13:22] So I fire cron every minute, but Coren doesn't like that so I convert them all to continuous tasks. [12:13:31] Now I can't run any more task. :/ [12:13:58] configuration of grid is just "FIXME" [12:14:03] in documentation for admins [12:14:43] petan, can wm-bot send a message when it sees Coren post here? [12:14:46] firing job in cron is usually better idea because it saves resources, but not every minute [12:14:56] you should start it like every 30 minutes or that [12:15:16] Cyberpower678: that's how @notify works [12:15:51] there is also memoserv but I personally hate it, because it's hard to use and nobody reads messages from it [12:16:00] @notify Coren Please remove the queue limit on tools. It's backing up on Cyberbot. [12:16:00] I doubt that anyone could have such a nick 'Coren Please remove the queue limit on tools. It's backing up on Cyberbot.' [12:16:10] eh [12:16:15] WTF? [12:16:19] well it notify YOU when he say something in here [12:16:46] I meant send a message to Coren when he is seen as active. [12:16:53] I was thinking of making wm-bot works like smart memoserv though [12:17:04] no, wm-bot can't do that [12:17:23] it would basically do same job as memoserv, just easier to use and incredibly more effective [12:29:26] petan, can you help me out? [12:30:11] petan, ping [12:30:18] hm [12:30:31] you just ask what you need instead of saying "ping" [12:30:43] like petan: "here goes the message" [12:31:07] Can you disrupt the PHP process for 920892 so it reboots. If I deleted and restarted it, it be sitting in the queue. [12:32:11] ? [12:32:15] disrupt? [12:32:21] you mean kill? [12:32:38] Yes, without deleting the actual process. [12:32:53] wait.. [12:32:55] not really easy... [12:33:02] I don't know which process it is? [12:33:44] I don't see any php process running on webservers for that user [12:34:05] jsub [12:34:06] [bz] (8RESOLVED - created by: 2denny vrandecic, priority: 4Normal - 6enhancement) [Bug 51107] set up public test repo with Wikivoyage links section - https://bugzilla.wikimedia.org/show_bug.cgi?id=51107 [12:34:10] ah on exec? [12:34:25] spambot? [12:34:28] yes [12:35:07] what is name of that php script it executes? [12:35:20] externallinks.php? [12:35:24] Yep. [12:35:35] I just found a bug. [12:35:44] killed [12:36:55] Can you kill it again? Sorry. [12:37:13] ok [12:38:06] done [12:38:19] Thanks. [12:38:22] yw [12:39:10] Seeya. [13:03:00] is Johannes Kroll in the house? [13:03:31] Silke_WMDE: Lydia_WMDE ? [13:23:20] wizardist: yes? [13:23:41] ah, Johannes_WMDE is working from a remote place [13:23:47] he's not here right now [13:24:08] wizardist: What's your question? [13:24:26] it's a shame :( e-mailing is bulky sometimes. [13:24:39] what do you mean? [13:24:50] it's easier to receive support via IRC :) [13:25:21] Did you e-mail Johannes? [13:25:24] sure [13:25:47] okay, I'll just go straightforward. I'm trying to add a new maintainer to the tool, and I added Johannes minutes ago. But the person I mean adding cannot be found in the Nova filters [13:25:53] username 'lvova' [13:26:36] Maybe lvova has not yet been added to the tools project... Coren petan could you check please? [13:27:00] yeah, I guessed that and if it appers so, please help :) [13:27:30] wizardist: Johannes is on deadline for the RENDER project, I think that's why he didn't answer you yet. [13:27:31] ok [13:27:40] !tr lvova [13:27:41] goes fishing, catches a trout, and smack lvova with it, over and over. [13:27:46] WTF [13:27:59] @infobot-detail tr [13:28:00] There is no such a key [13:28:03] Silke_WMDE: sure, he notified me about possible delays. It's understandable! [13:28:35] wizardist: Are you Anastacia'a colleague? [13:28:44] yep [13:28:51] :) [13:29:15] added [13:29:22] thanks! [13:29:24] yw [13:29:38] wizardist: try again to add her [13:30:26] like a charm! thanks petan & Silke_WMDE! [13:30:31] cool [13:35:19] * YuviPanda|away pokes Ryan_Lane [13:43:21] hmm, I see files in my /data/project/... directory owned by local-...:local-... (user&group), and it's 755, thus not giving me a change to write there. ??? [13:45:36] why not 775? [13:48:54] wizardist: are you working with the tool's account? You should "become" the tool. [13:49:19] hmmm, tools-login doesn't let me in as local-connectivity :) [13:49:39] wizardist: try 'become connectivity'? [13:49:46] you log in as yourself and then run "become connectivity" [13:49:51] ahahaha! [13:50:09] good joke is the one that is true! :) thanks [13:50:24] hehe [13:50:53] (If you may point me out the part of Docs that I missed, that would be very appreciated!) [13:50:56] That's here: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Joining_and_creating_a_Tool_account [13:51:12] and further below [13:51:39] I think the general rule is that all maintainers want to work as "the tool" all the time [13:51:41] aha, I think I did this before when creating the tool with coren, but it's been a while... [13:52:07] can two maintainers work as "the tool" simultaneously? [13:53:37] I suppose so. petan? ^^ [13:53:45] yeah, shouldn't be a problem, wizardist [13:54:03] wizardist: Yes, "being a tool" is just a switch of UID. [13:54:04] thanks YuviPanda [13:54:18] Oh, man, the jokes just write themselves. :-) [13:54:25] reasonable explanation :D [13:55:23] http://partiallyclips.com/2002/03/03/pool-table/ [13:55:28] Coren: thanks for merging my puppet patches (yesterday?) [13:56:08] YuviPanda: Yesterday. atm, I'm still on my first coffee as I drift towards the right time zone. [13:56:21] well, I just woke up 2 hours ago [13:56:44] I've no idea what happened to Monday and Tuesday. [13:56:58] LOL Be the ball :) [13:56:59] [bz] (8VERIFIED - created by: 2Aude, priority: 4Unprioritized - 6normal) [Bug 53158] setup devrepo on new, larger instance - https://bugzilla.wikimedia.org/show_bug.cgi?id=53158 [13:57:34] I should just start referring to things as 'X sleep cycles ago' [13:57:36] than in 'days' [14:00:20] Coren: also, do you know if NFS supports something like inotify? [14:00:22] Hasteur|ZzZ: no cloak yet [14:00:23] ? [14:00:26] or even inotify itself? [14:00:35] Coren: googling doesn't give me a definitive answer [14:00:50] !cloak [14:06:13] !cloak is If you have a verified e-mail address, at least 250 edits on Wikimedia projects, and have been registered for at least three months, you may request a Wikimedia cloak at: http://is.gd/2RwcCz - Full details are available at: http://meta.wikimedia.org/wiki/IRC/Cloaks [14:06:14] Key was added [14:11:21] YuviPanda: I'm... not sure. I don't think it does, but since v4 has opportunistic locking it actually might. [14:11:34] how does toolswatcher work? [14:11:47] does it just poll? I always thought it would use inotify or something [14:14:20] can someone tell me how much diskspace the english wikipedia takes [14:16:01] YuviPanda: *blush* It does the silly poll thing; but it polls LDAP not the filesystem. [14:16:14] poller! pfft. [14:16:22] well, that's not so bad [14:17:06] Coren: me and legoktm were talking about writing a small deamon that'll make scp not such a pain. The deamon wouldn't need setuid so people can just run them on their personal account. [14:17:14] but without inotify, I can't think of a nice way to do it :( [14:17:24] Ryan has a "invoke salt on $change from wikitech" plan for the long term that will be better. [14:17:38] (essentially, it should watch ~/ and copy it to /data/project/ and do a take) [14:17:58] ... how atrocious. [14:18:22] well, current thing is you scp to your home, then ssh in and copy it wherever, and then do a take [14:18:24] and then repeat [14:18:25] ad infinitum [14:18:27] painful [14:18:47] I won't poll NFS because that just is inelegant and most probably stupid. [14:19:06] * YuviPanda sighs [14:19:59] Wait, why sco to your home? Users have write access to their tools' homes. [14:20:28] Coren: i still have eto take [14:20:32] *to [14:21:05] and inotify for all of the tool account's subdirectories sounded bad :P [14:21:09] For the most part, yes. Although, strictly speaking, only executables from the website really need it. [14:21:22] it's a good practice in general, I thought [14:21:29] s/thought/think/ [14:21:37] so it essentially does 'autotake' [14:43:56] test [14:44:42] petan: test received [14:44:45] YuviPanda: are you interested in improving irc logs? eventually tools like log searching? [14:45:02] it's that wm-bot now stores logs in mysql... so we could create some nice interface for them [14:45:26] if you are not interested I will create it in php for which you would hate me :P but I just don't feel I can do that in python [14:45:48] Nettrom: nice :P [14:46:06] Is there a document anywhere on setting up https on a (non-toollabs) labs instance? [14:46:08] * Technical_13 is petan. [14:46:20] o.O [14:46:23] * petan is not Technical_13 [15:19:38] YuviPanda, try now? [15:35:39] andrewbogott: indeed, I see it as 1500 now [15:35:41] * YuviPanda installs [15:36:02] andrewbogott: 'warning - the following packages cannot be authenticated' - is that okay? [15:36:06] andrewbogott: it installs them fine :) [15:36:19] Yeah, it's OK -- I didn't sign it. [15:36:26] alright [15:36:33] now to apply the puppet classes for the proxy [15:38:36] andrewbogott: puppet fails [15:38:45] andrewbogott: because of lack of signing [15:38:53] hm [15:39:15] andrewbogott: https://dpaste.de/LsW1F/ [15:43:01] petan: I think IRC logs should go into something like logstash / elasticache. MySQL doesn't seem to be the right thing for the job of 'search logs fast' [15:43:08] but don't think I'll have the time anytime soon, sorry [15:43:28] YuviPanda, I'm tempted to say you should just force it for now, I don't immediately know how to sign a custom deb [15:43:35] :'() [15:43:43] :'( [15:43:44] ok [15:44:02] um… I will try to revisit later on. [15:44:05] YuviPanda: but logstash, neither elasticache is flexible enough. I need to perform sql queries on them in order to convert them to custom formats like wikitext [15:44:36] andrewbogott: okay. I'll poke Ryan when he comes back to see if there's a quick solution, and if not will force it [15:44:39] you can hardly implement logstash data to mediawiki easily [15:45:04] andrewbogott: at least will need to figure out a way to force it directly from puppet, rather than hand-install it [15:45:31] Is this a class that's going to get installed on multiple systems in multiple projects? [15:45:46] andrewbogott: it will be reused at least on toollabs [15:45:52] other than project-proxy [15:46:32] andrewbogott: there's already a role for it on toollabs. [15:57:03] Coren, ping [15:57:13] Yeeees? [15:57:29] Coren, did you get my meesages, from me a petan? [15:59:14] Talking about the queue limits? They're there to prevent runaway jobs (like peeps who schedule cron jobs every minute that last more than one minute and grow to fill all slots, by accident or otherwise). Slots are a finite resource; you can split your tools into a few components (which might be a good idea for [15:59:43] other reasons), or you can return some of them to cron that are more amenable to running at interval (but like /not/ every minute, please!) [15:59:52] Coren: ok but 16 is little but small isn't it? [16:00:06] I can think of many tools that would need more than 16 jobs [16:00:28] I can't think of a single, well-delineated task that needs that many jobs. What do you have in mind? [16:00:44] any wikipedia bot that executes 16 simultaneous tasks? [16:00:52] not everyone makes a new bot for single task [16:01:06] echoing petan [16:01:20] By definition, if you're running 16 jobs, you're not doing one thing sixteen times; you're doing sixteen things. [16:01:30] yes [16:01:45] also imagine a bot that works cross-wiki [16:01:54] each wiki in separate process [16:01:58] Coren, the grid seems already well configured enough to disallow duplicate jobs, so if the cron is starting a job every minute, and that job is already running, it won't start it. [16:02:01] that would be some 600+ jobs :P [16:02:34] petan: That would also be a very, very badly written bot. 600 times a couple hundred megs of ram in resources for a single bot? [16:02:46] well I admit it would be stupid [16:03:00] But the limit we have now is also dumb. [16:03:08] but on other hand the bot could be doing 16+ paralel tasks in 1 moment [16:03:09] It serves no purpose. [16:03:20] Like mine. [16:03:33] Cyberpower678: It serves to prevent one tool from consuming every slot. [16:03:39] maybe rising it to higher value could work? or make exceptions per tool? [16:03:50] Can't you up it to 30? [16:04:01] like adjust it for tool A by N [16:04:07] Yes, exceptions can be made per tool, given a compelling use case that clearly would be better served with a higher limit. [16:04:14] I've got several tasks design to run continuously. [16:04:33] Coren: ok but then a process to create such exception needs to be documented at Documentation/Admin [16:04:34] :P [16:04:45] That would otherwise be handled by executing cron every minute. [16:05:20] Cyberpower678: I have a really hard time buying this; you'll have to be considerably more explicit in your use case before I can understand why that's the case. [16:05:40] You're doing >16 different things that /need/ to be done in parallel? [16:06:22] And every minute, to boot? [16:07:06] The cross wiki runpage checker, status report, the RfX tally and Report, all run continuously. [16:07:31] The maintanence tag dating, was also meant to run continuously. [16:08:03] And you yourself said that firing the cron as little as possible conserves resources. [16:08:14] So I have been trying to keep that to a minimum. [16:08:35] Yes, but you're describing several independent tools to me, and about half a dozen of them. Why do you need >16 jobs? [16:08:55] Look at Cyberbot job list. [16:09:04] * Coren goes looks. [16:11:53] I see a number of bots that run on more than one wiki. Any reason they can't check the wikis sequentially? [16:12:28] ? [16:12:47] And, honestly, you might want to split off some of those to their own tools -- they seem completely unrelated to each other and probably could gain distinct maintainers over the years. [16:12:59] Let me elaborate on the first point... [16:13:47] I'm going to be creating a second Cyberbot project if the limit won't be raised. [16:13:49] Adminstats checks and tallies on wiki X, then sleeps a while, then loops back right? [16:14:07] Adminstats runs once a day. [16:14:30] It's not a continuous script. [16:14:37] Ah. Okay. [16:15:10] But I'd split up the RfX stuff away from the others, for one. Seems like a natural division to me. [16:15:17] And they are seperated by the hour to prevent massive simulataneous job loads. [16:15:29] Yeah, that's all done exactly right. [16:15:47] They're all Cyberbot [16:16:07] Why would I want to split up my bot? It makes it more confusing for me. [16:16:12] ... you mean you use the same code base for all those different things? [16:16:33] Adminstats? [16:16:47] Well, no, not adminstats; I mean your continuous bots. [16:16:59] Yep. [16:17:02] Ah, yes, adminstats is done right. [16:17:16] * Coren hates having two interleaved therads. :-) [16:18:49] Coren, like I said, keeping things in one place, makes it simpler for me. Less confusing. [16:18:52] Hm, allright. If they're all running the same code then obviously they need to be kept with each other. [16:19:34] I'm just really surprised that you need distinct processes for this. [16:19:42] * Coren ponders. [16:19:56] Maintanablilty. [16:20:08] If one breaks, everything else won't go to hell. [16:20:29] Well, I do have one question. Originally, you had most of those invoked from cron at (very brief) interval, right? [16:22:32] Like "X started every minute doing stuff then exit". You then converted them to "X doing stuff, sleep a while, start over". Right? [16:23:48] What would prevent you from having "Do X, do Y, do Z, sleep a while, start over"? I don't mean modifying your code to do everything; just wrap them in a shell script that calls each in turn then sleep? [16:24:20] I converted Do X everyminute, to keep doing x [16:24:20] pong Cyberpower678 [16:24:22] cyberbot.py do_one_thing; cyberbot.py do_something_else; sleep 60 etc [16:24:39] addshore, did you get my memo> [16:25:05] I wouldnt call it taken over :P [16:25:13] addshore, I would. [16:25:14] thats not have brfas work xD [16:25:15] :p [16:25:30] ill take a look at it as soon as I find the time though :/ [16:25:42] addshore, he said another BAGger should approve and not. [16:26:01] Coren, I don't do Python. [16:26:15] Cyberpower678: so I will, or O wont, but there is no point in me saying that I might xd [16:26:17] I used .py as example. I dunno what cyberbotI is written in. :-) [16:26:34] PHP [16:27:10] Coren loves php ;p [16:27:21] And like I said, if a script breaks, and it does a thouasand things, a good chunk of the bot will fail. [16:27:22] Cyberpower678: what are you trying to do? [16:27:30] addshore, ? [16:27:33] clarify. [16:27:48] cyberbot.py do_one_thing; cyberbot.py do_something_else; sleep 60 etc ? :P but in php? [16:28:28] addshore, I'm trying to convince Coren to give me more job slots. [16:28:33] :P [16:28:58] how many tasks are you running? :O [16:29:35] Cyberpower678: I can raise your slot limit, but I don't want to raise it too high nor forever. It's a finite resource, and I'd rather you look into ways to reduce the number of slots you are taking so that there aren't a disproportionate quantity of resources allocated to just your bot. [16:30:17] Cyberpower678: At some point, if you can't do that, you'll have to consider that you might have outgrown tool labs and look into spinning off into your own project. [16:30:35] Coren, can you raise to 25. That should satisfy everything for a good while. [16:30:44] Cure, 25 is reasonable. [16:30:46] Sure* [16:31:08] Coren, sorry. I'm unable to Cure 25 :p [16:31:50] addshore, 16 are running and 9 are attempting to run. [16:31:53] Cyberpower678: how are you running so many jobs simultaneously? :P [16:32:00] what are they doing? :P [16:32:09] Various stuff. [16:32:24] SoxBot, NoomBot, Snotbot [16:32:27] My bot [16:32:43] I'd say they should be different tool accounts, but hey - what do I know? [16:32:44] meh [16:32:46] Cyberpower678: Another alternative might be to give you a queue and exec host where you get to manage your own resources; this way if your tools consume too much they don't impact others. [16:32:48] * PissedPanda goes to do something else productive [16:32:52] PissedPanda: same ;p [16:33:23] Coren, fascinating. [16:33:58] What's the catch? [16:34:43] <{{Guy}}> What was his issue? [16:35:05] Read the logs. [16:36:08] Cyberpower678: Heh. There's no big catch except I'm not going to allocate an extra-large instance just for you, especially since you'd be the only one able to use it. Best I can reasonably give you is 1 CPU and 2G of ram. If all your bots use the same interpreter, though, your actual memory footprint shouldn't be too bad. [16:37:30] Coren, cool. [16:37:49] Will that give me limitless slots, only bounded by available memory? [16:38:01] I can configure it this way, yes. [16:38:16] Well, not limitless (there has to be a number), but arbitrarily large. [16:38:25] Then let's do that. [16:39:34] Allright, it'll take me a bit to do this; I've got no automation for that scenario. I'll probably manage to have it today if I'm done with FCGI soon enough. [16:39:53] Will I have to migrate to a different location? [16:40:09] No, just specifiy a different queue for your jobs. [16:40:15] Ok. [16:40:17] hey labs folks, is there a template for an IRC bot that I can easily augment with some new functionalities and deploy in labs (preferably written in Python) [16:42:20] <{{Guy}}> drdee: template? [16:42:31] wel not a template template [16:42:35] but an example [16:42:46] something that's already running [16:42:54] but doesn't do too much [16:43:00] so i can adjust it to my needs [16:43:35] <{{Guy}}> Most of the irc bots I know of are written in C# [16:44:05] really [16:44:20] no python irc bots on labs at all? [16:44:32] Mine is in Java [16:44:38] drdee: there are :) [16:44:40] Oyoyo is a python IRC base [16:44:52] does anybody know Willie: http://willie.dftba.net/ [16:45:45] drdee: https://github.com/mzmcbride/irc-bots [16:45:52] ty [17:34:40] !Logs [17:34:43] !logs [17:34:44] http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-labs/ [17:35:58] drdee: what you need that bot for? [17:36:37] I recommend you (if it is a simple task) to implement that into wm-bot it is heavily extensible, you can even call terminal wrapper to relay messages through it [17:37:05] <{{Guy}}> petan: wm-bot isn't in python. [17:37:13] something like echo Hi people | wm-bot #blah [17:37:16] you dont understand [17:37:20] <{{Guy}}> !python [17:37:21] EEEWWWWWWW!!!!! [17:37:25] {{Guy}}: it doesnt matter what language it is [17:37:28] it can be even python [17:37:29] !python del [17:37:30] Successfully removed python [17:37:43] PissedPanda now when you remove it we never find out who inserted it :P [17:37:50] heh [17:37:56] <{{Guy}}> Yuvi you're no fun... [17:38:17] it is a tcp / udp relay, it works in all languages that support tcp and python does :P [17:38:28] I myself created a python script that is using it [17:39:12] anyway if you insist on having own bot, there is plenty of python irc bots out there [17:40:23] !python [17:40:23] There are multiple keys, refine your input: pythonguy, pythonwalkthrough, [17:41:05] I am having a hard time deciding which key I should alias !python to :P that guy is just awesome [17:41:34] <{{Guy}}> !python is EEEWWWWWWW!!!!!!!!!! [17:41:35] Key was added [17:41:40] <{{Guy}}> xD [17:41:44] no we know who inserted it! [17:41:56] <{{Guy}}> This time. [17:42:13] everybody knows I love python XD it couldnt be me [17:42:48] Python isn't 'EEEWWWW!!!!!!' it's "Meh - I've had better." [17:42:54] but you are right that YuviPanda takes stuff too seriously, hence he is pissed [17:43:18] Coren: lol that sounds like something you would say after having sex with someone [17:43:21] * {{Guy}} hugs PissedPanda [17:44:57] @notify Ryan_Lane [17:44:57] I'll let you know when I see Ryan_Lane around here [17:45:29] * petan takes a cleenex and removes all that piss off YuviPanda [17:46:38] <{{Guy}}> Yuvi, avoid the monkeys. [17:46:46] <{{Guy}}> They fling poo [18:06:01] * PissedPanda pokes Ryan_Lane [18:06:10] :D [18:06:12] howdy [18:06:15] hello! [18:06:24] Ryan_Lane: saw the messages in labs-l about labsdebrepo? [18:06:39] Ryan_Lane: i can manually install the new package now, but puppet can't (since they aren't signed, and it can't verify them) [18:06:45] the package isn't signed by a trusted gpg key [18:07:01] sign them and trust the key? :) [18:07:07] andrewbogott: ^ [18:07:12] puppet should really generate one for the local repo [18:07:14] and add it [18:07:30] ben added support for this deb stuff a while ago and kind of half-assed it [18:08:18] Ryan_Lane: yeah, I don't even know if it worked at all until andrewbogott fixed parts of it [18:08:38] Ryan_Lane, would we be signing the packages in the repo, or the repo itself? [18:08:51] oh. sorry. the repo itself [18:09:02] wait. no. the packages [18:09:12] hm [18:09:32] heh [18:09:53] I guess it's the repo, which lists the hash of the packages [18:11:37] Now that I think about it, it must be the repo. Because I've dropped unsigned packages onto brewster before and puppet doesn't mind that. [18:12:22] But, ok, the repo is generated by puppet running on individual labs instances. So, not sure there's anyway to have a truly safe signature [18:14:31] andrewbogott: well, it's a local repo [18:14:38] so each project should generate their own [18:14:52] puppet should generate a gpg key if one doesn't exist [18:15:02] so just generate a keypair right there next to the repo? [18:15:06] yeah [18:15:15] the repo is relatively insecure anyway [18:15:21] make sure the permissions are root only [18:15:37] ok [18:33:47] Coren: you should probably remove the (known issue) from /topic [18:34:08] I should indeed. [18:34:23] andrewbogott: think you can add the puppet stuff for keypair addition? [18:35:01] PissedPanda, yeah, I'll look at it. [18:35:08] Unless you want to! [18:35:09] thank you, andrewbogott! [18:35:11] no :P [18:35:16] my apt is ~2 years old [18:44:52] So my mysql "just" stopped working labs [18:45:02] I'm not sure what happened but I suspect puppet [19:07:40] I just got a 'new notification' email which didn't tell me what the notification was [19:09:14] Coren, what does it mean that qstat shows several tasks with state 'Rr' and one with state 'r'? [19:09:33] R just means that the job has been restarted at some point in the past. [19:09:38] r means running [19:10:05] OK, so 'r' should be good news [19:10:10] So would Rr [19:10:31] Yep, I follow. Thanks. [19:10:38] np [19:43:57] !log wikistats - fixed stats updates for a couple hundred external mediawikis, by adding a trim() :p for some reason some API results have whitespace [19:43:58] Logged the message, Master [20:04:16] Coren, I'm back. [21:24:38] Coren: fy the "create new tool" link on tools.wmflabs.org is broken now [21:30:13] legoktm: yep [21:30:29] I'd fix it, but I don't know where that code it [21:30:30] *is [21:30:33] and it's probably perl [21:30:33] well, i managed to figure it out :) [21:30:36] :P [21:30:42] I wrote an email about the change ;) [21:30:51] Special:NovaServiceGroup [21:35:39] Ryan_Lane: Bah. It's html, and it's in git. [21:35:46] :D [21:35:57] is it in the tools repo? [21:36:04] a ugly dirty concotation of php and html, you mean :P [21:36:05] (Actually, it's HTML embedded in php, but same difference) [21:36:27] oh. wow. php? [21:36:36] Ryan_Lane, ? [21:36:40] the most horrible language of them all! :D [21:36:41] Ryan_Lane: www/ in labs/toollabs [21:36:47] Cyberpower678: ? [21:36:47] Ryan_Lane: look at it, it's fun :P [21:37:10] Meh. PHP does exactly what's needed when all you need is generate a bit of dynamic html in an otherwise static-ish page. [21:37:16] :) [21:38:42] I scheduled a reboot of virt11 [21:39:20] on a friday :D [21:39:32] * YuviPanda sells pitchforks outside the office [21:43:30] That's actually considerably more courageous than first appears; not only is this on a friday, but the weekend during which an eventual catastrophe could occur is immediately followed by all of the people likely to be woken up by said catastrophe being within arm's reach. :-) [21:44:11] oh that too [21:44:26] they'll be woken up right before long flinghts [21:44:29] *flights [21:44:40] so first deal with a server waking them up, then the TSA [21:44:42] mmm [21:44:56] * Coren wonders if there are torches and pitchfoks available at the duty free. :-) [21:45:53] Coren: alternatively you could buy alcohol at the dutyfree and then make Ryan_Lane drink them. [21:46:04] same as getting pitchforked [21:46:27] Coren: so, can I get a machine on tools, with the proxy role? [21:46:36] I don't need root on it, just need to be able to ssh in [21:46:47] to play with the proxy/grid service (which I'm going to call misty-unicorn) [21:46:55] (or something equally ridiculous) [21:47:17] Coren: no one cares about labs [21:47:21] no one from ops anyway [21:47:47] and other than me and andrewbogott, who's likely to fix anything? [21:47:52] Coren, hi. How's the new Exec node coming? [21:47:56] maybe you? :) [21:48:02] ... :-P [21:48:10] I go eat teh f00dz now. [21:49:59] !mob is http://cdn.head-fi.org/0/01/350x298px-015c657f_torchmob.jpeg [21:50:00] Key was added [21:52:16] hrm [21:52:22] YuviPanda: my flask tool isnt working :< [21:55:41] why, legoktm? [21:56:11] im not sure [21:56:15] i just added you to it [21:56:18] "periodssuck" [21:56:23] http://tools.wmflabs.org/periodssuck/cgi-bin/main.py/ [21:56:26] gives me a 500 [21:56:46] hmm, enable debugging? [21:56:52] i don't think we documented that, did we? [21:56:54] >.< [21:57:16] legoktm: i can help debug it in a while, finishing up the tool for Romaine + am on the phone with a friend [21:58:02] i added the debugging thing [21:58:04] ok, thanks [22:05:01] [bz] (8NEW - created by: 2spage, priority: 4Unprioritized - 6normal) [Bug 53778] empty Echo notification emails from Nova - https://bugzilla.wikimedia.org/show_bug.cgi?id=53778 [23:02:56] Coren|Dinner, just how much can you eat? :p [23:03:58] Nom nom nom... [23:20:36] Cyberpower678: Not going to be ready today; sorry. Working of FCGI atm. [23:20:46] why FCGI!? whyyyyy [23:20:55] we could have it all with the nginx proxy :P [23:21:13] Coren: did you get my message earlier about asking for a tools-proxy instance with that role? So I can build out the proxy? [23:21:13] YuviPanda, because of the ridiculous loads that accumulate at times. [23:21:14] YuviPanda: Because it's requested by many users. [23:21:33] Coren: oh sure, but it will work trivially with the nginx proxy, no? [23:21:44] spawn on grid, add port to proxy, and boom [23:22:09] needs no work [23:22:12] Coren, did Sean respond about archive? [23:22:24] Coren: also, can I still have a tools-proxy? :D [23:22:27] YuviPanda: Except that 99% of FCGI programs expect to talk to a unix socket on the webserver, and can't start on arbitrary TCP port? [23:22:35] Some people are beginning to ask questions. [23:23:02] Cyberpower678: No, not yet. "Some people" should ask their questions on labs-l. [23:23:07] there are ways around that do not involve apache. [23:23:26] anyway, Coren I can haz tools-proxy? [23:23:40] YuviPanda: Not today, I'm about to head off. [23:23:46] But tomorrow early. [23:24:04] Coren: that's good enough :) [23:24:16] Coren: I'll remember to poke you. [23:24:18] Coren: Thanks! [23:30:45] mutante: I thought you were using QR scan codes to manage keys with that !mob command-response above :) [23:32:34] spagewmf: hehe, last time i scanned one was to get bitcoin:) [23:35:32] Ryan_Lane: thanks for the extra instances, please may editor-engagement get another floating IP? (at 5/5 right now) [23:36:55] sure. one sec. [23:41:04] So... I know I'm behind the times; I have one script running on the bots project and I have not moved it to tool labs yet. [23:41:42] I need to modify it now, but a.) I'm apparently not in the bots project anymore, and b.) I have no idea where it's running from, though I know it is running. It used to be on -bnr3. [23:43:00] Where can I find it to modify/move it? [23:50:48] anyone, our labs instance toro keeps hanging in a "/bin/sh /usr/sbin/invoke-rc.d mysql start" I believe triggered by a puppet package update [23:52:03] andrewbogott: any luck with the signing? [23:52:18] I'm working on it now, making a bit of progress. [23:52:34] wheee [23:53:23] it's failing with "Can't start server : Bind on unix socket: No such file or directoryDo you already have another mysqld server running on socket: /mysqld/mysqld.sock ?" [23:54:33] that path looks wrong. First time it happened today I just ran `stop mysql; start mysql` but it's happened again (another puppet run?)