[02:47:44] 6Labs, 10Labs-Infrastructure, 3ToolLabs-Goals-Q4: Move LabsDB aliases to DNS - https://phabricator.wikimedia.org/T63897#1451228 (10yuvipanda) Designate is done, split horizon is done, etc. We've switched over to designate - time to pick this back up. [03:22:04] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Wbm1058 was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=170147 edit summary: [06:25:36] How to configure Labs server, so it doesn't download exe-files from
, but execute them through mono? I want to make CGI tool with C# exe file. Putting into .lighttpd.conf strings "static-file.exclude-extensions = ( ".php", ".pl", ".fcgi", ".exe" ) cgi.assign = ( ".pl" => "/usr/bin/perl", ".py" => "/usr/bin/python", ".pyc" => "/usr/bin/python", ".exe" => "/usr/bin/mono", )" not works [07:02:07] 6Labs, 10Labs-Infrastructure, 10wikitech.wikimedia.org: Remove [?] links from Special:NovaInstance - https://phabricator.wikimedia.org/T105770#1451321 (10Nemo_bis) 3NEW [08:40:09] PROBLEM - Puppet staleness on tools-bastion-01 is CRITICAL 11.11% of data above the critical threshold [43200.0] [09:54:17] 6Labs, 10wikitech.wikimedia.org, 3Labs-Sprint-105, 5Patch-For-Review: remove nutcracker from wikitech - https://phabricator.wikimedia.org/T102993#1451489 (10fgiunchedi) everything else being equal I'm in the "symmetry wins" camp, IOW nutcracker in wikitech [09:55:56] PROBLEM - Puppet failure on tools-static-02 is CRITICAL 100.00% of data above the critical threshold [0.0] [10:07:32] PROBLEM - Puppet failure on tools-static-01 is CRITICAL 100.00% of data above the critical threshold [0.0] [12:24:09] Damianz: CBNG seems more broke than usual. [12:39:12] (03PS1) 10Sitic: Stop click propagation for links in comments [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/224605 [12:39:25] (03CR) 10Sitic: [C: 032 V: 032] Stop click propagation for links in comments [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/224605 (owner: 10Sitic) [12:53:42] PROBLEM - Puppet failure on tools-exec-1209 is CRITICAL 50.00% of data above the critical threshold [0.0] [12:54:45] PROBLEM - Puppet failure on tools-exec-1215 is CRITICAL 40.00% of data above the critical threshold [0.0] [12:54:47] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1203 is CRITICAL 30.00% of data above the critical threshold [0.0] [12:55:33] PROBLEM - Puppet failure on tools-exec-1218 is CRITICAL 22.22% of data above the critical threshold [0.0] [12:56:13] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1201 is CRITICAL 66.67% of data above the critical threshold [0.0] [12:56:29] PROBLEM - Puppet failure on tools-exec-1207 is CRITICAL 60.00% of data above the critical threshold [0.0] [12:56:39] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1204 is CRITICAL 30.00% of data above the critical threshold [0.0] [12:56:57] PROBLEM - Puppet failure on tools-exec-1203 is CRITICAL 33.33% of data above the critical threshold [0.0] [12:58:51] PROBLEM - Puppet failure on tools-exec-1205 is CRITICAL 60.00% of data above the critical threshold [0.0] [12:59:15] PROBLEM - Puppet failure on tools-exec-1219 is CRITICAL 66.67% of data above the critical threshold [0.0] [12:59:26] I will let you know when I see hashar around here [12:59:26] @notify hashar [13:00:07] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1208 is CRITICAL 66.67% of data above the critical threshold [0.0] [13:00:35] PROBLEM - Puppet failure on tools-exec-1211 is CRITICAL 20.00% of data above the critical threshold [0.0] [13:01:04] PROBLEM - Puppet failure on tools-exec-1204 is CRITICAL 40.00% of data above the critical threshold [0.0] [13:03:00] PROBLEM - Puppet failure on tools-exec-wmt is CRITICAL 50.00% of data above the critical threshold [0.0] [13:03:28] PROBLEM - Puppet failure on tools-mailrelay-02 is CRITICAL 30.00% of data above the critical threshold [0.0] [13:28:44] RECOVERY - Puppet failure on tools-exec-1209 is OK Less than 1.00% above the threshold [0.0] [13:31:12] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1201 is OK Less than 1.00% above the threshold [0.0] [13:31:30] RECOVERY - Puppet failure on tools-exec-1207 is OK Less than 1.00% above the threshold [0.0] [13:33:52] RECOVERY - Puppet failure on tools-exec-1205 is OK Less than 1.00% above the threshold [0.0] [13:34:18] RECOVERY - Puppet failure on tools-exec-1219 is OK Less than 1.00% above the threshold [0.0] [13:34:44] RECOVERY - Puppet failure on tools-exec-1215 is OK Less than 1.00% above the threshold [0.0] [13:34:48] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1203 is OK Less than 1.00% above the threshold [0.0] [13:35:06] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1208 is OK Less than 1.00% above the threshold [0.0] [13:35:36] RECOVERY - Puppet failure on tools-exec-1218 is OK Less than 1.00% above the threshold [0.0] [13:36:02] RECOVERY - Puppet failure on tools-exec-1204 is OK Less than 1.00% above the threshold [0.0] [13:36:36] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1204 is OK Less than 1.00% above the threshold [0.0] [13:36:54] RECOVERY - Puppet failure on tools-exec-1203 is OK Less than 1.00% above the threshold [0.0] [13:37:56] RECOVERY - Puppet failure on tools-exec-wmt is OK Less than 1.00% above the threshold [0.0] [13:40:41] RECOVERY - Puppet failure on tools-exec-1211 is OK Less than 1.00% above the threshold [0.0] [13:43:27] RECOVERY - Puppet failure on tools-mailrelay-02 is OK Less than 1.00% above the threshold [0.0] [13:54:07] Hi, I have a question with regard to spam. I have created a abuse filter that should delete spam entries http://math.beta.wmflabs.org/wiki/Special:AbuseFilter/1 however I could not figure out how to aply this simple filter that should only delte pages created in a specific time period [13:54:48] I think asking in #wikimedia-dev or #mediawiki might be better physikerwelt [14:00:06] RECOVERY - Puppet staleness on tools-bastion-01 is OK Less than 1.00% above the threshold [3600.0] [14:01:54] YuviPanda|zzz... thanks... I created T105790 since it's not urgend [14:13:29] hello [14:13:33] is the DBA back? [14:14:36] no progress on https://phabricator.wikimedia.org/T105713 [14:18:26] hello? [14:25:18] [13intuition] 15siebrand pushed 1 new commit to 06master: 02https://github.com/Krinkle/intuition/commit/03cd7d72da5e21f0d66c7146ae6ce3f1e3e139f5 [14:25:19] 13intuition/06master 1403cd7d7 15Siebrand Mazeland: Localisation updates from https://translatewiki.net. [14:28:38] hey [14:30:49] andrewbogott: there? [14:31:54] Superyetkin: what’s up? [14:32:05] is the DBA back? [14:32:26] no progress on https://phabricator.wikimedia.org/T105713 yet [14:33:17] I think Jaime is still out sick. I’ll see if Sean is around [14:33:29] hmm [15:06:57] andrewbogott: any news? [15:10:06] AzaToth: are you there? [15:14:12] Superyetkin: nothing has changed since yesterday — Sean is sleeping, Jaime is out sick. [15:14:18] I emailed both of them [15:15:34] ok, thanks for you interest [15:30:17] Huji: yea [15:30:52] AzaToth: I need your help to get rolling with Twinkle for a bit; once that happens I will lead the i18nization of Twinkle on my own [15:31:00] We have a bug open for it (I reported in 2012) [15:31:15] I sent you an email earlier today; have you had a chance to look at it? [15:31:28] looking [15:31:40] AzaToth: thanks! [15:34:29] Huji: you don't want to export it from enwiki directly [15:34:44] that's what I think was the source of my problem AzaToth [15:34:58] I just need your help to "generate" the files from Twinkle source code itself [15:35:00] Huji: it's built and managed from https://github.com/azatoth/twinkle [15:35:32] there's a sync script available there, but's it's made by devs for devs [15:36:37] AzaToth: so I tried to run that script and got confused, how can I have it create individual JS and CSS files (so each would be uploaded to the respective MediaWiki namespace page on the wiki)? [15:37:32] it uploads them induvidually [15:38:23] for example twinkleprotect is uploaded to https://en.wikipedia.org/wiki/MediaWiki:Gadget-twinkleprotect.js [15:41:03] AzaToth: ok let me give it one more try on a fresh setting [15:41:36] Huji: the mapping for uploads can be read here: https://github.com/azatoth/twinkle/blob/master/sync.pl#L41-L65 [15:42:03] AzaToth: great! How can I specify the server address for the sync script? [15:42:18] is that .mwbotrc? [15:43:16] How do i properly announce downtime for the IRC server?:) global message as ircoper? [15:43:37] mutante: which IRC server? [15:43:50] AzaToth: irc.wikimedia.org [15:44:07] mutante: aint it pretty much only bots there? [15:44:11] ..and how do i restart the RC bot ?:) [15:44:17] not many live people to notice a message [15:44:23] AzaToth: i dont know, people claim i should announce it [15:44:38] and i remember vaguely how last time [15:44:45] it was a bigger deal than expected [15:44:55] Coren|MX, still trveling? [15:44:55] because the bot didnt come back and anti-vandals rely on it [15:45:11] No, I've arrived. [15:45:13] mutante: how would an announce help a bot? [15:45:16] and it has "pmtpa" in config :) [15:45:30] AzaToth: it doesn't, it's just the 2 things i need [15:46:39] Huji: host => "$opt->{lang}.$opt->{family}.org", [15:46:58] so run it with --lang=fawp --family=wmflabs [15:47:22] oh nice! [15:48:23] was a long time ago I made that script, not the best [15:48:43] mutante: via Tech News? Some bots might need to be restarted, it also affects Huggle users [15:49:35] the biggest reason was to have a link back to a commit in the rev history onwiki: https://en.wikipedia.org/w/index.php?title=MediaWiki:Gadget-twinkleprotect.js&action=history [15:50:12] AzaToth: I'm waiting on MediaWiki::Bot to be installed, hence the delay. It didn't install the first time; I am now using "force install" in cpan. [15:50:26] sitic: what is Tech News? [15:50:37] the thing compiled by guillom? [15:50:40] Huji: using cpanm? [15:50:53] The server is Ubuntu so I am using cpan command lind [15:50:55] line* [15:50:58] mutante: https://meta.wikimedia.org/wiki/Tech/News [15:51:06] AzaToth: is that a bad idea? [15:51:12] yeah [15:52:07] Huji: apt-get install cpanminus; cpanm --sudo install Git::Repository MediaWiki::Bot [15:52:40] cpanminus is much safter/easier to use than cpan directly [15:52:41] sitic: thanks, i guess i'm reading the "publication manual" :p [15:53:37] AzaToth: so it seems like I broke it. Now when I run cpanm it says "MediaWiki::Bot" is latest version. How can I uninstall a module and reinstall it? [15:54:32] mutante: You should probably ping #countervandalism staff before restarting [15:54:39] 6Labs: Don't run block-on-mount block-for-home-export if the volume is already mounted - https://phabricator.wikimedia.org/T105226#1451992 (10Andrew) 5Open>3Resolved resolved by https://gerrit.wikimedia.org/r/#/c/224461/ [15:54:53] Huji: cpanm install --reinstall ---sudo --notest MediaWiki::Bot [15:54:55] but I don't know who manages the antivandal bot at enwiki [15:55:08] can skip sudo if you installed it locally [15:55:15] and then there's wm-bot too [15:55:17] Glaisher: it would be good to know how to restart too :) [15:56:30] and these things turn 5 minutes into 5 days :) [15:56:37] 6Labs, 7Monitoring, 5Patch-For-Review: Monitor nova services - https://phabricator.wikimedia.org/T90784#1451996 (10Andrew) Once we have a read-only nova account, the monitoring can do proper queries. [15:56:45] 6Labs, 10Labs-Infrastructure: Give 'novaobserver' keystone account rights to read everything, everywhere, write or change nothing - https://phabricator.wikimedia.org/T104588#1451999 (10Andrew) [15:56:46] 6Labs, 7Monitoring, 5Patch-For-Review: Monitor nova services - https://phabricator.wikimedia.org/T90784#1451998 (10Andrew) [15:56:47] AzaToth: sorry but this happened: http://pastebin.com/ghy0wyJn [15:57:01] the more you ask the more comments who to announce it to... and the more new questions [15:57:32] Huji: ah, install that from the system: apt-get install libfile-slurp-perl [15:57:45] there might be some unspecified deps in the file [15:58:09] Huji: or install it via cpanm: cpanm install File::Slurp [15:58:33] now it says "Can't locate Getopt/Long/Descriptive.pm" [15:58:44] should I just keep intalling whatever it cannto find? [15:58:58] apt-get install libgetopt-long-descriptive-perl [15:58:59] yea [15:59:11] AzaToth: on it [16:00:19] AzaToth: progress! Now I don't get crash messages; instead it says "Mandatory parameter 'mode' missing in call to (eval)" [16:00:30] you should use only distro packages if you can get away with it. mixing some things from cpan and some from APT might just cause new issues [16:00:54] mutante: yeah I installed everything via apt-get to be safe [16:00:57] cool [16:01:16] mutante: but thanks for the recommendation I'll take a note [16:01:38] Huji: "mode" is the action to perform, pull, push, or deploy [16:01:57] so I should "push" everything now, right? (The wiki is wiped clean of any gadgets) [16:02:16] you want to deploy if you want to use it as a gadget [16:02:32] andrewbogott: I can't ssh to instance "huggle" same problem as with huggle-pg I guess [16:02:38] it's likely fucked up since outage of nfs [16:02:46] petan: ok — I’m in a meeting but will look shortly [16:02:52] ok [16:03:17] Huji: "deploy" _push_es it to where it needs to be as a gadget [16:03:42] Glaisher: what's up with wm-bot [16:03:54] AzaToth: but how do I deploy "all", rather than just one file? I just deployed Twinkle.js only [16:04:06] Huji: generally you use https://github.com/azatoth/twinkle/blob/master/Makefile for deploying all, i.e. "make deploy" [16:04:33] need to change that call to include the lang and family [16:04:39] * YuviPanda|zzz should build a git based gadget deploy system some day [16:04:42] as I said, the scipt isn't perfect ツ [16:04:49] please note that neither huggle nor wm-bot uses irc.wikimedia.org so it's safe to shutdown for those [16:04:49] AzaToth: it's okay, I'm learaning :) [16:04:52] YuviPanda|zzz: ツ [16:05:05] huggle and wm-bot use the RCStream now [16:05:35] petan: I read that as RCS-tream [16:05:41] petan: Puppet seems happy on that box, login works for me. Can you try again? [16:05:46] And if it still fails it might be worth trying a reboot [16:06:17] andrewbogott: it just started working [16:06:23] huh [16:06:27] what you did? [16:06:30] AzaToth: do you have a user database on replicas? [16:06:36] petan: ‘puppet agent -tv' [16:06:40] ok, that fixed it [16:06:41] Superyetkin: nope [16:06:41] which should be happening every 30 in a cron ayway [16:06:49] maybe that cron was disabled somehow? [16:06:53] maybe [16:06:58] maybe it crashed because of nfs? [16:06:59] ok [16:07:08] petan: no idea. But, glad it’s working :) [16:07:43] petan: do you have a user database on replicas? [16:07:44] AzaToth: so should I modify the makefile? or pass paramters to the 'make' command? [16:07:54] modify [16:08:05] Superyetkin: I have no idea what you talk about [16:08:22] are you a tool developer? [16:08:28] i.e. change ./sync.pl --deploy $^ to ./sync.pl --deploy --lang fawp --family wmflabs $^ [16:08:48] you are talking about tools project? yes I have some databases there, but probably only on that dedicated sql server which is not a replica [16:09:03] ok [16:09:13] Superyetkin: are you askin in person or in general? [16:09:16] that machine used to be a regular VM, but now it's no longer available to me (I can't ssh there) so I suppose it was moved somewhere, but it's definitely not replica [16:09:20] I am experinecing a weird issue there [16:09:46] describe them? [16:10:06] can you look at https://phabricator.wikimedia.org/T105713 ? [16:10:27] I am waiting for DBAs but they are not online... [16:10:41] unfortunatelly I have no powers over these dbs I can do nothing [16:10:50] Huji: seems you manage to deploy some [16:10:54] I need to have this fixed as soon as possible [16:10:56] you need to wait for someone who is allowed to login there [16:10:58] it is blocking... [16:11:18] AzaToth: I modified the makefile and the command line output shows that they should all be deployed [16:11:41] AzaToth: but Special:Recentchanges doesn't show them all, and the issue of not seeing "TW" persists [16:12:14] AzaToth: how can I force it to "overwrite" existing pages in MediaWIki namespace? [16:13:11] you can't [16:13:16] if no text has changed [16:13:57] AzaToth: I'm going to delete all those pages then, and run deploy again to see what happens [16:15:04] petan: oh cool :) [16:15:42] what's this [16:15:44] W: molly-guard: SSH session detected! [16:15:45] Please type in hostname of the machine to reboot: [16:15:46] Good thing I asked; I won't reboot huggle ... [16:15:47] W: aborting reboot due to 30-query-hostname exiting with code 1. [16:16:47] petan: https://packages.debian.org/sid/molly-guard [16:16:51] nice thingi ツ [16:17:08] AzaToth: ok, so everything is gone: https://fawp.wmflabs.org/w/index.php?title=Special%3AAllPages&from=&to=&namespace=8 [16:17:18] AzaToth: deploying again, fingers crossed [16:17:22] heh [16:18:04] AzaToth: everythign deployed fine, but still I don't see TW up there :( [16:18:06] AzaToth: it actually happened to me in past I executed shutdown command on remote server while thinking it's my pc, but I use sudo s2disk to shut it down, so nothing happened :P [16:19:06] petan: ツ [16:19:14] AzaToth: and what's really confusing is that no errors are logged in the console either; it's ... just not there! [16:19:43] petan: check out package "molly-guard". we installed it in production recently after exactly this thing happened to us [16:20:28] Huji: I'm getting some Twinkle is not defined [16:20:35] 6Labs, 10Labs-Infrastructure, 10wikitech.wikimedia.org: Remove [?] links from Special:NovaInstance - https://phabricator.wikimedia.org/T105770#1452049 (10scfc) 5Open>3declined a:3scfc Assuming you mean the configuration pages for the individual instances, the "[?]" links work fine for those classes wit... [16:20:45] AzaToth: I'm not. That's weird [16:20:57] AzaToth: note that you ahve also added a subpage for youself called Twinkle.js [16:21:15] or maybe I did [16:21:15] v [16:21:16] https://fawp.wmflabs.org/w/index.php?title=User:AzaToth/twinkle.js&action=history [16:21:30] Why is that there? I don't remember importing it? [16:22:03] Huji: default --push I think still does that [16:22:21] deleted [16:22:32] btw, you are admin too on that wiki so feel free to edit as you see fit [16:22:49] you shouldn't get that "Twinke lnot found" error anymore I suppose? [16:24:19] yea [16:27:59] when will Sean be online back again? [16:28:08] AzaToth: any idea what makes TW not show at all? [16:34:49] Huji: no idea [16:37:07] Huji: some things mught have changed since I last deployed it ヾ [16:37:18] you might want to ask TTO & C:o [16:37:36] AzaToth: so you think I should deploy an older than recent version? [16:38:06] AzaToth: also "TTO" and "C:o" are both unknown references to me? Are they users on Wikipedia/ [16:45:35] Huji: the gadget is running, it's just not getting visual [16:45:52] AzaToth: how does that happen? :p [16:45:54] :o [16:46:13] Huji: tto is https://en.wikipedia.org/wiki/User:This,_that_and_the_other [16:47:00] they've done a lot to the code so they might know better [16:52:50] AzaToth: I will ask [17:03:43] AzaToth: thanks a lot for your help today, I will be in touch. [17:07:18] 6Labs: Should keystone endpoints specify api version? - https://phabricator.wikimedia.org/T102806#1452150 (10Andrew) I've confirmed that in the latest docs, they suggest different url schemes for different services. Keystone: --publicurl http://controller:5000/v2.0 \ --internalurl http://controller:5000/v2... [17:08:25] 6Labs: Should keystone endpoints specify api version? - https://phabricator.wikimedia.org/T102806#1452151 (10Andrew) 5Open>3Resolved And, confirmed that prod.sh is consistent with this as well. [17:14:05] 6Labs: 'virt1' entry at markmonitor? - https://phabricator.wikimedia.org/T102689#1452160 (10Andrew) p:5Triage>3Low [17:15:16] 6Labs: 'virt1' entry at markmonitor? - https://phabricator.wikimedia.org/T102689#1452164 (10Dzahn) markmonitor or catchpoint? since the attached file is called catchpoint.jpg and i was wondering why the domain registrars would have this [17:19:18] 6Labs, 10Continuous-Integration-Infrastructure, 6operations, 7Puppet: Error "Duplicate declaration: File[/etc/ssh/userkeys] is already declared in file /private/modules/passwords/manifests/init.pp:36; cannot redeclare at /modules/ssh/manifests/server.pp:31" - https://phabricator.wikimedia.org/T92752#1452173... [17:19:21] 6Labs, 3Labs-Sprint-106, 7Puppet: puppetmaster::gitsync should update labs/private repository as well - https://phabricator.wikimedia.org/T92756#1452170 (10yuvipanda) 5Open>3Resolved a:3yuvipanda Resolved, me thinks [17:20:08] 6Labs, 10Labs-Infrastructure: Investigate keystone lockups - https://phabricator.wikimedia.org/T104884#1452176 (10Andrew) At least this is not T96256 -- I've just verified that that issue is still good and fixed. [17:20:22] 6Labs, 10Labs-Infrastructure: Investigate keystone lockups - https://phabricator.wikimedia.org/T104884#1452178 (10Andrew) p:5Triage>3Normal [17:21:06] andrewbogott: did you really mean markmonitor .. or catchpoint [17:21:36] mutante: virt1? I really meant markmonitor. We have an entry with our registrar left over from years ago. [17:22:30] 6Labs, 3Labs-Sprint-103, 5Patch-For-Review: decouple role::labs::instance puppet runs from the rest of puppet - https://phabricator.wikimedia.org/T103357#1452187 (10Andrew) p:5Triage>3Low I'm no longer sure this is a great idea... it adds a lot of complexity. Probably a nag/shame/notify system for broke... [17:22:55] andrewbogott: yea, oh. ok. just thought catchpoint because it was the name of the .jpg [17:23:06] so these are the glue records then [17:24:32] yeah [17:25:10] 6Labs, 10Labs-Infrastructure: Give 'novaobserver' keystone account rights to read everything, everywhere, write or change nothing - https://phabricator.wikimedia.org/T104588#1452189 (10Andrew) p:5Triage>3Normal [17:25:54] 6Labs, 10Labs-Infrastructure: Give 'novaobserver' keystone account rights to read everything, everywhere, write or change nothing - https://phabricator.wikimedia.org/T104588#1452191 (10yuvipanda) w00t! [17:36:19] Getting this error: Error: Could not retrieve catalog from remote server: Error 400 on SERVER: This service unit has a systemd script but nothing useful for upstart at /etc/puppet/modules/base/manifests/service_unit.pp:71 on node wdq-beta.wikidata-query.eqiad.wmflabs [17:36:34] does it mean I always have to make two sets of startup scripts? [17:40:04] SMalyshev: if you know you only want systemd I think you can set systemd=> true on the service unit? [17:40:23] There were docs on the base::service_unit description [17:40:27] YuviPanda|zzz: that's what I did [17:40:47] puppet refuses to accept it which I suspect means the host is running upstart and not systemd [17:40:58] but I have no idea why or how I can control it (if at all) [17:41:09] it works on different host though [17:41:13] Ih [17:41:16] Jessie is systemd [17:41:19] Ubuntu is upatart [17:41:24] Upstart [17:41:25] brb [17:41:29] aha... ok [17:41:40] So if you want to support both of them [17:41:43] YuviPanda|zzz: so effectively I have to have both then? [17:42:00] SMalyshev: if you want to support Ubuntu and trusty then yes [17:42:02] But do you? [17:42:13] YuviPanda|zzz: because I alread have hosts running on ubuntu, so I don't want to reinstall everything [17:42:17] If this is for wdqs I presume you would want it deployed in Jessie in the end [17:42:19] Ah I see [17:42:24] also, I have no idea what will be running in production [17:42:26] Yes then you have to write both I'm afraid [17:42:33] I'm pretty sure it will be running Debian [17:42:36] and on deployment-prep [17:42:47] A new services are going to be Debian Jessie [17:42:48] and on any of the other places I have no idea about... [17:43:07] *all [17:43:10] yes, we do want to switch to Jessie [17:43:11] YuviPanda|zzz: ok, good to know. what about deployment-prep, beta and such? [17:43:22] Will all be Jessie yeah [17:43:45] ok... so that's just my bad luck that I've made labs instances on ubuntu... [17:44:12] Heh unfortunately [17:44:14] ok then, will try to make upstart scripts too [17:44:20] I'm boarding a plane and will be gone soon [17:44:55] andrewbogott: is this on irccloud? [17:45:14] If on os x there is a nice app called nembus (I think?) [17:45:17] yep, just from browser so far [17:45:51] I will miss colloquy [17:46:34] andrewbogott: heh :) [17:50:27] YuviPanda|zzz: how do I set an away nick? Or is that not automatic? [17:50:52] andrewbogott: not sure - I always do it manually [17:50:59] 'k [17:56:20] andrewbogott: on irccloud, all you can do is set to go away automatically, no nick changes [17:56:52] JohnFLewis: dang. [17:57:14] get ZNC [17:57:22] it can do both [17:57:45] is ZNC a bouncer service? [17:58:28] andrewbogott: its an open source - install on your local server one [17:58:42] Oh, that I have already [17:59:00] you're using it right now? [17:59:10] IRCCloud is far more dynamic and easier to check up on quickly, the nick thing shouldn't be a deal breaker imho [17:59:14] No, just — until a minute ago I was using dircproxy [17:59:37] JohnFLewis: I agree, but I’ll bug ‘em about it anyway :) [17:59:48] andrewbogott: do it :) [18:04:25] Bot24: hi [18:04:25] Hello! [18:16:52] apparently afk nicks are “a terrible practice" [18:17:54] andrewbogott: I think so [18:18:23] also because my bot has to ping me at this nick [18:23:23] they are [18:23:52] but the problem is that other users dont respect the normal away status [18:23:59] or dont know that it exists [18:24:16] or that they can leave messages [18:25:07] AzaToth: I have made an interesting observation: the function "addPortlet" in Twinkle.js is never called in Twinkle.js and I cannot find any other script calling it. It's the function that show add that TW tab, isn't it? [18:28:07] does puppetmaster role (role::puppet:self) also set up firewall rules so other hosts can use this puppetmaster? [18:34:05] andrewbogott: is there a way to tell Vigrant to install a specific version of MW, rather than the latest dev version? [18:34:10] vagrant* [18:49:05] SMalyshev: you may want to see T104894 [18:54:43] Huji: I haven’t used vagrant in years. bd808 should know [19:59:22] hello [19:59:27] is the DBA back? [20:19:09] is there anyone who can answer questions about replica servers? [20:34:39] 6Labs, 10Labs-Infrastructure: Precise instances say "ImportError: No module named cc_power_state_change" on startup - https://phabricator.wikimedia.org/T103808#1452522 (10Andrew) p:5Normal>3Lowest [20:36:15] hi all, isn't anybody there that could help with a new project request (WikidataLDF)? https://phabricator.wikimedia.org/T105457?workflow=76375 [20:38:24] CristianCantoro: I’ll look [20:38:43] andrewbogott: thanks, a lot! [20:39:20] CristianCantoro: do you already have a labs account and shell access and such? [20:39:43] yep [20:39:59] andrewbogott: ^^ [20:40:14] Will that project need to host a wiki? Or is it a stand-alone web service, or what? [20:40:22] (wondering if you can get this done within toolbar) [20:40:32] bah, toollabs. [20:40:36] * andrewbogott disables spellcheck [20:41:35] andrewbogott: no, wiki hosted. The project is already online at http://wikidataldf.com/ [20:41:36] DBAs? [20:42:14] andrewbogott: the "problem" is that I am now hosting it on a DigitalOcean server and I am paying for it... but I have not the resources to maintain it anymore [20:42:24] so I would like to host it on the Wikimedia-labs [20:42:38] so that anybody else who is interested can help [20:43:00] CristianCantoro: this is the right approach [20:43:06] ‘wiki hosted’? [20:43:46] andrewbogott: yeah, I meant that I do not need to host a wiki... ^_^ [20:44:04] andrewbogott: (it is late here and commas are free to roam in my sentences) [20:44:18] ah, ok. So probably this is something you can do in toollabs so you don’t have to administrate a specific server. [20:44:54] andrewbogott: I have asked some time ago... the problem that was raised was storage [20:45:00] anyone here who can eloborate on replica servers? [20:45:19] andrewbogott: I am now using 32 GB of disk space [20:45:21] ah, so I see. hm. [20:45:46] yeah, a separate project might be necessary. [20:46:10] I have been suggested that NFS storage could be a problem for the application (i.e. NFS is too slow for the needs of the application) and vice versa (the application would make continuous calls to the NFS and stress it). [20:46:27] andrewbogott: I do not recall now with whom I talked [20:46:30] yeah, that’s almost certainly correct. [20:46:56] but it was a couple of months ago, on this channel [20:47:19] may be yuvipanda [20:47:40] Superyetkin: probably :) [20:47:54] he is one of the most active guys here [20:48:24] CristianCantoro: ok, if you visit https://wikitech.wikimedia.org/wiki/Special:NovaInstance and enter ‘wikidata-clf’ in the project filter you should be able to create an instance. [20:48:29] CristianCantoro: are you a Wikidata guy? [20:48:40] Best to think a bit about the firewall policy you want and set that up before creating the server though. [20:48:57] 6Labs, 7Tracking: New project: WikidataLDF - https://phabricator.wikimedia.org/T105457#1452548 (10Andrew) 5Open>3Resolved a:3Andrew Project 'wikidata-ldf' created. [20:48:58] 6Labs, 7Tracking: New Labs project requests (Tracking) - https://phabricator.wikimedia.org/T76375#1452551 (10Andrew) [20:49:56] Oh, which ‘firewall policy’ is called ‘security groups’ in this context. [20:51:31] andrewbogott: ok... what's "default"? [20:51:51] andrewbogott: do you know when Sean will be online? [20:51:56] ‘default’ is added to all new instances. Have a look at the ‘manage service groups’ link to see what the rules are. [20:52:12] ok [20:52:14] Superyetkin: he’s in Australia, where it is currently 6:50 AM. [20:52:34] does he work in the morning? [20:52:35] He’s also part-time, I’m not sure what hours he keeps. [20:52:39] Superyetkin: not too much... I mean, there are many many people that are way more active than I am [20:53:11] CristianCantoro: I am also interested in Wikidata topics [20:53:41] how about Jaime? [20:53:56] is s/he also a part-time worker? [20:54:17] I do not want to start over and recretae a crashed database... [20:54:51] I really wonder what happened to "some" user databases on c2 [20:55:00] andrewbogott: I have created the instance [20:55:02] most are alive, some are not [20:55:34] andrewbogott: it says BUILD (spwaning) [20:56:04] Superyetkin: jaime (male) is sick, as I mentioned before. He works part time, usually, and lives in Spain. [20:56:29] hmm, so all I can do is wait? [20:56:43] my tool is broken for 4 days [20:57:40] Have you never been too sick to work? [20:57:54] no, I have not [20:58:32] even if I have, someone else would have done what I was supposed to do [20:58:59] not being able to reach those guys really hurts [20:59:48] are there no "backup" DBAs out there? [21:00:07] this should not be a complex issue to track [21:05:00] Superyetkin: Yes, sorry, you just have to wait. It’s not usual for so many people to be unavailable, but it’s the week of wikimania /and/ people are sick and on holiday. [21:05:29] andrewbogott: I understand [21:05:52] we cannot always have what we want... [21:06:34] but after a major outage, this is really shocking [21:09:19] andrewbogott: I had to upload a new ssh key since the one that was on the wiki was old (and not available anymore) [21:10:04] CristianCantoro: sure — you’ll want to set up proxycommand if you’re not using that already. https://wikitech.wikimedia.org/wiki/Help:Access#Accessing_instances_with_ProxyCommand_ssh_option_.28recommended.29 [21:10:08] Unless you’re on windows [21:10:46] andrewbogott: I am on Linux... it is some months that I do not use Labs... I have lo learn everything again... :-S [21:11:15] ok :) hopefully the docs are somewhat accurate [21:12:20] andrewbogott: I have successfully logged in on the machine [21:12:22] yay :) [21:12:36] what should I do if I want to give access to other users? [21:13:02] CristianCantoro: you can add other labs users to your project. [21:13:16] Add them as members for ssh access or as admins if you want them to be able to muck with the system setup. [21:13:22] And you can customize sudo rules for either group. [21:14:25] ciao CristianCantoro [21:14:41] ciao Nemo_bis :) [21:17:43] andrewbogott: ok, I think everything is good and clear now... thanks a lot for your help! [21:18:02] sure thing — feel free to ask if you run into any trouble [21:22:58] 6Labs, 10Tool-Labs: missing database on replica server - https://phabricator.wikimedia.org/T105713#1452629 (10coren) It is possible that you might be connecting implicitly to a different server than you were previously - if you use a project name alias (like enwiki.labsdb) it'll get you a connection to /a/ dat... [21:23:57] 6Labs, 10Tool-Labs: missing database on replica server - https://phabricator.wikimedia.org/T105713#1452640 (10coren) Hmmm. On rereading, I note you specify `c2` explicitly? If that is the case, I cannot think of a reason why your DB would be unavailable. Adding our DBAs to the ticket so they can look into it. [21:29:43] 6Labs, 10Tool-Labs: missing database on replica server - https://phabricator.wikimedia.org/T105713#1452677 (10coren) As I note, I //do// see a database by that name, but on `tools.labsdb` (the general purpose database for tools, not one of the replicas) and not on `c2.labsdb` [21:55:46] andrewbogott: I have a question [21:55:55] 6Labs, 10Tool-Labs: missing database on replica server - https://phabricator.wikimedia.org/T105713#1452690 (10Superyetkin) >>! In T105713#1452677, @coren wrote: > As I note, I //do// see a database by that name, but on `tools.labsdb` (the general purpose database for tools, not one of the replicas) and not on... [21:55:58] if I look here: https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikidata-ldf.wikidata-ldf.eqiad.wmflabs [21:56:08] it says that the storage allocated is 40 GB [21:56:15] yep [21:56:35] that space is allocated but probably unpartitioned. [21:56:48] If you want to mount it… there’s a puppet class for that. Let me check the name [21:56:57] ... ok [21:57:09] because if I do df -H from inside the machine [21:57:11] I get this [21:57:12] Filesystem Size Used Avail Use% Mounted on [21:57:12] /dev/vda1 20G 2.1G 16G 12% / [21:57:12] none 4.1k 0 4.1k 0% /sys/fs/cgroup [21:57:12] udev 2.1G 13k 2.1G 1% /dev [21:57:13] tmpfs 415M 377k 415M 1% /run [21:57:13] none 5.3M 0 5.3M 0% /run/lock [21:57:13] none 2.1G 0 2.1G 0% /run/shm [21:57:13] none 105M 0 105M 0% /run/user [21:57:14] role::labs::lvm::mnt or role::labs::lvm::lvm, depending on where you want it to mount. [21:58:06] andrewbogott: why it is saying that my root partition in 20G at the moment? [21:58:43] …because your root partition is 20g. [21:58:53] andrewbogott :D [21:59:05] ok, so this 40 G volume is separated [21:59:07] space is allocated for the instance if you use it, but it isn’t automatically mounted and partitioned. [21:59:09] and I have to mount it [21:59:27] IT’s 40g total. So if you apply one of the above classes the balance (~20G) will be mounted as a separate volume. [21:59:42] k [22:00:25] ok, I can I mount it... any pointer to instrucntions or the like? [22:00:37] * how... [22:02:45] configure the instance, activate either role::labs::lvm::mnt or role::labs::lvm::lvm, depending on where you want it to mount. [22:03:05] then re-run puppet on the instance with ‘sudo puppet agent -tv' [22:06:22] andrewbogott: ok... I am too tired now to read docs... I'll check it out tomorrow [22:06:57] and probably I am going to mess things up if I do it now [22:07:11] andrewbogott: thanks again for your help [22:07:38] good $(time_zone), have a nice day [23:02:31] !log wikistats add missing Persian Wikivoyage (fa) [23:02:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikistats/SAL, Master [23:47:44] !log wikistats built and installed packaged 2.10 [23:47:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikistats/SAL, Master [23:48:09] !log wikistats added more missing projects, pnb.wikt, ca.wm, cn.wm .. [23:48:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikistats/SAL, Master