[00:06:00] Warning: There is 1 user waiting for shell: Kyzz (waiting 67 minutes) [00:19:35] Warning: There is 1 user waiting for shell: Kyzz (waiting 81 minutes) [00:33:06] Warning: There is 1 user waiting for shell: Kyzz (waiting 94 minutes) [00:46:36] Warning: There is 1 user waiting for shell: Kyzz (waiting 108 minutes) [01:00:11] Warning: There is 1 user waiting for shell: Kyzz (waiting 122 minutes) [01:11:18] I know what I want to replace gluster with now: http://docs.mongodb.org/manual/core/gridfs/ [01:11:26] we just need to write a fuse driver for it [01:11:46] oh [01:11:47] wait! [01:11:48] https://github.com/mikejs/gridfs-fuse [01:11:49] \o/ [01:13:41] Warning: There is 1 user waiting for shell: Kyzz (waiting 135 minutes) [01:23:15] Ryan_Lane: I do hope you are attempting some sort of humor. :-) [01:23:33] shut down NFS and switch to this immediately! [01:23:47] Coren: ;) [01:23:54] I'm thinking... no. [01:24:03] but it's mongo. it's FAST [01:24:38] I think I may try to upgrade to folsom release next week [01:25:13] Any expected breaking changes? [01:25:34] I just did it in labs on nova-precise2 [01:25:42] nope. it should be mostly transparent [01:25:43] Oooo. Hitch-less? [01:25:51] kind of [01:25:58] need to remake some config files [01:26:05] What could go wrong?™ [01:26:08] heh [01:26:15] well, worst case we have a full outage [01:26:54] the changes aren't massive between essex and folsom, though [01:26:59] so I don't forsee much issues [01:27:15] Warning: There is 1 user waiting for shell: Kyzz (waiting 149 minutes) [01:40:32] Ryan_Lane: On a more serious matter, we need to revisit the way service groups are managed. [01:40:46] Warning: There is 1 user waiting for shell: Kyzz (waiting 162 minutes) [01:41:02] Ryan_Lane: It gets unwieldy /really/ fast, and I've only got a tiny fraction of what will be there eventually. [01:42:13] how so? [01:42:27] the way it's listed on the page? [01:48:38] Ryan_Lane: That long column will only get worse, it's hard to find one's groups, and if you operate on one you get to reload the whole list and have to hunt it back down again. [01:54:20] Warning: There is 1 user waiting for shell: Kyzz (waiting 176 minutes) [02:07:51] Warning: There is 1 user waiting for shell: Kyzz (waiting 189 minutes) [02:21:21] Warning: There is 1 user waiting for shell: Kyzz (waiting 203 minutes) [02:34:51] Warning: There is 1 user waiting for shell: Kyzz (waiting 216 minutes) [02:48:21] Warning: There is 1 user waiting for shell: Kyzz (waiting 230 minutes) [03:01:47] Warning: There is 1 user waiting for shell: Kyzz (waiting 243 minutes) [03:15:17] Warning: There is 1 user waiting for shell: Kyzz (waiting 257 minutes) [03:28:48] Warning: There is 1 user waiting for shell: Kyzz (waiting 270 minutes) [03:36:45] Ryan_Lane: how do you test changes OpenStackManager changes? Is there a labs instance? [03:39:07] ori-l: There is a test wikitech [03:42:56] ah [03:43:26] at what URL? [03:49:28] ori-l: Predictably enough, https://wikitech-test.wmflabs.org/wiki/Main_Page [03:49:29] :-) [04:06:18] ori-l: create an account and I'll add you to a project [04:06:33] warning: I just upgrade openstack there to the folsom release [04:06:37] *upgraded [04:08:10] Ryan_Lane: 'There was either an authentication database error or you are not allowed to update your external account. ' [04:08:26] on account creation. I tried with username 'TEST Ori.livneh' and shell account 'testori' [04:10:09] . may be a bad character? [04:10:15] hm [04:10:18] it shouldn't be [04:10:34] I'll try 'TEST Ori' / 'testori' [04:11:07] yes, that worked [04:11:11] ah. ok [04:11:25] I want our new account creation javascript interface :) [04:12:39] :) I actually haven't been super involved in it except for providing reviews, but because I was the only dev on E3 for a while people think I'm responsible for everything [04:12:57] heh [04:13:09] so naturally I correct them only if it's criticism :D [04:13:17] hahaha [04:13:27] ok. added you as projectadmin on testing project [04:13:32] there's some instances there [04:13:40] they are all in the error state, naturally [04:13:50] since this instance doesn't have enough memory to launch instances [04:13:57] we'll move this to a larger node eventually :D [04:14:27] do you also need shell access to nova-precise2 to test any code changes? [04:14:44] sure, if it's alright to live-hack the JS a bit [04:15:15] yep. I have a branch that's live with the stuff that's currently in gerrit [04:15:20] you can either make a new branch or use that one [04:15:38] ok, cool [04:16:08] I wonder why Ori.livneh didn't work for you [04:16:44] ok, you're in the openstack project now [04:17:04] it has bitten me before in weird ways. my gerrit / labs shell account is 'ori' only because gerrit decided to truncate the period and everything after it, not by choice [04:17:17] hahaha [04:17:26] I'm using /srv/org/wikimedia/controller/wikis/w [04:17:40] we have some other wikis set up on there, but they may not be up to date [04:18:07] we really need to set up a better dev environment, but this hasn't annoyed us too much yet :D [04:18:17] seems ok to me [04:26:58] HTML5 added data-* attributes, meaning all element attributes prefixed with 'data-*' are permitted, and jQuery gives you a nice syntax for getting and setting these values [04:27:27] so for example if you had [04:27:54] var name = $('a').data('instance-name'); [04:29:02] though not everyone likes them, and grabbing the data from the URL is fine too [06:39:20] Ryan_Lane: not sure if addshore poked you already, but can we get a public ip for our new stats project? [06:46:29] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Needed Toolserver features was modified, changed by 70.59.16.167 link https://www.mediawiki.org/w/index.php?diff=678746 edit summary: [+270] /* Bots and webservices project */ cpan upfresh [07:05:58] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 0 minutes) [07:19:28] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 13 minutes) [07:28:34] !log tools petrb: installing GE tools to -dev so that we can develop new j|q* stuff there [07:28:37] Logged the message, Master [07:32:49] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 26 minutes) [07:40:42] also if someone wants to delete the spam… https://wikitech.wikimedia.org/wiki/User_talk:Jamesleo2149 [07:46:15] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 40 minutes) [07:57:55] !log tools petrb: added tools-dev to admin host list so that qtop works and fixing the bug of qtop [07:57:58] Logged the message, Master [07:59:45] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 53 minutes) [08:00:29] !log tools petrb: patching qtop [08:00:30] Logged the message, Master [08:13:16] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 67 minutes) [08:26:51] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 80 minutes) [08:40:16] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 94 minutes) [08:53:47] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 107 minutes) [09:07:20] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 121 minutes) [09:20:54] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 135 minutes) [09:34:25] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 148 minutes) [09:48:00] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 162 minutes) [10:01:30] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 175 minutes) [10:15:04] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 189 minutes) [10:28:35] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 202 minutes) [10:42:09] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 216 minutes) [10:55:34] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 229 minutes) [11:09:08] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 243 minutes) [11:22:34] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 256 minutes) [11:36:09] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 270 minutes) [11:49:39] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 283 minutes) [12:03:05] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 297 minutes) [12:16:40] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 310 minutes) [12:30:10] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 324 minutes) [12:43:46] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 337 minutes) [12:57:16] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 351 minutes) [13:01:37] ㅓ [13:02:20] hi devunt [13:03:13] hi [13:10:51] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 364 minutes) [13:17:59] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Needed Toolserver features was modified, changed by MPelletier (WMF) link https://www.mediawiki.org/w/index.php?diff=678902 edit summary: [+421] /* Bots and webservices project */ re [13:24:25] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 378 minutes) [13:31:38] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Needed Toolserver features was modified, changed by MPelletier (WMF) link https://www.mediawiki.org/w/index.php?diff=678964 edit summary: [+1085] /* Filesystem / shared storage */ updates [13:33:28] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Needed Toolserver features was modified, changed by MPelletier (WMF) link https://www.mediawiki.org/w/index.php?diff=678976 edit summary: [+283] /* Logs/Stats */ mo' updates [13:35:10] Krinkle|detached: Ping me? [13:37:55] Warning: There is 1 user waiting for shell: Jamesleo2149 (waiting 392 minutes) [13:38:24] Coren: You're an admin on wikitech right? that request that keeps pinging is a spambot [13:38:46] legoktm: I know. Apparently, even deleting the request doesn't shut the bot up. [13:39:17] * Coren doesn't know what, exactly, the bot looks for. [13:39:27] >.< [13:39:30] petan: ^ [13:39:40] hi [13:39:44] legoktm what u need? [13:39:57] the bot keeps pinging about a request that was deleted [13:40:01] (Deletion log); 13:23 . . Coren (Talk | contribs) deleted page Shell Request/Jamesleo2149 ‎(spambot) [13:40:07] oh yes [13:40:10] deleting is evil [13:40:15] let me undelete it [13:40:24] o_O? [13:40:32] it's sort of bug [13:40:46] when u delete the page, bot can't verify if it was processed or not [13:41:09] petan: If we delete the page, then the bot should presume that we want it to stfu about it? :-) [13:41:27] it tried to download the content of page until it see it's processed and if it fail to download it, it just skip because for example wikitech could be broken or server was down or whatever [13:41:43] I know there could be more elegant check and you guys are definitely welcome to submit a patch [13:41:44] Type @commands for list of commands. This bot is running http://meta.wikimedia.org/wiki/WM-Bot version wikimedia bot v. 1.10.6.8 source code licensed under GPL and located at https://github.com/benapetr/wikimedia-bot [13:41:48] here is source ^ [13:42:28] Deleted requests are misformed or spam; don't skip, just remove it. :-) If there was a problem reaching wikitech, the bot will just notice it coming back later. [13:43:44] Coren if there was problem reaching wikitech, it will throw some exception just as if it was unable to read the deleted page... of course it can be handled in a different way [13:44:08] which means writing some more code [13:44:17] that is not in balance with lazy petan [13:44:31] @requests [13:44:31] There are no shell requests waiting [13:47:57] Coren: I meant the bot see no difference between page being deleted or wikitech server being down - nobody wrote a code which check if page is deleted :P [13:48:20] in both cases the response is somewhat unknown to it [13:50:04] (15:39:16) >>>>>>Coren doesn't know what, exactly, the bot looks for. <<<<<< the bot is looking for completed=no or completed=false in a source code of page, as long as that the source code can be retrieved and text is present it is considered not processed (as you can read from source code of bot) [14:52:43] Coren: So I set ~/public_html/pywikibot as 777, but http://tools.wmflabs.org/legobot/pywikibot/ still doesn't have an index. Do you know what I'm not doing right? [14:59:01] legoktm: I'll check in a second, but please /please/ never ever set anything 777. :-) [14:59:13] meh. [15:00:24] changed to 774 [15:00:50] legoktm that pretty much suck [15:00:56] 775 is what you want [15:01:31] legoktm, how are you fairing with Pywikipediabot @ [15:01:33] 2 [15:01:36] <^demon> Cool kids `chmod -R 0777 /` [15:01:42] Cyberpower678: Very [15:01:42] Oh [15:01:45] <^demon> Then you never have permission problems ever ;-) [15:02:08] legoktm: There is no index in that directory. :-) [15:02:18] Yeah, I just want an autogenerated listing [15:02:25] ^demon I found a thread on unix exchange like "some hacker did chmod 777 -R / on my box, how can I revert that?" once, was fun [15:02:40] Autogenerated are disabled. [15:02:53] Why? [15:03:11] legoktm because if they weren't things would be too easy :P [15:03:22] we want it to be MOAR complicated [15:04:21] I gotta bbl actually, class time [15:05:06] addshore I just compiled some program I made: 721 warnings 0 errors :D [15:07:37] Because they are too error-prone in case of mistaken permissions, and they're generally a very bad idea. You can turn it on in your .htaccess if you need them. [15:10:16] Coren why bad idea? what is a security exploit on being able to see a directory structure of open source software? [15:10:28] all websites on labs must be open source [15:15:30] petan: You are presuming there are no configuration files with credentials, or non-public logs. [15:16:43] OH so you want to tell me that having a file that is secret but publicly available and is protected just by disabling the index is a proper way of protection? [15:16:53] such a file shouldn't be accessible at all [15:17:05] what if someone knew its name? or guessed it? [15:17:49] config file with credentials shouldn't be a plain text file that can be rendered just by entering its name to url [15:17:56] Coren: Ok so how would I turn them on ni my .htaccess? [15:17:59] in* [15:18:11] legoktm: "Option Indexes" [15:18:49] And I would save that file in my ~/public_html? [15:19:00] In the actual directory where you want the indexes. [15:19:20] Can I have it be recursive? [15:19:44] I really just want indexes everywhere [15:19:48] It is, by default. If you want it for everything, then yes. [15:19:53] Ok great [15:20:37] now http://tools.wmflabs.org/legobot/pywikibot/ is an "Internal Sever Error" [15:20:57] that string didn't need quotes right? [15:21:34] Oh, sorry, my error. "Options Indexes" [15:21:40] Coren: do we have an ETA for replication? [15:22:13] Betacommand: By Amsterdam remains the objective. It's my current WIP now that Gluster is out. [15:22:40] Coren: when is Amersterdam? [15:22:50] Coren: great thanks [15:23:07] Coren is there gluster remaining anywhere except -db? [15:23:10] Betacommand: https://www.mediawiki.org/wiki/Amsterdam_Hackathon_2013 [15:23:22] May 24-26 [15:23:54] !hackaton is https://www.mediawiki.org/wiki/Amsterdam_Hackathon_2013 May 24-26 [15:23:54] Key was added [15:24:16] petan: Not supposed to; and I should -db away from gluster tonight - the only reason I didn't move it during the maintenance is because that doesn't disrupt the tools so I could reduce the set of hosts to change. [15:24:41] ok [15:25:32] btw is gluster being used by some notable project / system? [15:25:42] I am wondering if it suck everywhere or just here [15:26:07] I like its logo [15:26:09] :D [15:26:13] it's a bug [15:26:24] or ant [15:26:25] no idea [15:27:21] petan: Not that I know of, but Ryan would be more aware than I. [15:28:09] well, it is a bug project, sponsored by redhat [15:28:14] I would expect it to work [15:28:20] * big project [15:28:22] not bug :D [15:33:43] NASA uses it [15:34:32] Well now we know why they can't get to the moon :/ [15:36:43] <^demon> No, that's just because we keep cutting NASAs budget. [15:38:16] legoktm XD [15:38:52] so that's why Obama canceled the program... gluster! [15:40:56] Coren did you discuss with Ryan any future alternative or are you going to stick with nfs? [15:41:16] actually gluster /seems/ like a good idea, just if it was all fixed and worked fine... [15:41:47] petan: Not yet; NFS is probably not a final solution since it won't scale well past a certain point - but we'll do *hard* testing before we deploy anything else. [15:42:44] Coren what do you mean by "won't scale" [15:42:49] you mean the size of volumes? [15:43:09] I thought that is depending on filesystem used on backend [15:43:31] petan: What? No, I mean on usage. There is just the one server. It's big, but not infinitely so. :-) [15:43:37] ah [15:43:51] true [15:55:29] hashar, role::beta::logging::mediawiki breaks deployment-bastion: err: Failed to apply catalog: Could not find dependency File[/usr/lib/ganglia/python_modules] for File[/usr/lib/ganglia/python_modules/udp_stats.py] at /etc/puppet/manifests/misc/monitoring.pp:42 [15:56:14] !log deployment-prep Enabled GeoData cronjobs [15:56:16] Logged the message, Master [15:58:39] MaxSem: ooops [15:58:45] MaxSem: yet another ops change I guess [15:58:56] that is on -bastion isn't it ? [15:59:01] yep [16:01:56] ah [16:02:36] hmm, it has an implicit dependency on ganglia [16:03:04] yeah and gmond is not defined apparently [16:04:01] MaxSem: sorry can't really handle that right now [16:04:06] maybe ops can help [16:04:19] I though we had gmond installed on labs [16:04:23] but maybe that is not handled via puppet [16:05:36] ahh [16:06:09] manifests/gangla.pp has an exception, whenever it is on labs there is no gmond service apparently [16:06:50] MaxSem: I will try to remember about it on monday. Filling a bug for now [16:08:11] MaxSem: https://bugzilla.wikimedia.org/show_bug.cgi?id=47738 :-D [16:08:12] [bz] (NEW - created by: Antoine "hashar" Musso, priority: Unprioritized - normal) [Bug 47738] role::beta::logging::mediawiki breaks on deployment-bastion - https://bugzilla.wikimedia.org/show_bug.cgi?id=47738 [16:19:59] MaxSem: that was mark migrating ganglia :-) [16:21:08] heh [16:28:06] I am off [16:28:10] have a nice weekend [16:30:23] hi JMarkOckerbloom [16:37:54] btw JMarkOckerbloom I think you & I both post at Making Light [16:50:35] andrewbogott: hey! [16:50:45] 'morning [16:50:53] would you be able to give mine and addshore's "stats" project a public ip? [16:51:12] sure. One minute... [16:51:18] :D awesome [16:54:04] legoktm, should be all set. [16:54:41] awesome. any chance you would know how set up the domain name? or what we need to do for that…? [16:54:48] we wanted to use stats.wmflabs.org [16:55:41] Should be easy, just visit https://wikitech.wikimedia.org/wiki/Special:NovaAddress [17:11:31] !log stats assigned the IP to the instance and made the hostname stats.wmflabs.org [17:11:32] I think. [17:11:33] Logged the message, Master [17:13:07] andrewbogott: Do we have docs somewhere on how to tweak those quotas? [17:13:22] I don't know -- possibly not. [17:13:55] sumanah: I believe you're right. [17:15:45] I signed on looking for MPelletier but he seems to not be on at the moment. I've sent him email instead. [17:15:55] JMarkOckerbloom: Thats Coren. [17:16:01] JMarkOckerbloom: That is I. :-) [17:16:24] Oh, okay. Coren, I've sent you email. Have to go back to my conference now, Thanks! [17:59:58] Ryan_Lane, I'm having a puppet cert problem with new instances today -- something we've seen before but I don't think it was diagnosed. [18:00:06] oh? [18:00:06] for example mwreview-dev3 [18:00:21] That's a new instance that I created an hour or so ago. [18:00:50] Can you tell what's happening? [18:02:11] andrewbogott: no server line [18:02:32] hm [18:02:41] in puppet.conf? [18:05:03] why are all the nodes failing to access home directories? [18:05:39] that doesn't make any sense [18:05:43] Ryan_Lane, you mean on mwreview-dev3? Or is this a separate issue? [18:06:04] well [18:06:08] an instance in testlabs [18:06:12] and on that system [18:06:35] Doesn't a puppet run need to complete before it can mount gluster vols? [18:07:00] hm [18:07:07] no [18:07:44] are all new instances like this? [18:07:59] I've tried four this morning, all failed in what looked like the same way [18:08:05] all in that project. [18:08:52] I wonder if there's a new version of puppet that gets installed or something... [18:09:08] I feel like we've seen this issue intermittently in the past. [18:09:13] Otto mentioned it [18:09:29] I was thinking it had to do with puppet congestion, but now i'm not so sure. [18:09:31] I'm not really sure how it's possible [18:09:41] let me check the image to make sure the puppet conf is ok [18:10:40] yeah. the image is fine [18:10:52] I wonder how that conf file is getting overridden [18:12:45] it looks like the postinstall is running fine [18:13:06] it must be that a newer version of puppet is somehow getting installed and is overwriting the puppet conf file [18:13:54] nope.... [18:13:59] dpkg.conf doesn't say that [18:14:24] ooohhh [18:14:29] the first puppet run is fucking this up [18:14:55] hm. or maybe not [18:15:19] Apr 26 16:45:15 mwreview-dev3 puppet-agent[1116]: Could not retrieve catalog from remote server: Error 400 on SERVER: Must pass gmond_port to Class[Ganglia_new::Monitor::Config] at /etc/puppet/modules/ganglia_new/manifests/monitor/config.pp:1 on node i-000006e7.pmtpa.wmflabs [18:15:44] ganglia-monitor isn't on the new image, so there are always ganglia failures [18:15:48] although that one doesn't look familiar [18:15:56] let me push in a new image [18:16:00] the new one has ganglia [18:16:16] Does it have apparmor too? That's the other thing that I notice often. [18:16:22] oh [18:16:23] right [18:16:34] let me add that [18:16:52] OK… it's easy to update the image, right? Otherwise, it's easy enough to add apparmor after the fact... [18:17:06] easy enoguh [18:17:09] just takes some time [18:20:21] Warning: There is 1 user waiting for shell: Mrcaps (waiting 0 minutes) [18:20:39] anything else missing in the package, while I'm at it? [18:20:45] err [18:20:47] the image [18:20:57] Not that i've noticed [18:22:03] ok building a new precise image [18:22:12] I need to puppetize this [18:22:21] apparently vmbuilder is deprecated too [18:22:31] so I guess I'll need to find a new method to do this as well [18:22:40] veewee or boxgrinder, I guess [18:29:54] ok. I know what needs to be added [18:30:00] the stupid icinga user [18:33:55] Warning: There is 1 user waiting for shell: Mrcaps (waiting 13 minutes) [18:38:44] good evening (for me in germany ;-) ) [18:38:59] I can't connect to bots-gs [18:38:59] hey [18:39:04] howcome [18:39:50] I tryed 5 times or so but it always raises "Permission denied (publickey)." [18:40:07] ok let me check [18:40:12] are you forwarding your key? [18:40:16] !forwarding [18:40:16] https://labsconsole.wikimedia.org/wiki/Access#Using_agent_forwarding [18:40:26] legoktm@bots-gs:~$ echo "Hi" [18:40:26] Hi [18:40:55] legoktm :D [18:41:54] legoktm: how is this new stats project different from analytics? [18:42:07] LOL [18:42:10] rschen7754|away [18:42:19] rschen7754|away don't run your bot directly on bots-gs [18:43:04] Ryan_Lane: I'm not sure what analytics is for, but addshore and I are basically going to set up a stathat clone [18:47:27] Warning: There is 1 user waiting for shell: Mrcaps (waiting 27 minutes) [18:48:46] petan: I think so. I am using putty, because that I can't do the steps described on the page, but in the Connection > SSH > Auth menu this is selected [18:49:21] Pyfisch can u try another box? [18:49:30] @labs-user Pyfisch [18:49:30] Pyfisch is member of 3 projects: Bastion, Bots, Tools, [18:49:32] yesterday I could access without problems [18:49:35] try tools-login [18:49:40] or tools-dev [18:49:42] legoktm: stathat? [18:49:49] stathat.com [18:50:11] you can set up new graphs in like 15 seconds [18:50:24] except its non free and has tracking code so its not optimal to use [18:50:26] what will this be used for? [18:50:42] petan: same error [18:50:44] you know our privacy policy disallows tracking, right? [18:51:05] it's non-free? [18:51:10] Pyfisch in that case you have problem with your connection [18:51:16] that's also not allowed by our terms of use :) [18:51:22] Ryan_Lane they want to create open source alternative of that service [18:51:32] exactly :D [18:51:36] petan: what to do? [18:51:46] !access | Pyfisch [18:51:46] Pyfisch: https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [18:52:17] Pyfisch I recommend you just create a local ssh key on bastion but u can also set up forwarding [18:52:21] Ryan_Lane: right now we have various tracking graphs of wikidata stats that we can't/don't want to link in interface messages because of the tracking. [18:52:29] it's both secure, just the later is harder [18:53:40] we have graphs tracking the dispatch lag, other ones tracking how fast bots are editing, how fast we're creating items, etc [18:54:09] this would be for production? [18:54:37] you guys really need to talk to the analytics folks :) [18:54:58] Extension:EventLogging [18:55:00] ori-l: ^^ [18:55:08] not really, this is just for us to track random things [18:55:14] and milimetric is helping us set it up right now :) [18:56:37] yeah... I can't figure out why the apache on that instance isn't serving publicly legoktm [18:56:55] does anyone here speak apache config? [18:56:56] :( [18:57:06] do we need to restart it after updating the config? [18:57:20] i restarted it [18:57:30] legoktm: sounds very cool [18:57:44] same problem, "curl localhost:80" works from the box, "curl stats.wmflabs.org" doesn't work from outside [18:57:55] milimetric: check the security groups in labsconsole [18:58:18]