[08:36:01] <grrrit-wm>	 (03PS1) 10Petrb: fixed some names [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111162 
[08:40:47] <petan>	 @labs-instance-info .
[08:40:55] <petan>	 @labs-info .
[08:40:55] <wm-bot>	 I don't know this instance, sorry, try browsing the list by hand, but I can guarantee there is no such instance matching this name, host or Nova ID unless it was created less than 35 seconds ago
[08:41:01] <petan>	 @labs-info catsort
[08:41:01] <wm-bot>	 [Name catsort doesn't exist but resolves to I-000004be.pmtpa.wmflabs] I-000004be.pmtpa.wmflabs is Nova Instance with name: catsort, host: virt5, IP: 10.4.1.11 of type: m1.small, with number of CPUs: 1, RAM of this size: 2048M, member of project: category-sorting, size of storage: 30 and with image ID: ubuntu-12.04-precise
[08:48:01] <hashar>	 petan: good morning :-D
[08:58:24] <petan>	 hi
[08:58:49] <petan>	 @labs-project category-sorting
[08:58:55] <petan>	 @labs-project-members category-sorting
[08:59:00] <petan>	 fuck
[08:59:01] <petan>	 @help
[08:59:01] <wm-bot>	 I am running http://meta.wikimedia.org/wiki/WM-Bot version wikimedia bot v. 1.20.2.1 my source code is licensed under GPL and located at https://github.com/benapetr/wikimedia-bot I will be very happy if you fix my bugs or implement new features
[08:59:34] <petan>	 @labs-project-users category-sorting
[08:59:34] <wm-bot>	 Following users are in this project (showing all 2 members): Liangent, Novaadmin, 
[08:59:55] <petan>	 liangent: are you here? I need to figure out why that catsort instance is down all time
[09:19:26] <petan>	 hashar: how do I ssh to deployment-staging-cache-mobile02.pmtpa.wmflabs
[09:19:31] <petan>	 ssh refuses
[09:19:56] <hashar>	 might be broken
[09:20:00] <hashar>	 na it works for me
[09:20:08] <hashar>	 petan: can you ssh on deployment-bastion.pmtpa.wmflabs ?
[09:20:12] <petan>	 it doesn't work for me as well as for nagios
[09:20:13] <petan>	 yes
[09:20:19] <hashar>	 no idea :(
[09:20:23] <hashar>	 let me run puppet on it
[09:20:28] <hashar>	 it is for mobile 
[09:20:36] <hashar>	 on beta some instances use firewall rules
[09:20:39] <hashar>	 which prevents nrpe access
[09:20:42] <hashar>	 need to fix it up
[09:20:42] <petan>	 ah
[09:20:45] <petan>	 ok
[09:20:56] <petan>	 but I can't ssh there even from bastion
[09:21:07] <petan>	 petrb@deployment-bastion:~$ ssh deployment-staging-cache-mobile02
[09:21:09] <petan>	 waiting forever
[09:21:14] <hashar>	 !log deployment-prep upgrading varnish on deployment-staging-cache-mobile02
[09:22:50] <hashar>	 petan: ran puppet on it
[09:35:53] <petan>	 ssh: connect to host deployment-staging-cache-mobile02 port 22: Connection timed out :o
[09:46:18] <liangent>	 petan: almost abandoned
[09:52:10] <hashar>	 !log deployment-prep upgrading varnish on deployment-cache-text1
[09:52:16] <hashar>	 poor boot
[09:52:18] <hashar>	 poor bot
[09:56:43] <andrewbogott>	 hashar, try again?
[09:57:12] <hashar>	 !log deployment-prep Upgrading all varnish 
[09:57:14] <labs-morebots>	 Logged the message, Master
[09:57:18] <hashar>	 andrewbogott: you are a hero
[09:57:29] <hashar>	 we need a bot to restart labs-morebots <--- petan 
[09:57:52] <andrewbogott>	 we need to use an irc framework that understands about netsplit
[09:58:08] <YuviPanda>	 we need to switch to Facebook Chat
[09:58:25] <hashar>	 !log deployment-prep rebooting all varnish caches
[09:58:27] <labs-morebots>	 Logged the message, Master
[10:00:39] <hashar>	 petan: any clue what is wrong with the puppet fresnesh check on labs ?
[10:02:01] <hashar>	 petan: puppet sends SNMP traps to  nagios-main.pmtpa.wmflabs
[10:07:50] <petan>	 hashar: yes, that is wrong
[10:07:58] <petan>	 it should go to icinga
[10:08:09] <petan>	 however I don't even like the way how puppet checks work now
[10:08:18] <petan>	 there should be post_run command defined
[10:08:23] <petan>	 with proper nrpe check
[10:08:51] <petan>	 I have this on my own icinga
[10:09:03] <petan>	 it checks even how many errors there were during last puppet run etc
[10:15:47] <hashar>	 petan: I guess we can fix the destination as a first step ? 
[10:15:54] <hashar>	 which instance should it point to ?
[10:16:24] <hashar>	 the trap should probably triggered using puppet stages
[12:13:29] <petan>	 @notify hashar
[12:13:29] <wm-bot>	 I'll let you know when I see hashar around here
[12:51:19] <hashar>	 re
[12:55:19] <grrrit-wm>	 (03CR) 10DamianZaremba: [C: 032 V: 032] Fixed lot of stuff in nagios builder [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/110968 (owner: 10Petrb)
[12:56:04] <grrrit-wm>	 (03CR) 10DamianZaremba: [C: 032 V: 032] fixed some names [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111162 (owner: 10Petrb)
[12:59:42] <grrrit-wm>	 (03PS1) 10DamianZaremba: General cleanup [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111182 
[13:00:10] <grrrit-wm>	 (03CR) 10DamianZaremba: [C: 032 V: 032] General cleanup [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111182 (owner: 10DamianZaremba)
[13:00:50] <Damianz>	 petan: I've regiggled it a little, but the functionality should be the same
[13:05:39] <grrrit-wm>	 (03PS1) 10DamianZaremba: Update README [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111183 
[13:05:53] <grrrit-wm>	 (03CR) 10DamianZaremba: [C: 032 V: 032] Update README [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111183 (owner: 10DamianZaremba)
[13:06:33] <YuviPanda>	 Damianz: Zaremba sounds like a really cool name
[13:06:39] <YuviPanda>	 and hehe at 'regiggled' :D
[13:07:04] <Damianz>	 Never really rated names on coolness lol
[13:07:18] <YuviPanda>	 :P
[13:14:42] <grrrit-wm>	 (03PS1) 10DamianZaremba: Fix travis image [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111184 
[13:14:53] <grrrit-wm>	 (03CR) 10DamianZaremba: [C: 032 V: 032] Fix travis image [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111184 (owner: 10DamianZaremba)
[13:22:18] <hashar>	 petan: Damianz: any idea where puppet should send its SNMP traps? 
[13:22:30] <hashar>	 they are currently being sent to  nagios-main.pmtpa.wmflabs
[13:22:40] <Damianz>	 nagios-icinga probably
[13:22:57] <Damianz>	 that hasn't worked properly in ages tbh - review for ops on changing an instance name in that config is sloooooow
[13:26:02] <hashar>	 Damianz: I can handle that part
[13:26:06] <hashar>	 I got high karma with ops :D
[13:26:18] <hashar>	 + it is already broken so its not going to change anything
[13:26:40] <hashar>	 just willing to confirm the nagios instance name, ideally confirming it listen for snmptraps somehow
[13:33:09] <hashar>	 !log deployment-prep removing role::memcached from both apache servers
[13:33:11] <labs-morebots>	 Logged the message, Master
[13:37:14] <petan>	 hashar: yes icinga
[13:37:27] <hashar>	 what is the full instance name ?
[13:37:31] <petan>	 icinga
[13:37:32] <petan>	 :D
[13:37:47] <petan>	 Damianz: I am happy to know I am not the only one who think that waiting for merge takes ages
[13:37:55] <petan>	 Damianz: speaking of which, can you merge my commits lol
[13:38:14] <Damianz>	 Which?
[13:38:20] <petan>	 these 2 I made
[13:38:21] <Damianz>	 The 2 from yesterday I already did
[13:38:22] <petan>	 to builder
[13:38:24] <Damianz>	 :P
[13:38:26] <petan>	 rly?
[13:38:28] <petan>	 :o
[13:38:48] <petan>	 @labs-instance icinga
[13:38:55] <petan>	 @labs-info icinga
[13:38:55] <wm-bot>	 [Name icinga doesn't exist but resolves to I-00000a48.pmtpa.wmflabs] I-00000a48.pmtpa.wmflabs is Nova Instance with name: icinga, host: virt8, IP: 10.4.1.88 of type: m1.medium, with number of CPUs: 2, RAM of this size: 4096M, member of project: nagios, size of storage: 55 and with image ID: ubuntu-12.04-precise
[13:38:59] <petan>	 hashar: that's it
[13:39:22] <petan>	 but I don't know if snmp is configured properly on that instance
[13:39:29] <petan>	 mutante knows how to do that
[13:39:46] <petan>	 I configured it with him on hackaton 2 years ago
[13:39:53] <petan>	 I don't remember how he did it
[13:43:34] <petan>	 !icinga
[13:43:34] <wm-bot>	 https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?host=$1
[13:43:51] <hashar>	 Damianz: petan: also there is no more any host group defined :D
[13:44:00] <petan>	 howcome
[13:44:03] <petan>	 I can see them
[13:44:16] <hashar>	 http://icinga.wmflabs.org/cgi-bin/icinga/status.cgi?hostgroup=deployment-prep&style=detail&scroll=772&servicestatustypes=29&limit=0&start=1
[13:44:25] <hashar>	 should show service alarms for the deployment-prep hostgroup
[13:44:27] <petan>	 oh wait
[13:44:29] <petan>	 that is new
[13:44:33] <petan>	 few hours ago it worked :D
[13:44:40] <Damianz>	 Hmm did I break that
[13:44:48] <petan>	 Damianz: did you touch it? :P
[13:45:01] <Damianz>	 >.>
[13:45:05] <petan>	 hashar: Damianz broke it I am innocent :D
[13:45:05] * Damianz  might have missed a thing

[13:45:14] <Damianz>	 give me a sec just writing an email
[13:45:23] <petan>	 Damianz: https://wikitech.wikimedia.org/wiki/Icinga/Labs
[13:45:27] <petan>	 document things there ^
[13:45:32] <petan>	 so that anyone can ssh there and fix things
[13:48:41] <hashar>	 for icinga traps:  https://gerrit.wikimedia.org/r/111187
[13:49:18] <hashar>	 double check the instance name, if that is fine with you +1 and I will get someone from ops to merge it
[13:50:55] <petan>	 instance name is "icinga"
[13:50:59] <petan>	 without quotes :D
[13:55:01] <petan>	 Damianz: /root/nagios-builder/wmflabs-ignored.host doesn't exist!!
[13:55:18] <Damianz>	 I know
[13:55:20] <petan>	 Damianz: if you fuck up the configuration, there is /backup/etc
[13:55:33] <Damianz>	 It doesn't use that file
[13:55:45] <petan>	 why it doesn't
[13:55:50] <petan>	 it's in crontab
[13:56:14] <Damianz>	 No it's not
[13:56:19] <Damianz>	 and that's not the problem anyway
[13:56:43] <petan>	 petrb@labs-vmbuilder-precise:~$ cat /etc/crontab
[13:56:49] <petan>	 */5 * * * * root cd /root/nagios-builder/labsnagiosbuilder/ && (git reset --hard; git pull origin master; ./build.py --ignored-hosts=/root/nagios-builder/labsnagiosbuilder/wmflabs-ignored.host >> /var/log/nagios.log 2>&1)
[13:59:08] <grrrit-wm>	 (03PS1) 10DamianZaremba: Fix indentation [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111190 
[13:59:34] <grrrit-wm>	 (03CR) 10DamianZaremba: [C: 032 V: 032] Fix indentation [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111190 (owner: 10DamianZaremba)
[14:06:46] <grrrit-wm>	 (03PS1) 10DamianZaremba: Rework cleanup logic [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111193 
[14:06:59] <grrrit-wm>	 (03CR) 10DamianZaremba: [C: 032 V: 032] Rework cleanup logic [labs/nagios-builder] - 10https://gerrit.wikimedia.org/r/111193 (owner: 10DamianZaremba)
[14:07:27] <Damianz>	 hashar: should be fixed
[14:07:47] <hashar>	 and akosiaris merged the change :-D
[14:09:46] <petan>	 it doesn't work
[14:10:01] <petan>	 @notify mutante
[14:10:01] <wm-bot>	 I'll let you know when I see mutante around here
[14:10:24] <Damianz>	 It needs that stupid hook stuff adding
[14:10:32] <petan>	 yes
[14:10:36] <petan>	 I don't know how to do that
[14:10:39] <Damianz>	 Tbh it would be better to delegate the monitoring project to be able to run salt commands then use that to check
[14:10:43] <Damianz>	 Since everything has salt already
[14:11:07] <petan>	 it could as well just use a nrpe check
[14:11:31] <petan>	 I wrote one myself for my servers
[14:11:47] <petan>	 but it would require puppet agent config changes
[14:11:56] <petan>	 and I am using puppet 3.4 dunno if it even works here
[14:13:16] <hashar>	 that works now thanks
[14:13:55] <hashar>	 the trap system, I have no clue how it is configured in icinga :(
[14:15:26] <Damianz>	 nrpe check requires root without changing permissions iirc
[14:15:56] <Damianz>	 Ignoring that really this should be done in a puppet reporter, not a snmp trap
[14:16:06] <Damianz>	 Since we don't know if puppet runs but fails
[14:16:25] <hashar>	 that is what akosiaris was saying earlier
[14:23:38] <andrewbogott>	 Coren, are you around/awake/working?  (No idea what TZ you're in today)
[14:25:34] <hashar>	 he might be flying around
[14:39:47] <petan>	 Damianz: my puppet generates last-run.log file which is readable by everyone, nagios check just test how old is it, and how many errors it contains
[15:01:21] <Betacommand>	 petan: why cant I do sql meta_p
[15:01:46] <Betacommand>	 I want to connect to the informational data?
[15:12:19] <Betacommand>	 petan or Coren anyone able to help me?
[15:32:02] <Betacommand>	 scfc_de: do you know how to connect to meta_p ?
[15:44:51] <Steinsplitter>	 sql metawiki
[15:47:33] <dschwen>	 scfc_de, http://www.sslshopper.com/ssl-checker.html#hostname=fastcci1.wmflabs.org
[15:47:43] <dschwen>	 still shows a warning
[15:54:22] <Betacommand>	 Steinsplitter: metawiki and meta_p are different
[15:54:42] <Betacommand>	 meta_p is the replacement for the toolserver db 
[15:54:54] <Betacommand>	 it contains information about all projects
[15:55:46] <anomie>	 Betacommand: Connect to any of the replica DBs and "use meta_p". Or just reference it in the select query without switching databases.
[15:56:16] <Steinsplitter>	 Betacommand: oh :)
[15:59:00] <Betacommand>	 anomie: thanks
[16:10:24] <scfc_de>	 dschwen: Sorry, I saw the ping too late.  Replied on bug.
[16:10:33] <dschwen>	 yep
[16:10:36] <dschwen>	 just saw it
[16:10:41] <dschwen>	 working on opening a new bug
[16:10:51] <scfc_de>	 Betacommand: ... isn't that available on every shard?
[16:11:03] <dschwen>	 sorry, I was a bit confused. I thought the bug was for instance proxy as well per the discussion on the mailing list
[16:11:11] <Silke_WMDE>	 Hi scfc_de! Is it okay for you if I rename https://bugzilla.wikimedia.org/show_bug.cgi?id=57876 to something like "Provide commonswiki, wikidatawiki and centralauth on all clusters"?
[16:11:43] <Betacommand>	 scfc_de: yes but couldnt figure out how to connect to it sql meta_p didnt work :P
[16:11:55] <Betacommand>	 I forgot about the use trick
[16:12:35] <scfc_de>	 Silke_WMDE: Okay with me; but I think there is a separate bug for centralauth IIRC.
[16:12:48] <Silke_WMDE>	 ok, I'll check
[16:14:06] <scfc_de>	 Silke_WMDE: https://bugzilla.wikimedia.org/show_bug.cgi?id=52559; you could mark it as a duplicate if you want to.
[16:14:17] <Silke_WMDE>	 ok, thanks!
[17:22:02] <hashar>	 !log fixed up beta-parsoid-update job so Parsoid should be up to date again. The issue is that the multigit job pointed to a wrong host (ZUUL_URL should be zuul.eqiad.wmnet)
[17:22:03] <labs-morebots>	 fixed is not a valid project.
[17:22:26] <hashar>	 !log deployment-prep fixed up beta-parsoid-update job so Parsoid should be up to date again. The issue is that the multigit job pointed to a wrong host (ZUUL_URL should be zuul.eqiad.wmnet)
[17:22:27] <labs-morebots>	 Logged the message, Master
[17:55:38] <dschwen>	 are projectstorage partitions easy to resize?
[17:55:54] <dschwen>	 I'm asking because 300G is a bit small for the maps project
[17:59:31] <scfc_de>	 andrewbogott, Coren: ^
[19:53:39] <Amir1>	 hi someone can say what's wrong about this query?
[19:53:41] <Amir1>	 SELECT /*SLOW OK */ CONCAT('* [[:commons:Image:', db1.img_name, ']] = [[:fa:Image:', db2.img_name, ']] (hash: ', db1.img_sha1, ')') AS collision
[19:53:43] <Amir1>	 FROM commonswiki.labsdb.image db1 
[19:53:44] <Amir1>	 JOIN fawiki.labsdb.image db2 ON db1.img_sha1 = db2.img_sha1 WHERE db1.img_sha1 <> '';
[19:54:19] <Amir1>	 Coren: ^
[19:54:24] <Amir1>	 hi by the way :)
[20:03:30] <scfc_de>	 Amir1: Are you trying to join Commons and fawiki?
[20:05:26] <valhallasw>	 commonswiki.labsdb.image / fawiki.labsdb.image looks wrong to me
[20:05:32] <valhallasw>	 ...but he's gone already
[20:10:38] <scfc_de>	 We still have to document how to do cross-wiki JOINs.  Amir1, should you read the backscroll: Try connecting to fawiki.labsdb and then "SELECT ... FROM fawiki_p.image JOIN commonswiki_f_p.page ON ...".
[20:36:23] <grrrit-wm>	 (03PS1) 10Legoktm: Send mediawiki/extension/Thanks to -corefeatures [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/111257 
[20:36:43] <grrrit-wm>	 (03CR) 10Legoktm: [V: 032] Send mediawiki/extension/Thanks to -corefeatures [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/111257 (owner: 10Legoktm)
[20:36:54] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] Send mediawiki/extension/Thanks to -corefeatures [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/111257 (owner: 10Legoktm)
[20:37:08] <YuviPanda>	 legoktm: such self merging :P
[20:37:22] <legoktm>	 :D
[20:38:10] <legoktm>	 !log tools restarting grrrit-wm: 'Send mediawiki/extension/Thanks to -corefeatures' https://gerrit.wikimedia.org/r/111257
[20:38:11] <labs-morebots>	 Logged the message, Master
[21:08:39] <jeremyb>	 scfc_de: idk about when you asked but i can get in now
[21:08:42] <dschwen>	 Hi DispenserAFK, any plans for migrating to labs?
[21:09:03] <jeremyb>	 bot seems to be back since 08:00:12 UTC
[21:09:03] <YuviPanda>	 legoktm: heh, I wanted to say 'much restarting' when you restarted, but forgot
[21:09:19] <scfc_de>	 jeremyb: bots-labs?  Yeah, and wm-bot has returned as well.
[21:09:31] <jeremyb>	 scfc_de: rightr
[21:09:34] <jeremyb>	 right*
[21:10:01] <jeremyb>	 YuviPanda: how is one supposed to know where one type of proxy starts and another ends?
[21:10:07] <YuviPanda>	 jeremyb: hmm?
[21:10:16] <YuviPanda>	 jeremyb: explain?
[21:10:20] <jeremyb>	 YuviPanda: seen labs-l recently?
[21:10:36] <YuviPanda>	 no, I've been on a mailing list vacation :P
[21:10:37] <YuviPanda>	 let me see
[21:11:30] <grrrit-wm>	 (03PS1) 10Gerrit Patch Uploader: Various IRC channel fixes. 1. Change #wmt to #wmt-bots in bot-wmt-delete.php (wmtbot-9) 2. remove private message to user `huh' from bot-wmt-es.php [labs/tools/WMT] - 10https://gerrit.wikimedia.org/r/111319 
[21:11:33] <grrrit-wm>	 (03CR) 10Gerrit Patch Uploader: "This commit was uploaded using the Gerrit Patch Uploader [1]." [labs/tools/WMT] - 10https://gerrit.wikimedia.org/r/111319 (owner: 10Gerrit Patch Uploader)
[21:11:39] <legoktm>	 oh
[21:11:41] <legoktm>	 found it
[21:11:41] <legoktm>	 https://gerrit.wikimedia.org/r/#/c/110830/
[21:12:00] <YuviPanda>	 jeremyb: instance proxy never supported ssl
[21:12:13] <YuviPanda>	 jeremyb: wait
[21:12:16] <YuviPanda>	 jeremyb: that's not instanceproxy
[21:12:44] <jeremyb>	 YuviPanda: right. i said "how is one supposed to know where one type of proxy starts and another ends?" :)
[21:13:01] <YuviPanda>	 jeremyb: Instance proxy has the name 'instance proxy' in the URL :D
[21:13:15] <jeremyb>	 and the newer proxy system?
[21:13:27] <jeremyb>	 anyway, see bug 60833
[21:13:42] <jeremyb>	 looks like nginx. is it puppetized?
[21:13:54] <grrrit-wm>	 (03CR) 10Legoktm: [C: 04-1] "I think we should just ignore the Jenkins-mwext-sync bot like we do L10n-bot rather than the entire repo entirely." [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/110830 (owner: 10Jforrester)
[21:13:57] <YuviPanda>	 jeremyb: newer proxy system is if you add it yourself from the sidebar
[21:14:33] <jeremyb>	 YuviPanda: right. but if it's not my service (e.g. the bug i just linked) how do i know what stack it's using?
[21:14:44] <dschwen>	 yay my bug is getting attention :-D
[21:14:53] <YuviPanda>	 jeremyb: commented :)
[21:15:22] <jeremyb>	 YuviPanda: puppetized?
[21:15:29] <YuviPanda>	 jeremyb: yeah, 'dynamicproxy' module
[21:15:37] <YuviPanda>	 jeremyb: and labsproxy for instance-proxy
[21:15:41] <YuviPanda>	 jeremyb: if you see a URL like http://fab.instance-proxy.wmflabs.org/
[21:15:48] <YuviPanda>	 jeremyb: it is instance-proxy - it has the name instance-proxy in it
[21:15:58] <YuviPanda>	 jeremyb: if you don't, it is either a public IP or a dynamic proxy.
[21:16:09] <YuviPanda>	 jeremyb: if it is your project, you can go to the special page and check if it is a proxy or IP
[21:16:31] <dschwen>	 wait, I was told fastcci1.wmflabs.org is an instance proxy url
[21:16:56] <YuviPanda>	 dschwen: let me check what proxy exactly it is
[21:16:59] <dschwen>	 https://wikitech.wikimedia.org/wiki/Special:NovaProxy
[21:17:12] <dschwen>	 title: Instance Proxy List
[21:17:19] <dschwen>	 that's where I set it up
[21:17:19] <YuviPanda>	 dschwen: gah, that's a stupid title :|
[21:17:27] <dschwen>	 :-p
[21:17:27] <jeremyb>	 hah
[21:17:30] <YuviPanda>	 dschwen: yeah, that's just the dynamic proxy. I should file a bug to have it renamed
[21:17:46] <YuviPanda>	 dschwen: yeah, so that is the star.wmflabs.org certificate. I updated the bug to explicitly refer to it
[21:17:51] <dschwen>	 whats the difference between instance and dynamic proxy
[21:18:02] <YuviPanda>	 dschwen: what is the name of your instance?
[21:18:04] <dschwen>	 (wait, I'll probably regret I asked this)
[21:18:10] <dschwen>	 fastcci-worker1
[21:18:24] <grrrit-wm>	 (03CR) 10John F. Lewis: [C: 032 V: 032] "Merge." [labs/tools/WMT] - 10https://gerrit.wikimedia.org/r/111319 (owner: 10Gerrit Patch Uploader)
[21:18:52] <YuviPanda>	 dschwen: http://fastcci-worker1.instance-proxy.wmflabs.org/ is the instance-proxy url to that particular instance. it is just literally <instance-name>.instance-proxy.wmflabs.org
[21:18:57] <YuviPanda>	 dschwen: you can not customize it
[21:19:23] <YuviPanda>	 dschwen: this was the only proxy available for a while. Then I wrote dynamic proxy, and you could map any domain name to any instance.
[21:20:04] <YuviPanda>	 dschwen: instance-proxy also does not support ssl - the certificate is completely invalid. This is because a SSL certificate for *.instance-proxy.wmflabs.org is too expensive(I am told?), while with dynamicproxy we could just use a *.wmflabs.org certificate
[21:20:07] <jeremyb>	 YuviPanda: so, instance-proxy is heading off to slaughter so i don't have to worry about whether it's working?
[21:20:25] <jeremyb>	 or that
[21:20:29] <YuviPanda>	 jeremyb: yeah, pretty much. andrewbogott_afk emailed labs-l a while ago about this
[21:20:34] <YuviPanda>	 instance-proxy is pretty much dead.
[21:20:38] <dschwen>	 YuviPanda, would it be ok to have multiple domains pointing to the same instance?
[21:20:40] <YuviPanda>	 use dynamicproxy
[21:20:43] <YuviPanda>	 dschwen: yup!
[21:20:47] <dschwen>	 ok
[21:21:02] <YuviPanda>	 dschwen: fwiw, dynamic proxy also gives you SPDY support for free :)
[21:21:10] <dschwen>	 Niiiiice
[21:21:24] <YuviPanda>	 dschwen: :) and websocket support as well
[21:21:25] <dschwen>	 I just read about that today and wondered how to implement it on my backend
[21:21:40] <YuviPanda>	 dschwen: :D You don't have to worry about it. it is just transparently taken care of for you
[21:21:52] <dschwen>	 that's great
[21:22:03] <YuviPanda>	 dschwen: it supports a slightly older version of SPDY (SPDY/2, rather than /3). Not too many differences though
[21:23:19] <dschwen>	 I was reading http://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft3-1
[21:23:29] <dschwen>	 since FF supports that now
[21:23:35] <dschwen>	 but 2 is fine too
[21:24:01] <YuviPanda>	 dschwen: yeah :)
[21:24:23] <YuviPanda>	 dschwen: do try it out and provide feedback. Some day in the glorious future we can have SPDY on wikipedia
[21:26:45] <scfc_de>	 YuviPanda: Is the proxy as currently running managed by manifests/role/labsproxy.pp?
[21:27:07] <YuviPanda>	 scfc_de: yeah, think so
[21:27:24] <jeremyb>	 YuviPanda: ugh. you also have to move the cert out of /etc/ssl/certs
[21:28:03] <YuviPanda>	 jeremyb: ?
[21:28:27] <YuviPanda>	 scfc_de: yeah, role::dynamicproxy
[21:29:12] <jeremyb>	 YuviPanda: no WMF cert should ever be in /etc/ssl/certs on any box. (unless you trust it to sign certs (as a CA) for any client / server on that box)
[21:29:49] <YuviPanda>	 jeremyb: hmm, Ryan_Lane set this up, I think?
[21:29:51] <YuviPanda>	 or andrewbogott_afk, not sure
[21:30:12] <jeremyb>	 YuviPanda: doesn't matter who...
[21:30:24] <YuviPanda>	 jeremyb: file a bug, maybe?
[21:30:47] <YuviPanda>	 jeremyb: I do know that nobody other than a few of us (all NDA'd + employees) have access to that box
[21:31:06] <scfc_de>	 jeremyb: That's the standard way at WMF; cf. manifests/certs.pp's install_certificate
[21:31:24] <YuviPanda>	 jeremyb: I'll also note that the actual star.wmflabs.org cert is in /etc/ssl/private
[21:31:32] <YuviPanda>	 jeremyb: which you need root to do anything with
[21:31:46] <scfc_de>	 YuviPanda: I think jeremyb is concerned that CAs in /etc/ssl/certs are automatically trusted.
[21:32:25] <jeremyb>	 YuviPanda: what's in private? cert or key?
[21:32:55] <YuviPanda>	 jeremyb: both, from what I see
[21:33:03] <jeremyb>	 hrmmm, k, weird
[21:33:20] <YuviPanda>	 either way, if you think it is insecure, do file a bug.
[21:33:45] <YuviPanda>	 I will admit to not knowing enough about these yet, and will leave those to other more knowledgeable people
[21:35:53] <scfc_de>	 YuviPanda: Does dynamicproxy have a /etc/ssl/certs/star.wmflabs.org.chained.pem (or something similar)?
[21:36:05] <YuviPanda>	 scfc_de: nope.
[21:36:07] <YuviPanda>	 scfc_de: pm?
[21:37:09] <jeremyb>	 YuviPanda: the subject of RT 6398 seemed relevant at first. but seems that's just host certs not service certs
[22:30:25] <PiRSquared>	 How can I move from /home/* to /data/project/* ?
[22:30:28] <PiRSquared>	 a file
[22:30:33] <PiRSquared>	 or copy
[22:32:18] <PiRSquared>	 ...?
[22:32:23] <PiRSquared>	 meh, I'll just copy
[22:32:26] <PiRSquared>	 and paste
[22:33:11] <PiRSquared>	 nope...
[22:51:28] <PiRSquared>	 Please
[22:51:37] <PiRSquared>	 YuviPanda: ping
[22:51:51] <PiRSquared>	 How can I move from my home to a project?
[23:01:02] <scfc_de>	 PiRSquared: You should be able to copy as your user account from your home directory to the directory of any tool you are a maintainer.  Afterwards, you can "become (yourtoolsname)" and "take (files)" to change the ownership to the tool account.
[23:16:57] * Coren  is home for ~36h

[23:17:08] <Coren>	 No explody bits while I was gone?
[23:24:10] <Damianz>	 Nothing major I don't think
[23:24:32] <Damianz>	 Though I was at FOSDEM for the past few days so am just going off none of my stuff being broken :D
[23:24:58] <Coren>	 Well, that's where I was too; so we probably are working from the same dataset.  :-)
[23:31:37] <jeremyb>	 Coren: what are the prospects on puppetizing tools? e.g. the ssl bit that was fixed manually last night
[23:32:50] <jeremyb>	 paravoid: /etc/ssl/certs seems to still be a common location for non-CA cert. is there anything tracking that yet? or plans to change it?
[23:32:52] <Coren>	 jeremyb: The current apache proxy is currently hearing the wail of the banshee; there's very little point in puppetizing it.  On the other hand, I believe that Yuvi has gotten the certificate woes handled right in the puppet classes for the new proxy.
[23:32:57] <jeremyb>	 certs*
[23:35:42] <jeremyb>	 Coren: i don't think i'm familiar with that wail. also, don't know what you mean by Yuvi. (https://gerrit.wikimedia.org/r/111342 is tim)
[23:36:54] <Coren>	 YuviPanda <-- is Yuvi; he's the one who worked on the new proxy system we will switch to in eqiad.
[23:38:01] <jeremyb>	 Coren: right, i know who he is. but what does "handled" mean?
[23:38:44] <Coren>	 Banshees, in some Celtic traditions, are spirits whose wails you ear just before you die; that was just a poetic way of saying tha tools-webproxy will not move to eqiad as it is obsolete.
[23:38:44] <jeremyb>	 > (https://gerrit.wikimedia.org/r/111342 is tim)
[23:39:12] <Coren>	 And Yuvi's new proxy system already has the intermediate certificate setup done right.
[23:39:16] <Coren>	 (In puppet)
[23:39:22] <jeremyb>	 no. no, it doesn't
[23:39:33] <jeremyb>	 have it right
[23:39:34] <Coren>	 Oh?  I was under the impression it did.
[23:39:46] <jeremyb>	 click the link i just pasted twice :)
[23:40:07] <jeremyb>	 so, tools.wmflabs.org will stop using apache ssl and switch to yuvi's method?
[23:40:17] <Coren>	 jeremyb: Yep.
[23:40:19] <jeremyb>	 cool
[23:40:37] <Coren>	 jeremyb: Ah, I hadn't noticed that the patch existed, but was not yet merged.
[23:40:54] <jeremyb>	 Coren: :)
[23:41:13] <Coren>	 The point remains that it's pointless to puppetize tools-webproxy since it's going to be decomissioned soon. :-)
[23:41:20] <jeremyb>	 ok, fine :)
[23:41:43] <jeremyb>	 i also didn't bother testing instance-proxy. and then someone said it already doesn't even have the right cert (name mismatch)
[23:41:52] <jeremyb>	 old is old
[23:42:26] <Damianz>	 I hope we can move soon tbh - lots of things, for a very long time have been 'after the move;
[23:42:31] <Damianz>	 s/;/'/
[23:42:45] <jeremyb>	 who's working on the move?
[23:44:44] <jeremyb>	 i see RT 6099 / 6158
[23:45:08] <jeremyb>	 6158 is rather ummmm, brief
[23:45:10] <jeremyb>	 :P
[23:45:44] <scfc_de>	 jeremyb: https://wikitech.wikimedia.org/wiki/Labs_Eqiad_Migration might be of interest.
[23:45:52] <jeremyb>	 *click*
[23:47:29] <jeremyb>	 scfc_de: linked from 6158 and watchlisted
[23:50:19] <jeremyb>	 Coren: anyway, welcome home to EST5EDT :)
[23:50:24] <jeremyb>	 where next?
[23:54:02] <Coren>	 Palo Alto for a couple days, then the SF office for a few more.
[23:55:01] <jeremyb>	 ah, going to visit [[Moffett Field]] and domas? :)
[23:55:19] <scfc_de>	 Coren: Speaking of https://gerrit.wikimedia.org/r/#/c/111342/, could you +2?
[23:55:58] <jeremyb>	 scfc_de: would be nice for someone to look in that file on the box and see if it has the right content
[23:56:02] <jeremyb>	 (i didn't +1)
[23:56:16] <scfc_de>	 I asked YuviPanda before.
[23:56:32] <Coren>	 jeremyb: Not a chance; FaceBook will keep be busy from 6AM to 8PM pretty much all the time I am there.  :-)
[23:56:39] <scfc_de>	 (Worst case: We'll revert.)
[23:56:43] * Coren  wishes he could visit the PARC though.

[23:56:54] <jeremyb>	 Coren: so, domas is not a chance??? :)
[23:57:03] <jeremyb>	 Coren: also, wtf, 6am?
[23:57:27] <jeremyb>	 scfc_de: well you took it to pm so how do i know what you asked :)
[23:58:25] * jeremyb  runs away

[23:58:42] <scfc_de>	 jeremyb: Because I just told you so :-).