[01:19:45] <mutante>	 cant ssh to my instance - permission denied (publickey)
[01:20:02] <mutante>	 have my regular labs key loaded as always. related to the message in topic?
[01:21:55] <mutante>	 i also still get to bastion-restricted
[01:24:22] <mutante>	 i can still login with my root key but not as regular user
[02:50:59] <grrrit-wm>	 (03PS1) 10Greg Grossmeier: Add #Differential for -releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/235962 
[03:04:08] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] Add #Differential for -releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/235962 (owner: 10Greg Grossmeier)
[03:04:32] <grrrit-wm>	 (03Merged) 10jenkins-bot: Add #Differential for -releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/235962 (owner: 10Greg Grossmeier)
[03:12:59] <wikibugs>	 !log tools.wikibugs Updated channels.yaml to: d18a53d6498e5faa77a03b35261f9d26fd51766d Add #Differential for -releng
[03:13:02] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL, Master
[05:16:50] <shinken-wm>	 PROBLEM - Puppet failure on tools-webgrid-lighttpd-1408 is CRITICAL 33.33% of data above the critical threshold [0.0]
[05:56:51] <shinken-wm>	 RECOVERY - Puppet failure on tools-webgrid-lighttpd-1408 is OK Less than 1.00% above the threshold [0.0]
[07:21:18] <addshore>	 YuviPanda: stoppwaffles
[07:21:23] <addshore>	 stroooopp!!
[09:22:28] <wikibugs>	 10MediaWiki-extensions-OpenStackManager, 10MediaWiki-Authentication-and-authorization, 6Reading-Infrastructure-Team: Update OpenStackManager to use AuthManager - https://phabricator.wikimedia.org/T110288#1605769 (10Liuxinyu970226)
[09:27:16] <shinken-wm>	 PROBLEM - Puppet failure on tools-webgrid-lighttpd-1402 is CRITICAL 71.43% of data above the critical threshold [0.0]
[10:36:04] <wikibugs>	 6Labs, 10Tool-Labs, 7Database: Potential templatelinks data integrity issue on Tool Labs' enwiki_p - https://phabricator.wikimedia.org/T111371#1606123 (10jcrespo) a:3jcrespo
[10:36:36] <wikibugs>	 6Labs, 10Tool-Labs, 7Database: Potential templatelinks data integrity issue on Tool Labs' enwiki_p - https://phabricator.wikimedia.org/T111371#1602483 (10jcrespo) p:5Triage>3Normal
[10:48:13] <wikibugs>	 6Labs, 10Tool-Labs, 7Database: Potential templatelinks data integrity issue on Tool Labs' enwiki_p - https://phabricator.wikimedia.org/T111371#1606151 (10jcrespo) I checked the checksums, and this table was checked/fixed on 10th August. The fact that this happened again, but only on db1069 and bellow means t...
[11:29:53] <wikibugs>	 6Labs, 10Tool-Labs: access.log files are not being updated - https://phabricator.wikimedia.org/T110861#1606218 (10scfc) 5Open>3Resolved a:3valhallasw
[11:47:51] <shinken-wm>	 PROBLEM - Puppet failure on tools-webgrid-lighttpd-1408 is CRITICAL 28.57% of data above the critical threshold [0.0]
[12:22:49] <shinken-wm>	 RECOVERY - Puppet failure on tools-webgrid-lighttpd-1408 is OK Less than 1.00% above the threshold [0.0]
[13:46:40] <wikibugs>	 10Tool-Labs-tools-Other, 6Community-Tech: Fix broken Edit Count tool on labs - https://phabricator.wikimedia.org/T111512#1606395 (10Aklapper)
[14:52:11] <wikibugs>	 6Labs, 10Tool-Labs, 7Database: Potential templatelinks data integrity issue on Tool Labs' enwiki_p - https://phabricator.wikimedia.org/T111371#1606563 (10jcrespo) This should be fixed now, but I am performing another checksum just to be sure.
[15:07:39] <Cyberpower678>	 anomie, hi
[15:08:23] <anomie>	 Cyberpower678: hi
[15:08:35] <Cyberpower678>	 anomie, I got your email and responded
[15:09:04] <anomie>	 Cyberpower678: I have no idea then
[15:09:12] * Cyberpower678 is also trying to clean up his Gmail.  He is apparently using 107% of allowed storage space. :/
[15:09:30] <Cyberpower678>	 It looks like a bug in the file generator.
[15:10:29] * gifti thinks it will not be easy to "fix" that
[15:10:54] * Cyberpower678 thinks that this fix is a must
[15:11:05] <gifti>	 good luck then ^^
[15:11:13] <Cyberpower678>	 I depend on those logs for bot maintanence.
[15:11:15] <anomie>	 All I can say is that I don't have a problem with AnomieBOT's log files
[15:11:21] <gifti>	 i worked around that "bug" for years
[15:11:35] * Cyberpower678 checks other logs.
[15:13:46] <Cyberpower678>	 The problem seems to be cropping in my newest task so far.
[15:14:08] <Cyberpower678>	 But it's running perfectly fine on my local machine.
[15:14:15] <Cyberpower678>	 With no obscure output.
[15:21:55] <Le0n>	 Hi, the Xtools is inactive, anyone here can restart he?
[15:24:29] <Cyberpower678>	 anomie, btw can you wrap up Cyberbot II 6 on wikipedia?
[15:24:40] <anomie>	 Cyberpower678: Not right this moment
[15:24:46] <Cyberpower678>	 aww
[15:25:07] <Cyberpower678>	 addshore, perhaps?
[15:25:18] <addshore>	 ?
[15:25:23] <Cyberpower678>	 Cyberbot II 6?
[15:25:41] <addshore>	 what about it? :O
[15:26:30] <Cyberpower678>	 addshore, approve it?
[15:26:40] <addshore>	 link? and I will look in a bit!
[15:27:24] <Cyberpower678>	 https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/Cyberbot_II_6
[15:27:28] <Cyberpower678>	 addshore, ^
[16:02:56] <mutante>	 still can't ssh to my instance with my regular user
[16:03:06] <mutante>	 only if i use the root access
[16:03:31] <valhallasw`cloud>	 mutante: maybe check /var/log/auth.log?
[16:04:08] <mutante>	 Invalid user dzahn
[16:04:53] <mutante>	 id: dzahn: no such user
[16:05:05] <mutante>	  /home/dzahn still exists
[16:05:38] <mutante>	 so why is my user deleted?
[16:05:54] <valhallasw`cloud>	 ldap having a bad day? not sure
[16:06:52] <valhallasw`cloud>	 given my lack of ldap skills, I'd just reboot the host ;-)
[16:06:55] <mutante>	 it's getting a bit frustrating.. debugging every time i just want to use my existing stuff
[16:08:35] <mutante>	 LDAP still knows me just fine
[16:08:43] <mutante>	 from that instance itself or elsewhere
[16:14:17] <mutante>	 valhallasw`cloud: the fix was "restart nslcd" (per Yuvi)
[16:15:09] <valhallasw`cloud>	 mutante: ah, right. I always forget what the ldap daemon is called
[16:15:17] <YuviPanda>	 Yeah
[16:15:43] <YuviPanda>	 There is nscd which is different 
[17:50:27] <grrrit-wm>	 (03PS1) 10Bgerstle: add notifs for #wikimedia-ios room [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/236050 
[18:02:14] <shinken-wm>	 RECOVERY - Puppet failure on tools-webgrid-lighttpd-1402 is OK Less than 1.00% above the threshold [0.0]
[18:20:25] <grrrit-wm>	 (03CR) 10Legoktm: [C: 04-1] "As I asked earlier, please document the channel's existence and purpose at https://meta.wikimedia.org/wiki/IRC/Channels#General_developmen" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/236050 (owner: 10Bgerstle)
[18:55:13] <wikibugs>	 10Tool-Labs-tools-Other, 6Commons, 6Community-Tech, 6Multimedia: [AOI] Create a new DerivativeFX after the Toolserver shutdown - https://phabricator.wikimedia.org/T110409#1609922 (10Jdforrester-WMF) p:5Triage>3Low
[19:40:13] <Cyberpower678>	 anomie, remind where I can find usage graphs of everything about labs?
[19:40:31] <anomie>	 Cyberpower678: I would if I knew.
[19:40:40] <Cyberpower678>	 Heh. :p
[19:41:03] * Cyberpower678 makes Coren magically feel better so he can answer that question.
[19:41:10] <Cyberpower678>	 YuviPanda, ^
[19:41:57] <grrrit-wm>	 (03CR) 10Bgerstle: "@legoktm, done! sorry it wasn't there already" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/236050 (owner: 10Bgerstle)
[19:44:21] <gifti>	 um
[19:49:14] <Cyberpower678>	 petan, can you tell me where the usage graphs of labs are?
[19:52:49] <YuviPanda>	 Not actually here 
[19:52:50] <YuviPanda>	 But 
[19:52:59] <YuviPanda>	 tools.wmflabs.org/nagf
[19:54:43] <Cyberpower678>	 YuviPanda, yay.  Thank you.
[20:54:00] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] add notifs for #wikimedia-ios room [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/236050 (owner: 10Bgerstle)
[20:54:18] <grrrit-wm>	 (03Merged) 10jenkins-bot: add notifs for #wikimedia-ios room [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/236050 (owner: 10Bgerstle)
[20:56:36] <wikibugs>	 !log tools.wikibugs Updated channels.yaml to: 0a914ec9da79d3e0b8a6dcbe825a2ac54eb03446 add notifs for #wikimedia-ios room
[20:56:40] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL, Master
[21:34:52] <wikibugs>	 6Labs, 6operations, 5Patch-For-Review: audit labs versus production ssh keys - https://phabricator.wikimedia.org/T108078#1610715 (10Krenair) @RobH: Any reason for this to remain open?
[21:35:33] <wikibugs>	 6Labs, 6operations, 5Patch-For-Review: audit labs versus production ssh keys - https://phabricator.wikimedia.org/T108078#1610722 (10RobH) 5Open>3Resolved Nope!  both sub-tasks resolved, resolving task.
[21:42:16] <wikibugs>	 6Labs, 6operations, 10wikitech.wikimedia.org: Determine whether wikitech should really depend on production search cluster - https://phabricator.wikimedia.org/T110987#1610756 (10chasemp) p:5Triage>3Normal iirc we have done other things to island off wikitech since it is the defacto place we check for inf...
[21:48:50] <shinken-wm>	 PROBLEM - Puppet failure on tools-webgrid-lighttpd-1408 is CRITICAL 60.00% of data above the critical threshold [0.0]
[22:23:49] <shinken-wm>	 RECOVERY - Puppet failure on tools-webgrid-lighttpd-1408 is OK Less than 1.00% above the threshold [0.0]
[22:37:42] <mutante>	 my instance says its image type is: (missing)  as opposed to debian-8.1-jessie or ubuntu-14.04-trusty or something
[22:38:27] <mutante>	 it actually is jessie
[22:38:47] <mutante>	 another instance i made with also jessie has the correct type
[22:42:35] <shinken-wm>	 PROBLEM - Free space - all mounts on tools-exec-1206 is CRITICAL tools.tools-exec-1206.diskspace.root.byte_percentfree (<33.33%)
[23:29:10] <YuviPanda>	 mutante: please file a bug
[23:29:11] <YuviPanda>	 thanks
[23:29:13] * YuviPanda goes back to bed
[23:31:55] <mutante>	 sorry for waking you up
[23:34:19] <mutante>	 (i dont know how i did)