[03:46:01] <wikibugs>	 10Tool-Labs: Reduce amount of Tools-local packages - https://phabricator.wikimedia.org/T91874#1311610 (10scfc) `python-celery-with-redis` is a "bundle" that "installs the following packages: ['celery>=3.0,<4.0', 'redis>=2.4.4']".  However, if those specific versions were indeed wanted, they wouldn't be fulfilled...
[06:27:18] <shinken-wm>	 PROBLEM - Puppet failure on tools-exec-1211 is CRITICAL 20.00% of data above the critical threshold [0.0]
[06:32:04] <shinken-wm>	 PROBLEM - Puppet failure on tools-webgrid-lighttpd-1410 is CRITICAL 55.56% of data above the critical threshold [0.0]
[06:57:18] <shinken-wm>	 RECOVERY - Puppet failure on tools-exec-1211 is OK Less than 1.00% above the threshold [0.0]
[07:02:02] <shinken-wm>	 RECOVERY - Puppet failure on tools-webgrid-lighttpd-1410 is OK Less than 1.00% above the threshold [0.0]
[07:04:18] <wikibugs>	 10Tool-Labs, 10Wikimedia-Hackathon-2015: Tool-labs meeting agenda for Lyon Hackathon - https://phabricator.wikimedia.org/T98912#1311868 (10Qgil) I guess this task can be resolved, or do you want to add something?
[10:55:46] <wikibugs>	 6Labs, 5Patch-For-Review: Get Labs openstack service dbs on a proper db server - https://phabricator.wikimedia.org/T92693#1312180 (10jcrespo) @Andrew When you have time, I want to perform this other pending failover, plus ask you some questions regarding backup needs.
[12:27:58] <Glaisher>	 Labs DB replicas have different indexes from production DBs?
[12:29:12] <Glaisher>	 I'm getting ERROR 1176 (42000): Key 'cat_title' doesn't exist in table 'category' but it should be there.
[12:30:12] <jynus>	 Glaisher, in some cases, indexes vary for performance reasons
[12:30:32] <Glaisher>	 hm
[12:30:34] <jynus>	 but be aware that labs users do not have access to the real tables
[12:30:48] <jynus>	 can you provide some context?
[12:31:24] <Glaisher>	 MariaDB [sourceswiki_p]> SELECT cat_title,cat_pages FROM `category` FORCE INDEX (cat_title) WHERE (cat_pages > 0) AND (cat_title>='User_ru') ORDER BY
[12:31:25] <Glaisher>	  cat_title LIMIT 1;
[12:31:26] <Glaisher>	 ERROR 1176 (42000): Key 'cat_title' doesn't exist in table 'category'
[12:31:29] <Glaisher>	 jynus: ^
[12:31:43] <jynus>	 yep, it is the _p
[12:31:58] <jynus>	 views do not support indexes
[12:32:33] <Glaisher>	 I think I'll just remove the FORCE INDEX then.
[12:32:55] <Glaisher>	 was directly copying from a real query that's done on wiki
[12:32:56] <jynus>	 yes, cat_title should be used automatically with no problem
[12:33:33] <Glaisher>	 Yeah, thanks.
[15:44:20] <shinken-wm>	 PROBLEM - Puppet failure on tools-webgrid-lighttpd-1401 is CRITICAL 37.50% of data above the critical threshold [0.0]
[16:09:20] <shinken-wm>	 RECOVERY - Puppet failure on tools-webgrid-lighttpd-1401 is OK Less than 1.00% above the threshold [0.0]
[17:03:15] <grrrit-wm>	 (03PS1) 10Sitic: Safe & read user preferences from local storage [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/213850 (https://phabricator.wikimedia.org/T100342) 
[17:03:30] <grrrit-wm>	 (03CR) 10Sitic: [C: 032 V: 032] Safe & read user preferences from local storage [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/213850 (https://phabricator.wikimedia.org/T100342) (owner: 10Sitic)
[17:20:39] <wikibugs>	 10Tool-Labs, 5Patch-For-Review: Reduce amount of Tools-local packages - https://phabricator.wikimedia.org/T91874#1312871 (10scfc)
[19:36:27] <wikibugs>	 10Tool-Labs, 5Patch-For-Review: Reduce amount of Tools-local packages - https://phabricator.wikimedia.org/T91874#1313068 (10scfc)
[23:50:18] <wikibugs>	 10Tool-Labs, 5Patch-For-Review: Reduce amount of Tools-local packages - https://phabricator.wikimedia.org/T91874#1313510 (10scfc) Ha!  I got confused by my own clean-up: `python-socketio-client` depends on `python-websocket-client` which depends on `python-backports.ssl-match-hostname`.  So one more package fo...
[23:50:56] <jgage>	 hi. i just created a new project with three instances running 12.04, 14.04, and 8.0. in all of them, my homedir is owned by root:root. known issue?