[00:12:16] <grrrit-wm>	 (03PS1) 10BryanDavis: Add parsley javascript library [labs/striker/staticfiles] - 10https://gerrit.wikimedia.org/r/313152 (https://phabricator.wikimedia.org/T144710) 
[00:25:48] <bd808>	 !log striker Deployed changes from T144710 for testing
[00:25:49] <stashbot>	 T144710: Create Wikitech/LDAP accounts via a new user friendly guided workflow - https://phabricator.wikimedia.org/T144710
[00:25:51] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL, Master
[01:55:25] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/逆襲的天邪鬼 was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=867017 edit summary: 
[03:16:07] <wikibugs>	 10Tool-Labs-tools-Pageviews, 06Community-Tech, 05Goal: Integrate Pageviews Analysis with Page Pile - https://phabricator.wikimedia.org/T134920#2673056 (10kaldari)
[03:17:16] <wikibugs>	 10Tool-Labs-tools-Pageviews, 06Community-Tech, 06translatewiki.net, 13Patch-For-Review: Register pageviews tool with translatewiki.net - https://phabricator.wikimedia.org/T128768#2673059 (10kaldari)
[03:32:35] <wikibugs>	 10Tool-Labs-tools-Pageviews, 06Community-Tech, 10GLAM-Tech: Massviews: bug with pagepile adding extra Wikipedia: on GLAM cultural partnership project pages - https://phabricator.wikimedia.org/T135437#2673128 (10MusikAnimal) I'm actually now routinely fetching siteinfo for other features in Massviews, so it's...
[03:57:17] <bd808>	 !log striker Fixed puppet on striker-deploy03 and striker-uwsgi02; apparently busted by some refactoring that effected /etc/puppet/puppet.conf
[03:57:21] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL, Master
[04:57:20] <shinken-wm>	 RECOVERY - Host tools-secgroup-test-103 is UP: PING OK - Packet loss = 0%, RTA = 0.91 ms
[05:03:18] <shinken-wm>	 PROBLEM - Host tools-secgroup-test-103 is DOWN: CRITICAL - Host Unreachable (10.68.21.22)
[06:46:01] <shinken-wm>	 PROBLEM - Host tools-secgroup-test-102 is DOWN: CRITICAL - Host Unreachable (10.68.21.170)
[06:46:09] <shinken-wm>	 RECOVERY - Host secgroup-lag-102 is UP: PING OK - Packet loss = 0%, RTA = 0.77 ms
[07:45:52] <shinken-wm>	 PROBLEM - Puppet staleness on tools-k8s-master-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0]
[07:46:58] <shinken-wm>	 PROBLEM - Host secgroup-lag-102 is DOWN: CRITICAL - Host Unreachable (10.68.17.218)
[08:53:53] <wikibugs>	 06Labs, 10Labs-Kubernetes, 10Tool-Labs, 07Tracking: Issues with 'webservice' kubernetes backend (tracking) - https://phabricator.wikimedia.org/T139107#2419652 (10Samwilson) I'm not able to get my `ws-search` tool to run under Kubernetes:  > 2016-09-28 08:27:51: (mod_fastcgi.c.2569) unexpected end-of-file (...
[10:23:47] <wikibugs>	 10Quarry: it would be useful to run the same Quarry query conveniently in several database - https://phabricator.wikimedia.org/T95582#2673720 (10Milimetric) The union all approach hits some limits as mysql doesn't let you do that beyond I think like 50 or 100 times or something like that.  @yuvipanda, yes, not p...
[13:03:56] <Dispenser_>	 Is SELECT * FROM interwiki; suppose to be empty?
[14:26:36] <nuria_>	 hello! on deployment-prep project instances
[14:26:37] <nuria_>	 (https://wikitech.wikimedia.org/wiki/Special:Ask/-5B-5BResource-20Type::instance-5D-5D-5B-5BProject::deployment-2Dprep-5D-5D/-3F%3DFQDN-23/-3FInstance-20Type/-3FImage-20Id/-3FPublic-20IP/-3FNumber-20of-20CPUs/-3FRAM-20Size/-3FAmount-20of-20Storage/-3FModification-20date/format%3Dbroadtable/limit%3D20/offset%3D20/sort%3DModification-20date/order%3DDESC/mainla
[14:26:37] <nuria_>	 bel%3DFQDN)
[14:27:04] <nuria_>	 i do not see: deployment-eventlogging03
[14:29:18] <Krenair>	 nuria_, you're only looking at results 21 through 58
[14:30:16] <nuria_>	 Krenair: indeed, there is a "further results" link , thnak you i had not seen it
[14:30:19] <Krenair>	 I'd go Previous and then limit 500
[14:39:28] <wikibugs>	 06Labs, 10Tool-Labs, 07Upstream: Add SSHFP dns records to bastions - https://phabricator.wikimedia.org/T132225#2674240 (10AlexMonk-WMF) So the Designate side of this (the bit that's really required) is being backported to Mitaka in https://review.openstack.org/#/c/374308/
[14:57:32] <bd808>	 Dispenser_: probably. we don't use the interwiki db table in WMF prod. Interwikis come from this file instead -- https://noc.wikimedia.org/conf/highlight.php?file=interwiki.php
[14:59:08] <Dispenser_>	 Thanks.  Hard coding them wasn't such a bad decision then...
[15:00:30] <bd808>	 the prod config file changes for sure, but holding the data in ram with hhvm makes it a lot faster to lookup interwiki things. We used to use a CDB file for similar reasons
[15:02:24] <Dispenser_>	 I was looking at script that converted external links => interwikis and I derived the hard coded patterns from an SQL query and was looking to update
[15:03:04] <Krenair>	 updated to that file don't happen very often
[15:03:37] <Krenair>	 updates*
[16:15:30] <bd808>	 !log striker Removed yuvipanda's cherry pick of https://gerrit.wikimedia.org/r/#/c/257275/4
[16:15:34] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL, Master
[16:35:16] <wikibugs>	 10Striker, 06Community-Tech-Tool-Labs, 13Patch-For-Review, 15User-bd808: Create Wikitech/LDAP accounts via a new user friendly guided workflow - https://phabricator.wikimedia.org/T144710#2674663 (10bd808) Here's what the happy path of adding a new account looks like on https://striker.wmflabs.org/:  {F4533...
[17:02:07] <halfak>	 o/ yuvipanda.  Where do I find a good DB connection example for PAWS?
[17:02:51] <yuvipanda>	 halfak: http://paws-public.wmflabs.org/paws-public/User:YuviPanda/examples/revisions-sql.ipynb or https://paws-public.wmflabs.org/paws-public/User:YuviPanda/replicahelper.ipynb
[17:05:30] <halfak>	 Thanks yuvipanda.  I added them to https://www.mediawiki.org/wiki/PAWS#Database_access
[17:05:38] <yuvipanda>	 \o/ awesome
[17:14:54] <wikibugs>	 06Labs, 10Labs-Infrastructure, 10DBA, 07Upstream: db1069: convert user_groups table to InnoDB across all the wikis - https://phabricator.wikimedia.org/T146121#2674775 (10jcrespo) I think user_groups on s3 failed today:  ``` *************************** 8. row ***************************       Id: 3472432...
[17:15:09] <wikibugs>	 06Labs, 10Labs-Infrastructure, 10DBA, 07Upstream: db1069: convert user_groups table to InnoDB across all the wikis - https://phabricator.wikimedia.org/T146121#2674776 (10jcrespo) p:05Triage>03High
[17:23:14] <wikibugs>	 10PAWS: PAWS can't edit SQL files - https://phabricator.wikimedia.org/T146920#2674793 (10Halfak)
[17:47:58] <halfak>	 Arg.  The pain of running sql queries to get TSV files in PAWS has convinced me to switch back to analytics-store for now
[18:35:05] <wikibugs>	 10Tool-Labs-tools-Pageviews, 06Community-Tech, 10GLAM-Tech: Massviews: bug with pagepile adding extra Wikipedia: on GLAM cultural partnership project pages - https://phabricator.wikimedia.org/T135437#2675034 (10Sadads) Yeah, I think I can hack around this for the time being: use the pagepile to create a repo...
[19:03:27] <wikibugs>	 10Quarry: it would be useful to run the same Quarry query conveniently in several database - https://phabricator.wikimedia.org/T95582#2675100 (10Quiddity) >>! In T95582#2667746, @Milimetric wrote: > @Quiddity, we're very close to allowing this kind of query in Hadoop, which you've worked with before, right?  Gra...
[20:11:52] <wikibugs>	 06Labs, 10Tool-Labs, 07Upstream: Add SSHFP dns records to bastions - https://phabricator.wikimedia.org/T132225#2675305 (10AlexMonk-WMF) Okay I mentioned it during the designate meeting today and it's been approved - that's now on the mitaka stable branch. It'd probably become available through the mitaka pac...
[20:18:00] <wikibugs>	 10Quarry: it would be useful to run the same Quarry query conveniently in several database - https://phabricator.wikimedia.org/T95582#2675345 (10matmarex) >>! In T95582#2673720, @Milimetric wrote: > The union all approach hits some limits as mysql doesn't let you do that beyond I think like 50 or 100 times or so...
[20:45:46] <wikibugs>	 10Tool-Labs-tools-Xtools: WMFLabs tool TOPEDITS brings up blank page - https://phabricator.wikimedia.org/T118138#2675498 (10Matthewrbowker) 05Open>03Resolved Give it a try now.  For the record, Peachy had been disabled for portions of the tool.  I re-enabled it.
[21:29:45] <yuvipanda>	 halfak: can you file a ticket describing the pain?
[21:30:24] <halfak>	 yuvipanda, yeah... Will try to get that documented.  Essentially, I want to write up "things I have on the stat machines that I don't have in PAWS"
[21:30:36] <yuvipanda>	 halfak: +1
[21:30:37] <yuvipanda>	 that sounds useful
[21:42:57] <wikibugs>	 10PAWS: Document reasons to use stat machines rather than PAWS for analysis - https://phabricator.wikimedia.org/T146949#2675759 (10Halfak)
[21:43:24] <wikibugs>	 10PAWS: Document reasons to use stat machines rather than PAWS for analysis - https://phabricator.wikimedia.org/T146949#2675759 (10Halfak)
[22:18:30] <grrrit-wm>	 (03CR) 10BryanDavis: Create LDAP and Striker users from registration form data (032 comments) [labs/striker] - 10https://gerrit.wikimedia.org/r/313143 (https://phabricator.wikimedia.org/T144710) (owner: 10BryanDavis)
[22:32:08] <wikibugs>	 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Track labs instances hanging - https://phabricator.wikimedia.org/T141673#2675873 (10AlexMonk-WMF) T146947 fqdn: deployment-ms-be01.deployment-prep.eqiad.wmflabs virt host: labvirt1010.eqiad.wmnet noticed: 2016-09-28T21:30ish, shinken check for puppet run bro...
[22:33:19] <Krenair>	 !log deployment-prep Rebooting deployment-ms-be01 - T146947, T141673
[22:33:21] <stashbot>	 T146947: Beta cluster database update job is stall, because of Swift? - https://phabricator.wikimedia.org/T146947
[22:33:21] <stashbot>	 T141673: Track labs instances hanging  - https://phabricator.wikimedia.org/T141673
[22:33:23] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL, Master
[23:43:08] <wikibugs>	 10Striker: Check for 2FA protection and enforce validation of 2FA tokens - https://phabricator.wikimedia.org/T144712#2676053 (10bd808) There are multiple possible paths to get this functionality as noted in the description. The best solution would be some internal network service (T120484 or an off-the-shelf sol...