[00:01:27] <wikibugs_>	 10Tool-Labs-tools-Xtools, 06Community-Tech: Output data for new XTools: Top edits - https://phabricator.wikimedia.org/T160139#3089968 (10kaldari)
[00:02:13] <wikibugs_>	 10Tool-Labs-tools-Xtools, 06Community-Tech: Output data for new XTools: Top edits - https://phabricator.wikimedia.org/T160139#3089968 (10kaldari)
[00:28:35] <wikibugs_>	 10Tool-Labs-tools-Xtools, 03Community-Tech-Sprint: Output data for new XTools: Top edits - https://phabricator.wikimedia.org/T160139#3089968 (10kaldari)
[07:45:45] <wikibugs_>	 06Labs, 10Gerrit, 06Operations, 06Release-Engineering-Team, 07LDAP: Remove user gerrit2 from ldap - https://phabricator.wikimedia.org/T160122#3090315 (10MoritzMuehlenhoff) @demon Can you confirm that the user is no longer needed?
[12:41:19] <wikibugs_>	 (03PS1) 10Lokal Profil: [WIP]Build fill_table_monuments_all from per dataset dicts. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/342198
[12:42:39] <wikibugs_>	 (03CR) 10jerkins-bot: [V: 04-1] [WIP]Build fill_table_monuments_all from per dataset dicts. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/342198 (owner: 10Lokal Profil)
[12:51:04] <wikibugs_>	 (03PS2) 10Lokal Profil: [WIP]Build fill_table_monuments_all from per dataset dicts. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/342198
[13:09:55] <wikibugs_>	 (03CR) 10Lokal Profil: "Primarilly ment a a proof-of-concept to get feedback from." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/342198 (owner: 10Lokal Profil)
[13:16:46] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Calexit was created, changed by Calexit link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Calexit edit summary: Created page with "{{Tools Access Request |Justification=I've been trying to use mediawiki-extension (https://github.com/redwerks/mediawiki-extension-command) which requires the extensionservice..."
[13:17:55] <wikibugs_>	 (03CR) 10Lokal Profil: [C: 04-1] "Looks good. Just minor superficial change" (033 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/338007 (owner: 10Jean-Frédéric)
[13:23:05] <Zppix>	 legoktm:  thank you!
[13:25:52] <wikibugs_>	 10Tool-Labs-tools-Zppixbot, 10Wikibugs, 10QuarryBot-enwiki, 13Patch-For-Review: Wikibugs doesnt send phab updates to ##Zppix-Wikipedia - https://phabricator.wikimedia.org/T159612#3090873 (10Zppix) 05Open>03Resolved a:03Zppix
[13:26:54] <wikibugs_>	 (03PS1) 10Lokal Profil: Minor cosmetic changes [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/342203
[13:34:25] <wikibugs_>	 (03CR) 10Lokal Profil: [C: 04-1] "Looks good. Two minor issues." (032 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/338008 (owner: 10Jean-Frédéric)
[13:35:24] <wikibugs_>	 (03CR) 10Lokal Profil: [C: 04-1] "Minus one to get on your radar (see previous comment)." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/338009 (owner: 10Jean-Frédéric)
[14:09:52] <wikibugs_>	 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Reassign service/pod IP ranges for kubernetes on tool labs - https://phabricator.wikimedia.org/T152399#3090960 (10chasemp)
[14:20:10] <wikibugs_>	 06Labs, 10MediaWiki-User-login-and-signup, 10wikitech.wikimedia.org: Error when attempting to login to Wikitech - https://phabricator.wikimedia.org/T160171#3090987 (10Zppix)
[14:22:25] <wikibugs_>	 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Reassign service/pod IP ranges for kubernetes on tool labs - https://phabricator.wikimedia.org/T152399#3091005 (10chasemp) Agreed, this works as long as assignments line up in such a way that it is an extension of the service range from Kubernetes' perspective.  That...
[14:36:30] <wikibugs_>	 (03CR) 10Jean-Frédéric: Minor cosmetic changes (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/342203 (owner: 10Lokal Profil)
[15:25:51] <halfak>	 Woops!  Looks like I missed the notice about postgres going down.  I think that might be the postgres that we use for wiki labels. 
[15:28:59] <halfak>	 It seems that the DB we use is behind the IP: 10.64.37.8
[15:30:20] <wikibugs_>	 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3091178 (10Andrew)
[15:30:46] <wikibugs_>	 06Labs, 10WM-Bot: Move wm-bot instance to Trusty - https://phabricator.wikimedia.org/T157838#3091180 (10Andrew)
[15:30:53] <wikibugs_>	 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#2964654 (10Andrew)
[15:30:58] <halfak>	 o/ yuvipanda 
[15:30:59] <halfak>	 Around?
[15:32:47] <andrewbogott>	 petan: I'm here again, if you are :)
[15:35:42] <halfak>	 andrewbogott, can you help me figure out if the postgres DBs that are going down for maintenance include the one used by wikilabels?  I have an IP address for our psql server. (10.64.37.8)
[15:36:55] <andrewbogott>	 halfak: I don't know much, but I'll see what I can find
[15:37:06] <halfak>	 Thanks!
[15:37:28] <halfak>	 I see a database with my username on it, so you'd think i would've gotten pinged. 
[15:37:33] <halfak>	 :)  But I just want to be sure. 
[15:38:41] <andrewbogott>	 that IP is labsdb1004, which is not listed in the email announcement and which is already running Jessie
[15:38:47] <andrewbogott>	 So at first glance it looks like you won't be affected
[15:38:48] <halfak>	 \o/ 
[15:38:56] <andrewbogott>	 unless those boxes are coupled somehow
[15:39:41] * andrewbogott surprised that postgres is running in multiple places
[15:40:14] <halfak>	 A long time ago, yuvipanda helped us get an account on that DB to use it as a centralized store for wikilabels. 
[15:41:51] <halfak>	 Anyway, thanks for your help andrewbogott. 
[15:41:57] <andrewbogott>	 yep!
[16:00:26] <wikibugs_>	 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Deprecate precise instances in Labs by 2017-03-31 - https://phabricator.wikimedia.org/T143349#3091233 (10Ricordisamoa)
[17:05:00] <wikibugs_>	 06Labs, 10Gerrit, 06Operations, 06Release-Engineering-Team, 07LDAP: Remove user gerrit2 from ldap - https://phabricator.wikimedia.org/T160122#3091314 (10demon) >>! In T160122#3090315, @MoritzMuehlenhoff wrote: > @demon Can you confirm that the user is no longer needed?  For a little history, it **used**...
[17:27:18] <andrewbogott>	 halfak: 	ores-web-05 and ores-worker-05 are in some weird in-between state where Nova thinks they've been rebooting for two weeks… is it OK if I force them to reboot, for real, now?
[17:27:31] <halfak>	 yes
[17:27:37] <halfak>	 Thanks for noticing that
[17:29:49] <andrewbogott>	 halfak: ok, done
[17:30:02] <andrewbogott>	 (I was hoping that would clear up another issue I'm seeing but, nope.)
[17:30:30] <halfak>	 bummer
[17:30:57] <wikibugs_>	 06Labs, 10Gerrit, 06Operations, 06Release-Engineering-Team, 07LDAP: Remove user gerrit2 from ldap - https://phabricator.wikimedia.org/T160122#3091387 (10MoritzMuehlenhoff) Ok, I'll make the change on Tuesday when you're around (since Monday is a holiday for US staff)
[17:42:44] <wikibugs_>	 06Labs, 10Gerrit, 06Operations, 06Release-Engineering-Team, 07LDAP: Remove user gerrit2 from ldap - https://phabricator.wikimedia.org/T160122#3091441 (10demon) p:05High>03Normal Sounds good
[18:17:02] <wikibugs_>	 06Labs, 10Tool-Labs, 06Operations: ssl certificate/key update: *.tools.wmflabs.org (expires on 2017-03-24) - https://phabricator.wikimedia.org/T160187#3091560 (10RobH)
[18:19:09] <wikibugs_>	 06Labs, 10Tool-Labs, 06Operations: ssl certificate/key update: *.tools.wmflabs.org (expires on 2017-03-24) - https://phabricator.wikimedia.org/T160187#3091579 (10RobH) I'm willing to assist on this as needed.  For other changes, we typically do the checklist as follows:  [] - stage new private key in private...
[18:20:16] <wikibugs_>	 06Labs, 10Tool-Labs, 06Operations: ssl certificate/key update: *.tools.wmflabs.org (expires on 2017-03-24) - https://phabricator.wikimedia.org/T160187#3091582 (10RobH)
[18:20:42] <wikibugs_>	 06Labs, 10Tool-Labs, 06Operations: ssl certificate/key update: *.tools.wmflabs.org (expires on 2017-03-24) - https://phabricator.wikimedia.org/T160187#3091560 (10RobH) a:05yuvipanda>03None
[18:20:51] <wikibugs_>	 06Labs, 10Tool-Labs, 06Operations: ssl certificate/key update: *.tools.wmflabs.org (expires on 2017-03-24) - https://phabricator.wikimedia.org/T160187#3091560 (10RobH) a:03RobH
[18:22:17] <madhuvishy>	 zhuyifei1999_: Hello! The labs (misc) nfs share is at super high usage, and I'd like to fast track cleanup of old files in the video project (T158786) - since you are fairly active on tickets related to this project - would you be able to help make that happen?
[18:22:17] <stashbot>	 T158786: Can we delete these old files on video project? - https://phabricator.wikimedia.org/T158786
[18:23:15] <madhuvishy>	 This project alone uses 40% of available storage
[18:24:19] <chasemp>	 thanks madhuvishy
[18:26:28] <andrewbogott>	 As far as I know we were saving those files at matanya's request.  matanya, can you comment?
[18:26:44] <andrewbogott>	 (I feel like I already asked about this a couple of weeks ago…)
[18:28:35] <chasemp>	 this does have a deja vu feeling andrewbogott
[18:33:48] <wikibugs_>	 10Quarry: Quarry runs thousands times slower in last months - https://phabricator.wikimedia.org/T160188#3091603 (10IKhitron)
[18:41:28] <wikibugs_>	 06Labs, 10Tool-Labs, 07Documentation: Document /data/project/.system/config/$TOOLNAME.web-memlimit for raising grid engine webservice memory limit - https://phabricator.wikimedia.org/T160189#3091632 (10bd808)
[18:42:14] <wikibugs_>	 06Labs, 10Tool-Labs, 07Documentation: Document /data/project/.system/config/$TOOLNAME.web-memlimit for raising grid engine webservice memory limit - https://phabricator.wikimedia.org/T160189#3091645 (10bd808) See also https://github.com/toollabs/validator/pull/1#issuecomment-285741073 where @Rillke pointed o...
[18:58:35] <wikibugs_>	 06Labs, 10Tool-Labs, 06Operations, 13Patch-For-Review: ssl certificate/key update: *.tools.wmflabs.org (expires on 2017-03-24) - https://phabricator.wikimedia.org/T160187#3091682 (10RobH)
[18:58:44] <wikibugs_>	 06Labs, 10Tool-Labs, 06Operations, 13Patch-For-Review: ssl certificate/key update: *.tools.wmflabs.org (expires on 2017-03-24) - https://phabricator.wikimedia.org/T160187#3091560 (10RobH) a:05RobH>03yuvipanda
[19:16:44] <wikibugs_>	 10Tool-Labs-tools-Pageviews: Massviews: resolve Project: to equivalent for the wiki project - https://phabricator.wikimedia.org/T160195#3091772 (10MusikAnimal)
[19:18:02] <wikibugs_>	 10Tool-Labs-tools-Pageviews: Massviews: resolve Project: to equivalent for the wiki project - https://phabricator.wikimedia.org/T160195#3091786 (10MusikAnimal)
[19:34:55] <wikibugs_>	 06Labs, 10Tool-Labs, 07Documentation: Document /data/project/.system/config/$TOOLNAME.web-memlimit for raising grid engine webservice memory limit - https://phabricator.wikimedia.org/T160189#3091863 (10Rillke) Added the instructions by Bryan to the Web help of Tool Labs. Right now it is under the Tomcat/ Jav...
[20:15:22] <wikibugs_>	 06Labs, 10Tool-Labs: Add interstitial to wikidata-externalid-url - https://phabricator.wikimedia.org/T160205#3092036 (10Dispenser)
[20:55:43] <wikibugs_>	 06Labs, 10Beta-Cluster-Infrastructure, 13Patch-For-Review: deployment-upload won't start, upload.beta.wmflabs.org down - https://phabricator.wikimedia.org/T131322#3092127 (10Andrew)
[20:55:46] <wikibugs_>	 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Sort out our libvirt qcow2 hack with the upstream - https://phabricator.wikimedia.org/T131548#3092125 (10Andrew) 05Open>03Resolved I wasn't able to get this change merged upstream.  Removing it on a labvirt seems to slightly increase disk usage, but I th...
[21:06:13] <hashar>	 andrewbogott: hello. So should we use raw images instead of qcow2?
[21:06:35] <andrewbogott>	 hashar: no, for the base images qcow2 is fine
[21:06:36] <hashar>	 I have on clue what are the differences or whether one is more interesting than the other
[21:06:45] <andrewbogott>	 That patch is only about newly created VM disk images
[21:06:55] <andrewbogott>	 which is more-or-less independent of the base image type
[21:07:56] <hashar>	 on contintcloud images and snapshots  are all qcow2
[21:09:10] <andrewbogott>	 right, but… 
[21:09:14] <andrewbogott>	 well maybe I don't know what you mean by 'images'
[21:09:38] <andrewbogott>	 you're talking about the base image, or something else?
[21:09:43] <hashar>	 the base images yes
[21:09:52] <hashar>	 the one sent to glance and we use to then boot instances
[21:10:08] <andrewbogott>	 right, those are qcow2 everywhere.
[21:10:14] <hashar>	 ok :}
[21:47:46] <wikibugs_>	 (03PS1) 10Hashar: Remove passwords::misc::contint::jenkins [labs/private] - 10https://gerrit.wikimedia.org/r/342277
[22:36:50] <wikibugs_>	 06Labs, 10wikitech.wikimedia.org, 15User-bd808: Fix or delete user accounts showing on wikitech but not found in LDAP after T149109 - https://phabricator.wikimedia.org/T159986#3092224 (10bd808) 05Open>03Resolved I deleted the `user` and `ldap_domains` rows for these users and emailed the account owners m...