[00:08:04] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Internal Server Error from new articleinfo interface in XTools - https://phabricator.wikimedia.org/T169767#3423390 (10kaldari) It looks like this is no longer an issue since the relevant config variables have been moved to `parameters.yml.dist`. Does anything... [00:09:31] 10Striker, 10Patch-For-Review: Striker error when logging non-ASCII strings - https://phabricator.wikimedia.org/T164034#3423395 (10bd808) Using the patch from https://gerrit.wikimedia.org/r/364342 in the striker labs testing project gives the result: ``` stdout: ANSI_X3.4-1968; stderr: ANSI_X3.4-1968 ``` I ca... [00:28:26] 10Data-Services, 10DBA, 10Patch-For-Review: Expose ar_content_format and ar_content_model columns of archive table on Labs replicas - https://phabricator.wikimedia.org/T89741#3423430 (10Bawolff) I'm fine with it, provided that its null when ar_deleted&1=1. This may be mildly paranoia, but I'd like to be str... [00:37:13] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Internal Server Error from new articleinfo interface in XTools - https://phabricator.wikimedia.org/T169767#3423456 (10Samwilson) It's true, this is probably solved. However, we're still getting this error in the logs because we haven't set up the XTools DB yet... [00:40:16] PROBLEM - Puppet errors on tools-exec-1438 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [00:42:34] !log striker Cherry-picking https://gerrit.wikimedia.org/r/#/c/364350/ for testing [00:42:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL [00:43:20] !log striker Cherry-picking https://gerrit.wikimedia.org/r/#/c/362210/ [00:43:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL [00:44:12] !log striker Cherry-picking https://gerrit.wikimedia.org/r/#/c/364174/ [00:44:14] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL [00:49:18] !log striker Fixed apache package and restarted on striker-puppet01 [00:49:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL [01:07:42] 10Data-Services, 10DBA, 10Patch-For-Review: Expose ar_content_format and ar_content_model columns of archive table on Labs replicas - https://phabricator.wikimedia.org/T89741#3423529 (10APalmer_WMF) 05Open>03Resolved [01:07:59] 10Data-Services, 10DBA, 10Patch-For-Review: Expose ar_content_format and ar_content_model columns of archive table on Labs replicas - https://phabricator.wikimedia.org/T89741#1044220 (10APalmer_WMF) 05Resolved>03Open [01:08:53] 10Striker, 10Patch-For-Review: Striker error when logging non-ASCII strings - https://phabricator.wikimedia.org/T164034#3423532 (10bd808) With https://gerrit.wikimedia.org/r/364350 applied: ``` stdout: UTF-8; stderr: UTF-8 ``` [01:09:31] (03Abandoned) 10BryanDavis: [DO NOT MERGE] Add view to show active encoding [labs/striker] - 10https://gerrit.wikimedia.org/r/364342 (https://phabricator.wikimedia.org/T164034) (owner: 10BryanDavis) [01:15:16] RECOVERY - Puppet errors on tools-exec-1438 is OK: OK: Less than 1.00% above the threshold [0.0] [01:38:35] PROBLEM - Puppet errors on tools-exec-1442 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [01:39:41] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Planning for Xtools beta - https://phabricator.wikimedia.org/T167217#3321093 (10Samwilson) a:03Samwilson [01:42:13] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Set up load balancing for new XTools - https://phabricator.wikimedia.org/T169590#3423574 (10Samwilson) a:03Samwilson The three-VPS set up is what I had been thinking was the way forward, but for now we'll just get things running on the medium-sized `xtools-p... [01:46:20] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [02:00:38] (03PS1) 10Krinkle: api: Create q=replag API [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364356 (https://phabricator.wikimedia.org/T170024) [02:00:40] (03PS1) 10Krinkle: frontend: Add warning if replag is high [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364357 (https://phabricator.wikimedia.org/T170024) [02:01:23] (03PS2) 10Krinkle: api: Create q=replag API [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364356 (https://phabricator.wikimedia.org/T170024) [02:01:25] (03CR) 10Krinkle: [C: 032] api: Create q=replag API [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364356 (https://phabricator.wikimedia.org/T170024) (owner: 10Krinkle) [02:01:27] (03PS2) 10Krinkle: frontend: Add warning if replag is high [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364357 (https://phabricator.wikimedia.org/T170024) [02:01:29] (03CR) 10Krinkle: [C: 032] frontend: Add warning if replag is high [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364357 (https://phabricator.wikimedia.org/T170024) (owner: 10Krinkle) [02:01:42] (03Merged) 10jenkins-bot: api: Create q=replag API [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364356 (https://phabricator.wikimedia.org/T170024) (owner: 10Krinkle) [02:01:58] (03Merged) 10jenkins-bot: frontend: Add warning if replag is high [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364357 (https://phabricator.wikimedia.org/T170024) (owner: 10Krinkle) [02:03:55] (03PS1) 10Krinkle: i18n: Add docs for new results-lag-warning message [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364358 [02:05:01] (03CR) 10jerkins-bot: [V: 04-1] i18n: Add docs for new results-lag-warning message [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364358 (owner: 10Krinkle) [02:05:31] (03PS2) 10Krinkle: i18n: Add docs for new results-lag-warning message [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364358 [02:13:31] RECOVERY - Puppet errors on tools-exec-1442 is OK: OK: Less than 1.00% above the threshold [0.0] [02:17:00] (03CR) 10Krinkle: [C: 032] i18n: Add docs for new results-lag-warning message [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364358 (owner: 10Krinkle) [02:17:03] (03Merged) 10jenkins-bot: i18n: Add docs for new results-lag-warning message [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/364358 (owner: 10Krinkle) [02:19:58] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Set up load balancing for new XTools - https://phabricator.wikimedia.org/T169590#3423594 (10bd808) >>! In T169590#3423574, @Samwilson wrote: > We're also wanting to track 404s, and if we're just running on a single server that's easier (are we allowed to insta... [02:31:20] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [02:42:00] PROBLEM - Puppet errors on tools-exec-1423 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [02:51:33] (03PS1) 10BryanDavis: Update fatal error pages [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/364359 [02:55:09] 10Striker, 10Patch-For-Review, 10User-bd808: Striker error when logging non-ASCII strings - https://phabricator.wikimedia.org/T164034#3423601 (10bd808) 05Open>03Resolved a:03bd808 I don't know exactly how to recreate this, but the stream encoding looks right now. [02:56:49] 10Striker: 502 Bad Gateway when trying to associate Phabricator account - https://phabricator.wikimedia.org/T170189#3423605 (10bd808) @demon could you try this action again now that {T164034} is thought to be fixed? We still may have some problem, but hopefully it won't melt the process with lame ascii encoding... [02:57:43] (03CR) 10BryanDavis: [C: 032] "Trivial" [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/364359 (owner: 10BryanDavis) [02:57:48] (03Merged) 10jenkins-bot: Update fatal error pages [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/364359 (owner: 10BryanDavis) [03:17:04] RECOVERY - Puppet errors on tools-exec-1423 is OK: OK: Less than 1.00% above the threshold [0.0] [03:28:12] (03PS1) 10BryanDavis: [WIP] Manage shared tool accounts [labs/striker/staticfiles] - 10https://gerrit.wikimedia.org/r/364360 (https://phabricator.wikimedia.org/T149458) [03:46:04] PROBLEM - Puppet errors on tools-exec-1428 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [04:11:02] RECOVERY - Puppet errors on tools-exec-1428 is OK: OK: Less than 1.00% above the threshold [0.0] [04:54:19] PROBLEM - Free space - all mounts on tools-logs-02 is CRITICAL: CRITICAL: tools.tools-logs-02.diskspace._srv.byte_percentfree (<55.56%) [04:56:47] (03CR) 10BryanDavis: Add toolinfo.json style data (031 comment) [labs/striker] - 10https://gerrit.wikimedia.org/r/353909 (https://phabricator.wikimedia.org/T149458) (owner: 10BryanDavis) [06:31:55] PROBLEM - Puppet errors on tools-exec-1436 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:40:05] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1427 is CRITICAL: CRITICAL: 25.00% of data above the critical threshold [0.0] [06:41:13] PROBLEM - Puppet errors on tools-exec-1438 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:54:20] RECOVERY - Free space - all mounts on tools-logs-02 is OK: OK: All targets OK [07:04:56] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: XTools prod environment: unknown "profiler_dump" function - https://phabricator.wikimedia.org/T170233#3423862 (10Samwilson) [07:06:53] RECOVERY - Puppet errors on tools-exec-1436 is OK: OK: Less than 1.00% above the threshold [0.0] [07:16:14] RECOVERY - Puppet errors on tools-exec-1438 is OK: OK: Less than 1.00% above the threshold [0.0] [07:18:40] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: 500 Internal Server Error when using Article Info interface in new XTools - https://phabricator.wikimedia.org/T170234#3423887 (10kaldari) [07:18:52] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: 500 Internal Server Error when using Article Info interface in new XTools - https://phabricator.wikimedia.org/T170234#3423899 (10kaldari) p:05Triage>03High [07:20:05] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1427 is OK: OK: Less than 1.00% above the threshold [0.0] [07:21:33] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Loading Edit Counter in new XTools triggers PUT error - https://phabricator.wikimedia.org/T170100#3423901 (10Samwilson) Okay, new prod server is recording hits correctly now: https://xtools.wmflabs.org/meta [07:29:50] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: 500 Internal Server Error when using Article Info interface in new XTools - https://phabricator.wikimedia.org/T170234#3423887 (10Samwilson) The log says: > [2017-07-11 07:26:20] request.CRITICAL: Uncaught PHP Exception Symfony\Component\Debug\Exception\FatalE... [07:30:56] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Set up load balancing for new XTools - https://phabricator.wikimedia.org/T169590#3423909 (10Samwilson) The new box is up and running, and is auto-deploying the master branch whenever there's a change (it checks every 10 minutes). I've created two new database... [07:38:22] PROBLEM - Free space - all mounts on tools-worker-1020 is CRITICAL: CRITICAL: tools.tools-worker-1020.diskspace.root.byte_percentfree (<100.00%) [07:45:55] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Planning for Xtools beta - https://phabricator.wikimedia.org/T167217#3423955 (10Samwilson) The new prod server is set up, and the former live01 turned off (but not deleted). Start of documentation: https://wikitech.wikimedia.org/wiki/Tool:XTools * Do we nee... [07:56:20] PROBLEM - Puppet errors on tools-worker-1021 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [08:31:20] RECOVERY - Puppet errors on tools-worker-1021 is OK: OK: Less than 1.00% above the threshold [0.0] [08:35:39] 10cloud-services-team, 10Wikimedia-Hackathon-2018-Organization, 10Developer-Relations (Jul-Sep 2017): Featured Projects related to Wikimedia Cloud Services and/or Technical Operations? - https://phabricator.wikimedia.org/T170242#3424101 (10Qgil) [11:32:31] PROBLEM - Puppet errors on tools-exec-1417 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [12:09:41] PROBLEM - Puppet errors on tools-exec-1405 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [12:12:34] RECOVERY - Puppet errors on tools-exec-1417 is OK: OK: Less than 1.00% above the threshold [0.0] [12:26:42] bd808: I did a strace of a test script of tail_lines, with time.sleep(0.001) after each iteration [12:28:14] comparing the syscalls with python 2 and 3, I noticed python 3 use a significatly larger buffer size. python 2 is 1048576 but python 3 is 8192 [12:29:42] might be related to that [12:36:11] 400 * 45000 * 7 is 126000000. I'm doing a test on about a tenth of the data, 1048576 * 16 = 16777216 [12:36:40] with no time.sleep python2 took 0m18.751s [12:36:54] python3 still running after a few minutes :/ [12:37:08] will look into it later, gtg now [12:49:42] RECOVERY - Puppet errors on tools-exec-1405 is OK: OK: Less than 1.00% above the threshold [0.0] [12:50:40] 10Tools: File uploads need to recognise urls on blacklist early in the process - https://phabricator.wikimedia.org/T157436#3424805 (10Magnus) Flickr2Commons issue tracker: https://bitbucket.org/magnusmanske/flickr2commons/issues?status=new&status=open [13:59:55] o/ [14:00:23] I'm here for labsdb1004 reboot. [14:01:13] halfak: \o [14:07:42] madhuvishy: I'm about if needed but trying to stay out of the way :) [14:23:18] !log wikilabels database maintenance has started for labsdb1004 [14:23:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [14:26:08] !log wikilabels confirmed back online [14:26:09] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [14:33:56] !log wikilabels staged wikilabels-wmflabs-deploy:bfe4a03 [14:33:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [14:34:31] !log wikilabels deployed wikilabels-wmflabs-deploy:bfe4a03 [14:34:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [14:36:57] 10Tools, 10cloud-services-team (Kanban), 10User-bd808: grid-jobs tool broken; loads forever with no actual response - https://phabricator.wikimedia.org/T168653#3425186 (10zhuyifei1999) bd808 gave me a log entry `GET /grid-jobs/?purge => generated 83797 bytes in 49443413 msecs`, which is approximately 13.73 h... [14:41:29] 10Tools, 10cloud-services-team (Kanban), 10User-bd808: grid-jobs tool broken; loads forever with no actual response - https://phabricator.wikimedia.org/T168653#3425198 (10bd808) We aren't explicitly opening in binary mode, but https://docs.python.org/3/library/functions.html#open says: > Binary files are buf... [14:43:47] NFS is being really slow on bastion-02... [14:46:45] zhuyifei1999_: are you testing that read loop on tools-dev (bastion-02)? I think you may be eating all of the NFS server quota [14:46:56] uh [14:46:59] yeah [14:47:09] which host should I test on? [14:47:44] hmmm... good question [14:48:00] I ctrl-c-ed [14:48:03] * bd808 looks at grid status [14:49:55] zhuyifei1999_: try tools-trusty.wmflabs.org [14:50:00] ok [14:50:10] it looks like everyone has forgotten it exists :) [14:51:22] I used to use it until someone forget to reboot it after the attribute cache was reenabled (it was the only bastion with that disabled) [14:52:23] zhuyifei1999_: did you see my note about the open() read buffer? [14:52:29] yeah [14:52:37] PROBLEM - Puppet errors on tools-prometheus-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [14:52:46] I'll look into it [14:52:46] I'm wondering if something changed how py3 is selecting the buffer size [14:55:53] 10cloud-services-team, 10DBA, 10Operations, 10Scoring-platform-team: Labsdb* servers need to be rebooted - https://phabricator.wikimedia.org/T168584#3425238 (10madhuvishy) [14:58:57] 10cloud-services-team, 10DBA, 10Operations, 10Scoring-platform-team: Labsdb* servers need to be rebooted - https://phabricator.wikimedia.org/T168584#3425254 (10madhuvishy) Status: labsdb1005 reboot is scheduled for July 12 at 1400 UTC. We've decided to wait on labsdb1001 and 1003 reboots for now - given t... [15:03:06] thanks for being around halfak o/ [15:03:32] :D Thanks for making sure I got looped in and knew when stuff was happening :D [15:07:44] :) [15:07:58] PROBLEM - Puppet errors on tools-prometheus-02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:10:01] Hello Harsh SHah! [15:10:05] Here [15:10:41] 10cloud-services-team, 10Wikimedia-Hackathon-2018-Organization, 10Developer-Relations (Jul-Sep 2017): Featured Projects related to Wikimedia Cloud Services and/or Technical Operations? - https://phabricator.wikimedia.org/T170242#3425293 (10bd808) The difficulty the #cloud-services-team would have in offerin... [15:11:45] I am working on my GSOC project [15:12:09] (03PS1) 10Alexandros Kosiaris: Add docker::registry::username [labs/private] - 10https://gerrit.wikimedia.org/r/364452 [15:12:16] I crated bot i need wikilab to test my BOT [15:12:19] harshcro_: hello! [15:12:22] hello [15:12:33] (03CR) 10Giuseppe Lavagetto: [C: 031] Add docker::registry::username [labs/private] - 10https://gerrit.wikimedia.org/r/364452 (owner: 10Alexandros Kosiaris) [15:12:47] (03CR) 10Alexandros Kosiaris: [V: 032 C: 032] Add docker::registry::username [labs/private] - 10https://gerrit.wikimedia.org/r/364452 (owner: 10Alexandros Kosiaris) [15:13:14] harshcro_: you are probably looking to create a tool for your bot [15:13:47] here's some background - https://wikitech.wikimedia.org/wiki/Help:Cloud_Services_Introduction and here's how to create a tool account - https://wikitech.wikimedia.org/wiki/Help:Getting_Started#Tools_Users [15:13:53] i need cluster for my Bot [15:14:04] i created node app [15:15:00] can you guide me for wiki tool lab [15:15:09] harshcro_: the best runtime environment we have for nodejs bots right now is the Kubernetes cluster in our Toolforge (still called Tool Labs) project. [15:15:55] okay [15:16:02] how i can use [15:16:26] can you guide? [15:16:26] The first step is to become a member of Toolforge. The links that madhuvishy gave cover this [15:17:07] I'll look for a description of the steps to run a nodejs bot. I know I've written about this recently in a couple of Phabricator tasks [15:19:19] harshcro_: this is an overview of running a nodejs bot on Kubernetes in Toolforge -- https://phabricator.wikimedia.org/T164910#3271720 [15:19:51] its not a simple process, but each step is described there [15:20:34] okay [15:22:58] how beacame i memeber of toolforge [15:23:01] ? [15:23:20] harshcro_: https://wikitech.wikimedia.org/wiki/Help:Getting_Started#Tools_Users [15:24:14] we are in the middle of renaming things so you will see documentation mentioning "tools", "tool labs", and "Toolforge" [15:24:20] they are all the same thing [15:24:32] okay [15:26:23] (03PS1) 10Alexandros Kosiaris: Add docker::registry::password [labs/private] - 10https://gerrit.wikimedia.org/r/364454 [15:32:53] (03CR) 10Alexandros Kosiaris: [V: 032 C: 032] Add docker::registry::password [labs/private] - 10https://gerrit.wikimedia.org/r/364454 (owner: 10Alexandros Kosiaris) [15:33:29] 10Tools, 10cloud-services-team (Kanban), 10User-bd808: grid-jobs tool broken; loads forever with no actual response - https://phabricator.wikimedia.org/T168653#3425415 (10zhuyifei1999) Testing setting buffering for both python2 and 3 and stracing it: buffering = 2097152, python2: ``` lseek(3, 1795162112, SE... [15:38:18] 10Tools, 10cloud-services-team (Kanban), 10User-bd808: grid-jobs tool broken; loads forever with no actual response - https://phabricator.wikimedia.org/T168653#3425436 (10zhuyifei1999) Ah, all question problems solved if I open the file in binary... [15:41:24] 10Tools, 10cloud-services-team (Kanban), 10User-bd808: grid-jobs tool broken; loads forever with no actual response - https://phabricator.wikimedia.org/T168653#3425469 (10zhuyifei1999) If the mode is set to 'rb' and keeping the buffering -1: python2: ``` open("/data/project/.system/accounting", O_RDONLY) =... [15:43:50] bd808: apparently all we have to do it set it binary explicitly... [15:44:21] that will make the lines yielded 'bytes' though. might need to deal with that [15:47:15] andrewbogott, just saw your note at https://phabricator.wikimedia.org/T169811#3425490 [15:47:44] halfak: I'm in meetings and interviews for a bit but might be able to pay attention to that in the afternoon :) [15:47:46] Would you like us to set up quotas so that they are *tight* or can we have some wiggle room to set up a new VM before taking an old one offline. [15:47:51] OK cool :) [15:47:59] I'll post in the task :) [15:51:50] General Cloud VPS question: If I want to transfer a big chunk of data off of one VM and onto another, what's the best way to do that? [15:52:41] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Planning for Xtools beta - https://phabricator.wikimedia.org/T167217#3425528 (10MusikAnimal) >>! In T167217#3423955, @Samwilson wrote: > The new prod server is set up, and the former live01 turned off (but not deleted). Start of documentation: https://wikitech... [15:55:43] bd808 : i have wikitech account [15:56:49] but not able to access Wikimedia Cloud Service [15:58:18] harshcro_: what URL are you having problems with? [15:58:36] https://horizon.wikimedia.org/auth/login/ [15:58:44] not able to login [15:58:58] that will use the same username and password that you use for wikitech [15:59:54] ya i am using same [16:00:00] but having error [16:00:19] error : You are not authorized for any projects or domains. [16:02:04] harshcro_: ah. right. You would not be a project member for anything in horizon yet [16:02:31] 10Striker: 502 Bad Gateway when trying to associate Phabricator account - https://phabricator.wikimedia.org/T170189#3425560 (10demon) Works! [16:02:31] so what is my next step [16:02:40] harshcro_: the place you need to go is https://toolsadmin.wikimedia.org/ to sing up for toolforge membership [16:04:01] i already have [16:05:26] harshcro_: what is your wikitech user name? [16:05:41] Harshhacki [16:07:11] harshcro_: I don't see a request for you in https://toolsadmin.wikimedia.org/tools/membership/ [16:07:36] can you see again [16:08:01] 10Cloud-Services, 10Cloud-VPS, 10Operations, 10netops, 10ops-codfw: codfw: labtestpuppetmaster2001 switch port configuration - https://phabricator.wikimedia.org/T167321#3425572 (10RobH) a:05RobH>03Papaul Assigned to Papaul to try another NIC on the server, and open a support case for a bad nic if so. [16:08:13] http://imgur.com/a/VQsMW [16:08:26] see this i am sharing screenshot [16:08:45] harshcro_: awesome. I see it now. [16:09:00] cool [16:10:27] harshcro_: you are approved :) [16:10:39] yeah thank you! [16:11:25] now i have to search about Kubernetes-testing [16:11:27] right [16:12:25] harshcro_: the next thing you will need is to add a ssh key so you can log into the shared hosts [16:12:41] okay [16:12:43] harshcro_: https://toolsadmin.wikimedia.org/profile/settings/ssh-keys [16:13:20] there is more info at https://wikitech.wikimedia.org/wiki/Help:Getting_Started#Uploading_an_SSH_key [16:13:54] 10Cloud-Services, 10Cloud-VPS, 10Operations, 10netops, 10ops-codfw: codfw: labtestpuppetmaster2001 switch port configuration - https://phabricator.wikimedia.org/T167321#3425612 (10Papaul) @Robh this is already done it was not switch problem it was DNS see T167157 [16:13:57] okay [16:14:23] 10Cloud-Services, 10Cloud-VPS, 10Operations, 10ops-codfw, 10Patch-For-Review: rack/setup/install labtestpuppetmaster2001 - https://phabricator.wikimedia.org/T167157#3425615 (10RobH) [16:14:25] 10Cloud-Services, 10Cloud-VPS, 10Operations, 10netops, 10ops-codfw: codfw: labtestpuppetmaster2001 switch port configuration - https://phabricator.wikimedia.org/T167321#3425614 (10RobH) 05Open>03Resolved [16:15:27] i add my ssh key [16:16:33] Now you can ssh to the bastion server for toolforge -- tools-login.wmflabs.org [16:17:47] okay [16:23:31] 10cloud-services-team (FY2017-18), 10Wikimedia-Blog-Content: Publish a blog post covering the Cloud Services rebranding process - https://phabricator.wikimedia.org/T170288#3425689 (10bd808) [16:24:00] 10cloud-services-team (FY2017-18), 10Goal, 10Patch-For-Review, 10User-bd808: Perform initial Cloud Services rebranding - https://phabricator.wikimedia.org/T168480#3365842 (10bd808) [16:33:20] bd808 i am not able to acsess run ssh [16:35:51] harshcro_: do you have any error messages you can share? I do not see any failed logins on tools-login that mention your username [16:36:29] 10Tool-Labs-tools-Xtools, 10JavaScript: Phabricator link on Internal Server Error page is reported as an XSS attack by noscript - https://phabricator.wikimedia.org/T170292#3425817 (10Matthewrbowker) [16:36:30] i am running this command ssh -v bastion.wmflabs.org - wikitool.pub [16:36:34] is that right [16:36:51] 10Tool-Labs-tools-Xtools, 10JavaScript: Phabricator link on Internal Server Error page is reported as an XSS attack by noscript - https://phabricator.wikimedia.org/T170292#3425830 (10Matthewrbowker) p:05Triage>03Lowest [16:37:43] that would try to execute a "wikitool.pub" command on the bastion.wmflabs.org host [16:37:51] so, probably not what you are wanting [16:38:50] try something like `ssh -i ~/.ssh/wikitool -v harshhacki@tools-login.wmflabs.org` [16:44:28] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint, 10JavaScript: Phabricator link on Internal Server Error page is reported as an XSS attack by noscript - https://phabricator.wikimedia.org/T170292#3425890 (10Matthewrbowker) [16:44:43] now! [16:44:46] ? [16:45:00] you can see i log in [16:45:43] harshcro_: nice. I see you have a shell session on tools-login :) [16:45:58] ya [16:46:04] 10Tool-Labs-tools-Xtools, 10Community-Tech, 10JavaScript: Phabricator link on Internal Server Error page is reported as an XSS attack by noscript - https://phabricator.wikimedia.org/T170292#3425904 (10MusikAnimal) [16:46:57] now in this what i have to do? [16:48:54] harshcro_: your next step is probably to create a tool account to run your bot under -- https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Tool_Accounts [16:50:24] okay [16:53:38] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: 500 Internal Server Error when using Article Info interface in new XTools - https://phabricator.wikimedia.org/T170234#3423887 (10Matthewrbowker) Taking a quick look at the code... We use date_diff(), which is an alias of DateTime::Diff. Docs for that functio... [16:55:06] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: 500 Internal Server Error when using Article Info interface in new XTools - https://phabricator.wikimedia.org/T170234#3423887 (10MusikAnimal) Yep, I've been working on this. I think it's memory-related. [16:55:12] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: 500 Internal Server Error when using Article Info interface in new XTools - https://phabricator.wikimedia.org/T170234#3425955 (10MusikAnimal) a:03MusikAnimal [17:01:40] for unix user and group [17:01:55] i have make my own or from tools project [17:01:56] ? [17:02:42] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Loading Edit Counter in new XTools triggers PUT error - https://phabricator.wikimedia.org/T170100#3425974 (10kaldari) 05Open>03Resolved a:03kaldari [17:07:42] 10cloud-services-team, 10Wikimedia-Hackathon-2018-Organization, 10Developer-Relations (Jul-Sep 2017): Featured Projects related to Wikimedia Cloud Services and/or Technical Operations? - https://phabricator.wikimedia.org/T170242#3426029 (10Qgil) Note that here we are talking specifically about projects done... [17:08:54] 10Tool-Labs-tools-Xtools, 10Community-Tech, 10JavaScript: Phabricator link on Internal Server Error page is reported as an XSS attack by noscript - https://phabricator.wikimedia.org/T170292#3426033 (10Matthewrbowker) It's the file name in the task description. Logging output: ``` [NoScript InjectionChecker... [17:17:26] harshcro_: I do not understand your question. Can you rephrase it or point to the document you are reading that is confusing? [17:17:58] Unix user: tools.toolname [17:17:59] Unix group: tools.toolname [17:18:06] i am taking about this [17:18:36] ah. so if you created a tool account named "my-tool" then this would create the "tools.my-tool" user and group [17:19:23] this is done behind the scenes when you submit the "Add service group" form [17:19:59] i need to run my boot so which tool i have to use? [17:20:51] *BOT [17:21:04] your bot will be a "tool". You need to create a tool account for it. Typically this would have a name that is related to the code you are going to run there. [17:21:42] that help section is very heard to read :/ [17:22:12] ya confuse me [17:22:18] https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Creating_a_new_Tool_account [17:22:41] I'll make a note about trying to clean up the wording there [17:22:53] 10cloud-services-team, 10Wikimedia-Hackathon-2018-Organization, 10Developer-Relations (Jul-Sep 2017): Featured Projects related to Wikimedia Cloud Services and/or Technical Operations? - https://phabricator.wikimedia.org/T170242#3424101 (10chasemp) >>! In T170242#3426029, @Qgil wrote: > Note that here we are... [17:23:30] okay [17:28:27] 10Cloud-Services, 10Tracking, 10User-bd808: New Labs project requests (tracking) - https://phabricator.wikimedia.org/T76375#3426173 (10bd808) [17:31:42] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Memory issues with ArticleInfo - https://phabricator.wikimedia.org/T170297#3426180 (10MusikAnimal) [17:32:13] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Memory issues with ArticleInfo - https://phabricator.wikimedia.org/T170297#3426180 (10MusikAnimal) p:05Triage>03High [17:33:53] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: 500 Internal Server Error when using Article Info interface in new XTools - https://phabricator.wikimedia.org/T170234#3426200 (10MusikAnimal) I got the same error on prod trying to load a page with a low revision count, so this apparently is different than T17... [17:35:43] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Memory issues with ArticleInfo - https://phabricator.wikimedia.org/T170297#3426206 (10MusikAnimal) [17:41:51] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Memory issues with ArticleInfo - https://phabricator.wikimedia.org/T170297#3426227 (10MusikAnimal) a:03MusikAnimal [17:48:28] i have my tool.wikiemojibot [17:48:32] now [17:54:40] bd808: if I want to know if someone has modified stewardbot files w/o going through gerrit, and want to sync both things, what am I suposed to do? [17:55:56] harshcro_: cool. Now you can start learning how our Kubernetes cluster works -- https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Kubernetes -- Your bot will probably be a "continuous job" unless it has a web UI [17:56:32] it's a twitter bot [17:57:56] TabbyCat: you want to get the current working tree into gerrit or ? [17:58:27] bd808: I suspect public_html files from stewardbots were modified directly on toolforge rather than uploading the patch to gerrit [17:58:40] is there any way to know that? [17:58:44] git blame? [17:59:27] git status says that we're up to date [18:00:14] TabbyCat: https://stackoverflow.com/questions/1800783/compare-local-git-branch-with-remote-branch [18:00:26] oh and git push asks me for the password for matanya [18:00:34] maybe the path should be changed [18:00:54] that's probably in the local repo config [18:02:31] how i supposed to start Kubernetes in sheel [18:02:34] shell [18:14:15] harshcro_: this isn't made completely clear in the docs apparently. [18:14:25] the command you need to run is: webservice --backend=kubernetes nodejs shell [18:15:03] that will create a kubernetes pod with the nodejs Docker image and then give you an interactive shell inside it [18:15:25] the command is mentioned at https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web#node.js_web_services [18:23:22] okay [18:26:30] * zhuyifei1999_ just realized harshcro_ is on https://phabricator.wikimedia.org/T164560 [18:26:55] ya [18:32:04] harshcro_: so if you are more familiar with Heroku or Amazon, one of the main difference between those and toolforge is that toolforge is more of a shared hosting [18:32:54] okay but i am not familiar with commands and all the stuff [18:33:02] in toolforge [18:33:08] PROBLEM - Puppet errors on tools-exec-1431 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [18:34:36] we sometimes create VPS projects, but they are usually for more complex stuffs and 'running a bot' usually don't qualify (since everything here is running on donor money, afaict) [18:34:54] the commands are basically linux commands [18:35:12] okay [18:35:13] and a few customized [18:35:21] okay [18:35:39] (03PS1) 10MarcoAurelio: Update privacy, cookie statement and code of conduct [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/364491 [18:36:14] since you will be interacting with kubernetes you can ignore quite a few customized commands related to grid [18:37:23] zhuyifei1999_: thanks for helping harshcro_ here. Maybe we can end up with a "if you know how to use Heroku then here's how to use Kubernetes in Toolforge" out of this at the end :) [18:37:26] (03CR) 10MarcoAurelio: [C: 032] Update privacy, cookie statement and code of conduct [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/364491 (owner: 10MarcoAurelio) [18:37:45] that would indeed be a swell matrix [18:37:49] (03Merged) 10jenkins-bot: Update privacy, cookie statement and code of conduct [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/364491 (owner: 10MarcoAurelio) [18:37:51] the not-really-linux-core commands you might have to use often is kubectl, become, and webservice [18:38:20] bd808: I actually never really touched heroku though [18:38:21] ya i am reading the document [18:38:44] zhuyifei1999_: but harshcro_ has! [18:39:00] ya i use heroku [18:39:09] heroku wraps all the things up in a nice cli [18:39:11] aws and digital ocean [18:39:19] ya [18:39:32] their buildpacks are like the webservice "flavors" we have [18:39:44] not exactly the same but close [18:39:52] !log tools.stewardbots Patch https://gerrit.wikimedia.org/r/364491 merged [18:39:52] while running a bot isn't really a webservice, the command currently has the simplest (imo) customized interface to toolforge k8s [18:39:54] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [18:40:27] TabbyCat: did you figure out your diff question? [18:40:28] bd808: I just can't help not typing an "s" after tool... [18:40:38] i.e. toolsforge [18:40:55] don't do it. chasemp will hunt you down :) [18:40:59] bd808: somewhat, I think I synced it and deleted a change in the meanwhile [18:41:10] * zhuyifei1999_ facepalm [18:41:11] their fault, should've gone through gerrit as everyone [18:42:46] thanks for the help [18:42:53] np [18:43:16] i am going to sleep now it's 12:20 in night here [18:43:45] good night harshcro_. Sleep is important :) [18:44:11] ya [18:44:16] good night [18:54:36] General Cloud VPS question: If I want to transfer a big chunk of data off of one VM and onto another, what's the best way to do that?  [18:55:09] I'd like to not download it to my workstation (tiny little laptop SSD + slowish xfer) [18:55:35] I could temporarily set up an SSH key and upload it to one of the VMs. [18:55:40] That doesn't sound to crazy. [18:56:14] rsync server could do it too [18:56:44] Is that a thing? Could I do it without agent-forwarding or uploading a key? [18:57:16] yeah you could setup rsync on the target VM to allow anon uploads [18:57:27] halfak: the "tiny SSD" problem can be addressed by scp -3 (but not the slowish xfer) [18:57:29] Oh! I see. hmm [18:57:36] otherwise... yeah, you can do rsync [18:57:39] or just a netcat :) [18:57:57] Ohhhhhh. I didn't know you could do -3 [18:57:59] That's interesting [18:58:00] yeah and as paravoid says "scp -3" will pass the data through your laptop but no store on disk [18:58:06] right [18:58:10] nc -l -p 1234 | tar xv [18:58:10] Interesting [18:58:44] tar -c foo bar baz | nc hostname 1234 [18:58:52] putting keys for your real account on a vm is not a good idea [18:59:32] I'd just make a temp key and not even tell wikitech about it. [18:59:34] paravoid's netcat pipe should actually work pretty easily [18:59:39] Then destroy it all when done. [19:00:06] that's easier than an rsync server for one-off [19:00:08] That's an interesting solution [19:04:23] (03Draft1) 10Paladox: add check_disk to ores-compute-01 [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364497 [19:04:26] (03PS2) 10Paladox: Fix check_disk_space [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364497 [19:04:29] (03CR) 10Paladox: [V: 032 C: 032] Fix check_disk_space [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364497 (owner: 10Paladox) [19:08:08] RECOVERY - Puppet errors on tools-exec-1431 is OK: OK: Less than 1.00% above the threshold [0.0] [19:08:39] (03PS1) 10BryanDavis: Cloud Services: Notify for Cloud-VPS milestones [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/364499 [19:17:12] 10Striker, 10Patch-For-Review, 10User-bd808: Striker error when logging non-ASCII strings - https://phabricator.wikimedia.org/T164034#3426686 (10bd808) [19:17:15] 10Striker: 502 Bad Gateway when trying to associate Phabricator account - https://phabricator.wikimedia.org/T170189#3426688 (10bd808) [19:31:34] 10Data-Services, 10Toolforge, 10cloud-services-team (Kanban), 10Wikimedia-Incident: 2017-07-02 Toolforge data loss for permissive data - https://phabricator.wikimedia.org/T169774#3426765 (10chasemp) 05Open>03Resolved a:03chasemp Data is available for user restore, notifications have been sent to the... [19:32:19] 10Data-Services, 10Toolforge, 10cloud-services-team (Kanban), 10Wikimedia-Incident: 2017-07-02 Toolforge data loss for permissive data - https://phabricator.wikimedia.org/T169774#3426773 (10chasemp) 05Resolved>03stalled [19:32:29] 10Data-Services, 10Toolforge, 10cloud-services-team (Kanban), 10Wikimedia-Incident: Cleanup: 2017-07-02 Toolforge data loss for permissive data - https://phabricator.wikimedia.org/T169774#3408084 (10chasemp) [19:32:40] 10Data-Services, 10Toolforge, 10cloud-services-team (Kanban), 10Wikimedia-Incident: Cleanup: 2017-07-02 Toolforge data loss for permissive data - https://phabricator.wikimedia.org/T169774#3408084 (10chasemp) marked stalled so we remember to remove the temp restore data [19:40:39] paravoid, do I need to kill the nc processes once the xfer finishes? [19:40:49] It seems to be just hanging :/ [19:40:49] the listening one, yes [19:40:57] the other one wouldn't be? [19:41:02] Gotcha. crtl-C feels wrong :) [19:41:47] * halfak does it anyway [19:43:05] it depends on the version of nc I think [19:43:39] 10Cloud-VPS, 10ORES, 10Scoring-platform-team: Set up larger ores-compute instance - https://phabricator.wikimedia.org/T169809#3426874 (10Halfak) I've moved everything from the home dir of ores-experiment to `ores-staging-02:/srv/ores-experiment-20170711/home` [19:47:51] (03CR) 10Legoktm: [C: 032] Cloud Services: Notify for Cloud-VPS milestones [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/364499 (owner: 10BryanDavis) [19:48:12] (03Merged) 10jenkins-bot: Cloud Services: Notify for Cloud-VPS milestones [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/364499 (owner: 10BryanDavis) [19:48:20] (03CR) 10jenkins-bot: Cloud Services: Notify for Cloud-VPS milestones [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/364499 (owner: 10BryanDavis) [19:49:23] !log tools.wikibugs Updated channels.yaml to: 05cf3b8f6a22cf79a5aa75a0702087c3659e646d Cloud Services: Notify for Cloud-VPS milestones [19:49:37] !log ores-staging deleted ores-experimental (backup on ores-staging-02) [19:49:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores-staging/SAL [19:49:49] !log ores-staging create ores-misc-01 (bigram) [19:49:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores-staging/SAL [19:49:53] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Memory issues with ArticleInfo - https://phabricator.wikimedia.org/T170297#3426947 (10MusikAnimal) Relevant commit, specially, see the changes to ArticleInfoController: https://github.com/x-tools/xtools-rebirth/commit/2d8d7f2a54b14bac9fdc6ddca949e04b91ddb061... [19:53:12] * awight looks forward to flying the Spruce Goose [19:53:14] 10Cloud-VPS (Project-requests), 10artificial-intelligence: Provide large disk space to WikiBrain for memory-mapped file - https://phabricator.wikimedia.org/T161554#3426960 (10Halfak) For the separate VMs, I'm imagining something like "wikibrain-atlasify-01" or something like that. [19:58:24] (03Draft1) 10Paladox: Add host ores-staging-02 [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364511 [19:58:26] (03PS2) 10Paladox: Add host ores-staging-02 and ores-misc-01 [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364511 [19:59:21] (03Abandoned) 10Paladox: Add host ores-staging-02 and ores-misc-01 [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364511 (owner: 10Paladox) [20:00:19] If I'm going to do the netcat trip to xfer files across Cloud VPS projects, do I need to set up a security group for the port? [20:00:26] It seems I'm getting a connection timeout [20:04:34] rsync isn't an option? /me reads backscroll [20:05:06] oic [20:05:50] I thought it was possible to run rsync over ssh with no rsync server? [20:06:18] as long as the package is installed [20:06:27] -._o_.- [20:06:44] I just set up a security group that only allows connections from a specific IP and the xfer is now working :D [20:06:59] kk [20:08:27] 10Cloud-Services, 10Cloud-VPS, 10Operations, 10ops-eqiad: rack/setup/install labpuppetmaster100[12].wikimedia.org - https://phabricator.wikimedia.org/T167905#3427136 (10Andrew) btw, these will be Jessie boxes, despite being in the Labs cluster. Thanks. [20:12:01] halfak: If you're already xferring, I shouldn't clean up in that case? [20:14:26] PROBLEM - Puppet errors on tools-exec-1419 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [20:18:52] 10cloud-services-team, 10Wikimedia-Hackathon-2018-Organization, 10Developer-Relations (Jul-Sep 2017): Featured Projects related to Wikimedia Cloud Services and/or Technical Operations? - https://phabricator.wikimedia.org/T170242#3427222 (10Qgil) (edited) 4-5 months. Note that these are not full time interns... [20:24:55] 10Tool-Labs-tools-Xtools, 10Community-Tech, 10JavaScript: Phabricator link on Internal Server Error page is reported as an XSS attack by noscript - https://phabricator.wikimedia.org/T170292#3427289 (10Matthewrbowker) https://github.com/x-tools/xtools-rebirth/pull/39 [20:39:44] 10Tool-Labs-tools-Xtools: Convert bash to use IRC quotes rather than Bugzilla quotes - https://phabricator.wikimedia.org/T170333#3427467 (10Matthewrbowker) [20:50:18] 10Toolforge: Delete tool yunomi - https://phabricator.wikimedia.org/T170070#3427597 (10bd808) p:05Triage>03Low [20:51:28] 10Tool-Labs-tools-XTools, 10Community-Tech: XTools prod environment: unknown "profiler_dump" function - https://phabricator.wikimedia.org/T170233#3427603 (10kaldari) [20:53:05] 10Cloud-Services, 10wikitech.wikimedia.org, 10Operations, 10RESTBase, and 3 others: Fix RESTBase support for wikitech.wikimedia.org - https://phabricator.wikimedia.org/T102178#3427614 (10GWicke) [20:54:25] RECOVERY - Puppet errors on tools-exec-1419 is OK: OK: Less than 1.00% above the threshold [0.0] [21:02:57] 10Toolforge, 10Tools: Performance and OOM issues with Flask tool on Kubernetes - https://phabricator.wikimedia.org/T169954#3427673 (10bd808) [21:22:41] woops. Missed the ping awight -- yeah don't clean up anymore now. [21:22:42] :) [21:22:56] I hadn't started it yet when you asked -- maybe 20 mins later. [21:23:01] But it's all done now. [21:28:33] 10Toolforge, 10Kubernetes: Kubernetes man pages missing from WMF packages - https://phabricator.wikimedia.org/T170346#3427841 (10bd808) [21:37:43] (03Draft1) 10Paladox: Remove ores-compute-01 [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364604 [21:37:46] (03PS2) 10Paladox: Remove ores-compute-01 [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364604 [21:37:49] (03CR) 10Paladox: [V: 032 C: 032] Remove ores-compute-01 [labs/icinga2] - 10https://gerrit.wikimedia.org/r/364604 (owner: 10Paladox) [21:46:59] 10Cloud-VPS (Quota-requests), 10ORES, 10Scoring-platform-team: Decrease quota for ores project from 80GB ram & 40 CPUs - https://phabricator.wikimedia.org/T170348#3427937 (10Halfak) [21:51:58] 10Toolforge, 10Tools: Performance and OOM issues with Flask tool on Kubernetes - https://phabricator.wikimedia.org/T169954#3414359 (10bd808) > Is there are problem with applications that use so much memory. I was under the impression that the service would be able to handle up to 4 GB. By default most webserv... [21:55:51] 10Toolforge, 10Tools: Performance and OOM issues with Flask tool on Kubernetes - https://phabricator.wikimedia.org/T169954#3428013 (10bd808) NOTE: All of the `kubectl` commands are easier if you first run `source <(kubectl completion bash)` in your shell. The completion code installed by this command knows how... [22:00:34] 10cloud-services-team, 10Wikimedia-Hackathon-2018-Organization, 10Developer-Relations (Jul-Sep 2017): Featured Projects related to Wikimedia Cloud Services and/or Technical Operations? - https://phabricator.wikimedia.org/T170242#3428049 (10Toniher) From what I understood from the different meetings last Frid... [22:06:25] 10Cloud-VPS (Quota-requests), 10ORES, 10Scoring-platform-team: Decrease quota for ores project from 80GB ram & 40 CPUs - https://phabricator.wikimedia.org/T170348#3428059 (10Halfak) [22:07:05] 10Cloud-VPS (Quota-requests), 10ORES, 10Scoring-platform-team, 10User-bd808: Request increase quota for ores-staging to 52GB RAM - https://phabricator.wikimedia.org/T169811#3428062 (10Halfak) @Andrew: see {T170348} [22:07:18] 10Cloud-VPS (Quota-requests), 10ORES, 10Scoring-platform-team: Decrease quota for ores project to 80GB ram & 40 CPUs - https://phabricator.wikimedia.org/T170348#3427937 (10Halfak) [22:09:07] 10cloud-services-team, 10Wikimedia-Hackathon-2018-Organization, 10Developer-Relations (Jul-Sep 2017): Featured Projects related to Wikimedia Cloud Services and/or Technical Operations? - https://phabricator.wikimedia.org/T170242#3428075 (10bd808) 300hrs is probably about 2/3 of the time estimate for a GSoC/O... [22:12:49] 10Tool-Labs-tools-XTools: Convert bash to use IRC quotes rather than Bugzilla quotes - https://phabricator.wikimedia.org/T170333#3427467 (10MusikAnimal) It's using them both currently. We can't do the conversation-style quotes or else they won't fit in the footer, so I hand-picked the ones that were both short e... [22:20:05] 10Tool-Labs-tools-XTools: Convert bash to use IRC quotes rather than Bugzilla quotes - https://phabricator.wikimedia.org/T170333#3428134 (10Matthewrbowker) Okay, I must have missed that update, I thought they were all bugzilla quotes. [22:20:24] 10Toolforge: Figure out process for deleting an unused tool - https://phabricator.wikimedia.org/T170355#3428146 (10bd808) [22:20:41] 10Toolforge: Figure out process for deleting an unused tool - https://phabricator.wikimedia.org/T170355#3428135 (10bd808) [22:20:43] 10Cloud-Services, 10Toolforge, 10Tracking: Tools that should get deleted (tracking) - https://phabricator.wikimedia.org/T133777#3428148 (10bd808) [22:22:24] 10Toolforge: Figure out process for deleting an unused tool - https://phabricator.wikimedia.org/T170355#3428135 (10bd808) @valhallasw, @scfc, @yuvipanda, @madhuvishy, @Andrew, @chasemp: what am I forgetting? Update the list in the description please and thank you. [22:40:29] "Toolforge"?? Is that the new name for Tool Labs? [22:40:35] yes [22:41:27] Okay. [23:10:56] 10Toolforge: Figure out process for deleting an unused tool - https://phabricator.wikimedia.org/T170355#3428384 (10Aklapper) For everyone's understanding: This is a complement to T102066 (abandoned tools that nobody wants to take over and/or should die because of technical reasons)? [23:26:17] 10Tool-Labs-tools-XTools, 10Community-Tech: XTools prod environment: unknown "profiler_dump" function - https://phabricator.wikimedia.org/T170233#3428447 (10kaldari) p:05Triage>03High [23:30:03] 10Tool-Labs-tools-XTools, 10Community-Tech: XTools prod environment: unknown "profiler_dump" function - https://phabricator.wikimedia.org/T170233#3423862 (10bd808) * https://github.com/symfony/symfony-standard/issues/982 * https://github.com/symfony/symfony/issues/19039 [23:38:18] 10Tool-Labs-tools-XTools, 10Community-Tech-Sprint, 10JavaScript: Phabricator link on Internal Server Error page is reported as an XSS attack by noscript - https://phabricator.wikimedia.org/T170292#3428485 (10kaldari) [23:48:33] 10Tool-Labs-tools-XTools, 10Community-Tech: EditCounter's "pages created" does not match Pages tool - https://phabricator.wikimedia.org/T169955#3428507 (10kaldari) [23:48:42] 10Toolforge: Figure out process for deleting an unused tool - https://phabricator.wikimedia.org/T170355#3428508 (10bd808) >>! In T170355#3428384, @Aklapper wrote: > For everyone's understanding: This is a complement to T102066 (abandoned tools that nobody wants to take over and/or should die because of technical... [23:51:27] 10Tool-Labs-tools-XTools, 10Community-Tech-Sprint: EditCounter's "pages created" does not match Pages tool - https://phabricator.wikimedia.org/T169955#3428522 (10kaldari) p:05Triage>03High [23:56:49] 10Tool-Labs-tools-XTools, 10Community-Tech: Replace automated edits interface in Edit Counter with link to http://xtools.wmflabs.org/autoedits - https://phabricator.wikimedia.org/T170185#3428537 (10kaldari) [23:57:59] 10Tool-Labs-tools-XTools, 10Community-Tech-Sprint: Replace automated edits interface in Edit Counter with link to http://xtools.wmflabs.org/autoedits - https://phabricator.wikimedia.org/T170185#3422164 (10kaldari) [23:59:34] 10Tool-Labs-tools-XTools, 10Community-Tech-Sprint: Replace automated edits interface in Edit Counter with link to http://xtools.wmflabs.org/autoedits - https://phabricator.wikimedia.org/T170185#3428540 (10kaldari) p:05Triage>03High