[00:21:28] 10Tool-Labs-tools-Pageviews, 07I18n: GRAMMAR doesn't seem to work in https://tools.wmflabs.org/siteviews/url_structure/ - https://phabricator.wikimedia.org/T156576#2980786 (10MusikAnimal) @Amire80 I tried upgrading Intuition and no dice :( The relevant function for Hebrew is [[ https://github.com/Krinkle/intui... [01:49:03] 06Labs, 10Labs-Infrastructure: Newly created instance requires password to ssh in - https://phabricator.wikimedia.org/T156580#2980173 (10bd808) I'm getting prompted for a password when trying to ssh to the instance using my root key as well. I would guess that the initial puppet run failed badly. Typically the... [03:02:52] 06Labs, 10wikitech.wikimedia.org: Enable Special:NovaSecurityGroup again in MediaWiki:Common.js - https://phabricator.wikimedia.org/T156604#2980833 (10scfc) [03:32:20] 06Labs, 10Tool-Labs, 06Tool-Labs-standards-committee: Create a wall for tools migration to trusty - https://phabricator.wikimedia.org/T156386#2980852 (10bd808) I'm getting something started at https://tools.wmflabs.org/precise-tools/. @zhuyifei1999 and myself are the initial maintainers. Code is at https://p... [05:03:53] 10Tool-Labs-tools-Pageviews, 07I18n: massviews-category-description lego for "category" - https://phabricator.wikimedia.org/T146973#2980865 (10MusikAnimal) @Nikerabbit Hey, so I still need to fix this issue, but I'm also having a similar issue for another feature I'm working on. Perhaps you could help... I wa... [05:33:54] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2980877 (10scfc) [05:34:03] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2980891 (10scfc) p:05Triage>03Normal [05:35:30] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2980877 (10scfc) [05:36:30] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2980893 (10scfc) [05:37:19] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2980877 (10scfc) (Copied and pasted verbatim from a new try to avoid any confusion.) [06:18:24] 06Labs: Bot (written in Mono, running on Labs) has lags during stats reading through API - https://phabricator.wikimedia.org/T147109#2681442 (10scfc) My (default) assumption is that this is due to Mono at the start of each web request reading some files from the tool's home directory (certificates?) that are on... [06:25:59] 06Labs, 10Tool-Labs: Bot (written in Mono, running on Labs) has lags during stats reading through API - https://phabricator.wikimedia.org/T147109#2980930 (10scfc) p:05Triage>03Low [06:27:33] 06Labs, 10Tool-Labs, 06Tool-Labs-standards-committee: Create a wall for tools migration to trusty - https://phabricator.wikimedia.org/T156386#2980933 (10bd808) The basic list is working. Now it needs: * trim off `tools.` prefix from unix account names * link each tool to `https://tools.wmflabs.org/?tool=$NAM... [06:38:08] RECOVERY - Free space - all mounts on tools-exec-1221 is OK: OK: tools.tools-exec-1221.diskspace._public_dumps.byte_percentfree (No valid datapoints found) [06:38:40] PROBLEM - Puppet run on tools-exec-1417 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:12:30] !log tools.precise-tools $ www/python/venv/bin/pip install -U pip (pip 1.5.4 => 9.0.1) [07:12:31] Unknown project "tools.precise-tools" [07:14:56] !log tools.precise-tools $ www/python/venv/bin/pip install -U pip (pip 1.5.4 => 9.0.1) [07:14:56] Unknown project "tools.precise-tools" [07:15:07] bd808: ^ o.O [07:18:37] RECOVERY - Puppet run on tools-exec-1417 is OK: OK: Less than 1.00% above the threshold [0.0] [07:43:23] 06Labs, 10Tool-Labs, 06Tool-Labs-standards-committee: Create a wall for tools migration to trusty - https://phabricator.wikimedia.org/T156386#2980991 (10zhuyifei1999) > * trim off tools. prefix from unix account names Why is `root` in the list? If it's unintentional, filter out everything that dos not start... [07:57:42] PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [08:25:24] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2980877 (10zhuyifei1999) I got a somehow-more-verbose error: ``` tools.precise-tools@tools-bastion-02:~$ webservice --backend=kubernetes python2 shell Traceback (most recent... [08:27:56] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2981042 (10zhuyifei1999) Happens for older tools as well, eg. `video2commons-test`. CC @yuvipanda [08:37:41] RECOVERY - Puppet run on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0] [08:44:41] 06Labs, 10Tool-Labs: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2981107 (10zhuyifei1999) Idk why the line numbers are different, but it's probably [[https://github.com/wikimedia/operations-software-tools-webservice/blob/faa20dd30cfd77c44c... [08:49:58] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2981154 (10zhuyifei1999) p:05Normal>03High a:03zhuyifei1999 Testing Popen on `/bin/ls` works fine, but the original fails, so I suspected `/usr/lo... [08:57:36] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: "webservice shell" fails with "No such file or directory" (with php5.6) - https://phabricator.wikimedia.org/T156605#2981206 (10zhuyifei1999) kubectl is `/usr/bin/kubectl` is across all 3 bastions, 02, 03 & 05. [09:09:06] !log tools.precise-tools migrating to k8s [09:09:06] Unknown project "tools.precise-tools" [10:12:45] 06Labs, 10Tool-Labs, 06Tool-Labs-standards-committee: Create a wall for tools migration to trusty - https://phabricator.wikimedia.org/T156386#2981404 (10zhuyifei1999) k8s reduced time to 1 second startup time, 01:10 response time. [10:16:25] zhuyifei1999_: ^ "create a wall" - but who's going to pay for it? [10:17:49] Mexico? :P [10:24:09] PROBLEM - Free space - all mounts on tools-exec-1221 is CRITICAL: CRITICAL: tools.tools-exec-1221.diskspace._public_dumps.byte_percentfree (No valid datapoints found)tools.tools-exec-1221.diskspace.root.byte_percentfree (<55.56%) [10:26:10] !log tools.precise-tools I got a optimization patch ready to be pushed, and since idk why I don't have write access to the repo, I'll put the diff inside the home dir and git apply it [10:26:10] Unknown project "tools.precise-tools" [10:28:05] !log tools.precise-tools $ git apply ~/169b40bff497eff780df28ba010cdf3564e62cdd.patch; webservice restart [10:28:05] Unknown project "tools.precise-tools" [10:30:23] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: k8s webservice restart failure with `ValueError: get() more than one object; use filter` - https://phabricator.wikimedia.org/T156626#2981472 (10zhuyifei1999) [10:32:38] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: k8s webservice restart failure with `ValueError: get() more than one object; use filter` - https://phabricator.wikimedia.org/T156626#2981485 (10zhuyifei1999) Workaround: ``` tools.precise-tools@tools-bastion-05:~/www/python/src$ webservice stop Stopping webservice too... [10:51:38] 06Labs, 10Tool-Labs, 10Prod-Kubernetes, 10Tools-Kubernetes, 07kubernetes: Kubernetes discussion at Dev Summit - https://phabricator.wikimedia.org/T153943#2981513 (10akosiaris) Has this meeting occured after all ? Meeting nodes ? [10:57:27] 06Labs, 10Analytics-Tech-community-metrics: http://korma.wmflabs.org/ got erased - https://phabricator.wikimedia.org/T156253#2981529 (10Aklapper) a:03Aklapper [11:10:56] 06Labs, 10Tool-Labs, 06Tool-Labs-standards-committee: Create a wall for tools migration to trusty - https://phabricator.wikimedia.org/T156386#2981559 (10zhuyifei1999) @bd808 Idk why I can't write to that repo, but I applied P4832 (saved in `~/169b40bff497eff780df28ba010cdf3564e62cdd.patch`) that decreased th... [11:17:16] 10Tool-Labs-tools-LTA-Knowledgebase: Add continuous integration - https://phabricator.wikimedia.org/T155944#2981577 (10hashar) If you code is on Github (assuming it is https://github.com/lta-knowledgebase/lta-tool ) there are several third parties CI system that hook into Github workflow: https://github.com/int... [11:29:29] 10Tool-Labs-tools-LTA-Knowledgebase: Add continuous integration - https://phabricator.wikimedia.org/T155944#2959711 (10Samtar) Just to note, I'm currently testing CircleCI - at the moment it is only linting so some unit tests need the be written.. [11:51:12] 06Labs, 10Tool-Labs, 06Tool-Labs-standards-committee: Create a wall for tools migration to trusty - https://phabricator.wikimedia.org/T156386#2981624 (10zhuyifei1999) Applied P4833 instead, now about 34 seconds (3 tests: 36, 32, 33). The root job entry is `continuous:tools-exec-1220.tools.eqiad.wmflabs:root... [11:56:55] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: k8s webservice restart failure with `ValueError: get() more than one object; use filter` - https://phabricator.wikimedia.org/T156626#2981631 (10zhuyifei1999) Seems to be related to the new webservice starting up too early, and the query selector find both the stopping... [11:59:21] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs: Make a nag system to email maintainers of tools still running on precise grid hosts - https://phabricator.wikimedia.org/T149214#2981632 (10zhuyifei1999) [12:53:17] bd808: oh thanks for adding me as maintainer (I think I somehow forgot to say that :)) [13:26:35] 10Tool-Labs-tools-Database-Queries, 06Project-Admins: Archive Tool-Labs-tools-Database-Queries project - https://phabricator.wikimedia.org/T107699#2981752 (10Aklapper) 05Open>03declined No reply by @Bugreporter. Hence declining. [13:28:06] I'm sure we all miss GlusterFS https://fosdem.org/2017/schedule/track/software_defined_storage/ [13:49:33] 06Labs, 10Tool-Labs: Bot (written in Mono, running on Labs) has lags during stats reading through API - https://phabricator.wikimedia.org/T147109#2981777 (10MaxBioHazard) p:05Low>03Normal All these bots runs under mbh account. In last 3 months this bot works unstable, occasionally don't save pages (see ht... [13:56:49] 06Labs, 10Tool-Labs: Bot (written in Mono, running on Labs) has lags during stats reading through API - https://phabricator.wikimedia.org/T147109#2981782 (10zhuyifei1999) >>! In T147109#2981777, @MaxBioHazard wrote: > There is important factor, that I don't mention earlier: this problem occurs when process run... [14:00:53] 06Labs, 10Tool-Labs: Bot (written in Mono, running on Labs) has lags during stats reading through API - https://phabricator.wikimedia.org/T147109#2981790 (10MaxBioHazard) Without jsub. I never specify release. I run bot directly with jsub now to test. [14:21:12] 06Labs, 10Tool-Labs: Bot (written in Mono, running on Labs) has lags during stats reading through API - https://phabricator.wikimedia.org/T147109#2981832 (10MaxBioHazard) Problem occurs, when bot runned directly (not from cron) with jsub. [14:58:41] PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [15:15:50] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Labs instance ci-jessie-wikimedia-498353 can not be deleted - https://phabricator.wikimedia.org/T156636#2981991 (10hashar) [15:24:53] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Labs instance ci-jessie-wikimedia-498353 can not be deleted - https://phabricator.wikimedia.org/T156636#2982038 (10Paladox) I believe this may be a bug, i saw some updates related to deleting node pool instances but we doint have any of... [15:37:37] The namespaces for Goan Konkani were updated last year https://phabricator.wikimedia.org/T129315 but the X! Tools editcounter still shows the old namespaces https://tools.wmflabs.org/xtools-ec/?user=The%20Discoverer&project=gom.wikipedia.org [15:38:31] TheDiscoverer: afaik, xtools is being rewritten [15:38:42] RECOVERY - Puppet run on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0] [15:39:11] zhuyifei1999_: ok.. so no point updating it now? [15:40:08] I'm not sure. You might want to file a ticket on phabricator [15:40:51] zhuyifei1999_: Phabricator or https://github.com/x-Tools/xtools/issues? [15:41:17] zhuyifei1999_: https://github.com/x-Tools/xtools/issues [15:41:37] either I guess. I'm not really sure how the tool operates [15:42:24] musikanimal: are you around? [15:42:37] Cyberpower678: are you around? [15:43:20] ? [15:44:30] Cyberpower678: The namespaces for Goan Konkani were updated last year https://phabricator.wikimedia.org/T129315 but the X! Tools editcounter still shows the old namespaces https://tools.wmflabs.org/xtools-ec/?user=The%20Discoverer&project=gom.wikipedia.org [15:44:41] Matthew_: ^ [15:44:50] TheDiscoverer: xtools is being rewritten. [15:45:03] Cyberpower678: so no point updating it now? [15:45:29] I'll let Matthew_ decide that. [15:45:41] Cyberpower678: Ok.. thanks [15:45:58] TheDiscoverer: do you have a Phabricator account? If so, could you file a task? [15:46:22] If not, Cyberpower678 could you? I'll dig into it when I'm in front of my computer. [15:46:55] Matthew_: I have an account and will file a task [15:47:21] Thank you. [15:47:39] Matthew_: Thanks to you :) good bye and good day! [15:47:58] Cyberpower678: Thanks [15:48:03] zhuyifei1999_: Thanks [15:49:33] np [15:49:37] np [15:51:05] TheDiscoverer: make sure to tag the task. And send me a link please. [15:51:21] Matthew_: doing it right now [15:51:54] Cool. [15:52:26] 10Tool-Labs-tools-Xtools: Update Goan Konkani namespace names in X!'s Tools edit counter - https://phabricator.wikimedia.org/T156641#2982134 (10The_Discoverer) [15:53:11] Thank you. [15:53:52] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: keystone admin api occasionally refuses connection - https://phabricator.wikimedia.org/T156337#2982150 (10Andrew) [15:54:02] 10Tool-Labs-tools-Xtools: Update Goan Konkani namespace names in X!'s Tools edit counter - https://phabricator.wikimedia.org/T156641#2982151 (10Matthewrbowker) p:05Triage>03Low [15:54:42] PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [0.0] [16:04:41] RECOVERY - Puppet run on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:18:36] zhuyifei1999_: I had added you to the project ACLs, but not the ACLs for the diffusion project. That seems to be fixed now. [16:18:50] ok thx [16:18:53] so you should be able to push your patch(es) [16:18:56] will work on it [16:19:34] shall I create new branches and merge after test or commit directly to master? [16:20:04] zhuyifei1999_: commit to master is fine I think. we can always revert things that don't work out. [16:20:13] k [16:20:36] I try to not get to fancy with tool workflows :) [16:20:52] *too [16:21:23] btw, the !log thing seems broken. any ideas? [16:24:38] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Labs instance ci-jessie-wikimedia-498353 can not be deleted - https://phabricator.wikimedia.org/T156636#2982246 (10Paladox) @hashar a little tool http://markmail.org/message/gygnjvdy3njwfrtg https://lists.launchpad.net/openstack/msg1462... [16:26:00] zhuyifei1999_: it would be something in the LDAP lookups. I'll peek at the logs for stashbot when I get a chance and see if I can figure it out. There was some weird lag when I first created the tool before I could become. [16:26:22] k [16:27:08] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Labs instance ci-jessie-wikimedia-498353 can not be deleted - https://phabricator.wikimedia.org/T156636#2982263 (10Paladox) @hashar i think this is the fix https://review.openstack.org/#/c/230448/ https://bugs.launchpad.net/tripleo/+bug/... [16:32:14] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Labs instance ci-jessie-wikimedia-498353 can not be deleted - https://phabricator.wikimedia.org/T156636#2982283 (10Paladox) Theres also this fix https://github.com/openstack-infra/nodepool/commit/2cc94909017d40f48742de99ac7eb2a3c2ee1234 [16:32:22] wow now it generates in like 17/18 seconds. time to add features :) [16:33:55] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Labs instance ci-jessie-wikimedia-498353 can not be deleted - https://phabricator.wikimedia.org/T156636#2982309 (10Paladox) ah ha, this looks like the fix https://github.com/openstack-infra/nodepool/commit/8b76b3dca03bd2fbec1b1723ab4c991... [16:37:57] 06Labs, 10Labs-Infrastructure: Newly created instance requires password to ssh in - https://phabricator.wikimedia.org/T156580#2982337 (10Nemo_bis) I already deleted and recreated the instance, but the result was the same. I have no access to Horizon, so I can't check logs there. [16:46:04] RECOVERY - Puppet run on tools-services-02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:54:12] (03PS8) 10Paladox: Connect wikibugs to irc over ssl [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/328663 (https://phabricator.wikimedia.org/T141089) [16:54:15] 06Labs, 06Operations, 10netops: asw-c2-eqiad reboots & fdb_mac_entry_mc_set() issues - https://phabricator.wikimedia.org/T155875#2982398 (10faidon) p:05Unbreak!>03High Looks stable for now, lowering priority. [17:08:30] bd808: shall I store status in redis? [17:09:03] easier to work with than in a sql database [17:09:10] zhuyifei1999_: that was my initial thought, yes [17:09:20] k [17:09:50] mysql is not a great cache layer, better than files on NFS for sure, but not great [17:15:47] 06Labs, 10Labs-Infrastructure: Newly created instance requires password to ssh in - https://phabricator.wikimedia.org/T156580#2982453 (10bd808) >>! In T156580#2982337, @Nemo_bis wrote: > I have no access to Horizon, so I can't check logs there. This is probably a tangent, but is your blocker for using Horizon... [17:59:13] 06Labs, 10Horizon, 07Upstream: Clicking on a project name in Identity logs you out of Horizon - https://phabricator.wikimedia.org/T138809#2411037 (10Andrew) This is fixed in mitaka -- in that version, the project names are only links for users that can actually view the appropriate pages. [18:03:24] 06Labs, 10Tool-Labs, 10Continuous-Integration-Config: operations/software/tools-webservice (and operations/software/tools-manifest?) do not run Debian tests - https://phabricator.wikimedia.org/T156651#2982620 (10scfc) [18:05:25] !log tools.precise-tools test [18:05:25] Unknown project "tools.precise-tools" [18:05:52] such detail in my logging :/ "2017-01-30T18:05:25Z Stashbot WARNING : Invalid project tools.precise-tools" [18:09:31] zhuyifei1999_: I think the problem may be one of the total number of service groups exceeding the anon LDAP query limit :/ [18:09:55] uh [18:10:04] there is indeed a limit there but I thought it was clownishly high but that may have changed w/ the openldap switch [18:11:17] 10Tool-Labs-tools-Other, 15User-bd808: Stashbot not seeing tools.precise-tools as a valid project - https://phabricator.wikimedia.org/T156652#2982646 (10bd808) [18:15:58] 10Tool-Labs-tools-Other, 15User-bd808: Stashbot not seeing tools.precise-tools as a valid project - https://phabricator.wikimedia.org/T156652#2982675 (10bd808) Possibly related: ``` $ ldapsearch -xLLL '(objectclass=groupofnames)'|grep precise dn: cn=tools.test-lighttpd-precise,ou=servicegroups,dc=wikimedia,dc=... [18:20:12] 06Labs, 10Tool-Labs, 10Continuous-Integration-Config: operations/software/tools-webservice (and operations/software/tools-manifest?) do not run Debian tests - https://phabricator.wikimedia.org/T156651#2982620 (10zhuyifei1999) I wasn't whitelisted at the time for CI, but now I am :) One test was triggered, `t... [18:22:43] 06Labs: horizon puppet panel vs. hiera parsing - https://phabricator.wikimedia.org/T156655#2982714 (10Andrew) [18:26:06] 10Tool-Labs-tools-Other, 15User-bd808: Stashbot not seeing tools.precise-tools as a valid project - https://phabricator.wikimedia.org/T156652#2982733 (10bd808) The [[https://github.com/wikimedia/labs-tools-stashbot/blob/master/stashbot/sal.py#L197-L212|real LDAP query and logic]] is basically: ``` $ ldapsearch... [18:38:55] !log tools.precise-tools test [18:38:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.precise-tools/SAL [18:57:14] 10Tool-Labs-tools-Other, 15User-bd808: Stashbot not seeing tools.precise-tools as a valid project - https://phabricator.wikimedia.org/T156652#2982856 (10bd808) I added a bit of live logging to stashbot to track this down and restarted the bot. That restart seems to have 'fixed' the problem which is not entirel... [19:12:40] 06Labs, 10Horizon, 06Operations, 07Puppet: Puppet tab in Horizon unusably slow - https://phabricator.wikimedia.org/T149589#2757207 (10greg) UBN! for over a week? [19:17:48] 06Labs, 10Horizon, 06Operations, 07Puppet: Puppet tab in Horizon unusably slow - https://phabricator.wikimedia.org/T149589#2982953 (10Andrew) p:05Unbreak!>03Normal [19:23:36] 06Labs: labs (labvirt/labservices) in "misc" cluster - https://phabricator.wikimedia.org/T123000#2982985 (10fgiunchedi) [19:26:03] 06Labs: labtest in "misc" cluster - https://phabricator.wikimedia.org/T123000#2983007 (10fgiunchedi) [20:21:56] hi [20:22:06] webservice --backend=kubernetes python shell is not working for me [20:22:12] I'm trying to follow https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web/Kubernetes#python_.28uwsgi_.2B_python3.4.29 [20:23:52] yuvipanda: ? ^ [20:24:12] https://paste.fedoraproject.org/541265/80784714/ is the actual error [20:24:42] * legoktm uses py2 for now [20:24:53] legoktm: try now? [20:25:00] ok, one sec [20:25:09] !log tools sudo ln -s /usr/bin/kubectl /usr/local/bin/kubectl to temporarily fix webservice shell not working [20:25:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [20:25:24] yuvipanda: working, thanks :D [20:25:28] legoktm: np [20:28:36] yuvipanda: also to make sure I understand k8s properly, my webservice will have full filesystem access right? but it somehow doesn't use nfs? [20:28:59] legoktm: it does use NFS [20:29:26] ah, ok [20:29:43] I thought k8s had something to do with reducing depenency upon nfs though? [20:29:59] maybe I'm confusing things [20:30:14] legoktm: the k8s infrastructure does not depend on NFS [20:30:28] legoktm: the PaaS (whenever that happens) will be completely NFS free... [20:30:35] ah [20:30:38] ok, thanks :D [20:31:42] !screen [20:31:42] script /dev/null [20:35:07] 06Labs, 10Tool-Labs: Delete http://wikimedia-ui.wmflabs.org/ instance - https://phabricator.wikimedia.org/T156679#2983368 (10Volker_E) [20:35:34] 06Labs, 10Tool-Labs: Delete http://wikimedia-ui.wmflabs.org/ instance - https://phabricator.wikimedia.org/T156679#2983381 (10Volker_E) [20:41:43] yuvipanda: https://phabricator.wikimedia.org/T156605 [20:42:29] thanks zhuyifei1999_ [20:42:31] just catching up - been an, uh, intense weekend [20:42:39] k [20:44:40] zhuyifei1999_: my symlink should fix it for now, I've merged your patch. I'll try to roll it out laster today [20:44:49] k [20:46:25] when you have time, could you look at https://phabricator.wikimedia.org/T156626 ? [20:47:24] zhuyifei1999_: will do! restart seems to 'work' even if it's failing with that message rn [20:47:34] ok [21:33:19] 06Labs: Request increased quota for etytree labs project - https://phabricator.wikimedia.org/T156021#2983597 (10Epantaleo) Thanks! [22:41:15] yuvipanda: I bookmarked this from reddit this weekend -- https://github.com/jakevdp/WhirlwindTourOfPython [22:41:39] maybe looking at that will finally make notebooks 'click' for me [22:59:17] 06Labs, 10Wikimedia-Stream: Provide useful diffs to high-volume consumers of recent changes - https://phabricator.wikimedia.org/T100082#2983911 (10Krinkle) [23:31:20] if any OAuth admins are around, c5b79da447beda86fe61e03a6127910d pls [23:31:31] * AmandaNP prays this one will be the final one