[00:21:54] 06Labs, 10Tool-Labs, 05Prometheus-metrics-monitoring: Remove prometheus-node-exporter from Tool Labs apt repository (if no longer needed) - https://phabricator.wikimedia.org/T158824#3048591 (10scfc) [00:29:02] madhuvishy, yuvipanda: i keep getting 502 Bad Gateway after clicking "Start My Server" at https://paws.wmflabs.org/paws/hub/home ... [00:29:14] HaeB: looking [00:29:36] (works fine for me) [00:29:44] 06Labs, 10Tool-Labs: jmail is not puppetized - https://phabricator.wikimedia.org/T158722#3048613 (10scfc) Actually, there seems to be a bug in that script: The first run of `cat >>/tmp/jmail.log` after a reboot creates `/tmp/jmail.log` being owned by that particular user (mostly `drtrigonbot`) and subsequent i... [00:29:51] HaeB: what's your wiki username? [00:30:33] ..or also when trying to access eg https://paws.wmflabs.org/paws/user/Tbayer_%28WMF%29 directly [00:30:41] ^ this one [00:41:58] HaeB: still debugging btw [00:42:07] thanks! [00:43:17] HaeB: try now? [00:43:46] 06Labs, 10Tool-Labs: jmail is not puppetized - https://phabricator.wikimedia.org/T158722#3048640 (10scfc) a:03scfc I believe the part about `email="/data/project/.system/store/mail/.deliver.$(date +%s).$$"` should also be replaced by files created by `mktemp` in the tool's home directory. I wasn't aware of... [00:44:06] yuvipanda: still the same :( [00:44:13] (in two different browsers) [00:44:48] HaeB: can you try a user without special chars in it? [00:44:53] I'll be so pissed if that's it, but should check [00:45:08] ok.. [00:45:40] works for User:HaeB [00:46:21] HaeB: sigh. is that something you can live with for a day or two? [00:46:37] I'm in the middle of setting up 'paws-test', which has much newer code and is all around better in every way :) [00:48:35] yuvipanda: OK, i can use that account for a bit (thankfully the notebook i was working on is still accessible under paws-public, so I'll just copy it over) [00:48:54] no worries, and looking forward to the shiny new version ;) [00:49:19] HaeB: :D it is exactly the same, just less hacky in the background :) [00:50:05] shiny under the hood, then ;) [01:02:26] 06Labs, 10Tool-Labs: jmail is not puppetized - https://phabricator.wikimedia.org/T158722#3048662 (10bd808) It would be swell to have this rewritten in python and added to the `labs/toollabs.git` repo as part of the jobutils deb. [01:45:11] 06Labs, 10Tool-Labs-tools-Other, 06Community-Tech-Tool-Labs, 06Developer-Relations, and 3 others: Create an authoritative and well promoted catalog of Wikimedia tools - https://phabricator.wikimedia.org/T115650#3048688 (10Ricordisamoa) [01:45:39] 06Labs, 10Tool-Labs-tools-Other, 06Community-Tech-Tool-Labs, 06Developer-Relations, and 3 others: Create an authoritative and well promoted catalog of Wikimedia tools - https://phabricator.wikimedia.org/T115650#1729232 (10Ricordisamoa) [01:57:39] 06Labs, 10MediaWiki-extensions-MolHandler: Replace or delete labs instance mediahandler-tests-static - https://phabricator.wikimedia.org/T157990#3048703 (10Rillke) Contents have been migrated to the new instance `static2`. Old one (`mediahandler-tests-static`) still in place in case forgot something; can be sa... [02:02:21] 06Labs, 10MediaWiki-extensions-MolHandler: Replace or delete labs instance mediahandler-tests-static - https://phabricator.wikimedia.org/T157990#3048708 (10Rillke) 05Open>03Resolved [02:02:24] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3048709 (10Rillke) [02:09:52] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3048715 (10Rillke) [05:53:11] 06Labs, 10Horizon: Shutting down an instance via Horizon does not work - https://phabricator.wikimedia.org/T158796#3048860 (10zhuyifei1999) [06:26:06] 06Labs, 10video2commons: Cleanup /data/project on Labs project video - https://phabricator.wikimedia.org/T158831#3048870 (10madhuvishy) [06:26:34] 06Labs, 10video2commons: Cleanup /data/project on Labs project video - https://phabricator.wikimedia.org/T158831#3048882 (10madhuvishy) Do feel free to ping for any help! [06:39:35] PROBLEM - Puppet run on tools-exec-1414 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:14:32] RECOVERY - Puppet run on tools-exec-1414 is OK: OK: Less than 1.00% above the threshold [0.0] [08:05:29] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/339354 (owner: 10L10n-bot) [08:27:51] 06Labs, 10Horizon: Shutting down an instance via Horizon does not work - https://phabricator.wikimedia.org/T158796#3048990 (10Paladox) But when you shut it down how is it meant to start up? [08:41:55] 06Labs, 10Labs-Infrastructure, 10wikitech.wikimedia.org: add useful content to Wikitech:Shell - https://phabricator.wikimedia.org/T56697#3049015 (10MarcoAurelio) [08:41:57] 06Labs, 10WikimediaMessages, 10wikitech.wikimedia.org, 13Patch-For-Review, 05WMF-deploy-2017-02-28_(1.29.0-wmf.14): Group messages for Wikitech - https://phabricator.wikimedia.org/T158417#3049013 (10MarcoAurelio) 05Open>03Resolved Will be live on #wmf-deploy-2017-02-28_1.29.0-wmf.14. Thinking about d... [09:51:49] 10Wikibugs: Wikibugs always points to the creation of the task - https://phabricator.wikimedia.org/T158704#3044585 (10tom29739) It only happens sometimes, for example this message a few hours before the task in the description links to the correct task: ` Labs, Tool-Labs, Tool-Labs-tools-Database-Queri... [13:29:52] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations: labsdb1006/1007 (postgresql) maintenance - https://phabricator.wikimedia.org/T157359#3049509 (10chasemp) I think @akosiaris is the only person with a lot of context for this setup. If I understand correctly the users are https://wikitech.wikimedia.org/wik... [13:42:05] 06Labs, 10Tool-Labs: jmail is not puppetized - https://phabricator.wikimedia.org/T158722#3045246 (10chasemp) Email as an API seems like an idea fraught with peril to me. Can we remove this entirely? What's the must-have use case here? [13:44:04] 06Labs, 10PAWS: paws returns 502 bad gateway - https://phabricator.wikimedia.org/T158685#3049545 (10chasemp) 05Open>03Resolved It has been up since. For now I am resolving this to be revisited if we can narrow down the trigger and/or mitigate. [14:09:24] 06Labs, 10video2commons: Cleanup /data/project on Labs project video - https://phabricator.wikimedia.org/T158831#3049584 (10zhuyifei1999) [14:09:27] 06Labs, 10video2commons: Can we delete these old files on video project? - https://phabricator.wikimedia.org/T158786#3049587 (10zhuyifei1999) [14:09:36] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations: labsdb1006/1007 (postgresql) maintenance - https://phabricator.wikimedia.org/T157359#3049588 (10dschwen) Yes, it is used by me! I'm pulling data from that server for the client-side rendered tiles and 3D buildings in WikiMiniAtlas. [14:32:32] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations: labsdb1006/1007 (postgresql) maintenance - https://phabricator.wikimedia.org/T157359#3049645 (10chasemp) >>! In T157359#3049588, @dschwen wrote: > Yes, it is used by me! I'm pulling data from that server for the client-side rendered tiles and 3D buildings... [14:35:09] 06Labs, 10video2commons: Can we delete these old files on video project? - https://phabricator.wikimedia.org/T158786#3049648 (10scfc) [14:35:39] ^ can we, yes, will we, tbd [14:42:51] Zppix: ... [14:43:16] zhuyifei1999_: yes? [14:43:45] I was referring to your "^ can we, yes, will we, tbd" [14:44:12] zhuyifei1999_: oh i thought i broke something or whatever [15:03:47] 06Labs, 10Tool-Labs, 06Tool-Labs-standards-committee, 10Tool-Labs-tools-Quentinv57's-tools: Consider setting a Diffusion mirror of GitHub's Quentinv57's-tools - https://phabricator.wikimedia.org/T158405#3049753 (10zhuyifei1999) a:03Cyberpower678 (Assigning to tool maintainer) [15:38:52] andrewbogott hi, why dosent df -h show how much i really have in terms of storage [15:38:54] i see /dev/vda3 19G 15G 2.7G 85% / [15:39:07] but on my instance it should have 80gb total storage. [15:44:11] paladox: instances don't partition all of their allocated space by default [15:44:18] oh [15:44:25] There's a puppet class to create an additional volume if you need the space [15:44:41] oh ah. Could i do that please? [15:44:49] I'm looking, just a second [15:45:04] ok thanks [15:45:20] role::labs::lvm::srv iirc [15:45:27] yep, that's the one :) [15:45:30] oh [15:45:31] :) [15:45:34] * paladox looks [15:46:31] so i add that role, will i need to do anything else after adding the role. [15:46:34] also thanks. [15:46:41] run puppet [15:46:58] ok [15:46:58] puppet updates automatically every 20 minutes, you can force an update with 'sudo puppet agent -tv' [15:47:00] thanks [15:51:19] andrewbogott so i can only access the extended storage through /srv? Is there a way to do it for /? [15:51:23] please [15:52:25] Only /srv . Your OS is on / so you def don't want to mount something on top of that. [15:52:54] oh [15:52:56] ok [15:53:13] So i can access the extended storage on for example /var or /home ? [15:54:18] Well, again, you already have things you need on /var. You can of course symlink as.much as you like. [15:54:28] paladox: there is https://wikitech.wikimedia.org/wiki/Help:Resize_root_partition but be careful [15:54:43] oh [15:54:46] thanks [15:55:47] i.e. make backups, if it fails without backup uh oh [15:56:15] oh. [15:56:20] How do i make a backup? [15:57:35] I mean backup any important data. if it's a throwaway test instance then you can just destroy and recreate it [15:59:15] resizing partitions aren't exactly safe afaik, (though I never encountered issued with that) [15:59:18] ok, well it only has gerrit, jenkins and zuul (im doing testing) Nothing important though. [16:01:36] it shows /dev/vda4 40894464 167770111 126875648 60.5G Linux LVM as having the most [16:01:40] storage wise. [16:10:06] Usually it's better to keep whatever crazy-growing-storage you have someplace other than / so you don't break your whole OS when you fill it up :) [16:10:50] ok [16:22:12] 06Labs, 10Horizon: Shutting down an instance via Horizon does not work - https://phabricator.wikimedia.org/T158796#3049987 (10Andrew) 05Open>03Invalid I'm not able to reproduce this. It's most likely an intermittent issue, possibly T156337. If you're able to make it happen reliably then please re-open.... [16:42:11] andrewbogott is there a way we can get it so when we create instances it will give the full storage, or a bit more storage then it currently does? Or is it set like this to conserve storage on the hosts. [16:42:30] It's the way it is on purpose. Most users never use even the originally allocated 20G [16:43:13] oh ok [16:43:45] the mount role is made for this purpose (access the additional quota capacity) [16:53:32] oh [17:26:28] 06Labs, 10video2commons: Can we delete these old files on video project? - https://phabricator.wikimedia.org/T158786#3050294 (10madhuvishy) From my ticket closed as duplicate of this one - video2commons (on the video Labs project) is using 2.034TB/5TB of available NFS storage. This is a lot and should be clean... [17:43:53] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Simon-kempf was created, changed by Simon-kempf link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Simon-kempf edit summary: Created page with "{{Tools Access Request |Justification=Hello ! I'm a founder of http://skygen.alwaysdata.net/ a community with softwares to edit wikis and bots. We need a server to run Revoluc..." [17:44:07] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Nodepool quota bump - https://phabricator.wikimedia.org/T158320#3050369 (10chasemp) Currently: ```+----------------------+--------------+ | Field | Value | +----------------------+--------------+ | cores... [17:45:21] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Simon-kempf was modified, changed by Simon-kempf link https://wikitech.wikimedia.org/w/index.php?diff=1576469 edit summary: [17:47:42] 06Labs, 07Tracking: Existing Labs project quota increase requests (Tracking) - https://phabricator.wikimedia.org/T140904#3050407 (10Andrew) [17:47:45] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Nodepool quota bump - https://phabricator.wikimedia.org/T158320#3050404 (10Andrew) 05Open>03Resolved a:03Andrew done! [18:13:01] 06Labs: adding project to nfs-mounts.yaml does not create directories - https://phabricator.wikimedia.org/T158883#3050493 (10chasemp) [18:13:09] 06Labs: adding project to nfs-mounts.yaml does not create directories - https://phabricator.wikimedia.org/T158883#3050505 (10chasemp) p:05Triage>03Normal [18:55:09] !log ores restarted uwsgi-ores and precached on ores-web-03 (memory issues) [18:55:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL [18:55:20] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations: labsdb1006/1007 (postgresql) maintenance - https://phabricator.wikimedia.org/T157359#3050688 (10MaxSem) Yes, I'm using it (from tool labs, actually), but feel free to take it offline any time. [18:58:00] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations: labsdb1006/1007 (postgresql) maintenance - https://phabricator.wikimedia.org/T157359#3050709 (10chasemp) Thanks @maxsem AFAIK we can take this down with proper notice (and really we must). My thinking is to send a general 1 week notice to labs-announce a... [18:58:24] madhuvishy: well the 2TB videos aren't really from v2c, but from the old wikimania 2014 videos [18:58:46] hopefully matanya will look into them [18:59:35] v2c already has an auto cleanup strategy built in (remove old files after two months) [19:01:10] 06Labs, 10Tool-Labs: jmail is not puppetized - https://phabricator.wikimedia.org/T158722#3050715 (10bd808) jmail is documented on wiki: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Processing_email_programatically I don't know if this was something carried over from Toolserver or a local invention. [19:01:21] thanks for notifying though [19:03:27] zhuyifei1999_: ah okay - but yes it's in definite need of cleanup [19:05:41] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations: labsdb1006/1007 (postgresql) maintenance - https://phabricator.wikimedia.org/T157359#3050727 (10dschwen) @chasemp yes a few days downtime should be OK. I have a cache layer that should serve most of the requests. [19:10:00] 06Labs, 10Labs-Infrastructure, 10DBA, 06Operations: labsdb1006/1007 (postgresql) maintenance - https://phabricator.wikimedia.org/T157359#3050755 (10jcrespo) > @jcrespo how does that sound? Good to me. [19:26:05] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Simon-kempf was modified, changed by BryanDavis link https://wikitech.wikimedia.org/w/index.php?diff=1577657 edit summary: [19:36:25] 10Tool-Labs-tools-Pageviews: Massviews export data is incorrect (both CSV and JSON) - https://phabricator.wikimedia.org/T158505#3050892 (10MusikAnimal) I could add monthly stats, weekly I'd have to compute client-side which I guess I could also do... but I should then add it to the main Pageviews app too. I'll c... [19:38:55] 10Tool-Labs-tools-Pageviews: Add daily date type to all Pageviews apps - https://phabricator.wikimedia.org/T158900#3050926 (10MusikAnimal) [19:39:50] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Nodepool quota bump - https://phabricator.wikimedia.org/T158320#3050943 (10hashar) And Chase kindly updated the Nodepool config via https://gerrit.wikimedia.org/r/#/c/339463/ with: max servers: 25 min jessie: 12 min trusty: 5 Thank... [19:41:57] 10Tool-Labs-tools-Pageviews: Add "weekly" date type to Pageviews apps - https://phabricator.wikimedia.org/T158901#3050953 (10MusikAnimal) [19:42:14] 10Tool-Labs-tools-Pageviews: Massviews export data is incorrect (both CSV and JSON) - https://phabricator.wikimedia.org/T158505#3050969 (10MusikAnimal) See T158900 and T158901 [19:57:08] 06Labs, 10Tool-Labs: jmail is not puppetized - https://phabricator.wikimedia.org/T158722#3051077 (10scfc) (I disagree that mail APIs are worse than others; in fact IMHO the inherent queue system saves much trouble. Being able to use Phabricator or Debian Bugs asynchronously is priceless.) It looks as if `drt... [19:59:52] 10Tool-Labs-tools-Other, 13Patch-For-Review, 15User-bd808: Jouncebot failing to parse Deployments page - https://phabricator.wikimedia.org/T158715#3045001 (10Danmichaelo) Fixed in mwclient 0.8.4 [19:59:55] chasemp: andrewbogott: Nodepool managed to instantly consume the larger pool :) [20:00:21] https://grafana.wikimedia.org/dashboard/db/nodepool?panelId=1&fullscreen&from=now-6h&to=now [20:52:48] (03CR) 10Jean-Frédéric: [C: 032] Change DB engine to InnoDB [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/324396 (https://phabricator.wikimedia.org/T138517) (owner: 10Lokal Profil) [20:53:14] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: keystone admin api occasionally refuses connection - https://phabricator.wikimedia.org/T156337#3051245 (10Andrew) 05Open>03Resolved I've rolled us back to eventlet, and the problem is resolved. I still don't know what the story was with uwsgi/keystone b... [20:55:03] (03Merged) 10jenkins-bot: Change DB engine to InnoDB [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/324396 (https://phabricator.wikimedia.org/T138517) (owner: 10Lokal Profil) [20:56:04] (03CR) 10jenkins-bot: Change DB engine to InnoDB [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/324396 (https://phabricator.wikimedia.org/T138517) (owner: 10Lokal Profil) [21:05:11] Jamesofur: Thank you again for cleaning up the sugarcrm labs project. Can I delete the project altogether, now? [21:05:47] Also, there's still a fair amount of data leftover on NFS, which is getting backed up. Do need it? And/or is it sensitive such that I should delete and try to destroy backups as well? [21:10:22] Jamesofur: (I asked those questions on our old friend https://phabricator.wikimedia.org/T128819 ) [21:14:26] andrewbogott: good morning :]  I started working on the removing the CI Precise instances [21:14:41] will take a bit of time to complete though. There is a huge cleanup needed in the CI config files [21:15:14] but I dont see a reason I would not be able to complete by end of March [21:17:24] Great! [21:37:42] bd808, chasemp I'm getting a "502 bad gateway" error on PAWS again. >_< [21:38:48] J-Mo: stop breaking it... ;) [21:39:03] yeah, I assume it's me. [21:39:12] but seriously, is it? [21:39:19] madhuvishy, yuvipanda: ^ can you poke that? chasemp and I are in a meeting [21:39:32] bd808: sure [21:39:39] J-Mo: i'm looking [21:39:43] doesn't need to be fixed right now, bd808, madhuvishy. But is there a diagnosis for this? [21:39:49] J-Mo: hi [21:39:52] J-Mo: looking. [21:39:56] J-Mo: madhuvishy actually knows how PAWS works, I don't so much [21:40:03] wondering if I should plan on doing more of my analysis locally going forward [21:40:17] i think in general yuvipanda is working on upgrading it and making it all better [21:40:22] yeah [21:40:29] k. makes sense. [21:40:42] J-Mo: I've spent about 12h in the last 24h on a big upgrade, which is coming before CSCW hopefully [21:40:51] w00t! [21:40:53] J-Mo: so sorry about it until then [21:41:14] J-Mo: you should keep using PAWS! :) hopefully we can make up for these downtimes by responding quickly until then! [21:41:38] you're all very responsive, madhuvishy. and I am eternally grateful for the work you do :) [21:41:44] okay i have to ask what in the world is paws? [21:42:15] Zppix: https://www.paws.org/ [21:42:21] Zppix also https://wikitech.wikimedia.org/wiki/PAWS [21:42:27] paws.wmflabs.org [21:42:32] yuvipanda: lol [21:42:36] Zppix: as part of our charitable mission, we help take care of some animals [21:42:57] it's the adorablest research platform ever! [21:43:01] hmmm what link should i click? [21:43:03] J-Mo: try now? [21:43:21] Zppix: mine [21:43:27] you guys are a bunch of trolls [21:43:33] Zppix: paws.wmflabs.org is where our installation of Jupyterhub is at [21:43:45] explanations at https://wikitech.wikimedia.org/wiki/PAWS [21:43:50] madhuvishy that sounds a little rated M to me xD [21:43:52] Zppix: madhuvishy is not a guy! [21:43:53] yuvipanda working! [21:44:25] Zppix I apologize for my trollish compatriots [21:44:27] J-Mo: cool! I hope to have a 'bandaid' in a bit [21:44:32] perf [21:44:34] thanks :) [21:44:43] yuvipanda looks interesting mind if i try to break it? [21:44:52] that's my job! [21:44:59] J-Mo not no more [21:45:10] * J-Mo was demoted :`-( [21:45:19] J-Mo no you were i was just promoted [21:45:25] uh-oh [21:45:27] to please call yuvipanda again... [21:45:46] andrewbogott: yes :) [21:45:47] xD [21:45:51] yuvipanda mostly ignores me (possibly for good reason) [21:46:22] J-Mo its a panda thing, can't help it [21:48:14] J-Mo: am putting bandaids in place, should not affect you tho (if you're already in your notebook) [21:49:08] cool. I just needed access to copy paste something today. [21:49:57] J-Mo i hear air raid sirens going off on paws is that a good thing xD [21:51:53] J-Mo: paws-public is always available tho [21:53:35] yuvipanda did i break something, creating a new txt doc doesnt do anything [21:55:30] Zppix: my bandaid is breaking things [21:56:00] yuvipanda well could you not xD [21:57:04] Zppix: fixed now [21:57:05] Zppix: nope [21:57:06] J-Mo: bandaid in place [21:57:47] yuvipanda dont forget it has to be band-aid brand :P [22:02:18] (03PS1) 10Jean-Frédéric: Change database docker image to mariadb:5.5.54 [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/339549 (https://phabricator.wikimedia.org/T158911) [22:02:46] (03CR) 10Jean-Frédéric: "WIP. Does not actually solve the issue." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/339549 (https://phabricator.wikimedia.org/T158911) (owner: 10Jean-Frédéric) [22:23:55] 06Labs, 06DC-Ops, 06Operations: Move labstore1002 and labstore1002-array1 and labstore1002-array2 to different rack (currently in C3) - https://phabricator.wikimedia.org/T158913#3051441 (10madhuvishy) [22:25:13] 06Labs, 06DC-Ops, 06Operations: Move labstore1002 and labstore1002-array1 and labstore1002-array2 to different rack (currently in C3) - https://phabricator.wikimedia.org/T158913#3051441 (10madhuvishy) [22:30:37] !log wikilabels disabling finished cswiki campaign [22:30:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [22:32:31] twentyafterfour not the right channel but its not as noisy here, but on 338965 (gerrit) couldnt we sort the query ascending instead or would that defeat the purpose? [22:39:44] Zppix: The query returns the page Special:WantedPages. How you order the query defines how the output will be ordered and that should be by "most wanted pages" first and alphabetically second if the first one is identical [22:44:43] so why descending? [22:45:32] because you want the most wanted first, not the least wanted [22:45:35] The special page shows how many redlinks exist for a specific title. When you order ascending the page outputs the pages with 1 redlink to it first [22:52:58] Zppix: See the two images at https://phabricator.wikimedia.org/M201 for comparison. [23:03:13] 06Labs: sugarcrm project using 12G of NFS - https://phabricator.wikimedia.org/T158778#3051609 (10Andrew) 05Open>03Resolved All gone! [23:42:53] 06Labs, 13Patch-For-Review: track growth of project and home directories - https://phabricator.wikimedia.org/T126623#3051738 (10madhuvishy) 05Open>03Resolved a:03madhuvishy The dashboard is updating fine the last 3 days - closing this. [23:42:57] 06Labs, 06Operations, 13Patch-For-Review, 07Tracking: overhaul labstore setup [tracking] - https://phabricator.wikimedia.org/T126083#3051741 (10madhuvishy)