[00:00:53] valhallasw`cloud, valhallasw`vecto: Hi. [00:01:00] What are we doing with tsreports? [00:59:03] 06Labs, 10Tool-Labs: Tool Labs users .bashrc file does not exist for tools accounts - https://phabricator.wikimedia.org/T131561#3011797 (10scfc) 05duplicate>03Open T91235 would be one possible solution (and IMHO the most beautiful), but there are other ways to do this, for example by copying `/etc/skel` in... [00:59:54] bd808: no luck [00:59:57] ? [01:11:52] mutante, Zppix|Away: I have good news. enwiki_p's externallinks table has a nearly identical URL storage structure as IABot's global meta table has. I can reconstruct the table with a simple script. Shouldn't take too long either. :D [01:42:20] bd808: how is python 2 versus 3 accessed on labs again [01:43:38] 06Labs, 10Tool-Labs: Sorting by CPU/VMEM columns doesn't sort by their value on http://tools.wmflabs.org/?status - https://phabricator.wikimedia.org/T69737#3011865 (10scfc) 05Invalid>03Open Actually, you found a bug :-). [01:44:38] JustBerry: webservice --backend=kubernetes python shell for Python 3 [01:44:46] ok [01:45:05] python2 for Python 2 [01:46:01] 06Labs, 10Tool-Labs: http://tools.wmflabs.org/?status is no longer sortable - https://phabricator.wikimedia.org/T157648#3011868 (10scfc) [02:03:17] 10Quarry: Users blocked from account creation on meta can not use Quarry - https://phabricator.wikimedia.org/T157342#3002152 (10Harej) >>! In T157342#3008064, @bd808 wrote: > Switching the wiki contacted for the OAuth handshake would really be a game of whack-a-mole. Today someone is affected by a meta ban, tomo... [03:14:05] 10Tool-Labs-tools-Erwin's-tools: https://github.com/erwin85/erwin85/issues should be disabled - https://phabricator.wikimedia.org/T157652#3011970 (10scfc) [03:14:42] 10Tool-Labs-tools-Erwin's-tools: https://github.com/erwin85/erwin85/wiki should be disabled - https://phabricator.wikimedia.org/T157653#3011982 (10scfc) [03:15:04] 10Tool-Labs-tools-Erwin's-tools: Unknown Error/MySQL errors - https://phabricator.wikimedia.org/T140421#2464064 (10scfc) I first tried to silence the deprecation warning with `error_reporting((E_ERROR|E_CORE_ERROR|E_ALL|E_COMPILE_ERROR) ^ E_DEPRECATED);` in `inc/webStart.inc.php` (inspired by http://stackoverflo... [03:16:08] 06Labs, 10Tool-Labs, 07Epic: Phase out precise instances from Tool Labs - https://phabricator.wikimedia.org/T94790#3011997 (10scfc) [03:16:11] 10Tool-Labs-tools-Erwin's-tools: Unknown Error/MySQL errors - https://phabricator.wikimedia.org/T140421#3011996 (10scfc) [04:05:09] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: keystone admin api occasionally refuses connection - https://phabricator.wikimedia.org/T156337#3012106 (10Andrew) >>! In T156337#3011653, @bd808 wrote: >>>! In T156337#3011599, @Andrew wrote: >> I've convinced myself that this is indeed a result of the uwsgi... [04:08:49] 06Tool-Labs-standards-committee, 10Wikimedia-Mailing-lists: Create mailing list for Tool-Labs-standards-committee - https://phabricator.wikimedia.org/T156218#3012107 (10Quiddity) * Admin email fwd'd to Huji. * List settings changed to private archives, require "Confirm and approve" for new subscriptions, plus... [06:22:09] 06Labs, 10Tool-Labs: Get rid of toolwatcher, use skeleton homedirs instead - https://phabricator.wikimedia.org/T91235#3012191 (10zhuyifei1999) [06:22:12] 06Labs, 10Tool-Labs: Tool Labs users .bashrc file does not exist for tools accounts - https://phabricator.wikimedia.org/T131561#3012190 (10zhuyifei1999) [06:48:20] PROBLEM - Puppet run on tools-exec-1412 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:09:34] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/336751 (owner: 10L10n-bot) [07:19:07] 06Labs, 10DBA, 06Operations, 10netops, 13Patch-For-Review: DBA plan to mitigate asw-c2-eqiad reboots - https://phabricator.wikimedia.org/T155999#3012217 (10Marostegui) [07:22:52] 06Labs, 06Operations, 10netops: asw-c2-eqiad reboots & fdb_mac_entry_mc_set() issues - https://phabricator.wikimedia.org/T155875#3012222 (10Marostegui) [07:22:56] 06Labs, 10DBA, 06Operations, 10netops, 13Patch-For-Review: DBA plan to mitigate asw-c2-eqiad reboots - https://phabricator.wikimedia.org/T155999#3012218 (10Marostegui) 05Open>03Resolved All the initial actions listed on the original ticket to mitigate this issue have been completed, the only pending... [07:28:19] RECOVERY - Puppet run on tools-exec-1412 is OK: OK: Less than 1.00% above the threshold [0.0] [08:02:04] 10Tool-Labs-tools-Erwin's-tools: Unknown Error/MySQL errors - https://phabricator.wikimedia.org/T140421#3012240 (10scfc) I had to switch from the Kubernetes backend to the (Trusty) grid with `webservice stop && webservice --backend=gridengine lighttpd start` because `catanalyzer.php` uses `dot` from the package... [08:05:56] 10Tool-Labs-tools-Erwin's-tools: Unknown Error/MySQL errors - https://phabricator.wikimedia.org/T140421#3012243 (10scfc) … and I saved @Nemo_bis's work as branch `pu/nemos-work` – IMHO Git makes it easier to work with branches than with stashes: If you rebase a branch onto `master` (with conflicts), you can eith... [08:09:13] 10Tool-Labs-tools-Other, 07Epic: Toolserver.org tools that have not been migrated (tracking) - https://phabricator.wikimedia.org/T60865#3012247 (10scfc) [08:09:15] 10Tool-Labs-tools-Erwin's-tools: Migrate https://toolserver.org/~erwin85/delete.php to Tool Labs - https://phabricator.wikimedia.org/T62877#3012245 (10scfc) 05Open>03Resolved AFAICT https://tools.wmflabs.org/erwin85/delete.php works. [08:10:13] 10Tool-Labs-tools-Other, 07Epic: Toolserver.org tools that have not been migrated (tracking) - https://phabricator.wikimedia.org/T60865#605796 (10scfc) [08:10:15] 10Tool-Labs-tools-Erwin's-tools: Migrate https://toolserver.org/~erwin85/categorycount.php to Tool Labs - https://phabricator.wikimedia.org/T62869#3012249 (10scfc) 05Open>03Resolved AFAICT https://tools.wmflabs.org/erwin85/categorycount.php works. [08:10:57] 10Tool-Labs-tools-Erwin's-tools: xwiki.php not working - https://phabricator.wikimedia.org/T86976#3012255 (10scfc) [08:11:01] 10Tool-Labs-tools-Other, 07Epic: Toolserver.org tools that have not been migrated (tracking) - https://phabricator.wikimedia.org/T60865#605796 (10scfc) [08:12:17] 10Tool-Labs-tools-Other, 07Epic: Toolserver.org tools that have not been migrated (tracking) - https://phabricator.wikimedia.org/T60865#3012260 (10scfc) [08:12:20] 10Tool-Labs-tools-Erwin's-tools, 13Patch-For-Review: Migrate https://toolserver.org/~erwin85/talkcatintersect.php to Tool Labs - https://phabricator.wikimedia.org/T62874#3012258 (10scfc) 05Open>03Resolved AFAICT https://tools.wmflabs.org/erwin85/talkcatintersect.php works. [08:21:52] 10Tool-Labs-tools-Erwin's-tools: xwiki.php not working - https://phabricator.wikimedia.org/T86976#3012274 (10scfc) 05Open>03Resolved >>! In T86976#2469111, @MarcoAurelio wrote: > […] > The tool is suposed to parse the data from the COIBot XWiki reports to help spam fighters look where the link is still prese... [08:29:39] 06Labs, 10Tool-Labs: Add 'file a bug' link to tool labs error pages - https://phabricator.wikimedia.org/T90570#3012277 (10scfc) 05Open>03declined Dito. I'd especially dislike the default #Tool-Labs tag for Phabricator tasks. This could be interpreted that #Tool-Labs administrators will fix anything :-).... [08:32:46] 06Labs, 10Tool-Labs: Error pages link to obsolete wiki page - https://phabricator.wikimedia.org/T157661#3012288 (10scfc) [08:32:49] 06Labs, 10Tool-Labs: Error pages link to obsolete wiki page - https://phabricator.wikimedia.org/T157662#3012300 (10scfc) [08:32:58] 06Labs, 10Tool-Labs: Error pages link to obsolete wiki page - https://phabricator.wikimedia.org/T157662#3012316 (10scfc) p:05Triage>03Normal [08:38:52] 06Labs, 10Tool-Labs: Make gridengine exec hosts also submit hosts - https://phabricator.wikimedia.org/T123270#3012325 (10scfc) To make this a bit less confusing, let's make this task about setting up execution nodes as submit hosts and T67777 about limitting the number of parallel tasks a user can execute. [08:40:01] 06Labs, 10Tool-Labs: Limit number of jobs users can execute in parallel - https://phabricator.wikimedia.org/T67777#709752 (10scfc) 05duplicate>03Open To make this a bit less confusing, let's make this task about limitting the number of parallel tasks a user can execute and T123270 about setting up executio... [08:48:24] 06Labs, 10Tool-Labs: Error pages link to obsolete wiki page - https://phabricator.wikimedia.org/T157662#3012352 (10scfc) [08:48:27] 06Labs, 10Tool-Labs: Error pages link to obsolete wiki page - https://phabricator.wikimedia.org/T157661#3012354 (10scfc) [09:14:50] 10Tool-Labs-tools-Other: Migrate http://toolserver.org/~purodha/sample/dbswithuser.php to Tool Labs - https://phabricator.wikimedia.org/T63028#677367 (10scfc) I think it's easier to add the option to display the local `user_id` to another tool. @Nemo_bis, what similar tool meets your needs otherwise? [09:26:43] 10Tool-Labs-tools-Other: Migrate https://toolserver.org/~magnus/coord_gen.php - https://phabricator.wikimedia.org/T63188#3012414 (10scfc) 05Open>03declined The tool is dormant at http://tools.wmflabs.org/magnus-toolserver/coord_gen.php with @Magnus offering to prioritize resurrecting it (cf. http://tools.wmf... [09:26:49] 10Tool-Labs-tools-Other, 07Epic: Toolserver.org tools that have not been migrated (tracking) - https://phabricator.wikimedia.org/T60865#3012416 (10scfc) [09:36:32] 06Labs, 10Tool-Labs: Rewrite /usr/local/bin/crontab in python; fix bugs - https://phabricator.wikimedia.org/T156174#3012434 (10zhuyifei1999) a:03zhuyifei1999 [09:39:12] 10Tool-Labs-tools-Other: Migrate http://toolserver.org/~purodha/sample/dbswithuser.php to Tool Labs - https://phabricator.wikimedia.org/T63028#3012441 (10Nemo_bis) >>! In T63028#3012377, @scfc wrote: > I think it's easier to add the option to display the local `user_id` to another tool. @Nemo_bis, what similar... [09:49:44] PROBLEM - Puppet staleness on tools-checker-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [10:52:53] 10Quarry: Users blocked from account creation on meta can not use Quarry - https://phabricator.wikimedia.org/T157342#3002152 (10Base) Can't the user just change the URL address in the browser and handle OAuth manually through another wiki? Or OAuth does not allow it to be done like that? [10:58:17] 10Quarry: Users blocked from account creation on meta can not use Quarry - https://phabricator.wikimedia.org/T157342#3012592 (10Base) I have just logged in to Quarry by changing Meta to angwiki so I guess it is possible. I am not banned on Meta though, if needed I can ban myself for test purposes. [12:06:28] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Tarawneh was created, changed by Tarawneh link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Tarawneh edit summary: Created page with "{{Tools Access Request |Justification=have live statistics linking geolocations with articles reading in Jordan. |Completed=false |User Name=Tarawneh }}" [12:41:29] 06Labs, 10Tool-Labs: Improve outcome of http://tools.wmflabs.org/?status proposal - https://phabricator.wikimedia.org/T149405#3012748 (10Wesalius) 05Open>03Resolved a:03Wesalius This is duplicate of T69737 [12:51:12] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012775 (10Magnus) [13:00:50] 10Tool-Labs-tools-Xtools, 06Community-Tech: XTools: Articleinfo - https://phabricator.wikimedia.org/T157602#3012807 (10Aklapper) (This task's summary welcomes a verb to describe what you'd like to achieve in this task when it comes to `XTools: Articleinfo`.) [13:07:29] 10Tool-Labs-tools-Xtools, 03Community-Tech-Sprint, 07Documentation: Improve documentation for setting up XTools on a local machine - https://phabricator.wikimedia.org/T157609#3012827 (10Aklapper) [13:22:14] (03PS1) 10Addshore: Rename MoveToCommons extensions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/336789 [13:22:36] (03PS2) 10Addshore: Rename MoveToCommons extensions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/336789 (https://phabricator.wikimedia.org/T157539) [13:22:54] (03CR) 10Addshore: [C: 032] Rename MoveToCommons extensions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/336789 (https://phabricator.wikimedia.org/T157539) (owner: 10Addshore) [13:23:18] (03Merged) 10jenkins-bot: Rename MoveToCommons extensions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/336789 (https://phabricator.wikimedia.org/T157539) (owner: 10Addshore) [13:23:28] (03CR) 10jenkins-bot: Rename MoveToCommons extensions [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/336789 (https://phabricator.wikimedia.org/T157539) (owner: 10Addshore) [13:43:47] 06Labs, 10Tool-Labs: Support @weekly et al in crontab - https://phabricator.wikimedia.org/T86446#968407 (10zhuyifei1999) This was supposed to be fixed in ca4568e. [13:54:11] 10Tool-Labs-tools-LTA-Knowledgebase, 15User-Samtar: Static user testing - https://phabricator.wikimedia.org/T157682#3012922 (10Samtar) [13:56:11] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012775 (10chasemp) Hi @magnus I tried a few things to get this going and at the moment I'm seeing 502 bad gateway instead of 404. > tools.autodesc@tools-bastion-03:~$ webservice nodejs status > Your webservice is not... [13:57:32] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012957 (10chasemp) > tools.autodesc@tools-bastion-03:~$ nodejs -v > v0.10.25 [14:00:58] 10Tool-Labs-tools-LTA-Knowledgebase: Recent changes should link to LTA entry - https://phabricator.wikimedia.org/T157684#3012970 (10AntiCompositeNumber) [14:03:49] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012993 (10chasemp) Then I see a series of new job ID's as the webservice is auto-restarted I believe: > 970166 > 970174 > 970183 I see more: > autodesc@0.0.0 start /mnt/nfs/labstore-secondary-tools-project/autodesc... [14:04:38] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012997 (10Magnus) Tried npm update, which worked fine, but no joy. 1. Can I update this myself? Probably not. 2. Why did this suddenly fail? Who changed what? [14:05:20] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012998 (10chasemp) Trusty node version example > rush@tools-webgrid-lighttpd-1413:~$ nodejs -v > v0.10.25 [14:05:41] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012999 (10Magnus) With "this", I meant "npm start" and the subsequent call to "node" [14:09:27] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3013029 (10chasemp) @Magnus, do you have an idea how long it's been down for? >>! In T157678#3012997, @Magnus wrote: > Tried npm update, which worked fine, but no joy. > > 1. Can I update this myself? Probably not.... [14:12:34] bd808: about this morning? I'm wondering about https://phabricator.wikimedia.org/T157678 and if this is somehow webservice precise/trusty switchup related [14:21:37] chasemp: was he running on k8s before? I think that the node version for jessie was bumped for prod & yuvipanda rebuilt the Docker images Tuesday evening [14:23:59] I'm not fully functioning yet this AM, but I can look at it in an hour or so. [14:26:06] 10Tool-Labs-tools-LTA-Knowledgebase: Recent changes should link to LTA entry - https://phabricator.wikimedia.org/T157684#3013080 (10Samtar) p:05Triage>03Low a:03Samtar On it! I'll get this sorted this evening [14:44:34] bd808: ah interesting I didn't realize that, very possibly part of the deal [14:46:21] 06Labs, 10Tool-Labs: Rewrite /usr/local/bin/crontab in python; fix bugs - https://phabricator.wikimedia.org/T156174#3013123 (10zhuyifei1999) How about something similar to this: {P4916} The regex insanity has not been written yet, though. [15:16:46] 06Labs, 10Tool-Labs: Error pages link to obsolete wiki page - https://phabricator.wikimedia.org/T157662#3012300 (10zhuyifei1999) There are no //common causes of errors// in that page, nor anywhere that I can remember. If that were actually the intended target, I'd suggest to change it to ``` You may wish to chasemp: so many moving parts to keep track of :/ [15:55:57] I am half here half away atm but yeah, I also wonder if updating our containers isn't going to be an issue in this manner in general [15:56:16] we are going to have to version our nodejs container for instance and keep backwards compat or something for awhile? [15:58:05] the "real" fix for this would be per-tool containers and abandoning the idea that we will rebuild existing containers when something changes in our base image. That is in some ways the point of containers in the first place; to provide a stable build over time for a given deployment. [15:58:28] That doesn't match with some of the k8s container thinking I've heard though [15:58:40] bd808 chasemp: what do you think of reformatting https://phabricator.wikimedia.org/tag/tool-labs/ into something like https://phabricator.wikimedia.org/tag/commons/ that organizes by component instead of progress? whether showing all tasks or only open tasks triage + backlog accounts for 80-90% of the tickets [15:59:32] zhuyifei1999_: not a horrible idea, but I'd like to wait a bit on major phab work. We are likely going to do some renaming and that might be the right time to rethink what goes where. [15:59:45] k [15:59:49] using sub-projects may be a good idea as well [16:00:12] yep, you can't let a task on two columns [16:01:20] yeah. In the grand picture I think we want to separate the board that the WMF team uses for tracking/reporting and the boards that are used to show all the things that need attention. [16:01:58] the model of all the things in one pile is really not working [16:02:44] I'll file a ticket about this [16:02:55] :) great idea [16:05:36] 10Quarry: Users blocked from account creation on meta can not use Quarry - https://phabricator.wikimedia.org/T157342#3013392 (10Reguyla) @Base, I don't think Oauth works like that. I think the developer has to assign a home wiki to use for the login process and although they can choose just about anything, in th... [16:08:36] bd808: good idea to use anaconda on labs? ;p [16:08:50] (or is it not a good idea for that matter) [16:11:20] JustBerry: you could file a phab task and we could look to see if it has packages supported by Debian. Otherwise probably not going to happen [16:12:16] bd808: unless matplotlib can get its act together with pip [16:12:18] might be a hurdle ;p [16:19:39] test. [16:21:21] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3012775 (10bd808) I think there is a possibility that things were broken by a combination of a nodejs package upgrade in the WMF apt repo ({T149331}), the base Docker images used for Kubernetes being rebuilt on 2017-02... [16:21:27] melodykramer: ack [16:31:38] FFS. what's wrong with nodejs pods? [16:31:48] * bd808 beats head against floor [16:35:17] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3013509 (10bd808) ``` tools.autodesc@tools-bastion-02:~$ webservice --backend=kubernetes nodejs shell If you don't see a command prompt, try pressing enter. tools.autodesc@interactive:~$ tools.autodesc@interactive:~$ l... [16:40:13] 06Labs, 10Tool-Labs: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3013519 (10bd808) ``` tools.autodesc@interactive:/data/project/autodesc/www/js$ mv node_modules/ node_modules-busted tools.autodesc@interactive:/data/project/autodesc/www/js$ npm install (node:39) fs: re-evaluating nat... [16:42:41] 06Labs, 10Tool-Labs: #tool-labs workboard is crazy - https://phabricator.wikimedia.org/T157698#3013537 (10zhuyifei1999) [16:46:24] 06Labs, 10Tool-Labs: #tool-labs workboard is crazy - https://phabricator.wikimedia.org/T157698#3013559 (10zhuyifei1999) [16:51:12] 06Labs, 10Tool-Labs, 15User-bd808: webservice for nodejs fails - https://phabricator.wikimedia.org/T157678#3013569 (10bd808) 05Open>03Resolved a:03bd808 I sent an email to labs-l about this issue as there are likely other tools that are affected: https://lists.wikimedia.org/pipermail/labs-l/2017-Februa... [16:56:36] 10Tool-Labs-tools-Xtools, 06Community-Tech: Rewrite XTools: Articleinfo - https://phabricator.wikimedia.org/T157602#3013598 (10Matthewrbowker) [17:02:21] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3013613 (10Andrew) [17:13:08] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3013652 (10Andrew) [17:20:53] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3013688 (10Andrew) [17:21:30] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#2921104 (10Andrew) [17:25:39] 06Labs: Delete or recreate docs.editor-engagement.wmflabs.org - https://phabricator.wikimedia.org/T157701#3013713 (10Andrew) [17:26:10] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3013726 (10Andrew) [17:30:31] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3013759 (10Andrew) [17:55:36] 10Tool-Labs-tools-Xtools, 06Community-Tech: Output data for new XTools: Articleinfo - https://phabricator.wikimedia.org/T157706#3013867 (10kaldari) [17:58:09] 10Tool-Labs-tools-Xtools, 06Community-Tech: Output data for new XTools: Articleinfo - https://phabricator.wikimedia.org/T157706#3013901 (10kaldari) [17:58:42] 10Tool-Labs-tools-Xtools, 06Community-Tech: [Epic] Rewrite XTools: Articleinfo - https://phabricator.wikimedia.org/T157602#3013905 (10kaldari) [17:59:09] 10Tool-Labs-tools-Xtools, 06Community-Tech: Output data for new XTools: Articleinfo - https://phabricator.wikimedia.org/T157706#3013867 (10kaldari) p:05Triage>03Normal [18:00:32] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3013922 (10Andrew) [18:06:36] 06Labs, 06Discovery, 06Maps: Update, replace or delete maps-tiles1 - https://phabricator.wikimedia.org/T157708#3013957 (10Andrew) [18:07:00] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3013971 (10Andrew) [18:10:32] 06Labs, 10Huggle, 15User-Addshore: Labs instance huggle.huggle.wmflabs needs to be replaced or deleted - https://phabricator.wikimedia.org/T157710#3013987 (10Andrew) [18:10:57] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3014001 (10Andrew) [18:20:06] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3014054 (10Andrew) [18:27:19] bd808: MaxSem can't check which version of debian labs is running on with uname -or [18:30:22] 06Labs, 10Huggle: Labs instance huggle.huggle.wmflabs needs to be replaced or deleted - https://phabricator.wikimedia.org/T157710#3014097 (10Addshore) a:05Addshore>03Petrb [18:30:53] JustBerry, lsb_release -a [18:31:11] MaxSem: wait, it's ubuntu...? [18:31:27] works on debian too [18:32:42] MaxSem: I thought labs ran on debian [18:32:44] not ubuntu [18:32:51] tools.spiarticleanalyzer@tools-bastion-03:~$ lsb_release -a [18:32:52] No LSB modules are available. [18:32:52] Distributor ID: Ubuntu [18:32:53] Description: Ubuntu 14.04.5 LTS [18:32:55] Release: 14.04 [18:32:57] Codename: trusty [18:32:58] we have a mix of both [18:34:28] the trend is towards Debian though. feel encouraged to use jessie and replace trusty [18:38:18] * MaxSem remembers the time when we ran Fedora [18:38:35] MaxSem: goodness [18:38:36] wasn't involved in technical side back then though [18:38:40] should I test on debian or ubuntu then [18:38:46] I need to know which environment to test [18:38:47] outside of labs [18:38:51] how good was Fedora [18:39:01] Any new servers in production will run Jessie [18:39:04] cause it's a pain to write code in nano MaxSem lol [18:39:06] zhuyifei1999_: o/ [18:39:17] Labs supports Trusty because there are still some old Trusty boxes running there. [18:39:21] * zhuyifei1999_ only used centos outside debian-based systems [18:39:28] We're in the process of killing off all the remaining Precise systems though [18:39:37] MaxSem: what version of debian is being run [18:39:39] I only see ubuntu [18:39:41] Jessie [18:39:43] Jessie [18:40:23] JustBerry: you can ssh to newer instances, or to k8s interactive [18:40:53] they are jessie [18:41:08] JustBerry: also, you keep saying 'Labs' and I don't think that's the word you want. https://wikitech.wikimedia.org/wiki/Labs_labs_labs [18:41:24] tool labs andrewbogott [18:41:59] JustBerry: the bastions and the kubernetes containers are different [18:42:14] the k8s containers run Debian Jessie [18:42:48] k8s is like vm in vm lol [18:42:59] you have to do `webservice --backend=kubernetes shell` to get inside a k8s container [18:43:35] but not exactly vm [18:43:44] turtles all the way down [18:43:45] zhuyifei1999_: deeper than that ;) virthost -> VM -> kuberentes -> Docker [18:44:13] bd808: amd64/arm64/armel/armhf/i386/etc. [18:44:24] although k8s is really a management layer I guess [18:44:33] JustBerry: amd64 I would think. let me check [18:44:46] oh btw how different is docker from lxc? or is docker based on lxc? [18:44:57] JustBerry: yup. amd64 [18:45:31] zhuyifei1999_: docker is 'mostly' lxc or at least the same kernel level pieces that comprise lxc [18:45:49] okay [18:47:53] at some point we may switch from Docker to rkt containers but that's largely an implementation detail [18:48:23] https://www.ctl.io/developers/blog/post/what-is-rocket-and-how-its-different-than-docker/ [18:48:29] bd808: so lsb_release a gives me ubuntu [18:48:37] should I test my code in ubuntu or debian [18:48:42] JustBerry: only on a bastion, not on k8s [18:48:51] where does the code run then [18:48:56] what are you trying to do? Lets start from there [18:49:11] what language is your code? [18:49:25] mutante: .py [18:49:28] python [18:49:37] bd808: I'm just making a few mediawiki calls [18:49:40] and using flask [18:49:40] code runs on Precise (for a few more weeks), Trusty, or Jessie depending on if it is a grid job or a kubernetes pod [18:49:42] to display [18:49:47] bd808: rocket o.O [18:50:05] bd808: so which one would it be [18:50:12] ok [18:50:14] if I create a file [18:50:17] .py [18:50:20] so, jessie has python 2.7.9 https://packages.debian.org/jessie/python [18:50:24] JustBerry: we built that all as a kubernetes web service (with my tutorial) [18:50:24] make a few api calls [18:50:36] bd808: so we're talking ubuntu then? [18:50:40] mutante: his container is actually python3 [18:50:48] JustBerry: no, Debian Jessie [18:51:15] does this work bd808 [18:51:16] debian-8.7.1-amd64-netinst [18:51:18] .iso [18:51:24] JustBerry: are you developing live on tool labs? [18:51:28] ok, so i was just gonna say you should look at the python version [18:51:29] trying not to bd808 [18:51:30] JustBerry: jessie is the codename for debian [18:51:35] I'd prefer developing in virtualbox [18:51:37] nano is annoying [18:51:41] I'd prefer on mac ideally [18:51:42] again not sure what problems you are trying to solve here today [18:51:46] and unless you see differences in the python version that means you have to change code... [18:51:46] well [18:51:49] then it doesnt really matter [18:51:52] I'm trying to install a bunch of libs [18:51:54] if you use Debian or Ubuntu [18:51:55] and having issues on mac [18:51:57] my concern is [18:51:59] if I debug on mac [18:52:01] * a version of [18:52:04] I'm getting nowhere with debugging on linux [18:52:14] and I can't debug the code and debug the libs on mac and then go to linux and start all over [18:52:17] that just doesn't make sense [18:52:24] so I'm trying to setup a virtualbox on mac [18:52:25] JustBerry: ah. ok [18:52:27] yeah [18:52:37] wouldn't you recommend that, bd808? [18:53:08] JustBerry: sure. that's reasonable. You could actually use mediawiki-vagrant to setup the base VM that you test in. [18:53:16] bd808: what is that [18:53:21] bd808: why I said toollabs-on-vagrant a while ago :P [18:53:25] https://www.mediawiki.org/wiki/MediaWiki-Vagrant [18:54:14] JustBerry: correct. There is one change you need to make to get a Debian Jessie VM from mw-vagrant [18:54:18] but toollabs has so much exvironment set up that's difficult replicate with a new VM [18:54:53] zhuyifei1999_: that's the whole problem [18:54:59] I mean look [18:55:01] I have code on my machine [18:55:04] JustBerry: after git clone --recursive https://gerrit.wikimedia.org/r/mediawiki/vagrant and cd vagrant you would need to `git checkout jessie-migration` to get the experimental Jessie branch [18:55:09] how do I get it onto tool labs [18:55:13] without git commiting 500 times [18:55:14] bd808: [18:55:25] JustBerry: though, idk why you are nano-ing, you should be git-pull-ing :P [18:55:31] so commit [18:55:33] 500 times? [18:55:37] for each thing [18:55:43] like if I'm just testign [18:55:48] testing printing a list or something [18:55:50] or some small thing [18:55:53] that's like 500 commits [18:56:00] uh [18:56:49] that was the whole issue before [18:56:58] I'm commiting nontested stuff [18:57:05] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014203 (10mmodell) @bd808: We can allow patches via direct push and then review them via audit. We'd just need to have secur... [18:57:08] unless you want me to test locally [18:57:15] you want = you suggest [18:57:22] for the core of what you are talking about (making MW api calls) you should be able to dev and test anywhere. [18:57:23] zhuyifei1999_: which requires me to debug two environments [18:57:26] one way to test locally is python app.py [18:57:34] zhuyifei1999_: I can't [18:57:39] why? [18:57:46] I haven't been able to install libicui18n.54.dylib locally [18:57:50] did brew [18:57:51] pip [18:57:52] php56 [18:57:55] did tons of things [18:58:03] libicui18n.54.dylib? [18:58:09] yup [18:58:10] import icu [18:58:11] gives [18:58:16] ImportError: dlopen(/Library/Python/2.7/site-packages/_icu.so, 2): Library not loaded: libicui18n.54.dylib [18:58:16] Referenced from: /Library/Python/2.7/site-packages/_icu.so [18:58:17] Reason: image not found [18:58:25] google? [18:58:27] very annoying error [18:58:29] tried many times [18:58:34] I did a lot to debug this [18:58:40] bd808: [19:01:04] JustBerry: sorry, supporting your local dev environment is a bit out of scope for me. [19:01:21] I do understand the need, but I can't fix it today [19:01:37] * zhuyifei1999_ must admit that I don't know too much about python on bsd-based system to know what is wrong [19:06:20] JustBerry: oh with a vm you'll know how to play with root. and don't $ sudo rm -rf / --no-preserve-root [19:06:22] * zhuyifei1999_ is stuffing beans up my nose [19:24:44] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014268 (10bd808) >>! In T117071#3014203, @mmodell wrote: > @bd808: We can allow patches via direct push and then review them... [19:38:13] 06Labs, 10Tool-Labs: #tool-labs workboard is crazy - https://phabricator.wikimedia.org/T157698#3014312 (10bd808) Thanks for kicking off this discussion @zhuyifei1999. I completely agree with the general problem statement. I think we need to take a bit of time to think about what the 'ideal' world looks like.... [19:44:51] 06Labs, 10Tool-Labs: Rewrite /usr/local/bin/crontab in python; fix bugs - https://phabricator.wikimedia.org/T156174#3014327 (10bd808) @zhuyifei1999 Looks like a good start. I think we should move the script from Puppet to https://gerrit.wikimedia.org/r/#/admin/projects/labs/toollabs and make the change in Pupp... [19:50:34] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014381 (10mmodell) >>! In T117071#3014268, @bd808 wrote: > So that would be some form of recreating Gerrit in Diffusion? We... [19:54:13] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014383 (10Legoktm) I'm a little confused, AIUI the requirements are that a group of tool owners can direct push or go throug... [20:02:15] 10Tool-Labs-tools-LTA-Knowledgebase: Add history - https://phabricator.wikimedia.org/T157725#3014453 (10DatGuy) [20:06:56] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014474 (10demon) >>! In T117071#3014383, @Legoktm wrote: > I'm a little confused, AIUI the requirements are that a group of... [20:07:07] 06Labs, 10Tool-Labs: Rewrite /usr/local/bin/crontab in python; fix bugs - https://phabricator.wikimedia.org/T156174#3014481 (10zhuyifei1999) Yeah sure. Part of this task anyways. :) [20:09:57] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014489 (10Legoktm) OK, gotcha. So probably the real task left here is documenting somewhere (maybe it already exists) how to... [20:12:24] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014496 (10demon) >>! In T117071#3014489, @Legoktm wrote: > OK, gotcha. So probably the real task left here is documenting so... [20:13:20] zhuyifei1999_: bd808 MaxSem how much space per tool in terms of hard disk space [20:13:42] JustBerry: there is currently no hard quota [20:13:53] bd808: I assume the admins just monitor the space used then [20:14:30] more or less. we have scripts that hunt for things that are huge and causing systemic problems [20:14:37] https://phabricator.wikimedia.org/T136212 [20:15:26] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014502 (10mmodell) >>! In T117071#3014489, @Legoktm wrote: > OK, gotcha. So probably the real task left here is documenting... [20:16:27] bd808: this module I'm downloading is eating space [20:19:47] 06Labs, 10Tool-Labs, 10labs-sprint-119, 06Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#3014515 (10mmodell) @legoktm: would it be helpful if we indexed repositories->tools mapping in the phabricator search index?... [20:27:07] -ansi -Wno-long-long -ffloat-store -g -O2 -MT RobustDeterminant.lo -MD -MP -MF .deps/RobustDeterminant.Tpo -c -o RobustDeterminant.lo RobustDeterminant.cpp [20:27:07] libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../include -I../../include/geos -I../../include -DGEOS_INLINE -pedantic -Wall -ansi -Wno-long-long -ffloat-store -g -O2 -MT RobustDeterminant.lo -MD -MP -MF .deps/RobustDeterminant.Tpo -c RobustDeterminant.cpp -fPIC -DPIC -o .libs/ [20:27:44] whops, compile stuff [20:27:47] zhuyifei1999_: o/ [20:28:30] compile parameters are crazy [20:29:57] zhuyifei1999_: make and make install be crazy [20:35:50] !log wikilabels staged wikilabels-wmflabs-deploy:a74d240 [20:35:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [20:38:50] !log wikilabels deployed wikilabels-wmflabs-deploy:a74d240 [20:38:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [20:51:06] bd808: can't install these guys: http://packages.ubuntu.com/search?suite=trusty&searchon=names&keywords=basemap [20:51:08] halfak: ^^ [20:51:46] pip install matplotlib gives what error? [20:52:29] halfak: not the poitn [20:52:32] basemap ain't working [20:52:37] lemme post [20:52:38] pastbein [20:52:42] Oh. What do you mean "not working" [20:52:43] yeah [20:52:46] PROBLEM - Puppet run on tools-worker-1010 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [20:52:46] http://pastebin.com/S1w1BmSf [20:52:58] http://pastebin.com/JZm6MJsy [20:53:06] halfak: [20:53:15] pip install? [20:53:31] JustBerry: you don't have rights to install debs certainly. We only do that via Puppet patches after a Phab task and discussion of the general usefulness for Tools. [20:53:32] zhuyifei1999_, that'd be the standard way to install a python package [20:53:36] PROBLEM - Puppet run on tools-docker-registry-01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [20:53:48] PROBLEM - Puppet run on tools-webgrid-lighttpd-1404 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [20:53:58] halfak: been using pip install [20:53:58] PROBLEM - Puppet run on tools-worker-1008 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [20:53:58] and those are Ubutunu packages that won't help you in a Jessie kubernetes container [20:53:59] halfak: exactly [20:54:10] PROBLEM - Puppet run on tools-webgrid-lighttpd-1203 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [20:54:29] ah yeah. I see what you were saying. [20:54:30] PROBLEM - Puppet run on tools-docker-registry-02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:54:58] * halfak tries installing in his virtualenv [20:55:09] halfak: okay wait [20:55:17] halfak: it's too large for pip [20:55:18] I used http://matplotlib.org/basemap/users/installing.html [20:55:31] "too large for pip"? [20:55:37] PROBLEM - Puppet run on tools-webgrid-lighttpd-1204 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [20:55:44] bd808: not hosted on pip [20:55:49] too large or whatever [20:55:53] that's why the instructions above [20:55:54] is used [20:55:55] PROBLEM - Puppet run on tools-proxy-01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [20:55:56] but that didn't work [20:56:00] I extracted the tarball and whatnot [20:56:13] PROBLEM - Puppet run on tools-logs-02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [20:56:15] PROBLEM - Puppet run on tools-redis-1001 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:56:46] bd808: halfak ^^ [20:56:54] * halfak downloads tarball [20:56:56] via the matplotlib.org instructions [20:57:03] PROBLEM - Puppet run on tools-exec-1218 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:57:06] ... from sourceforge [20:57:08] * halfak barfs [20:57:14] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3014703 (10Andrew) [20:57:29] PROBLEM - Puppet run on tools-static-10 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [20:57:32] so you need to configure a prefix for ./configure? [20:57:35] http://stackoverflow.com/questions/5979513/python-manually-install-package-using-virtualenv [20:57:41] shouldn't that essentially be the idea? [20:57:49] ^ that yeah [20:57:55] PROBLEM - Puppet run on tools-worker-1005 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [20:58:01] PROBLEM - Puppet run on tools-exec-1415 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [20:58:07] PROBLEM - Puppet run on tools-webgrid-lighttpd-1414 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:58:07] PROBLEM - Puppet run on tools-webgrid-lighttpd-1402 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [20:58:09] PROBLEM - Puppet run on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:58:10] JustBerry: what prefix are you using? [20:58:19] PROBLEM - Puppet run on tools-worker-1019 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [20:58:25] PROBLEM - Puppet run on tools-exec-1219 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [20:58:29] PROBLEM - Puppet run on tools-webgrid-lighttpd-1406 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [20:58:35] zhuyifei1999_: prefix for [20:58:37] PROBLEM - Puppet run on tools-webgrid-lighttpd-1407 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [20:58:38] well I followed that [20:58:39] PROBLEM - Puppet run on tools-worker-1022 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [20:58:50] $GEOS_DIR [20:58:54] zhuyifei1999_: that's the prefix [20:59:11] * halfak waits for the slowest download of 2017 [20:59:22] export GEOS_DIR= [20:59:32] JustBerry: ^ [20:59:33] PROBLEM - Puppet run on tools-webgrid-lighttpd-1208 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [20:59:43] PROBLEM - Puppet run on tools-exec-1404 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:59:45] PROBLEM - Puppet run on tools-exec-1405 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [21:00:03] whatever set there is your $GEOS_DIR [21:00:17] PROBLEM - Puppet run on tools-webgrid-lighttpd-1413 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [21:00:21] and idk what you set it to [21:00:38] puppet is going crazy... [21:00:39] PROBLEM - Puppet run on tools-exec-1408 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [21:01:09] PROBLEM - Puppet run on tools-exec-1419 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [21:01:21] PROBLEM - Puppet run on tools-worker-1004 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [21:01:41] zhuyifei1999_: yeah I did /usr/libs [21:01:46] halfak: lol I know [21:01:48] that's why I said [21:01:49] too large for pip [21:01:53] and it says it on one of the doc pages [21:01:55] but halfak [21:01:57] source $HOME/www/python/venv/bin/python setup.py install [21:01:57] -bash: source: /data/project/spiarticleanalyzer/www/python/venv/bin/python: cannot execute binary file [21:02:06] chasemp: rip the venv idae ^^ [21:02:08] idea* [21:02:09] JustBerry: you don't have write permissions to that dir [21:02:09] PROBLEM - Puppet run on tools-exec-1413 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [21:02:10] http://stackoverflow.com/questions/5979513/python-manually-install-package-using-virtualenv [21:02:11] PROBLEM - Puppet run on tools-webgrid-lighttpd-1207 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [21:02:13]