[00:45:15] 10PAWS: os.environ doesn't get client ID, etc. necessary for PAWS session to use OAuth - https://phabricator.wikimedia.org/T128108#2065470 (10yuvipanda) 5Open>3Resolved I've identified and fixed the issue! It was due to me using a different version of the MW Authenticator than required. It works now! Thanks... [01:47:15] 6Labs, 15User-bd808: Create project for AuthManager testing - https://phabricator.wikimedia.org/T125320#2065660 (10bd808) Setup new host at am-01.authmanager.eqiad.wmflabs: * Enabled role `role::labs::mediawiki_vagrant` and forced puppet run with `sudo puppet agent --test --verbose` * Setup MediaWiki-Vagrant h... [01:53:07] !log authmanager Setup initial wiki farm on am-01.authmanager.eqiad.wmflabs (T125320) [01:53:08] T125320: Create project for AuthManager testing - https://phabricator.wikimedia.org/T125320 [01:53:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Authmanager/SAL, Master [01:58:17] 6Labs, 10Tool-Labs: Tools web interface for tool authors (Brainstorming ticket) - https://phabricator.wikimedia.org/T128158#2065676 (10yuvipanda) [01:58:19] bd808: https://phabricator.wikimedia.org/T128158 [02:01:40] 6Labs, 10Tool-Labs, 7Epic: Tools web interface for tool authors (Brainstorming ticket) - https://phabricator.wikimedia.org/T128158#2065690 (10bd808) [02:01:48] so epic [02:01:59] bd808: indeed [02:02:04] bd808: it has several controversial ideas [02:02:21] I think we have some related tasks we can group in under it [02:02:23] such as 'not everyone who could be a person who develops tools is happy using the commandline' [02:02:25] yeah [02:02:45] bd808: I also definitely want to somehow merge/transition LDAP accounts to be not so different from other ones [02:02:54] I don't think that non-cli folks should be controversial [02:03:26] I was rambling about that (wikitech using SUL) last night [02:03:29] bd808: ah, but that is 'should' :) [02:03:59] there was certainly some "omg wft bbq!!1!" complaints from those listening [02:04:09] indeed [02:04:12] it was a bit funny actually [02:04:42] "don't take away that horrible experience because we have built other things that depend on it" [02:04:59] :D [02:05:22] I'm thinking that ldap should just be a thing that sits over in the corner quietly and gets used when it needs to [02:05:34] * bd808 just got called for dinner [02:05:55] 6Labs, 10Tool-Labs, 7Epic: Tools web interface for tool authors (Brainstorming ticket) - https://phabricator.wikimedia.org/T128158#2065691 (10bd808) p:5Triage>3Normal [02:35:02] PROBLEM - Puppet failure on tools-worker-1008 is CRITICAL: CRITICAL: 12.50% of data above the critical threshold [0.0] [02:36:28] PROBLEM - Puppet failure on tools-worker-1006 is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [0.0] [02:38:12] PROBLEM - Puppet failure on tools-worker-1007 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [02:43:10] RECOVERY - Puppet failure on tools-worker-1007 is OK: OK: Less than 1.00% above the threshold [0.0] [02:44:55] RECOVERY - Puppet failure on tools-worker-1008 is OK: OK: Less than 1.00% above the threshold [0.0] [02:46:25] RECOVERY - Puppet failure on tools-worker-1006 is OK: OK: Less than 1.00% above the threshold [0.0] [03:16:07] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1203 is OK: OK: Less than 1.00% above the threshold [0.0] [05:25:26] 10PAWS: Shared code area - https://phabricator.wikimedia.org/T128163#2065815 (10jayvdb) [06:07:17] 6Labs, 10Tool-Labs, 13Patch-For-Review: setup-tomcat does not work - https://phabricator.wikimedia.org/T118094#2065837 (10Shoichi) @scfc @zhuyifei1999 Thanks, waiting you! [07:10:18] (03CR) 10Ricordisamoa: "They should be added in a different patch then :)" [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 (owner: 10ArthurPSmith) [08:34:53] PROBLEM - Host tools-bastion-01 is DOWN: CRITICAL - Host Unreachable (10.68.17.228) [10:44:18] (03PS1) 10Addshore: Remove analytics/limn-wikidata-data [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/273428 [10:46:00] (03PS1) 10Addshore: Add analytics/wmde/.* to #wikimedia-de-tech [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/273429 [13:04:06] 6Labs, 10Labs-Infrastructure, 10DBA: Database upgrade MariaDB 10: Metadata access in INFORMATION_SCHEMA causes complete blocks - https://phabricator.wikimedia.org/T71182#2066430 (10jcrespo) ^Reason why the ban on information_schema you asked about. [13:20:37] 6Labs, 10Labs-Infrastructure, 10DBA: Database upgrade MariaDB 10: Metadata access in INFORMATION_SCHEMA causes complete blocks - https://phabricator.wikimedia.org/T71182#738054 (10AlexMonk-WMF) yeah, thanks @jcrespo. moving this subscription to my other account [14:26:31] PROBLEM - SSH on tools-webgrid-lighttpd-1204 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:56:22] RECOVERY - SSH on tools-webgrid-lighttpd-1204 is OK: SSH OK - OpenSSH_6.6.1p1 Ubuntu-2ubuntu2~wmfprecise2 (protocol 2.0) [17:05:01] I'm going to upgrade Elasticsearch on the tools-elastic-* hosts today. I don't think anyone but me is using them yet. [17:08:03] ok [17:08:23] bd808: out of my complete ignorance, what is their use case? [17:09:02] right now they are powering stashbot stuff (sal, bash) [17:09:27] Yuvi asked me to set them up because ORES is interested in using ES as storage for something [17:09:35] I think it was ORES anyway [17:10:31] gotcha [17:10:38] https://phabricator.wikimedia.org/T120040 [17:10:55] something for teahouse I guess [17:15:42] 6Labs, 10Tool-Labs, 13Patch-For-Review, 15User-bd808: Setup an experimental, user accessible (read+write) ES cluster for Tool Labs - https://phabricator.wikimedia.org/T120040#1843341 (10EBernhardson) i saw this mentioned securing elasticsearch. I dunno how useful it is, but here are some notes i took at el... [17:16:58] 6Labs, 10Tool-Labs, 13Patch-For-Review, 15User-bd808: Setup an experimental, user accessible (read+write) ES cluster for Tool Labs - https://phabricator.wikimedia.org/T120040#2066872 (10bd808) >>! In T120040#2066869, @EBernhardson wrote: > i saw this mentioned securing elasticsearch. I dunno how useful it... [17:36:38] kaldari, ping [17:36:57] howdy [17:37:09] kaldari, can't talk long but I removed the template pointer class,. [17:37:46] The solution to resolving templates was so simple, I'm surprised I never thought of it before [17:38:20] did you use the parser API? [17:38:29] action=parse yes [17:38:55] It returns externallinks found in the wikitext. :p [17:39:25] that should make things easier :) [17:39:46] I reduced it to one single function in the API class. [17:41:47] kaldari, I sent you a PM [17:45:23] Cyberpower678: Also, I wanted to ask if it would be useful in the Tool Labs API that we're building if it could return the last page that the bot processed, for when it crashes or restarts and needs to start over. Do you already have a way to pick up where it left off or would that be useful to you? [17:46:42] kaldari, From conversations many months ago, I had sophisticated memory management. Cyberbot automatically resumes where it leaves off even in multithreaded modes, with complete stats of what it did so far. [17:47:29] But thanks though [17:47:45] Cool, I was just asking since the IA tends to freak out whenever Cyberbot stops for a while. Is there anything we can do to lessen the gaps in it running? [17:48:07] The gaps usually stem from a failure in labs/toollabs [17:48:21] Cyberbot is stable [17:48:35] It only uses 30MB of RAM on average [17:48:57] Even less than that actually. [17:49:16] And it has a 1G memory allocation [17:49:36] is the labs instability just that labs totally goes down sometimes, or more specific problems? [17:50:29] It goes down, or it restarts, or their most notable incident, them trying to truncate output logs that are excessively large, and ended up deleting mine, stalling the bot tasks. [17:51:25] I'm thinking a Cyberbot project with sufficient RAM is still a viable option to house IABot, especially when it goes global. [17:51:55] But for now, that's on hold [17:52:29] Though I have to say the initial RAM demands then and now, is greatly reduced. [17:57:46] 6Labs, 10Labs-Infrastructure, 10DBA: Get HA db support for labs services - https://phabricator.wikimedia.org/T126251#2066936 (10jcrespo) As I got no answer, I went ahead created a slave on db2030. This will allow moving this service (m5) to codfw if something goes wrong with eqiad. Combined with the regular... [17:58:02] Cyberpower678: circling back on that I did not uncover any clear indicator of the logging issues experienced 2(?) weeks ago and I'm sorry that was an issue, let me know if happens again so I can try to draw some conclusions [17:58:39] chasemp, okay [18:07:47] 6Labs, 10Labs-Infrastructure, 10DBA: Get HA db support for labs internal services - https://phabricator.wikimedia.org/T126251#2066965 (10jcrespo) [18:45:31] (03PS3) 10ArthurPSmith: Fixes for nuclides to handle problem of duplicate (and some wrong) returns from SPARQL query [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 [18:46:43] (03CR) 10ArthurPSmith: [C: 031] "Ok, I removed the magic numbers piece. Ready to merge?" [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 (owner: 10ArthurPSmith) [19:08:48] !log tools Upgraded Elasticsearch on tools-elastic-0[123] to 1.7.5 [19:08:54] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master [20:07:19] 6Labs, 6Operations, 10wikitech.wikimedia.org: Update wikitech-static OS/PHP version - https://phabricator.wikimedia.org/T126385#2012874 (10Dzahn) [20:09:44] 6Labs, 6Operations, 10wikitech.wikimedia.org: Update wikitech-static OS/PHP version - https://phabricator.wikimedia.org/T126385#2067313 (10Dzahn) @Krenair i made a procurement ticket to request a second VM with jessie. The idea would be to give you access to that so you could migrate data from the old one, w... [20:33:10] 6Labs, 6Operations, 10wikitech.wikimedia.org: Update wikitech-static OS/PHP version - https://phabricator.wikimedia.org/T126385#2067409 (10Krenair) Sounds good to me [20:43:31] Krinkle: Is the JS support in Intuition usable? I see there is a JS interface, but I don't see the API that it needs. [21:46:40] PROBLEM - SSH on tools-exec-1216 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:56:31] RECOVERY - SSH on tools-exec-1216 is OK: SSH OK - OpenSSH_6.6.1p1 Ubuntu-2ubuntu2~wmfprecise2 (protocol 2.0) [22:33:50] (03CR) 10Ricordisamoa: "recheck" [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 (owner: 10ArthurPSmith) [22:40:53] (03PS4) 10Ricordisamoa: Fixes for nuclides to handle problem of duplicate (and some wrong) returns from SPARQL query [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 (owner: 10ArthurPSmith) [22:42:33] (03CR) 10Ricordisamoa: "PS4 drops magic_qid and numeric_pid which didn't belong here." [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 (owner: 10ArthurPSmith) [22:43:44] (03CR) 10Ricordisamoa: [C: 032] Fixes for nuclides to handle problem of duplicate (and some wrong) returns from SPARQL query [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 (owner: 10ArthurPSmith) [22:45:51] (03Merged) 10jenkins-bot: Fixes for nuclides to handle problem of duplicate (and some wrong) returns from SPARQL query [labs/tools/ptable] - 10https://gerrit.wikimedia.org/r/271551 (owner: 10ArthurPSmith) [22:50:31] Sorry, I have ask a request at [[Nova Resource:Tools/Access Request/Kanashimi]], may someone help me to complete the request? Thank you. [23:00:17] Thank you! [23:15:08] 10Labs-project-wikistats: Add 300 000 wikia wikis to stats table - https://phabricator.wikimedia.org/T38291#2068014 (10Dzahn) python wikia.py Traceback (most recent call last): File "wikia.py", line 24, in from wikitools import wiki, api ImportError: No module named wikitools [23:21:35] 10Labs-project-wikistats: Add 300 000 wikia wikis to stats table - https://phabricator.wikimedia.org/T38291#418526 (10Dzahn) Nevermind, there is already a generated list in the repo. git clone https://github.com/WikiTeam/wikiteam.git ~/wikiteam/listsofwikis/mediawiki$ wc -l wikia.com 410373 wikia.com we can... [23:52:07] 10Labs-project-wikistats: Add 300 000 wikia wikis to stats table - https://phabricator.wikimedia.org/T38291#2068316 (10Dzahn) imported! number of wikis: before: 115212 now: 410703 now gotta run a loooon update i guess