[00:00:07] And I find it impossible to use the CLI to do MySQL stuff. [00:00:40] I have a hard time processing that information when laid out in plain text like that. [00:03:31] Well this is depressing. [00:05:43] Cyberpower678: so your script works fine now, and mysql workbench does not? [00:06:00] yuvipanda: correct. The timeout went away on it's own. [00:06:06] I've never used mysql workbench, so am not sure what to do. I can verify that python works by default [00:07:03] Cyberpower678: is 'SSL connection error: protocol version mismatch' the error you are getting? [00:07:37] if so, from googling, the advice seems to be to upgrade mysql workbench (https://bugs.mysql.com/bug.php?id=83503) [00:07:42] https://bugs.mysql.com/bug.php?id=81977 [00:08:08] yuvipanda: [00:08:11] Your connection attempt failed for user 's51059' from your host to server at tools-db:3306: [00:08:11] SSL connection error: unknown error number [00:08:13] no, that will not work [00:08:24] don't think that'll work, Reedy [00:08:26] since it's a private CA [00:08:27] the error he is getting is normal when using SSL [00:08:35] SSL must be disable [00:08:50] yuvipanda: as you can see I have a very helpful error message. [00:09:34] Cyberpower678: as bd808 said, please tone down the snark / sarcasm. [00:09:56] :/ [00:10:04] Reedy: are you on windows? [00:10:06] Cyberpower678: you're on windows right? [00:10:11] yuvipanda: no [00:10:14] macOS [00:10:31] Cyberpower678: ok! what version of mysql workbench are you using? and are you 100% sure you have SSL / TLS disabled? [00:10:38] Yes. [00:10:42] he has SSL enabled [00:10:51] otherwise, he would not get an SSL error [00:11:04] SSL is not negotiated when disabled [00:11:16] I agree. so I think maybe the UI is confusing, and somehow SSL is getting enabled even though Cyberpower678 thinks it is not [00:11:32] Use SSL should be to no [00:11:48] yuvipanda: I have 6.3.8. It hasn't received an update in forever. [00:11:49] not to Require or If available [00:12:45] Cyberpower678: can you verify (with a screenshot, so we can see exactly what you are seeing) that 'Use SSL' is set to 'No' and not anything else? [00:13:24] yuvipanda: jynus: https://snag.gy/enkcZH.jpg [00:13:28] There's 6.3.9 on https://dev.mysql.com/downloads/workbench/ [00:13:37] (not saying it will fix it) [00:13:46] http://imgur.com/a/na4Rm [00:14:25] interesting [00:14:47] Cyberpower678: can you create a new connection and set its parameters accordingly and try again? [00:14:57] something is enabling ssl for him, then [00:15:13] yuvipanda: I'm trying Reedy's update. It's about time the app gets an update. [00:15:25] Cyberpower678: ok. try creating a new connection too [00:15:32] I suspect maybe it's using an older value there [00:15:42] what do you have in Advanced? [00:18:58] yuvipanda: jynus: https://snag.gy/rnk3Rp.jpg [00:19:54] cut others, try connection again [00:19:58] FWIW. On a clean install of workbench on osx.. SSL set to No, It complains about protocols [00:20:00] But works fine [00:20:05] Set to If Available "SSL connection error: unknown error number" [00:20:11] yes [00:20:13] that is normal [00:20:20] that is expected [00:20:25] Right [00:20:40] I was mostly testing that the macos version isn't completely borked [00:20:49] So like you say, it seems something else is still setting an ssl flag somewhere [00:21:00] maybe the driver or the config [00:21:03] or something [00:21:11] I would try deleting the connection [00:21:15] restarting [00:21:19] creating one from 0 [00:22:19] the problem is that I cannot just disable tls [00:22:31] it requires another maintenance window to restart the server [00:22:40] and I need it for replication reasons [00:23:02] If it was broken for a lot of users, there'd be a case to disable it [00:23:06] Broken for one, is not a case [00:23:39] Especially when it works for others, under the same configuration etc for all intents and purposes [00:23:50] jynus: dropping others still fails. [00:23:54] there are 74 connected users right now [00:25:32] for a 0.0.1 update it looks very different. [00:25:33] Cyberpower678: what do you mean by 'dropping others'? [00:25:38] did you create a new connection from scratch? [00:25:42] jynus: ^ [00:26:37] did you do what yuvi asked? [00:26:48] jynus: yuvipanda: Updating MySQL fixed the problem. [00:26:53] No new connection needed. [00:26:57] ah [00:27:16] Didn't know there was even an update. It has been updated in years. [00:27:43] Eh? [00:27:43] Changes in MySQL Workbench 6.3.9 (2017-02-07, General Availability) [00:27:44] Changes in MySQL Workbench 6.3.8 (2016-10-21, General Availability) [00:27:44] Changes in MySQL Workbench 6.3.7 (2016-06-13, General Availability) [00:27:53] https://dev.mysql.com/doc/relnotes/workbench/en/wb-news-6-3-9.html [00:27:55] Interface is a lot cleaner though. :-) [00:28:41] well, another client satidfied [00:28:52] Reedy: they were all minor bug fixes. None of those updates ever fixed any real issues. [00:28:52] you will also enjoy faster performance [00:29:07] This update changes the entire look and feel of the app. [00:29:08] Maybe not for you. But I'm sure they did for other people [00:29:08] and more features such as innodb fulltext [00:29:18] "In some cases, executing a query caused MySQL Workbench to become unresponsive when the host was macOS Sierra. (Bug #25036263, Bug #83658)" [00:29:19] etc. [00:29:23] Things like that sound relevant [00:29:51] "Changes made with the Options File editor were committed to my.cnf correctly, but also generated an error." [00:29:54] Reedy: is that fixed in 6.3.9? [00:29:58] Yes [00:30:05] Stuff like that sound of the ilk to the issue at hand [00:30:21] "New connections configured with the Standard TCP/IP over SSH connection method failed and caused MySQL Workbench to become unresponsive or to exit." [00:30:43] There's a lot in the changelog [00:31:39] Reedy: these were all major issues in the previous builds and none of them got fixed until now. [00:31:44] Took them long enough. [00:35:37] * Cyberpower678 is happy now. [00:37:40] Cyberpower678: cool :) glad the upgrade sorted it out [00:37:48] :-) [00:52:59] 06Labs: Delete or recreate docs.editor-engagement.eqiad.wmflabs - https://phabricator.wikimedia.org/T157701#3031806 (10Mattflaschen-WMF) The only things there were: * e3doc redirecting to growthdoc (see next) * growthdoc redirecting old GuidedTour doc URLs to https://doc.wikimedia.org/GuidedTour/master/js/ . *... [00:53:16] 06Labs: Delete or recreate docs.editor-engagement.eqiad.wmflabs - https://phabricator.wikimedia.org/T157701#3031807 (10Mattflaschen-WMF) 05Open>03Resolved [01:26:03] 10Labs-project-Wikistats, 13Patch-For-Review: Update for wikistats.wmflabs.org/display.php - https://phabricator.wikimedia.org/T157181#3031869 (10Dzahn) @Rsterbal I added tooltip texts. Try to mouse-over the column headers and you will see text now, some including links as requested. https://wikistats.wmfl... [01:30:11] 10Labs-project-Wikistats, 13Patch-For-Review: Update for wikistats.wmflabs.org/display.php - https://phabricator.wikimedia.org/T157181#3031892 (10Dzahn) 05Open>03Resolved [01:30:19] 10Labs-project-Wikistats: Update for wikistats.wmflabs.org/display.php - https://phabricator.wikimedia.org/T157181#2998242 (10Dzahn) [01:30:32] 10Labs-project-Wikistats: add column definitions on wikistats.wmflabs.org/display.php - https://phabricator.wikimedia.org/T157181#2998242 (10Dzahn) [01:36:27] thanks bd808 that was quick :) [01:36:57] samwilson: thank tgr for finally getting echo notifications hooked up to the requests :) [01:37:10] ah cool :) that sounds clever [02:04:20] PROBLEM - Puppet run on tools-docker-builder-03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [02:08:25] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Bckddn was created, changed by Bckddn link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Bckddn edit summary: Created page with "{{Tools Access Request |Justification=my app development |Completed=false |User Name=Bckddn }}" [03:05:57] 06Labs: Delete or recreate docs.editor-engagement.eqiad.wmflabs - https://phabricator.wikimedia.org/T157701#3032097 (10Andrew) \o/ thank you! [03:07:04] 06Labs, 10Labs-Infrastructure: Deprecate precise instances in Labs by 03/31/2017 - https://phabricator.wikimedia.org/T143349#3032099 (10Andrew) [04:25:18] Change on 12www.mediawiki.org a page OAuth/es was modified, changed by GermanX link https://www.mediawiki.org/w/index.php?diff=2399043 edit summary: Created page with "==Obsoleto==" [04:27:24] (03PS1) 10BryanDavis: Support Python3 [labs/tools/stashbot] - 10https://gerrit.wikimedia.org/r/338046 [04:27:26] (03PS1) 10BryanDavis: flake8 fixes [labs/tools/stashbot] - 10https://gerrit.wikimedia.org/r/338047 [04:27:28] (03PS1) 10BryanDavis: Guard against multiple bots competing in channels [labs/tools/stashbot] - 10https://gerrit.wikimedia.org/r/338048 (https://phabricator.wikimedia.org/T156895) [05:03:04] (03CR) 10BryanDavis: [C: 032] Support Python3 [labs/tools/stashbot] - 10https://gerrit.wikimedia.org/r/338046 (owner: 10BryanDavis) [05:03:49] (03CR) 10BryanDavis: [C: 032] flake8 fixes [labs/tools/stashbot] - 10https://gerrit.wikimedia.org/r/338047 (owner: 10BryanDavis) [05:04:03] (03Merged) 10jenkins-bot: Support Python3 [labs/tools/stashbot] - 10https://gerrit.wikimedia.org/r/338046 (owner: 10BryanDavis) [05:05:00] (03Merged) 10jenkins-bot: flake8 fixes [labs/tools/stashbot] - 10https://gerrit.wikimedia.org/r/338047 (owner: 10BryanDavis) [06:34:05] PROBLEM - Puppet run on tools-exec-1221 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:14:05] RECOVERY - Puppet run on tools-exec-1221 is OK: OK: Less than 1.00% above the threshold [0.0] [07:32:57] 06Labs, 10Labs-Infrastructure, 10DBA, 10Datasets-General-or-Unknown: Rebuild old timestamp format tables - https://phabricator.wikimedia.org/T151607#3032303 (10Marostegui) a:05Marostegui>03None Unassigning as there is nothing we can really do now as we are not in 10.1 in general. [09:47:42] 10Tool-Labs-tools-Quentinv57's-tools: Raw PHP code displayed on Quentinv57's stewardcounter - https://phabricator.wikimedia.org/T158289#3032479 (10MarcoAurelio) [09:48:07] 06Labs, 10Tool-Labs, 10Tool-Labs-tools-Quentinv57's-tools: Raw PHP code displayed on Quentinv57's stewardcounter - https://phabricator.wikimedia.org/T158289#3032492 (10MarcoAurelio) [09:48:58] 10Tool-Labs-tools-Quentinv57's-tools: Raw PHP code displayed on Quentinv57's stewardcounter - https://phabricator.wikimedia.org/T158289#3032479 (10MarcoAurelio) [09:50:53] 10Tool-Labs-tools-Quentinv57's-tools: Please fix bugzilla link in Tools-Quentinv57 - https://phabricator.wikimedia.org/T158290#3032495 (10MarcoAurelio) [10:48:36] Reedy: around? [10:50:43] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Bckddn was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=1528480 edit summary: [11:42:38] addshore: Am now [11:43:10] Reedy: do you think you could help tarrow with https://phabricator.wikimedia.org/T158199 (as you have access to the host)? [11:43:28] Do I? :P [11:43:47] he can read the elasticsearch logs but wants to see in the nginx logs to see if anything obvious happens [11:43:55] and yus you do ;) [11:43:55] :) [11:48:25] I should probably fix my ssh config for labs [11:48:33] reedy@tools-bastion-03:~$ ssh tools-elastic-01.tools.eqiad.wmflabs [11:48:34] * Reedy waits [11:48:54] And waits [11:49:05] debug1: Connecting to tools-elastic-01.tools.eqiad.wmflabs [10.68.20.9] port 22. [11:49:07] Is the host up? [11:49:13] It's looking kinda dead [11:50:12] Yeah, this isn't normal [11:50:20] tarrow: The host seems to be some amount of down [11:50:28] Proably explains the problem [11:50:32] hm... [11:50:55] I'll grab my phone in a few and login to wikitech and see if I can get a console/reboot [11:51:04] the actual elasticlogs don't seem to have any details about what is causing the down though... [11:51:29] it responds to ping, and that seems to be about it [11:51:40] So it would look like it's crashed [11:51:52] ssh: connect to host tools-elastic-01.tools.eqiad.wmflabs port 22: Connection timed out [11:52:17] Console may tell us more [11:52:34] actually hitting the http works though... [11:52:56] curl http://tools-elastic-01.tools.eqiad.wmflabs [11:52:58] Oh [11:53:05] So apparently you can't ssh to it from the tools bastion [11:53:08] fu---- [11:53:10] That's just stupid [11:54:05] And I can't sudo on the host [11:55:36] Reedy: o/ [11:55:50] I can ssh to it from primary.bastion.wmflabs.org [11:55:53] but I can't log in [11:56:30] I can from bastion-01 [11:56:33] But I can't sudo [11:56:42] I'm trying to remember the weird way that it's done in tools [11:57:09] !log video `pip install -U youtube-dl` on all v2c worker instances, version 2017.1.10 -> 2017.2.16 [11:57:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Video/SAL [11:59:44] !log video depooled v2c worker at encoding01, depooled & repooled encoding02, repooled encoding03 [11:59:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Video/SAL [12:00:08] zhuyifei1999_: o/ [12:00:12] btw it works now ;p [12:00:31] JustBerry: (meh) [12:00:39] zhuyifei1999_: took too long lol [12:00:49] Now, I actually have to design the diff comparing stuff and whatnot [12:00:55] Still waiting on the mediawiki API upgrade though [12:01:00] Unless I use the pywikibot [12:01:05] use pywikibot* heh [12:01:07] This page isn’t working [12:01:07] wikitech.wikimedia.org is currently unable to handle this request. [12:01:08] HTTP ERROR 500 [12:01:40] Reedy: regular browsing or you're sending an HTTP request via a program you wrote? [12:01:45] (website loads for me) [12:11:44] 06Labs, 10MediaWiki-extensions-OpenStackManager, 10wikitech.wikimedia.org: HTTP 500 on Special:NovaSudoers - https://phabricator.wikimedia.org/T158299#3032736 (10Reedy) [12:22:28] Reedy: I don't suppose you got anywhere with reading the nginx logs? [12:23:54] Trying to work out how the feck I'm supposed to sudo on tools [12:25:48] Reedy: can't you modify LDAP records? [12:26:00] zhuyifei1999_: I'm supposed to have sudo access [12:26:02] iirc, members of tools.admin can sudo [12:26:09] I'm a project admin on tools [12:26:21] hmm [12:26:33] There was something quirky about it [12:26:52] 06Labs: Provide snapshot of http://tools-elastic-01.tools.eqiad.wmflabs logs - https://phabricator.wikimedia.org/T158199#3032800 (10Tarrow) I've discovered that I can read the elasticsearch logs by connecting to tools-elastic-01.tools.eqiad.wmflabs and looking in /var/log/elasticsearch. These don't seems to con... [12:27:34] sudo -l seems to work [12:47:09] 06Labs, 10wikitech.wikimedia.org: Array to string conversion in SMW_SQLStore3_Writers.php - https://phabricator.wikimedia.org/T158302#3032819 (10Reedy) [12:48:48] 06Labs, 10wikitech.wikimedia.org: Array to string conversion in SMW_SQLStore3_Writers.php - https://phabricator.wikimedia.org/T158302#3032834 (10Reedy) https://github.com/wikimedia/mediawiki-extensions-SemanticMediaWiki/blob/1.8.x/includes/storage/SQLStore/SMW_SQLStore3_Writers.php#L383 [12:56:27] 06Labs, 10Tool-Labs, 07Tracking: Please delete tool 'ut-iw-bot' - https://phabricator.wikimedia.org/T158303#3032846 (10Udo_T) [12:56:55] 06Labs, 10Tool-Labs: Please delete tool 'ut-iw-bot' - https://phabricator.wikimedia.org/T158303#3032859 (10zhuyifei1999) [12:57:51] 06Labs, 06Operations, 10wikitech.wikimedia.org: Expand list of people who can create new Labs project - https://phabricator.wikimedia.org/T101688#3032860 (10scfc) 05Open>03Resolved >>! In T101688#1390474, @Legoktm wrote: > Do we currently have an issue with projects not being created in a timely manner?... [12:58:10] 06Labs, 06Operations, 10wikitech.wikimedia.org: Expand list of people who can create new Labs project - https://phabricator.wikimedia.org/T101688#3032862 (10scfc) 05Resolved>03declined [13:00:22] 06Labs, 07Tracking: New Labs project requests (tracking) - https://phabricator.wikimedia.org/T76375#3032869 (10scfc) [13:00:24] 06Labs: Request creation of Mathematical Refresh Rate Policies labs project - https://phabricator.wikimedia.org/T143901#3032867 (10scfc) 05Open>03declined @Agaherbert, if you still need a Labs project, please reopen this task and comment on the questions @Andrew and @chasemp raised. Thanks! [13:02:18] 06Labs, 10Tool-Labs: Create Phabricator tag for tasks that require extended permissions - https://phabricator.wikimedia.org/T157856#3032889 (10scfc) p:05Triage>03Normal [13:02:35] 06Labs, 10Tool-Labs: Delete tool hujibot - https://phabricator.wikimedia.org/T157916#3032892 (10scfc) p:05Triage>03Normal [13:10:44] 06Labs, 10Tool-Labs, 10Tools-Kubernetes, 13Patch-For-Review: Allow running cronjobs on k8s - https://phabricator.wikimedia.org/T158155#3032897 (10scfc) p:05Triage>03Normal @yuvipanda: I successfully created a scheduled job by copying the example `kubectl run hello --schedule="*/1 * * * *" --restart=OnF... [13:11:18] 06Labs, 10Tool-Labs: Improve `webservice status` output - https://phabricator.wikimedia.org/T158244#3032901 (10scfc) p:05Triage>03Normal [13:12:26] 06Labs: Request creation of wikifactmine labs project - https://phabricator.wikimedia.org/T157385#3032903 (10scfc) @Andrew, @chasemp: Could you take a look at this, please? Thanks. [13:17:05] andrewbogott: is there any general purpose postgre DB on labs? I guess not [13:28:27] 06Labs, 06Operations, 10wikitech.wikimedia.org: wikitech regularly looses session directly after login - https://phabricator.wikimedia.org/T118395#3032915 (10scfc) 05Open>03Invalid I cannot reproduce this. Please reopen if the problem reoccurs. [13:33:55] 06Labs, 10wikitech.wikimedia.org: Wikitech sign-up page has bad styling - https://phabricator.wikimedia.org/T136032#3032920 (10scfc) 05Open>03Resolved a:03Krinkle AFAICT, all issues have been addressed. Thanks, @Krinkle. [13:35:28] 06Labs, 10wikitech.wikimedia.org, 13Patch-For-Review: Labs front-page statistics are very wrong - https://phabricator.wikimedia.org/T139773#3032924 (10scfc) 05Open>03Resolved >>! In T139773#2809983, @Andrew wrote: > Instance counts are now accurate! [13:45:58] 06Labs, 10MediaWiki-extensions-OAuth, 10wikitech.wikimedia.org, 13Patch-For-Review: OAuth api access on wikitech fails with consumed nonce error - https://phabricator.wikimedia.org/T149150#2743542 (10scfc) @bd808: Is there anything left to do here? [14:01:27] 06Labs, 10Wikimedia-Site-requests, 10wikitech.wikimedia.org: Determine file strategy for wikitech - https://phabricator.wikimedia.org/T144495#2601575 (10scfc) AFAIUI, `wikitech` is meant to be somewhat separate from the cluster so that, if the cluster goes down, https://wikitech-static.wikimedia.org/ can sti... [14:03:36] 06Labs, 10WM-Bot: Move wm-bot instance to Trusty - https://phabricator.wikimedia.org/T157838#3032966 (10Petrb) @Andrew is it possible to request some general purpose storage? We used to have /data/project for this in past, but this new project doesn't have this mount point and instances are created incredibly... [14:06:12] 06Labs, 10SyntaxHighlight, 10wikitech.wikimedia.org: Extension:SyntaxHighlight_GeSHi reports for pages with syntax highlighting errors are bogus - https://phabricator.wikimedia.org/T153616#3032970 (10scfc) Some false positives have disappeared from the list, and https://wikitech.wikimedia.org/w/api.php?actio... [14:12:44] 06Labs, 10Labs-Infrastructure, 10MediaWiki-extensions-OpenStackManager, 10wikitech.wikimedia.org: Wikitech API is many hours behind, at least - https://phabricator.wikimedia.org/T149969#3033017 (10scfc) 05Open>03Resolved IIRC since this task was filed, @Andrew fixed some OpenStack hooks, and @Krenair m... [14:24:10] How lovely, my current write speed on /data/scratch seems to be about 100 KiB/s [14:48:11] 06Labs, 10wikitech.wikimedia.org: Wikitech: rename 'shellmanagers' to 'shellmanager' for consistency - https://phabricator.wikimedia.org/T158039#3033105 (10scfc) I don't find any matches in `operations/puppet` and `mediawiki/extensions/OpenStackManager`, but until now I wasn't even aware of `mediawiki/extensio... [15:19:47] 06Labs, 10wikitech.wikimedia.org: Can't reset password on wikitech (Unicode passwords not accepted), due to LDAP/opendj? - https://phabricator.wikimedia.org/T58114#3033181 (10scfc) @Nemo_bis: Were you able to reproduce this with the new LDAP server? [15:22:03] 06Labs: Request creation of wikifactmine labs project - https://phabricator.wikimedia.org/T157385#3033184 (10chasemp) seems reasonable, +1 from me. Default quota should suffice. @andrew second? [15:22:32] 06Labs: Provide snapshot of http://tools-elastic-01.tools.eqiad.wmflabs logs - https://phabricator.wikimedia.org/T158199#3033186 (10chasemp) is this superceded by {T157385}? [15:24:22] 06Labs: Provide snapshot of http://tools-elastic-01.tools.eqiad.wmflabs logs - https://phabricator.wikimedia.org/T158199#3033188 (10Tarrow) Not really. If I can get away with not having to run my own elastic cluster but can instead use the communal tool-labs one that would be cool. [15:53:34] 06Labs, 10wikitech.wikimedia.org: Wikitech: rename 'shellmanagers' to 'shellmanager' for consistency - https://phabricator.wikimedia.org/T158039#3033309 (10Dereckson) WikimediaMessages is an extension to provide localisation especially for Wikimedia projects, by opposite to the MediaWiki software. [16:13:43] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Nodepool quota bump - https://phabricator.wikimedia.org/T158320#3033347 (10hashar) [16:14:40] 06Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure: Nodepool quota bump - https://phabricator.wikimedia.org/T158320#3033347 (10hashar) [16:20:20] andrewbogott: the new Horizon is more friendly, specially when creating instances. Thx !!! [16:30:17] 06Labs, 10wikitech.wikimedia.org: Project Bastion has service groups - https://phabricator.wikimedia.org/T64537#3033454 (10scfc) Now there are `deadlinkfinder`, `jybot` and `linktranslator`. [16:30:59] 06Labs, 10wikitech.wikimedia.org: Project Bastion has service groups - https://phabricator.wikimedia.org/T64537#3033456 (10scfc) [16:44:11] 06Labs, 10MediaWiki-extensions-OpenStackManager, 10wikitech.wikimedia.org: Restrict creating service groups to white-listed projects - https://phabricator.wikimedia.org/T158328#3033516 (10scfc) [16:50:53] Hello! Is there anybody who can help me with a problem regarding a bot written in C#, a problem that exists since today (two bots run every day and they never failed this way, but today both keep failing) [16:51:35] Some Exception which is triggered deep within MySql.Data.MySqlClient.MySqlConnection.Open() and has to do something with the character set [16:56:44] 06Labs, 10Labs-Infrastructure: Have shell requests marked as uncompleted or completed automatically - https://phabricator.wikimedia.org/T47456#509687 (10scfc) [16:56:46] 06Labs, 10wikitech.wikimedia.org: Cleanup and enable UserFunctions extension on wikitech - https://phabricator.wikimedia.org/T47455#3033584 (10scfc) 05stalled>03declined T47456 has been closed, so there is no longer an incentive to deploy this extension on #wikitech, and given that https://www.mediawiki.or... [17:21:02] Wurgl: we saw a webservice yesterday with a problem related to trying to negotiate SSL connections with the db server. Haven't heard of charset issues but certainly possible. [17:22:00] We switched to a new database server version yesterday and clients may be negotiating different default settings with the new servers [17:22:53] Wurgl: I would suggest looking at the connection settings possible for the db connector you are using and trying to set things explicitly to work around any issues you are seeing [17:23:32] bd808: don't suppose I can poke you into taking a look at https://phabricator.wikimedia.org/T158199? I'm trying to work out why I'm getting errors back from the tools elasticsearch [17:24:11] tarrow: I think I can look in about 30-45 minutes. Stuck in meetings right now. [17:24:22] I have the bug open in a tab at least :) [17:24:37] bd808: thanks :) [17:37:19] I use actually two database servers: conn_user = new MySql.Data.MySqlClient.MySqlConnection("server=tools-db;uid=s51412;pwd=;database=s51412__data;"); [17:37:42] bd808: conn_dewp = new MySql.Data.MySqlClient.MySqlConnection("server=dewiki.labsdb;uid=s51412;pwd=;database=dewiki_p;charset=utf8"); [17:38:04] Which one changed? or both? [17:38:26] What version of the mysql .net provider are you using? a recent one? [17:38:39] Wurgl: both (actually all) db servers changed yesterday [17:38:46] 3.15 which was the newest somehen in January when I looked [17:38:57] Yes, now 3.5 ist the newest [17:39:05] Aha! [17:39:08] Wait [17:39:22] 3.15 was the framework f�r the bot-part [17:39:42] Wurgl: you may want to browse this page -- https://dev.mysql.com/doc/connector-net/en/connector-net-connection-options.html [17:40:06] Wurgl: Try adding... "SSL Mode=No" to your connection string [17:40:19] bd808: If it's the same issue as workbench was having... [17:40:31] yeah, that was the fix for the bug in another c# tool yesterday [17:40:36] With .net i do not know. The executable was build by the author (which is currently not accessible), I am just the maintainer [17:40:49] if you are having charset issues maybe look at CharSet settings [17:40:55] Ah. No [17:40:58] SSL mode=None [17:42:24] I am a C/C++ guy (and a Linux-User/Windows-avoider). I have never ever touched this C# thing. Is there a good page how to set up everything? [17:42:44] You'll need to change the code, and recompile [17:43:02] Yes! I konw that [17:43:40] You just need a text editor (possibly a version control tool if used) and then just run mcs? [17:43:54] I mean how to set up the developer studio and that parts. I already have dev. studio in Win/7 f�r a large C++ project [17:45:33] 10Tool-Labs-tools-Other, 07Epic: Toolserver.org tools that have not been migrated (tracking) - https://phabricator.wikimedia.org/T60865#3033812 (10scfc) [17:45:36] 10Tool-Labs-tools-Other: Migrate http://toolserver.org/~purodha/sample/dbswithuser.php to Tool Labs - https://phabricator.wikimedia.org/T63028#3033810 (10scfc) 05stalled>03declined In that case, please file an RFE at https://github.com/Pathoschild/Wikimedia-contrib/issues or in #MediaWiki-extensions-CentralA... [17:46:00] Text editor is no problem, I use vi since *hmm* 1987? Version tool � The author does not use one, so I do not too :-) [17:46:37] To change the connection string, you do not need visual studio [17:51:33] I will try. Maybe today, maybe tomorrow � Thanks [17:51:59] 06Labs, 10Striker, 10Tool-Labs, 10community-labs-monitoring: Preperation for api for community-labs-monitoring - https://phabricator.wikimedia.org/T157847#3018687 (10scfc) p:05Triage>03Normal [18:44:19] 06Labs, 10Tool-Labs: templatetiger is using 613G in Tools out of 8T - https://phabricator.wikimedia.org/T136192#3034042 (10Kolossos) @yuvipanda Yes it works fine. [18:53:11] PROBLEM - Puppet run on tools-worker-1021 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [18:55:31] o/ I've followed https://wikitech.wikimedia.org/wiki/Help:MediaWiki-Vagrant_in_Labs#Setting_up_your_instance_with_MediaWiki-Vagrant and have got to step 11 (https://extlinkschange-testwiki.wmflabs.org/) and am getting 502 Bad Gateway [18:55:34] any ideas? [18:56:10] (step 10 - proxy name is the same as hostname) [18:58:55] 06Labs, 10Tool-Labs, 10Tools-Kubernetes, 13Patch-For-Review: Allow running cronjobs on k8s - https://phabricator.wikimedia.org/T158155#3034120 (10Legoktm) Shouldn't image be something like `docker-registry.tools.wmflabs.org/toollabs-python-web:latest`? To resolve this task I think we need documentation so... [19:01:26] (doing a `vagrant provision`) [19:07:46] and now it's fixed! :D [19:16:52] 06Labs, 10Tool-Labs: /usr/local/bin/killgridjobs.sh is outdated - https://phabricator.wikimedia.org/T157792#3016568 (10scfc) https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Admin references neither `exec-manage` nor the scripts under `modules/toollabs/files/gridscripts/` at the moment :-) (and the SOP... [19:17:23] 06Labs, 10Tool-Labs: /usr/local/bin/killgridjobs.sh is outdated - https://phabricator.wikimedia.org/T157792#3034180 (10scfc) p:05Triage>03Low [19:28:12] RECOVERY - Puppet run on tools-worker-1021 is OK: OK: Less than 1.00% above the threshold [0.0] [19:35:35] what's the right way to temp. disable puppet on a labs instance. puppet agent --disable is enabled again after a reboot [19:35:57] but would like to test something that involves systemd units and if they come up after reboot [19:36:20] paladox: ^ right? [19:36:49] mutante: good q. idk [19:37:06] how would you do this in prod? [19:37:50] i dont know :) usually i dont need it to survive a reboot [19:38:03] :D we're all in the same boat then :) [19:38:30] maybe with "systemctl mask puppet" [19:38:33] hmm [19:38:38] let's try this [19:38:41] https://docs.puppet.com/puppet/latest/man/agent.html#OPTIONS [19:38:59] paladox: let's try "mask" to disable the puppet service [19:39:08] oh what's mask? [19:39:40] mutante ^^ [19:40:03] it's to permanently disable a service [19:40:08] more than just stopping it [19:40:13] https://fedoramagazine.org/systemd-masking-units/ [19:40:46] ah [19:40:52] so systemctl disable puppet.service [19:40:52] ? [19:40:58] "third level of 'off'" [19:41:43] sudo systemctl mask httpd.service [19:41:48] sudo systemctl mask puppet.service [19:41:52] disable != mask [19:41:55] but you can try both [19:42:02] root@phabricator:/home/paladox# sudo systemctl mask puppet.service [19:42:02] Created symlink from /etc/systemd/system/puppet.service to /dev/null. [19:42:12] root@phabricator:/home/paladox# systemctl status puppet [19:42:12] ● puppet.service [19:42:12] Loaded: masked (/dev/null) [19:42:13] Active: inactive (dead) [19:42:21] Failed to start puppet.service: Unit puppet.service is masked. [19:42:39] does it prevent it from overwriting your unit file now? [19:42:39] root@phabricator:/home/paladox# sudo puppet agent -tv [19:42:40] Info: Retrieving pluginfacts [19:42:41] Info: Retrieving plugin [19:42:42] I [19:42:45] mutante ^^ puppet agent works [19:42:45] this should survive a reboot [19:43:19] oh but puppet agent still works [19:43:31] presummingly because it can run without the puppet service. [19:44:07] mutante ^^ [19:44:49] hrmm.. was this a puppet::self instance? [19:45:05] well puppet-phabricator is [19:45:12] phabricator is a client of puppet-paladox [19:45:13] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Provide a migration path for tools running tomcat - https://phabricator.wikimedia.org/T141396#3034303 (10scfc) p:05Triage>03Low [19:45:25] i mean puppet-phabricator [19:45:29] mutante ^^ [19:45:49] paladox: heh, well, you can just stop the master i guess :p [19:45:57] oh [19:45:58] then it cant run for the test.. then turn it on again [19:46:02] ok [19:46:11] should this bug be reported? [19:46:28] i dont know if it can be called bug [19:47:07] mutante stopping puppetmaster still lets puppet agent work [19:47:25] 06Labs, 10Tool-Labs: About 71 users are missing replica.my.cnf - https://phabricator.wikimedia.org/T140592#3034313 (10scfc) There are now 84 users with missing `replica.my.cnf`, so this task is not resolved: ``` root@tools-bastion-03:~# (cd /home && for dir in *; do if ! [ -f "$dir/replica.my.cnf" ]; then ech... [19:47:32] i think all this may be fixed in puppet 4.x. But who knows :) [19:47:46] 06Labs, 10Tool-Labs: About 71 users are missing replica.my.cnf - https://phabricator.wikimedia.org/T140592#3034314 (10scfc) p:05Triage>03Normal [19:47:48] paladox: eh, wait. it must say something about "cant talk to master" , right [19:48:04] nope [19:48:05] otherwise i'd doubt it's actually talking to the master you think it talks to [19:48:06] it works [19:48:16] is it actually talking to itself? [19:48:21] and is puppet::self [19:48:39] let me kill the port [19:48:40] 8140 [19:49:03] are you positive the master service is stopped on the master? check the log file there while you run agent ? [19:52:19] 06Labs, 10Tool-Labs: Document all tables missing in replicas - https://phabricator.wikimedia.org/T139289#3034332 (10scfc) p:05Triage>03Low [19:52:21] yep i have checked by running systemctl status puppetmaster [19:52:23] mutante ^^ [19:52:50] root@puppet-phabricator:/home/paladox# systemctl start puppetmaster [19:52:50] syJob for puppetmaster.service failed. See 'systemctl status puppetmaster.service' and 'journalctl -xn' for details. [19:53:04] seems something is running on the port. [19:53:08] 06Labs, 10Tool-Labs: enwiki_p DB corruption - https://phabricator.wikimedia.org/T138954#3034348 (10scfc) p:05Triage>03Low [19:53:21] wait, it wasn't running the whole time? [19:53:25] Could not run: Address already in use - bind(2) for 0.0.0.0:8140 [19:53:31] Yep [19:53:34] looks like that [19:53:35] but you say "puppet agent -tv" still works ? [19:53:39] yep [19:53:41] then it was never the master for this [19:54:04] oh, but the port is on puppet-phabricator [19:54:05] what is running on 8140 [19:54:09] per error message Could not run: Address already in use - bind(2) for 0.0.0.0:8140 [19:54:12] how do i check that? [19:54:26] netstat -tulpen | grep 8140 [19:54:39] thanks [19:54:40] tcp6 0 0 :::8140 :::* LISTEN 0 59530 17609/apache2 [19:55:02] 06Labs, 10Tool-Labs: DNS resolution sometimes fails on tools-bastion-03 - https://phabricator.wikimedia.org/T143194#3034356 (10scfc) p:05Triage>03Low >>! In T143194#2846046, @Samwilson wrote: > […] > But I'm wondering if it's actually related to being rate-limited, and the Guzzle library is mis-reporting t... [19:55:20] i guess "puppetmaster" does not have a valid systemd unit file then [19:55:25] is it jessie? [19:55:25] ah [19:55:34] shutting down apache on puppet-phabricator [19:55:39] makes the agent not work [19:55:40] :) [19:55:53] Error: Could not send report: Connection refused - connect(2) for "puppet-phabricator.phabricator.eqiad.wmflabs" port 8140 [19:56:10] paladox: ok [19:56:23] well now you confirmed that was the master , good [19:56:30] you can try the reboot of the client [19:56:39] ok [19:56:45] meanwhile, #puppet said that "systemctl disable puppet" should have worked too [19:56:52] to disable it on that side [19:57:27] we don't run puppet as a service [19:57:28] it did not keep you from manually running the agent, but it should have kept it from automatically running and overwriting your file [19:57:29] so that wouldn't work [19:57:38] puppet runs with cron in our infrastructure [19:58:12] oh [19:58:23] oh, right, is it the same in labs as in prod? [19:58:50] ok, then it'd be to disable the cron [19:59:13] puppet agent --disable [19:59:29] and then for a manual run puppet agent --enable && puppet agent -tv && puppet agent --disable [19:59:37] is what I normally do in that situation [19:59:42] did that, but then when you reboot it re enables it's self. [19:59:51] valhallasw`cloud: yes, but it's because of the reboot [20:00:00] in this case [20:00:19] we wanted to replace a systemd unit file and see if services come up after boot [20:01:22] why would the lock file disappear on reboot? /me is confused [20:01:32] Hm, maybe we have a 'smart cleanup' that cleans them up on reboot, as our reboots are typically unclean and thus leave lock files around. [20:02:31] yes, same here. i was just going by the report that it was enabled again after reboot [20:03:59] that's where this started, basically [20:04:58] 06Labs, 06Operations, 06Release-Engineering-Team: contintcloud project thinks it is using 206 fixed-ip quota errantly - https://phabricator.wikimedia.org/T158350#3034394 (10chasemp) [20:05:03] 06Labs, 06Operations, 06Release-Engineering-Team: contintcloud project thinks it is using 206 fixed-ip quota errantly - https://phabricator.wikimedia.org/T158350#3034406 (10chasemp) p:05Triage>03High [20:05:10] and yea, the thing we wanted to test, unit file for phab phd service.. does fail.. so good that we tried [20:05:58] 06Labs, 06Operations, 06Release-Engineering-Team: contintcloud project thinks it is using 206 fixed-ip quota errantly - https://phabricator.wikimedia.org/T158350#3034394 (10chasemp) a:03Andrew currently nodepool is going along fine except the quota is clearly wrong. I don't yet understand why the current... [20:08:56] Hey all, trying to get eventlogging working on https://extlinkschange-testwiki.wmflabs.org - have added the role and added the extension the hook is in (SpamBlacklist) - however nothing is getting logged in /vagrant/logs/eventlogging.log and /var/log/upstart/eventlogging-devserver.log isn't showing anything either [20:09:03] any ideas what I might be missing? [20:14:48] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Setup Kubernetes Masters in a HA setup - https://phabricator.wikimedia.org/T142862#3034434 (10scfc) p:05Triage>03Normal [20:42:10] samtar: are the EL events server side or client side? If client side, can you see the HTTP requests being made in your browser's debug tools? [20:58:00] 06Labs, 10Labs-Infrastructure, 10Tool-Labs, 07Wikimedia-Incident: Write a simple script that handles failovering proxies - https://phabricator.wikimedia.org/T143639#3034525 (10scfc) p:05Triage>03Low [21:01:11] 06Labs, 10Tool-Labs: bigbrother hosts missing exec packages - https://phabricator.wikimedia.org/T143458#3034531 (10scfc) p:05Triage>03Normal [21:01:39] 06Labs, 10Tool-Labs: s51704 had multiple long-running (~1 hour) concurrent queries before labsdb crashed - https://phabricator.wikimedia.org/T142358#3034537 (10scfc) p:05Triage>03Low [21:03:24] 06Labs, 10Tool-Labs, 06Discovery, 06Maps: p50380g50921 has 20+ open persistent connections to labsdb1001 & labsdb1003 - https://phabricator.wikimedia.org/T142356#3034543 (10scfc) p:05Triage>03Low [21:04:06] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Add a nice and simple perl5 webservice type based on modern perl5 principles - https://phabricator.wikimedia.org/T141572#3034550 (10scfc) p:05Triage>03Low [21:06:30] 10Quarry: Users blocked from account creation on meta can not use Quarry - https://phabricator.wikimedia.org/T157342#3002152 (10Tgr) Should be fixed with the next train. [21:06:48] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Move kubernetes authentication to using X.509 client certs - https://phabricator.wikimedia.org/T144153#3034563 (10scfc) p:05Triage>03Low [21:07:06] 06Labs, 10Tool-Labs: Setup monitoring for cdnjs git pull - https://phabricator.wikimedia.org/T144215#3034565 (10scfc) p:05Triage>03Low [21:14:38] 06Labs, 10Tool-Labs: Rethink use of floating IPs in tools - https://phabricator.wikimedia.org/T141445#3034575 (10scfc) p:05Triage>03Low Cf. also T151704 to ask Freenode to lift/ease restrictions on IRC bots. For applications on the grid we could define a resource `has_own_ip` that only execution nodes wit... [21:22:43] 06Labs: Provide snapshot of http://tools-elastic-01.tools.eqiad.wmflabs logs - https://phabricator.wikimedia.org/T158199#3034582 (10bd808) There are a lot of error log entries that look like: ``` 2017/02/15 09:55:45 [crit] 714#0: *1437567 connect() to 127.0.0.1:9200 failed (99: Cannot assign requested address) w... [21:23:32] tarrow: ^ some info for you there. TL;DR your tool is making too many connections for nginx to keep up [21:25:18] bd808: thanks! I figured it might be like that. I'm using the official client and I thought it should be limited to 10 concurrent connections but obviously it isn't. :-) [21:27:17] The problem may not be concurrency, but instead the bursts. [21:28:22] nginx opens a TCP socket to talk to Elasticsearch and only has a pool of 30k port numbers to that with. The sockets linger for a while (not sure how long at the moment) after your client is done [21:28:53] so you are effectively rate limited to 30k total connections in (unspecified) time [21:29:11] Ah, that makes sense! I should really be using bulk for the indexing [21:29:23] yes [21:29:27] that should help a lot [21:29:46] But previously I was running it on a cluster without a proxy in front of it and it 'just worked' so I didn't bother [21:30:03] yeah. makes sense [21:30:22] Thanks so much for taking a look! :-) [21:30:41] 06Labs, 10Tool-Labs: Freenode sometimes throttles bot connections from tools - https://phabricator.wikimedia.org/T151704#3034592 (10scfc) p:05Triage>03Low [21:30:50] batch inserts are much nicer for the Elasticsearch side too if lots of things are going into the same index [21:34:50] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 06Developer-Relations, and 2 others: Developing community norms for vital bots and tools - https://phabricator.wikimedia.org/T149312#3034611 (10scfc) p:05Triage>03Low [21:35:12] 06Labs, 10MediaWiki-extensions-OAuth, 10wikitech.wikimedia.org, 15User-bd808: OAuth api access on wikitech fails with consumed nonce error - https://phabricator.wikimedia.org/T149150#3034613 (10bd808) 05Open>03Resolved a:03bd808 >>! In T149150#3032939, @scfc wrote: > @bd808: Is there anything left to... [21:36:10] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 10Tools-Kubernetes: My first kubernetes + python3 + django app tutorial - https://phabricator.wikimedia.org/T149191#3034623 (10scfc) p:05Triage>03Low [21:37:35] 06Labs: Provide snapshot of http://tools-elastic-01.tools.eqiad.wmflabs logs - https://phabricator.wikimedia.org/T158199#3034625 (10bd808) h/t to https://ma.ttias.be/nginx-cannot-assign-requested-address-for-upstream/ for leading me to think about TCP port exhaustion as the possible culprit here. [21:51:59] (03CR) 10Lokal Profil: [C: 032] Refactor database_statistics.getStatistics [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/331407 (owner: 10Jean-Frédéric) [21:53:31] (03Merged) 10jenkins-bot: Refactor database_statistics.getStatistics [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/331407 (owner: 10Jean-Frédéric) [21:54:19] 06Labs, 10SyntaxHighlight, 10wikitech.wikimedia.org: Extension:SyntaxHighlight_GeSHi reports for pages with syntax highlighting errors are bogus - https://phabricator.wikimedia.org/T153616#3034659 (10bd808) ``` silver:~ bd808$ mwscript showJobs.php --wiki=labswiki --group categoryMembershipChange: 7 queued;... [21:57:52] (03CR) 10jenkins-bot: Refactor database_statistics.getStatistics [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/331407 (owner: 10Jean-Frédéric) [21:58:20] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Hall1467 was created, changed by Hall1467 link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Hall1467 edit summary: Created page with "{{Tools Access Request |Justification=I would like to access Wikidata database replicas to run queries |Completed=false |User Name=Hall1467 }}" [21:59:04] Reedy? [21:59:10] aren't you able to query the wikidata db without even needing access to tools or labs [21:59:14] xD [21:59:18] Wurgl: ? [21:59:25] p.s im referencing that hall1467 request [22:00:03] Reedy: I can build an executable. But it failes during init of DotNetWikiBot [22:00:28] Tries to download the file http://sourceforge.net/p/dotnetwikibot/svn/HEAD/tree/cache/CommonData.xml?format=raw but this does not work? [22:00:53] WFM locally? [22:01:21] Sorry, what is WFM? [22:01:30] works for me [22:01:34] Where are you running it? [22:01:39] on tools-bastion [22:02:16] I think at home I do not have those databases [22:02:38] I can't remember if labs needs a proxy for things [22:02:50] Does it work if you use curl/wget on the command line? [22:03:17] wget works fine [22:04:15] Is the source code public? [22:04:18] Visual C# 2010 Express [22:04:38] Ist this to new? [22:04:47] Shouldn't be [22:04:52] MS have released many versions since [22:05:34] The sources I got (after asking several times) hold a .csproj file which was converted during opening into *.sln [22:06:05] So it was developed/compiled with some older version (just guessing) [22:06:13] probably, or sln not commited [22:06:20] shouldn't matter [22:08:43] However, tomorrow is soon, its beer time now :-) [22:08:48] Wurgl: that link redirects to https://sourceforge.net/p/dotnetwikibot/svn/HEAD/tree/cache/CommonData.xml?format=raw [22:08:57] so it may be the http -> https redirect is not handled correctly [22:09:11] try changing http to https and see if that fixes it? [22:10:03] Wonder why it worked before? [22:10:11] Or is the tool running an old version? [22:10:49] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 10Tools-Kubernetes: My first kubernetes + python3 + django app tutorial - https://phabricator.wikimedia.org/T149191#3034697 (10Tobias1984) @scfc Should we perhaps close this issue. It seems the issue I had with the paths has been solved and it is now also do... [22:12:20] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 10Tools-Kubernetes: My first kubernetes + python3 + django app tutorial - https://phabricator.wikimedia.org/T149191#3034702 (10bd808) I still want something like https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/My_first_Flask_OAuth_tool but with a Django f... [22:13:22] (03Draft2) 10Zppix: Adding configuration for tool-quarrybot-enwiki [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/338265 [22:13:37] Reedy: the redirect might be new [22:14:01] serving all downloads over https is a fairly new concept :-) [22:14:05] https is the same as http � it hangs :-( [22:14:13] 06Labs: Update Labslogbot to authenticate to wikitech using bot password or OAuth - https://phabricator.wikimedia.org/T150569#3034720 (10scfc) p:05Triage>03Low (Because Extension:OpenStackManager's (dead?) code confused me and I couldn't find the strings from https://wikitech.wikimedia.org/w/index.php?title=... [22:14:36] Wurgl: no errors at all, just hangs? [22:14:52] The code where it hangs is public, ist is DotNetWikiBot.cs from here http://dotnetwikibot.sourceforge.net/ [22:15:06] (03PS3) 10Zppix: Adding configuration for tool-quarrybot-enwiki [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/338265 (https://phabricator.wikimedia.org/T158355) [22:15:11] Console.WriteLine("Before init " + address); [22:15:12] Bot.InitWebClient(); [22:15:12] Console.WriteLine("Init done " + postData); [22:15:12] if (string.IsNullOrEmpty(postData)) [22:15:13] webResourceText = Bot.webClient.DownloadString(address); [22:15:13] else [22:15:14] webResourceText = Bot.webClient.UploadString(address, postData); [22:15:16] Console.WriteLine("loop end"); [22:15:55] 10Wikibugs, 10QuarryBot-enwiki, 13Patch-For-Review: wikibugs config for repo messages - https://phabricator.wikimedia.org/T158355#3034729 (10Zppix) [22:16:20] The do not like too many lines :-( [22:16:25] Where's the source code? [22:16:45] http://dotnetwikibot.sourceforge.net/ <-- click on download. [22:16:59] This is the framework which is used [22:17:19] Wurgl please either pastebin.com for sending lines of code for future reference (this is to prevent flood prevention from kicking you from the irc server) [22:17:21] 06Labs, 10Tool-Labs: Make webservice command read default cli arguments from ~/.webservicerc - https://phabricator.wikimedia.org/T148872#3034732 (10scfc) p:05Triage>03Low +1 for `~/.webservicerc` being `argparse` and identical to the known command line arguments and not YAML. [22:17:52] And I use the first debugger ever: I add messages to the console [22:17:55] 06Labs, 10Tool-Labs: Send "are you there?" email to tool labs members every 3 months to revalidate email address - https://phabricator.wikimedia.org/T148792#3034735 (10scfc) p:05Triage>03Low [22:18:28] webResourceText = Bot.webClient.DownloadString(address); <-- this is the line where it throws an exception [22:19:17] public static WebClient webClient = new WebClient(); <-- the type of "webClient" [22:19:22] catch it and get more information from it? [22:19:46] The request timed out <-- this is what the exception says [22:22:06] I think tomorow is another day [22:22:47] I will try from at home without the database on Windows native � [22:22:54] However, thanks! [22:37:08] 06Labs, 10Tool-Labs: Expose tool-labs service names via environment variables - https://phabricator.wikimedia.org/T151002#3034792 (10scfc) p:05Triage>03Low Switching the Tool Labs Redis server between `tools-redis-1001` and `tools-redis-1002` is currently done by changing the alias `tools-redis` which is i... [22:38:00] 06Labs, 10Tool-Labs, 10MediaWiki-extensions-WikimediaMaintenance: Make maintance script for sending annual survey emails - https://phabricator.wikimedia.org/T148783#3034796 (10scfc) p:05Triage>03Low [22:38:26] 06Labs, 10Tool-Labs: Java Runtime on Toollabs - not up to date? - https://phabricator.wikimedia.org/T147833#3034798 (10scfc) p:05Triage>03Low [22:39:52] 06Labs, 10Tool-Labs, 10Gerrit, 10Pywikibot-core: Fresh clone of pywikibot from gerrit fails with error: RPC failed; result=56, HTTP code = 200 on Toollabs (NFS) - https://phabricator.wikimedia.org/T151351#3034803 (10scfc) p:05Triage>03Low [22:40:21] 06Labs, 10Tool-Labs: Grid jobs often stuck after Tool Labs maintenance - https://phabricator.wikimedia.org/T151603#3034805 (10scfc) p:05Triage>03Normal [22:41:12] 06Labs, 10Tool-Labs: Install librsvg 2.40.16 on Jessie for K8s webservice - https://phabricator.wikimedia.org/T151656#3034813 (10scfc) p:05Triage>03Normal [22:41:49] 06Labs, 10Tool-Labs, 10Tools-Kubernetes, 07Tracking: Packages to be installed in Tool Labs Kubernetes Images (Tracking) - https://phabricator.wikimedia.org/T140110#3034817 (10scfc) [22:41:51] 06Labs, 10Tool-Labs: Install librsvg 2.40.16 on Jessie for K8s webservice - https://phabricator.wikimedia.org/T151656#2823930 (10scfc) [22:43:02] 06Labs, 10Tool-Labs: define a prebaked way to temporarily disable access to a tool - https://phabricator.wikimedia.org/T147242#3034819 (10scfc) p:05Triage>03Low [22:43:31] 06Labs, 10Tool-Labs: Setup Failoverable Puppetmasters for tools - https://phabricator.wikimedia.org/T145883#3034821 (10scfc) p:05Triage>03Low [22:43:42] 06Labs, 10Tool-Labs: Expose tool-labs service names via environment variables - https://phabricator.wikimedia.org/T151002#2804324 (10bd808) "Real" service discovery is something that we should get from a PaaS layer or other orchestration system for containers. In reality all that will do however is manage some... [22:43:53] 06Labs, 10Tool-Labs: Make gridengine exec hosts also submit hosts - https://phabricator.wikimedia.org/T123270#3034825 (10scfc) p:05Triage>03Low [22:44:32] 06Labs, 10Tool-Labs: Create sqldump script - https://phabricator.wikimedia.org/T151680#3034829 (10scfc) p:05Triage>03Low [22:46:08] 06Labs, 10Tool-Labs, 06WMF-Legal: Install unrar on Tool Labs - https://phabricator.wikimedia.org/T151794#3034843 (10scfc) p:05Triage>03Low [22:47:00] 06Labs, 10Quarry, 10Tool-Labs: Clarify Tool Labs' rules to see if Quarry and PAWS are allowed to be hosted there - https://phabricator.wikimedia.org/T152212#3034852 (10scfc) p:05Triage>03Low [22:51:22] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Reassign service/pod IP ranges for kubernetes on tool labs - https://phabricator.wikimedia.org/T152399#3034859 (10scfc) p:05Triage>03Normal [22:52:17] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Reassign service/pod IP ranges for kubernetes on tool labs - https://phabricator.wikimedia.org/T152399#3034868 (10yuvipanda) p:05Normal>03High Since we can't move too many services to k8s until this gets fixed. [22:59:29] 06Labs, 10Labs-Infrastructure, 10Tool-Labs: Update Mono to newer stable release - https://phabricator.wikimedia.org/T152949#3034880 (10scfc) 05Open>03declined Declining this task given that Trusty and Jessie only offer 3.2.8 and AFAICT no #Tool-Labs administrator has knowledge or free workload to maintai... [23:02:53] 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs, 07Software-Licensing: Expand the Tool Labs definition of "free license" to include FSF-approved and DFSG-compatible licenses - https://phabricator.wikimedia.org/T152581#3034890 (10scfc) p:05Triage>03Normal [23:09:59] 06Labs, 10Tool-Labs, 10Tools-Kubernetes, 07Tracking: Make webservice backend default to kubernetes - https://phabricator.wikimedia.org/T154504#3034899 (10scfc) p:05Triage>03Normal -1. There are many differences between SGE webservices and Kubernetes ones, and thus changing the default offers quite a s... [23:10:43] 06Labs, 10Tool-Labs, 10Tools-Kubernetes: Make webservice backend for lighttpd default to kubernetes - https://phabricator.wikimedia.org/T154506#3034904 (10scfc) p:05Triage>03Normal -1 per parent task. [23:11:43] 06Labs, 10Tool-Labs: Bigbrother doesn't restart wikilinkbot - https://phabricator.wikimedia.org/T154527#3034907 (10scfc) p:05Triage>03Low [23:12:09] 06Labs, 10Tool-Labs: Several users and tools have invalid credentials in replica.my.cnf - https://phabricator.wikimedia.org/T154933#3034909 (10scfc) p:05Triage>03Low [23:13:11] 06Labs, 10Tool-Labs: Create Updated NodeJS container for Tool Labs - https://phabricator.wikimedia.org/T155063#3034923 (10scfc) p:05Triage>03Low [23:14:45] 06Labs, 10Tool-Labs: giftbot webservice outages and/or issues - https://phabricator.wikimedia.org/T155494#3034928 (10scfc) p:05Triage>03Low [23:21:38] 06Labs, 10Tool-Labs: Explain/Investigate low number of giftbot queue jobs - https://phabricator.wikimedia.org/T155789#3034977 (10scfc) p:05Triage>03Low I don't remember any changes in the configuration, but looking just now at `qstat -j 1227342`: ``` […] scheduling info: queue instance "webgrid... [23:22:44] 06Labs, 10Tool-Labs, 13Patch-For-Review: Rewrite /usr/local/bin/crontab in python; fix bugs - https://phabricator.wikimedia.org/T156174#3034980 (10scfc) p:05Triage>03Normal [23:23:10] 06Labs, 10Tool-Labs, 10MassAction, 15User-Addshore: Delete tool 'massaction' - https://phabricator.wikimedia.org/T157079#3034982 (10scfc) p:05Triage>03Low [23:24:41] 06Labs, 10Tool-Labs, 10Tools-Kubernetes, 05Prometheus-metrics-monitoring: Labs Promethius not recording k8s stats since 2017-01-24T06:00 - https://phabricator.wikimedia.org/T157355#3034985 (10scfc) p:05Triage>03Normal [23:28:57] 06Labs, 10Tool-Labs: Upgrade bootstrap-vz version for tools docker builder - https://phabricator.wikimedia.org/T157526#3035009 (10scfc) p:05Triage>03Normal [23:29:20] 06Labs, 10Tool-Labs: #tool-labs workboard is crazy - https://phabricator.wikimedia.org/T157698#3035011 (10scfc) p:05Triage>03Normal [23:30:21] 06Labs, 10Tool-Labs: #tool-labs workboard is crazy - https://phabricator.wikimedia.org/T157698#3013537 (10yuvipanda) Indeed, and I don't use it at all :) [23:49:35] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Hall1467 was modified, changed by Hall1467 link https://wikitech.wikimedia.org/w/index.php?diff=1533898 edit summary: