[00:18:13] RECOVERY - Puppet run on tools-worker-1012 is OK: OK: Less than 1.00% above the threshold [0.0] [01:38:55] !checkuser [01:39:01] whoops, wrong channel [01:40:33] RECOVERY - Puppet run on tools-proxy-02 is OK: OK: Less than 1.00% above the threshold [0.0] [02:28:58] Krenair: you know of the nova network situation? it's kinda sortof deprecated, you are supposed to use Neutron, but there's no direct migration path... [02:29:13] labtest will hopefully help us move to neutron [02:29:27] So I got all the recovery alerts, and *then* the problem alerts. Interesting [02:29:36] Krenair: on SMS? [02:29:57] I don't get production paging [02:30:09] shinken emails me about deployment-prep [02:30:21] YuviPanda, I don't know the details re. nova network situation [02:31:02] ah, right [02:31:07] Krenair: Thanks again for the help: https://en.wikipedia.org/w/index.php?title=User_talk:Dung247&diff=prev&oldid=720309159 [02:31:36] you're welcome TParis [02:32:36] 06Labs: Investigate labnet1002 kernel panic - https://phabricator.wikimedia.org/T135322#2295290 (10yuvipanda) [02:34:15] 06Labs: Investigate labnet1002 kernel panic - https://phabricator.wikimedia.org/T135322#2295303 (10yuvipanda) I copied kern.log to /home/yuvipanda/kern.log [02:47:27] RECOVERY - Puppet run on tools-webgrid-lighttpd-1403 is OK: OK: Less than 1.00% above the threshold [0.0] [02:52:33] RECOVERY - Puppet run on tools-exec-1208 is OK: OK: Less than 1.00% above the threshold [0.0] [02:52:33] RECOVERY - Puppet run on tools-k8s-master-01 is OK: OK: Less than 1.00% above the threshold [0.0] [02:52:47] RECOVERY - Puppet run on tools-exec-1214 is OK: OK: Less than 1.00% above the threshold [0.0] [02:52:47] RECOVERY - Puppet run on tools-flannel-etcd-02 is OK: OK: Less than 1.00% above the threshold [0.0] [02:53:17] RECOVERY - Puppet run on tools-webgrid-lighttpd-1210 is OK: OK: Less than 1.00% above the threshold [0.0] [02:57:18] RECOVERY - Puppet run on tools-exec-1220 is OK: OK: Less than 1.00% above the threshold [0.0] [02:57:32] RECOVERY - Puppet run on tools-exec-cyberbot is OK: OK: Less than 1.00% above the threshold [0.0] [02:58:02] RECOVERY - Puppet run on tools-bastion-03 is OK: OK: Less than 1.00% above the threshold [0.0] [03:02:24] RECOVERY - Puppet run on tools-webgrid-generic-1405 is OK: OK: Less than 1.00% above the threshold [0.0] [04:22:30] 10Tool-Labs-tools-Other: Use OAuth for http://tools.wmflabs.org/cluebot/ - https://phabricator.wikimedia.org/T135323#2295320 (10bd808) [04:23:25] 10Tool-Labs-tools-Other: Use OAuth for http://tools.wmflabs.org/cluebot/ - https://phabricator.wikimedia.org/T135323#2295333 (10bd808) [04:23:27] 06Labs, 06WMF-Legal, 07Epic, 07Privacy: [EPIC] Protect end-user privacy by restricting non-consentual third-party browser interactions - https://phabricator.wikimedia.org/T133919#2295332 (10bd808) [06:58:06] PROBLEM - ToolLabs Home Page on toollabs is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Not Available - string 'Magnus' not found on 'http://tools.wmflabs.org:80/' - 530 bytes in 0.017 second response time [07:23:04] RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 3669 bytes in 0.093 second response time [09:01:45] 10Tool-Labs-tools-Other: Use OAuth for http://tools.wmflabs.org/cluebot/ - https://phabricator.wikimedia.org/T135323#2295320 (10tom29739) The required 'account creation' notice isn't there, and it's using reCaptcha so loading content from other sites without use consent, so all of this is a Tool Labs Terms of Us... [09:13:40] RECOVERY - Puppet run on tools-exec-1212 is OK: OK: Less than 1.00% above the threshold [0.0] [10:20:57] RECOVERY - Puppet run on tools-web-static-01 is OK: OK: Less than 1.00% above the threshold [0.0] [10:22:57] RECOVERY - Puppet run on tools-exec-1201 is OK: OK: Less than 1.00% above the threshold [0.0] [14:24:38] RECOVERY - Puppet run on tools-flannel-etcd-01 is OK: OK: Less than 1.00% above the threshold [0.0] [14:29:15] RECOVERY - Puppet run on tools-services-02 is OK: OK: Less than 1.00% above the threshold [0.0] [14:36:13] PROBLEM - SSH on tools-webgrid-lighttpd-1408 is CRITICAL: Server answer [15:58:55] grrrit is offline :( [16:00:59] oh [16:01:03] I'll take a look [16:02:40] Glaisher: should be back up now, I was experimenting with it earlier and forgot to put it back in afixed up state [16:02:47] grrrit-wm: welcome back [16:02:58] yay, thanks [16:14:04] hi [16:14:09] hello [16:17:10] anyone here? [16:17:11] Hi cat_, I am here, if you need anything, please ask, otherwise no one is going to help you... Thank you [16:17:32] i want to know how to create a page [16:17:54] cat_: what kind of page? [16:18:24] a page on wikipedia [16:18:26] normal page [16:18:42] are there different type of pages? [16:18:59] cat_: https://en.wikipedia.org/wiki/Wikipedia:Your_first_article would be a good place to start, then [16:19:37] cat_: for queries related to wikipedia (I'm assuming you're referring to the English wikipedia), you'll also get more response in the #wikipedia-en channel on IRC. [17:52:54] How do I increase the memory limit for webservice generic? [18:00:47] tom29739: you can request it on phabricator. You get 4GB by default, which should really be enough for anything. [18:01:38] I'm getting 'There is insufficient memory for the Java Runtime Environment to continue'. [18:02:29] webservice nodes are overcommitted, so it might be that we need to add more nodes [18:02:42] please file a task in phabricator; I don't have the time to look at it in detail at the moment [18:03:15] OK. [18:11:54] 06Labs, 10Tool-Labs: Increase webservice memory limit for piagetbot - https://phabricator.wikimedia.org/T135346#2295936 (10tom29739) [19:42:46] 06Labs: "webservice" command (with jsub) doesn't work in crontab - https://phabricator.wikimedia.org/T135348#2295987 (10Nemo_bis) [19:50:13] 06Labs, 10Tool-Labs: "webservice" command (with jsub) doesn't work in crontab - https://phabricator.wikimedia.org/T135348#2296003 (10valhallasw) [19:51:35] 06Labs, 10Labs-Infrastructure: Misleading "release is deprecated and has no effect, using trusty" - https://phabricator.wikimedia.org/T135349#2296004 (10Nemo_bis) [19:51:47] 06Labs, 10Labs-Infrastructure: Misleading "release is deprecated and has no effect, using trusty" - https://phabricator.wikimedia.org/T135349#2296019 (10Nemo_bis) [19:53:00] 06Labs, 10Tool-Labs: Misleading "release is deprecated and has no effect, using trusty" - https://phabricator.wikimedia.org/T135349#2296020 (10valhallasw) [19:53:18] Nemo_bis: for tool labs bugs, please just tag them as tool-labs (herald will add #labs; #labs-infra is for openstack related stuff) [20:17:44] valhallasw`cloud: how do I know if something is openstack-related? [20:18:10] Anyway, please update https://phabricator.wikimedia.org/project/view/474/ [20:18:43] (I'm sure I will forget otherwise) [20:22:22] Nemo_bis: if you have to ask, it probably isn't ;-) [20:22:31] And that description is correct [20:22:58] Webservice = only tool labs = single labs project [20:25:44] The description says "specific", Tool Labs doesn't seem very specific to me [20:41:14] 10Tool-Labs-tools-Other: OAuth time out is too short - https://phabricator.wikimedia.org/T135279#2296047 (10Aklapper) [20:46:27] hi valhallasw`cloud the api processes at dewiki are very laggy. replication lags since 17:50 UTC. Whats going on? [20:47:05] doctaxon: I wouldn't know. [20:47:24] who'd know? [21:26:33] https://phabricator.wikimedia.org/T131266 is important and should be dealt with at some point. [21:26:37] Thank you in advance [22:39:09] (03CR) 10Rush: "I think it would be better to do in hiera, we try to keep if $realm checks to a minimum in general and I'm not a big fan of them in templa" [labs/private] - 10https://gerrit.wikimedia.org/r/288736 (owner: 10Alex Monk) [23:00:29] 06Labs, 10Tool-Labs: Update python-wikitools on Tool Labs - https://phabricator.wikimedia.org/T102862#2296124 (10MZMcBride) >>! In T102862#1375842, @valhallasw wrote: > - There is no v1.3 available in git; https://github.com/alexz-enwp/wikitools/commit/72a00c96e97c6d1017f607b9ebc8e1a520bd0d3e jumps from 1.2 to... [23:05:03] (03PS2) 10Alex Monk: Add my (Alex M's) key to labtest-instances root [labs/private] - 10https://gerrit.wikimedia.org/r/288736 [23:41:56] (03CR) 10Andrew Bogott: [C: 032] Add my (Alex M's) key to labtest-instances root [labs/private] - 10https://gerrit.wikimedia.org/r/288736 (owner: 10Alex Monk) [23:42:54] (03CR) 10Andrew Bogott: [V: 032] Add my (Alex M's) key to labtest-instances root [labs/private] - 10https://gerrit.wikimedia.org/r/288736 (owner: 10Alex Monk)