[00:00:09] I guess only via bastion.wmflabs.org, but I guess at first we should use tools-dev [00:00:12] It’s working… slowly... [00:00:30] My become just succeeded. ~10 mins later [00:00:51] wow :D [00:01:02] And an ls is doing the same thing... [00:01:11] Just finished. 2 minutes. [00:01:18] Matthew_: My login still needs time -.- [00:01:26] Wait no. [00:01:51] enterprisey: BTW, I never asked. That tweak I made to the template - Will it work? [00:03:00] Just finished. 3 minutes. [00:03:14] ah, my login too [00:03:23] tom29739 Luke081515 Is it working faster for you? [00:03:24] seems like it is fast again [00:03:27] Mine just picked up really quick. [00:03:31] become ~1 second [00:03:32] I can access tools-bastion-05. [00:03:33] yep [00:03:45] qstat in ~0.1 seconds [00:03:48] what changed? [00:04:11] Someone doing CPU intensive on the bastion again I should imagine. [00:04:17] -.- [00:04:20] * Matthew_ sighs [00:04:27] It wasn’t me this time! [00:04:28] Matthew_: which template are you talking about? [00:04:32] I can take a look what grafana says [00:04:36] enterprisey: The article request one. [00:04:46] * enterprisey goes and looks [00:04:59] It's very unreliable, that bastion. [00:05:11] I guess tools is two big for grafana, my browser is just hanging [00:05:47] checking grafana now [00:06:20] I seen to remember reading that tools-bastion-01 was only replaced by 05a few months ago. [00:06:47] Certainly while I've been here, and that's only a couple of months. [00:06:56] seems like there was no big CPU usage at bastion.05 from nagf [00:07:05] but ram is nearly full [00:07:14] why don't create a xlarge bastion instance? [00:07:39] The RAM is always full. [00:07:46] Why not. [00:09:14] 6Labs, 10Tool-Labs: Tools-bastion-05 is hanging, login is broken - https://phabricator.wikimedia.org/T131122#2156675 (10Luke081515) Maybe we should replace this large instance through an xlarge instance? [00:10:51] 6Labs, 10Tool-Labs: Tools-bastion-05 is hanging, login is broken - https://phabricator.wikimedia.org/T131122#2156676 (10tom29739) And put in those cgroups and ulimits that have been talked about on it. [00:18:19] MusikAnimal: I believe I’ve found the problem. [00:18:31] awesome [00:18:42] xtools-ec is in the xtools group, however the inverse is not true. [00:19:03] So the xtools-ec user cannot write to /data/projects/xtools/tmp/session by permissions. [00:20:43] hmm the files show up though, or at least they were [00:21:00] e.g. you'll see files in that folder that have the group xtools, and others that have xtools-ec [00:21:09] and xtools-articleinfo [00:21:25] However, I’m wondering if the permissions changed. Or how they’re being handled. [00:21:42] And I see no sessions that have anything other than tools.tools. [00:21:44] *xtools [00:21:46] Ugh. [00:22:23] I deleted all the files earlier [00:22:31] which is supposed to happen when you run the restart script [00:22:39] that was after it went down, though [00:23:02] Hm. [00:23:08] Then I got nothing, my apologies :/ [00:37:17] !log rcm.cac updating all repos to master [00:37:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm.cac/SAL, Master [00:37:34] Fwiw hanging like that is almost def NFS being abused [00:37:52] !log tools.articlerequest Updated dev to version 0.1devel8 [00:37:55] and bastion 10 kis for testing throttling of these issurs [00:38:00] issues even [00:38:47] chasemp: I guess your "puppet is disabled" message is not full of logic :D [00:39:33] did you reread it? ;) [00:40:17] It confused me for a bit. 'Please feel free to disable' -- but it is disabled :D [00:40:49] Disable the disable :) [00:41:03] But that has nothing to do with this itself [00:41:17] puppet staleness wont cause a perf issue [00:42:39] I know next to nothing about puppet :) [00:42:43] !log tools.xtools Temporarily changed permissions of session directory to test session errors. [00:42:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.xtools/SAL, Master [00:43:43] chasemp: Is there a reason to don't replace bastion by a xlarge instance? [00:43:46] In general? [00:44:27] Nope its on my todo [00:44:43] Cyberpower678: Ok, I know you are burned on tools but I need help with this. OAuth, how do I troubleshoot those errors? [00:45:07] chasemp: Do you think bastion-05 needs investigation at the moment (the task), or can I close it? Otherwise I will only reset the prio [00:45:31] Why was puppet disabled? [00:46:38] tom29739: I think the message said it? [00:46:53] It need attention but no mpre specific than the other tasks made for bastion overload [00:47:03] so don't close it? [00:47:21] Luke081515, globalblocking patch updated. [00:47:28] ok, I will try it [00:48:36] matt_flaschen: At the moment I only can control, if the db is created, because at the moment I can't create a proxy, so if you need a live test, this maybe takes one or two days from labs [00:48:51] Matthew_, to troubleshoot you have two options, try and figure out what's causing the generic error message to show, or rewrite the OAuth module. [00:49:08] Leaving it open is fine thanks [00:49:09] I think the xTools OAuth is pretty sloppy./ [00:49:12] Erm… I’ll troubleshoot. Got to learn oath first. [00:49:13] It is. [00:49:17] 6Labs, 10Tool-Labs: Tools-bastion-05 is hanging, login is broken - https://phabricator.wikimedia.org/T131122#2156788 (10Luke081515) p:5Unbreak!>3Triage At the moment it's normal, but maybe it needs investigation? [00:49:19] ok [00:49:41] Matthew_, I wish you lot's of luck for the next few hours then. :p [00:49:50] Thank you. [00:50:15] Luke081515, I tested it locally. I suggest we don't make testing on Labs a blocker. Hopefully the new patch set works, but if not you can file a bug and ping me to follow up. [00:50:32] ok :) [00:51:08] As a matter of fact, the xTools code is pretty shitty to begin with. ACC's old code was better. [00:51:37] matt_flaschen: This time without provision errors, looks good, I will check the DB now [00:53:43] Waaaaaahooo! [00:54:03] matt_flaschen: your patch has a +1 now :) [00:54:09] Cyberpower678: Permissions issue. [00:54:51] Thanks, Luke081515 [00:55:08] !log tools.xtools Added read to oath.cnf [00:55:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.xtools/SAL, Master [00:55:30] Matthew_, you lucky bestard [00:55:39] Yeah. [01:09:13] !log rcm deleted instance gbtest, test is done [01:09:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL, Master [01:50:41] !log tools.xtools xtools-ec is still down, Database permissions issue related to the inability for xtools-ec to read the replica.my.cnf file for xtools. [01:50:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.xtools/SAL, Master [02:02:26] 6Labs, 10Tool-Labs: tools-bastion-01 is in SHUTOFF state - https://phabricator.wikimedia.org/T131135#2156906 (10scfc) [02:56:42] (03PS1) 10Mattflaschen: Add two Collaboration team boards [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/280151 [04:27:04] kaldari, Niharika: bot's DB has amassed to almost 6 million links now. Going to be now though [04:46:31] !log tools.xtools Attempted to fix permission issue for replica.my.cnf, succeeded. However, database selection is now broken. Notice placed on edit counter page. [04:46:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.xtools/SAL, Master [06:57:31] PROBLEM - Puppet run on tools-webgrid-lighttpd-1414 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [08:34:52] PROBLEM - Host tools-bastion-01 is DOWN: CRITICAL - Host Unreachable (10.68.17.228) [09:29:12] 6Labs, 10Horizon, 13Patch-For-Review, 7Tracking: Increase horizon session length - https://phabricator.wikimedia.org/T130621#2157250 (10hashar) Nice catch :-D [12:10:32] RECOVERY - Puppet run on tools-webgrid-lighttpd-1414 is OK: OK: Less than 1.00% above the threshold [0.0] [12:47:36] Should logrotate be run on the grid or not? [13:19:30] qstat [13:26:22] doctaxon: Wrong window? :D [13:40:52] !log deployment-prep Added ores-related classes and roles [13:40:52] Please !log in #wikimedia-releng for beta cluster SAL [13:40:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL, Master [13:41:07] :D [14:57:39] Hi, I'm currently working on a wikidata project. Thus I'm intrested in seeing the web proxy settings for my project. Is there a ui where I can have a look at the settings? [14:58:25] not right now [14:58:43] ok thank you [14:58:46] usually you could [14:58:51] I can get data out for you though [14:59:19] what's the project name? [15:00:12] it's math. I just wanted to confirm that there should be 3 proxies en-drmf2016, drmf2016 and wikidata-drmf2016 all pointing to drmf2016 port 8080 [15:02:57] yep, it has those three [15:02:58] Cyberpower678: What did you mean by: "bot's DB has amassed to almost 6 million links now. Going to be now though"? [15:03:07] Cyberpower678: Also, can you please join commtech? [15:03:22] Krenair: thank you [15:03:45] physikerwelt, no problem. you may have seen labs-l - this is in migration at the moment and the new interface is disabled until everything is ready [15:06:59] Krenair: Yes I saw that, but I did not know if that was already completed. [15:11:44] when it's done you'll be able to see the new ui in horizon [15:14:47] 6Labs, 10MediaWiki-extensions-OATHAuth, 10wikitech.wikimedia.org: wikitech 2fa provisioning form does so without confirmation - https://phabricator.wikimedia.org/T130892#2157762 (10jcrespo) > when jcrespo is back from Easter, and hopefully has a backup of that table I have in short-term storage that table o... [15:53:50] 6Labs, 10MediaWiki-extensions-OATHAuth, 10wikitech.wikimedia.org: wikitech 2fa provisioning form does so without confirmation - https://phabricator.wikimedia.org/T130892#2157838 (10csteipp) >>! In T130892#2157762, @jcrespo wrote: >> when jcrespo is back from Easter, and hopefully has a backup of that table >... [16:07:16] andrewbogott, so how does one get a shared filesystem setup between instances? [16:07:26] chasemp, ^ [16:07:48] Cyberpower678: unless you have *reallllllly* good reasons, the answer is 'you do not' [16:07:55] k [16:07:57] yes please don't rely on it [16:08:07] Just wondering [16:09:21] Yeah, we removed the self-serve interface for that because it was a disaster 98% of the time :) [16:09:32] 6Labs, 10Tool-Labs, 13Patch-For-Review: Instrument jsub/jstart/webservices usage - https://phabricator.wikimedia.org/T123444#2157848 (10yuvipanda) a:5yuvipanda>3None [16:09:50] and part of the disaster is it's almost always the wrong solution to the problem in question [16:11:03] 6Labs, 10MediaWiki-extensions-OATHAuth, 10wikitech.wikimedia.org: wikitech 2fa provisioning form does so without confirmation - https://phabricator.wikimedia.org/T130892#2157851 (10jcrespo) > The only thing I really need is the list of id's where is_validate was 0 at 2016-03-23 22:00 UTC. Do you have capacit... [16:15:32] chasemp, so this is my first time setting up a project. [16:16:26] And also my first time setting up a linux environment in general. [16:16:36] I want to install php for the command line. [16:18:28] Cyberpower678: you would install the php5-cli package I imagine [16:22:30] chasemp, okay, where do I find it. [16:22:48] Like I said, I've never setup a linux environment before. [16:23:02] 6Labs, 10MediaWiki-extensions-OATHAuth, 10wikitech.wikimedia.org: wikitech 2fa provisioning form does so without confirmation - https://phabricator.wikimedia.org/T130892#2157891 (10csteipp) The 19th is close enough! Thanks! [16:23:10] https://help.ubuntu.com/lts/serverguide/package-management.html [16:23:15] apt-get install php5-cli [16:32:04] chasemp, do I have to sudo that? [16:32:14] It only worked when I sudo'd it. [16:33:02] That would make sense, you'll have to figure out sudo as well. It's a testimate to how powerful the Tools environment that it is a tall order to move out from under it's umbrella but you'll get there. [16:33:49] chasemp, but the tradeoff to that, is the massive resources now under your control. :p [16:34:04] So I successfully installed PHP [16:34:08] I think. [16:34:32] But how do I control the sudo rules. [16:35:07] https://help.ubuntu.com/community/RootSudo [16:39:49] https://wikitech.wikimedia.org/wiki/Special:NovaSudoer [17:14:35] 10Tool-Labs-tools-Other, 6Community-Tech, 7Community-Wishlist-Survey, 7Milestone: Pageview Stats tool - https://phabricator.wikimedia.org/T120497#2158100 (10DannyH) [17:19:47] 6Labs, 10Tool-Labs, 10labs-sprint-119, 6Community-Tech-Tool-Labs, 10Diffusion: Figure out a git hosting solution for tools/kubernetes - https://phabricator.wikimedia.org/T117071#2158141 (10mmodell) I think almost all of the requirements for a really minimal deployment server can be satisfied by `scap::de... [17:21:18] 6Labs, 10Tool-Labs, 13Patch-For-Review, 3Scap3: Setup a proper deployment strategy for Kubernetes - https://phabricator.wikimedia.org/T129311#2158172 (10mmodell) I think almost all of the requirements for a really minimal deployment server can be satisfied by `scap::deploy_host` once this merges: https://g... [18:02:40] what was the cure for instances in SHUTOFF? I got test-prometheus2.monitoring.eqiad.wmflabs [18:03:51] godog: depending on reason you can start it via 'nova start UUID' on labcontrol [18:04:44] 6Labs, 7Tracking: Make OpenStack Horizon useful for production labs - https://phabricator.wikimedia.org/T87279#2158360 (10Andrew) [18:04:46] 6Labs, 10Horizon, 13Patch-For-Review, 7Tracking: Increase horizon session length - https://phabricator.wikimedia.org/T130621#2158358 (10Andrew) 5Open>3stalled Stalled pending upstream fix. If people complain about this I can make a local hack in the meantime. [18:04:52] 6Labs, 10Horizon, 13Patch-For-Review, 7Tracking: Increase horizon session length - https://phabricator.wikimedia.org/T130621#2158362 (10Andrew) a:5Andrew>3None [18:05:18] chasemp: I think it was fallout from the reboots, I've tried that now, thanks! [18:11:51] godog: we had a few of those man sorry [18:12:24] Krenair, what is the allow running as option. Can I sudo execute as a different user if I chose to? [18:18:44] chasemp: np, that did the trick, thanks! [18:29:01] chasemp, Sorry, user cyberpower678 is not allowed to execute '/usr/bin/apt-get install' as root on cyberbot-exec-01.cyberbot.eqiad.wmflabs. [18:29:30] chasemp, I'm trying to configure sudo policies, but I'm not sure what I'm doing wrong. [18:30:52] Cyberpower678, I don't remember, sorry [18:30:58] Cyberpower678: leave the original in place and don't touch the options [18:39:29] Cyberpower678: the standard policy gives people who are set as admin in wikitech sudo rights (all rights) [18:39:51] valhallasw`cloud: nope, it gives *everyone* part of the project sudo rights [18:40:01] valhallasw`cloud: that's the default, at least. [18:40:04] oh, even better [18:40:07] yuvipanda, unless, you tell it not to. [18:40:13] indeed. [18:40:53] yuvipanda, question, I have restricted sudo to me. Can any member of the project restore their sudo access on Wikitech? [18:41:04] all admins can [18:41:12] sudo rules on wikitech are editable by admins [18:41:13] projectadmins? [18:41:17] yes [18:41:19] k [18:41:37] but in general, unless you know exactly what you are doing: do not add people to your project unless you trust them enough :) [18:42:10] yuvipanda, what is cyberbot.iabot-chmod and can it be safely deleted? [18:42:12] in tools we have a bunch of other stuff in place to make it securer (PAM stuff) but it's easy to break and hardish to maintain. [18:42:18] Cyberpower678: I've no idea [18:42:34] Did that spawn from the service group? [18:42:44] maybe? [18:43:07] I'm guessing so. [18:43:16] ok! [18:43:18] I deleted those service groups per your suggestion. [18:43:32] So can I delete these sudo policies? [18:50:17] (03CR) 10Legoktm: [C: 032] Add two Collaboration team boards [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/280151 (owner: 10Mattflaschen) [18:51:13] (03Merged) 10jenkins-bot: Add two Collaboration team boards [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/280151 (owner: 10Mattflaschen) [18:53:07] !log tools.wikibugs Updated channels.yaml to: ad14e0d1a0a07b775d014e6f8f1edaf145349116 Add two Collaboration team boards [18:53:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL, Master [18:55:28] 6Labs, 13Patch-For-Review, 5WMF-deploy-2016-03-22_(1.27.0-wmf.18), 5WMF-deploy-2016-03-29_(1.27.0-wmf.19): Switch to using Horizon/Designate for labs public dns - https://phabricator.wikimedia.org/T124184#2158578 (10Andrew) [19:09:49] 10Tool-Labs-tools-wikiloves: Criar pagina de configuração da ferramenta no commons - https://phabricator.wikimedia.org/T130240#2158591 (10Danilo) 5Open>3Resolved a:3Danilo Desenvolvi um script para ler a configuração do módulo, fazer as consultas no banco de dados, processar os dados e salvar em um JSON pa... [19:15:34] cwd, have a moment? [19:16:00] Krenair: sure, in a meeting so semi distracted but i'm here [19:16:57] cwd, please try logging in to wikitech now [19:17:56] Krenair: still no dice [19:18:50] but i have clicked the reset pw link a lot of times, not sure if that nukes the pw before the email link is clicked [19:20:08] yeah I turned on logging, "Failed to bind as uid=cwdent,ou=people,dc=wikimedia,dc=org" [19:20:58] huh, the user should say cdentinger... [19:22:09] my user on phab is cwdent but i log in through mediawiki.org rather than ldap [19:22:36] it should? I thought your cn was Cdentinger, not uid [19:22:45] krenair@bastion-01:~$ ldaplist -l passwd cwdent | grep cn [19:22:45] cn: Cdentinger [19:23:11] yes, your phabricator account is not currently linked to ldap [19:23:18] oh, ok i see [19:23:23] i am not very familiar with ldap [19:23:59] my shell login is cwdent [19:24:12] yes [19:25:34] but gerrit and the wiki use common name for login? [19:26:07] yes [19:26:35] I believe the wiki then looks up your uid from it and tries to bind using your password [19:27:11] so is that "failure to bind" just a pw error? [19:27:16] not clear [19:27:31] wait, you said you clicked the reset password link? [19:27:53] I thought you didn't get the email with the link in it? [19:28:36] Krenair: no i didn't, i was just wondering if the pw might reset when i clicked the "forgot password" link [19:28:47] i never got the email [19:29:30] "A failure to bind means the user doesn't exist." <- from a random forum [19:30:34] dn: uid=cwdent,ou=people,dc=wikimedia,dc=org [19:30:34] objectClass: person [19:30:34] objectClass: inetorgperson [19:30:35] etc. etc. [19:32:42] it definitely exists [19:34:03] Krenair: and it's got my correct email? or is it derived from cn@dc [19:36:26] this is not just a system for internal staff/contractors [19:36:35] it deals with all emails, it can't possibly assume cn@dc [19:36:41] Ooooh! [19:36:43] * Krenair facedesks [19:36:52] * Krenair facedesks hard [19:36:58] krenair@bastion-01:~$ ldapsearch -x uid=cwdent | grep mail [19:36:59] mail: cdentinger@wikimedia.og [19:37:01] Spot the missing character [19:37:14] ha! [19:37:43] I'll figure out how to change that for you, one moment [19:37:48] thank you! [19:38:17] it took us far too long to find that :p [19:39:12] typos remain the worst thing [19:42:03] yuvipanda, I'm having trouble setting up mysql on my instances. [19:42:05] cyberpower678@cyberbot-exec-01:~$ mysql tools-db [19:42:05] ERROR 1045 (28000): Access denied for user 's51059'@'localhost' (using password: YES) [19:42:16] cwd, try it now [19:42:20] (password reset email) [19:42:43] yuvipanda, for some reason it's pointing to localhost [19:42:48] Cyberpower678: read the error. [19:42:51] yes. [19:43:02] you're not providing a host [19:43:26] God. Toollabs has spoiled me. [19:43:28] Krenair: just did it, will let you know if i don't see it in like 10m [19:44:04] valhallasw`cloud, cyberpower678@cyberbot-exec-01:~$ mysql --host tools-db [19:44:05] ERROR 2005 (HY000): Unknown MySQL server host 'tools-db' (0) [19:44:53] cwd, I missed a place to update it, try again [19:45:12] * cwd winces [19:45:15] valhallasw`cloud, why won't it resolve to the tools-db host? [19:45:21] Krenair: it will only send once every 24hrs [19:45:26] because the server isn't called tools-db [19:45:26] ugh. [19:45:48] Huh? [19:46:21] yuvipanda, told me that these all resolve to the proper server locations across projects. [19:46:37] no, he didn't. The servers are called xxwiki.labsdb [19:46:46] the toolsdb server is called tools.labsdb [19:47:02] But I've always been using tools-db on toollabs [19:48:50] cwd, try again [19:49:16] valhallasw`cloud, that works, so what's with tools-db. Why can I use that on toollabs? [19:49:24] because /etc/hosts [19:49:35] well, because backwards compatibiliy [19:49:45] which is why it's in /etc/hosts [19:49:47] Krenair: got it! thank you so much [19:49:53] I see. [19:50:18] didn't realize what a spof gerrit is for my job until i lost access [19:50:25] but I see the docs page still refers to the old name -- let me fix that [19:52:25] cwd, you're welcome, does your new password work as expected? [19:56:57] Krenair: yep! [19:57:00] valhallasw`cloud, where is that located. /etc/hosts doesn't seem to exist. :/ [19:57:47] valhallasw@tools-bastion-05:~$ wc -l /etc/hosts [19:57:47] 15 /etc/hosts [20:02:28] Will logrotate work on the grid? [20:03:10] valhallasw`cloud, using SFTP, I can't seem to find it when I login via tools-login.wmflabs.org as cyberpower678 [20:03:22] So I'm rather confusedl. [20:03:40] truncating works, moving the file doesnt work without restarting the job (assuming it's set as stdout file for SGE) [20:04:43] not that logrotate will not make NFS cry less [20:05:00] so if you really need to log on a verbose level, please log to /tmp and logrotate there [20:05:11] ? [20:05:20] (/tmp is per-exec host and doesn't hit NFS) [20:05:29] Cyberpower678: I don't know why your sftp client doesn't understand /etc. [20:05:51] The file exists on tools-bastion-05, and should exist on every other tools host as well. [20:06:26] Oh wait. The connection somehow stalled. [20:06:31] I see it now./ [20:08:04] 10Tool-Labs-tools-wikiloves: Desenvolver gráficos e tabelas para exibir os dados - https://phabricator.wikimedia.org/T131192#2158781 (10Danilo) [20:12:00] Crontab is modifying my tool's cron to add jsub when the entry in the crontab already uses jsub. [20:12:54] hello someone can add me to ssh bastion ? [20:13:34] damiano64: if you're in a project, you should have access to the bastion [20:13:45] if you're not, you have no reason to connect there [20:14:20] damiano64: if you want access to tool labs, please follow the procedure there: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Quick_start [20:14:45] no i am not now... but i read before to open a new project or join to exeistent i need a shell on Bastion .. its'nt [20:14:49] tom29739: uh, that's odd. What's the command? [20:15:34] damiano64: ah, you might have stumbled on old documentation. [20:15:59] (until maybe a year ago you needed a seperate 'bastion' right) [20:16:03] I am helping organization of Wikimania 2016 in Esino LArio [20:16:06] valhallasw`cloud, "@daily /usr/bin/jsub -N cron-tools.hall-of-fame-1 -once -quiet 'logrotate /data/project/hall-of-fame/logrotate.conf'" That's my crontab entry. [20:16:19] tom29739: the @daily throws it off. [20:16:29] tom29739: also @daily is a bad idea given that it fires at midnight [20:17:08] (things that do not need to happen at midnight specifically should schedule for some other moment, e.g. 00:41) [20:17:32] What should I use? It just needs to run daily. [20:18:30] 'not midnight' is the most important, then 'not on the whole hour' is a good second heuristic [20:19:29] So something completely random like say 2:11am, would be fine? [20:19:32] yep [20:21:46] In order to access a labs instance Sentence: you must be a member of the bastion project. Verify whether you are a member at 'bastion' project page by clicking "Members [Expand]" [20:21:51] It seems to have worked, it hasn't changed it anyway. [20:23:47] please fill a Shell access request form [20:24:36] damiano64: it's more helpful if you paste a link, but I found the text and removed it. [20:24:53] it's outdated and hasn't been true for a while [20:33:33] ok valhallasw.. but i follow your link https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Quick_start [20:33:57] perhaps i wrong something [20:36:11] damiano64: > To support organization of Wikimania 2016 in Esino Lario -- why do you need access to tool labs for that? [20:36:17] I need a vm to support Wikimania event 2016 .. i need a space where to save our local DB and documentation... it's possible to have it [20:37:20] i am a newbie of wikitech [20:38:37] damiano64: sorry, I mean one conceptual level higher. What do you want to use it for? A website? [20:39:12] No i don't need tool labs but wikimedia labs [20:40:29] No backup our local DB that support renting apartment to host attendee of event [20:41:24] ...why do you need labs to make a backup? I'm completely confused here. [20:42:01] it's true labs is a bad place to host backups [20:43:24] If I understand well, wikimedia labs is a openstack cloud. Openstack cloud have virtual machine. virtual machine can manage owner storage.. [20:44:07] Yes, but that doesn't mean that it's the right solution for your problem. [20:45:25] storage is without a doubt the most contentious resource in labs and VM persistent and robustness is not guaranteed anywhere in labs other than Tools and few exceptions [20:45:38] Now i use a VM Azure machine who cost 100 Dollar at month... may be that is worse ... have you other solution [20:46:10] damiano64: http://rsync.net/ [20:46:30] they will give you a discount if nonprofit as well [20:46:41] chasemp: might be problematic in terms of EU privacy laws, but there are probably EU-based solutions as well [20:46:54] (Azure is likely to have the same issue, though, and labs definitely has the same issue) [20:47:05] :) [20:47:09] so if Labs is OK, rsync.net should also be OK [20:47:18] so basically ignore what I juts said [20:48:02] for clarity we are figuring out our own backup solution within labs itself for labs itself [20:48:15] so someone else putting their backups on a VM which we don't backup [20:48:21] doesn't add up [20:49:08] I try to see rsync.net [20:49:20] Or you could use something like AWS. [20:52:34] Yes, I use AWS to support my customer (I work as ICT consultant). I hope to use Wikimedia resource to support wikimedia itself... no problem [20:53:15] 6Labs, 10Monitoring, 6Operations, 10wikitech.wikimedia.org: Bacula recovery of sql files from silver/wikitech fails - https://phabricator.wikimedia.org/T131195#2158892 (10jcrespo) [20:56:58] 6Labs, 10Monitoring, 6Operations, 10wikitech.wikimedia.org: Bacula recovery of sql files from silver/wikitech fails - https://phabricator.wikimedia.org/T131195#2158913 (10jcrespo) I did a last try trying to recover all possible files within a month, and I got some extra errors: ``` 29-Mar 20:39 helium.eq... [20:57:57] As i am newbie, i don't know wikitech platform and its policy... what i have imagined is to heve a VM with linux installed with a RDP access to its GUI and put and get forth and back DB, document, presentation as need to backup and may be show directly on remote desktop [21:00:15] The hypotesis to create a site (ie LAMP platform) to manage event data it's not possible due time to begin of event [21:00:32] Tool Labs can host websites. [21:06:35] damiano64: tool labs allows you to host a website (lighttpd, mysql, php) [21:07:28] damiano64: the RDP scenario might be technically possible with some sort of headless (VNC) X server but is not something that has been done before, and is therefore likely to exhibit issues we haven't seen before [21:08:24] hi, is it possible to create federated tables on tools-db and connect them with a replica server (e.g. enwiki.labsdb)? [21:09:11] damiano64: in Linux, it's not common to use a GUI for a remote server. We all work in terminals via SSH. [21:09:15] so far I'm getting error 1434: Foreign data src error [21:09:32] PeterBowman: uh. Probably not. [21:09:37] PeterBowman: why do you want to do that? [21:10:07] I'd like to join on 'page' table to retrieve page titles [21:10:45] PeterBowman: you can create databases on the replica hosts [21:11:15] yes, but it's not recommended according to wikitech help pages [21:11:35] it's not recommended /if you don't need to join with replica databases/ [21:11:48] if you need to join with replica databases, there is no alternative, but there are several caveats. [21:12:04] I believe that's as they are limited perf wise, prone to removal if abused, and not backed up but for ephemeral needs...it may needed [21:12:14] be even [21:13:02] right valhallasw`cloud, I must have misread that [21:13:20] are there any caveats I should be aware of compared with tools-db? [21:13:59] PeterBowman: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Database#Steps_to_create_a_user_database_on_the_replica_servers [21:14:08] the warning and two cautions, basically. [21:14:36] 75% of the performance on replica servers goes away on just replicating [21:14:50] use them, but use them wisely [21:15:16] oh right, then that's why I finally chose tools-db :( [21:15:51] I'm currently mapping page ids and page titles in a custom table, so far it's working fine [21:17:01] but that is already at the page table (page_id, page_namespace, page_title)? [21:18:03] yes, so the idea was to create a federated table on my db (tools-db) and connect with the page table in replica db [21:18:28] and avoid populating a custom page table on my side [21:42:14] autocracy: I know, but i recently implement that solution for a lazy customer (few workstation) and i discover it work well... [21:47:21] autocracy: h3.ddesystems.it (psw vncuser1)... it's a test environment [22:03:26] bd808: if I make an API request from Tool Labs, is that always going to come from the same IP address? And if so, is the address 208.80.155.131? [22:05:56] 6Labs, 10Monitoring, 6Operations, 10wikitech.wikimedia.org: Bacula recovery of sql files from silver/wikitech fails - https://phabricator.wikimedia.org/T131195#2158892 (10Dzahn) I also tried this multiple times and to different restore clients. Either i got an empty file with that same error about decrypti... [22:07:58] kaldari: I'm pretty sure that the answer to that is no — different exec nodes have different public IPs. [22:09:22] kaldari: or maybe I'm wrong… is that the behavior you're seeing currently? [22:09:37] 208.80.155.131 is a proxy, which I would've thought was only used for inbound traffic [22:09:42] perhaps yuvipanda can explain [22:09:45] andrewbogott: hmm, I'm trying to register to use Yandex's search API, but they require that the queries come from 1 IP. Do you know of any way to keep external requests to a single IP? [22:10:38] andrewbogott: Proxy outbound makes sense if you want to not allow arbitary requests etc [22:10:59] It makes sense, I'm just not clear on how that would've been accomplished :) [22:11:24] so kaldari, sorry, I guess the answer is that I don't know. You can hang around here for an answer from yuvi or send a question to labs-l [22:11:26] squid caching proxy? [22:11:46] Like we do for wgCopyUploadProxy in MW land [22:12:41] which is a squid reverse proxy IIRC [22:13:57] Reedy: you're describing how it could be done, not what you think happens to outbound tools traffic by default, right? [22:14:17] Vaguely, yeah [22:14:39] But if we reused infrastructure we've already made for this similar purpose elsewhere.. [22:15:02]