[01:01:42] halfak quick question [01:02:36] There's no reason why it's necessary to have my vagrantfile in the ores directory is there? [01:03:32] I could just put the vagrantfile, the provision.bash, and the uwsgi-dev.service in a new Dev folder in my home directory and vagrant up/ssh from there? [01:05:02] Sorry, if this is obvious. I'm a vagrant virgin and they say an oz. of prevention is worth a lb of cure.... [01:07:34] aawww he left [01:07:41] aetilley: hey [01:07:45] so the reason they are in the ores directory [01:07:59] is that all the files in that directory on your host (your computer) are made available in the VM under /vagrant [01:08:12] this allows you to edit files locally and just have them automatically run inside the virtual machine [01:09:09] Right but if I vagrant up from this new Dev directory that becomes /vagrant [01:09:15] I can still edit locally. [01:10:10] aetilley: yes but the systemd and other unit files assume that /vagrant/ refers to the ores directory itself [01:10:13] since I'm going to clone several things (including ores) into what will be /vagrant...Is there another reason to vagrant up from Ores [01:10:15] to locate source files and stuff [01:10:23] hm [01:10:39] so if you move them elsewhere it probably won't actually work [01:10:45] * YuviPanda is on a shitty connection on a bus to an airport [01:17:52] So just to be clear, I understant that this will make a new VM instance. But if I'm willing to do that... [01:18:18] It would be nice if I had some indicator of whether or not this was working. [01:18:55] Ok, I'll put this on the backburner. [01:20:56] aetilley: if you do 'vagrant up' on a fresh git clone, it should churn for a while and 'just work' [01:20:57] you shouldn't have to do any manual steps [01:21:16] hitting localhost:5000 on your browser should give you ores [01:26:57] Are you sure it's port 5000? [01:30:03] aetilley: uh, one of the ports... [01:30:22] aetilley: let me check [01:30:26] aetilley: 8080 [01:52:49] halfak: hey, if you are around [01:53:03] I finished diff-related features [01:55:04] bad words need some work [02:00:27] YuviPanda: I'm afraid that doesn't work for me on an old nor a fresh clone of ores. I tried every port I can think of. [02:00:43] vagrant ssh works just fine. [02:01:08] I'm going to get on a plane now. I'll debug when I'm off it! [02:01:14] Safe travels. [02:01:45] Ty [02:03:38] aetilley, o/ [02:04:02] greetings Aaron. [02:04:15] Want me to start from the top? [02:04:25] Nope. Read scrollback. [02:04:34] Got here right on time it seems. [02:05:10] port 8080 [02:05:14] (sorry was looking it up) [02:05:17] localhost:8080 [02:05:23] Doesn't work [02:05:30] but I think we might have changed yours since I remember you had a conflict. [02:05:37] 8081 doesn't work either [02:05:38] In the ores dir on the host, look at vagrantfile [02:05:44] that one. [02:05:47] line 11 [02:07:51] Yeah, doesn't work for what Yuvi wanted (in the browser). Now if I get a message about [02:08:07] SSH-2.0-OpenSSH_6.7PL Debian-5 [02:08:12] Protocol mismatch. [02:08:18] Hmm... That's weird. [02:08:22] So you can't even SSH to it? [02:08:28] sorry that's port 2222 [02:08:34] ssh is fine [02:08:38] kk [02:08:43] * halfak thinks [02:08:54] but my original question [02:09:05] I think might be independent [02:09:14] but I'll let you think. [02:09:19] Oh I guess I didn't scroll back that far. [02:09:26] Let's consider the original question. [02:09:29] k [02:09:34] Oh but before we do , copy-paste that line [02:09:36] line 11 [02:09:41] of the config [02:09:43] the whole line [02:09:48] Vagrantfile [02:10:30] My suspicion is [02:10:36] That the pip install failed [02:10:42] Leading to uwsgi not running [02:11:16] Are you on a plane right now? [02:11:20] No [02:11:27] Apparently I'm here too early [02:11:32] And everything is closed [02:11:37] tegel is like that [02:11:37] Including the check in counter [02:11:50] My flight is a 6 20 [02:11:55] So it is 4 10 here [02:12:03] "config.vm.network "forwared_port", guest: 8080, host: 8081" [02:12:08] Chairs also closed Lol [02:12:25] it's actually on my other machine. A fresh clone like yuvi suggested. [02:12:54] YuviPanda, bummer. Been in that situation before. [02:13:04] Actually, it was the trip out of lyon. [02:13:18] Seems like they should open two hours before a flight. [02:13:39] OK. Fresh clone is running a vagrant where? [02:14:50] huh? [02:15:10] What machine [02:15:19] When you vagrant up [02:15:20] My macbook [02:15:28] Debian? [02:15:45] I'm not sure what you're asking. [02:16:29] but when I point my browser to 127.0.0.1:8081 there's no data received. [02:17:35] You there? I think I might have lost intarwebz [02:17:39] Vagrant ssh [02:17:41] Then [02:17:50] Journalctl -u uwsgi* [02:17:54] (Small j) [02:17:59] And paste output? [02:18:40] No journal files were found. Failed to add filter for units: No data available. [02:19:02] sudo! [02:19:14] You have to sudo that journalctl command. [02:19:20] oog. ok one sec [02:19:31] (sorry, I'm in my living room with wife and dog, so I am a bit slow. [02:19:51] I'm in a strange airport and on my phone :D [02:19:53] It's on my other machine so let me go get the output. [02:20:47] But it appears that it can't find /srv/ores/venv/bin/python [02:21:01] Ah hmm [02:21:07] So, you should try to load localhost:8080 [02:21:12] *localhost:8081 [02:21:19] aetilley: can you run 'vagrant provision' on your host machine? [02:21:20] Oh! Yay! [02:21:24] And see what that iutputs? [02:21:26] Outputs [02:22:06] hi ai team [02:22:30] Vagrant provision ends with same complaint about can't find the ..../python file [02:22:43] aetilley: can you pastebin the full output? [02:22:51] o/ Emufarmers [02:22:55] :v [02:23:00] I was brought here [02:23:50] I actually thought YuviPanda was joking at first, because all the Wikimedia research I look at is usually AI-based anyway >.> [02:24:08] Let me get on irc on the machine where this is happening. One sec. [02:24:10] i'm here for legacy reasons [02:24:21] harej is the token non-AI researcher [02:24:21] Legacy!? [02:24:25] I'm the token guy-with-root [02:24:36] or maybe he's using AI now, for all I know! [02:24:48] I write mountains of python? [02:25:27] I wonder what is the term for a group of snakes [02:25:40] 'A tangle of snakes'? [02:25:57] A knot [02:26:16] http://www.thealmightyguru.com/Pointless/AnimalGroups.html <-- First google result [02:26:18] Must be fact [02:26:33] Do you have a knot of snakes? [02:27:31] you said a 'mountain of python' [02:27:46] was going to suggest you say 'a knot of pytohn' instead [02:28:34] Of course :) [02:28:42] aetilley, still switching machines? [02:28:51] halfak: ok back sorry, you want to me to pastebin the result of provision? [02:28:57] Yes plz [02:29:31] halfak: do we still need the indonesian aspell? [02:29:51] yes [02:30:03] We aren't running a model now, but I'd like to get one up. [02:30:06] Is it a problem? [02:30:13] We could consider dropping it. [02:30:29] Now that I think of it, it's not an install requirement. [02:30:38] We'll only need it if we try to deploy a model that requires it. [02:30:50] halfak: it is a... slightly odd package since it isn't in debian but is in ubuntu and we have to munge around to get it installed [02:31:21] also, https://en.wikipedia.org/wiki/Quarry_(disambiguation) has a link to quarry [02:31:30] :D [02:31:56] YuviPanda: someone wrote a post with an interwiki-style link to quarry [02:32:00] So... I think we can skip it for now if it is a pain and look for a better option. [02:32:34] We might even be able to include it as a dat if we need to. [02:32:52] right [02:32:54] cool [02:33:52] http://pastebin.com/ibMDb46q [02:34:04] I can see if I can do it with line breaks.... [02:34:41] fascinating [02:35:50] WTF [02:36:02] So, python3-virtualenv is installed. [02:36:02] ? [02:36:08] And then commend not found. [02:36:29] harej: yes because the packages aren't actually installed [02:36:30] or is it trying to use virtualenv in the aspell package? [02:36:50] YuviPanda: you might be confusing me and aaron again [02:37:00] err [02:37:02] halfak: [02:37:04] there, no more confusion [02:37:14] Samoyed, :P [02:37:20] * Samoyed curls up in the corner [02:37:30] There can be only one ha[TAB] [02:38:14] So may I ask my original question again? [02:38:46] Yeah [02:38:51] (I'm not sure how far up you scrolled...) [02:39:15] See my original disclaimer about my Vagrant virginity... [02:39:26] That sounds weird [02:39:29] lol [02:39:46] Since I'm going to clone ores (and other repos) inside of this VM.... [02:40:09] Is there any reason why I need to vagrant up from inside a copy of the ores directory? [02:41:01] Oh... hmmm... I guess that if you moved the vagrant file elsewhere, it would still work, but it wouldn't find OREs in the /vagrant dir on the guest. [02:41:18] But as I said, [02:41:22] or you said [02:41:23] YuviPanda, hacking while hanging out outside the airport [02:41:24] halfak: ^ [02:41:27] halfak: I suspect that might be a problem [02:41:35] the directory in which you call vagrant up Becomes /vagrant [02:41:44] YuviPanda, do you need me to test? [02:41:46] so you can customize that however you want. [02:42:00] halfak: you are aetilley :D [02:42:43] oh nevermind [02:42:52] aleph, I dunno. give it a try? [02:42:58] Well I did [02:43:01] arge [02:43:03] one sec [02:43:11] who's who? [02:43:23] idek anymore [02:43:27] I can't test at the moment because airport and don't have vagrant already setup ont his laptop [02:44:19] halfak: I did. It instantiates a new machine of course, but vagrant up and vagrant ssh work just fine. [02:44:36] Huh. Cool :) [02:44:45] I just want to make sure that there's not something I missing: some other reason why one needs to vagrant up from inside of ores. [02:44:46] so I made this pull request https://github.com/wiki-ai/ores/pull/83 that might make ores work [02:45:10] aetilley: it sets up a virtualenv, installs the requirements.txt, sets up and starts celery and uwsgi [02:45:25] all of which require that you run vagrant up from inside of the ores folder [02:46:02] You mean your new commit? [02:46:08] YuviPanda^ [02:46:30] YuviPanda, do I need to clear out my old VM to properly test it? [02:46:46] aetilley: no... 'it' as in the current vagrant provisioner script [02:46:49] aetilley: my new commit just tries to fix the problem you are having [02:46:58] halfak: yeah, 'vagrant destroy', apply patch, vagrant up? [02:47:22] ok, how about I wipe my ores and re-clone? [02:47:33] or that [02:47:43] * aetilley sprints to bathroom [02:48:22] halfak: What's the next step for wb-vandalism? [02:48:52] We need labeled data. [02:48:56] aetilley: re-clone won't help because my patch hasn't been merged yet [02:49:31] So, we could try the revert detection strategy in WikiData and spot check it. I think we can do a better job than Martin and Stefan if we look for identity reverts. [02:49:38] Amir1, ^ [02:49:57] After merge then. [02:50:47] * halfak waits for goddamn scipy to compile [02:51:08] ok :) I try to dig their "labeled data" to figure out if there is something useful in it :) [02:51:25] aetilley, "vagrant up" [02:52:01] You can use the ORE "label_reverted" script to get identity reverts. [02:52:33] halfak, and nothing else? I'm already up [02:52:35] er [02:52:40] done [02:53:01] oh thanks [02:53:09] Amir1, $ cat pairs_of_revid_and_pageid.tsv | ores label_reverted --api=... --revert-window=48 --revert-radius=3 > revids_and_revert_status [02:53:56] * Amir1 writes this down [02:55:03] there is a damn checkin counter now [02:55:03] and i gotta go stand in line [02:55:03] boo [02:55:06] I'll go [02:56:25] o/ YuviPanda [02:56:42] Amir1, wouldn't trust that command 100% [02:56:54] So check the docs [02:56:59] sure [02:57:22] aetilley, working like you had hoped? [02:58:37] halfak [02:58:39] no [02:58:45] sorry, what did you want me to do? [02:59:08] So, you pulled down the new vagrant file. [02:59:17] no [02:59:27] "After merge then." [02:59:29] What merge? [02:59:38] Oh! [02:59:41] I see what you mean. [02:59:42] I was waiting for you to say it was merged. [02:59:52] Gotcha. Running my test. Going to take a while [03:00:00] I can give you the commands to pull it in. [03:00:04] ok [03:00:14] In the ores repo [03:00:18] $ git pull [03:00:33] wait is this a good idea? [03:00:38] shouldn't I wait for your ok? [03:00:42] Na. [03:00:51] ok [03:00:52] We're not going to affect master :) [03:01:19] k [03:01:23] So... you're on a fork, right? [03:01:34] ya [03:01:55] refork? [03:02:00] no sorry [03:02:02] yes? [03:02:13] I have a fork of an old ores. [03:02:34] $ git checkout -b provision-simplify [03:03:03] $git pull https://github.com/wiki-ai/ores.git provision-simplify [03:03:20] $ git merge master [03:04:24] It's asking me for a commit mess. Any preference? [03:04:35] default [03:04:38] k [03:04:40] Just save and get out of there [03:05:04] We're going to take a big ol' dump in this branch. [03:05:08] :) [03:05:14] And then delete it [03:06:19] already up to date when merge [03:06:52] 'vagrant destroy' then 'vagrant up' [03:06:54] but yeah, mess saved [03:07:10] destroying... [03:07:13] Then I'm guessing you're going to spend a lot of time installing scipy/numpy [03:07:40] slowly getting up [03:07:55] Bah! All workers are offline again. [03:07:59] ORES WTF!? [03:08:47] We can eat through about 400 in the queue in like 3 seconds. [03:08:58] so in general when I want to "re-clone" I should always call vagrant destroy to uninstall? [03:09:15] Usually no. [03:09:34] or just pull from wiki-ai? [03:09:42] In this case, there was something wrong in provisioning and I figured that it would be a good idea to try again. [03:09:53] Yeah. Just pull from wiki-ai. [03:10:01] Oh! And run "vagrant provision" [03:10:38] go with anaconda. it's wonderful [03:11:01] Yeah... not if you're the one developing libraries. [03:11:13] Anaconda is great for the python user. [03:11:42] Ok, so is this the general workflow: Pull from github/wiki-ai/, push to github/aetilley/, submit pull request. [03:13:11] Yup :) [03:13:19] ok. [03:13:35] Hopefully, 'vagrant up' will take far less time soon. [03:14:01] Also, i was going through a basic web.py tutorial online. This is probably a terribly noob question, but what "web framework" is ores? [03:14:18] Oh dear [03:14:18] woof, my machine is starting to get really hot... [03:14:28] flight 2 of my 3 flight journey is cancelled [03:14:56] aetilley, FLASK [03:15:01] ah [03:15:05] accidental caps [03:15:19] k [03:20:37] vagrant up appears to have frozen [03:20:54] I'll give it more time though [03:21:08] yeah. If it's compiling, there are lines that hang a long time. [03:21:12] k [03:23:25] YuviPanda, BTW, ORES went down twice today because the workers just shut down. [03:23:37] Logs suggest an stream of errors. [03:23:46] E.g. timeouts and revision not found errors. [03:24:38] Ok [03:24:52] I think my trip back to the US just turned into a bit of a nightmare [03:24:58] Going to be afk for a while lol [03:24:59] Brb [03:25:00] Boo. [03:25:04] Good luck dude. [03:25:19] I'm seriously considering running fab restart_workers once every could of hours through cron until we figure this out. [03:25:31] Ah the php solution [03:25:37] yeah. [03:25:52] Either that or stay up all night working on it. [03:25:56] And I'm already in trouble. [03:27:21] Just tried a live restart and the precached didn't slow down. [03:27:31] I think this is going to happen. [03:27:37] For tonight. [03:32:18] Yes halfak doit [03:34:49] BTW, the best I have found about workers just going offline after a while: https://github.com/celery/celery/issues/1209 [03:35:01] Closed in 2013 as fixed in 3.1.x [03:35:05] We are running 3.1.8 [03:35:06] halfak: vagrant up finished, vagrant provision called and finished. Chrome still cant see 127.0.0.1:8081 [03:35:43] aetilley: any errors from provision? [03:35:55] no [03:36:01] Also try 8080 I think that's our actual port? [03:36:01] So, the last provision had an error, but this one did not? [03:36:14] Yeah [03:36:18] Yeah. I had aetilley change it because he had a local conflict [03:36:23] on the host [03:36:36] I have tried 8080, 8081, 5000, 2222 [03:36:44] What am I supposed to see? [03:36:52] "Hello I'm ores" ? [03:37:05] Yes [03:37:11] yeah no. [03:37:36] * YuviPanda disappears into flight checks [03:37:46] Journalctl will show what is up [03:37:48] (I only knew that from the intarwebz [03:37:50] oh [03:37:56] lemme try that [03:38:10] sudo journalctl -u uwsgi* [03:38:21] Or even ores* [03:38:31] (Don't remember fully which one it is lol) [03:38:48] derp, journalctl not found [03:38:58] After vagrant ssh [03:39:01] And with sudo [03:39:02] ah [03:40:17] omg very long output [03:40:32] * halfak watches his crontab run [03:40:46] And we're still online. [03:40:55] Good enough for the night. [03:41:17] The error messages appear to be clipped. [03:41:22] aetilley, sudo journalctl -u | less [03:41:30] clipped on the right? [03:42:16] yes [03:42:24] piping it into less helped. [03:42:34] But what would you like me to do with it? [03:43:42] Look at the bottom [03:43:50] I coule $(sudo journalctl -u uwsgi*) > file.txt then email it to you. [03:43:57] ok [03:44:04] su, su, sudio [03:44:06] sorry, i'm useless [03:44:09] If that is easier, sure. [03:44:27] https://en.wikipedia.org/wiki/Sussudio [03:45:15] Bottom says unit uwsgi-dev.service entered failed state [03:46:13] Anything interesting-looking above that? [03:46:41] If now, do sudo journalctl -u uwsgi* | tail -n 100 > file.txt [03:46:48] And email it to me [03:46:51] or pastebin [03:46:56] whatever is easiesty [03:51:15] sent [03:51:21] (gmail) [03:51:36] lies. [03:51:41] It came to the wikimedia.org [03:51:49] my gmail, sorry [03:51:50] haha [03:52:03] and it was the whole thing [03:52:10] not the tail [03:52:39] It's important that you read All of that, Aaron.... [03:53:38] :P [03:53:52] rebooting. brb [03:54:39] You no have line breaks? [03:54:49] Reading this file is torture. [03:54:53] lol [03:55:54] OK. It looks like your local ORES is old! [03:55:57] yay! [03:55:59] Easy fix [03:57:38] back [03:59:04] OK. It looks like your ores is behind. [03:59:11] Did you pull the latest from wiki-ai? [04:01:04] I don't remember exactly what steps we followed, but I didn't do anything other than what you said to do. [04:01:10] oog [04:01:40] k [04:01:50] git fetch upstream [04:01:53] Does that work? [04:02:00] from the host in the ores dir [04:02:32] gotta go. [04:02:38]