[00:02:54] Arguably, the least. useful. error message. evar. [00:10:54] Coren: Hi. [00:11:04] Heyas. [00:11:24] http://tools.wmflabs.org/sulinfo/sulinfo.php?username=MZMcBride [00:11:31] That's giving me a fairly lengthy warning about s3. [00:11:38] Listing every database that's allegedly unavailable. [00:11:41] Is the tool broken? [00:12:17] It's not clear that it is, but I also can't see what it could possibly find objectionable about s3 either. [00:12:26] You see the error messages? [00:12:48] I can access metawiki_p. [00:12:53] From sql. [00:12:54] I see the error in the tool, but every test I run on s3 (and s7 for that matter) indicate they're in full health. [00:13:04] Okay, so we'll assume the tool is broken. :-) [00:13:36] YuviPanda|zzz or petan: Can you add me to "sulinfo"? [00:13:41] petan: Can you add me to wm-bot? [00:13:47] The funny part is I've looked at the tool's source and, as far as I can tell, it outputs the message because it genuinely can't seem to connect to s3 -- but I've no idea why and I'm not familiar enough with it to guess. [00:14:03] He talks about "maintainers" on the mailing list every ten fucking minutes and yet he's the only one with access to wm-bot? [00:14:23] I don't think he's the only one. [00:14:33] I'm looking at http://tools.wmflabs.org/ [00:14:46] Is there a better way to get info? [00:14:55] group info * [00:14:56] Elsie: wm-bot runs from the bots project [00:14:59] Oh, AFAIK, that's a placeholder for when he'll move the bot to tools -- it's actually running on a very old instance in the obsolete 'bots' project. [00:15:11] and bots is actually bots.wmflabs.org [00:15:20] Sigh. [00:15:28] I think I actually somewhat knew that. [00:15:30] I asked for access to bots. [00:15:47] * Coren mumbles something about cobblers walking bare footed. [00:16:00] I don't know the analogy. [00:16:09] Also, is there some sort of wiki page I can request access or something? [00:16:30] You could just find a root and ask them to add you. [00:16:34] * legoktm points to Coren [00:16:39] https://wikitech.wikimedia.org/wiki/Nova_Resource:Bots [00:16:47] I thought it required approval from a current group member. [00:16:58] Lets find out! [00:17:06] Let's. [00:17:12] > You must be a member of the projectadmin role in project bots to perform this action. [00:17:17] Right. [00:17:23] I meant social approval. [00:17:33] Though that project is already overrun by users. [00:17:33] I socially approve you. [00:17:38] ty [00:17:45] I haven't eaten in a while and it's showing in my mood. [00:17:59] Labs is annoying me. [00:18:17] Every time I see a tools on Toolserver or Labs, I find the fastest way in my mind to make it obsolete. [00:18:18] It's better if you are added by a project admin Elsie. We'll do it to salvage an abandonned project but opsen will generally not fiddle with access to volunteer-run things on their own impetus. [00:18:24] Everything should be production. [00:18:41] You're a project admin for bots. [00:18:46] Elsie: Much should be, but not everything. [00:18:47] Or so the wiki says. [00:19:05] Well, for example, sulinfo. [00:19:11] Clearly it has value. [00:19:12] Elsie: The wiki is correct, but it's mostly a "technical" adminishy thing. Then again, I know that project has pretty much open membership so I see no harm. [00:19:17] Why can't I just use Special:CentralAuth? [00:19:29] Coren: I'm not in a rush if you want to wait. :-) [00:19:43] Yeah, SULinfo should be brought into the centralauth extension. [00:19:47] I'll tell you why I can't use Special:CentralAuth: because it doesn't include edit counts. [00:19:53] Did you just volunteer? :-) [00:20:09] I bitched about this a few weeks ago on Bugzilla somewhere. [00:20:22] I also apparently have several Toolserver tools I don't remember writing. [00:20:28] {{sofixit}} applies to dev too. :-) [00:20:34] You don't say. [00:20:41] Elsie: Need help porting them over? [00:20:52] (Presuming they are still useful) [00:20:53] Yes, but not to Labs. :-) [00:21:06] They should be in MediaWiki core or MediaWiki extensions. [00:21:30] I'm going to try to trick legoktm to porting one tonight. [00:21:38] Hey [00:21:42] I just killed an extension today! [00:21:46] Which? [00:21:47] That should be good enough I think. [00:21:49] AssertEdit [00:21:53] Oh, really? [00:22:03] You should have gotten 3 bug emails for it. [00:22:09] I'm sure I got many more. [00:22:10] https://gerrit.wikimedia.org/r/#/c/90263/ [00:22:27] I research hit counters today. [00:22:36] * legoktm hits Elsie [00:22:37] Coren: Want two fun facts? [00:22:42] legoktm: +1 [00:22:54] Coren: Fun fact 1: ISBN magic auto-linking support was added in r40. [00:23:00] I reserve my right to agree that they are factual until after I heard them. :-) [00:23:26] Coren: Fun fact 2: Hit counters were added in r51. [00:23:30] https://www.mediawiki.org/wiki/Special:Code/MediaWiki/40 [00:23:31] legoktm: There's always a usecase for something - http://xkcd.com/1172/ [00:23:36] https://www.mediawiki.org/wiki/Special:Code/MediaWiki/51 [00:23:56] This was back when the page table didn't exist. [00:23:58] And it was the cur table. [00:24:15] Anyway! [00:24:17] MrZ-man: :D [00:24:31] Hello MrZ-man. [00:24:36] Yeah, for hit counters I know -- they were then killed because they inflicted much pain and suffering upon the database. :-) [00:24:45] Yeah. [00:24:49] I tracked all that down as well. [00:24:56] I'm drafting an RFC to kill them completely from core. [00:25:03] Because they're ancient and unmaintained. [00:25:09] And the old concept is pretty stupid. [00:25:34] There's a whole hitcounter database table to update page.page_counter. [00:25:49] Even though all of the code is disabled on any high-traffic site. [00:26:03] And on low-traffic sites, it just makes things slower and provides no real value. [00:26:11] Okay, that's enough rambling for now. [00:26:25] What about Special:MergeHistory? It's been in core since 2007 but disabled by default [00:26:36] One thing at a time. [00:26:59] MrZ-man: I was told that that's unfinished [00:27:03] half-baked [00:27:08] No kidding. [00:27:10] That's why it's disabled. [00:38:02] Thanks legoktm [00:38:09] np [00:38:39] But I do have a python / mysql related issue not sure if anyone here can answer. [00:38:45] ProgrammingError: (2014, "Commands out of sync; you can't run this command now") [00:39:19] here is the query i am running: ur'''SELECT /* SLOW_OK_LIMIT: 1800 */ user_id, user_editcount from user''' [00:39:59] after i do execute : self.dbCursor.execute(query) [00:40:23] I do this: for user in self.dbCursor.fetchall(): [00:40:34] umm [00:40:34] I am using SSDictCursor [00:40:37] can i see the full code? [00:41:02] is it ok to past here..? [00:41:10] use a pastebin [00:41:12] dpaste.de [00:42:21] https://dpaste.de/fBZG [00:43:42] legoktm: Link works? [00:43:46] if so line 128 [00:43:49] yeah im reading it now [00:44:13] Since its a SSDictCursor fetchall will exhaust the cursor [00:44:30] so I don't get why its complaining [00:44:38] you should really use batched queries [00:44:44] fetching the entire user table at once is a bad idea [00:45:07] it will not do that since its SSDictCursor [00:45:21] thats the beauty of SSDictCursor [00:46:31] huh, interesting. [00:47:20] and if when I run this on interactive mode without submitting to job queue it works find [00:47:23] *fine [00:47:31] http://stackoverflow.com/questions/11583083/python-commands-out-of-sync-you-cant-run-this-command-now <-- maybe you need to close the cursor? idk [00:47:43] so I am not sure why it gives me this error on the job q [00:47:56] not sure either, maybe Coren knows? [00:48:22] "I'm afraid I can't recall, senator" [00:48:49] :P [00:49:49] Seriously know, I'm a python newbie and know even less about its DB interface. I'm sure you'll have more help from one of our Python experts. andrewbogott_afk is a good bet. [00:49:56] s/know/now/ [01:17:13] Coren: Any chance you could look at why AndyRussG is getting "Permission Denied" on logon please? [01:17:34] He's in the bastion project but doesn't seem to have a homedir [01:18:10] The two are normally unrelated; homes are created on first succesful login. [01:18:30] First thing to check: case significance of username when logging in. That's what catches most people. [01:19:39] OK, that was it! I'm in with my username all set to lowercase [01:21:05] Thanks! [01:23:56] Coren: do you have shell access to beta.wmflabs? [01:24:10] That sounds a redundant question [01:24:21] PiRSquared: I don't, but I can grab it at need. What's up? [01:25:00] I forgot my password there, didn't set my email, but have rights [01:27:48] Are the production replicas only available from tool labs? [01:29:20] Reedy: no, but it takes some effort to make them available elsewhere [01:29:49] Ok that's fine [01:29:54] So what you're saying is yes [01:29:55] ;) [01:30:04] well, I mean effort on your part ;) [01:30:10] it's technically available on all projects [03:23:22] Hi folks, I have another quick issue here [03:23:37] Trying to dump a table from the database [03:23:58] On tools [03:24:03] "SELECT * FROM ep_articles INTO OUTFILE '/home/andyrussg/ep_articles.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';" [03:24:17] Says "ERROR 1045 (28000): Access denied for user 'u3609'@'%' (using password: NO)" [03:27:28] Hmm, I can do "SELECT * FROM ep_articles", I guess it's the "INTO" part that it doesn't like [03:30:14] OK, I think I see what's wrong and another solution [03:37:36] OK, solved it..! [14:23:11] Coren: Job 886235: the script re-execed itself, logged its re-startup, and then the job disappeared. All the other AnomieBOT jobs re-execed themselves at around the same time without issue. Any ideas? [14:23:52] anomie: Without so much as a log entry? [14:24:28] Coren: Nothing written to stderr or stdout after the message it writes to stderr during startup. [14:27:49] anomie: According to qacct, it failed to restart because the sheperd itself failed -- a quick google reports a few people saying they get this error very infrequently but completely unpredictably, which doesn't help us. [14:28:49] * Coren ponders. [14:28:53] Wait, when was this? [14:30:29] About 12:50 UTC today [14:43:34] * Coren checks the logs for correlation but finds nothing of interest. [14:43:53] Do you know what caused the re-exec? [14:46:02] I pushed a revision into the bot's private git repo. That job was the one that checks the private git repo for updates periodically: it pulled the new version, puts info into redis to tell all the other jobs to re-exec themselves, and then re-execs itself. [14:48:11] Hmmm. I could see how NFS might play a role in a scenario like this; there is caching at a number of layers and while NFS guarantees that any single client's view of the filesystem is time-consistent, there is no guarantee that they are so /between/ clients; if there is little/no delay, it's possible that the actual update was only partially completed when one of them restarted. [14:48:55] (I.e., that bot would have restarted while that node's view of the filesystem was still "during the pull" and might have had partial files or such) [14:49:47] Wait, the job that /didn't/ restart is the one that did the pull? [14:50:23] That kills that hypothesis, by definition this'd be the one guaranteed to have the latest view. [14:51:12] The job did restart. It got at least as far as line 40 in /data/project/anomiebot/bot/bot-updater.pl, and I think even line 49, before it disappeared. [14:53:08] Well, according to the sheperd, your job exited with a status of 0 at Thu Oct 17 12:51:04 2013 without having even gotten near its resource limit. [14:54:02] So it wasn't killed by the grid itself. [14:59:28] Hmm. Every "exit 0" in the bot is preceeded by writing something to stderr. This may have to remain a mystery, I suppose. [14:59:37] Coren: BTW, is there any command that does "wait for NFS to update" to avoid that caching issue you mentioned earlier? [14:59:40] * Coren doesn't like mysteries. [15:00:36] anomie: Not as such. Good defensive coding on NFS is to synchronize processes through the file system since that means you're at the same point in "time". [15:01:30] I.e. your update job finished by creating a status file of some kind, and the clients include waiting for that file to be there before they actually restart. [15:02:40] Although, honestly, that's usually overkill unless you have processes on different nodes trying to write to the same place. [15:06:06] The point here is just to have the various processes re-exec themselves when the code changed, so I don't have to go do it manually. [15:42:09] Coren, thanks for making the webserver change happen. [15:42:50] Hey, it's my job to make better mousetraps. :-) [15:42:51] You may remember that I opened a bug about getting fcgi to work. In theory this should work with the new setup. [15:43:14] It should indeed although, right now, I think it's only been tested with PHP [15:43:20] I just cannot figure out how. I need a custon fcgi binary to run, not just PHP via fcgi [15:43:49] dschwen: You'll need to add a configuration stanza to your .lighttpd.conf [15:44:12] http://iipimage.sourceforge.net/category/new-release/ I tried that (with adjusted paths) [15:44:39] but I assume I cannot just add a location like "/iipserv" to the configuration [15:45:23] it is not quite clear to me how much freedom I have in my lighthttpd config if it gets merged with an unknown master config [15:46:08] lighthttpd throws error messages about parse errors in the config, but the errors refer the merged config, which is gone after ther server fails to start [15:46:18] how can I debug this black box? [16:13:34] 2013-10-17 16:13:07: (configfile.c.912) source: /var/run/lighttpd/zoomviewer.conf line: 572 pos: 1 parser failed somehow near here: (EOL) [16:13:49] and cat: /var/run/lighttpd/zoomviewer.conf: No such file or directory [16:13:54] :-( [16:38:25] Coren: Re the line number problem in ~/.lighttpd.conf, could the generated config use 'include' instead of just catting the user's config file? [16:56:14] anomie: It could, that's an artifact of the previous method I forgot to undo [16:57:07] dschwen: Give me a minute to do a proper include so you get correct line numbers. [17:44:27] in any case, when I try to define a fastcgi.server in my config I get a "Duplicate config variable in conditional 0 global: fastcgi.server" error [17:48:51] oh! I think I've got it running! [17:49:26] the key is using fastcgi.server += (... rather than fastcgi.server = ( [18:10:17] now I have intermittently working fcgi. 5% of the requests work, the rest returns error 403 [18:10:19] http://tools.wmflabs.org/zoomviewer/iipsrv.fcgi?FIF=/data/project/zoomviewer/cache/779543aa14d92a2dff180a4cbc0eb2f6.tif&cnt=1&sds=0,90&jtl=7,861 [18:12:41] hm.. I'll check first if that method of access is safe... [18:49:12] dschwen: Sorry for the delay in my answering -- I was stuck in a meeting. [18:49:27] That took *way* longer than expected. [18:52:31] Coren, what's the update on cyberbot? [18:53:01] It's next on my todo. Prolly this PM. [18:53:28] Provided I don't get stuck in anothe meeting. :-0 Thankfully, I've got none planned. :-) [18:54:47] Coren, What's been keeping you these past days? "Office meetings" aka hot tub and 80 flat screen? [18:54:49] :p [18:54:58] * Coren wishes. [18:55:14] :D [18:55:44] * Cyberpower678 any opening position for me at the foundation? [18:55:50] Coren, ^ [18:56:13] https://wikimediafoundation.org/wiki/Work_with_us [18:56:24] http://wikimediafoundation.org/wiki/Work_with_us [18:56:29] * Coren grins.' [18:57:32] Reedy: Coren: blank page on mobile... lol [18:58:09] You mean the site doesn't work on a mobile device? [18:58:33] ... "we don't hire no steenking mobile users"? :-) Seriously, I think that's a serious bug and should be raised. [18:58:54] That page has no content on my Android. Just the desktop link on bottom of page. [18:59:15] T13|needsCoffee, well�that's Android. [18:59:33] Cyberpower678: "Wikimedia’s Mobile Web team needs you to optimize and evolve the infrastructure that powers all of our mobile projects." :) [18:59:34] :D [18:59:38] Just took a screenshot.. lol [19:00:15] mutante, who said I wanted to work with mobile page design. [19:01:12] Cyberpower678: ah, i should have said T13|sleeps , he found the mobile issue [19:01:41] btw, this used to work too, we made special rewrites for it: [19:01:45] http://wikimediafoundation.org/wiki/Job_openings/foo [19:01:52] but it seems now they are all just jobvite links [19:01:59] and not subpages on wiki anymore [19:02:35] eh, by rewrites i meant: from here http://jobs.wikimedia.org/foo to http://wikimediafoundation.org/wiki/Job_openings/foo [19:06:53] Reedy https://commons.wikimedia.org/wiki/File:Work_with_us_-_Wikimedia_Foundation.png [19:07:55] it says "Go to mobile view" but also that Mobile view is already selected? [19:08:20] what happens when you follow the link to mobile view? [19:08:42] It loads the desktop version [19:08:53] I think, let me make sure. [19:09:33] #wikimedia-mobile [19:09:40] yea, i was gonna say, please report it there [19:10:29] I double checked and it loads a bullet list of openings [19:32:12] YuviPanda: Around? [19:32:18] bd808: sure [19:32:52] I haven't talked to you for a while. Having Panda withdrawl [19:32:57] bd808: awww [19:32:59] bd808: :D [19:33:00] bd808: <3 [19:33:14] I have some labs related questions too :) [19:33:18] bd808: ah! :) [19:33:37] bd808: been reducing my keyboard time with book time and exercise time [19:33:37] so [19:33:38] bd808: 'sup [19:33:50] I'm the proud new owner of the project to bring the wikimania scholarship app in house [19:34:21] I've been told that getting the existing code running in labs would be a good place to start [19:34:39] bd808: haha! [19:34:41] bd808: indeed [19:34:48] So…. I need a new labs project I think. And I figured you would point me to the right wiki pages to read [19:35:08] bd808: getting a new one is mostly... 'poke andrewbogott or Ryan_Lane or Coren' [19:35:38] bd808: https://wikitech.wikimedia.org/wiki/Help:Instances helps [19:35:53] bd808: but I think https://wikitech.wikimedia.org/wiki/Help:Self-hosted_puppetmaster is the most important bit especially if you are going to be puppetizing stuff [19:35:59] since it lets you test out patches to ops/puppet [19:36:00] in labs [19:36:35] * bd808 fumbles for 2-factor token [19:36:55] bd808: hehe [19:38:14] YuviPanda: Thanks for the links. The puppet one looks especially useful [19:38:18] bd808: indeed [19:41:28] bd808: so what I usually do is have a 'test' instance on which I test my puppet changes, and then an 'actual' instance where the puppet changes are aplied from ops/puppet [19:41:40] bd808: there's also a good amount of begging in -operations to get your patches merged :P [19:42:23] YuviPanda: I think it will take a while before I'm even ready for begging [19:43:04] I haven't looked at the app code much yet, but I've been led to believe it needs a fair amount of love before we'd put it on the cluster [19:43:51] "disconnected by services" geez [19:44:00] So step 1) run it somewhere, 2) clean it up, 3) bring git in-house, 4) get security review, 5) Profit! [19:44:35] bd808: heh, right [19:44:45] bd808: yeah, (1) can be done locally too [19:45:18] YuviPanda: Sure, but I'd like to have a place to demo and share sooner rather than later [19:45:41] of course [19:45:51] I think Chad will be giving some help and Katie from WMDE [19:46:07] oh! [19:46:09] that's nice :D [19:46:33] I think they've both maintained the product at some point. Katie may be the original author [19:47:08] PHP? [19:47:13] Yeah. https://github.com/filbertkm/Scholarships [19:47:31] Not wiki based, just a data app [19:47:56] I think it takes in scholarship requests and then gives a UI to evaluate them [19:48:20] AH [19:48:21] NICE [19:48:26] grr [19:48:29] dammit, need to remap caps lock to esc [19:49:10] I did that last week but haven't trained my fingers to find esc at caps-lock yet [19:50:38] My new keyboard has esc on a chiclet key that sucks to hit [19:50:45] bd808: new laptop? [19:51:04] New desktop keyboard [19:51:08] ah [19:51:18] bd808: i'm using a (borrowed)mechanical one [19:51:23] it's pretty awesome :D [19:52:02] real keyboards are nice. [19:52:08] bd808: true [19:52:58] I'm giving this one a try: www.amazon.com/gp/product/B00CYX26BC/ [19:53:35] I like that it's narrower than my old natural. Reach for the mouse is better [19:53:44] but the escape key really sucks [19:58:26] YuviPanda: What would be a good name for the project? Just "scholarship" or a wider something like "wikimania" ot "wikimania-support"? [19:58:43] bd808: i like the last [19:59:14] That sounds good to me too. Then it could be used for whatever comes up. [19:59:20] indeed [19:59:26] bd808: and lesser projects == good [20:00:05] There's only 175 currently. :) [20:00:35] bd808: heh [20:04:46] soo... my little cgi server crashes and upon analyzing the coredump it seems it crashes in libjpeg.so.8 [20:05:13] is there any chance that the dev and webgrid environments diverge with respect to libjpeg? [20:05:58] dschwen: what exactly is the error? [20:06:11] dschwen: is it probably just running ouit of memory there? [20:10:56] hm [20:10:59] unlikely [20:11:09] it is just preparing a 256x256px image [20:11:39] dschwen: Coren should be able to help better [20:20:28] andrewbogott, Ryan_Lane, Coren: New project request at https://wikitech.wikimedia.org/wiki/New_Project_Request/wikimania-support. I'm going to be working on hardening a community developed app that is used to handle Wikimania scholarship requests. The plan is to have this running on our prod cluster in January. [20:21:58] dschwen: They shouldn't diverge; they're both deployed with puppet from the same config. Are you running a compiled executable you built on -dev? [20:23:21] bd808: Seems legit. [20:23:36] Coren: Excellent [20:24:16] The last project request I submitted was quashed, but for good reasons. [20:24:23] bd808: they've fallen for your ruse! [20:24:40] * bd808 giggle maniacally [20:24:55] Coren, are you creating or shall I? [20:25:55] andrewbogott: Way ahead of you, man. :-) [20:26:02] 'k [20:26:31] bd808: Remember that if you intend for this to end up in production, it needs to be 100% puppetized. :-) [20:27:00] Coren: You shouldn't expect anything less [20:27:28] * bd808 crosses fingers that will be easy enough to pull off [20:27:58] I think the final app will be an apache config, some php code and a mysql db [20:28:20] That describes 80% of the web. :-) [20:29:08] Maybe I'll shop it to a VC for seed funding too [20:29:20] Make sure you don't rely on the database living alongside your app though; if this ends up in production it'll have to use the "real" databases. [20:30:45] Coren: Sure. I'm actually more accustomed to that being the case. [20:31:11] * bd808 has a seedy corporate past full of separation of concerns training [20:33:22] Eeew! Suit! [20:33:50] Coren: No suits, but lots of process and audits and TPS reports [20:42:41] Coren, I built on login... yikes. [20:43:57] dschwen: That's impolite, but shouldn't change a thing. It's possible that some nodes have very slighly newer minor versions of some libraries, but I'd be surprised if that broke the ABI. If you give me a bit, I'll be glad to look at your core dumps soon (better yet if you rebuild with -g3 first) [20:44:42] will do [20:44:46] on -dev :-) [20:45:51] builds from login and dev have the exact same md5sum [20:49:34] Coren, I built with -g3 and got a new coredump [20:49:52] however the bt is still meaningless [20:50:11] probably have to compile against a libjpeg-debug [20:56:33] Coren: Thanks for setting that project up by the way. [21:15:00] Coren: re: toolserver wiki [21:15:08] Coren: is it possible to do an import into a namespace? [21:15:16] I thought it does a direct import [21:15:36] ah, seems you can transfer into a namespace [21:16:00] they seem to be under the same license, which is good [21:47:30] Ryan_Lane: Yeah, you can to import to namespace; with a bit of sed on the XML if nothing else. [22:37:31] Ryan_Lane, is it likely that the labs base image used to include php, and no longer does? [22:37:42] no [22:37:49] it never included php [22:38:05] hm. I actually need to make a new image [22:38:41] it seems this image version is missing man [22:38:49] Hm... [22:38:52] :) [22:38:54] why do you ask? [22:38:56] * andrewbogott asks the perennial question "how did this ever work?" [22:39:24] :D [23:47:03] I'm trying to bring up a new instance in the wikimania-support project that was created today. [23:47:35] Everything works great until I add the nfs:client role, run puppet and rebot [23:47:48] after that I can no longer ssh into the instance [23:48:23] I get connected to the ssh server on scholarships-alpha.pmtpa.wmflabs, see the motd and then it closes the connection [23:49:04] I'm wondering if this is a known problem or caused by me doing something wrong [23:49:23] I've had the same behavior from two instances in a row so far [23:50:51] andrewbogott, Ryan_Lane: Any idea why I wouldn't be able to ssh in after enabling role::labsnfs::client? [23:53:50] ssh -vvv output at https://dpaste.de/uyFx [23:54:54] Looks like the issue is "Unable to create and initialize directory '/home/bd808'" [23:55:21] Which probably means that the nfs mount didn't happen?