[01:04:26] where in the world is yuvipanda? [01:04:49] I want to check out his github/gerrit bot [01:04:58] Physically? India IIRC [01:05:04] heh [01:05:13] I mean why isn't he in this channel? :) [01:05:27] Someone probably got peckish [01:06:44] anyone know what his github/gerrit bot is called? [01:07:19] I really just want to steal his github polling code [01:08:27] found it [01:08:43] ugh. WTFPL [01:08:46] :D [01:08:56] Ryan_Lane2: https://github.com/yuvipanda?tab=repositories any look familiar? [01:08:58] NVM XD [01:09:10] it's SuchABot [01:09:12] https://github.com/yuvipanda/SuchABot? [01:09:20] That's what I was looking at too XD [01:09:29] * Damianz goes back to eating [01:14:48] heh. of course there's no docs [01:14:55] and I have no clue what the entry point is [01:18:24] I think the Gerrit pull is in a different bot. [01:19:25] https://github.com/yuvipanda/labs-tools-gerrit-to-redis? [01:20:09] Oh, you were looking for the "github polling code" ... Okay, Suchabot would probably the place to look at. [01:55:54] Coren: will you handle https://gerrit.wikimedia.org/r/#/c/102617/ ? [01:56:44] (03PS1) 10Tim Landscheidt: Fix Lintian errors in jobutils man pages [labs/toollabs] - 10https://gerrit.wikimedia.org/r/106648 [02:00:28] (03CR) 10Tim Landscheidt: "Ready to merge." [labs/toollabs] - 10https://gerrit.wikimedia.org/r/106648 (owner: 10Tim Landscheidt) [02:24:36] Coren: ping [02:26:29] this is just awesome [02:26:50] wikipedia articles have a box that says "coordinates: ...", that links to the geohack toollabs tool [02:27:14] https://tools.wmflabs.org/geohack/geohack.php?... has a logo of wikimedia labs, from wikitech [02:27:28] hits are going to wikitech like crazy, reaching maxclients 150, and DoSing it [02:27:44] puppetmaster is on the same apache instance as wikitech, and puppet for labs gets down too [02:28:21] so many things wrong in this picture... [02:30:41] Heh. [02:32:09] did GeoHack just get migrated to Toollabs recently, or is this just a new spike? [02:32:21] It doesn't matter when GeoHack got migrated, per se. [02:32:25] It matters when the logo was added. [02:34:49] > Powered by WMF Labs

[02:34:52] So that should be bits? [02:35:04] Or upload, I guess. [02:35:38] yeah, it should be uploaded somewhere more clustery :-) [02:37:36] I wonder if wikitech uses Commons as a foreign file repo. [02:38:06] I'd assume so. [02:38:36] I tried to load wikitech to find out... then remembered it's down. [02:39:07] gj gloria [02:54:33] Gloria: wikitech-static should work [08:58:58] (03CR) 10Legoktm: [C: 032] "Lets see what breaks :)" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106555 (owner: 10Nemo bis) [09:00:24] !log tools grrrit-wm: setting up #mediawiki-feed, https://gerrit.wikimedia.org/r/106555 [09:00:25] Logged the message, Master [09:02:45] (03CR) 10Nemo bis: "something" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106555 (owner: 10Nemo bis) [09:05:52] (03Abandoned) 10Nemo bis: Add gerritfeed-wm for a full feed relayed to IRC [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/87663 (owner: 10Nemo bis) [10:14:09] (03PS1) 10Nemo bis: Avoid double posting to #wikimedia-dev [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106670 [10:40:27] (03CR) 10Legoktm: [C: 032] Avoid double posting to #wikimedia-dev [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106670 (owner: 10Nemo bis) [10:41:03] !log tools grrrit-wm: restarting https://gerrit.wikimedia.org/r/106670 [10:41:05] Logged the message, Master [11:04:09] suppose hashar is not around [11:04:23] * aude would like to do sudo -H -u mwdeploy on beta but is denied [11:05:48] found a way but still want to know [11:27:39] (03CR) 10Yuvipanda: "This should probably be reverted, since this re-clutters -dev again and also makes the rest of the mediawiki/ entries under wikimedia-dev " [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106555 (owner: 10Nemo bis) [11:28:31] (03CR) 10Yuvipanda: "The way to implement -feed is to make a change to grrrit-wm that also sends a raw feed there." [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106555 (owner: 10Nemo bis) [11:29:15] (03PS1) 10Yuvipanda: Revert "Setup/restore #mediawiki-feed" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106680 [11:29:18] (03CR) 10jenkins-bot: [V: 04-1] Revert "Setup/restore #mediawiki-feed" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106680 (owner: 10Yuvipanda) [11:33:39] (03PS2) 10Yuvipanda: Revert "Setup/restore #mediawiki-feed" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106680 [11:33:46] (03CR) 10Nemo bis: "I never heard complaints of #wikimedia-dev being cluttered due to MediaWiki extensions, what do you mean? Maybe you can explain on bug 599" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106555 (owner: 10Nemo bis) [11:34:14] (03PS3) 10Nemo bis: Revert "Setup/restore #mediawiki-feed" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106680 (owner: 10Yuvipanda) [11:34:40] (03CR) 10Yuvipanda: [C: 032 V: 032] Revert "Setup/restore #mediawiki-feed" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106680 (owner: 10Yuvipanda) [11:35:03] (03CR) 10Nemo bis: "Do you mean https://gerrit.wikimedia.org/r/87663 should be (restored and) merged?" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106680 (owner: 10Yuvipanda) [11:39:24] (03PS1) 10Yuvipanda: Add support for a Firehose Channel that gets *all* changes [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106681 [11:39:27] (03CR) 10jenkins-bot: [V: 04-1] Add support for a Firehose Channel that gets *all* changes [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106681 (owner: 10Yuvipanda) [11:39:40] (03PS2) 10Yuvipanda: Add support for a Firehose Channel that gets *all* changes [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106681 [11:44:42] (03CR) 10Yuvipanda: "Reimplemented in Ia7ef98f2146781c073ba09d9bef53cd4147e5903" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/87663 (owner: 10Nemo bis) [11:46:39] (03PS3) 10Yuvipanda: Add support for a Firehose Channel that gets *all* changes [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106681 [11:48:21] (03CR) 10Yuvipanda: "Reimplemented in Ia7ef98f2146781c073ba09d9bef53cd4147e5903" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106680 (owner: 10Yuvipanda) [11:50:01] (03PS4) 10Yuvipanda: Add support for a Firehose Channel that gets *all* changes [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106681 [11:51:47] (03PS5) 10Yuvipanda: Add support for a Firehose Channel that gets *all* changes [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106681 [11:52:42] (03CR) 10Yuvipanda: [C: 032 V: 032] Add support for a Firehose Channel that gets *all* changes [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106681 (owner: 10Yuvipanda) [11:55:45] (03PS1) 10Yuvipanda: Have corefeatures repos post to both -dev & -corefeatures [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106682 [11:55:57] (03CR) 10Yuvipanda: [C: 032 V: 032] Have corefeatures repos post to both -dev & -corefeatures [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/106682 (owner: 10Yuvipanda) [13:17:43] Coren: you about? [13:45:08] Cyberpower678: is there any way to find out that who voiced me or opped me in a channel? [13:46:01] IDK [13:46:25] is there anybody who knows this? [13:54:32] Is there a way to identify (for instance with a qualifier) that an identifier to an external source has been verified ? [13:56:25] sorry wrong channel [15:08:13] Betacommand: Am now. [15:08:29] can you tell me what happened to job 2134264 [15:13:48] Betacommand: killed by the grid. maxvmem 7.986G [15:14:05] Did you really start a job with an 8G vmem limit? [15:15:02] Actually, the interesting question isn't "did you start a job with an 8G limit" but "how in hell did it /reach/ that limit?" :-) [15:16:10] In about 1h; so if it's a leak it's a darn big one. [15:19:13] Coren: did you see your backlog re: virt0/wikitech? [15:19:49] paravoid: The DDOS caused by the inclusion of the image? I did, since it's gone I'm guessing you guys axed it already? [15:20:09] no [15:20:13] the traffic is just lower now [15:20:21] * Coren nods. [15:20:45] I didn't do anything and I doubt anyone else did [15:21:10] The general rules should be "don't hotlink to wikitech"; Ima go make sure geohack doesn't do it anymore. [15:21:47] Also, will email labs-l on the topic. [15:23:20] Coren: the script is fairly massive [15:23:31] Betacommand: Clearly! [15:24:38] Coren: its parsing ~500k pages [15:25:41] ... that's still a 4000x increase in magnitude. I dunno how you're parsing them but it's not very frugal! [15:27:39] Im looking into it [15:29:10] hashar, Silke_WMDE: Is User:WikidataJenkins one of yours? [15:29:32] scfc_de: not mine [15:29:39] let me see [15:30:21] I mean https://wikitech.wikimedia.org/wiki/User:WikidataJenkins, if that's unclear. [15:30:58] Sounds like those folks across the corridor [15:31:04] hang on [15:31:05] scfc_de: Tobi @ wikidata [15:31:17] ah [15:31:18] scfc_de: used to experiment with Wikimedia Deutchland cloud bee instance [15:31:28] will ask him to document [15:33:35] Does the account need shell access? [15:58:06] Coren: is there a way to monitor the number of child processes a script has? [15:58:22] Im not sure python's is working correctly [15:58:45] Betacommand: As a rule, not trivially. You have to enumerate the processes in /proc and compare their ppid. [15:59:03] dam [15:59:13] scfc_de: https://wikitech.wikimedia.org/wiki/User:WikidataJenkins got updated :-] [16:01:12] hashar: So if they only connect to Gerrit, they don't shell access on Labs? [16:01:24] scfc_de: yup no need for shell access [16:01:40] hashar: k, thanks. [16:01:58] scfc_de: and I think you asked me about a user Jenkins-17 previously [16:02:24] scfc_de: must be a user for the QA jenkins which run selenium tests [16:03:20] Oh, so many Jenkinses ... :-) [16:04:53] hashar: Have you had a change to think about if https://gerrit.wikimedia.org/r/#/c/106295/ would work? [16:17:48] scfc_de: haven't looked sorry :( [16:27:31] hashar: I'd like to have a new jenkins job for Scholarships that lints the json i18n files. Should I file a request for that in the Wikimedia/Continuous integration component in bugzilla or somewhere else? [16:32:00] do I need shell access to be able to grep exception logs on production machines? [16:32:47] I'm trying to assess how often a given exception happens and get more data (the UI doesn't show much of it to users) [16:34:42] gi11es: Yes. The logs are on fluorine.eqiad.wmnet and you need at least "moral" access to the prod environment to log in there. [16:35:09] Getting access is relatively painless but requires a 3 day wait [16:35:23] alright, I've put my request on the wiki already [16:36:54] *mortal not "moral" [16:38:10] moral would be good too for log access:) [16:38:29] but yea, please create an RT ticket for it [16:38:43] you can find the email address in -operations channel topic [16:39:11] then you'll need somebody to sign off on it, and add your SSH key .. and a gerrit patch needs to be made [16:39:36] Betacommand: Alternaly, you can call /bin/ps which does just that and parse its output. Not clear that's much better. [16:39:48] thanks guys [16:40:20] "Moral access" is when one really /deserves/ it. [16:41:14] gi11es: You can look at https://rt.wikimedia.org/Ticket/Display.html?id=5872 and https://gerrit.wikimedia.org/r/#/c/86764/ to see an example from when I was granted access. [16:41:38] if you cant login to that via browser, use the email i mentioned to request RT account [16:46:46] bd808: hi [16:46:54] bd808: I guess you can do it yourself more or less :-] [16:47:20] Cool. Do you think that jshint can do that? [16:47:25] bd808: more seriously, we have a bug opened to lint json files. Turns out jshint doesn't play well with json :( [16:47:38] Ah. [16:47:48] bd808: for example json require double quotes whereas jshint enforce single quotes [16:48:00] bd808: there is probably more culprit :( bug is https://bugzilla.wikimedia.org/show_bug.cgi?id=58279 [16:49:09] Okey doke. If it's a known issue then I'll either wait for that to be resolved or figure out how to help resolve it. :) [16:49:42] bd808: I think one can write a basic PHP script which would be given a directory / list of files, open them then pass the content via PHP json_decode() [16:50:23] Yeah. That would actually be pretty easy to add as a phpunit test for me [16:50:43] * bd808 opens a bug to remind himself to do that soon [16:50:54] bd808: just take over https://bugzilla.wikimedia.org/show_bug.cgi?id=58279 :D [16:51:06] we could use a json linter for other repositories as well [16:54:37] Php's json_decode would work but there are probably better linters. The error codes from php are not very granular and it doesn't give line number/position information on failure. [16:55:03] I think I've used some node module to do it in the past [17:03:21] bd808: but then we could hit some corner case where node js will choke on some json that PHP considers valid [17:03:33] bd808: PHP json is a superset of the usual json :( [17:04:16] Yikes. The php script route is probably best then [17:05:24] then you have json_last_error() which gives a INT error code :/ [17:05:28] no line number grr [17:05:49] Yeah. Even in 5.5 they haven't added line number information [17:06:06] I would add it if I knew how to code C for PHP :D [17:12:04] Their parser would need some tweaks. It doesn't track line numbers internally at the moment. https://github.com/php/php-src/blob/master/ext/json/JSON_parser.c [17:13:00] You could take a look at jq: "jq . data/i18n/qqq.json" => "parse error: Expected another key-value pair at line 216, column 1". [17:27:29] anomie, is what I'm seeing on tools-exec-04 normal in regards to the CPU consumption. [17:27:30] ? [17:29:41] Cyberpower678: Ask Coren. But I note it's slowly going down from where it was a few days ago, when there was apparently some task stuck in a busy loop. I'd have thought it'd go down faster though. [17:30:28] !log wikimania-support Updated scholarship-alpha to fbffb96 [17:30:29] Logged the message, Master [17:30:48] anomie, it doesn't affect me because my bot runs on its own node, but I'd thought I ask as it might be affecting other bots. [17:31:40] @seen Coren [17:31:40] Cyberpower678: I have never seen Coren [17:31:43] @seen Coren [17:31:43] Cyberpower678: Coren is in here, right now [17:38:34] I look at it, and it was Hazard-SJ's bot. But looking at the source, I didn't even find a loop. Does Pywikibot do some magic there? [17:38:51] *'ve looked [17:42:18] 5.3G/7.8G (peak 5.3G) [17:42:25] thats a lotta ram [17:47:39] Hmmmmmmm... I appear to have broken the "matthewrbowker" tool... even though I uploaded correctly. PHP error log shows only an "out of memory" error, access log isn't even logging. The weird part about the out of memory error is that it is a duplicate of "matthewrbowker-dev", my staging site, and I haven't had memory errors there. [18:12:25] (03PS1) 10Tim Landscheidt: Retrieve $PACKAGE_VERSION from debian/changelog [labs/toollabs] - 10https://gerrit.wikimedia.org/r/106747 [19:04:24] Coren, do you use nova-precise2 ever? [19:15:27] hello, does someone know what I'm doing wrong here: barras@bastion1:~$ ssh tools-login.wmflabs.org [19:15:27] ssh: connect to host tools-login.wmflabs.org port 22: Connection timed out [19:15:52] Barras2, I can look. [19:16:06] Well, hm. Are you able to reach other labs instances, or is this your first try? [19:16:11] And, are you on windows, mac, linux, ??? [19:16:15] mac [19:16:18] first try [19:16:23] total noob :) [19:16:51] Barras2: Can you reach other SSH hosts? [19:17:23] well, cvn-app1.wmflabs.org works [19:17:37] scfc_de: He's getting to bastion but not the second hop. So probably a key-forwarding thing. [19:18:01] Barras2, I think you'll have the best luck with this solution: https://wikitech.wikimedia.org/wiki/Access#Accessing_instances_with_ProxyCommand_ssh_option_.28recommended.29 [19:18:15] That means that you don't log into bastion directly, your local ssh handles the first hop for you. [19:18:18] Want to give that a shot? [19:18:19] andrewbogott: tools-login.wmflabs.org (not ...pmtpa.wmflabs) is directly accessible. [19:18:31] Ah, true. [19:18:48] andrewbogott: ping [19:18:52] So, in that case Barras2, just leave bastion out of it and ssh directly from your mac. Should work, if you can access bastion already. [19:19:05] Barras2: Do you think someone added you to tools already? [19:19:09] mutex: what's up? [19:19:29] andrewbogott: I came across your nova plugin on github and was hoping I could ask you some questions ;-) [19:20:00] mutex: sure! That code is a little bit stale, but I'm using it for at least one thing on labs... [19:20:24] andrewbogott: pm ? or in channel... might be off topic [19:20:41] Howsabout in #openstack [19:20:49] andrewbogott: With what you linked above, how can I access the ~/.ssh/config file on my mac? (/me is really clueless on that stuff) [19:21:02] andrewbogott: sure [19:21:11] Barras2: Actually you can maybe disregard that link if you're just doing tools stuff. [19:21:21] Try just opening a new shell and 'ssh tools-login.wmflabs.org' [19:21:36] If you are having access problems, please see:https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [19:21:36] Permission denied (publickey,hostbased). [19:22:37] Barras2: But if you do the same for bastion.wmflabs.org it works? [19:23:00] scfc_de: It looks like the high load avg on tools-exec-04 is coming from about 8 processes stuck in the "uninterruptible sleep" state. :/ [19:23:01] andrewbogott: yes [19:23:14] Barras2: Above you said "port 22: Connection timed out". Are you a member of the Tools project? [19:23:29] scfc_de: petan added me earlier [19:23:48] https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Tools&curid=3085&diff=94631&oldid=94620 [19:24:26] It was also my first guess that something there happened wrong probably. [19:24:40] Barras2: try again while I watch the log? [19:24:55] andrewbogott: ok [19:25:31] still getting the permission denied thing [19:26:01] anomie: But "uninterruptible sleep" shouldn't increase the load?! [19:26:48] scfc_de: On Linux, "uninterruptible sleep" is counted in loadavg. I don't know why. [19:26:58] barras, what's your shell name? And, can you paste the command you're using? [19:27:22] andrewbogott: barras is my shell name ;-) [19:28:07] anomie: Those are beetstra's I/O hangers. AFAIUI, they will be there till the next reboot. [19:28:46] :~ Paul$ ssh tools-login.wmflabs.org [19:28:46] If you are having access problems, please see:https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances [19:28:46] Permission denied (publickey,hostbased). [19:29:02] Oh, um… ssh barras@tools-login.wmflabs.org? [19:29:24] amazing, that works! [19:29:48] andrewbogott: thanks a lot :-) [19:29:49] anomie - I seem to be unable to kill those - they do not listen to any form of murder [19:30:00] scfc_de, Beetstra: Yeah, that's typical for something really stuck in uninterruptible sleep. Unfortunately, it's stopping any other jobs from being scheduled on -04 because np_load_avg is higher than 1.75. [19:30:15] Barras2: You understand the difference? [19:31:12] andrewbogott: No really, to be honest. Not really something I have done before ;-) [19:31:30] So, your shell name on your local machine is 'Paul' [19:31:35] But on the remote machine is Barras [19:31:49] So, you were trying to log in with the name 'paul' but with a key associated with the name 'barras' [19:31:54] ahhhh [19:31:54] scfc_de: I don't suppose there's any way to raise the np_load_avg limit on -04 to something like 3.75 until next reboot? (with 8 hung processes and 4 CPU cores, minimum possible is 2) [19:32:07] andrewbogott: I understand, that makes sense. [19:32:09] the barras@ just specifies the name change during the login [19:32:11] ok, cool. [19:32:16] welcome, btw! [19:33:08] anomie, scfc_de: I am afraid the instance needs a reboot .. there is even a 'ls -al' hanging there - one that crashed my putty [19:34:13] Beetstra: Do you remember what directory you were trying to ls? (not that it'll fix anything, I'm just curious) [19:34:16] anomie: That's one possibility; the other question would be why SGE tries to schedule your pending jobs there. The other hosts aren't that much full? [19:34:18] (more info, I found the job was gone, so I started it again - that did not start the bot, I went to the instance, saw that the bot was 'running' twice .. tried to kill it, no avail [19:34:29] come back to the main, job again gone [19:34:54] one sec, have to look [19:35:38] oh shit, that was the wrong command [19:36:03] anomie: Did you specify that queue explicitely? [19:36:13] scfc_de: When I git push to my private repo, I have a post-receive hook that git pulls in the bot's checkout directory, kicks off jobs to all the nodes where my tasks are running, makes sure that NFS actually synced the changed dir, and then signals the tasks on that node to re-exec themselves. [19:36:34] in /home/beetstra/unblockbot/unblockbot [19:37:29] (which I just tried again, and that crashed as well) [19:37:51] the instance is even slow in putty [19:39:52] What MAY have triggered it (but I don't really recall doing that there) is that I have a script that clears the syslogs that I generate (only keeping the last ## lines) [19:40:08] (unblockbot hardly does anything worth syslogging) [19:40:33] maybe they managed to conflict in writing to the syslogs [19:40:36] anomie: "Re-exec themselves" = stop & start? Not that it would fix the underlying problem, but I think you can make SGE do that without having to be on the individual hosts yourself. [19:41:20] Beetstra: It's more which directories you use. /home, /data/project, something "strange"? [19:41:50] scfc_de: "re-exec themselves" = calling exec(). Is there a way to get SGE to wait until NFS synced on the node first? [19:42:27] I don't think I use something strange, especially not that bot - it just is a single perl script reading Wikipedia and reporting to IRC, no logs, no mysql, no wiki-login, nothing [19:42:30] anomie: NFS synced? [19:42:42] it does not edit [19:44:59] scfc_de: I know when log files are written on the exec nodes it sometimes takes a minute or two before the writes show up on -login. If the same happens vice versa when the hook does git pull, the bot process might well re-exec itself but still see the old version of the code. [19:45:37] I do the clearing indeed on the -login server, not on the instance the bot is actually on [19:46:16] And what anomie says I see sometimes (clearing of the syslogs which work one sec, and shortly after the old log is back) [19:46:47] Ah, that is then probably also the reason that bots login and loose their cookies shortly after! [19:47:22] anomie: AFAIK, there's only one NFS server, so there shouldn't be any delays. [19:47:24] I have to make sure that I do that on the instance the bot is running on [19:51:54] scfc_de: Only one server, yes. But each node has to communicate with the server to find out when the filesystem changed, and that seems to sometimes be delayed. [19:54:35] * anomie could just have the hook ssh into each server to launch the little "wait for NFS then signal jobs" script. But that seems excessively evil. [20:00:02] anomie: (I think you can't ssh as a tool to another host.) But when your bot restarts, it has to read from NFS, and that data will be the live stuff, and not some old copy. [20:01:38] scfc_de: The hook runs as my user, and has to sudo to the tool to submit the job (: [20:02:58] scfc_de: So NFS guarantees that the open calls on the files won't return an old cached version, blocking the process if necessary? That'd be helpful. [20:04:55] anomie: That's my understanding, but not based on reading the spec :-); maybe Coren could confirm that? [20:59:55] bd808: re (PHP json parser https://github.com/php/php-src/blob/master/ext/json/JSON_parser.c ) totally need to be hacked in to track line numbers hehe [21:00:21] bd808: iirc that file is no more used in debian because the license is non free ( The Software shall be used for Good, not Evil. ) [21:00:38] can't remember how Debian went around that [21:01:50] It would be possible to track lines there but it would be a non-trivial patch. I haven't looked at the src-deb to see what they hack out with their patches. [21:02:14] well potentially we could json_decode with php [21:02:24] and on failure, use another json decoder that gives better clues [21:24:26] Hi all [21:24:40] Does anyone here have problems attracting women and getting laid? [21:24:53] Nathan184: this is not the channel for spam [21:25:23] Im sure my new ebook will help create a new entality that is attractive to women [21:25:31] mentality* [21:25:33] !ops [21:25:48] yes? [21:26:09] Barras: offtopic spam [21:26:41] !ops [21:26:50] Nathan184: stop [21:27:01] lol ok [21:27:18] Would you be interested in buying an ebook? [21:27:33] no, and please keep on-topic [21:27:55] fine [21:28:41] heh [21:28:58] wow [21:42:16] Yoohoo [21:42:50] I'm thinking about doing a study of the category system usage on commons. [21:43:24] I'd like to collect anonymous data points [user logged in, namespace] for users browsing commons [21:43:53] I'm thinking of doing random samples (1 in X page views sends data to labs) [21:44:04] with X = 100 or 1000 [21:44:30] does anyone see a problem with that? [21:44:43] by "user logged in" I mean a yes/no value! [21:44:58] no IP, nothing else [21:45:14] well, except a timestamp generated on the server [21:56:50] dschwen: From Labs or using the existing methods by the UX people? [21:56:51] Sorry for my absence; pool pump failed. I don't have much time to replace it before there is danger of freezing. [21:56:58] * Coren reads backlog. [21:58:32] anomie: Yes; the high load is "fake" and caused by those dead processes. [21:58:35] Coren: IDK if you saw my message earlier, but I've once again broken a tool :/ I'm sorry. [22:01:28] Matthew_: How so? [22:02:06] Coren: I'm getting an internal error. Only logged thing in ~/php_error.log is out of memory, but these same tools work on my staging site just fine. [22:02:30] "Take" throws no errors, but the couple files I checked are 644, while others are 755. [22:03:24] (or 775, I don't have it infront of me) [22:03:24] scfc_de: what existing methods do we have? Do you have a pointer? [22:03:32] The tool in question is matthewrbowker [22:04:02] I was thinking of just shooting a get request at a tool, which then immediately closes the connection [22:05:16] Matthew_: Take doesn't change file permissions at all; only owner. [22:05:30] Hmmm... so I should chmod manually? [22:05:37] Matthew_: Also, use the new web service setup [22:05:40] !newweb [22:05:40] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help/NewWeb [22:05:45] ^^ [22:05:52] Better error reporting. [22:06:25] dschwen: I'm not sure, but there was a method that the UX people used to gather information about which links were clicked & Co. [22:07:07] Coren: OK, I can do that... let me use the new web service here. I have to do it for two tools (as I have a staging tool set up), one second. [22:07:23] dschwen: http://blog.wikimedia.org/tag/clicktracking/ (from 2009, though). [22:09:12] dschwen: http://www.mediawiki.org/wiki/Extension:EventLogging "Extensions used on Wikimedia" [22:09:40] yeah, well, that is great [22:09:50] but the data I need is not accessible anywhere [22:09:59] not on stats.wikimedia.org at least [22:10:07] and I asked Erik Zachte about it [22:11:52] Coren: No luck, but it's not me: [22:11:53] 2014-01-10 22:11:31: (log.c.166) server started [22:11:54] 2014-01-10 22:11:31: (log.c.118) opening errorlog '/data/project/matthewrbowker/access.log' failed: Permission denied [22:11:54] 2014-01-10 22:11:31: (server.c.938) Configuration of plugins failed. Going down. [22:12:08] dschwen: Sure, you would need to have a new test set up. [22:12:18] chmod access.log? IFF so, what mode? [22:12:21] Matthew_: You have to remove the old access.log [22:12:29] That one is owned by root from the apache setup. [22:12:32] OK, standby. [22:12:42] I /thought/ the instructions told so. Lemme go make sure it does. [22:13:18] Coren: Yeah, it does. I missed it, because that didn't happen on my dev... it's really weird. [22:15:00] looks like they are running this in short campaigns [22:17:36] dschwen: I'm not saying that this is the only approach. But it has been used and there are probably less privacy and performance concerns than forwarding every view of a category page to Tools :-). [22:17:39] Coren: Good news, it started. That appears to have fixed it, that and a recursive CHMOD for some weird reason... Thanks :) [22:20:16] Now if only I could get to replica.my.cnf [22:21:21] Got it, nvm. I'm really thick today :/ [22:29:49] thanks scfc_de I'm working on a schema now [22:30:31] (however I don't think there would be any privacy concerns with the data set I intend, and random sampling could be ajusted to minimize the performance impact) [22:35:20] dschwen: k [22:38:26] Coren: Hmmmm... following links to my tools generates an odd URL: http://www.tools-webgrid-01.com:4080/matthewrbowker/cnrd/ [22:39:17] https://en.wikipedia.org/wiki/Wikipedia:Cross-namespace_redirects#See_also <-- the last link in this section is where I'm getting that behavior. [22:39:28] Matthew_: You're not supposed to try to talk to it directly. http://tools.wikimedia.org/matthewbroker/ does that for you. :-) [22:39:44] Coren: I'm not, that's the confusing part. Look at the link :3 [22:41:30] "http://tools.wmflabs.org/matthewrbowker/cnrd" [22:42:15] Matthew_: don't use _SERVER["HTTP_HOST"], but _SERVER["HTTP_X_FORWARDED_HOST"] [22:42:36] valhallasw: In terms of what? includes.php? [22:42:44] in how you create your URL [22:42:47] Matthew_: That's probably your tool construction a Location: header from _SERVER["HTTP_HOST"]. You might want to give a relative URI instead so that it works fine regardless of whether there is a proxy or not. [22:43:14] Coren: maybe we can fix that in the default lighttpd config? [22:43:22] Hmmmm... that's just it, I don't think I am. The only thing I use a HTTP_HOST for is including a file... [22:44:38] https://github.com/Matthewrbowker/labs I'm not even that... I'm using _SERVER["SERVER_NAME"] [22:44:58] server_name and server_port combined would also give you that, yes [22:45:41] Huh... [22:46:20] I'm just using server_name... and that's to include a file... https://github.com/Matthewrbowker/labs/blob/master/includes.php Hmmmm... [22:47:57] You know the even more confusing part... it only happens when I click a link. Navigating to the URL manually is fine. [22:53:49] Coren: I think it's because crnd is just a directory and the redirect comes from lighttpd itself?! [22:54:24] Hmmm. That's possible, if it tries to imp... oh! Missing slash! [22:55:12] Gah purple! [22:56:04] So yeah; it's the implicit redirect on missing slash that fails badly. Please open a bz; this may not be trivial to fix right for the general case. In the meantime, make sure your links have the obligatory trailing slash. :-) [22:56:10] Matthew_: ^^ [22:57:13] Coren: OK, can do. Will you open a bz, or shall I? [22:57:44] It'd be great if you could; I'm on my iPad. [22:57:51] OK, can do, standby one. [23:01:01] Coren: https://bugzilla.wikimedia.org/show_bug.cgi?id=59926 [23:01:33] Thanks for looking into that, Coren and scfc_de :)