[00:00:49] Analytics-EventLogging, Analytics-Kanban, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1094294 (Nuria) [00:01:07] mforns: ok [00:05:14] Analytics-EventLogging, Analytics-Kanban, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1094305 (mforns) a:mforns [00:06:32] mforns: in varnishcsa the relevant bit is http://sourcecodebrowser.com/varnish/2.0.3/lib_2libvarnishapi_2shmlog_8c_source.html#l00352 I think [00:06:48] which reads the length straight from the memory record [00:06:57] which stores it on two bytes [00:07:10] so a limit of 1014 seems unlikely [00:07:18] tgr, aha [00:16:51] tgr: man, how did found that piece of code so fast. [00:17:52] milimetric, ottomata, you see why Momofuk Ando hit the top views? [00:18:12] nuria, BTW, I'm investigating the GettingStarted/GuidedTour button click issue. [00:18:29] superm401: many thanks [00:35:54] good night everyone, see you tomorrow! [01:45:10] Analytics-EventLogging, MediaWiki-extensions-Sentry, Multimedia: Log EventLogging schema validation errors in Sentry - https://phabricator.wikimedia.org/T90083#1094638 (Tgr) [04:18:57] Analytics-EventLogging: A bunch of GuidedTourButtonClicksNotValidating - https://phabricator.wikimedia.org/T91412#1094789 (Mattflaschen) a:Mattflaschen [04:20:07] Analytics-EventLogging, MediaWiki-extensions-GuidedTour: A bunch of GuidedTourButtonClicksNotValidating - https://phabricator.wikimedia.org/T91412#1082305 (Mattflaschen) [04:35:30] Analytics-EventLogging, MediaWiki-extensions-GuidedTour, Patch-For-Review: A bunch of GuidedTourButtonClicksNotValidating - https://phabricator.wikimedia.org/T91412#1094804 (Mattflaschen) I haven't tested the old versions, but I believe the regression was introduced [here](https://git.wikimedia.org/bl... [05:30:56] Analytics-General-or-Unknown, Possible-Tech-Projects: Pageviews for Wikiprojects and Task Forces in Languages other than English - https://phabricator.wikimedia.org/T56184#1094839 (NiharikaKohli) @Capt_Swing ping. Do you think this task has the volume of work and complexity suitable for a 3-month GSoC/Ou... [06:16:22] Analytics-General-or-Unknown, Possible-Tech-Projects: Pageviews for Wikiprojects and Task Forces in Languages other than English - https://phabricator.wikimedia.org/T56184#1094902 (Doc_James) By the way analysis by Andrew West per this publication has determined that what medical content people look at v... [06:18:45] Analytics-General-or-Unknown, Possible-Tech-Projects: Pageviews for Wikiprojects and Task Forces in Languages other than English - https://phabricator.wikimedia.org/T56184#1094906 (Doc_James) Also would love to see mobile added to the popular page tool. Currently it is only desktop views. Mobile now is o... [10:47:07] (PS1) QChris: Make custom file ending optional for thumbnails in MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194843 [10:47:09] (PS1) QChris: Ban dash from hex digits in MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194844 [10:47:11] (PS1) QChris: Add basic Java implementation of guard framework [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194845 [10:47:13] (PS1) QChris: Add basic shell glue for guard framework [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194846 [10:47:15] (PS1) QChris: Add guard for MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194847 [10:47:17] (PS1) QChris: Allow guard to ignore failures (based on total count) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194848 [10:47:19] (PS1) QChris: Allow guard to ignore failures (based on per-kind count) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194849 [10:53:22] (PS2) QChris: Add basic shell glue for guard framework [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194846 [10:53:24] (PS2) QChris: Add guard for MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194847 [10:53:26] (PS2) QChris: Allow guard to ignore failures (based on per-kind count) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194849 [10:53:28] (PS2) QChris: Allow guard to ignore failures (based on total count) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194848 [12:03:54] (PS3) QChris: Add basic shell glue for guard framework [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194846 [12:03:56] (PS3) QChris: Add guard for MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194847 [12:03:58] (PS2) QChris: Ban dash from hex digits in MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194844 [12:04:00] (PS2) QChris: Add basic Java implementation of guard framework [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194845 [12:04:02] (PS3) QChris: Allow guard to ignore failures (based on per-kind count) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194849 [12:04:04] (PS3) QChris: Allow guard to ignore failures (based on total count) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194848 [12:04:06] (PS1) QChris: Fail less hard for misrepresented urls in MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194855 [12:54:36] Analytics, Analytics-Cluster: Log the X-Cache header in the webrequest logs - https://phabricator.wikimedia.org/T91749#1095420 (faidon) NEW [13:42:36] Analytics-Engineering, Analytics-Wikimetrics: Unable to add a custom cohort user - https://phabricator.wikimedia.org/T91751#1095456 (Chankalun) NEW a:Chankalun [14:42:11] hi milimetric [14:43:02] hey YuviPanda :) heard you've got iPythons for us :) [14:43:09] milimetric: :D yesssssssss. [14:43:16] milimetric: tied to Wiki logins, no less... [14:43:22] that's pretty sweet, I'm not gonna lie [14:43:33] milimetric: and isolated via docker containers, with dumps / replica / persistante home... [14:43:48] milimetric: it’s not puppetized / productionized yet, but OMG IT IS SO AWESOME [14:43:52] oh ok, so now you're just showing off :P [14:43:55] milimetric: I sent halfak several ALL CAPS emails [14:43:56] milimetric: :D [14:44:01] haha [14:44:05] no that's awesome [14:44:15] I’ve a few more kinks to work out.. [14:44:51] yeah, but the docker idea solves all the problems I can think of [14:44:59] and makes this a tool with great potential, good work [14:45:32] milimetric: yup, yup :D need to put up a way to easily publish them as well. this will be a nice complement to quarry [14:45:47] esp. since you don’t need a wikitech account or anything to be able to use them [14:47:09] hmMMMmmmm [14:47:10] http://blog.cloudera.com/blog/2014/08/how-to-use-ipython-notebook-with-apache-spark/ [14:47:10] :) [14:47:53] ottomata: niiice :) [14:48:00] ottomata: there’s also R, Julia, etc kernels... [14:48:12] and because it’s docker, there’s also plenty of ways to scale this out.. [14:49:15] imagine a world where data streams flow from all directions, are forked for shaping into iPython notebooks, and joined back to a central stream for public consumption [14:50:34] :D [14:50:45] Quarry is about to hit 2500 individual queries (and more than 10k query runs) [14:52:05] that's awesome [14:55:14] milimetric: :D I should give it more love in some time… raise limits to 20mins instead of 10, and actually publicize it a bit.. [14:55:28] yeah, I think it'd be very useful [14:56:34] (CR) Ottomata: [C: 2 V: 2] Make custom file ending optional for thumbnails in MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194843 (owner: QChris) [14:57:07] (CR) Ottomata: [C: 2 V: 2] Fail less hard for misrepresented urls in MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194855 (owner: QChris) [14:59:57] (CR) Ottomata: [C: 2 V: 2] Ban dash from hex digits in MediaFileUrlParser [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194844 (owner: QChris) [15:00:55] Seems I am coming /back just at the right time :-) [15:02:14] standup interuption! [15:02:14] :) [15:02:22] Ah. I see. [15:02:27] :-D [15:02:45] Anyways ... maybe I can talk you into doing a release today with the above changes? [15:03:05] Because (if possible) I think I should rerun the mediacounts with those fixed. [15:03:09] s/fixed/fixes/ [15:11:45] milimetric: halfak am off for dinner. current users will continue working, but new users will get a permission denied. I’ll fix it when I come back. [15:12:56] np, bon apetit [15:15:53] kk o/ [15:18:59] mforns: http://bl.ocks.org/yuuniverse4444/8325617 [15:19:18] milimetric, oh, nice [15:19:28] it could use some color work and the hover's a bit wonky [15:19:31] but it's a great start [15:19:35] basically exactly what we need [15:19:46] aha [15:19:58] yes, the other day I was looking at this: http://code.shutterstock.com/rickshaw/examples/status.html [15:20:25] mforns: oh yeah, but that's timeseries no? [15:20:44] yes, it seems it would need more adaptation [15:20:59] yes and your example has also the hover [15:21:19] ok, I'll grab that for today [15:21:59] mforns: no, I mean, rickshaw doesn't let you do anything else except for timeseries [15:22:09] oh I see! [15:22:26] yeah, it's the main reason I didn't want to use it [15:22:37] but it's handy for the simple timeseries stuff [15:22:49] milimetric, aha [15:22:58] ok, thanks for the idea [15:23:21] if I get stuck, I'll ping you :] [15:24:39] (CR) Ottomata: [C: 2 V: 2] Add basic Java implementation of guard framework [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194845 (owner: QChris) [15:41:32] nuria, those jobs timed out waiting for some data to exist i am rerunning them now [15:41:56] i wonder if the timeout is too low for daily jobs, since they often get instantiated before the full day exists [15:45:10] (PS1) Ottomata: Set timeout to -1 for mobile apps jobs. These operate on long periods of data (daily, monthly) [analytics/refinery] - https://gerrit.wikimedia.org/r/194868 [15:45:55] ottomata: but i think that is when you switched the cluster right? [15:46:04] ottomata: so there was "backlog" [15:46:16] ottomata: they run for almost a month w/o issues [15:46:22] right? [15:47:17] nuria: They also timed out before. At least some of them. [15:47:30] qchris: ah yeah? [15:47:41] (CR) Ottomata: "Hey yalls," [analytics/refinery] - https://gerrit.wikimedia.org/r/194868 (owner: Ottomata) [15:47:53] yup. [15:48:00] nuria, i don't think they had [15:48:06] qchris: did not know that, so then maybe we need to amp timeouts [15:48:19] i think i reran them before too, from when I had to wrangle a bunch of jobs after the cluster upgrade [15:48:46] the 25th is after I upgraded the cluster, i thought maybe our cluster resource contention issue might have contributed, but idon't htink so [15:48:54] timeout was set at 5 hours after instantiation [15:49:01] i think for daily and monthly this won't work [15:51:46] nuria: 0001459-150216211537130-oozie-oozi-C@20 is one of the jobs that timed out. That was from 2015-02-18. [15:51:48] ottomata: but probably cause we need to parametize jobs differently, and count backwards rather than forward when it comes to daily partitions [15:52:20] ottomata: so on jan 5th we execute over data from jan 4th [15:52:35] ottomata: that would make more sense with a 5 hour timeout [15:52:53] hmm [15:53:00] joseph was arguing for that too [15:53:12] i didn't buy it, because i like the fact that nominal time matches the data for which you are running the job [15:53:20] * qchris agrees with ottomata. [15:53:48] ottomata, qchris : ya, i get that is more intuitive [15:53:58] why not just a high or no timeout [15:54:00] ottomata, qchris : soooo... what can we do? [15:54:03] i don't really see why we need a timeout [15:54:06] we can do this: [15:54:12] https://gerrit.wikimedia.org/r/194868 [15:54:13] ottomata: unbounded executions lead to problems [15:54:19] it isn't execution [15:54:21] ottomata: on my humble opinion [15:54:33] it is a timeout on waiting for data to exist [15:54:40] before oozie decides it isn't going to happen [15:55:04] timeout: The maximum time, in minutes, that a materialized action will be waiting for the additional conditions to be satisfied before being discarded. [15:56:49] ottomata: as long as "checking" whether that data exists does not take up much resources on the cluster [15:57:07] ottomata: cause oozie will be checking for a longer period [15:57:37] i mean, it just looks for existence of a SUCCESS file [15:57:50] for each of its datasets [15:59:15] ottomata: ok, if you and qqchris agree that is a good compromise let's do it [15:59:19] sorry qchris [16:02:53] so qchris, convince me that it is good to have a top level directory called guard that contains shell scripts :) [16:03:14] :-) [16:03:25] Is there a better place for the shell script wrappers? [16:03:49] They certainly do not belong one of the maven directories. [16:03:59] (So refinery-{tools, hive, ...} is out) [16:04:25] not in resources/ [16:04:25] ? [16:05:09] You mean ... resources in the top level, or refinery-tools/resources ? [16:05:12] src/resources/ [16:05:15] yeah [16:05:30] the main stuff maybe in refinery-core/src/resrouces [16:05:38] and the specific guards with their own projects [16:05:39] Ah. No. Those scripts are just external tooling. The refinery-tools jar can live without them. [16:05:40] like tests are [16:05:49] resources get packed in jars? [16:05:54] test resources go there, no? [16:05:56] Not necessarily. [16:06:16] But regardless ... we do not want the scripts in the jar, or do we? [16:06:19] hm, maybe they shoudl be part of refinery instead of refinery-source? [16:06:25] hmm [16:06:26] hmm [16:06:29] hm. not sure [16:06:30] maybe not. [16:06:38] I would keep them in refinery-source. [16:06:39] it is specific for source [16:06:40] yeah [16:06:43] the guard classes are there [16:06:44] hm [16:06:46] Right. [16:06:51] That's the argument. [16:07:06] wait, why not resources? because of jar? they don't get packed in the jar, do they? do the test resources get packed in the jar? [16:07:30] refinery-core/src/test/resources/ [16:07:31] One can tune what gets packaged into the jar. [16:07:33] GeoIP2-City-Test.mmdb GeoIP2-Country-Test.mmdb access_method_test_data.csv isCrawler_test_data.csv pageview_test_data.csv x_analytics_test_data.csv [16:07:36] aye [16:07:56] But refinery-tools/.../resources feel wrong, because that [16:08:14] direcotry would hould resources that tie to refinery-tools jar. [16:08:25] But the shell wrappers are decoupled from the jar. [16:09:04] hm, put it in refinery-tools/src/main/bash ? [16:09:06] Also ... no one would find them if we hid them underneath refinery-tools/... [16:09:17] true [16:09:26] hmmMMMm [16:09:43] I'd only keep Java stuff (and really mandatory resources) in refinery-tools/... [16:09:53] haha [16:10:03] that is what you said about refinery/source in general [16:10:06] ottomata, nuria : I ran january unique_monthly, and there almost twice the number of iOS [16:10:08] and that is why we have two repositories [16:10:13] in comparison to feb [16:10:50] True. But the separation between refinery and refinery/source is still valid. [16:10:53] I checked duration as well: 1:03 [16:11:04] For daily query, it was 3minutes [16:11:04] Even if there is some shell scripting in refinery/source. [16:11:17] joal: did you look at the e-mail from mobile folks? I was about to do that now so i can answer before they get to the office [16:11:19] I think we could deploy that as well today if you wish :) [16:11:25] For me, running the guard ties way more to the sources than how to create Hive tables. [16:11:28] I have seen it yes [16:11:49] joal: same query with vastly different results in both months suggests data loss (if IOS team hasn't changed anything) [16:11:57] ottomata: and the guards can run completely outside of a refinery setup [16:12:07] I am going to double check numbers with the nem definition for sections [16:12:53] aye, qchris, but we don't deploy refinery/source [16:13:23] qchris: do you intend for this to be automated, or to run it manually occassionly? [16:13:46] ottomata: True. But a simple checkout (outside of the cluster. can be plain fs on any machine) will do. './run_all_guards.sh --rebuild-jar' in a cron will do the rest [16:13:56] ottomata: Automatically. in a cron. [16:14:13] failed output emailed i suppose? [16:14:29] joal: ok, looking at adam's e-mail now [16:14:30] ottomata: Last time you said that it's ok if you get those emails. Yes. [16:14:37] yes, i kinda remember :) [16:15:00] ottomata: But I am not sold on it. If you have better suggestions ... let's hear them. [16:15:28] ottomata: The shell scripting decouples this on purpose, so one can switch from cron+email to whatever one likes. [16:15:37] aye [16:15:41] (CR) Joal: [C: 1] "I think it's a good idea. I would also monitor automatically the date of most ancient waiting job -> it can delay everything for a given j" [analytics/refinery] - https://gerrit.wikimedia.org/r/194868 (owner: Ottomata) [16:21:14] qchris: do these scripts depend on the cwd from which you are launching them? [16:21:15] joal: ok, our queries should pick up data just fine so (query -wise) i cannot find a reason why data should differ greatly between jan and feb .Were android results very different also for january? [16:21:32] ottomata: no. [16:21:59] ottomata: (At least I tried hard that they do not rely on cwd. If they fail for a certain cwd ... that's a bug) [16:22:33] nuria: Last email from Dan suggests having sections=all [16:22:42] I am trying it now :) [16:23:12] joal: no need [16:23:18] ah ? [16:23:21] joal: we only use [16:23:47] joal: ah no, wait [16:24:07] :D [16:24:17] trying it right now [16:24:23] joal: me -> read too fast [16:24:29] np [16:24:45] joal: i think we should remove the "sections" entirely, we are -after all- counting "distinct" [16:24:50] joal: right? [16:25:32] qchris: [16:25:33] I am not sure to fully understand why it was here in the first place, so I am not sure either if removing it is a good idea :) [16:25:33] reset_guard_arguments() { [16:25:33] javascript:; [16:25:33] GUARD_ARGUMENTS=() [16:25:38] can't you just do [16:25:42] unset GUARD_ARGUMENTS [16:25:42] ? [16:25:54] nuria: I am gonna double check numbers with and without [16:26:48] qchris: GUARD="$(basename "$(pwd)")" [16:26:48] ? [16:26:48] ottomata: Yes, one could. But we want GUARD_ARGUMENTS to be an array. So if we unset it, [16:26:53] oh ok [16:26:55] got it [16:27:00] ottomata: we'd have to check whether or not GUARD_ARGUMENTS got initialized (upon adding arguments). [16:27:28] joal: teh thing is that parameter is to distinguish used initiated requests vs not [16:27:32] ottomata: The "basename "$(pwd)"" is just "convention over configuration" [16:27:45] joal: but since we are counting distinct appinstallids it doesn't matter [16:27:46] ottomata: So naming the directory will choose the right Guard class. [16:28:06] ottomata: So e.g.: in https://gerrit.wikimedia.org/r/#/c/194847/ [16:28:23] ottomata: the directory is called MediaFileUrlParser, hence it will [16:28:24] nuria: ok [16:28:34] ottomata: pick the MediaFileUrlParserGuard . [16:28:35] but pwd means you'd ahve to be cded into that dir? [16:28:40] I am still going to double check, it doesn't cost much [16:28:53] tools/common.inc takes care of that. [16:29:22] Sorry. That was wrong. [16:30:11] ? [16:30:12] No it was right :-) [16:30:26] tools/common.inc takes care of "cd"-ing to the script's directory. [16:30:31] cd "$(dirname "$0")" [16:30:39] (CR) Ottomata: Add guard for MediaFileUrlParser (1 comment) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/194847 (owner: QChris) [16:31:04] ??? [16:31:14] oh because common is included from the script in the subdir [16:31:14] uhh [16:31:15] $0 is the name of the script [16:31:24] right [16:31:40] no, its not though, itt isn't included, it is run [16:31:45] https://gerrit.wikimedia.org/r/#/c/194847/3/guard/MediaFileUrlParser/run_guard.sh [16:31:52] ../tools/run_guard.sh [16:31:55] that runs the top level run_guard [16:32:02] oh, that reminds me [16:32:06] * Ironholds whaps qchris [16:32:10] Let's dissect that command :-) [16:32:11] won't $0 be tols/run_guard.sh [16:32:12] haha [16:32:13] oh man [16:32:16] commit messages should explain the desired outcome, not just the script name! [16:32:17] :P [16:32:35] I woke up to a dozen gerrit emails that were like "implement a guard" "implement a guard...in Java" [16:33:05] Ironholds: They do! [16:33:18] ottomata: $0 is the script name [16:33:22] ja, qchris, i might ask something simliar. could you add a big ol README in guard/ about 1. how to use, and 2. how to implement new guards? [16:33:27] i think i'm having trouble following [16:33:35] since it is all about relative imports and directory names [16:33:36] joal: just sent e-mail to dan to understand this better [16:33:40] ottomata: like "guard/MediaFileUrlParser/run_guard.sh" [16:33:46] Thx Nuria [16:33:50] testing qchris... [16:34:06] ottomata: dirname "$0" is then "guard/MediaFileUrlParser" [16:34:36] k. I'll add a README. [16:34:45] ja qchris [16:34:59] [:/tmp] 1 $ cat f1.sh [16:35:00] ./f2.sh [16:35:00] [:/tmp] $ cat f2.sh [16:35:00] echo "\$0 is $0" [16:35:03] ag, will make gist [16:35:33] ottomata: so 'cd "$(dirname "$0")" ' cds to the directory of the script [16:35:56] https://gist.github.com/ottomata/26b69544497d8432c394 [16:36:02] * qchris looks [16:36:10] oh [16:36:11] sorry [16:36:14] that shows that you are right [16:36:15] weird [16:36:22] $0 doesn't get reset when the script runs another script? [16:36:31] oh [16:36:32] yes it does [16:36:33] sorry [16:36:34] ha [16:36:34] yesh [16:36:40] inside of f2.sh [16:36:43] $0 is always f2.sh [16:37:05] Now I am getting lost in what you wanted to say. Sorry. [16:37:09] ok [16:37:13] guard/MediaFileUrlParser/run_guard.sh [16:37:14] does [16:37:18] ../tools/run_guard.sh [16:37:35] "does" means "is a link to" [16:37:39] ? [16:37:41] that means [16:37:46] runs, or executes [16:37:52] OH [16:37:58] OH [16:38:00] it is symlnk. [16:38:00] doh [16:38:01] ok. [16:38:09] i just saw the content in the gerrit change [16:38:13] which shows the path to the file [16:38:18] Ah. True. [16:38:20] which is the same as executing it in a shell script [16:38:21] haha [16:38:21] ok ok [16:38:22] got it [16:38:26] https://gerrit.wikimedia.org/r/#/c/194847/3/guard/MediaFileUrlParser/run_guard.sh [16:38:52] Hahaha. True. That diff /is/ misleading. [16:38:54] joel: will be here, let me know what you find [16:38:55] it does look like you committed a file with the contents ..tools/run_guard.sh [16:38:55] haha [16:39:13] The "Type: Symbolic Link" on the far right is ... well it's invisible. [16:39:20] I had to look for it too to find it. [16:40:04] nuria: https://phabricator.wikimedia.org/P366 [16:40:10] Sounds like we can remove :) [16:40:14] And recompute [16:40:51] joal: that actually makes a lot of sense right? specially when counting distinct ocurrences [16:40:54] hmmm [16:40:59] qchris, the link does the same thing though, no? [16:41:12] ls -l sub/ [16:41:17] link.sh -> ../f1.sh [16:41:33] $ sub/link.sh [16:41:33] $0 is ./f2.sh' [16:42:02] joal: let's wait to see what dan answers to the e-mail just in case and we can re-run jan/feb daily/monthly, right? [16:42:36] nuria: Yes sure :) [16:42:42] Analytics, MediaWiki-Core-Team, Wikimedia-Site-requests: Ran out of captcha images - https://phabricator.wikimedia.org/T91760#1095716 (Nemo_bis) Note, captchas were made a lot harder by df4806c64c48c2cd2cee063611b3193a47c069c8; side effects of new generation FancyCaptcha images have yet to be assessed. [16:42:47] joal: many thnks for looking into this [16:42:59] nuria: But we could have had mobile sessions without sections IN (0, all) no ? [16:43:01] ottomata: Sorry ... I guess I still don't get the issue. [16:43:05] ha, ok [16:43:22] nuria: no problemo, that part af the job, isn't it ? [16:43:25] MediaFileUrlParser/run_guard.sh -> ../tools/run_guard.sh [16:43:31] right. [16:43:46] joal: without this line entirely you mean" AND uri_query LIKE('%sections=0%')" [16:43:47] you still manage the communication with Dan, so it's easier for me :) [16:44:00] then [16:44:02] source "$(dirname "$0")/../tools/common.inc" [16:44:08] from tools/run_guard.sh [16:44:25] right. [16:44:42] nuria: one run as it now, onr with sections in (0, all), one with no check on section parameter [16:44:50] then there are uses of $(pwd) [16:44:53] and [16:45:00] $0 [16:45:04] joal: ok [16:45:53] Removing the check prevents us no to count mobile sessions that wouldn't access sections 0 or all (if it even exists) [16:46:03] in my test at least, $0 will be tools/run_guard.sh [16:46:06] nuria: --^ [16:46:07] ottomata: $0 in all instances should be "the command that was used to invoke run_guard.sh" [16:46:13] ottomata: right. [16:46:54] joal: right, which might be "not user initiated requests" [16:47:30] joal: which we should not do on a mobile connection as it east bandwidth, so .. [16:47:43] joal: that is why i was waiting for dan to answer [16:47:47] nuria: if you say so :) [16:47:48] sorry what? [16:47:50] nuria: ok [16:47:52] no prob [16:47:56] naw, $0 is the current exectuting script [16:48:01] file [16:48:08] I prepare a new patch for the daily refacto [16:48:32] hmmm [16:48:33] including 'sections in ('0', 'all') [16:48:36] nuria: what were you waiting on my answer for? [16:48:43] maybe i am wrong ( i usually am when arguing with qchris) [16:48:58] milimetric: "mobile-dan" not "analytics-dan" [16:49:03] sry, k [16:49:25] ottomata, hive question? [16:49:38] qchris i think in my test my symlink still pointed at a file that exectued another file [16:49:38] sorry. keyboard died. [16:49:41] why the heck do column names always come out as table.col_name now? I swear that didn't used to happen [16:49:45] ungh, i dunno whatever qchris, i trust that you are right on this one :) [16:49:53] anwyay, yeah, right a readme so I can follow better [16:50:04] hive 0.13? [16:50:05] maybe Ironholds? [16:50:06] dunno [16:50:14]