[06:01:48] <wikibugs>	 3Wikimedia Labs / 3tools: jsub not installed on the queue machines - 10https://bugzilla.wikimedia.org/64988 (10bgwhite) 3UNC p:3Unprio s:3critic a:3Marc A. Pelletier On queue machines: Can't exec "/usr/bin/jsub": No such file or directory.                                Can't exec "/usr/bin/qsub": No...
[08:27:19] <wikibugs>	 3Wikimedia Labs / 3Infrastructure: Virtual image for Ubuntu Trusty (14.04) - 10https://bugzilla.wikimedia.org/60684#c3 (10Antoine "hashar" Musso) a:3Andrew Bogott Apparently Andrew Boggot (ops) did some work on this.  Labs now has an image type labeled 'ubuntu-14.04-trusty (testing)'.  Seems to be a good f...
[12:40:07] <Betacommand>	 Coren: when we get in Ive got an isuse
[12:55:47] <manybubbles>	 matanya: I updated the hebrew analyzer settings in beta again last night.  please give it a try when you get the chance.
[12:56:09] <matanya>	 thanks, two minutes manybubbles 
[12:58:07] <matanya>	 manybubbles: excellent!!
[12:58:53] <manybubbles>	 matanya: that is hebrew_exact
[12:59:09] <matanya>	 no false results at all
[12:59:24] <manybubbles>	 are you missing results that you'd expect?
[12:59:29] <matanya>	 but nothing surprising too 
[12:59:30] <manybubbles>	 that is sort of the tradeoff there
[12:59:40] <matanya>	 i'm looking deeper
[13:00:10] <matanya>	 it does give the plural, which is good. the current doesn't
[13:01:09] <matanya>	 and prefix and suffix
[13:01:11] <matanya>	 really nice
[13:01:58] <matanya>	 hmm. not so great with suffix and prefix
[13:02:15] <matanya>	 manybubbles: this is a very good starting point
[13:02:22] <manybubbles>	 sweet!
[13:02:25] <matanya>	 need to tewak from here
[13:02:38] <matanya>	 to get more features without breaking this base 
[13:02:40] <manybubbles>	 so tweaking from here might mean editing hebmorph
[13:02:46] <manybubbles>	 sure
[13:02:55] <matanya>	 i feared you are going to say that
[13:03:14] <manybubbles>	 matanya: I can probably setup special cases
[13:03:26] <manybubbles>	 but I'm not 100% sure on that one either
[13:03:58] <matanya>	 look for example: http://he.wikipedia.beta.wmflabs.org/w/index.php?search=%D7%94%D7%A6%D7%9C%D7%97%D7%94&title=%D7%9E%D7%99%D7%95%D7%97%D7%93%3A%D7%97%D7%99%D7%A4%D7%95%D7%A9&go=%D7%9C%D7%93%D7%A3 
[13:03:59] <Krinkle>	 Coren: crontab for tools.krinklebot and tools.visualeditor disappeared May 1st as well. The one for krinklebot I found in crontab.backup and migrated to jsub (it was failing because command jstart was not found?) 
[13:04:07] <matanya>	 vs http://he.wikipedia.beta.wmflabs.org/w/index.php?search=%D7%91%D7%94%D7%A6%D7%9C%D7%97%D7%94&title=%D7%9E%D7%99%D7%95%D7%97%D7%93%3A%D7%97%D7%99%D7%A4%D7%95%D7%A9&go=%D7%9C%D7%93%D7%A3
[13:04:07] <Krinkle>	 Coren: The one for visusaleditor however I'm unable to find.
[13:04:21] <Krinkle>	 I don't see any ...DATA.crontab nor a crontab.backup and crontab -l is empty
[13:04:51] <matanya>	 manybubbles: those too describe the work success and successfully in hebrew 
[13:05:06] <matanya>	 but one search doesn't find the other
[13:05:27] <manybubbles>	 matanya: hmm - so the trouble is - what I've turned on is "exact" matching
[13:05:29] <manybubbles>	 the other options, 
[13:05:36] <manybubbles>	 "query" and "query_light"
[13:05:58] <manybubbles>	 are all https://en.wikipedia.org/wiki/Lemmatisation analyzers
[13:06:04] <manybubbles>	 which means maybe that they are going too far
[13:06:57] <matanya>	 ok, i would stay with what we have at the moment 
[13:07:10] <manybubbles>	 but to build something in between, a more exact version of "query_light" is going to require java and hebrew knowledge and time
[13:07:16] <matanya>	 i know it is very hard to achieve this in hebrew
[13:07:24] <manybubbles>	 matanya: at least it is better
[13:08:03] <manybubbles>	 another option is to search on both the exact and the query_light analyzers and heavilly boost the exact analyzers
[13:08:15] <manybubbles>	 which _should_ pull those to the top
[13:08:21] <manybubbles>	 then you'll end up with more stuff mixed in
[13:08:42] <manybubbles>	 but, maybe, that is something for another time.  if just getting this deployed makes us better then we were we shuold do it
[13:08:56] <matanya>	 manybubbles: i would poke amir. he has a degree in linguistics, and is a native speaker
[13:09:14] <manybubbles>	 yeah!
[13:10:50] <matanya>	 manybubbles: so basically this is the same level as the current search? 
[13:11:13] <manybubbles>	 is it?  I can't read enough hebrew to tell if it is better or worse
[13:11:17] <manybubbles>	 thats the trouble
[13:11:40] <matanya>	 it is at least at the same level if not better
[13:12:30] <manybubbles>	 ah, well, that feels like a big gain from really really bad
[13:12:44] * manybubbles  sending an email to matanya and amir

[13:13:44] <matanya>	 thanks for all the effort manybubbles 
[13:22:39] <manybubbles>	 matanya: thanks for reviewing it.  I feel totally lost in the vast majority of languages and have to rely on real speakers.  I spend so much time on the code....
[13:47:08] <matanya>	 andrewbogott: hi, does virt1000 need any outside access? i.e web access ?
[13:47:33] <andrewbogott>	 matanya: yes, virt1000 = wikitech
[13:47:38] <andrewbogott>	 so it's a public webserver
[13:47:57] <matanya>	 andrewbogott: anything else needed there? asking regarding ferm rules ?
[13:48:04] <andrewbogott>	 and a dns server and an ldap server
[13:48:22] <matanya>	 isn't that internal ?
[13:48:32] <scfc_de>	 How do the instances' Puppet connect to virt1000?  http?
[13:48:57] <scfc_de>	 Isn't ferm DROP by default for all networks?
[13:48:58] <andrewbogott>	 ldap and puppet are internal but need to be open to labs
[13:49:04] <andrewbogott>	 dns is public I believe.
[13:49:33] <andrewbogott>	 puppet uses http for file access but I think it uses another port as well…
[13:49:48] <matanya>	 ok, so 80/443,53 from outside, 8124 (or whatever puppet is) 
[13:49:49] <andrewbogott>	 matanya: is it possible to just have ferm duplicate the rules that are in place now?  Or are there big gaping holes in the current config?
[13:50:10] <matanya>	 i see no rules atm
[13:50:30] <matanya>	 if you can print the iptable for me, i can for sure duplicate that
[13:55:05] <andrewbogott>	 matanya: sent
[13:55:14] <matanya>	 thanks
[13:56:52] <andrewbogott>	 matanya: coming to Zurich?
[13:57:11] <matanya>	 no, don't have time, going to london
[13:57:43] <matanya>	 but if you video, i'll for sure watch
[14:06:27] <andrewbogott>	 matanya: I'm not presenting anything anyway, just planning to lurk about at Coren's workshop.
[14:06:31] <andrewbogott>	 I'm sure we will meet eventually :)
[15:11:33] <hedonil>	 Cyberpower678: finals over yet?
[15:12:14] <Cyberpower678>	 NOOOO
[15:12:15] <Cyberpower678>	 :p
[15:12:23] <hedonil>	 hehe
[15:12:28] <Cyberpower678>	 I still have to fail it.
[15:12:30] <hedonil>	 :P
[15:12:47] <hedonil>	 Cyberpower678: B+ isn't fail
[15:12:58] <Cyberpower678>	 Look up MATH 251 for PSU
[15:13:23] <Cyberpower678>	 hedonil, what makes you sure that I have a B+
[15:13:38] <hedonil>	 Cyberpower678: educated guess
[15:13:57] <Cyberpower678>	 You guessed wrong.
[15:14:05] <hedonil>	 lol
[15:14:51] <hedonil>	 Cyberpower678: always keep in mind 50 Cent's goal
[15:15:17] <hedonil>	 Cyberpower678: a /real/ hedonistic goal
[15:51:51] <Betacommand>	 Coren: You around yet?
[16:36:58] <notconfusing>	 I creating puppet instances keeps on failing. I get  error 
[16:37:00] <notconfusing>	 (/Stage[main]/Role::Labs::Instance/Mount[/data/project]) Could not evaluate: Execution of '/bin/mount -o rw,vers=4,bg,hard,intr,sec=sys,proto=tcp,port=0,noatime,nofsc /data/project' returned 32: mount.nfs: mounting labstore.svc.eqiad.wmnet:/project/wikidumpparse/project failed, reason given by server:#012  No such file or directory
[16:48:08] <manybubbles>	 !log help
[16:48:08] <labs-morebots>	 I am a logbot running on tools-exec-08.
[16:48:08] <labs-morebots>	 Messages are logged to wikitech.wikimedia.org/wiki/Server_Admin_Log.
[16:48:08] <labs-morebots>	 To log a message, type !log <msg>.
[16:48:59] <manybubbles>	 !log integration installed elasticsearch highlighter plugin that cirrus needs for integration tests
[16:49:01] <labs-morebots>	 Logged the message, Master
[18:03:39] <wikibugs>	 3Wikimedia Labs / 3tools: querycache and querycachetwo tables aren't available on labs sql dbs - 10https://bugzilla.wikimedia.org/63782#c2 (10Luis Villa (personal-for work use lvilla@wikimedia.org)) Only concern that I can see is how often these get stale? is there any risk that a page with sensitive content...
[18:06:54] <wikibugs>	 3Wikimedia Labs / 3tools: Provide filearchive table with fa_storage_key or, if it exists and is sufficiently indexed and populated, fa_sha1 for commonswiki - 10https://bugzilla.wikimedia.org/57697#c3 (10Luis Villa (personal-for work use lvilla@wikimedia.org)) What's the difference between this one and 58993?...
[18:08:54] <wikibugs>	 3Wikimedia Labs / 3tools: Create views for user_daily_contribs table - 10https://bugzilla.wikimedia.org/61300#c3 (10Luis Villa (personal-for work use lvilla@wikimedia.org)) Aggregate count that could be computed from the revision table, if I understand correctly - am I missing something?
[18:16:54] <wikibugs>	 3Wikimedia Labs / 3tools: Tool Labs: Provide anonymized view of the user_properties table - 10https://bugzilla.wikimedia.org/58196#c42 (10Luis Villa (WMF Legal)) (In reply to Tisza Gergő from comment #35) > Maybe instead of a live view, this table could be generated by a query which > runs at the end of the...
[18:47:17] <Damianz>	 Does anyone know of a labs graphite instance I can throw metrics at?
[18:47:32] * Damianz  might be on a mission to track down bot bugs this evening

[18:48:58] <johang>	 why did you guys remove qsub and friends on the tools-exec-* hosts?
[18:49:18] <mutante>	 Damianz: i dunno if/which existing instance, but how about just firing up a new instance, then apply "role::graphite" via webui 
[18:49:37] <Damianz>	 If I'm going to do that, I might just run carbon on tools ;)
[18:49:39] <mutante>	 ideally new instance in existing "graphite" project
[18:50:00] <Damianz>	 I did suggest a general project a while ago thinking about it
[18:50:36] <mutante>	 no biggie, projects and instances are easily created and destroyed again
[18:51:08] <Damianz>	 Well... project creation is a bit hard for non admins :P
[18:51:35] <mutante>	 did you check there is no existing one called graphite?
[18:51:41] <mutante>	 i can make one right now
[18:53:29] <Damianz>	 Gah project listing is slow
[18:53:57] <Damianz>	 Doesn't seem to be... if you make one I'll spend a small while setting it up
[18:54:13] <mutante>	 hold on, brb
[18:54:19] <mutante>	 needs 2-factor auth
[18:54:22] * Damianz  notes this is how he never fixes the bots - gets distracted doing other stuff to monitor the errors on the bots to fix

[18:54:37] <Damianz>	 Yeah 2fa was awesome... until I lost my phone and couldn't login for like 4 months D:
[18:55:01] <mutante>	 backup codes on paper .. saved me
[18:55:10] <mutante>	 but yea, i always need to first get my phone to boot
[18:55:21] <mutante>	 it's always out of battery in that wrong moment
[18:56:24] <Damianz>	 RSA SecureID tokens or Yubikeys are the way forward
[18:56:41] <mutante>	 !log graphite - initial project creation, general project for all things graphite in one place
[18:56:42] <labs-morebots>	 Logged the message, Master
[18:56:58] <mutante>	 !log graphite - made damianz a project admin
[18:56:59] <labs-morebots>	 Logged the message, Master
[18:57:11] <Damianz>	 Tyvm
[18:57:15] <mutante>	 yw
[18:57:37] <mutante>	 eh, wait, that didnt actually work
[18:58:01] <mutante>	 is your user not damianz?
[18:58:21] <Damianz>	 hmm, shell is damian
[18:59:08] <mutante>	 Damianz: gotcha, this worked better.. 
[18:59:10] <mutante>	 Successfully added DamianZaremba to projectadmin. 
[18:59:15] <Damianz>	 :)
[18:59:22] <Damianz>	 Silly wiki vs shell names
[18:59:42] <mutante>	  "damian" works, "DamianZaremba" works, but i tried "damianz"
[18:59:59] <mutante>	 so it's actually the IRC nick, heh:
[19:00:00] <Damianz>	 damianz could work, depending on what thing it is... heh
[19:27:54] <wikibugs>	 3Wikimedia Labs / 3tools: jsub not installed on the queue machines - 10https://bugzilla.wikimedia.org/64988#c1 (10Johan) This affects me to. I run python script in crontab that constructs and submits SGE array jobs. That's no longer possible. Did some research and it seems the exec instances are no longer pe...
[19:52:36] <kaldari>	 Coren: My bots are broken again :( This time they are complaining that bots-sql2.pmtpa.wmflabs is an unknown MySQL server.
[19:52:59] <kaldari>	 looks like they've been broken since March 25 actually
[19:54:33] <kaldari>	 so far the uptime on my bots since migrating to tool labs has been about 10%
[19:55:33] <kaldari>	 :P
[20:00:30] <kaldari>	 if I try entering 'sql local' from my tools account, it says 'Access denied for user 'p50380g50647'@'10.68.16.7' (using password: YES)'
[20:01:30] <kaldari>	 ah, looks like I still have old credentials in my my.cnf
[20:05:24] <kaldari>	 so how do I get access to my old database so I can import it?
[20:06:15] <gifti>	 kaldari: it should still be there
[20:06:39] <kaldari>	 gifti: still be where? on bots-sql2.pmtpa.wmflabs?
[20:06:59] <gifti>	 ah, ok, about this place i don't know
[20:07:27] <gifti>	 hm, iirc there was once an email
[20:07:41] <Betacommand>	 Think the data is gone
[20:07:46] <kaldari>	 keeping up with toollabs changes is a full-time job :)
[20:08:09] <Betacommand>	 kaldari: that happened several months ago
[20:08:25] <kaldari>	 so they just wiped the old stuff instead of importing it to the new DB server?
[20:09:36] * kaldari  cries into his beer

[20:10:07] <Betacommand>	 kaldari: the bot-* servers where decommissioned with at least a dozen emails and warnings 
[20:10:16] <kaldari>	 who reads emails?
[20:10:32] <kaldari>	 I expect a singing telegram
[20:10:54] <Betacommand>	 kaldari: Well the fat lady already sang and you missed it :P
[20:10:57] <kaldari>	 actually, I just wish they would leave shit along and quit changing everything
[20:11:03] <kaldari>	 alone
[20:11:13] <gifti>	 the fat lady?
[20:11:20] <kaldari>	 :)
[20:11:36] <Betacommand>	 gifti: its a joke/reference to a common saying
[20:11:45] <gifti>	 hm
[20:11:51] <kaldari>	 https://en.wiktionary.org/wiki/it_ain%27t_over_%27til_the_fat_lady_sings
[20:12:06] <rohit-dua>	 sorry to interrupt.   os.environ["REMOTE_ADDR"] in python cgi, gives me the server address every-time instead of the clients address. What can be the possible bug? :( 
[20:12:34] <Betacommand>	 rohit-dua: its a privacy issue
[20:12:37] <gifti>	 there were two email from 2014-09-07 and 2013-09-10 about deleting -sql2 but not that they were actually deleted
[20:12:41] <gifti>	 *emails
[20:13:14] <kaldari>	 gifti: must be somewhere in my 14,000 unread emails :(
[20:14:14] <hedonil>	 kaldari: was bots-sql2 an instance? (sounds so). poke andrewbogott_afk, maybe it's mothballed somewhere
[20:14:17] <rohit-dua>	 Betacommand: ok. how can it be solved?
[20:14:25] <kaldari>	 hope so
[20:14:43] <kaldari>	 maybe a have a local back-up around here somewhere...
[20:14:50] <Betacommand>	 rohit-dua: why do you want the user's IP address?
[20:15:05] <gifti>	 user agents are so much better :>
[20:15:17] <hedonil>	 kaldari: if it was on tools-db (unlikely) take a look here  https://tools.wmflabs.org/tools-info/explorer/?dir=public/backups/tools-db
[20:15:41] <rohit-dua>	 Betacommand: storing it for some stat purpose.  do i need to change some permissions?
[20:16:06] <Betacommand>	 rohit-dua: that data isnt exposed 
[20:16:36] <Betacommand>	 Some people could use it like the CheckUser tool
[20:16:40] <kaldari>	 hedonil: nope, not there
[20:16:57] <hedonil>	 kaldari: andrewbogott_afk is your hero then ;)
[20:17:11] <kaldari>	 cool, I'll bug him :)
[20:17:46] <rohit-dua>	 Betacommand: do u mean that REMOTE_ADDR cannot be found on tool-labs?
[20:17:52] <valhallasw>	 rohit-dua: correct.
[20:17:54] <Betacommand>	 rohit-dua: correct
[20:18:15] <rohit-dua>	 ok thank you. 
[20:18:16] <valhallasw>	 rohit-dua: if you want to do statistics, you a) have to ask the user in advance, and b) do in in some other way than by IP
[20:18:38] <rohit-dua>	 maybe i'll use google analytics or smthing
[20:19:06] <hedonil>	 rohit-dua: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Web_logs
[20:19:08] <valhallasw>	 again, you have to ask the user for permission before serving analytics content
[20:19:11] <hedonil>	 rohit-dua:  In general, the privacy policy precludes the logging of personally identifiable information; special permission from Foundation legal counsel is required if such information is required
[20:19:46] <Betacommand>	 rohit-dua: and your access will be revoked
[20:20:18] <kaldari>	 hedonil: actually, looks like I still have most of the data on the toolserver db server. Thank god they never delete anything :)
[20:20:56] <hedonil>	 kaldari: then you are the lucky one :-)
[20:20:58] <rohit-dua>	 thank you for informing. I definatley dont want my access revoked. :)
[20:22:41] <rohit-dua>	 can i store the user-agent?
[20:23:09] <hedonil>	 kaldari: for the records, here's a lis of migrated / mothballed / nuked projects
[20:23:11] <wikibugs>	 3Tool Labs tools / 3Commons Delinker: Wikidata not included in Usage Global of Commons Delinker - 10https://bugzilla.wikimedia.org/65019 (10Shonagon) 3UNC p:3Unprio s:3normal a:3None The Usage Global of Commons Delinker doesn't seem to look in wikidata. Ex : . https://commons.wikimedia.org/wiki/Speci...
[20:23:12] <hedonil>	 kaldari: https://wikitech.wikimedia.org/wiki/Labs_Eqiad_Migration/Progress
[20:23:37] <hedonil>	 rohit-dua: user-agent is ok, it's also in your lighttpd weblog
[20:24:06] <kaldari>	 thanks
[20:24:09] <hedonil>	 *access.log
[20:24:32] <rohit-dua>	 thanks hedonil
[20:25:09] <Betacommand>	 rohit-dua: when foundation legal is involved its not a good idea to try to bypass it
[20:26:12] <mutante>	 using Google analytics is not compliant with the law in .de
[20:26:15] <mutante>	 fwiw
[20:26:27] <mutante>	 piwik is ok though
[20:26:49] <rohit-dua>	 wont piwik be against privacy policy?
[20:26:49] <Betacommand>	 mutante: piwik ?
[20:27:24] <mutante>	 i did not say it's ok with WMF privacy policy, but 
[20:27:26] <mutante>	 http://piwik.org/what-is-piwik/
[20:27:28] <Betacommand>	 rohit-dua: probably
[20:28:44] <valhallasw>	 *any* analytics require *explicit* consent from the user
[20:29:52] <valhallasw>	 I don't see how the german law is relevant in the case of tool labs, though.
[20:32:09] <mutante>	 it's not, i should have shut up. geee
[20:32:22] <mutante>	 i just wanted to say using Google Analytics is even worse
[20:32:47] <valhallasw>	 aah, I see
[20:32:51] <mutante>	 and tools labs has tons of German users
[20:32:56] <mutante>	 because it replaced toolserver
[20:46:30] <kaldari>	 shit, I created my new database with only 1 underscore instead of 2. Can someone drop the database s51249_hotarticles?
[20:47:11] <kaldari>	 I don't seem to have permission to
[20:48:00] <kaldari>	 nevermind, it seems I do actually :)
[20:49:06] <Damianz>	 Hmm... is there a simple way to determin if a user/ip is blocked via the api? As far as I can tell there's about 10 different ways someone could be blocked (or just blocked from certain things like anon editing from ranges)...
[20:58:03] <valhallasw>	 Damianz: petrb had a question about thta on wikitech-l
[20:58:08] <valhallasw>	 let me see if I can find the thread
[20:58:30] <valhallasw>	 Damianz: meta=userinfo&uiprop=blockinfo
[20:58:39] <valhallasw>	 or  list=users&usprop=blockinfo.
[20:58:52] <valhallasw>	 and finally "betacommand suggested to use list=blocks with bkusers or bkip and that
[20:58:55] <valhallasw>	 works fine"
[21:00:16] <valhallasw>	 Damianz: http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/76775
[21:02:57] <Damianz>	 sweet thanks
[21:03:27] <Damianz>	 now just to figure out if openid is enabled or just oauth
[21:40:32] <a930913>	 Out of interest, why can't REMOTE_ADDR be set to a hash of the IP? To track uniques and the like.
[21:56:40] <Damianz>	 What /is/ REMOTE_ADDR set to? Surly an ip address... or that's going to cause me issues
[22:00:53] <hedonil>	 Damianz: http://tools.wmflabs.org/newwebtest/info.php  (scroll to bottom)
[22:01:19] <hedonil>	 _SERVER["SERVER_ADDR"] = 10.68.17.9  (=webgrid-server)
[22:01:37] <hedonil>	 _SERVER["REMOTE_ADDR"] = 10.68.16.4  (=proxy)
[22:03:39] <Damianz>	 Guess I'm not improving the experience for anon users then... that really sucks for transfering anon experience, I'm left with force people to login (apparently people hate that) or captcha (people hate that) for reporting :(
[22:04:48] <mutante>	 "anon" = IP address is public , "non-anon" = all i get is a nickname
[22:04:52] <mutante>	 i never got that
[22:40:34] <hedonil>	 Cyberpower678: ?
[22:41:35] <Cyberpower678>	 hedonil, you just made me die in Borderlands 2. :/
[22:42:08] <hedonil>	 he
[22:42:47] <hedonil>	 Cyberpower678: "The discussion has been closed, with a consensus to remove the opt-in requirement"
[22:42:51] <hedonil>	 really
[22:43:22] <Cyberpower678>	 Where are you quoting that?
[22:43:53] <hedonil>	 Cyberpower678: your talk page
[22:44:17] <Cyberpower678>	 hedonil, ok.  I saw it already and responded.~~~~
[22:44:25] <hedonil>	 yep
[22:45:24] <hedonil>	 I'm finishing the I18 rewrite for xtools
[22:45:48] <hedonil>	 Cyberpower678: shakes his head
[22:46:03] <hedonil>	 bleh
[22:46:12] * hedonil  shakes his head

[22:47:17] <hedonil>	 with the means of an iron broom ;)
[23:38:44] <Damianz>	 mutante: I never got why for signed in users the IP is hidden :P
[23:39:44] <mutante>	 Damianz: yea, let them ask for cloaks like on IRC if they want to hide? *gg*