[00:27:37] 06Revision-Scoring-As-A-Service, 10Beta-Cluster-Infrastructure: deployment-sca03 can't call puppetmaster - https://phabricator.wikimedia.org/T143958#2584624 (10mmodell) Looks like that error is related to {bf86143048c9ef821f37f0bfe5e1d2b72ce6d4a5}, that modified the pick_initscript function which probably requ... [00:30:33] 06Revision-Scoring-As-A-Service, 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 13Patch-For-Review, 15User-Ladsgroup: Switch beta to use the proper wiki models for scoring (rather than "testwiki") - https://phabricator.wikimedia.org/T143567#2584704 (10mmodell) [00:30:36] 06Revision-Scoring-As-A-Service, 10Beta-Cluster-Infrastructure: deployment-sca03 can't call puppetmaster - https://phabricator.wikimedia.org/T143958#2584701 (10mmodell) 05Open>03Resolved a:03mmodell After restarting the puppetmaster, a puppet run on `deployment-sca03` succeeded: `twentyafterfour@deploym... [00:45:41] (03PS1) 10Catrope: Add UNIQUE indexes to ores_classification [extensions/ORES] - 10https://gerrit.wikimedia.org/r/306854 (https://phabricator.wikimedia.org/T143962) [08:59:50] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES, 13Patch-For-Review, 07Schema-change: Add uniqueness constraints to ores_classification - https://phabricator.wikimedia.org/T143962#2585296 (10Ladsgroup) [10:31:32] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES: $wgOresModels set all models true - https://phabricator.wikimedia.org/T143988#2585410 (10Ladsgroup) [10:56:14] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES: $wgOresModels set all models true - https://phabricator.wikimedia.org/T143988#2585440 (10Ladsgroup) Update this only happen in enwiki, dewiki, fawiki, and wikidatawiki are okay but ores_classification is exploding for enwiki [10:58:12] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES: $wgOresModels set all models true - https://phabricator.wikimedia.org/T143988#2585441 (10Ladsgroup) ``` mysql:wikiadmin@db1055 [enwiki]> select count(*) from ores_classification group by oresc_model; +----------+ | count(*) | +----------+ | 492691... [11:07:13] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES, 13Patch-For-Review, 15User-Ladsgroup: $wgOresModels set all models true - https://phabricator.wikimedia.org/T143988#2585454 (10Ladsgroup) a:03Ladsgroup [11:07:58] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES, 13Patch-For-Review, 15User-Ladsgroup: $wgOresModels set all models true - https://phabricator.wikimedia.org/T143988#2585456 (10Ladsgroup) Okay, I found the issue. We should find a way to clean up those later [11:59:48] 06Revision-Scoring-As-A-Service, 10Wikilabels, 15User-Ladsgroup: Deploy wikilabels - https://phabricator.wikimedia.org/T143992#2585553 (10Ladsgroup) [12:08:40] 06Revision-Scoring-As-A-Service, 10Wikilabels, 15User-Ladsgroup: Deploy wikilabels - https://phabricator.wikimedia.org/T143992#2585576 (10Ladsgroup) 05Open>03Invalid [12:08:57] 06Revision-Scoring-As-A-Service, 10Wikilabels, 15User-Ladsgroup: Deploy wikilabels - https://phabricator.wikimedia.org/T143992#2585553 (10Ladsgroup) Already done three days ago by @halfak [12:15:26] (03CR) 10Thiemo Mättig (WMDE): [C: 031] "Code looks good. Can't say much about the impact this haves. I believe all WHERE clauses executed on this table should be considered for a" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/306854 (https://phabricator.wikimedia.org/T143962) (owner: 10Catrope) [16:57:39] o/ sabya [16:57:47] I saw you looking for a better ipython notebook [16:57:48] o/ halfak [16:57:53] yes [16:57:55] do you know about paws.wmflabs.org? [16:58:19] didn't. just checked out. but that does not solve my problem.... i think [16:58:45] say I have a long running job running at labs server [16:59:03] i initiated that from my computer [16:59:21] now I leave for work [17:00:02] Yeah. that'll work on paws. [17:00:13] Just log in from home and it will be running still -- or finished [17:00:25] I've got to run. I'll be back in ~40 mins [17:00:33] ok [17:35:25] back! [17:37:39] sabya, did you check out PAWS? Does it look like it will work for your needs? [17:39:52] halfak: not yet. working to evaluate first model with 50 selected features. Right now I have a jupyter notebook hosted in an instance at ores-staging. [17:40:42] using ssh tunnel to that instance and opening from local browser. [17:48:16] sabya, I see. Maybe you could start up your jupyter instance in a screen to avoid the disconnection issue. Would that work? [17:48:47] yes. using tmux. [17:49:49] The prob was if my computer loses connection, the browser session is lost. on reconnect kernel i don't really see the output. [17:49:54] nice [17:51:01] Oh... weird. Can't you just re-tunnel and open a new browser window with the server? [17:51:08] I know this works with PAWS [17:54:30] then i was using http://millejoh.github.io/emacs-ipython-notebook/ so not sure whose issue was that. Now I switched to Chrome :-) if it happens will try. [17:56:38] Oh! I see. So the jupyter/ipython server was managed through the ssh connection? That does seem fragile-ish. [17:59:51] in ores-staging I had tmux... from tmux instantiated jyputer server... from my local i setup ssh port forwarding... had my emacs connect to forwarded port... [17:59:54] :-D [18:03:40] Now I am connecting to the forwarded port from Chrome. Will see if issue happens again in Chrome. Thanks for PAWS tip. Will try that too. [18:04:16] 06Revision-Scoring-As-A-Service, 10ORES: ORES vision blog post - https://phabricator.wikimedia.org/T141275#2586977 (10Halfak) @MelodyKramer and I talked about communication around #ORES yesterday. Quick background: Jeff and I discussed trying to communicate what ORES is and how Wikipedians use it more clear... [18:04:33] \o/ no problem :) [18:08:47] o/ halfak [18:09:14] o/ Amir1 [18:09:18] One thing. We have enwiki for beta cluster now [18:09:32] only for enwiki in beta cluster. Other wikis use testwiki [18:09:36] we can simply fix it [18:10:27] I also found a bug in ORES review tool in enwiki, I accidentally enabled it for all models [18:10:45] the table is growing really fast, I made the patch and it can be merged and deployed Monday [18:11:14] Amir1, I saw that and I should have asked about it. [18:11:28] Oh and I talked with Jaime and got his okay to do the purge (I was worried it might cause hiccups) [18:11:42] which one halfak :D [18:13:01] Accidentally enabled all models [18:13:21] I'm not sure what I was thinking -- maybe that you were looking forward towards enabling the goodfaith models. [18:17:51] :D [18:18:03] I will enable good faith model soon [18:18:15] Next on my monthly agenda [18:18:22] Cool :) [18:18:28] mostly waiting on UX design [18:28:59] halfak: https://github.com/wikimedia/change-propagation/pull/87 [18:29:26] We're overloaded? [18:30:43] Looks like we had some bursts of requests 3 hours ago [18:30:58] Amir1, do you know what could have caused that? [18:31:41] yeah [18:32:01] I guess it's because of the mistake I did for enwiki [18:32:03] It looks like we were handling about 90 requests per second! [18:32:41] It can handle a lot [18:32:44] fantastic [18:34:17] Hmmm shouldn't this all have been cached? [18:34:51] Hmm... it seems the cache hit rate seemed stable [18:54:26] 06Revision-Scoring-As-A-Service, 10ORES: ORES vision blog post - https://phabricator.wikimedia.org/T141275#2587112 (10DarTar) Thanks for the great summary @halfak. I also wanted to add that given that the prod announcement is mostly relevant to community members and tool builders, we were considering the [[ ht... [19:50:18] 06Revision-Scoring-As-A-Service, 10MediaWiki-API, 10MediaWiki-extensions-ORES, 07Epic: [Epic] Implement ORES service proxy in api.php - https://phabricator.wikimedia.org/T143895#2587270 (10Tgr) If I read the code correctly, [[https://www.mediawiki.org/wiki/Extension:ORES|Extension:ORES]] copies ORES scores... [19:58:17] 06Revision-Scoring-As-A-Service, 10MediaWiki-API, 10MediaWiki-extensions-ORES, 07Epic: [Epic] Implement ORES service proxy in api.php - https://phabricator.wikimedia.org/T143895#2587273 (10Ladsgroup) Hey @Tgr, thanks for the comment >>! In T143895#2587270, @Tgr wrote: > If I read the code correctly, [[http... [20:01:19] Woops. Looks like I accidentally this channel :) [20:40:48] 06Revision-Scoring-As-A-Service, 10MediaWiki-API, 10MediaWiki-extensions-ORES, 07Epic: [Epic] Implement ORES service proxy in api.php - https://phabricator.wikimedia.org/T143895#2587443 (10Tgr) >>! In T143895#2587273, @Ladsgroup wrote: > Not correct, we do it for every revision after the deployment. Well,... [21:03:08] 06Revision-Scoring-As-A-Service, 10MediaWiki-API, 10MediaWiki-extensions-ORES, 07Epic: [Epic] Implement ORES service proxy in api.php - https://phabricator.wikimedia.org/T143895#2587510 (10Anomie) >>! In T143895#2587270, @Tgr wrote: > for watchlist/contribs Note watchlist is just a filtered recentchanges,... [21:03:39] 06Revision-Scoring-As-A-Service, 10revscoring, 07Spike: [Spike] Investigate HashingVectorizer - https://phabricator.wikimedia.org/T128087#2587515 (10Halfak) I got a bit of a start here: http://paws-public.wmflabs.org/paws-public/User:EpochFail/projects/vectors_demo/damage_detection_test.ipynb I'll be implem... [21:49:36] 10Revision-Scoring-As-A-Service-Backlog, 10Edit-Review-Improvements, 03Collab-Team-Q1-July-Sep-2016, 07Design: Design system for signifying ORES Good-Faith and Damaging scores - https://phabricator.wikimedia.org/T143455#2587585 (10Quiddity)