[13:45:36] o/ kevinbazira [13:45:41] how's documentation work going? [13:49:02] I was just looking at the articlequality PR. We should work from a shared branch on the master repo. That way I could push some changes. Right now, the PR is based on your fork. You could create a branch on the wikimedia/articlequality repo directly. [13:52:39] halfak o/ [13:53:11] Alright, I'll create a new branch for the PR [13:56:50] About the documentation work, I looked at the Model info documentation: https://www.mediawiki.org/wiki/ORES/Model_info/Statistics [13:57:36] Observation(Question): The figures being refered to in the PDF documentation are missing in the web page documentation.Is this by choice? [13:59:24] Oh! It's just because we haven't done the work to adapt them. [14:13:33] o/ kevinbazira. Did you see my answer to your question? We could cut out those figures, but I think maybe we could also design more intuitive ones too. [14:18:08] halfak alright, I'll work with the PDF figures and share new figures with you to see whether they're more intuitive [14:18:50] We might need to have the original author upload them to commons since that's the best way to handle copyright. If you choose to create new ones, we can upload those directly. [14:19:05] I could ask him to cut them out for us. [14:20:04] Alright, that will be great [15:07:32] kevinbazira, I left some notes on one of the PRs. Would you be willing to move your work to a branch on the main repo? It's more difficult to collaborate on a branch of your fork. [15:07:45] I can help set up that branch if you want. [15:13:25] Yes, I'll move the work to a branch on the main repo. [15:49:08] * halfak thinks through how to know if we're looking at a function that returns a Dependent or not. [15:49:51] We could stipulate that methods that do not return new Features or Datasources should start with "_" to imply they are protected. [15:50:17] In practice, it looks like all of those methods are static or defined outside of the DependentSet class itself. [19:15:45] https://arxiv.org/abs/1811.10154 [19:15:58] "Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead" by Cynthia RUdin [19:16:17] to appear in Nature Machine Intelligence [19:26:22] Looks like we're having an event. [19:26:36] halfak: want to move our meeting? [19:26:37] I just got a ping for overload in prod. [19:26:42] Not yet [19:26:51] K [19:27:53] Our cache hit rate in CODFW is bad. someone is hitting us hard. [19:28:20] Wait I take that back. [19:28:25] Traffic looks pretty minimal. [19:30:04] CPU usage is rising exponentially... but maybe stabilized in the last 10 minutes. Still below 10%. [19:30:29] oresrdb2001 is pretty heavily active though. [19:30:42] More active than eqiad's oresrdb1001 which is seeing a lot more traffic. [19:32:09] oresrdb2001.codfw.wmnet doesn't accept my public key -- which is pretty lame. [19:33:46] NEvermind. It does look like CODFW is seeing more traffic. [19:34:32] cool [19:34:45] So maybe we're getting hammered. [19:34:59] Now to see if I can figure out if someone his giving us a user-agent or not. [19:35:52] ... and I can't read our logs because I don't have the rights to do so. [19:36:10] akosiaris, any chance you're still up. Looks like ORES is getting hammered in CODFW. [19:37:44] It looks like someone is doing some sort of historical analysis and they are using multiple IPs to get around our rate limiting. [19:44:41] not good [19:45:09] Logstash isn't looking super helpful. It mostly just locks up when I try to query for requests. [19:45:31] But on the bright side, it seems we can process almost 120 requests per second without much of a hiccup. [19:45:37] Going over that is bad. [19:46:28] Looks like the IPs I see in logstash are internal to WMF> [19:49:48] 10ORES, 10Scoring-platform-team (Current): Heavy traffic on ORES CODFW (2019-11-01 @ 1900 UTC) - https://phabricator.wikimedia.org/T237134 (10Halfak) [21:21:10] 10ORES, 10Scoring-platform-team (Current), 10Patch-For-Review: Heavy traffic on ORES CODFW (2019-11-01 @ 1900 UTC) - https://phabricator.wikimedia.org/T237134 (10CDanis) 05Open→03Resolved