[08:29:14] 10Scoring-platform-team, 10DBA, 10MediaWiki-Database, 10Blocked-on-schema-change, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) [09:15:12] 10Scoring-platform-team, 10DBA, 10MediaWiki-Database, 10Blocked-on-schema-change, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) [09:33:36] 10JADE, 10Scoring-platform-team (Current), 10DBA, 10Operations, and 2 others: Introduce a new namespace for collaborative judgments about wiki entities - https://phabricator.wikimedia.org/T200297 (10Marostegui) >>! In T200297#4787352, @awight wrote: > @Marostegui Hello! I've added a few summary columns an... [09:34:23] 10Scoring-platform-team, 10DBA, 10MediaWiki-Database, 10Blocked-on-schema-change, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) s5 eqiad progress [] labsdb1011 [] labsdb1010 [] labsdb1009 [x] dbstore1002 [] db1124 [] db1113 [] db1110 [] db110... [09:34:49] 10Scoring-platform-team, 10DBA, 10MediaWiki-Database, 10Blocked-on-schema-change, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) [13:21:29] 10ORES, 10Scoring-platform-team, 10Operations, 10Release Pipeline (Blubber): Build blubber file for ORES - https://phabricator.wikimedia.org/T210268 (10jijiki) p:05Triage>03Normal [13:22:15] 10ORES, 10Scoring-platform-team, 10Operations, 10Release Pipeline (Blubber), 10Release-Engineering-Team (Backlog): Blubber should be able to make multi docker files per repo - https://phabricator.wikimedia.org/T210267 (10jijiki) p:05Triage>03Normal [13:22:57] 10ORES, 10Scoring-platform-team (Current), 10Operations: Investigate memory usage of ORES in kubernetes - https://phabricator.wikimedia.org/T210264 (10jijiki) p:05Triage>03Normal [13:33:55] 10Scoring-platform-team, 10Icinga, 10Operations: Add ahalfaker to ORES-related icinga contacts - https://phabricator.wikimedia.org/T210742 (10jijiki) p:05Triage>03Normal [13:43:34] 10ORES, 10Scoring-platform-team, 10Operations, 10Puppet, 10Wikimedia-Incident: Logrotate should restart services when more people are around - https://phabricator.wikimedia.org/T210720 (10jijiki) p:05Triage>03Normal @Ladsgroup feel free to mark this as "Resolved" if you feel we don't have other options. [14:38:55] o/ [14:53:12] (03CR) 10Ladsgroup: [V: 032 C: 032] "Deploying to beta." [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/476847 (owner: 10Ladsgroup) [15:02:05] o/ [15:02:11] * halfak jumped right into a meeting [15:04:34] akosiaris: hey, around to deploy a puppet change if possible? [15:05:57] akosiaris: Also I made this patch but I'm not sure if it's correct: https://gerrit.wikimedia.org/r/c/operations/puppet/+/476850 [15:43:09] now ores-sentinel-01 uses ores-puppetmaster-01 as its puppetmaster [15:43:39] now it's time to make use of the sentinel puppet patch [15:43:47] (need to write profile for it first) [15:48:51] Amir1, what's the status with https://github.com/wikimedia/ores/pull/285 (Try to run travis with docker) [15:49:40] * halfak looks for things he can unblock. [15:50:51] I ended up working a half-day on Friday yesterday, so I'm looking to take some time for myself this afternoon. [15:51:32] *Friday afternoon [16:14:25] Amir1: I am now [16:14:41] I did see https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/476850/1/modules/ores/manifests/config.pp as well. I am just not sure yet it's the best way to solve the issue [16:14:44] hey, thanks. [16:17:12] akosiaris: my knowledge about puppet is very novice. I looked up and any changes to the file define here notifies the service and causes it to restart [16:17:55] according to the documentation. That would help in case we push wrong config to puppet, it would restart right away and alarms will go off [16:17:59] yeah but this is code. This is more or less akin to use global variables in another module [16:18:25] look how that define suddenly has dependencies on something that is not internal to it and it has nothing to do with [16:18:43] it would work, don't get me wrong [16:18:47] it's just not very elegant [16:19:09] s/very// :D [16:19:18] hmm, I see. We can make the service subscribe to this the problem is ores::web calls uwsgi and uwsgi calls the service [16:19:46] yeah I started looking at the code a few hours ago. It's a bit of a mess, I 'll give you that [16:19:48] so I need to pass around subscribe in several class [16:20:17] I have already something in the works, lemme see if I can finish it today [16:20:51] cool [16:21:32] akosiaris: I also have this thing I will babysit it: https://gerrit.wikimedia.org/r/c/operations/puppet/+/477289 [16:22:05] we can't change the task_serializer but result serializer is fine [16:25:41] so... wait [16:25:47] what will break if I change this ? [16:25:53] s/change/merge/ [16:26:06] ah wait, nothing [16:26:15] uwsgi will still look into the cache [16:26:47] and the workers will pickup the new results and populate the cache [16:27:11] some times, in case of race condition, it might end up reading from the celery result backend but 1- it's rare 2- I tested it and it worked with json [16:27:32] so what will happen when workers not restarted yet (and hence not having picked up yet the json config change) entounter a json serialized object ? [16:28:02] it will probably error out. [16:28:14] let me find a way to make it smoother. I think I can [16:29:54] akosiaris: We can make it first accept json and then make it json [16:30:00] let me split the patch [16:30:26] ah ok. sounds good [16:35:22] Amir1: celery manages workers per-machine, right? So if you restart both services one machine at a time, we should be safe? [16:40:12] awight: but the task results are stored in redis centrally [16:40:39] Ah yeah, sorry I hadn't seen this patch yet. [16:40:53] Was imagining it was one of the in-flight Celery tweaks. [16:41:45] +1 the plan to soft migrate if you feel like doing that work--otherwise, can we just purge Redis and have a few minutes of down time? [16:42:11] Remember that cache only helps with a small percentage of our total volume. [16:42:41] So the hit from purging cache is just a few moments of high latency for the most recent changes. [16:42:53] Amir1: ^ [16:43:40] awight: nah, let's just let it accept both pickle and json [16:43:49] and tomorrow we change it to json [16:43:54] that should be okay [16:43:59] Up to you of course. [16:44:27] I guess you'll just try to parse as json, and if it errors out, fall back to pickle [16:45:16] celery does it automatically when you have it in accept content config [16:45:21] awight: see https://gerrit.wikimedia.org/r/477289 [16:46:25] akosiaris: so the plan is to deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/477289/2/modules/ores/manifests/web.pp, wait for two hours and then deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/477302/1 [16:46:40] it will still accept pickle [16:47:12] Thanks [16:47:23] yeah I am already deploying https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/477289/ [16:47:31] \o/ [16:47:56] let's not do the 2 hours btw, let's do the followup tomorrow EU morning [16:48:06] I 'll make sure to restart all uwsgi+celery after I merge this [16:48:35] thanks [16:48:56] akosiaris: one last thing: I have a WIP for redis sentinel https://gerrit.wikimedia.org/r/c/operations/puppet/+/476891 [16:49:33] I'm trying to see if it actually works by using it in a standalone puppetmaster in labs [16:49:49] does the whole approach makes sense to you? [16:50:09] I 'll review but I have a meeting in 10 [16:50:57] hmm quite a bit of ERB you got there [16:51:16] yeah :D [16:52:01] logs on ores1001 seem fine [16:52:24] I think the change did not cause any undesired consequences [16:52:49] ah, wait, pebkac. testing it again [16:53:18] (sorry, got disconnected) [16:53:20] > celery does it automatically when you have it in accept content config [16:53:28] ^ awesome! [16:53:44] This Celery 4 thing is one of the few times an upgrade seems to pay off :-) [16:55:48] [2018-12-03T16:55:07] /srv/deployment/ores/deploy/venv/lib/python3.5/site-packages/sklearn/base.py:311: UserWarning: Trying to unpickle estimator RobustScaler from version 0.19.1 when using version 0.19.2. This might lead to breaking code or invalid results. Use at your own risk. [16:55:48] UserWarning) [16:55:54] is this worrying ? [16:55:58] or should I ignore ? [16:56:36] akosiaris: that's unrelated but I think the whole thing is worrying [16:56:45] When did we update that wheel... [16:56:51] We must have a version mismatch somewhere. [16:56:57] It's sklearn [16:58:19] The wheel was updated Oct 31 [16:58:40] Should have been deployed Nov 7 [16:58:46] I think I did it as part of celery4 [16:58:56] Interesting. We certainly didn't rebuild our models since then [16:58:58] So, I think this is just our failure to rebuild models [16:59:02] Usually we pin sklearn at one version. [16:59:03] yup [16:59:12] Because that's the one where version matters the most. [16:59:23] We should never update sklearn unintentionally. [16:59:27] But, akosiaris: the bottom line is that today's deployment didn't cause this warning. [16:59:33] Right. [17:00:19] Revscoring should probably have that version pinned. [17:00:25] +1 [17:00:33] anyway, want me to stall rolling out the json serializer change ? [17:01:18] Amir1: ^ In my uneducated opinion, seems like we should continue? [17:01:47] * akosiaris meeting [17:02:24] akosiaris: we should continue, it's unrelated [17:02:42] meeting! [17:02:44] Amir1, ^ [17:02:55] on my way [17:03:06] finding a room at WMDE is getting harder and harder [17:03:10] ok [17:14:25] 10Scoring-platform-team, 10Growth-Team: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Halfak) [17:15:10] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train/test damaging & goodfaith models for Italian Wikipedia (itwiki) - https://phabricator.wikimedia.org/T208779 (10Halfak) [17:15:13] 10Scoring-platform-team, 10Growth-Team: Enable ORES filters on RC for Italian Wikipedia - https://phabricator.wikimedia.org/T211032 (10Halfak) [17:56:03] I'll have a quick patch up for the mwparserfromhell issue. [17:56:09] oh F*@7, this cafe is playing "sandy claws is coming to town". [17:56:15] * awight jams earbuds deep into skull [17:56:28] For clarity, this does not solve the one parser issue that caused the problem. It makes mwparserfromhell *interruptable* [17:57:09] Nice, so it'll also be a test of the Celery heartbeat feature perhaps. [17:58:48] Sandy Claws? That's a new twist [17:59:05] (03PS1) 10Halfak: Bumps mwparserfromhell to 0.5.2 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/477310 [17:59:47] 10ORES, 10Scoring-platform-team (Current): ORES workers using dramatically higher CPU, increasing linearly with time - https://phabricator.wikimedia.org/T206654 (10Halfak) https://gerrit.wikimedia.org/r/#/c/research/ores/wheels/+/477310 [17:59:56] (03PS2) 10Halfak: Bumps mwparserfromhell to 0.5.2 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/477310 (https://phabricator.wikimedia.org/T206654) [18:00:33] awight, https://gerrit.wikimedia.org/r/#/c/research/ores/wheels/+/477310/ this one should be easy :) [18:01:47] 10ORES, 10Scoring-platform-team (Current), 10Patch-For-Review: ORES workers using dramatically higher CPU, increasing linearly with time - https://phabricator.wikimedia.org/T206654 (10Halfak) a:05awight>03Halfak [18:06:00] harej: Well it's the white bread song, but the "Sandy Claws" concept is stolen from "the nightmare before christmas". Worth watching if you haven't. [18:06:36] * awight cheerfully whistles "frosty tha G-D-- snowman" [18:06:59] (03CR) 10Awight: [C: 032] Bumps mwparserfromhell to 0.5.2 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/477310 (https://phabricator.wikimedia.org/T206654) (owner: 10Halfak) [18:07:06] halfak: Thanks! ^ [18:07:46] No thank you :) [18:11:43] For the +2? Thank Mr. Sippy Bird: https://www.youtube.com/watch?v=8OkKhkJiJyo [18:11:45] 10JADE: In a multi-part judgment, can one of the parts be null? - https://phabricator.wikimedia.org/T209524 (10Harej) As discussed in last week's Scoring Staff meeting, the issue is that semantically, "null" doesn't mean "I don't know" – it means "there is no data." So we need a more explicit "I don't know" indi... [18:11:55] lol [18:12:13] 10JADE: In a multi-part judgment, can one of the parts be null? - https://phabricator.wikimedia.org/T209524 (10Harej) Related Etherpad: https://etherpad.wikimedia.org/p/Jade_summary_data_use_cases [18:21:52] 10ORES, 10Scoring-platform-team (Current): drafttopic score JSONSchema should specify array of strings for prediction type - https://phabricator.wikimedia.org/T205343 (10Halfak) https://github.com/wikimedia/revscoring/pull/417 Got both issues here. We'll need to rebuild all of the models. That's something w... [18:22:11] halfak: creepy--Jenkins isn't merging that patch yet. [18:22:52] (03CR) 10Awight: "kicking Jenkins" [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/477310 (https://phabricator.wikimedia.org/T206654) (owner: 10Halfak) [18:22:55] (03CR) 10Awight: [C: 032] Bumps mwparserfromhell to 0.5.2 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/477310 (https://phabricator.wikimedia.org/T206654) (owner: 10Halfak) [18:23:57] harej: Thanks for the board maintenance! [18:24:21] Kanban board grooming is my favorite work activity that feels the least like work [18:25:20] haha i know what you mean. [18:26:02] Amir1: Do you know anything about Zuul being broken today? Or know whether the wheels repo is set to not auto-submit after V+2 CR+2? [18:26:33] awight: wheels won't get merged automatically, I wanted to fix it for a very long time [18:26:45] I hate when deployment repos are like that but there's no obvious indication in Gerrit... and if we force-submit incorrectly, it kills Gearman >.< [18:26:53] Amir1: thanks for confirmation! [18:28:12] OK I'm going AFK for a bit. I'll be available via gchat and telegram. [18:28:21] Amir1: How do you feel about the ores-prod-deploy repo? Sometimes not auto-merging is nice at the top level, but you're deploying more often than any of us so IMO your opinion is the most important. [18:28:25] o/ [18:28:52] 10ORES, 10Scoring-platform-team, 10Continuous-Integration-Config: ORES wheels deployment repo Jenkins job should gate-and-submit - https://phabricator.wikimedia.org/T211041 (10awight) [18:29:01] awight: the wmflabs repo is on github so no jenkins there :D [18:29:27] sure, I was thinking about the top-level production repo though [18:30:17] for prod it doesn't matter much, we don't have any tests on it [18:30:26] if we have tests, that would be good [18:30:28] In my past experience, it's been slightly sane to have fine-grained control to merge the top level only when I'm actually deploying, but the CR+2 is safe at a random time. [18:30:33] oh hmm [18:31:30] halAFK: dang, I did have a pretty major question for you. It can wait, but this is the heart of it: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/476994/4/jsonschema/judgment/v1.json [18:32:10] Amir1: how's your experience been when articlequality is stored as an integer vs. stored as a string? [18:32:33] I'm thinking that Jade should map to an integer for storage, but wanted to use ORES as a precedent. [18:32:39] integer is better for db reasons [18:32:44] +1 [18:32:51] It seems nicer for querying also. [18:33:51] Both for DB reasons and because the client just has to know "better than {'good article' -> 2}" turns into "where x <= 2' [18:40:47] (03CR) 10Legoktm: [C: 04-1] "I don't think catching DBErrors is something that makes sense...what errors are you expecting? e.g. if getConnection() fails, then MW has " [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476446 (owner: 10Awight) [18:47:23] 10ORES, 10Scoring-platform-team: Support CIDR range whitelist for ORES throttling - https://phabricator.wikimedia.org/T210103 (10awight) Okay, thanks for the feedback. I like your suggestion and will try to extract all IP calculations into a third class, ParallelHostLimiter or something of the sort. [18:48:04] 10ORES, 10Scoring-platform-team (Current): Support CIDR range whitelist for ORES throttling - https://phabricator.wikimedia.org/T210103 (10awight) a:03awight [18:48:40] 10JADE, 10Scoring-platform-team: JADE literature review - https://phabricator.wikimedia.org/T201887 (10awight) [18:49:20] 10JADE, 10Scoring-platform-team (Current), 10Documentation, 10Epic: [Epic] Write user-facing documentation for JADE - https://phabricator.wikimedia.org/T199069 (10awight) [18:50:56] going to be afk because I'll be eating a nice doner :D [18:51:28] uh, stomachache just mentioning that word lol [18:57:23] (03CR) 10Awight: "> I don't think catching DBErrors is something that makes" [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476446 (owner: 10Awight) [18:58:29] Amir1: yum--donors make the most generous kebab. [19:30:49] back now :D [19:43:38] 10JADE, 10Scoring-platform-team, 10Gerrit: Rename "JADE" extension to "Jade" - https://phabricator.wikimedia.org/T211046 (10awight) [19:44:13] 10JADE, 10Scoring-platform-team, 10Gerrit: Rename "JADE" extension to "Jade" - https://phabricator.wikimedia.org/T211046 (10awight) [19:44:18] 10JADE, 10Scoring-platform-team (Current), 10Operations, 10TechCom, and 3 others: Deploy JADE extension to production - https://phabricator.wikimedia.org/T183381 (10awight) [19:45:22] (03CR) 10Awight: "recheck" [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476771 (owner: 10Awight) [19:45:42] 10JADE, 10Scoring-platform-team, 10Gerrit: Rename "JADE" extension to "Jade" - https://phabricator.wikimedia.org/T211046 (10awight) https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/476771/ [19:46:00] +1 for integer [19:46:07] Oh wait. [19:46:14] In the content? I think a string is better [19:46:19] hmm [19:46:23] In the DB, an integer mapping is better [19:46:33] I feel less strong about the content-string, than the db-integer [19:46:34] Why is that? To make it more human-readable? [19:46:44] Content string is because storage is mega-cheap. [19:46:51] Compared to DB fields. [19:46:59] And yeah, human readability. [19:47:03] Ah well consider the case where we need to rename a quality class... [19:47:17] I don't see that happening. [19:47:28] Can you imagine if Wikipedians wanted to rename a quality class? [19:47:39] Also, consider that the code in the content will be either in the "native" language or will be a machine-readable code i.e. English. [19:47:52] Oh yes. That is a good point. [19:48:02] * awight digs in patch [19:48:05] In wikilabels, we used the english name and had i18n strings. [19:48:38] Interesting. [19:48:46] Check this out, https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/476994/4/i18n/en.json [19:48:54] No matter what, there will be english in the schema. [19:49:12] after changing JADE to jade, should we change ORES to Ores? [19:49:19] One quick note, the numbers are reversed. [19:49:29] We usually start low for stub and high for featured article. [19:49:34] Makes math make more sense. [19:49:47] Amir1, no way dude. I like ORES. :D [19:49:59] halAFK: okay, I was thinking of it as a ranking, but happy to do it the other way around! [19:50:16] :D [19:50:32] awight, this scale won't work for some wikis. [19:50:43] halAFK: So an issue I couldn't get around without using numbers like this, is that a custom scale like wikidata's "E" will require its own i18n string [19:50:53] ^ that will be horrible for translators, IMO [19:51:23] well, "E" is not the best example because it's on an explicitly multilingual wiki [19:51:47] Yeah, this almost-the-same scale is complicated. [19:52:00] halAFK: What's wrong with the numeric scale? Or are you referring to the almost-the-same customization? [19:52:08] Hard to benefit from the partial sameness [19:52:21] For that, I'm imagining that the local wiki can have their own message strings to override the defaults, but that's bad for i18n as well. [19:53:30] awight, I'm starting to like the idea of numbering. [19:53:39] Another thought about how to handle partial sameness is that we have a scale identifier, like "ruwiki_wp10" (really bad example), and then we calculate the message keys like "jade-contentquality-scale-ruwiki_wp10-1-label" [19:53:39] This could work for *any* ordinal. [19:53:48] In fact, it could work for booleans too. [19:54:01] (03CR) 10jerkins-bot: [V: 04-1] Throw a fit and s/JADE/Jade/ [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476771 (owner: 10Awight) [19:54:09] ^ both of that. actually, I don't like how it generalizes to non-ordinals, like the draftquality enum. [19:54:33] Hm. That's a good point. Is there a non-ordinal way we could store draftquality stuff? [19:54:44] Maybe a bitfield. [19:54:49] Oh wait. no. [19:55:00] We could do drafttopic as a bitfield. [19:55:02] lol [19:55:30] FYI there's no custom logic for contentquality vs damaging, so we're free to still use code strings: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/476994/4/includes/JudgmentPageWikitextRenderer.php [19:55:45] Especially if the scale itself will not change across wikis... [19:56:10] If the scale might vary, then we should take a serious look at my awful "scale identifier" alternative. [19:56:31] halAFK: https://meta.wikimedia.org/wiki/Grants:Project/University_of_Virginia/Machine_learning_to_predict_Wikimedia_user_blocks [19:56:42] I'm OK with numbers in the end. It'll be weight because ORES and JADE will output different values. But in theory, if we have a mapping, we can make it work. [19:57:10] harej, this is relevant to notconfusing's work. [19:57:51] halAFK: alright, thanks for the help. I'll flip the order. [19:58:04] *weird. [19:58:16] Cool, awight. Glad I checked back in :) [19:58:41] harej: That's terrifying. Unless we have a separate model for each admin's likelihood of blocking a user :p [19:58:43] harej, Looks like this might have some interesting outcomes. The budget is too small for sure. [19:58:58] awight, I think we also need to model different block reasons. [19:59:11] +1 good point [19:59:27] For the same reason as damaging vs. goodfaith... [19:59:35] Could rescope this to a data exploration of blocks and that would be an interesting research project. [20:00:03] OK I'm going to head out. Thanks for the notice about this harej. I'll make some time to comment. [20:00:15] * halAFK --> really AFK [20:00:47] (03CR) 10Awight: "Note from IRC: the order should be flipped to better match how we do this in ORES." [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476994 (owner: 10Awight) [20:01:25] I can't get on wifi here, so gonna relocate. Back in a few. [20:44:41] 10ORES, 10Scoring-platform-team (Current): Support CIDR range whitelist for ORES throttling - https://phabricator.wikimedia.org/T210103 (10awight) I'm backing away from the scope of this—going to implement the feature for this task in the quick way, and save refactoring for a rainier day. [21:22:57] Hey a small victory: I accidentally pushed to git@github.com:wiki-ai/ores.git, and the push was magically submitted to our new org wikimedia/ores.git. [21:23:46] Amir1: https://github.com/wikimedia/ores/pull/298, meanwhile I'm about to smoke test it. [21:26:57] gtg for a minute! [22:04:22] wikimedia/revscoring#1558 (schema_cleanup - 96a9e58 : halfak): The build passed. https://travis-ci.org/wikimedia/revscoring/builds/462939390 [22:27:41] Internet in the U.S. has been worse than in Peru for me lately. [22:28:06] awight: what happened? Weren't you in NL? [22:28:09] Second nominally wifi-enabled place to not actually connect me to the 'tubes. [22:28:59] Hauskatze: I also lived in a rural corner of the Andes for a few months last year. [22:29:08] ah, I thought it was because of the termination of the net neutrality there? [22:29:19] I take things less for granted now [22:29:32] Oho! That's very possible, thanks for pointing out the link. [22:29:35] a law was recently passed here to protect net neutrality [22:29:49] It would also make me feel special for having used "too much" Internet for my SLA [22:29:51] though it is already protected in the EU-wide so a bit redundant :) [22:37:28] wikimedia/ores#1178 (cidr_whitelist - b136d23 : Adam Wight): The build passed. https://travis-ci.org/wikimedia/ores/builds/463026149 [22:44:24] 10ORES, 10Scoring-platform-team, 10Operations, 10Puppet, 10Wikimedia-Incident: Logrotate should restart services when more people are around - https://phabricator.wikimedia.org/T210720 (10Ladsgroup) >>! In T210720#4784938, @akosiaris wrote: > So we should make a better job of surfacing and fixing the iss... [22:47:58] (03PS5) 10Awight: Store content quality as an integer index [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476994 [22:52:42] (03CR) 10jerkins-bot: [V: 04-1] Store content quality as an integer index [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476994 (owner: 10Awight) [22:54:05] (03CR) 10jerkins-bot: [V: 04-1] Store content quality as an integer index [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476994 (owner: 10Awight) [23:01:01] (03PS6) 10Awight: Store content quality as an integer index [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476994 [23:01:50] I'm done for the day [23:02:05] see you on Wednesday [23:04:02] I forget, do we amend on GitHub... [23:04:13] Amir1: o/ have a nice Wikidata day [23:12:31] (03PS14) 10Awight: Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) [23:12:33] (03PS11) 10Awight: Add indexes to filter by judgment value [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476447 (https://phabricator.wikimedia.org/T200297) [23:12:35] (03PS7) 10Awight: Throw a fit and s/JADE/Jade/ [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476771 (https://phabricator.wikimedia.org/T211046) [23:16:05] (03PS7) 10Awight: Store content quality as an integer index [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476994 [23:16:08] (03PS15) 10Awight: Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) [23:16:09] (03PS12) 10Awight: Add indexes to filter by judgment value [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476447 (https://phabricator.wikimedia.org/T200297) [23:16:12] (03PS8) 10Awight: Throw a fit and s/JADE/Jade/ [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476771 (https://phabricator.wikimedia.org/T211046) [23:41:08] 10ORES, 10Scoring-platform-team, 10Analytics, 10Dumps-Generation, and 3 others: Decide whether we will include raw features - https://phabricator.wikimedia.org/T211069 (10awight) [23:42:04] harej: ^ this might be fun although maybe outside the scope of what your PM focii [23:42:09] s/what/ [23:42:21] brain cold from this outdoor seating [23:43:50] 10ORES, 10Scoring-platform-team, 10Analytics, 10Dumps-Generation, and 3 others: Decide whether we will include raw features - https://phabricator.wikimedia.org/T211069 (10awight) [23:46:28] I'm starting to realize that this could be a huge efficiency boost. Currently, it takes 24 hours of medium-big machine time to rebuild all our models, but we might be able to reuse the most expensive processing stage most of the time. [23:46:42] Space-X-esquely [23:47:06] awight: what might be fun? keep in mind that i have the spam bots ignored in my IRC client [23:47:16] right, ty [23:47:19] 23:43 < wikibugs> ORES, Scoring-platform-team, Analytics, Dumps-Generation, and 3 others: Decide whether we will [23:47:22] include raw features - https://phabricator.wikimedia.org/T211069 (awight) [23:47:34] actually--are you highlighting on the name of the bot? [23:47:42] can you see me? (wikibugs) [23:48:27] I can see it when you copy and paste lines of chat from them; I don't see the original lines [23:48:46] harej: I'm pushing on the epic T209611, and what started looking fun was T211069 [23:48:46] T211069: Decide whether we will include raw features - https://phabricator.wikimedia.org/T211069 [23:48:47] T209611: [Epic] Make ORES scores available in Hadoop and as a dump - https://phabricator.wikimedia.org/T209611 [23:54:33] too cold, relocating. [23:58:07] mebbe this heat lamp will help