[00:02:50] I wrote something most likely off-base [00:04:37] * harej searches for the right place in the document to put it [00:08:12] I added: "Surfacing activity in watchlists: will people think to use filters and highlighters?" [00:08:21] I feel like at some point we should all meet together to figure out if we're on the same page. [00:12:08] awight: I'm concerned about your exchange with Daniel Kinzler on Phabricator. I am not sure that the proposed change is worth yet another round of DBA reviews. [00:12:49] harej: Thanks for looking! [00:13:03] I think it's going to need DBA review no matter what... [00:13:42] At the IRC meeting I was asked to add the judgment summary columns, which is the bulk of the work and includes a schema change that has to be reviewed. [00:14:38] the indexes for filtering are just two lines of DDL and no code, FYI [00:16:33] So after the RFC meeting, exactly how much delay has been introduced? [00:19:32] Um [00:20:58] I need to spend a week coding and dealing with Phabricator, and it'll take a minute (0-4 weeks?) to get DBA review. But I'm not comfortable with saying that the RFC meeting introduced delay, the point of the thing was to get feedback that could alter our course. [00:21:15] So it introduced delay like any design process does... [00:22:13] Right. I'm concerned about the delay itself more than whose fault is it [00:23:26] I think we're just icing the cake here [00:23:50] That's good to hear! [00:24:08] Ah, I love this winter thing where it's dark out at 4:23 in the afternoon. AND it's raining. Best time of the year! [00:25:24] reopening the RFC sounds a bit dramatic, I know. IMO, dropping my notes into that task was all the RFC-related paperwork I should be doing. [00:25:52] I'm going to write this patch, wait for a nod from marostegui and we're good to go. [00:26:38] What I didn't like and what that exchange is about at heart, is that I'm being asked to do something which would clearly be incomplete, when everyone involved so far agrees that the complete version is what we need to do. [00:27:56] Ideally the two of us decide which use cases to support and I just code from there. If more review is required, that's life... [00:28:48] What do you think of my proposal to implement both the summary columns and the indexes, in two patches so we can split hairs later? [00:28:57] harej: ^ [00:29:24] Implementing it as two patches seems reasonable. [00:29:45] I take it you won't need me to the bad guy and kill the feature, since it won't make a difference in terms of our timeline? [00:30:35] hehe [00:31:19] My preference is * don't include the shit at all, no summary columns, or * provide the summary and index it [00:31:43] The only thing I don't want to do is write half the feature doing 95% of the work, then regret it immediately... [00:32:01] Really I'd like to just deploy the blasted thing and run an ALTER [00:32:07] That is my heart speaking ;-) [00:32:27] But I'm okay with crossing more "t"s [00:33:47] Note that dues said, > I do think people will ask for filtering immediately. [00:34:21] It seems like augmentation itself might not be a huge use case, however once editors see the augmentation, of course they'll want to filter for it. [00:34:45] To clarify, this is different from filtering on the mere existence of a Jade page, right? [00:34:59] But rather, filtering based on the Jade page's *contents* [00:35:12] yes [00:35:26] just "damaging" and "goodfaith" [00:36:23] So, adding this new feature has a cost. What benefit do we get from it in return? Who is yearning for this feature? [00:36:38] I suggest that we don't overthink "filtering" though, and just refer to this feature as "summary columns for judgment data". [00:36:42] Which feature are you referring to? [00:36:52] Being able to filter based on a judgment page's contents [00:37:13] A call might be easier? [00:37:23] Let's do that [00:37:38] Feel free to send the invite [01:08:31] harej: Line 3, "for where"...? [01:10:46] ty [01:12:35] "Jade as an override" is solid btw, that's the original guiding spirit of this whole thing. [01:13:00] And it clearly requires both augmentation and filtering... [01:14:03] I was struck with inspiration :) [01:15:14] gtg o/ [05:18:25] o/ [05:18:42] Don't ask why I'm so early [06:00:36] 10ORES, 10Scoring-platform-team: Tuning broken in some repos, needs revscoring 2 update - https://phabricator.wikimedia.org/T184727 (10Ladsgroup) >>! In T184727#4453085, @Ladsgroup wrote: > It seems done to me. Anything needed here @awight? ping :D [06:03:16] 10Scoring-platform-team, 10Wikilabels: Labeling campaign for dewiki shows 'revision not found' for all revisions - https://phabricator.wikimedia.org/T194457 (10Ladsgroup) >>! In T194457#4319048, @Cirdan wrote: > Without using a private window, it works fine. Previously, I have always used a private window, hen... [06:03:50] 10Scoring-platform-team, 10Wikilabels: Can't label edits in private browsing - https://phabricator.wikimedia.org/T194457 (10Ladsgroup) p:05Triage>03Lowest [06:04:34] 10ORES, 10Scoring-platform-team, 10Performance: Profile ORES code memory use, optimize pre-fork allocation - https://phabricator.wikimedia.org/T182350 (10Ladsgroup) p:05Triage>03Normal [06:05:52] 10Scoring-platform-team: Some revscoring data utilities should report warnings at the end, rather than with console loglines - https://phabricator.wikimedia.org/T190747 (10Ladsgroup) p:05Triage>03Low [06:07:17] 10Scoring-platform-team, 10DBA, 10MediaWiki-Database, 10Blocked-on-schema-change, and 2 others: Schema change for rc_this_oldid index - https://phabricator.wikimedia.org/T202167 (10Marostegui) [06:11:11] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: ores-beta lacking a firewall rule to allow port 8081 - https://phabricator.wikimedia.org/T194603 (10Ladsgroup) a:03Ladsgroup [06:12:15] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: ores-beta lacking a firewall rule to allow port 8081 - https://phabricator.wikimedia.org/T194603 (10Ladsgroup) 05Open>03Resolved I just added web security group to the node in horizon which opens the ports from 8000 - 8100 [06:13:00] 10Scoring-platform-team, 10drafttopic-modeling: Refactor scripts fetching text and other metadata - https://phabricator.wikimedia.org/T181074 (10Ladsgroup) p:05Triage>03Low [06:14:30] 10ORES, 10Scoring-platform-team, 10revscoring, 10artificial-intelligence: Add a parameter to cv_train to output model information - https://phabricator.wikimedia.org/T195603 (10Ladsgroup) Isn't this already done? [06:14:50] 10ORES, 10Scoring-platform-team, 10Services (designing): Merge ORES precaching with ORESFetchScoreJob - https://phabricator.wikimedia.org/T201868 (10Ladsgroup) p:05Triage>03Normal [06:24:36] 10Scoring-platform-team, 10Cleanup, 10Gerrit, 10Repository-Admins, 10Release-Engineering-Team (Watching / External): Investigate and clean up deprecated ORES repos - https://phabricator.wikimedia.org/T179610 (10Ladsgroup) p:05Triage>03Low [06:26:33] 10ORES, 10Scoring-platform-team: Look at overhead of json codec and data passing to feature extraction workers - https://phabricator.wikimedia.org/T181809 (10Ladsgroup) p:05Triage>03Low [06:30:22] 10ORES, 10Scoring-platform-team, 10Graphite: ORES timeout error graph is incorrect - https://phabricator.wikimedia.org/T182876 (10Ladsgroup) p:05Triage>03Normal [06:32:27] 10MediaWiki-extensions-ORES, 10Scoring-platform-team, 10Performance: Clean up ORES thresholds cache: pre-emptively check before expiry - https://phabricator.wikimedia.org/T182256 (10Ladsgroup) p:05Triage>03Normal [06:33:30] 10ORES, 10Scoring-platform-team, 10Analytics, 10Dumps-Generation, and 3 others: [Epic] Make ORES scores available in Hadoop and as a dump - https://phabricator.wikimedia.org/T209611 (10Ladsgroup) p:05Triage>03High [06:33:46] 10ORES, 10Scoring-platform-team, 10Analytics, 10Dumps-Generation, and 3 others: [Epic] Make ORES scores available in Hadoop and as a dump - https://phabricator.wikimedia.org/T209611 (10Ladsgroup) p:05High>03Triage [06:34:09] 10ORES, 10Scoring-platform-team, 10Analytics, 10Dumps-Generation, and 3 others: [Epic] Make ORES scores available in Hadoop and as a dump - https://phabricator.wikimedia.org/T209611 (10Ladsgroup) p:05Triage>03High [06:35:22] 10ORES, 10Scoring-platform-team, 10Analytics: Include feature values in ORES changeprop stream - https://phabricator.wikimedia.org/T209734 (10Ladsgroup) p:05Triage>03Normal [06:35:29] 10ORES, 10Scoring-platform-team, 10Analytics: Backfill ORES Hadoop scores with historical data - https://phabricator.wikimedia.org/T209737 (10Ladsgroup) p:05Triage>03Normal [06:35:37] 10ORES, 10Scoring-platform-team, 10Analytics, 10Dumps-Generation: Produce dump files for ORES scores - https://phabricator.wikimedia.org/T209739 (10Ladsgroup) p:05Triage>03High [06:35:45] 10ORES, 10Scoring-platform-team, 10Analytics: Purge ORES scores from Hadoop and begin backfill when model version changes - https://phabricator.wikimedia.org/T209742 (10Ladsgroup) p:05Triage>03Normal [06:36:16] 10ORES, 10Scoring-platform-team, 10Analytics, 10Dumps-Generation: Produce dump files for ORES scores - https://phabricator.wikimedia.org/T209739 (10Ladsgroup) p:05High>03Triage [06:36:20] 10ORES, 10Scoring-platform-team, 10Analytics: Include feature values in ORES changeprop stream - https://phabricator.wikimedia.org/T209734 (10Ladsgroup) p:05Normal>03Triage [06:36:37] 10ORES, 10Scoring-platform-team, 10Analytics: Backfill ORES Hadoop scores with historical data - https://phabricator.wikimedia.org/T209737 (10Ladsgroup) p:05Normal>03Triage [06:36:43] 10ORES, 10Scoring-platform-team, 10Analytics: Purge ORES scores from Hadoop and begin backfill when model version changes - https://phabricator.wikimedia.org/T209742 (10Ladsgroup) p:05Normal>03Triage [06:36:58] 10ORES, 10Scoring-platform-team, 10Discovery, 10Wikidata, and 2 others: Include ORES predictions in RDF export - https://phabricator.wikimedia.org/T200716 (10Ladsgroup) p:05Normal>03Triage [06:37:04] 10JADE, 10Scoring-platform-team, 10Advanced-Search, 10Discovery-Search, and 3 others: Advanced field indexing to support search. - https://phabricator.wikimedia.org/T206352 (10Ladsgroup) p:05Low>03Triage [06:37:07] 10JADE, 10Scoring-platform-team: Document group polarization as a risk to JADE - https://phabricator.wikimedia.org/T208349 (10Ladsgroup) p:05Low>03Triage [06:38:10] 10MediaWiki-extensions-ORES, 10Scoring-platform-team, 10Growth-Team, 10MediaWiki-extensions-PageCuration: ORES "is model really enabled?" PHP API - https://phabricator.wikimedia.org/T205323 (10Ladsgroup) p:05Triage>03Normal [06:38:26] 10MediaWiki-extensions-ORES, 10Scoring-platform-team, 10Growth-Team, 10MediaWiki-extensions-PageCuration: ORES "is model really enabled?" PHP API - https://phabricator.wikimedia.org/T205323 (10Ladsgroup) p:05Normal>03Triage [06:38:43] 10Scoring-platform-team, 10editquality-modeling, 10artificial-intelligence: Simplify and modularize the Makefile template - https://phabricator.wikimedia.org/T190968 (10Ladsgroup) p:05Triage>03Normal [06:38:50] 10Scoring-platform-team, 10editquality-modeling, 10artificial-intelligence: Simplify and modularize the Makefile template - https://phabricator.wikimedia.org/T190968 (10Ladsgroup) p:05Normal>03Triage [06:39:08] 10Scoring-platform-team, 10User-Ladsgroup: Audit deployed editquality models and figure out why if the models are bad - https://phabricator.wikimedia.org/T194742 (10Ladsgroup) p:05Triage>03Low [06:39:23] 10Scoring-platform-team, 10User-Ladsgroup: Audit deployed editquality models and figure out why if the models are bad - https://phabricator.wikimedia.org/T194742 (10Ladsgroup) p:05Low>03Triage [06:39:30] 10JADE, 10Scoring-platform-team, 10Design: Find way to re-surface judgments for continuous evaluation - https://phabricator.wikimedia.org/T204064 (10Ladsgroup) p:05Low>03Triage [06:39:42] 10ORES, 10Scoring-platform-team, 10Operations, 10Release Pipeline (Blubber): Build blubber file for ORES - https://phabricator.wikimedia.org/T210268 (10Ladsgroup) p:05Low>03Triage [06:39:45] 10ORES, 10Scoring-platform-team: User-facing tool to explain scoring algorithms - https://phabricator.wikimedia.org/T204560 (10Ladsgroup) p:05Low>03Triage [06:40:05] 10ORES, 10Scoring-platform-team, 10User-Ladsgroup: Rewrite ORES "reference" UI using React - https://phabricator.wikimedia.org/T195274 (10Ladsgroup) p:05Triage>03Normal [06:40:34] 10ORES, 10Scoring-platform-team, 10User-Ladsgroup: Rewrite ORES "reference" UI using React - https://phabricator.wikimedia.org/T195274 (10Ladsgroup) p:05Normal>03Triage [06:40:38] 10ORES, 10Scoring-platform-team, 10Upstream: Make ORES dependency solving upstreamable - https://phabricator.wikimedia.org/T201657 (10Ladsgroup) p:05Triage>03Low [06:41:23] 10ORES, 10Scoring-platform-team, 10Upstream: Make ORES dependency solving upstreamable - https://phabricator.wikimedia.org/T201657 (10Ladsgroup) p:05Low>03Triage [06:41:28] 10ORES, 10Scoring-platform-team: Read timeout from enwiki when requesting non-existent revision - https://phabricator.wikimedia.org/T204984 (10Ladsgroup) p:05Triage>03High [06:41:30] 10ORES, 10Scoring-platform-team: Read timeout from enwiki when requesting non-existent revision - https://phabricator.wikimedia.org/T204984 (10Ladsgroup) p:05High>03Triage [06:41:50] 10ORES, 10Scoring-platform-team: ORES command line service sometimes hangs - https://phabricator.wikimedia.org/T205909 (10Ladsgroup) p:05Triage>03High [06:41:52] 10ORES, 10Scoring-platform-team: ORES command line service sometimes hangs - https://phabricator.wikimedia.org/T205909 (10Ladsgroup) p:05High>03Triage [06:42:22] 10MediaWiki-extensions-ORES, 10Scoring-platform-team, 10Growth-Team: RC filters: Un-scored Bot edits in Namespaces displayed with ORES full-coverage filters - https://phabricator.wikimedia.org/T206271 (10Ladsgroup) [06:42:57] 10Scoring-platform-team, 10Wikilabels: Wikilabels needs manual reboot when DB connection is broken - https://phabricator.wikimedia.org/T209604 (10Ladsgroup) p:05Triage>03Normal [06:43:02] 10Scoring-platform-team, 10Wikilabels: Wikilabels needs manual reboot when DB connection is broken - https://phabricator.wikimedia.org/T209604 (10Ladsgroup) p:05Normal>03Triage [07:00:54] 10ORES, 10Scoring-platform-team: Clean up 500 errors in ORES service - https://phabricator.wikimedia.org/T210575 (10Ladsgroup) [07:01:56] 10ORES, 10Scoring-platform-team: Clean up 500 errors in ORES service - https://phabricator.wikimedia.org/T210575 (10Ladsgroup) p:05Triage>03Normal [08:00:41] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Build a test setup for redis sentinel in cloud VPS - https://phabricator.wikimedia.org/T210577 (10Ladsgroup) p:05Triage>03High [08:15:51] 10Scoring-platform-team, 10Wikibase-Containers, 10Wikidata, 10Wikilabels, 10Release Pipeline (Blubber): Stretch in docker registry forces ascii encoding - https://phabricator.wikimedia.org/T210260 (10hashar) Seems like Blubber for python should add `locale` and set `LC_ALL` to something having UTF-8. May... [08:20:07] 10ORES, 10Scoring-platform-team (Current): Add support for redis-sentinel in score cache - https://phabricator.wikimedia.org/T210579 (10Ladsgroup) p:05Triage>03High [08:22:48] 10ORES, 10Scoring-platform-team (Current), 10Puppet: Write puppet for redis-sentinel - https://phabricator.wikimedia.org/T210580 (10Ladsgroup) p:05Triage>03High [08:27:55] 10ORES, 10Scoring-platform-team (Current): New node request: oresrdb[12]003 - https://phabricator.wikimedia.org/T210582 (10Ladsgroup) p:05Triage>03High [08:27:55] El búfer 12 está vacío. [08:30:11] This bot is funnily stupid :D [08:30:18] akosiaris: Hey, this is for you: https://phabricator.wikimedia.org/T210582 [08:32:30] lol [08:49:29] 10ORES, 10Scoring-platform-team (Current), 10Performance, 10User-Ladsgroup: Make celery queues transient - https://phabricator.wikimedia.org/T210584 (10Ladsgroup) [09:14:01] 10ORES, 10Scoring-platform-team (Current), 10Performance, 10User-Ladsgroup: Make celery queues transient - https://phabricator.wikimedia.org/T210584 (10akosiaris) This has already been implemented (albeit not in celery but redis). https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/394022/2/modules/ores... [09:35:47] akosiaris: btw. Are you around for a quick ores deployment? [09:59:42] Amir1: yup [09:59:55] Thanks [10:51:28] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Add support for redis-sentinel in score cache - https://phabricator.wikimedia.org/T210579 (10Ladsgroup) a:03Ladsgroup https://github.com/wikimedia/ores/pull/295 [10:54:03] wikimedia/ores#1162 (redis_sentinel_for_cache - 3bf6541 : Amir Sarabadani): The build passed. https://travis-ci.org/wikimedia/ores/builds/460711641 [11:15:18] PROBLEM - puppet on ORES-redis02.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [11:16:31] PROBLEM - puppet on ORES-worker02.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [11:17:58] wikimedia/ores#1164 (more_json - 0010919 : Amir Sarabadani): The build passed. https://travis-ci.org/wikimedia/ores/builds/460720763 [11:18:47] PROBLEM - puppet on ORES-web02.Experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [11:25:33] PROBLEM - puppet on ORES-web01.Experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [11:27:03] ^ These are related to puppetmaster having problem [11:29:16] PROBLEM - puppet on ORES-worker01.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [11:41:24] these should recover soon [11:43:18] RECOVERY - puppet on ORES-redis02.experimental is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:44:31] RECOVERY - puppet on ORES-worker02.experimental is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [11:46:47] RECOVERY - puppet on ORES-web02.Experimental is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [11:53:33] RECOVERY - puppet on ORES-web01.Experimental is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [11:54:38] akosiaris: hey, sorry again. This is for you https://gerrit.wikimedia.org/r/c/operations/puppet/+/476250 [11:57:03] Amir1: so wait, why are we removing them now ? are they the default value, no longer needed or what ? [11:57:16] RECOVERY - puppet on ORES-worker01.experimental is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:57:52] They were the default value https://github.com/wikimedia/ores/blob/master/config/00-main.yaml#L60 [11:58:12] we had to add them because you can't mix old and new configs [11:59:02] and some of them were coming from this repo (and ores prod repo) https://github.com/wikimedia/mediawiki-services-ores-deploy/blob/master/config/00-main.yaml [11:59:10] so we had race condition issue [12:00:13] so I added these old configs to puppet then I removed all of these configs, deployed, changed the configs in puppet to new, then readded those in code repos, then deployed again [12:00:23] and now it's time to remove them from puppet [12:00:29] akosiaris: I hope this helps [12:04:24] ah ok, that explains it [12:04:25] thanks! [12:04:49] Sorry for all of the trouble, I couldn't come up with any easier way to handle this [12:06:47] no trouble, I was just wondering [12:07:08] I keep monitoring the grafana [12:07:54] just in case [12:30:07] akosiaris: also this labs-only patch https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/476255 [12:32:45] Thank you so much [13:25:12] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Change default serializer of celery from pickle to json - https://phabricator.wikimedia.org/T206333 (10Ladsgroup) The task tracker needs more work: ` "Traceback (most recent call last):\n File \"/srv/ores/venv/lib/python3.5/site-packages/kombu/ser... [13:32:34] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Change default serializer of celery from pickle to json - https://phabricator.wikimedia.org/T206333 (10Ladsgroup) https://github.com/wikimedia/ores/pull/297 https://github.com/wikimedia/ores/pull/296 [13:34:32] wikimedia/ores#1166 (even_more_json - 817915c : Amir Sarabadani): The build passed. https://travis-ci.org/wikimedia/ores/builds/460772415 [13:51:44] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Implement sentinel for ORES production Redis - https://phabricator.wikimedia.org/T122676 (10Ladsgroup) a:03Ladsgroup [13:52:23] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup: Run a test failover in labs before migrating prod to sentinel - https://phabricator.wikimedia.org/T210605 (10Ladsgroup) p:05Triage>03High [14:09:41] I try to get a nap, will be back soon [14:32:29] 10ORES, 10Scoring-platform-team, 10Wikimedia-production-error: PHP Fatal Error: Argument 2 passed to ORES\Hooks\ApiHooksHandler::addScoresForAPI() must be an instance of array, null given - https://phabricator.wikimedia.org/T210610 (10hashar) [14:32:47] 10ORES, 10Scoring-platform-team, 10Wikimedia-production-error: PHP Fatal Error: Argument 2 passed to ORES\Hooks\ApiHooksHandler::addScoresForAPI() must be an instance of array, null given - https://phabricator.wikimedia.org/T210610 (10hashar) [14:47:00] o/ hoo. I just heard you'd be in the WMDE offices during our offsite in berlin. :) [14:47:24] Yes :) I'll get there on Tuesday [14:47:32] I figure a lot of our meeting time will be less interesting (unless you have opinions about the direction of the team -- which are off course welcome) [14:47:44] But a lot of our other activites and group time might be interesting to you. [14:49:59] Sounds good… I obviously have a lot of stuff going on w/ WMDE there also… so might be hard making time sometimes [14:50:21] Right. We're hoping to scoop up a few WMDE folks for some evening activities too. [14:51:20] We'll be there over the weekend (the 8th and 9th) so I hope to make one of those days a "team building" activity -- like ice skating or something. [14:58:32] 10ORES, 10Scoring-platform-team (Current), 10User-Ladsgroup, 10Wikimedia-production-error: PHP Fatal Error: Argument 2 passed to ORES\Hooks\ApiHooksHandler::addScoresForAPI() must be an instance of array, null given - https://phabricator.wikimedia.org/T210610 (10Ladsgroup) a:03Ladsgroup [14:59:55] halfak: The top three here is for you :D https://github.com/wikimedia/ores/pulls [15:02:18] Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: @Thiemo_WMDE & @chiborg - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [15:11:57] Amir1, the sentinel work looks good. I just made some notes about structure of score_caches and one note about a warning string. [15:13:54] (03PS1) 10Ladsgroup: Don't try to add scores in API where there is nothing to add [extensions/ORES] - 10https://gerrit.wikimedia.org/r/476279 (https://phabricator.wikimedia.org/T210610) [15:14:40] halfak: Thanks. Regarding first note, Sentinel on its own is another service to coordinate between the redis cluster, it's not a proxy [15:15:00] you can only get the name of the master and you need to connect it in the client [15:15:17] that's why I think redis sentinel is a better class name [15:15:21] OK. I'm not sure if that is relevant to the note. [15:15:46] Well, we still use the same constants when interacting with it. [15:16:52] "Maybe from .redis import Sentinel" Didn't you suggest to rename the class from RedisSentinel to Sentinel? [15:18:14] halfak: ^ [15:18:36] Ahh yes. And to put it in the same module as "Redis" [15:20:21] What does it gain us? I like keep classes separate as much as possible. [15:21:13] They are both from redis.io and you have copy-pasted the constants. So DRY would be one good thing. Logical coupling is another. [15:21:29] I don't know what philosophy you are working from to "keep classes separate" [15:26:18] It's ISP + LSP IMO but meh [15:27:18] Struggling to see how https://en.wikipedia.org/wiki/Interface_segregation_principle is relevant [15:27:55] I assume by LSP you mean https://en.wikipedia.org/wiki/Liskov_substitution_principle [15:28:42] Which really just means that the APIs have a common interface that they implement so they can be switched out. It seems that under both of our proposals, this is true. [15:28:59] Note that you can have these two classes appear in the same module while allowing them to be in different files. [15:29:48] ores/score_caches/redis/(__init__.py, redis.py, sentinel.py, defaults.py) [15:30:08] I wanted to keep different implantations of an interface (score_cache) separate [15:30:18] They should not be tightly coupled [15:30:37] so e.g. I can drop the redis class and this doesn't get affected [15:31:25] Amir1, what do you mean by "tightly coupled"? It seems that sharing defaults and a coherent module structure does not result in problematic coupling. [15:31:49] You should be able to drop the redis class without having an effect. [15:31:58] I think I'm not explaining my suggestion clearly. [15:32:15] I'm not suggesting that you depend on Redis in Sentinel. [15:34:44] I'm suggesting you put "Sentinel" inside of the "redis" module (not the "Redis" class). [15:35:18] And that you import from a common submodule (e.g. "defaults") to get a TTL, SOCKET_TIMEOUT, PREFIX [15:40:46] wikimedia/ores#1172 (redis_sentinel_for_cache - 5f7da18 : Amir Sarabadani): The build was broken. https://travis-ci.org/wikimedia/ores/builds/460832877 [15:43:36] I personally think it's more of the style than an real issue so I just do it [15:44:35] So you don't see the point I'm making or you see it but disagree with it? *sigh* It would be nice to be understood. [15:45:29] To me, structuring modules in python should make sense and aid in applying DRY. [15:46:15] wikimedia/ores#1173 (redis_sentinel_for_cache - c85282b : Amir Sarabadani): The build was fixed. https://travis-ci.org/wikimedia/ores/builds/460835357 [15:52:07] Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: @Thiemo_WMDE & @chiborg - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [15:58:05] 10JADE: In a multi-part judgment, can one of the parts be null? - https://phabricator.wikimedia.org/T209524 (10Halfak) Hey! Yes. So I believe that no one intentionally selects "null" in Wikilabels. The "nulls" we get are when information is unavailable (e.g. the revision was deleted). "null" doesn't carry th... [15:58:08] 10ORES, 10Scoring-platform-team (Current), 10Patch-For-Review, 10User-Ladsgroup, 10Wikimedia-production-error: PHP Fatal Error: Argument 2 passed to ORES\Hooks\ApiHooksHandler::addScoresForAPI() must be an instance of array, null given - https://phabricator.wikimedia.org/T210610 (10zeljkofilipin) p:05Tr... [16:26:24] Last call for SoS [16:39:01] * halfak watches Rosiestep talk about ORES @ Wikicite [16:39:33] https://www.youtube.com/watch?v=9i37kj7DS28#t=33m35s [16:41:27] notconfusing, gets a shout out for WIGI too [16:41:48] nice thansk! I need to fix that tool acutally, so a bit of guilt there [16:44:53] ORES reference starts at 38:40 [16:49:56] its cool to be a part of that movement [16:50:17] this is what its all about creating tools to activists can make a difference [16:53:50] right on :) [16:56:55] i'm really sad that finally wikicite was in my backyard but i wasnt in my own backyard [17:02:18] halfak: just checking, mwapi doesn't have flow-api bindings, right? maybe when i finsih this subtask i'll look into adding them [17:03:38] I don't think so. That would be cool. [17:23:19] 10Scoring-platform-team, 10Wikibase-Containers, 10Wikidata, 10Wikilabels, 10Release Pipeline (Blubber): Stretch in docker registry forces ascii encoding - https://phabricator.wikimedia.org/T210260 (10LarsWirzenius) I confirm that generating the locales in the image, or installing locales-all (which has a... [17:38:10] 10Scoring-platform-team, 10Wikibase-Containers, 10Wikidata, 10Wikilabels, 10Release Pipeline (Blubber): Stretch in docker registry forces ascii encoding - https://phabricator.wikimedia.org/T210260 (10Ladsgroup) What about `en_US.UTF8`? Most servers has this locale installed [17:53:50] Don't forget about BS session [17:54:52] haha [17:55:10] I'm basically at 13th hour of my work today [17:55:44] My overtime is reaching a point that I need to take one week off to compensate :P [17:55:46] o/ see you tomorrow :) [17:55:49] please do [17:55:59] 10Scoring-platform-team, 10Wikibase-Containers, 10Wikidata, 10Wikilabels, 10Release Pipeline (Blubber): Stretch in docker registry forces ascii encoding - https://phabricator.wikimedia.org/T210260 (10LarsWirzenius) 19:21 exolobe4:~/uni $ docker run unitest env LC_ALL=en_US.UTF8 python3 -c "print('étoile'... [17:56:58] awight: well, I'm waiting for this gerrit patch to be merged so I backport it to unblock the train [17:56:59] :D [17:57:16] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ORES/+/476279 [17:57:21] Amir1: I can take that over if you want [17:57:33] awight: I already tested on localhost, fixes the issue [17:57:42] Nah, it's easy, I finish it [17:58:22] (03CR) 10Awight: [C: 032] "FYI we don't need to return true any more, "return;" is sufficient." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/476279 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [17:59:45] Thanks! [18:00:20] This end-run around the API is pretty scary... [18:00:49] Ideally, it seems like the API would have native hooks meant for this type of batch processing. [18:03:47] First of all it needs tests but when I asked for tests in the patch that introduced it I was told "It's a rabbit hole I'm going into" [18:05:10] (03CR) 10Hashar: "Please cherry pick it to wmf/1.33.0-wmf.6 so we can hotfix production as well :)" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/476279 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [18:05:45] (03CR) 10Ladsgroup: "Already on it. Waiting for jenkins :D" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/476279 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [18:08:11] That's a hell of a chunk of code to not test. [18:09:50] btw. This morning I triaged a big chunk of tasks, feel free to change priority or move things around [18:12:21] 10MediaWiki-extensions-ORES, 10Scoring-platform-team: Write tests for ApiHooksHandler - https://phabricator.wikimedia.org/T210638 (10awight) [18:12:42] 10Scoring-platform-team, 10Wikibase-Containers, 10Wikidata, 10Wikilabels, 10Release Pipeline (Blubber): Stretch in docker registry forces ascii encoding - https://phabricator.wikimedia.org/T210260 (10Ladsgroup) That sounds good to add to blubber. My knowledge in go (and docker) is not well to add it on m... [18:15:20] notconfusing: holler if you want to show off any patches today,e tc [18:15:26] and etc. [18:15:45] awight: you mean with regards to newcomerquality? [18:15:53] yah [18:15:58] I know you're on a coding spree... [18:17:06] awight: Also there is one open PR in ORES if you feel like it :D [18:17:08] 10Scoring-platform-team, 10Wikibase-Containers, 10Wikidata, 10Wikilabels, 10Release Pipeline (Blubber): Stretch in docker registry forces ascii encoding - https://phabricator.wikimedia.org/T210260 (10hashar) On my local Debian Stretch machine: ` $ locale -a C C.UTF-8 fr_FR.utf8 POSIX ` ` $ LC_ALL=C.UTF-8... [18:17:41] awight: i finished doing the prediction pipeline, but only in ipython, i still need to convert it to python-standalone, so there aren't patches yet, but another set of eyes on the ipython doc would be good, or would you rather wait? [18:18:06] I'd be interested in seeing it, totally. [18:18:17] u has in git? [18:18:41] yes. let me just make sure its pushed up [18:19:51] Amir1: looking now. [18:26:21] Thanks [18:29:47] 10Scoring-platform-team, 10Wikibase-Containers, 10Wikidata, 10Wikilabels, 10Release Pipeline (Blubber): Stretch in docker registry forces ascii encoding - https://phabricator.wikimedia.org/T210260 (10hashar) The base container has the appropriate locale apparently and we can have the container to set `EN... [18:31:55] (03Merged) 10jenkins-bot: Don't try to add scores in API where there is nothing to add [extensions/ORES] - 10https://gerrit.wikimedia.org/r/476279 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [18:35:34] (03PS1) 10Ladsgroup: Don't try to add scores in API where there is nothing to add [extensions/ORES] (wmf/1.33.0-wmf.6) - 10https://gerrit.wikimedia.org/r/476334 (https://phabricator.wikimedia.org/T210610) [18:36:30] (03CR) 10Ladsgroup: [C: 032] Don't try to add scores in API where there is nothing to add [extensions/ORES] (wmf/1.33.0-wmf.6) - 10https://gerrit.wikimedia.org/r/476334 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [18:40:15] (03CR) 10jenkins-bot: Don't try to add scores in API where there is nothing to add [extensions/ORES] - 10https://gerrit.wikimedia.org/r/476279 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [18:40:40] (03Merged) 10jenkins-bot: Don't try to add scores in API where there is nothing to add [extensions/ORES] (wmf/1.33.0-wmf.6) - 10https://gerrit.wikimedia.org/r/476334 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [18:42:50] (03CR) 10jenkins-bot: Don't try to add scores in API where there is nothing to add [extensions/ORES] (wmf/1.33.0-wmf.6) - 10https://gerrit.wikimedia.org/r/476334 (https://phabricator.wikimedia.org/T210610) (owner: 10Ladsgroup) [19:09:57] 10ORES, 10Scoring-platform-team (Current), 10MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), 10Patch-For-Review, and 2 others: PHP Fatal Error: Argument 2 passed to ORES\Hooks\ApiHooksHandler::addScoresForAPI() must be an instance of array, null given - https://phabricator.wikimedia.org/T210610 (10Ladsgroup) 05... [19:36:58] I'm done for the day [19:37:02] see you tomorrow [19:37:04] o/ [19:37:06] o/ see you later Amir1 [19:37:17] I'm working on Italian models :) [19:37:21] o/ have fun people! [19:37:38] Italian models are the best models :D [19:37:48] O_o [19:37:50] Sorry I had to make the pun joke [19:38:14] Italy has a problem with not feeding their models [19:40:34] I'm out for ~45min [19:40:46] You're not supposed to feed data to the models? instructions unclear. [19:41:15] * halfak suddenly realizes a whole world of punniness involved in thinking of models as people. [19:52:29] Wow. The itwiki sample is really old. [20:36:21] ERROR: Authentication error: Authentication required: You must have push access to verify locks [20:36:22] What [20:39:30] Ha! I needed to remove a faulty pre-push hook [20:39:37] What a complicated mess you are, git. [20:39:48] I wish the world would have gone with mercurial instead. [20:40:46] mercurial is to subversion as git is to cvs [20:40:54] Similar, but better and more consistent. [20:40:55] I remember it being pretty much equivalent. Were the improvements around storage semantics? [20:41:13] There were magic incantations, occasionally. [20:41:23] I think the rate of magic was much lower. [20:41:23] I mean nothing compared to git. [20:41:26] In my experience anyway [20:41:26] that [20:41:27] Right [20:41:51] wikimedia/editquality#389 (itwiki_advanced - 758f0b2 : halfak): The build passed. https://travis-ci.org/wikimedia/editquality/builds/460968572 [20:41:56] Thankfully we have google to help us apply magic [20:42:02] *web search :) [20:45:55] * halfak watches the tuning proceed. :) [20:46:34] 10Scoring-platform-team: AWight to summarize CSCW notes to a blog - https://phabricator.wikimedia.org/T210662 (10awight) [20:52:22] 10Scoring-platform-team, 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Build article quality model for Galician Wikipedia - https://phabricator.wikimedia.org/T201146 (10Halfak) Finally getting a chance to work on this. Sorry for the delay. Looks like we have 67 Stubs, 100 Starts,... [20:58:18] halfak, awight: is there still a JADE demo script (was it a userscript maybe?) that I can show to people? I'd like to show it to some students tomorrow. [20:58:29] doesn't need to be pretty or up-to-date; just want to get the point across. [20:58:32] harej, ^ [20:58:37] ty [20:58:42] :) [20:59:35] J-Mo: it's running on this demo wiki https://jade-demo.wmflabs.org/mediawiki/index.php?title=Jade&action=history [21:00:03] marvelous! Thank you harej [21:00:05] the gadget itself is https://en.wikipedia.org/w/index.php?title=User%3APrtksxna%2Fjade-test.js&action=raw&ctype=text%2Fjavascript [21:00:12] https://jade-demo.wmflabs.org/mediawiki/index.php/MediaWiki:Common.js [21:13:53] hmmm... do we not have git lfs installed on stat1006? [21:15:43] How have other people been updating models? [21:15:47] awight, ^ [21:16:38] *sigh* [21:16:42] halfak: I haven't in a while, but pretty sure I did have LFS [21:16:58] I'm so confused. I'm using /usr/bin/git [21:17:11] It's on stat1005 [21:17:47] * halfak curses the science gods [21:18:05] I never understand why stat1005 and 1006 are configured differently. [21:18:06] Does seem inefficient to randomly install half the tools on some machines [21:19:01] I'd probably elect to go maximal if possible, e.g. include the hive clients on the non-hive servers [21:24:11] wikimedia/articlequality#104 (glwiki - de50f65 : halfak): The build failed. https://travis-ci.org/wikimedia/articlequality/builds/460986960 [21:41:30] 10Scoring-platform-team, 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Build article quality model for Galician Wikipedia - https://phabricator.wikimedia.org/T201146 (10Halfak) https://github.com/wikimedia/articlequality/pull/70 [21:41:43] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Build article quality model for Galician Wikipedia - https://phabricator.wikimedia.org/T201146 (10Halfak) [21:42:32] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train/test damaging & goodfaith models for Italian Wikipedia (itwiki) - https://phabricator.wikimedia.org/T208779 (10Halfak) https://github.com/wikimedia/editquality/pull/172 [21:42:53] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train/test damaging & goodfaith models for Italian Wikipedia (itwiki) - https://phabricator.wikimedia.org/T208779 (10Halfak) a:03Halfak [21:44:24] (03PS2) 10Awight: [WIP] Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) [21:44:43] -- might be gone for 30min [21:44:51] awight, just added a couple modeling things to the "review" column [21:45:02] Oh! no worries. I'll just ping you in the task if that's OK :) [21:45:23] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train/test damaging & goodfaith models for Italian Wikipedia (itwiki) - https://phabricator.wikimedia.org/T208779 (10Halfak) @awight :D [21:45:46] Actually you might leave these to Am.ir1 if you're busy. [21:46:40] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) (owner: 10Awight) [21:47:39] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) (owner: 10Awight) [22:39:53] Random thought, we could include a suggested thresholds chart like https://en.wikipedia.org/wiki/Special:ORESModels in the model_info output. [22:41:37] Thinking about the Echo algorithm, it makes a lot of sense. We should adopt the strategy when making dumps. [23:04:38] halfak: Argh! I just noticed that the itwiki models I merged are now in the main git history rather than LFS. We'll have to rewrite history to clean up... [23:09:44] 10Scoring-platform-team: Accidentally merged models in editquality; need to rewrite history - https://phabricator.wikimedia.org/T210678 (10awight)