[00:02:48] hoo: Hi, I'm looking for the example you suggested re: cached messageLocalizer, it's a wild world in Wikibase... [00:03:29] Is the solution to forsake service wiring? [00:03:58] (03Merged) 10jenkins-bot: Split user schema into local ID, central ID, and IP [extensions/JADE] - 10https://gerrit.wikimedia.org/r/461502 (https://phabricator.wikimedia.org/T206573) (owner: 10Awight) [00:04:09] includes/WikibaseRepo.php includes a metric crapload of pseudo-service getters [00:05:34] awight: In WikibaseRepo (the part running on Wikidata.org, not the part running on Wikipedia) we just always force render stuff in user language [00:05:46] but that's not an option for JADE [00:05:52] as this might also run on like enwiki [00:06:13] and you don't want to split the parser cache this wildly there for the two users having their interface set to Greek [00:06:34] I was thinking that content language was appropriate here [00:07:23] The main argument being that freeform wikitext fields won't be translated, and the page language should be consistent. [00:07:47] * awight tries to remember if ULS is a thing for anons [00:11:07] Not really [00:11:19] Ah, yes… in that case content language it is [00:11:23] except on maybe commons [00:11:58] Wikibase client splits by user language on commons, metawiki, … [00:12:20] Will the cache already be split on $parser->getContentLanguage ? [00:12:25] Users expect things in their language on these… but not sure in which language they'd comment [00:12:36] No, only on getUserLang [00:12:37] Sorry I'm ignorant about this detail, feel free to link to docs so I can figure it out. [00:13:03] I'm not sure there are decent docs :S [00:13:12] kk, reusing the existing cache variation seems prudent at least. [00:13:13] We got this wrong so so often with Wikibase [00:13:17] hehe I bet [00:13:33] We've been head-scratching about this exact question for CentralNotice banners since the dawn of time [00:14:07] Oh, I can guess [00:14:20] Anyway, it's past 2am here and I should really go to bed [00:14:26] But if interface language can only be != content language for logged-in users, I think we're good with parser->getUserLang, right? [00:14:46] ok please do; I was wondering if you were clinging to the bottom of the planet or on a timezone safari... [00:14:49] o/ [00:15:40] Yeah, in some cases there's no way around getUserLang… but it's a performance killer, so beware [00:15:42] See you :) [00:33:58] (03CR) 10jenkins-bot: Comment fixups [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468077 (owner: 10Awight) [00:36:39] (03CR) 10jenkins-bot: Test helper for judgment text; add defaults [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468076 (owner: 10Awight) [00:39:27] (03CR) 10jenkins-bot: Split user schema into local ID, central ID, and IP [extensions/JADE] - 10https://gerrit.wikimedia.org/r/461502 (https://phabricator.wikimedia.org/T206573) (owner: 10Awight) [00:39:38] (03PS19) 10Awight: Render Judgment pages as wikitext [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464737 (https://phabricator.wikimedia.org/T206346) [00:39:40] (03PS4) 10Awight: Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 [00:39:42] (03PS18) 10Awight: Tests to demonstrate SpamBlacklist integration [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464727 (https://phabricator.wikimedia.org/T206255) [00:39:44] (03PS53) 10Awight: Secondary schema for JADE indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) [00:39:47] (03PS3) 10Awight: Link table model [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) [00:39:48] (03PS12) 10Awight: Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) [00:39:50] (03PS12) 10Awight: Maintenance scripts for judgment indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466808 (https://phabricator.wikimedia.org/T202596) [00:39:52] (03PS1) 10Awight: Clean up test annotations; coversNothing for integration test [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468192 [00:39:54] (03PS1) 10Awight: [WIP] More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 [00:40:00] (03CR) 10Awight: Render Judgment pages as wikitext (036 comments) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464737 (https://phabricator.wikimedia.org/T206346) (owner: 10Awight) [00:44:15] (03CR) 10jerkins-bot: [V: 04-1] [WIP] More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 (owner: 10Awight) [00:45:51] (03CR) 10jerkins-bot: [V: 04-1] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [00:46:30] (03CR) 10jerkins-bot: [V: 04-1] Maintenance scripts for judgment indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466808 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [00:46:39] (03CR) 10jerkins-bot: [V: 04-1] Link table model [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [00:46:41] (03CR) 10jerkins-bot: [V: 04-1] Tests to demonstrate SpamBlacklist integration [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464727 (https://phabricator.wikimedia.org/T206255) (owner: 10Awight) [00:47:26] (03CR) 10jerkins-bot: [V: 04-1] Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [00:48:58] (03CR) 10jerkins-bot: [V: 04-1] Clean up test annotations; coversNothing for integration test [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468192 (owner: 10Awight) [00:49:07] (03CR) 10jerkins-bot: [V: 04-1] Secondary schema for JADE indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [00:49:29] (03CR) 10jerkins-bot: [V: 04-1] Tests to demonstrate SpamBlacklist integration [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464727 (https://phabricator.wikimedia.org/T206255) (owner: 10Awight) [00:52:10] (03CR) 10jerkins-bot: [V: 04-1] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [00:54:45] (03CR) 10jerkins-bot: [V: 04-1] Secondary schema for JADE indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [00:58:09] (03CR) 10jerkins-bot: [V: 04-1] Link table model [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [01:04:34] (03CR) 10jerkins-bot: [V: 04-1] Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [01:07:58] (03CR) 10jerkins-bot: [V: 04-1] Maintenance scripts for judgment indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466808 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [01:12:57] (03CR) 10jerkins-bot: [V: 04-1] Clean up test annotations; coversNothing for integration test [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468192 (owner: 10Awight) [01:16:21] (03CR) 10jerkins-bot: [V: 04-1] [WIP] More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 (owner: 10Awight) [06:19:04] (03PS19) 10Awight: Tests to demonstrate SpamBlacklist integration [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464727 (https://phabricator.wikimedia.org/T206255) [06:19:06] (03PS54) 10Awight: Secondary schema for JADE indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) [06:19:08] (03PS4) 10Awight: Link table model [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) [06:19:10] (03PS13) 10Awight: Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) [06:19:12] (03PS13) 10Awight: Maintenance scripts for judgment indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466808 (https://phabricator.wikimedia.org/T202596) [06:19:14] (03PS2) 10Awight: Clean up test annotations; coversNothing for integration test [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468192 [06:19:16] (03PS5) 10Awight: [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 [06:19:19] (03PS2) 10Awight: [WIP] More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 [06:22:59] (03CR) 10jerkins-bot: [V: 04-1] [WIP] More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 (owner: 10Awight) [06:27:43] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [06:45:39] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [06:48:45] (03CR) 10jerkins-bot: [V: 04-1] [WIP] More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 (owner: 10Awight) [08:24:01] (03PS14) 10Awight: Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) [08:24:03] (03PS14) 10Awight: Maintenance scripts for judgment indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466808 (https://phabricator.wikimedia.org/T202596) [08:24:05] (03PS3) 10Awight: Clean up test annotations; coversNothing for integration test [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468192 [08:24:07] (03PS3) 10Awight: More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 [08:24:09] (03PS6) 10Awight: [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 [08:30:44] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [08:44:00] (03PS4) 10Awight: Clean up test annotations; coversNothing for integration test [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468192 [08:44:02] (03PS4) 10Awight: More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 [08:44:04] (03PS7) 10Awight: [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 [08:44:06] (03PS1) 10Awight: Tests for JudgmentTarget [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468277 [08:53:13] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [09:02:34] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [09:08:20] (03PS20) 10Awight: Render Judgment pages as wikitext [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464737 (https://phabricator.wikimedia.org/T206346) [09:08:22] (03PS20) 10Awight: Tests to demonstrate SpamBlacklist integration [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464727 (https://phabricator.wikimedia.org/T206255) [09:08:24] (03PS55) 10Awight: Secondary schema for JADE indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) [09:08:26] (03PS5) 10Awight: Link table model [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) [09:08:28] (03PS15) 10Awight: Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) [09:08:30] (03PS15) 10Awight: Maintenance scripts for judgment indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466808 (https://phabricator.wikimedia.org/T202596) [09:08:32] (03PS5) 10Awight: Clean up test annotations; coversNothing for integration test [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468192 [09:08:34] (03PS5) 10Awight: More MoveHooks tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468193 [09:08:36] (03PS2) 10Awight: Tests for JudgmentTarget [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468277 [09:08:38] (03PS1) 10Awight: Services tests [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468280 [09:08:40] (03PS8) 10Awight: [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 [09:21:32] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [09:33:51] o/ [09:34:54] (03PS1) 10Awight: [WIP] link table helper can be stateful [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468284 [09:37:02] (03CR) 10jerkins-bot: [V: 04-1] [WIP] link table helper can be stateful [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468284 (owner: 10Awight) [09:49:38] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Unit tests for JudgmentContent [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468180 (owner: 10Awight) [09:50:20] (03PS2) 10Awight: [WIP] link table helper can be stateful [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468284 [09:55:08] (03CR) 10jerkins-bot: [V: 04-1] [WIP] link table helper can be stateful [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468284 (owner: 10Awight) [09:59:37] (03PS1) 10Awight: [WIP] Rename shouty classes to "Jade" [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468292 [10:04:43] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Rename shouty classes to "Jade" [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468292 (owner: 10Awight) [10:13:16] (03CR) 10jerkins-bot: [V: 04-1] [WIP] link table helper can be stateful [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468284 (owner: 10Awight) [10:15:52] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Rename shouty classes to "Jade" [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468292 (owner: 10Awight) [11:50:07] 10MediaWiki-extensions-ORES, 10Scoring-platform-team (Current), 10Wikimedia-Extension-setup, 10Patch-For-Review, 10User-Ladsgroup: Enable ORES Review Tool on Czech Wikipedia - https://phabricator.wikimedia.org/T151611 (10Urbanecm) [12:08:52] afk for lunch, will be back soon [12:46:39] back now [13:24:28] (03PS1) 10Ladsgroup: Add uwsgi logging [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468314 (https://phabricator.wikimedia.org/T169586) [13:25:08] (03CR) 10Ladsgroup: [V: 032 C: 032] "Going beta" [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468314 (https://phabricator.wikimedia.org/T169586) (owner: 10Ladsgroup) [13:39:14] o/ [13:39:16] Feeling better today. [14:32:24] (03PS1) 10Ladsgroup: Fix typo [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468325 [14:32:48] (03CR) 10Ladsgroup: [V: 032 C: 032] Fix typo [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468325 (owner: 10Ladsgroup) [14:45:46] I successfully managed to explode a browser tab while trying to implement JS thread pooling. [14:45:47] :) [14:46:11] I had a look at that threads.js and the examples don't look like a thread pool to me. [14:54:44] halfak: Reviewing this would be amazing: https://github.com/wikimedia/ores/pull/275 (+ reviewing the redis one) [14:55:04] I want to push these two to prod today [14:55:56] I must be missing something from that PR [14:56:05] I don't see the logging config getting moved to the main config. [14:56:19] I also don't see a reference to the relevant section of the main config. [14:57:01] Amir1, ^ it looks like I am reviewing something that is far less than functional. Is that intentional? [14:57:10] it's already done [14:57:19] let me show you the old pr, this is the fix up [14:57:32] wikimedia/ores#1067 (fix_logging - 47e1f81 : Amir Sarabadani): The build passed. https://travis-ci.org/wikimedia/ores/builds/443228397 [14:57:37] https://github.com/wikimedia/ores/pull/272/files [14:58:27] and https://gerrit.wikimedia.org/r/c/mediawiki/services/ores/deploy/+/466708 [14:58:29] halfak: ^ [15:00:43] Merged the first one. [15:01:24] Looks like that gerrit one is already merged. Am I reading that right? [15:02:45] yeah, just wanted to show what we has done [15:02:58] Thanks [15:02:59] gotcha. makes sense to me. [15:13:37] halfak: do you think we can get this deployed today? https://github.com/wikimedia/ores/pull/270 [15:14:27] (03PS1) 10Ladsgroup: Bump ores to HEAD [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468336 [15:15:09] Amir1, where can I see the results of your most recent analysis? [15:15:14] (03CR) 10Ladsgroup: [V: 032 C: 032] Bump ores to HEAD [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468336 (owner: 10Ladsgroup) [15:16:12] https://grafana-labs.wikimedia.org/dashboard/db/ores-labs?orgId=1&from=now-48h&to=now&panelId=16&fullscreen&edit&tab=general [15:16:55] it was okay yesterday but it seems logstash in labs is broken today [15:16:58] * halfak waits for graphs to load. [15:18:23] check cpu usage of worker nodes in yesterday [15:19:03] graphs don't load [15:22:02] halfak: you can load them in graphite-labs.wikimedia.org [15:23:06] https://grafana-labs.wikimedia.org/dashboard/db/ores-labs?orgId=1&from=now-48h&to=now&panelId=7&fullscreen [15:23:09] it loads now [15:24:27] How do I read this? [15:24:36] Whend did you run what? [15:27:01] halfak: at 14:20 I started the double precaching using celery [15:27:17] at 15:03 I did it with redis [15:27:51] the spikes vary but it didn't change with one precache or two [15:29:48] I think we need a clear discussion of this in the task before we move forward. If something goes wrong, we'll want to know exactly what we tested. [15:34:20] Amir1, ^ [15:39:27] I have tested everything possible. You can try it too [15:39:38] halfak: it's still there in ores.wmflabs [15:40:09] Amir1, right. I could take an hour or so and test this myself or I could work from your notes when I review in 5 minutes. [15:40:29] And regardless, we'll want this type of clear documentation for future reference. [15:42:39] halfak: I have tested with first one precache using celery deduplicator, then tested it with two precache which one had 0.2 delay, then I tested it with redis deduplicator, one precache, then two precache (similar) and then one precache again. the web cpu went up, but worker stayed the same [15:43:00] that's basically all of ti [15:43:29] ^ Needs to be in the task with timestamps and reports of the CPU seen in the graph. [15:45:12] E.g. how long did you run each precache. What % CPU did you see in celery/uwsgi? Was it even across all of the workers? Did the cache hit rate reflect the task tracking catching the duplicate requests? [16:23:33] PROBLEM - ORES web node labs ores-web-01 on ores.wmflabs.org is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 234 bytes in 0.018 second response time [16:24:22] PROBLEM - ORES worker labs on ores.wmflabs.org is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 234 bytes in 0.037 second response time [16:24:48] Amir1, ^ you or something else? [16:24:58] me [16:25:02] kk [16:25:52] PROBLEM - ORES web node labs ores-web-02 on ores.wmflabs.org is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 234 bytes in 0.026 second response time [17:08:23] I finally have thread pooling working. :))) [17:31:04] awight: is there an "easy" way of creating a new judgment by hand, or should I just read the schema and come up with something? [17:32:06] harej: yes, local wiki? [17:32:13] http://jade-demo.wmflabs.org/mediawiki/index.php/Jade [17:32:19] ooh [17:34:17] harej: Here's some conforming sample data, https://en.wikipedia.beta.wmflabs.org/wiki/Judgment:Diff/376901 [17:34:29] The only other trick is that you need to have a valid revid in the title [17:36:50] harej: What commit of E:JADE do you have on that site? maybe it's older? [17:37:55] just updated it to master [17:37:56] harej: Please add me to the project group at your leisure [17:37:58] hmm [17:38:14] I'm trying to create http://jade-demo.wmflabs.org/mediawiki/index.php?title=Judgment:Diff/4439&action=submit [17:38:19] "internal error" [17:39:02] I'm running 1.31; does JADE require master? [17:39:27] E:JADE version does look OK [17:39:46] JADE might require something from master, maybe check the exception log? [17:40:03] the... apache exception log? [17:40:07] Try not to fix it too fast ;-) failures are helpful [17:40:09] umm [17:40:25] $wgShowExceptionDetails = true; [17:41:15] done [17:41:50] I also added you to the WPX project; this wiki is hosted on wpx-mediawiki-02 [17:41:52] .eqiad.wmflabs [17:42:42] ty [17:44:08] harej: [b9fee4ba4978737296a104a9] /mediawiki/index.php?title=Judgment:Diff/4439&action=submit Error from line 291 of /var/lib/mediawiki/extensions/JADE/includes/JudgmentValidator.php: Class 'JsonSchema\Validator' not found [17:44:15] needs composer install [17:44:16] https://gist.github.com/halfak/5e0c4951c52f57ff3320aff9b51b757e [17:44:22] ^ ThreadPool.js that works. [17:44:55] d'oh [17:46:36] okay, I installed everything [17:47:56] harej: O_o same error [17:48:00] * awight ssh's [17:49:15] argh, where is the webroot? [17:49:25] debian-mediawiki is a weird beast [17:49:31] there's /etc/mediawiki and /var/lib/mediawiki [17:49:36] i think you're looking for the latter [17:50:17] really easy to install -- "sudo apt install mediawiki" -- then to work with? a bit of a learning curve [17:51:03] harr [17:51:34] * awight scratches head [17:52:04] /var/lib/mediawiki/extensions/JADE/vendor/justinrainbow/json-schema/src/JsonSchema/Validator.php [17:52:08] is there [17:53:28] I even restarted apache [17:55:39] how *does* extensions/*/vendor get pulled into the autoloads? [17:55:57] i may have forgotten to run composer for core [17:57:14] weird, i thought debian would handle that automatically [17:57:40] JADE/vendor/composer/autoload_* looks correct FWIW [17:58:13] there's... something that might need to be done to pull in the extensions vendor stuff? [17:58:51] * awight tried to avoid pulling legoktm's bell on a string [17:59:29] something about merge-plugin? [17:59:57] I'm re-thinking how threadpool workers can work more effectively. I think we should maintain an independent queue of workers processing score requests and use promises to handle everything. The only problem is handling the first N requests. How do you know when the queue is filling up so you can batch appropriately? [18:00:15] harej: What's this about... https://www.mediawiki.org/wiki/Composer#Using_composer-merge-plugin [18:00:30] * halfak thinks out loud. Feel free to ignore. [18:00:40] :) Don't worry, we are. Looks great though. [18:01:02] Composer is not my thing, unfortunately [18:01:10] I just type "composer update" and it does the magic thing for me [18:03:04] harej: well... I ran "composer update" from mediawiki and it works now [18:03:07] http://jade-demo.wmflabs.org/mediawiki/index.php/Judgment:Diff/4439 [18:03:35] btw you really want to go out on a branch to get the "render wikitext" patch [18:03:41] totally different experience. [18:03:57] indeed; that was my intent with setting up this wiki [18:04:07] harej: https://commons.wikimedia.org/wiki/File:JADE_rendered_diff.png [18:04:09] to be able to experiment with branches ~*ahead of master*~ [18:04:13] :D [18:04:26] Take the last commit before my WIPs... [18:04:32] * awight fumbles for an ID [18:04:48] find me a git checkout command to use [18:05:05] https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/468280/ [18:05:10] git fetch ssh://awight@gerrit.wikimedia.org:29418/mediawiki/extensions/JADE refs/changes/80/468280/1 && git checkout FETCH_HEAD [18:05:20] s/awight/harej or what have you [18:07:05] harej: You'll also need to: cd mediawiki; php maintenance/update.php --quick [18:09:11] http://jade-demo.wmflabs.org/mediawiki/index.php/Judgment:Diff/4439 meh [18:10:19] harej: ooh that would have to do with my code sucking [18:10:22] one moment please [18:10:52] harej: You'll need 1.32 [18:10:54] dang it [18:13:52] harej: Is that even possible with the deb package?? [18:14:03] i think the debian package only does LTS versions [18:14:09] legoktm can confirm [18:14:24] Oof, k I'll workaround momentarily [18:14:40] or I can just install a newer version of MediaWiki [18:14:54] we may want to update the docs to say that version 1.32 is a minimum requirement. i'll do that [18:17:27] RECOVERY - ORES worker labs on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 443 bytes in 0.544 second response time [18:17:33] (03PS1) 10Awight: [DNM] Compat for MediaWiki 1.31 [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468372 [18:17:47] RECOVERY - ORES web node labs ores-web-02 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 457 bytes in 0.030 second response time [18:17:52] harej: git fetch ssh://awight@gerrit.wikimedia.org:29418/mediawiki/extensions/JADE refs/changes/72/468372/1 && git checkout FETCH_HEAD [18:17:56] thank you! [18:18:06] RECOVERY - ORES web node labs ores-web-01 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 458 bytes in 0.545 second response time [18:18:39] Heads-up, I'm restarting ORES services for a DNS update [18:19:16] new error http://jade-demo.wmflabs.org/mediawiki/index.php/Judgment:Diff/4439 [18:19:39] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Compat for MediaWiki 1.31 [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468372 (owner: 10Awight) [18:21:25] * awight facepalms [18:21:29] ok this time I will test [18:22:59] (03PS2) 10Awight: [DNM] Compat for MediaWiki 1.31 [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468372 [18:23:09] harej: ready to re-up [18:23:28] git fetch ssh://awight@gerrit.wikimedia.org:29418/mediawiki/extensions/JADE refs/changes/72/468372/2 && git checkout FETCH_HEAD [18:23:57] hey this looks great! [18:24:31] there are section edit links that don't work; is the plan to eventually enable section editing? [18:24:34] <3 right on [18:24:37] Good question [18:24:52] I think the wiki page-ness will be both a strength, and really confusing. [18:24:59] We should chat [18:25:13] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Compat for MediaWiki 1.31 [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468372 (owner: 10Awight) [18:25:44] When I'm not occupied with this conference business I'm going to want to create a user story map thing that highlights what interfaces we need to create [18:25:55] harej: Are you watching the GenderMag talk btw? [18:26:07] I'm watching Alex Stinson talk about Wikidata :D [18:26:13] oh huh [18:27:35] harej: http://gendermag.org these are some proven tools that we could benefit from, "gender inclusiveness magnification" to help with our user stories. [18:27:51] IMO it's very relevant to our project since we're in the business of battling bias. [18:30:31] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Compat for MediaWiki 1.31 [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468372 (owner: 10Awight) [18:30:47] PROBLEM - ORES worker labs on ores.wmflabs.org is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 325 bytes in 0.031 second response time [18:32:22] harej: Now, with user IDs [18:32:45] neat! [18:33:06] RECOVERY - ORES worker labs on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 443 bytes in 0.547 second response time [18:34:19] harej: I'm not really sold on the table format for endorsements, it's pretty distracting. If you have any improvements to make, take a look here: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/JADE/+/506b0e820887a9523e6183c6a8e9da7e3825fc24/templates/judgment_page.wiki.mustache [18:35:28] awight & harej: section-edit-wise, it seems like we could go the wikibase route. [18:39:36] * awight clicks edit on wikidata [18:40:37] (03CR) 10Hoo man: [C: 032] "In the future, we should diable the useless/ defunct section edit links, but this is a very good start. Also manually tested." [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464737 (https://phabricator.wikimedia.org/T206346) (owner: 10Awight) [18:40:38] ah yeah, in-place ajaxy editing. I like it. [18:40:48] \o/ [18:41:34] Yeah. I think that's the future. It could be that we can't quite handle it in the MVP. [18:41:44] what do you think awight? [18:41:54] I think in any case we shouldn't have section edit links that don't work. Either the links should work or they should not be there. [18:41:58] It seems our user-testing will be around an ajaxy editing mode. [18:42:09] awight: halfak: Shall we wait for DBA approval for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/JADE/+/456078? [18:42:18] Because this is blocking the whole stack of changes [18:42:39] hoo, what do you mean by "wait"? [18:42:42] halfak: harej: hoo: +1 about section editing, I'll turn that off [18:43:19] I actually looked into thsi right before the discussion started here… offhand I found no nice way to do that [18:43:26] but I probably miss something [18:44:08] Worst-case, I can just replace headings with bold... [18:45:37] hoo: as for the DBA review... akosiaris did you have a moment to chat with marostegui by any chance? [18:47:12] I'm okay with blocking for maybe one more week--I had pinged a few times now, but now that I've escalated to SoS request I feel like we can wait one more windowful. [18:47:38] [11:14:03] i think the debian package only does LTS versions <-- yes. There's https://phabricator.wikimedia.org/T156543 but I doubt it's going to happen for pre-release versions. [18:48:02] However. I'm also fine with merging the DB patches ourselves, since we're not deployed to production. [18:48:09] DBAs can review on the master branch. [18:48:58] (03Merged) 10jenkins-bot: Render Judgment pages as wikitext [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464737 (https://phabricator.wikimedia.org/T206346) (owner: 10Awight) [18:49:00] (03Merged) 10jenkins-bot: Tests to demonstrate SpamBlacklist integration [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464727 (https://phabricator.wikimedia.org/T206255) (owner: 10Awight) [18:51:26] (03CR) 10jenkins-bot: Render Judgment pages as wikitext [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464737 (https://phabricator.wikimedia.org/T206346) (owner: 10Awight) [18:55:38] (03CR) 10jenkins-bot: Tests to demonstrate SpamBlacklist integration [extensions/JADE] - 10https://gerrit.wikimedia.org/r/464727 (https://phabricator.wikimedia.org/T206255) (owner: 10Awight) [18:56:15] awight: I guess this might be a good way forward… otherwise the hole stack will be stuck in limbo [19:00:24] +1 [19:02:03] Btw, shall I move https://phabricator.wikimedia.org/T206346 on the team workboard and close it? [19:11:27] (03CR) 10Hoo man: [C: 032] "While this still should be reviewed by a DBA at some point, we shouldn't block development on this (the extension is not deployed anywhere" [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [19:17:55] (03Merged) 10jenkins-bot: Secondary schema for JADE indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [19:20:24] (03CR) 10jenkins-bot: Secondary schema for JADE indexes [extensions/JADE] - 10https://gerrit.wikimedia.org/r/456078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [19:40:26] awight: I also created http://jade-demo.wmflabs.org/mediawiki/index.php/Judgment:Revision/4438 as a parallel [19:41:04] diffs (damaging, goodfaith); revision (contentquality); am I missing anything? [19:48:33] this demo wiki is a good idea; it helps give structure to the thoughts that I was already having, which is "we need to do a much better job surfacing the features of this extension" [19:51:03] harej, one thing I think prateek could work with is a list of locations where we ought to surface JADE :) [19:51:25] His gadget represents a lot of progress there [19:51:49] At some point I'll need to resume those conversations and get some testing on the calendar [19:52:07] -back-, sorry I forgot to away during a meeting [19:56:20] 10JADE, 10Scoring-platform-team (Current), 10SpamBlacklist, 10MW-1.33-notes (1.33.0-wmf.1; 2018-10-23), 10Patch-For-Review: SpamBlacklist integration for JADE - https://phabricator.wikimedia.org/T206255 (10awight) [19:56:27] 10JADE, 10Scoring-platform-team (Current), 10MW-1.33-notes (1.33.0-wmf.1; 2018-10-23): Support wikitext in judgment notes and endorsement comment fields - https://phabricator.wikimedia.org/T206346 (10awight) 05Open>03Resolved [19:57:36] harej: I'd love any input on the .wiki.mustache template, I think the presentation could be better. [19:57:47] We can also include a CSS and JS module... [20:02:11] Plain and straightforward is probably the best approach here, but I'll let you know if I think of something. [20:02:34] One thing is that I think preferred judgments should be made more prominent, somehow. Right now the preferred judgment and non-preferred judgments are visually given equal weight. [20:03:00] +1, the asterisk is going to piss people off IMO [20:03:12] Because there's no corresponding footnote 8D [20:05:37] Yeah, I was initially confused by the asterisk as well [20:06:27] Is there a convention for how to mark the consensus after a !vote? [20:06:44] Wikipedia often uses a green page filling box [20:06:50] and the consensus as the top [20:08:47] I'm not seeing one yet... but yeah IMO that's the visual language we should be referring to. [20:09:27] Like this https://meta.wikimedia.org/wiki/Requests_for_comment/Global_ban_of_INeverCry [20:09:58] this is meta, but enwiki (at least used to) does it the same way [20:11:13] hmm that is a lot of green [20:12:08] Yeah… it's probably not quite the best UX wise :S [20:21:00] 10JADE, 10Scoring-platform-team, 10Design: Make JADE "preferred" consensus clear; borrow from !vote styling - https://phabricator.wikimedia.org/T207415 (10awight) [20:27:19] offically TODOing that [20:32:42] OMG. I think I have it figured out. [20:32:49] This javascript batching ORES thingie. [20:34:21] It's going to look like you're sending ORES a massive pile of requests, but you're really batching and parallelizing! Mwahahaha! [20:36:07] halfak: Didn't you have a requirement that the client starts getting updates as soon as the first response comes back? [20:36:13] Yes. [20:36:32] kk [20:36:32] I'll have it all. If I'm right, you won't even need to package up context with your request. [20:36:58] You'll just be able to treat the OresScoreBatchPool as though it is sending a billion one-off requests. [20:45:13] (03CR) 10Hoo man: [C: 032] Link table model (031 comment) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [20:52:05] (03Merged) 10jenkins-bot: Link table model [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [20:54:47] (03CR) 10Awight: Link table model (031 comment) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [21:02:33] awight: yup I 've informed the DBAs [21:03:05] akosiaris: Thanks! I'll update the task with a pointer to where it can be reviewed [21:06:51] (03CR) 10Hoo man: [C: 032] Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [21:07:01] 10JADE, 10Scoring-platform-team (Current), 10DBA, 10Operations, and 4 others: Write our anticipated "phase two" schemas and submit for review - https://phabricator.wikimedia.org/T202596 (10awight) @Marostegui We've merged the DDL to our repo in order to unblock development, so here are the new coordinates... [21:13:24] (03Merged) 10jenkins-bot: Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [21:16:02] (03CR) 10Hoo man: [C: 031] "I want to play around with this a bit before merging, but looks good at a glance." (031 comment) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466808 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [21:16:31] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468439 (owner: 10L10n-bot) [21:29:09] It's alive! [21:29:14] OMG. [21:29:38] Amir1, if you're still around, I got the crazy batching thing to work. You're gonna like this. It's crazy. [21:30:33] https://meta.wikimedia.org/wiki/User:EpochFail/ArticleQuality-system.js [21:30:54] See the OresScoreBatchPool at the top. That's where all of the magic happens. [21:31:29] You can basically call score() as many times as you want. E.g. in a loop or in other functions. And it'll just queue and batch process the jobs as needed. [21:32:02] The batch processing works by starting up worker functions that process a single batch and then recurse if necessary. [21:32:15] If no recursion is necessary (queue is empty) they shut down. [21:32:29] Every time you add something to the queue, a notification bumps the threads back to life. [21:32:53] I put a little delay into the worker start-ups so that there's a chance for the queue to fill up before the first batches start being cut and that seems to work magnificently. [21:33:17] I think we can all sneeze at 50ms of startup time. [21:33:17] :) [21:34:56] And with that, I'm AFK. Have a good evening, folks! [21:36:04] Woo I'm so excited about getting that working :) [21:40:31] 10ORES, 10Scoring-platform-team (Current), 10Patch-For-Review, 10User-Ladsgroup: Silence or address E_WOULDBLOCK warning - https://phabricator.wikimedia.org/T152012 (10Ladsgroup) This is number of scores processed by celery: {F26624764} The first spike is when there was one precaching enabled. The second s... [21:53:42] (03PS1) 10Ladsgroup: Increase verbosity of logs and use redis for task tracking [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468487 (https://phabricator.wikimedia.org/T152012) [21:59:40] (03CR) 10Ladsgroup: [V: 032 C: 032] Increase verbosity of logs and use redis for task tracking [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/468487 (https://phabricator.wikimedia.org/T152012) (owner: 10Ladsgroup) [22:12:12] I'm done for the day, I let ores with redis task tracker work until Monday [22:12:26] Also, tomorrow I'm mostly off to reduce the overtime [22:12:36] (As I said in the staff meeting) [22:14:47] Amir1: Just to clarify, it's not on production right? [22:14:55] nope [22:15:03] kk! [22:15:10] Good luck not working for 3 days :p [22:16:25] 10JADE, 10Scoring-platform-team (Current), 10MW-1.33-notes (1.33.0-wmf.1; 2018-10-23), 10Patch-For-Review: Validate global user ID; revisit user schema - https://phabricator.wikimedia.org/T206573 (10awight) 05Open>03Resolved [22:16:42] 10JADE, 10Scoring-platform-team (Current), 10SpamBlacklist, 10MW-1.33-notes (1.33.0-wmf.1; 2018-10-23), 10Patch-For-Review: SpamBlacklist integration for JADE - https://phabricator.wikimedia.org/T206255 (10awight) 05Open>03Resolved [22:16:54] 10JADE, 10Scoring-platform-team (Current), 10Patch-For-Review: Avoid polymorphic database columns in the JADE schema - https://phabricator.wikimedia.org/T203037 (10awight) 05Open>03Resolved [22:16:57] 10JADE, 10Scoring-platform-team (Current), 10Operations, 10TechCom, and 3 others: Deploy JADE extension to production - https://phabricator.wikimedia.org/T183381 (10awight) [22:19:25] 10JADE, 10Scoring-platform-team (Current), 10Discovery-Search, 10Elasticsearch: Extract judgment data for search indexing - https://phabricator.wikimedia.org/T206352 (10awight) [22:29:49] (03CR) 10jenkins-bot: Link table model [extensions/JADE] - 10https://gerrit.wikimedia.org/r/468078 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight) [22:37:05] (03CR) 10jenkins-bot: Hooks to maintain judgment link tables [extensions/JADE] - 10https://gerrit.wikimedia.org/r/466806 (https://phabricator.wikimedia.org/T202596) (owner: 10Awight)