[01:10:19] apergos: I'm realizing that T209737 would benefit from your dumps-related knowledge, but it's late there. [01:10:19] T209737: Backfill ORES Hadoop scores with historical data - https://phabricator.wikimedia.org/T209737 [01:10:52] This is about recalculating old revision scores with new ORES model versions, as they come out. [01:11:33] So for example, I'm thinking we want to include the most complete model version's scores in a dump, rather than preferring the newest possible model version. [01:11:48] Maybe ping me tomorrow to share your thoughts! [02:45:26] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review: Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10awight) [04:30:25] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review: Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10awight) Something tricky I ran into: success files aren't written for hours where there are zero changeprop events through codfw. Maybe we ha... [06:27:58] PROBLEM - ORES web node labs ores-web-02 on ores.wmflabs.org is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 325 bytes in 0.023 second response time [06:50:49] awight [06:51:31] do 'histrical' dumps and 'currnt' ones, if that makes sense [07:04:34] RECOVERY - ORES web node labs ores-web-02 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 540 bytes in 0.099 second response time [07:42:28] people who analyze the impacts of various scores on wp revisions will want the historical data; people who just want to use the last scores for something, or to set up a mirror with the latest info will wnt the current data [07:43:11] you should also be thinking about how we can tell if historical data has changed (so we can reuse old dump files in creating new ones) [07:43:29] typically such changes, for revisions, are caused by deletion only; I don't know what the case will be for ores scores [13:49:31] (03CR) 10Hoo man: [C: 03+2] Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) (owner: 10Awight) [13:52:56] 10Jade: JADE allows leading zeros in ids in Judgment titles - https://phabricator.wikimedia.org/T213816 (10hoo) [13:58:05] (03Merged) 10jenkins-bot: Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) (owner: 10Awight) [14:02:21] (03CR) 10jenkins-bot: Summarize preferred judgment values in link table [extensions/JADE] - 10https://gerrit.wikimedia.org/r/475932 (https://phabricator.wikimedia.org/T200297) (owner: 10Awight) [14:02:51] (03CR) 10Hoo man: Add indexes to filter by judgment value (032 comments) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476447 (https://phabricator.wikimedia.org/T200297) (owner: 10Awight) [14:21:57] (03CR) 10Hoo man: [C: 04-1] "While this is not important now, it might become once someone attempts to use wikitext in these messages." (031 comment) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/479578 (https://phabricator.wikimedia.org/T211346) (owner: 10Awight) [14:44:08] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review: Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10Ottomata) Yeah, this isn't the first time we've had this problem. It isn't actually that easy to solve, because the Kafka consumer doesn't ad... [15:00:34] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review: Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10Nuria) So i understand, the expectation here will be that files are written for all hours but empty for those of which there was no data? [16:11:51] o/ [16:11:56] o/ [16:11:57] oops, I forgot to /away lat night [16:12:05] *lat = last or late [16:12:58] awight, I'm going to run away at noon PST [16:13:09] I submitted a last minute vacation request so I can go skating today. [16:13:36] We've been having warm/cold/warm/cold weather which means all of the local lakes are pristine ice. I wanna go skating this afternoon. [16:13:39] halfak: Sounds great! [16:13:52] While it is still light out. [16:14:21] haha can't have it all [16:14:36] pristine ice; but subzero air temp and no light [16:15:14] and everyone in a perpetual state of hung-over, from my experience in cold-weather communities [16:16:19] Na. MSP is much more dry than the rural areas around here. It's more common to go to a brewery and have one or two nice beers than to go get smashed. I heard DC is crazy WRT getting drunk/hung-over. [16:17:00] Wisconsin and rural Minnesota has a bit of reputation for drinking. [16:17:38] I think the whole micro-brew culture has been undermining that though. [16:17:46] In a good way :) [16:18:36] too expensive to get really sick [16:20:03] Yeah. Seems like a part of it. Then again, there are those 12% imperial stouts. :| [16:21:52] * awight shudders [16:21:57] * halfak tries to work out how to control preferred status for a proposal. [16:22:10] Do we have a good verb for that? [16:22:16] not yet [16:22:18] "prefer" [16:22:23] "promote" [16:22:24] nominate for consensus... [16:22:29] too verbose :) [16:22:32] + [16:22:33] 1 [16:22:35] Also, not final enough. [16:22:42] "set consensus" [16:22:45] "flag consensus" [16:23:04] c.f. wikidata? [16:23:54] I say run with "promote" [16:24:40] Cool. I'll do that for now in the wireframes :) [16:24:53] "prefer" sounds too subjective, when we need to emphasize that you should be reflecting an existing consensus [16:25:20] +1 [16:26:28] Wikidata is careful not to verbify :) https://www.wikidata.org/wiki/Help:Ranking [16:27:33] 10ORES, 10Scoring-platform-team (Current), 10Analytics, 10Patch-For-Review: Wire ORES recent_score events into Hadoop - https://phabricator.wikimedia.org/T209732 (10awight) >>! In T209732#4881126, @Ottomata wrote: > We could emit a single test event per hour into the topic in each dc... :) That works for... [16:36:13] 10Jade, 10Scoring-platform-team (Current): JADE allows leading zeros in ids in Judgment titles - https://phabricator.wikimedia.org/T213816 (10awight) a:05hoo→03None Good find, thanks! [16:36:26] hoo: ^ oops, did you claim that task intentionally? [16:36:34] And, thanks for all the cR! [16:37:29] 10Jade, 10Scoring-platform-team (Current): JADE allows leading zeros in ids in Judgment titles - https://phabricator.wikimedia.org/T213816 (10Harej) Do we, as a general rule, want to prevent the creation of Jade pages that do not correspond to legal revision IDs? [16:38:48] Special:Diff/012345 works. [16:39:38] halfak: exactly, we just need a bit more validation [16:39:39] We should not allow it though, I agree. [16:40:08] assert str(int(x)) === x, maybe [16:40:22] that also handles 12345.0 [16:42:19] Special:Diff resolves to a different URL so it makes sense the input would be liberal; this is about page titles which should be more conservative [16:44:05] 10Jade, 10Scoring-platform-team (Current): JADE allows leading zeros in ids in Judgment titles - https://phabricator.wikimedia.org/T213816 (10awight) >>! In T213816#4881623, @Harej wrote: > Do we, as a general rule, want to prevent the creation of Jade pages that do not correspond to legal revision IDs? For s... [16:44:07] awight/harej: I'm not putting a comment/summary field into my editing wireframes with the assumption that we'll do something like a VE-popover save dialog that includes an edit summary field. Sound good? [16:44:25] Make sure to write that down somewhere [16:44:53] halfak: harej: ah thanks, I had missed that we were talking about "Special:Diff". I agree with harej about that. [16:45:23] halfak: ooh I really loathe the popup save dialog. [16:45:42] Hmm. Wikidata doesn't even give you the option of providing a summary ^_^ [16:45:49] O_O [16:46:00] in Wikidata, the summary is pretty much written for you [16:46:02] let's move out of the Jones's neighborhood [16:46:30] we can also write summaries. But I think some actions -- like consensus changes -- should have a custom summary. [16:49:01] There's a nice convention we can borrow from section editing, where the automatic summary is written in a /* C-style comment */ and the custom notes can follow [16:58:42] awight, yeah. I like that for consensus changes but maybe other changes don't really need a custom summary. [16:59:08] OK I'll add the field to consensus changes for now, but we can continue to discuss :) [17:02:34] (03CR) 10Awight: "I think the selectivity concerns are legitimate, but there's a decent chance we do get some benefit from the indexes so my preference is t" (031 comment) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476447 (https://phabricator.wikimedia.org/T200297) (owner: 10Awight) [17:03:02] +1 that endorsement doesn't need a summary [17:03:08] Hmm. Maybe edits to a proposal needs a summary. [17:03:13] +1 [17:03:32] I'm going to leave this like it is for now. I'll post on the task about this issue. Maybe harej can think about it for me :) [17:04:00] I think I see what you mean though, that if you're adding a judgment.notes, it's a bit odd to also include an edit summary. [17:04:01] And we can get a statement about where we want summaries editable and how for this initial integration. [17:04:06] In most cases [17:04:28] awight, yeah. On creation, it doesn't make much sense to me. But on edit, it makes a lot of sense. [17:05:10] E.g. "added a link to [[WP:NOT]]" or "Removed #badgrammer per discussion on the talk page." [17:05:11] 10[1] 04https://meta.wikimedia.org/wiki/WP:NOT [17:05:25] Last meeting of the day is starting for me ^_^ [17:05:33] yep. I can think of cases where it would make sense on creation, probably the exception rather than the rule, though. For example, a summary of "I was asked to comment here by [[User:Timid]] [17:05:34] 10[2] 04https://meta.wikimedia.org/wiki/User:Timid [17:07:15] lol @ User:Timid. Nice :) [17:07:26] (03CR) 10Awight: Kludge to localize string constants in template (031 comment) [extensions/JADE] - 10https://gerrit.wikimedia.org/r/479578 (https://phabricator.wikimedia.org/T211346) (owner: 10Awight) [17:08:37] (03PS18) 10Awight: Add indexes to filter by judgment value [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476447 (https://phabricator.wikimedia.org/T200297) [17:08:39] (03PS14) 10Awight: Throw a fit and s/JADE/Jade/ [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476771 (https://phabricator.wikimedia.org/T211046) [17:08:41] (03PS5) 10Awight: Kludge to localize string constants in template [extensions/JADE] - 10https://gerrit.wikimedia.org/r/479578 (https://phabricator.wikimedia.org/T211346) [17:10:01] "TimId" would have been cooler [17:14:48] hoo: I'm open to reconsidering the indexes, but lmk what you think of my justifications in the CR... [17:15:53] What I'm thinking is that we'll probably see something like 90% (number pulled out of arse) damaging judgments, so the index will be helpful for tasks like filtering and reviewing the non-damaging records. [17:16:43] Actually... this deserves a code comment to remind us to review later. [17:17:37] (03PS19) 10Awight: Add indexes to filter by judgment value [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476447 (https://phabricator.wikimedia.org/T200297) [17:17:39] (03PS15) 10Awight: Throw a fit and s/JADE/Jade/ [extensions/JADE] - 10https://gerrit.wikimedia.org/r/476771 (https://phabricator.wikimedia.org/T211046) [17:17:41] (03PS6) 10Awight: Kludge to localize string constants in template [extensions/JADE] - 10https://gerrit.wikimedia.org/r/479578 (https://phabricator.wikimedia.org/T211346) [17:17:57] harej, have you revived the Topic discussion with Search yet? [17:18:01] * halfak is looking fo rit. [17:18:12] I haven’t [17:20:07] halfak: Uh, oh--looks like this conflicts with your recent work? https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/ORES/+/483338/ [17:21:37] This is OK. I think the core idea is important and it's good to have code that demonstrates it. We should fit it into this in a wholistic way. [17:23:12] Also, I still need to implement some of the patterns we discussed in the task. E.g. hiding the model-pool logic. [17:23:27] Been putting that off for Jade work this quarter ^_^ [17:26:46] * awight high-five [17:49:09] gargle. Why do MediaWiki unit tests never work for me after a few weeks of quiescence? [17:49:30] My least favorite detail is that they usually fail to run silently, with no log output anywhere. [17:50:05] Silent but deadly (to uptime) [17:52:33] (03PS1) 10Awight: Validate for canonical page title [extensions/JADE] - 10https://gerrit.wikimedia.org/r/484514 (https://phabricator.wikimedia.org/T213816) [17:52:35] moving a car, back in 20min [17:58:06] Looks like I had a couple of meetings get canceled, so I'm looking to head out a little earlier than planned for my half-day ski adventure. [17:58:16] harej, anything you need from me before I go? [17:58:38] Not at this point. Have anything else for me? [18:01:13] Just some updates to the edit view. [18:01:13] https://docs.google.com/drawings/d/1fP-4Tmn4bDDLWKD_Uu_W0xQNM-QyRc4LDKQTkKNp_fc/edit [18:01:23] Includes a comment field for changing the consensus. [18:02:04] I'd like to know where else we should allow for custom comments and/or if you think the popover/modal pattern of VE is fine for edit summaries. [18:02:21] Any thoughts you have would be helpful. [18:02:39] awight and I had a brief discussion about it in the scrollback. [18:08:49] OK I got all my emails cleaned up. I'm going to go AFK for my afternoon. I'll check back in around 2200 UTC. [18:08:55] o/ [18:08:59] srrodlund, that means I'll miss the docs meeting. [18:09:01] o/ [18:09:37] okie dokie [18:32:07] srrodlund: if I’m late it’s because muni service between Stonestown and downtown is apparently very sparse [18:32:49] harej: No problem! Don't rush or anything. I think we'll be pretty informal. [18:33:09] harej: Muni can get fun [18:33:41] Not as fun as the 14 bus at 2 AM [18:33:52] I hope you're not doing that [18:35:44] Exactly once :) [18:35:59] haha wisdom comes easily to you [18:36:44] Late-night Transbay can be thrilling, too. [18:37:17] Waiting on Fremont for an hour while workers power-wash urine into the sky is the highlight [18:37:56] One neat thing about SF after midnight though is watching the fog descend through skyscrapers [21:16:57] harej: Not sure if you're on scoring-internal, but I need to disappear for the rest of the day. Sorry to abandon! [21:17:05] There's a scoring-internal? [21:17:36] :) [21:17:41] Isn't that just called you and Aaron sending emails back and forth to each other? LD [21:17:44] harej: OIT can add you, I believe [21:17:46] hahaha [21:17:49] no it's also Amir1 [21:17:54] you should be on the list [21:17:54] In the meantime, can you forward me the email? [21:17:57] I'll email OIT [21:18:33] harej: oh actually, it's the listserv. https://lists.wikimedia.org/mailman/listinfo/scoring-internal [21:18:53] I'm an admin so can approve if you need perms to subscribe. [21:19:07] gotta run!