[08:27:56] 10ORES, 10Scoring-platform-team (Current), 10Operations: ORES uwsgi consumes a large amount of memory and CPU when shutting down (as part of a restart) - https://phabricator.wikimedia.org/T242705 (10Gilles) Do you think this is Flask-specific? How hard would it be to port this code to another Python micro WS... [13:28:31] 10ORES, 10Scoring-platform-team (Current), 10Operations: ORES uwsgi consumes a large amount of memory and CPU when shutting down (as part of a restart) - https://phabricator.wikimedia.org/T242705 (10Halfak) We've been testing with `gunicorn` as well. The stackoverflow question has notes about running the te... [14:44:06] 10Scoring-platform-team (Current), 10drafttopic-modeling: Why does loading the drafttopic models take so much memory? - https://phabricator.wikimedia.org/T250435 (10Halfak) # Testing memory maps We use the `mmap="r"` argument to instruct gensim to load our vectors as a memory map. This has lots of theoretica... [14:46:51] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Build article quality model for ptwikipedia - https://phabricator.wikimedia.org/T246663 (10Halfak) https://github.com/wikimedia/articlequality/pull/115 [18:01:06] posting our async update notes -- [18:01:40] halfak - [18:01:42] Y: I have a bunch of things up for review. https://phabricator.wikimedia.org/T249917 & https://phabricator.wikimedia.org/T246663 & https://phabricator.wikimedia.org/T246668 & https://phabricator.wikimedia.org/T249559 I spent some time exploring memory usage issues with out topic models. See https://phabricator.wikimedia.org/T250435 Isaac and I selected a short-list for outreachy candidates. chtnnh [18:01:44] and a few product engineers and I put together a plan for the NSFW classifier. [18:01:46] T: I started working on memory usage in uwsgi/topic models issues again -- specifically experimenting with memory-maps vs raw memory usage. No wins there. I have another VP interview today. After that, I hope to wrap up some topic model memory usage exploration work. [18:01:55] kevinbazira - [18:01:58] Y: [18:01:59] Fixed double endorsement on facet [18:02:01] User A was able to endorse a proposal in a facet, then was also able to create and endorse a new proposal. [18:02:03] This caused User A to endorse 2 proposals in the same facet. [18:02:05] This patch fixes this issue. Now User A is not able to create and endorse a new proposal if they've already endorsed a proposal in the same facet. [18:02:07] T: [18:02:09] Finally fixed a phan issue that persisted from yesterday about userAlreadyEndorsed taking a non-nulled array. [18:02:11] Thanks to Andy who helped with ideas that we built upon. [18:02:18] haksoat - [18:02:20] Y: [18:02:22] I took a deeper look at the built-in Elasticsearch tokenizers. Looks like the pattern tokenizer best suites our use case. Tried getting our generated regex from the deltas package to work on Elasticsearch. [18:02:24] T: [18:02:26] Continue with getting the regex to work on Elasticsearch, struggling to convert the expressions to something suits on Java's regex (Elasticsearch is written in Java), but I hope to have huge progress before the day runs out. [18:02:28] and me - [18:02:35] Y: Did some code review for Kevin related to the double endorsements bug, also continued re-enabling Jade hooks related the Link summary classes [18:02:36] T: More of the same, mostly will be working on the adhoc solution for Jade db schemas and integrating with the DatabaseSchema hooks [18:35:08] 10Scoring-platform-team (Research), 10Structured-Data-Backlog, 10artificial-intelligence: Implement NSFW image classifier using Open NSFW - https://phabricator.wikimedia.org/T214201 (10Chtnnh) The new service request has been filed (T250110) and if you have any input on it please feel free to comment on the... [19:24:35] Hello halfak [19:24:59] I need help knowing what r'\U00026100-\U000275FF' indicates and others in that form [19:25:30] That is a unicode range. I identified those by going through documentation about various character sets. [19:25:30] I am only able to identify unicode like r'\u3041-\u3096' with a small u [19:25:42] Aha! looks like this is hex. [19:26:04] The ones with U right? [19:26:45] Looks like \U00026100 is ord 155904 [19:26:58] Also is it possible to get a link to the docs? [19:27:15] "The \U escape sequence is similar, but expects 8 hex digits, not 4" [19:27:23] https://docs.python.org/3/howto/unicode.html [19:27:26] Okay [19:27:28] Thanks [19:27:33] No problem :) [19:38:13] o/ Helder around? [19:38:24] I'm looking to finish up https://github.com/wikimedia/articlequality/pull/115 [19:39:17] There's one outstanding issue. We still match "{{Carece de fontes/bloco/doc}}" but I think that is a minor issue. If you agree, we can merge. If not, let's chat. [19:39:18] 10[1] 04https://meta.wikimedia.org/wiki/Template:Carece_de_fontes/bloco/doc [19:39:24] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Build draft quality model for ptwikipedia - https://phabricator.wikimedia.org/T246667 (10Chtnnh) https://github.com/wikimedia/draftquality/pull/36 Model has been built 😄 [19:48:54] 10Scoring-platform-team, 10revscoring, 10artificial-intelligence: Template matching regex features only matches prefix - https://phabricator.wikimedia.org/T250522 (10Halfak) [19:49:55] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Build draft quality model for ptwikipedia - https://phabricator.wikimedia.org/T246667 (10Halfak) a:03Chtnnh [19:51:37] I want to get this on beta today, but it might be a stretch. I'll start working on cutting a version of revscoring adap. [19:51:40] *asap [19:51:53] chtnnh, once we have this deployed on beta, people can start testing it. [19:52:08] wouldnt that be amazing :D [19:52:13] A prod deployment on Monday isn't out of the question. [19:52:33] chtnnh is elated [19:52:50] xD [19:54:29] let me know if i can do anything to help with beta on this halfak [20:23:37] wikimedia/revscoring#1901 (master - c278821 : Aaron Halfaker): The build was broken. https://travis-ci.org/wikimedia/revscoring/builds/676377883 [20:24:42] >:( [20:25:43] * halfak restarts the build. [20:25:47] That shouldn't have failed. [20:30:00] Looks like build completed successfull this time, travis. [20:30:03] Where are you now? [20:34:21] xd [20:34:40] *xD [20:46:20] halfak, is this the test that was missing? https://github.com/wikimedia/revscoring/pull/483/files#diff-eb8a1a946669e9299ead6f634defe88dR138 [20:46:38] (still untested) [20:47:04] Yes. You'll want to add raw words to watch examples in a "WORDS_TO_WATCH" variable above too. [20:50:28] hmmm.. I missed that [21:19:48] wikimedia/revscoring#1905 (master - 9fbf5eb : Aaron Halfaker): The build has errored. https://travis-ci.org/wikimedia/revscoring/builds/676398973 [21:20:21] wikimedia/revscoring#1907 (fix_fetch_text - 56cdb03 : Aaron Halfaker): The build has errored. https://travis-ci.org/wikimedia/revscoring/builds/676399278 [21:20:57] sssh travis. [21:24:06] wikimedia/revscoring#1910 (master - 9569c60 : Aaron Halfaker): The build passed. https://travis-ci.org/wikimedia/revscoring/builds/676400236 [21:39:52] halfak, Travis is happy with my patch now: https://github.com/wikimedia/revscoring/pull/483 [21:40:03] Anything else? [21:40:53] Helder, merged! Thank you! [21:40:57] Nice work [21:41:03] \0/ [21:42:08] (03PS1) 10Halfak: Adds ptwiki article and draftquality [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/589713 [21:42:26] accraze, have a couple minutes for https://gerrit.wikimedia.org/r/#/c/mediawiki/services/ores/deploy/+/589713 ? [21:43:00] 10Scoring-platform-team (Current), 10articlequality-modeling, 10draftquality-modeling, 10artificial-intelligence: Mid-April 2020 ORES deployment - https://phabricator.wikimedia.org/T250536 (10Halfak) [21:43:29] 10Scoring-platform-team (Current), 10articlequality-modeling, 10draftquality-modeling, 10artificial-intelligence: Mid-April 2020 ORES deployment - https://phabricator.wikimedia.org/T250536 (10Halfak) [21:43:31] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Build draft quality model for ptwikipedia - https://phabricator.wikimedia.org/T246667 (10Halfak) [21:43:33] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Build article quality model for ptwikipedia - https://phabricator.wikimedia.org/T246663 (10Halfak) [21:43:36] 10Scoring-platform-team (Current), 10articlequality-modeling, 10draftquality-modeling, 10artificial-intelligence: Mid-April 2020 ORES deployment - https://phabricator.wikimedia.org/T250536 (10Halfak) https://gerrit.wikimedia.org/r/#/c/mediawiki/services/ores/deploy/+/589713 [21:43:55] (03PS2) 10Halfak: Adds ptwiki article and draftquality [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/589713 (https://phabricator.wikimedia.org/T250536) [21:44:39] If not, no big deal. I think I'll head out now, but I'll swing back later. If things are merged, I'll send 'em to beta and we can target a deployment window on Monday :) [21:44:43] Take care y'all