[10:16:26] o/ [11:38:48] o/ Amir1 [11:38:52] hey dude. You around? [11:38:59] o/ [11:39:01] yeah [11:39:08] :) [11:39:10] Handling a CoC case, be back ASAP [11:39:14] kk [11:42:14] * halfak make some coffee [11:53:17] okay, what should we do today [11:56:52] Back! [11:57:12] How about we look at fawiki article quality. I think I should dig into the makefile to see if I can find where the observations are being dropped. [11:57:31] Any other things you think would benefit from some fast iteration? ^_^ [11:59:29] Amir1, ^ [11:59:55] I have some things [12:00:03] Great :) [12:00:04] but let's get this handled first [12:01:27] halfak: I'm pretty sure I added both campaigns [12:01:48] I just rebased your branch FYI :) [12:02:39] Woops. Actually I did it wrong. fixing... [12:03:12] Is now articlequality-ready [12:03:20] yay [12:05:07] Sure enough, you have ~700 obs in the 700 set [12:05:09] hmm [12:05:22] maybe you did some work since I looked at the model generation output? [12:05:51] I didn't do anything with that [12:05:59] Oh I see that there are 250 FA. [12:06:03] And 250 GA. [12:06:18] Now that I think about it, that is an oddly large number [12:06:21] * halfak explores [12:06:25] They all got merged [12:06:35] halfak: because we got them from quarry and merged them [12:08:26] In the human-labeled 300, we expected to get basically zero GA/FA articles [12:08:34] Instead we go 95% GA/FA articles! [12:08:43] Someting got mixed up! [12:08:58] I think maybe the 300 human labels were of the GA/FA article sample O_o [12:09:36] yeah [12:10:02] I don't who did that, maybe we mistakenly mislabeled GA and FA with stub and start [12:10:18] Oh! That'd make sense. [12:10:30] * halfak looks at some examples. [12:10:35] Aha! There is some overlap [12:11:18] https://github.com/wiki-ai/wikilabels-wmflabs-deploy/blob/master/forms/i18n/wp10/fa.json [12:11:25] they are correct [12:11:52] So. Somewhere is definitely weird with the sample we sent for labeling. [12:12:07] btw. We deployed unsourced statements campaigns last week in case you missed it [12:13:19] Oh great! I did miss that :) [12:13:20] Good work [12:13:40] I think this was the query that should have loaded into wikilabels: https://quarry.wmflabs.org/query/25462 [12:13:45] It's 600 obs. [12:13:46] Not 300 [12:14:42] Arg. I think this is the mixup. And it means we need to label 600 more things. [12:14:44] :( [12:14:53] :(((( [12:15:05] https://phabricator.wikimedia.org/T174684#4044579 [12:15:19] https://phabricator.wikimedia.org/T174684#4043323 [12:16:10] On the bright side, the 300 human-labeled observations are a far better window into GA and FA [12:16:24] It looks like some of those articles were actually stubs! [12:17:55] Another bright side: These 600 should go really fast because they'll be lower quality/easier judgement calls [12:18:03] Stub/Start is super easy to call :) [12:18:27] Amir1, you want to get the 600 loaded or should I? [12:18:50] as you wish, I can work on wp10 for mediawiki in the mean time [12:19:54] Great! [12:20:03] * halfak goes to look for somewhere to be productive. [12:59:54] Got stuck doing some program committee work. [13:00:00] *paper reviewing stuff [13:00:20] hi halfak [13:00:28] o/ Vermont [13:14:27] Was gonna check on the draft topic memory usage but it looks like the model won't load. I'll need to do some more work to get that running or maybe I'll ping codezee [13:14:38] I need to leave soon. Amir1, need anything before I walk away. [13:15:00] halfak: oh yeah