[00:01:33] but that will still have 50 rev id per request, 2 requests per seconds, right? [00:07:27] xinbenlv, yes. you can provide the system with any number of rev_ids and it will manage making sure that 50 X 2 requests happen. [00:09:02] I'm out of here for the evening. Will be back on around 1500 UTC [00:09:04] o/ [02:34:46] 10Scoring-platform-team, 10User-Zppix: The future of Wikimedia Scoring Platform's CI for repos - https://phabricator.wikimedia.org/T183294#3850195 (10Zppix) Gitlab runs docker, it seems to be cool, however its mobile interface lacks a good ui, and it appears gitlab doesnt feature an true web editor unlike gith... [07:28:23] PROBLEM - https://grafana.wikimedia.org/dashboard/db/ores-extension grafana alert on einsteinium is CRITICAL: CRITICAL: https://grafana.wikimedia.org/dashboard/db/ores-extension is alerting: Failure rate alert. [07:29:23] RECOVERY - https://grafana.wikimedia.org/dashboard/db/ores-extension grafana alert on einsteinium is OK: OK: https://grafana.wikimedia.org/dashboard/db/ores-extension is not alerting. [12:49:12] awight: could you check your email please? [12:49:28] o/ [12:50:02] Not such a bad answer [12:50:09] Lol [12:50:20] Its not what i wanted to hear [12:50:27] Zppix: weren’t you saying that Gitlab’s CI is docker-based? [12:50:32] It is [12:50:39] Also, we’re migrating to Kubernetes which seems to be as well [12:50:50] But they provide images on gitlab [12:50:50] so as annoying as docker might be, it’s our future [12:51:32] Also fyi, Travis has something to help with caching heavy downloads… lemme see here [12:51:36] awight: should i bother with doing docker atm? [12:51:42] Yes i know [12:52:10] 10Scoring-platform-team (Current): Drafttopic: add article text fetching utility - https://phabricator.wikimedia.org/T183355#3851037 (10Sumit) [12:52:34] 10Scoring-platform-team (Current): Drafttopic: add article text fetching utility - https://phabricator.wikimedia.org/T183355#3851049 (10Sumit) https://github.com/wiki-ai/drafttopic/pull/14 [12:53:00] awight: ill update ticket when i get a chance [12:53:03] Zppix: IMO it’s useful in the medium-term, so any effort you feel like putting into that would be valued. [12:53:13] Ok [12:53:19] Ill look further into [12:53:38] If anything ill borrow docker images from releng :P and customize them from there [12:53:42] I don’t have a sense for how these integrations differ, but my uninformed opinion is that it would be best to have a docker integration which is decoupled from any of the external frameworks. [12:55:45] awight: ok thanks, ill work on it when i can make time, juggling between all my activities is fun :P [12:56:18] Zppix: well understood. Only do however much is fun. [12:56:37] I think learning more about ci is fun [12:56:45] Its not really hard its just effort [12:58:56] +1 I’m partial to writing CI myself [12:59:36] Zppix: I’ve somehow managed to avoid using Docker—would you educate me about how it works? We would be using the host kernel but replacing the entire user space, right? [13:00:04] Docker is essentialy a vm [13:00:24] Think docker is virtualbox [13:00:39] But not a full hardware-level VM, right? So for example we couldn’t run BSD in a docker container? [13:00:40] And the images are the os it will run [13:01:03] That i dont know for sure never tried [13:01:38] I would assume it could be possible but a hack job [13:03:05] if bsd uses the linux kernel and drivers then you can get away with it, otherwise not [13:03:34] eg I can run ubuntu in a docker container on my fedora box but I will have the apparmour/selinux issue, or need to just turn that stuff off [13:03:48] Lol [13:05:45] I believe that bsd derivatives by and large use their own kernel [13:05:52] with some sort of "compatibiliy layer" [13:25:56] apergos: That answers my question. Yeah, the BSD kernel is unrelated to linux. [14:23:33] 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#3851398 (10akosiaris) I 'll start this with a reiteration of some common... [14:39:16] Zppix: fwiw, here’s an example Dockerfile written by releng: https://phabricator.wikimedia.org/source/scap/browse/master/Dockerfile.ci [14:47:28] awight: yes i know [14:52:47] 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#3851468 (10awight) Hi, thanks for your thoughts! >>! In T181071#3851398,... [14:54:06] 10Scoring-platform-team (Current), 10JADE, 10Epic: Deploy JADE API prototype in labs - https://phabricator.wikimedia.org/T176333#3851472 (10awight) [15:00:58] halfak: o/ [15:01:05] Something fun to start the morning with ;-). https://www.mediawiki.org/wiki/Topic:U458eohqqxukl6m5 [15:57:52] 10Scoring-platform-team, 10JADE, 10Epic: Deploy JADE MVP to production - https://phabricator.wikimedia.org/T183381#3851603 (10awight) [16:00:29] 10Scoring-platform-team, 10JADE, 10Design: Design curation/suppression integration with MediaWiki (for JADE) - https://phabricator.wikimedia.org/T183276#3851622 (10awight) [16:00:31] 10Scoring-platform-team, 10JADE: Build consumable dumps of JADE - https://phabricator.wikimedia.org/T183204#3851623 (10awight) [16:00:33] 10Scoring-platform-team, 10JADE, 10Epic: Deploy JADE MVP to production - https://phabricator.wikimedia.org/T183381#3851621 (10awight) [16:00:35] 10Scoring-platform-team, 10JADE, 10Design: Design conceptual prototype of JADE integration with MediaWiki - https://phabricator.wikimedia.org/T182829#3851624 (10awight) [17:44:11] awight: so looking at docker, do we still want to do jessie or do we want to just do stretch? [17:44:32] I can attempt to do both but i want to know what is the absolute min [17:44:41] oh cool. both, ideally, since those are our current and future platforms. [17:44:48] absolute minimum would be just jessie. [17:45:01] Ok [17:45:07] Ill see what i can hack [17:48:42] Okay setting up travis with docker will be difficult... [17:48:53] Hmm [17:49:07] I wonder if we should just hold off til we migrate [17:49:11] awight: ^ [17:49:38] Don’t feel obliged. What part will be difficult? [17:49:56] * awight squints at https://docs.travis-ci.com/user/docker/ [17:50:49] Setting it up to use docker images [17:51:07] It requires setting eviroment vars and such which im uncomfortable with [17:51:25] I mean once thats done im able to do the rest [17:51:42] But i dont feel comfortable setting env vars myself [17:55:05] 10Scoring-platform-team (Current): Drafttopic: Add utility to extract dependents - https://phabricator.wikimedia.org/T183392#3851910 (10Sumit) [17:55:17] 10Scoring-platform-team (Current): Drafttopic: Add utility to extract dependents - https://phabricator.wikimedia.org/T183392#3851923 (10Sumit) https://github.com/wiki-ai/drafttopic/pull/15 [17:56:57] halfak: the two scripts that I've added address text fetching and dependents extraction in drafttopic, I have been managing them in separate branches, it'd be great if you can have a look and merge them [17:57:17] since managing them in two separate branches has become difficult [19:15:40] 10Scoring-platform-team, 10ORES, 10Operations, 10Patch-For-Review: rack/setup/install ores2001-2009 - https://phabricator.wikimedia.org/T165170#3852233 (10Dzahn) So looks like we need a new role class for "regular ores server in production" (regular as opposed to ores::redis). Because we have currently on... [19:23:46] 10Scoring-platform-team (Current), 10ORES, 10Operations, 10Patch-For-Review: Investigate why ORES logs are being written to syslog despite explicit logging config. Fix. - https://phabricator.wikimedia.org/T182614#3852237 (10awight) https://github.com/wiki-ai/ores/pull/243 [19:24:37] awight: if someone could setup the env vars and setup a base docker image i could configure it to our needs [19:25:09] Zppix: That sounds interesting. Feel free to assign me to a subtask. [19:25:53] wiki-ai/ores#896 (console_logging - aa2dabe : Adam Roses Wight): The build failed. https://travis-ci.org/wiki-ai/ores/builds/319335017 [19:27:02] Ok let me make one awight [19:28:07] wiki-ai/ores#898 (console_logging - 4bfe7fc : Adam Roses Wight): The build failed. https://travis-ci.org/wiki-ai/ores/builds/319335661 [19:28:25] back in 10. [19:31:35] wiki-ai/ores#900 (console_logging - 1eca3a7 : Adam Roses Wight): The build was fixed. https://travis-ci.org/wiki-ai/ores/builds/319337539 [19:33:23] 10Scoring-platform-team, 10User-Zppix: Create base docker image and set env vars for Wiki-AI CI - https://phabricator.wikimedia.org/T183410#3852260 (10Zppix) [19:43:28] Zppix: per above, I was mucking around the other night and actually created an ORES docker image :) [19:44:27] TheresNoTime: not what we need what we need is for Continous Integration [19:44:35] Thanks though! [19:55:12] TheresNoTime, pull request? :D [19:55:36] Put it in a docker dir [19:55:42] Pls [19:56:04] It's the first docker image I've created mind you :p [19:56:13] Its 1 more than me [19:56:26] TheresNoTime: yeah, sounds like your Dockerfile would be a great addition to this project :) [19:56:35] awight: i made that task fyi [19:58:16] halfak: This should be the end of the logging work, https://git.io/vbDwb [20:00:15] awight, looking at it now. [20:02:55] awight: I'll PR when I'm back in front of my pc ^^ [20:03:45] halfak: should we do a seperate repo for ci images or just put them in each repo? [20:04:30] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10artificial-intelligence: Experiment with using English Wikipedia models on Simple English - https://phabricator.wikimedia.org/T181848#3852343 (10awight) a:03awight [20:05:28] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10artificial-intelligence: Experiment with using English Wikipedia models on Simple English - https://phabricator.wikimedia.org/T181848#3804284 (10awight) Now throwing a stack trace that no goodfaith model exists in the database... [20:05:40] awight: hi [20:05:42] Zppix: Separate repo please. Is there a standard for any of the CIs you’ve looked at? [20:06:01] Adotchar: o/ slow progress on simplewiki, for some reason it’s been the poster problem child [20:06:13] :/ [20:06:14] awight: not as far as i know... [20:06:27] awight: ill set one up [20:06:32] ty! [20:06:41] Do we need an "image"? [20:06:44] awight: the code/script (idk the difference) that halfak gave me earlier works, so that was the original experiment thing [20:06:50] I thought everything was built from a Dockerfile [20:06:51] awight: anything I can do to help [20:07:47] halfak: good point, there seems to be a centralized registry of docker images, but we wouldn’t be hosting those. [20:08:04] awight: https://github.com/wiki-ai/Continous-Integration-Images [20:08:18] No we want our own [20:08:21] Trust me [20:08:42] Ive tried a few of the registered ones and i wanted to do some things to the creators [20:08:44] what do other people do, Zppix [20:09:03] halfak: they treat ci likes its a vanity item [20:09:13] Adotchar: At the moment I’m mired in glitches specific to Extension:ORES, but you could hit the API manually if you want to play around. I’ll get you an example URL… [20:09:20] I doubt that Zppix [20:09:25] They spend time making it pretty rather than function [20:09:35] Atleast the ones i used [20:09:50] Plus customization is better than being stuck [20:10:22] Adotchar: https://ores.wikimedia.org/v3/scores/simplewiki/12345 [20:10:31] That last number is the revision_id being scored. [20:11:30] awight, I think adotchar is already using the models to score recentchanges with the ScoredRevisions gadget [20:12:32] Halfak awight the new ci image repo is setup with irc notifis and stuff [20:13:20] Zppix, I'm still skeptical that we want to run our own images. [20:13:35] What do other people do when they are doing CI using docker on gitlabs [20:13:35] ? [20:13:40] What does gitlabs recommend? [20:13:45] They make their own [20:13:52] Or we use slow shared gitlab ones [20:14:05] I can maintain them once we have a base one setup [20:14:43] Oh? Which repos did you look at? How do they store their images? [20:15:09] They do seperate repos if they use similar ci setup across their repos like we do [20:15:28] Small one of projects ive noticed put them in a docker dir [20:15:34] One off* [20:15:54] I just randomly looked [20:16:04] I didnt write them down sorry [20:17:31] It would be good to start from an example. [20:17:45] Awight is working on a base image [20:17:49] (I think) [20:18:43] halfak: if your worried about how the images will work its painless the only human interaction needed is the setup mostly [20:19:03] Zppix, I'm worried about making a mess we don't want to maintain. [20:19:16] Its not difficult to maintain either [20:19:27] Following best practices (or even just common practices) is a good idea in all cases. [20:19:34] The most maintain youll need to do is if you need to change something [20:19:46] Besides that it can just sit there and be fine [20:20:19] One image can be used across our repos considering we feed off of each other for the most part [20:21:03] You may have to worry about this image once every like few months really [20:21:14] Even if that [20:21:36] That sounds right. [20:21:42] Again, following best practices is best. [20:22:11] Most people do use their own dockers as they are easy to maintain afaic [20:22:13] We don't want someone to show up to our project and be confused by how we're managing images unless we have a good reason for why we chose to do it that way. [20:22:24] Releng has a seperate docker repo [20:22:28] Zppix, what's one example of "most people"? [20:23:31] Let me grab one [20:23:51] Oracle [20:24:03] https://github.com/oracle/docker-images [20:24:34] Got it. [20:24:43] Looks like the base dir is a bunch of different images. Right? [20:24:56] Yes but we wouldnt need more than 1 [20:25:15] For ci [20:25:23] We might. [20:25:28] Why? [20:25:33] Also, we'll need a docker image for kubernetes. [20:25:44] Im talking docker for CI [20:25:48] We might have a separate image for web/celery workers. [20:25:49] Sure. [20:26:00] But it seems that the design is to put it all in one place. [20:26:20] The reason were doing this is to make ci migration painless [20:26:50] You're also making a docker-images repo [20:27:22] No i created a ci images repo [20:28:21] halfak: Suspicious—the ores-beta service has been reporting overload for days at least, yet it’s quiescent. [20:28:30] I think I’ll investigate rather than restart. [20:30:15] Dec 19 20:04:37 deployment-sca03 celery[31055]: MemoryError: [Errno 12] Cannot allocate memory [20:30:23] celery worker forkbombed itself. [20:30:29] well now that that’s over… I’ll restart. [20:31:39] I’m not going to ask what “celery worker forkbombed itself” because it’s probably something technical that I can’t understand and just sounds funny to a layperson like me. [20:32:19] Adotchar: celery is a pain in the ass, is that a simple enough explanation? :P [20:32:45] Adotchar: lol thank you. See, it has to do with making a salad… [20:33:03] Adotchar: But more seriously, this page is ready to try out, at your convenience: https://simple.wikipedia.beta.wmflabs.org/wiki/Special:RecentChanges [20:33:55] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10artificial-intelligence: Experiment with using English Wikipedia models on Simple English - https://phabricator.wikimedia.org/T181848#3852454 (10awight) I ran CheckModelVersions manually, which brought the database models back... [20:39:48] 10Scoring-platform-team, 10ORES, 10Wikidata, 10Beta-Cluster-reproducible, and 2 others: wikidata.beta.wmflabs.org/wiki/Special:RecentChanges InvalidArgumentException No model available for [goodfaith] - https://phabricator.wikimedia.org/T183266#3852471 (10awight) @Catrope is right, all it took was kicking... [20:47:31] (03PS1) 10Awight: Don't double-quote model version [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399464 (https://phabricator.wikimedia.org/T183266) [21:00:18] (03PS1) 10Awight: Catch another double-quote, this may change the behavior of the three changes feeds. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399465 [21:02:04] (03CR) 10jerkins-bot: [V: 04-1] Catch another double-quote, this may change the behavior of the three changes feeds. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399465 (owner: 10Awight) [21:02:18] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10Patch-For-Review, 10artificial-intelligence: Experiment with using English Wikipedia models on Simple English - https://phabricator.wikimedia.org/T181848#3852513 (10awight) a:05awight>03None @Catrope This is unstalled an... [21:04:41] (03PS2) 10Awight: Catch another double-quote, this may change the behavior of the three changes feeds. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399465 [21:09:44] (03CR) 10Awight: [C: 032] Use ScoreParser instead of Cache::processRevision in ApiHooksHandler (031 comment) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/397597 (https://phabricator.wikimedia.org/T181334) (owner: 10Ladsgroup) [21:09:48] (03CR) 10jerkins-bot: [V: 04-1] Catch another double-quote, this may change the behavior of the three changes feeds. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399465 (owner: 10Awight) [21:12:36] (03Merged) 10jenkins-bot: Use ScoreParser instead of Cache::processRevision in ApiHooksHandler [extensions/ORES] - 10https://gerrit.wikimedia.org/r/397597 (https://phabricator.wikimedia.org/T181334) (owner: 10Ladsgroup) [21:20:06] So after writing more about cramming JADE into the page/revision format, I'm starting to wonder why not use wikibase. [21:20:52] If a page is a collection of judgements about a single wiki-entity, then those judgements could just be property/value pairs. [21:21:30] The problems I see here are that we're using an industrial strength ontology to solve a simple schema problem. [21:21:46] And also that we'll need to write a lot of adapting code to get what we want out of it. [21:21:54] But otherwise, I'm not seeing a big downside. [21:27:02] halfak: I’ve had the urge to use wikibase on a few occasions, and could never convince myself that it’s actually a good idea. But it’s a perfect conceptual mapping, in this case. [21:27:31] and “schema” matches property nicely, AIUI [21:28:08] I guess we don’t have to worry about performance because we still have events flowing into a less botiquey DB [21:28:14] boutiquey. [21:28:18] I blame the French. [21:29:07] If we go that direction JADE would be mostly software for managing the *use* of wikibase. [21:29:13] (03PS3) 10Awight: Catch another double-quote, this may change the behavior of the three changes feeds. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399465 [21:29:32] E.g. making sure that a diff loads at the top of JADE:Diff/123124 [21:30:05] I would want to decouple from any assumptions that wikibase is *the* storage backend, cos IMO that would be prohibitive for anyone else to adopt. [21:30:30] Is there JADE for wikidata claims? :) [21:30:47] Not sure what you mean. [21:32:30] Will wikidata editors want to use JADE to make judgments of wikidata statements such as https://www.wikidata.org/wiki/Q42037 “country of citizenship”? [21:33:17] there we go—a statement linked from recentchanges… https://www.wikidata.org/w/index.php?title=Q46256711&curid=47392680&diff=611484428&oldid=611484396 [21:33:49] this edit was clearly bollocks—I can’t even pronounce that place name. [21:34:01] Im not a experienced wikidata editor so take this opinion with a grain of salt, yes it should [21:34:15] :) Zppix is always down for a thought experiment [21:34:23] ( [21:34:24] Yeah. I guess I could see wanting to judge properties. They're just pages after all. [21:34:31] “down” in the US is “up”) [21:34:46] a property is a page, but I think a statement is not a page [21:34:53] Oh I see. [21:34:57] Hmm [21:35:14] * awight dives down the rabbit hole [21:35:25] Hey man i have plenty opinions to share just ask xD [21:35:39] then there is the business of judging judgments :) That might actually make sense in wikibase [21:36:10] And judging judging judgments? [21:36:19] OK so thinking this through... In order to enable JADE in a wiki, we'd enable wikibase for a namespace and we'd write code to handle title prefixes. [21:36:45] E.g. "Revision/" and "Diff/" will expect a rev_id and will render differently. [21:37:33] We'll recommend a well-defined set of properties based on whether the page is a Diff/ page or Revision/ page. [21:37:40] 10Scoring-platform-team (Current), 10JADE, 10Epic: Implement basic path structure for JADE (judgements) - https://phabricator.wikimedia.org/T181098#3852633 (10awight) This task might be done now? [21:37:45] And based on those properties, a set of statements could be added. [21:37:50] awight, ^ lol [21:38:06] This MW integration via content handler has me upside down. [21:38:31] 10Scoring-platform-team (Current), 10JADE, 10Epic: Deploy JADE API prototype in labs - https://phabricator.wikimedia.org/T176333#3852649 (10Halfak) [21:38:33] 10Scoring-platform-team (Current), 10JADE, 10Epic: Implement basic path structure for JADE (judgements) - https://phabricator.wikimedia.org/T181098#3852647 (10Halfak) 05Open>03Resolved [21:38:34] Definitely "done" [21:38:34] halfak: It’s disturbing, but the nice part about ContentHandler is that we get to define the UI. Which we don’t want to do ;-) [21:38:43] \o/ [21:38:45] haha right. [21:39:07] Though, I think defining a reference UI isn't the worst. [21:39:17] I want to make sure that other alternative UIs are easy. [21:39:57] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10Patch-For-Review, 10artificial-intelligence: Experiment with using English Wikipedia models on Simple English - https://phabricator.wikimedia.org/T181848#3852651 (10awight) @Catrope I should have read the title of the task..... [21:41:09] halfak: Mind dropping a few sentences about this into https://www.mediawiki.org/wiki/JADE/Implementations#Technical_alternatives ? [21:41:30] Not sure i'm ready to do that [21:41:37] Still trying to think it through. [21:41:51] MW really pushes back against design-first approach. [21:42:23] Because we don’t know how deep the puddles are? [21:42:58] Do we want to? [21:43:42] halfak: It might be amusing to tug Amir1’s coat or another Wikidata dev, they seem to be experienced at telling other people that Wikibase isn’t right for their use case... [21:44:21] awight, yeah. Deep puddles and also I feel compromise everywhere :| [21:44:26] awight, +1 for asking Amir1 [21:44:35] haha realpolitikiwiki [21:44:38] Speaking of Amir1, i haven't seen him around since the meeting this morning. [21:44:39] Wikibase is a compromise itself imo [21:44:48] halfak: I think he said it was the end of his day [21:44:49] halfak: I just got back [21:44:52] what's up [21:44:54] oh! [21:44:55] ohey! [21:44:59] good timing. [21:45:08] Amir1, having a crisis. Somehow wikibase is starting to make sense to me for JADE. [21:45:20] LAWL [21:45:32] * awight stays for the peanuts [21:45:49] * Zppix grabs popcorn [21:46:34] halfak: hmm [21:46:37] OK. So, imagine we install wikibase in enwiki. [21:46:52] And set up a namespace for judging things. [21:47:08] E.g. JADE:Diff/1234 would contain a set of judgements about that diff [21:47:23] damaging, goodfaith, edittype, etc. [21:47:27] Good luck getting community approval :P [21:47:37] That's gonna be a thing too. [21:47:53] I honestly want wikibase no where near my baby [21:49:33] OK so reason #1 why wikibase is a bad idea: we'd need federated Wikibase or we'd end up re-defining the property ontology in every wiki [21:49:34] hmm [21:49:51] federation is actually implemented [21:50:03] I have two wikis working in federated mode in labs [21:50:06] Oh. Would we want to use it? [21:50:14] yeah definitely [21:50:19] Interesting. OK. [21:51:15] So with wikibase we can't apply schemas nicely. [21:51:42] E.g. the entity JADE:Diff/1234 could have "damaging: spaceship". [21:51:56] Potential reason #2 ^ [21:53:10] hmm, yeah, there are ways to implement that [21:53:16] constraints [21:53:28] OK reason #3, we'd have to convince enwiki to let us enable it. [21:53:37] halfak: i randomly brought up wikibase in enwiki on irc and no one was keen on it just fyi [21:53:47] ^ that [21:54:04] Could we make it so that JADE *uses* Wikibase but *isn't* wikibase? [21:54:15] You could api? [21:54:22] yeah, technically enwiki is already a wikibase, just a client and not a repo [21:54:36] Sure. It seems like we'd like to have a repo. [21:54:46] Couldnt jade live on wikidata and just use api and stuff to enwiki and such [21:54:50] I was thinking of a central wiki for JADE not per client [21:54:59] otherwise wikibase doesn't make sense [21:55:00] Amir1, that could work. [21:55:24] Amir1, one problem is curation. We'd want to show JADE stuff in enwiki's RC feed. [21:55:35] Or at least the enwiki-relevant JADE stuff. [21:56:14] We do that wikidata [21:56:21] Already [21:56:30] Same concept i would assume [21:56:49] that's actually already possible with the way change dispatching works in wikidata [21:57:01] it's super complex and I rewrote half of it recently [21:57:04] but possible [21:58:00] OK. [21:58:07] Should JADE be a namespace on wikidata.org? [21:58:15] Doesn't seem like it fits... but hmm. [21:58:29] I think it can be even a dedicated wiki [21:58:41] Oh yeah. I forgot about the federation angle. [21:58:42] with federation and stuff [21:58:49] *but* [21:58:57] I have doubts for two things [22:00:01] I think it would have to be its own instance, cos it wouldn’t satisfy wikidata’s notability guidelines & they don’t want a kitchen sink. [22:00:11] 1- writing something like this requires a at least five-full-time-software-engineer team with at least half year of work [22:00:23] /o\ [22:00:27] 2- in matter of database, it definitely will explode [22:00:29] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10Patch-For-Review, 10artificial-intelligence: Experiment with using English Wikipedia models on Simple English - https://phabricator.wikimedia.org/T181848#3852728 (10Halfak) Oh this is done. It has been reviewed by @Adotchar [22:00:44] What [22:00:59] Why would the database explode? [22:03:00] the way wikibase is designed is to make lots of rows for lots of things [22:04:15] Adotchar: LMK if that’s not right and we can reopen, or if you did review then please put some notes on that task… [22:04:30] Amir1, when you estimate that level of dev effort, what do you have in mind? What would these engineers be focused on building? [22:04:49] awight: I reviewed using en-wiki models on simple-wiki. It works. [22:05:14] Adotchar: ah, great thanks! I guess we can tune the threholds later & stuff. [22:05:34] halfak: there is a need to define a new entity type called jade [22:05:58] (we are introducing two new entity type beside item and properties, lexeme and mediainfo) [22:06:32] with its own serializer and deserializer factories, rdf exports and lots of other things [22:06:49] so wikibase repo as whole can work with it [22:07:09] Amir1, I figure that a "jade" could just borrow from item and property. [22:07:20] Kind of like how librarybase did. [22:07:31] in theory yes, but in the codebase, it's a mess :D [22:07:36] Damn. [22:08:10] tbh it might take less effort, I probably need to do a little bit of investigation [22:08:33] Amir1, what would be the best way I could help you think about this easier? [22:08:51] Maybe we should set up a wikibase and try to make an enwiki/Diff/1234 item? [22:09:07] something similar [22:09:15] I will write an extension, really fast [22:09:25] and enable it in a test system [22:10:32] OK. So dominant proposals in my brain: (1) Implement JADE as an API/Service and have an event processor write to a special namespace in client wikis + a JSON content handler. [22:10:36] (2) try this wikibase thing. [22:11:30] (03CR) 10Ladsgroup: [C: 032] Don't double-quote model version [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399464 (https://phabricator.wikimedia.org/T183266) (owner: 10Awight) [22:11:38] ty! [22:12:31] thank you for spotting [22:13:11] (03CR) 10Ladsgroup: [C: 032] Catch another double-quote, this may change the behavior of the three changes feeds. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399465 (owner: 10Awight) [22:13:22] (03Merged) 10jenkins-bot: Don't double-quote model version [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399464 (https://phabricator.wikimedia.org/T183266) (owner: 10Awight) [22:13:36] I read through everything and only found those three… [22:14:06] BTW, do you know if CheckModelVersions is on a cronjob in production? [22:14:59] (03Merged) 10jenkins-bot: Catch another double-quote, this may change the behavior of the three changes feeds. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/399465 (owner: 10Awight) [22:18:13] operations-puppet/modules/mediawiki/templates/jobrunner/jobrunner.conf.erb ? [22:21:05] (A bit more thinking out loud) If we follow the JADE API + Content handler direction, then we wouldn't want someone to directly edit the page on-wiki. We'd rather that they submitted a request directly to the JADE API and that the change showed up as a job. [22:21:18] This would look like editing a specific claim on a wikibase entity. [22:21:43] So our content handler would need to be able to render an editing interface like wikibase. [22:21:46] awight|away: it's not [22:22:02] that means we should keep b/c [22:22:09] I will write a patch for that tomorrow [22:22:17] Amir1: awesome [22:22:53] Alternatively, we could work like EventLogging and let people edit the JSON directly. [22:23:08] E.g. https://meta.wikimedia.org/wiki/Schema:Edit [22:24:24] Sorry, gtg. This is fun to think about, though :) [22:25:09] I will definitely think and comment more, specially on ores isde [22:25:18] but I don't think I have the mentality now [22:25:26] Amir1, no worries. [22:25:39] Please do try to find some time when you've got the brain cycles to spare [22:25:50] I'm pretty burnt out on this as it is. [23:03:31] OK I'm heading out for the day. I'm hoping some sleep is going to cure my thoughts WRT JADE and content handler. See y'all tomorrow!