[03:43:08] Scar(y|ed) YuviPanda: https://www.flickr.com/photos/ragesoss/19982819646/in/album-72157655939543150/ [03:43:46] Meanwhile, halfak plays it cool: https://www.flickr.com/photos/ragesoss/19821250680/in/album-72157655939543150/ [04:04:24] :> [07:47:19] basile, I object. I did not witness this. [09:30:47] ragesoss: you should add a wikimania tag, or how can people find the photos? :( [10:24:17] ToAruShiroiNeko: do you see something at https://photos.google.com/search/wikimania [11:11:30] Nemo_bis nop [11:32:57] ok [11:33:11] ToAruShiroiNeko: asked at http://webapps.stackexchange.com/questions/80755/search-public-images-in-google-photos in the meanwhile [11:35:37] Oh I would have loved to watch that show [11:35:43] unfortunately I was too far [11:35:49] chained away by need for power [11:35:55] 110V [11:35:59] not OTHER kind of power [11:36:24] https://picasaweb.google.com/lh/view?feat=tags&psc=G&filter=0&tags=wikimania#6171605560038421762 <- I can totally see myself [15:20:13] Nemo_bis: I'll get them onto Commons soon. [15:49:48] Sweet :) [16:04:41] guillom, \o/ good photo :D [16:04:59] halfak: Congratulate ragesoss :) [16:05:06] Oh yeah. Just noticed [16:58:37] o/ notconfusing__ [16:59:02] halfak, yes? [16:59:27] 50% just saying hi and 50% wanting to ask you about celery/futures for cocytus. [16:59:44] :) [17:03:03] notconfusing__, ^ [17:03:14] Any progress with futures/celery [17:03:19] we reimplemented it with futures [17:03:25] Coool! [17:03:26] halfak, ^ [17:03:26] And? [17:03:34] it works and is much simpler [17:03:37] \o/ [17:03:40] Great! [17:03:49] but i think its all happening in one system thread [17:04:10] so how do you have it on a dual-core machine [17:04:11] Should be OK since the CPU intensive diff happens on mediawiki servers, right? [17:04:20] Ahh... When I do multicore, I use celery [17:04:27] I see [17:04:30] Same concept as a future, but distributed processing. [17:04:35] right [17:04:42] You'll need to manage a client and a server (pretty easy, honestly) [17:04:54] You can have multiple servers to host more workers if you need it. [17:05:14] But I think it is better if it is not CPU intensive to just use futures. [17:05:31] If you start maxing out a CPU, I'd like to take a look at implementing it in celery. [17:05:47] You could use the saem architecture that we're using for ORES. [17:05:47] it didn't seem like it was maxing out CPU [17:05:59] I don't think it will. [17:06:14] Esp. since you're just *looking* at the diff that the API returns. [17:06:22] right [17:06:24] If you needed to parse the whole page -- different story. [17:06:30] precisely [17:06:32] :) [17:09:03] if i want to make it a tool that allows users to register their own alerts then ill probably have to make it celery [17:09:22] notconfusing__, yeah. That'll be an interesting problem. [17:09:31] I've been looking into standing up socket.io servers. [17:09:45] but im not even entirely sure its a useful tool, its more a solution looking for a problem right now [17:09:49] It seems like having people listen to events and source their own computational framework would be a good first step. [17:10:21] Having the computation system integrated could get complicated fast. [17:10:45] Needs sandboxing and all that. [17:11:17] But I might write a service that looks up DOIs and does something cool when a CSCW paper gets cited in Wikipedia. [17:11:33] All I need is a feed of new DOIs [17:11:47] And a x-small instance on labs [17:11:51] *an [17:15:08] hmm [19:50:58] halfak: I have been thinking of building a simple rcstream processing proxy framework [19:51:21] Reads rcstream proxy does some processing on it (augment / filter) and reemits it [21:01:36] YuviPanda, +1 [21:02:16] e.g. https://meta.wikimedia.org/wiki/File:MediaWiki_Events_Conceptual.svg [21:02:26] So I could listen to RCstream and emit reverts. [21:02:43] Or emit probably good-faith new editors. Or edits that probably need to be reverted. [21:03:01] And someone else could listen to these events to build next generation quality control tools -- or something I haven't thought of? [21:27:23] halfak: indeed and I am thinking of a nice setup where Kafka is the entire thing that filters / augmenters consume, and at the end there is a websocket / socketio proxy for web apps / client side js / external tools [21:28:07] halfak: and you can just write your code in any language and just do Kafka jnput and output [21:28:08] Mmm [21:28:31] YuviPanda, we'll want a process by which experimental augmenters can become part of a core/maintained system. Or maybe we don't need that? [21:28:55] YuviPanda, yeah... if we're managing things ourself, then cool, but will we delay the end of the pipeline? [21:28:59] halfak: we do, yes. This is another one of those things for which we have no process yet [21:29:07] halfak: what do you mean by delay the end? [21:29:34] halfak: you will each emit on different topics. reverts on enwiki could go on reverts-enwiki and won't delay anything else [21:29:56] halfak: Kafka also allows you to not miss any events - you can catch up afterwards :D [21:29:56] YuviPanda, makes sense. [21:30:41] halfak: have a nicely designed system in my head :) [21:30:57] I should go eat first tho [21:41:31] L( [21:41:34] *:)