[13:29:32] o/ schana [13:29:41] hey halfak [13:29:52] Looks like we're rebooting ores redis today. [13:30:06] So it'll be a good opportunity to examine what happens to the workers :) [13:30:08] interesting [13:30:48] Wanna join us in -ai to discuss? [13:30:57] be right there [13:35:46] heya [13:39:31] o/ [13:44:11] how is the land of research [13:44:33] Full of systems engineering these days [14:25:57] what systems are you building, halfak [14:26:13] Guerillero, getting ORES onto production hardware. [14:26:23] :D [14:26:25] exciting [14:26:34] It turns out that this is harder than building the system and configuring it for wmflabs. :\ [14:26:49] how so? [14:27:21] Basically, we're gutting our configuration and deployment process to match what releng was looking for. [14:27:43] Turns out that switching from the best-available-tool to the officially-supported-tool is a little painful. [14:58:38] it will be fab when it is done! [14:58:58] halfak: also fyi, http://paws-public.wmflabs.org/paws-public/ is a real webserver now. so the big file you could not download earlier, you can now [14:59:22] Yay! [14:59:23] halfak: but that means the ipynb files will not render and old links are broken already [14:59:29] lol [14:59:31] halfak: the URL structure is sitll not fully stable, justfyi [14:59:31] booo [14:59:35] kk [14:59:44] halfak: so you can get files out, but links given out will still break [15:00:05] I'm still trying to figure out if I can put in usernames easily and performantly, or if I should just not even bother [16:31:19] yuvipanda, re. usernames, what about just applying MediaWiki's URL normalization pattern? [16:31:30] Convert spaces to "_" and then URL encode [16:31:45] the problem is one of using them on filesystem names [16:31:54] putting unicode in there makes me feel very iffy [16:32:04] URL encode the filesystem names too? [16:32:13] I could use a rewrite from the browser [16:32:15] Maybe they could be IDs in the filesystem [16:32:40] halfak: yeah, so question has been wether I should rewrite the url from username to id [16:32:45] I can probably do that with some lua [16:33:23] but that'll slow down operations [16:34:17] yuvipanda, gotcha. That is rough. [16:34:19] * halfak thinks [16:38:40] halfak: so it'll be much easier to scale without the rewriting. also I'll have to figure out what to do when users get renamed on wiki [16:38:54] halfak: since links will breakk and there's no redirect [16:40:21] yuvipanda, could we set up redirects? [16:40:33] Seems like doing that lookup could be expensive. [16:40:51] halfak: from renamed accounts? [16:41:00] we could do redirects much more easily [16:41:16] halfak: we'll also add a 'public link' option to the notebooks via an extension [16:42:32] OK... I think I have an idea. [16:42:47] So, stuff that is beneath a user-name should get second-class support [16:43:05] Stuff that is the "main space" or whatever should get first-class support. [16:43:22] So, if we make it work 99% to have username in the URL, that should be OK [16:43:35] That would be what we'd get by ignoring the fact that users can be renamed. [16:43:39] right but we won't have a mainspace for a few weeks [16:44:07] err [16:44:09] months [16:44:11] probably [16:44:25] halfak: I have wanted to have it be 'User:' for a while [16:45:08] /users/? [16:45:18] Or trying to stick with the MediaWiki style [16:50:10] yuvipanda, ^ [16:50:26] halfak: yeah, stick with the MW style [16:50:30] to imply similar connections [16:50:40] I'll also be building a nice filebrowser anyway [16:50:48] that'll render markdown, rst, etc [16:51:29] so you can put up a nice 'front page' for your username [16:52:27] Good deal. [16:52:33] IMO, should be a bit like github. [16:52:50] Some customization up front, but everyone has a list of projects too [16:53:12] Have you considered the 1 jupyter instance == 1 git repo pattern? [16:53:23] Or would you prefer many git repos in the same instance? [16:53:34] I don't have a real answer to that at all [16:53:41] I'd definitely like many git repos [16:53:49] but dunno how that'll play out in the long term [16:54:02] I'm getting more and more convinced that any public publishing we do should be based off IPFS [16:55:01] Just so long as I can think productively about versioning. [16:55:58] I like browsing git repos to see what people are up to. It helps me think about provenance and containers. [16:56:09] halfak: yup, you can kindof think of IPFS as one massive, unified git repo of sorts [16:56:15] that's constant across the whole world [16:56:24] /halfak/python-mwapi is a fork of /mediawiki-utilities/python-mwapi [16:56:46] * halfak runs in terror from the idea of one massive repo [16:56:58] you should really read the IPFS paper :D it is quite short and well written [16:59:03] I think I get the gist of it. I guess I really mean to ask about containers and what is contained. [16:59:11] E.g. there's the "research objects" crowd. [16:59:18] right [16:59:25] datasets, code, documentation for 1 project. [16:59:33] that's what we sohuld have, I think [16:59:36] Which sounds a lot like a repo per instance. [16:59:47] indeed [16:59:51] definitely not one repo per person [16:59:53] But maybe that is just because I organize my projects into a repo [16:59:57] Yeah. [17:00:03] I'm thinking many repos per person. [17:00:07] * yuvipanda nods [22:08:40] anyone know the quickest way to get the number of biographies of Women on enwiki? I tried this SPARQL query https://www.mediawiki.org/wiki/Wikibase/Indexing/SPARQL_Query_Examples#Women_that_are_artists (minus 'artists') on query.wikidata.org but it timed out. [22:10:01] I haven't used SPARQL before, so I'm just making guesses based on MySQL syntax right now. [22:18:25] let's see… bearloga, guillom, HaeB, halfak know how I can get the answer to this, or anyone I can contact directly? [22:18:32] I^^ [22:18:48] request is coming from Comms. Was an external press request. [22:18:49] J-Mo, saw the Q, but I don't grok SPARQL yet :\ [22:19:32] yeah, is Max Klein on IRC these days? Or Piotrus? They ran a study on this. [22:20:01] J-Mo, you might try Smalyshev [22:20:07] J-Mo: SMalyshev on #wikimedia-discovery is the best for this [22:20:07] It looks like hes in -tech now [22:20:13] jinx! [22:20:27] thanks! [22:20:56] J-Mo: he's also the lead on query.wikidata.org [22:23:47] pinged him. thanks bearloga! [23:14:07] J-Mo: you could also ask the people on #wikidata [23:14:22] * yuvipanda is stuck in flight purgatory - his flight is delayed until 7PM [23:43:02] oof sorry to hear about the travel woes, yuvipanda.