[05:22:59] (03PS1) 10Prtksxna: Unify BetaFeatures SVG screenshot markup [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394521 (https://phabricator.wikimedia.org/T180427) [05:23:27] (03Abandoned) 10Prtksxna: Unify BetaFeatures SVG screenshot markup [extensions/ORES] - 10https://gerrit.wikimedia.org/r/393989 (https://phabricator.wikimedia.org/T180427) (owner: 10Prtksxna) [09:24:00] (03CR) 10Thiemo Mättig (WMDE): [C: 031] "A -1? For what? I seriously can not see what should be wrong with this code. It's a small, easy to understand, nicely separated top-level " (031 comment) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394316 (https://phabricator.wikimedia.org/T181334) (owner: 10Ladsgroup) [11:49:44] 10Scoring-platform-team, 10Operations, 10Wikimedia-Incident: Create an incident report - https://phabricator.wikimedia.org/T181795#3802542 (10akosiaris) [11:52:43] 10Scoring-platform-team, 10Operations, 10Wikimedia-Incident: Create an incident report for ORES overload incident 2017 - https://phabricator.wikimedia.org/T181795#3802557 (10jcrespo) [11:53:30] 10Scoring-platform-team, 10Operations, 10Wikimedia-Incident: Create an incident report for ORES overload incident 2017 - https://phabricator.wikimedia.org/T181795#3802542 (10jcrespo) I know this is just a todo for yourself, but the title appears in many places out of the original context, sorry for the edit. [14:02:47] o/ [14:09:10] halfak: Remind me, why would codfw have such a low cache hit rate? Are we directing some type of external requests to only eqiad? [14:09:40] It looks like both clusters are getting precache and external traffic. [14:10:30] I suspect it's cause precache definitely lowers the cache hit rate and external traffic is researchers [14:10:52] which have erratic patterns and are not necessarily asking for items in cache [14:11:09] also various tools in labs are using eqiad due to proximity [14:14:42] akosiaris: OK thanks, the tools explanation sounds most likely. It’s quite a striking difference, but probably not oeprationally significant. I hear that the cache hit rate graph only includes the external traffic, which is about 10% of total requests. [14:16:14] akosiaris: Also, thanks for your work on the available memory graph! I don’t see the changes though, maybe they weren’t saved? [14:16:34] I think I can do that based on your math fix :) [14:19:01] hmm, conflicting changes, you must have just gone in there... [14:19:06] I don’t save. [14:19:09] *won't [14:19:12] yeah reload :-) [14:19:19] yup there it is! [14:19:54] whew, we gotta abandon scb1001-2 [14:21:21] akosiaris: Do you have any thoughts about T181661? Seems to have releng stumped. [14:21:21] T181661: Connection timeout from tin to new ores servers - https://phabricator.wikimedia.org/T181661 [14:23:04] err, looks like someone ?else? deployed successfully since my last attempt. [14:23:17] not me [14:23:23] I haven't had time to look into this yet [14:25:00] No rush [14:25:17] I’m experimenting there now, seeing redis connection errors. I’ll debug the config. [14:30:52] 10Scoring-platform-team, 10ORES, 10Operations: Problem with Redis server configuration on new ORES cluster - https://phabricator.wikimedia.org/T181806#3802940 (10awight) [14:37:23] 10Scoring-platform-team (Current), 10Operations, 10monitoring: Investigate scb1001 and scb1002 available memory graphs in Grafana - https://phabricator.wikimedia.org/T181544#3802968 (10awight) One more little glitch, the `{cluster="scb"}` list doesn't include the codfw nodes. [14:38:08] 10Scoring-platform-team, 10Operations, 10Patch-For-Review, 10Wikimedia-Incident: ORES overload incident, 2017-11-28 - https://phabricator.wikimedia.org/T181538#3802973 (10awight) [14:38:10] 10Scoring-platform-team (Current), 10Operations, 10monitoring: Investigate scb1001 and scb1002 available memory graphs in Grafana - https://phabricator.wikimedia.org/T181544#3802972 (10awight) 05Resolved>03Open [14:45:58] awight: see my pr again ? [14:49:42] 10Scoring-platform-team (Current), 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#3802985 (10awight) [14:52:43] Zppix: Can I ask what happened with the commit and revert yesterday? [14:53:00] I fucked up :P [14:53:06] Was that just about the direct push, or the minor flake8 thing? [14:53:32] I asked amir to revert my direct push as i meant to create a new branch [14:54:25] ah np [14:54:48] What’s all the #noqa for? [14:55:22] halfak told me to do it to stop flake8 complaining [14:55:29] Awight [14:55:37] hum [14:56:09] And you do both __init__ and setup to cover pip and direct python setup? [14:56:16] Sorry I’m just catching up. [14:56:18] Yes [14:56:25] If you look @ phab task [15:01:27] 10Scoring-platform-team (Current), 10Operations, 10monitoring: Investigate scb1001 and scb1002 available memory graphs in Grafana - https://phabricator.wikimedia.org/T181544#3803010 (10akosiaris) 05Open>03Resolved Yeah that's because our prometheus per machine stats is per DC, not global. Anyway, simples... [15:01:31] 10Scoring-platform-team, 10Operations, 10Patch-For-Review, 10Wikimedia-Incident: ORES overload incident, 2017-11-28 - https://phabricator.wikimedia.org/T181538#3803012 (10akosiaris) [15:03:35] o/ [15:03:44] awight, I did a good analysis in that phab task :) [15:04:07] halfak: Super helpful, thanks for that. [15:04:28] I’m happy with everything but the #noqa, just taking a look [15:04:57] right. I think that flake8 will complain about doing imports late in the file. I think we need to do all of our checks before we start importing a bunch of stuff. [15:05:12] It did ^ [15:08:30] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803018 (10Halfak) Well.. I've had my github account locked, so I'm working on experimenting with gitlab. I've completed the upload of LFS'd content... [15:08:42] All the repos are LFS'd on gitlab ^ [15:10:59] awight, would love to talk to you more about moving to gitlab. [15:11:39] git...lab [15:11:40] huh [15:12:15] Yeah. Much better support for open source projects and it seems they are slowly winning over from github using a similar model -- except more open source. [15:12:20] gitlab itself is open source. [15:13:36] Sounds great, if you think it’ll be around in a year? [15:13:55] The only blocker I can think of is that WMF only has their mirroring set up for GH [15:14:35] I talked to people about that. There's no blocker with mirroring from gitlab. [15:14:47] But yeah, it would be yet another platform [15:14:56] On the other hand, it's so easy to migrate, I'm not too worried. [15:15:20] And gitlab isn't new. I considered hosting us on there originally, but I didn't finish my homework and forgot to seriously consider it in the past. [15:15:34] Migrating our stuff from github to gitlab was easy :) [15:15:37] Will they send us T-shirts? :p [15:15:47] Oh! Also your github account works on gitlab :D [15:16:03] How can I help? [15:16:09] The closedness of GH is infuriating [15:16:18] and their “tools” are nonsense IMO [15:16:34] btw.. a quick q [15:16:43] I have all day :) [15:16:50] how feasible would it be to change the serialization format for the models ? [15:16:59] I am guessing they are by far the largest objects there [15:17:01] akosiaris: I’ve been thinking the same thing. [15:17:02] causing all this pain [15:17:08] But not to save space. To use more :) [15:17:19] both with scap, git-lfs and yada yada [15:17:25] why would changing the serialization format help? [15:17:43] zipping could be made possible [15:17:51] compression in general [15:17:57] We do have the functionality to compress the model files. [15:17:59] No proble, [15:18:11] We handle bz2 natively -- just haven't made the switch yet. [15:18:18] Only for one model :) [15:18:29] akosiaris: This is what I’ve been thinking about, https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language [15:18:31] doesn't this lower repo size considerable ? [15:18:37] considerably* [15:19:09] akosiaris, in the LFS case it will matter a lot. [15:19:10] akosiaris: The thing about LFS is that we only have to download the *current* models though, which is already a >2x win [15:19:23] awight: well that or even just yaml/json, something anyway [15:19:28] In the repo case the history sucks [15:19:43] awight, yeah! Like 10x [15:19:52] The nice part about PMML is that we can train, tune, test, and run our models with other frameworks if we dare. [15:19:54] Then with compression, we can get another ~3-4X [15:20:23] * halfak does not want to implement some crazy new serialization strategy [15:20:29] But I would use it if someone else did :D [15:22:18] the good thing about serialization and compression in our case is that the objects are effectively being uncompressed/deserialized only once per worker's lifetime which means we won't suffer the usual high CPU usage issues that come with constant serialization/deserialization [15:22:35] as anyone experienced enough in java can tell you [15:22:50] sounds terrible [15:22:50] wait, not even that [15:22:52] akosiaris: FYI the one model we’re using bz2 for went from 40MB -> 12MB, FYI [15:23:00] But +1 to what you said akosiaris [15:23:02] only once per master's lifetime, right ? [15:23:31] akosiaris: yes, the models are loaded before forking workers [15:23:32] Every time we load a model into memory, we'll need to decompress (cheap) [15:23:43] Every time we rebuild a model, we'll need to compress (rare, expensive) [15:24:09] … and when we build a model it’s already 24hr of CPU so the bzip2 is nuthin [15:24:55] +1 [15:25:17] awight, time to sed our Makefile? [15:25:31] I can’t wait... [15:25:41] just need the weekend :) [15:26:12] akosiaris: I’m not sure what you’re referring to with CPU for deserialization though, cos we would still need to do that after decompressing. [15:28:34] Zppix: Check the last commit on the branch, ^ I think I made flake8 happy. [15:29:14] awight: so one usual problem with serialization/deserialization is the CPU required for doing that (it's not free). If that is happening often it can lead to wasted CPU cycles. But often is not the case here, so no problem [15:29:47] for example https://blog.hartleybrody.com/python-serialize/ [15:29:57] "After letting the script run for awhile, the bottleneck jumped out right away – the workers were spending about 40% of their time serializing and deserializing data." [15:30:06] K. I think the tradeoff with compression is just CPU for I/O, and the repos don’t seem to be very huge honestly. [15:30:36] akosiaris: aha thanks. yeah we never do any serialization codec’ing during runtime [15:30:43] the reason I made the Java joke is that serialization/deserialization is happening all the time in the java world [15:31:00] in fact it's quite often the source of security problems/exploits and so on [15:31:03] Oh! We're definitely spending a lot of time serializing/deserializing data between workers. [15:31:07] I can imagine… CORBA’ing to death [15:31:32] halfak: What data is that? Only a hair’s worth of json, right? [15:31:37] Worker gets a big chunk of serialized data to extract features from and returns a small chunk of serialized data that represents the prediction. [15:31:43] ooh [15:31:48] one thing most people don't know is that the equifax hack was the result of unvalidated deserialized user-input [15:32:15] but we don't have that fear [15:33:27] it seems that escaping and sanitization is weirdly misunderstood [15:34:34] (03PS5) 10Ladsgroup: Wire ModelLookup service using OresServices and remove methods from Cache.php [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394316 (https://phabricator.wikimedia.org/T181334) [15:34:47] 10Scoring-platform-team, 10ORES: Look at overhead of json codec and data passing to feature extraction workers - https://phabricator.wikimedia.org/T181809#3803084 (10awight) [15:36:07] (03CR) 10jerkins-bot: [V: 04-1] Wire ModelLookup service using OresServices and remove methods from Cache.php [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394316 (https://phabricator.wikimedia.org/T181334) (owner: 10Ladsgroup) [15:41:13] halfak: Amir1: I like our immediate TODO system. Working for you two, too? [15:42:03] Oooh I need to make updates to my section [15:44:51] halfak: Care to share why GitHub locked your account? There’s nothing visible on your user page. [15:44:59] Exceeded their quotas? [15:45:12] University had superior bandwidth? [15:45:47] They only allow 1GB of storage and 1GB of xfer per month out of LFS. [15:46:02] They charge you for xfer every time someone forks a repo [15:46:17] For the entirety of the LFS history [15:46:21] * awight vomit-takes [15:46:31] Super predatory [15:46:32] What’re they scared of... [15:46:48] Yeah let’s ditch like a brother in middle school. [15:48:37] Or more accurately, ditch GitHub as if it were a friend who got into Amway [15:49:11] I 'll take that as a deeply American joke [15:49:42] what on earth is amway.... the site is utterly useless [15:49:55] oh... spam ? [15:49:59] no idea either [15:50:38] oh no.. ponzi schemes [15:50:42] mult-level marketing scheme to make people sell stuff [15:50:48] eer sorry "multi-level marketing" [15:50:53] aka .. ponzi schemes [15:53:42] 10Scoring-platform-team (Current), 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Write reports about why Ext:ORES is helping cause server 500s and alternatives to fix - https://phabricator.wikimedia.org/T181010#3803155 (10awight) [15:54:34] 10Scoring-platform-team (Current), 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: [Spike] Write reports about why Ext:ORES is helping cause server 500s and write tasks to fix - https://phabricator.wikimedia.org/T181010#3776331 (10awight) [15:54:49] lolol [15:54:56] Sorry to rabbit-trap you all. [15:55:44] There are a lot of these still active in the U.S…. Amway, Avon, Tupperware, Partylite…. it gets worse than that, even. [15:55:49] you know I have 2 acquaintances that do this work for an extra income [15:56:10] so at some point one of them shows me a web page with "their products" [15:56:57] so .. 1lt of olive oil around here is around 6 euros [15:57:03] they were selling it 17! [15:58:09] not that aren't worse out there... ther's this for example https://www.greeceandgrapes.com/en/olive-oil-extra-virgin-200ml-7m [15:58:24] which is above absurd [15:58:44] snake olive oil [15:58:53] yup [15:58:59] very well said [15:59:03] so next is subscription boxes, wouldn't it be convenient if i shipped you a box with random samples of olive oil each month, with a auto-renewing sucscription [15:59:04] :) [15:59:06] Selling me something is at least dignified, but I lost a friend when he tried to rope me into the actual Ponzi part of some garbage. [15:59:28] they get a lot more money when they pull in a new low-level recruit. [16:00:09] Looking awight [16:00:39] awight, which #noqa's are a problem for you? [16:05:41] halfak: check my commit on that branch. Why was the import conditional? Seems to exist in py2 [16:05:59] https://github.com/wiki-ai/ores/commit/2cfab4371d63fc514df056568a3f0dfbf70e3ca2 [16:06:16] * awight double-checks that [16:06:51] awight, oh! It doesn't need to be :) [16:11:40] akosiaris: olive oil ain't cheap in the US, even in California [16:11:42] it's ridiculous [16:11:54] 10Scoring-platform-team (Current), 10ORES, 10Operations, 10Release-Engineering-Team: Connection timeout from tin to new ores servers - https://phabricator.wikimedia.org/T181661#3803203 (10akosiaris) A quick look at graphs for various ores hosts[1] and tin [2] does not show anything network related. A look... [16:11:55] 10[1] 04https://meta.wikimedia.org/wiki/:en:Cortisol13 => [16:11:58] El búfer 2 está vacío. [16:12:13] O_o AsimovBot is out on a limb [16:12:45] Lmao AsimovBot is speaking spanish [16:13:19] ahh I think it was the footnotes [3] [16:13:20] El búfer 3 está vacío. [16:13:21] [0] [16:13:27] :p [16:13:32] AsimovBot: is on steroids [16:13:33] 04Error: Command “is” not recognized. Please review and correct what you’ve written. [16:13:39] apergos: yeah I know. I used to joke about I would become rich very quickly if I could teleport at 0 cost olive oil and olives in SF [16:13:54] you could [16:14:11] plus feta that doesn't stink to high heaven [16:14:24] I can't believe what they pass off as feta there (except if it has no flavor at all) [16:16:43] feta in the US just means “bad cheese" [16:18:14] Fettuccini [16:18:40] halfak: Zppix: ok, VersionConflict has been in that same place since 2005 (setuptools 0.0.1) so I think we’re good including my change. [16:18:52] Ok [16:19:53] awight: you guys know best im just along for the ride. [16:20:54] Phew thats done [16:21:03] Zppix: footage from our “ride” https://www.youtube.com/watch?v=C44SE0bzxq8 [16:21:18] 10Scoring-platform-team, 10ORES, 10User-Zppix: [ORES] Make setup.py only work when using Python 3+ - https://phabricator.wikimedia.org/T181693#3803243 (10Zppix) 05Open>03Resolved [16:21:27] [do not try this at home] [16:22:15] That’s a nice fix though, should be very helpful for new contributors! [16:23:14] awight: I DIED [16:23:28] Luckily, I don’t have a car to do that with. [16:24:54] awight: its okay i wouldnt let you do that shit id be scared shitless [16:26:06] Fortunately, I got old at some point and I drive like a gangster now. For the record, gangsters drive like grandpas cos they do not want to be pulled over, ever. [16:29:20] halfak: I’m editing ORES/Deployment, and wondering what to do about the ores.wmflabs.org section. How about I make that a separate page? AFAIK that’s not a part of the production deployment workflow. [16:30:14] Or is it? [16:30:36] It isn't really, but it's something we should keep in sync mostly. [16:30:47] I thought that was only for the experimental ores-wmflabs-deploy repo? [16:32:10] +1 it is. [16:33:57] awight: old? Please your all young [16:34:10] Hell halfak looks like he just graduated high school [16:34:21] lol this makes me glad I don’t have a photo anywhere on the intertubes. [16:34:57] (03PS6) 10Ladsgroup: Wire ModelLookup service using OresServices and remove methods from Cache.php [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394316 (https://phabricator.wikimedia.org/T181334) [16:35:10] Zppix, :P I'm 33 [16:35:18] halfak: bs [16:35:36] Zppix: halfak did not graduate high school, please don’t pick at his scabs. [16:35:50] :P [16:39:34] https://en.wikipedia.org/wiki/Aaron_Halfaker [16:39:45] Woah! They finally changed my photo to a better one [16:39:48] \o/ [16:39:58] * halfak intentionally doesn't watch this page. [16:41:04] Still not a notable person from my home town. https://en.wikipedia.org/wiki/Virginia,_Minnesota#Notable_people [16:41:16] Halfak gets an enwiki article but NOOO i dont get one wtf [16:41:55] Halfak i can add you to that list if you want? [16:42:13] Mention your conflict if you do! [16:42:23] * halfak won't touch any of that. [16:42:26] I really dont have a COI with you [16:42:37] Mostly not, right. [16:43:05] awight, any concerns around moving forward with gitlab migration? [16:43:13] Want more time to think about it? [16:43:14] nope! [16:43:17] I mean i work with your team and you in a volunteer capacity thats about it [16:43:24] +1 [16:43:30] +3 [16:43:35] xD [16:43:38] Still a bit of a COI but I'd say putting me in that list is pretty safe. [16:43:42] halfak: Only the mirroring concern, but you’ve asked about that already. [16:43:54] awight: no mirroring wont be an issue [16:43:58] Maybe we should run a test with one of the repos first. [16:44:06] I'll see if I can get source/editquality working. [16:44:17] https://phabricator.wikimedia.org/source/editquality/ [16:44:19] FYI [16:44:24] awight: halfak phab can pull from any service that can pull from http [16:45:01] * awight immediately checks for “worse” picture. You can thank the editor here for complete CoI :). https://en.wikipedia.org/w/index.php?title=Aaron_Halfaker&oldid=782168105 [16:45:46] halfak: btw, please. What have you done for the sport of hockey? [16:45:49] Oh! I met Gamaliel. He nominated the page for deletion as soon as I showed it to him :) [16:45:54] lol right? [16:46:05] Hockey is the main export of my home town [16:46:06] “local handsome guy” needs speedy delete [16:46:11] lol [16:46:13] link? [16:46:30] among your VA MN notables [16:46:39] Halfak https://en.wikipedia.org/w/index.php?title=Virginia,_Minnesota&oldid=813077084 [16:46:40] Oh! no page. But could be removed from the list [16:47:04] {{done}} [16:47:04] You rule, halfak! [16:47:16] :D [16:47:28] No you rule, AsimovBot [16:47:50] AsimovBot: rule us [16:47:51] 04Error: Command “rule” not recognized. Please review and correct what you’ve written. [16:47:52] Zppix, you messed up alphabetical order [16:48:11] Fuck it, let someone else fix it halfak xD [16:48:15] lol [16:48:17] /o\ [16:48:25] Zppix: ORES is gonna find you [16:48:45] awight: ill remind ORES that i contribute to its source code xD [16:49:43] ORES has no sympathy [16:49:47] ORES does not feel [16:49:59] ORES has no fear [16:50:13] https://muchmorethannear.files.wordpress.com/2008/11/slide1.jpg [16:50:38] halfak: thats mean, ORES has feelings [16:51:10] http://www.motifake.com/image/demotivational-poster/0805/big-brother-is-watching-you-bigbrother-demotivational-poster-1210575930.jpg [16:51:24] ^ its me in the morning [16:51:29] We take this stuff vawwy sewiously [16:51:42] We do? [16:53:37] * halfak attempts to observe from gitlab [16:53:44] We'll see what happens! [16:54:06] Let me know if you need help halfak [16:54:27] I have no idea what's going to happen with the conversion to LFS. [16:54:57] If anything goes wrong just blame paladox [16:55:13] heh [16:55:28] * paladox hasent tested the conversion to lfs, only tested adding lfs to a repo [16:55:47] awight, our prod deploys might get broken while we work this out. [16:56:05] paladox, is lfs enabled for https://phabricator.wikimedia.org/source/articlequality/ ? [16:56:07] I am concerned. [16:56:09] Used to be "wikiclass" [16:56:21] Rollback might not be possible. [16:56:29] awight, right. Just realized this. [16:56:31] is that a gerrit repo? [16:56:34] or a github one? [16:56:34] well. I was going to say “before” something but it’s too late. [16:56:42] paladox, "github" soon to be gitlab [16:56:47] halfak: Major thing to check is that scap will splat out LFS files. [16:56:56] I doubt it will. [16:57:00] ah, nope lfs wont be enabled. Not sure if phabricator supports it. [16:57:18] hahaha! [16:57:25] halfak: Also, scap rewrites our submodule URLs differently on beta vs production. Please pull releng into that discussion... [16:57:29] Well... We might need to reverse everything [16:57:44] paladox, the editquality repo is also a github mirror. [16:57:59] ah [16:58:06] it may allow you to do it like that [16:58:17] we should ask mmodell. [16:59:00] fatal: unable to access 'https://gitlab.com/wiki-ai/editquality/': Failed to connect to gitlab.com port 443: Connection timed out [16:59:09] I've confirmed that the URL works [16:59:14] Might be firewalled [16:59:22] port 443? [16:59:28] that's ssl i think. [17:00:27] * paladox has to go [17:00:41] Grr I’m trying to lock our deployment but can’t get the syntax right. [17:00:47] Just… don’t deploy production :) [17:01:04] Amir1: ^ we’re in the git-lfs danger zone. [17:01:32] halfak: almost certainly firewalled. [17:01:39] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803356 (10Halfak) I'm working on updating https://phabricator.wikimedia.org/source/editquality to pull from gitlab and I'm getting ``` Error updating w... [17:02:01] halfak: This is wmflabs deployment? [17:02:11] I thought that had a more lenient firewall... [17:02:19] awight, this is phab [17:02:39] Phab is telling me it can't "git ls-remote" in the management interface [17:02:56] That sounds horrific [17:03:13] awight, worst case, we can take our old repos, upload them to a temp repo, re-observe those repos and deploy [17:03:19] It will take time but I know how to do it. [17:03:31] Someone else will need to do it with their account though because mine is borked ;) [17:04:01] Old repos: https://analytics.wikimedia.org/datasets/archive/public-datasets/all/ores/ [17:04:44] halfak: So did you get locked out for using their LFS feature? [17:04:54] halfak: pls drop that into the task if it’s not there already. [17:05:03] I got locked out of doing anything with my account. [17:05:06] It is there :) [17:05:17] https://phabricator.wikimedia.org/T181678#3803018 [17:05:21] kk [17:05:35] See email forwarded to scoring-internal [17:08:22] awight, one more thing to run past you. [17:08:46] I accidentally did something dumb in gitlab with the name "wikiclass" so I decided it was about time we renamed to "articlequality". What do you think? [17:09:25] If you think it's a good idea. I'll make a task and get all of the details worked out. [17:11:30] +1 for consistency, thanks! [17:11:43] 10Scoring-platform-team, 10ORES, 10articlequality-modeling, 10artificial-intelligence: Rename "wikiclass" to "articlequality" - https://phabricator.wikimedia.org/T181820#3803377 (10Halfak) [17:11:49] cool :) [17:12:04] Now back to work on something to maybe make our py>=3 stuff cleaner :) [17:12:15] If this works the way I hope I think y'all are going to like it. [17:13:36] hmm? [17:13:48] Zppix’s PR is merged [17:14:06] Right. That's cool. We can still do more if you like it :D [17:17:24] Halfak yw btw [17:19:12] :P thanks [17:19:36] halfak: packaging in a separate module? [17:20:31] awight, right. I'm building a better python version checker [17:20:40] One that will allow for ranges and produce more useful error messages. [17:23:50] * awight refrains from outburst of profanity and obliquely suggests it instead. [17:23:57] VisualEditor. [17:24:10] I made a heap of changes to ORES/Deployment and cannot save or even copy + paste [17:24:17] at least I was able to take screenshots [17:24:20] wtf man [17:24:27] Seriously wtf [17:25:56] I like dogfooding but not when it involves my psychological harm [17:26:07] * awight mutters into the distance [17:26:17] https://phabricator.wikimedia.org/T181822 [17:26:27] * awight gathers flaming pitchforks [17:27:10] awight, agreed. I'm turning off the fancy new source editor too because there are some bugs that are just not getting fixed :( [17:27:33] Like being able to preview before clicking "save" [17:28:46] You can click back to WYSIWIG for that [17:29:22] … not that I’m in the mood to defend the whole Sisyphean pursuit [17:29:25] awight, check out https://gitlab.com/halfak/python_version [17:29:56] Is my python version check not good enough xD [17:29:57] Jk [17:29:59] Awesome. Will it work from our __init__.py and setup.py? [17:30:16] Yes. I have a test repo for that. Just about to push the change. [17:30:21] Zppix: Look at it like, your version checker was like the Wright Brother’s plane [17:30:41] https://pypi.python.org/pypi/python_version [17:30:53] I wish pypi supported markdown [17:31:03] I was kidding awight [17:31:45] so is awight ;) [17:31:51] o/ Nettrom [17:31:59] o/ halfak [17:32:04] We're looking to rename the "wikiclass" repo to "articlequality" [17:32:07] Any concerns? [17:33:03] Not really, it makes sense to me since “wikiclass” suggests something more generic than an article quality model. Will there be redirects from the old repo? In case people follow links from papers and such, I mean. [17:33:23] Yup! There will be redirects :) [17:33:40] Also, looking to move off of github because of their crazy LFS limits. [17:33:48] But github will remain a mirror at worst [17:34:01] Yeah, I saw some comments from you about that, understandable given the mess they’ve made [17:34:06] * paladox is back [17:34:31] Cool. I'm heading to lunch. Back in ~ an hour [17:34:40] Great, enjoy lunch! [17:34:48] I’ll go grab a morning coffee [17:36:42] awight: If you rework the error msgs i will look at them and I will make suggestions aswell [17:38:12] 10Scoring-platform-team, 10ORES: Make it easier to keep production and labs deployments in sync - https://phabricator.wikimedia.org/T181824#3803452 (10awight) [17:40:52] 10Scoring-platform-team, 10ORES: Make it easier to keep production and labs deployments in sync - https://phabricator.wikimedia.org/T181824#3803464 (10awight) [17:48:50] (03PS1) 10Petar.petkovic: Use ORES preference on Related Changes page [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394611 (https://phabricator.wikimedia.org/T180866) [17:58:48] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803540 (10demon) >>! In T181678#3798554, @Halfak wrote: > Trying start a gerrit review for wheels. Got this: > > ``` > Do you really want to submit t... [17:59:46] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803543 (10demon) >>! In T181678#3803356, @Halfak wrote: > I'm working on updating https://phabricator.wikimedia.org/source/editquality to pull from git... [18:16:56] Halfak what if i brought that py3 requirement stuff to all our repos? Would that work? [18:26:59] 10Scoring-platform-team (Current), 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: [Spike] Write reports about why Ext:ORES is helping cause server 500s and write tasks to fix - https://phabricator.wikimedia.org/T181010#3803643 (10awight) [18:27:23] 10Scoring-platform-team (Current), 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: [Spike] Write reports about why Ext:ORES is helping cause server 500s and write tasks to fix - https://phabricator.wikimedia.org/T181010#3776331 (10awight) Deployment documentation is updated, struck-th... [18:30:59] 10Scoring-platform-team (Current), 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: How can we test all the wiki/page combinations that can be affected by ORES? - https://phabricator.wikimedia.org/T181830#3803654 (10awight) [18:34:39] 10Scoring-platform-team, 10Documentation, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Document ORES architecture from a robustness perspective - https://phabricator.wikimedia.org/T181831#3803671 (10awight) [18:35:00] 10Scoring-platform-team (Current), 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: [Spike] Write reports about why Ext:ORES is helping cause server 500s and write tasks to fix - https://phabricator.wikimedia.org/T181010#3776331 (10awight) [18:35:25] 10Scoring-platform-team (Current), 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: [Spike] Write reports about why Ext:ORES is helping cause server 500s and write tasks to fix - https://phabricator.wikimedia.org/T181010#3776331 (10awight) 05Open>03Resolved p:05Triage>03High Cl... [18:35:56] (03CR) 10Catrope: "ORES disabled on related changes, right? (Although there's a task asking to undo that.) So does this change do anything?" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394611 (https://phabricator.wikimedia.org/T180866) (owner: 10Petar.petkovic) [18:37:23] RoanKattouw: “related changed”? ^ [18:37:27] err *changes [18:38:03] We’re disabled on RecentChangesLinked but I actually don’t know what that is yet. [18:38:10] Yes the same thing [18:38:26] (03CR) 10Petar.petkovic: "> ORES disabled on related changes, right? (Although there's a task" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394611 (https://phabricator.wikimedia.org/T180866) (owner: 10Petar.petkovic) [18:38:37] It's RC filtered to pages that link to a given page [18:38:46] cool! [18:39:00] * awight wonders how I’ve been getting to such a page by mistake [18:39:30] (03CR) 10Catrope: [C: 032] Use ORES preference on Related Changes page [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394611 (https://phabricator.wikimedia.org/T180866) (owner: 10Petar.petkovic) [18:39:52] Recent Changes and Related Changes look similar at a glance. [18:40:13] Zppix, I think we should bring it to ores and revscoring [18:40:24] All of the others depend on those so it should work as intended. [18:40:34] Yeah they're basically the same [18:40:47] ORES was on RCL but Amir1 disabled it there for DB perf reasons [18:41:06] (03Merged) 10jenkins-bot: Use ORES preference on Related Changes page [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394611 (https://phabricator.wikimedia.org/T180866) (owner: 10Petar.petkovic) [18:41:09] But when bawolff helped us fix the WL+ORES DB perf issue, he said that we should be able to fix RCL with the same trick [18:42:02] Halfak ok [18:42:11] It is already on ores halfak lmao [18:42:23] Right [18:42:54] Ill add it to revs... same setup? Init and setup.py halfak? [18:46:44] (03PS1) 10Catrope: Revert "Use ORES preference on Related Changes page" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394630 [18:48:23] +1 [18:48:42] (03CR) 10Sbisson: [C: 032] Revert "Use ORES preference on Related Changes page" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394630 (owner: 10Catrope) [18:51:11] (03Merged) 10jenkins-bot: Revert "Use ORES preference on Related Changes page" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/394630 (owner: 10Catrope) [18:52:49] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803805 (10Halfak) @demon, right, I'm still not able to push the wheels LFS migration. Can you help us get gitlabs proxied? [18:53:36] wiki-ai/revscoring#1301 (PY3-hard-requirements - ffa3222 : Devin/Zppix): The build failed. https://travis-ci.org/wiki-ai/revscoring/builds/310187053 [18:54:05] wiki-ai/revscoring#1302 (PY3-hard-requirements-1 - f46a0cb : Devin/Zppix): The build failed. https://travis-ci.org/wiki-ai/revscoring/builds/310187236 [18:54:15] halfak: Maybe this kind of proxying will work? https://wikitech.wikimedia.org/wiki/HTTP_proxy [18:54:47] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803812 (10bd808) Related: {T143969} [18:54:49] But it seems like Chad might not be considering that we’re hosting elsewhere and mirroring into WMF git-lfs? [18:55:24] 10Scoring-platform-team, 10Diffusion, 10Gerrit, 10ORES, and 4 others: Add gitlab to proxies/whitelist for mirroring to phabricator - https://phabricator.wikimedia.org/T181835#3803814 (10Halfak) [18:55:56] Whoa that other bug is long. [18:56:25] Halfak awight https://github.com/wiki-ai/revscoring/pull/374 [18:56:27] which bug awight? [18:56:35] T143969 [18:56:36] T143969: Unable to mirror repository from git.legoktm.com into diffusion - https://phabricator.wikimedia.org/T143969 [18:58:03] 10Scoring-platform-team, 10Diffusion, 10Gerrit, 10ORES, and 4 others: Add gitlab to proxies/whitelist for mirroring to phabricator - https://phabricator.wikimedia.org/T181835#3803838 (10Halfak) See also {T143969} [18:59:35] oh damn. [18:59:39] I just realized a thing. [19:00:12] You can't just import a thing in setup.py because setup.py needs to be read to identify the requirements of installation. [19:00:18] hmmm... [19:00:34] Maybe it's good that we stick with what zppix did rather than using my fancy library. [19:00:52] * halfak runs a test [19:01:00] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803864 (10demon) >>! In T181678#3803805, @Halfak wrote: > @demon, right, I'm still not able to push the wheels LFS migration. Can you help us get gitl... [19:05:10] 10Scoring-platform-team, 10Gerrit, 10ORES, 10Operations, and 3 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3803891 (10Halfak) @demon, it seems this is a different conversation. We do want to use lfs internally on gerrit for our wheels repository. I've read... [19:07:16] Can i just throw flake8 against the wall [19:07:38] Its so damn picky [19:12:40] Nope. It'll raise a E292 THROWN AGAINST WALL warning [19:13:35] halfak: Sometimes flake i think errors just to annoy me lol [19:14:00] E827 ZPPIX DID SOMETHING [19:14:55] That seems more like a real flake8 error to me [19:16:54] halfak: yeah and I doubt there’s a reliable way to force and order for required module installation, so we’ll hit py3 code in a dependency [19:16:59] LOL E* life [19:17:46] E500000 Im flake8 and i complain because of that extra single press of the spacebar [19:20:30] Yeah. That's what flake8's job is though. [19:20:44] I have flake8 running in my text editor. [19:21:30] I just wish it wouldnt fail the entire build [19:21:42] And is there a reason revs doesnt have codecov? [19:21:54] I can set it up if need be [19:26:34] Amir1: you got a min? I have a special request for you to look at (its regarding wikibase) [19:33:22] Should have codecov [19:34:36] halfak: doesnt appear to [19:34:39] Ill look [19:37:03] halfak: its enabled on codecov.io but i dont think travis is running it [19:38:16] Zppix, https://github.com/wiki-ai/revscoring/blob/master/.travis.yml#L62 [19:38:50] brb [19:39:27] Hmm [19:39:51] Its not showing as a check on my pr like it should regardless of what state of the build [20:08:40] back [20:19:55] 10Scoring-platform-team, 10Bad-Words-Detection-System, 10revscoring, 10artificial-intelligence: Add language support for Simple English - https://phabricator.wikimedia.org/T181848#3804284 (10Adotchar) [20:20:21] 10Scoring-platform-team, 10Bad-Words-Detection-System, 10revscoring, 10artificial-intelligence: Add language support for Simple English - https://phabricator.wikimedia.org/T181848#3804300 (10Adotchar) a:05Adotchar>03None [20:23:17] 10Scoring-platform-team, 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Edit quality campaign for simple.wikipedia.org - https://phabricator.wikimedia.org/T181849#3804302 (10Adotchar) [20:23:50] 10Scoring-platform-team (Current), 10ORES, 10Easy, 10Patch-For-Review: Wire statistics into test model included with our Vagrant role. - https://phabricator.wikimedia.org/T176914#3804315 (10awight) 05Open>03Resolved [20:24:36] wiki-ai/revscoring#1306 (pytz_any - 0e7c67b : Adam Roses Wight): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/310225773 [20:24:41] 10Scoring-platform-team: Clean up ORES vagrant role - https://phabricator.wikimedia.org/T181850#3804319 (10awight) [20:28:31] Zppix, you can break on the + [20:30:03] Oh [20:30:05] Well [20:30:23] Should of pinged me sooner xD [20:31:42] wiki-ai/revscoring#1313 (PY3-hard-requirements-1 - d43ea9b : Devin/Zppix): The build was fixed. https://travis-ci.org/wiki-ai/revscoring/builds/310228772 [20:32:58] 10Scoring-platform-team (Current), 10ORES, 10Operations, 10Release-Engineering-Team: Connection timeout from tin to new ores servers - https://phabricator.wikimedia.org/T181661#3804344 (10awight) I just ran scap with `-l "ores1001.*" and deployment went smoothly. However, with scap running in parallel and... [20:33:45] Zppix: lmk if I can help you set up flake8 locally, too. I don’t mind the spam, though, it’s up to you. [20:34:38] awight: see i do normally have it locally, its just atm im unable to access my desktop so im stuck on mobile atm [20:34:53] okedoke [20:35:04] I’ve done crazier things in my old age :p [20:35:16] wiki-ai/revscoring#1317 (PY3-hard-requirements-1 - 16ae029 : Devin/Zppix): The build was broken. https://travis-ci.org/wiki-ai/revscoring/builds/310229999 [20:35:18] halfak: now im getting more errors from flake [20:35:30] Told you flake hates me xD [20:39:18] Hi. [20:40:12] Hello! [20:40:40] o/ Adotchar! [20:41:13] So regarding simplewiki. I think we *might* be able to get away with just using English Wikipedia's models there. Would you be willing to help us test them out? [20:41:36] Sure. [20:41:43] OK Amir1 what do you think [20:41:47] Now remember the grammar is different. [20:41:56] So there should be some change for that. [20:41:57] We'd be running a model trained on one wiki on another wiki [20:42:02] halfak: what's going on? [20:42:04] halfak: i can help test too [20:42:14] Thinking of running enwiki's damaging/goodfaith models on simplewiki [20:43:26] Features can be the same but i highly doubt it would get any good results using a model on another wiki [20:43:31] the data is not same [20:44:34] The grammar is different and there’s a lot less words. [20:44:57] which means it needs its own data to train [20:45:06] But that really doesn’t matter much. All it means is the page’ll be tagged. [20:45:12] !log Uploaded revscoring 2.0.11 to pypi btw [20:45:12] awight: Not expecting to hear !log here [20:45:13] No hay log abierto en #wikimedia-ai - log on para abrirlo, log list para listar los logs disponibles. [20:45:17] Using complex words shouldn’t be tagged as vandalism. [20:45:19] Amir1, curse words should still be the same. We don't have features that really use grammar. [20:45:28] But word length might be a thing. [20:45:38] lol awight [20:45:53] We should have stashbot work here [20:46:06] halfak: i can look into stashbot [20:46:15] That'd be cool. [20:46:18] Zppix: no worries, I was just messing with stashbot [20:46:24] :) [20:46:25] I didn’t expect !log to work, either. [20:46:32] awight: ik but it should work in here [20:46:40] I mean its already present in channel [20:46:46] Honestly we just need blatant vandalism tagged, or a bot that will revert it. [20:46:57] The bot op for simple’s version of ClueBot went missing [20:47:08] Let's give this a try. We can probably get this up on wmflabs pretty quickly and you can try out some of the 3rd party tools. [20:47:24] If it looks OK, we'll get it up in production and then you can try the new RecentChanges interface. [20:47:30] hmm, okay [20:47:45] lol it's going to be pretty weird to try to deploy ORES now though :/ [20:47:56] We might want to *not* try to deploy ORES :D [20:48:05] On simple? Or something else? [20:48:14] Also, we need to install git lfs on wmflabs to get this to work, I think. [20:48:34] halfak: I did just do a deployment to ores1001 to test a bug. It’s working fine, as long as you don’t… increment the revision :p [20:48:40] actually, I did that too. [20:48:45] curious but it worked. [20:49:19] halfak: Maybe you haven’t overwritten any of the repos we point to from ores-prod-deploy?? [20:49:30] Or maybe because the rewritten submodules… *gulp* [20:50:05] oh dear. url = http://tin.eqiad.wmnet/ores/deploy/.git/modules/submodules/wikiclass [20:50:21] Yeah we’re sitting on a landmine. [20:50:28] awight, right [20:50:34] halfak where would we need to !log in here [20:50:54] GOod Q. Not sure. [20:51:07] halfak: Well… maybe we want to UBN this git-lfs work so we don’t get in a position where we can’t deploy hotfixes. [20:51:09] halfak: where do you normally send y'all logs? [20:51:36] awight, I could try to switch back to non-lfs version of the repos on github. [20:51:44] * awight mops brow [20:51:45] We'd want to leave those RO [20:52:19] I don’t want to make a bunch of unnecessary work, but OTOH we don’t want to have to do that work in an emergency context. [20:53:20] halfak: deploy prep, ops work for log? [20:55:50] halfak: Oh how about disconnecting the GH mirrors? [20:55:53] that might be good enough. [20:56:12] Wondering if that would let us still push fixes to gerrit, though? [20:59:03] awight, I think it could work. [20:59:23] It doesn't help for wmflabs because those are direct from github ATM. [20:59:35] hehe that can be our guinea pig [20:59:37] If I can get git lfs on them, we can just operate like normal there. [20:59:51] I'll see if I can get phab disconnected quick. [20:59:56] ty! [21:00:05] Can you make me a task while I rush through the work? [21:00:27] yes [21:00:35] great :) [21:04:03] 10Scoring-platform-team, 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): Disconnect scoring repos to stop mirroring from GitHub - https://phabricator.wikimedia.org/T181851#3804438 (10awight) [21:04:17] 10Scoring-platform-team (Current), 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): Disconnect scoring repos to stop mirroring from GitHub - https://phabricator.wikimedia.org/T181851#3804454 (10awight) [21:04:42] Halfak, would !log to deploy prep, and prod sal be ok? [21:05:09] Zppix, now that I think about it, I'm not sure it makes sense to log to either from here. [21:05:18] It's good that we have to do it from other channels. [21:05:21] halfak: I didn’t UBN any of the other work, in the hopes that this one change will be enough to get us into a safe place. If it’s not (if we can’t write to the WMF repos or they’re already broken), we should probably raise priority on whatever it takes. [21:05:26] ok [21:06:30] * awight goes back to innocently playing with mw-vagrant [21:10:32] 10Scoring-platform-team (Current), 10ORES, 10Operations, 10Scap, 10Release-Engineering-Team (Watching / External): Disconnect scoring repos to stop mirroring from GitHub - https://phabricator.wikimedia.org/T181851#3804492 (10Halfak) Done! https://phabricator.wikimedia.org/source/editquality/manage/uris/... [21:10:47] O_O [21:10:56] What did that do [21:11:18] halfak: You’re still “observing” gitlab tho? [21:11:24] awight, nope. It's disabled. [21:11:31] kk [21:12:01] Theoretically should work [21:12:03] halfak: Did you happen to check where our phab repos stopped mirroring? [21:13:17] awight, I'll check to see if we can update a submodule on deployment-tin. [21:13:25] That should tell us whether everything is OK or not. [21:14:02] I’m not 100% sure that proves it, cos of proxying, caching, URL rewriting [21:14:16] awight, it will be pulling from phab [21:14:29] There are tons of hidden caches there [21:14:37] Hmm.. fair point. [21:14:47] maybe try a fresh checkout in your tin:~ [21:14:48] It'll prove that we can still pull from a phab URL and have it work :) [21:14:51] sure [21:14:58] yeah I guess that doesn’t prove it either [21:14:59] argh [21:15:31] tin doesn't have git-lfs installed [21:15:33] We can just… check for the current production revisions in phab itself? [21:15:39] SO I don't know how we were supposed to update and deploy anything [21:15:45] right [21:16:16] Could someone be nice and fix my pr for revs [21:16:22] Maybe then flake wont yell [21:17:16] https://phabricator.wikimedia.org/T180628 [21:17:45] We should back-peddle entirely. There's lots to do before we can make use of git-lfs [21:17:57] I'm seriously considering just reverting the github repos. [21:18:02] We have the gitlab repos anyway. [21:18:17] And I made backups just so we could do this. [21:18:24] awight, thoughts? [21:18:41] +1 [21:18:58] Sorry for the lost time, it was a noble goal! [21:19:36] No worries. Learn things. Rewind sometimes :) [21:19:45] So what are we doing? Im lost [21:19:59] * halfak prepares to have github come to his house and shake him for spare change [21:20:01] halfak: I agree that it’s best to not have to pressure anyone to do LFS things [21:20:02] lo [21:20:20] Threaten them with a blog post [21:21:11] halfak: just do your legal thing, just go to github's lawyers and yell "no" [21:21:25] I always forget about that solution [21:21:49] I mean you should be training wmf legal [21:23:37] * halfak is a legal guru with his loud "NO" yelling [21:24:48] halfak: id hire [21:25:45] awight, gitlab's review interface: https://gitlab.com/wiki-ai/articlequality/merge_requests/1 [21:26:09] 10Scoring-platform-team, 10ORES, 10articlequality-modeling, 10artificial-intelligence: Rename "wikiclass" to "articlequality" - https://phabricator.wikimedia.org/T181820#3804549 (10Halfak) https://gitlab.com/wiki-ai/articlequality/merge_requests/1 [21:26:57] Halfak are we offically moving to gitlab! [21:26:59] ?* [21:27:13] Zppix, probably. I just had that in my stack so I pushed it :) [21:27:27] Ok [21:32:14] halfak: I like it. I think. Not seeing how to leave a comment at a line, but that could be related to my logged-out status. [21:34:09] Let me know when you setup the org on gitlab halfak [21:34:34] Zppix, org exists but I won't pull everything over until we're ready. [21:34:41] K [22:10:04] halfak: can you explain what i did wrong in my revs pr [22:13:38] looking [22:14:48] $ flake8 . --max-line-length=85 --exclude=.svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.eggs,*.egg,doc [22:14:48] ./setup.py:11:72: W291 trailing whitespace [22:14:48] ./setup.py:12:11: W503 line break before binary operator [22:14:48] ./revscoring/__init__.py:141:86: E501 line too long (88 > 85 characters) [22:14:48] ./revscoring/__init__.py:142:27: W503 line break before binary operator [22:21:00] Ok? I dont understand how to fix the w503s and to cut up e501 any more halfak [22:21:53] Break in the middle of the string [22:22:11] "I have a lovely bunch of coconuts! " + \ [22:22:17] "Here they are standing in a row." [22:22:32] Ok so break after the plus? [22:23:52] right [22:25:19] Ok ty [22:32:30] Now the part before platform.python_version is at 90... halfak [22:32:55] You did it wrong :P [22:33:07] Halfak go figure [22:33:10] What do i do [22:35:10] Show me the line. [22:36:12] halfak: of the source code? Or travis? [22:37:21] source code [22:39:55] Halfak: https://www.irccloud.com/pastebin/e3G6ffNs [22:40:04] 10Scoring-platform-team (Current), 10ORES, 10Operations, 10Release-Engineering-Team (Kanban), and 2 others: Git refusing to clone some ORES submodules - https://phabricator.wikimedia.org/T181552#3804694 (10mmodell) a:03mmodell [22:45:28] Zppix, https://etherpad.wikimedia.org/p/shortentheline [22:50:28] wiki-ai/revscoring#1323 (PY3-hard-requirements-1 - 35846da : Devin/Zppix): The build was fixed. https://travis-ci.org/wiki-ai/revscoring/builds/310282811 [22:51:34] Yay [23:20:22] Yay! Got some JADE done :) [23:20:47] I'm AFK for the night. Have a good one folks. [23:21:01] Zppix, BTW, I put one little note on your PR. Probably good after that [23:21:09] o/ [23:22:27] Ok [23:24:03] Halfak fixed