[01:43:43] ostriches: so i'm testing gerrit-new. what is the chance of you accepting and deploying a patch to one of the .js files it loads? [01:44:13] What js file? Is it part of gwt's massive pile of shiz? [01:45:08] We'd have to rebuild all of gerrit, which is ugly since we're using vanilla upstream for once :p [01:45:40] ostriches: https://gerrit-new.wikimedia.org/r/gerrit_ui/gerrit_ui.nocache.js [01:46:05] Yeahhhh that would be a gerrit rebuild [01:46:56] ew [01:47:15] So gerrit's javascript is actually java. [01:47:19] Yay, gwt. [01:47:38] yeah, i know [01:47:43] Anyway, I'm tired and been messing with this all day. I'm going to go eat and play pokemon. If you've got follow-ups please respond on the thread or on the gerrit 2.12 upgrade task [01:47:45] :) [01:47:52] my patch would be: find "D(l+gc)", replace with "D((l||f.gecko1_8)+gc)" [01:47:56] guess what's broken [01:48:07] A browser I haven't used in years? [01:48:08] :p [01:48:45] it's doing some extra fucked user agent detection [01:48:56] and when the user agent doesn't match any it knows about, it shits itself [01:49:10] I wonder if that's gerrit or just gwt :p [01:49:14] gwt tends to shit itself easy [01:49:18] ostriches: alternatively. can we redirect https://gerrit-new.wikimedia.org/r/gerrit_ui/undefined.cache.js to https://gerrit-new.wikimedia.org/r/gerrit_ui/D39174379837CB12534D3B279AEAC59F.cache.js ? that would also work, i think :P [01:49:34] anyway. for now, i can just spoof the user-agent… [01:50:03] MatmaRex: Write up a patch for the redirect and put me + daniel on it and I'll look later. [01:50:07] * ostriches is gone for real now [01:50:12] i don't even know where to start [01:50:20] but it's more likely than the previous thing? okay, good to know [01:50:25] see you :) [01:50:38] gerrit.wikimedia.org.erb in puppetz :) [01:50:43] Adios! [13:58:00] Hmm, new Gerrit... Ugh, new crappy UI! I'm used to the old crappy UI! [13:58:12] ;) [13:58:40] anomie: you can switch to the new change screen today and start getting used to it in production ;) i switched like a year ago! [13:58:55] MatmaRex_: "new change screen"? [13:59:14] https://gerrit.wikimedia.org/r/#/settings/preferences [13:59:21] :o yes! [13:59:29] https://gerrit.wikimedia.org/r/#/settings/preferences Change View: New Screen [13:59:47] It's scary :( [14:00:12] it looks almost the same as the new gerrit. but they apparently swapped the positions of the commit message and the commit metadata/action block, for some silly reason. [14:00:45] I think I'll stick to the old one until the new gerrit is out, instead of getting used to a different version. [14:01:15] On the plus side, T49408 seems to be fixed in the new Gerrit. [14:01:15] T49408: Inline comments wind up on wrong line in lines shown by +10⇧, +10⇩, etc - https://phabricator.wikimedia.org/T49408 [18:24:27] bd808: Do you know who I'd poke to find out what timestamp exactly there was an EventLogging for ServerSideAccountCreation for the creation of user "Knock CBRN" on enwiki? [18:25:56] anomie: hmmmm.... I have access but haven't looked at EL data much. I think dr0ptp4kt is an El data mining wizard [18:32:10] anomie: you should be able to see that from stat1003 [18:32:37] It's in the context of T119736#2459160. That user creation logged the beginning of the highlighted section of code at 18:17:33 and the end at 18:17:40. The irc.wikimedia.org feed spit out a log entry at local timestamp 14:17:39, but I have no idea about clock sync or delay between the log publish and when it gets to my IRC client. [18:32:37] T119736: Could not find local user data for {Username}@{wiki} - https://phabricator.wikimedia.org/T119736 [18:32:48] MatmaRex_: I don't think I have access to that. [18:34:15] huh. [18:34:30] i can probably look it up then… [18:38:00] anomie: hmm. [18:38:16] mysql:research@dbstore1002.eqiad.wmnet [log]> select timestamp from ServerSideAccountCreation_5487345 where event_userName='Knock CBRN'; [18:38:18] Empty set (47.69 sec) [18:38:30] MatmaRex_: Try 'Knock_CBRN'? [18:39:07] Although it shouldn't be that. [18:39:36] also empty [18:39:59] is that a recent thing? or should i look in the older versions of that table? [18:40:47] It was today at 18:17 [18:41:59] Although looking at the code in extensions/EventLogging/includes/EventLogging.php it doesn't seem to log the exact timestamp the event was sent, just when it was received after being sent by a DeferredUpdate. So never mind. [18:44:36] anomie: sorry, my connection died. 18:17 UTC? [18:44:46] MatmaRex: Although looking at the code in extensions/EventLogging/includes/EventLogging.php it doesn't seem to log the exact timestamp the event was sent, just when it was received after being sent by a DeferredUpdate. So never mind. [18:46:21] hm, okay. [18:54:49] bd808: Does https://gerrit.wikimedia.org/r/#/c/298817/ look good to you? [19:03:25] anomie: It's worth a try, sure. wmf.9 is not going out anywhere. wmf.10 is on group0 now and headed to group1 in the next 30 minutes or so. Maybe cherry pick to both? [19:03:41] err both .8 and .10 [19:06:12] greg-g, ostriches: Any objection to cherry-picking https://gerrit.wikimedia.org/r/#/c/298817/ to wmf.8 and wmf.10 so we can try to debug the latest iteration of T119736 a bit better? [19:06:13] T119736: Could not find local user data for {Username}@{wiki} - https://phabricator.wikimedia.org/T119736 [19:06:41] anomie: No objections. [19:07:45] ostriches: And since it looks like you're about to do the train, would you want to do it or do you want to ping me when the train is done? [19:08:03] Train's done for group0. [19:08:56] I have a hunch that this is going to end up having something to do with db write contention for commonswiki and wikidatawiki [19:09:26] almost all the manual cleanup I did was for those 2 [19:09:50] ostriches: So does that mean I should go ahead? [19:10:14] * Reedy hands anomie a jfdi hat [19:10:46] anomie: DO THE NEEDFUL! [19:10:48] :) [21:02:44] RFC meeting starting in #wikimedia-office: image and oldimage tables [23:00:06] TimStarling: how do you think we should handle https://phabricator.wikimedia.org/T135483 ? [23:05:41] could disable the preprocessor cache [23:06:58] it's unconditional at the moment, at Preprocessor_Hash.php line 714 [23:07:00] only a partial fix, since we serialize() also for backtraces [23:07:45] the arguments are serialized? [23:08:17] just when an exception is thrown? or during profiling? [23:08:24] yes, at least in certain cases or by certain log handlers. I don't know the exact circumstances, but I hit it earlier. [23:09:28] I think I proposed a few other fixes when this came up before [23:09:37] I think this bug is just a duplicate [23:09:41] we could also have custom __sleep() / __wakeup() that pack / unpack nextNeighbor into an array [23:10:19] OK, I'll see if I can find the original [23:10:48] well, I previously suggested a custom serializer, but otherwise, yes [23:11:18] i.e. not use serialize(), it's not really gaining much if you have to call __sleep() a million times [23:11:22] I don't know how those work, I need to read up [23:11:41] no, I mean just write your own function which converts an object to a string [23:11:50] without integrating with anything [23:12:31] right [23:12:59] https://phabricator.wikimedia.org/T73486 [23:13:49] and you were all over this task [23:13:56] and I proposed 5 different solutions [23:14:09] I'm basically senile [23:15:03] I remember this now [23:15:44] if we go with a custom serializer, we'd have to ensure it is used everywhere a PPNode could be serialized [23:17:49] declining to cache on very large input makes it a bigger ddos vector [23:18:14] g2g [23:18:50] preprocessToObj() is pretty fast, it's expand() that's slow [23:19:14] having HHVM crash is a more serious DoS vector, right?