[00:02:07] RoanKattouw: I figured out what was up with https://gerrit.wikimedia.org/r/#/c/351120/ ; passes now [00:02:32] Cool; will look in a minute [00:39:20] maybe we should radically simplify postgres support, make it map to MySQL in a more obvious way [00:39:35] maybe use a single schema file like we do for sqlite [00:40:08] migrate any old installations [00:40:37] it is tech debt, right? we pay interest on this [00:51:03] Yup [00:51:09] TimStarling: What do you mean by mapping to mySQL? [00:52:53] I mean, for example, fixing the table names that are arbitrarily different, e.g. user instead of mwuser [00:53:03] removing the foreign key constraints [00:55:30] remove the triggers, wtf? [00:56:44] there's a trigger on insertion into the text table (renamed pagecontent for no good reason) which updates the search index [00:56:52] why? we have perfectly good search update hooks [00:57:48] change all the TIMESTAMPTZ fields to char(14) etc. [00:59:13] what do you suppose the biggest postgresql wiki is? maybe we waste a megabyte or two by switching to char(14), but in return, get some developer sanity [00:59:52] presumably postgresql users want upgrades to work more than they want a schema that optimises every last byte [00:59:58] the larget public one must be Citizendium [01:00:06] * MaxSem emits an evil grin [01:00:25] Swapping timestamps is probably the easiest lowest hanging fruit [01:01:15] I pointed out at the time, all those years ago, that you can just quote table names instead of renaming to names that are not reserved words [01:01:39] people are lazy typists [01:01:46] Table quoting is hard, as we've seen :p [01:02:21] all quoting is hard. Bobby Tables we call him [01:02:22] yeah, the response was that it might be inconvenient when you log into the server and manually write SQL queries [01:03:55] serial/bigserial as replacement for sequences? [01:04:04] * RainbowSprinkles is just thinking aloud [01:04:32] serial/bigserial is basically same as int/bigint w/ a sequence [01:04:38] less typing tho (so easier to just transform mysql?) [01:05:37] OMG WHATCHA DOIN YA WANNA PARSE SQL WITH REGEXES AGAIN? [01:06:50] I wasn't! [01:07:02] I wrote our schema as a gigantic php array! [01:07:18] yaml would probably be nicer [01:08:29] well, now PHP arrays are like JSON so why not keep it that way [01:09:33] Yay short array syntax: break blame and old patches from applying nicely. But hey, we saved 5 keystrokes a couple of times per file! [01:09:55] also, can we kill mssql? I won't shut up about it [01:10:10] Have fun with that, I got yelled at last time [01:10:47] I feel bad for Oracle cuz I liked Jure, but he's not around these days and it's bitrotting :( [01:11:14] but think of all those 3 corporate users! [01:11:40] I mean when he was around the support was good [01:11:46] So I didn't mind :) [01:12:40] is there any way to migrate a wiki from one DBMS to another? [01:12:49] XML dump + import [01:12:52] Should work [01:13:05] but you will lose your user table, probably a lot of other things [01:13:08] but fuck all those user account [01:13:12] s [01:13:25] Hmm, true [01:13:27] maybe we should add an option to it to export everything [01:13:27] You'd miss a lot [01:14:04] There's probably a number of tools in this area [01:14:08] "convert X to Postgres" [01:14:11] "convert Y to mysql" [01:14:44] eg: https://github.com/dimitri/pgloader [01:14:48] sure, you can do an SQL dump, but it doesn't help that the data format is different [01:15:03] for the sake of Citizendium, let's not make such tools. let that poor thing die with dignity! [01:15:16] TimStarling: Well, presumably you'd want a tool that allows you to massage the data [01:16:09] maybe MW could have an SQL dump tool, it could change the table names as it goes [01:20:23] * MaxSem has just poked AaronSchulz IRL [01:22:07] Where the heck did we stash the pingback data? [01:22:08] Hmmm [01:24:27] It's in eventlogging, but no useful graphana graphs [01:24:59] | 4602 | mysql | [01:24:59] | 213 | sqlite | [01:24:59] | 100 | postgres | [01:25:32] where be dat? [01:25:58] | 12233 | mysql | [01:25:58] | 354 | sqlite | [01:25:58] | 244 | postgres | [01:25:58] | 4 | mssql | [01:26:09] ^ from the older version of schema [01:26:41] I want something like https://grafana.wikimedia.org/dashboard/db/extension-distributor-downloads?orgId=1 :) [01:26:46] MediaWikiPingback_15781718 [01:26:56] MediaWikiPingback_15781718_15423246 [01:27:07] https://phabricator.wikimedia.org/T75174#3250320 [01:28:14] also, can we kill mssql? [01:29:16] also, I'm not sure if we can even kill Oracle at this point. we should probably be piercing its heart with wooden stakes by now [01:30:15] let's have an RFC [01:32:29] postgres failure could probably be cleaned up in a weekend by someone with some free time [01:32:51] interesting how often people forget db->timestamp() in code or wfTimestamp() in test assertions [01:33:54] If we didn't use timestamp fields like Tim suggested that future proofs itself [01:33:55] I don't really like that in prod with mysql either since it's too close to sql injection for comfort [01:34:54] maybe I should made quotedTimestamp() instead of people needing >addQuotes( >timestamp( x ) ) all over the place [01:35:34] as for compatibility, I'd focus on constants and nonsense ID=0 row hacks first [01:36:18] triggers and constraints, yeah [01:36:55] I think I killed much of the triggers/cascade stuff [01:38:00] CREATE TRIGGER page_deleted AFTER DELETE ON page [01:38:01] FOR EACH ROW EXECUTE PROCEDURE page_deleted(); [01:38:15] Hm.. I think wmf-config in siteinfo broke WikiApiary for wmf wikis. [01:38:20] To....remove recentchanges rows? [01:38:21] https://wikiapiary.com/wiki/Wikimedia_Commons tab: General [01:38:29] wmf-config={'wmfMasterDatacenter': 'eqiad'} [01:38:40] I guess it doesn't expect that in the middle of already very complex SMW and wikitext [01:39:33] anyway.. [01:41:45] add_interwiki() function is unused afaict [01:42:22] RainbowSprinkles: the conditional code in php was removed. In effect that means the app and DDL logic runs redundantly. [01:42:43] the later should be deleted at least from new installs, though I never got around to removing that. [01:43:01] Yeah [01:43:42] The problem with wanting to wait a release or two before doing the rest is forgetting stuff... [01:44:16] I don't know why anyone ever thought it was a good idea to have that. [01:45:07] If the app logic is good enough for Wikipedia, why add a bunch of DDL for far-less used backend (for MediaWiki), and have to have if/else sprinkled all over the code checking if DDL is there or not... [01:45:15] Eh, there's a school of thought that says data constraints like that should be enforced by the database and not left up to applications. [01:45:20] it's like something on dailywtf [01:45:22] But yes, in this case it's kinda dumb. [01:45:41] I think you can make a better argument for it when you've got multiple applications touching the same data store. [01:45:51] I mean if we *only* supported PG, and it was the only code path, I might buy that. [01:46:08] *buy that DDL approach [01:48:26] -- These are added by the setup script due to version compatibility issues [01:48:26] -- If using 8.1, we switch from "gin" to "gist" [01:48:43] freaking necromancy [01:50:12] What's the current stable postgres? Like what's in distros too? [01:50:31] 9.6, Jessie is 9.4 [01:50:42] 9.2.x - 9.6.x seem supported [01:50:51] So at the *bare* minimum, we should target 9.2.x+ [01:52:48] precise was 9.1 and it sucked [01:53:10] 127.0.0.1/32 vs 127.0.0.1...sigh, cause rc_ip has to be a CIDR... [01:54:57] Swapping timestamps should be possible via an alter, don't need a maint script. to_char(foo, 'YYYYMMDDHHMISSMS') [01:56:49] AaronSchulz, and of course ipblocks don't need no new trendy data types! [01:59:31] ALTER using blah blah blah [02:30:57] interesting that Scribunto is more popular than ParserFunctions in ExtensionDistributor [02:33:54] Probably due to bundling? [02:34:01] Most installs have ParserFunctions out of the box [04:25:12] created T164898 PostgreSQL schema change for consistency with MySQL [04:25:12] T164898: PostgreSQL schema change for consistency with MySQL - https://phabricator.wikimedia.org/T164898 [15:30:11] subbu: I don't know how Parsoid deploys work, when will https://gerrit.wikimedia.org/r/#/c/352711/ be testable in Beta Labs? [15:30:54] we can deploy it to beta labs whenever. [15:31:05] let me ping arlo [15:32:22] I wanted to see whether we can close T164790 and/or T164791 thanks to your Parsoid update [15:32:22] T164790: Add class mw-parser-output to VE's editing area - https://phabricator.wikimedia.org/T164790 [15:32:23] T164791: Add class mw-parser-output to Flow comment- or content- containing elements - https://phabricator.wikimedia.org/T164791 [15:33:16] got it. am going afk for a bit but once arlo deploys, will let you know [15:33:38] Thanks [15:52:52] subbu|afk, anomie: deployed parsoid with mw-parser-output to beta labs [15:55:35] arlolra: Thanks [16:12:28] Hrm, either I'm testing it wrong or neither VE nor Flow are picking up the class from Parsoid's output :( [16:12:50] * anomie put some details on the bugs [16:38:17] pretty sure they request with body_only, so they'd never see it [16:39:31] for example, see the source at [16:39:31] https://en.wikipedia.beta.wmflabs.org/api/rest_v1/page/html/Selenium_Echo_link_test_0.6771523815609475 [16:39:42] [16:39:54] but body_only returns the innerHTML [22:35:10] Yeah Flow uses body_only I'm pretty sure [22:35:11] For bad resaons [22:36:02] Then again, I don't think normal VE really looks at the much either