Fork me on GitHub

Wikimedia IRC logs browser - #wikimedia-tech

Filter:
Start date
End date

Displaying 118 items:

2019-12-18 09:05:01 <neuhaus> Hi, i think I've come across a bug. I made a change on en.wikipedia.org and the history shows that I've removed 12 thousand bytes but i added a two sentences instead
2019-12-18 09:05:31 <neuhaus> check out the latest change at https://en.wikipedia.org/w/index.php?title=Android_One&action=history
2019-12-18 09:06:30 <andre__> neuhaus: Congrats! :D Looks like, indeed. See https://www.mediawiki.org/wiki/How_to_report_a_bug
2019-12-18 09:06:50 <neuhaus> it was a visual edit. There seems to be another issue with the history, it should hilight the added sentence just like it does with the first line
2019-12-18 09:07:31 <andre__> First thing is about page history (#MediaWiki-Page-History in Phabricator), second issue is about the diff algorithm (#MediaWiki-Page-Diffs in Phabricator)
2019-12-18 09:07:46 <neuhaus> yes
2019-12-18 09:11:47 <neuhaus> The accounts on wikipedia have nothing in common with the accounts on mediawiki, right?
2019-12-18 09:14:03 <andre__> neuhaus, they should be the same nowadays
2019-12-18 09:14:15 <neuhaus> oh ok
2019-12-18 09:14:21 <andre__> neuhaus: If you're logged in on a Wikipedia, you should also be logged in on mediawiki.org, or?
2019-12-18 09:15:04 <neuhaus> no, but i was able to login manually
2019-12-18 09:15:21 <andre__> hmmm
2019-12-18 10:03:39 <[1997kB]> What is the maximum count of tables in toolsdb on Labs?
2019-12-18 13:44:16 <Urbanecm> [1997kB]: I'm not aware of any - maybe ask in #wikimedia-cloud
2019-12-18 13:46:00 <[1997kB]> ty, I'll
2019-12-18 15:00:29 <wm-bot> Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: @CFisch_WMDE - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
2019-12-18 15:36:14 <zig> hello, is the technical meeting happening here?
2019-12-18 15:36:44 <zig> It is in 30 minutes, sorry for the noise!
2019-12-18 15:38:59 <zig> btw I am the one that just sent a mail to wikidata-tech. ref: https://lists.wikimedia.org/pipermail/wikidata-tech/2019-December/001510.html
2019-12-18 15:45:35 <andre__> zig: Looking at https://meta.wikimedia.org/wiki/Grants:Project/WDQS_On_FoundationDB , is there also a text which avoids subjective terms like "perfect combination", "state-of-the-art" or "industry grade" but describes actual technical problems?
2019-12-18 15:46:43 <andre__> zig: I don't think anyone will be convinced by a marketing pitch, so describing actual technical issues would be welcome. Currently they are hard to find in that text.
2019-12-18 15:49:17 <andre__> zig: I don't understand the "hardware cost" part in those proposals. See https://wikitech.wikimedia.org/wiki/Portal:Toolforge
2019-12-18 15:50:18 <wm-bot> Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: @addshore - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
2019-12-18 15:50:55 <addshore> o/
2019-12-18 15:55:29 <zig> andre__: I used Google Cloud Calculator to get those cost, but they are much lower say at online.net, thanks for the feedback.
2019-12-18 15:56:20 <zig> how is toolforge supposed to help? I can message them about my requirements and can deliver an estimation?
2019-12-18 15:58:13 <zig> s/can/they can/
2019-12-18 15:58:53 <andre__> zig: Why are there hardware costs?
2019-12-18 15:58:57 <andre__> at all?
2019-12-18 16:00:27 <addshore> o/ Hi all! I'm here for the Technical Advice IRC meeting :) To help you wherever possible!
2019-12-18 16:01:00 <zig> I need to benchmark the solution to see if it really an improvement over the current solution. The workload I want to reproduce is the following: https://iccl.inf.tu-dresden.de/web/Wissensbasierte_Systeme/WikidataSPARQL/en
2019-12-18 16:01:11 <andre__> \o
2019-12-18 16:02:06 <andre__> zig: so you need money to buy (or rent) hardware yourself, to benchmark, after implementing something and not knowing if it's better?
2019-12-18 16:02:35 <zig> Basically, the goal of the project is to make WDQS more scalable both in terms of read and write and reduce the lag between the moment an edit is done on wikibase, and the time it will appear on WDQS. I want to at least proove that reads are as fast as current solution.
2019-12-18 16:03:06 <zig> andre__: I somewhat know over small dataset (latest-lexemes.nt) that it is faster. But I figured a proof should be done over the real data.
2019-12-18 16:03:58 <andre__> So that's entirely I research project, I see. Still the vague subjective terms confuse me. :-/
2019-12-18 16:04:38 <zig> I know that nomunofu is faster than blazegraph (current solution) over 5.6G
2019-12-18 16:04:41 <andre__> Phrases like "it is reported to be difficult to scale" feel a bit like https://en.wikipedia.org/wiki/WP:WEASEL
2019-12-18 16:05:44 <KF19LivesForever> Will there be a meeting on the 25th???
2019-12-18 16:05:52 <zig> andre__: that is not my word, but a quote from https://lists.wikimedia.org/pipermail/wikidata/2019-June/013124.html
2019-12-18 16:06:26 <addshore> There will not be a TAIM meeting on the 25th
2019-12-18 16:06:27 <andre__> KF19LivesForever, I doubt :)
2019-12-18 16:06:38 <addshore> I believe this is the last TAIM meeting!
2019-12-18 16:06:48 <andre__> zig: Ah, thanks! Then feel free to mark the quote as a quote :)
2019-12-18 16:07:34 <addshore> I see that email says "If anyone has experience with scaling large graph databases, please reach out to us, we're always happy to share ideas!"
2019-12-18 16:07:50 <addshore> Zig, have you reached out to the team directly?
2019-12-18 16:07:58 <andre__> zig: What I'm basically trying to say is that I'm not convinced of the two proposals as they don't really describe actual technical problems. But I don't have authority, so I'm just sharing impressions. :)
2019-12-18 16:08:03 <KF19LivesForever> last of the decade!
2019-12-18 16:08:15 <zig> andre__: tx
2019-12-18 16:08:26 <andre__> zig: Plus from reading it it is entirely unclear to me why you need money for hardware
2019-12-18 16:08:48 <zig> addshore: I was under the impression that emailing wikidata-tech was direct.
2019-12-18 16:09:06 <addshore> It is fairly direct, they are also on IRC.
2019-12-18 16:11:23 <zig> The current status is that I have a portable binary of the project that can load .nt files. It is 3 or 4 times faster than blazegraph over latest-lexemes (1.6GB) on a single machine. But I need to money and hardware to expand the experiment to full wikidata-scale (1.7TB).
2019-12-18 16:12:23 <zig> also, even if I sent mail to wikidata (and now wikidata-tech) and filled the grant proposal, outside the current session, I had not feedback.
2019-12-18 16:12:46 <addshore> So, lexemes are much smaller than items, have you tried loading items?
2019-12-18 16:13:15 <zig> I recognize that it might sounds like sales pitch, I just try to find the words that convey the idea I have and how I see it.
2019-12-18 16:13:40 <zig> addshore: yes and no. The import process is not finished.
2019-12-18 16:13:53 <addshore> I don't think the full set of triples is near 1.7TB, it should be less than half of that afaik
2019-12-18 16:15:25 <addshore> How long do you predict the import of all items would take?
2019-12-18 16:15:33 <zig> well, I am sure it not half less.
2019-12-18 16:18:04 <zig> My last calculation estimated the total time to load all the data to 1 month. I thought about an optimization to reduce the bulk loading time, I don't know how much time it will take with the optimization. People have reported 2 weeks on SSD using virtuoso. So I guess more than 2 weeks.
2019-12-18 16:18:41 <addshore> Oooof, 1 month would be painful in production. The current load time takes about a week and that is painful.
2019-12-18 16:18:54 <zig> (the total size of 1.7TB is uncompressed file)
2019-12-18 16:19:33 <addshore> The TTL dump?
2019-12-18 16:20:10 <zig> I use latest-all.nt.bz2 dump, ref: https://dumps.wikimedia.org/wikidatawiki/entities/
2019-12-18 16:20:57 <zig> the current approach that takes one month is not optimized for bulk loading.
2019-12-18 16:21:35 <zig> In what occasion do you need to bulk load the data?
2019-12-18 16:22:54 <zig> also it depends on the available RAM. On my test server i have 32GB of RAM. With more RAM you can load more stuff at once, hence it is faster.
2019-12-18 16:23:14 <zig> I think wdqs servers have 256GB of RAM?
2019-12-18 16:23:56 <addshore> Bulk loading is needed for provisioning new servers, or reloading old servers
2019-12-18 16:24:30 <zig> there is no backup of the database file?
2019-12-18 16:24:57 <dcausse> wdqs servers have 127Gb currently, importing the 9B triples take about ~2weeks at the moment
2019-12-18 16:25:07 <zig> Ah maybe it is not possible to hot-copy the database file, I am not sure how blazegraph works in this regard.
2019-12-18 16:25:21 <dcausse> no it's not, writes have to be suspended
2019-12-18 16:26:07 <addshore> Hi dcausse !
2019-12-18 16:26:12 <dcausse> hi! :)
2019-12-18 16:26:20 <addshore> was just looking up the memory of the servers but you beat me!
2019-12-18 16:26:34 <zig> with write-ahead-log enabled, I think one can backup the database files.
2019-12-18 16:27:42 <addshore> is going to try using suggestions here https://addshore.com/2019/10/your-own-wikidata-query-service-with-no-limits-part-1/#comment-390 when I try to make my own copy of the service again
2019-12-18 16:29:37 <zig> what is the purpose of toolforge? can they provide me access to a machine similar to wdqs?
2019-12-18 16:29:58 <addshore> So, you probably want the cloudy bit rather than the tooly bit
2019-12-18 16:30:01 <addshore> finds some links
2019-12-18 16:30:12 <addshore> https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS
2019-12-18 16:30:45 <addshore> You can also find them in #wikimedia-cloud
2019-12-18 16:31:42 <addshore> Default is 16GB I believe
2019-12-18 16:32:07 <addshore> But I imagine for specific projects or tests a larger allowance might be allowed
2019-12-18 16:33:14 <andre__> also see https://phabricator.wikimedia.org/project/view/2880/
2019-12-18 16:34:36 <[1997kB]> hey, In mobile site when someone click edit icon on top of talk pages, it doesn't shows source of all page.. is it a bug? https://en.m.wikipedia.org/wiki/Talk:Main_Page
2019-12-18 16:35:31 <James_F> [1997kB]: That's how edit works on mobile. It's section-edit only.
2019-12-18 16:35:41 <addshore> [1997kB]: hmm, https://en.m.wikipedia.org/wiki/Talk:Main_Page doesn't look idea for mobile when comparing it to what is on the desktop site
2019-12-18 16:35:49 <addshore> I would probably consider that a bug that needs fixing!
2019-12-18 16:36:07 <andre__> The mouse-over tooltip explicitly says "first section"
2019-12-18 16:36:34 <addshore> oh, indeed
2019-12-18 16:37:08 <addshore> Not the best UX in the world
2019-12-18 16:37:35 <[1997kB]> James_F, doesn't that limit the usage of mobile site for editing?
2019-12-18 16:38:09 <addshore> I guess the desired user interaction is to click into "Active discussions" and then use the further edit pens there
2019-12-18 16:38:37 <addshore> having a page level "watch" element right next to a section level "edit" element seems a bit evil though, like it is designed to tick people
2019-12-18 16:39:59 <James_F> [1997kB]: Mobile devices are generally bandwidth/data and CPU/memory constrained.
2019-12-18 16:40:22 <James_F> [1997kB]: E.g. trying to edit one of the 100KiB epic articles might crash their browser. It's a balance.
2019-12-18 16:41:03 <James_F> To a first approximation, no-one edits on both mobile and desktop, so behaviours expected in one don't need to be replicated on the other.
2019-12-18 16:44:21 <[1997kB]> yeah I agree, but in my last 2 years of contributions on mobile I think it would be great if all of the page can be editing at once.. like if I'm reading an article an found two typos in different sections, I always switch to desktop to fix them at once..
2019-12-18 16:49:49 <[1997kB]> anyone discussed this or filed a phab report?
2019-12-18 16:56:09 <addshore> [1997kB]: I havn't seen one and I havn't written one
2019-12-18 16:57:27 <James_F> [1997kB]: Not in the past five years, but we discussed it back in 2012/13 a fair bit.
2019-12-18 16:58:28 <andre__> [1997kB], if you want to go to Phab, it's probably #Advanced_Mobile_Contributions after https://www.mediawiki.org/wiki/How_to_report_a_bug
2019-12-18 17:00:35 <addshore> I think thats the end of the final TAIM!
2019-12-18 17:00:37 <addshore> thanks for coming!
2019-12-18 17:01:56 <[1997kB]> ty..
2019-12-18 17:29:02 <addshore> andre__: do you have the ability to edit comments on phab?
2019-12-18 17:29:26 <andre__> addshore, only remove.
2019-12-18 17:29:32 <andre__> edit is up to each author
2019-12-18 17:29:40 <addshore> would live the ability to replace a bunch of P123 in https://phabricator.wikimedia.org/T237984 with quoted P numbers so they don't show up in the linked objects
2019-12-18 17:29:44 <addshore> aaah okay,
2019-12-18 17:30:19 <andre__> once linked means that they probably stay, not sure
2019-12-18 17:30:22 <addshore> there are 5 random pastes linked, which is lame. I'm being very careful to quote my P numbers now, hah
2019-12-18 17:30:48 <andre__> hehe
2019-12-18 19:50:38 <akoopal> hi, I just noticed on some dutch talkpages that the signatures give wrong time
2019-12-18 19:51:00 <akoopal> seems they put the time in UTC, but mentions CET

This page is generated from SQL logs, you can also download static txt files from here