|
2019-07-10 14:00:18
|
<wm-bot>
|
Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: @amir1 & @ottomata - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
|
|
2019-07-10 14:01:18
|
<ottomata>
|
:)
|
|
2019-07-10 14:26:37
|
<Golbez>
|
did someone break sorting
|
|
2019-07-10 14:26:45
|
<Golbez>
|
oh it's fixed
|
|
2019-07-10 14:50:17
|
<wm-bot>
|
Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: @amir1 & @ottomata - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
|
|
2019-07-10 15:00:20
|
<Amir1>
|
o/
|
|
2019-07-10 15:00:56
|
<Amir1>
|
Hello and welcome to the TAIM, ottomata and I are your hosts this evening/morning/whatever and we wish you a pleasant meeting
|
|
2019-07-10 15:01:26
|
<jem>
|
Hello, thanks and the same for you :)
|
|
2019-07-10 15:02:16
|
<ottomata>
|
hello!
|
|
2019-07-10 15:02:26
|
<sario528>
|
Hi
|
|
2019-07-10 15:02:53
|
<sario528>
|
Do we have a specif topic for this meeting?
|
|
2019-07-10 15:02:59
|
<sario528>
|
specific*
|
|
2019-07-10 15:03:35
|
<Amir1>
|
sario528: not so far, you can ask any questions you like
|
|
2019-07-10 15:08:58
|
<jem>
|
Ok, it seems nobody has questions so I'll make an easy one (I think)
|
|
2019-07-10 15:09:15
|
<Amir1>
|
sure
|
|
2019-07-10 15:09:33
|
<jem>
|
It's more a thing to solve than a question, I think
|
|
2019-07-10 15:09:46
|
<jem>
|
The Spanish locale isn't available in the toolforge bastions
|
|
2019-07-10 15:10:09
|
<jem>
|
(I write locale -a and I can see others but es_ES isn't there)
|
|
2019-07-10 15:10:22
|
<jem>
|
And to have proper sorting I have to use fr_FR at the moment
|
|
2019-07-10 15:11:20
|
<Amir1>
|
jem: technically, fixing it should not be hard, you can make a change in puppet to get it installed (I can find the exact module for you)
|
|
2019-07-10 15:11:48
|
<Amir1>
|
but before moving forward I think it's better to make a phabricator ticket (if doesn't exist) and discuss it there
|
|
2019-07-10 15:12:10
|
<jem>
|
Hum
|
|
2019-07-10 15:12:24
|
<jem>
|
I don't think it can create any doubt
|
|
2019-07-10 15:12:44
|
<jem>
|
Spanish for sure isn't less than Portuguese or French :)
|
|
2019-07-10 15:13:00
|
<jem>
|
Or Norwegian or Polish or Romanian
|
|
2019-07-10 15:13:40
|
<jem>
|
Was there a discussion about adding all of them? :)
|
|
2019-07-10 15:14:06
|
<Amir1>
|
I have no idea, bstorm_ might be able to answer that
|
|
2019-07-10 15:18:25
|
<jem>
|
Ok, there is no hurry at the moment so I'll be checking
|
|
2019-07-10 15:19:36
|
<Amir1>
|
jem: In general, I would recommand making a phabricator ticket so a) It's tracked down b) seen by people who know more
|
|
2019-07-10 15:20:23
|
<jem>
|
I understand it's important and useful but maybe it's not required for every little detail
|
|
2019-07-10 15:20:42
|
<jem>
|
(Or maybe this isn't as little as I think, but we don't know yet)
|
|
2019-07-10 15:22:01
|
<jem>
|
If all locale additions are in fact tracked down, then it's clear, that's the right way
|
|
2019-07-10 15:22:19
|
<Reedy>
|
An audit trail is always good - "why did we do this?"
|
|
2019-07-10 15:25:04
|
<yannf>
|
There are now many JS errors on Commons
|
|
2019-07-10 15:25:48
|
<yannf>
|
I was told the community is responsible for fixing that, but I don't see anyone willing or being able to do it.
|
|
2019-07-10 15:27:25
|
<Amir1>
|
yannf: There's also some errors due to https://phabricator.wikimedia.org/T227504
|
|
2019-07-10 15:29:52
|
<no_gravity>
|
Hello! Not sure if this is the right channel/time to ask. I am trying to build a pure text version of the current articles in the english wikipedia. It seems to be quite some work: Parse the xml dumps and then parse the wiki format. Is there a more elegant way?
|
|
2019-07-10 15:31:07
|
<Amir1>
|
no_gravity: if you're not asking for all articles, the rest API endpoint would be a good place to hit
|
|
2019-07-10 15:31:33
|
<Amir1>
|
https://en.wiktionary.org/api/rest_v1/#/Page_content/get_page_definition_term
|
|
2019-07-10 15:31:37
|
<no_gravity>
|
Amir1: In fact, this api endpoint produces the exact output I need: https://en.wikipedia.org/w/api.php?action=query&format=json&prop=extracts&explaintext&exlimit=1&exsectionformat=plain&pageids=600744
|
|
2019-07-10 15:32:13
|
<no_gravity>
|
Amir1: I am just not sure how often I can call it. I would be fine to call it once every second for 3 months if that is ok.
|
|
2019-07-10 15:32:41
|
<no_gravity>
|
It is to teach an NLP algorithm I am working on.
|
|
2019-07-10 15:33:25
|
<Amir1>
|
no_gravity: it should not be a problem, specially if you follow the UA policy
|
|
2019-07-10 15:33:38
|
<Amir1>
|
https://meta.wikimedia.org/wiki/User-Agent_policy
|
|
2019-07-10 15:34:21
|
<no_gravity>
|
Amir1: Great, I will try that then.
|
|
2019-07-10 15:34:30
|
<Amir1>
|
Because these are supposed to be cached in the ParserCache, so it's not a big deal on server side
|
|
2019-07-10 15:35:20
|
<no_gravity>
|
Amir1: "these" = the api calls and the responses?
|
|
2019-07-10 15:35:25
|
<Amir1>
|
yup
|
|
2019-07-10 15:35:52
|
<no_gravity>
|
Is it cached on different levels? Because I would think nobody else will do the *exact* same api call.
|
|
2019-07-10 15:36:42
|
<Amir1>
|
yes, there are cached in different levels
|
|
2019-07-10 15:37:00
|
<Amir1>
|
Varnish (VCL) and then we have ParserCache
|
|
2019-07-10 15:37:04
|
<no_gravity>
|
Nice. Ok. Then I will try that.
|
|
2019-07-10 15:38:26
|
<jem>
|
Amir1: I just found https://phabricator.wikimedia.org/T223777 about ca_ES also missing, but the original title was "UTF-8 support in toolforge, after migration to stretch", which would fit perfectly
|
|
2019-07-10 15:39:21
|
<jem>
|
bd808 changed it but maybe a generic title is possible, for sure there are a lot of missing languages and the problem is really one, not many
|
|
2019-07-10 15:39:36
|
<yann_>
|
Amir1, I got errors only when I edit
|
|
2019-07-10 15:40:58
|
<Amir1>
|
yann_: oh, maybe you should talk to ComTech?
|
|
2019-07-10 15:41:11
|
<Amir1>
|
jem: but we migrated to stretch already
|
|
2019-07-10 15:41:14
|
<jem>
|
Anyway.. almost two months open now isn't encouraging
|
|
2019-07-10 15:41:50
|
<jem>
|
Amir1: I understand the problem is in stretch nodes
|
|
2019-07-10 15:43:12
|
<Amir1>
|
that would take long time to be fixed, the new release (buster) just got out ten days ago
|
|
2019-07-10 15:43:54
|
<jem>
|
But bastion nodes aren't migrating again soon... or are they?
|
|
2019-07-10 15:44:01
|
<Amir1>
|
jem: do you have this issue in the grid as well?
|
|
2019-07-10 15:45:00
|
<Amir1>
|
in general, use of bastion should stay at minimum
|
|
2019-07-10 15:47:37
|
<cormacparle__>
|
yann_: can you give more details about the errors you're getting?
|
|
2019-07-10 15:51:04
|
<jem>
|
Amir1: The node I was in is login-stretch = tools-sgebastion-07, but let me check others
|
|
2019-07-10 16:03:24
|
<c>
|
any reason why spliting up database load across multiple replicas would share CPU but not connections?
|
|
2019-07-10 16:12:11
|
<ottomata>
|
c, not sure I understand your question
|
|
2019-07-10 16:14:27
|
<c>
|
ottomata: okay, this scenario I have 1 master and 5 replica databases for a single wiki with a load balance of 0, 1, 1, 1, 1, 1. During a load spike (editing a high use template for example), all dbs share the increased CPU load but not the increase in db connections. In this specific scenario all the extra connections went to the master instance
|
|
2019-07-10 16:15:41
|
<ottomata>
|
c, all the writes have to go the master directly, and then are replicated, right?
|
|
2019-07-10 16:15:52
|
<ottomata>
|
so mw will have to apply all the template updates directly to the master
|
|
2019-07-10 16:16:14
|
<ottomata>
|
the replicas only help balance read connections/load
|
|
2019-07-10 16:16:40
|
<c>
|
this is assuming that all the connections are writes, there have been scenarios before where connections were spread across the board
|
|
2019-07-10 16:22:13
|
<yannf>
|
oops, everybody's gone
|
|
2019-07-10 16:22:23
|
<yannf>
|
sorry, it was dinner time
|
|
2019-07-10 16:26:50
|
<c>
|
we use Amazon RDS but perhaps I should find a way to better analyze db traffic, their metrics for type of connections show me counts over a time period but don't get more forensic than that https://imgur.com/a/Ja1xtvY with regard to trying to come up with an ideal and cost effective balance
|