Fork me on GitHub

Wikimedia IRC logs browser - #wikimedia-cloud

Start date
End date

Displaying 227 items:

2018-08-09 00:26:57 <MacFan4000> Hi, I need my 2fa reset. I got a new phone. I am still logged in on my tablet. I don’t have my recovery codes
2018-08-09 00:31:10 <MacFan4000> !help
2018-08-09 00:31:10 <wm-bot> MacFan4000: If you don't get a response in 15-30 minutes, please create a phabricator task --
2018-08-09 01:04:23 <MacFan4000> Hello?
2018-08-09 01:38:22 <wikibugs_> (PS1) GoranSMilovanovic: Initial [labs/tools/WiktionaryCognateDashboard] -
2018-08-09 01:40:02 <wikibugs_> (CR) GoranSMilovanovic: [V: 2 C: 2] Initial [labs/tools/WiktionaryCognateDashboard] - (owner: GoranSMilovanovic)
2018-08-09 01:42:47 <awight> !log deployment-prep T201518: ORES, fawiki wp10, misc updates
2018-08-09 01:42:58 <stashbot> Logged the message at
2018-08-09 01:42:59 <stashbot> T201518: ORES deployment (Early August) -
2018-08-09 10:27:43 <arturo> !log tools T201602 upgrade packages from trusty-updates
2018-08-09 10:27:48 <stashbot> Logged the message at
2018-08-09 10:27:49 <stashbot> T201602: toolforge: round of package upgrades -
2018-08-09 10:30:50 <arturo> !log tools T201602 upgrade packages from jessie-wikimedia
2018-08-09 10:30:55 <stashbot> Logged the message at
2018-08-09 10:33:05 <WikidataFacts> I’m getting “MySQL server has gone away” errors in my tool (errno 32, broken pipe), and I’m wondering about the best way to fix that wihout making the DBAs sad ;)
2018-08-09 10:33:17 <WikidataFacts> I’m currently initializing the DB connection at startup and that’s it
2018-08-09 10:33:25 <WikidataFacts> should I keep that approach, and try to reconnect when the connection dies
2018-08-09 10:33:48 <WikidataFacts> or instead open new connections on each web request and close them when the request finishes, to avoid stale connections hanging around?
2018-08-09 10:40:23 <arturo> !log tools T201602 upgrade packages from jessie-backports (excluding python-designateclient)
2018-08-09 10:40:28 <stashbot> Logged the message at
2018-08-09 10:40:29 <stashbot> T201602: toolforge: round of package upgrades -
2018-08-09 10:41:25 <arturo> WikidataFacts: I understand your problem, but can't properly answer your question
2018-08-09 10:41:54 <arturo> I would say stale connections hanging around is totally something to avoid
2018-08-09 10:42:19 <jynus> I can answer that
2018-08-09 10:42:23 <jynus> it is documented
2018-08-09 10:42:49 <WikidataFacts> ah
2018-08-09 10:42:51 <arturo> thanks jynus
2018-08-09 10:42:51 <WikidataFacts> ?
2018-08-09 10:42:52 <jynus> WikidataFacts:
2018-08-09 10:42:56 <jynus> yep
2018-08-09 10:43:00 <arturo> reading
2018-08-09 10:43:02 <WikidataFacts> amazing how I only needed to know it was documented ^^
2018-08-09 10:43:03 <jynus> for webrequests
2018-08-09 10:43:04 <WikidataFacts> thanks jynus
2018-08-09 10:43:15 <jynus> it is highly recommended to open a connection each time
2018-08-09 10:43:15 <WikidataFacts> (I was searching for the error message, that didn’t bring up anything)
2018-08-09 10:43:19 <jynus> this is for several reasons
2018-08-09 10:43:28 <jynus> 1- idle connections are killed automtaically
2018-08-09 10:43:43 <jynus> 2-backends can change due to proxy balancing/maintenance
2018-08-09 10:43:57 <jynus> so you want to open a new connection normally to get the freshest conffiguration
2018-08-09 10:44:16 <jynus> 3-supporing pool of connections would overload the highly shared db infrastrcture
2018-08-09 10:44:27 <jynus> we have thousands of db users, so we have to be conservative
2018-08-09 10:45:05 <jynus> this would be different if you had dedicated resources (a db just for you), but that is not the case for toolsdb/wikireplicas
2018-08-09 10:45:14 <jynus> WikidataFacts: does it make sense?
2018-08-09 10:45:22 <WikidataFacts> totally, thank you
2018-08-09 10:45:48 <jynus> regarding retries, if you do, retry once to reconnect, then fail
2018-08-09 10:56:07 <arturo> MacFan4000: we would have to follow these steps:
2018-08-09 11:06:53 <WikidataFacts> jynus: implemented at, thanks again for your help!
2018-08-09 11:08:33 <jynus> you are welcome!
2018-08-09 12:23:04 <MacFan4000> arturo: ok, I’ll make a file on a bastion
2018-08-09 12:23:41 <MacFan4000> On tools-bastion-03 my home dir doesn’t exist
2018-08-09 12:24:07 <MacFan4000> I haven’t become a tool at all
2018-08-09 12:24:34 <MacFan4000> I can’t create the dir
2018-08-09 12:28:01 <arturo> mmm
2018-08-09 12:28:15 <arturo> and what if you become a tool?
2018-08-09 12:31:37 <MacFan4000> Still can’t create it
2018-08-09 12:34:38 <Murbaut> Hi
2018-08-09 12:34:40 <arturo> MacFan4000 what is the error message?
2018-08-09 12:36:40 <MacFan4000>
2018-08-09 12:36:58 <MacFan4000> It’s cause /home is owned by root
2018-08-09 12:36:58 <arturo> I see a `legofan400` home directory which belongs to the `macfan4000` user
2018-08-09 12:37:06 <MacFan4000> Oh
2018-08-09 12:37:23 <MacFan4000> Guess my dir was never renamed
2018-08-09 12:39:23 <arturo> <slow NFS>
2018-08-09 12:40:16 <MacFan4000>
2018-08-09 12:40:30 <arturo> that's legit
2018-08-09 12:42:14 <arturo> MacFan4000: don't try to write anything outside your home
2018-08-09 12:42:43 <arturo> and /home is not a home of an user ... @_@
2018-08-09 12:42:56 <arturo> so, for example, this works for me
2018-08-09 12:43:43 <arturo> `cd && touch test` as your macfan4000 user
2018-08-09 12:43:56 <arturo> ^^^ MacFan4000 try that please :-P
2018-08-09 12:45:48 <Samwalton9> Maybe an odd request, and also a long shot, but would anyone here have time to help with deploying a tool on a horizon project instance? The developers who worked on it don't have the time, and I'm fumbling around in the dark trying to set it up. It's using docker, and I've managed to get it running, but don't seem to be able to hook everything up correctly for external web access.
2018-08-09 12:46:42 <arturo> Samwalton9: with horizon project instance I understand a Cloud VPS instance/virtual machine, right?
2018-08-09 12:46:50 <Samwalton9> Yep
2018-08-09 12:48:24 <Samwalton9> Specifically I'm trying to deploy via the wikicitevis project.
2018-08-09 12:49:05 <arturo> for inbound connections we would need either a NAT or a proxy
2018-08-09 12:49:26 <arturo> (if I understand correctly)
2018-08-09 12:52:11 <Samwalton9> So I think I have a web proxy set up
2018-08-09 12:52:53 <Samwalton9> From my limited understanding I think the issue is most likely in the docker and file structure setup somewhere
2018-08-09 12:54:43 <arturo> Samwalton9: you set the web proxy, using horizon, right? let me jump to your project and review the setup
2018-08-09 12:54:51 <Samwalton9> Yeah
2018-08-09 12:54:54 <Samwalton9> Thanks :)
2018-08-09 12:57:05 <arturo> ok, so your docker setup should listen in this URL:
2018-08-09 12:58:16 <arturo> is this file how your are configuring docker?
2018-08-09 12:58:31 <arturo> bc it contains not a single mention to :8080
2018-08-09 12:58:44 <arturo> Samwalton9: ^^^
2018-08-09 12:58:53 <Samwalton9> Yeah I think the only things I tried to change were some nginx config files that were pointing to localhost
2018-08-09 12:59:25 <Samwalton9> Hmm, I think I went with 8080 because of, but perhaps it should be 8000 then?
2018-08-09 13:00:01 <arturo> well, I would say the proxy backend and the port you are actually using in docker should match
2018-08-09 13:00:24 <arturo> (i.e, the number in horizon and your docker config)
2018-08-09 13:00:48 <Samwalton9> That would make sense - let me play around with that
2018-08-09 13:01:07 <arturo> ok, let us know
2018-08-09 13:01:17 <Murbaut> y
2018-08-09 13:01:38 <arturo> hey Murbaut, do you need some help?
2018-08-09 13:01:57 <Murbaut> no, thanks very much
2018-08-09 13:02:07 <arturo> ok :-)
2018-08-09 13:02:35 <MacFan4000> arturo: /home/legofan4000/2fa.txt
2018-08-09 13:03:13 <MacFan4000> Sorry for delay but my WiFi connection is having issues
2018-08-09 13:03:20 <arturo> MacFan4000: ok :-)
2018-08-09 13:05:40 <arturo> MacFan4000: what is your wikitech username?
2018-08-09 13:05:42 <Samwalton9> Hmm. Modified all 8000 ports to 8080, and added the horizon IP everywhere it seemed it should be, gunicorn now reporting it's listening on :8090 for some reason.
2018-08-09 13:06:10 <Samwalton9> One of the devs is going to give me some notes on what needs doing, so I think I'll wait for that. Thanks for the help so far arturo :)
2018-08-09 13:06:27 <arturo> you are welcome Samwalton9 :-)
2018-08-09 13:07:22 <MacFan4000> MacFan4000
2018-08-09 13:07:47 <arturo> OATHAuth disabled for MacFan4000 <--- done :-)
2018-08-09 16:04:40 <wikibugs_> (PS211) Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] -
2018-08-09 16:08:56 <wikibugs_> (CR) jerkins-bot: [V: -1] Initial commit [labs/tools/wikidata-slicer] - (owner: Ricordisamoa)
2018-08-09 16:10:54 <Reedy> wonders why they haven't just merged that commit yet
2018-08-09 16:11:33 <paladox> 211 patch sets wow
2018-08-09 16:11:40 <wikibugs_> (CR) Ricordisamoa: [C: -2] "PS211 adds help text to invalid entity inputs" [labs/tools/wikidata-slicer] - (owner: Ricordisamoa)
2018-08-09 16:12:13 <Reedy> And nearly 3 years
2018-08-09 16:13:16 <wikibugs_> (CR) Reedy: "Can I ask... Why are you just doing commits/amends onto this change set... It seems well beyond an initial commit now, and going on 3 year" [labs/tools/wikidata-slicer] - (owner: Ricordisamoa)
2018-08-09 16:13:39 <wikibugs_> (PS212) Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] -
2018-08-09 16:13:44 <zhuyifei1999_> @seen Framawiki
2018-08-09 16:13:44 <wm-bot> zhuyifei1999_: I have never seen Framawiki
2018-08-09 16:16:19 <WikidataFacts> can someone take a brief look at T201625 ? it’s not urgent, I just want to know if I added the right projects so the right people will see it :)
2018-08-09 16:16:20 <stashbot> T201625: Review Wikidata Shape Expressions Inference tool regarding Toolforge Rule 6 -
2018-08-09 16:25:45 <wm-bot> I will let you know when I see WikidataFacts and I will deliver that message to them
2018-08-09 16:25:45 <zhuyifei1999_> @notify WikidataFacts I /think/ 'The tool only allows authenticated users ... attributed to the user who started it' is okay, although I can't say for sure
2018-08-09 16:31:32 <wikibugs_> (CR) Ricordisamoa: [C: -2] "@Reedy" [labs/tools/wikidata-slicer] - (owner: Ricordisamoa)
2018-08-09 16:32:21 <wikibugs_> (CR) Ricordisamoa: [C: -2] "PS212 removes unused defaultdict import" [labs/tools/wikidata-slicer] - (owner: Ricordisamoa)
2018-08-09 17:56:29 <framawiki> zhuyifei1999_: sure :)
2018-08-09 17:56:39 <zhuyifei1999_> ok :)
2018-08-09 18:01:04 <zhuyifei1999_> I'm splitting the db out of the web server
2018-08-09 18:40:52 <zhuyifei1999_> framawiki: do you think we should make mwoauth a submodule?
2018-08-09 18:41:03 <zhuyifei1999_> I can't find the APT package for it
2018-08-09 18:41:18 <zhuyifei1999_> (submodule as in git)
2018-08-09 18:42:09 <framawiki> i don't really like git submodules, but if it's necessary...
2018-08-09 18:42:23 <framawiki> why not moving all py dependencies to a venv ?
2018-08-09 18:42:56 <framawiki> probably not easy to puppetize
2018-08-09 18:43:43 <zhuyifei1999_> I really wonder how ORES does this
2018-08-09 18:43:54 <zhuyifei1999_> halfak|Lunch:
2018-08-09 18:44:32 <zhuyifei1999_> how does ORES deal with pip packages not in apt?
2018-08-09 18:45:00 <awight> zhuyifei1999_: Ah yeah good question. By distributing a submodule of wheels in our deployment repo. Let me send links,
2018-08-09 18:45:55 <awight>
2018-08-09 18:46:40 <zhuyifei1999_> okay. how do they get applied?
2018-08-09 18:47:24 <awight> Here’s how they’re built:
2018-08-09 18:47:34 <awight> They’re applied at runtime with virtualenv,
2018-08-09 18:47:49 <awight>
2018-08-09 18:50:37 <zhuyifei1999_> I don't think changing the out deployment mechanism to scap is do-able in the near-term without investing a lot of time on the setup
2018-08-09 18:51:06 <zhuyifei1999_> but thanks for the pointers. I'm looking into it
2018-08-09 19:12:55 <twentyafterfour> Can anyone help me figure out why labtestweb2001 is still trying to send writes to a read-only database master? it's spamming logstash
2018-08-09 19:43:27 <Krenair> twentyafterfour, I think the labtest keystone DB is read-only
2018-08-09 19:43:34 <Krenair> because it lives in codfw
2018-08-09 19:43:50 <Krenair> or something
2018-08-09 19:44:35 <twentyafterfour> right I know why the db is read only. I guess I'm trying to figure out how to get labtestweb to stop connecting to it
2018-08-09 19:44:49 <twentyafterfour> I guess I can just ignore it for now
2018-08-09 19:47:16 <Krenair> twentyafterfour, oh I shouldn't think you'd be able to do that properly
2018-08-09 19:50:52 <Krenair> maybe you could try telling mediawiki that the wiki is read only
2018-08-09 19:51:15 <Krenair> but I don't know if that'd get everything to stop
2018-08-09 19:52:08 <Krenair> real solution really is just to use a DB host that isn't read-only
2018-08-09 19:55:12 <zhuyifei1999_> framawiki: I'm thinking of generating requireemets.txt using, considering we are no longer pinned by the version provided by APT and can choose whichever version we want
2018-08-09 19:55:24 <bd808> twentyafterfour, Krenair: yeah, we need to point labtestweb at a db server in eqiad rather than codfw. I think there is an open ticket about that
2018-08-09 19:57:32 <Krenair> at some point I lost access to my labtestwiki account but couldn't get it restored because the db is readonly :(
2018-08-09 20:13:25 <framawiki> zhuyifei1999_: yes, why not
2018-08-09 20:13:37 <zhuyifei1999_> k :)
2018-08-09 20:16:42 <framawiki> will take a close look this weekend
2018-08-09 23:53:26 <huh_> !help Is anyone around to help debug a prominent tool labs project? (not mine)
2018-08-09 23:53:26 <wm-bot> huh_: If you don't get a response in 15-30 minutes, please create a phabricator task --
2018-08-09 23:55:29 <huh_> The problem is with WikiMiniAtlas. If you don't know what that is, go to the top right corner of any location article on en.wikipedia (and a bunch of other wikis) and click the globe icon
2018-08-09 23:55:39 <huh_> I became aware of this issue from seeing
2018-08-09 23:56:27 <huh_> WikiMiniAtlas used to display links to nearby articles on the map, e.g. see
2018-08-09 23:57:06 <huh_> Now it doesn't, because the server-side database query fails.
2018-08-09 23:58:04 <huh_> This bug was reported in December 2017 ( and the OP of the Wikimedia Forum section says they have tried to contact the author multiple times since then but has gotten no response. (Granted, I'm guilty of this too)
2018-08-10 00:00:28 <huh_> I was able to debug this a bit on the client site. Namely, I saw that WikiMiniAtlas is making XHR requests such as, which just returns a page containing the comment "<!-- error: Too many database connections -->"
2018-08-10 00:02:16 <huh_> label.php used to return a JSON list of articles within a specified coordinate range, e.g.
2018-08-10 00:03:00 <huh_> If someone could determine why this is broken now and how to fix it, that would be great
2018-08-10 00:03:20 <huh_> "There was a database restructuring on the WMF cloud services that - besides reliability improvements - broke a ton of stuff. I'll work on this tomorrow and see how much can be salvaged :-/"
2018-08-10 00:05:02 <chicocvenancio> Is that last message quoting someone?
2018-08-10 00:05:38 <huh_> Yeah, it's from the author of the script (from the GitHub issue)
2018-08-10 00:07:16 <huh_> Here is the code for label.php where it tries to connect to SQL and execute the query, but returns the "error: Too many database connections"
2018-08-10 00:07:18 <huh_>
2018-08-10 00:08:51 <chicocvenancio> huh_: if there are active members of the project, it is better to ask them for support
2018-08-10 00:09:11 <huh_> how can I see the list of members?
2018-08-10 00:13:32 <huh_> I guess it's somewhere on but I don't know where to look. chicocvenancio, any clue?
2018-08-10 00:16:39 <huh_> Okay, found it:
2018-08-10 00:20:26 <chicocvenancio> sorry I was on mobile. good you found it
2018-08-10 00:25:40 <chicocvenancio> huh_: looking at the code you pointed to, it'll take some work to get this working
2018-08-10 00:27:22 <huh_> Darn, I was hoping it was going to be a simple fix. What makes this it seem difficult to fix?
2018-08-10 00:32:06 <huh_> s/this it/it/
2018-08-10 00:32:48 <chicocvenancio> well, in the past the database server for wiki replicas and for user databases were the same
2018-08-10 00:32:55 <chicocvenancio> now they are separate
2018-08-10 00:33:50 <chicocvenancio> so things like are not possible in sql
2018-08-10 00:34:13 <huh_> Oh dang
2018-08-10 00:34:24 <chicocvenancio> tools need to open separate connections to the servers and join with application logic
2018-08-10 00:38:42 <huh_> Thank you for explaining the issue! I wasn't aware of that
2018-08-10 00:39:27 <chicocvenancio> there is documentation for this on wikitech somewhere... and there were a few emails on the cloud mailing lists as well
2018-08-10 00:43:18 <huh_> Looks like this is the thread:
2018-08-10 00:44:50 <huh_> "Death blow for GHEL coordinate extraction and WikiMiniAtlas. 🙁" -dschwen (WMA author), cf
2018-08-10 00:48:08 <huh_> Daniel also has a couple comments on that mailing list thread, both pretty pessimistic about porting WMA to use application logic joins
2018-08-10 00:48:17 <chicocvenancio> it sounds like you might need to convince another developer to do this
2018-08-10 00:53:41 <huh_> Yeah, it sounds like Daniel just gave up
2018-08-10 00:54:23 <huh_> because he has a limited time budget, and also because application joins would likely be very slow
2018-08-10 00:54:49 <huh_> I guess I'll just post a link to this log on the Wikimedia Forum thread and ping some of the administrators of the project.
2018-08-10 00:55:19 <chicocvenancio> well, there are technical solutions, but yeah, volunteers choose on what they work
2018-08-10 00:55:23 <huh_> Also here is the wikitech article about the change:
2018-08-10 00:56:32 <huh_> Thanks, you've been extremely helpful
2018-08-10 00:56:48 <chicocvenancio> you're welcome
2018-08-10 01:48:37 <huh_> chicocvenancio, I responded at
2018-08-10 01:51:20 <huh_> Wouldn't the complexity of an application logic join be O(M * N)?
2018-08-10 01:55:41 <chicocvenancio> Ideally the complexity should be the same as the original join, IMHO.
2018-08-10 01:56:13 <chicocvenancio> The db side join doesn't magically do away with complexity
2018-08-10 01:57:52 <chicocvenancio> Coding it in a sane way is not trivial, but not impossible. It is an app with two data sources.
2018-08-10 01:59:22 <huh_> So you'd end up rewriting an SQL join operation from scratch in PHP?
2018-08-10 01:59:50 <SQL> huh_: sometimes that's faster - yes.
2018-08-10 01:59:57 <huh_> relevant username
2018-08-10 02:00:04 <huh_> lol
2018-08-10 02:00:11 <SQL> Yeah, you've pinged me a few times now lol
2018-08-10 02:00:21 <huh_> Sorry about that.
2018-08-10 02:00:26 <SQL> nbd
2018-08-10 02:00:52 <SQL> There have been times tho that I've found it quicker to do the processing / filtering in a scripting language
2018-08-10 02:00:55 <SQL> than in a query
2018-08-10 02:02:42 <chicocvenancio> huh_: *I'd* end up writing the whole thing from scratch in python. But yeah, any language will do
2018-08-10 02:08:10 <chicocvenancio> has a bad history with php
2018-08-10 02:15:03 <huh_> strongly prefers python as well
2018-08-10 12:45:20 <wikibugs_> (PS1) D3r1ck01: Add few more channels to #wikimedia-dev-africa [labs/tools/wikibugs2] -
2018-08-10 12:47:37 <wikibugs_> (PS2) D3r1ck01: Add few more gerrit projects to #wikimedia-dev-africa [labs/tools/wikibugs2] -
2018-08-10 19:01:52 <wikibugs_> (CR) Merlijn van Deen: [C: 2] Add few more gerrit projects to #wikimedia-dev-africa [labs/tools/wikibugs2] - (owner: D3r1ck01)
2018-08-10 19:02:21 <wikibugs_> (Merged) jenkins-bot: Add few more gerrit projects to #wikimedia-dev-africa [labs/tools/wikibugs2] - (owner: D3r1ck01)
2018-08-10 19:02:32 <wikibugs_> (CR) jenkins-bot: Add few more gerrit projects to #wikimedia-dev-africa [labs/tools/wikibugs2] - (owner: D3r1ck01)
2018-08-10 20:40:04 <wikibugs_> (PS1) MarcoAurelio: Remove #Anti-Harassment notifications from #wikimedia-commtech [labs/tools/wikibugs2] - (
2018-08-10 20:41:36 <wikibugs_> (PS2) MarcoAurelio: Remove #Anti-Harassment notifications from #wikimedia-commtech [labs/tools/wikibugs2] - (
2018-08-10 21:45:37 <wikibugs_> (CR) Merlijn van Deen: [C: 2] Remove #Anti-Harassment notifications from #wikimedia-commtech [labs/tools/wikibugs2] - ( (owner: MarcoAurelio)
2018-08-10 21:46:07 <wikibugs_> (Merged) jenkins-bot: Remove #Anti-Harassment notifications from #wikimedia-commtech [labs/tools/wikibugs2] - ( (owner: MarcoAurelio)
2018-08-10 21:46:15 <wikibugs_> (CR) jenkins-bot: Remove #Anti-Harassment notifications from #wikimedia-commtech [labs/tools/wikibugs2] - ( (owner: MarcoAurelio)
2018-08-10 21:51:09 <wikibugs_> !log tools.wikibugs Updated channels.yaml to: 181a9422666bbb5d8ad2c25253909aabe44dddec Merge branch 'master' of
2018-08-10 21:51:11 <stashbot> Logged the message at

This page is generated from SQL logs, you can also download static txt files from here