[00:11:52] 10Wikimedia-Apache-configuration, 06Discovery, 06Operations, 07Mobile, 13Patch-For-Review: m.wikipedia.org incorrectly redirects to en.m.wikipedia.org - https://phabricator.wikimedia.org/T69015#2840368 (10Deskana) >>! In T69015#2840271, @debt wrote: > Is there a chance that this will actually be implemen... [01:28:20] 10Wikimedia-Apache-configuration, 06Discovery, 06Operations, 07Mobile, 13Patch-For-Review: m.wikipedia.org incorrectly redirects to en.m.wikipedia.org - https://phabricator.wikimedia.org/T69015#730151 (10MaxSem) ``` ServerName m.wikipedia.<%= @domain_suffix %> ServerAlias zero.... [06:02:13] 10netops, 06Operations: asw2-d-eqiad.mgmt.eqiad - JNX_ALARMS CRITICAL - 2 red alarms, - https://phabricator.wikimedia.org/T152182#2840869 (10Dzahn) [08:32:37] 10Wikimedia-Apache-configuration, 06Discovery, 06Operations, 07Mobile, 13Patch-For-Review: m.wikipedia.org incorrectly redirects to en.m.wikipedia.org - https://phabricator.wikimedia.org/T69015#2840986 (10Krenair) We could just split zero to a separate VHost? [10:04:18] 10Wikimedia-Apache-configuration, 06Discovery, 06Operations, 07Mobile, 13Patch-For-Review: m.wikipedia.org incorrectly redirects to en.m.wikipedia.org - https://phabricator.wikimedia.org/T69015#2841133 (10Dereckson) a:05Dereckson>03None [10:05:16] 10Wikimedia-Apache-configuration, 06Discovery, 06Operations, 07Mobile, 13Patch-For-Review: m.wikipedia.org incorrectly redirects to en.m.wikipedia.org - https://phabricator.wikimedia.org/T69015#730151 (10Dereckson) >>! In T69015#2840986, @Krenair wrote: > We could just split zero to a separate VHost? Ye... [10:07:37] bblack: Hi! re: T152122 - I can see that ottomata followed up in the #analytics channel but not in the ticket, will try to update it. Moreover the pivot deployment was for thorium, new replacement for stat1001, not related :) [10:07:37] T152122: Central Notice: possible CN issue early on December 1st UTC - https://phabricator.wikimedia.org/T152122 [10:07:58] Krenair: interesting! If you could add me to them I can take a look :) [10:13:18] it seems that the reporter of the Central Notice issue is going to check webrequest logs on hive [10:13:36] but after that I can't see anything else in the chan logs [10:34:42] 10Traffic, 06Operations: Block hotlinking - https://phabricator.wikimedia.org/T152091#2841156 (10Gilles) Great points @valhallasw. It'd be nice to know how much it really costs, though, even if it's already priced into the current infrastructure. When I heard that it represented a large share of our traffic i... [14:35:20] 10netops, 06Operations, 10ops-eqiad: asw2-d-eqiad.mgmt.eqiad - JNX_ALARMS CRITICAL - 2 red alarms, - https://phabricator.wikimedia.org/T152182#2841599 (10faidon) Thanks @Dzahn. The alerts seem to be: ``` 2016-12-01 16:04:45 UTC Major FPC 1 PEM 0 is not powered 2016-12-01 16:04:41 UTC Major Management Eth... [14:35:43] varnish 4.1.4 is out https://github.com/varnishcache/varnish-cache/blob/4.1/doc/changes.rst [14:35:53] including a fix for https://github.com/varnishcache/varnish-cache/issues/2035 [14:52:50] 10Traffic, 10Citoid, 06Operations, 10RESTBase, and 5 others: Set-up Citoid behind RESTBase - https://phabricator.wikimedia.org/T108646#2841648 (10faidon) Citoid is already fronted by our #Traffic infrastructure (Varnish), which is obviously a layer capable of caching, with cache hits there being obviously... [15:21:45] 10Traffic, 06Analytics-Kanban, 06Operations, 13Patch-For-Review: Ganglia varnishkafka python module crashing repeatedly - https://phabricator.wikimedia.org/T152093#2841735 (10elukey) a:03elukey [16:26:00] 10Traffic, 10Citoid, 06Operations, 10RESTBase, and 5 others: Set-up Citoid behind RESTBase - https://phabricator.wikimedia.org/T108646#2841876 (10GWicke) @faidon: Request rates are very low, and have a wide spread. By the time a given URL or DOI is requested again, the response will very likely have fallen... [16:46:26] 10netops, 06Labs, 10Labs-Infrastructure, 06Operations, 10wikitech.wikimedia.org: Move novaobserver (and novaadmin) users out of ldap - https://phabricator.wikimedia.org/T152215#2841937 (10Andrew) [16:51:16] 10netops, 06Labs, 10Labs-Infrastructure, 06Operations, and 2 others: Provide instance level ro access to OpenStack APIs - https://phabricator.wikimedia.org/T150092#2841969 (10chasemp) [16:55:02] ema: nice [16:56:36] 10netops, 06Labs, 10Labs-Infrastructure, 06Operations, and 2 others: Provide read-only access to OpenStack APIs from WMF IP space - https://phabricator.wikimedia.org/T150092#2841976 (10chasemp) [16:57:23] ema, it's the chain starting at https://gerrit.wikimedia.org/r/#/c/322601/ [16:58:05] technically there's still a realm distinction buried in there but it's scope is massively reduced and we can look at killing that later [16:58:57] 10netops, 06Operations, 10ops-eqiad: asw2-d-eqiad.mgmt.eqiad - JNX_ALARMS CRITICAL - 2 red alarms, - https://phabricator.wikimedia.org/T152182#2841978 (10Cmjohnson) I fixed the power issue but not sure what this error is referencing. Everything looks up based on physical inspection. 2016-12-01 16:04:41 UTC... [16:59:22] I split the changes over 4 commits that should all be valid in sequence [17:04:03] 10Traffic, 10Citoid, 06Operations, 10RESTBase, and 5 others: Set-up Citoid behind RESTBase - https://phabricator.wikimedia.org/T108646#2841986 (10mobrovac) [17:04:46] 10Traffic, 10Citoid, 06Operations, 10RESTBase, and 5 others: Set-up Citoid behind RESTBase - https://phabricator.wikimedia.org/T108646#1526094 (10mobrovac) @Mvolz would you mind looking into making the Citoid extension using the RESTBase endpoint instead of placing a call to `citoid.wikimedia.org` ? [17:07:15] It'll need more approvals to get done, but should be easier to review [17:12:48] will try to check it next week :) [17:18:17] ty [21:45:19] 10Traffic, 10Citoid, 06Operations, 10RESTBase, and 5 others: Set-up Citoid behind RESTBase - https://phabricator.wikimedia.org/T108646#2842822 (10kaldari)