[10:11:27] 10Traffic, 10Operations, 10serviceops, 10Patch-For-Review: Certificate *.wikipedia.org valid until 2020-06-20 - https://phabricator.wikimedia.org/T251726 (10Dzahn) >>! In T251726#6109023, @Vgutierrez wrote: > the icinga check on cp hosts currently warns 30 days before and goes critical 15 days before cert... [10:18:51] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) 05Resolved→03Open IC-313592 is down again Transport: cr2-eqord:xe-0/1/1 (Telia, IC-313592, 51ms 10Gbps wave) {#1062}; Transport: cr3-ulsfo:xe-0/1/1 (Telia, IC-313592, 51ms 10Gbps wave)... [10:24:21] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) sent mail to Telia [13:55:38] 10Traffic, 10Operations, 10serviceops, 10Patch-For-Review: Certificate *.wikipedia.org valid until 2020-06-20 - https://phabricator.wikimedia.org/T251726 (10Vgutierrez) if every LE certificate checked by that icinga check it's issued by acme-chief then yes, it's good [14:22:42] 10Traffic, 10Operations, 10Patch-For-Review, 10Performance-Team (Radar): Better handling for one-hit-wonder objects - https://phabricator.wikimedia.org/T144187 (10ema) I have tested the exp admission policy based on probability exponentially decreasing with object size on cp2027 (cache_text) and cp2028 (ca... [15:52:19] 10Traffic, 10Analytics, 10Analytics-Kanban, 10Operations, 10Patch-For-Review: Remove North Korea from data quality traffic entropy reports - https://phabricator.wikimedia.org/T251546 (10Milimetric) p:05Triage→03High [16:31:30] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations: Caching of https://doc.wikimedia.org/cover/mediawiki-libs-IPUtils/IPUtils.php.html is inconsistent - https://phabricator.wikimedia.org/T252131 (10Reedy) [16:32:39] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations: Caching of https://doc.wikimedia.org/cover/mediawiki-libs-IPUtils/IPUtils.php.html is inconsistent - https://phabricator.wikimedia.org/T252131 (10Reedy) [16:36:10] 10Traffic, 10Operations: Deploy Wikidough: Experimental DNS-over-HTTPS (DoH) public resolver - https://phabricator.wikimedia.org/T252132 (10ssingh) [16:38:28] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations: Caching of https://doc.wikimedia.org/cover/mediawiki-libs-IPUtils/IPUtils.php.html is inconsistent - https://phabricator.wikimedia.org/T252131 (10hashar) Must be a stale cache somewhere. It would help to have the headers. I haven't looked at t... [16:39:19] 10Traffic, 10Cloud-Services, 10Operations, 10Wikimedia-Incident, 10cloud-services-team (Kanban): Requests to production are sometimes timing out or giving empty response - https://phabricator.wikimedia.org/T249035 (10CDanis) Are these issues still ongoing? [16:41:48] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations: Caching of https://doc.wikimedia.org/cover/mediawiki-libs-IPUtils/IPUtils.php.html is inconsistent - https://phabricator.wikimedia.org/T252131 (10Reedy) ` HTTP/2 200 OK date: Thu, 07 May 2020 15:42:30 GMT server: Apache content-security-policy... [16:48:02] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations: Caching of https://doc.wikimedia.org/cover/mediawiki-libs-IPUtils/IPUtils.php.html is inconsistent - https://phabricator.wikimedia.org/T252131 (10Reedy) ` HTTP/2 200 OK date: Thu, 07 May 2020 16:46:02 GMT server: Apache content-security-policy... [16:48:27] Well, this is fun [17:09:29] 10Traffic, 10Operations: Deploy Wikidough: Experimental DNS-over-HTTPS (DoH) public resolver - https://phabricator.wikimedia.org/T252132 (10ssingh) p:05Triage→03Medium [17:10:00] Reedy: just a moment [17:18:39] across all the cp-texts I see the same output hash [17:18:43] different date headers but I think that's fine? [17:21:01] https://phabricator.wikimedia.org/P11172 [17:30:06] 10Traffic, 10Cloud-Services, 10Operations, 10Wikimedia-Incident, 10cloud-services-team (Kanban): Requests to production are sometimes timing out or giving empty response - https://phabricator.wikimedia.org/T249035 (10DB111) In my case last time at 5 a.m. (UTC) of 04/12, so running fine for nearly one mon... [17:58:06] cdanis: But it's showing different content [17:58:13] I am not sure why or how [17:58:17] is it still? [17:59:14] Seemingly not now [17:59:31] Seems to be consistently 304-ing [20:13:01] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations: Caching of https://doc.wikimedia.org/cover/mediawiki-libs-IPUtils/IPUtils.php.html is inconsistent - https://phabricator.wikimedia.org/T252131 (10colewhite) p:05Triage→03Medium