[01:03:59] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3916113 (10Tbayer) >>! In T185350#3913845, @Nuria wrote: ... > it is likely that for media downloaded in chunks the field doesn't reflect file s... [14:06:17] 10Traffic, 10Operations, 10Puppet: Puppet hosts with signed certificate present on agent but not master - https://phabricator.wikimedia.org/T185239#3917317 (10herron) [15:54:11] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3917572 (10Tbayer) [16:11:10] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3917622 (10Nuria) >What is the argument for assuming that this data is correct? The data for the pdf file transfer looks as you would expect if... [16:25:51] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3917729 (10Tbayer) >>! In T185350#3917622, @Nuria wrote: >>What is the argument for assuming that this data is correct? > The data for the pdf f... [17:53:01] 10Traffic, 10Operations, 10Puppet: Puppet hosts with signed certificate present on agent but not master - https://phabricator.wikimedia.org/T185239#3910613 (10akosiaris) ganeti2001, wtp2005, wtp2020 only use that cert for rsyslog and not for any service so we can refresh the puppet certs for them whenever we... [19:37:48] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3918246 (10Nuria) >Also, note in both cases (response_size 18550 and 15722), these weren't partial requests either (the status code was 200, not... [19:58:51] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3918346 (10Tbayer) @ottomata notes that the response_size field should correspond to the "Size of response in bytes, excluding HTTP headers" out... [20:01:43] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3918360 (10Ottomata) Thanks @Tbayer, meant to write that here too :) I'd be very surprised if the value of `%b` from varnishlog which is sent a... [20:06:07] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3918374 (10Tbayer) In the meantime, I ran a query to estimate how much data was transferred in the download direction last month overall *if* th... [20:29:06] 10Traffic, 10Operations, 10Puppet: Puppet hosts with signed certificate present on agent but not master - https://phabricator.wikimedia.org/T185239#3918682 (10herron) [20:30:02] 10Traffic, 10Operations, 10Puppet: Puppet hosts with signed certificate present on agent but not master - https://phabricator.wikimedia.org/T185239#3910613 (10herron) Thanks @akosiaris! ganeti2001, wtp2005, wtp2020 puppet certs have been refreshed. [21:35:20] 10netops, 10Operations, 10ops-codfw: rack spare switches in c1-codfw - https://phabricator.wikimedia.org/T185336#3918812 (10Papaul) serial ports information ex4300-spare1-codfw = port 39 ex4300-spare2-codfw = port 40 qfx5100-spare1-codfw = port 41 qfx5100-spare2-codfw =port 42 [21:35:37] 10netops, 10Operations, 10ops-codfw: rack spare switches in c1-codfw - https://phabricator.wikimedia.org/T185336#3918813 (10Papaul) [21:50:01] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3913807 (10faidon) >>! In T185350#3918374, @Tbayer wrote: > In the meantime, I ran a query to estimate how much data was transferred in the down... [22:05:35] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3918834 (10Tbayer) OK, I just launched the below query for ulsfo - will report the result here once it has completed. ```lang=sql SELECT SUM(... [22:58:14] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3919049 (10Tbayer) @faidon: The ulsfo result for December is 2018 (decimal) terabytes. Plausible? ``` total_bytes requests 2017852005519238... [23:06:45] 10Traffic, 10Analytics-Data-Quality, 10Operations: Vet reliability of the response_size field for data analysis purposes - https://phabricator.wikimedia.org/T185350#3919078 (10faidon) Interesting! So with a ratio in:out of approximately 25:1 (based on January's figures), this means that we could estimate the...