[01:05:29] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3242374 (10Niharika) Alright, so I got the same Out of memory error on tool labs for project Biography. Here's the stack trace: ``` PHP Fatal error: Out of mem... [06:17:24] PROBLEM - Webrequests Varnishkafka log producer on cp4007 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [06:22:25] RECOVERY - Webrequests Varnishkafka log producer on cp4007 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf [06:40:05] cp4007 died because of OOM this morning --^ [08:22:58] 06Analytics-Kanban: Update mediawiki history oozie SLA - https://phabricator.wikimedia.org/T164713#3242850 (10JAllemandou) [08:23:24] 06Analytics-Kanban: Update mediawiki history oozie SLA - https://phabricator.wikimedia.org/T164713#3242862 (10JAllemandou) a:03JAllemandou [08:23:48] (03PS1) 10Joal: Update mediawiki history oozie job SLA [analytics/refinery] - 10https://gerrit.wikimedia.org/r/352548 (https://phabricator.wikimedia.org/T164713) [08:24:17] 06Analytics-Kanban, 13Patch-For-Review: Update per-hosts-uniques oozie job to match new global ones - https://phabricator.wikimedia.org/T164607#3242868 (10JAllemandou) a:03JAllemandou [08:27:30] (03CR) 10Joal: [V: 032 C: 032] "Answering to mforns and merging." (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/349266 (https://phabricator.wikimedia.org/T163479) (owner: 10Joal) [08:30:53] 06Analytics-Kanban, 13Patch-For-Review: Add zero carrier to pageview_hourly data on druid - https://phabricator.wikimedia.org/T161824#3144399 (10JAllemandou) Field is now present in pageview-hourly, but still missing from pageview-daily :( [08:31:57] 06Analytics-Kanban: Unique Devices on Pivot, initial screen should not add values by default, is this configurable? - https://phabricator.wikimedia.org/T164194#3225083 (10JAllemandou) Waiting for new versions of druid/pivot. [08:44:30] joal: plan for the day - I am going to reboot two kafka nodes for kernel upgrades plus all the cassandra instances on AQS for jvm upgrades [08:45:02] elukey: ok, well noted [08:45:09] elukey: thanks for that - not fun work [08:45:33] joal: it improves our infrastructure's procedures :P [08:46:10] elukey: knowing you're a cumin expert now, I guess it's becomning less and less of pain [08:47:30] joal: Cassandra is always Cassandra, even if with a bit of cumin :D [08:47:45] huhuhu :) [10:20:16] Hi nuria_, it's fun to have notifications from you at this local-time of the day :) [10:20:54] joal: ayay [11:15:30] milimetric: not working much longer today but if you deploy dashiki extension config today send me a note and i will deploy changes to reportcard tomorrow [11:29:55] 06Analytics-Kanban: Load pivot pageview-hourly dataset every hour - https://phabricator.wikimedia.org/T164730#3243337 (10Nuria) [11:30:44] AQS jvm rolling restart completed [11:47:25] just discovered that I'll also need to perform reboots for kernel upgrades in there, but I'll schedule it later on this week [11:49:08] * elukey lunch! [12:40:49] taking a break a-team [12:52:53] * fdans lunch! [13:25:40] (03PS1) 10Filippo Giunchedi: Reset signal disposition and unblock signals for children [analytics/kafkatee] - 10https://gerrit.wikimedia.org/r/352591 [13:53:07] 10Analytics, 10Analytics-General-or-Unknown: Provide regular cross-wiki reports on flagged revisions status - https://phabricator.wikimedia.org/T44360#3243851 (10Milimetric) Ok, thank you very much, I understand now, and I have added this topic of discussion in our feedback for the design / implementation of t... [14:02:36] 06Analytics-Kanban: Create purging script for mediawiki-history data - https://phabricator.wikimedia.org/T162034#3150343 (10Milimetric) p:05Triage>03Normal [14:02:53] 06Analytics-Kanban: Update druid unique Devices Dataset to only contain hosts having more than 1000 uniques - https://phabricator.wikimedia.org/T164183#3243921 (10Milimetric) p:05Triage>03Normal [14:03:05] 06Analytics-Kanban, 13Patch-For-Review: Update restbase oozie job - https://phabricator.wikimedia.org/T163479#3243922 (10Milimetric) p:05Triage>03Normal [14:03:12] 06Analytics-Kanban, 13Patch-For-Review: Update per-hosts-uniques oozie job to match new global ones - https://phabricator.wikimedia.org/T164607#3243923 (10Milimetric) p:05Triage>03Normal [14:03:14] 06Analytics-Kanban: Finalize list of metrics, breakdowns, and filters for Wikistats 2.0 backend - https://phabricator.wikimedia.org/T163356#3243924 (10Milimetric) p:05Triage>03Normal [14:03:25] 06Analytics-Kanban, 13Patch-For-Review: Update mediawiki history oozie SLA - https://phabricator.wikimedia.org/T164713#3243925 (10Milimetric) p:05Triage>03Normal [14:03:36] 06Analytics-Kanban: Unique Devices on Pivot, initial screen should not add values by default, is this configurable? - https://phabricator.wikimedia.org/T164194#3243926 (10Milimetric) p:05Triage>03Normal [14:03:42] 10Analytics-Dashiki, 06Analytics-Kanban, 05MW-1.29-release (WMF-deploy-2017-04-04_(1.29.0-wmf.19)), 13Patch-For-Review: Move Dashiki config from CommonSettings to extension - https://phabricator.wikimedia.org/T161038#3243939 (10Milimetric) p:05Triage>03Normal [14:03:43] 06Analytics-Kanban: Add monthly unique devices dataset to pivot - https://phabricator.wikimedia.org/T163327#3243940 (10Milimetric) p:05Triage>03Normal [14:03:58] 06Analytics-Kanban, 10Analytics-Wikistats, 13Patch-For-Review: Add "Interwicket" to the list of bots - https://phabricator.wikimedia.org/T154090#3243942 (10Milimetric) p:05Triage>03Normal [14:04:03] 06Analytics-Kanban: Collaborate with zero on asiacell report - https://phabricator.wikimedia.org/T161326#3243943 (10Milimetric) p:05Triage>03Normal [14:04:44] 06Analytics-Kanban: Load pivot pageview-hourly dataset every hour - https://phabricator.wikimedia.org/T164730#3243944 (10Milimetric) p:05Triage>03Normal [14:04:49] 06Analytics-Kanban: Provide unqiues estimate/offset breakdowns externally - https://phabricator.wikimedia.org/T164593#3243948 (10Milimetric) p:05Triage>03Normal [14:05:58] 06Analytics-Kanban: AQS alarms need to log to analytics channel - https://phabricator.wikimedia.org/T162407#3243949 (10Milimetric) p:05Triage>03Normal [14:06:05] 06Analytics-Kanban: Webrequest tagging and distribution. Measuring non-pageview requests - https://phabricator.wikimedia.org/T164019#3243950 (10Milimetric) p:05Triage>03Normal [14:06:08] 10Analytics-Cluster, 06Analytics-Kanban: Provision new Kafka clusters in eqiad and codfw with security features - https://phabricator.wikimedia.org/T152015#3243951 (10Milimetric) p:05Triage>03Normal [14:24:35] 10Analytics: Provide uniques estimate/offset breakdowns available in dumps - https://phabricator.wikimedia.org/T164597#3244026 (10Milimetric) p:05Triage>03Normal [14:24:48] 10Analytics: Provide uniques estimate/offset breakdowns available externally - https://phabricator.wikimedia.org/T164597#3239315 (10Milimetric) p:05Normal>03Triage [14:25:14] 06Analytics-Kanban: Provide uniques estimate/offset breakdowns available in dumps - https://phabricator.wikimedia.org/T164597#3239315 (10Milimetric) p:05Triage>03Normal [14:25:27] 10Analytics: Provide uniques offset/underestimate breakdowns in AQS - https://phabricator.wikimedia.org/T164596#3244036 (10Milimetric) p:05Triage>03Normal [14:26:17] 06Analytics-Kanban: Pageview hourly data in Pivot is not showing up correctly - https://phabricator.wikimedia.org/T164586#3244041 (10Milimetric) a:03JAllemandou [14:27:06] 10Analytics: Investigate whether we could calculate "hourly unique devices" - https://phabricator.wikimedia.org/T163789#3244045 (10Milimetric) p:05Triage>03Normal [14:27:11] 06Analytics-Kanban: Provide uniques estimate/offset breakdowns available in dumps - https://phabricator.wikimedia.org/T164597#3244050 (10JAllemandou) [14:27:13] 06Analytics-Kanban, 13Patch-For-Review: Update per-hosts-uniques oozie job to match new global ones - https://phabricator.wikimedia.org/T164607#3244052 (10JAllemandou) [14:27:26] 06Analytics-Kanban: Provide uniques estimate/offset breakdowns available in dumps - https://phabricator.wikimedia.org/T164597#3239315 (10JAllemandou) a:03JAllemandou [14:27:50] 10Analytics: Preserve userAgent field in apps schemas - https://phabricator.wikimedia.org/T164125#3244057 (10Milimetric) p:05Triage>03Normal [14:27:59] 10Analytics, 10Pageviews-API: Endpoint for average view rate in Pageview API - https://phabricator.wikimedia.org/T162933#3244058 (10Milimetric) p:05Triage>03Normal [14:28:28] 10Analytics: Non existing article is one of the most viewed according to the data returned by the /metrics/pageviews/top/ API - https://phabricator.wikimedia.org/T149178#3244076 (10Milimetric) p:05Triage>03Normal [14:29:11] 10Analytics, 06Editing-Analysis: Pivot "MediaWiki history" data lake: Feature request for "Time" dimension to split by calendar month / quarter / year - https://phabricator.wikimedia.org/T161186#3244079 (10Milimetric) p:05Triage>03Normal [14:29:19] 10Analytics, 10Analytics-Dashiki: Clean up remaining Dashiki configs on meta - https://phabricator.wikimedia.org/T159269#3244080 (10Milimetric) p:05Triage>03Normal [14:30:14] (03PS2) 10Joal: Update per host last access uniques oozie jobs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/352182 (https://phabricator.wikimedia.org/T164597) [14:31:08] 10Analytics, 06Editing-Analysis: Pivot "MediaWiki history" data lake: Feature request for "Event Users" - https://phabricator.wikimedia.org/T161185#3244098 (10Milimetric) p:05Triage>03Normal [14:31:20] 10Analytics: Prototype counting of requests with real time (streaming data) - https://phabricator.wikimedia.org/T159264#3244099 (10Milimetric) p:05Triage>03Normal [14:33:06] 10Analytics: Bot Identification: Inconsistent data in #all-sites-by-os-and-browser for IE7 - https://phabricator.wikimedia.org/T148461#3244109 (10Milimetric) [14:33:18] 10Analytics: Bot Identification: Inconsistent data in #all-sites-by-os-and-browser for IE7 - https://phabricator.wikimedia.org/T148461#2723684 (10Milimetric) p:05Triage>03Normal [14:33:29] 10Analytics, 10Pageviews-API: Track page views by page ID rather than title (handles moved pages) - https://phabricator.wikimedia.org/T159046#3244111 (10Milimetric) p:05Triage>03Normal [14:34:27] 10Analytics, 10Analytics-EventLogging, 10MediaWiki-extensions-General-or-Unknown, 07Technical-Debt: JsonData and EventLogging have multiple classes with the same name - https://phabricator.wikimedia.org/T159079#3244112 (10Milimetric) p:05Triage>03High [14:34:40] 10Analytics: Import 2001 wikipedia data - https://phabricator.wikimedia.org/T155014#3244114 (10Milimetric) p:05Triage>03Low [14:35:07] 10Analytics, 10ChangeProp, 10EventBus, 13Patch-For-Review, 06Services (later): Create schema for Job event - https://phabricator.wikimedia.org/T157094#3244116 (10Milimetric) p:05Triage>03Normal [14:35:18] 10Analytics: Measure Community Backlog. - https://phabricator.wikimedia.org/T155497#3244117 (10Milimetric) p:05Triage>03Normal [14:40:22] 10Analytics, 10Pageviews-API, 10RESTBase-API, 06Services (watching): Pageviews Data : removes 1000 limit in the most viewed articles for a given project and timespan API - https://phabricator.wikimedia.org/T153081#3244144 (10Milimetric) p:05Triage>03Low [14:41:15] 10Analytics: Adding breakdowns to mw edit history reconstruction: wiki projects, categories (cohort) - https://phabricator.wikimedia.org/T163113#3244145 (10Milimetric) p:05Triage>03Normal [14:43:48] 10Analytics: Pull data for edit reconstruction from labs and push it back after reconstruction - https://phabricator.wikimedia.org/T152788#3244149 (10Milimetric) p:05Triage>03Normal a:03JAllemandou [14:44:09] 10Analytics, 10Analytics-Dashiki: Just an idea: poly-graph - https://phabricator.wikimedia.org/T148469#3244164 (10Milimetric) p:05Triage>03Low [14:45:12] 10Analytics, 10Analytics-Dashiki: Provide filterable line graph for browser-family/browser-major - https://phabricator.wikimedia.org/T150713#3244165 (10Milimetric) p:05Triage>03Low [14:46:27] 10Analytics, 07Documentation: Document a proposal for bundling other than load-refine jobs together (see refine/diagram) - https://phabricator.wikimedia.org/T130734#3244166 (10Milimetric) 05Open>03declined We currently think that tagging/pipeline work we're doing now replaces this idea. [14:47:58] 10Analytics, 10Data-release: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#3244172 (10Milimetric) p:05Triage>03Normal [14:48:04] 10Analytics: productionize ClickStream dataset - https://phabricator.wikimedia.org/T158972#3244173 (10Milimetric) p:05Triage>03High [14:49:42] 10Analytics: Edit analysis dashboard Failures by User Type chart does not update correctly - https://phabricator.wikimedia.org/T148656#2729192 (10Milimetric) I wouln't just remove this chart from the dashboard, we can remove the stacked bar chart from Dashiki but that seems like a shame. [14:49:50] 10Analytics: Edit analysis dashboard Failures by User Type chart does not update correctly - https://phabricator.wikimedia.org/T148656#3244181 (10Milimetric) p:05Triage>03Normal [14:50:08] 10Analytics: Describe threat model for sanitized pageview data {mole} - https://phabricator.wikimedia.org/T131158#3244182 (10Milimetric) p:05Triage>03Normal [14:51:37] 10Analytics, 10Pageviews-API: Yearly endpoint for the /pageviews/top API - https://phabricator.wikimedia.org/T154381#3244183 (10Milimetric) p:05Triage>03Normal [14:53:54] 10Analytics, 06Labs, 10Pageviews-API, 10wikitech.wikimedia.org: wikitech.wikimedia.org missing from pageviews API - https://phabricator.wikimedia.org/T153821#3244188 (10Milimetric) p:05Triage>03High [15:15:15] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3244266 (10MusikAnimal) I'm still trying to wrap my head around why the heck it's using 1.5+ gigabytes of memory. For starters, would it be worth looking into h... [15:16:33] milimetric, halfak, I recall us talking once upon a time about auto-updating metrics that are based on dumps...has there been any movement on that? [15:17:31] :) Joal has made a bunch of progress on text-based analysis. I'd like to hear the larger answer though. I have some metrics I want to turn into regular processes soon too :D [15:19:15] 10Analytics, 06Labs, 10Pageviews-API, 10wikitech.wikimedia.org: wikitech.wikimedia.org missing from pageviews API - https://phabricator.wikimedia.org/T153821#2892095 (10bd808) >>! In T153821#2965325, @Milimetric wrote: > Are there plans on putting wikitech behind varnish? Indirectly, yes via {T161859}. At... [15:21:27] marktraceur: depending on what exactly you're talking about, yes, what's an example metric? [15:21:57] milimetric: We're using dumps to count the number of "illustrated" articles (i.e. with at least one image), and total number of illustrations, etc. [15:22:39] marktraceur: ok so the full wikitext dumps (there's other big datasets on dumps, that's why I ask) [15:22:42] 10Analytics, 06Labs, 10Pageviews-API, 10wikitech.wikimedia.org: wikitech.wikimedia.org missing from pageviews API - https://phabricator.wikimedia.org/T153821#3244283 (10bd808) p:05High>03Normal Lowering priority from high to normal. Having pageview data on wikitech would be nice, but I don't see that i... [15:23:10] marktraceur: we have it on our backlog starting next fiscal (July) [15:23:23] parsing wikitext and extracting metrics, that is [15:23:25] milimetric: Cool. [15:23:41] We'll be creating a new task for it in a bit here [15:23:43] marktraceur: best thing to do is to file a task with specifically what metrics you need, we'll prioritize it along with the others [15:23:53] :) great minds [15:28:26] hey... sorry, forgot to log in :S [15:56:27] (03CR) 10BBlack: Reset signal disposition and unblock signals for children (031 comment) [analytics/kafkatee] - 10https://gerrit.wikimedia.org/r/352591 (owner: 10Filippo Giunchedi) [16:00:39] (03CR) 10Joal: [C: 04-1] "Close to good, but missing important spark arguments:" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/201009 (https://phabricator.wikimedia.org/T94596) (owner: 10Ottomata) [16:35:59] (03CR) 10Joal: "Some things to discuss, but globally very ok!" (0310 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/346291 (https://phabricator.wikimedia.org/T161924) (owner: 10Joal) [17:37:34] * elukey off!! [17:37:36] byeee [17:44:21] (03CR) 10Mforns: [C: 032] "LGTM! I guess you already tested that, so please go ahead and merge!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/352182 (https://phabricator.wikimedia.org/T164597) (owner: 10Joal) [18:35:42] PROBLEM - Webrequests Varnishkafka log producer on cp4006 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [18:39:20] 10Analytics, 10Analytics-Cluster, 06Operations, 10Research-management: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843#3245106 (10DarTar) [18:39:49] 10Analytics, 10Analytics-Cluster, 06Operations, 10Research-management: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843#2734568 (10DarTar) We removed the #rd tag and will follow up if there's any additional approval needed. [18:41:42] RECOVERY - Webrequests Varnishkafka log producer on cp4006 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf [18:59:44] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3245190 (10Niharika) >>! In T164178#3244266, @MusikAnimal wrote: > So thinking of things that use memory that we can cut back on... what if we only kept track... [19:39:58] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3245416 (10Stevietheman) #4 sounds like it could be done in a fairly straightforward manner, using an temporary indexed database table. 1) Go through each proj... [19:52:38] (03CR) 10Mforns: [C: 031] "LGTM I added a DRY suggestion and a couple super-minor things like typos in comments or annoying uppercase observations :]" (0316 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/352181 (https://phabricator.wikimedia.org/T143928) (owner: 10Joal) [19:53:32] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3245463 (10kaldari) @Niharika: Let's try throwing a `gc_collect_cycles()` at the end of the `foreach ( $pages as $page ) {` loop and re-running WikiProject Biog... [20:56:27] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3245627 (10MusikAnimal) >>! In T164178#3245463, @kaldari wrote: > @Niharika: Let's try throwing a `gc_collect_cycles()` at the end of the `foreach ( $pages as $... [20:57:05] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3245629 (10Niharika) >>! In T164178#3245463, @kaldari wrote: > @Niharika: Let's try throwing a `gc_collect_cycles()` at the end of the `foreach ( $pages as $pag... [20:59:09] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3245637 (10Niharika) >>! In T164178#3245627, @MusikAnimal wrote: >>>! In T164178#3245463, @kaldari wrote: >> @Niharika: Let's try throwing a `gc_collect_cycles(... [21:01:27] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3245655 (10Niharika) >>! In T164178#3245637, @Niharika wrote: >>>! In T164178#3245627, @MusikAnimal wrote: >>>>! In T164178#3245463, @kaldari wrote: >>> @Nihari... [21:31:22] milimetric, I seem to remember viewing a UI that had "new editors" and "surviving new editors" and that sort of stuff. [21:31:25] Where can I find that? [21:31:32] * halfak feels shame for not remembering. [21:31:39] I'm looking for graphs over time. [21:40:37] halfak: sorry that was just a mockup, here one sec [21:40:47] halfak: https://analytics.wikimedia.org/dashboards/standard-metrics/ [21:40:56] halfak: not mockup, "beta" [21:41:02] Gotcha. Thanks :) [21:41:02] so the numbers aren't updated [21:41:08] but they're good as of last fall [21:41:20] these are some of the new metrics making their way into wikistats 2 [21:42:06] 10Analytics: Preserve userAgent field in apps schemas - https://phabricator.wikimedia.org/T164125#3245865 (10mforns) Sorry for the delay @Tbayer > That said, one thing that has changed since then is that we are now (going forward) dealing with a sanitized user agent field that has already been cleared of a lot... [21:43:15] Cool. This is perfect. Thank you :) [21:43:40] 06Analytics-Kanban, 15User-Elukey: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#2990326 (10mforns) @elukey I commented on @Tbayer 's task T164125 There might be some changes that can affect this task. Can you read them and give your opinion? THX! [21:44:02] bye team! see ya tomorrow [21:48:35] PROBLEM - Webrequests Varnishkafka log producer on cp3035 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [21:49:34] RECOVERY - Webrequests Varnishkafka log producer on cp3035 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf [23:06:03] 10Analytics, 03Community-Tech-Sprint: Investigation: How can we improve the speed of the popular pages bot - https://phabricator.wikimedia.org/T164178#3246056 (10Stevietheman) >>! In T164178#3245629, @Niharika wrote: > @Stevietheman I can't find a reference for it now, but the pageviews API does do caching on...