[06:19:50] 10Analytics, 10Contributors-Analysis: Bring the Editor Engagement Dashboard back - https://phabricator.wikimedia.org/T166877#3587323 (10Nemo_bis) [08:28:15] elukey: https://gerrit.wikimedia.org/r/#/c/375811/ thoughts? [08:32:10] addshore: on paper it is perfectly fine, although I am thinking if the $user variable name would become a bit overloaded/confusing.. Is there a more profound reason/benefit for this refactoring or just simplification? [08:35:39] well, either it should go this was, or the sub classes I guess should also get $group passed into them [08:36:39] because in graphite.pp we have this https://github.com/wikimedia/puppet/blob/production/modules/statistics/manifests/wmde/graphite.pp#L58 [08:39:02] sure [08:41:26] addshore: merged [08:41:32] thanks! [08:41:34] !log Rerun Workflow banner_activity-druid-daily-wf-2017-9-6 [08:41:38] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:14:43] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#3587560 (10mobrovac) p:05Normal>03High [09:19:58] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#3587592 (10mobrovac) [09:53:16] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 2 others: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure. - https://phabricator.wikimedia.org/T175212#3587681 (10mobrovac) [10:12:36] 10Analytics-Kanban: Chose how to deal with "Infinity" value for Banners - https://phabricator.wikimedia.org/T175248#3587723 (10JAllemandou) [10:13:01] 10Analytics-Kanban: Chose how to deal with "Infinity" value for Banners - https://phabricator.wikimedia.org/T175248#3587739 (10JAllemandou) a:03JAllemandou [10:24:07] * elukey lunch! [10:55:46] (03PS7) 10Joal: Add oozie mediawiki-history-reduced job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/376235 (https://phabricator.wikimedia.org/T174915) [10:58:26] (03PS8) 10Joal: Add oozie mediawiki-history-reduced job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/376235 (https://phabricator.wikimedia.org/T174915) [11:00:35] (03PS9) 10Joal: Add oozie mediawiki-history-reduced job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/376235 (https://phabricator.wikimedia.org/T174915) [11:02:53] (03PS10) 10Joal: Add oozie mediawiki-history-reduced job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/376235 (https://phabricator.wikimedia.org/T174915) [12:20:19] so I finally got why the eventlogging_cleaner scripts were degrading in performance (on both slaves) [12:20:31] https://grafana.wikimedia.org/dashboard/file/server-board.json?var-server=db1047&refresh=1m&orgId=1&from=now-7d&to=now&panelId=19&fullscreen [12:20:39] they were saturating the disks [12:21:16] batch sizes too big and possibily a short sleep time [12:21:34] now I found out that 10s of sleep between batches + 1000 rows maximum at the time seems to work [12:21:41] but it is really really low [12:33:39] 10k batches every 4seconds seems to be acceptable [12:33:53] but it will be ~10M records per hours [12:33:55] *hour [12:34:00] that is not a big number [12:51:19] Taking abreak a-team [12:56:01] the biggest table is 1151423188 rows [12:56:16] that one alone would take 4 days to complete fully [12:56:21] not super bad [12:56:29] no sorry, 6 days [13:01:35] next try, 100k with 10s of sleep, that would be ~1d for 1b rows [13:02:30] muuuuch better! [13:27:45] 10Analytics, 10Analytics-Wikistats: Productionise list view - https://phabricator.wikimedia.org/T175265#3588157 (10fdans) [13:29:34] 10Analytics, 10Analytics-Wikistats: Add top articles by pageviews metric - https://phabricator.wikimedia.org/T175266#3588171 (10fdans) [13:30:13] nope, trashing again for db1047 sigh [13:32:12] 10Analytics, 10Analytics-Wikistats: Stub new mediawiki history-based metrics - https://phabricator.wikimedia.org/T175268#3588195 (10fdans) [13:32:33] 10Analytics, 10Analytics-Wikistats: Stub new mediawiki history-based metrics - https://phabricator.wikimedia.org/T175268#3588208 (10fdans) a:03fdans [13:38:29] so with 10k every ~6s it should be 100k/m --> 1M every 10min --> 60M every hour --> 600M every 10h --> 1.4B every day [13:38:42] so if these calculations are ok we should be fine with the current speed [13:38:48] that is not trashing [13:39:00] as soon as I use the 100k batch card disk saturation goes to 100% [13:40:45] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review, 10Readers-Web-Backlog (Tracking): Schema:Popups suddenly stopped logging events in MariaDB, but they are still being sent according to Grafana - https://phabricator.wikimedia.org/T174815#3588308 (10elukey) So I reverted the batch size to 100... [14:03:36] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Set up LVS and VirtualHost for RunSingleJob.php - https://phabricator.wikimedia.org/T174599#3588369 (10Joe) Everything is set up and you can reach the correct LVS endpoint via the discovery DNS system at `jobrunner.discovery.wmnet`,... [14:03:43] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: [EPIC] Develop a JobQueue backend based on EventBus - https://phabricator.wikimedia.org/T157088#3588371 (10Joe) [14:03:46] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Set up LVS and VirtualHost for RunSingleJob.php - https://phabricator.wikimedia.org/T174599#3588370 (10Joe) 05Open>03Resolved [14:06:02] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#3588374 (10elukey) The run of the script was too aggressive, the dbs started to trash due to disk saturation: {F9375108} I started again the script on... [14:24:09] * elukey coffee! [14:36:07] 10Analytics-Kanban, 10Operations, 10hardware-requests: Decommission stat1002.eqiad.wmnet - https://phabricator.wikimedia.org/T173097#3588483 (10Dzahn) p:05Triage>03Normal [14:36:19] 10Analytics-Kanban, 10Operations, 10hardware-requests: Decommission stat1002.eqiad.wmnet - https://phabricator.wikimedia.org/T173097#3518505 (10Dzahn) 05Open>03stalled [14:36:20] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Replacement of stat1002 and stat1003 - https://phabricator.wikimedia.org/T152712#3588485 (10Dzahn) [14:36:38] 10Analytics-Kanban, 10Operations, 10hardware-requests: Decommission stat1002.eqiad.wmnet - https://phabricator.wikimedia.org/T173097#3518505 (10Dzahn) [14:36:43] 10Analytics, 10Operations, 10ops-eqiad: Remove stat1002 - https://phabricator.wikimedia.org/T173094#3588487 (10Dzahn) [14:37:27] 10Analytics-Kanban, 10Operations, 10ops-eqiad: Decommission stat1003.eqiad.wmnet - https://phabricator.wikimedia.org/T175150#3588490 (10Dzahn) p:05Triage>03Normal [14:44:33] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: Separate off ChangePropagation for JobQueue as a new deployment - https://phabricator.wikimedia.org/T175281#3588533 (10mobrovac) [14:44:55] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: Separate off ChangePropagation for JobQueue as a new deployment - https://phabricator.wikimedia.org/T175281#3588552 (10mobrovac) [14:45:03] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: [EPIC] Develop a JobQueue backend based on EventBus - https://phabricator.wikimedia.org/T157088#3588551 (10mobrovac) [14:45:05] 10Analytics, 10Operations, 10Traffic: Invalid "wikimedia" family in unique devices data due to misplaced WMF-Last-Access-Global cookie - https://phabricator.wikimedia.org/T174640#3568769 (10BBlack) The model's a bit different in the wikimedia.org case, I'm not even sure there's a rational answer here. Can w... [14:49:50] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Separate off ChangePropagation for JobQueue as a new deployment - https://phabricator.wikimedia.org/T175281#3588564 (10mobrovac) [14:59:03] 10Analytics, 10Operations, 10Traffic: Invalid "wikimedia" family in unique devices data due to misplaced WMF-Last-Access-Global cookie - https://phabricator.wikimedia.org/T174640#3568769 (10Nuria) I think here we should not think of global unique devices for wikimedia.org domains and rather use just per-doma... [14:59:28] 10Analytics, 10Operations, 10Traffic: Invalid "wikimedia" family in unique devices data due to misplaced WMF-Last-Access-Global cookie - https://phabricator.wikimedia.org/T174640#3568769 (10JAllemandou) Thanks @BBlack for the detailed explanations :) As for using the full `host` header value for wikimedia.or... [14:59:53] hehe nuria_ - Comment colide :) [15:00:39] a-team: standdduppp [15:00:54] ping ottomata [15:16:32] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 5 others: Separate off ChangePropagation for JobQueue as a new deployment - https://phabricator.wikimedia.org/T175281#3588604 (10mobrovac) [15:30:23] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 3 others: Visualize page create events for all wikis - https://phabricator.wikimedia.org/T170850#3588687 (10Nuria) Super thanks @Nettrom , ping us on irc if you need help with config , there are several examples , the config yo... [16:09:20] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588797 (10zhuyifei1999) [16:16:23] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588868 (10zhuyifei1999) (Originally reported by @IKhitron on https://www.mediawiki.org/wiki/Topic:Txnc2f9ndpwke6w4) [16:24:41] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588895 (10zhuyifei1999) https://quarry.wmflabs.org/query/21425, 2048 characters plain text can be downloaded. [16:29:29] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588907 (10zhuyifei1999) https://quarry.wmflabs.org/query/21426, 151 characters url can be but 279 cannot. [16:40:11] * elukey off! [16:43:33] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588921 (10zhuyifei1999) The same happens with xlsxwriter 0.9.9 instead of 0.5.2. [16:44:36] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588922 (10zhuyifei1999) ``` Sep 7 16:42:41 jessie python[26945]: /srv/venv/local/lib/python2.7/site-packages/xlsxwriter/worksheet.py:831: UserWarning: Ignoring URL 'http://www.example.com/aaaaaaaaaaaaaaaaaaa... [16:45:55] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588924 (10IKhitron) Wait a moment, it's excel's bug??? [16:51:28] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588928 (10zhuyifei1999) This is raised in https://github.com/jmcnamara/XlsxWriter/blob/b4c4b499ffb3db8e0fa1b306880bcbcb3675fd4d/xlsxwriter/worksheet.py#L828 What do you think of forcing string instead of url... [16:52:14] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588931 (10zhuyifei1999) >>! In T175285#3588924, @IKhitron wrote: > Wait a moment, it's excel's bug??? Yes, you can't possible add urls longer than 255 chars apparently. [16:52:20] 10Quarry: Quarry XLSX cells for long urls are wrongly empty - https://phabricator.wikimedia.org/T175285#3588932 (10IKhitron) Excellent for me. [17:10:25] (03CR) 10Ottomata: [V: 032 C: 032] Add webrequest sample bundle for creating a sampled webrequest table (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/376334 (owner: 10Ottomata) [17:31:06] (03PS1) 10Ottomata: Use default oozie concurrencty for sample/coordinator [analytics/refinery] - 10https://gerrit.wikimedia.org/r/376552 [17:31:17] (03CR) 10Ottomata: [V: 032 C: 032] Use default oozie concurrencty for sample/coordinator [analytics/refinery] - 10https://gerrit.wikimedia.org/r/376552 (owner: 10Ottomata) [17:38:03] hi channel: anyone here using pyspark on the stats machines? [18:00:47] (03PS1) 10Zhuyifei1999: output: Fallback to write_string when some error occur in write for xlsx [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/376561 (https://phabricator.wikimedia.org/T175285) [18:14:14] Hi dsaez, I usually use scala, but hit, I might have clues [18:17:47] (03PS2) 10Zhuyifei1999: output: Fallback to write_string when some error occur in write for xlsx [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/376561 (https://phabricator.wikimedia.org/T175285) [18:20:57] joal , I need to leave now, I'll ask you tomorrow thanks [18:21:06] no prob dsaez, sorry for the lag [18:26:00] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review, 10Readers-Web-Backlog (Tracking): Schema:Popups suddenly stopped logging events in MariaDB, but they are still being sent according to Grafana - https://phabricator.wikimedia.org/T174815#3573902 (10Nuria) @Tbayer We talked about this and the... [18:42:41] 10Analytics: Provide top domain and data to truly test superset - https://phabricator.wikimedia.org/T166689#3589421 (10Nuria) ping @Ottomata and @JAllemandou please update ticket if superset is running on screen accessible by tunnel [18:45:23] 10Analytics, 10Analytics-Wikistats: Punch hole so AQS can access druid hosts - https://phabricator.wikimedia.org/T175299#3589434 (10Nuria) [19:09:46] Hey ottomata, would have minute for a quick braindump? [19:10:50] joal: ya gimme 1 min [19:12:14] k batcave! [19:12:26] yup ottomata [19:26:40] 10Analytics: Provide top domain and data to truly test superset - https://phabricator.wikimedia.org/T166689#3589592 (10Ottomata) Now running in a screen on stat1004 under my user: ssh -N stat1004.eqiad.wmnet -L 8088:stat1004.eqiad.wmnet:8088 http://localhost:8080 Login is admin / admin [20:12:00] (03PS2) 10Joal: [WIP] Add mediawiki history metrics endpoints [analytics/aqs] - 10https://gerrit.wikimedia.org/r/373961 (https://phabricator.wikimedia.org/T174174) [20:12:07] Yay !!! Testing is on its way :) [20:12:23] Leaving for tonight, see you tomorrow a-team :) [20:20:15] latrrrsss [20:43:26] Does anybody happen to know the right database / table to look in for the article -> wikidata entity mapping? [20:53:35] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#3589791 (10Pchelolo) [21:38:13] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#3589884 (10Pchelolo) [21:48:21] ottomata: cergen? [21:48:46] ottomata: man, the wmf is awesome at so many things, but naming things is definitely not one! :) [22:04:41] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#3589982 (10Pchelolo) [22:05:42] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review, 10Readers-Web-Backlog (Tracking): Schema:Popups suddenly stopped logging events in MariaDB, but they are still being sent according to Grafana - https://phabricator.wikimedia.org/T174815#3589983 (10Nuria) @Tbayer : also please take a look at... [22:29:06] (03CR) 10Nuria: [V: 031 C: 031] "I think looks good ping @elukey to confirm that the python logging looks good, if so after we verify is working well we should abstract it" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/355601 (https://phabricator.wikimedia.org/T162034) (owner: 10Mforns)