[02:44:53] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3358797 (10Aklapper) [08:34:27] 10Analytics: Refactor puppet code for the Hadoop Analytics cluster to roles/profiles - https://phabricator.wikimedia.org/T167790#3359234 (10elukey) [08:35:18] 10Analytics: Send burrow lag statistics to statsd/graphite {hawk} - https://phabricator.wikimedia.org/T120852#3359238 (10elukey) This has been stalled for a looong time, do we really want to do it? [08:59:27] 10Analytics, 10Analytics-Cluster, 10User-Elukey: Understand Kafka ACLs and figure out what ACLs we want for production topics - https://phabricator.wikimedia.org/T167304#3359293 (10elukey) [09:23:34] 10Analytics-Kanban, 10DBA, 10Operations, 10ops-eqiad, 10User-Elukey: db1046 BBU looks faulty - https://phabricator.wikimedia.org/T166141#3359355 (10Marostegui) @elukey looks like the BBU is now almost completely dead. After Jaime's relearn attempt, almost 3 hours ago the battery status hasn't changed: ``... [09:39:58] 10Analytics, 10Analytics-Cluster, 10Operations, 10ops-eqiad: rack/setup/install new kafka nodes - https://phabricator.wikimedia.org/T167992#3359422 (10elukey) My preference would be either `jumbo` or `aggregate` (the latter sounds better probably) [09:58:26] hello a-team, I'm working from a train back from Santiago today so my connection may be a bit flaky [09:58:52] o/ [11:04:08] * elukey lunch! [11:07:44] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#2990326 (10Volans) What @jcrespo said, see also my comment on https://gerrit.wikimedia.org/r/#/c/356383/12/modules/role/files/mariadb/eventlogging_clean... [11:08:08] Hi a-team - Reminder I'm not here today [11:21:39] I'm off for holiday as well [11:25:03] Reedy: I'm off today, but if valhalla comes around again, I can help you both work with EL data. You can log into stat1003 and access it from there, all our docs are on wikitech: https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging#Accessing_data [11:25:44] Usually if you google "wikitech <>" you'll get what you need [12:10:34] !log disable BBU auto learn on all the hadoop workers [12:10:35] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:17:20] 10Analytics-Kanban, 10Operations, 10User-Elukey: New analytic hosts with BBU learning cycle enabled - https://phabricator.wikimedia.org/T167809#3360322 (10elukey) Updated https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Administration with instructions on how to set the BBU auto-learn fun... [12:17:37] 10Analytics-Kanban, 10Operations, 10User-Elukey: New analytic hosts with BBU learning cycle enabled - https://phabricator.wikimedia.org/T167809#3360324 (10elukey) [12:41:52] 10Analytics-Kanban, 10Operations, 10User-Elukey: New analytic hosts with BBU learning cycle enabled - https://phabricator.wikimedia.org/T167809#3360362 (10elukey) ``` elukey@neodymium:~$ sudo cumin 'R:class = role::analytics_cluster::hadoop::worker' 'megacli -AdpBbuCmd -GetBbuProperties -aALL -nolog | grep "... [13:19:46] hey team :] [13:19:51] o/ [13:28:26] hey elukey :] [13:28:49] elukey, have you seen the performance notes? do you think it makes sense? [13:33:45] elukey, I worked a bit on this patch: https://gerrit.wikimedia.org/r/#/c/359938 [13:34:32] it's a minimal change (I think) that avoids using offsets [13:35:40] we already talked about this before, I think it was one of our options before deciding to go with limit/offset [13:36:32] the only drawback of this solution is that some events <<1% might be re-processed 2 times [13:36:44] but the outcome is the same [13:41:27] mforns: I have, really great work! [13:44:06] mforns: ah nice! This is what I wanted to try out :) [13:44:39] cool, I'm glad then, I wasn't sure if I should have done it [13:45:12] haven't tested it with a db yet though [13:46:29] nono it makes sense! [13:47:19] let me review the changes and understand them fully [13:48:24] k [13:54:26] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3360580 (10Qgil) p:05Triage>03Normal Assuming priority Normal. [14:15:44] (Marcel still working on other stuff, sorry, will give you feedback asap) [14:17:19] elukey, np no rush [14:34:55] 10Analytics: dbstore1002 /srv filling up - https://phabricator.wikimedia.org/T168303#3360724 (10Marostegui) [14:38:32] 10Analytics, 10User-Elukey: dbstore1002 /srv filling up - https://phabricator.wikimedia.org/T168303#3360739 (10elukey) p:05Triage>03Normal [14:47:53] (03CR) 10Mforns: [C: 031] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/353287 (https://phabricator.wikimedia.org/T164021) (owner: 10Nuria) [14:55:27] (03CR) 10Mforns: Use native timestamps in mediawiki history (034 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/358916 (https://phabricator.wikimedia.org/T161150) (owner: 10Joal) [15:00:41] ping elukey [15:00:54] cominggg [15:34:40] Reedy: all our stuff is documented in wikitech see: https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging#Accessing_data [15:35:50] Reedy: CommandInvocation schema is not receiving events though, teh client needs to change its user agent for those to be received [15:53:51] elukey, pushed last changes, with comments addressed, on top of your patch :] [15:54:13] mforns: \o/ [16:09:38] mforns: SELECT DISTINCT TABLE_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA='log' AND COLUMN_NAME LIKE 'event%' [16:09:43] wdyt as filter? [16:10:01] elukey, in a meeting, will look into it in a bit [16:10:06] suuure [16:12:50] heyaaaa, pip / setup.py q for the educated if anybody has an idea: [16:13:09] i want to install a script with the same name as my package [16:13:21] but pip gets a little wonky when i try to do that [16:13:38] it tries to make the script wrapper it installs execute a directory path [16:13:40] the path to my package [16:13:56] since the script has the same name as the package, i guess the package is being installed in the place where the script should [16:20:00] ottomata: o/ - IIRC it was not suggested to name the python package as one of the scripts to avoid pip getting confused [16:25:16] ottomata: would something like this work: https://github.com/schana/design-maker/blob/develop/setup.py#L10 [16:26:05] schana: i tried something like that, but then i got relative import errors [16:26:25] then don't use relative imports :D (a personal style opinion of mine) [16:26:53] fine, what's best practice for importing within the same package? [16:26:55] i got [16:27:18] ottomata: https://www.python.org/dev/peps/pep-0008/#imports [16:27:30] "Absolute imports are recommended, as they are usually more readable and tend to be better behaved" [16:27:48] sure, but, within my own files i have [16:27:48] certpy/util.py [16:27:48] certpy/main.py [16:27:56] (assume main is where i'll put the entrypoint script) [16:28:05] main.py needs to do somethign like [16:28:10] from .util import setup_logging [16:28:49] just do [16:28:49] from certpy.util import setup_logging [16:28:49] although I would go so far as to say don't import functions [16:28:55] only import modules [16:29:04] ya maybe [16:29:04] but [16:29:06] IMO it makes it cleaner to tell where the code is coming from [16:29:14] ok, how to I run my main function if i haven't pip installed? [16:29:22] if i do from certpy.util import setup_logging [16:29:25] i can't do [16:29:32] python ./certpy/main.py [16:29:39] hmm, or can I? [16:31:08] in a virtualenv you can always "pip install -e ." [16:31:17] and local changes will be reflected [16:31:45] yeahhh, but shouldn't have to use a virtualenv if i don't want to [16:31:59] well, you could do it system-wide [16:32:13] same effect, just with more mess [16:32:37] yeah but that's mess [16:32:39] especially in prod [16:32:43] its annoying, because we can't pip install [16:32:50] we often use deb packages for dependencies [16:32:58] and then just set PYTHONPATH for the lib we are working with [16:33:00] e.g. eventlogging [16:33:15] eventlogging repo is deployed via scap to /srv/deployment/eventlogging/analytics [16:33:23] all python dependencies are installed as .deb packages [16:33:31] building python deb packages isn't too bad: https://github.com/wikimedia-research/translation-recs-app/compare/deb-package [16:33:37] doing export PYTHONPATH=/srv/deployment/eventlogging/analytics [16:33:42] let's me do [16:33:52] /srv/deployment/eventlogging/analytics/bin/eventlogging-consumer -h [16:33:54] and just use it from there [16:34:34] oo cool schana, i think your previous suggestion with entry points actually works [16:34:45] my certpy/main.py has [16:34:46] from certpy import instantiate_manifest, setup_logging [16:34:48] and also [16:34:52] def main(): [16:34:52] and [16:34:55] if __name__ == '__main__': [16:34:55] main() [16:35:00] seems to work as [16:35:08] ./certpy/main.py [16:35:12] and also after pip install [16:35:18] certpy [16:35:32] entry_points={'console_scripts': ['certpy = certpy.main:main']}, [16:36:12] cool [16:36:50] thanks! [16:37:11] no problem [16:50:07] mforns: your change works fine on my test env, but it does UPDATE already sanitized rows each time [16:50:29] the "last" timestamp of the batch to be precise [16:50:44] since it becomes that starting point of the subsequent batch [16:50:47] elukey, you mean only sanitized rows? [16:50:49] and we use >= in there [16:51:06] elukey, yes, that was the downside of this solution [16:51:19] but it should be like <<1% of the events no? [16:51:41] I think it depends on how big the batch is no? [16:51:51] the batch is always 1000 events no? [16:52:01] yes but it is a parameter [16:52:05] it is not set in stone [16:52:07] :) [16:52:10] ah of course [16:52:55] mmmmm maybe we could sort by uuid? [16:53:23] and batch by uuid [16:53:28] it might be super slow but we can try [16:53:51] going afk now but I am going to think about the solution :) [16:54:51] * elukey afk! [17:24:08] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Write generic certificate management software for use with Puppet and Self Signing CAs. - https://phabricator.wikimedia.org/T166167#3361187 (10Ottomata) [17:26:09] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Write generic certificate management software for use with Puppet and Self Signing CAs. - https://phabricator.wikimedia.org/T166167#3287261 (10Ottomata) I've moved my work from my github account to gerrit under a new project name 'certpy'. It is s... [20:39:26] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3361580 (10Aklapper) [20:41:48] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3361582 (10Aklapper) * The [[ https://github.com/Bitergia/mediawiki-identities... [20:42:26] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3361584 (10Aklapper)