[10:24:11] FIRING: SystemdUnitFailed: arclamp_compress_logs.service on arclamp2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [10:37:06] FIRING: [2x] SystemdUnitFailed: arclamp_compress_logs.service on arclamp2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [10:42:06] RESOLVED: [2x] SystemdUnitFailed: arclamp_compress_logs.service on arclamp2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [14:10:55] Hello! I'm seeing different behavior between local and CI testing on a CR: https://gerrit.wikimedia.org/r/c/operations/alerts/+/1098950. CI fails (like the original CR, see related task), but I can’t reproduce locally. I suspect a `promtool` version mismatch. I’ve updated `test_alerts.py` to run locally (`./test_alerts.py`) and print the [14:10:55] `promtool` version used. [15:05:18] FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:05:56] FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:06:22] FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:07:42] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:08:36] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:08:46] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:08:46] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:09:00] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:09:01] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:09:01] @jinxer-wm: DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA NOW! I NEED MY SEEDBOX! [15:09:37] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:09:37] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:09:37] @jinxer-wm: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:11] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:11] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:11] @jinxer-wm: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:20] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:20] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:20] @jinxer-wm: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:27] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:28] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:29] @jinxer-wm: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:40] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:40] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:40] @jinxer-wm: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:54] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:54] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:10:54] @jinxer-wm: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:11:05] @tantalum.libera.chat 005 EBOT2024 DEAF=D TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: EXTBAN=$,agjrxz :are supported by this server: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:11:05] @Guest23: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [15:11:05] @jinxer-wm: FUCKING DELETE THE ENTRETENIMIENTO PAGE FROM SPANISH WIKIPEDIA RIGHT NOW! I NEED MY SEEDBOX AND I CANNOT GET IT UNTIL THAT PAGE IS DELETED! [16:01:35] Hey arnaudb, I'll take a look! I tried pulling your patchset and running the tests via Docker locally, but they failed. Are you running the tests via Docker as well? [16:03:59] ah no! I'll try, I was running them directly with my own promtool, its why I suspected a version difference between the CI and mine! thanks for pointing out the docker tappof I'll go and try with it! [16:06:04] arnaudb you'll find the commands into the README file :)