[05:38:28] 10Countervandalism-Network: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415 (10Krinkle) 03NEW [05:43:38] 10Countervandalism-Network: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415#10690270 (10Krinkle) The CVN API is hosted as part of the `cvn-apache` service: https://meta.wikimedia.org/wiki/Countervandalism_Network/Infrastructure#cvn-apache The current server for tha... [05:48:30] 10Countervandalism-Network: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415#10690271 (10Krinkle) > * The cronjob on the CVNBot server that copies the latest database to NFS (shared disk volume). ` krinkle@cvn-app11:/data/project/cvn-common/dropbox$ ls -halF … -rw-r... [05:50:30] 10Countervandalism-Network: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415#10690272 (10Krinkle) > * The cronjob on the Apache server that copies the latest database from there to the API. ` krinkle@cvn-apache10:/srv/cvn/git/cvn-api/data$ ls -halF … -rw------- 1 ro... [06:55:14] !log krinkle@cvn-apache10: Run `sudo chmod 644 /srv/cvn/git/infrastructure/crontab-config/*.cron`, per T390415 [06:55:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cvn/SAL [06:55:16] T390415: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415 [06:55:58] 10Countervandalism-Network: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415#10690294 (10Krinkle) 05Open→03Resolved p:05Triage→03Medium a:03Krinkle Looking at `/var/log/syslog` on cvn-apache10 reveals why: ` … Mar 30 06:40:01 cvn-apache10 cron[705]: (*... [07:05:15] 10Countervandalism-Network: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415#10690300 (10Peachey88) [07:57:23] (03PS1) 10Krinkle: docroot: Add real-time API status check [labs/countervandalism/cvn-infrastructure] - 10https://gerrit.wikimedia.org/r/1132150 (https://phabricator.wikimedia.org/T390415) [07:57:35] (03CR) 10Krinkle: [C:03+2] docroot: Add real-time API status check [labs/countervandalism/cvn-infrastructure] - 10https://gerrit.wikimedia.org/r/1132150 (https://phabricator.wikimedia.org/T390415) (owner: 10Krinkle) [07:58:45] (03CR) 10Krinkle: [V:03+2 C:03+2] docroot: Add real-time API status check [labs/countervandalism/cvn-infrastructure] - 10https://gerrit.wikimedia.org/r/1132150 (https://phabricator.wikimedia.org/T390415) (owner: 10Krinkle) [07:59:50] 10Countervandalism-Network, 13Patch-For-Review: CVN API no longer syncing, last updated 2023-02-13 - https://phabricator.wikimedia.org/T390415#10690318 (10Krinkle) https://cvn.wmcloud.org/ {F58946814 height=50}