[04:37:25] 10Analytics, 10Analytics-Data-Quality, 10Product-Analytics: Some registered users have null values for event_user_text and event_user_text_historical in mediawiki_history - https://phabricator.wikimedia.org/T218463 (10Neil_P._Quinn_WMF) [08:21:08] 10Quarry: Quarry is down with 502 Bad Gateway message - https://phabricator.wikimedia.org/T218468 (10Framawiki) p:05Triage→03Unbreak! [08:29:55] 10Quarry: Quarry is down with 502 Bad Gateway message - https://phabricator.wikimedia.org/T218468 (10Framawiki) 05Open→03Resolved a:03Framawiki Solved. ` framawiki@quarry-web-01:/var/log/nginx$ sudo tail access.log -n 1 172.16.0.164 - - [16/Mar/2019:08:22:28 +0000] "GET /query/20733 HTTP/1.1" 502 173 "-" "... [08:40:27] 10Quarry, 10Regression: Bad resultset number case is not handled - https://phabricator.wikimedia.org/T218470 (10Framawiki) p:05Triage→03Low [08:53:59] 10Quarry: Quarry is down with 502 Bad Gateway message - https://phabricator.wikimedia.org/T218468 (10Framawiki) I see that in logs, just after the first down alert (02:55:39): ` Mar 16 03:01:16 quarry-web-01 nslcd[632]: [13c5e3] ldap_start_tls_s() failed (uri=ldap://ldap-labs.codfw.wikim... [13:38:31] 10Quarry: Quarry is down with 502 Bad Gateway message - https://phabricator.wikimedia.org/T218468 (10zhuyifei1999) Could it be related to T217280? [20:49:16] 10Analytics, 10Product-Analytics, 10Epic, 10User-Elukey: Provide feature parity between the wiki replicas and the Analytics Data Lake - https://phabricator.wikimedia.org/T212172 (10Neil_P._Quinn_WMF) [20:49:18] 10Analytics, 10Analytics-Data-Quality, 10Product-Analytics: Some registered users have null values for event_user_text and event_user_text_historical in mediawiki_history - https://phabricator.wikimedia.org/T218463 (10Neil_P._Quinn_WMF)