[00:02:22] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [00:10:22] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [00:32:23] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [00:40:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [00:48:43] PROBLEM Free ram is now: WARNING on bots-3 i-000000e5.pmtpa.wmflabs output: Warning: 15% free memory [00:53:42] PROBLEM Free ram is now: CRITICAL on bots-3 i-000000e5.pmtpa.wmflabs output: Critical: 1% free memory [00:59:13] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [01:03:12] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [01:07:33] PROBLEM Total processes is now: WARNING on bots-salebot i-00000457.pmtpa.wmflabs output: PROCS WARNING: 174 processes [01:10:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [01:12:32] RECOVERY Total processes is now: OK on bots-salebot i-00000457.pmtpa.wmflabs output: PROCS OK: 99 processes [01:29:42] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs CRITICAL - Host Unreachable (i-000000e5.pmtpa.wmflabs) [01:33:13] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [01:40:33] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [01:59:43] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs CRITICAL - Host Unreachable (i-000000e5.pmtpa.wmflabs) [02:04:02] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [02:11:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [02:22:33] RECOVERY Total processes is now: OK on parsoid-spof i-000004d6.pmtpa.wmflabs output: PROCS OK: 148 processes [02:26:52] PROBLEM Free ram is now: WARNING on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Warning: 11% free memory [02:30:13] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs CRITICAL - Host Unreachable (i-000000e5.pmtpa.wmflabs) [02:34:03] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [02:37:52] RECOVERY Free ram is now: OK on wikistream-1 i-0000016e.pmtpa.wmflabs output: OK: 27% free memory [02:38:42] RECOVERY Free ram is now: OK on bots-sql2 i-000000af.pmtpa.wmflabs output: OK: 20% free memory [02:41:52] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [02:42:52] RECOVERY Current Load is now: OK on parsoid-roundtrip4-8core i-000004ed.pmtpa.wmflabs output: OK - load average: 3.98, 4.20, 4.85 [02:46:43] PROBLEM Free ram is now: WARNING on bots-sql2 i-000000af.pmtpa.wmflabs output: Warning: 13% free memory [03:00:52] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs CRITICAL - Host Unreachable (i-000000e5.pmtpa.wmflabs) [03:04:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [03:10:52] PROBLEM Free ram is now: WARNING on wikistream-1 i-0000016e.pmtpa.wmflabs output: Warning: 13% free memory [03:11:52] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [03:30:52] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs CRITICAL - Host Unreachable (i-000000e5.pmtpa.wmflabs) [03:34:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [03:42:43] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [04:00:52] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs CRITICAL - Host Unreachable (i-000000e5.pmtpa.wmflabs) [04:04:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [04:12:43] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [04:30:53] PROBLEM host: i-000000e5.pmtpa.wmflabs is DOWN address: i-000000e5.pmtpa.wmflabs CRITICAL - Host Unreachable (i-000000e5.pmtpa.wmflabs) [04:33:43] RECOVERY Free ram is now: OK on bots-3 i-000000e5.pmtpa.wmflabs output: OK: 105% free memory [04:33:53] RECOVERY host: i-000000e5.pmtpa.wmflabs is UP address: i-000000e5.pmtpa.wmflabs PING OK - Packet loss = 0%, RTA = 0.55 ms [04:34:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [04:35:12] PROBLEM Current Load is now: CRITICAL on bots-3 i-000000e5.pmtpa.wmflabs output: CRITICAL - load average: 7.87, 28.35, 36.29 [04:42:43] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [04:50:13] PROBLEM Current Load is now: WARNING on bots-3 i-000000e5.pmtpa.wmflabs output: WARNING - load average: 2.23, 3.56, 15.13 [05:04:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [05:13:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [05:15:12] RECOVERY Current Load is now: OK on bots-3 i-000000e5.pmtpa.wmflabs output: OK - load average: 2.05, 2.25, 4.91 [05:36:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [05:43:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [06:06:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [06:13:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [06:28:33] PROBLEM Total processes is now: CRITICAL on incubator-apache i-00000211.pmtpa.wmflabs output: PROCS CRITICAL: 204 processes [06:30:32] PROBLEM Total processes is now: WARNING on parsoid-spof i-000004d6.pmtpa.wmflabs output: PROCS WARNING: 153 processes [06:35:32] RECOVERY Total processes is now: OK on parsoid-spof i-000004d6.pmtpa.wmflabs output: PROCS OK: 148 processes [06:36:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [06:38:32] PROBLEM Total processes is now: WARNING on incubator-apache i-00000211.pmtpa.wmflabs output: PROCS WARNING: 199 processes [06:38:42] PROBLEM Free ram is now: WARNING on swift-be1 i-000001c7.pmtpa.wmflabs output: Warning: 19% free memory [06:43:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [07:06:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [07:14:02] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [07:36:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [07:44:02] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [07:48:33] PROBLEM Total processes is now: WARNING on parsoid-spof i-000004d6.pmtpa.wmflabs output: PROCS WARNING: 154 processes [07:55:52] PROBLEM Current Load is now: WARNING on parsoid-roundtrip4-8core i-000004ed.pmtpa.wmflabs output: WARNING - load average: 9.74, 7.74, 5.86 [08:06:44] PROBLEM Current Load is now: WARNING on parsoid-roundtrip7-8core i-000004f9.pmtpa.wmflabs output: WARNING - load average: 9.34, 9.10, 6.70 [08:06:44] PROBLEM Current Load is now: WARNING on parsoid-roundtrip3 i-000004d8.pmtpa.wmflabs output: WARNING - load average: 8.93, 8.21, 5.91 [08:06:44] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [08:11:52] PROBLEM Current Load is now: WARNING on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: WARNING - load average: 5.06, 5.98, 5.34 [08:14:02] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [08:21:52] RECOVERY Current Load is now: OK on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: OK - load average: 3.97, 4.61, 4.95 [08:36:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [08:44:13] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [08:48:52] PROBLEM Current Load is now: WARNING on parsoid-roundtrip6-8core i-000004f8.pmtpa.wmflabs output: WARNING - load average: 7.46, 6.30, 5.37 [08:49:52] PROBLEM Current Load is now: WARNING on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: WARNING - load average: 7.93, 7.20, 5.94 [08:58:43] RECOVERY Free ram is now: OK on swift-be1 i-000001c7.pmtpa.wmflabs output: OK: 21% free memory [09:06:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [09:14:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [09:37:22] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [09:43:52] RECOVERY Current Load is now: OK on parsoid-roundtrip6-8core i-000004f8.pmtpa.wmflabs output: OK - load average: 3.73, 4.37, 5.00 [09:44:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [10:07:22] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [10:14:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [10:26:53] PROBLEM Free ram is now: CRITICAL on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Critical: 5% free memory [10:31:53] PROBLEM Free ram is now: WARNING on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Warning: 6% free memory [10:34:53] RECOVERY Current Load is now: OK on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: OK - load average: 3.30, 3.97, 4.84 [10:37:22] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [10:44:43] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [10:45:52] Damianz: [10:51:42] RECOVERY Current Load is now: OK on parsoid-roundtrip3 i-000004d8.pmtpa.wmflabs output: OK - load average: 5.06, 4.48, 4.88 [11:07:22] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [11:14:29] !htmllogs [11:14:29] experimental: http://bots.wmflabs.org/~wm-bot/html/%23wikimedia-labs [11:14:43] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [11:15:22] d [11:21:43] RECOVERY Current Load is now: OK on parsoid-roundtrip7-8core i-000004f9.pmtpa.wmflabs output: OK - load average: 3.71, 4.41, 4.83 [11:37:23] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [11:45:22] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [11:51:52] PROBLEM Free ram is now: CRITICAL on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Critical: 5% free memory [12:06:52] PROBLEM Free ram is now: WARNING on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Warning: 6% free memory [12:08:12] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [12:15:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [12:21:52] PROBLEM Free ram is now: CRITICAL on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Critical: 5% free memory [12:38:13] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [12:46:43] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [13:09:02] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [13:16:54] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [13:39:03] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [13:47:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [14:06:53] PROBLEM Free ram is now: CRITICAL on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Critical: 5% free memory [14:09:03] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [14:18:33] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [14:21:53] PROBLEM Free ram is now: WARNING on sube i-000003d0.pmtpa.wmflabs output: Warning: 19% free memory [14:39:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [14:48:33] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [15:09:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [15:19:12] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [15:21:53] RECOVERY Free ram is now: OK on sube i-000003d0.pmtpa.wmflabs output: OK: 20% free memory [15:37:52] PROBLEM Free ram is now: WARNING on sube i-000003d0.pmtpa.wmflabs output: Warning: 19% free memory [15:39:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [15:40:02] Damianz: are you here? [15:40:52] RECOVERY Current Load is now: OK on parsoid-roundtrip4-8core i-000004ed.pmtpa.wmflabs output: OK - load average: 3.60, 4.19, 4.77 [15:49:12] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [16:09:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [16:19:13] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [16:39:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [16:49:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [16:52:52] RECOVERY Free ram is now: OK on sube i-000003d0.pmtpa.wmflabs output: OK: 20% free memory [17:11:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [17:19:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [17:41:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [17:49:32] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [17:55:52] PROBLEM Free ram is now: WARNING on sube i-000003d0.pmtpa.wmflabs output: Warning: 19% free memory [17:56:32] PROBLEM Total processes is now: WARNING on parsoid-spof i-000004d6.pmtpa.wmflabs output: PROCS WARNING: 155 processes [17:56:52] PROBLEM Free ram is now: WARNING on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Warning: 10% free memory [18:03:52] PROBLEM Current Load is now: WARNING on parsoid-roundtrip4-8core i-000004ed.pmtpa.wmflabs output: WARNING - load average: 12.56, 10.48, 7.17 [18:09:43] PROBLEM Current Load is now: WARNING on parsoid-roundtrip7-8core i-000004f9.pmtpa.wmflabs output: WARNING - load average: 8.51, 8.92, 6.73 [18:11:42] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [18:14:42] PROBLEM Current Load is now: WARNING on parsoid-roundtrip3 i-000004d8.pmtpa.wmflabs output: WARNING - load average: 8.08, 7.47, 5.83 [18:15:53] PROBLEM Current Load is now: WARNING on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: WARNING - load average: 7.70, 7.29, 5.81 [18:19:33] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [18:21:11] [star] can you make a choice? [18:21:22] it's becoming annoying [18:21:27] <[christmas_star]> sorry [18:26:52] PROBLEM Current Load is now: WARNING on parsoid-roundtrip6-8core i-000004f8.pmtpa.wmflabs output: WARNING - load average: 10.46, 8.56, 6.28 [18:41:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [18:49:33] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [19:11:43] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [19:19:33] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [19:42:22] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [19:47:01] <[christmas_star]> Damianz: hi [19:49:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [19:56:52] RECOVERY Current Load is now: OK on parsoid-roundtrip6-8core i-000004f8.pmtpa.wmflabs output: OK - load average: 2.89, 3.78, 4.95 [20:04:43] RECOVERY Current Load is now: OK on parsoid-roundtrip3 i-000004d8.pmtpa.wmflabs output: OK - load average: 2.67, 2.99, 4.48 [20:05:52] RECOVERY Current Load is now: OK on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: OK - load average: 5.16, 4.20, 4.94 [20:12:22] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [20:19:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [20:19:52] PROBLEM Current Load is now: WARNING on parsoid-roundtrip6-8core i-000004f8.pmtpa.wmflabs output: WARNING - load average: 5.00, 6.12, 5.42 [20:20:53] PROBLEM Current Load is now: WARNING on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: WARNING - load average: 6.19, 6.03, 5.50 [20:22:43] PROBLEM Current Load is now: WARNING on parsoid-roundtrip3 i-000004d8.pmtpa.wmflabs output: WARNING - load average: 6.68, 6.09, 5.35 [20:34:52] RECOVERY Current Load is now: OK on parsoid-roundtrip6-8core i-000004f8.pmtpa.wmflabs output: OK - load average: 3.40, 4.42, 4.88 [20:42:23] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [20:50:23] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [21:10:53] RECOVERY Current Load is now: OK on ve-roundtrip2 i-0000040d.pmtpa.wmflabs output: OK - load average: 4.47, 4.58, 4.89 [21:12:23] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [21:18:32] PROBLEM Total processes is now: CRITICAL on incubator-apache i-00000211.pmtpa.wmflabs output: PROCS CRITICAL: 208 processes [21:20:23] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [21:22:43] RECOVERY Current Load is now: OK on parsoid-roundtrip3 i-000004d8.pmtpa.wmflabs output: OK - load average: 4.72, 4.46, 4.84 [21:22:53] PROBLEM Current Load is now: WARNING on parsoid-roundtrip6-8core i-000004f8.pmtpa.wmflabs output: WARNING - load average: 6.51, 5.79, 5.67 [21:23:33] PROBLEM Total processes is now: WARNING on incubator-apache i-00000211.pmtpa.wmflabs output: PROCS WARNING: 199 processes [21:42:24] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [21:49:42] RECOVERY Current Load is now: OK on parsoid-roundtrip7-8core i-000004f9.pmtpa.wmflabs output: OK - load average: 4.94, 4.68, 4.94 [21:51:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [22:12:18] Hi. How I can request access on fa.wikipedia.beta.wmflabs.org? I simply want copy wikipedia CSSs (font configuration) and tools and import some wikipedia page on it. My labs console account is https://labsconsole.wikimedia.org/wiki/User:Ebrahim but my user name on that wiki is http://fa.wikipedia.beta.wmflabs.org/wiki/%DA%A9%D8%A7%D8%B1%D8%A8%D8%B1:Ebraminio [22:13:12] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [22:15:52] Hello ebraminio. [22:16:22] Dereckson, hi [22:16:54] Basically, what you need is an administrator access on this wiki if I understand? [22:18:42] Dereckson, I think so, but for importing I think I need a temporary import permission I think. I want import persian featured articles on that wiki [22:21:04] Problem is these wikis are mainly to test deployment stuff. So there aren't really any userrights configured. [22:21:52] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [22:25:58] Dereckson, oh okay. Can you (or someone else) export some pages from a category in wikipedia (with their templates) and import to them and copy Persian Wikipedia Mediawiki:Common.css on that wikipedia? [22:31:36] Dereckson, This is the dump that I want to export and import on that wiki: http://i.imgur.com/OFvaf.png [22:34:27] Could you also indicate What for? [22:35:43] Dereckson, if we had these pages and this testwiki (a mini model of original wiki), our community could make better decision before enabling https://bugzilla.wikimedia.org/show_bug.cgi?id=42280 [22:38:52] RECOVERY Current Load is now: OK on parsoid-roundtrip4-8core i-000004ed.pmtpa.wmflabs output: OK - load average: 3.28, 3.55, 4.55 [22:39:03] Hmmm I think the *.wikipedia.beta.wmflabs.org sites pursue the goal to deploy a new configuration and see it doesn't break website. It's not a testing place. You would need to create a new testing place. [22:42:05] Dereckson, hmm. But labconsoles single node instance making tutorial wasreally confusing for me. Could you make a wiki for us? (and enable webfonts on it?) [22:43:12] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [22:46:27] ebraminio: ok [22:47:03] Dereckson, I believe community will accept other [free] fonts than original deployment had. In fact I am sad about disabling webfonts on Persian wikipedia. It could be remained disabled by default but not removed totally [22:47:03] I'm currently a little busy, I will handle that tomorrow evening or Monday morning (UTC); is that fine for you? [22:48:25] As discussed in the bug, to let a broken feature on the wiki is a little annoying: people has generally the expectation a feature availalbe works. [22:48:54] Dereckson, okay, please tell Mardetanha when you enabled it, he will tell me when I am not online on IRC [22:49:12] Ok. [22:49:32] Dereckson, hmm, okay, it is understandable. [22:49:37] Dereckson, thanks anyway [22:51:52] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [23:13:13] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [23:21:52] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [23:32:52] PROBLEM Current Load is now: WARNING on parsoid-roundtrip4-8core i-000004ed.pmtpa.wmflabs output: WARNING - load average: 5.02, 5.18, 5.11 [23:37:52] RECOVERY Current Load is now: OK on parsoid-roundtrip4-8core i-000004ed.pmtpa.wmflabs output: OK - load average: 4.71, 4.88, 5.00 [23:43:13] PROBLEM host: i-0000051a.pmtpa.wmflabs is DOWN address: i-0000051a.pmtpa.wmflabs PING CRITICAL - Packet loss = 100% [23:51:52] PROBLEM Free ram is now: CRITICAL on dumps-bot1 i-000003ed.pmtpa.wmflabs output: Critical: 5% free memory [23:52:42] PROBLEM host: i-000004de.pmtpa.wmflabs is DOWN address: i-000004de.pmtpa.wmflabs PING CRITICAL - Packet loss = 100%