[00:16:20] 06Revision-Scoring-As-A-Service, 10ORES, 07Pywikibot-Wikidata, 15User-Ladsgroup, 03Wikidata-Sprint-2016-11-15: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2902891 (10Ladsgroup) [00:20:44] 06Revision-Scoring-As-A-Service, 10ORES, 07Pywikibot-Wikidata, 15User-Ladsgroup, 03Wikidata-Sprint-2016-11-15: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2902908 (10Ladsgroup) p:05Triage>03Unbreak! [00:53:44] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2902936 (10Ladsgroup) ^ will fix this. Tested in ores-experiment. http://ores-experiment.wmflabs.org/v2/scores/wikidatawiki/?models=damagi... [01:31:24] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2902941 (10Ladsgroup) Okay, It seems there are some undeployed changes to wheels repo, I need to deploy all (like revscoring, etc.) or non... [02:13:25] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2902957 (10Ladsgroup) Now, https://phabricator.wikimedia.org/diffusion/1915/repository/master/ is stuck :/ [02:51:22] 10Revision-Scoring-As-A-Service-Backlog, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#1750473 (10Shizhao) this is done? see https://meta.wikimedia.org/w/index.php?title=Talk:Wiki_labels&oldid=16156223#wikilabels_in_zhwiki [02:59:07] 06Revision-Scoring-As-A-Service, 10Wikilabels, 15User-Ladsgroup: Minification and gzip compression for wikilabels assets - https://phabricator.wikimedia.org/T154122#2902978 (10Ladsgroup) minify doesn't work :/ http://labels.wmflabs.org/gadget/WikiLabels.js?minify=1 [02:59:54] PROBLEM - ORES worker labs on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:00:44] RECOVERY - ORES worker labs on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 457 bytes in 0.581 second response time [03:01:47] 06Revision-Scoring-As-A-Service, 10Wikilabels, 15User-Ladsgroup: Minification and gzip compression for wikilabels assets - https://phabricator.wikimedia.org/T154122#2902979 (10Ladsgroup) Strangely it works in my localhost. :))) [03:06:38] 06Revision-Scoring-As-A-Service, 10Wikilabels, 15User-Ladsgroup: Minification and gzip compression for wikilabels assets - https://phabricator.wikimedia.org/T154122#2902980 (10Ladsgroup) From logs of wikilabels-experiment: ``` Dec 27 03:03:05 wikilabels-experiment uwsgi-wikilabels-web[18598]: 2016-12-27 03:0... [03:08:22] 06Revision-Scoring-As-A-Service, 10Wikilabels, 15User-Ladsgroup: Minification and gzip compression for wikilabels assets - https://phabricator.wikimedia.org/T154122#2902981 (10Ladsgroup) Fixed by installing nodejs \o/ [03:43:35] 06Revision-Scoring-As-A-Service, 10Wikilabels, 13Patch-For-Review, 15User-Ladsgroup: Minification and gzip compression for wikilabels assets - https://phabricator.wikimedia.org/T154122#2903016 (10Ladsgroup) I started https://github.com/wiki-ai/wikilabels/pull/143 [05:03:44] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2903103 (10Ladsgroup) Deployed in beta, [[https://ores-beta.wmflabs.org/v2/scores/wikidatawiki/?models=damaging&revids=421063984 | got it... [05:26:53] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2903117 (10Ladsgroup) And now everything is fine, I'll monitor `#wikimedia-operations` channel in IRC for any issues and keep an eye on [[... [05:27:03] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2903118 (10Ladsgroup) 05Open>03Resolved [05:54:34] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2903134 (10Ladsgroup) [06:15:18] 06Revision-Scoring-As-A-Service, 10ORES, 15User-Ladsgroup: Clean up failure ratio monitoring and set up an alarm when it goes more than a certain threshold - https://phabricator.wikimedia.org/T154175#2903147 (10Ladsgroup) [06:16:02] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review, 07Pywikibot-Wikidata, and 2 others: Quantity changes broke ORES - https://phabricator.wikimedia.org/T154168#2903162 (10Ladsgroup) Incident report: https://wikitech.wikimedia.org/wiki/Incident_documentation/20161227-ores [06:21:10] 10Revision-Scoring-As-A-Service-Backlog, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#2903166 (10Shizhao) [06:21:24] 10Revision-Scoring-As-A-Service-Backlog, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#1750473 (10Shizhao) [06:36:51] 06Revision-Scoring-As-A-Service, 10Wikilabels, 10rsaas-editquality, 07Chinese-Sites: Create zhwiki edit quality campaign - https://phabricator.wikimedia.org/T152561#2903178 (10Shizhao) [07:40:43] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#2903256 (10Shizhao) [07:42:06] 06Revision-Scoring-As-A-Service, 10Wikilabels, 10rsaas-editquality, 07Chinese-Sites: Create zhwiki edit quality campaign - https://phabricator.wikimedia.org/T152561#2903257 (10Shizhao) [07:42:08] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#1750473 (10Shizhao) [07:44:32] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#2903263 (10Shizhao) [07:44:35] 06Revision-Scoring-As-A-Service, 10Wikilabels, 10rsaas-editquality, 07Chinese-Sites: Create zhwiki edit quality campaign - https://phabricator.wikimedia.org/T152561#2903261 (10Shizhao) 05Open>03Resolved [07:46:42] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#2903264 (10Shizhao) [07:47:41] 06Revision-Scoring-As-A-Service, 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 07Chinese-Sites: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#1750473 (10Shizhao) [09:18:14] 06Revision-Scoring-As-A-Service, 10Monitoring, 10ORES, 15User-Ladsgroup: Clean up failure ratio monitoring and set up an alarm when it goes more than a certain threshold - https://phabricator.wikimedia.org/T154175#2903312 (10Peachey88)