[00:02:50] <jonas_agx_>	 I'll have to leave now. Can we continue it by email?
[00:07:23] <halfak>	 jonas_agx_, yeah.  Ping me when you want to hack some more. 
[00:07:32] <halfak>	 I'll try to figure out where para was missing from. 
[00:07:36] <halfak>	 Thanks for working today :) 
[00:07:54] <jonas_agx_>	 Can I commit the changes on requirements
[00:07:55] <jonas_agx_>	 ?
[00:08:06] <jonas_agx_>	 requirements.txt
[00:08:14] <halfak>	 Oh yeah!  If you want to submit a PR, that would be great
[00:09:12] <halfak>	 I just sent you an invite to the wiki-ai group. 
[00:09:27] <halfak>	 If you move your change to a branch and push that up, we can turn that into a pull request (PR)
[00:09:37] <halfak>	 You'll need to accept the invite first though
[00:09:42] <halfak>	 We can do this later if you need to run. 
[00:09:56] <jonas_agx_>	 Okay
[00:10:35] <jonas_agx_>	 Can you invite me, please
[00:10:44] <halfak>	 You should have an invite now
[00:10:57] <jonas_agx_>	 Okay tks
[00:11:04] <halfak>	 Try to go to https://github.com/wiki-ai
[00:13:12] <halfak>	 Weird that I don't seen to crash without para. 
[00:13:16] <halfak>	 I'll keep digging into this
[00:17:36] <wikibugs>	 10Revision-Scoring-As-A-Service-Backlog, 10Research Ideas: Conversation summarizer - https://phabricator.wikimedia.org/T155587#2947788 (10Halfak)
[00:18:29] <wikibugs>	 10Revision-Scoring-As-A-Service-Backlog, 10Research Ideas: General image classifier for commons - https://phabricator.wikimedia.org/T155538#2947802 (10Halfak)
[00:20:44] <wikibugs>	 10Revision-Scoring-As-A-Service-Backlog, 10Research Ideas: General image classifier for commons - https://phabricator.wikimedia.org/T155538#2947804 (10Halfak)
[00:22:30] <wikibugs>	 10Revision-Scoring-As-A-Service-Backlog, 10Research Ideas: Taidi: Translation AI for documentation and information - https://phabricator.wikimedia.org/T155588#2947807 (10Halfak)
[00:23:49] <wikibugs>	 10Revision-Scoring-As-A-Service-Backlog, 10Research Ideas: Wikivoyage Travel Agent - https://phabricator.wikimedia.org/T155589#2947820 (10Halfak)
[01:39:04] <halfak>	 o/ Amir1 
[01:39:05] <halfak>	 :) 
[01:39:24] <Amir1>	 halfak: Hey
[01:39:30] <Amir1>	 Check mail please
[01:39:38] <halfak>	 Just saw the broken form builder
[01:39:51] <Amir1>	 Thanks
[01:40:16] <Amir1>	 Today I bought a 24" monitor so I use it alongside with laptop
[01:40:19] <Amir1>	 so fancy :D
[01:40:22] <halfak>	 Nice!
[01:40:32] <halfak>	 Much better I imagine :) 
[01:40:39] <halfak>	 Are you looking into the form builder or should I take a quick look?
[01:40:55] <Amir1>	 I won't leave my house easily, probably I'll start talking to myself after a week :P
[01:41:07] <Amir1>	 halfak: I already made a PR
[01:41:52] <Amir1>	 halfak: it'll fix the front-end but I didn't investigate the backend 
[01:42:59] <halfak>	 There's a backend problem?
[01:45:28] <Amir1>	 halfak: I didn't check
[01:45:34] <Amir1>	 that's what I'm saying 
[01:46:08] <halfak>	 Gotcha. The form builder shouldn't affect the backend at all. 
[01:46:15] <halfak>	 It's really just a single page app
[01:47:41] <Amir1>	 oh, I see
[02:29:21] <Amir1>	 halfak: Do you think we can deploy it now?
[07:46:17] <wikibugs>	 10Revision-Scoring-As-A-Service-Backlog, 10DBA, 10MediaWiki-General-or-Unknown, 10ORES: Fatal exception of type "DBQueryError" on sorting ORES contributions - https://phabricator.wikimedia.org/T155500#2945272 (10Marostegui) Hello,   Thanks for the ticket. I have been taking a look and this looks related to...
[19:27:44] <icinga-wm>	 PROBLEM - ORES web node labs ores-web-03 on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[19:29:44] <icinga-wm>	 PROBLEM - ORES worker labs on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[19:31:34] <icinga-wm>	 RECOVERY - ORES worker labs on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 457 bytes in 0.066 second response time
[19:38:58] <halfak>	 I'll check on web-03 in a couple of minutes
[19:45:44] <icinga-wm>	 PROBLEM - ORES worker labs on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[19:46:34] <icinga-wm>	 RECOVERY - ORES worker labs on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 442 bytes in 0.596 second response time
[19:47:34] <icinga-wm>	 RECOVERY - ORES web node labs ores-web-03 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 457 bytes in 2.237 second response time
[19:47:43] <halfak>	 Looks like precached is having a memory usage issue again 
[20:06:55] <halfak>	 Amir1, have a minute to look at https://github.com/wiki-ai/ores-wmflabs-deploy/pull/72 ?
[20:14:27] <halfak>	 Amir1, I noticed we have an urgent disk space situation on ores compute. 
[20:14:38] <halfak>	 I'm moving some of your folders to the /srv drive that has way more space. 
[20:14:41] <halfak>	 I'll leave symlinks. 
[20:14:47] <halfak>	 And some notes for you. 
[20:26:39] <halfak>	 $ sudo ls -al ~ladsgroup/ | grep "/srv"
[20:26:39] <halfak>	 lrwxrwxrwx  1 ladsgroup wikidev      35 Jan 18 20:22 editquality -> /srv/ladsgroup/projects/editquality
[20:26:39] <halfak>	 lrwxrwxrwx  1 ladsgroup wikidev      42 Jan 18 20:23 gerrit-ores-config -> /srv/ladsgroup/projects/gerrit-ores-config
[20:26:39] <halfak>	 lrwxrwxrwx  1 ladsgroup wikidev      45 Jan 18 20:23 ores-wikimedia-config -> /srv/ladsgroup/projects/ores-wikimedia-config
[20:26:48] <halfak>	 OK.  That freed up 4GB
[20:30:22] <halfak>	 Just opened up 2.4GB more by moving agx's stuff around
[21:13:29] <wikibugs>	 (03PS1) 10Sbisson: [WIP] Fetch thresholds from live stats for filters [extensions/ORES] - 10https://gerrit.wikimedia.org/r/332825 (https://phabricator.wikimedia.org/T152161)
[21:16:19] <wikibugs>	 (03PS2) 10Sbisson: [WIP] Fetch thresholds from live stats for filters [extensions/ORES] - 10https://gerrit.wikimedia.org/r/332825 (https://phabricator.wikimedia.org/T152161)
[21:18:36] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] [WIP] Fetch thresholds from live stats for filters [extensions/ORES] - 10https://gerrit.wikimedia.org/r/332825 (https://phabricator.wikimedia.org/T152161) (owner: 10Sbisson)
[21:25:18] <wikibugs>	 (03PS3) 10Sbisson: [WIP] Fetch thresholds from live stats for filters [extensions/ORES] - 10https://gerrit.wikimedia.org/r/332825 (https://phabricator.wikimedia.org/T152161)
[21:34:59] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] [WIP] Fetch thresholds from live stats for filters [extensions/ORES] - 10https://gerrit.wikimedia.org/r/332825 (https://phabricator.wikimedia.org/T152161) (owner: 10Sbisson)
[22:19:06] <wikibugs>	 (03PS4) 10Sbisson: [WIP] Fetch thresholds from live stats for filters [extensions/ORES] - 10https://gerrit.wikimedia.org/r/332825 (https://phabricator.wikimedia.org/T152161)
[22:25:56] <wikibugs>	 06Revision-Scoring-As-A-Service, 10Edit-Review-Improvements, 10rsaas-editquality, 06Collaboration-Team-Triage (Collab-Team-Q3-Jan-Mar-2017), 13Patch-For-Review: Automatically adjust ORES threshold settings when ORES models are updated - https://phabricator.wikimedia.org/T152161#2950775 (10SBisson) @Halfa...