[06:38:18] <moritzm>	 FYI, I'll switch one of the etcd nodes for the ml cluster away from direct disk storage to DRBD for a bit, so latency alerts may kick in for a bit
[06:38:59] <moritzm>	 the Ganeti server where it's running on is going to be replaced and this is needed for the migration to a new host
[07:06:01] <isaranto>	 Ack!
[07:06:08] <isaranto>	 Morning folks o/
[07:37:20] <moritzm>	 ml-etcd1002 is back on "plain" disk storage
[08:35:27] <aiko>	 morning!
[08:52:21] <isaranto>	 hi Aiko!
[10:21:47] <wikibugs>	 (03PS1) 10Kevin Bazira: langid: normalize text input [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751)
[10:24:51] <wikibugs>	 (03CR) 10CI reject: [V:04-1] langid: normalize text input [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751) (owner: 10Kevin Bazira)
[10:25:17] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team, 13Patch-For-Review: [langid] fasttext only processes one line at a time - https://phabricator.wikimedia.org/T377751#10253451 (10kevinbazira) Text normalization has been added to the langid model-server and it fixed this issue as shown below: `lang=bash, name=Before Norma...
[10:28:24] <wikibugs>	 (03PS2) 10Kevin Bazira: langid: normalize text input [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751)
[10:33:11] <wikibugs>	 (03CR) 10Kevin Bazira: "This patch has been tested on ml-testing, and the results can be seen here: https://phabricator.wikimedia.org/T377751#10253451" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751) (owner: 10Kevin Bazira)
[10:37:14] <kevinbazira>	 isaranto: aiko: o/ here is the patch to fix the fasttext issue in langid: https://gerrit.wikimedia.org/r/1082438
[10:37:15] <kevinbazira>	 please review whenever you get a minute. thanks!
[10:40:19] <isaranto>	 Thanks kevin! I'm going for an interview and will review later
[12:11:10] <wikibugs>	 (03CR) 10Ilias Sarantopoulos: langid: normalize text input (032 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751) (owner: 10Kevin Bazira)
[12:29:39] * isaranto lunch
[13:37:48] <elukey>	 hey folks, not sure if you saw the last updates, but after the I/F team hackathon this is possible:
[13:37:51] <elukey>	 elukey@deploy2002:~$ dig -x 10.67.16.186 +short
[13:37:54] <elukey>	 10-67-16-186.revertrisk-language-agnostic-predictor-default-00025.revertrisk.svc.cluster.local.
[13:37:57] <elukey>	 10-67-16-186.revertrisk-language-agnostic-predictor-default-00025-private.revertrisk.svc.cluster.local.
[13:38:06] <klausman>	 oh, neat!
[13:38:13] <elukey>	 so if you have an IP address of a pod logged somewhere, you can dns reverse it
[13:38:22] <klausman>	 I saw the initial proposal for RRs, I am very happy to see it implemented
[13:38:32] <elukey>	 still not fully automated, but it works
[14:00:39] <wikibugs>	 (03PS3) 10Kevin Bazira: langid: normalize text input [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751)
[14:02:25] <wikibugs>	 (03CR) 10Kevin Bazira: langid: normalize text input (032 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751) (owner: 10Kevin Bazira)
[15:02:20] <wikibugs>	 (03CR) 10Klausman: langid: normalize text input (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1082438 (https://phabricator.wikimedia.org/T377751) (owner: 10Kevin Bazira)
[16:08:39] <isaranto>	 going afk folks o/
[16:37:33] <wikibugs>	 06Machine-Learning-Team, 10Section-Level-Image-Suggestions, 10Section-Topics, 06Structured-Data-Backlog: [XL] Let the model that learns section alignments consume section topics output - https://phabricator.wikimedia.org/T331968#10255192 (10AUgolnikova-WMF)
[20:37:53] <wikibugs>	 06Machine-Learning-Team, 06Data-Platform-SRE: Investigate Label functionality of AMD GPU device plugin on k8s - https://phabricator.wikimedia.org/T373806#10256280 (10Ottomata)