{
  "id": 320850,
  "title": "Arcface with adaptive margin - Tensorflow",
  "url": "/competitions/happy-whale-and-dolphin/discussion/320850",
  "author_name": "",
  "post_date": "2022-04-23T20:28:34.458068900Z",
  "votes": 5,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Can anybody share Tensorflow implementation of \"Arcface with adaptive margin\" used in this competition? I really appreciate that. Thank you!</p>",
  "messages": [
    {
      "id": "1765771",
      "postDate": "04/23/2022 20:28:34",
      "content": "<p>Can anybody share Tensorflow implementation of \"Arcface with adaptive margin\" used in this competition? I really appreciate that. Thank you!</p>",
      "rawMarkdown": "Can anybody share Tensorflow implementation of \"Arcface with adaptive margin\" used in this competition? I really appreciate that. Thank you!",
      "votes": null
    },
    {
      "id": "1765866",
      "postDate": "04/24/2022 01:49:09",
      "content": "<p>I have implemented the modified arcface posted by <a href=\"https://www.kaggle.com/c/landmark-recognition-2020/discussion/187757\" target=\"_blank\">this</a> based on the public arcface implementation in tf, I'm not sure if it is what you need.</p>\n<pre><code>class AdaptiveArcMarginProduct(tf.keras.layers.Layer):\n    def __init__(self, n_classes, s=30, a=0.5, b=0.05, n=-0.25, easy_margin=False,\n                 ls_eps=0.0, **kwargs):\n\n        super(AdaptiveArcMarginProduct, self).__init__(**kwargs)\n\n        self.n_classes = n_classes\n        self.s = s\n        self.margin = tf.constant(a * (freq.sort_index().values ** n) + b, dtype=tf.float32)\n        self.ls_eps = ls_eps\n        self.easy_margin = easy_margin\n\n    def get_config(self):\n\n        config = super().get_config().copy()\n        config.update({\n            'n_classes': self.n_classes,\n            's': self.s,\n            'ls_eps': self.ls_eps,\n            'easy_margin': self.easy_margin,\n        })\n        return config\n\n    def build(self, input_shape):\n        super(AdaptiveArcMarginProduct, self).build(input_shape[0])\n\n        self.W = self.add_weight(\n            name='W',\n            shape=(int(input_shape[0][-1]), self.n_classes),\n            initializer='glorot_uniform',\n            dtype='float32',\n            trainable=True,\n            regularizer=None)\n\n    def call(self, inputs):\n        X, y = inputs\n        y = tf.cast(y, dtype=tf.int32)\n        cosine = tf.matmul(\n            tf.math.l2_normalize(X, axis=1),\n            tf.math.l2_normalize(self.W, axis=0)\n        )\n        sine = tf.math.sqrt(1.0 - tf.math.pow(cosine, 2))\n\n        m = tf.expand_dims(tf.gather(self.margin, y), -1)\n        cos_m = tf.math.cos(m)\n        sin_m = tf.math.sin(m)\n        th = tf.math.cos(math.pi - m)\n        mm = tf.math.sin(math.pi - m) * m\n\n        phi = cosine * cos_m - sine * sin_m\n        if self.easy_margin:\n            phi = tf.where(cosine &gt; 0, phi, cosine)\n        else:\n            phi = tf.where(cosine &gt; th, phi, cosine - mm)\n        one_hot = tf.cast(\n            tf.one_hot(y, depth=self.n_classes),\n            dtype=cosine.dtype\n        )\n        if self.ls_eps &gt; 0:\n            one_hot = (1 - self.ls_eps) * one_hot + self.ls_eps / self.n_classes\n\n        output = (one_hot * phi) + ((1.0 - one_hot) * cosine)\n        output *= self.s\n        return output\n</code></pre>\n<p><code>freq</code> is computed by the train data :</p>\n<pre><code>f = open ('../input/happywhale-splits/individual_ids.json', \"r\")\ntarget_encodings = json.loads(f.read())\ntarget_encodings = {target_encodings[x] :x for x in target_encodings}\ntrain_df = pd.read_csv('../input/happy-whale-and-dolphin/train.csv')\nfreq = train_df.individual_id.map(dict((v ,k) for k ,v in target_encodings.items())).value_counts()\n</code></pre>",
      "rawMarkdown": "I have implemented the modified arcface posted by [this](https://www.kaggle.com/c/landmark-recognition-2020/discussion/187757) based on the public arcface implementation in tf, I'm not sure if it is what you need.\n\n    class AdaptiveArcMarginProduct(tf.keras.layers.Layer):\n        def __init__(self, n_classes, s=30, a=0.5, b=0.05, n=-0.25, easy_margin=False,\n                     ls_eps=0.0, **kwargs):\n    \n            super(AdaptiveArcMarginProduct, self).__init__(**kwargs)\n    \n            self.n_classes = n_classes\n            self.s = s\n            self.margin = tf.constant(a * (freq.sort_index().values ** n) + b, dtype=tf.float32)\n            self.ls_eps = ls_eps\n            self.easy_margin = easy_margin\n    \n        def get_config(self):\n    \n            config = super().get_config().copy()\n            config.update({\n                'n_classes': self.n_classes,\n                's': self.s,\n                'ls_eps': self.ls_eps,\n                'easy_margin': self.easy_margin,\n            })\n            return config\n    \n        def build(self, input_shape):\n            super(AdaptiveArcMarginProduct, self).build(input_shape[0])\n    \n            self.W = self.add_weight(\n                name='W',\n                shape=(int(input_shape[0][-1]), self.n_classes),\n                initializer='glorot_uniform',\n                dtype='float32',\n                trainable=True,\n                regularizer=None)\n    \n        def call(self, inputs):\n            X, y = inputs\n            y = tf.cast(y, dtype=tf.int32)\n            cosine = tf.matmul(\n                tf.math.l2_normalize(X, axis=1),\n                tf.math.l2_normalize(self.W, axis=0)\n            )\n            sine = tf.math.sqrt(1.0 - tf.math.pow(cosine, 2))\n    \n            m = tf.expand_dims(tf.gather(self.margin, y), -1)\n            cos_m = tf.math.cos(m)\n            sin_m = tf.math.sin(m)\n            th = tf.math.cos(math.pi - m)\n            mm = tf.math.sin(math.pi - m) * m\n    \n            phi = cosine * cos_m - sine * sin_m\n            if self.easy_margin:\n                phi = tf.where(cosine > 0, phi, cosine)\n            else:\n                phi = tf.where(cosine > th, phi, cosine - mm)\n            one_hot = tf.cast(\n                tf.one_hot(y, depth=self.n_classes),\n                dtype=cosine.dtype\n            )\n            if self.ls_eps > 0:\n                one_hot = (1 - self.ls_eps) * one_hot + self.ls_eps / self.n_classes\n    \n            output = (one_hot * phi) + ((1.0 - one_hot) * cosine)\n            output *= self.s\n            return output\n\n`freq` is computed by the train data :\n    \n    f = open ('../input/happywhale-splits/individual_ids.json', \"r\")\n    target_encodings = json.loads(f.read())\n    target_encodings = {target_encodings[x] :x for x in target_encodings}\n    train_df = pd.read_csv('../input/happy-whale-and-dolphin/train.csv')\n    freq = train_df.individual_id.map(dict((v ,k) for k ,v in target_encodings.items())).value_counts()",
      "votes": null
    },
    {
      "id": "1766066",
      "postDate": "04/24/2022 06:48:49",
      "content": "<p>Thank you very much! </p>",
      "rawMarkdown": "Thank you very much!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1765866,
      "author_name": "kejiewang",
      "author_url": "",
      "post_date": "04/24/2022 01:49:09",
      "content": "<p>I have implemented the modified arcface posted by <a href=\"https://www.kaggle.com/c/landmark-recognition-2020/discussion/187757\" target=\"_blank\">this</a> based on the public arcface implementation in tf, I'm not sure if it is what you need.</p>\n<pre><code>class AdaptiveArcMarginProduct(tf.keras.layers.Layer):\n    def __init__(self, n_classes, s=30, a=0.5, b=0.05, n=-0.25, easy_margin=False,\n                 ls_eps=0.0, **kwargs):\n\n        super(AdaptiveArcMarginProduct, self).__init__(**kwargs)\n\n        self.n_classes = n_classes\n        self.s = s\n        self.margin = tf.constant(a * (freq.sort_index().values ** n) + b, dtype=tf.float32)\n        self.ls_eps = ls_eps\n        self.easy_margin = easy_margin\n\n    def get_config(self):\n\n        config = super().get_config().copy()\n        config.update({\n            'n_classes': self.n_classes,\n            's': self.s,\n            'ls_eps': self.ls_eps,\n            'easy_margin': self.easy_margin,\n        })\n        return config\n\n    def build(self, input_shape):\n        super(AdaptiveArcMarginProduct, self).build(input_shape[0])\n\n        self.W = self.add_weight(\n            name='W',\n            shape=(int(input_shape[0][-1]), self.n_classes),\n            initializer='glorot_uniform',\n            dtype='float32',\n            trainable=True,\n            regularizer=None)\n\n    def call(self, inputs):\n        X, y = inputs\n        y = tf.cast(y, dtype=tf.int32)\n        cosine = tf.matmul(\n            tf.math.l2_normalize(X, axis=1),\n            tf.math.l2_normalize(self.W, axis=0)\n        )\n        sine = tf.math.sqrt(1.0 - tf.math.pow(cosine, 2))\n\n        m = tf.expand_dims(tf.gather(self.margin, y), -1)\n        cos_m = tf.math.cos(m)\n        sin_m = tf.math.sin(m)\n        th = tf.math.cos(math.pi - m)\n        mm = tf.math.sin(math.pi - m) * m\n\n        phi = cosine * cos_m - sine * sin_m\n        if self.easy_margin:\n            phi = tf.where(cosine &gt; 0, phi, cosine)\n        else:\n            phi = tf.where(cosine &gt; th, phi, cosine - mm)\n        one_hot = tf.cast(\n            tf.one_hot(y, depth=self.n_classes),\n            dtype=cosine.dtype\n        )\n        if self.ls_eps &gt; 0:\n            one_hot = (1 - self.ls_eps) * one_hot + self.ls_eps / self.n_classes\n\n        output = (one_hot * phi) + ((1.0 - one_hot) * cosine)\n        output *= self.s\n        return output\n</code></pre>\n<p><code>freq</code> is computed by the train data :</p>\n<pre><code>f = open ('../input/happywhale-splits/individual_ids.json', \"r\")\ntarget_encodings = json.loads(f.read())\ntarget_encodings = {target_encodings[x] :x for x in target_encodings}\ntrain_df = pd.read_csv('../input/happy-whale-and-dolphin/train.csv')\nfreq = train_df.individual_id.map(dict((v ,k) for k ,v in target_encodings.items())).value_counts()\n</code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 1766066,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "04/24/2022 06:48:49",
          "content": "<p>Thank you very much! </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1765771": "Can anybody share Tensorflow implementation of \"Arcface with adaptive margin\" used in this competition? I really appreciate that. Thank you!",
    "1765866": "I have implemented the modified arcface posted by [this](https://www.kaggle.com/c/landmark-recognition-2020/discussion/187757) based on the public arcface implementation in tf, I'm not sure if it is what you need.\n\n    class AdaptiveArcMarginProduct(tf.keras.layers.Layer):\n        def __init__(self, n_classes, s=30, a=0.5, b=0.05, n=-0.25, easy_margin=False,\n                     ls_eps=0.0, **kwargs):\n    \n            super(AdaptiveArcMarginProduct, self).__init__(**kwargs)\n    \n            self.n_classes = n_classes\n            self.s = s\n            self.margin = tf.constant(a * (freq.sort_index().values ** n) + b, dtype=tf.float32)\n            self.ls_eps = ls_eps\n            self.easy_margin = easy_margin\n    \n        def get_config(self):\n    \n            config = super().get_config().copy()\n            config.update({\n                'n_classes': self.n_classes,\n                's': self.s,\n                'ls_eps': self.ls_eps,\n                'easy_margin': self.easy_margin,\n            })\n            return config\n    \n        def build(self, input_shape):\n            super(AdaptiveArcMarginProduct, self).build(input_shape[0])\n    \n            self.W = self.add_weight(\n                name='W',\n                shape=(int(input_shape[0][-1]), self.n_classes),\n                initializer='glorot_uniform',\n                dtype='float32',\n                trainable=True,\n                regularizer=None)\n    \n        def call(self, inputs):\n            X, y = inputs\n            y = tf.cast(y, dtype=tf.int32)\n            cosine = tf.matmul(\n                tf.math.l2_normalize(X, axis=1),\n                tf.math.l2_normalize(self.W, axis=0)\n            )\n            sine = tf.math.sqrt(1.0 - tf.math.pow(cosine, 2))\n    \n            m = tf.expand_dims(tf.gather(self.margin, y), -1)\n            cos_m = tf.math.cos(m)\n            sin_m = tf.math.sin(m)\n            th = tf.math.cos(math.pi - m)\n            mm = tf.math.sin(math.pi - m) * m\n    \n            phi = cosine * cos_m - sine * sin_m\n            if self.easy_margin:\n                phi = tf.where(cosine > 0, phi, cosine)\n            else:\n                phi = tf.where(cosine > th, phi, cosine - mm)\n            one_hot = tf.cast(\n                tf.one_hot(y, depth=self.n_classes),\n                dtype=cosine.dtype\n            )\n            if self.ls_eps > 0:\n                one_hot = (1 - self.ls_eps) * one_hot + self.ls_eps / self.n_classes\n    \n            output = (one_hot * phi) + ((1.0 - one_hot) * cosine)\n            output *= self.s\n            return output\n\n`freq` is computed by the train data :\n    \n    f = open ('../input/happywhale-splits/individual_ids.json', \"r\")\n    target_encodings = json.loads(f.read())\n    target_encodings = {target_encodings[x] :x for x in target_encodings}\n    train_df = pd.read_csv('../input/happy-whale-and-dolphin/train.csv')\n    freq = train_df.individual_id.map(dict((v ,k) for k ,v in target_encodings.items())).value_counts()",
    "1766066": "Thank you very much!"
  },
  "source": "meta"
}