{
  "id": 347595,
  "title": "TF / Keras custom loss",
  "url": "/competitions/open-problems-multimodal/discussion/347595",
  "author_name": "Lucas Morin",
  "post_date": "2022-08-24T18:26:18.476000",
  "votes": 32,
  "comment_count": 3,
  "views": 0,
  "content": "<p>To build a custom loss I adapted some code I found on a past competition (G-research) to the present competition.</p>\n<p>I came up with the code below. It seems to give the same result as a simple numpy mean/corrcoef implementation and appears to work as a loss within a tf/keras model. As I am using it in a float16 framework it appears some downscaling (division by 100 of all terms) is needed. Feel free to comment if you know how to improve this.</p>\n<pre><code>def correlation_loss(y_true, y_pred):\n    x = y_true\n    y = y_pred\n    mx = K.mean(tf.convert_to_tensor(x),axis=1)\n    my = K.mean(tf.convert_to_tensor(y),axis=1)\n    mx = tf.tile(tf.expand_dims(mx,axis=1),(1,x.shape[1]))\n    my = tf.tile(tf.expand_dims(my,axis=1),(1,x.shape[1]))\n    xm, ym = (x-mx)/100, (y-my)/100\n    r_num = K.sum(tf.multiply(xm,ym),axis=1)\n    r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n    r = tf.reduce_mean(r_num / r_den)\n    r = K.maximum(K.minimum(r, 1.0), -1.0)\n    return 1 - r\n</code></pre>\n<p>More generally, I've shared a custom NN baseline for the CITE part <a href=\"https://www.kaggle.com/lucasmorin/msci-cite-tf-keras-nn-custom-loss/edit\" target=\"_blank\">here</a>. I am trying to make it work on the multimodal part. </p>",
  "messages": [
    {
      "id": 1912458,
      "postDate": "2022-08-24T18:26:18.477Z",
      "content": "<p>To build a custom loss I adapted some code I found on a past competition (G-research) to the present competition.</p>\n<p>I came up with the code below. It seems to give the same result as a simple numpy mean/corrcoef implementation and appears to work as a loss within a tf/keras model. As I am using it in a float16 framework it appears some downscaling (division by 100 of all terms) is needed. Feel free to comment if you know how to improve this.</p>\n<pre><code>def correlation_loss(y_true, y_pred):\n    x = y_true\n    y = y_pred\n    mx = K.mean(tf.convert_to_tensor(x),axis=1)\n    my = K.mean(tf.convert_to_tensor(y),axis=1)\n    mx = tf.tile(tf.expand_dims(mx,axis=1),(1,x.shape[1]))\n    my = tf.tile(tf.expand_dims(my,axis=1),(1,x.shape[1]))\n    xm, ym = (x-mx)/100, (y-my)/100\n    r_num = K.sum(tf.multiply(xm,ym),axis=1)\n    r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n    r = tf.reduce_mean(r_num / r_den)\n    r = K.maximum(K.minimum(r, 1.0), -1.0)\n    return 1 - r\n</code></pre>\n<p>More generally, I've shared a custom NN baseline for the CITE part <a href=\"https://www.kaggle.com/lucasmorin/msci-cite-tf-keras-nn-custom-loss/edit\" target=\"_blank\">here</a>. I am trying to make it work on the multimodal part. </p>",
      "rawMarkdown": "To build a custom loss I adapted some code I found on a past competition (G-research) to the present competition.\n\nI came up with the code below. It seems to give the same result as a simple numpy mean/corrcoef implementation and appears to work as a loss within a tf/keras model. As I am using it in a float16 framework it appears some downscaling (division by 100 of all terms) is needed. Feel free to comment if you know how to improve this.\n\n```\ndef correlation_loss(y_true, y_pred):\n    x = y_true\n    y = y_pred\n    mx = K.mean(tf.convert_to_tensor(x),axis=1)\n    my = K.mean(tf.convert_to_tensor(y),axis=1)\n    mx = tf.tile(tf.expand_dims(mx,axis=1),(1,x.shape[1]))\n    my = tf.tile(tf.expand_dims(my,axis=1),(1,x.shape[1]))\n    xm, ym = (x-mx)/100, (y-my)/100\n    r_num = K.sum(tf.multiply(xm,ym),axis=1)\n    r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n    r = tf.reduce_mean(r_num / r_den)\n    r = K.maximum(K.minimum(r, 1.0), -1.0)\n    return 1 - r\n```\n\nMore generally, I've shared a custom NN baseline for the CITE part [here](https://www.kaggle.com/lucasmorin/msci-cite-tf-keras-nn-custom-loss/edit). I am trying to make it work on the multimodal part. ",
      "votes": 32
    },
    {
      "id": 1952967,
      "postDate": "2022-09-24T05:54:23.503Z",
      "content": "<p>I found this <a href=\"https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch/notebook\" target=\"_blank\">great places</a> having kinds of loss functions in keras and torch.</p>",
      "rawMarkdown": "I found this [great places](https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch/notebook) having kinds of loss functions in keras and torch.",
      "votes": 3
    },
    {
      "id": 1916337,
      "postDate": "2022-08-27T19:17:23.250Z",
      "content": "<p>Thanks for sharing <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> , great work.  For tf.keras.losses class version ,  I added rearranged code below. Thanks again.</p>\n<pre><code>class CorrelationScore(tf.keras.losses.Loss):\n    def __init__(self):\n        super().__init__()\n    def call(self,y_true, y_pred):\n        mx = K.mean(y_true,axis=1)\n        my = K.mean(y_pred,axis=1)\n        mx = tf.tile(tf.expand_dims(mx,axis=1),(1,y_true.shape[1]))\n        my = tf.tile(tf.expand_dims(my,axis=1),(1,y_true.shape[1]))\n        xm, ym = (y_true-mx)/100, (y_pred-my)/100\n        r_num = K.sum(tf.multiply(xm,ym),axis=1)\n        r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n        r = tf.reduce_mean(r_num / r_den)\n        r = K.maximum(K.minimum(r, 1.0), -1.0)\n        return 1 - r\n</code></pre>",
      "rawMarkdown": "Thanks for sharing @lucasmorin , great work.  For tf.keras.losses class version ,  I added rearranged code below. Thanks again.\n\n```python\nclass CorrelationScore(tf.keras.losses.Loss):\n    def __init__(self):\n        super().__init__()\n    def call(self,y_true, y_pred):\n        mx = K.mean(y_true,axis=1)\n        my = K.mean(y_pred,axis=1)\n        mx = tf.tile(tf.expand_dims(mx,axis=1),(1,y_true.shape[1]))\n        my = tf.tile(tf.expand_dims(my,axis=1),(1,y_true.shape[1]))\n        xm, ym = (y_true-mx)/100, (y_pred-my)/100\n        r_num = K.sum(tf.multiply(xm,ym),axis=1)\n        r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n        r = tf.reduce_mean(r_num / r_den)\n        r = K.maximum(K.minimum(r, 1.0), -1.0)\n        return 1 - r\n```\n",
      "votes": 4
    },
    {
      "id": 2016735,
      "postDate": "2022-11-04T07:46:35.763Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1952967,
      "author_name": "no-magic",
      "author_url": "",
      "post_date": "2022-09-24T05:54:23.503000",
      "content": "<p>I found this <a href=\"https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch/notebook\" target=\"_blank\">great places</a> having kinds of loss functions in keras and torch.</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 1916337,
      "author_name": "Mustafa Keser",
      "author_url": "",
      "post_date": "2022-08-27T19:17:23.250000",
      "content": "<p>Thanks for sharing <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> , great work.  For tf.keras.losses class version ,  I added rearranged code below. Thanks again.</p>\n<pre><code>class CorrelationScore(tf.keras.losses.Loss):\n    def __init__(self):\n        super().__init__()\n    def call(self,y_true, y_pred):\n        mx = K.mean(y_true,axis=1)\n        my = K.mean(y_pred,axis=1)\n        mx = tf.tile(tf.expand_dims(mx,axis=1),(1,y_true.shape[1]))\n        my = tf.tile(tf.expand_dims(my,axis=1),(1,y_true.shape[1]))\n        xm, ym = (y_true-mx)/100, (y_pred-my)/100\n        r_num = K.sum(tf.multiply(xm,ym),axis=1)\n        r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n        r = tf.reduce_mean(r_num / r_den)\n        r = K.maximum(K.minimum(r, 1.0), -1.0)\n        return 1 - r\n</code></pre>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 2016735,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-11-04T07:46:35.763000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1912458": "To build a custom loss I adapted some code I found on a past competition (G-research) to the present competition.\n\nI came up with the code below. It seems to give the same result as a simple numpy mean/corrcoef implementation and appears to work as a loss within a tf/keras model. As I am using it in a float16 framework it appears some downscaling (division by 100 of all terms) is needed. Feel free to comment if you know how to improve this.\n\n```\ndef correlation_loss(y_true, y_pred):\n    x = y_true\n    y = y_pred\n    mx = K.mean(tf.convert_to_tensor(x),axis=1)\n    my = K.mean(tf.convert_to_tensor(y),axis=1)\n    mx = tf.tile(tf.expand_dims(mx,axis=1),(1,x.shape[1]))\n    my = tf.tile(tf.expand_dims(my,axis=1),(1,x.shape[1]))\n    xm, ym = (x-mx)/100, (y-my)/100\n    r_num = K.sum(tf.multiply(xm,ym),axis=1)\n    r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n    r = tf.reduce_mean(r_num / r_den)\n    r = K.maximum(K.minimum(r, 1.0), -1.0)\n    return 1 - r\n```\n\nMore generally, I've shared a custom NN baseline for the CITE part [here](https://www.kaggle.com/lucasmorin/msci-cite-tf-keras-nn-custom-loss/edit). I am trying to make it work on the multimodal part. ",
    "1952967": "I found this [great places](https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch/notebook) having kinds of loss functions in keras and torch.",
    "1916337": "Thanks for sharing @lucasmorin , great work.  For tf.keras.losses class version ,  I added rearranged code below. Thanks again.\n\n```python\nclass CorrelationScore(tf.keras.losses.Loss):\n    def __init__(self):\n        super().__init__()\n    def call(self,y_true, y_pred):\n        mx = K.mean(y_true,axis=1)\n        my = K.mean(y_pred,axis=1)\n        mx = tf.tile(tf.expand_dims(mx,axis=1),(1,y_true.shape[1]))\n        my = tf.tile(tf.expand_dims(my,axis=1),(1,y_true.shape[1]))\n        xm, ym = (y_true-mx)/100, (y_pred-my)/100\n        r_num = K.sum(tf.multiply(xm,ym),axis=1)\n        r_den = tf.sqrt(tf.multiply(K.sum(K.square(xm),axis=1), K.sum(K.square(ym),axis=1)))\n        r = tf.reduce_mean(r_num / r_den)\n        r = K.maximum(K.minimum(r, 1.0), -1.0)\n        return 1 - r\n```\n",
    "2016735": ""
  }
}