{
  "id": 244125,
  "title": "【rerun】[CV: 0.983 -> 0.825, LB: 0.974 -> 0.726] ResNet18d Baseline w/ Mixup",
  "url": "/competitions/seti-breakthrough-listen/discussion/244125",
  "author_name": "Tawara",
  "post_date": "2021-06-05T09:23:32.302000",
  "votes": 69,
  "comment_count": 22,
  "views": 0,
  "content": "<p>I've published a new baseline using resnet18d trained with Mixup:<br>\n<a href=\"https://www.kaggle.com/ttahara/seti-e-t-resnet18d-baseline\" target=\"_blank\">https://www.kaggle.com/ttahara/seti-e-t-resnet18d-baseline</a>  </p>\n<p>It seems currently the best score notebook (according to sorting by <code>best score</code>) although it use a very small model. </p>\n<p>I had to put limits on training epochs, model input size, etc. because I wanted to include all folds training and test inference in one notebook.</p>\n<p>There is  a lot of room for improvement. Happy Kaggling!</p>\n<p><br>  </p>\n<h6>Points</h6>\n<ul>\n<li>5-Fold Cross Validation (split by Stratified K-Fold)</li>\n<li>Augmentation: Resize -&gt; HFlip -&gt; VFlip -&gt; ShiftScaleRotate -&gt; RandomResizedCrop -&gt; Mixup</li>\n<li>Model Input Size: 1x320x320<ul>\n<li>I use only on-target(\"A\") observations</li>\n<li>how to read a cadence snippet file is as follows:</li></ul></li>\n</ul>\n<pre><code>   img = np.load(path)[[0, 2, 4]]          # shape: (3, 273, 256)\n   img = np.vstack(img)                    # shape: (819, 256)\n   img = img.transpose(1, 0)               # shape: (256, 819)\n</code></pre>",
  "messages": [
    {
      "id": 1336904,
      "postDate": "2021-06-05T09:23:32.303Z",
      "content": "<p>I've published a new baseline using resnet18d trained with Mixup:<br>\n<a href=\"https://www.kaggle.com/ttahara/seti-e-t-resnet18d-baseline\" target=\"_blank\">https://www.kaggle.com/ttahara/seti-e-t-resnet18d-baseline</a>  </p>\n<p>It seems currently the best score notebook (according to sorting by <code>best score</code>) although it use a very small model. </p>\n<p>I had to put limits on training epochs, model input size, etc. because I wanted to include all folds training and test inference in one notebook.</p>\n<p>There is  a lot of room for improvement. Happy Kaggling!</p>\n<p><br>  </p>\n<h6>Points</h6>\n<ul>\n<li>5-Fold Cross Validation (split by Stratified K-Fold)</li>\n<li>Augmentation: Resize -&gt; HFlip -&gt; VFlip -&gt; ShiftScaleRotate -&gt; RandomResizedCrop -&gt; Mixup</li>\n<li>Model Input Size: 1x320x320<ul>\n<li>I use only on-target(\"A\") observations</li>\n<li>how to read a cadence snippet file is as follows:</li></ul></li>\n</ul>\n<pre><code>   img = np.load(path)[[0, 2, 4]]          # shape: (3, 273, 256)\n   img = np.vstack(img)                    # shape: (819, 256)\n   img = img.transpose(1, 0)               # shape: (256, 819)\n</code></pre>",
      "rawMarkdown": " I've published a new baseline using resnet18d trained with Mixup:\nhttps://www.kaggle.com/ttahara/seti-e-t-resnet18d-baseline  \n\n  \nIt seems currently the best score notebook (according to sorting by `best score`) although it use a very small model. \n  \nI had to put limits on training epochs, model input size, etc. because I wanted to include all folds training and test inference in one notebook.\n  \n\nThere is  a lot of room for improvement. Happy Kaggling!\n  \n\n<br>  \n###### Points\n* 5-Fold Cross Validation (split by Stratified K-Fold)\n* Augmentation: Resize -> HFlip -> VFlip -> ShiftScaleRotate -> RandomResizedCrop -> Mixup\n* Model Input Size: 1x320x320\n    * I use only on-target(\"A\") observations\n    * how to read a cadence snippet file is as follows:\n    ```python\n    img = np.load(path)[[0, 2, 4]]          # shape: (3, 273, 256)\n    img = np.vstack(img)                    # shape: (819, 256)\n    img = img.transpose(1, 0)               # shape: (256, 819)\n    ```",
      "votes": 68
    },
    {
      "id": 1340087,
      "postDate": "2021-06-07T16:53:49.820Z",
      "content": "<p>thanks for sharing,it's useful for me to reference</p>",
      "rawMarkdown": "thanks for sharing,it's useful for me to reference",
      "votes": 1
    },
    {
      "id": 1338257,
      "postDate": "2021-06-06T09:59:50.967Z",
      "content": "<p>Thanks for this, I am still trying to break 0.98 on a single model, so far I think my single model (efficientnetb0) is around 0.978-0.979. I should draw some insights from your notebook since your resnet34d broke 0.98 ~~</p>\n<p>It may be because my network has not converged…I only train 16 epochs.</p>",
      "rawMarkdown": "Thanks for this, I am still trying to break 0.98 on a single model, so far I think my single model (efficientnetb0) is around 0.978-0.979. I should draw some insights from your notebook since your resnet34d broke 0.98 ~~\n\nIt may be because my network has not converged...I only train 16 epochs.",
      "votes": 1,
      "replies": [
        {
          "id": 1338314,
          "postDate": "2021-06-06T10:58:20.023Z",
          "content": "<p>single model with LB 0.978x ??<br>\nwow……..</p>",
          "rawMarkdown": "single model with LB 0.978x ??\nwow........",
          "votes": 1
        },
        {
          "id": 1338353,
          "postDate": "2021-06-06T11:25:14.980Z",
          "content": "<p>You can go to the CV vs LB discussion forum to see. There’s a handful who reached 0.98 LB using single model. </p>",
          "rawMarkdown": "You can go to the CV vs LB discussion forum to see. There’s a handful who reached 0.98 LB using single model. ",
          "votes": 2
        },
        {
          "id": 1338396,
          "postDate": "2021-06-06T12:21:29.157Z",
          "content": "<p><a href=\"https://www.kaggle.com/reighns\" target=\"_blank\">@reighns</a><br>\nIn my case, Mixup requires  more and more training epochs. I'm sure that you'll break 0.98 by single model in the near future :)</p>",
          "rawMarkdown": "@reighns\nIn my case, Mixup requires  more and more training epochs. I'm sure that you'll break 0.98 by single model in the near future :)",
          "votes": 5
        },
        {
          "id": 1339234,
          "postDate": "2021-06-07T06:02:43.490Z",
          "content": "<p><a href=\"https://www.kaggle.com/assign\" target=\"_blank\">@assign</a> I'm sure that <a href=\"https://www.kaggle.com/reighns\" target=\"_blank\">@reighns</a> isn't talking about LB, as it only shows you 2 digits, not 3.</p>",
          "rawMarkdown": "@assign I'm sure that @reighns isn't talking about LB, as it only shows you 2 digits, not 3."
        },
        {
          "id": 1342627,
          "postDate": "2021-06-09T15:54:26.687Z",
          "content": "<p>Can you please correct me if I misunderstand something? That would be helpful.</p>",
          "rawMarkdown": "Can you please correct me if I misunderstand something? That would be helpful."
        },
        {
          "id": 1343693,
          "postDate": "2021-06-10T11:31:37.537Z",
          "content": "<p>I am referring to LB here. As I sort my LB scores and compare it with the LB standing. I’m like 2-3 places away from 0.98 so I deduce it should be 0.978-0979</p>",
          "rawMarkdown": "I am referring to LB here. As I sort my LB scores and compare it with the LB standing. I’m like 2-3 places away from 0.98 so I deduce it should be 0.978-0979",
          "votes": 2
        },
        {
          "id": 1344179,
          "postDate": "2021-06-10T17:08:26.303Z",
          "content": "<p>Oh I see. Interesting!<br>\nThanks!</p>",
          "rawMarkdown": "Oh I see. Interesting!\nThanks!"
        }
      ]
    },
    {
      "id": 1338555,
      "postDate": "2021-06-06T14:45:36.223Z",
      "content": "<p>Thanks for sharing your findings with the community!</p>",
      "rawMarkdown": "Thanks for sharing your findings with the community!",
      "votes": 2
    },
    {
      "id": 1338307,
      "postDate": "2021-06-06T10:46:54.380Z",
      "content": "<p>Thank you for sharing excellent code and results.<br>\nI have two (may be noob) questions.</p>\n<ul>\n<li>Is it necessary to transpose img at last (img = img.transpose(1, 0))? As almost everyone does it, I might be missing something.</li>\n<li>Is it necessary to Resize and ShiftScaleRotate? I thought it might be enough if I rotate and RandomResizedCrop at last.</li>\n</ul>",
      "rawMarkdown": "Thank you for sharing excellent code and results.\nI have two (may be noob) questions.\n- Is it necessary to transpose img at last (img = img.transpose(1, 0))? As almost everyone does it, I might be missing something.\n- Is it necessary to Resize and ShiftScaleRotate? I thought it might be enough if I rotate and RandomResizedCrop at last.",
      "votes": 2,
      "replies": [
        {
          "id": 1338402,
          "postDate": "2021-06-06T12:25:38.270Z",
          "content": "<blockquote>\n  <p>Is it necessary to transpose img at last (img = img.transpose(1, 0))?</p>\n</blockquote>\n<p>I think it is not necessary. For example, <a href=\"https://www.kaggle.com/yukia18\" target=\"_blank\">@yukia18</a> achieved good score without transposing:  <br>\n<a href=\"https://www.kaggle.com/c/seti-breakthrough-listen/discussion/241195#1325779\" target=\"_blank\">https://www.kaggle.com/c/seti-breakthrough-listen/discussion/241195#1325779</a></p>\n<blockquote>\n  <p>Is it necessary to Resize and ShiftScaleRotate? </p>\n</blockquote>\n<p>I don't know, but removing them made CV worse in my case.</p>",
          "rawMarkdown": "> Is it necessary to transpose img at last (img = img.transpose(1, 0))?\n\nI think it is not necessary. For example, @yukia18 achieved good score without transposing:  \nhttps://www.kaggle.com/c/seti-breakthrough-listen/discussion/241195#1325779\n\n> Is it necessary to Resize and ShiftScaleRotate? \n\nI don't know, but removing them made CV worse in my case.",
          "votes": 1
        },
        {
          "id": 1338430,
          "postDate": "2021-06-06T13:00:19.947Z",
          "content": "<p>Thank you for your reply.<br>\nI will learn from your code and investigate more detail.</p>",
          "rawMarkdown": "Thank you for your reply.\nI will learn from your code and investigate more detail.",
          "votes": 1
        },
        {
          "id": 1338534,
          "postDate": "2021-06-06T14:24:24.913Z",
          "content": "<p>if i  may ask what size did you resize you images to and what size did you for randomresizedcrop ?</p>",
          "rawMarkdown": "if i  may ask what size did you resize you images to and what size did you for randomresizedcrop ?"
        },
        {
          "id": 1338613,
          "postDate": "2021-06-06T15:29:51.657Z",
          "content": "<p>In this baseline, image sizes used for Resize and RandomResizedCrop are same. (height, width) = (320, 320)</p>",
          "rawMarkdown": "In this baseline, image sizes used for Resize and RandomResizedCrop are same. (height, width) = (320, 320)"
        },
        {
          "id": 1339213,
          "postDate": "2021-06-07T05:38:28.833Z",
          "content": "<p>Its a bit weird because I think Your randomresizedcrop is not doing anything but your cv is going down if you remove resize </p>",
          "rawMarkdown": "Its a bit weird because I think Your randomresizedcrop is not doing anything but your cv is going down if you remove resize "
        }
      ]
    },
    {
      "id": 1395709,
      "postDate": "2021-07-21T13:35:45.587Z",
      "content": "<p><strong>Update</strong>: I ran the notebook again for the updated competition dataset.</p>",
      "rawMarkdown": "**Update**: I ran the notebook again for the updated competition dataset."
    },
    {
      "id": 1341705,
      "postDate": "2021-06-08T22:09:05.853Z",
      "content": "<p><a href=\"https://www.kaggle.com/ttahara\" target=\"_blank\">@ttahara</a> thanks for this! So far I've used it to run about half a dozen experiments on various augmentations.</p>\n<p>Does anyone know if there is a way to see where ResNet thinks the alien signal is? Maybe some kind of heat map?</p>",
      "rawMarkdown": "@ttahara thanks for this! So far I've used it to run about half a dozen experiments on various augmentations.\n\nDoes anyone know if there is a way to see where ResNet thinks the alien signal is? Maybe some kind of heat map?\n",
      "replies": [
        {
          "id": 1341709,
          "postDate": "2021-06-08T22:22:38.313Z",
          "rawMarkdown": "",
          "votes": 2,
          "isDeleted": true
        },
        {
          "id": 1345686,
          "postDate": "2021-06-11T19:13:29.803Z",
          "content": "<p>Also if you are using PyTorch, you can have a look at <a href=\"url\" target=\"_blank\">https://captum.ai/</a>. This library makes it very easy to apply many model interpretation methods to a PyTorch model.</p>",
          "rawMarkdown": "Also if you are using PyTorch, you can have a look at [https://captum.ai/](url). This library makes it very easy to apply many model interpretation methods to a PyTorch model.",
          "votes": 2
        }
      ]
    },
    {
      "id": 1341579,
      "postDate": "2021-06-08T18:21:23.067Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true
    },
    {
      "id": 1338986,
      "postDate": "2021-06-06T23:30:30.747Z",
      "content": "<p>thank you a lot.i try to use it.</p>",
      "rawMarkdown": "thank you a lot.i try to use it.",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 1340087,
      "author_name": "M.Fauzan Alfariz",
      "author_url": "",
      "post_date": "2021-06-07T16:53:49.820000",
      "content": "<p>thanks for sharing,it's useful for me to reference</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1338257,
      "author_name": "gao-hongnan",
      "author_url": "",
      "post_date": "2021-06-06T09:59:50.967000",
      "content": "<p>Thanks for this, I am still trying to break 0.98 on a single model, so far I think my single model (efficientnetb0) is around 0.978-0.979. I should draw some insights from your notebook since your resnet34d broke 0.98 ~~</p>\n<p>It may be because my network has not converged…I only train 16 epochs.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1338314,
          "author_name": "assign",
          "author_url": "",
          "post_date": "2021-06-06T10:58:20.023000",
          "content": "<p>single model with LB 0.978x ??<br>\nwow……..</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1338353,
          "author_name": "gao-hongnan",
          "author_url": "",
          "post_date": "2021-06-06T11:25:14.980000",
          "content": "<p>You can go to the CV vs LB discussion forum to see. There’s a handful who reached 0.98 LB using single model. </p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1338396,
          "author_name": "Tawara",
          "author_url": "",
          "post_date": "2021-06-06T12:21:29.157000",
          "content": "<p><a href=\"https://www.kaggle.com/reighns\" target=\"_blank\">@reighns</a><br>\nIn my case, Mixup requires  more and more training epochs. I'm sure that you'll break 0.98 by single model in the near future :)</p>",
          "votes": 5,
          "replies": []
        },
        {
          "id": 1339234,
          "author_name": "nofreewill42",
          "author_url": "",
          "post_date": "2021-06-07T06:02:43.490000",
          "content": "<p><a href=\"https://www.kaggle.com/assign\" target=\"_blank\">@assign</a> I'm sure that <a href=\"https://www.kaggle.com/reighns\" target=\"_blank\">@reighns</a> isn't talking about LB, as it only shows you 2 digits, not 3.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1342627,
          "author_name": "nofreewill42",
          "author_url": "",
          "post_date": "2021-06-09T15:54:26.687000",
          "content": "<p>Can you please correct me if I misunderstand something? That would be helpful.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1343693,
          "author_name": "gao-hongnan",
          "author_url": "",
          "post_date": "2021-06-10T11:31:37.537000",
          "content": "<p>I am referring to LB here. As I sort my LB scores and compare it with the LB standing. I’m like 2-3 places away from 0.98 so I deduce it should be 0.978-0979</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1344179,
          "author_name": "nofreewill42",
          "author_url": "",
          "post_date": "2021-06-10T17:08:26.303000",
          "content": "<p>Oh I see. Interesting!<br>\nThanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1338555,
      "author_name": "Old Monk",
      "author_url": "",
      "post_date": "2021-06-06T14:45:36.223000",
      "content": "<p>Thanks for sharing your findings with the community!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1338307,
      "author_name": "tomoo inubushi",
      "author_url": "",
      "post_date": "2021-06-06T10:46:54.380000",
      "content": "<p>Thank you for sharing excellent code and results.<br>\nI have two (may be noob) questions.</p>\n<ul>\n<li>Is it necessary to transpose img at last (img = img.transpose(1, 0))? As almost everyone does it, I might be missing something.</li>\n<li>Is it necessary to Resize and ShiftScaleRotate? I thought it might be enough if I rotate and RandomResizedCrop at last.</li>\n</ul>",
      "votes": 2,
      "replies": [
        {
          "id": 1338402,
          "author_name": "Tawara",
          "author_url": "",
          "post_date": "2021-06-06T12:25:38.270000",
          "content": "<blockquote>\n  <p>Is it necessary to transpose img at last (img = img.transpose(1, 0))?</p>\n</blockquote>\n<p>I think it is not necessary. For example, <a href=\"https://www.kaggle.com/yukia18\" target=\"_blank\">@yukia18</a> achieved good score without transposing:  <br>\n<a href=\"https://www.kaggle.com/c/seti-breakthrough-listen/discussion/241195#1325779\" target=\"_blank\">https://www.kaggle.com/c/seti-breakthrough-listen/discussion/241195#1325779</a></p>\n<blockquote>\n  <p>Is it necessary to Resize and ShiftScaleRotate? </p>\n</blockquote>\n<p>I don't know, but removing them made CV worse in my case.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1338430,
          "author_name": "tomoo inubushi",
          "author_url": "",
          "post_date": "2021-06-06T13:00:19.947000",
          "content": "<p>Thank you for your reply.<br>\nI will learn from your code and investigate more detail.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1338534,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-06-06T14:24:24.913000",
          "content": "<p>if i  may ask what size did you resize you images to and what size did you for randomresizedcrop ?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1338613,
          "author_name": "Tawara",
          "author_url": "",
          "post_date": "2021-06-06T15:29:51.657000",
          "content": "<p>In this baseline, image sizes used for Resize and RandomResizedCrop are same. (height, width) = (320, 320)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1339213,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-06-07T05:38:28.833000",
          "content": "<p>Its a bit weird because I think Your randomresizedcrop is not doing anything but your cv is going down if you remove resize </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1395709,
      "author_name": "Tawara",
      "author_url": "",
      "post_date": "2021-07-21T13:35:45.587000",
      "content": "<p><strong>Update</strong>: I ran the notebook again for the updated competition dataset.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1341705,
      "author_name": "John Clarke",
      "author_url": "",
      "post_date": "2021-06-08T22:09:05.853000",
      "content": "<p><a href=\"https://www.kaggle.com/ttahara\" target=\"_blank\">@ttahara</a> thanks for this! So far I've used it to run about half a dozen experiments on various augmentations.</p>\n<p>Does anyone know if there is a way to see where ResNet thinks the alien signal is? Maybe some kind of heat map?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1341709,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-06-08T22:22:38.313000",
          "content": "",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1345686,
          "author_name": "Ilya Makarov",
          "author_url": "",
          "post_date": "2021-06-11T19:13:29.803000",
          "content": "<p>Also if you are using PyTorch, you can have a look at <a href=\"url\" target=\"_blank\">https://captum.ai/</a>. This library makes it very easy to apply many model interpretation methods to a PyTorch model.</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1341579,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-06-08T18:21:23.067000",
      "content": "",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1338986,
      "author_name": "tensor choko",
      "author_url": "",
      "post_date": "2021-06-06T23:30:30.747000",
      "content": "<p>thank you a lot.i try to use it.</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1336904": " I've published a new baseline using resnet18d trained with Mixup:\nhttps://www.kaggle.com/ttahara/seti-e-t-resnet18d-baseline  \n\n  \nIt seems currently the best score notebook (according to sorting by `best score`) although it use a very small model. \n  \nI had to put limits on training epochs, model input size, etc. because I wanted to include all folds training and test inference in one notebook.\n  \n\nThere is  a lot of room for improvement. Happy Kaggling!\n  \n\n<br>  \n###### Points\n* 5-Fold Cross Validation (split by Stratified K-Fold)\n* Augmentation: Resize -> HFlip -> VFlip -> ShiftScaleRotate -> RandomResizedCrop -> Mixup\n* Model Input Size: 1x320x320\n    * I use only on-target(\"A\") observations\n    * how to read a cadence snippet file is as follows:\n    ```python\n    img = np.load(path)[[0, 2, 4]]          # shape: (3, 273, 256)\n    img = np.vstack(img)                    # shape: (819, 256)\n    img = img.transpose(1, 0)               # shape: (256, 819)\n    ```",
    "1340087": "thanks for sharing,it's useful for me to reference",
    "1338257": "Thanks for this, I am still trying to break 0.98 on a single model, so far I think my single model (efficientnetb0) is around 0.978-0.979. I should draw some insights from your notebook since your resnet34d broke 0.98 ~~\n\nIt may be because my network has not converged...I only train 16 epochs.",
    "1338555": "Thanks for sharing your findings with the community!",
    "1338307": "Thank you for sharing excellent code and results.\nI have two (may be noob) questions.\n- Is it necessary to transpose img at last (img = img.transpose(1, 0))? As almost everyone does it, I might be missing something.\n- Is it necessary to Resize and ShiftScaleRotate? I thought it might be enough if I rotate and RandomResizedCrop at last.",
    "1395709": "**Update**: I ran the notebook again for the updated competition dataset.",
    "1341705": "@ttahara thanks for this! So far I've used it to run about half a dozen experiments on various augmentations.\n\nDoes anyone know if there is a way to see where ResNet thinks the alien signal is? Maybe some kind of heat map?\n",
    "1341579": "",
    "1338986": "thank you a lot.i try to use it."
  }
}