{
  "id": 266394,
  "title": "5th place solution",
  "url": "/competitions/seti-breakthrough-listen/discussion/266394",
  "author_name": "SiNpcw",
  "post_date": "2021-08-19T01:38:22.457000",
  "votes": 37,
  "comment_count": 13,
  "views": 0,
  "content": "<p>Thank you to all the participants for their hard work.<br>\nThanks also to the organizers and hosts.</p>\n<p><strong>Working Topics:</strong></p>\n<ul>\n<li>eca_nfnet_l2 (4fold) + efficientnet_v2_m (4fold)</li>\n<li>x5 tta (base, h-flip, v-flip, h&amp;v-flip, scaling)</li>\n<li>augment (Horizontal flip, Vertical flip, ShiftScaleRotate)</li>\n<li>512x512 use 0/2/4 channel<br>\nIt tends to be better to use images with larger resolution for both LB and CV.</li>\n<li>mixup</li>\n<li>SHOT (<a href=\"https://github.com/tim-learn/SHOT\" target=\"_blank\">https://github.com/tim-learn/SHOT</a>)</li>\n<li>pseudo labeling\nAs it was said in the discussion, we needed to find the difference between training and testing in this competition. As a method to resolve the domain shift we are using pseudo labels. In this competition, we assumed that there would be less swing between public and private tests, since the host had said that they would be split randomly.\nFor the pseudo-labels, we used the following two methods for ensembling. We believe that this creates diversity and contributes to improved robustness.<ol>\n<li>use all pseudo labels</li>\n<li>use only the high reliable pseudo-labels</li></ol></li>\n</ul>\n<p><strong>Not Working Topics:</strong></p>\n<ul>\n<li>focal loss, weighted cross entropy loss</li>\n<li>Using off-target images</li>\n<li>Using preprocessing images<br>\nLearning by merging images differentiated in the frequency direction.</li>\n<li>K-means clustering<br>\nRemove clusters that do not contain test images from the training data.</li>\n<li>Using old SETI's data for Pretraining</li>\n<li>DANN (<a href=\"https://arxiv.org/abs/1505.07818\" target=\"_blank\">https://arxiv.org/abs/1505.07818</a>)</li>\n<li>Audio Spectrogram Transformer (<a href=\"https://arxiv.org/pdf/2104.01778.pdf\" target=\"_blank\">https://arxiv.org/pdf/2104.01778.pdf</a>)</li>\n<li>Unsupervised Data Augmentation for Consistency Training (<a href=\"https://arxiv.org/pdf/1904.12848.pdf\" target=\"_blank\">https://arxiv.org/pdf/1904.12848.pdf</a>)</li>\n<li>ROC_star (<a href=\"https://github.com/iridiumblue/roc-star\" target=\"_blank\">https://github.com/iridiumblue/roc-star</a>)</li>\n</ul>",
  "messages": [
    {
      "id": 1480355,
      "postDate": "2021-08-19T01:38:22.457Z",
      "content": "<p>Thank you to all the participants for their hard work.<br>\nThanks also to the organizers and hosts.</p>\n<p><strong>Working Topics:</strong></p>\n<ul>\n<li>eca_nfnet_l2 (4fold) + efficientnet_v2_m (4fold)</li>\n<li>x5 tta (base, h-flip, v-flip, h&amp;v-flip, scaling)</li>\n<li>augment (Horizontal flip, Vertical flip, ShiftScaleRotate)</li>\n<li>512x512 use 0/2/4 channel<br>\nIt tends to be better to use images with larger resolution for both LB and CV.</li>\n<li>mixup</li>\n<li>SHOT (<a href=\"https://github.com/tim-learn/SHOT\" target=\"_blank\">https://github.com/tim-learn/SHOT</a>)</li>\n<li>pseudo labeling\nAs it was said in the discussion, we needed to find the difference between training and testing in this competition. As a method to resolve the domain shift we are using pseudo labels. In this competition, we assumed that there would be less swing between public and private tests, since the host had said that they would be split randomly.\nFor the pseudo-labels, we used the following two methods for ensembling. We believe that this creates diversity and contributes to improved robustness.<ol>\n<li>use all pseudo labels</li>\n<li>use only the high reliable pseudo-labels</li></ol></li>\n</ul>\n<p><strong>Not Working Topics:</strong></p>\n<ul>\n<li>focal loss, weighted cross entropy loss</li>\n<li>Using off-target images</li>\n<li>Using preprocessing images<br>\nLearning by merging images differentiated in the frequency direction.</li>\n<li>K-means clustering<br>\nRemove clusters that do not contain test images from the training data.</li>\n<li>Using old SETI's data for Pretraining</li>\n<li>DANN (<a href=\"https://arxiv.org/abs/1505.07818\" target=\"_blank\">https://arxiv.org/abs/1505.07818</a>)</li>\n<li>Audio Spectrogram Transformer (<a href=\"https://arxiv.org/pdf/2104.01778.pdf\" target=\"_blank\">https://arxiv.org/pdf/2104.01778.pdf</a>)</li>\n<li>Unsupervised Data Augmentation for Consistency Training (<a href=\"https://arxiv.org/pdf/1904.12848.pdf\" target=\"_blank\">https://arxiv.org/pdf/1904.12848.pdf</a>)</li>\n<li>ROC_star (<a href=\"https://github.com/iridiumblue/roc-star\" target=\"_blank\">https://github.com/iridiumblue/roc-star</a>)</li>\n</ul>",
      "rawMarkdown": "Thank you to all the participants for their hard work.\nThanks also to the organizers and hosts.\n\n**Working Topics:**\n- eca_nfnet_l2 (4fold) + efficientnet_v2_m (4fold)\n- x5 tta (base, h-flip, v-flip, h&v-flip, scaling)\n- augment (Horizontal flip, Vertical flip, ShiftScaleRotate)\n- 512x512 use 0/2/4 channel\nIt tends to be better to use images with larger resolution for both LB and CV.\n- mixup\n- SHOT (https://github.com/tim-learn/SHOT)\n- pseudo labeling\nAs it was said in the discussion, we needed to find the difference between training and testing in this competition. As a method to resolve the domain shift we are using pseudo labels. In this competition, we assumed that there would be less swing between public and private tests, since the host had said that they would be split randomly.\nFor the pseudo-labels, we used the following two methods for ensembling. We believe that this creates diversity and contributes to improved robustness.\n  1. use all pseudo labels\n  1. use only the high reliable pseudo-labels\n\n**Not Working Topics:**\n- focal loss, weighted cross entropy loss\n- Using off-target images\n- Using preprocessing images\n  Learning by merging images differentiated in the frequency direction.\n- K-means clustering\n  Remove clusters that do not contain test images from the training data.\n- Using old SETI's data for Pretraining\n- DANN (https://arxiv.org/abs/1505.07818)\n- Audio Spectrogram Transformer (https://arxiv.org/pdf/2104.01778.pdf)\n- Unsupervised Data Augmentation for Consistency Training (https://arxiv.org/pdf/1904.12848.pdf)\n- ROC_star (https://github.com/iridiumblue/roc-star)",
      "votes": 37
    },
    {
      "id": 1495847,
      "postDate": "2021-08-29T20:41:20.477Z",
      "content": "<p>Great works!</p>",
      "rawMarkdown": "Great works!\n",
      "votes": 1
    },
    {
      "id": 1482242,
      "postDate": "2021-08-20T01:10:16.937Z",
      "content": "<p>Congratulations team. Great job! For pseudo labeling, did you use the predicted value as it, i.e. soft labels. Or did you convert predictions to 0s and 1s, i.e hard labels?</p>",
      "rawMarkdown": "Congratulations team. Great job! For pseudo labeling, did you use the predicted value as it, i.e. soft labels. Or did you convert predictions to 0s and 1s, i.e hard labels?",
      "votes": 2,
      "replies": [
        {
          "id": 1482250,
          "postDate": "2021-08-20T01:18:59.080Z",
          "content": "<p>We used soft labels for pseudo labeling.</p>",
          "rawMarkdown": "We used soft labels for pseudo labeling.",
          "votes": 2
        }
      ]
    },
    {
      "id": 1515864,
      "postDate": "2021-09-17T15:21:22.073Z",
      "content": "<p><a href=\"https://www.kaggle.com/sinpcw\" target=\"_blank\">@sinpcw</a> have u tried any denoising method ?</p>",
      "rawMarkdown": "@sinpcw have u tried any denoising method ?"
    },
    {
      "id": 1486488,
      "postDate": "2021-08-23T02:49:06.257Z",
      "content": "<p>Congratulations! Obviously, you have put a lot of work into this project. I have one question only. Have you extracted and listened to correctly classified ET signals? If you have, can you give us an example, just to hear how the cosmic events sound like?</p>",
      "rawMarkdown": "Congratulations! Obviously, you have put a lot of work into this project. I have one question only. Have you extracted and listened to correctly classified ET signals? If you have, can you give us an example, just to hear how the cosmic events sound like?"
    },
    {
      "id": 1482232,
      "postDate": "2021-08-20T00:56:08.680Z",
      "content": "<p>Great works!<br>\nWould you mind to share your single model scores?</p>",
      "rawMarkdown": "Great works!\nWould you mind to share your single model scores?",
      "replies": [
        {
          "id": 1482240,
          "postDate": "2021-08-20T01:08:43.403Z",
          "content": "<p>The scores for the single model are as follows:</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>private</th>\n<th>public</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>eca_nfnet_l2</td>\n<td>0.79763</td>\n<td>0.79806</td>\n</tr>\n<tr>\n<td>efficientnet_v2_m</td>\n<td>0.79592</td>\n<td>0.79690</td>\n</tr>\n</tbody>\n</table>\n<p>The CV has not been properly verified because of the different folds between models, but it is about publicLB +0.1.</p>",
          "rawMarkdown": "The scores for the single model are as follows:\n|  | private | public | \n| --- | --- | --- |\n| eca_nfnet_l2 | 0.79763 | 0.79806 |\n| efficientnet_v2_m | 0.79592 | 0.79690 |\n\nThe CV has not been properly verified because of the different folds between models, but it is about publicLB +0.1.",
          "votes": 2
        }
      ]
    },
    {
      "id": 1481051,
      "postDate": "2021-08-19T09:37:46.547Z",
      "content": "<p>You did it! I believed you would make it.<br>\nGreat work!</p>",
      "rawMarkdown": "You did it! I believed you would make it.\nGreat work!"
    },
    {
      "id": 1480452,
      "postDate": "2021-08-19T03:16:01.443Z",
      "content": "<p>Nice works，tanks for you sharing。I first get SHOT (<a href=\"https://github.com/tim-learn/SHOT\" target=\"_blank\">https://github.com/tim-learn/SHOT</a>)</p>",
      "rawMarkdown": "Nice works，tanks for you sharing。I first get SHOT (https://github.com/tim-learn/SHOT)\n"
    },
    {
      "id": 1487846,
      "postDate": "2021-08-23T22:15:02.887Z",
      "rawMarkdown": "",
      "isDeleted": true,
      "replies": [
        {
          "id": 1487975,
          "postDate": "2021-08-24T01:58:49.593Z",
          "content": "<p>TTA is Test Time Augmentation. TTA is inference the original image and augmentatied image (such as horizontal-flip), and takes the average.</p>\n<p>for example following:</p>\n<pre><code>x0 = x.to(device)\nx1 = torch.flip(x0, [-1])\ny = (model(x0) + model(x1)) / 2\n</code></pre>",
          "rawMarkdown": "TTA is Test Time Augmentation. TTA is inference the original image and augmentatied image (such as horizontal-flip), and takes the average.\n\nfor example following:\n```\nx0 = x.to(device)\nx1 = torch.flip(x0, [-1])\ny = (model(x0) + model(x1)) / 2\n```\n",
          "votes": 1
        },
        {
          "id": 1487988,
          "postDate": "2021-08-24T02:17:21.353Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1488005,
          "postDate": "2021-08-24T02:41:37.537Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1495847,
      "author_name": "Fuco",
      "author_url": "",
      "post_date": "2021-08-29T20:41:20.477000",
      "content": "<p>Great works!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1482242,
      "author_name": "Chris Deotte",
      "author_url": "",
      "post_date": "2021-08-20T01:10:16.937000",
      "content": "<p>Congratulations team. Great job! For pseudo labeling, did you use the predicted value as it, i.e. soft labels. Or did you convert predictions to 0s and 1s, i.e hard labels?</p>",
      "votes": 2,
      "replies": [
        {
          "id": 1482250,
          "author_name": "SiNpcw",
          "author_url": "",
          "post_date": "2021-08-20T01:18:59.080000",
          "content": "<p>We used soft labels for pseudo labeling.</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1515864,
      "author_name": "Jaideep",
      "author_url": "",
      "post_date": "2021-09-17T15:21:22.073000",
      "content": "<p><a href=\"https://www.kaggle.com/sinpcw\" target=\"_blank\">@sinpcw</a> have u tried any denoising method ?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1486488,
      "author_name": "Dac-Thanh Van",
      "author_url": "",
      "post_date": "2021-08-23T02:49:06.257000",
      "content": "<p>Congratulations! Obviously, you have put a lot of work into this project. I have one question only. Have you extracted and listened to correctly classified ET signals? If you have, can you give us an example, just to hear how the cosmic events sound like?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1482232,
      "author_name": "yabea",
      "author_url": "",
      "post_date": "2021-08-20T00:56:08.680000",
      "content": "<p>Great works!<br>\nWould you mind to share your single model scores?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1482240,
          "author_name": "SiNpcw",
          "author_url": "",
          "post_date": "2021-08-20T01:08:43.403000",
          "content": "<p>The scores for the single model are as follows:</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>private</th>\n<th>public</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>eca_nfnet_l2</td>\n<td>0.79763</td>\n<td>0.79806</td>\n</tr>\n<tr>\n<td>efficientnet_v2_m</td>\n<td>0.79592</td>\n<td>0.79690</td>\n</tr>\n</tbody>\n</table>\n<p>The CV has not been properly verified because of the different folds between models, but it is about publicLB +0.1.</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1481051,
      "author_name": "WOOSUNG YOON",
      "author_url": "",
      "post_date": "2021-08-19T09:37:46.547000",
      "content": "<p>You did it! I believed you would make it.<br>\nGreat work!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1480452,
      "author_name": "yanqiangmiffy",
      "author_url": "",
      "post_date": "2021-08-19T03:16:01.443000",
      "content": "<p>Nice works，tanks for you sharing。I first get SHOT (<a href=\"https://github.com/tim-learn/SHOT\" target=\"_blank\">https://github.com/tim-learn/SHOT</a>)</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1487846,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-08-23T22:15:02.887000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1487975,
          "author_name": "SiNpcw",
          "author_url": "",
          "post_date": "2021-08-24T01:58:49.593000",
          "content": "<p>TTA is Test Time Augmentation. TTA is inference the original image and augmentatied image (such as horizontal-flip), and takes the average.</p>\n<p>for example following:</p>\n<pre><code>x0 = x.to(device)\nx1 = torch.flip(x0, [-1])\ny = (model(x0) + model(x1)) / 2\n</code></pre>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1487988,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-08-24T02:17:21.353000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1488005,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-08-24T02:41:37.537000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1480355": "Thank you to all the participants for their hard work.\nThanks also to the organizers and hosts.\n\n**Working Topics:**\n- eca_nfnet_l2 (4fold) + efficientnet_v2_m (4fold)\n- x5 tta (base, h-flip, v-flip, h&v-flip, scaling)\n- augment (Horizontal flip, Vertical flip, ShiftScaleRotate)\n- 512x512 use 0/2/4 channel\nIt tends to be better to use images with larger resolution for both LB and CV.\n- mixup\n- SHOT (https://github.com/tim-learn/SHOT)\n- pseudo labeling\nAs it was said in the discussion, we needed to find the difference between training and testing in this competition. As a method to resolve the domain shift we are using pseudo labels. In this competition, we assumed that there would be less swing between public and private tests, since the host had said that they would be split randomly.\nFor the pseudo-labels, we used the following two methods for ensembling. We believe that this creates diversity and contributes to improved robustness.\n  1. use all pseudo labels\n  1. use only the high reliable pseudo-labels\n\n**Not Working Topics:**\n- focal loss, weighted cross entropy loss\n- Using off-target images\n- Using preprocessing images\n  Learning by merging images differentiated in the frequency direction.\n- K-means clustering\n  Remove clusters that do not contain test images from the training data.\n- Using old SETI's data for Pretraining\n- DANN (https://arxiv.org/abs/1505.07818)\n- Audio Spectrogram Transformer (https://arxiv.org/pdf/2104.01778.pdf)\n- Unsupervised Data Augmentation for Consistency Training (https://arxiv.org/pdf/1904.12848.pdf)\n- ROC_star (https://github.com/iridiumblue/roc-star)",
    "1495847": "Great works!\n",
    "1482242": "Congratulations team. Great job! For pseudo labeling, did you use the predicted value as it, i.e. soft labels. Or did you convert predictions to 0s and 1s, i.e hard labels?",
    "1515864": "@sinpcw have u tried any denoising method ?",
    "1486488": "Congratulations! Obviously, you have put a lot of work into this project. I have one question only. Have you extracted and listened to correctly classified ET signals? If you have, can you give us an example, just to hear how the cosmic events sound like?",
    "1482232": "Great works!\nWould you mind to share your single model scores?",
    "1481051": "You did it! I believed you would make it.\nGreat work!",
    "1480452": "Nice works，tanks for you sharing。I first get SHOT (https://github.com/tim-learn/SHOT)\n",
    "1487846": ""
  }
}