{
  "id": 266662,
  "title": "58th place solution",
  "url": "/competitions/seti-breakthrough-listen/discussion/266662",
  "author_name": "imori",
  "post_date": "2021-08-19T22:37:34.417000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Thank you for competition hosts and participants!</p>\n<p>This is 58th place solution. (LB:83th place)</p>\n<h1>Method</h1>\n<ul>\n<li>ensemble 5 models and weighted using ridge regression</li>\n</ul>\n<p>efficientnet_b4(6 channel, 512) CV:0.8450 LB:0.73689 PB:0.74290<br>\nefficientnetv2_m(576) CV:0.8749 LB:0.77383 PB:0.77542<br>\nefficientnetv2_s(576) CV:0.8738 LB:0.77502 PB:0.77084<br>\nefficientnetv2_s(576) CV:0.8738 LB:0.77360 PB:0.77222<br>\nefficientnet_b4(512)  CV:0.8731 LB:0.77383 PB:0.77542</p>\n<ul>\n<li>stratified 5 fold</li>\n<li>mixup alpha=1(some model used alpha=0.4~0.6)</li>\n<li>data aug (hflip, vflip, shift scale rotate, coarse dropout, motion blur, IAA sharpen)</li>\n<li>TTA (hflip, vflip, hflip+vflip)</li>\n<li>around 15 epoch</li>\n<li>adamW optimizer + cosine annealing LR</li>\n<li>use 3 channel  (may be competition key idea)</li>\n<li>large input boosts LB</li>\n</ul>\n<p>Ensembling with other model(LB around 0.77) is not effective.</p>\n<p>We started 2 weeks ago, so it is hard time to fine-tune training parameters and selecting models.</p>\n<h1>not working</h1>\n<ul>\n<li>other model(seresnext50, efficientnet_b0,b3, efficientnet_v2_b3, swin-transformer, nfnet-0, resizer model, volo)</li>\n<li>mean output (LB:0.77987  PB:0.77936)  This doesn't look so bad.</li>\n<li>weight from logistic regression</li>\n<li>6channel (ensembling with 3chanel looks effective)</li>\n<li>stacking some classifier (svc, nn, logistic regression etc)  This improves oof val score, but it may be overfitted.</li>\n<li>multi loss (BCE loss + L1 smooth loss)</li>\n<li>retraining (I may made mistakes to select learing rate)</li>\n<li>pseudo labeling  (overfit)</li>\n</ul>\n<h1>Regret</h1>\n<ul>\n<li>try other efficientnet models (ensembling with these improves pb and I may be able to get silver).</li>\n<li>adversarial validation to fill in CV-LB gap   I posted here <a href=\"https://www.kaggle.com/c/seti-breakthrough-listen/discussion/266652\" target=\"_blank\">https://www.kaggle.com/c/seti-breakthrough-listen/discussion/266652</a></li>\n<li>use 3 channel earlier   ( I wasted a few days….)</li>\n<li>EDA (we took almost much time to create models)</li>\n</ul>\n<p>We didn't have enough time and climbing LB was very hard because of late joining, but I'm glad to join and got some new idea like ResizerModel and VOLO (thank you authors).</p>\n<p>Thanks</p>",
  "messages": [
    {
      "id": 1482147,
      "postDate": "2021-08-19T22:37:34.417Z",
      "content": "<p>Thank you for competition hosts and participants!</p>\n<p>This is 58th place solution. (LB:83th place)</p>\n<h1>Method</h1>\n<ul>\n<li>ensemble 5 models and weighted using ridge regression</li>\n</ul>\n<p>efficientnet_b4(6 channel, 512) CV:0.8450 LB:0.73689 PB:0.74290<br>\nefficientnetv2_m(576) CV:0.8749 LB:0.77383 PB:0.77542<br>\nefficientnetv2_s(576) CV:0.8738 LB:0.77502 PB:0.77084<br>\nefficientnetv2_s(576) CV:0.8738 LB:0.77360 PB:0.77222<br>\nefficientnet_b4(512)  CV:0.8731 LB:0.77383 PB:0.77542</p>\n<ul>\n<li>stratified 5 fold</li>\n<li>mixup alpha=1(some model used alpha=0.4~0.6)</li>\n<li>data aug (hflip, vflip, shift scale rotate, coarse dropout, motion blur, IAA sharpen)</li>\n<li>TTA (hflip, vflip, hflip+vflip)</li>\n<li>around 15 epoch</li>\n<li>adamW optimizer + cosine annealing LR</li>\n<li>use 3 channel  (may be competition key idea)</li>\n<li>large input boosts LB</li>\n</ul>\n<p>Ensembling with other model(LB around 0.77) is not effective.</p>\n<p>We started 2 weeks ago, so it is hard time to fine-tune training parameters and selecting models.</p>\n<h1>not working</h1>\n<ul>\n<li>other model(seresnext50, efficientnet_b0,b3, efficientnet_v2_b3, swin-transformer, nfnet-0, resizer model, volo)</li>\n<li>mean output (LB:0.77987  PB:0.77936)  This doesn't look so bad.</li>\n<li>weight from logistic regression</li>\n<li>6channel (ensembling with 3chanel looks effective)</li>\n<li>stacking some classifier (svc, nn, logistic regression etc)  This improves oof val score, but it may be overfitted.</li>\n<li>multi loss (BCE loss + L1 smooth loss)</li>\n<li>retraining (I may made mistakes to select learing rate)</li>\n<li>pseudo labeling  (overfit)</li>\n</ul>\n<h1>Regret</h1>\n<ul>\n<li>try other efficientnet models (ensembling with these improves pb and I may be able to get silver).</li>\n<li>adversarial validation to fill in CV-LB gap   I posted here <a href=\"https://www.kaggle.com/c/seti-breakthrough-listen/discussion/266652\" target=\"_blank\">https://www.kaggle.com/c/seti-breakthrough-listen/discussion/266652</a></li>\n<li>use 3 channel earlier   ( I wasted a few days….)</li>\n<li>EDA (we took almost much time to create models)</li>\n</ul>\n<p>We didn't have enough time and climbing LB was very hard because of late joining, but I'm glad to join and got some new idea like ResizerModel and VOLO (thank you authors).</p>\n<p>Thanks</p>",
      "rawMarkdown": "Thank you for competition hosts and participants!\n\nThis is 58th place solution. (LB:83th place)\n\n# Method\n\n- ensemble 5 models and weighted using ridge regression\n\nefficientnet_b4(6 channel, 512) CV:0.8450 LB:0.73689 PB:0.74290\nefficientnetv2_m(576) CV:0.8749 LB:0.77383 PB:0.77542\nefficientnetv2_s(576) CV:0.8738 LB:0.77502 PB:0.77084\nefficientnetv2_s(576) CV:0.8738 LB:0.77360 PB:0.77222\nefficientnet_b4(512)  CV:0.8731 LB:0.77383 PB:0.77542\n\n- stratified 5 fold\n- mixup alpha=1(some model used alpha=0.4~0.6)\n- data aug (hflip, vflip, shift scale rotate, coarse dropout, motion blur, IAA sharpen)\n- TTA (hflip, vflip, hflip+vflip)\n- around 15 epoch\n- adamW optimizer + cosine annealing LR\n- use 3 channel  (may be competition key idea)\n- large input boosts LB\n\nEnsembling with other model(LB around 0.77) is not effective.\n\nWe started 2 weeks ago, so it is hard time to fine-tune training parameters and selecting models.\n\n# not working\n\n- other model(seresnext50, efficientnet_b0,b3, efficientnet_v2_b3, swin-transformer, nfnet-0, resizer model, volo)\n- mean output (LB:0.77987  PB:0.77936)  This doesn't look so bad.\n- weight from logistic regression\n- 6channel (ensembling with 3chanel looks effective)\n- stacking some classifier (svc, nn, logistic regression etc)  This improves oof val score, but it may be overfitted.\n- multi loss (BCE loss + L1 smooth loss)\n- retraining (I may made mistakes to select learing rate)\n- pseudo labeling  (overfit)\n\n# Regret\n\n- try other efficientnet models (ensembling with these improves pb and I may be able to get silver).\n- adversarial validation to fill in CV-LB gap   I posted here https://www.kaggle.com/c/seti-breakthrough-listen/discussion/266652\n- use 3 channel earlier   ( I wasted a few days....)\n- EDA (we took almost much time to create models)\n\n\n\nWe didn't have enough time and climbing LB was very hard because of late joining, but I'm glad to join and got some new idea like ResizerModel and VOLO (thank you authors).\n\nThanks",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1482147": "Thank you for competition hosts and participants!\n\nThis is 58th place solution. (LB:83th place)\n\n# Method\n\n- ensemble 5 models and weighted using ridge regression\n\nefficientnet_b4(6 channel, 512) CV:0.8450 LB:0.73689 PB:0.74290\nefficientnetv2_m(576) CV:0.8749 LB:0.77383 PB:0.77542\nefficientnetv2_s(576) CV:0.8738 LB:0.77502 PB:0.77084\nefficientnetv2_s(576) CV:0.8738 LB:0.77360 PB:0.77222\nefficientnet_b4(512)  CV:0.8731 LB:0.77383 PB:0.77542\n\n- stratified 5 fold\n- mixup alpha=1(some model used alpha=0.4~0.6)\n- data aug (hflip, vflip, shift scale rotate, coarse dropout, motion blur, IAA sharpen)\n- TTA (hflip, vflip, hflip+vflip)\n- around 15 epoch\n- adamW optimizer + cosine annealing LR\n- use 3 channel  (may be competition key idea)\n- large input boosts LB\n\nEnsembling with other model(LB around 0.77) is not effective.\n\nWe started 2 weeks ago, so it is hard time to fine-tune training parameters and selecting models.\n\n# not working\n\n- other model(seresnext50, efficientnet_b0,b3, efficientnet_v2_b3, swin-transformer, nfnet-0, resizer model, volo)\n- mean output (LB:0.77987  PB:0.77936)  This doesn't look so bad.\n- weight from logistic regression\n- 6channel (ensembling with 3chanel looks effective)\n- stacking some classifier (svc, nn, logistic regression etc)  This improves oof val score, but it may be overfitted.\n- multi loss (BCE loss + L1 smooth loss)\n- retraining (I may made mistakes to select learing rate)\n- pseudo labeling  (overfit)\n\n# Regret\n\n- try other efficientnet models (ensembling with these improves pb and I may be able to get silver).\n- adversarial validation to fill in CV-LB gap   I posted here https://www.kaggle.com/c/seti-breakthrough-listen/discussion/266652\n- use 3 channel earlier   ( I wasted a few days....)\n- EDA (we took almost much time to create models)\n\n\n\nWe didn't have enough time and climbing LB was very hard because of late joining, but I'm glad to join and got some new idea like ResizerModel and VOLO (thank you authors).\n\nThanks"
  }
}