{
  "id": 239552,
  "title": "Things that didn't work",
  "url": "/competitions/seti-breakthrough-listen/discussion/239552",
  "author_name": "Salman",
  "post_date": "2021-05-16T17:24:30.937000",
  "votes": 52,
  "comment_count": 34,
  "views": 0,
  "content": "<p><strong>Following things didn't work</strong></p>\n<p>Bigger models <br>\nLabel Smoothing<br>\nFocal Loss<br>\nVision Transformer<br>\nHard Example Mining<br>\nTriplet Network<br>\nPseudo Labels<br>\nConvLSTM<br>\nRNN on Embeddings time wise</p>\n<p><strong>Now moving towards, signal processing + Computer Vision</strong><br>\n<strong>My best model till now is B0 - 5 Epochs - Spatial Image - CV 0.976 - LB 0.97</strong></p>\n<p>Update:<br>\nArcFace didn't work either.</p>",
  "messages": [
    {
      "id": 1310476,
      "postDate": "2021-05-16T17:24:30.937Z",
      "content": "<p><strong>Following things didn't work</strong></p>\n<p>Bigger models <br>\nLabel Smoothing<br>\nFocal Loss<br>\nVision Transformer<br>\nHard Example Mining<br>\nTriplet Network<br>\nPseudo Labels<br>\nConvLSTM<br>\nRNN on Embeddings time wise</p>\n<p><strong>Now moving towards, signal processing + Computer Vision</strong><br>\n<strong>My best model till now is B0 - 5 Epochs - Spatial Image - CV 0.976 - LB 0.97</strong></p>\n<p>Update:<br>\nArcFace didn't work either.</p>",
      "rawMarkdown": "**Following things didn't work**\n\nBigger models \nLabel Smoothing\nFocal Loss\nVision Transformer\nHard Example Mining\nTriplet Network\nPseudo Labels\nConvLSTM\nRNN on Embeddings time wise\n\n**Now moving towards, signal processing + Computer Vision**\n**My best model till now is B0 - 5 Epochs - Spatial Image - CV 0.976 - LB 0.97**\n\n\nUpdate:\nArcFace didn't work either.",
      "votes": 52
    },
    {
      "id": 1341216,
      "postDate": "2021-06-08T14:10:23.857Z",
      "content": "<p>This is interesting but more details would be good.  In my latest competition, one top team listed 3 things that did not work for them, and I used them all because they worked for me.  Devil is in detail, and maybe some failure is not due to what you think is the cause.  </p>",
      "rawMarkdown": "This is interesting but more details would be good.  In my latest competition, one top team listed 3 things that did not work for them, and I used them all because they worked for me.  Devil is in detail, and maybe some failure is not due to what you think is the cause.  ",
      "votes": 13
    },
    {
      "id": 1311522,
      "postDate": "2021-05-17T13:03:28.317Z",
      "content": "<p>Best Model:</p>\n<ul>\n<li>Eff-B4 with original image dimension</li>\n<li>6 epochs </li>\n<li>Average Ensemble</li>\n<li>Stratified 4-fold</li>\n<li>BCE loss with logits</li>\n</ul>\n<p>To do: </p>\n<ul>\n<li>Mixup</li>\n<li>Label Smoothing</li>\n</ul>\n<p>Things that work:</p>\n<ol>\n<li>EfficientNet-B0 [Good boost in performance]</li>\n<li>Less Aggressive augmentations </li>\n<li>Original Image size</li>\n<li>B4 Improves the performance </li>\n</ol>\n<p>Things that didn't work:</p>\n<ol>\n<li>Apex - Slows the code. Reduces Memory Usage</li>\n<li>Nvidia Apex - Finding it tough to fix the gradient overflow error. </li>\n<li>Focal Loss - tried multiple gamma values </li>\n<li>Hard negative sampling</li>\n<li>Weighted Sampler </li>\n<li>Oversampling</li>\n</ol>\n<p>Notes:<br>\nI chose value/settings based on </p>\n<ol>\n<li>All fold showed noticeable improvement w.r.t loss</li>\n<li>Loss difference between all folds is minimal<br>\n3.Gave importance to loss over AUC for choosing the best models </li>\n</ol>\n<p>Spl thanks to <br>\n<a href=\"https://www.kaggle.com/yasufuminakama/seti-nfnet-l0-starter-training\" target=\"_blank\">https://www.kaggle.com/yasufuminakama/seti-nfnet-l0-starter-training</a> kernel </p>\n<p>Hope this helps :-)</p>",
      "rawMarkdown": "Best Model:\n- Eff-B4 with original image dimension\n- 6 epochs \n- Average Ensemble\n- Stratified 4-fold\n- BCE loss with logits\n\nTo do: \n- Mixup\n- Label Smoothing\n\nThings that work:\n1. EfficientNet-B0 [Good boost in performance]\n2. Less Aggressive augmentations \n3. Original Image size\n4. B4 Improves the performance \n\nThings that didn't work:\n1. Apex - Slows the code. Reduces Memory Usage\n2. Nvidia Apex - Finding it tough to fix the gradient overflow error. \n3. Focal Loss - tried multiple gamma values \n4. Hard negative sampling\n5. Weighted Sampler \n6. Oversampling\n\nNotes:\nI chose value/settings based on \n1. All fold showed noticeable improvement w.r.t loss\n2. Loss difference between all folds is minimal\n3.Gave importance to loss over AUC for choosing the best models \n\nSpl thanks to \nhttps://www.kaggle.com/yasufuminakama/seti-nfnet-l0-starter-training kernel \n\nHope this helps :-)",
      "votes": 10,
      "replies": [
        {
          "id": 1340626,
          "postDate": "2021-06-08T06:34:38.240Z",
          "content": "<p>Hi, would you happen to know any useful articles you came across that helped you learn the models you most prefer?</p>",
          "rawMarkdown": "Hi, would you happen to know any useful articles you came across that helped you learn the models you most prefer?"
        },
        {
          "id": 1341221,
          "postDate": "2021-06-08T14:15:11.090Z",
          "content": "<p>Why are you trying apex instead of torch.amp? The former was the prototype for the latter.</p>",
          "rawMarkdown": "Why are you trying apex instead of torch.amp? The former was the prototype for the latter.",
          "votes": 3
        }
      ]
    },
    {
      "id": 1320502,
      "postDate": "2021-05-24T06:16:38.710Z",
      "content": "<p>Focal Loss didn't work in many cases seemingly.</p>",
      "rawMarkdown": "Focal Loss didn't work in many cases seemingly.",
      "votes": 4
    },
    {
      "id": 1315032,
      "postDate": "2021-05-19T14:05:15.157Z",
      "content": "<p>Things that work:</p>\n<ol>\n<li>Mixup augmentation</li>\n<li>Change Input from Channel to Spatial</li>\n</ol>",
      "rawMarkdown": "Things that work:\n  1. Mixup augmentation\n  2. Change Input from Channel to Spatial",
      "votes": 1,
      "replies": [
        {
          "id": 1315060,
          "postDate": "2021-05-19T14:29:39.433Z",
          "content": "<p>How did You Implement MixUp ?.</p>",
          "rawMarkdown": "How did You Implement MixUp ?."
        },
        {
          "id": 1315072,
          "postDate": "2021-05-19T14:39:21.497Z",
          "content": "<p>Same as the code snippet in <a href=\"https://openreview.net/attachment?id=r1Ddp1-Rb&amp;name=pdf\" target=\"_blank\">Mixup paper</a>, Figure 1(a).</p>",
          "rawMarkdown": "Same as the code snippet in [Mixup paper](https://openreview.net/attachment?id=r1Ddp1-Rb&name=pdf), Figure 1(a)."
        },
        {
          "id": 1315076,
          "postDate": "2021-05-19T14:45:48.353Z",
          "content": "<p>And how did you change Input from Channel to spatial ?</p>",
          "rawMarkdown": "And how did you change Input from Channel to spatial ?"
        },
        {
          "id": 1315086,
          "postDate": "2021-05-19T14:52:58.423Z",
          "content": "<p>Ha, I learn it from your discussion with Salman.</p>",
          "rawMarkdown": "Ha, I learn it from your discussion with Salman."
        },
        {
          "id": 1315090,
          "postDate": "2021-05-19T14:55:30.080Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 1311073,
      "postDate": "2021-05-17T06:33:08.540Z",
      "content": "<p><a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> Can You share with us How did you convert Normal Images Into spatial Ones</p>",
      "rawMarkdown": "@micheomaano Can You share with us How did you convert Normal Images Into spatial Ones",
      "votes": 1,
      "replies": [
        {
          "id": 1311157,
          "postDate": "2021-05-17T07:24:07.373Z",
          "content": "<p>I used this.</p>\n<p>def f(each):<br>\n    image = np.load(each).astype(float)<br>\n    image = np.vstack(image).transpose((1, 0))<br>\n    x = np.zeros(shape = (3, image.shape[0], image.shape[1]))<br>\n    x[0, :, :] = image  <br>\n    x[1, :, :] = image  <br>\n    x[2, :, :] = image <br>\n    np.save(\"combined/\" + each, x)</p>",
          "rawMarkdown": "I used this.\n\ndef f(each):\n    image = np.load(each).astype(float)\n    image = np.vstack(image).transpose((1, 0))\n    x = np.zeros(shape = (3, image.shape[0], image.shape[1]))\n    x[0, :, :] = image  \n    x[1, :, :] = image  \n    x[2, :, :] = image \n    np.save(\"combined/\" + each, x)",
          "votes": 1
        },
        {
          "id": 1311185,
          "postDate": "2021-05-17T08:00:33.580Z",
          "content": "<p>Thank you very much</p>",
          "rawMarkdown": "Thank you very much"
        },
        {
          "id": 1311221,
          "postDate": "2021-05-17T08:43:13.243Z",
          "content": "<p>Can i ask you how much Time it took you To make The dataset. its taking 16 hours for me ?. Is it normal or am i doing something wrong</p>",
          "rawMarkdown": "Can i ask you how much Time it took you To make The dataset. its taking 16 hours for me ?. Is it normal or am i doing something wrong"
        },
        {
          "id": 1311224,
          "postDate": "2021-05-17T08:45:54.863Z",
          "content": "<p>It took 20 minutes i guess.</p>",
          "rawMarkdown": "It took 20 minutes i guess."
        },
        {
          "id": 1311227,
          "postDate": "2021-05-17T08:47:22.370Z",
          "content": "<p>did you use Tf.glob or os ? this is what code i am using <br>\n`import math<br>\nfrom pathlib import Path</p>\n<p>import numpy as np<br>\nimport pandas as pd<br>\nimport tensorflow as tf<br>\nfrom sklearn.model_selection import train_test_split<br>\nfrom tensorflow.keras import mixed_precision<br>\nfrom tensorflow.keras.utils import Sequence<br>\nimport os<br>\nfrom tqdm import tqdm<br>\nimport shlex</p>\n<p>path = tf.io.gfile.glob(\"F:\\Pycharm_projects\\SETI\\data/train/<em>/</em>.npy\")<br>\nfor i in tqdm(path):<br>\n    image = np.load(i).astype(np.float32)</p>\n<pre><code>i = i.split(\"\\\\\")\nimage = np.vstack(image).transpose((1, 0))\nx = np.zeros(shape=(3, image.shape[0], image.shape[1]))\nx[0, :, :] = image\nx[1, :, :] = image\nx[2, :, :] = image\nnp.save(\"F:\\Pycharm_projects\\SETI/data/spatial/\" + i[5] + \"/\" + i[6] , x)\n</code></pre>\n<p>`</p>",
          "rawMarkdown": "did you use Tf.glob or os ? this is what code i am using \n`import math\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow.keras import mixed_precision\nfrom tensorflow.keras.utils import Sequence\nimport os\nfrom tqdm import tqdm\nimport shlex\n\npath = tf.io.gfile.glob(\"F:\\Pycharm_projects\\SETI\\data/train/*/*.npy\")\nfor i in tqdm(path):\n\timage = np.load(i).astype(np.float32)\n\n\ti = i.split(\"\\\\\")\n\timage = np.vstack(image).transpose((1, 0))\n\tx = np.zeros(shape=(3, image.shape[0], image.shape[1]))\n\tx[0, :, :] = image\n\tx[1, :, :] = image\n\tx[2, :, :] = image\n\tnp.save(\"F:\\Pycharm_projects\\SETI/data/spatial/\" + i[5] + \"/\" + i[6] , x)\n`"
        },
        {
          "id": 1311247,
          "postDate": "2021-05-17T09:07:41.647Z",
          "content": "<p>I guess you should use python multiprocessing</p>\n<p>from multiprocessing import Pool<br>\npool = Pool(number of cores)<br>\npool.map(f, image_paths)</p>",
          "rawMarkdown": "I guess you should use python multiprocessing\n\nfrom multiprocessing import Pool\npool = Pool(number of cores)\npool.map(f, image_paths)",
          "votes": 1
        },
        {
          "id": 1311258,
          "postDate": "2021-05-17T09:16:18.910Z",
          "content": "<p>Can you please upload the dataset to kaggle ?. It would be a great help.Also the images i am saving are having 10 times the Size of normal ones . Is this behaviour normal <a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a></p>",
          "rawMarkdown": "\nCan you please upload the dataset to kaggle ?. It would be a great help.Also the images i am saving are having 10 times the Size of normal ones . Is this behaviour normal @micheomaano"
        },
        {
          "id": 1312717,
          "postDate": "2021-05-18T07:47:55.567Z",
          "content": "<p>its because of data types.<br>\nI'll try to upload soon.</p>",
          "rawMarkdown": "its because of data types.\nI'll try to upload soon.",
          "votes": 1
        },
        {
          "id": 1312766,
          "postDate": "2021-05-18T08:28:24.540Z",
          "content": "<p><a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> thank you very much.  Btw I have figured out a way of doing it </p>",
          "rawMarkdown": "@micheomaano thank you very much.  Btw I have figured out a way of doing it "
        }
      ]
    },
    {
      "id": 1310905,
      "postDate": "2021-05-17T03:54:42.983Z",
      "content": "<p>Hard sample mining then oversampling works bad for me too. I'm wondering is that hard or just noise, cause the competition host seems to put some \"Easter Eggs\" in the dataset.</p>",
      "rawMarkdown": "Hard sample mining then oversampling works bad for me too. I'm wondering is that hard or just noise, cause the competition host seems to put some \"Easter Eggs\" in the dataset.",
      "votes": 1
    },
    {
      "id": 1310574,
      "postDate": "2021-05-16T18:24:19.370Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> ,<br>\nI was too late to read this post… Just spent a whole evening on ViT and trying to figure out why was it performing the way it was 😓😓<br>\nBut also, Thanks for saving my time because I was going to do focal loss next 😅</p>",
      "rawMarkdown": "Hi @micheomaano ,\nI was too late to read this post... Just spent a whole evening on ViT and trying to figure out why was it performing the way it was 😓😓\nBut also, Thanks for saving my time because I was going to do focal loss next 😅",
      "votes": 1,
      "replies": [
        {
          "id": 1310580,
          "postDate": "2021-05-16T18:26:18.693Z",
          "content": "<p>lol.<br>\nYou should alway try tho. <br>\nMay be a single epoch. xD<br>\nBest of luck.  </p>",
          "rawMarkdown": "lol.\nYou should alway try tho. \nMay be a single epoch. xD\nBest of luck.  ",
          "votes": 2
        },
        {
          "id": 1311145,
          "postDate": "2021-05-17T07:17:54.727Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> ,<br>\nI was curious to know if you got any improvement in Spatial over 6-channel?<br>\nIt doesn't show any improvements in my case…</p>",
          "rawMarkdown": "Hi @micheomaano ,\nI was curious to know if you got any improvement in Spatial over 6-channel?\nIt doesn't show any improvements in my case...",
          "votes": 1
        }
      ]
    },
    {
      "id": 1312081,
      "postDate": "2021-05-17T19:47:30.560Z",
      "content": "<p>Balancing the training classes - by oversampling the 1's - made my results a lot worse.</p>",
      "rawMarkdown": "Balancing the training classes - by oversampling the 1's - made my results a lot worse.",
      "votes": 2
    },
    {
      "id": 1310625,
      "postDate": "2021-05-16T19:10:44.810Z",
      "content": "<p>bigger models worked for me</p>",
      "rawMarkdown": "bigger models worked for me",
      "votes": 2
    },
    {
      "id": 1320609,
      "postDate": "2021-05-24T07:35:33.323Z",
      "content": "<p>I'm experimenting with a bigger model, open source code with GRU only once, lb96 +. Pseudo Labels do make LB worse</p>",
      "rawMarkdown": "I'm experimenting with a bigger model, open source code with GRU only once, lb96 +. Pseudo Labels do make LB worse",
      "replies": [
        {
          "id": 1320645,
          "postDate": "2021-05-24T08:05:40.247Z",
          "content": "<p>You only used <a href=\"https://www.kaggle.com/xhlulu\" target=\"_blank\">@xhlulu</a> 's GRU with only one layer, and got LB96+? And you did not do anything else?<br>\nThat is very interesting. Did you figure out about the transposing and time dim. issue?</p>",
          "rawMarkdown": "You only used @xhlulu 's GRU with only one layer, and got LB96+? And you did not do anything else?\nThat is very interesting. Did you figure out about the transposing and time dim. issue?"
        }
      ]
    },
    {
      "id": 1320547,
      "postDate": "2021-05-24T06:38:10.657Z",
      "content": "<p>Has anyone tried weighted BCEWithLogitsLoss wrt to classes?</p>",
      "rawMarkdown": "Has anyone tried weighted BCEWithLogitsLoss wrt to classes?",
      "replies": [
        {
          "id": 1320588,
          "postDate": "2021-05-24T07:20:50.213Z",
          "content": "<p>I tried fairly gentle weighting (2.5 for positives) and it didn't make any difference.</p>",
          "rawMarkdown": "I tried fairly gentle weighting (2.5 for positives) and it didn't make any difference.",
          "votes": 3
        }
      ]
    },
    {
      "id": 1316725,
      "postDate": "2021-05-20T20:07:07.083Z",
      "content": "<p>hflip vflip blur didnt work</p>",
      "rawMarkdown": "hflip vflip blur didnt work"
    },
    {
      "id": 1313324,
      "postDate": "2021-05-18T14:08:07.847Z",
      "content": "<p>What is your implementation of weighted loss?</p>",
      "rawMarkdown": "What is your implementation of weighted loss?",
      "replies": [
        {
          "id": 1313429,
          "postDate": "2021-05-18T15:01:24.983Z",
          "content": "<p>I didn't use that.</p>",
          "rawMarkdown": "I didn't use that."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1341216,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2021-06-08T14:10:23.857000",
      "content": "<p>This is interesting but more details would be good.  In my latest competition, one top team listed 3 things that did not work for them, and I used them all because they worked for me.  Devil is in detail, and maybe some failure is not due to what you think is the cause.  </p>",
      "votes": 13,
      "replies": []
    },
    {
      "id": 1311522,
      "author_name": "Balaji Selvaraj",
      "author_url": "",
      "post_date": "2021-05-17T13:03:28.317000",
      "content": "<p>Best Model:</p>\n<ul>\n<li>Eff-B4 with original image dimension</li>\n<li>6 epochs </li>\n<li>Average Ensemble</li>\n<li>Stratified 4-fold</li>\n<li>BCE loss with logits</li>\n</ul>\n<p>To do: </p>\n<ul>\n<li>Mixup</li>\n<li>Label Smoothing</li>\n</ul>\n<p>Things that work:</p>\n<ol>\n<li>EfficientNet-B0 [Good boost in performance]</li>\n<li>Less Aggressive augmentations </li>\n<li>Original Image size</li>\n<li>B4 Improves the performance </li>\n</ol>\n<p>Things that didn't work:</p>\n<ol>\n<li>Apex - Slows the code. Reduces Memory Usage</li>\n<li>Nvidia Apex - Finding it tough to fix the gradient overflow error. </li>\n<li>Focal Loss - tried multiple gamma values </li>\n<li>Hard negative sampling</li>\n<li>Weighted Sampler </li>\n<li>Oversampling</li>\n</ol>\n<p>Notes:<br>\nI chose value/settings based on </p>\n<ol>\n<li>All fold showed noticeable improvement w.r.t loss</li>\n<li>Loss difference between all folds is minimal<br>\n3.Gave importance to loss over AUC for choosing the best models </li>\n</ol>\n<p>Spl thanks to <br>\n<a href=\"https://www.kaggle.com/yasufuminakama/seti-nfnet-l0-starter-training\" target=\"_blank\">https://www.kaggle.com/yasufuminakama/seti-nfnet-l0-starter-training</a> kernel </p>\n<p>Hope this helps :-)</p>",
      "votes": 10,
      "replies": [
        {
          "id": 1340626,
          "author_name": "Erik Kaufman",
          "author_url": "",
          "post_date": "2021-06-08T06:34:38.240000",
          "content": "<p>Hi, would you happen to know any useful articles you came across that helped you learn the models you most prefer?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1341221,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-06-08T14:15:11.090000",
          "content": "<p>Why are you trying apex instead of torch.amp? The former was the prototype for the latter.</p>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 1320502,
      "author_name": "blueboy-97",
      "author_url": "",
      "post_date": "2021-05-24T06:16:38.710000",
      "content": "<p>Focal Loss didn't work in many cases seemingly.</p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 1315032,
      "author_name": "opdoop",
      "author_url": "",
      "post_date": "2021-05-19T14:05:15.157000",
      "content": "<p>Things that work:</p>\n<ol>\n<li>Mixup augmentation</li>\n<li>Change Input from Channel to Spatial</li>\n</ol>",
      "votes": 1,
      "replies": [
        {
          "id": 1315060,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-05-19T14:29:39.433000",
          "content": "<p>How did You Implement MixUp ?.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1315072,
          "author_name": "opdoop",
          "author_url": "",
          "post_date": "2021-05-19T14:39:21.497000",
          "content": "<p>Same as the code snippet in <a href=\"https://openreview.net/attachment?id=r1Ddp1-Rb&amp;name=pdf\" target=\"_blank\">Mixup paper</a>, Figure 1(a).</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1315076,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-05-19T14:45:48.353000",
          "content": "<p>And how did you change Input from Channel to spatial ?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1315086,
          "author_name": "opdoop",
          "author_url": "",
          "post_date": "2021-05-19T14:52:58.423000",
          "content": "<p>Ha, I learn it from your discussion with Salman.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1315090,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-05-19T14:55:30.080000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1311073,
      "author_name": "Mithil Salunkhe",
      "author_url": "",
      "post_date": "2021-05-17T06:33:08.540000",
      "content": "<p><a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> Can You share with us How did you convert Normal Images Into spatial Ones</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1311157,
          "author_name": "Salman",
          "author_url": "",
          "post_date": "2021-05-17T07:24:07.373000",
          "content": "<p>I used this.</p>\n<p>def f(each):<br>\n    image = np.load(each).astype(float)<br>\n    image = np.vstack(image).transpose((1, 0))<br>\n    x = np.zeros(shape = (3, image.shape[0], image.shape[1]))<br>\n    x[0, :, :] = image  <br>\n    x[1, :, :] = image  <br>\n    x[2, :, :] = image <br>\n    np.save(\"combined/\" + each, x)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1311185,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-05-17T08:00:33.580000",
          "content": "<p>Thank you very much</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1311221,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-05-17T08:43:13.243000",
          "content": "<p>Can i ask you how much Time it took you To make The dataset. its taking 16 hours for me ?. Is it normal or am i doing something wrong</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1311224,
          "author_name": "Salman",
          "author_url": "",
          "post_date": "2021-05-17T08:45:54.863000",
          "content": "<p>It took 20 minutes i guess.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1311227,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-05-17T08:47:22.370000",
          "content": "<p>did you use Tf.glob or os ? this is what code i am using <br>\n`import math<br>\nfrom pathlib import Path</p>\n<p>import numpy as np<br>\nimport pandas as pd<br>\nimport tensorflow as tf<br>\nfrom sklearn.model_selection import train_test_split<br>\nfrom tensorflow.keras import mixed_precision<br>\nfrom tensorflow.keras.utils import Sequence<br>\nimport os<br>\nfrom tqdm import tqdm<br>\nimport shlex</p>\n<p>path = tf.io.gfile.glob(\"F:\\Pycharm_projects\\SETI\\data/train/<em>/</em>.npy\")<br>\nfor i in tqdm(path):<br>\n    image = np.load(i).astype(np.float32)</p>\n<pre><code>i = i.split(\"\\\\\")\nimage = np.vstack(image).transpose((1, 0))\nx = np.zeros(shape=(3, image.shape[0], image.shape[1]))\nx[0, :, :] = image\nx[1, :, :] = image\nx[2, :, :] = image\nnp.save(\"F:\\Pycharm_projects\\SETI/data/spatial/\" + i[5] + \"/\" + i[6] , x)\n</code></pre>\n<p>`</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1311247,
          "author_name": "Salman",
          "author_url": "",
          "post_date": "2021-05-17T09:07:41.647000",
          "content": "<p>I guess you should use python multiprocessing</p>\n<p>from multiprocessing import Pool<br>\npool = Pool(number of cores)<br>\npool.map(f, image_paths)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1311258,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-05-17T09:16:18.910000",
          "content": "<p>Can you please upload the dataset to kaggle ?. It would be a great help.Also the images i am saving are having 10 times the Size of normal ones . Is this behaviour normal <a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1312717,
          "author_name": "Salman",
          "author_url": "",
          "post_date": "2021-05-18T07:47:55.567000",
          "content": "<p>its because of data types.<br>\nI'll try to upload soon.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1312766,
          "author_name": "Mithil Salunkhe",
          "author_url": "",
          "post_date": "2021-05-18T08:28:24.540000",
          "content": "<p><a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> thank you very much.  Btw I have figured out a way of doing it </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1310905,
      "author_name": "Chenglu",
      "author_url": "",
      "post_date": "2021-05-17T03:54:42.983000",
      "content": "<p>Hard sample mining then oversampling works bad for me too. I'm wondering is that hard or just noise, cause the competition host seems to put some \"Easter Eggs\" in the dataset.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1310574,
      "author_name": "Manav",
      "author_url": "",
      "post_date": "2021-05-16T18:24:19.370000",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> ,<br>\nI was too late to read this post… Just spent a whole evening on ViT and trying to figure out why was it performing the way it was 😓😓<br>\nBut also, Thanks for saving my time because I was going to do focal loss next 😅</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1310580,
          "author_name": "Salman",
          "author_url": "",
          "post_date": "2021-05-16T18:26:18.693000",
          "content": "<p>lol.<br>\nYou should alway try tho. <br>\nMay be a single epoch. xD<br>\nBest of luck.  </p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1311145,
          "author_name": "Manav",
          "author_url": "",
          "post_date": "2021-05-17T07:17:54.727000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/micheomaano\" target=\"_blank\">@micheomaano</a> ,<br>\nI was curious to know if you got any improvement in Spatial over 6-channel?<br>\nIt doesn't show any improvements in my case…</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1312081,
      "author_name": "John Clarke",
      "author_url": "",
      "post_date": "2021-05-17T19:47:30.560000",
      "content": "<p>Balancing the training classes - by oversampling the 1's - made my results a lot worse.</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1310625,
      "author_name": "Awsaf",
      "author_url": "",
      "post_date": "2021-05-16T19:10:44.810000",
      "content": "<p>bigger models worked for me</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1320609,
      "author_name": "zhangeng",
      "author_url": "",
      "post_date": "2021-05-24T07:35:33.323000",
      "content": "<p>I'm experimenting with a bigger model, open source code with GRU only once, lb96 +. Pseudo Labels do make LB worse</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1320645,
          "author_name": "Baran Hashemi",
          "author_url": "",
          "post_date": "2021-05-24T08:05:40.247000",
          "content": "<p>You only used <a href=\"https://www.kaggle.com/xhlulu\" target=\"_blank\">@xhlulu</a> 's GRU with only one layer, and got LB96+? And you did not do anything else?<br>\nThat is very interesting. Did you figure out about the transposing and time dim. issue?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1320547,
      "author_name": "Baran Hashemi",
      "author_url": "",
      "post_date": "2021-05-24T06:38:10.657000",
      "content": "<p>Has anyone tried weighted BCEWithLogitsLoss wrt to classes?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1320588,
          "author_name": "James Howard",
          "author_url": "",
          "post_date": "2021-05-24T07:20:50.213000",
          "content": "<p>I tried fairly gentle weighting (2.5 for positives) and it didn't make any difference.</p>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 1316725,
      "author_name": "Gustavo Corradi",
      "author_url": "",
      "post_date": "2021-05-20T20:07:07.083000",
      "content": "<p>hflip vflip blur didnt work</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1313324,
      "author_name": "Baran Hashemi",
      "author_url": "",
      "post_date": "2021-05-18T14:08:07.847000",
      "content": "<p>What is your implementation of weighted loss?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1313429,
          "author_name": "Salman",
          "author_url": "",
          "post_date": "2021-05-18T15:01:24.983000",
          "content": "<p>I didn't use that.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1310476": "**Following things didn't work**\n\nBigger models \nLabel Smoothing\nFocal Loss\nVision Transformer\nHard Example Mining\nTriplet Network\nPseudo Labels\nConvLSTM\nRNN on Embeddings time wise\n\n**Now moving towards, signal processing + Computer Vision**\n**My best model till now is B0 - 5 Epochs - Spatial Image - CV 0.976 - LB 0.97**\n\n\nUpdate:\nArcFace didn't work either.",
    "1341216": "This is interesting but more details would be good.  In my latest competition, one top team listed 3 things that did not work for them, and I used them all because they worked for me.  Devil is in detail, and maybe some failure is not due to what you think is the cause.  ",
    "1311522": "Best Model:\n- Eff-B4 with original image dimension\n- 6 epochs \n- Average Ensemble\n- Stratified 4-fold\n- BCE loss with logits\n\nTo do: \n- Mixup\n- Label Smoothing\n\nThings that work:\n1. EfficientNet-B0 [Good boost in performance]\n2. Less Aggressive augmentations \n3. Original Image size\n4. B4 Improves the performance \n\nThings that didn't work:\n1. Apex - Slows the code. Reduces Memory Usage\n2. Nvidia Apex - Finding it tough to fix the gradient overflow error. \n3. Focal Loss - tried multiple gamma values \n4. Hard negative sampling\n5. Weighted Sampler \n6. Oversampling\n\nNotes:\nI chose value/settings based on \n1. All fold showed noticeable improvement w.r.t loss\n2. Loss difference between all folds is minimal\n3.Gave importance to loss over AUC for choosing the best models \n\nSpl thanks to \nhttps://www.kaggle.com/yasufuminakama/seti-nfnet-l0-starter-training kernel \n\nHope this helps :-)",
    "1320502": "Focal Loss didn't work in many cases seemingly.",
    "1315032": "Things that work:\n  1. Mixup augmentation\n  2. Change Input from Channel to Spatial",
    "1311073": "@micheomaano Can You share with us How did you convert Normal Images Into spatial Ones",
    "1310905": "Hard sample mining then oversampling works bad for me too. I'm wondering is that hard or just noise, cause the competition host seems to put some \"Easter Eggs\" in the dataset.",
    "1310574": "Hi @micheomaano ,\nI was too late to read this post... Just spent a whole evening on ViT and trying to figure out why was it performing the way it was 😓😓\nBut also, Thanks for saving my time because I was going to do focal loss next 😅",
    "1312081": "Balancing the training classes - by oversampling the 1's - made my results a lot worse.",
    "1310625": "bigger models worked for me",
    "1320609": "I'm experimenting with a bigger model, open source code with GRU only once, lb96 +. Pseudo Labels do make LB worse",
    "1320547": "Has anyone tried weighted BCEWithLogitsLoss wrt to classes?",
    "1316725": "hflip vflip blur didnt work",
    "1313324": "What is your implementation of weighted loss?"
  }
}