{
  "id": 46982,
  "title": "Let's help the beginners : LB=0.82 cnn_trad_pool2_net solution",
  "url": "/competitions/tensorflow-speech-recognition-challenge/discussion/46982",
  "author_name": "hengck23",
  "post_date": "2018-01-06T08:50:09.944000",
  "votes": 52,
  "comment_count": 74,
  "views": 0,
  "content": "<p>Please refer to the attachment PPTX for details.</p>",
  "messages": [
    {
      "id": 265667,
      "postDate": "2018-01-06T08:50:09.943Z",
      "content": "<p>Please refer to the attachment PPTX for details.</p>",
      "rawMarkdown": "Please refer to the attachment PPTX for details.",
      "votes": 52
    },
    {
      "id": 269128,
      "postDate": "2018-01-16T09:00:15.600Z",
      "content": "<p>@Shitian Ni</p>\n\n<p>from your notebook torch.ipynb, i convert to main.py below. it can run fine on my system. I attached the log file, which shows the results:</p>\n\n<pre><code>** start training here! **\noptimizer=&lt;torch.optim.sgd.SGD object at 0x7fa2cf688518&gt;\nmomentum=0.900000\nLR=None\n\nwaves_per_epoch = 51088\n\n  rate   iter_k   epoch  num_m| valid_loss/acc | train_loss/acc | batch_loss/acc |  time   \n  --------------------------------------------------------------------------------------------\n  0.0000    0.0 k  0.00   0.0 | 2.5337  0.0826 | 0.0000  0.0000 | 0.0000  0.0000 |  0 hr 00 min \n  0.0050    0.5 k  1.25   0.1 | 0.6781  0.7852 | 1.0782  0.6578 | 0.7809  0.7344 |  0 hr 02 min \n  0.0050    1.0 k  2.51   0.1 | 0.6302  0.8107 | 0.9056  0.7074 | 0.7444  0.7344 |  0 hr 03 min \n</code></pre>\n\n<p>you may want to check your your system, e.g. run pytorch mnist example, etc.</p>\n\n<p>My environment is:</p>\n\n<pre><code>set cuda environment\n    torch.__version__              = 0.3.0.post4\n    torch.version.cuda             = 9.0.176\n    torch.backends.cudnn.version() = 7003\n    os['CUDA_VISIBLE_DEVICES']  = 0\n    torch.cuda.device_count()   = 1\n    torch.cuda.current_device() = 0\n</code></pre>\n\n<p>but i don't think cuda8/9 will make a difference.</p>",
      "rawMarkdown": "@Shitian Ni\n\nfrom your notebook torch.ipynb, i convert to main.py below. it can run fine on my system. I attached the log file, which shows the results:\n\n    ** start training here! **\n    optimizer=",
      "votes": 1
    },
    {
      "id": 266602,
      "postDate": "2018-01-09T06:12:32.083Z",
      "content": "<p>Thank you for sharing. I am new to pytorch, and have several issues running the code.</p>\n\n<p>I got </p>\n\n<pre><code>RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1512378360668/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\n</code></pre>\n\n<p>when using Cnn_Trad_Pool2_Net. I want to know how 26624 in the final Linear layer is calculated.\nI tried using other networks, but the accuracies are not improving, \nlogs look like</p>\n\n<pre><code>0.0050  174.0 k  436.00  22.3 | 2.9176  0.1000 | 2.4834  0.0930 | 2.4858  0.0938 | 12 hr 48 min  174121,174121, torch.Size([128, 1, 40, 101])\n</code></pre>\n\n<p>for running vggnet.\nWhat can be possibly wrong?\nThank you very much.</p>",
      "rawMarkdown": "Thank you for sharing. I am new to pytorch, and have several issues running the code.\n\nI got \n\n    RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1512378360668/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\nwhen using Cnn_Trad_Pool2_Net. I want to know how 26624 in the final Linear layer is calculated.\nI tried using other networks, but the accuracies are not improving, \nlogs look like\n\n    0.0050  174.0 k  436.00  22.3 | 2.9176  0.1000 | 2.4834  0.0930 | 2.4858  0.0938 | 12 hr 48 min  174121,174121, torch.Size([128, 1, 40, 101])\n\nfor running vggnet.\nWhat can be possibly wrong?\nThank you very much.",
      "votes": 1,
      "replies": [
        {
          "id": 266605,
          "postDate": "2018-01-09T06:20:10.940Z",
          "content": "<blockquote>\n  <blockquote>\n    <p>I want to know how 26624 in the final Linear layer is calculated. \n    you can do a print  in forward(), see #print(x.size()) below. From there you can compute the size.</p>\n  </blockquote>\n</blockquote>\n\n<pre><code>class Cnn_Trad_Pool2_Net(nn.Module):\ndef __init__(self, in_shape=(1,40,101), num_classes=12 ):\n    super(Cnn_Trad_Pool2_Net, self).__init__()\n    self.num_classes = num_classes\n\n    self.conv1 = nn.Conv2d(1,  64, kernel_size=(20, 8), stride=(1, 1))\n    self.conv2 = nn.Conv2d(64, 64, kernel_size=(10, 4), stride=(1, 1))\n    self.fc = nn.Linear(26624,num_classes)\n\n\ndef forward(self, x):\n\n    x = self.conv1(x)\n    x = F.relu(x,inplace=True)\n    x = F.max_pool2d(x,kernel_size=(2,2),stride=(2,2))\n\n    x = self.conv2(x)\n    x = F.relu(x,inplace=True)\n    x = x.view(x.size(0), -1)\n\n    #print(x.size())\n    x = F.dropout(x,p=0.5,training=self.training)\n    x = self.fc(x)\n\n    return x  #logits\n</code></pre>",
          "rawMarkdown": "&gt;&gt;I want to know how 26624 in the final Linear layer is calculated. \nyou can do a print  in forward(), see #print(x.size()) below. From there you can compute the size.\n\n    class Cnn_Trad_Pool2_Net(nn.Module):\n    def __init__(self, in_shape=(1,40,101), num_classes=12 ):\n        super(Cnn_Trad_Pool2_Net, self).__init__()\n        self.num_classes = num_classes\n\n        self.conv1 = nn.Conv2d(1,  64, kernel_size=(20, 8), stride=(1, 1))\n        self.conv2 = nn.Conv2d(64, 64, kernel_size=(10, 4), stride=(1, 1))\n        self.fc = nn.Linear(26624,num_classes)\n\n\n    def forward(self, x):\n\n        x = self.conv1(x)\n        x = F.relu(x,inplace=True)\n        x = F.max_pool2d(x,kernel_size=(2,2),stride=(2,2))\n\n        x = self.conv2(x)\n        x = F.relu(x,inplace=True)\n        x = x.view(x.size(0), -1)\n        \n        #print(x.size())\n        x = F.dropout(x,p=0.5,training=self.training)\n        x = self.fc(x)\n\n        return x  #logits\n",
          "votes": 3
        },
        {
          "id": 267151,
          "postDate": "2018-01-10T17:30:54.263Z",
          "content": "<p>Hi Heng CherKeng,\nThanks for sharing, I have the same error. Can you please help me about this?\nRuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1503963423183/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243</p>\n\n<p>Print(x.size()) give me torch.Size([128, 2816]).\nDo you think I should change the some parameters in CNN?</p>",
          "rawMarkdown": "Hi Heng CherKeng,\nThanks for sharing, I have the same error. Can you please help me about this?\nRuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1503963423183/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\n\nPrint(x.size()) give me torch.Size([128, 2816]).\nDo you think I should change the some parameters in CNN?\n"
        },
        {
          "id": 267162,
          "postDate": "2018-01-10T17:59:02.007Z",
          "content": "<p>I solved the issue by copying Linear class and corresponding codes from pytorch repository to the local file, rename them and replace Linear class in the original code</p>",
          "rawMarkdown": "I solved the issue by copying Linear class and corresponding codes from pytorch repository to the local file, rename them and replace Linear class in the original code",
          "votes": 1
        },
        {
          "id": 267184,
          "postDate": "2018-01-10T19:33:04.960Z",
          "content": "<p>Hi Shitian\nThank you for response\nDo you mean you sync linear.py from pytorch repository with your local installed one?\nI also read other related posts. It looks like the error related to python version too.</p>",
          "rawMarkdown": "Hi Shitian\nThank you for response\nDo you mean you sync linear.py from pytorch repository with your local installed one?\nI also read other related posts. It looks like the error related to python version too.\n\n \n"
        },
        {
          "id": 267345,
          "postDate": "2018-01-11T04:31:28.303Z",
          "content": "<p>I don't know what you mean by sync. I mean copy code from pytorch github repository to your local file, replace linear class you import from your code.  Then the error disappears. My environment was Nvidia GPU Cloud pytorch docker. But after that I also got some other errors like expect Variable(CPU_LONG) got Variable(GPU_LONG).</p>",
          "rawMarkdown": "I don't know what you mean by sync. I mean copy code from pytorch github repository to your local file, replace linear class you import from your code.  Then the error disappears. My environment was Nvidia GPU Cloud pytorch docker. But after that I also got some other errors like expect Variable(CPU_LONG) got Variable(GPU_LONG)."
        },
        {
          "id": 267346,
          "postDate": "2018-01-11T04:50:37.103Z",
          "content": "<p>Thank you. Let us keep trying.</p>",
          "rawMarkdown": "Thank you. Let us keep trying."
        },
        {
          "id": 267935,
          "postDate": "2018-01-12T20:02:58.723Z",
          "content": "<p>Any news on fixing the errors? I also encountered the same issue.</p>\n\n<p>However, my error is slightly different: </p>\n\n<p><code>\nRuntimeError: sizes do not match at /opt/conda/conda-bld/pytorch_1501969512886/work/pytorch-0.1.12/torch/lib/THC/generated/../generic/THCTensorMathPointwise.cu:296\n</code></p>",
          "rawMarkdown": "Any news on fixing the errors? I also encountered the same issue.\n\nHowever, my error is slightly different: \n\n```\nRuntimeError: sizes do not match at /opt/conda/conda-bld/pytorch_1501969512886/work/pytorch-0.1.12/torch/lib/THC/generated/../generic/THCTensorMathPointwise.cu:296\n```"
        },
        {
          "id": 267944,
          "postDate": "2018-01-12T20:27:19.040Z",
          "content": "<p>No. I changed 26624  to some other number. It didn't work</p>",
          "rawMarkdown": "No. I changed 26624  to some other number. It didn't work"
        },
        {
          "id": 268035,
          "postDate": "2018-01-13T03:50:07.913Z",
          "content": "<p>I solved my issue. I just update the pytorch from 0.1.2 to 0.3.0. Hope this could also apply to yours.</p>",
          "rawMarkdown": "I solved my issue. I just update the pytorch from 0.1.2 to 0.3.0. Hope this could also apply to yours."
        },
        {
          "id": 268307,
          "postDate": "2018-01-14T04:14:59Z",
          "content": "<p>RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1512387374934/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\nHow solve this error?\nMy Pytorch vesion is \"0.3.0.post4\"</p>",
          "rawMarkdown": "RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1512387374934/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\nHow solve this error?\nMy Pytorch vesion is \"0.3.0.post4\""
        },
        {
          "id": 268358,
          "postDate": "2018-01-14T09:09:09.320Z",
          "content": "<p>I solved the error after setting <code>self.fc = nn.Linear(2816,self.num_classes)</code></p>\n\n<p>View Examples in\n<a href=\"http://pytorch.org/docs/master/nn.html#linear-layers\">Linear layers docs</a></p>",
          "rawMarkdown": "I solved the error after setting `self.fc = nn.Linear(2816,self.num_classes)`\n\nView Examples in\n[Linear layers docs][1]\n\n\n  [1]: http://pytorch.org/docs/master/nn.html#linear-layers"
        },
        {
          "id": 268360,
          "postDate": "2018-01-14T09:13:39.367Z",
          "content": "<p>use this instead. kenel size is transposed, e.g kernel_size=(8, 20)</p>\n\n<p>class Cnn_Trad_Pool2_Net(nn.Module):</p>\n\n<pre><code>def __init__(self, in_shape=(1,40,101), num_classes=12 ):\n\n    super(Cnn_Trad_Pool2_Net, self).__init__()\n    self.num_classes = num_classes\n\n    self.conv1 = nn.Conv2d(1,  64, kernel_size=(8, 20), stride=(1, 1))\n    self.conv2 = nn.Conv2d(64, 64, kernel_size=(4, 10), stride=(1, 1))\n    self.fc = nn.Linear(26624,num_classes)\n\n\ndef forward(self, x):\n\n    x = self.conv1(x)\n    x = F.relu(x,inplace=True)\n    x = F.max_pool2d(x,kernel_size=(2,2),stride=(2,2))\n\n    x = self.conv2(x)\n    x = F.relu(x,inplace=True)\n    x = x.view(x.size(0), -1)\n\n    #print(x.size())\n    x = F.dropout(x,p=0.5,training=self.training)\n    x = self.fc(x)\n\n    return x  #logits\n</code></pre>",
          "rawMarkdown": "use this instead. kenel size is transposed, e.g kernel_size=(8, 20)\n\n    \n\n\n\n   class Cnn_Trad_Pool2_Net(nn.Module):\n\n    def __init__(self, in_shape=(1,40,101), num_classes=12 ):\n\n        super(Cnn_Trad_Pool2_Net, self).__init__()\n        self.num_classes = num_classes\n\n        self.conv1 = nn.Conv2d(1,  64, kernel_size=(8, 20), stride=(1, 1))\n        self.conv2 = nn.Conv2d(64, 64, kernel_size=(4, 10), stride=(1, 1))\n        self.fc = nn.Linear(26624,num_classes)\n\n\n    def forward(self, x):\n\n        x = self.conv1(x)\n        x = F.relu(x,inplace=True)\n        x = F.max_pool2d(x,kernel_size=(2,2),stride=(2,2))\n\n        x = self.conv2(x)\n        x = F.relu(x,inplace=True)\n        x = x.view(x.size(0), -1)\n\n        #print(x.size())\n        x = F.dropout(x,p=0.5,training=self.training)\n        x = self.fc(x)\n\n        return x  #logits",
          "votes": 1
        },
        {
          "id": 268465,
          "postDate": "2018-01-14T14:51:46.817Z",
          "content": "<p>Thank you.\nThe error is gone, but my accuracies still at 0.09 after number of epochs.</p>\n\n<p>If I run <code>net(tensors)</code>,</p>\n\n<p>I get</p>\n\n<pre><code> Variable containing:\n\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n          ...             ⋱             ...          \n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n[torch.cuda.FloatTensor of size 128x12 (GPU 0)]\n</code></pre>\n\n<p>Outputs from the network are the same for all tensors from</p>\n\n<pre><code>for tensors, labels, indices in train_loader:\n    tensors = Variable(tensors).cuda()\n</code></pre>\n\n<p>But tensors are different.</p>\n\n<p>Any suggestions?</p>",
          "rawMarkdown": "Thank you.\nThe error is gone, but my accuracies still at 0.09 after number of epochs.\n\nIf I run `net(tensors)`,\n\nI get\n\n     Variable containing:\n\n     0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n     0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n     0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n              ...             ⋱             ...          \n     0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n     0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n     0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n    [torch.cuda.FloatTensor of size 128x12 (GPU 0)]\nOutputs from the network are the same for all tensors from\n\n    for tensors, labels, indices in train_loader:\n        tensors = Variable(tensors).cuda()\n\nBut tensors are different.\n\nAny suggestions?"
        },
        {
          "id": 268653,
          "postDate": "2018-01-15T06:16:18.620Z",
          "content": "<p>Thank you. have fix this!</p>\n\n<p>Awsome work, you are my hero~~</p>",
          "rawMarkdown": "Thank you. have fix this!\n\nAwsome work, you are my hero~~"
        },
        {
          "id": 268889,
          "postDate": "2018-01-15T22:20:07.477Z",
          "content": "<p>@Shitian Ni</p>\n\n<p>can you post your all files that produce \"0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\" . I try to debug</p>",
          "rawMarkdown": "@Shitian Ni\n\ncan you post your all files that produce \"0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\" . I try to debug",
          "votes": 1
        },
        {
          "id": 269117,
          "postDate": "2018-01-16T08:21:41.737Z",
          "content": "<p>Thank you. I think that is something with .cuda() and .cpu(). Now that is solved, but my accuracy still doesn't improve.\nCould you review my file is possible?\nI put the files on my Github account\n<a href=\"https://github.com/shitian-ni/torch-speech\">https://github.com/shitian-ni/torch-speech</a>\nMain file is torch.ipynb, you can see that the accuracy is not improving.\nMy torch version is '0.3.0.post4'\nThank you very much</p>",
          "rawMarkdown": "Thank you. I think that is something with .cuda() and .cpu(). Now that is solved, but my accuracy still doesn't improve.\nCould you review my file is possible?\nI put the files on my Github account\nhttps://github.com/shitian-ni/torch-speech\nMain file is torch.ipynb, you can see that the accuracy is not improving.\nMy torch version is '0.3.0.post4'\nThank you very much\n"
        }
      ]
    },
    {
      "id": 266469,
      "postDate": "2018-01-08T21:21:17.013Z",
      "content": "<p>Hello</p>\n\n<p>I am trying to set up a Jupyter notebook from your PyCharm files ; yet work to be done , got head on collision with many errors .  any suggestions ? </p>",
      "rawMarkdown": "Hello\n\nI am trying to set up a Jupyter notebook from your PyCharm files ; yet work to be done , got head on collision with many errors .  any suggestions ? ",
      "votes": 1
    },
    {
      "id": 266353,
      "postDate": "2018-01-08T15:31:58.037Z",
      "content": "<p>@Heng CherKeng</p>\n\n<p>Can you explain how (-1) is possible as index in  Dataset? I guess it's because custom collate function ?</p>\n\n<p>So Im trying to implement dataloader in pytorch, that on every epoch returns only subset of \"unknowns\". But I can't understand how it's done in you code. </p>",
      "rawMarkdown": "@Heng CherKeng\n\nCan you explain how (-1) is possible as index in  Dataset? I guess it's because custom collate function ?\n\nSo Im trying to implement dataloader in pytorch, that on every epoch returns only subset of \"unknowns\". But I can't understand how it's done in you code. ",
      "votes": 1,
      "replies": [
        {
          "id": 266354,
          "postDate": "2018-01-08T15:33:52.887Z",
          "content": "<p>yes. there is a custom collate function. see audio_dataset.py : def collate(batch):</p>",
          "rawMarkdown": "yes. there is a custom collate function. see audio_dataset.py : def collate(batch):",
          "votes": 2
        },
        {
          "id": 266355,
          "postDate": "2018-01-08T15:34:58.270Z",
          "content": "<p>you should see: audio_processing_tf.py:</p>\n\n<pre><code>class TFRandomSampler(Sampler):\n\n    def __init__(self, data, silence_probability=0.1, unknown_probability=0.1):\n           ...\n\n   def __iter__(self):\n          data = self.data\n          ....\n\n           if self.unknown_num&gt;0:#unknown\n               unknown_list = data.index_by_class[1]*math.ceil(self.unknown_num/len(data.index_by_class[1]))\n               random.shuffle(unknown_list)\n               unknown_list = unknown_list[:self.unknown_num]\n               l +=  unknown_list\n</code></pre>",
          "rawMarkdown": "you should see: audio_processing_tf.py:\n\n\n    class TFRandomSampler(Sampler):\n\n        def __init__(self, data, silence_probability=0.1, unknown_probability=0.1):\n               ...\n\n       def __iter__(self):\n              data = self.data\n              ....\n\n               if self.unknown_num&gt;0:#unknown\n                   unknown_list = data.index_by_class[1]*math.ceil(self.unknown_num/len(data.index_by_class[1]))\n                   random.shuffle(unknown_list)\n                   unknown_list = unknown_list[:self.unknown_num]\n                   l +=  unknown_list\n\n",
          "votes": 3
        }
      ]
    },
    {
      "id": 266122,
      "postDate": "2018-01-07T22:33:13.473Z",
      "content": "<p>Almost everything is working with provided files, however, just tiny little functions are missing. It should be trivial to re-implement but for the sake of completeness, Heng CherKeng could your provide <code>top_accuracy(), adjust_learning_rate(), and get_learning_rate()</code>, the only missing functions? :)</p>",
      "rawMarkdown": "Almost everything is working with provided files, however, just tiny little functions are missing. It should be trivial to re-implement but for the sake of completeness, Heng CherKeng could your provide `top_accuracy(), adjust_learning_rate(), and get_learning_rate()`, the only missing functions? :)",
      "votes": 1
    },
    {
      "id": 266068,
      "postDate": "2018-01-07T18:07:50.233Z",
      "content": "<p>Hi CherKeng,\nI tried to run the code but got this error message even I installed the module 'utility'. Could you or anyone help? Thanks.</p>\n\n<p>Traceback (most recent call last):</p>\n\n<p>File \"~/PyTorch/train_cnn_trad_pool2_net.py\", line 7, in </p>\n\n<pre><code>from utility.file import *\n</code></pre>\n\n<p>ModuleNotFoundError: No module named 'utility'</p>",
      "rawMarkdown": "Hi CherKeng,\nI tried to run the code but got this error message even I installed the module 'utility'. Could you or anyone help? Thanks.\n\nTraceback (most recent call last):\n\n  File \"~/PyTorch/train_cnn_trad_pool2_net.py\", line 7, in ",
      "votes": 1,
      "replies": [
        {
          "id": 266145,
          "postDate": "2018-01-07T23:36:44.813Z",
          "content": "<p>You don't need to install any <code>utility</code> module. Just create folder <code>utility</code> and place <code>file.py</code> there</p>",
          "rawMarkdown": "You don't need to install any `utility` module. Just create folder `utility` and place `file.py` there",
          "votes": 1
        },
        {
          "id": 266154,
          "postDate": "2018-01-08T00:21:30.513Z",
          "content": "<p>Thank you, Sir. </p>",
          "rawMarkdown": "Thank you, Sir. ",
          "votes": 1
        },
        {
          "id": 266788,
          "postDate": "2018-01-09T18:38:10.800Z",
          "content": "<p>Thanks for the suggestion.</p>",
          "rawMarkdown": "Thanks for the suggestion."
        }
      ]
    },
    {
      "id": 265702,
      "postDate": "2018-01-06T10:44:38.617Z",
      "content": "<p>Hi Heng, glad you are posting in this competition!</p>\n\n<p>\"Silence.wav\" is it zero wav file? </p>",
      "rawMarkdown": " Hi Heng, glad you are posting in this competition!\n\n\"Silence.wav\" is it zero wav file? ",
      "votes": 1,
      "replies": [
        {
          "id": 265704,
          "postDate": "2018-01-06T10:47:59.217Z",
          "content": "<p>yes. i could have just use np.zeros(), but i choose to keep it as a wav file to simplify my code for other purposes</p>",
          "rawMarkdown": "yes. i could have just use np.zeros(), but i choose to keep it as a wav file to simplify my code for other purposes",
          "votes": 2
        },
        {
          "id": 265926,
          "postDate": "2018-01-07T05:36:00.283Z",
          "content": "<p>Thanks CherKeng for your great sharing, I have question here. What is different between Silence.wav and empty.wav.  I created the empty.wav with your code. Do I need similar method to create Silence.wav?  </p>",
          "rawMarkdown": "Thanks CherKeng for your great sharing, I have question here. What is different between Silence.wav and empty.wav.  I created the empty.wav with your code. Do I need similar method to create Silence.wav?  "
        },
        {
          "id": 269786,
          "postDate": "2018-01-17T09:57:29.603Z",
          "content": "<p>This zero wav file improved my LB from 0.84 to 0.88, really amazing! I find it reasonable after I listen to test data which contains some mute data</p>",
          "rawMarkdown": "This zero wav file improved my LB from 0.84 to 0.88, really amazing! I find it reasonable after I listen to test data which contains some mute data"
        }
      ]
    },
    {
      "id": 266150,
      "postDate": "2018-01-08T00:02:21.510Z",
      "content": "<p>@Sergey Mushinskiy</p>\n\n<p>The files as requested.</p>",
      "rawMarkdown": "@Sergey Mushinskiy\n\nThe files as requested.",
      "votes": 2,
      "replies": [
        {
          "id": 266152,
          "postDate": "2018-01-08T00:18:38.813Z",
          "content": "<p>Thanks! Hope it attracts even more guys to compete :)</p>",
          "rawMarkdown": "Thanks! Hope it attracts even more guys to compete :)",
          "votes": 1
        },
        {
          "id": 266155,
          "postDate": "2018-01-08T00:23:27.970Z",
          "content": "<p>Hi, CherKeng, do you mind drawing a data/file structure? It would be great to people who is a python newbie. Thanks again for the sharing.  </p>",
          "rawMarkdown": "Hi, CherKeng, do you mind drawing a data/file structure? It would be great to people who is a python newbie. Thanks again for the sharing.  ",
          "votes": 1
        },
        {
          "id": 266366,
          "postDate": "2018-01-08T16:04:24.360Z",
          "content": "<p>@Terracotta</p>\n\n<p>Please refer to this post: <a href=\"https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/46988\">https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/46988</a></p>\n\n<p>look for the below message:</p>\n\n<p>\"please refer to ppt for details.</p>\n\n<p>here, the files contain: - full pycharm project, include train, evaluate, submit code - trained model at LB=0.86 - data split</p>\n\n<p>build.tar.gz (82.37 KB)</p>\n\n<p>data.tar.gz (2.27 MB)</p>\n\n<p>results.tar.gz (4.45 MB)</p>\n\n<p>readme.pptx (524.8 KB)\"</p>",
          "rawMarkdown": "@Terracotta\n\n\nPlease refer to this post: https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/46988\n\nlook for the below message:\n\n\"please refer to ppt for details.\n\nhere, the files contain: - full pycharm project, include train, evaluate, submit code - trained model at LB=0.86 - data split\n\nbuild.tar.gz (82.37 KB)\n\ndata.tar.gz (2.27 MB)\n\nresults.tar.gz (4.45 MB)\n\nreadme.pptx (524.8 KB)\"",
          "votes": 1
        },
        {
          "id": 266386,
          "postDate": "2018-01-08T17:15:25.433Z",
          "content": "<p>Many thanks. Great discussion posts!</p>",
          "rawMarkdown": "Many thanks. Great discussion posts!"
        },
        {
          "id": 266674,
          "postDate": "2018-01-09T11:57:58.593Z",
          "content": "<p>Hi @Heng. I have a question regarding the implementation of the samplers such as <code>TFRandomSampler(Sampler)</code> and <code>TFSequentialSampler(Sampler)</code>. In the implementation of those classes you inherit/extend the Sampler class. But nowhere in your files I could find the <code>Sampler</code> class or the <code>RandomSampler(dataset)</code> for that matter which you call at line 252 in file audio_processing_tf.py. The only thing I could find is a sampler object which is initialized by the <code>TFSequentialSampler</code>. Would you mind elaborating a little bit on this? Is there a possibility that you meant to call <code>TFRandomSampler</code> instead of <code>RandomSampler</code> in file <code>audio_processing_tf.py</code></p>\n\n<p>I've tried multiple times to run the provided files to get the dataset split correctly and this is what I get:</p>\n\n<pre><code>audio_processing_tf.py: calling main function ...\ntime = 0.00 min\nnum_ids = 6798\nnum_classes = 12\n0            silence =     0 (0.000)\n1            unknown =  4221 (0.621)\n2                yes =   261 (0.038)\n3                 no =   270 (0.040)\n4                 up =   260 (0.038)\n5               down =   264 (0.039)\n6               left =   247 (0.036)\n7              right =   256 (0.038)\n8                 on =   257 (0.038)\n9                off =   256 (0.038)\n10               stop =   246 (0.036)\n11                 go =   260 (0.038)\n-----------------------------------\n0 stop\n9 off\n4 up\n3 no\n6 left\n4 up\n6 left\n11 go\n8 on\n6 left\n11 go\n7 right\n7 right\n2 yes\nlabels: 3221, waves: 3221\nsucess!\n</code></pre>\n\n<p>The output is from running <code>audio_processing_tf.py</code> and replacing RandomSampler with TFRandomSampler. The file is <code>train_valid_6798</code>, the question here which I cannot answer is why do I get 3221 instead of 6798 processed files?</p>",
          "rawMarkdown": "Hi @Heng. I have a question regarding the implementation of the samplers such as `TFRandomSampler(Sampler)` and `TFSequentialSampler(Sampler)`. In the implementation of those classes you inherit/extend the Sampler class. But nowhere in your files I could find the `Sampler` class or the `RandomSampler(dataset)` for that matter which you call at line 252 in file audio_processing_tf.py. The only thing I could find is a sampler object which is initialized by the `TFSequentialSampler`. Would you mind elaborating a little bit on this? Is there a possibility that you meant to call `TFRandomSampler` instead of `RandomSampler` in file `audio_processing_tf.py`\n\nI've tried multiple times to run the provided files to get the dataset split correctly and this is what I get:\n\n    audio_processing_tf.py: calling main function ...\n    time = 0.00 min\n    num_ids = 6798\n    num_classes = 12\n    0            silence =     0 (0.000)\n    1            unknown =  4221 (0.621)\n    2                yes =   261 (0.038)\n    3                 no =   270 (0.040)\n    4                 up =   260 (0.038)\n    5               down =   264 (0.039)\n    6               left =   247 (0.036)\n    7              right =   256 (0.038)\n    8                 on =   257 (0.038)\n    9                off =   256 (0.038)\n    10               stop =   246 (0.036)\n    11                 go =   260 (0.038)\n    -----------------------------------\n    0 stop\n    9 off\n    4 up\n    3 no\n    6 left\n    4 up\n    6 left\n    11 go\n    8 on\n    6 left\n    11 go\n    7 right\n    7 right\n    2 yes\n    labels: 3221, waves: 3221\n    sucess!\n\nThe output is from running `audio_processing_tf.py` and replacing RandomSampler with TFRandomSampler. The file is `train_valid_6798`, the question here which I cannot answer is why do I get 3221 instead of 6798 processed files?"
        },
        {
          "id": 266785,
          "postDate": "2018-01-09T18:23:50.710Z",
          "content": "<p>Sampler is pytorch class. 3221 is random subsample version of 6798 , to ensure that prob_silence=0.1 and prob_unknown=0.1</p>",
          "rawMarkdown": "Sampler is pytorch class. 3221 is random subsample version of 6798 , to ensure that prob_silence=0.1 and prob_unknown=0.1"
        },
        {
          "id": 266787,
          "postDate": "2018-01-09T18:35:32.503Z",
          "content": "<p>Nice thanks for the clarifications. You see what I was doing wrong was the fact that I was trying to process all the data and save them, then train the model but in your case you had the iterator to feed the data into the model. Now I know that I'll have to run the sampler a couple of times if I want to process the data once and save them. Cheers!</p>",
          "rawMarkdown": "Nice thanks for the clarifications. You see what I was doing wrong was the fact that I was trying to process all the data and save them, then train the model but in your case you had the iterator to feed the data into the model. Now I know that I'll have to run the sampler a couple of times if I want to process the data once and save them. Cheers!"
        },
        {
          "id": 266966,
          "postDate": "2018-01-10T08:18:14.827Z",
          "content": "<p>Hi @Heng, sorry bothering you but I need to learn and the only way is by asking. If you get annoyed by my question please just let me know and I'll stop asking ;). I've used your technique to create the train,valid, and test generators and I've tested them on a model which previously was giving ~0.73% on LB. After using your provided technique for generating the data and the same model it seems that the mode is unable to learn anything at all. The loss oscillates between particular values but never improves. Below is an output from the training procedure:</p>\n\n<pre><code>512/512 [==============================] - 96s - loss: 2.0777 - acc: 0.2480 - val_loss: 2.0285 - \nval_acc: 0.2988--\nEpoch 26/50\n512/512 [==============================] - 95s - loss: 2.0339 - acc: 0.2852 - val_loss: 2.0035 - \nval_acc: 0.3105--\nEpoch 27/50\n512/512 [==============================] - 102s - loss: 2.0812 - acc: 0.2871 - val_loss: 2.0661 - \nval_acc: 0.2539-\nEpoch 28/50\n512/512 [==============================] - 105s - loss: 2.0969 - acc: 0.2383 - val_loss: 1.9392 - \nval_acc: 0.3086-\nEpoch 29/50\n512/512 [==============================] - 105s - loss: 2.0443 - acc: 0.2773 - val_loss: 2.1398 - \nval_acc: 0.2559-\nEpoch 30/50\n512/512 [==============================] - 103s - loss: 1.9928 - acc: 0.2715 - val_loss: 2.0004 - \nval_acc: 0.3262-\nEpoch 31/50\n512/512 [==============================] - 105s - loss: 2.1038 - acc: 0.2363 - val_loss: 2.0440 - \nval_acc: 0.3086-\nEpoch 32/50\n512/512 [==============================] - 103s - loss: 2.1193 - acc: 0.2520 - val_loss: 1.9881 - \nval_acc: 0.2988-\nEpoch 33/50\n512/512 [==============================] - 103s - loss: 2.0528 - acc: 0.2695 - val_loss: 1.9842 - \nval_acc: 0.2383-\nEpoch 34/50\n512/512 [==============================] - 101s - loss: 2.0756 - acc: 0.2559 - val_loss: 2.0211 - \nval_acc: 0.2969-\nEpoch 35/50\n512/512 [==============================] - 103s - loss: 2.0462 - acc: 0.2773 - val_loss: 1.9498 - \nval_acc: 0.3086-\nEpoch 36/50\n512/512 [==============================] - 99s - loss: 2.0386 - acc: 0.3066 - val_loss: 2.0807 - \n</code></pre>\n\n<p>val_acc: 0.2598--</p>\n\n<p>One thing that I did was to add random padding as augmentation, do think that might be affecting the quality of the data?</p>",
          "rawMarkdown": "Hi @Heng, sorry bothering you but I need to learn and the only way is by asking. If you get annoyed by my question please just let me know and I'll stop asking ;). I've used your technique to create the train,valid, and test generators and I've tested them on a model which previously was giving ~0.73% on LB. After using your provided technique for generating the data and the same model it seems that the mode is unable to learn anything at all. The loss oscillates between particular values but never improves. Below is an output from the training procedure:\n\n    512/512 [==============================] - 96s - loss: 2.0777 - acc: 0.2480 - val_loss: 2.0285 - \n    val_acc: 0.2988--\n    Epoch 26/50\n    512/512 [==============================] - 95s - loss: 2.0339 - acc: 0.2852 - val_loss: 2.0035 - \n    val_acc: 0.3105--\n    Epoch 27/50\n    512/512 [==============================] - 102s - loss: 2.0812 - acc: 0.2871 - val_loss: 2.0661 - \n    val_acc: 0.2539-\n    Epoch 28/50\n    512/512 [==============================] - 105s - loss: 2.0969 - acc: 0.2383 - val_loss: 1.9392 - \n    val_acc: 0.3086-\n    Epoch 29/50\n    512/512 [==============================] - 105s - loss: 2.0443 - acc: 0.2773 - val_loss: 2.1398 - \n    val_acc: 0.2559-\n    Epoch 30/50\n    512/512 [==============================] - 103s - loss: 1.9928 - acc: 0.2715 - val_loss: 2.0004 - \n    val_acc: 0.3262-\n    Epoch 31/50\n    512/512 [==============================] - 105s - loss: 2.1038 - acc: 0.2363 - val_loss: 2.0440 - \n    val_acc: 0.3086-\n    Epoch 32/50\n    512/512 [==============================] - 103s - loss: 2.1193 - acc: 0.2520 - val_loss: 1.9881 - \n    val_acc: 0.2988-\n    Epoch 33/50\n    512/512 [==============================] - 103s - loss: 2.0528 - acc: 0.2695 - val_loss: 1.9842 - \n    val_acc: 0.2383-\n    Epoch 34/50\n    512/512 [==============================] - 101s - loss: 2.0756 - acc: 0.2559 - val_loss: 2.0211 - \n    val_acc: 0.2969-\n    Epoch 35/50\n    512/512 [==============================] - 103s - loss: 2.0462 - acc: 0.2773 - val_loss: 1.9498 - \n    val_acc: 0.3086-\n    Epoch 36/50\n    512/512 [==============================] - 99s - loss: 2.0386 - acc: 0.3066 - val_loss: 2.0807 - \n   val_acc: 0.2598--\n\nOne thing that I did was to add random padding as augmentation, do think that might be affecting the quality of the data?"
        },
        {
          "id": 266968,
          "postDate": "2018-01-10T08:20:23.223Z",
          "content": "<p>there could be some bug in your implementation. I suggest you can install pytorch and run my code</p>",
          "rawMarkdown": "there could be some bug in your implementation. I suggest you can install pytorch and run my code"
        },
        {
          "id": 266983,
          "postDate": "2018-01-10T09:03:58.527Z",
          "content": "<p>I ran your code, but has the same issue </p>",
          "rawMarkdown": "I ran your code, but has the same issue "
        },
        {
          "id": 266987,
          "postDate": "2018-01-10T09:23:53.870Z",
          "content": "<p>At first I had some issues with the code but they were gone after updating to the latest Pytorch.  I hope this could help. The issue I have now is that I only got LB 0.78. Still in a learning process. </p>",
          "rawMarkdown": "At first I had some issues with the code but they were gone after updating to the latest Pytorch.  I hope this could help. The issue I have now is that I only got LB 0.78. Still in a learning process. "
        },
        {
          "id": 266989,
          "postDate": "2018-01-10T09:37:37.843Z",
          "content": "<p>Remember to change your learning rate. You need to run twice. Check my log files and graph in pptx</p>",
          "rawMarkdown": "Remember to change your learning rate. You need to run twice. Check my log files and graph in pptx"
        },
        {
          "id": 267642,
          "postDate": "2018-01-11T23:43:18.047Z",
          "content": "<p>Thanks. It worked.</p>",
          "rawMarkdown": "Thanks. It worked."
        },
        {
          "id": 279288,
          "postDate": "2018-02-07T18:11:20.930Z",
          "content": "<p>@Heng CherKeng i started working on your project but i dont have gpu and i am trying to make the required change in the project to run it on CPU (without GPU).</p>\n\n<p>I am stuck and i dont know how to proceed.</p>\n\n<p>any help will be very much appreciated!</p>\n\n<p>Thanks</p>\n\n<h2>the error shows</h2>\n\n<pre><code>** start evaluation here! **\nTraceback (most recent call last):\nFile \"/home/hrcule/competitionKaggle/speechTensorFlow/fullProject/build/resnet-tf-speech /evaluate.py\", line 127, in &lt;module&gt;\nrun_evaluate()\nFile \"/home/hrcule/competitionKaggle/speechTensorFlow/fullProject/build/resnet-tf-speech /evaluate.py\", line 93, in run_evaluate\nfor i, (tensors, labels, indices) in enumerate(test_loader, 0):\nFile \"/home/hrcule/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py\", line 210, in __next__\nreturn self._process_next_batch(batch)\n</code></pre>",
          "rawMarkdown": "@Heng CherKeng i started working on your project but i dont have gpu and i am trying to make the required change in the project to run it on CPU (without GPU).\n\nI am stuck and i dont know how to proceed.\n\nany help will be very much appreciated!\n\nThanks\n\n##the error shows\n\n    ** start evaluation here! **\n    Traceback (most recent call last):\n    File \"/home/hrcule/competitionKaggle/speechTensorFlow/fullProject/build/resnet-tf-speech /evaluate.py\", line 127, in "
        }
      ]
    },
    {
      "id": 266026,
      "postDate": "2018-01-07T14:24:28.067Z",
      "content": "<p>data split and other files as requested  </p>",
      "rawMarkdown": "data split and other files as requested  ",
      "votes": 2
    },
    {
      "id": 269810,
      "postDate": "2018-01-17T10:47:38.273Z",
      "content": "<p>Thank you Lots of great tips how to setup the code &amp; Congratulations!</p>",
      "rawMarkdown": "Thank you Lots of great tips how to setup the code &amp; Congratulations!"
    },
    {
      "id": 267365,
      "postDate": "2018-01-11T06:34:26.567Z",
      "content": "<p>@Heng CherKeng，</p>\n\n<p>Your code is great. I am new to pytorch. It is a great example. I notice you have great graphs in your report. I wonder whether it is a built-in function in pytorch like tensorboard in tensorflow, or you created your self. Could you please share some information about this? Thanks!</p>",
      "rawMarkdown": "@Heng CherKeng，\n\nYour code is great. I am new to pytorch. It is a great example. I notice you have great graphs in your report. I wonder whether it is a built-in function in pytorch like tensorboard in tensorflow, or you created your self. Could you please share some information about this? Thanks!",
      "replies": [
        {
          "id": 267492,
          "postDate": "2018-01-11T14:36:00.580Z",
          "content": "<p>it is created by hand in spreadsheet software</p>",
          "rawMarkdown": "it is created by hand in spreadsheet software",
          "votes": 1
        }
      ]
    },
    {
      "id": 266677,
      "postDate": "2018-01-09T12:07:59.997Z",
      "content": "<p>Nice</p>",
      "rawMarkdown": "Nice"
    },
    {
      "id": 266046,
      "postDate": "2018-01-07T16:19:32.580Z",
      "content": "<p>I was wondering, did you use GPU to train the model? If yes, how many GPUs did you use?</p>",
      "rawMarkdown": "I was wondering, did you use GPU to train the model? If yes, how many GPUs did you use?",
      "replies": [
        {
          "id": 266047,
          "postDate": "2018-01-07T16:20:55.593Z",
          "content": "<p>the log files are based on single GPU machine (titanX pascal) </p>",
          "rawMarkdown": "the log files are based on single GPU machine (titanX pascal) ",
          "votes": 1
        },
        {
          "id": 266051,
          "postDate": "2018-01-07T16:45:37.863Z",
          "content": "<p>Ok, thanks! That explains the short training duration... I'm struggling to train my models in my laptop with an Nvidia 840M GPU which takes at least 8 hours!</p>",
          "rawMarkdown": "Ok, thanks! That explains the short training duration... I'm struggling to train my models in my laptop with an Nvidia 840M GPU which takes at least 8 hours!"
        }
      ]
    },
    {
      "id": 266022,
      "postDate": "2018-01-07T14:18:23.257Z",
      "content": "<p>Thanks CherKeng for the great sharing.</p>",
      "rawMarkdown": "Thanks CherKeng for the great sharing.",
      "replies": [
        {
          "id": 266027,
          "postDate": "2018-01-07T14:24:57.757Z",
          "content": "<p>see post below for file.py</p>",
          "rawMarkdown": "see post below for file.py"
        }
      ]
    },
    {
      "id": 265928,
      "postDate": "2018-01-07T05:38:32.490Z",
      "content": "<p>@JohnsonTang\nHere is my file.  I think they are the same file</p>",
      "rawMarkdown": "@JohnsonTang\nHere is my file.  I think they are the same file\n"
    },
    {
      "id": 265915,
      "postDate": "2018-01-07T04:55:19.460Z",
      "content": "<p>@kirk . No. There are no special reasons. My whole project files is large and I have just upload the most important files to illustrate the method. You should be able to reproduce the results with my files. </p>\n\n<p>The missing files are less important, e.g. for data read/write, etc . Some of the import (e.g.  dataset.sampler) are not used at all.</p>\n\n<p>Note that the intend of this post is not to encourage kagglers to download scripts, run and make submission. Rather kagglers should take my files as reference and modify or \"debug\" their existing code, method.</p>\n\n<p>If there is anything unclear, please ask again in this post. I added two more files here:</p>\n\n<ol>\n<li>submit.py to make submission</li>\n<li>common.py which is  a list of packages i used (but not all are necessary)</li>\n</ol>",
      "rawMarkdown": "@kirk . No. There are no special reasons. My whole project files is large and I have just upload the most important files to illustrate the method. You should be able to reproduce the results with my files. \n\nThe missing files are less important, e.g. for data read/write, etc . Some of the import (e.g.  dataset.sampler) are not used at all.\n\nNote that the intend of this post is not to encourage kagglers to download scripts, run and make submission. Rather kagglers should take my files as reference and modify or \"debug\" their existing code, method.\n\nIf there is anything unclear, please ask again in this post. I added two more files here:\n\n 1.  submit.py to make submission\n 2. common.py which is  a list of packages i used (but not all are necessary)\n\n",
      "replies": [
        {
          "id": 266017,
          "postDate": "2018-01-07T13:44:58.217Z",
          "content": "<p>Hi Heng. Thanks a lot for your reply. The reason for me asking was because I noticed that parts such as sampler were used for splitting the data (correct me if I am wrong). The main problem that I am trying to understand and solve is how to properly split the dataset. When trying to use pseudo-labeling I had a reference model of 0.73% and used that to predict labels {unknown, silence} on the test set with prob &gt; 0.95, but the model couldn't find any labels{unknown, silence} with prob. &gt; 0.95 which made the whole pseudo-labeling technique unusable for me.</p>",
          "rawMarkdown": "Hi Heng. Thanks a lot for your reply. The reason for me asking was because I noticed that parts such as sampler were used for splitting the data (correct me if I am wrong). The main problem that I am trying to understand and solve is how to properly split the dataset. When trying to use pseudo-labeling I had a reference model of 0.73% and used that to predict labels {unknown, silence} on the test set with prob &gt; 0.95, but the model couldn't find any labels{unknown, silence} with prob. &gt; 0.95 which made the whole pseudo-labeling technique unusable for me."
        },
        {
          "id": 266024,
          "postDate": "2018-01-07T14:23:01.130Z",
          "content": "<p>for this \"LB=0.82 cnn_trad_pool2_net \", pseudo-labeling is not required.</p>\n\n<p>For the split, use the original split provided in kaggle. Anyway, I have attached the files below. </p>\n\n<p>Also, the confidence for \"silence\" LB test samples is usually low. You should try a \"much\" lower threshold. (but again pseudo-labeling is not required)</p>",
          "rawMarkdown": "for this \"LB=0.82 cnn_trad_pool2_net \", pseudo-labeling is not required.\n\n\nFor the split, use the original split provided in kaggle. Anyway, I have attached the files below. \n\n\nAlso, the confidence for \"silence\" LB test samples is usually low. You should try a \"much\" lower threshold. (but again pseudo-labeling is not required)"
        },
        {
          "id": 266063,
          "postDate": "2018-01-07T17:47:05.960Z",
          "content": "<p>Thanks a lot Heng. What I've noticed is that even if I use the <code>validation_list.txt</code> and <code>testing_list.txt</code> provided with the data I still have a difference of 20% from the LB.  Even if I try lower thresholds still there's no predicted label{unknown, silence} with prob. &gt; 0.6. If you don't mind me asking, from what I understand is that you split your data by probability of 0.1 for each class{unknown, silence}. You consider only the original classes i.e. 10, and from that you account a 0.1% to be unknown and another 0.1% to be silence.</p>\n\n<pre><code>known_num = 0\n    for i in range(2,AUDIO_NUM_CLASSES):\n        known_num += len(data.index_by_class[i])\n\n    self.known_num   = known_num\n    self.silence_num = int((self.known_num/self.known_probability)*self.silence_probability)\n    self.unknown_num = int((self.known_num/self.known_probability)*self.unknown_probability)\n    self.length = self.silence_num + self.unknown_num + self.known_num\n</code></pre>\n\n<p>How did you came up with that probability of 0.1. Is that a heuristic? On another note what is the intuition in this particular splitting technique?</p>",
          "rawMarkdown": "Thanks a lot Heng. What I've noticed is that even if I use the `validation_list.txt` and `testing_list.txt` provided with the data I still have a difference of 20% from the LB.  Even if I try lower thresholds still there's no predicted label{unknown, silence} with prob. &gt; 0.6. If you don't mind me asking, from what I understand is that you split your data by probability of 0.1 for each class{unknown, silence}. You consider only the original classes i.e. 10, and from that you account a 0.1% to be unknown and another 0.1% to be silence.\n\n    known_num = 0\n        for i in range(2,AUDIO_NUM_CLASSES):\n            known_num += len(data.index_by_class[i])\n\n        self.known_num   = known_num\n        self.silence_num = int((self.known_num/self.known_probability)*self.silence_probability)\n        self.unknown_num = int((self.known_num/self.known_probability)*self.unknown_probability)\n        self.length = self.silence_num + self.unknown_num + self.known_num\n\nHow did you came up with that probability of 0.1. Is that a heuristic? On another note what is the intuition in this particular splitting technique?"
        },
        {
          "id": 266149,
          "postDate": "2018-01-07T23:59:34.470Z",
          "content": "<p>someone make a kernel to probe the LB dataset by submitting 12 csv files, each csv file has only one label. It turns out the LB score of each  csv file is about 0.08 to 0.09, meaning that the distribution of each class is almost equal in the public LB set.</p>\n\n<p>However, it seems that the kernel is removed?</p>\n\n<p>To confirm, you can submit a csv file with all test samples labelled as unknown. You can use this to probe the probability odf \"unknown\" of the LB public set  Assume that the private LB set = LB public set  +/- delta, make make sure your model works for both public and private cases.</p>",
          "rawMarkdown": "someone make a kernel to probe the LB dataset by submitting 12 csv files, each csv file has only one label. It turns out the LB score of each  csv file is about 0.08 to 0.09, meaning that the distribution of each class is almost equal in the public LB set.\n\nHowever, it seems that the kernel is removed?\n\nTo confirm, you can submit a csv file with all test samples labelled as unknown. You can use this to probe the probability odf \"unknown\" of the LB public set  Assume that the private LB set = LB public set  +/- delta, make make sure your model works for both public and private cases."
        },
        {
          "id": 266151,
          "postDate": "2018-01-08T00:12:00.637Z",
          "content": "<p>i think i got validation=0.93 and LB=0.73 if i use random sampling for all train samples during training. But once i start to use class balance sampling (i.e. posted code, with silence and unknown train samples at 0.1 each at each epoch), results are improved. You should get about 12% difference between validation and LB set.</p>\n\n<p>Some experts can get to about 8 to 6% from the posts i read in the forum</p>",
          "rawMarkdown": "i think i got validation=0.93 and LB=0.73 if i use random sampling for all train samples during training. But once i start to use class balance sampling (i.e. posted code, with silence and unknown train samples at 0.1 each at each epoch), results are improved. You should get about 12% difference between validation and LB set.\n\nSome experts can get to about 8 to 6% from the posts i read in the forum",
          "votes": 3
        },
        {
          "id": 266368,
          "postDate": "2018-01-08T16:08:26.563Z",
          "content": "<p>Thanks Heng appreciate it :). That makes a lot of sense now. If you don't follow closely the discussions in the forum that's what happens. It's not that easy though to keep track of everything! </p>",
          "rawMarkdown": "Thanks Heng appreciate it :). That makes a lot of sense now. If you don't follow closely the discussions in the forum that's what happens. It's not that easy though to keep track of everything! "
        },
        {
          "id": 267762,
          "postDate": "2018-01-12T08:04:55.363Z",
          "content": "<p>Indeed it seems that for the public LB, the distribution is about equal for each label. However for the whole test data set this is not the case. I got typically something like this for my submission file (around 0.86 score):</p>\n\n<p>yes     →  3.87%</p>\n\n<p>no      →  4.63%</p>\n\n<p>up      →  4.53%</p>\n\n<p>down    →  3.49%</p>\n\n<p>left    →  4.40%</p>\n\n<p>right   →  3.99%</p>\n\n<p>on      →  4.47%</p>\n\n<p>off     →  4.31%</p>\n\n<p>stop    →  3.85%</p>\n\n<p>go      →  4.38%</p>\n\n<p>silence →  7.29%</p>\n\n<p>unknown → 50.78%</p>\n\n<p>So Unknown is about 50%, silence above average and the rest around 4%. So there could be a major shake-up if some of the top submissions favour the known words right now. </p>\n\n<p>In theory you could not include any \"unknow\" labels in your submission and score 0.90 right now and once the you are tested against the whole set drop back to 0.50 :) </p>",
          "rawMarkdown": "Indeed it seems that for the public LB, the distribution is about equal for each label. However for the whole test data set this is not the case. I got typically something like this for my submission file (around 0.86 score):\n\nyes     →  3.87%\n\nno      →  4.63%\n\nup      →  4.53%\n\ndown    →  3.49%\n\nleft    →  4.40%\n\nright   →  3.99%\n\non      →  4.47%\n\noff     →  4.31%\n\nstop    →  3.85%\n\ngo      →  4.38%\n\nsilence →  7.29%\n\nunknown → 50.78%\n\nSo Unknown is about 50%, silence above average and the rest around 4%. So there could be a major shake-up if some of the top submissions favour the known words right now. \n\nIn theory you could not include any \"unknow\" labels in your submission and score 0.90 right now and once the you are tested against the whole set drop back to 0.50 :) "
        },
        {
          "id": 267771,
          "postDate": "2018-01-12T08:50:10.320Z",
          "content": "<p>We tested against our validation set. We can get both good results for subset ( equal distribution) or full set( unknown distribution is greater than 50%)</p>",
          "rawMarkdown": "We tested against our validation set. We can get both good results for subset ( equal distribution) or full set( unknown distribution is greater than 50%)",
          "votes": 1
        },
        {
          "id": 267774,
          "postDate": "2018-01-12T09:02:51.083Z",
          "content": "<p>That seems indeed a good strategy. Out of interest, do you see higher validation scores lead to higher LB scores, or is this correlation not there once you get into the high 0.8x ?</p>\n\n<p>For this competition I use the validation score as a method to guard against overfitting, however there is no direct link to LB score. So a validation score can be higher while the LB score is lower.</p>",
          "rawMarkdown": "That seems indeed a good strategy. Out of interest, do you see higher validation scores lead to higher LB scores, or is this correlation not there once you get into the high 0.8x ?\n\nFor this competition I use the validation score as a method to guard against overfitting, however there is no direct link to LB score. So a validation score can be higher while the LB score is lower."
        }
      ]
    },
    {
      "id": 265872,
      "postDate": "2018-01-07T00:04:25.490Z",
      "content": "<p>Hi Heng, thanks for sharing. I just have a quick question. Is there a particular reason for not sharing some specific parts. For instance the Sampler class is missing, as well as some other functionality coming from files such as dataset.sampler,  common, utility, etc. ?</p>",
      "rawMarkdown": "Hi Heng, thanks for sharing. I just have a quick question. Is there a particular reason for not sharing some specific parts. For instance the Sampler class is missing, as well as some other functionality coming from files such as dataset.sampler,  common, utility, etc. ?"
    },
    {
      "id": 265857,
      "postDate": "2018-01-06T22:18:28.567Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 265672,
      "postDate": "2018-01-06T09:05:45.517Z",
      "content": "<p>thank you!</p>",
      "rawMarkdown": "thank you!",
      "votes": 1
    },
    {
      "id": 269134,
      "postDate": "2018-01-16T09:31:39.197Z",
      "content": "<p>Thanks for the post</p>",
      "rawMarkdown": "Thanks for the post"
    },
    {
      "id": 268799,
      "postDate": "2018-01-15T17:22:16.427Z",
      "content": "<p>Thanks.</p>",
      "rawMarkdown": "Thanks."
    },
    {
      "id": 267633,
      "postDate": "2018-01-11T23:07:31.613Z",
      "content": "<p>Thank you! This is really helpful</p>",
      "rawMarkdown": "Thank you! This is really helpful"
    },
    {
      "id": 266673,
      "postDate": "2018-01-09T11:56:35.510Z",
      "content": "<p>Thanks bro.</p>",
      "rawMarkdown": "Thanks bro."
    }
  ],
  "comments": [
    {
      "id": 269128,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2018-01-16T09:00:15.600000",
      "content": "<p>@Shitian Ni</p>\n\n<p>from your notebook torch.ipynb, i convert to main.py below. it can run fine on my system. I attached the log file, which shows the results:</p>\n\n<pre><code>** start training here! **\noptimizer=&lt;torch.optim.sgd.SGD object at 0x7fa2cf688518&gt;\nmomentum=0.900000\nLR=None\n\nwaves_per_epoch = 51088\n\n  rate   iter_k   epoch  num_m| valid_loss/acc | train_loss/acc | batch_loss/acc |  time   \n  --------------------------------------------------------------------------------------------\n  0.0000    0.0 k  0.00   0.0 | 2.5337  0.0826 | 0.0000  0.0000 | 0.0000  0.0000 |  0 hr 00 min \n  0.0050    0.5 k  1.25   0.1 | 0.6781  0.7852 | 1.0782  0.6578 | 0.7809  0.7344 |  0 hr 02 min \n  0.0050    1.0 k  2.51   0.1 | 0.6302  0.8107 | 0.9056  0.7074 | 0.7444  0.7344 |  0 hr 03 min \n</code></pre>\n\n<p>you may want to check your your system, e.g. run pytorch mnist example, etc.</p>\n\n<p>My environment is:</p>\n\n<pre><code>set cuda environment\n    torch.__version__              = 0.3.0.post4\n    torch.version.cuda             = 9.0.176\n    torch.backends.cudnn.version() = 7003\n    os['CUDA_VISIBLE_DEVICES']  = 0\n    torch.cuda.device_count()   = 1\n    torch.cuda.current_device() = 0\n</code></pre>\n\n<p>but i don't think cuda8/9 will make a difference.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 266602,
      "author_name": "Shitian Ni",
      "author_url": "",
      "post_date": "2018-01-09T06:12:32.083000",
      "content": "<p>Thank you for sharing. I am new to pytorch, and have several issues running the code.</p>\n\n<p>I got </p>\n\n<pre><code>RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1512378360668/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\n</code></pre>\n\n<p>when using Cnn_Trad_Pool2_Net. I want to know how 26624 in the final Linear layer is calculated.\nI tried using other networks, but the accuracies are not improving, \nlogs look like</p>\n\n<pre><code>0.0050  174.0 k  436.00  22.3 | 2.9176  0.1000 | 2.4834  0.0930 | 2.4858  0.0938 | 12 hr 48 min  174121,174121, torch.Size([128, 1, 40, 101])\n</code></pre>\n\n<p>for running vggnet.\nWhat can be possibly wrong?\nThank you very much.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 266605,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-09T06:20:10.940000",
          "content": "<blockquote>\n  <blockquote>\n    <p>I want to know how 26624 in the final Linear layer is calculated. \n    you can do a print  in forward(), see #print(x.size()) below. From there you can compute the size.</p>\n  </blockquote>\n</blockquote>\n\n<pre><code>class Cnn_Trad_Pool2_Net(nn.Module):\ndef __init__(self, in_shape=(1,40,101), num_classes=12 ):\n    super(Cnn_Trad_Pool2_Net, self).__init__()\n    self.num_classes = num_classes\n\n    self.conv1 = nn.Conv2d(1,  64, kernel_size=(20, 8), stride=(1, 1))\n    self.conv2 = nn.Conv2d(64, 64, kernel_size=(10, 4), stride=(1, 1))\n    self.fc = nn.Linear(26624,num_classes)\n\n\ndef forward(self, x):\n\n    x = self.conv1(x)\n    x = F.relu(x,inplace=True)\n    x = F.max_pool2d(x,kernel_size=(2,2),stride=(2,2))\n\n    x = self.conv2(x)\n    x = F.relu(x,inplace=True)\n    x = x.view(x.size(0), -1)\n\n    #print(x.size())\n    x = F.dropout(x,p=0.5,training=self.training)\n    x = self.fc(x)\n\n    return x  #logits\n</code></pre>",
          "votes": 3,
          "replies": []
        },
        {
          "id": 267151,
          "author_name": "kmaster",
          "author_url": "",
          "post_date": "2018-01-10T17:30:54.263000",
          "content": "<p>Hi Heng CherKeng,\nThanks for sharing, I have the same error. Can you please help me about this?\nRuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1503963423183/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243</p>\n\n<p>Print(x.size()) give me torch.Size([128, 2816]).\nDo you think I should change the some parameters in CNN?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267162,
          "author_name": "Shitian Ni",
          "author_url": "",
          "post_date": "2018-01-10T17:59:02.007000",
          "content": "<p>I solved the issue by copying Linear class and corresponding codes from pytorch repository to the local file, rename them and replace Linear class in the original code</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 267184,
          "author_name": "kmaster",
          "author_url": "",
          "post_date": "2018-01-10T19:33:04.960000",
          "content": "<p>Hi Shitian\nThank you for response\nDo you mean you sync linear.py from pytorch repository with your local installed one?\nI also read other related posts. It looks like the error related to python version too.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267345,
          "author_name": "Shitian Ni",
          "author_url": "",
          "post_date": "2018-01-11T04:31:28.303000",
          "content": "<p>I don't know what you mean by sync. I mean copy code from pytorch github repository to your local file, replace linear class you import from your code.  Then the error disappears. My environment was Nvidia GPU Cloud pytorch docker. But after that I also got some other errors like expect Variable(CPU_LONG) got Variable(GPU_LONG).</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267346,
          "author_name": "kmaster",
          "author_url": "",
          "post_date": "2018-01-11T04:50:37.103000",
          "content": "<p>Thank you. Let us keep trying.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267935,
          "author_name": "Josan",
          "author_url": "",
          "post_date": "2018-01-12T20:02:58.723000",
          "content": "<p>Any news on fixing the errors? I also encountered the same issue.</p>\n\n<p>However, my error is slightly different: </p>\n\n<p><code>\nRuntimeError: sizes do not match at /opt/conda/conda-bld/pytorch_1501969512886/work/pytorch-0.1.12/torch/lib/THC/generated/../generic/THCTensorMathPointwise.cu:296\n</code></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267944,
          "author_name": "kmaster",
          "author_url": "",
          "post_date": "2018-01-12T20:27:19.040000",
          "content": "<p>No. I changed 26624  to some other number. It didn't work</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 268035,
          "author_name": "Josan",
          "author_url": "",
          "post_date": "2018-01-13T03:50:07.913000",
          "content": "<p>I solved my issue. I just update the pytorch from 0.1.2 to 0.3.0. Hope this could also apply to yours.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 268307,
          "author_name": "yyll008",
          "author_url": "",
          "post_date": "2018-01-14T04:14:59",
          "content": "<p>RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1512387374934/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\nHow solve this error?\nMy Pytorch vesion is \"0.3.0.post4\"</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 268358,
          "author_name": "Shitian Ni",
          "author_url": "",
          "post_date": "2018-01-14T09:09:09.320000",
          "content": "<p>I solved the error after setting <code>self.fc = nn.Linear(2816,self.num_classes)</code></p>\n\n<p>View Examples in\n<a href=\"http://pytorch.org/docs/master/nn.html#linear-layers\">Linear layers docs</a></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 268360,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-14T09:13:39.367000",
          "content": "<p>use this instead. kenel size is transposed, e.g kernel_size=(8, 20)</p>\n\n<p>class Cnn_Trad_Pool2_Net(nn.Module):</p>\n\n<pre><code>def __init__(self, in_shape=(1,40,101), num_classes=12 ):\n\n    super(Cnn_Trad_Pool2_Net, self).__init__()\n    self.num_classes = num_classes\n\n    self.conv1 = nn.Conv2d(1,  64, kernel_size=(8, 20), stride=(1, 1))\n    self.conv2 = nn.Conv2d(64, 64, kernel_size=(4, 10), stride=(1, 1))\n    self.fc = nn.Linear(26624,num_classes)\n\n\ndef forward(self, x):\n\n    x = self.conv1(x)\n    x = F.relu(x,inplace=True)\n    x = F.max_pool2d(x,kernel_size=(2,2),stride=(2,2))\n\n    x = self.conv2(x)\n    x = F.relu(x,inplace=True)\n    x = x.view(x.size(0), -1)\n\n    #print(x.size())\n    x = F.dropout(x,p=0.5,training=self.training)\n    x = self.fc(x)\n\n    return x  #logits\n</code></pre>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 268465,
          "author_name": "Shitian Ni",
          "author_url": "",
          "post_date": "2018-01-14T14:51:46.817000",
          "content": "<p>Thank you.\nThe error is gone, but my accuracies still at 0.09 after number of epochs.</p>\n\n<p>If I run <code>net(tensors)</code>,</p>\n\n<p>I get</p>\n\n<pre><code> Variable containing:\n\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n          ...             ⋱             ...          \n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n 0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\n[torch.cuda.FloatTensor of size 128x12 (GPU 0)]\n</code></pre>\n\n<p>Outputs from the network are the same for all tensors from</p>\n\n<pre><code>for tensors, labels, indices in train_loader:\n    tensors = Variable(tensors).cuda()\n</code></pre>\n\n<p>But tensors are different.</p>\n\n<p>Any suggestions?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 268653,
          "author_name": "yyll008",
          "author_url": "",
          "post_date": "2018-01-15T06:16:18.620000",
          "content": "<p>Thank you. have fix this!</p>\n\n<p>Awsome work, you are my hero~~</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 268889,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-15T22:20:07.477000",
          "content": "<p>@Shitian Ni</p>\n\n<p>can you post your all files that produce \"0.1912  0.1844 -0.0294  ...  -0.0409 -0.0181 -0.0262\" . I try to debug</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 269117,
          "author_name": "Shitian Ni",
          "author_url": "",
          "post_date": "2018-01-16T08:21:41.737000",
          "content": "<p>Thank you. I think that is something with .cuda() and .cpu(). Now that is solved, but my accuracy still doesn't improve.\nCould you review my file is possible?\nI put the files on my Github account\n<a href=\"https://github.com/shitian-ni/torch-speech\">https://github.com/shitian-ni/torch-speech</a>\nMain file is torch.ipynb, you can see that the accuracy is not improving.\nMy torch version is '0.3.0.post4'\nThank you very much</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 266469,
      "author_name": "Fizmath",
      "author_url": "",
      "post_date": "2018-01-08T21:21:17.013000",
      "content": "<p>Hello</p>\n\n<p>I am trying to set up a Jupyter notebook from your PyCharm files ; yet work to be done , got head on collision with many errors .  any suggestions ? </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 266353,
      "author_name": "Anton M",
      "author_url": "",
      "post_date": "2018-01-08T15:31:58.037000",
      "content": "<p>@Heng CherKeng</p>\n\n<p>Can you explain how (-1) is possible as index in  Dataset? I guess it's because custom collate function ?</p>\n\n<p>So Im trying to implement dataloader in pytorch, that on every epoch returns only subset of \"unknowns\". But I can't understand how it's done in you code. </p>",
      "votes": 1,
      "replies": [
        {
          "id": 266354,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-08T15:33:52.887000",
          "content": "<p>yes. there is a custom collate function. see audio_dataset.py : def collate(batch):</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 266355,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-08T15:34:58.270000",
          "content": "<p>you should see: audio_processing_tf.py:</p>\n\n<pre><code>class TFRandomSampler(Sampler):\n\n    def __init__(self, data, silence_probability=0.1, unknown_probability=0.1):\n           ...\n\n   def __iter__(self):\n          data = self.data\n          ....\n\n           if self.unknown_num&gt;0:#unknown\n               unknown_list = data.index_by_class[1]*math.ceil(self.unknown_num/len(data.index_by_class[1]))\n               random.shuffle(unknown_list)\n               unknown_list = unknown_list[:self.unknown_num]\n               l +=  unknown_list\n</code></pre>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 266122,
      "author_name": "Cut Onion",
      "author_url": "",
      "post_date": "2018-01-07T22:33:13.473000",
      "content": "<p>Almost everything is working with provided files, however, just tiny little functions are missing. It should be trivial to re-implement but for the sake of completeness, Heng CherKeng could your provide <code>top_accuracy(), adjust_learning_rate(), and get_learning_rate()</code>, the only missing functions? :)</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 266068,
      "author_name": "Yibing Wu",
      "author_url": "",
      "post_date": "2018-01-07T18:07:50.233000",
      "content": "<p>Hi CherKeng,\nI tried to run the code but got this error message even I installed the module 'utility'. Could you or anyone help? Thanks.</p>\n\n<p>Traceback (most recent call last):</p>\n\n<p>File \"~/PyTorch/train_cnn_trad_pool2_net.py\", line 7, in </p>\n\n<pre><code>from utility.file import *\n</code></pre>\n\n<p>ModuleNotFoundError: No module named 'utility'</p>",
      "votes": 1,
      "replies": [
        {
          "id": 266145,
          "author_name": "Cut Onion",
          "author_url": "",
          "post_date": "2018-01-07T23:36:44.813000",
          "content": "<p>You don't need to install any <code>utility</code> module. Just create folder <code>utility</code> and place <code>file.py</code> there</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 266154,
          "author_name": "Yibing Wu",
          "author_url": "",
          "post_date": "2018-01-08T00:21:30.513000",
          "content": "<p>Thank you, Sir. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 266788,
          "author_name": "Nagendra Singh",
          "author_url": "",
          "post_date": "2018-01-09T18:38:10.800000",
          "content": "<p>Thanks for the suggestion.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 265702,
      "author_name": "Anton M",
      "author_url": "",
      "post_date": "2018-01-06T10:44:38.617000",
      "content": "<p>Hi Heng, glad you are posting in this competition!</p>\n\n<p>\"Silence.wav\" is it zero wav file? </p>",
      "votes": 1,
      "replies": [
        {
          "id": 265704,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-06T10:47:59.217000",
          "content": "<p>yes. i could have just use np.zeros(), but i choose to keep it as a wav file to simplify my code for other purposes</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 265926,
          "author_name": "FullySelfDrivingTang",
          "author_url": "",
          "post_date": "2018-01-07T05:36:00.283000",
          "content": "<p>Thanks CherKeng for your great sharing, I have question here. What is different between Silence.wav and empty.wav.  I created the empty.wav with your code. Do I need similar method to create Silence.wav?  </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 269786,
          "author_name": "Taka",
          "author_url": "",
          "post_date": "2018-01-17T09:57:29.603000",
          "content": "<p>This zero wav file improved my LB from 0.84 to 0.88, really amazing! I find it reasonable after I listen to test data which contains some mute data</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 266150,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2018-01-08T00:02:21.510000",
      "content": "<p>@Sergey Mushinskiy</p>\n\n<p>The files as requested.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 266152,
          "author_name": "Cut Onion",
          "author_url": "",
          "post_date": "2018-01-08T00:18:38.813000",
          "content": "<p>Thanks! Hope it attracts even more guys to compete :)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 266155,
          "author_name": "Yibing Wu",
          "author_url": "",
          "post_date": "2018-01-08T00:23:27.970000",
          "content": "<p>Hi, CherKeng, do you mind drawing a data/file structure? It would be great to people who is a python newbie. Thanks again for the sharing.  </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 266366,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-08T16:04:24.360000",
          "content": "<p>@Terracotta</p>\n\n<p>Please refer to this post: <a href=\"https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/46988\">https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/46988</a></p>\n\n<p>look for the below message:</p>\n\n<p>\"please refer to ppt for details.</p>\n\n<p>here, the files contain: - full pycharm project, include train, evaluate, submit code - trained model at LB=0.86 - data split</p>\n\n<p>build.tar.gz (82.37 KB)</p>\n\n<p>data.tar.gz (2.27 MB)</p>\n\n<p>results.tar.gz (4.45 MB)</p>\n\n<p>readme.pptx (524.8 KB)\"</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 266386,
          "author_name": "Yibing Wu",
          "author_url": "",
          "post_date": "2018-01-08T17:15:25.433000",
          "content": "<p>Many thanks. Great discussion posts!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266674,
          "author_name": "kirk",
          "author_url": "",
          "post_date": "2018-01-09T11:57:58.593000",
          "content": "<p>Hi @Heng. I have a question regarding the implementation of the samplers such as <code>TFRandomSampler(Sampler)</code> and <code>TFSequentialSampler(Sampler)</code>. In the implementation of those classes you inherit/extend the Sampler class. But nowhere in your files I could find the <code>Sampler</code> class or the <code>RandomSampler(dataset)</code> for that matter which you call at line 252 in file audio_processing_tf.py. The only thing I could find is a sampler object which is initialized by the <code>TFSequentialSampler</code>. Would you mind elaborating a little bit on this? Is there a possibility that you meant to call <code>TFRandomSampler</code> instead of <code>RandomSampler</code> in file <code>audio_processing_tf.py</code></p>\n\n<p>I've tried multiple times to run the provided files to get the dataset split correctly and this is what I get:</p>\n\n<pre><code>audio_processing_tf.py: calling main function ...\ntime = 0.00 min\nnum_ids = 6798\nnum_classes = 12\n0            silence =     0 (0.000)\n1            unknown =  4221 (0.621)\n2                yes =   261 (0.038)\n3                 no =   270 (0.040)\n4                 up =   260 (0.038)\n5               down =   264 (0.039)\n6               left =   247 (0.036)\n7              right =   256 (0.038)\n8                 on =   257 (0.038)\n9                off =   256 (0.038)\n10               stop =   246 (0.036)\n11                 go =   260 (0.038)\n-----------------------------------\n0 stop\n9 off\n4 up\n3 no\n6 left\n4 up\n6 left\n11 go\n8 on\n6 left\n11 go\n7 right\n7 right\n2 yes\nlabels: 3221, waves: 3221\nsucess!\n</code></pre>\n\n<p>The output is from running <code>audio_processing_tf.py</code> and replacing RandomSampler with TFRandomSampler. The file is <code>train_valid_6798</code>, the question here which I cannot answer is why do I get 3221 instead of 6798 processed files?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266785,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-09T18:23:50.710000",
          "content": "<p>Sampler is pytorch class. 3221 is random subsample version of 6798 , to ensure that prob_silence=0.1 and prob_unknown=0.1</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266787,
          "author_name": "kirk",
          "author_url": "",
          "post_date": "2018-01-09T18:35:32.503000",
          "content": "<p>Nice thanks for the clarifications. You see what I was doing wrong was the fact that I was trying to process all the data and save them, then train the model but in your case you had the iterator to feed the data into the model. Now I know that I'll have to run the sampler a couple of times if I want to process the data once and save them. Cheers!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266966,
          "author_name": "kirk",
          "author_url": "",
          "post_date": "2018-01-10T08:18:14.827000",
          "content": "<p>Hi @Heng, sorry bothering you but I need to learn and the only way is by asking. If you get annoyed by my question please just let me know and I'll stop asking ;). I've used your technique to create the train,valid, and test generators and I've tested them on a model which previously was giving ~0.73% on LB. After using your provided technique for generating the data and the same model it seems that the mode is unable to learn anything at all. The loss oscillates between particular values but never improves. Below is an output from the training procedure:</p>\n\n<pre><code>512/512 [==============================] - 96s - loss: 2.0777 - acc: 0.2480 - val_loss: 2.0285 - \nval_acc: 0.2988--\nEpoch 26/50\n512/512 [==============================] - 95s - loss: 2.0339 - acc: 0.2852 - val_loss: 2.0035 - \nval_acc: 0.3105--\nEpoch 27/50\n512/512 [==============================] - 102s - loss: 2.0812 - acc: 0.2871 - val_loss: 2.0661 - \nval_acc: 0.2539-\nEpoch 28/50\n512/512 [==============================] - 105s - loss: 2.0969 - acc: 0.2383 - val_loss: 1.9392 - \nval_acc: 0.3086-\nEpoch 29/50\n512/512 [==============================] - 105s - loss: 2.0443 - acc: 0.2773 - val_loss: 2.1398 - \nval_acc: 0.2559-\nEpoch 30/50\n512/512 [==============================] - 103s - loss: 1.9928 - acc: 0.2715 - val_loss: 2.0004 - \nval_acc: 0.3262-\nEpoch 31/50\n512/512 [==============================] - 105s - loss: 2.1038 - acc: 0.2363 - val_loss: 2.0440 - \nval_acc: 0.3086-\nEpoch 32/50\n512/512 [==============================] - 103s - loss: 2.1193 - acc: 0.2520 - val_loss: 1.9881 - \nval_acc: 0.2988-\nEpoch 33/50\n512/512 [==============================] - 103s - loss: 2.0528 - acc: 0.2695 - val_loss: 1.9842 - \nval_acc: 0.2383-\nEpoch 34/50\n512/512 [==============================] - 101s - loss: 2.0756 - acc: 0.2559 - val_loss: 2.0211 - \nval_acc: 0.2969-\nEpoch 35/50\n512/512 [==============================] - 103s - loss: 2.0462 - acc: 0.2773 - val_loss: 1.9498 - \nval_acc: 0.3086-\nEpoch 36/50\n512/512 [==============================] - 99s - loss: 2.0386 - acc: 0.3066 - val_loss: 2.0807 - \n</code></pre>\n\n<p>val_acc: 0.2598--</p>\n\n<p>One thing that I did was to add random padding as augmentation, do think that might be affecting the quality of the data?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266968,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-10T08:20:23.223000",
          "content": "<p>there could be some bug in your implementation. I suggest you can install pytorch and run my code</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266983,
          "author_name": "Shitian Ni",
          "author_url": "",
          "post_date": "2018-01-10T09:03:58.527000",
          "content": "<p>I ran your code, but has the same issue </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266987,
          "author_name": "Yibing Wu",
          "author_url": "",
          "post_date": "2018-01-10T09:23:53.870000",
          "content": "<p>At first I had some issues with the code but they were gone after updating to the latest Pytorch.  I hope this could help. The issue I have now is that I only got LB 0.78. Still in a learning process. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266989,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-10T09:37:37.843000",
          "content": "<p>Remember to change your learning rate. You need to run twice. Check my log files and graph in pptx</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267642,
          "author_name": "Yibing Wu",
          "author_url": "",
          "post_date": "2018-01-11T23:43:18.047000",
          "content": "<p>Thanks. It worked.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 279288,
          "author_name": "pooh",
          "author_url": "",
          "post_date": "2018-02-07T18:11:20.930000",
          "content": "<p>@Heng CherKeng i started working on your project but i dont have gpu and i am trying to make the required change in the project to run it on CPU (without GPU).</p>\n\n<p>I am stuck and i dont know how to proceed.</p>\n\n<p>any help will be very much appreciated!</p>\n\n<p>Thanks</p>\n\n<h2>the error shows</h2>\n\n<pre><code>** start evaluation here! **\nTraceback (most recent call last):\nFile \"/home/hrcule/competitionKaggle/speechTensorFlow/fullProject/build/resnet-tf-speech /evaluate.py\", line 127, in &lt;module&gt;\nrun_evaluate()\nFile \"/home/hrcule/competitionKaggle/speechTensorFlow/fullProject/build/resnet-tf-speech /evaluate.py\", line 93, in run_evaluate\nfor i, (tensors, labels, indices) in enumerate(test_loader, 0):\nFile \"/home/hrcule/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py\", line 210, in __next__\nreturn self._process_next_batch(batch)\n</code></pre>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 266026,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2018-01-07T14:24:28.067000",
      "content": "<p>data split and other files as requested  </p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 269810,
      "author_name": "Andrzej",
      "author_url": "",
      "post_date": "2018-01-17T10:47:38.273000",
      "content": "<p>Thank you Lots of great tips how to setup the code &amp; Congratulations!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 267365,
      "author_name": "JamesGoGo",
      "author_url": "",
      "post_date": "2018-01-11T06:34:26.567000",
      "content": "<p>@Heng CherKeng，</p>\n\n<p>Your code is great. I am new to pytorch. It is a great example. I notice you have great graphs in your report. I wonder whether it is a built-in function in pytorch like tensorboard in tensorflow, or you created your self. Could you please share some information about this? Thanks!</p>",
      "votes": 0,
      "replies": [
        {
          "id": 267492,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-11T14:36:00.580000",
          "content": "<p>it is created by hand in spreadsheet software</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 266677,
      "author_name": "Masamune O",
      "author_url": "",
      "post_date": "2018-01-09T12:07:59.997000",
      "content": "<p>Nice</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 266046,
      "author_name": "Arbyn Acosta",
      "author_url": "",
      "post_date": "2018-01-07T16:19:32.580000",
      "content": "<p>I was wondering, did you use GPU to train the model? If yes, how many GPUs did you use?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 266047,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-07T16:20:55.593000",
          "content": "<p>the log files are based on single GPU machine (titanX pascal) </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 266051,
          "author_name": "Arbyn Acosta",
          "author_url": "",
          "post_date": "2018-01-07T16:45:37.863000",
          "content": "<p>Ok, thanks! That explains the short training duration... I'm struggling to train my models in my laptop with an Nvidia 840M GPU which takes at least 8 hours!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 266022,
      "author_name": "Yibing Wu",
      "author_url": "",
      "post_date": "2018-01-07T14:18:23.257000",
      "content": "<p>Thanks CherKeng for the great sharing.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 266027,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-07T14:24:57.757000",
          "content": "<p>see post below for file.py</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 265928,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2018-01-07T05:38:32.490000",
      "content": "<p>@JohnsonTang\nHere is my file.  I think they are the same file</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 265915,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2018-01-07T04:55:19.460000",
      "content": "<p>@kirk . No. There are no special reasons. My whole project files is large and I have just upload the most important files to illustrate the method. You should be able to reproduce the results with my files. </p>\n\n<p>The missing files are less important, e.g. for data read/write, etc . Some of the import (e.g.  dataset.sampler) are not used at all.</p>\n\n<p>Note that the intend of this post is not to encourage kagglers to download scripts, run and make submission. Rather kagglers should take my files as reference and modify or \"debug\" their existing code, method.</p>\n\n<p>If there is anything unclear, please ask again in this post. I added two more files here:</p>\n\n<ol>\n<li>submit.py to make submission</li>\n<li>common.py which is  a list of packages i used (but not all are necessary)</li>\n</ol>",
      "votes": 0,
      "replies": [
        {
          "id": 266017,
          "author_name": "kirk",
          "author_url": "",
          "post_date": "2018-01-07T13:44:58.217000",
          "content": "<p>Hi Heng. Thanks a lot for your reply. The reason for me asking was because I noticed that parts such as sampler were used for splitting the data (correct me if I am wrong). The main problem that I am trying to understand and solve is how to properly split the dataset. When trying to use pseudo-labeling I had a reference model of 0.73% and used that to predict labels {unknown, silence} on the test set with prob &gt; 0.95, but the model couldn't find any labels{unknown, silence} with prob. &gt; 0.95 which made the whole pseudo-labeling technique unusable for me.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266024,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-07T14:23:01.130000",
          "content": "<p>for this \"LB=0.82 cnn_trad_pool2_net \", pseudo-labeling is not required.</p>\n\n<p>For the split, use the original split provided in kaggle. Anyway, I have attached the files below. </p>\n\n<p>Also, the confidence for \"silence\" LB test samples is usually low. You should try a \"much\" lower threshold. (but again pseudo-labeling is not required)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266063,
          "author_name": "kirk",
          "author_url": "",
          "post_date": "2018-01-07T17:47:05.960000",
          "content": "<p>Thanks a lot Heng. What I've noticed is that even if I use the <code>validation_list.txt</code> and <code>testing_list.txt</code> provided with the data I still have a difference of 20% from the LB.  Even if I try lower thresholds still there's no predicted label{unknown, silence} with prob. &gt; 0.6. If you don't mind me asking, from what I understand is that you split your data by probability of 0.1 for each class{unknown, silence}. You consider only the original classes i.e. 10, and from that you account a 0.1% to be unknown and another 0.1% to be silence.</p>\n\n<pre><code>known_num = 0\n    for i in range(2,AUDIO_NUM_CLASSES):\n        known_num += len(data.index_by_class[i])\n\n    self.known_num   = known_num\n    self.silence_num = int((self.known_num/self.known_probability)*self.silence_probability)\n    self.unknown_num = int((self.known_num/self.known_probability)*self.unknown_probability)\n    self.length = self.silence_num + self.unknown_num + self.known_num\n</code></pre>\n\n<p>How did you came up with that probability of 0.1. Is that a heuristic? On another note what is the intuition in this particular splitting technique?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266149,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-07T23:59:34.470000",
          "content": "<p>someone make a kernel to probe the LB dataset by submitting 12 csv files, each csv file has only one label. It turns out the LB score of each  csv file is about 0.08 to 0.09, meaning that the distribution of each class is almost equal in the public LB set.</p>\n\n<p>However, it seems that the kernel is removed?</p>\n\n<p>To confirm, you can submit a csv file with all test samples labelled as unknown. You can use this to probe the probability odf \"unknown\" of the LB public set  Assume that the private LB set = LB public set  +/- delta, make make sure your model works for both public and private cases.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 266151,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-08T00:12:00.637000",
          "content": "<p>i think i got validation=0.93 and LB=0.73 if i use random sampling for all train samples during training. But once i start to use class balance sampling (i.e. posted code, with silence and unknown train samples at 0.1 each at each epoch), results are improved. You should get about 12% difference between validation and LB set.</p>\n\n<p>Some experts can get to about 8 to 6% from the posts i read in the forum</p>",
          "votes": 3,
          "replies": []
        },
        {
          "id": 266368,
          "author_name": "kirk",
          "author_url": "",
          "post_date": "2018-01-08T16:08:26.563000",
          "content": "<p>Thanks Heng appreciate it :). That makes a lot of sense now. If you don't follow closely the discussions in the forum that's what happens. It's not that easy though to keep track of everything! </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267762,
          "author_name": "Peter",
          "author_url": "",
          "post_date": "2018-01-12T08:04:55.363000",
          "content": "<p>Indeed it seems that for the public LB, the distribution is about equal for each label. However for the whole test data set this is not the case. I got typically something like this for my submission file (around 0.86 score):</p>\n\n<p>yes     →  3.87%</p>\n\n<p>no      →  4.63%</p>\n\n<p>up      →  4.53%</p>\n\n<p>down    →  3.49%</p>\n\n<p>left    →  4.40%</p>\n\n<p>right   →  3.99%</p>\n\n<p>on      →  4.47%</p>\n\n<p>off     →  4.31%</p>\n\n<p>stop    →  3.85%</p>\n\n<p>go      →  4.38%</p>\n\n<p>silence →  7.29%</p>\n\n<p>unknown → 50.78%</p>\n\n<p>So Unknown is about 50%, silence above average and the rest around 4%. So there could be a major shake-up if some of the top submissions favour the known words right now. </p>\n\n<p>In theory you could not include any \"unknow\" labels in your submission and score 0.90 right now and once the you are tested against the whole set drop back to 0.50 :) </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 267771,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-01-12T08:50:10.320000",
          "content": "<p>We tested against our validation set. We can get both good results for subset ( equal distribution) or full set( unknown distribution is greater than 50%)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 267774,
          "author_name": "Peter",
          "author_url": "",
          "post_date": "2018-01-12T09:02:51.083000",
          "content": "<p>That seems indeed a good strategy. Out of interest, do you see higher validation scores lead to higher LB scores, or is this correlation not there once you get into the high 0.8x ?</p>\n\n<p>For this competition I use the validation score as a method to guard against overfitting, however there is no direct link to LB score. So a validation score can be higher while the LB score is lower.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 265872,
      "author_name": "kirk",
      "author_url": "",
      "post_date": "2018-01-07T00:04:25.490000",
      "content": "<p>Hi Heng, thanks for sharing. I just have a quick question. Is there a particular reason for not sharing some specific parts. For instance the Sampler class is missing, as well as some other functionality coming from files such as dataset.sampler,  common, utility, etc. ?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 265857,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-01-06T22:18:28.567000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 265672,
      "author_name": "Vadim Borisov",
      "author_url": "",
      "post_date": "2018-01-06T09:05:45.517000",
      "content": "<p>thank you!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 269134,
      "author_name": "Srikanth Reddy Metlakunta",
      "author_url": "",
      "post_date": "2018-01-16T09:31:39.197000",
      "content": "<p>Thanks for the post</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 268799,
      "author_name": "sisyphusishappy",
      "author_url": "",
      "post_date": "2018-01-15T17:22:16.427000",
      "content": "<p>Thanks.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 267633,
      "author_name": "Rishil Jacob",
      "author_url": "",
      "post_date": "2018-01-11T23:07:31.613000",
      "content": "<p>Thank you! This is really helpful</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 266673,
      "author_name": "Bo Ju",
      "author_url": "",
      "post_date": "2018-01-09T11:56:35.510000",
      "content": "<p>Thanks bro.</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "265667": "Please refer to the attachment PPTX for details.",
    "269128": "@Shitian Ni\n\nfrom your notebook torch.ipynb, i convert to main.py below. it can run fine on my system. I attached the log file, which shows the results:\n\n    ** start training here! **\n    optimizer=",
    "266602": "Thank you for sharing. I am new to pytorch, and have several issues running the code.\n\nI got \n\n    RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1512378360668/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243\nwhen using Cnn_Trad_Pool2_Net. I want to know how 26624 in the final Linear layer is calculated.\nI tried using other networks, but the accuracies are not improving, \nlogs look like\n\n    0.0050  174.0 k  436.00  22.3 | 2.9176  0.1000 | 2.4834  0.0930 | 2.4858  0.0938 | 12 hr 48 min  174121,174121, torch.Size([128, 1, 40, 101])\n\nfor running vggnet.\nWhat can be possibly wrong?\nThank you very much.",
    "266469": "Hello\n\nI am trying to set up a Jupyter notebook from your PyCharm files ; yet work to be done , got head on collision with many errors .  any suggestions ? ",
    "266353": "@Heng CherKeng\n\nCan you explain how (-1) is possible as index in  Dataset? I guess it's because custom collate function ?\n\nSo Im trying to implement dataloader in pytorch, that on every epoch returns only subset of \"unknowns\". But I can't understand how it's done in you code. ",
    "266122": "Almost everything is working with provided files, however, just tiny little functions are missing. It should be trivial to re-implement but for the sake of completeness, Heng CherKeng could your provide `top_accuracy(), adjust_learning_rate(), and get_learning_rate()`, the only missing functions? :)",
    "266068": "Hi CherKeng,\nI tried to run the code but got this error message even I installed the module 'utility'. Could you or anyone help? Thanks.\n\nTraceback (most recent call last):\n\n  File \"~/PyTorch/train_cnn_trad_pool2_net.py\", line 7, in ",
    "265702": " Hi Heng, glad you are posting in this competition!\n\n\"Silence.wav\" is it zero wav file? ",
    "266150": "@Sergey Mushinskiy\n\nThe files as requested.",
    "266026": "data split and other files as requested  ",
    "269810": "Thank you Lots of great tips how to setup the code &amp; Congratulations!",
    "267365": "@Heng CherKeng，\n\nYour code is great. I am new to pytorch. It is a great example. I notice you have great graphs in your report. I wonder whether it is a built-in function in pytorch like tensorboard in tensorflow, or you created your self. Could you please share some information about this? Thanks!",
    "266677": "Nice",
    "266046": "I was wondering, did you use GPU to train the model? If yes, how many GPUs did you use?",
    "266022": "Thanks CherKeng for the great sharing.",
    "265928": "@JohnsonTang\nHere is my file.  I think they are the same file\n",
    "265915": "@kirk . No. There are no special reasons. My whole project files is large and I have just upload the most important files to illustrate the method. You should be able to reproduce the results with my files. \n\nThe missing files are less important, e.g. for data read/write, etc . Some of the import (e.g.  dataset.sampler) are not used at all.\n\nNote that the intend of this post is not to encourage kagglers to download scripts, run and make submission. Rather kagglers should take my files as reference and modify or \"debug\" their existing code, method.\n\nIf there is anything unclear, please ask again in this post. I added two more files here:\n\n 1.  submit.py to make submission\n 2. common.py which is  a list of packages i used (but not all are necessary)\n\n",
    "265872": "Hi Heng, thanks for sharing. I just have a quick question. Is there a particular reason for not sharing some specific parts. For instance the Sampler class is missing, as well as some other functionality coming from files such as dataset.sampler,  common, utility, etc. ?",
    "265857": "",
    "265672": "thank you!",
    "269134": "Thanks for the post",
    "268799": "Thanks.",
    "267633": "Thank you! This is really helpful",
    "266673": "Thanks bro."
  }
}