{
  "id": 89585,
  "title": "Is your LB score made by model which is trained in only-kernel?",
  "url": "/competitions/freesound-audio-tagging-2019/discussion/89585",
  "author_name": "",
  "post_date": "2019-04-16T02:16:26.482839600Z",
  "votes": 4,
  "comment_count": 18,
  "views": 0,
  "content": "<p>My current score is made by model, trained in only-kernel in 1hour.</p>\n\n<p>How about yours??</p>\n\n<p>If you don't want to share, just skip :) </p>\n\n<p>Hope you get good scores, thanks.</p>",
  "messages": [
    {
      "id": "517420",
      "postDate": "04/16/2019 02:16:26",
      "content": "<p>My current score is made by model, trained in only-kernel in 1hour.</p>\n\n<p>How about yours??</p>\n\n<p>If you don't want to share, just skip :) </p>\n\n<p>Hope you get good scores, thanks.</p>",
      "rawMarkdown": "My current score is made by model, trained in only-kernel in 1hour.\n\nHow about yours??\n\nIf you don't want to share, just skip :) \n\nHope you get good scores, thanks.",
      "votes": null
    },
    {
      "id": "517440",
      "postDate": "04/16/2019 03:10:03",
      "content": "<p>Mine is only-kernel. I guess we will be seeing better LB score range in next couple of weeks...</p>",
      "rawMarkdown": "Mine is only-kernel. I guess we will be seeing better LB score range in next couple of weeks...",
      "votes": null
    },
    {
      "id": "517448",
      "postDate": "04/16/2019 03:31:19",
      "content": "<p>I think so. Masters are coming! </p>\n\n<p>I have a question. Did you train your model several times in kernel or just one time?</p>",
      "rawMarkdown": "I think so. Masters are coming! \n\nI have a question. Did you train your model several times in kernel or just one time?",
      "votes": null
    },
    {
      "id": "517471",
      "postDate": "04/16/2019 04:13:12",
      "content": "<p>My score is also in kernel but given that it is right at 1 hour, I will move training to a separate kernel.</p>",
      "rawMarkdown": "My score is also in kernel but given that it is right at 1 hour, I will move training to a separate kernel.",
      "votes": null
    },
    {
      "id": "517475",
      "postDate": "04/16/2019 04:21:13",
      "content": "<p>Hi, it's single model, and run <code>learn.fit_one_cycle(epochs, lr)</code> several times (using fast.ai), but a single run of kernel of the last created weight. I should start saving best weight in the next kernel...</p>",
      "rawMarkdown": "Hi, it's single model, and run `learn.fit_one_cycle(epochs, lr)` several times (using fast.ai), but a single run of kernel of the last created weight. I should start saving best weight in the next kernel...",
      "votes": null
    },
    {
      "id": "517517",
      "postDate": "04/16/2019 06:04:33",
      "content": "<p>I am very surprised to see such scores with single models. I am using an ensemble of models trained (some of them inside kernels and some on external GPU).  In a nutshell I perform only inference in the kernels and strictly keep time &lt; 20 minutes. So it is ensemble of 5 models for me no training included no noisy data yet... </p>\n\n<p>Just to note that I 've used only curated data to train my models.  Eventually it all boils down to this: If let's say we need ~1 minute per model to inference the ~1100 files of test set, then we have time to create the best enseble of 20 models....</p>",
      "rawMarkdown": "I am very surprised to see such scores with single models. I am using an ensemble of models trained (some of them inside kernels and some on external GPU).  In a nutshell I perform only inference in the kernels and strictly keep time &lt; 20 minutes. So it is ensemble of 5 models for me no training included no noisy data yet... \n\nJust to note that I 've used only curated data to train my models.  Eventually it all boils down to this: If let's say we need ~1 minute per model to inference the ~1100 files of test set, then we have time to create the best enseble of 20 models....",
      "votes": null
    },
    {
      "id": "517759",
      "postDate": "04/16/2019 13:33:52",
      "content": "<p>This is also what I do, training+testing in a single kernel, in less than 1hour. But if people start to train their model offline, it will be difficult to remain at a high place in this competition.</p>\n\n<p>Good luck to you ! </p>",
      "rawMarkdown": "This is also what I do, training+testing in a single kernel, in less than 1hour. But if people start to train their model offline, it will be difficult to remain at a high place in this competition.\n\nGood luck to you !",
      "votes": null
    },
    {
      "id": "518261",
      "postDate": "04/17/2019 02:45:53",
      "content": "<p>Oh, very helpful tips. :) \nI hope you will get a good score in this competition! Thanks!</p>",
      "rawMarkdown": "Oh, very helpful tips. :) \nI hope you will get a good score in this competition! Thanks!",
      "votes": null
    },
    {
      "id": "518263",
      "postDate": "04/17/2019 02:46:26",
      "content": "<p>Good result! Hope get a good result!</p>",
      "rawMarkdown": "Good result! Hope get a good result!",
      "votes": null
    },
    {
      "id": "518264",
      "postDate": "04/17/2019 02:47:03",
      "content": "<p>Yes, I think so. I need to make more models to get a high rank! \nThanks!</p>",
      "rawMarkdown": "Yes, I think so. I need to make more models to get a high rank! \nThanks!",
      "votes": null
    },
    {
      "id": "518274",
      "postDate": "04/17/2019 03:22:35",
      "content": "<p>Hi, I have to admit that it got worse if I make it keeping best validation weight for predicting tests...\nI think this would be caused by noise, one of key difficulties to solve.\nUmmm... over LB 0.6 would be the range to suffer label (&amp; maybe audio) noise.</p>",
      "rawMarkdown": "Hi, I have to admit that it got worse if I make it keeping best validation weight for predicting tests...\nI think this would be caused by noise, one of key difficulties to solve.\nUmmm... over LB 0.6 would be the range to suffer label (&amp; maybe audio) noise.",
      "votes": null
    },
    {
      "id": "518488",
      "postDate": "04/17/2019 11:26:48",
      "content": "<p>Hi, you gave us a solution! Thanks. \nI'm also using curated data only. I will do stacking.\nI have a question. Do you use your own models (not famous series, resnet, vgg, etc.)?</p>\n\n<p>Currently, I'm using my own models.</p>",
      "rawMarkdown": "Hi, you gave us a solution! Thanks. \nI'm also using curated data only. I will do stacking.\nI have a question. Do you use your own models (not famous series, resnet, vgg, etc.)?\n\nCurrently, I'm using my own models.",
      "votes": null
    },
    {
      "id": "518690",
      "postDate": "04/17/2019 15:57:02",
      "content": "<p>Everything plays as long as is not pretrained.... \"Famous\" models have huge capacity and need to be trained carefully...</p>",
      "rawMarkdown": "Everything plays as long as is not pretrained.... \"Famous\" models have huge capacity and need to be trained carefully...",
      "votes": null
    },
    {
      "id": "518825",
      "postDate": "04/17/2019 22:11:06",
      "content": "<p>I've got leaked cv 0.94 resnet model just to check how it works, then it marked LB 0.55.\nConfirmed to have too much capacity to remember most of training samples...</p>",
      "rawMarkdown": "I've got leaked cv 0.94 resnet model just to check how it works, then it marked LB 0.55.\nConfirmed to have too much capacity to remember most of training samples...",
      "votes": null
    },
    {
      "id": "518865",
      "postDate": "04/18/2019 01:07:47",
      "content": "<p>@Vogils, daisukelab \nThanks for sharing your experience.\nI've tested my own model and pretrained model(not using pretrained weights, just using structure). \nAnd, I got a result that resnet series were too big and showed less performance than my own model. \nMaybe, I think mel spectrogram is somewhat different from the traditional picture.\nFinally, I've decided to use hand-made model :)</p>",
      "rawMarkdown": "Vogils, daisukelab \nThanks for sharing your experience.\nI've tested my own model and pretrained model(not using pretrained weights, just using structure). \nAnd, I got a result that resnet series were too big and showed less performance than my own model. \nMaybe, I think mel spectrogram is somewhat different from the traditional picture.\nFinally, I've decided to use hand-made model :)",
      "votes": null
    },
    {
      "id": "522836",
      "postDate": "04/25/2019 05:34:14",
      "content": "<p>Yes my best LB score is from models trained in kernels only + inference in kernel.</p>",
      "rawMarkdown": "Yes my best LB score is from models trained in kernels only + inference in kernel.",
      "votes": null
    },
    {
      "id": "527395",
      "postDate": "05/05/2019 10:50:31",
      "content": "<p>My best single model with LB 0.650, trained in kernel less than 1 hour, but prepared data in another kernel</p>",
      "rawMarkdown": "My best single model with LB 0.650, trained in kernel less than 1 hour, but prepared data in another kernel",
      "votes": null
    },
    {
      "id": "540659",
      "postDate": "05/31/2019 19:59:33",
      "content": "<p>I also got 0.93 local lwlrap, though with LB at 0.51, single model trained in kernel.\n<a href=\"/daisukelab\">@daisukelab</a>, you said you had a \"leaked resnet model.\" What was leaking, and how did you solve it? I may have the same problem... </p>",
      "rawMarkdown": "I also got 0.93 local lwlrap, though with LB at 0.51, single model trained in kernel.\n@daisukelab, you said you had a \"leaked resnet model.\" What was leaking, and how did you solve it? I may have the same problem...",
      "votes": null
    },
    {
      "id": "540703",
      "postDate": "05/31/2019 22:16:57",
      "content": "<p><a href=\"/nswitanek\">@nswitanek</a>, in this competition, it's important to properly split training samples. This basic step would solve the issue. sklearn's <code>train_test_split</code> would help you I guess... hope it works for you.</p>",
      "rawMarkdown": "nswitanek, in this competition, it's important to properly split training samples. This basic step would solve the issue. sklearn's `train_test_split` would help you I guess... hope it works for you.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 517440,
      "author_name": "daisukelab",
      "author_url": "",
      "post_date": "04/16/2019 03:10:03",
      "content": "<p>Mine is only-kernel. I guess we will be seeing better LB score range in next couple of weeks...</p>",
      "votes": null,
      "replies": [
        {
          "id": 517448,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "04/16/2019 03:31:19",
          "content": "<p>I think so. Masters are coming! </p>\n\n<p>I have a question. Did you train your model several times in kernel or just one time?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 517475,
          "author_name": "daisukelab",
          "author_url": "",
          "post_date": "04/16/2019 04:21:13",
          "content": "<p>Hi, it's single model, and run <code>learn.fit_one_cycle(epochs, lr)</code> several times (using fast.ai), but a single run of kernel of the last created weight. I should start saving best weight in the next kernel...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 518261,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "04/17/2019 02:45:53",
          "content": "<p>Oh, very helpful tips. :) \nI hope you will get a good score in this competition! Thanks!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 518274,
          "author_name": "daisukelab",
          "author_url": "",
          "post_date": "04/17/2019 03:22:35",
          "content": "<p>Hi, I have to admit that it got worse if I make it keeping best validation weight for predicting tests...\nI think this would be caused by noise, one of key difficulties to solve.\nUmmm... over LB 0.6 would be the range to suffer label (&amp; maybe audio) noise.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 517471,
      "author_name": "tanlikesmath",
      "author_url": "",
      "post_date": "04/16/2019 04:13:12",
      "content": "<p>My score is also in kernel but given that it is right at 1 hour, I will move training to a separate kernel.</p>",
      "votes": null,
      "replies": [
        {
          "id": 518263,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "04/17/2019 02:46:26",
          "content": "<p>Good result! Hope get a good result!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 517517,
      "author_name": "voglinio",
      "author_url": "",
      "post_date": "04/16/2019 06:04:33",
      "content": "<p>I am very surprised to see such scores with single models. I am using an ensemble of models trained (some of them inside kernels and some on external GPU).  In a nutshell I perform only inference in the kernels and strictly keep time &lt; 20 minutes. So it is ensemble of 5 models for me no training included no noisy data yet... </p>\n\n<p>Just to note that I 've used only curated data to train my models.  Eventually it all boils down to this: If let's say we need ~1 minute per model to inference the ~1100 files of test set, then we have time to create the best enseble of 20 models....</p>",
      "votes": null,
      "replies": [
        {
          "id": 518488,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "04/17/2019 11:26:48",
          "content": "<p>Hi, you gave us a solution! Thanks. \nI'm also using curated data only. I will do stacking.\nI have a question. Do you use your own models (not famous series, resnet, vgg, etc.)?</p>\n\n<p>Currently, I'm using my own models.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 518690,
          "author_name": "voglinio",
          "author_url": "",
          "post_date": "04/17/2019 15:57:02",
          "content": "<p>Everything plays as long as is not pretrained.... \"Famous\" models have huge capacity and need to be trained carefully...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 518825,
          "author_name": "daisukelab",
          "author_url": "",
          "post_date": "04/17/2019 22:11:06",
          "content": "<p>I've got leaked cv 0.94 resnet model just to check how it works, then it marked LB 0.55.\nConfirmed to have too much capacity to remember most of training samples...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 518865,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "04/18/2019 01:07:47",
          "content": "<p>@Vogils, daisukelab \nThanks for sharing your experience.\nI've tested my own model and pretrained model(not using pretrained weights, just using structure). \nAnd, I got a result that resnet series were too big and showed less performance than my own model. \nMaybe, I think mel spectrogram is somewhat different from the traditional picture.\nFinally, I've decided to use hand-made model :)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 540659,
          "author_name": "nswitanek",
          "author_url": "",
          "post_date": "05/31/2019 19:59:33",
          "content": "<p>I also got 0.93 local lwlrap, though with LB at 0.51, single model trained in kernel.\n<a href=\"/daisukelab\">@daisukelab</a>, you said you had a \"leaked resnet model.\" What was leaking, and how did you solve it? I may have the same problem... </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 540703,
          "author_name": "daisukelab",
          "author_url": "",
          "post_date": "05/31/2019 22:16:57",
          "content": "<p><a href=\"/nswitanek\">@nswitanek</a>, in this competition, it's important to properly split training samples. This basic step would solve the issue. sklearn's <code>train_test_split</code> would help you I guess... hope it works for you.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 517759,
      "author_name": "nathanh12",
      "author_url": "",
      "post_date": "04/16/2019 13:33:52",
      "content": "<p>This is also what I do, training+testing in a single kernel, in less than 1hour. But if people start to train their model offline, it will be difficult to remain at a high place in this competition.</p>\n\n<p>Good luck to you ! </p>",
      "votes": null,
      "replies": [
        {
          "id": 518264,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "04/17/2019 02:47:03",
          "content": "<p>Yes, I think so. I need to make more models to get a high rank! \nThanks!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 522836,
      "author_name": "jamesrequa",
      "author_url": "",
      "post_date": "04/25/2019 05:34:14",
      "content": "<p>Yes my best LB score is from models trained in kernels only + inference in kernel.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 527395,
      "author_name": "lftuwujie",
      "author_url": "",
      "post_date": "05/05/2019 10:50:31",
      "content": "<p>My best single model with LB 0.650, trained in kernel less than 1 hour, but prepared data in another kernel</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "517420": "My current score is made by model, trained in only-kernel in 1hour.\n\nHow about yours??\n\nIf you don't want to share, just skip :) \n\nHope you get good scores, thanks.",
    "517440": "Mine is only-kernel. I guess we will be seeing better LB score range in next couple of weeks...",
    "517448": "I think so. Masters are coming! \n\nI have a question. Did you train your model several times in kernel or just one time?",
    "517471": "My score is also in kernel but given that it is right at 1 hour, I will move training to a separate kernel.",
    "517475": "Hi, it's single model, and run `learn.fit_one_cycle(epochs, lr)` several times (using fast.ai), but a single run of kernel of the last created weight. I should start saving best weight in the next kernel...",
    "517517": "I am very surprised to see such scores with single models. I am using an ensemble of models trained (some of them inside kernels and some on external GPU).  In a nutshell I perform only inference in the kernels and strictly keep time &lt; 20 minutes. So it is ensemble of 5 models for me no training included no noisy data yet... \n\nJust to note that I 've used only curated data to train my models.  Eventually it all boils down to this: If let's say we need ~1 minute per model to inference the ~1100 files of test set, then we have time to create the best enseble of 20 models....",
    "517759": "This is also what I do, training+testing in a single kernel, in less than 1hour. But if people start to train their model offline, it will be difficult to remain at a high place in this competition.\n\nGood luck to you !",
    "518261": "Oh, very helpful tips. :) \nI hope you will get a good score in this competition! Thanks!",
    "518263": "Good result! Hope get a good result!",
    "518264": "Yes, I think so. I need to make more models to get a high rank! \nThanks!",
    "518274": "Hi, I have to admit that it got worse if I make it keeping best validation weight for predicting tests...\nI think this would be caused by noise, one of key difficulties to solve.\nUmmm... over LB 0.6 would be the range to suffer label (&amp; maybe audio) noise.",
    "518488": "Hi, you gave us a solution! Thanks. \nI'm also using curated data only. I will do stacking.\nI have a question. Do you use your own models (not famous series, resnet, vgg, etc.)?\n\nCurrently, I'm using my own models.",
    "518690": "Everything plays as long as is not pretrained.... \"Famous\" models have huge capacity and need to be trained carefully...",
    "518825": "I've got leaked cv 0.94 resnet model just to check how it works, then it marked LB 0.55.\nConfirmed to have too much capacity to remember most of training samples...",
    "518865": "Vogils, daisukelab \nThanks for sharing your experience.\nI've tested my own model and pretrained model(not using pretrained weights, just using structure). \nAnd, I got a result that resnet series were too big and showed less performance than my own model. \nMaybe, I think mel spectrogram is somewhat different from the traditional picture.\nFinally, I've decided to use hand-made model :)",
    "522836": "Yes my best LB score is from models trained in kernels only + inference in kernel.",
    "527395": "My best single model with LB 0.650, trained in kernel less than 1 hour, but prepared data in another kernel",
    "540659": "I also got 0.93 local lwlrap, though with LB at 0.51, single model trained in kernel.\n@daisukelab, you said you had a \"leaked resnet model.\" What was leaking, and how did you solve it? I may have the same problem...",
    "540703": "nswitanek, in this competition, it's important to properly split training samples. This basic step would solve the issue. sklearn's `train_test_split` would help you I guess... hope it works for you."
  },
  "source": "meta"
}