{
  "id": 211816,
  "title": "need some help",
  "url": "/competitions/rfcx-species-audio-detection/discussion/211816",
  "author_name": "",
  "post_date": "2021-01-16T12:02:26.423229100Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>hi guys in fold 0 i am not even able to reach 0.9+ score  but in fold5 as you can see i am getting very high score surely be overfitting or leakage  , i tried to find  any leakage but i didnt  able to find any  ,  i am doing stratified k fold with 6 splits , any help will be helpful to find the possible leakage .</p>\n<p>fold:5 epoch: 32 train_loss :0.016653743898771722 Comp Metric:0.9527245439217173<br>\nValid E:32 - Loss:0.0654: 100%<br>\n31/31 [00:06&lt;00:00, 4.72it/s]<br>\nfold:5 epoch: 32 Valid_loss :0.06542863238900289 Comp Metric:0.9422105672105673<br>\nTrain E:33 - Loss0.0120: 100%<br>\n171/171 [05:16&lt;00:00, 1.85s/it]<br>\nfold:5 epoch: 33 train_loss :0.011998938920466884 Comp Metric:0.95640672201787<br>\nValid E:33 - Loss:0.0591: 100%<br>\n31/31 [00:44&lt;00:00, 1.45s/it]<br>\nfold:5 epoch: 33 Valid_loss :0.05905768358736003 Comp Metric:0.9589531680440772<br>\nTrain E:34 - Loss0.0152: 100%<br>\n171/171 [02:33&lt;00:00, 1.11it/s]<br>\nfold:5 epoch: 34 train_loss :0.01518326856857044 Comp Metric:0.9508906271932299<br>\nValid E:34 - Loss:0.0307: 100%<br>\n31/31 [00:07&lt;00:00, 3.99it/s]<br>\nfold:5 epoch: 34 Valid_loss :0.03072895531114821 Comp Metric:0.9539944903581268</p>",
  "messages": [
    {
      "id": "1155455",
      "postDate": "01/16/2021 12:02:26",
      "content": "<p>hi guys in fold 0 i am not even able to reach 0.9+ score  but in fold5 as you can see i am getting very high score surely be overfitting or leakage  , i tried to find  any leakage but i didnt  able to find any  ,  i am doing stratified k fold with 6 splits , any help will be helpful to find the possible leakage .</p>\n<p>fold:5 epoch: 32 train_loss :0.016653743898771722 Comp Metric:0.9527245439217173<br>\nValid E:32 - Loss:0.0654: 100%<br>\n31/31 [00:06&lt;00:00, 4.72it/s]<br>\nfold:5 epoch: 32 Valid_loss :0.06542863238900289 Comp Metric:0.9422105672105673<br>\nTrain E:33 - Loss0.0120: 100%<br>\n171/171 [05:16&lt;00:00, 1.85s/it]<br>\nfold:5 epoch: 33 train_loss :0.011998938920466884 Comp Metric:0.95640672201787<br>\nValid E:33 - Loss:0.0591: 100%<br>\n31/31 [00:44&lt;00:00, 1.45s/it]<br>\nfold:5 epoch: 33 Valid_loss :0.05905768358736003 Comp Metric:0.9589531680440772<br>\nTrain E:34 - Loss0.0152: 100%<br>\n171/171 [02:33&lt;00:00, 1.11it/s]<br>\nfold:5 epoch: 34 train_loss :0.01518326856857044 Comp Metric:0.9508906271932299<br>\nValid E:34 - Loss:0.0307: 100%<br>\n31/31 [00:07&lt;00:00, 3.99it/s]<br>\nfold:5 epoch: 34 Valid_loss :0.03072895531114821 Comp Metric:0.9539944903581268</p>",
      "rawMarkdown": "hi guys in fold 0 i am not even able to reach 0.9+ score  but in fold5 as you can see i am getting very high score surely be overfitting or leakage  , i tried to find  any leakage but i didnt  able to find any  ,  i am doing stratified k fold with 6 splits , any help will be helpful to find the possible leakage .\n\nfold:5 epoch: 32 train_loss :0.016653743898771722 Comp Metric:0.9527245439217173\nValid E:32 - Loss:0.0654: 100%\n31/31 [00:06<00:00, 4.72it/s]\nfold:5 epoch: 32 Valid_loss :0.06542863238900289 Comp Metric:0.9422105672105673\nTrain E:33 - Loss0.0120: 100%\n171/171 [05:16<00:00, 1.85s/it]\nfold:5 epoch: 33 train_loss :0.011998938920466884 Comp Metric:0.95640672201787\nValid E:33 - Loss:0.0591: 100%\n31/31 [00:44<00:00, 1.45s/it]\nfold:5 epoch: 33 Valid_loss :0.05905768358736003 Comp Metric:0.9589531680440772\nTrain E:34 - Loss0.0152: 100%\n171/171 [02:33<00:00, 1.11it/s]\nfold:5 epoch: 34 train_loss :0.01518326856857044 Comp Metric:0.9508906271932299\nValid E:34 - Loss:0.0307: 100%\n31/31 [00:07<00:00, 3.99it/s]\nfold:5 epoch: 34 Valid_loss :0.03072895531114821 Comp Metric:0.9539944903581268",
      "votes": null
    },
    {
      "id": "1157415",
      "postDate": "01/17/2021 21:43:24",
      "content": "<p>We have a very limited number of positive instances per class. With a small dataset it's usual to have diverging results per fold. If you had genuine leakage, the loss would be approaching zero.</p>",
      "rawMarkdown": "We have a very limited number of positive instances per class. With a small dataset it's usual to have diverging results per fold. If you had genuine leakage, the loss would be approaching zero.",
      "votes": null
    },
    {
      "id": "1157437",
      "postDate": "01/17/2021 22:39:30",
      "content": "<p>hi <a href=\"https://www.kaggle.com/trooperog\" target=\"_blank\">@trooperog</a>, maybe it is the particular fold split, you can check the classes instances or try with less number of folds/other seeds and see if changes.. to be able to help you more we need to see the train loop though </p>",
      "rawMarkdown": "hi @trooperog, maybe it is the particular fold split, you can check the classes instances or try with less number of folds/other seeds and see if changes.. to be able to help you more we need to see the train loop though",
      "votes": null
    },
    {
      "id": "1157830",
      "postDate": "01/18/2021 07:25:44",
      "content": "<p>hmm this might be the case for the diverging results for different folds .</p>",
      "rawMarkdown": "hmm this might be the case for the diverging results for different folds .",
      "votes": null
    },
    {
      "id": "1157831",
      "postDate": "01/18/2021 07:26:31",
      "content": "<p>yeah i will try with different seeds and folds thnx for the help !</p>",
      "rawMarkdown": "yeah i will try with different seeds and folds thnx for the help !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1157415,
      "author_name": "bigironsphere",
      "author_url": "",
      "post_date": "01/17/2021 21:43:24",
      "content": "<p>We have a very limited number of positive instances per class. With a small dataset it's usual to have diverging results per fold. If you had genuine leakage, the loss would be approaching zero.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1157830,
          "author_name": "trooperog",
          "author_url": "",
          "post_date": "01/18/2021 07:25:44",
          "content": "<p>hmm this might be the case for the diverging results for different folds .</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1157437,
      "author_name": "imeintanis",
      "author_url": "",
      "post_date": "01/17/2021 22:39:30",
      "content": "<p>hi <a href=\"https://www.kaggle.com/trooperog\" target=\"_blank\">@trooperog</a>, maybe it is the particular fold split, you can check the classes instances or try with less number of folds/other seeds and see if changes.. to be able to help you more we need to see the train loop though </p>",
      "votes": null,
      "replies": [
        {
          "id": 1157831,
          "author_name": "trooperog",
          "author_url": "",
          "post_date": "01/18/2021 07:26:31",
          "content": "<p>yeah i will try with different seeds and folds thnx for the help !</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1155455": "hi guys in fold 0 i am not even able to reach 0.9+ score  but in fold5 as you can see i am getting very high score surely be overfitting or leakage  , i tried to find  any leakage but i didnt  able to find any  ,  i am doing stratified k fold with 6 splits , any help will be helpful to find the possible leakage .\n\nfold:5 epoch: 32 train_loss :0.016653743898771722 Comp Metric:0.9527245439217173\nValid E:32 - Loss:0.0654: 100%\n31/31 [00:06<00:00, 4.72it/s]\nfold:5 epoch: 32 Valid_loss :0.06542863238900289 Comp Metric:0.9422105672105673\nTrain E:33 - Loss0.0120: 100%\n171/171 [05:16<00:00, 1.85s/it]\nfold:5 epoch: 33 train_loss :0.011998938920466884 Comp Metric:0.95640672201787\nValid E:33 - Loss:0.0591: 100%\n31/31 [00:44<00:00, 1.45s/it]\nfold:5 epoch: 33 Valid_loss :0.05905768358736003 Comp Metric:0.9589531680440772\nTrain E:34 - Loss0.0152: 100%\n171/171 [02:33<00:00, 1.11it/s]\nfold:5 epoch: 34 train_loss :0.01518326856857044 Comp Metric:0.9508906271932299\nValid E:34 - Loss:0.0307: 100%\n31/31 [00:07<00:00, 3.99it/s]\nfold:5 epoch: 34 Valid_loss :0.03072895531114821 Comp Metric:0.9539944903581268",
    "1157415": "We have a very limited number of positive instances per class. With a small dataset it's usual to have diverging results per fold. If you had genuine leakage, the loss would be approaching zero.",
    "1157437": "hi @trooperog, maybe it is the particular fold split, you can check the classes instances or try with less number of folds/other seeds and see if changes.. to be able to help you more we need to see the train loop though",
    "1157830": "hmm this might be the case for the diverging results for different folds .",
    "1157831": "yeah i will try with different seeds and folds thnx for the help !"
  },
  "source": "meta"
}