This is my implementation of @cdeotte WaveNet Starter - [LB 0.52]. I tried to resemble it as closely as possible.
\nYou can find the code here:
\n\nThis notebook has a little better CV and public LB scores. It achieves 0.77 in CV score and 0.50 score in public LB.
\nHope you like it 👍🏼
", "messages": [ { "id": "2655513", "postDate": "02/17/2024 00:53:51", "content": "This is my implementation of @cdeotte WaveNet Starter - [LB 0.52]. I tried to resemble it as closely as possible.
\nYou can find the code here:
\n\nThis notebook has a little better CV and public LB scores. It achieves 0.77 in CV score and 0.50 score in public LB.
\nHope you like it 👍🏼
", "rawMarkdown": "This is my implementation of @cdeotte [WaveNet Starter - [LB 0.52]](https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52). I tried to resemble it as closely as possible.\n\nYou can find the code here:\n\n- [HMS | WaveNet PyTorch [Train]](https://www.kaggle.com/code/alejopaullier/hms-wavenet-pytorch-train)\n- [HMS | WaveNet PyTorch [Inference]](https://www.kaggle.com/code/alejopaullier/hms-wavenet-pytorch-inference)\n\nThis notebook has a little better CV and public LB scores. It achieves 0.77 in CV score and 0.50 score in public LB.\n\nHope you like it 👍🏼", "votes": null }, { "id": "2655549", "postDate": "02/17/2024 01:59:29", "content": "Great work! I was just going to look into doing a similar thing!
", "rawMarkdown": "Great work! I was just going to look into doing a similar thing!", "votes": null }, { "id": "2655558", "postDate": "02/17/2024 02:35:17", "content": "Thanks @cody11null I hope I saved you some time :)
", "rawMarkdown": "Thanks @cody11null I hope I saved you some time :)", "votes": null }, { "id": "2655560", "postDate": "02/17/2024 02:45:49", "content": "you are awesome .. I was feeling so lazy to do it .. You saved a lot of my time ..
", "rawMarkdown": "you are awesome .. I was feeling so lazy to do it .. You saved a lot of my time ..", "votes": null }, { "id": "2655563", "postDate": "02/17/2024 02:57:18", "content": "Thanks for your kind words, I am glad people find the code useful
", "rawMarkdown": "Thanks for your kind words, I am glad people find the code useful", "votes": null }, { "id": "2655807", "postDate": "02/17/2024 08:04:10", "content": "Amazing! Great work!
", "rawMarkdown": "Amazing! Great work!", "votes": null }, { "id": "2655936", "postDate": "02/17/2024 10:07:47", "content": "Great you are awesome brilliant work and helped us too much
", "rawMarkdown": "Great you are awesome brilliant work and helped us too much", "votes": null }, { "id": "2656523", "postDate": "02/17/2024 19:19:24", "content": "Great work! Saves a lot of time
", "rawMarkdown": "Great work! Saves a lot of time", "votes": null }, { "id": "2666190", "postDate": "02/24/2024 07:52:04", "content": "Thanks a lot!
", "rawMarkdown": "Thanks a lot!", "votes": null }, { "id": "2727770", "postDate": "04/01/2024 23:06:10", "content": "Thank you for this! We are new to this and have some confusion about why it would achieve so much lower LB score than CV score (0.77 vs 0.50). Are they not both measuring KL divergence -- one on the validation set and one on the test set (which is hidden by Kaggle)? In which case we would expect them to be similar since the model wasn't trained on either of those sets right?
", "rawMarkdown": "Thank you for this! We are new to this and have some confusion about why it would achieve so much lower LB score than CV score (0.77 vs 0.50). Are they not both measuring KL divergence -- one on the validation set and one on the test set (which is hidden by Kaggle)? In which case we would expect them to be similar since the model wasn't trained on either of those sets right?", "votes": null }, { "id": "2727790", "postDate": "04/01/2024 23:30:03", "content": "Hi. See discussion here
", "rawMarkdown": "Hi. See discussion [here][1]\n\n[1]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/485697", "votes": null }, { "id": "2727815", "postDate": "04/02/2024 00:13:19", "content": "Thanks for the quick response! Can I check if I understand correctly?
\nThe training data from kaggle seems to have merged two training sets, one with a large number of labelers and one small. The test set on the other hand seems to only have samples from the set with the large number of labelers. (This has been inferred because when the training data is divided based on number of voters, models perform similarly on the high voter training set as on the hidden test set).
\nIn general the high-voter data is easier to learn than the low voter data (ie models generally perform better on the high-voter data). As a result leaderboard scores tend to be higher than scores on randomly sampled validation data.
\nIs that the right conclusion?
", "rawMarkdown": "Thanks for the quick response! Can I check if I understand correctly?\n\nThe training data from kaggle seems to have merged two training sets, one with a large number of labelers and one small. The test set on the other hand seems to only have samples from the set with the large number of labelers. (This has been inferred because when the training data is divided based on number of voters, models perform similarly on the high voter training set as on the hidden test set).\n\nIn general the high-voter data is easier to learn than the low voter data (ie models generally perform better on the high-voter data). As a result leaderboard scores tend to be higher than scores on randomly sampled validation data.\n\nIs that the right conclusion?", "votes": null }, { "id": "2727825", "postDate": "04/02/2024 00:24:10", "content": "Yes your understanding is correct.
\nAnd when we compute CV scores, it appears that CV score computed from train data subset votes>=10 correlates better with public LB score. And there is less difference between CV score and LB score when doing so.
(And the original discussion post here computes CV score from all data which explains the large CV score LB score gap).
Got it. Thanks so much!
", "rawMarkdown": "Got it. Thanks so much!", "votes": null } ], "comments": [ { "id": 2655549, "author_name": "cody11null", "author_url": "", "post_date": "02/17/2024 01:59:29", "content": "Great work! I was just going to look into doing a similar thing!
", "votes": null, "replies": [ { "id": 2655558, "author_name": "alejopaullier", "author_url": "", "post_date": "02/17/2024 02:35:17", "content": "Thanks @cody11null I hope I saved you some time :)
", "votes": null, "replies": [] } ] }, { "id": 2655560, "author_name": "phoenix9032", "author_url": "", "post_date": "02/17/2024 02:45:49", "content": "you are awesome .. I was feeling so lazy to do it .. You saved a lot of my time ..
", "votes": null, "replies": [ { "id": 2655563, "author_name": "alejopaullier", "author_url": "", "post_date": "02/17/2024 02:57:18", "content": "Thanks for your kind words, I am glad people find the code useful
", "votes": null, "replies": [] } ] }, { "id": 2655807, "author_name": "aidlre001", "author_url": "", "post_date": "02/17/2024 08:04:10", "content": "Amazing! Great work!
", "votes": null, "replies": [] }, { "id": 2655936, "author_name": "tanishqdublish", "author_url": "", "post_date": "02/17/2024 10:07:47", "content": "Great you are awesome brilliant work and helped us too much
", "votes": null, "replies": [] }, { "id": 2656523, "author_name": "dabumts", "author_url": "", "post_date": "02/17/2024 19:19:24", "content": "Great work! Saves a lot of time
", "votes": null, "replies": [] }, { "id": 2666190, "author_name": "jasperpieterse", "author_url": "", "post_date": "02/24/2024 07:52:04", "content": "Thanks a lot!
", "votes": null, "replies": [] }, { "id": 2727770, "author_name": "karinana", "author_url": "", "post_date": "04/01/2024 23:06:10", "content": "Thank you for this! We are new to this and have some confusion about why it would achieve so much lower LB score than CV score (0.77 vs 0.50). Are they not both measuring KL divergence -- one on the validation set and one on the test set (which is hidden by Kaggle)? In which case we would expect them to be similar since the model wasn't trained on either of those sets right?
", "votes": null, "replies": [ { "id": 2727790, "author_name": "cdeotte", "author_url": "", "post_date": "04/01/2024 23:30:03", "content": "Hi. See discussion here
", "votes": null, "replies": [ { "id": 2727815, "author_name": "karinana", "author_url": "", "post_date": "04/02/2024 00:13:19", "content": "Thanks for the quick response! Can I check if I understand correctly?
\nThe training data from kaggle seems to have merged two training sets, one with a large number of labelers and one small. The test set on the other hand seems to only have samples from the set with the large number of labelers. (This has been inferred because when the training data is divided based on number of voters, models perform similarly on the high voter training set as on the hidden test set).
\nIn general the high-voter data is easier to learn than the low voter data (ie models generally perform better on the high-voter data). As a result leaderboard scores tend to be higher than scores on randomly sampled validation data.
\nIs that the right conclusion?
", "votes": null, "replies": [ { "id": 2727825, "author_name": "cdeotte", "author_url": "", "post_date": "04/02/2024 00:24:10", "content": "Yes your understanding is correct.
\nAnd when we compute CV scores, it appears that CV score computed from train data subset votes>=10 correlates better with public LB score. And there is less difference between CV score and LB score when doing so.
(And the original discussion post here computes CV score from all data which explains the large CV score LB score gap).
Got it. Thanks so much!
", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2655513": "This is my implementation of @cdeotte [WaveNet Starter - [LB 0.52]](https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52). I tried to resemble it as closely as possible.\n\nYou can find the code here:\n\n- [HMS | WaveNet PyTorch [Train]](https://www.kaggle.com/code/alejopaullier/hms-wavenet-pytorch-train)\n- [HMS | WaveNet PyTorch [Inference]](https://www.kaggle.com/code/alejopaullier/hms-wavenet-pytorch-inference)\n\nThis notebook has a little better CV and public LB scores. It achieves 0.77 in CV score and 0.50 score in public LB.\n\nHope you like it 👍🏼", "2655549": "Great work! I was just going to look into doing a similar thing!", "2655558": "Thanks @cody11null I hope I saved you some time :)", "2655560": "you are awesome .. I was feeling so lazy to do it .. You saved a lot of my time ..", "2655563": "Thanks for your kind words, I am glad people find the code useful", "2655807": "Amazing! Great work!", "2655936": "Great you are awesome brilliant work and helped us too much", "2656523": "Great work! Saves a lot of time", "2666190": "Thanks a lot!", "2727770": "Thank you for this! We are new to this and have some confusion about why it would achieve so much lower LB score than CV score (0.77 vs 0.50). Are they not both measuring KL divergence -- one on the validation set and one on the test set (which is hidden by Kaggle)? In which case we would expect them to be similar since the model wasn't trained on either of those sets right?", "2727790": "Hi. See discussion [here][1]\n\n[1]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/485697", "2727815": "Thanks for the quick response! Can I check if I understand correctly?\n\nThe training data from kaggle seems to have merged two training sets, one with a large number of labelers and one small. The test set on the other hand seems to only have samples from the set with the large number of labelers. (This has been inferred because when the training data is divided based on number of voters, models perform similarly on the high voter training set as on the hidden test set).\n\nIn general the high-voter data is easier to learn than the low voter data (ie models generally perform better on the high-voter data). As a result leaderboard scores tend to be higher than scores on randomly sampled validation data.\n\nIs that the right conclusion?", "2727825": "Yes your understanding is correct. \n\nAnd when we compute CV scores, it appears that CV score computed from train data subset `votes>=10` correlates better with public LB score. And there is less difference between CV score and LB score when doing so.\n\n(And the original discussion post here computes CV score from `all data` which explains the large CV score LB score gap).", "2727828": "Got it. Thanks so much!" }, "source": "meta" }