![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2745910%2Fffc019d305437ad348358b191da41c31%2FScreen%20Shot%202020-12-23%20at%2011.14.25%20AM.png?generation=1608740135979369&alt=media\")
Today I stumbled upon this kernel:
\nhttps://www.kaggle.com/lanniegal/riiid-sakt-model-inference-public
Upon checking the difference:
\n
The result is shocking. Not only is this notebook a no effort fork from @manikanthr5 's modification from several sakt models with no version differences. The author also deleted all the references @manikanthr5 put up and tributes to the sakt's other contributors. This notebook getting upvote is an insult to the Kaggle sharing spirit to be honest.
\nAnother example is this one: https://www.kaggle.com/pandaman817/riiid-lgbm-bagging2-sakt-0-781
", "messages": [ { "id": 1123993, "postDate": "2020-12-23T16:19:03.823Z", "content": "Today I stumbled upon this kernel:
\nhttps://www.kaggle.com/lanniegal/riiid-sakt-model-inference-public
Upon checking the difference:
\n
The result is shocking. Not only is this notebook a no effort fork from @manikanthr5 's modification from several sakt models with no version differences. The author also deleted all the references @manikanthr5 put up and tributes to the sakt's other contributors. This notebook getting upvote is an insult to the Kaggle sharing spirit to be honest.
\nAnother example is this one: https://www.kaggle.com/pandaman817/riiid-lgbm-bagging2-sakt-0-781
", "rawMarkdown": "Today I stumbled upon this kernel:\nhttps://www.kaggle.com/lanniegal/riiid-sakt-model-inference-public\n\nUpon checking the difference:\n\n\n\nThe result is shocking. Not only is this notebook a no effort fork from @manikanthr5 's modification from several sakt models with no version differences. The author also deleted all the references @manikanthr5 put up and tributes to the sakt's other contributors. This notebook getting upvote is an insult to the Kaggle sharing spirit to be honest.\n\nAnother example is this one: https://www.kaggle.com/pandaman817/riiid-lgbm-bagging2-sakt-0-781", "votes": 29 }, { "id": 1125140, "postDate": "2020-12-24T12:13:21.047Z", "content": "Although I don't really mind, and I know this happens, my (very hard) effort to put this TPU notebook
\nTPU - Track knowledge states of 1M+ students
\nonly got 20 votes …
\nI believed, when it was published, if I included the submission (inference) pipeline and make it scored (LB 0.781 at the published time), it will get > 100 votes and forks.
\nI would rather to get fewer votes from the people really use it or at least get some ideas from my notebook, or even people not use it but appreciate my effort.
", "rawMarkdown": "Although I don't really mind, and I know this happens, my (very hard) effort to put this TPU notebook\n\n[TPU - Track knowledge states of 1M+ students](https://www.kaggle.com/yihdarshieh/tpu-track-knowledge-states-of-1m-students)\n\nonly got 20 votes ...\n\nI believed, when it was published, if I included the submission (inference) pipeline and make it scored (LB 0.781 at the published time), it will get > 100 votes and forks.\n\nI would rather to get fewer votes from the people really use it or at least get some ideas from my notebook, or even people not use it but appreciate my effort.", "votes": 14, "replies": [ { "id": 1125719, "postDate": "2020-12-25T02:19:55.677Z", "content": "Hi @yihdarshieh It's a great notebook. I would like to make one suggestion. Please go through the comment by Andrew Lukyanenko in this thread. Your notebook could be optimized so that beginners can understand. There was a lot of content and some part of it can be hidden (like warnings). I hope this helps for you.
", "rawMarkdown": "Hi @yihdarshieh It's a great notebook. I would like to make one suggestion. Please go through the comment by [Andrew Lukyanenko in this thread](https://www.kaggle.com/general/89512#517007). Your notebook could be optimized so that beginners can understand. There was a lot of content and some part of it can be hidden (like warnings). I hope this helps for you. ", "votes": 1 }, { "id": 1125974, "postDate": "2020-12-25T08:02:59.033Z", "content": "Thank you for the information. I will improve the style once the competition is finished. As I mentioned, it already took me quite a lot of time to have this current version - many options but not very well documented. After that version, I came back to work on my models …
", "rawMarkdown": "Thank you for the information. I will improve the style once the competition is finished. As I mentioned, it already took me quite a lot of time to have this current version - many options but not very well documented. After that version, I came back to work on my models ...", "votes": 1 }, { "id": 1125991, "postDate": "2020-12-25T08:14:42.533Z", "content": "@yihdarshieh Thanks for the notebook, I've referred to it quite a few times for understanding how to setup the TPU pipeline. Took me 5 days of continuous effort to get it up and running. Currently checking against old model architecture which ran on GPU as benchmark to ensure it's working properly. Speed-up vs GPU is 3.5~4x.
\nNow at least I can test 2-3 models in each of the remaining weeks.
Great to know it helps!
", "rawMarkdown": "Great to know it helps!", "votes": 1 }, { "id": 1126112, "postDate": "2020-12-25T10:51:16.243Z", "content": "@abdurrafae If you are ok to use Colab Pro, you can even test 3 - 4 models per day … However, setup on Colab might again take some time, so you need to consider what's the best for you, considering it remains only 2 weeks.
", "rawMarkdown": "@abdurrafae If you are ok to use Colab Pro, you can even test 3 - 4 models per day ... However, setup on Colab might again take some time, so you need to consider what's the best for you, considering it remains only 2 weeks." }, { "id": 1126121, "postDate": "2020-12-25T11:01:57.943Z", "content": "Thanks for the advice, I won't be able to use Colab pro since it's limited to US and Canada for now. I'll try on normal Colab session in a day or two, need a breather after working on this for 5 days. :D
\nJust a query, for comparing models do you let them run for a pre-determined epochs or use earlystopping?
\nI had been training for a constant num of epochs (26) and comparing models according to results achieved till then. I can see that the validation loss is still decreasing on the last 2-3 epochs.
Another advice needed: Do you scale learning rate (lr) when training on TPU vs GPU? I read that it helps to scale the lr when training on TPU. I tried up-scaling by a factor of Sqrt(Accelerators) but it leads to divergence.
", "rawMarkdown": "Thanks for the advice, I won't be able to use Colab pro since it's limited to US and Canada for now. I'll try on normal Colab session in a day or two, need a breather after working on this for 5 days. :D\n\nJust a query, for comparing models do you let them run for a pre-determined epochs or use earlystopping?\nI had been training for a constant num of epochs (26) and comparing models according to results achieved till then. I can see that the validation loss is still decreasing on the last 2-3 epochs. \n\nAnother advice needed: Do you scale learning rate (lr) when training on TPU vs GPU? I read that it helps to scale the lr when training on TPU. I tried up-scaling by a factor of Sqrt(Accelerators) but it leads to divergence. " }, { "id": 1126126, "postDate": "2020-12-25T11:15:42.890Z", "content": "I actually keep the global_batch size the same on GPU / TPU, that means, on GPU, batch is 128, but on TPU, batch is 16 (each replica) * 8.
\nThis is not good in general, since TPU can run faster if we feed it larger batches (of course, under its limit). With larger batch size, the optimization steps are fewer, so a larger lr is better for the convergence within the same time.
\nI tried larger batch with large lr - a bit worse than original batch (128) with normal lr. But at that time, my pipeline has some problem. I haven't check again larger batch/lr yet.
\nMy model usually get the best CV at epoch 14 - 17. I have an upper limit of epoch 20, but I manually stopped it once I found the CV decrease for 2-3 epoch.
\nI am a bit too lazy to implement earlystop by myself :)
", "rawMarkdown": "I actually keep the global_batch size the same on GPU / TPU, that means, on GPU, batch is 128, but on TPU, batch is 16 (each replica) * 8.\n\nThis is not good in general, since TPU can run faster if we feed it larger batches (of course, under its limit). With larger batch size, the optimization steps are fewer, so a larger lr is better for the convergence within the same time.\n\nI tried larger batch with large lr - a bit worse than original batch (128) with normal lr. But at that time, my pipeline has some problem. I haven't check again larger batch/lr yet.\n\nMy model usually get the best CV at epoch 14 - 17. I have an upper limit of epoch 20, but I manually stopped it once I found the CV decrease for 2-3 epoch.\n\nI am a bit too lazy to implement earlystop by myself :) ", "votes": 1 } ] }, { "id": 1124275, "postDate": "2020-12-23T19:37:06.027Z", "content": "(It's also a problem we cannot downvote notebooks for some strange reason ;))
", "rawMarkdown": "(It's also a problem we cannot downvote notebooks for some strange reason ;))", "votes": 3, "replies": [ { "id": 1127330, "postDate": "2020-12-26T12:55:30.063Z", "content": "This topic was already debated some time ago and there's pro and cons to enable downvotes on kernels. You can see the topic there.
", "rawMarkdown": "This topic was already debated some time ago and there's pro and cons to enable downvotes on kernels. You can see the topic [there](https://www.kaggle.com/product-feedback/76394).\n\n" } ] }, { "id": 1124537, "postDate": "2020-12-24T03:14:45.667Z", "content": "@scaomath thank you for pointing it out. I have seen in my previous competitions as well that people just copy public kernels and remove the references to the original kernels to make them look like genuine work. I think these things will continue as long as people upvote them. It would make sense that beginners might not be able to understand what is original or copied work, but It is rather sad to see people who are Experts and Masters upvoting those low effort kernels and making them look authentic.
", "rawMarkdown": "@scaomath thank you for pointing it out. I have seen in my previous competitions as well that people just copy public kernels and remove the references to the original kernels to make them look like genuine work. I think these things will continue as long as people upvote them. It would make sense that **beginners** might not be able to understand what is original or copied work, but It is rather sad to see people who are **Experts** and **Masters** upvoting those low effort kernels and making them look authentic. ", "votes": 4 }, { "id": 1127738, "postDate": "2020-12-26T19:57:58.157Z", "content": "I also noticed it and I have to admit that I found it is quite… what's the word, 'unpleasant'. (I have some strong words in my mind though…
\nFortunately, Kaggle has some features to mitigate this issue.
\nAt least we can report to the Kaggle admin that the notebook was just copied from others' hard work.
\nI don't know what will happen after we report the notebook though.
Issues with intelectual property wrt. to kernel's forks exist since quite a long time on Kaggle and some actions were already taken from Kaggle's team to make the game as fair as possible. I guess that they would take into account any new suggestions that would help improving the plateform.
", "rawMarkdown": "Issues with intelectual property wrt. to kernel's forks exist since quite a long time on Kaggle and [some actions were already taken from Kaggle's team](https://www.kaggle.com/general/89570) to make the game as fair as possible. I guess that they would take into account any new suggestions that would help improving the plateform.", "votes": 1 }, { "id": 1126198, "postDate": "2020-12-25T12:20:34.080Z", "content": "I wish kaggle come up with a solution to this problem. This has become a common occurence in kaggle. There is a similar thread in RANZCR CLiP competition as well. I wonder how these people feel with stolen upvotes? The first upvote should wake them up if it were a geniune mistake.
", "rawMarkdown": "I wish kaggle come up with a solution to this problem. This has become a common occurence in kaggle. There is a similar thread in [RANZCR CLiP](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/205147) competition as well. I wonder how these people feel with stolen upvotes? The first upvote should wake them up if it were a geniune mistake.", "votes": 2 }, { "id": 1128542, "postDate": "2020-12-27T14:31:59.260Z", "content": "Created this thread in the beginners community to increase the reach of this discussion.
", "rawMarkdown": "Created [this](https://www.kaggle.com/discussion/206639) thread in the beginners community to increase the reach of this discussion." }, { "id": 1124389, "postDate": "2020-12-23T22:11:20.613Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1125140, "author_name": "Yih-Dar SHIEH", "author_url": "", "post_date": "2020-12-24T12:13:21.047000", "content": "Although I don't really mind, and I know this happens, my (very hard) effort to put this TPU notebook
\nTPU - Track knowledge states of 1M+ students
\nonly got 20 votes …
\nI believed, when it was published, if I included the submission (inference) pipeline and make it scored (LB 0.781 at the published time), it will get > 100 votes and forks.
\nI would rather to get fewer votes from the people really use it or at least get some ideas from my notebook, or even people not use it but appreciate my effort.
", "votes": 14, "replies": [ { "id": 1125719, "author_name": "Manikanth Reddy", "author_url": "", "post_date": "2020-12-25T02:19:55.677000", "content": "Hi @yihdarshieh It's a great notebook. I would like to make one suggestion. Please go through the comment by Andrew Lukyanenko in this thread. Your notebook could be optimized so that beginners can understand. There was a lot of content and some part of it can be hidden (like warnings). I hope this helps for you.
", "votes": 1, "replies": [] }, { "id": 1125974, "author_name": "Yih-Dar SHIEH", "author_url": "", "post_date": "2020-12-25T08:02:59.033000", "content": "Thank you for the information. I will improve the style once the competition is finished. As I mentioned, it already took me quite a lot of time to have this current version - many options but not very well documented. After that version, I came back to work on my models …
", "votes": 1, "replies": [] }, { "id": 1125991, "author_name": "AbdurRafae", "author_url": "", "post_date": "2020-12-25T08:14:42.533000", "content": "@yihdarshieh Thanks for the notebook, I've referred to it quite a few times for understanding how to setup the TPU pipeline. Took me 5 days of continuous effort to get it up and running. Currently checking against old model architecture which ran on GPU as benchmark to ensure it's working properly. Speed-up vs GPU is 3.5~4x.
\nNow at least I can test 2-3 models in each of the remaining weeks.
Great to know it helps!
", "votes": 1, "replies": [] }, { "id": 1126112, "author_name": "Yih-Dar SHIEH", "author_url": "", "post_date": "2020-12-25T10:51:16.243000", "content": "@abdurrafae If you are ok to use Colab Pro, you can even test 3 - 4 models per day … However, setup on Colab might again take some time, so you need to consider what's the best for you, considering it remains only 2 weeks.
", "votes": 0, "replies": [] }, { "id": 1126121, "author_name": "AbdurRafae", "author_url": "", "post_date": "2020-12-25T11:01:57.943000", "content": "Thanks for the advice, I won't be able to use Colab pro since it's limited to US and Canada for now. I'll try on normal Colab session in a day or two, need a breather after working on this for 5 days. :D
\nJust a query, for comparing models do you let them run for a pre-determined epochs or use earlystopping?
\nI had been training for a constant num of epochs (26) and comparing models according to results achieved till then. I can see that the validation loss is still decreasing on the last 2-3 epochs.
Another advice needed: Do you scale learning rate (lr) when training on TPU vs GPU? I read that it helps to scale the lr when training on TPU. I tried up-scaling by a factor of Sqrt(Accelerators) but it leads to divergence.
", "votes": 0, "replies": [] }, { "id": 1126126, "author_name": "Yih-Dar SHIEH", "author_url": "", "post_date": "2020-12-25T11:15:42.890000", "content": "I actually keep the global_batch size the same on GPU / TPU, that means, on GPU, batch is 128, but on TPU, batch is 16 (each replica) * 8.
\nThis is not good in general, since TPU can run faster if we feed it larger batches (of course, under its limit). With larger batch size, the optimization steps are fewer, so a larger lr is better for the convergence within the same time.
\nI tried larger batch with large lr - a bit worse than original batch (128) with normal lr. But at that time, my pipeline has some problem. I haven't check again larger batch/lr yet.
\nMy model usually get the best CV at epoch 14 - 17. I have an upper limit of epoch 20, but I manually stopped it once I found the CV decrease for 2-3 epoch.
\nI am a bit too lazy to implement earlystop by myself :)
", "votes": 1, "replies": [] } ] }, { "id": 1124275, "author_name": "AmorfEvo", "author_url": "", "post_date": "2020-12-23T19:37:06.027000", "content": "(It's also a problem we cannot downvote notebooks for some strange reason ;))
", "votes": 3, "replies": [ { "id": 1127330, "author_name": "FabienDaniel", "author_url": "", "post_date": "2020-12-26T12:55:30.063000", "content": "This topic was already debated some time ago and there's pro and cons to enable downvotes on kernels. You can see the topic there.
", "votes": 0, "replies": [] } ] }, { "id": 1124537, "author_name": "Manikanth Reddy", "author_url": "", "post_date": "2020-12-24T03:14:45.667000", "content": "@scaomath thank you for pointing it out. I have seen in my previous competitions as well that people just copy public kernels and remove the references to the original kernels to make them look like genuine work. I think these things will continue as long as people upvote them. It would make sense that beginners might not be able to understand what is original or copied work, but It is rather sad to see people who are Experts and Masters upvoting those low effort kernels and making them look authentic.
", "votes": 4, "replies": [] }, { "id": 1127738, "author_name": "Kouki", "author_url": "", "post_date": "2020-12-26T19:57:58.157000", "content": "I also noticed it and I have to admit that I found it is quite… what's the word, 'unpleasant'. (I have some strong words in my mind though…
\nFortunately, Kaggle has some features to mitigate this issue.
\nAt least we can report to the Kaggle admin that the notebook was just copied from others' hard work.
\nI don't know what will happen after we report the notebook though.
Issues with intelectual property wrt. to kernel's forks exist since quite a long time on Kaggle and some actions were already taken from Kaggle's team to make the game as fair as possible. I guess that they would take into account any new suggestions that would help improving the plateform.
", "votes": 1, "replies": [] }, { "id": 1126198, "author_name": "A Legacy Grandmaster!", "author_url": "", "post_date": "2020-12-25T12:20:34.080000", "content": "I wish kaggle come up with a solution to this problem. This has become a common occurence in kaggle. There is a similar thread in RANZCR CLiP competition as well. I wonder how these people feel with stolen upvotes? The first upvote should wake them up if it were a geniune mistake.
", "votes": 2, "replies": [] }, { "id": 1128542, "author_name": "A Legacy Grandmaster!", "author_url": "", "post_date": "2020-12-27T14:31:59.260000", "content": "Created this thread in the beginners community to increase the reach of this discussion.
", "votes": 0, "replies": [] }, { "id": 1124389, "author_name": "", "author_url": "", "post_date": "2020-12-23T22:11:20.613000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1123993": "Today I stumbled upon this kernel:\nhttps://www.kaggle.com/lanniegal/riiid-sakt-model-inference-public\n\nUpon checking the difference:\n\n\n\nThe result is shocking. Not only is this notebook a no effort fork from @manikanthr5 's modification from several sakt models with no version differences. The author also deleted all the references @manikanthr5 put up and tributes to the sakt's other contributors. This notebook getting upvote is an insult to the Kaggle sharing spirit to be honest.\n\nAnother example is this one: https://www.kaggle.com/pandaman817/riiid-lgbm-bagging2-sakt-0-781", "1125140": "Although I don't really mind, and I know this happens, my (very hard) effort to put this TPU notebook\n\n[TPU - Track knowledge states of 1M+ students](https://www.kaggle.com/yihdarshieh/tpu-track-knowledge-states-of-1m-students)\n\nonly got 20 votes ...\n\nI believed, when it was published, if I included the submission (inference) pipeline and make it scored (LB 0.781 at the published time), it will get > 100 votes and forks.\n\nI would rather to get fewer votes from the people really use it or at least get some ideas from my notebook, or even people not use it but appreciate my effort.", "1124275": "(It's also a problem we cannot downvote notebooks for some strange reason ;))", "1124537": "@scaomath thank you for pointing it out. I have seen in my previous competitions as well that people just copy public kernels and remove the references to the original kernels to make them look like genuine work. I think these things will continue as long as people upvote them. It would make sense that **beginners** might not be able to understand what is original or copied work, but It is rather sad to see people who are **Experts** and **Masters** upvoting those low effort kernels and making them look authentic. ", "1127738": "I also noticed it and I have to admit that I found it is quite... what's the word, 'unpleasant'. (I have some strong words in my mind though...\nFortunately, Kaggle has some features to mitigate this issue. \nAt least we can report to the Kaggle admin that the notebook was just copied from others' hard work. \nI don't know what will happen after we report the notebook though. ", "1127338": "Issues with intelectual property wrt. to kernel's forks exist since quite a long time on Kaggle and [some actions were already taken from Kaggle's team](https://www.kaggle.com/general/89570) to make the game as fair as possible. I guess that they would take into account any new suggestions that would help improving the plateform.", "1126198": "I wish kaggle come up with a solution to this problem. This has become a common occurence in kaggle. There is a similar thread in [RANZCR CLiP](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/205147) competition as well. I wonder how these people feel with stolen upvotes? The first upvote should wake them up if it were a geniune mistake.", "1128542": "Created [this](https://www.kaggle.com/discussion/206639) thread in the beginners community to increase the reach of this discussion.", "1124389": "" } }