![\"Submission](\"https://raw.githubusercontent.com/motloch/kaggle_images/main/birdclef22-baseline-on-lb.jpg\")
Just sharing a fun plot, time of last submission against the leaderboard position for teams with public score 0.71:
Not wanting to shame anyone - did this myself in several competitions where I ended up not having time to actually compete. Got this plot while I was wondering how many people are actually just submitting the best public notebook, given I am yet to surpass this mark.
\nSome takeaways:
\n1) The clearly visible linear feature (highlighted in blue) corresponds to people who (presumably) submitted the best public notebook without any changes. They all have the same score, so their LB position is given by the submission time. The envelope is formed by people who have not submitted since, the points below the line are people who have submitted since but did not improve their score.
\n2) Once I finally get there, I will immediately jump up by 72 spots - encouragement for those of us in the second half of the leaderboard!
\n3) Some people are still submitting new solutions after not making any LB progress for three weeks - respect!
\n4) Most of the people eventually find a way to improve - see the three long lines between LB 240 and 260? Those are people who submitted in the past three days. Given we do not typically see similar features earlier, it seems that the natural explanation is that many people start with the best public notebook \"as is\", but eventually get better.
Just sharing a fun plot, time of last submission against the leaderboard position for teams with public score 0.71:
Not wanting to shame anyone - did this myself in several competitions where I ended up not having time to actually compete. Got this plot while I was wondering how many people are actually just submitting the best public notebook, given I am yet to surpass this mark.
\nSome takeaways:
\n1) The clearly visible linear feature (highlighted in blue) corresponds to people who (presumably) submitted the best public notebook without any changes. They all have the same score, so their LB position is given by the submission time. The envelope is formed by people who have not submitted since, the points below the line are people who have submitted since but did not improve their score.
\n2) Once I finally get there, I will immediately jump up by 72 spots - encouragement for those of us in the second half of the leaderboard!
\n3) Some people are still submitting new solutions after not making any LB progress for three weeks - respect!
\n4) Most of the people eventually find a way to improve - see the three long lines between LB 240 and 260? Those are people who submitted in the past three days. Given we do not typically see similar features earlier, it seems that the natural explanation is that many people start with the best public notebook \"as is\", but eventually get better.
This is a great post and it's really encouraging to see that many people are eventually able to improve their score. Keep up the good work, everyone!
", "rawMarkdown": "This is a great post and it's really encouraging to see that many people are eventually able to improve their score. Keep up the good work, everyone!", "votes": null }, { "id": "1781040", "postDate": "05/08/2022 06:21:36", "content": "Dear @motloch
\nIndeed it is not unusual to see people forkmitting (i.e. individuals who fork someone else's public notebook, often the highest scoring one they can find, and then directly submit it with no modifications) which leads to isoscore strings in the final shakeup scatter-plot, created by a series of teams having exactly the same both Public and Private scores.
\nSometimes one can observe several isoscore strings within the same competition, presumably by people who forkmitted early, and then did not return to the competition to see if there was an even better scoring public notebook later on.
\nLike yourself I am not at all judging; I have \"agreed to the rules\" on many competitions to take a look and see what it is about, forking (+ upvoting!) a popular notebook that I find readable and intelligible and tinkering to learn more. However, I have also found, again like you said, that to really take part in a competition is a lot of work; one has to be very attentive for the whole three months, constantly reading the forum to pick up on the latest ideas, reading through tens of notebooks to find little code snippets that give an edge. There is also the guessing game of seeing the Public leaderboard fairly static for a while, then suddenly someone submits a score that is significantly better than the rest ¿what did they discover? what new technique did they use? did they find some magic in the data?
\nVery time consuming, but very rewarding. I have the utmost respect for the Competition Ms & GMs who perform well time after time.
All the best and good luck,
\ncarl
Very interesting find. I firmly believe that only those who put in the effort in this competition deserve that medal, which is what motivates me and my teammates to move forward.
", "rawMarkdown": "Very interesting find. I firmly believe that only those who put in the effort in this competition deserve that medal, which is what motivates me and my teammates to move forward.", "votes": null }, { "id": "1781273", "postDate": "05/08/2022 11:21:58", "content": "\n\n\n…which is what motivates me and my teammates to move forward.
\n
What teammates are you speaking about? You officially go solo and this means that private sharing with anybody else is a rules' violation!
", "rawMarkdown": "yangranran \n> ...which is what motivates me and my teammates to move forward.\n\nWhat teammates are you speaking about? You **officially** go solo and this means that private sharing with anybody else is a rules' violation!", "votes": null }, { "id": "1781296", "postDate": "05/08/2022 11:50:11", "content": "Calm down, of course I know this rule and we plan to finish teaming up in two or three days.
", "rawMarkdown": "Calm down, of course I know this rule and we plan to finish teaming up in two or three days.", "votes": null }, { "id": "1781315", "postDate": "05/08/2022 12:03:59", "content": "\n\n\nCalm down, of course I know this rule and we plan to finish teaming up in two or three days.
\n
I am calmly telling you that you are violating the competition rules just now.
\nAren't I right, @addisonhoward ?
Why? We just decided we were going to team up together and didn't share anything privately.If this is also a violation of the rules, then how should we team up? Don't you need time to decide if the team is a good fit for you?
", "rawMarkdown": "Why? We just decided we were going to team up together and didn't share anything privately.If this is also a violation of the rules, then how should we team up? Don't you need time to decide if the team is a good fit for you?", "votes": null }, { "id": "1781343", "postDate": "05/08/2022 12:28:09", "content": "@yangranran
\nYour comments are typically contradictory as those of someone who has something to hide.
\nYour original statement was \"me and my teammates\" and you don't have any official teammates now and future doesn't matter.
First of all, the teammates in my comment \"me and my teammates\" refer to teammates from previous competitions and teammates we are about to team up in this competition, and we put in the effort in the competitions. Secondly, I firmly believe that I did not break any rules, and I wonder what exactly you are trying to express? Thirdly, my teammates and I will officially finish forming the team as soon as possible, and if we finally check out any violations, I am willing to accept punishment!
", "rawMarkdown": "First of all, the teammates in my comment \"me and my teammates\" refer to teammates from previous competitions and teammates we are about to team up in this competition, and we put in the effort in the competitions. Secondly, I firmly believe that I did not break any rules, and I wonder what exactly you are trying to express? Thirdly, my teammates and I will officially finish forming the team as soon as possible, and if we finally check out any violations, I am willing to accept punishment!", "votes": null }, { "id": "1781375", "postDate": "05/08/2022 12:58:13", "content": "I feel like this is more escalated than needs be, assuming what ran ran is saying is true. @yangranran, why do not you form the team right now? Sounds like the decision has been made and that would stop this debate straight away. Thanks!
", "rawMarkdown": "I feel like this is more escalated than needs be, assuming what ran ran is saying is true. @yangranran, why do not you form the team right now? Sounds like the decision has been made and that would stop this debate straight away. Thanks!", "votes": null }, { "id": "1781386", "postDate": "05/08/2022 13:09:32", "content": "@motloch I will immediately finish teaming up and end this debate. But suppose you are questioned and criticized for no reason, wouldn't you be angry and sad? I just wanted to prove that I didn't break the rules. Sorry to talk about some other things under your topic.
", "rawMarkdown": "motloch I will immediately finish teaming up and end this debate. But suppose you are questioned and criticized for no reason, wouldn't you be angry and sad? I just wanted to prove that I didn't break the rules. Sorry to talk about some other things under your topic.", "votes": null }, { "id": "1781394", "postDate": "05/08/2022 13:19:47", "content": "No worry. I have a feeling there is a bit of cultural misunderstanding here. In some cultures a \"handshake\" kind of deal is perfectly valid and honored, in others unless \"it is on paper and signed\" it does not count. And it is true that at the time you wrote the previous posts, you officially had no teammates (but presumably had them in the \"handshake\" kind of way), which rubbed Allie the wrong way. I did not personally find anything wrong with what you wrote, but I also understand some people can be very frustrated from placing lower in competitions because of various forms of cheating.
\nLet's hope this discussion ends here and all the best in the competition!
", "rawMarkdown": "No worry. I have a feeling there is a bit of cultural misunderstanding here. In some cultures a \"handshake\" kind of deal is perfectly valid and honored, in others unless \"it is on paper and signed\" it does not count. And it is true that at the time you wrote the previous posts, you officially had no teammates (but presumably had them in the \"handshake\" kind of way), which rubbed Allie the wrong way. I did not personally find anything wrong with what you wrote, but I also understand some people can be very frustrated from placing lower in competitions because of various forms of cheating.\n\nLet's hope this discussion ends here and all the best in the competition!", "votes": null }, { "id": "1781404", "postDate": "05/08/2022 13:27:50", "content": "You are right and wish us all the best.
", "rawMarkdown": "You are right and wish us all the best.", "votes": null }, { "id": "1781462", "postDate": "05/08/2022 14:28:07", "content": "Pavel,
\nI used \"your\" discussion thread for my replies because I have had to react. I have had a very good reason for questioning Yangranran's comments.
\nPlease don't try to (mis)interpret my actions. It is ridiculous to speak about \"cultural misunderstanding\" here, we are on Kaggle platform and it doesn't recognize anything like handshake agreement and Yangranran knows it very well.
\nFor me there is a history of more than strange former teammates' activity of this competitor so I wanted to keep everything transparent.
\nIn general I am always not frustrated but sad and disgusted seeing anybody unashamedly cheating no matter what my place on the leaderboard is and whether I take part in the competition or not.
I am finishing here and wish all the best to everybody who reaches his/her results without cheating.🙂
", "rawMarkdown": "Pavel,\nI used \"your\" discussion thread for my replies because I have had to react. I have had a very good reason for questioning Yangranran's comments.\nPlease don't try to (mis)interpret my actions. It is ridiculous to speak about \"cultural misunderstanding\" here, we are on Kaggle platform and it doesn't recognize anything like handshake agreement and Yangranran knows it very well. \nFor me there is a history of more than strange former teammates' activity of this competitor so I wanted to keep everything transparent.\n**In general** I am always not frustrated but sad and disgusted seeing anybody unashamedly cheating no matter what my place on the leaderboard is and whether I take part in the competition or not. \n\nI am finishing here and wish all the best to everybody who reaches his/her results without cheating.🙂", "votes": null }, { "id": "1781514", "postDate": "05/08/2022 15:23:54", "content": "The public kernel has an unusually high score this time. Few lucky seeds and/or folds with that could take another .02. So possibly one needs to get .73 or more to make a huge jump. Doing this with a new model from scratch can be difficult. Just couple of days back I could somehow manage to scrape thru.
\nConsidering the high public kernel score, I feel one should focus on strengthening the model/CV and not worry too much about the forks and public leaderboard scores. Of course this is easier said than done, because one tends to get confidence in one's model only after seeing a good public leaderboard score (human tendency)
\nI lost a whole weekend resolving a Kaggle issue https://www.kaggle.com/competitions/birdclef-2022/discussion/322092. With 2 weekends still left, will definitely try to experiment and see if I can do anything interesting.
", "rawMarkdown": "The public kernel has an unusually high score this time. Few lucky seeds and/or folds with that could take another .02. So possibly one needs to get .73 or more to make a huge jump. Doing this with a new model from scratch can be difficult. Just couple of days back I could somehow manage to scrape thru. \n\nConsidering the high public kernel score, I feel one should focus on strengthening the model/CV and not worry too much about the forks and public leaderboard scores. Of course this is easier said than done, because one tends to get confidence in one's model only after seeing a good public leaderboard score (human tendency)\n\nI lost a whole weekend resolving a Kaggle issue https://www.kaggle.com/competitions/birdclef-2022/discussion/322092. With 2 weekends still left, will definitely try to experiment and see if I can do anything interesting.", "votes": null }, { "id": "1781530", "postDate": "05/08/2022 15:39:42", "content": "I am not a fan of \"fork\" competing but in this particular case I wouldn't be so worried either. With unclear metrics/evaluation and only 16% in public test set I think that only the golden top of the leaderboard shows any real value of the solutions.
", "rawMarkdown": "I am not a fan of \"fork\" competing but in this particular case I wouldn't be so worried either. With unclear metrics/evaluation and only 16% in public test set I think that only the golden top of the leaderboard shows any real value of the solutions.", "votes": null }, { "id": "1781598", "postDate": "05/08/2022 17:35:14", "content": "You are right. Private is 84%, so some of the forks+seed/fold could be badly overfitted to public LB. Secondly there is no test soundscapes for CV, so unless somehow one has derived a CV that works, most likely rest are fitting to LB which may/may not work at the end. Lastly as you have said the exact evaluation metric is unclear (though @bilzard has provided a great many useful posts analyzing the same).
\nBut like I said, it is psychological to strive to get a decent rank on public LB. I am guilty of experimenting with the thresholds and trying to improve my score(of my baseline model) instead of focussing on strengthening it. Maybe next weekend I try to improve my models and see if I can move higher.
The interesting thing is that the best public notebook is surprisingly simple to improve on in terms of validation (+0.04). Public LB doesn't follow however in this case.
", "rawMarkdown": "The interesting thing is that the best public notebook is surprisingly simple to improve on in terms of validation (+0.04). Public LB doesn't follow however in this case.", "votes": null } ], "comments": [ { "id": 1781020, "author_name": "", "author_url": "", "post_date": "05/08/2022 06:05:50", "content": "This is a great post and it's really encouraging to see that many people are eventually able to improve their score. Keep up the good work, everyone!
", "votes": null, "replies": [] }, { "id": 1781040, "author_name": "carlmcbrideellis", "author_url": "", "post_date": "05/08/2022 06:21:36", "content": "Dear @motloch
\nIndeed it is not unusual to see people forkmitting (i.e. individuals who fork someone else's public notebook, often the highest scoring one they can find, and then directly submit it with no modifications) which leads to isoscore strings in the final shakeup scatter-plot, created by a series of teams having exactly the same both Public and Private scores.
\nSometimes one can observe several isoscore strings within the same competition, presumably by people who forkmitted early, and then did not return to the competition to see if there was an even better scoring public notebook later on.
\nLike yourself I am not at all judging; I have \"agreed to the rules\" on many competitions to take a look and see what it is about, forking (+ upvoting!) a popular notebook that I find readable and intelligible and tinkering to learn more. However, I have also found, again like you said, that to really take part in a competition is a lot of work; one has to be very attentive for the whole three months, constantly reading the forum to pick up on the latest ideas, reading through tens of notebooks to find little code snippets that give an edge. There is also the guessing game of seeing the Public leaderboard fairly static for a while, then suddenly someone submits a score that is significantly better than the rest ¿what did they discover? what new technique did they use? did they find some magic in the data?
\nVery time consuming, but very rewarding. I have the utmost respect for the Competition Ms & GMs who perform well time after time.
All the best and good luck,
\ncarl
Very interesting find. I firmly believe that only those who put in the effort in this competition deserve that medal, which is what motivates me and my teammates to move forward.
", "votes": null, "replies": [ { "id": 1781273, "author_name": "blankaf", "author_url": "", "post_date": "05/08/2022 11:21:58", "content": "\n\n\n…which is what motivates me and my teammates to move forward.
\n
What teammates are you speaking about? You officially go solo and this means that private sharing with anybody else is a rules' violation!
", "votes": null, "replies": [] }, { "id": 1781296, "author_name": "yangranran", "author_url": "", "post_date": "05/08/2022 11:50:11", "content": "Calm down, of course I know this rule and we plan to finish teaming up in two or three days.
", "votes": null, "replies": [] }, { "id": 1781315, "author_name": "blankaf", "author_url": "", "post_date": "05/08/2022 12:03:59", "content": "\n\n\nCalm down, of course I know this rule and we plan to finish teaming up in two or three days.
\n
I am calmly telling you that you are violating the competition rules just now.
\nAren't I right, @addisonhoward ?
Why? We just decided we were going to team up together and didn't share anything privately.If this is also a violation of the rules, then how should we team up? Don't you need time to decide if the team is a good fit for you?
", "votes": null, "replies": [] }, { "id": 1781343, "author_name": "blankaf", "author_url": "", "post_date": "05/08/2022 12:28:09", "content": "@yangranran
\nYour comments are typically contradictory as those of someone who has something to hide.
\nYour original statement was \"me and my teammates\" and you don't have any official teammates now and future doesn't matter.
First of all, the teammates in my comment \"me and my teammates\" refer to teammates from previous competitions and teammates we are about to team up in this competition, and we put in the effort in the competitions. Secondly, I firmly believe that I did not break any rules, and I wonder what exactly you are trying to express? Thirdly, my teammates and I will officially finish forming the team as soon as possible, and if we finally check out any violations, I am willing to accept punishment!
", "votes": null, "replies": [] }, { "id": 1781375, "author_name": "motloch", "author_url": "", "post_date": "05/08/2022 12:58:13", "content": "I feel like this is more escalated than needs be, assuming what ran ran is saying is true. @yangranran, why do not you form the team right now? Sounds like the decision has been made and that would stop this debate straight away. Thanks!
", "votes": null, "replies": [] }, { "id": 1781386, "author_name": "yangranran", "author_url": "", "post_date": "05/08/2022 13:09:32", "content": "@motloch I will immediately finish teaming up and end this debate. But suppose you are questioned and criticized for no reason, wouldn't you be angry and sad? I just wanted to prove that I didn't break the rules. Sorry to talk about some other things under your topic.
", "votes": null, "replies": [] }, { "id": 1781394, "author_name": "motloch", "author_url": "", "post_date": "05/08/2022 13:19:47", "content": "No worry. I have a feeling there is a bit of cultural misunderstanding here. In some cultures a \"handshake\" kind of deal is perfectly valid and honored, in others unless \"it is on paper and signed\" it does not count. And it is true that at the time you wrote the previous posts, you officially had no teammates (but presumably had them in the \"handshake\" kind of way), which rubbed Allie the wrong way. I did not personally find anything wrong with what you wrote, but I also understand some people can be very frustrated from placing lower in competitions because of various forms of cheating.
\nLet's hope this discussion ends here and all the best in the competition!
", "votes": null, "replies": [] }, { "id": 1781404, "author_name": "yangranran", "author_url": "", "post_date": "05/08/2022 13:27:50", "content": "You are right and wish us all the best.
", "votes": null, "replies": [] }, { "id": 1781462, "author_name": "blankaf", "author_url": "", "post_date": "05/08/2022 14:28:07", "content": "Pavel,
\nI used \"your\" discussion thread for my replies because I have had to react. I have had a very good reason for questioning Yangranran's comments.
\nPlease don't try to (mis)interpret my actions. It is ridiculous to speak about \"cultural misunderstanding\" here, we are on Kaggle platform and it doesn't recognize anything like handshake agreement and Yangranran knows it very well.
\nFor me there is a history of more than strange former teammates' activity of this competitor so I wanted to keep everything transparent.
\nIn general I am always not frustrated but sad and disgusted seeing anybody unashamedly cheating no matter what my place on the leaderboard is and whether I take part in the competition or not.
I am finishing here and wish all the best to everybody who reaches his/her results without cheating.🙂
", "votes": null, "replies": [] } ] }, { "id": 1781514, "author_name": "allohvk", "author_url": "", "post_date": "05/08/2022 15:23:54", "content": "The public kernel has an unusually high score this time. Few lucky seeds and/or folds with that could take another .02. So possibly one needs to get .73 or more to make a huge jump. Doing this with a new model from scratch can be difficult. Just couple of days back I could somehow manage to scrape thru.
\nConsidering the high public kernel score, I feel one should focus on strengthening the model/CV and not worry too much about the forks and public leaderboard scores. Of course this is easier said than done, because one tends to get confidence in one's model only after seeing a good public leaderboard score (human tendency)
\nI lost a whole weekend resolving a Kaggle issue https://www.kaggle.com/competitions/birdclef-2022/discussion/322092. With 2 weekends still left, will definitely try to experiment and see if I can do anything interesting.
", "votes": null, "replies": [ { "id": 1781530, "author_name": "blankaf", "author_url": "", "post_date": "05/08/2022 15:39:42", "content": "I am not a fan of \"fork\" competing but in this particular case I wouldn't be so worried either. With unclear metrics/evaluation and only 16% in public test set I think that only the golden top of the leaderboard shows any real value of the solutions.
", "votes": null, "replies": [] }, { "id": 1781598, "author_name": "allohvk", "author_url": "", "post_date": "05/08/2022 17:35:14", "content": "You are right. Private is 84%, so some of the forks+seed/fold could be badly overfitted to public LB. Secondly there is no test soundscapes for CV, so unless somehow one has derived a CV that works, most likely rest are fitting to LB which may/may not work at the end. Lastly as you have said the exact evaluation metric is unclear (though @bilzard has provided a great many useful posts analyzing the same).
\nBut like I said, it is psychological to strive to get a decent rank on public LB. I am guilty of experimenting with the thresholds and trying to improve my score(of my baseline model) instead of focussing on strengthening it. Maybe next weekend I try to improve my models and see if I can move higher.
The interesting thing is that the best public notebook is surprisingly simple to improve on in terms of validation (+0.04). Public LB doesn't follow however in this case.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1780897": "Just sharing a fun plot, time of last submission against the leaderboard position for teams with public score 0.71:\n\nNot wanting to shame anyone - did this myself in several competitions where I ended up not having time to actually compete. Got this plot while I was wondering how many people are actually just submitting the best [public notebook](https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer), given I am yet to surpass this mark.\n\nSome takeaways:\n1) The clearly visible linear feature (highlighted in blue) corresponds to people who (presumably) submitted the best [public notebook](https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer) without any changes. They all have the same score, so their LB position is given by the submission time. The envelope is formed by people who have not submitted since, the points below the line are people who have submitted since but did not improve their score.\n2) Once I finally get there, I will immediately jump up by 72 spots - encouragement for those of us in the second half of the leaderboard!\n3) Some people are still submitting new solutions after not making any LB progress for three weeks - respect!\n4) Most of the people eventually find a way to improve - see the three long lines between LB 240 and 260? Those are people who submitted in the past three days. Given we do not typically see similar features earlier, it seems that the natural explanation is that many people start with the best public notebook \"as is\", but eventually get better.", "1781020": "This is a great post and it's really encouraging to see that many people are eventually able to improve their score. Keep up the good work, everyone!", "1781040": "Dear @motloch \n\nIndeed it is not unusual to see people *forkmitting* (i.e. individuals who fork someone else's public notebook, often the highest scoring one they can find, and then directly submit it with no modifications) which leads to [isoscore strings](https://www.kaggle.com/code/carlmcbrideellis/shakeup-scatterplots-boxes-strings-and-things) in the final shakeup scatter-plot, created by a series of teams having *exactly* the same both Public and Private scores.\n\nSometimes one can observe several isoscore strings within the same competition, presumably by people who forkmitted early, and then did not return to the competition to see if there was an even better scoring public notebook later on.\n\nLike yourself I am not at all judging; I have \"agreed to the rules\" on many competitions to take a look and see what it is about, forking (+ upvoting!) a popular notebook that I find readable and intelligible and tinkering to learn more. However, I have also found, again like you said, that to *really* take part in a competition is *a lot of work*; one has to be very attentive for the whole three months, constantly reading the forum to pick up on the latest ideas, reading through tens of notebooks to find little code snippets that give an edge. There is also the guessing game of seeing the Public leaderboard fairly static for a while, then suddenly someone submits a score that is significantly better than the rest ¿what did they discover? what new technique did they use? did they find some magic in the data?\nVery time consuming, but *very* rewarding. I have the utmost respect for the Competition Ms & GMs who perform well time after time.\n\nAll the best and good luck,\ncarl", "1781072": "Very interesting find. I firmly believe that only those who put in the effort in this competition deserve that medal, which is what motivates me and my teammates to move forward.", "1781273": "yangranran \n> ...which is what motivates me and my teammates to move forward.\n\nWhat teammates are you speaking about? You **officially** go solo and this means that private sharing with anybody else is a rules' violation!", "1781296": "Calm down, of course I know this rule and we plan to finish teaming up in two or three days.", "1781315": "yangranran \n> Calm down, of course I know this rule and we plan to finish teaming up in two or three days.\n\nI am calmly telling you that you are violating the competition rules just now.\nAren't I right, @addisonhoward ?", "1781322": "Why? We just decided we were going to team up together and didn't share anything privately.If this is also a violation of the rules, then how should we team up? Don't you need time to decide if the team is a good fit for you?", "1781343": "yangranran \nYour comments are typically contradictory as those of someone who has something to hide.\nYour original statement was \"**me and my teammates**\" and you don't have any official teammates **now** and future doesn't matter.", "1781369": "First of all, the teammates in my comment \"me and my teammates\" refer to teammates from previous competitions and teammates we are about to team up in this competition, and we put in the effort in the competitions. Secondly, I firmly believe that I did not break any rules, and I wonder what exactly you are trying to express? Thirdly, my teammates and I will officially finish forming the team as soon as possible, and if we finally check out any violations, I am willing to accept punishment!", "1781375": "I feel like this is more escalated than needs be, assuming what ran ran is saying is true. @yangranran, why do not you form the team right now? Sounds like the decision has been made and that would stop this debate straight away. Thanks!", "1781386": "motloch I will immediately finish teaming up and end this debate. But suppose you are questioned and criticized for no reason, wouldn't you be angry and sad? I just wanted to prove that I didn't break the rules. Sorry to talk about some other things under your topic.", "1781394": "No worry. I have a feeling there is a bit of cultural misunderstanding here. In some cultures a \"handshake\" kind of deal is perfectly valid and honored, in others unless \"it is on paper and signed\" it does not count. And it is true that at the time you wrote the previous posts, you officially had no teammates (but presumably had them in the \"handshake\" kind of way), which rubbed Allie the wrong way. I did not personally find anything wrong with what you wrote, but I also understand some people can be very frustrated from placing lower in competitions because of various forms of cheating.\n\nLet's hope this discussion ends here and all the best in the competition!", "1781404": "You are right and wish us all the best.", "1781462": "Pavel,\nI used \"your\" discussion thread for my replies because I have had to react. I have had a very good reason for questioning Yangranran's comments.\nPlease don't try to (mis)interpret my actions. It is ridiculous to speak about \"cultural misunderstanding\" here, we are on Kaggle platform and it doesn't recognize anything like handshake agreement and Yangranran knows it very well. \nFor me there is a history of more than strange former teammates' activity of this competitor so I wanted to keep everything transparent.\n**In general** I am always not frustrated but sad and disgusted seeing anybody unashamedly cheating no matter what my place on the leaderboard is and whether I take part in the competition or not. \n\nI am finishing here and wish all the best to everybody who reaches his/her results without cheating.🙂", "1781514": "The public kernel has an unusually high score this time. Few lucky seeds and/or folds with that could take another .02. So possibly one needs to get .73 or more to make a huge jump. Doing this with a new model from scratch can be difficult. Just couple of days back I could somehow manage to scrape thru. \n\nConsidering the high public kernel score, I feel one should focus on strengthening the model/CV and not worry too much about the forks and public leaderboard scores. Of course this is easier said than done, because one tends to get confidence in one's model only after seeing a good public leaderboard score (human tendency)\n\nI lost a whole weekend resolving a Kaggle issue https://www.kaggle.com/competitions/birdclef-2022/discussion/322092. With 2 weekends still left, will definitely try to experiment and see if I can do anything interesting.", "1781530": "I am not a fan of \"fork\" competing but in this particular case I wouldn't be so worried either. With unclear metrics/evaluation and only 16% in public test set I think that only the golden top of the leaderboard shows any real value of the solutions.", "1781598": "You are right. Private is 84%, so some of the forks+seed/fold could be badly overfitted to public LB. Secondly there is no test soundscapes for CV, so unless somehow one has derived a CV that works, most likely rest are fitting to LB which may/may not work at the end. Lastly as you have said the exact evaluation metric is unclear (though @bilzard has provided a great many useful posts analyzing the same).\nBut like I said, it is psychological to strive to get a decent rank on public LB. I am guilty of experimenting with the thresholds and trying to improve my score(of my baseline model) instead of focussing on strengthening it. Maybe next weekend I try to improve my models and see if I can move higher.", "1782818": "The interesting thing is that the best public notebook is surprisingly simple to improve on in terms of validation (+0.04). Public LB doesn't follow however in this case." }, "source": "meta" }