Thanks. (They are still hiding their score)
\nhttps://imgur.com/a/J1ts0rh
Thanks. (They are still hiding their score)
\nhttps://imgur.com/a/J1ts0rh
Ohh god. I feel bad for you. Although, it's just public leaderboard.๐ฅฒ
", "rawMarkdown": "Ohh god. I feel bad for you. Although, it's just public leaderboard.๐ฅฒ", "votes": null }, { "id": "1312180", "postDate": "05/17/2021 22:05:39", "content": "Insane haha. Hats off @tvdwiele. Looking forward to the solution!
", "rawMarkdown": "Insane haha. Hats off @tvdwiele. Looking forward to the solution!", "votes": null }, { "id": "1312185", "postDate": "05/17/2021 22:14:16", "content": "We have immense respect for you and your team, and look forward to learn from your approach.
\nMay the best win after the shakedown.
", "rawMarkdown": "We have immense respect for you and your team, and look forward to learn from your approach.\n\nMay the best win after the shakedown.", "votes": null }, { "id": "1312186", "postDate": "05/17/2021 22:18:09", "content": "Thank you, I'm also looking forward to seeing your approach. I guess you have a much better postprocessing strategy than us, and I guess it should be something like a heuristics contest (may be wrong).
", "rawMarkdown": "Thank you, I'm also looking forward to seeing your approach. I guess you have a much better postprocessing strategy than us, and I guess it should be something like a heuristics contest (may be wrong).", "votes": null }, { "id": "1312189", "postDate": "05/17/2021 22:21:22", "content": "We donโt really do postprocessing but incorporate the training grid into the core of our approach. All details will be explained after the deadline.
", "rawMarkdown": "We donโt really do postprocessing but incorporate the training grid into the core of our approach. All details will be explained after the deadline.", "votes": null }, { "id": "1312191", "postDate": "05/17/2021 22:22:57", "content": "Even if our teams can use hand-labelled grids (of course our team doesn't use hand-labelled grids because it's prohibited), I'm sure we couldn't get to 1.0. (I think we could probably get to 1.28, though.)
", "rawMarkdown": "Even if our teams can use hand-labelled grids (of course our team doesn't use hand-labelled grids because it's prohibited), I'm sure we couldn't get to 1.0. (I think we could probably get to 1.28, though.)", "votes": null }, { "id": "1312193", "postDate": "05/17/2021 22:27:11", "content": "We got to a point where hand labelling would only help in rare occasions (which we donโt use in our submissions).
", "rawMarkdown": "We got to a point where hand labelling would only help in rare occasions (which we donโt use in our submissions).", "votes": null }, { "id": "1312195", "postDate": "05/17/2021 22:28:05", "content": "Hmm, seems my guess is completely wrong. BTW, How about showing your best score? I think you are still hiding it.
", "rawMarkdown": "Hmm, seems my guess is completely wrong. BTW, How about showing your best score? I think you are still hiding it.", "votes": null }, { "id": "1312197", "postDate": "05/17/2021 22:29:51", "content": "Our best score is 0.891.
", "rawMarkdown": "Our best score is 0.891.", "votes": null }, { "id": "1312201", "postDate": "05/17/2021 22:39:17", "content": "Thank you, It's just unbelievable :). Did you use leakages (e.g. start/end leakage)? Our team doesn't use any leakage, because they didn't improve score at all.
", "rawMarkdown": "Thank you, It's just unbelievable :). Did you use leakages (e.g. start/end leakage)? Our team doesn't use any leakage, because they didn't improve score at all.", "votes": null }, { "id": "1312202", "postDate": "05/17/2021 22:40:04", "content": "I am excited to hear about your approach!
", "rawMarkdown": "I am excited to hear about your approach!", "votes": null }, { "id": "1312204", "postDate": "05/17/2021 22:40:57", "content": "Our score can not be improved by any leakages beyond the ones shared on the forum a while back. So far, it looks like this has been a clean challenge.
", "rawMarkdown": "Our score can not be improved by any leakages beyond the ones shared on the forum a while back. So far, it looks like this has been a clean challenge.", "votes": null }, { "id": "1312205", "postDate": "05/17/2021 22:41:08", "content": "\n\nWe donโt really do postprocessing but incorporate the training grid into the core of our approach.
\n
Amazing! I'm trying so hard to do exactly that but failed. Looking forward to your approach!
", "rawMarkdown": ">We donโt really do postprocessing but incorporate the training grid into the core of our approach. \n\nAmazing! I'm trying so hard to do exactly that but failed. Looking forward to your approach!", "votes": null }, { "id": "1312206", "postDate": "05/17/2021 22:42:12", "content": "The only additional leakage we tried to add was to use the ends of public trajectories to predict the start of the private ones, but we already had them right :-)
", "rawMarkdown": "The only additional leakage we tried to add was to use the ends of public trajectories to predict the start of the private ones, but we already had them right :-)", "votes": null }, { "id": "1312209", "postDate": "05/17/2021 22:45:27", "content": "Yeah, using public leakage kernel (pseudo-device kernel) didn't change the score (or sometimes degraded the score).
", "rawMarkdown": "Yeah, using public leakage kernel (pseudo-device kernel) didn't change the score (or sometimes degraded the score).", "votes": null }, { "id": "1312230", "postDate": "05/17/2021 23:34:40", "content": "I hope this (hiding score) does not become a habit in Kaggle competitions. It is not against competition rules but it is against the spirit of a fun and fair competition. Our team didn't have such high score strategy, but we might have played differently. it was unpleasant.
", "rawMarkdown": "I hope this (hiding score) does not become a habit in Kaggle competitions. It is not against competition rules but it is against the spirit of a fun and fair competition. Our team didn't have such high score strategy, but we might have played differently. it was unpleasant.", "votes": null }, { "id": "1312324", "postDate": "05/18/2021 01:52:56", "content": "congrats on 2nd place and becoming GM @mamasinkgs
", "rawMarkdown": "congrats on 2nd place and becoming GM @mamasinkgs", "votes": null }, { "id": "1312346", "postDate": "05/18/2021 02:12:36", "content": "congrats on becoming GM @mamasinkgs ๐๐
\nYour hunch on 1st place Tom and Dott hiding their scores was true!! You are very perceptiveโฆ
\nand THANK YOU!! besides being so helpful. I can only thank YOU for my position. Floors was the key and I had missed that. I used the other set of floors and was able to manage my rankโฆ would not have been possible without your suggestions and guidance ๐๐๐๐
\nThank you so muchโฆ learnt so much from you!! ๐๐
", "rawMarkdown": "congrats on becoming GM @mamasinkgs ๐๐\n\nYour hunch on 1st place Tom and Dott hiding their scores was true!! You are very perceptive...\n\nand THANK YOU!! besides being so helpful. I can only thank YOU for my position. Floors was the key and I had missed that. I used the other set of floors and was able to manage my rank... would not have been possible without your suggestions and guidance ๐๐๐๐\n\nThank you so much... learnt so much from you!! ๐๐", "votes": null }, { "id": "1312752", "postDate": "05/18/2021 08:16:47", "content": "Thanks a lot! I'll take a rest and will start writing the overview of our team's solution and upload my postprocessing code.
", "rawMarkdown": "Thanks a lot! I'll take a rest and will start writing the overview of our team's solution and upload my postprocessing code.", "votes": null }, { "id": "1312755", "postDate": "05/18/2021 08:18:01", "content": "Thanks, and congrats your silver medal!
", "rawMarkdown": "Thanks, and congrats your silver medal!", "votes": null }, { "id": "1312828", "postDate": "05/18/2021 09:21:12", "content": "While I haven't been on Kaggle for long enough fully to understand the local etiquette, I have lots sympathy for @mamasinkgs and the MYRCJ team on being overtaken at the last moment having led for so long. I've enjoyed very much watching this competition unfold, and especially that many top competitors have been very active in sharing ideas on this forum.
", "rawMarkdown": "While I haven't been on Kaggle for long enough fully to understand the local etiquette, I have lots sympathy for @mamasinkgs and the MYRCJ team on being overtaken at the last moment having led for so long. I've enjoyed very much watching this competition unfold, and especially that many top competitors have been very active in sharing ideas on this forum.", "votes": null }, { "id": "1312927", "postDate": "05/18/2021 10:25:44", "content": "I don't want to complain about it because it's not against rule, but I think this competition should have been more interesting if they don't hide their score. The battle in public LB is one of the most exciting part in kaggle. I think we couldn't win this competition anyway whether they hide their score or not, but I'm a bit sad because I couldn't enjoy the battle in Public LB with Tom's team.
", "rawMarkdown": "I don't want to complain about it because it's not against rule, but I think this competition should have been more interesting if they don't hide their score. The battle in public LB is one of the most exciting part in kaggle. I think we couldn't win this competition anyway whether they hide their score or not, but I'm a bit sad because I couldn't enjoy the battle in Public LB with Tom's team.", "votes": null }, { "id": "1312937", "postDate": "05/18/2021 10:31:27", "content": "Though I'm a bit sad I couldn't enjoy the battle in Public LB, I'm sure the solution that will be provided by Tom will be really interesting and valuable. So, I'm really happy they participate in this competition and sincerely respect them.
", "rawMarkdown": "Though I'm a bit sad I couldn't enjoy the battle in Public LB, I'm sure the solution that will be provided by Tom will be really interesting and valuable. So, I'm really happy they participate in this competition and sincerely respect them.", "votes": null }, { "id": "1313189", "postDate": "05/18/2021 12:42:17", "content": "We understand that it is frustrating to lose first place on the last day (I have been there!). However, we stand by the tactic we followed and want to explain our reasoning.
\nWe didn't want to get distracted by the public leaderboard. Both Dmitry and myself have experienced being in first place on the public leaderboard on at least three occasions. It is not good for sleep quality and just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data. We decided to focus on our internal validation and putting all the pieces together instead.
\nUp until the last week of the competition, our best public LB score was around 2.8 (probably because we were not focusing on it too much), so it would likely not have changed the dynamics of the competition either way.
\nWe also enjoyed the irony of doing very well at indoor tracking and not being tracked at the same time (hence the team name).
", "rawMarkdown": "We understand that it is frustrating to lose first place on the last day (I have been there!). However, we stand by the tactic we followed and want to explain our reasoning.\n\nWe didn't want to get distracted by the public leaderboard. Both Dmitry and myself have experienced being in first place on the public leaderboard on at least three occasions. It is not good for sleep quality and just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data. We decided to focus on our internal validation and putting all the pieces together instead.\n\nUp until the last week of the competition, our best public LB score was around 2.8 (probably because we were not focusing on it too much), so it would likely not have changed the dynamics of the competition either way.\n\nWe also enjoyed the irony of doing very well at indoor tracking and not being tracked at the same time (hence the team name).", "votes": null }, { "id": "1313210", "postDate": "05/18/2021 12:53:59", "content": "Thanks to your team for making the prediction of a team breaking 1.0 on public LB come true, looking forward to your team's solution @tvdwiele
", "rawMarkdown": "Thanks to your team for making the prediction of a team breaking 1.0 on public LB come true, looking forward to your team's solution @tvdwiele", "votes": null }, { "id": "1313254", "postDate": "05/18/2021 13:22:53", "content": "Just like with our submissions, we are hiding our writeup from you for a bit longer, so we can make it extra worth it ;-)
", "rawMarkdown": "Just like with our submissions, we are hiding our writeup from you for a bit longer, so we can make it extra worth it ;-)", "votes": null }, { "id": "1313884", "postDate": "05/18/2021 19:06:11", "content": "I understand your frustration Reza, and am incredibly impressed with your 2nd place score. But I'm not terribly sympathetic to your argument. What strategy exactly would you pursue with the additional information other than trying to get the lowest possible score??
\nSome people outside of Kaggle complain that it's not applicable to real-world machine learning, and I think one of the most important lessons from kaggle contests is to learn how to get your own representative validation metrics and to not rely on the LB. If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.
\nIf looking backwards in time over a path, or using leakage from training data, or countless other unrealistic strategies, are considered acceptable, I think covering your tracks on the LB should be too. especially if, as you claim, if actually did provide the #1 team a strategic advantage.
\nJust my two cents :) but seriously, congrats again, very impressive score, and I look forward to reading all the solutions!
", "rawMarkdown": "I understand your frustration Reza, and am incredibly impressed with your 2nd place score. But I'm not terribly sympathetic to your argument. What strategy exactly would you pursue with the additional information other than trying to get the lowest possible score??\n\nSome people outside of Kaggle complain that it's not applicable to real-world machine learning, and I think one of the most important lessons from kaggle contests is to learn how to get your own representative validation metrics and to not rely on the LB. If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.\n\nIf looking backwards in time over a path, or using leakage from training data, or countless other unrealistic strategies, are considered acceptable, I think covering your tracks on the LB should be too. especially if, as you claim, if actually did provide the #1 team a strategic advantage.\n\nJust my two cents :) but seriously, congrats again, very impressive score, and I look forward to reading all the solutions!", "votes": null }, { "id": "1313924", "postDate": "05/18/2021 19:56:15", "content": "To be honest, What Tom's team did was not so frustrating at least for me and I sincerely respect them, and his comment is reasonable and can understand it. However, I don't agree with @paulfornia 's opinion at all.
\n If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.\nHowever, Tom said his best score was 2.8 until the last week of the competition. I remember our score is about 1.6 meter then and of course Tom checked this (he said our score pushed their team forward and they worked so hard, and it's an honor :) ). So Tom's team heard the voice telling them that \"an error of 1.6 m is possible, so keep trying\", which is not realistic as you say.
", "rawMarkdown": "To be honest, What Tom's team did was not so frustrating at least for me and I sincerely respect them, and his comment is reasonable and can understand it. However, I don't agree with @paulfornia 's opinion at all.\n\n```\n If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.\n```\n\nHowever, Tom said his best score was 2.8 until the last week of the competition. I remember our score is about 1.6 meter then and of course Tom checked this (he said our score pushed their team forward and they worked so hard, and it's an honor :) ). So Tom's team heard the voice telling them that \"an error of 1.6 m is possible, so keep trying\", which is not realistic as you say.", "votes": null }, { "id": "1313925", "postDate": "05/18/2021 20:02:08", "content": "I don't think Tom's action made the competition more realistic and less gamified, and I believe Tom doesn't think so. If Tom's kaggle account can not check public LB, I agree with this opinion, though.
", "rawMarkdown": "I don't think Tom's action made the competition more realistic and less gamified, and I believe Tom doesn't think so. If Tom's kaggle account can not check public LB, I agree with this opinion, though.", "votes": null }, { "id": "1313929", "postDate": "05/18/2021 20:04:30", "content": "Hey @mamasinkgs , I'm sorry, I definitely didn't mean to upset you, and I appreciate everything you and your team did for this contest! That is a very fair point about Tom having that advantage that you didn'tโฆ however, couldn't one argue that it strengthens the rationale for his strategy? My point was simply that the public LB is a part of kaggle that is clearly a game, not related to ML. So either it doesn't affect your strategy, in which case it doesn't matter, or it does affect your strategy, in which case Tom was very smart to hide his score. No?
", "rawMarkdown": "Hey @mamasinkgs , I'm sorry, I definitely didn't mean to upset you, and I appreciate everything you and your team did for this contest! That is a very fair point about Tom having that advantage that you didn't... however, couldn't one argue that it strengthens the rationale for his strategy? My point was simply that the public LB is a part of kaggle that is clearly a game, not related to ML. So either it doesn't affect your strategy, in which case it doesn't matter, or it does affect your strategy, in which case Tom was very smart to hide his score. No?", "votes": null }, { "id": "1313945", "postDate": "05/18/2021 20:15:28", "content": "What strategy exactly would you pursue with the additional information other than trying to get the lowest possible score?? \nIf I saw the LB score of 0.9 in the last week of the competition, I would try some ideas that can give a big boost (I tried some stable ideas that gave a small boost, actually). Probably I would have focused on generating grids strategy. (To be honest I think we would have lost even if we succeed in it, though.) So, as Tom said, it is clear hiding a score is an advantage.
\n just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data.\nHowever, the advantage is not so big and I'm sure Tom's team should have won even if they didn't hide their score, and of course I really respect them.
", "rawMarkdown": "``` \nWhat strategy exactly would you pursue with the additional information other than trying to get the lowest possible score?? \n```\nIf I saw the LB score of 0.9 in the last week of the competition, I would try some ideas that can give a big boost (I tried some stable ideas that gave a small boost, actually). Probably I would have focused on generating grids strategy. (To be honest I think we would have lost even if we succeed in it, though.) So, as Tom said, it is clear hiding a score is an advantage. \n```\n just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data.\n```\nHowever, the advantage is not so big and I'm sure Tom's team should have won even if they didn't hide their score, and of course I really respect them.", "votes": null }, { "id": "1313954", "postDate": "05/18/2021 20:24:43", "content": "Okay yeah that's completely fair. And just to be totally clear, I don't mean any ill content or sarcasm when I say congratulations and that I am impressed by your score :) I say this as someone who busted their butt for the last three months and still got absolutely slaughtered by your team and others. I look forward to seeing you in the next contest!
", "rawMarkdown": "Okay yeah that's completely fair. And just to be totally clear, I don't mean any ill content or sarcasm when I say congratulations and that I am impressed by your score :) I say this as someone who busted their butt for the last three months and still got absolutely slaughtered by your team and others. I look forward to seeing you in the next contest!", "votes": null }, { "id": "1314009", "postDate": "05/18/2021 21:44:49", "content": "I'm sorry my writing of overall 2nd place solution writeup is very slow, but I will finish writing it by May 21.
", "rawMarkdown": "I'm sorry my writing of overall 2nd place solution writeup is very slow, but I will finish writing it by May 21.", "votes": null } ], "comments": [ { "id": 1312178, "author_name": "nischaydnk", "author_url": "", "post_date": "05/17/2021 22:05:00", "content": "Ohh god. I feel bad for you. Although, it's just public leaderboard.๐ฅฒ
", "votes": null, "replies": [] }, { "id": 1312180, "author_name": "olaf2000", "author_url": "", "post_date": "05/17/2021 22:05:39", "content": "Insane haha. Hats off @tvdwiele. Looking forward to the solution!
", "votes": null, "replies": [] }, { "id": 1312185, "author_name": "tvdwiele", "author_url": "", "post_date": "05/17/2021 22:14:16", "content": "We have immense respect for you and your team, and look forward to learn from your approach.
\nMay the best win after the shakedown.
", "votes": null, "replies": [ { "id": 1312186, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/17/2021 22:18:09", "content": "Thank you, I'm also looking forward to seeing your approach. I guess you have a much better postprocessing strategy than us, and I guess it should be something like a heuristics contest (may be wrong).
", "votes": null, "replies": [] }, { "id": 1312189, "author_name": "tvdwiele", "author_url": "", "post_date": "05/17/2021 22:21:22", "content": "We donโt really do postprocessing but incorporate the training grid into the core of our approach. All details will be explained after the deadline.
", "votes": null, "replies": [] }, { "id": 1312195, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/17/2021 22:28:05", "content": "Hmm, seems my guess is completely wrong. BTW, How about showing your best score? I think you are still hiding it.
", "votes": null, "replies": [] }, { "id": 1312197, "author_name": "tvdwiele", "author_url": "", "post_date": "05/17/2021 22:29:51", "content": "Our best score is 0.891.
", "votes": null, "replies": [] }, { "id": 1312201, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/17/2021 22:39:17", "content": "Thank you, It's just unbelievable :). Did you use leakages (e.g. start/end leakage)? Our team doesn't use any leakage, because they didn't improve score at all.
", "votes": null, "replies": [] }, { "id": 1312204, "author_name": "tvdwiele", "author_url": "", "post_date": "05/17/2021 22:40:57", "content": "Our score can not be improved by any leakages beyond the ones shared on the forum a while back. So far, it looks like this has been a clean challenge.
", "votes": null, "replies": [] }, { "id": 1312205, "author_name": "jiweiliu", "author_url": "", "post_date": "05/17/2021 22:41:08", "content": "\n\nWe donโt really do postprocessing but incorporate the training grid into the core of our approach.
\n
Amazing! I'm trying so hard to do exactly that but failed. Looking forward to your approach!
", "votes": null, "replies": [] }, { "id": 1312206, "author_name": "tvdwiele", "author_url": "", "post_date": "05/17/2021 22:42:12", "content": "The only additional leakage we tried to add was to use the ends of public trajectories to predict the start of the private ones, but we already had them right :-)
", "votes": null, "replies": [] }, { "id": 1312209, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/17/2021 22:45:27", "content": "Yeah, using public leakage kernel (pseudo-device kernel) didn't change the score (or sometimes degraded the score).
", "votes": null, "replies": [] } ] }, { "id": 1312191, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/17/2021 22:22:57", "content": "Even if our teams can use hand-labelled grids (of course our team doesn't use hand-labelled grids because it's prohibited), I'm sure we couldn't get to 1.0. (I think we could probably get to 1.28, though.)
", "votes": null, "replies": [ { "id": 1312193, "author_name": "tvdwiele", "author_url": "", "post_date": "05/17/2021 22:27:11", "content": "We got to a point where hand labelling would only help in rare occasions (which we donโt use in our submissions).
", "votes": null, "replies": [] } ] }, { "id": 1312202, "author_name": "areehdot", "author_url": "", "post_date": "05/17/2021 22:40:04", "content": "I am excited to hear about your approach!
", "votes": null, "replies": [] }, { "id": 1312230, "author_name": "vaghefi", "author_url": "", "post_date": "05/17/2021 23:34:40", "content": "I hope this (hiding score) does not become a habit in Kaggle competitions. It is not against competition rules but it is against the spirit of a fun and fair competition. Our team didn't have such high score strategy, but we might have played differently. it was unpleasant.
", "votes": null, "replies": [ { "id": 1312828, "author_name": "jbomitchell", "author_url": "", "post_date": "05/18/2021 09:21:12", "content": "While I haven't been on Kaggle for long enough fully to understand the local etiquette, I have lots sympathy for @mamasinkgs and the MYRCJ team on being overtaken at the last moment having led for so long. I've enjoyed very much watching this competition unfold, and especially that many top competitors have been very active in sharing ideas on this forum.
", "votes": null, "replies": [] }, { "id": 1312927, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 10:25:44", "content": "I don't want to complain about it because it's not against rule, but I think this competition should have been more interesting if they don't hide their score. The battle in public LB is one of the most exciting part in kaggle. I think we couldn't win this competition anyway whether they hide their score or not, but I'm a bit sad because I couldn't enjoy the battle in Public LB with Tom's team.
", "votes": null, "replies": [] }, { "id": 1312937, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 10:31:27", "content": "Though I'm a bit sad I couldn't enjoy the battle in Public LB, I'm sure the solution that will be provided by Tom will be really interesting and valuable. So, I'm really happy they participate in this competition and sincerely respect them.
", "votes": null, "replies": [] }, { "id": 1313189, "author_name": "tvdwiele", "author_url": "", "post_date": "05/18/2021 12:42:17", "content": "We understand that it is frustrating to lose first place on the last day (I have been there!). However, we stand by the tactic we followed and want to explain our reasoning.
\nWe didn't want to get distracted by the public leaderboard. Both Dmitry and myself have experienced being in first place on the public leaderboard on at least three occasions. It is not good for sleep quality and just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data. We decided to focus on our internal validation and putting all the pieces together instead.
\nUp until the last week of the competition, our best public LB score was around 2.8 (probably because we were not focusing on it too much), so it would likely not have changed the dynamics of the competition either way.
\nWe also enjoyed the irony of doing very well at indoor tracking and not being tracked at the same time (hence the team name).
", "votes": null, "replies": [] }, { "id": 1313210, "author_name": "duykhanh99", "author_url": "", "post_date": "05/18/2021 12:53:59", "content": "Thanks to your team for making the prediction of a team breaking 1.0 on public LB come true, looking forward to your team's solution @tvdwiele
", "votes": null, "replies": [] }, { "id": 1313254, "author_name": "tvdwiele", "author_url": "", "post_date": "05/18/2021 13:22:53", "content": "Just like with our submissions, we are hiding our writeup from you for a bit longer, so we can make it extra worth it ;-)
", "votes": null, "replies": [] }, { "id": 1313884, "author_name": "paulfornia", "author_url": "", "post_date": "05/18/2021 19:06:11", "content": "I understand your frustration Reza, and am incredibly impressed with your 2nd place score. But I'm not terribly sympathetic to your argument. What strategy exactly would you pursue with the additional information other than trying to get the lowest possible score??
\nSome people outside of Kaggle complain that it's not applicable to real-world machine learning, and I think one of the most important lessons from kaggle contests is to learn how to get your own representative validation metrics and to not rely on the LB. If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.
\nIf looking backwards in time over a path, or using leakage from training data, or countless other unrealistic strategies, are considered acceptable, I think covering your tracks on the LB should be too. especially if, as you claim, if actually did provide the #1 team a strategic advantage.
\nJust my two cents :) but seriously, congrats again, very impressive score, and I look forward to reading all the solutions!
", "votes": null, "replies": [] }, { "id": 1313924, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 19:56:15", "content": "To be honest, What Tom's team did was not so frustrating at least for me and I sincerely respect them, and his comment is reasonable and can understand it. However, I don't agree with @paulfornia 's opinion at all.
\n If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.\nHowever, Tom said his best score was 2.8 until the last week of the competition. I remember our score is about 1.6 meter then and of course Tom checked this (he said our score pushed their team forward and they worked so hard, and it's an honor :) ). So Tom's team heard the voice telling them that \"an error of 1.6 m is possible, so keep trying\", which is not realistic as you say.
", "votes": null, "replies": [] }, { "id": 1313925, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 20:02:08", "content": "I don't think Tom's action made the competition more realistic and less gamified, and I believe Tom doesn't think so. If Tom's kaggle account can not check public LB, I agree with this opinion, though.
", "votes": null, "replies": [] }, { "id": 1313929, "author_name": "paulfornia", "author_url": "", "post_date": "05/18/2021 20:04:30", "content": "Hey @mamasinkgs , I'm sorry, I definitely didn't mean to upset you, and I appreciate everything you and your team did for this contest! That is a very fair point about Tom having that advantage that you didn'tโฆ however, couldn't one argue that it strengthens the rationale for his strategy? My point was simply that the public LB is a part of kaggle that is clearly a game, not related to ML. So either it doesn't affect your strategy, in which case it doesn't matter, or it does affect your strategy, in which case Tom was very smart to hide his score. No?
", "votes": null, "replies": [] }, { "id": 1313945, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 20:15:28", "content": "What strategy exactly would you pursue with the additional information other than trying to get the lowest possible score?? \nIf I saw the LB score of 0.9 in the last week of the competition, I would try some ideas that can give a big boost (I tried some stable ideas that gave a small boost, actually). Probably I would have focused on generating grids strategy. (To be honest I think we would have lost even if we succeed in it, though.) So, as Tom said, it is clear hiding a score is an advantage.
\n just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data.\nHowever, the advantage is not so big and I'm sure Tom's team should have won even if they didn't hide their score, and of course I really respect them.
", "votes": null, "replies": [] }, { "id": 1313954, "author_name": "paulfornia", "author_url": "", "post_date": "05/18/2021 20:24:43", "content": "Okay yeah that's completely fair. And just to be totally clear, I don't mean any ill content or sarcasm when I say congratulations and that I am impressed by your score :) I say this as someone who busted their butt for the last three months and still got absolutely slaughtered by your team and others. I look forward to seeing you in the next contest!
", "votes": null, "replies": [] } ] }, { "id": 1312324, "author_name": "duykhanh99", "author_url": "", "post_date": "05/18/2021 01:52:56", "content": "congrats on 2nd place and becoming GM @mamasinkgs
", "votes": null, "replies": [ { "id": 1312752, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 08:16:47", "content": "Thanks a lot! I'll take a rest and will start writing the overview of our team's solution and upload my postprocessing code.
", "votes": null, "replies": [] } ] }, { "id": 1312346, "author_name": "kmldas", "author_url": "", "post_date": "05/18/2021 02:12:36", "content": "congrats on becoming GM @mamasinkgs ๐๐
\nYour hunch on 1st place Tom and Dott hiding their scores was true!! You are very perceptiveโฆ
\nand THANK YOU!! besides being so helpful. I can only thank YOU for my position. Floors was the key and I had missed that. I used the other set of floors and was able to manage my rankโฆ would not have been possible without your suggestions and guidance ๐๐๐๐
\nThank you so muchโฆ learnt so much from you!! ๐๐
", "votes": null, "replies": [ { "id": 1312755, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 08:18:01", "content": "Thanks, and congrats your silver medal!
", "votes": null, "replies": [] } ] }, { "id": 1314009, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/18/2021 21:44:49", "content": "I'm sorry my writing of overall 2nd place solution writeup is very slow, but I will finish writing it by May 21.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1312176": "Thanks. (They are still hiding their score)\nhttps://imgur.com/a/J1ts0rh", "1312178": "Ohh god. I feel bad for you. Although, it's just public leaderboard.๐ฅฒ", "1312180": "Insane haha. Hats off @tvdwiele. Looking forward to the solution!", "1312185": "We have immense respect for you and your team, and look forward to learn from your approach.\n\nMay the best win after the shakedown.", "1312186": "Thank you, I'm also looking forward to seeing your approach. I guess you have a much better postprocessing strategy than us, and I guess it should be something like a heuristics contest (may be wrong).", "1312189": "We donโt really do postprocessing but incorporate the training grid into the core of our approach. All details will be explained after the deadline.", "1312191": "Even if our teams can use hand-labelled grids (of course our team doesn't use hand-labelled grids because it's prohibited), I'm sure we couldn't get to 1.0. (I think we could probably get to 1.28, though.)", "1312193": "We got to a point where hand labelling would only help in rare occasions (which we donโt use in our submissions).", "1312195": "Hmm, seems my guess is completely wrong. BTW, How about showing your best score? I think you are still hiding it.", "1312197": "Our best score is 0.891.", "1312201": "Thank you, It's just unbelievable :). Did you use leakages (e.g. start/end leakage)? Our team doesn't use any leakage, because they didn't improve score at all.", "1312202": "I am excited to hear about your approach!", "1312204": "Our score can not be improved by any leakages beyond the ones shared on the forum a while back. So far, it looks like this has been a clean challenge.", "1312205": ">We donโt really do postprocessing but incorporate the training grid into the core of our approach. \n\nAmazing! I'm trying so hard to do exactly that but failed. Looking forward to your approach!", "1312206": "The only additional leakage we tried to add was to use the ends of public trajectories to predict the start of the private ones, but we already had them right :-)", "1312209": "Yeah, using public leakage kernel (pseudo-device kernel) didn't change the score (or sometimes degraded the score).", "1312230": "I hope this (hiding score) does not become a habit in Kaggle competitions. It is not against competition rules but it is against the spirit of a fun and fair competition. Our team didn't have such high score strategy, but we might have played differently. it was unpleasant.", "1312324": "congrats on 2nd place and becoming GM @mamasinkgs", "1312346": "congrats on becoming GM @mamasinkgs ๐๐\n\nYour hunch on 1st place Tom and Dott hiding their scores was true!! You are very perceptive...\n\nand THANK YOU!! besides being so helpful. I can only thank YOU for my position. Floors was the key and I had missed that. I used the other set of floors and was able to manage my rank... would not have been possible without your suggestions and guidance ๐๐๐๐\n\nThank you so much... learnt so much from you!! ๐๐", "1312752": "Thanks a lot! I'll take a rest and will start writing the overview of our team's solution and upload my postprocessing code.", "1312755": "Thanks, and congrats your silver medal!", "1312828": "While I haven't been on Kaggle for long enough fully to understand the local etiquette, I have lots sympathy for @mamasinkgs and the MYRCJ team on being overtaken at the last moment having led for so long. I've enjoyed very much watching this competition unfold, and especially that many top competitors have been very active in sharing ideas on this forum.", "1312927": "I don't want to complain about it because it's not against rule, but I think this competition should have been more interesting if they don't hide their score. The battle in public LB is one of the most exciting part in kaggle. I think we couldn't win this competition anyway whether they hide their score or not, but I'm a bit sad because I couldn't enjoy the battle in Public LB with Tom's team.", "1312937": "Though I'm a bit sad I couldn't enjoy the battle in Public LB, I'm sure the solution that will be provided by Tom will be really interesting and valuable. So, I'm really happy they participate in this competition and sincerely respect them.", "1313189": "We understand that it is frustrating to lose first place on the last day (I have been there!). However, we stand by the tactic we followed and want to explain our reasoning.\n\nWe didn't want to get distracted by the public leaderboard. Both Dmitry and myself have experienced being in first place on the public leaderboard on at least three occasions. It is not good for sleep quality and just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data. We decided to focus on our internal validation and putting all the pieces together instead.\n\nUp until the last week of the competition, our best public LB score was around 2.8 (probably because we were not focusing on it too much), so it would likely not have changed the dynamics of the competition either way.\n\nWe also enjoyed the irony of doing very well at indoor tracking and not being tracked at the same time (hence the team name).", "1313210": "Thanks to your team for making the prediction of a team breaking 1.0 on public LB come true, looking forward to your team's solution @tvdwiele", "1313254": "Just like with our submissions, we are hiding our writeup from you for a bit longer, so we can make it extra worth it ;-)", "1313884": "I understand your frustration Reza, and am incredibly impressed with your 2nd place score. But I'm not terribly sympathetic to your argument. What strategy exactly would you pursue with the additional information other than trying to get the lowest possible score??\n\nSome people outside of Kaggle complain that it's not applicable to real-world machine learning, and I think one of the most important lessons from kaggle contests is to learn how to get your own representative validation metrics and to not rely on the LB. If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.\n\nIf looking backwards in time over a path, or using leakage from training data, or countless other unrealistic strategies, are considered acceptable, I think covering your tracks on the LB should be too. especially if, as you claim, if actually did provide the #1 team a strategic advantage.\n\nJust my two cents :) but seriously, congrats again, very impressive score, and I look forward to reading all the solutions!", "1313924": "To be honest, What Tom's team did was not so frustrating at least for me and I sincerely respect them, and his comment is reasonable and can understand it. However, I don't agree with @paulfornia 's opinion at all.\n\n```\n If you are building a state-of-the-art system in the real world, there is not going to be some divine force telling you that \"an error of 1m is possible, so keep trying\". Tom throwing a wrench in the public LB actually makes the contest more realistic, and less gamified IMO.\n```\n\nHowever, Tom said his best score was 2.8 until the last week of the competition. I remember our score is about 1.6 meter then and of course Tom checked this (he said our score pushed their team forward and they worked so hard, and it's an honor :) ). So Tom's team heard the voice telling them that \"an error of 1.6 m is possible, so keep trying\", which is not realistic as you say.", "1313925": "I don't think Tom's action made the competition more realistic and less gamified, and I believe Tom doesn't think so. If Tom's kaggle account can not check public LB, I agree with this opinion, though.", "1313929": "Hey @mamasinkgs , I'm sorry, I definitely didn't mean to upset you, and I appreciate everything you and your team did for this contest! That is a very fair point about Tom having that advantage that you didn't... however, couldn't one argue that it strengthens the rationale for his strategy? My point was simply that the public LB is a part of kaggle that is clearly a game, not related to ML. So either it doesn't affect your strategy, in which case it doesn't matter, or it does affect your strategy, in which case Tom was very smart to hide his score. No?", "1313945": "``` \nWhat strategy exactly would you pursue with the additional information other than trying to get the lowest possible score?? \n```\nIf I saw the LB score of 0.9 in the last week of the competition, I would try some ideas that can give a big boost (I tried some stable ideas that gave a small boost, actually). Probably I would have focused on generating grids strategy. (To be honest I think we would have lost even if we succeed in it, though.) So, as Tom said, it is clear hiding a score is an advantage. \n```\n just the fact that you are at a certain score can give others a competitive advantage because it shows the minimum that can be achieved with the given data.\n```\nHowever, the advantage is not so big and I'm sure Tom's team should have won even if they didn't hide their score, and of course I really respect them.", "1313954": "Okay yeah that's completely fair. And just to be totally clear, I don't mean any ill content or sarcasm when I say congratulations and that I am impressed by your score :) I say this as someone who busted their butt for the last three months and still got absolutely slaughtered by your team and others. I look forward to seeing you in the next contest!", "1314009": "I'm sorry my writing of overall 2nd place solution writeup is very slow, but I will finish writing it by May 21." }, "source": "meta" }