Toward the end of the game, I would say, this was a really terrible competition, all because of a terrible metric. The organizers wasted money and data, and the kagglers wasted time and made some poor models. I was expecting to see some wonderful feature engineering and model ideas, but now I am really disappointed.
\nThe organizer should re-evaluate the results of this game and choose simple and effective evaluation mertric, such as week mean auc.
Toward the end of the game, I would say, this was a really terrible competition, all because of a terrible metric. The organizers wasted money and data, and the kagglers wasted time and made some poor models. I was expecting to see some wonderful feature engineering and model ideas, but now I am really disappointed.
\nThe organizer should re-evaluate the results of this game and choose simple and effective evaluation mertric, such as week mean auc.
Kaggle should add a leave the competition button for such situations.
", "rawMarkdown": "Kaggle should add a leave the competition button for such situations. ", "votes": 19, "replies": [ { "id": 2821402, "postDate": "2024-05-18T05:19:11.870Z", "content": "@greysky this is a very useful feature and in my opinion, should be present in all competitions.
\nI think this should be present in teams as well- named as disband team and exit competition/ disassociate from team and exit competition as well.
\nAlso, Kaggle could consider a system where team-members rate one another. This will help future teams understand the potential value-add from a given member should they choose to collaborate.
", "rawMarkdown": "@greysky this is a very useful feature and in my opinion, should be present in all competitions.\n\nI think this should be present in teams as well- named as **disband team and exit competition**/ **disassociate from team and exit competition** as well.\n\nAlso, Kaggle could consider a system where team-members rate one another. This will help future teams understand the potential value-add from a given member should they choose to collaborate. ", "votes": 17, "replies": [ { "id": 2829637, "postDate": "2024-05-22T17:42:57.187Z", "content": "@ravi20076 any idea why they don't have this? I imagine leaving/disbanding teams creates the opportunity for abuse (eg. taking a teammate's work and then splitting off) but I see no downside for exiting a competition…
", "rawMarkdown": "@ravi20076 any idea why they don't have this? I imagine leaving/disbanding teams creates the opportunity for abuse (eg. taking a teammate's work and then splitting off) but I see no downside for exiting a competition...", "votes": 2 }, { "id": 2837235, "postDate": "2024-05-26T11:35:38.543Z", "content": "I agree, when one asks to leave a team, he/ she could leave the competition also @tztang
", "rawMarkdown": "I agree, when one asks to leave a team, he/ she could leave the competition also @tztang ", "votes": 1 }, { "id": 2838209, "postDate": "2024-05-26T23:35:14.020Z", "content": "@ravi20076,I strongly believe not having an option to exit a competition is not an oversight but a deliberate business decision. They want to sell to potential client how well attended their competitions are. Having said that, if they keep running this kind of problematic competitions, they will chase good companies away from running a competition on this platform anyway. What is the point if the whole thing is diluted and becomes a joke in the ML/ Data Science world.
", "rawMarkdown": "@ravi20076,I strongly believe not having an option to exit a competition is not an oversight but a deliberate business decision. They want to sell to potential client how well attended their competitions are. Having said that, if they keep running this kind of problematic competitions, they will chase good companies away from running a competition on this platform anyway. What is the point if the whole thing is diluted and becomes a joke in the ML/ Data Science world.", "votes": 1 }, { "id": 2838540, "postDate": "2024-05-27T05:22:16.567Z", "content": "@sheriytm
\nI totally understand your point. Such an activity may impact the host’s marketing activity also.
Yup, that's a good idea.
", "rawMarkdown": "Yup, that's a good idea.", "votes": 2 }, { "id": 2829784, "postDate": "2024-05-22T19:35:02.617Z", "content": "Sounds great in theory, but then just like with anything else people will abuse it like crazy. Just think about it… people will join lots of competitions in search for gold, silver or bronze… then with only hours left until deadline those who clearly don't have a chance at the podium will leave in droves such competitions. This in turn would cause sudden recalculations of gold, silver or bronze positions in such competitions b/c they are based on percentages of the total number of competitors. It would be madness.
\nAny new idea, and new product that you try to introduce (not only in Kaggle) should start with the critical question \"how is this product going to be abused?… in what ways?\". When you really think about it, a \"leave button\" would bring way more harm than good to Kaggle. It will never happen.
", "rawMarkdown": "Sounds great in theory, but then just like with anything else people will abuse it like crazy. Just think about it... people will join lots of competitions in search for gold, silver or bronze... then with only hours left until deadline those who clearly don't have a chance at the podium will leave in droves such competitions. This in turn would cause sudden recalculations of gold, silver or bronze positions in such competitions b/c they are based on percentages of the total number of competitors. It would be madness. \n\nAny new idea, and new product that you try to introduce (not only in Kaggle) should start with the critical question \"how is this product going to be abused?... in what ways?\". When you really think about it, a \"leave button\" would bring way more harm than good to Kaggle. It will never happen.", "votes": 4, "replies": [ { "id": 2830327, "postDate": "2024-05-23T06:33:45.477Z", "content": "You make a fair point. In the same way that there's a competition \"Merger & Entry\" deadline, maybe have an \"Exit & Disband\" deadline as well? Just wishful thinking of course.
", "rawMarkdown": "You make a fair point. In the same way that there's a competition \"Merger & Entry\" deadline, maybe have an \"Exit & Disband\" deadline as well? Just wishful thinking of course.", "votes": 1 }, { "id": 2837239, "postDate": "2024-05-26T11:37:22.590Z", "content": "Perfect @tztang
\nThese deadlines could be on the same day. We may have a system where we may not perform any activity related to the team formation in the last 7 days of any competition!
I just recently started using kaggle and doing machine learning. I would love to use the \"leave\" button, to be honest. It's too difficult for me somehow 😔
\nso far 😜
Good idea to leave disband team but completion will not benefits both.
", "rawMarkdown": "Good idea to leave disband team but completion will not benefits both.", "votes": 1 } ] }, { "id": 2821276, "postDate": "2024-05-18T03:37:00.730Z", "content": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.
", "rawMarkdown": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.", "votes": 16 }, { "id": 2822583, "postDate": "2024-05-18T17:13:18.810Z", "content": "Competition is very interesting by itself, but the metric and the attitude of the hosts that initially said that will disqualify cheaters by manually checking top notebooks and then changed their minds by allowing to do anything - broke everything and dissapointed a lot of people.
", "rawMarkdown": "Competition is very interesting by itself, but the metric and the attitude of the hosts that initially said that will disqualify cheaters by manually checking top notebooks and then changed their minds by allowing to do anything - broke everything and dissapointed a lot of people.", "votes": 10 }, { "id": 2821680, "postDate": "2024-05-18T07:36:48.587Z", "content": "It's not within our purview to assess the organizers' choices. They have the prerogative to select the metric they deem appropriate. It's possible that the models they have deployed in their production environment frequently deteriorate over time, prompting them to explore new avenues. I'm not in a position to ascertain their circumstances with certainty. This is merely my conjecture regarding the rationale behind such a metric. In any case, the customer (organizer) has the prerogative to set own requirements. This is essentially how it is in real-world scenarios. I would like to share a case from my professional experience. Frequently, I'm tasked with demonstrating to the customer that their suggestions or ideas are not optimal for a specific task. However, in many instances, we are required to adhere to the customer's specifications, even when it goes against common sense. In this competition, I initially planned to create a model based on log regression and binning. This would have allowed to interpret the model and perform secondary validation based on PSI and HHI. However, I have since realized that such a model would be uncompetitive in comparison with ensembles.
", "rawMarkdown": "It's not within our purview to assess the organizers' choices. They have the prerogative to select the metric they deem appropriate. It's possible that the models they have deployed in their production environment frequently deteriorate over time, prompting them to explore new avenues. I'm not in a position to ascertain their circumstances with certainty. This is merely my conjecture regarding the rationale behind such a metric. In any case, the customer (organizer) has the prerogative to set own requirements. This is essentially how it is in real-world scenarios. I would like to share a case from my professional experience. Frequently, I'm tasked with demonstrating to the customer that their suggestions or ideas are not optimal for a specific task. However, in many instances, we are required to adhere to the customer's specifications, even when it goes against common sense. In this competition, I initially planned to create a model based on log regression and binning. This would have allowed to interpret the model and perform secondary validation based on PSI and HHI. However, I have since realized that such a model would be uncompetitive in comparison with ensembles. ", "votes": 7, "replies": [ { "id": 2821765, "postDate": "2024-05-18T08:28:50.377Z", "rawMarkdown": "", "votes": -1, "isDeleted": true }, { "id": 2822487, "postDate": "2024-05-18T16:11:39.773Z", "content": "very true :)
", "rawMarkdown": "very true :)", "votes": 1 }, { "id": 2826277, "postDate": "2024-05-20T20:06:44.713Z", "content": "It's also possible that this decision didn't come from guys responsible for contact with Kagglers. They (at least at the beginning) were available and responded to the questions etc.. Nevertheless, it leaves a bad aftertaste and is generally seen as a bad move.
", "rawMarkdown": "It's also possible that this decision didn't come from guys responsible for contact with Kagglers. They (at least at the beginning) were available and responded to the questions etc.. Nevertheless, it leaves a bad aftertaste and is generally seen as a bad move.", "votes": 2 } ] }, { "id": 2821116, "postDate": "2024-05-17T23:33:40.857Z", "content": "Organizers should reconsider ever hosting another competition. Please do not.
", "rawMarkdown": "Organizers should reconsider ever hosting another competition. Please do not.", "votes": 7 }, { "id": 2822993, "postDate": "2024-05-19T00:27:25.230Z", "content": "I'm glad I didn't invest a lot of time.
", "rawMarkdown": "I'm glad I didn't invest a lot of time.", "votes": 3 }, { "id": 2822823, "postDate": "2024-05-18T19:31:03.333Z", "content": "Home Credit Group should simply hire Upstart, or some other company that knows what it's doing in this space. I'm not exactly sure what useful stuff they learned from this competition, other than the fact that their metric has been exposed.
", "rawMarkdown": "Home Credit Group should simply hire Upstart, or some other company that knows what it's doing in this space. I'm not exactly sure what useful stuff they learned from this competition, other than the fact that their metric has been exposed. ", "votes": 3 }, { "id": 2839067, "postDate": "2024-05-27T11:21:42.170Z", "content": "I wouldnt day it was a terrible competition. Even thought it was a bit chaotic I learned a lot. And nothing is perfect in live. The important thing is to keep a flexible mindset.
", "rawMarkdown": "I wouldnt day it was a terrible competition. Even thought it was a bit chaotic I learned a lot. And nothing is perfect in live. The important thing is to keep a flexible mindset. ", "votes": 4 }, { "id": 2823927, "postDate": "2024-05-19T13:35:32.677Z", "content": "It's outrageous that I've spent almost two weeks working on improving models and feature engineering, and the boosted scores aren't as high as gently changing a few numbers in a day today!😅
", "rawMarkdown": "It's outrageous that I've spent almost two weeks working on improving models and feature engineering, and the boosted scores aren't as high as gently changing a few numbers in a day today!😅", "votes": 4 }, { "id": 2824289, "postDate": "2024-05-19T17:28:39.707Z", "content": "In the afternoon, I saw a public post with a high score. I saved it to read again at night. Before reading it, I submitted the post to see how much training results came out, but to my surprise, after reading that code, it came out. It's a trick, not a training process. At first I didn't understand why there were 2 submitted files haha, let's continue working.
", "rawMarkdown": "In the afternoon, I saw a public post with a high score. I saved it to read again at night. Before reading it, I submitted the post to see how much training results came out, but to my surprise, after reading that code, it came out. It's a trick, not a training process. At first I didn't understand why there were 2 submitted files haha, let's continue working.", "votes": 2 }, { "id": 2821974, "postDate": "2024-05-18T10:57:36.567Z", "content": "\n\nThe organizer should re-evaluate the results of this game and choose simple and effective evaluation mertric, such as week mean auc.
\n
Didn't they already change their minds twice? From disallowing metric hack to allowing it? This is the result of that flip flop
", "rawMarkdown": ">The organizer should re-evaluate the results of this game and choose simple and effective evaluation mertric, such as week mean auc. \n\nDidn't they already change their minds twice? From disallowing metric hack to allowing it? This is the result of that flip flop", "votes": 2 }, { "id": 2827597, "postDate": "2024-05-21T15:13:27.567Z", "content": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.
", "rawMarkdown": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.", "replies": [ { "id": 2829787, "postDate": "2024-05-22T19:43:01.597Z", "content": "In this competition almost everybody was serious about feature engineering and development of ML models until they weren't. Only those who joined in the last 10 days entered directly into the realm 0.6XX scores. It's just like with anything else in life: You adapt, or you are left behind! This isn't any different.
", "rawMarkdown": "In this competition almost everybody was serious about feature engineering and development of ML models until they weren't. Only those who joined in the last 10 days entered directly into the realm 0.6XX scores. It's just like with anything else in life: You adapt, or you are left behind! This isn't any different.", "votes": 1 } ] }, { "id": 2821803, "postDate": "2024-05-18T09:00:09.393Z", "content": "I think one could exit a competition silently if it does not align with the person's goals and objectives.
", "rawMarkdown": "\nI think one could exit a competition silently if it does not align with the person's goals and objectives.", "votes": -13 }, { "id": 2835412, "postDate": "2024-05-25T10:02:08.817Z", "content": "Datascience competition -> metric hacking competition
", "rawMarkdown": "Datascience competition -> metric hacking competition" }, { "id": 2834259, "postDate": "2024-05-24T16:30:31.860Z", "content": "Participating in my first competition, I saw a lot of excellent approaches
", "rawMarkdown": "Participating in my first competition, I saw a lot of excellent approaches" }, { "id": 2833228, "postDate": "2024-05-24T05:48:43.627Z", "content": "……………
", "rawMarkdown": "..............." }, { "id": 2830121, "postDate": "2024-05-23T02:49:08.040Z", "content": "Sorry for those who invested much
", "rawMarkdown": "Sorry for those who invested much" }, { "id": 2829754, "postDate": "2024-05-22T18:58:57.843Z", "content": "@yangq369 this was horrible indeed, or unfortunately.
", "rawMarkdown": "@yangq369 this was horrible indeed, or unfortunately. " }, { "id": 2827792, "postDate": "2024-05-21T16:37:17.600Z", "content": "not so well
", "rawMarkdown": "not so well\n" }, { "id": 2823597, "postDate": "2024-05-19T10:14:37.397Z", "content": "lol i agreee it swas not good
", "rawMarkdown": "lol i agreee it swas not good" }, { "id": 2822729, "postDate": "2024-05-18T18:23:11.240Z", "content": "Very true @bratkovskyevgeny!
", "rawMarkdown": "Very true @bratkovskyevgeny!" }, { "id": 2821760, "postDate": "2024-05-18T08:24:35.070Z", "content": "@tomasjeline2 It isn't fair. The judging criteria for the competition need to be revised.
", "rawMarkdown": "@tomasjeline2 It isn't fair. The judging criteria for the competition need to be revised.\n" }, { "id": 2829524, "postDate": "2024-05-22T16:02:07.977Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 2823576, "postDate": "2024-05-19T09:57:30.140Z", "content": "Very true .
", "rawMarkdown": "Very true . ", "isDeleted": true } ], "comments": [ { "id": 2821390, "author_name": "Farukcan Saglam", "author_url": "", "post_date": "2024-05-18T05:13:56.827000", "content": "Kaggle should add a leave the competition button for such situations.
", "votes": 19, "replies": [ { "id": 2821402, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-05-18T05:19:11.870000", "content": "@greysky this is a very useful feature and in my opinion, should be present in all competitions.
\nI think this should be present in teams as well- named as disband team and exit competition/ disassociate from team and exit competition as well.
\nAlso, Kaggle could consider a system where team-members rate one another. This will help future teams understand the potential value-add from a given member should they choose to collaborate.
", "votes": 17, "replies": [ { "id": 2829637, "author_name": "Taizhuo Tang", "author_url": "", "post_date": "2024-05-22T17:42:57.187000", "content": "@ravi20076 any idea why they don't have this? I imagine leaving/disbanding teams creates the opportunity for abuse (eg. taking a teammate's work and then splitting off) but I see no downside for exiting a competition…
", "votes": 2, "replies": [] }, { "id": 2837235, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-05-26T11:35:38.543000", "content": "I agree, when one asks to leave a team, he/ she could leave the competition also @tztang
", "votes": 1, "replies": [] }, { "id": 2838209, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2024-05-26T23:35:14.020000", "content": "@ravi20076,I strongly believe not having an option to exit a competition is not an oversight but a deliberate business decision. They want to sell to potential client how well attended their competitions are. Having said that, if they keep running this kind of problematic competitions, they will chase good companies away from running a competition on this platform anyway. What is the point if the whole thing is diluted and becomes a joke in the ML/ Data Science world.
", "votes": 1, "replies": [] }, { "id": 2838540, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-05-27T05:22:16.567000", "content": "@sheriytm
\nI totally understand your point. Such an activity may impact the host’s marketing activity also.
Yup, that's a good idea.
", "votes": 2, "replies": [] }, { "id": 2829784, "author_name": "Eduard Stefanescu", "author_url": "", "post_date": "2024-05-22T19:35:02.617000", "content": "Sounds great in theory, but then just like with anything else people will abuse it like crazy. Just think about it… people will join lots of competitions in search for gold, silver or bronze… then with only hours left until deadline those who clearly don't have a chance at the podium will leave in droves such competitions. This in turn would cause sudden recalculations of gold, silver or bronze positions in such competitions b/c they are based on percentages of the total number of competitors. It would be madness.
\nAny new idea, and new product that you try to introduce (not only in Kaggle) should start with the critical question \"how is this product going to be abused?… in what ways?\". When you really think about it, a \"leave button\" would bring way more harm than good to Kaggle. It will never happen.
", "votes": 4, "replies": [ { "id": 2830327, "author_name": "Taizhuo Tang", "author_url": "", "post_date": "2024-05-23T06:33:45.477000", "content": "You make a fair point. In the same way that there's a competition \"Merger & Entry\" deadline, maybe have an \"Exit & Disband\" deadline as well? Just wishful thinking of course.
", "votes": 1, "replies": [] }, { "id": 2837239, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-05-26T11:37:22.590000", "content": "Perfect @tztang
\nThese deadlines could be on the same day. We may have a system where we may not perform any activity related to the team formation in the last 7 days of any competition!
I just recently started using kaggle and doing machine learning. I would love to use the \"leave\" button, to be honest. It's too difficult for me somehow 😔
\nso far 😜
Good idea to leave disband team but completion will not benefits both.
", "votes": 1, "replies": [] } ] }, { "id": 2821276, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "2024-05-18T03:37:00.730000", "content": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.
", "votes": 16, "replies": [] }, { "id": 2822583, "author_name": "Danu A.", "author_url": "", "post_date": "2024-05-18T17:13:18.810000", "content": "Competition is very interesting by itself, but the metric and the attitude of the hosts that initially said that will disqualify cheaters by manually checking top notebooks and then changed their minds by allowing to do anything - broke everything and dissapointed a lot of people.
", "votes": 10, "replies": [] }, { "id": 2821680, "author_name": "Evgeny Bratkovsky", "author_url": "", "post_date": "2024-05-18T07:36:48.587000", "content": "It's not within our purview to assess the organizers' choices. They have the prerogative to select the metric they deem appropriate. It's possible that the models they have deployed in their production environment frequently deteriorate over time, prompting them to explore new avenues. I'm not in a position to ascertain their circumstances with certainty. This is merely my conjecture regarding the rationale behind such a metric. In any case, the customer (organizer) has the prerogative to set own requirements. This is essentially how it is in real-world scenarios. I would like to share a case from my professional experience. Frequently, I'm tasked with demonstrating to the customer that their suggestions or ideas are not optimal for a specific task. However, in many instances, we are required to adhere to the customer's specifications, even when it goes against common sense. In this competition, I initially planned to create a model based on log regression and binning. This would have allowed to interpret the model and perform secondary validation based on PSI and HHI. However, I have since realized that such a model would be uncompetitive in comparison with ensembles.
", "votes": 7, "replies": [ { "id": 2821765, "author_name": "", "author_url": "", "post_date": "2024-05-18T08:28:50.377000", "content": "", "votes": -1, "replies": [] }, { "id": 2822487, "author_name": "Bellamkonda Harshitha", "author_url": "", "post_date": "2024-05-18T16:11:39.773000", "content": "very true :)
", "votes": 1, "replies": [] }, { "id": 2826277, "author_name": "Marek Przybyłowicz", "author_url": "", "post_date": "2024-05-20T20:06:44.713000", "content": "It's also possible that this decision didn't come from guys responsible for contact with Kagglers. They (at least at the beginning) were available and responded to the questions etc.. Nevertheless, it leaves a bad aftertaste and is generally seen as a bad move.
", "votes": 2, "replies": [] } ] }, { "id": 2821116, "author_name": "NxGTR", "author_url": "", "post_date": "2024-05-17T23:33:40.857000", "content": "Organizers should reconsider ever hosting another competition. Please do not.
", "votes": 7, "replies": [] }, { "id": 2822993, "author_name": "Bruce", "author_url": "", "post_date": "2024-05-19T00:27:25.230000", "content": "I'm glad I didn't invest a lot of time.
", "votes": 3, "replies": [] }, { "id": 2822823, "author_name": "Eduard Stefanescu", "author_url": "", "post_date": "2024-05-18T19:31:03.333000", "content": "Home Credit Group should simply hire Upstart, or some other company that knows what it's doing in this space. I'm not exactly sure what useful stuff they learned from this competition, other than the fact that their metric has been exposed.
", "votes": 3, "replies": [] }, { "id": 2839067, "author_name": "LucaMTB", "author_url": "", "post_date": "2024-05-27T11:21:42.170000", "content": "I wouldnt day it was a terrible competition. Even thought it was a bit chaotic I learned a lot. And nothing is perfect in live. The important thing is to keep a flexible mindset.
", "votes": 4, "replies": [] }, { "id": 2823927, "author_name": "ZhangYuX1012", "author_url": "", "post_date": "2024-05-19T13:35:32.677000", "content": "It's outrageous that I've spent almost two weeks working on improving models and feature engineering, and the boosted scores aren't as high as gently changing a few numbers in a day today!😅
", "votes": 4, "replies": [] }, { "id": 2824289, "author_name": "Ngọc Thương", "author_url": "", "post_date": "2024-05-19T17:28:39.707000", "content": "In the afternoon, I saw a public post with a high score. I saved it to read again at night. Before reading it, I submitted the post to see how much training results came out, but to my surprise, after reading that code, it came out. It's a trick, not a training process. At first I didn't understand why there were 2 submitted files haha, let's continue working.
", "votes": 2, "replies": [] }, { "id": 2821974, "author_name": "JM", "author_url": "", "post_date": "2024-05-18T10:57:36.567000", "content": "\n\nThe organizer should re-evaluate the results of this game and choose simple and effective evaluation mertric, such as week mean auc.
\n
Didn't they already change their minds twice? From disallowing metric hack to allowing it? This is the result of that flip flop
", "votes": 2, "replies": [] }, { "id": 2827597, "author_name": "Maryam Awadallah", "author_url": "", "post_date": "2024-05-21T15:13:27.567000", "content": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.
", "votes": 0, "replies": [ { "id": 2829787, "author_name": "Eduard Stefanescu", "author_url": "", "post_date": "2024-05-22T19:43:01.597000", "content": "In this competition almost everybody was serious about feature engineering and development of ML models until they weren't. Only those who joined in the last 10 days entered directly into the realm 0.6XX scores. It's just like with anything else in life: You adapt, or you are left behind! This isn't any different.
", "votes": 1, "replies": [] } ] }, { "id": 2821803, "author_name": "Dugyala Sharanya", "author_url": "", "post_date": "2024-05-18T09:00:09.393000", "content": "I think one could exit a competition silently if it does not align with the person's goals and objectives.
", "votes": -13, "replies": [] }, { "id": 2835412, "author_name": "Ning Wang", "author_url": "", "post_date": "2024-05-25T10:02:08.817000", "content": "Datascience competition -> metric hacking competition
", "votes": 0, "replies": [] }, { "id": 2834259, "author_name": "Kazybek Kassym", "author_url": "", "post_date": "2024-05-24T16:30:31.860000", "content": "Participating in my first competition, I saw a lot of excellent approaches
", "votes": 0, "replies": [] }, { "id": 2833228, "author_name": "PeterPccc", "author_url": "", "post_date": "2024-05-24T05:48:43.627000", "content": "……………
", "votes": 0, "replies": [] }, { "id": 2830121, "author_name": "Benjamin Atiemo", "author_url": "", "post_date": "2024-05-23T02:49:08.040000", "content": "Sorry for those who invested much
", "votes": 0, "replies": [] }, { "id": 2829754, "author_name": "Muhammed Tausif", "author_url": "", "post_date": "2024-05-22T18:58:57.843000", "content": "@yangq369 this was horrible indeed, or unfortunately.
", "votes": 0, "replies": [] }, { "id": 2827792, "author_name": "divyanshu_3333", "author_url": "", "post_date": "2024-05-21T16:37:17.600000", "content": "not so well
", "votes": 0, "replies": [] }, { "id": 2823597, "author_name": "Aviral Jain", "author_url": "", "post_date": "2024-05-19T10:14:37.397000", "content": "lol i agreee it swas not good
", "votes": 0, "replies": [] }, { "id": 2822729, "author_name": "Sheema Zain", "author_url": "", "post_date": "2024-05-18T18:23:11.240000", "content": "Very true @bratkovskyevgeny!
", "votes": 0, "replies": [] }, { "id": 2821760, "author_name": "Chinnn", "author_url": "", "post_date": "2024-05-18T08:24:35.070000", "content": "@tomasjeline2 It isn't fair. The judging criteria for the competition need to be revised.
", "votes": 0, "replies": [] }, { "id": 2829524, "author_name": "", "author_url": "", "post_date": "2024-05-22T16:02:07.977000", "content": "", "votes": 1, "replies": [] }, { "id": 2823576, "author_name": "", "author_url": "", "post_date": "2024-05-19T09:57:30.140000", "content": "Very true .
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2819590": "Toward the end of the game, I would say, this was a really terrible competition, all because of a terrible metric. The organizers wasted money and data, and the kagglers wasted time and made some poor models. I was expecting to see some wonderful feature engineering and model ideas, but now I am really disappointed.\nThe organizer should re-evaluate the results of this game and choose simple and effective evaluation mertric, such as week mean auc.", "2821390": "Kaggle should add a leave the competition button for such situations. ", "2821276": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.", "2822583": "Competition is very interesting by itself, but the metric and the attitude of the hosts that initially said that will disqualify cheaters by manually checking top notebooks and then changed their minds by allowing to do anything - broke everything and dissapointed a lot of people.", "2821680": "It's not within our purview to assess the organizers' choices. They have the prerogative to select the metric they deem appropriate. It's possible that the models they have deployed in their production environment frequently deteriorate over time, prompting them to explore new avenues. I'm not in a position to ascertain their circumstances with certainty. This is merely my conjecture regarding the rationale behind such a metric. In any case, the customer (organizer) has the prerogative to set own requirements. This is essentially how it is in real-world scenarios. I would like to share a case from my professional experience. Frequently, I'm tasked with demonstrating to the customer that their suggestions or ideas are not optimal for a specific task. However, in many instances, we are required to adhere to the customer's specifications, even when it goes against common sense. In this competition, I initially planned to create a model based on log regression and binning. This would have allowed to interpret the model and perform secondary validation based on PSI and HHI. However, I have since realized that such a model would be uncompetitive in comparison with ensembles. ", "2821116": "Organizers should reconsider ever hosting another competition. Please do not.", "2822993": "I'm glad I didn't invest a lot of time.", "2822823": "Home Credit Group should simply hire Upstart, or some other company that knows what it's doing in this space. I'm not exactly sure what useful stuff they learned from this competition, other than the fact that their metric has been exposed. ", "2839067": "I wouldnt day it was a terrible competition. Even thought it was a bit chaotic I learned a lot. And nothing is perfect in live. The important thing is to keep a flexible mindset. ", "2823927": "It's outrageous that I've spent almost two weeks working on improving models and feature engineering, and the boosted scores aren't as high as gently changing a few numbers in a day today!😅", "2824289": "In the afternoon, I saw a public post with a high score. I saved it to read again at night. Before reading it, I submitted the post to see how much training results came out, but to my surprise, after reading that code, it came out. It's a trick, not a training process. At first I didn't understand why there were 2 submitted files haha, let's continue working.", "2821974": ">The organizer should re-evaluate the results of this game and choose simple and effective evaluation mertric, such as week mean auc. \n\nDidn't they already change their minds twice? From disallowing metric hack to allowing it? This is the result of that flip flop", "2827597": "Today I looked at the ranking list and it was all 0.6XX, which is very scary. If these people achieve good results on the private list, it would be unfriendly to the contestants who are serious about feature engineering.", "2821803": "\nI think one could exit a competition silently if it does not align with the person's goals and objectives.", "2835412": "Datascience competition -> metric hacking competition", "2834259": "Participating in my first competition, I saw a lot of excellent approaches", "2833228": "...............", "2830121": "Sorry for those who invested much", "2829754": "@yangq369 this was horrible indeed, or unfortunately. ", "2827792": "not so well\n", "2823597": "lol i agreee it swas not good", "2822729": "Very true @bratkovskyevgeny!", "2821760": "@tomasjeline2 It isn't fair. The judging criteria for the competition need to be revised.\n", "2829524": "", "2823576": "Very true . " } }