Just reviewed my score submissions and found that I could have won a bronze - had i selected this as my final submission, but my public lb score was too low to convince me
\n\nHere was my solution for the bronze medal\nhttps://www.kaggle.com/roydatascience/eda-iso-pca-lle-stratified-lstm-attention?scriptVersionId=11248695
", "messages": [ { "id": 496171, "postDate": "2019-03-22T00:27:40.433Z", "content": "Just reviewed my score submissions and found that I could have won a bronze - had i selected this as my final submission, but my public lb score was too low to convince me
\n\nHere was my solution for the bronze medal\nhttps://www.kaggle.com/roydatascience/eda-iso-pca-lle-stratified-lstm-attention?scriptVersionId=11248695
", "rawMarkdown": "Just reviewed my score submissions and found that I could have won a bronze - had i selected this as my final submission, but my public lb score was too low to convince me\n\nHere was my solution for the bronze medal\nhttps://www.kaggle.com/roydatascience/eda-iso-pca-lle-stratified-lstm-attention?scriptVersionId=11248695 ", "votes": 8 }, { "id": 496175, "postDate": "2019-03-22T00:34:28.987Z", "content": "Reviewed our submissions too and best private is 0.704 (Top 3) but we would never have selected this one because public LB was really too low (0.593) and we had better local CV with other models.
", "rawMarkdown": "Reviewed our submissions too and best private is 0.704 (Top 3) but we would never have selected this one because public LB was really too low (0.593) and we had better local CV with other models.", "votes": 1, "replies": [ { "id": 496179, "postDate": "2019-03-22T00:36:09.883Z", "content": "Yes this was truly unexpected.
", "rawMarkdown": "Yes this was truly unexpected.", "votes": 4 }, { "id": 496185, "postDate": "2019-03-22T00:43:33.617Z", "content": "Yeap, it seems we miss something because we followed the golden rules. I'm impressed to see great results with only 2 or 3 submissions in top 50.
", "rawMarkdown": "Yeap, it seems we miss something because we followed the golden rules. I'm impressed to see great results with only 2 or 3 submissions in top 50.", "votes": 1 } ] }, { "id": 496173, "postDate": "2019-03-22T00:30:19.747Z", "content": "CV Score = 0.6787. Thats why the experts suggest to trust the CV or LB.
", "rawMarkdown": "CV Score = 0.6787. Thats why the experts suggest to trust the CV or LB.", "votes": 2, "replies": [ { "id": 496180, "postDate": "2019-03-22T00:36:33.280Z", "content": "@roydatascience, there is a reason you did not chose it. i.e. you did not trust your local CV . It was really hard to setup a goood CV for this comeptition. I posted a thread to discuss this issue here. By the way I have a \nbetter submission that would have landed me in bronze as well but dod not select it :-(
\n\nMy best on hindsight is a CatBoost model with public LB = 0.648, private LB=0.655, and CV scores 0.719 ± 0.094
", "rawMarkdown": "@roydatascience, there is a reason you did not chose it. i.e. you did not trust your local CV . It was really hard to setup a goood CV for this comeptition. I posted a [thread to discuss this issue here](https://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/85143). By the way I have a \nbetter submission that would have landed me in bronze as well but dod not select it :-(\n\nMy best on hindsight is a [CatBoost model with public LB = 0.648, private LB=0.655, a](https://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/85146)nd CV scores 0.719 ± 0.094\n", "votes": 1 }, { "id": 496189, "postDate": "2019-03-22T00:44:32.187Z", "content": "all my subs using adversarial validation worked better overall than the others on final lb, but like a lot of people i did not trust them because of really low public lb score :/
", "rawMarkdown": "all my subs using adversarial validation worked better overall than the others on final lb, but like a lot of people i did not trust them because of really low public lb score :/" }, { "id": 496198, "postDate": "2019-03-22T00:59:01.950Z", "content": "Thats true, sometimes i learn the things hard way
", "rawMarkdown": "Thats true, sometimes i learn the things hard way", "votes": 3 }, { "id": 496214, "postDate": "2019-03-22T01:22:33.137Z", "content": "@roydatascience, do not worry. If you had chosen that model it would have been a guessing game right? So going with our intuition and following the data facts is what would serve us best in the long run. Hence, I do not have any regrets about not choosing my best model either :-)
", "rawMarkdown": "@roydatascience, do not worry. If you had chosen that model it would have been a guessing game right? So going with our intuition and following the data facts is what would serve us best in the long run. Hence, I do not have any regrets about not choosing my best model either :-)" }, { "id": 496228, "postDate": "2019-03-22T01:39:09.737Z", "content": "Moreover making notes would be a good practice, i shall note down the CV and LB’s after every submission and then select the two submission, one best with CV, other best in LB
", "rawMarkdown": "Moreover making notes would be a good practice, i shall note down the CV and LB’s after every submission and then select the two submission, one best with CV, other best in LB", "votes": 4 } ] }, { "id": 496174, "postDate": "2019-03-22T00:32:44.353Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 496487, "postDate": "2019-03-22T09:11:45.470Z", "content": "I have a submission with Private=0.414 and Public=0.142! Ok, both of them are poor scores ... but I cannot realize why a difference like this can happen. Could anyone help me in understanding this? Just to better figure out how Kaggle works for future competitions ...
", "rawMarkdown": "I have a submission with Private=0.414 and Public=0.142! Ok, both of them are poor scores ... but I cannot realize why a difference like this can happen. Could anyone help me in understanding this? Just to better figure out how Kaggle works for future competitions ..." } ] } ], "comments": [ { "id": 496175, "author_name": "MPWARE", "author_url": "", "post_date": "2019-03-22T00:34:28.987000", "content": "Reviewed our submissions too and best private is 0.704 (Top 3) but we would never have selected this one because public LB was really too low (0.593) and we had better local CV with other models.
", "votes": 1, "replies": [ { "id": 496179, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-03-22T00:36:09.883000", "content": "Yes this was truly unexpected.
", "votes": 4, "replies": [] }, { "id": 496185, "author_name": "MPWARE", "author_url": "", "post_date": "2019-03-22T00:43:33.617000", "content": "Yeap, it seems we miss something because we followed the golden rules. I'm impressed to see great results with only 2 or 3 submissions in top 50.
", "votes": 1, "replies": [] } ] }, { "id": 496173, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-03-22T00:30:19.747000", "content": "CV Score = 0.6787. Thats why the experts suggest to trust the CV or LB.
", "votes": 2, "replies": [ { "id": 496180, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-03-22T00:36:33.280000", "content": "@roydatascience, there is a reason you did not chose it. i.e. you did not trust your local CV . It was really hard to setup a goood CV for this comeptition. I posted a thread to discuss this issue here. By the way I have a \nbetter submission that would have landed me in bronze as well but dod not select it :-(
\n\nMy best on hindsight is a CatBoost model with public LB = 0.648, private LB=0.655, and CV scores 0.719 ± 0.094
", "votes": 1, "replies": [] }, { "id": 496189, "author_name": "Guillaume", "author_url": "", "post_date": "2019-03-22T00:44:32.187000", "content": "all my subs using adversarial validation worked better overall than the others on final lb, but like a lot of people i did not trust them because of really low public lb score :/
", "votes": 0, "replies": [] }, { "id": 496198, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-03-22T00:59:01.950000", "content": "Thats true, sometimes i learn the things hard way
", "votes": 3, "replies": [] }, { "id": 496214, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-03-22T01:22:33.137000", "content": "@roydatascience, do not worry. If you had chosen that model it would have been a guessing game right? So going with our intuition and following the data facts is what would serve us best in the long run. Hence, I do not have any regrets about not choosing my best model either :-)
", "votes": 0, "replies": [] }, { "id": 496228, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-03-22T01:39:09.737000", "content": "Moreover making notes would be a good practice, i shall note down the CV and LB’s after every submission and then select the two submission, one best with CV, other best in LB
", "votes": 4, "replies": [] } ] }, { "id": 496174, "author_name": "", "author_url": "", "post_date": "2019-03-22T00:32:44.353000", "content": "", "votes": 1, "replies": [ { "id": 496487, "author_name": "Ludovico Ristori", "author_url": "", "post_date": "2019-03-22T09:11:45.470000", "content": "I have a submission with Private=0.414 and Public=0.142! Ok, both of them are poor scores ... but I cannot realize why a difference like this can happen. Could anyone help me in understanding this? Just to better figure out how Kaggle works for future competitions ...
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "496171": "Just reviewed my score submissions and found that I could have won a bronze - had i selected this as my final submission, but my public lb score was too low to convince me\n\nHere was my solution for the bronze medal\nhttps://www.kaggle.com/roydatascience/eda-iso-pca-lle-stratified-lstm-attention?scriptVersionId=11248695 ", "496175": "Reviewed our submissions too and best private is 0.704 (Top 3) but we would never have selected this one because public LB was really too low (0.593) and we had better local CV with other models.", "496173": "CV Score = 0.6787. Thats why the experts suggest to trust the CV or LB.", "496174": "" } }