When I'm only using the new dataset, my best single model get 0.8837 cv/ 0.768 lb. But when I conduct experiment on combined dataset, the cv easily passes 0.91 after a few epochs. I'm not sure the whether the high offline score of using combined dataset would contribute to the LB.
\nSo which dataset are you using that achieves the best lb result? new one or combined one?
", "messages": [ { "id": 1421332, "postDate": "2021-08-03T04:00:25.987Z", "content": "When I'm only using the new dataset, my best single model get 0.8837 cv/ 0.768 lb. But when I conduct experiment on combined dataset, the cv easily passes 0.91 after a few epochs. I'm not sure the whether the high offline score of using combined dataset would contribute to the LB.
\nSo which dataset are you using that achieves the best lb result? new one or combined one?
", "rawMarkdown": "When I'm only using the new dataset, my best single model get **0.8837 cv/ 0.768 lb**. But when I conduct experiment on combined dataset, the cv easily passes **0.91** after a few epochs. I'm not sure the whether the high offline score of using combined dataset would contribute to the LB. \n\nSo which dataset are you using that achieves the best lb result? new one or combined one?", "votes": 2 }, { "id": 1427884, "postDate": "2021-08-03T08:53:26.910Z", "content": "One tip: When using the combined dataset in order to get a better feedback for the performance improvement that old data gives, use in cv-test folds only data from the new set. In this way you can accurately make a comparison between the 2 approaches. Practically keep the 5 testing folds the same like in the approach with the only the new dataset(in a 5 cv methodology) while enhance with the old data only the 5 training folds
", "rawMarkdown": "One tip: When using the combined dataset in order to get a better feedback for the performance improvement that old data gives, use in cv-test folds only data from the new set. In this way you can accurately make a comparison between the 2 approaches. Practically keep the 5 testing folds the same like in the approach with the only the new dataset(in a 5 cv methodology) while enhance with the old data only the 5 training folds", "votes": 1, "replies": [ { "id": 1428637, "postDate": "2021-08-03T09:40:27.313Z", "content": "Good idea ! Validation on new_train_only is much more reliable than validation on combined dataset.
", "rawMarkdown": "Good idea ! Validation on new_train_only is much more reliable than validation on combined dataset." } ] }, { "id": 1430288, "postDate": "2021-08-03T11:29:27.583Z", "content": "combined data didn't work for me
", "rawMarkdown": "combined data didn't work for me", "votes": 2, "replies": [ { "id": 1430782, "postDate": "2021-08-03T11:55:20.497Z", "content": "Yeah, I just submitted the combined-data model, the lb score is around 0.71, which is a pretty bad result.
", "rawMarkdown": "Yeah, I just submitted the combined-data model, the lb score is around 0.71, which is a pretty bad result.", "votes": 1 } ] } ], "comments": [ { "id": 1427884, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2021-08-03T08:53:26.910000", "content": "One tip: When using the combined dataset in order to get a better feedback for the performance improvement that old data gives, use in cv-test folds only data from the new set. In this way you can accurately make a comparison between the 2 approaches. Practically keep the 5 testing folds the same like in the approach with the only the new dataset(in a 5 cv methodology) while enhance with the old data only the 5 training folds
", "votes": 1, "replies": [ { "id": 1428637, "author_name": "YTEP (Jiazhi Yang)", "author_url": "", "post_date": "2021-08-03T09:40:27.313000", "content": "Good idea ! Validation on new_train_only is much more reliable than validation on combined dataset.
", "votes": 0, "replies": [] } ] }, { "id": 1430288, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-08-03T11:29:27.583000", "content": "combined data didn't work for me
", "votes": 2, "replies": [ { "id": 1430782, "author_name": "YTEP (Jiazhi Yang)", "author_url": "", "post_date": "2021-08-03T11:55:20.497000", "content": "Yeah, I just submitted the combined-data model, the lb score is around 0.71, which is a pretty bad result.
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1421332": "When I'm only using the new dataset, my best single model get **0.8837 cv/ 0.768 lb**. But when I conduct experiment on combined dataset, the cv easily passes **0.91** after a few epochs. I'm not sure the whether the high offline score of using combined dataset would contribute to the LB. \n\nSo which dataset are you using that achieves the best lb result? new one or combined one?", "1427884": "One tip: When using the combined dataset in order to get a better feedback for the performance improvement that old data gives, use in cv-test folds only data from the new set. In this way you can accurately make a comparison between the 2 approaches. Practically keep the 5 testing folds the same like in the approach with the only the new dataset(in a 5 cv methodology) while enhance with the old data only the 5 training folds", "1430288": "combined data didn't work for me" } }