I use all day 6 as cv, and get 0.91x score in LB, but in local day 6 cv I get 0.997x ;I have used part of day 9 as cv, and get higher score like 0.979x with same features before.
\n\nI saw some threads like https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55821, said it should choose same hour as test set;
\n\nSo I can't choose day 6 as cv?
", "messages": [ { "id": "322946", "postDate": "05/04/2018 01:41:57", "content": "I use all day 6 as cv, and get 0.91x score in LB, but in local day 6 cv I get 0.997x ;I have used part of day 9 as cv, and get higher score like 0.979x with same features before.
\n\nI saw some threads like https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55821, said it should choose same hour as test set;
\n\nSo I can't choose day 6 as cv?
", "rawMarkdown": "I use all day 6 as cv, and get 0.91x score in LB, but in local day 6 cv I get 0.997x ;I have used part of day 9 as cv, and get higher score like 0.979x with same features before.\n\nI saw some threads like https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55821, said it should choose same hour as test set; \n\nSo I can't choose day 6 as cv?", "votes": null }, { "id": "322952", "postDate": "05/04/2018 02:16:28", "content": "The nature of click behavior changes as the days go by.
\n\nIt's important to have some of the later dates for cross-validation.
", "rawMarkdown": "The nature of click behavior changes as the days go by.\n\nIt's important to have some of the later dates for cross-validation.", "votes": null }, { "id": "322965", "postDate": "05/04/2018 03:10:09", "content": "try the last 10M data in day9
", "rawMarkdown": "try the last 10M data in day9", "votes": null }, { "id": "323022", "postDate": "05/04/2018 07:14:29", "content": "yes,I used to try day 9 last 2.5M as cv. Someday I have a idea that use a whole day as cv set, and I choose day 6(it's smaller than other day )
", "rawMarkdown": "yes,I used to try day 9 last 2.5M as cv. Someday I have a idea that use a whole day as cv set, and I choose day 6(it's smaller than other day )", "votes": null }, { "id": "323037", "postDate": "05/04/2018 07:41:50", "content": "thanks for replying, as you say, clicking behavior changes day by day, so maybe should choose a whole day apart from train set as validation data.
", "rawMarkdown": "thanks for replying, as you say, clicking behavior changes day by day, so maybe should choose a whole day apart from train set as validation data.", "votes": null }, { "id": "323039", "postDate": "05/04/2018 07:42:56", "content": "Maybe I'm wrong.Because LB has decrease when I choose whole day 6 as cv
", "rawMarkdown": "Maybe I'm wrong.Because LB has decrease when I choose whole day 6 as cv", "votes": null }, { "id": "323395", "postDate": "05/05/2018 03:05:08", "content": "if you use \"Day\" as a feature, then you maybe use some later days data in train, I use one day for both train and split 10% for validation, and the feature importance for \"Day\" is 0. Hope this help.
", "rawMarkdown": "if you use \"Day\" as a feature, then you maybe use some later days data in train, I use one day for both train and split 10% for validation, and the feature importance for \"Day\" is 0. Hope this help.", "votes": null } ], "comments": [ { "id": 322952, "author_name": "danieljbrooks", "author_url": "", "post_date": "05/04/2018 02:16:28", "content": "The nature of click behavior changes as the days go by.
\n\nIt's important to have some of the later dates for cross-validation.
", "votes": null, "replies": [ { "id": 323037, "author_name": "liuhdsgoal", "author_url": "", "post_date": "05/04/2018 07:41:50", "content": "thanks for replying, as you say, clicking behavior changes day by day, so maybe should choose a whole day apart from train set as validation data.
", "votes": null, "replies": [] }, { "id": 323039, "author_name": "liuhdsgoal", "author_url": "", "post_date": "05/04/2018 07:42:56", "content": "Maybe I'm wrong.Because LB has decrease when I choose whole day 6 as cv
", "votes": null, "replies": [] } ] }, { "id": 322965, "author_name": "cnzjhdx", "author_url": "", "post_date": "05/04/2018 03:10:09", "content": "try the last 10M data in day9
", "votes": null, "replies": [ { "id": 323022, "author_name": "liuhdsgoal", "author_url": "", "post_date": "05/04/2018 07:14:29", "content": "yes,I used to try day 9 last 2.5M as cv. Someday I have a idea that use a whole day as cv set, and I choose day 6(it's smaller than other day )
", "votes": null, "replies": [] } ] }, { "id": 323395, "author_name": "creamiracle", "author_url": "", "post_date": "05/05/2018 03:05:08", "content": "if you use \"Day\" as a feature, then you maybe use some later days data in train, I use one day for both train and split 10% for validation, and the feature importance for \"Day\" is 0. Hope this help.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "322946": "I use all day 6 as cv, and get 0.91x score in LB, but in local day 6 cv I get 0.997x ;I have used part of day 9 as cv, and get higher score like 0.979x with same features before.\n\nI saw some threads like https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55821, said it should choose same hour as test set; \n\nSo I can't choose day 6 as cv?", "322952": "The nature of click behavior changes as the days go by.\n\nIt's important to have some of the later dates for cross-validation.", "322965": "try the last 10M data in day9", "323022": "yes,I used to try day 9 last 2.5M as cv. Someday I have a idea that use a whole day as cv set, and I choose day 6(it's smaller than other day )", "323037": "thanks for replying, as you say, clicking behavior changes day by day, so maybe should choose a whole day apart from train set as validation data.", "323039": "Maybe I'm wrong.Because LB has decrease when I choose whole day 6 as cv", "323395": "if you use \"Day\" as a feature, then you maybe use some later days data in train, I use one day for both train and split 10% for validation, and the feature importance for \"Day\" is 0. Hope this help." }, "source": "meta" }