I have been trying this for a 2 weeks now, not too long but tried a bunch of different models and all models seem to be performing horribly. I seems to be nowhere compared to where i was 2 weeks ago.
\n\nresnet34\nresnet50\ndensenet201\nefficientnetB4
\n\ni split the data according to id_code and create a stratified split. for all 1108 classes. i use fastai custom head so that the head is 1108. I trained for 300 epochs to 400 epochs with different batch sizes in different runs. apart from heating my room for 2 weeks, the networks gave very poor LB scores like 0.05 or 0.07.
\n\nwhat is the best way to create a good CV split ? CV improves to .7 but LB is like 0.005.
", "messages": [ { "id": "632920", "postDate": "09/24/2019 08:10:50", "content": "I have been trying this for a 2 weeks now, not too long but tried a bunch of different models and all models seem to be performing horribly. I seems to be nowhere compared to where i was 2 weeks ago.
\n\nresnet34\nresnet50\ndensenet201\nefficientnetB4
\n\ni split the data according to id_code and create a stratified split. for all 1108 classes. i use fastai custom head so that the head is 1108. I trained for 300 epochs to 400 epochs with different batch sizes in different runs. apart from heating my room for 2 weeks, the networks gave very poor LB scores like 0.05 or 0.07.
\n\nwhat is the best way to create a good CV split ? CV improves to .7 but LB is like 0.005.
", "rawMarkdown": "I have been trying this for a 2 weeks now, not too long but tried a bunch of different models and all models seem to be performing horribly. I seems to be nowhere compared to where i was 2 weeks ago.\n\nresnet34\nresnet50\ndensenet201\nefficientnetB4\n\ni split the data according to id_code and create a stratified split. for all 1108 classes. i use fastai custom head so that the head is 1108. I trained for 300 epochs to 400 epochs with different batch sizes in different runs. apart from heating my room for 2 weeks, the networks gave very poor LB scores like 0.05 or 0.07. \n\nwhat is the best way to create a good CV split ? CV improves to .7 but LB is like 0.005.", "votes": null }, { "id": "632957", "postDate": "09/24/2019 08:56:11", "content": "It has been discussed here. \nhttps://www.kaggle.com/c/recursion-cellular-image-classification/discussion/98116
", "rawMarkdown": "It has been discussed here. \nhttps://www.kaggle.com/c/recursion-cellular-image-classification/discussion/98116", "votes": null }, { "id": "632970", "postDate": "09/24/2019 09:11:42", "content": "If you compare the test and training sets you'll see they have different experiments, so to be fair on your CV you should also use different experiments right?\nI've been using about 1/3 for for validation (3 folds)
\n\nWith proper hyper params and regularisation you should get to CV and LB 0.3 in only a few epochs (like 6?), no need to heat up your room! lol\nmaybe try higher lr and momentum ;)
", "rawMarkdown": "If you compare the test and training sets you'll see they have different experiments, so to be fair on your CV you should also use different experiments right?\nI've been using about 1/3 for for validation (3 folds)\n\nWith proper hyper params and regularisation you should get to CV and LB 0.3 in only a few epochs (like 6?), no need to heat up your room! lol\nmaybe try higher lr and momentum ;)", "votes": null }, { "id": "633018", "postDate": "09/24/2019 10:50:53", "content": "do you mean regularization on network architecture?
", "rawMarkdown": "do you mean regularization on network architecture?", "votes": null }, { "id": "633252", "postDate": "09/24/2019 15:55:09", "content": "yes, makes sense, will try this split. thank you very much for the help.
", "rawMarkdown": "yes, makes sense, will try this split. thank you very much for the help.", "votes": null }, { "id": "633367", "postDate": "09/24/2019 20:14:24", "content": "@projdev just weight decay and a bit of augmentation should be enough for LB 0.3, even without any submission tricks, or is your experience different?
", "rawMarkdown": "projdev just weight decay and a bit of augmentation should be enough for LB 0.3, even without any submission tricks, or is your experience different?", "votes": null } ], "comments": [ { "id": 632957, "author_name": "analokamus", "author_url": "", "post_date": "09/24/2019 08:56:11", "content": "It has been discussed here. \nhttps://www.kaggle.com/c/recursion-cellular-image-classification/discussion/98116
", "votes": null, "replies": [] }, { "id": 632970, "author_name": "hmendonca", "author_url": "", "post_date": "09/24/2019 09:11:42", "content": "If you compare the test and training sets you'll see they have different experiments, so to be fair on your CV you should also use different experiments right?\nI've been using about 1/3 for for validation (3 folds)
\n\nWith proper hyper params and regularisation you should get to CV and LB 0.3 in only a few epochs (like 6?), no need to heat up your room! lol\nmaybe try higher lr and momentum ;)
", "votes": null, "replies": [ { "id": 633018, "author_name": "projdev", "author_url": "", "post_date": "09/24/2019 10:50:53", "content": "do you mean regularization on network architecture?
", "votes": null, "replies": [] }, { "id": 633252, "author_name": "darshansonde", "author_url": "", "post_date": "09/24/2019 15:55:09", "content": "yes, makes sense, will try this split. thank you very much for the help.
", "votes": null, "replies": [] }, { "id": 633367, "author_name": "hmendonca", "author_url": "", "post_date": "09/24/2019 20:14:24", "content": "@projdev just weight decay and a bit of augmentation should be enough for LB 0.3, even without any submission tricks, or is your experience different?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "632920": "I have been trying this for a 2 weeks now, not too long but tried a bunch of different models and all models seem to be performing horribly. I seems to be nowhere compared to where i was 2 weeks ago.\n\nresnet34\nresnet50\ndensenet201\nefficientnetB4\n\ni split the data according to id_code and create a stratified split. for all 1108 classes. i use fastai custom head so that the head is 1108. I trained for 300 epochs to 400 epochs with different batch sizes in different runs. apart from heating my room for 2 weeks, the networks gave very poor LB scores like 0.05 or 0.07. \n\nwhat is the best way to create a good CV split ? CV improves to .7 but LB is like 0.005.", "632957": "It has been discussed here. \nhttps://www.kaggle.com/c/recursion-cellular-image-classification/discussion/98116", "632970": "If you compare the test and training sets you'll see they have different experiments, so to be fair on your CV you should also use different experiments right?\nI've been using about 1/3 for for validation (3 folds)\n\nWith proper hyper params and regularisation you should get to CV and LB 0.3 in only a few epochs (like 6?), no need to heat up your room! lol\nmaybe try higher lr and momentum ;)", "633018": "do you mean regularization on network architecture?", "633252": "yes, makes sense, will try this split. thank you very much for the help.", "633367": "projdev just weight decay and a bit of augmentation should be enough for LB 0.3, even without any submission tricks, or is your experience different?" }, "source": "meta" }