I want to see how far can we go with just one model(and for best one fold), or it's just a ensembling game.
\nFor me:
\nmodel: efficientnet_b0
\nfold: trained on single fold (4/5 of the dataset)
\nlocal ROC AUC: 0.986
\npublic ROC AUC: 0.97 ( based on \"Score Sorting\", I guess it should be around ~ 0.977 )
\nTTA: no tta
Edit:
\nI have tried my best pipeline(above) with 5 folds to ensemble, the public score is better but does not achieve 0.98, so I think the naive ensemble(average) boost is small.
", "messages": [ { "id": 1311691, "postDate": "2021-05-17T14:56:18.113Z", "content": "I want to see how far can we go with just one model(and for best one fold), or it's just a ensembling game.
\nFor me:
\nmodel: efficientnet_b0
\nfold: trained on single fold (4/5 of the dataset)
\nlocal ROC AUC: 0.986
\npublic ROC AUC: 0.97 ( based on \"Score Sorting\", I guess it should be around ~ 0.977 )
\nTTA: no tta
Edit:
\nI have tried my best pipeline(above) with 5 folds to ensemble, the public score is better but does not achieve 0.98, so I think the naive ensemble(average) boost is small.
", "rawMarkdown": "I want to see how far can we go with just one model(and for best one fold), or it's just a ensembling game.\n\nFor me:\n\nmodel: efficientnet_b0\nfold: trained on single fold (4/5 of the dataset)\nlocal ROC AUC: 0.986\npublic ROC AUC: 0.97 ( based on \"Score Sorting\", I guess it should be around ~ 0.977 )\nTTA: no tta\n\n\nEdit:\n\nI have tried my best pipeline(above) with 5 folds to ensemble, the public score is better but does not achieve 0.98, so I think the naive ensemble(average) boost is small.\n", "votes": 14 }, { "id": 1318750, "postDate": "2021-05-22T14:41:25.670Z", "content": "model : Nfnet_l0
\nLocal Roc Auc : 0.9837 [4 folds ] , 0.9821[OOF]
\nPublic Roc Auc : 0.97
\nI think there is smaller number of +ve samples in Public Test and hence the Public Score may not be a Good Indicator of Generalization of Model , given that lot of people have got 0.97 till now !
It's weired that NFNet never works for me. I will give it another chance someday.
", "rawMarkdown": "It's weired that NFNet never works for me. I will give it another chance someday.", "votes": 1 }, { "id": 1319258, "postDate": "2021-05-23T04:27:28.753Z", "content": "@snaker it's all about training pipeline :)
", "rawMarkdown": "@snaker it's all about training pipeline :)" } ] }, { "id": 1319260, "postDate": "2021-05-23T04:30:11.710Z", "content": "Why is there a big gap between your CV and LB, and I also use B0 network cv97.7 LB 97 + which is very close？
", "rawMarkdown": "Why is there a big gap between your CV and LB, and I also use B0 network cv97.7 LB 97 + which is very close？", "replies": [ { "id": 1319295, "postDate": "2021-05-23T05:18:27.257Z", "content": "Not sure, I'm using stratified K fold
", "rawMarkdown": "Not sure, I'm using stratified K fold" } ] }, { "id": 1313343, "postDate": "2021-05-18T14:14:21.510Z", "content": "May I ask why did you choose a single fold and not e.g 4 folds?
", "rawMarkdown": "May I ask why did you choose a single fold and not e.g 4 folds?", "replies": [ { "id": 1314085, "postDate": "2021-05-19T00:25:41.730Z", "content": "I split dataset into 5 folds, but currently only trained on 4 of them for quickly validation. May be in the last month of this game i will start to train on 5 fold and ensemble.
", "rawMarkdown": "I split dataset into 5 folds, but currently only trained on 4 of them for quickly validation. May be in the last month of this game i will start to train on 5 fold and ensemble." } ] }, { "id": 1312385, "postDate": "2021-05-18T02:54:44.123Z", "content": "What image size are you using?
", "rawMarkdown": "What image size are you using?", "replies": [ { "id": 1312655, "postDate": "2021-05-18T07:06:15.650Z", "content": "original, no resize
", "rawMarkdown": "original, no resize" }, { "id": 1314965, "postDate": "2021-05-19T13:24:06.227Z", "content": "'original' means (273, 256) ? or (1638, 256) ?
", "rawMarkdown": "'original' means (273, 256) ? or (1638, 256) ?" }, { "id": 1315643, "postDate": "2021-05-20T03:25:08.767Z", "content": "it should be 1638, 256
", "rawMarkdown": "it should be 1638, 256", "votes": 2 } ] }, { "id": 1311767, "postDate": "2021-05-17T15:48:33.630Z", "content": "You used sapatial image or channel wise?
", "rawMarkdown": "You used sapatial image or channel wise?", "replies": [ { "id": 1312285, "postDate": "2021-05-18T00:58:51.463Z", "content": "Spatial, channel wise did not work for me
", "rawMarkdown": "Spatial, channel wise did not work for me", "votes": 1 }, { "id": 1312992, "postDate": "2021-05-18T11:08:40.753Z", "content": "@snaker Can you please tell how you converted channel wise images into spatial ones ?
", "rawMarkdown": "@snaker Can you please tell how you converted channel wise images into spatial ones ?" }, { "id": 1313336, "postDate": "2021-05-18T14:12:38.173Z", "content": "Oddly, channel-wise is better for me.
", "rawMarkdown": "Oddly, channel-wise is better for me. " }, { "id": 1314141, "postDate": "2021-05-19T01:25:25.197Z", "content": "Stack them vertically, just like the EDA notebooks looks like
", "rawMarkdown": "Stack them vertically, just like the EDA notebooks looks like" }, { "id": 1318911, "postDate": "2021-05-22T17:05:42.163Z", "content": "Did you train on Kaggle? I tried it is not running on Kaggle.
", "rawMarkdown": "Did you train on Kaggle? I tried it is not running on Kaggle." }, { "id": 1319245, "postDate": "2021-05-23T03:54:37.227Z", "content": "Not Kaggle, using my own resource.
", "rawMarkdown": "Not Kaggle, using my own resource." } ] } ], "comments": [ { "id": 1318750, "author_name": "Athar Sayed", "author_url": "", "post_date": "2021-05-22T14:41:25.670000", "content": "model : Nfnet_l0
\nLocal Roc Auc : 0.9837 [4 folds ] , 0.9821[OOF]
\nPublic Roc Auc : 0.97
\nI think there is smaller number of +ve samples in Public Test and hence the Public Score may not be a Good Indicator of Generalization of Model , given that lot of people have got 0.97 till now !
It's weired that NFNet never works for me. I will give it another chance someday.
", "votes": 1, "replies": [] }, { "id": 1319258, "author_name": "Athar Sayed", "author_url": "", "post_date": "2021-05-23T04:27:28.753000", "content": "@snaker it's all about training pipeline :)
", "votes": 0, "replies": [] } ] }, { "id": 1319260, "author_name": "zhangeng", "author_url": "", "post_date": "2021-05-23T04:30:11.710000", "content": "Why is there a big gap between your CV and LB, and I also use B0 network cv97.7 LB 97 + which is very close？
", "votes": 0, "replies": [ { "id": 1319295, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-23T05:18:27.257000", "content": "Not sure, I'm using stratified K fold
", "votes": 0, "replies": [] } ] }, { "id": 1313343, "author_name": "Baran Hashemi", "author_url": "", "post_date": "2021-05-18T14:14:21.510000", "content": "May I ask why did you choose a single fold and not e.g 4 folds?
", "votes": 0, "replies": [ { "id": 1314085, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-19T00:25:41.730000", "content": "I split dataset into 5 folds, but currently only trained on 4 of them for quickly validation. May be in the last month of this game i will start to train on 5 fold and ensemble.
", "votes": 0, "replies": [] } ] }, { "id": 1312385, "author_name": "Chandan Verma", "author_url": "", "post_date": "2021-05-18T02:54:44.123000", "content": "What image size are you using?
", "votes": 0, "replies": [ { "id": 1312655, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-18T07:06:15.650000", "content": "original, no resize
", "votes": 0, "replies": [] }, { "id": 1314965, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-05-19T13:24:06.227000", "content": "'original' means (273, 256) ? or (1638, 256) ?
", "votes": 0, "replies": [] }, { "id": 1315643, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-20T03:25:08.767000", "content": "it should be 1638, 256
", "votes": 2, "replies": [] } ] }, { "id": 1311767, "author_name": "Salman", "author_url": "", "post_date": "2021-05-17T15:48:33.630000", "content": "You used sapatial image or channel wise?
", "votes": 0, "replies": [ { "id": 1312285, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-18T00:58:51.463000", "content": "Spatial, channel wise did not work for me
", "votes": 1, "replies": [] }, { "id": 1312992, "author_name": "Mithil Salunkhe", "author_url": "", "post_date": "2021-05-18T11:08:40.753000", "content": "@snaker Can you please tell how you converted channel wise images into spatial ones ?
", "votes": 0, "replies": [] }, { "id": 1313336, "author_name": "Baran Hashemi", "author_url": "", "post_date": "2021-05-18T14:12:38.173000", "content": "Oddly, channel-wise is better for me.
", "votes": 0, "replies": [] }, { "id": 1314141, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-19T01:25:25.197000", "content": "Stack them vertically, just like the EDA notebooks looks like
", "votes": 0, "replies": [] }, { "id": 1318911, "author_name": "Aman Deep Gupta", "author_url": "", "post_date": "2021-05-22T17:05:42.163000", "content": "Did you train on Kaggle? I tried it is not running on Kaggle.
", "votes": 0, "replies": [] }, { "id": 1319245, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-23T03:54:37.227000", "content": "Not Kaggle, using my own resource.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1311691": "I want to see how far can we go with just one model(and for best one fold), or it's just a ensembling game.\n\nFor me:\n\nmodel: efficientnet_b0\nfold: trained on single fold (4/5 of the dataset)\nlocal ROC AUC: 0.986\npublic ROC AUC: 0.97 ( based on \"Score Sorting\", I guess it should be around ~ 0.977 )\nTTA: no tta\n\n\nEdit:\n\nI have tried my best pipeline(above) with 5 folds to ensemble, the public score is better but does not achieve 0.98, so I think the naive ensemble(average) boost is small.\n", "1318750": "model : Nfnet_l0 \nLocal Roc Auc : 0.9837 [4 folds ] , 0.9821[OOF]\nPublic Roc Auc : 0.97\nI think there is smaller number of +ve samples in Public Test and hence the Public Score may not be a Good Indicator of Generalization of Model , given that lot of people have got 0.97 till now ! ", "1319260": "Why is there a big gap between your CV and LB, and I also use B0 network cv97.7 LB 97 + which is very close？", "1313343": "May I ask why did you choose a single fold and not e.g 4 folds?", "1312385": "What image size are you using?", "1311767": "You used sapatial image or channel wise?" } }