![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F991320%2F3fa4d3a563388f552a6302bc050c53e9%2FScreen%20Shot%202019-09-02%20at%2012.52.41%20PM.png?generation=1567443170774352&alt=media\")
This is the last chance to rethink the image size because we have little time.\nI am currently using only 256. I've already tried a larger size but it only took a long time to train and didn't give good score. What is your best image size？
", "messages": [ { "id": 610122, "postDate": "2019-08-28T13:06:03.877Z", "content": "This is the last chance to rethink the image size because we have little time.\nI am currently using only 256. I've already tried a larger size but it only took a long time to train and didn't give good score. What is your best image size？
", "rawMarkdown": "This is the last chance to rethink the image size because we have little time.\nI am currently using only 256. I've already tried a larger size but it only took a long time to train and didn't give good score. What is your best image size？", "votes": 27 }, { "id": 612806, "postDate": "2019-08-30T02:39:47.713Z", "content": "My single models are,
\n1. 300 Regression EfficientNet B3, LB 0.830\n2. 260 Regression EfficientNet B2, LB 0.820
Our current place is from Model 1.
", "rawMarkdown": "My single models are, \n1. 300 Regression EfficientNet B3, LB 0.830\n2. 260 Regression EfficientNet B2, LB 0.820\n\nOur current place is from Model 1. ", "votes": 9, "replies": [ { "id": 614241, "postDate": "2019-08-31T08:46:05.973Z", "content": "hi max well did you try any preprocessing like bens cropping or any other augs..and how much is you train loss for each at the time of making prediction
", "rawMarkdown": "hi max well did you try any preprocessing like bens cropping or any other augs..and how much is you train loss for each at the time of making prediction" }, { "id": 614447, "postDate": "2019-08-31T13:55:30.117Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 614458, "postDate": "2019-08-31T14:14:21.790Z", "content": "@pranavpulijala \nMy CV on APTOS 2019 train dataset is about 0.93.\nIf I add higher grade examples of DRD2015 for evaluation, that will go down.\nI think it would be meaningless to compare CV without using same datasets.
", "rawMarkdown": "@pranavpulijala \nMy CV on **APTOS 2019 train dataset** is about 0.93.\nIf I add higher grade examples of DRD2015 for evaluation, that will go down.\nI think it would be meaningless to compare CV without using same datasets. " }, { "id": 614514, "postDate": "2019-08-31T15:50:24.747Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 611126, "postDate": "2019-08-29T05:50:34.947Z", "content": "I use 256 preprocessed by mean subtraction.\nMy best single model (EfficientNet-B4) got 0.829.\nI've tried 320 and 512, but it was a waste of time.
", "rawMarkdown": "I use 256 preprocessed by mean subtraction.\nMy best single model (EfficientNet-B4) got 0.829.\nI've tried 320 and 512, but it was a waste of time.", "votes": 9, "replies": [ { "id": 611224, "postDate": "2019-08-29T07:10:17.477Z", "content": "Please give some tips. What data you used, any other preprocessing than mean subtraction, data augmentations and whether you treated the problem as regression or classification ? Thanks in advance
", "rawMarkdown": "Please give some tips. What data you used, any other preprocessing than mean subtraction, data augmentations and whether you treated the problem as regression or classification ? Thanks in advance" }, { "id": 611268, "postDate": "2019-08-29T07:37:13.153Z", "content": "@octpath0302 You are still use old and current data together？
", "rawMarkdown": "@octpath0302 You are still use old and current data together？" }, { "id": 611327, "postDate": "2019-08-29T08:02:59.667Z", "content": "@virajbagal \nI use only rotation and horizontal flip in training phase. no tta. regression.\nSince there are many class 2 data in public test set, training data are sampled with adjusted distribution. My LB score so far improved best when the distribution was adjusted.\n@garybios Yes. I use all the datasets listed in 'external data thread' at the same time.
", "rawMarkdown": "@virajbagal \nI use only rotation and horizontal flip in training phase. no tta. regression.\nSince there are many class 2 data in public test set, training data are sampled with adjusted distribution. My LB score so far improved best when the distribution was adjusted.\n@garybios Yes. I use all the datasets listed in 'external data thread' at the same time.", "votes": 3 }, { "id": 611434, "postDate": "2019-08-29T08:58:39.057Z", "content": "nice, thanks a lot. Could you tell me how do you adjust the data distribution?
", "rawMarkdown": "nice, thanks a lot. Could you tell me how do you adjust the data distribution?" }, { "id": 611504, "postDate": "2019-08-29T09:48:49.343Z", "content": "A trained model was used to predict the class of public test sets and estimate the distribution.\nThere are various ways of sampling. Whether to do it only first, or every epoch, or to make a K-Fold.
", "rawMarkdown": "A trained model was used to predict the class of public test sets and estimate the distribution.\nThere are various ways of sampling. Whether to do it only first, or every epoch, or to make a K-Fold.", "votes": 1 }, { "id": 611534, "postDate": "2019-08-29T10:06:30.907Z", "content": "@octpath0302 that this mean you do not crop your images? (you said you mean subtract only)
", "rawMarkdown": "@octpath0302 that this mean you do not crop your images? (you said you mean subtract only)" }, { "id": 611548, "postDate": "2019-08-29T10:13:33.163Z", "content": "@octpath0302 I thought of sampling based on the class distribution of public test dataset. But I thought that way would overfit public test dataset. I think the distribution of private test dataset is more similar to train dataset than private test dataset. What do you think about this？
", "rawMarkdown": "@octpath0302 I thought of sampling based on the class distribution of public test dataset. But I thought that way would overfit public test dataset. I think the distribution of private test dataset is more similar to train dataset than private test dataset. What do you think about this？", "votes": 1 }, { "id": 611563, "postDate": "2019-08-29T10:20:47.900Z", "content": "@hirune924 you can probe it and see. someone from the top has done it for sure.
", "rawMarkdown": "@hirune924 you can probe it and see. someone from the top has done it for sure." }, { "id": 611566, "postDate": "2019-08-29T10:21:53.580Z", "content": "@valanm Sorry for misleading! In the preprocessing stage, I use auto cropping which is similar to the 1st solution on the previous competition, and padding.\nI make preprocessing to get robustness against black area at corner of each image.
", "rawMarkdown": "@valanm Sorry for misleading! In the preprocessing stage, I use auto cropping which is similar to the 1st solution on the previous competition, and padding.\nI make preprocessing to get robustness against black area at corner of each image.", "votes": 2 }, { "id": 611568, "postDate": "2019-08-29T10:22:22.813Z", "content": "@hirune924 I also thought that way is easy to overfit public lb, but we do not know the distribution of private test dataset.
", "rawMarkdown": "@hirune924 I also thought that way is easy to overfit public lb, but we do not know the distribution of private test dataset." }, { "id": 611577, "postDate": "2019-08-29T10:27:00.067Z", "content": "@hirune924 I also think there is a danger of overfit. However, it is generally possible that there is a lot of data in class 0 (normal) and class 2 (middle). Therefore, I assume that the public test set and the private test set are randomly divided, so far.
", "rawMarkdown": "@hirune924 I also think there is a danger of overfit. However, it is generally possible that there is a lot of data in class 0 (normal) and class 2 (middle). Therefore, I assume that the public test set and the private test set are randomly divided, so far.", "votes": 2 }, { "id": 611765, "postDate": "2019-08-29T12:24:41.440Z", "content": "@garybios @valanm @virajbagal How about you? Which size/preprocessing do you use?
", "rawMarkdown": "@garybios @valanm @virajbagal How about you? Which size/preprocessing do you use?" }, { "id": 611792, "postDate": "2019-08-29T12:46:45.253Z", "content": "300px b3 classification gives me .822, for preprocessing just cropping the object.
", "rawMarkdown": "300px b3 classification gives me .822, for preprocessing just cropping the object.", "votes": 2 }, { "id": 611811, "postDate": "2019-08-29T12:59:18.987Z", "content": "@octpath0302 224, no preprocessing.\n@valanm nice job, you made a great classification model.
", "rawMarkdown": "@octpath0302 224, no preprocessing.\n@valanm nice job, you made a great classification model.", "votes": 1 }, { "id": 611880, "postDate": "2019-08-29T13:37:50.370Z", "content": "@octpath0302 256, cropping image and training only on new data gave me 0.796. Despite adding augmentations, my model trained on OLD Data with New Data as validation is overfitting and always predicting 0. I tried to balance the classes by downsampling and upsampling with each class around 5000 images but the score is really poor. I am desperately trying again and again to train the model on old data.
", "rawMarkdown": "@octpath0302 256, cropping image and training only on new data gave me 0.796. Despite adding augmentations, my model trained on OLD Data with New Data as validation is overfitting and always predicting 0. I tried to balance the classes by downsampling and upsampling with each class around 5000 images but the score is really poor. I am desperately trying again and again to train the model on old data. ", "votes": 2 }, { "id": 611929, "postDate": "2019-08-29T14:05:24.153Z", "content": "@virajbagal Hmm. . that's strange. But in my case too, augmentation is hardly working. And I use only downsampling (I have not tried upsampling).\nAre CV and LB (or losses) somewhat correlated? Do your CV get >0.85?
", "rawMarkdown": "@virajbagal Hmm. . that's strange. But in my case too, augmentation is hardly working. And I use only downsampling (I have not tried upsampling).\nAre CV and LB (or losses) somewhat correlated? Do your CV get >0.85?", "votes": 1 }, { "id": 611945, "postDate": "2019-08-29T14:11:41.657Z", "content": "My CV goes upto 92. But predicts all zeros.
", "rawMarkdown": "My CV goes upto 92. But predicts all zeros. " }, { "id": 611950, "postDate": "2019-08-29T14:18:49.520Z", "content": "My CV has never reached to 0.9. So, I think that overfit or something is happening in your case..\n(too small validation set or .. I don't know.)\nIn my case, CV is around 0.87 to 0.88 and LB is around 0.81 to 0.83.
", "rawMarkdown": "My CV has never reached to 0.9. So, I think that overfit or something is happening in your case..\n(too small validation set or .. I don't know.)\nIn my case, CV is around 0.87 to 0.88 and LB is around 0.81 to 0.83.", "votes": 1 }, { "id": 612069, "postDate": "2019-08-29T15:30:18.047Z", "content": "Me too..
", "rawMarkdown": "Me too.. " }, { "id": 612070, "postDate": "2019-08-29T15:31:28.803Z", "content": "@octpath0302 , thanks for sharing the details. A quick question on the external data usage. External datasets have different labels. Some of them have DR levels (0 to 4 or 0 to 3), some have segmented masks, some have counts of microaneurisms, exudates, etc. How are you managing with such desparate labels. Are you somehow creating uniform labels for all the external datasets ?
", "rawMarkdown": "@octpath0302 , thanks for sharing the details. A quick question on the external data usage. External datasets have different labels. Some of them have DR levels (0 to 4 or 0 to 3), some have segmented masks, some have counts of microaneurisms, exudates, etc. How are you managing with such desparate labels. Are you somehow creating uniform labels for all the external datasets ?" }, { "id": 612090, "postDate": "2019-08-29T15:46:40.437Z", "content": "@ravivadapalli I use datasets which DR levels are 0 to 4 as labeled dataset, and the others are as unlabeled (pseudo-labeled or something) dataset. \nI don't try to use any other labels and masks because there is little time to Kaggle..
", "rawMarkdown": "@ravivadapalli I use datasets which DR levels are 0 to 4 as labeled dataset, and the others are as unlabeled (pseudo-labeled or something) dataset. \nI don't try to use any other labels and masks because there is little time to Kaggle..", "votes": 3 }, { "id": 612215, "postDate": "2019-08-29T16:58:51.740Z", "content": "I see. thank you @octpath0302 for sharing the details.
", "rawMarkdown": "I see. thank you @octpath0302 for sharing the details. " }, { "id": 612309, "postDate": "2019-08-29T18:02:23.947Z", "content": "Hi, @octpath0302. Thank you for sharing so much of your approach. Could you explain more how you apply mean subtraction to your images?
", "rawMarkdown": "Hi, @octpath0302. Thank you for sharing so much of your approach. Could you explain more how you apply mean subtraction to your images? " }, { "id": 612602, "postDate": "2019-08-29T22:19:20.673Z", "content": "I refer to the 1st place solution of the previous competition.\nhttps://www.kaggle.com/c/diabetic-retinopathy-detection/discussion/15801#latest-370950
", "rawMarkdown": "I refer to the 1st place solution of the previous competition.\nhttps://www.kaggle.com/c/diabetic-retinopathy-detection/discussion/15801#latest-370950", "votes": 1 }, { "id": 612609, "postDate": "2019-08-29T22:40:32.370Z", "content": "Got it, thanks!
", "rawMarkdown": "Got it, thanks! " }, { "id": 613421, "postDate": "2019-08-30T13:47:13.103Z", "content": "May I ask what is your train setting?
", "rawMarkdown": "May I ask what is your train setting?" }, { "id": 613587, "postDate": "2019-08-30T16:31:48.790Z", "content": "Hi @octpath0302 , did you observe big drop in CV after applying downsampling but LB may be the same or even better?
", "rawMarkdown": "Hi @octpath0302 , did you observe big drop in CV after applying downsampling but LB may be the same or even better? " }, { "id": 613917, "postDate": "2019-08-31T00:15:57.010Z", "content": "@andalpha I have provided a lot of information, so I can't do any more..\n@naivelamb I'm not sure because I've never trained by applying downsampling only. After applying downsampling and data addition, I observed that CV was almost unchanged and LB was getting better.
", "rawMarkdown": "@andalpha I have provided a lot of information, so I can't do any more..\n@naivelamb I'm not sure because I've never trained by applying downsampling only. After applying downsampling and data addition, I observed that CV was almost unchanged and LB was getting better." }, { "id": 613932, "postDate": "2019-08-31T00:37:24.717Z", "content": "By data addition you mean mixing external data (2015 for example) with current data together to train the model, right?
", "rawMarkdown": "By data addition you mean mixing external data (2015 for example) with current data together to train the model, right?" }, { "id": 613943, "postDate": "2019-08-31T00:43:31.403Z", "content": "yes.
", "rawMarkdown": "yes." }, { "id": 614227, "postDate": "2019-08-31T08:28:11.460Z", "content": "hi oct when you training distribution sampling wrt to public test set.. what you mean\nis it taking more class 2 data or how is it ...
", "rawMarkdown": "hi oct when you training distribution sampling wrt to public test set.. what you mean\nis it taking more class 2 data or how is it ...\n" }, { "id": 614234, "postDate": "2019-08-31T08:40:13.860Z", "content": "@valann by b3 classification you mean not using regression?
", "rawMarkdown": "@valann by b3 classification you mean not using regression?" }, { "id": 614247, "postDate": "2019-08-31T08:52:22.140Z", "content": "yes I only use classification
", "rawMarkdown": "yes I only use classification" }, { "id": 614256, "postDate": "2019-08-31T09:23:07.180Z", "content": "@octpath which external data set you using.. \nThanks in advance
", "rawMarkdown": "@octpath which external data set you using.. \nThanks in advance" }, { "id": 614660, "postDate": "2019-08-31T21:13:24.103Z", "rawMarkdown": "", "isDeleted": true }, { "id": 614702, "postDate": "2019-08-31T23:52:08.133Z", "content": "do you use optimized thresholds?
", "rawMarkdown": "do you use optimized thresholds?" }, { "id": 614756, "postDate": "2019-09-01T02:53:29.483Z", "content": "@valan how did you work with wierd crop patches produced by fastai ...I found some blacks borders with zero padding mode,or circle arcs at border with reflection ...
", "rawMarkdown": "@valan how did you work with wierd crop patches produced by fastai ...I found some blacks borders with zero padding mode,or circle arcs at border with reflection ..." }, { "id": 615136, "postDate": "2019-09-01T14:01:40.210Z", "content": "@ggoldszmidt I mean local mean subtraction (high-pass filtering), so not standardized. Almost the same as 1st place solution of the previous competition.\n@tanlikesmath Yes. Optimized.\n@jaideepvalani \n> hi oct when you training distribution sampling wrt to public test set.. what you mean\n> is it taking more class 2 data or how is it …
\n\nSorry, I can't catch your meaning. You can calculate the public test class distribution with a model (e. g. from public kernel). \nExternal data I'm using are listed up on the official thread.
", "rawMarkdown": "@ggoldszmidt I mean local mean subtraction (high-pass filtering), so not standardized. Almost the same as 1st place solution of the previous competition.\n@tanlikesmath Yes. Optimized.\n@jaideepvalani \n> hi oct when you training distribution sampling wrt to public test set.. what you mean\n> is it taking more class 2 data or how is it …\n\nSorry, I can't catch your meaning. You can calculate the public test class distribution with a model (e. g. from public kernel). \nExternal data I'm using are listed up on the official thread." }, { "id": 615160, "postDate": "2019-09-01T14:50:28.137Z", "content": "hi oct thanks for reply..\n1) in training set we have 3000 eg . how we decide on the ratio of distribution while sampling say we have batch size of 32..\n2) I tried to follow the external thread none of them have labels ,if i missed any link could u past the one that you used in ..\n3) which public kernel are talking about
", "rawMarkdown": "hi oct thanks for reply..\n1) in training set we have 3000 eg . how we decide on the ratio of distribution while sampling say we have batch size of 32..\n2) I tried to follow the external thread none of them have labels ,if i missed any link could u past the one that you used in ..\n3) which public kernel are talking about\n" }, { "id": 615176, "postDate": "2019-09-01T15:05:11.730Z", "content": "@jaideepvalani \n1) I mean not sampling mini-batch, but how to re-construct labeled dataset.\n2) 3) Most of the external datasets are unlabeled datasets. But you can pseudo-label any images by using high-performance model from public kernels. If you already have good models, you can use them.
", "rawMarkdown": "@jaideepvalani \n1) I mean not sampling mini-batch, but how to re-construct labeled dataset.\n2) 3) Most of the external datasets are unlabeled datasets. But you can pseudo-label any images by using high-performance model from public kernels. If you already have good models, you can use them. " }, { "id": 615200, "postDate": "2019-09-01T15:46:54.470Z", "content": "@jaideepvalani i read the documentation and adopted the code. I default would be good enough then data science wouldn't be hot on job market.
", "rawMarkdown": "@jaideepvalani i read the documentation and adopted the code. I default would be good enough then data science wouldn't be hot on job market. " } ] }, { "id": 615912, "postDate": "2019-09-02T13:47:37.107Z", "content": "Here are some of my results. I did on image size. All 5x CV
\n\nEfficientNet B5, 224 LB: 0.807\nEfficientNet B5, 300 LB: 0.812\nEfficientNet B5, 380 LB: 0.816\nEfficientNet B5, 456 LB: 0.823
EfficientNet B4, 224 LB: 0.800\nEfficientNet B4, 300 LB: 0.811\nEfficientNet B4, 380 LB: 0.816
@dir HB good results,your kernel public appreaciated by many kagglers and including me :)\nwould you like to tell the class distribution..
", "rawMarkdown": "@dir HB good results,your kernel public appreaciated by many kagglers and including me :)\nwould you like to tell the class distribution.. ", "votes": 2 }, { "id": 615973, "postDate": "2019-09-02T14:55:46.567Z", "content": "Glad to hear that kernel helped =) My final distributions are very similar to some that found in this thread . =)
", "rawMarkdown": "Glad to hear that kernel helped =) My final distributions are very similar to some that found in this [thread](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/106844#614198) . =) " }, { "id": 615975, "postDate": "2019-09-02T14:59:08.097Z", "content": "I tried doing 5x CV with EfficientNet B5 but I only get .79-.792. I dont know what else to do.
", "rawMarkdown": "I tried doing 5x CV with EfficientNet B5 but I only get .79-.792. I dont know what else to do.", "votes": 1 }, { "id": 615985, "postDate": "2019-09-02T15:07:34.797Z", "content": "They are diffrent ways how you can improve score. Have a look some threads like [this] (https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/105100#latest-615864) . All of them have good tips.
\n\nBut in genral. \n1) Try diffrent preprocessing \n2) Try diffrent augmentations \n3) Try combining of diffrent dataset.
\n\nForum is full of tips =) \nGood luck to you =)
", "rawMarkdown": "They are diffrent ways how you can improve score. Have a look some threads like [this] (https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/105100#latest-615864) . All of them have good tips. \n\nBut in genral. \n1) Try diffrent preprocessing \n2) Try diffrent augmentations \n3) Try combining of diffrent dataset. \n\nForum is full of tips =) \nGood luck to you =) \n\n", "votes": 2 }, { "id": 615987, "postDate": "2019-09-02T15:08:00.310Z", "content": "@drhabib when you increase img size, do you also increase your parameters, like learning rate or epochs? I'm asking this because for me it's harder to keep models with large img size learning.
", "rawMarkdown": "@drhabib when you increase img size, do you also increase your parameters, like learning rate or epochs? I'm asking this because for me it's harder to keep models with large img size learning.", "votes": 1 }, { "id": 615996, "postDate": "2019-09-02T15:17:51.187Z", "content": "Good question.
\n\nI ususally run lr_finder and select learning rate based on this, but in most casses its very similar across diffrent sizes. Also regarding number of Epochs if you are intrested, for 224 I run for 20 epochs and for rest image sizes 5 epochs is enogh. Again it depends also on which learning rate policy you are using in my case its one_cycle_policy.
\n\nHope it helps =)
", "rawMarkdown": "Good question. \n\nI ususally run lr_finder and select learning rate based on this, but in most casses its very similar across diffrent sizes. Also regarding number of Epochs if you are intrested, for 224 I run for 20 epochs and for rest image sizes 5 epochs is enogh. Again it depends also on which learning rate policy you are using in my case its one_cycle_policy.\n\nHope it helps =)\n", "votes": 5 }, { "id": 616011, "postDate": "2019-09-02T15:44:01.563Z", "content": "Thanks @drhabib , I have tried many policies, also I use Keras, but the bigger models don't train longer than 10 epochs reliably.
", "rawMarkdown": "Thanks @drhabib , I have tried many policies, also I use Keras, but the bigger models don't train longer than 10 epochs reliably.", "votes": 1 }, { "id": 616016, "postDate": "2019-09-02T15:47:02.880Z", "content": "which optimizer are you using ?
", "rawMarkdown": "which optimizer are you using ?" }, { "id": 616022, "postDate": "2019-09-02T15:58:23.130Z", "content": "For me what seems to work is either Adam or RAdam, but also with LR policies, like Cosine or cycle.
\n\nSo far my best single model result comes from B5 224, using Adam and cosine LR (almost 0.8)
", "rawMarkdown": "For me what seems to work is either Adam or RAdam, but also with LR policies, like Cosine or cycle.\n\nSo far my best single model result comes from B5 224, using Adam and cosine LR (almost 0.8)", "votes": 1 }, { "id": 616051, "postDate": "2019-09-02T16:30:58.663Z", "content": "Do you use weights from lower resolution or u always start from imagenet?
", "rawMarkdown": "Do you use weights from lower resolution or u always start from imagenet?", "votes": 1 }, { "id": 616062, "postDate": "2019-09-02T16:41:17.920Z", "content": "@valanm , I'm starting from imagenet. By lower resolution, do you mean pretraining from the older dataset?
", "rawMarkdown": "@valanm , I'm starting from imagenet. By lower resolution, do you mean pretraining from the older dataset?", "votes": 1 }, { "id": 616068, "postDate": "2019-09-02T16:49:07.713Z", "content": "For instance, train 20 epochs on 224 and then 5 more with increased resolution to 456
", "rawMarkdown": "For instance, train 20 epochs on 224 and then 5 more with increased resolution to 456", "votes": 4 }, { "id": 616072, "postDate": "2019-09-02T16:54:05.180Z", "content": "I think the issue might be that when you go to higher size you have to use small batch size. \n1) Have you tried to do gradient accumulation ?\n2) Try Mish activation function. Basicity its very simple to implement here is the paper. (https://arxiv.org/pdf/1908.08681v1.pdf) . You can see in on figure 7 that decreasing batch size has little effect in accuracy in comparison to ReLu. In my case it helped a bit.
\n\n
@valanm I reuse weight for lower resolution to train higher \nHope this helps
", "rawMarkdown": "I think the issue might be that when you go to higher size you have to use small batch size. \n1) Have you tried to do gradient accumulation ?\n2) Try Mish activation function. Basicity its very simple to implement here is the [paper](https://arxiv.org/pdf/1908.08681v1.pdf). (https://arxiv.org/pdf/1908.08681v1.pdf) . You can see in on figure 7 that decreasing batch size has little effect in accuracy in comparison to ReLu. In my case it helped a bit. \n\n\n\n\n@valanm I reuse weight for lower resolution to train higher \nHope this helps ", "votes": 8 }, { "id": 616094, "postDate": "2019-09-02T17:17:21.433Z", "content": "Thank @valanm and @drhabib , both suggestions are very interesting, I'll give a try.
", "rawMarkdown": "Thank @valanm and @drhabib , both suggestions are very interesting, I'll give a try.", "votes": 1 }, { "id": 617975, "postDate": "2019-09-04T17:18:55.830Z", "content": "@drhabib i was just wondering , if increasing the image size should take more training as we have more to learn. Can you explain something about this?
", "rawMarkdown": "@drhabib i was just wondering , if increasing the image size should take more training as we have more to learn. Can you explain something about this?", "votes": 1 }, { "id": 618051, "postDate": "2019-09-04T19:04:55.393Z", "content": "Hi.. I will refer you to this two blog posts. \nhttps://miguel-data-sc.github.io/2017-11-23-second/\nhttps://towardsdatascience.com/boost-your-cnn-image-classifier-performance-with-progressive-resizing-in-keras-a7d96da06e20
", "rawMarkdown": "Hi.. I will refer you to this two blog posts. \nhttps://miguel-data-sc.github.io/2017-11-23-second/\nhttps://towardsdatascience.com/boost-your-cnn-image-classifier-performance-with-progressive-resizing-in-keras-a7d96da06e20", "votes": 1 }, { "id": 619846, "postDate": "2019-09-06T16:50:48.257Z", "content": "@drhb i was wondering were u able to run wit b5 456 sz image on kaggles kernel,how much was bs..
", "rawMarkdown": "@drhb i was wondering were u able to run wit b5 456 sz image on kaggles kernel,how much was bs..\n", "votes": 1 }, { "id": 619848, "postDate": "2019-09-06T16:52:08.863Z", "content": "for interference only =) \nbs = 64 did a job, but i think going higher is possible but i havent tried =)
", "rawMarkdown": "for interference only =) \nbs = 64 did a job, but i think going higher is possible but i havent tried =) " }, { "id": 619853, "postDate": "2019-09-06T16:54:57.507Z", "content": "ok so training on smaller image size and but inference on a higher one could work well ? this is new for me i was thinking that we should always give same size image for inference as training..\nPlease confirm.. i will add this in my learning book :)
", "rawMarkdown": "ok so training on smaller image size and but inference on a higher one could work well ? this is new for me i was thinking that we should always give same size image for inference as training..\nPlease confirm.. i will add this in my learning book :)", "votes": 1 }, { "id": 619871, "postDate": "2019-09-06T17:19:27.677Z", "content": "When you are doing interference you only do forward pass and you dont have to compute gradients as a result more memory is available for gpu and therefore you can use big batch size =)
", "rawMarkdown": "When you are doing interference you only do forward pass and you dont have to compute gradients as a result more memory is available for gpu and therefore you can use big batch size =) " }, { "id": 620725, "postDate": "2019-09-07T22:51:17.453Z", "content": "in my last submission i get some thing similar to what is posted without doing any mechanical adjustments much.. \nbut dont know will i get also similar score as tilll now for all submissions only the ones having 2 above 1200 have seen score above 0.80.. ,1 and 3 between 130 and 160, the posted ones are very different to these..\nWas that bar graph post a competition gimmick hahah :) ...
", "rawMarkdown": "in my last submission i get some thing similar to what is posted without doing any mechanical adjustments much.. \nbut dont know will i get also similar score as tilll now for all submissions only the ones having 2 above 1200 have seen score above 0.80.. ,1 and 3 between 130 and 160, the posted ones are very different to these..\nWas that bar graph post a competition gimmick hahah :) ..." }, { "id": 621395, "postDate": "2019-09-08T13:33:19.477Z", "content": "hi hb good to see you gained the rank in private lb..\ncongrats\ncould you tell me how you replaced the default act functions for the model..
", "rawMarkdown": "hi hb good to see you gained the rank in private lb..\ncongrats\ncould you tell me how you replaced the default act functions for the model.. \n" } ] }, { "id": 610215, "postDate": "2019-08-28T14:49:24.983Z", "content": "456, efficientnetb5 lb:0,824 single model
", "rawMarkdown": "456, efficientnetb5 lb:0,824 single model", "votes": 6, "replies": [ { "id": 610243, "postDate": "2019-08-28T15:20:50.643Z", "content": "a nice lb score.
", "rawMarkdown": "a nice lb score." }, { "id": 610699, "postDate": "2019-08-29T00:19:29.260Z", "content": "nice work! my best lb score of 256 single model is 0.821.
", "rawMarkdown": "nice work! my best lb score of 256 single model is 0.821." }, { "id": 610786, "postDate": "2019-08-29T01:15:25.363Z", "content": "whats kind of network?
", "rawMarkdown": "whats kind of network?" }, { "id": 610896, "postDate": "2019-08-29T02:44:11.283Z", "content": "efficientnetb5
", "rawMarkdown": "efficientnetb5" }, { "id": 611029, "postDate": "2019-08-29T04:21:47.390Z", "content": "@hirune924 @leixiang Nice scores !!\nWhat kind of preprocessing you guys used?\nDid you first pretrain on old train and finetune on new
", "rawMarkdown": "@hirune924 @leixiang Nice scores !!\nWhat kind of preprocessing you guys used?\nDid you first pretrain on old train and finetune on new " }, { "id": 611757, "postDate": "2019-08-29T12:19:18.590Z", "content": "@leixiang @hirune924 nice to see single model reach the score. May I ask for If you have any change of the output header of the efficientnet like adding BN or concat [avgpool, maxpool]? Thank you all.
", "rawMarkdown": "@leixiang @hirune924 nice to see single model reach the score. May I ask for If you have any change of the output header of the efficientnet like adding BN or concat [avgpool, maxpool]? Thank you all." }, { "id": 612317, "postDate": "2019-08-29T18:06:14.153Z", "content": "Hi,
\n\nYou are asking others, but I would like to offer my experience: Using MaxPool with EfficientNet seemed to hurt the model's performance. I tried several times with different experiments and no good. I thought ConcatPool should improve performance, but it seems that the information from the MaxPool only brings the performance of the AvgPool down. Of course, this is only data from my limited experiments.
", "rawMarkdown": "Hi, \n\nYou are asking others, but I would like to offer my experience: Using MaxPool with EfficientNet seemed to hurt the model's performance. I tried several times with different experiments and no good. I thought ConcatPool should improve performance, but it seems that the information from the MaxPool only brings the performance of the AvgPool down. Of course, this is only data from my limited experiments. " }, { "id": 612656, "postDate": "2019-08-29T23:28:22.537Z", "content": "This idea is borrowed from the bi-lstm model. It seems no one who has high rank in the discussion has told this feature recently... you are right and I make a mistake... I just search my log and find that using concat just decrease my CV from 0.8091 to 0.7993 because I have too many trials and i get confused... Maybe I need to figure out the difference between them.
", "rawMarkdown": "This idea is borrowed from the bi-lstm model. It seems no one who has high rank in the discussion has told this feature recently... you are right and I make a mistake... I just search my log and find that using concat just decrease my CV from 0.8091 to 0.7993 because I have too many trials and i get confused... Maybe I need to figure out the difference between them.", "votes": 1 } ] }, { "id": 612000, "postDate": "2019-08-29T14:52:56.757Z", "content": "328 B5 Classification, reach 0.825.
\n328 B5 Regression, reach 0.823.\n256 B4 Regression, reach 0.821
[Uptate 2019/08/30 13:18]\n256 B4 Classification, reach 0.820
", "rawMarkdown": "328 B5 Classification, reach 0.825. \n328 B5 Regression, reach 0.823.\n256 B4 Regression, reach 0.821\n\n[Uptate 2019/08/30 13:18]\n256 B4 Classification, reach 0.820", "votes": 1, "replies": [ { "id": 612012, "postDate": "2019-08-29T15:04:45.767Z", "content": "and ensemble gives you your current score 0.833 !?
", "rawMarkdown": "and ensemble gives you your current score 0.833 !?" }, { "id": 612058, "postDate": "2019-08-29T15:27:09.777Z", "content": "Yes.... Only ensemble B5 regression and B4 regression
", "rawMarkdown": "Yes.... Only ensemble B5 regression and B4 regression", "votes": 1 }, { "id": 612124, "postDate": "2019-08-29T15:58:43.767Z", "content": "0.823 & 0.812 -> 0.833 ? This is a great synergy effect!!\nDo you think that the difference in size has an effect?
", "rawMarkdown": "0.823 & 0.812 -> 0.833 ? This is a great synergy effect!!\nDo you think that the difference in size has an effect?" }, { "id": 612148, "postDate": "2019-08-29T16:11:55.400Z", "content": "i got similar result with him just ensemble a lb 0.824 eftnet b5 456 and lb 0.814 eftnet b4 380，maybe different image size boost the score
", "rawMarkdown": "i got similar result with him just ensemble a lb 0.824 eftnet b5 456 and lb 0.814 eftnet b4 380，maybe different image size boost the score", "votes": 1 }, { "id": 612182, "postDate": "2019-08-29T16:34:44.243Z", "content": "I tried voting between 4 classification models \nb3 300 0.808\nb3 416 0.812\nb5 384 0.812\nb5 456 0.817\nbut got 0.785, couldn't find any bug so I concluded its better to use averaging, but i haven't tried yet.
", "rawMarkdown": "I tried voting between 4 classification models \nb3 300 0.808\nb3 416 0.812\nb5 384 0.812\nb5 456 0.817\nbut got 0.785, couldn't find any bug so I concluded its better to use averaging, but i haven't tried yet. ", "votes": 1 }, { "id": 612660, "postDate": "2019-08-29T23:33:59.357Z", "content": "@octpath0302 I am not sure maybe not.\nIn my experience, combining different structure model output together may boost score.\n I am still trying to improve my single model score right now. (struggling)
", "rawMarkdown": "@octpath0302 I am not sure maybe not.\nIn my experience, combining different structure model output together may boost score.\n I am still trying to improve my single model score right now. (struggling)", "votes": 2 }, { "id": 612667, "postDate": "2019-08-29T23:44:59.013Z", "content": "@valanm, I think you use majority voting, it did not work for my.
\n\nI use this way to ensemble two classification models. \nit slightly improve my score(b4:0.815, b5:0.825 -> 0.829),
\n\nfor one model prediction\nfor _ in range(TTA):\nwith torch.no_grad():\n for i, data in tqdm(enumerate(test_loader)):\n images = data\n images = images.cuda()\n pred = model(images)\n pred = torch.softmax(pred, dim=-1)\n test_pred[i * 64:(i + 1) * 64] += pred
add two model prediction together\noutput = np.argmax(((test_pred1 / TTA) + (test_pred2 / TTA)) / 2, axis=1)
@chanhu This also happened to our case. Ensembling (averaging) B5 (0. 819) and B0 (0.812) made our current leaderboard score (0.837). Although, the training procedure for each model was completely different, which might have caused such synergy!
", "rawMarkdown": "@chanhu This also happened to our case. Ensembling (averaging) B5 (0. 819) and B0 (0.812) made our current leaderboard score (0.837). Although, the training procedure for each model was completely different, which might have caused such synergy!", "votes": 3 }, { "id": 613251, "postDate": "2019-08-30T11:01:34.387Z", "content": "@udaykamal that's just awesome synergy
", "rawMarkdown": "@udaykamal that's just awesome synergy", "votes": 1 }, { "id": 614383, "postDate": "2019-08-31T12:21:09.477Z", "content": "@chanhu @udaykamal how many tta transform do you use? flip,transpose and multi-scale?
", "rawMarkdown": "@chanhu @udaykamal how many tta transform do you use? flip,transpose and multi-scale?" }, { "id": 614397, "postDate": "2019-08-31T12:41:10.700Z", "content": "tta 5 or 10. only flip, and rotation.
", "rawMarkdown": "tta 5 or 10. only flip, and rotation." }, { "id": 614420, "postDate": "2019-08-31T13:10:01.410Z", "content": "@chanhu have you checked how much boost this gives you. i tried horizontal flip but the score got worse 0.001
", "rawMarkdown": "@chanhu have you checked how much boost this gives you. i tried horizontal flip but the score got worse 0.001" }, { "id": 614483, "postDate": "2019-08-31T14:49:45.930Z", "content": "@hustkevin1037 we have not tried with TTA yet. Out current leaderboard score is simple average of two predictions from two different model. No TTA, no K-fold.
", "rawMarkdown": "@hustkevin1037 we have not tried with TTA yet. Out current leaderboard score is simple average of two predictions from two different model. No TTA, no K-fold." }, { "id": 614515, "postDate": "2019-08-31T15:54:09.760Z", "content": "Hey uday did you try regression only for both ?
", "rawMarkdown": "Hey uday did you try regression only for both ?" }, { "id": 614745, "postDate": "2019-09-01T01:55:32.897Z", "content": "@valanm do you use torch or keras?
", "rawMarkdown": "@valanm do you use torch or keras? " }, { "id": 614748, "postDate": "2019-09-01T02:07:41.613Z", "content": "@udaykamal did you use optimized threshold or just [0.5,1,5,2,5,3,5] ?
", "rawMarkdown": "@udaykamal did you use optimized threshold or just [0.5,1,5,2,5,3,5] ?" }, { "id": 614903, "postDate": "2019-09-01T08:03:41.890Z", "content": "@valanm about +0.004.
", "rawMarkdown": "@valanm about +0.004." }, { "id": 615011, "postDate": "2019-09-01T11:03:34.163Z", "content": "@jaideepvalani yes\n@hustkevin1037 optimized threshold resulted in lower score in our case. We are using just [0.5,1,5,2,5,3,5]
", "rawMarkdown": "@jaideepvalani yes\n@hustkevin1037 optimized threshold resulted in lower score in our case. We are using just [0.5,1,5,2,5,3,5]" }, { "id": 615207, "postDate": "2019-09-01T15:56:06.917Z", "content": "@custodiogabriel i am trying fastai for this competition\nthis might be a dumb question but are you guys ensembling logits or softmax outputs, does this matter at all?
", "rawMarkdown": "@custodiogabriel i am trying fastai for this competition\nthis might be a dumb question but are you guys ensembling logits or softmax outputs, does this matter at all?" }, { "id": 615508, "postDate": "2019-09-02T03:15:49.100Z", "content": "@valanm If you are doing classification, softmax outputs makes more sense.
", "rawMarkdown": "@valanm If you are doing classification, softmax outputs makes more sense. ", "votes": 1 }, { "id": 615547, "postDate": "2019-09-02T04:54:49.937Z", "content": "@ uday thanks...\ncould you please tell what is distribution of each class in submission score..
", "rawMarkdown": "@ uday thanks...\ncould you please tell what is distribution of each class in submission score.." }, { "id": 615574, "postDate": "2019-09-02T05:44:04.953Z", "content": "@naivelamb thanks. Is the reason scaling? Would ligits make sense for stacking?
", "rawMarkdown": "@naivelamb thanks. Is the reason scaling? Would ligits make sense for stacking?" }, { "id": 615596, "postDate": "2019-09-02T06:23:07.523Z", "content": "It depends on what stacking methods you are using. Some are obviously sensitive to scaling.
", "rawMarkdown": "It depends on what stacking methods you are using. Some are obviously sensitive to scaling. " }, { "id": 615733, "postDate": "2019-09-02T09:58:29.877Z", "content": "may I know what batch size u are using? I am training with 16 only due to limited gpu ram. Considering to rent a machine if batch size matters
", "rawMarkdown": "may I know what batch size u are using? I am training with 16 only due to limited gpu ram. Considering to rent a machine if batch size matters" }, { "id": 616046, "postDate": "2019-09-02T16:25:06.247Z", "content": "Averaging softmax outputs instead of logits gave the same score. However, 12 images (0.6%) got different label.
", "rawMarkdown": "Averaging softmax outputs instead of logits gave the same score. However, 12 images (0.6%) got different label." }, { "id": 620049, "postDate": "2019-09-07T00:08:12.497Z", "content": "@valanm I hope you can get good score in private test set
", "rawMarkdown": "@valanm I hope you can get good score in private test set" } ] }, { "id": 616493, "postDate": "2019-09-03T07:21:07.007Z", "content": "EfficientNet b5, no CV or LB improvement after 256
", "rawMarkdown": "EfficientNet b5, no CV or LB improvement after 256" }, { "id": 612028, "postDate": "2019-08-29T15:11:56.193Z", "content": "Is there a reason to use img size 256x256 with efficient nets? I ask this because it's not the default size for any of its models, but people seem to get good results anyway.
", "rawMarkdown": "Is there a reason to use img size 256x256 with efficient nets? I ask this because it's not the default size for any of its models, but people seem to get good results anyway.", "replies": [ { "id": 612076, "postDate": "2019-08-29T15:37:22.917Z", "content": "In my understanding, the default sizes have been searched on a specific dataset. And the dataset are different from this competition dataset. So, I think it is not always necessary to resize this dataset according to the default size.\nMany people have reported that the accuracy is good even if it is different from the default size, so I think it supports this idea.
", "rawMarkdown": "In my understanding, the default sizes have been searched on a specific dataset. And the dataset are different from this competition dataset. So, I think it is not always necessary to resize this dataset according to the default size.\nMany people have reported that the accuracy is good even if it is different from the default size, so I think it supports this idea.", "votes": 3 }, { "id": 612098, "postDate": "2019-08-29T15:50:34.097Z", "content": "I understand @octpath0302 , this makes sense.
", "rawMarkdown": "I understand @octpath0302 , this makes sense." } ] } ], "comments": [ { "id": 612806, "author_name": "Maxwell", "author_url": "", "post_date": "2019-08-30T02:39:47.713000", "content": "My single models are,
\n1. 300 Regression EfficientNet B3, LB 0.830\n2. 260 Regression EfficientNet B2, LB 0.820
Our current place is from Model 1.
", "votes": 9, "replies": [ { "id": 614241, "author_name": "Jaideep", "author_url": "", "post_date": "2019-08-31T08:46:05.973000", "content": "hi max well did you try any preprocessing like bens cropping or any other augs..and how much is you train loss for each at the time of making prediction
", "votes": 0, "replies": [] }, { "id": 614447, "author_name": "", "author_url": "", "post_date": "2019-08-31T13:55:30.117000", "content": "", "votes": 1, "replies": [] }, { "id": 614458, "author_name": "Maxwell", "author_url": "", "post_date": "2019-08-31T14:14:21.790000", "content": "@pranavpulijala \nMy CV on APTOS 2019 train dataset is about 0.93.\nIf I add higher grade examples of DRD2015 for evaluation, that will go down.\nI think it would be meaningless to compare CV without using same datasets.
", "votes": 0, "replies": [] }, { "id": 614514, "author_name": "", "author_url": "", "post_date": "2019-08-31T15:50:24.747000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 611126, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T05:50:34.947000", "content": "I use 256 preprocessed by mean subtraction.\nMy best single model (EfficientNet-B4) got 0.829.\nI've tried 320 and 512, but it was a waste of time.
", "votes": 9, "replies": [ { "id": 611224, "author_name": "Viraj Bagal", "author_url": "", "post_date": "2019-08-29T07:10:17.477000", "content": "Please give some tips. What data you used, any other preprocessing than mean subtraction, data augmentations and whether you treated the problem as regression or classification ? Thanks in advance
", "votes": 0, "replies": [] }, { "id": 611268, "author_name": "Gary", "author_url": "", "post_date": "2019-08-29T07:37:13.153000", "content": "@octpath0302 You are still use old and current data together？
", "votes": 0, "replies": [] }, { "id": 611327, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T08:02:59.667000", "content": "@virajbagal \nI use only rotation and horizontal flip in training phase. no tta. regression.\nSince there are many class 2 data in public test set, training data are sampled with adjusted distribution. My LB score so far improved best when the distribution was adjusted.\n@garybios Yes. I use all the datasets listed in 'external data thread' at the same time.
", "votes": 3, "replies": [] }, { "id": 611434, "author_name": "Gary", "author_url": "", "post_date": "2019-08-29T08:58:39.057000", "content": "nice, thanks a lot. Could you tell me how do you adjust the data distribution?
", "votes": 0, "replies": [] }, { "id": 611504, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T09:48:49.343000", "content": "A trained model was used to predict the class of public test sets and estimate the distribution.\nThere are various ways of sampling. Whether to do it only first, or every epoch, or to make a K-Fold.
", "votes": 1, "replies": [] }, { "id": 611534, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-29T10:06:30.907000", "content": "@octpath0302 that this mean you do not crop your images? (you said you mean subtract only)
", "votes": 0, "replies": [] }, { "id": 611548, "author_name": "hirune924", "author_url": "", "post_date": "2019-08-29T10:13:33.163000", "content": "@octpath0302 I thought of sampling based on the class distribution of public test dataset. But I thought that way would overfit public test dataset. I think the distribution of private test dataset is more similar to train dataset than private test dataset. What do you think about this？
", "votes": 1, "replies": [] }, { "id": 611563, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-29T10:20:47.900000", "content": "@hirune924 you can probe it and see. someone from the top has done it for sure.
", "votes": 0, "replies": [] }, { "id": 611566, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T10:21:53.580000", "content": "@valanm Sorry for misleading! In the preprocessing stage, I use auto cropping which is similar to the 1st solution on the previous competition, and padding.\nI make preprocessing to get robustness against black area at corner of each image.
", "votes": 2, "replies": [] }, { "id": 611568, "author_name": "Gary", "author_url": "", "post_date": "2019-08-29T10:22:22.813000", "content": "@hirune924 I also thought that way is easy to overfit public lb, but we do not know the distribution of private test dataset.
", "votes": 0, "replies": [] }, { "id": 611577, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T10:27:00.067000", "content": "@hirune924 I also think there is a danger of overfit. However, it is generally possible that there is a lot of data in class 0 (normal) and class 2 (middle). Therefore, I assume that the public test set and the private test set are randomly divided, so far.
", "votes": 2, "replies": [] }, { "id": 611765, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T12:24:41.440000", "content": "@garybios @valanm @virajbagal How about you? Which size/preprocessing do you use?
", "votes": 0, "replies": [] }, { "id": 611792, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-29T12:46:45.253000", "content": "300px b3 classification gives me .822, for preprocessing just cropping the object.
", "votes": 2, "replies": [] }, { "id": 611811, "author_name": "Gary", "author_url": "", "post_date": "2019-08-29T12:59:18.987000", "content": "@octpath0302 224, no preprocessing.\n@valanm nice job, you made a great classification model.
", "votes": 1, "replies": [] }, { "id": 611880, "author_name": "Viraj Bagal", "author_url": "", "post_date": "2019-08-29T13:37:50.370000", "content": "@octpath0302 256, cropping image and training only on new data gave me 0.796. Despite adding augmentations, my model trained on OLD Data with New Data as validation is overfitting and always predicting 0. I tried to balance the classes by downsampling and upsampling with each class around 5000 images but the score is really poor. I am desperately trying again and again to train the model on old data.
", "votes": 2, "replies": [] }, { "id": 611929, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T14:05:24.153000", "content": "@virajbagal Hmm. . that's strange. But in my case too, augmentation is hardly working. And I use only downsampling (I have not tried upsampling).\nAre CV and LB (or losses) somewhat correlated? Do your CV get >0.85?
", "votes": 1, "replies": [] }, { "id": 611945, "author_name": "Viraj Bagal", "author_url": "", "post_date": "2019-08-29T14:11:41.657000", "content": "My CV goes upto 92. But predicts all zeros.
", "votes": 0, "replies": [] }, { "id": 611950, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T14:18:49.520000", "content": "My CV has never reached to 0.9. So, I think that overfit or something is happening in your case..\n(too small validation set or .. I don't know.)\nIn my case, CV is around 0.87 to 0.88 and LB is around 0.81 to 0.83.
", "votes": 1, "replies": [] }, { "id": 612069, "author_name": "Chanhu", "author_url": "", "post_date": "2019-08-29T15:30:18.047000", "content": "Me too..
", "votes": 0, "replies": [] }, { "id": 612070, "author_name": "Ravi Kumar Vadapalli", "author_url": "", "post_date": "2019-08-29T15:31:28.803000", "content": "@octpath0302 , thanks for sharing the details. A quick question on the external data usage. External datasets have different labels. Some of them have DR levels (0 to 4 or 0 to 3), some have segmented masks, some have counts of microaneurisms, exudates, etc. How are you managing with such desparate labels. Are you somehow creating uniform labels for all the external datasets ?
", "votes": 0, "replies": [] }, { "id": 612090, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T15:46:40.437000", "content": "@ravivadapalli I use datasets which DR levels are 0 to 4 as labeled dataset, and the others are as unlabeled (pseudo-labeled or something) dataset. \nI don't try to use any other labels and masks because there is little time to Kaggle..
", "votes": 3, "replies": [] }, { "id": 612215, "author_name": "Ravi Kumar Vadapalli", "author_url": "", "post_date": "2019-08-29T16:58:51.740000", "content": "I see. thank you @octpath0302 for sharing the details.
", "votes": 0, "replies": [] }, { "id": 612309, "author_name": "Andrew Minh Nguyen", "author_url": "", "post_date": "2019-08-29T18:02:23.947000", "content": "Hi, @octpath0302. Thank you for sharing so much of your approach. Could you explain more how you apply mean subtraction to your images?
", "votes": 0, "replies": [] }, { "id": 612602, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T22:19:20.673000", "content": "I refer to the 1st place solution of the previous competition.\nhttps://www.kaggle.com/c/diabetic-retinopathy-detection/discussion/15801#latest-370950
", "votes": 1, "replies": [] }, { "id": 612609, "author_name": "Andrew Minh Nguyen", "author_url": "", "post_date": "2019-08-29T22:40:32.370000", "content": "Got it, thanks!
", "votes": 0, "replies": [] }, { "id": 613421, "author_name": "RAMPAGE", "author_url": "", "post_date": "2019-08-30T13:47:13.103000", "content": "May I ask what is your train setting?
", "votes": 0, "replies": [] }, { "id": 613587, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-08-30T16:31:48.790000", "content": "Hi @octpath0302 , did you observe big drop in CV after applying downsampling but LB may be the same or even better?
", "votes": 0, "replies": [] }, { "id": 613917, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-31T00:15:57.010000", "content": "@andalpha I have provided a lot of information, so I can't do any more..\n@naivelamb I'm not sure because I've never trained by applying downsampling only. After applying downsampling and data addition, I observed that CV was almost unchanged and LB was getting better.
", "votes": 0, "replies": [] }, { "id": 613932, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-08-31T00:37:24.717000", "content": "By data addition you mean mixing external data (2015 for example) with current data together to train the model, right?
", "votes": 0, "replies": [] }, { "id": 613943, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-31T00:43:31.403000", "content": "yes.
", "votes": 0, "replies": [] }, { "id": 614227, "author_name": "Jaideep", "author_url": "", "post_date": "2019-08-31T08:28:11.460000", "content": "hi oct when you training distribution sampling wrt to public test set.. what you mean\nis it taking more class 2 data or how is it ...
", "votes": 0, "replies": [] }, { "id": 614234, "author_name": "Jaideep", "author_url": "", "post_date": "2019-08-31T08:40:13.860000", "content": "@valann by b3 classification you mean not using regression?
", "votes": 0, "replies": [] }, { "id": 614247, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-31T08:52:22.140000", "content": "yes I only use classification
", "votes": 0, "replies": [] }, { "id": 614256, "author_name": "Jaideep", "author_url": "", "post_date": "2019-08-31T09:23:07.180000", "content": "@octpath which external data set you using.. \nThanks in advance
", "votes": 0, "replies": [] }, { "id": 614660, "author_name": "", "author_url": "", "post_date": "2019-08-31T21:13:24.103000", "content": "", "votes": 0, "replies": [] }, { "id": 614702, "author_name": "ilovescience", "author_url": "", "post_date": "2019-08-31T23:52:08.133000", "content": "do you use optimized thresholds?
", "votes": 0, "replies": [] }, { "id": 614756, "author_name": "Jaideep", "author_url": "", "post_date": "2019-09-01T02:53:29.483000", "content": "@valan how did you work with wierd crop patches produced by fastai ...I found some blacks borders with zero padding mode,or circle arcs at border with reflection ...
", "votes": 0, "replies": [] }, { "id": 615136, "author_name": "oct_path", "author_url": "", "post_date": "2019-09-01T14:01:40.210000", "content": "@ggoldszmidt I mean local mean subtraction (high-pass filtering), so not standardized. Almost the same as 1st place solution of the previous competition.\n@tanlikesmath Yes. Optimized.\n@jaideepvalani \n> hi oct when you training distribution sampling wrt to public test set.. what you mean\n> is it taking more class 2 data or how is it …
\n\nSorry, I can't catch your meaning. You can calculate the public test class distribution with a model (e. g. from public kernel). \nExternal data I'm using are listed up on the official thread.
", "votes": 0, "replies": [] }, { "id": 615160, "author_name": "Jaideep", "author_url": "", "post_date": "2019-09-01T14:50:28.137000", "content": "hi oct thanks for reply..\n1) in training set we have 3000 eg . how we decide on the ratio of distribution while sampling say we have batch size of 32..\n2) I tried to follow the external thread none of them have labels ,if i missed any link could u past the one that you used in ..\n3) which public kernel are talking about
", "votes": 0, "replies": [] }, { "id": 615176, "author_name": "oct_path", "author_url": "", "post_date": "2019-09-01T15:05:11.730000", "content": "@jaideepvalani \n1) I mean not sampling mini-batch, but how to re-construct labeled dataset.\n2) 3) Most of the external datasets are unlabeled datasets. But you can pseudo-label any images by using high-performance model from public kernels. If you already have good models, you can use them.
", "votes": 0, "replies": [] }, { "id": 615200, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-09-01T15:46:54.470000", "content": "@jaideepvalani i read the documentation and adopted the code. I default would be good enough then data science wouldn't be hot on job market.
", "votes": 0, "replies": [] } ] }, { "id": 615912, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-02T13:47:37.107000", "content": "Here are some of my results. I did on image size. All 5x CV
\n\nEfficientNet B5, 224 LB: 0.807\nEfficientNet B5, 300 LB: 0.812\nEfficientNet B5, 380 LB: 0.816\nEfficientNet B5, 456 LB: 0.823
EfficientNet B4, 224 LB: 0.800\nEfficientNet B4, 300 LB: 0.811\nEfficientNet B4, 380 LB: 0.816
@dir HB good results,your kernel public appreaciated by many kagglers and including me :)\nwould you like to tell the class distribution..
", "votes": 2, "replies": [] }, { "id": 615973, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-02T14:55:46.567000", "content": "Glad to hear that kernel helped =) My final distributions are very similar to some that found in this thread . =)
", "votes": 0, "replies": [] }, { "id": 615975, "author_name": "Benson Jin", "author_url": "", "post_date": "2019-09-02T14:59:08.097000", "content": "I tried doing 5x CV with EfficientNet B5 but I only get .79-.792. I dont know what else to do.
", "votes": 1, "replies": [] }, { "id": 615985, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-02T15:07:34.797000", "content": "They are diffrent ways how you can improve score. Have a look some threads like [this] (https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/105100#latest-615864) . All of them have good tips.
\n\nBut in genral. \n1) Try diffrent preprocessing \n2) Try diffrent augmentations \n3) Try combining of diffrent dataset.
\n\nForum is full of tips =) \nGood luck to you =)
", "votes": 2, "replies": [] }, { "id": 615987, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2019-09-02T15:08:00.310000", "content": "@drhabib when you increase img size, do you also increase your parameters, like learning rate or epochs? I'm asking this because for me it's harder to keep models with large img size learning.
", "votes": 1, "replies": [] }, { "id": 615996, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-02T15:17:51.187000", "content": "Good question.
\n\nI ususally run lr_finder and select learning rate based on this, but in most casses its very similar across diffrent sizes. Also regarding number of Epochs if you are intrested, for 224 I run for 20 epochs and for rest image sizes 5 epochs is enogh. Again it depends also on which learning rate policy you are using in my case its one_cycle_policy.
\n\nHope it helps =)
", "votes": 5, "replies": [] }, { "id": 616011, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2019-09-02T15:44:01.563000", "content": "Thanks @drhabib , I have tried many policies, also I use Keras, but the bigger models don't train longer than 10 epochs reliably.
", "votes": 1, "replies": [] }, { "id": 616016, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-02T15:47:02.880000", "content": "which optimizer are you using ?
", "votes": 0, "replies": [] }, { "id": 616022, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2019-09-02T15:58:23.130000", "content": "For me what seems to work is either Adam or RAdam, but also with LR policies, like Cosine or cycle.
\n\nSo far my best single model result comes from B5 224, using Adam and cosine LR (almost 0.8)
", "votes": 1, "replies": [] }, { "id": 616051, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-09-02T16:30:58.663000", "content": "Do you use weights from lower resolution or u always start from imagenet?
", "votes": 1, "replies": [] }, { "id": 616062, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2019-09-02T16:41:17.920000", "content": "@valanm , I'm starting from imagenet. By lower resolution, do you mean pretraining from the older dataset?
", "votes": 1, "replies": [] }, { "id": 616068, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-09-02T16:49:07.713000", "content": "For instance, train 20 epochs on 224 and then 5 more with increased resolution to 456
", "votes": 4, "replies": [] }, { "id": 616072, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-02T16:54:05.180000", "content": "I think the issue might be that when you go to higher size you have to use small batch size. \n1) Have you tried to do gradient accumulation ?\n2) Try Mish activation function. Basicity its very simple to implement here is the paper. (https://arxiv.org/pdf/1908.08681v1.pdf) . You can see in on figure 7 that decreasing batch size has little effect in accuracy in comparison to ReLu. In my case it helped a bit.
\n\n
@valanm I reuse weight for lower resolution to train higher \nHope this helps
", "votes": 8, "replies": [] }, { "id": 616094, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2019-09-02T17:17:21.433000", "content": "Thank @valanm and @drhabib , both suggestions are very interesting, I'll give a try.
", "votes": 1, "replies": [] }, { "id": 617975, "author_name": "Nitish Kumar", "author_url": "", "post_date": "2019-09-04T17:18:55.830000", "content": "@drhabib i was just wondering , if increasing the image size should take more training as we have more to learn. Can you explain something about this?
", "votes": 1, "replies": [] }, { "id": 618051, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-04T19:04:55.393000", "content": "Hi.. I will refer you to this two blog posts. \nhttps://miguel-data-sc.github.io/2017-11-23-second/\nhttps://towardsdatascience.com/boost-your-cnn-image-classifier-performance-with-progressive-resizing-in-keras-a7d96da06e20
", "votes": 1, "replies": [] }, { "id": 619846, "author_name": "Jaideep", "author_url": "", "post_date": "2019-09-06T16:50:48.257000", "content": "@drhb i was wondering were u able to run wit b5 456 sz image on kaggles kernel,how much was bs..
", "votes": 1, "replies": [] }, { "id": 619848, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-06T16:52:08.863000", "content": "for interference only =) \nbs = 64 did a job, but i think going higher is possible but i havent tried =)
", "votes": 0, "replies": [] }, { "id": 619853, "author_name": "Jaideep", "author_url": "", "post_date": "2019-09-06T16:54:57.507000", "content": "ok so training on smaller image size and but inference on a higher one could work well ? this is new for me i was thinking that we should always give same size image for inference as training..\nPlease confirm.. i will add this in my learning book :)
", "votes": 1, "replies": [] }, { "id": 619871, "author_name": "DrHB", "author_url": "", "post_date": "2019-09-06T17:19:27.677000", "content": "When you are doing interference you only do forward pass and you dont have to compute gradients as a result more memory is available for gpu and therefore you can use big batch size =)
", "votes": 0, "replies": [] }, { "id": 620725, "author_name": "Jaideep", "author_url": "", "post_date": "2019-09-07T22:51:17.453000", "content": "in my last submission i get some thing similar to what is posted without doing any mechanical adjustments much.. \nbut dont know will i get also similar score as tilll now for all submissions only the ones having 2 above 1200 have seen score above 0.80.. ,1 and 3 between 130 and 160, the posted ones are very different to these..\nWas that bar graph post a competition gimmick hahah :) ...
", "votes": 0, "replies": [] }, { "id": 621395, "author_name": "Jaideep", "author_url": "", "post_date": "2019-09-08T13:33:19.477000", "content": "hi hb good to see you gained the rank in private lb..\ncongrats\ncould you tell me how you replaced the default act functions for the model..
", "votes": 0, "replies": [] } ] }, { "id": 610215, "author_name": "leixiang@AInnovation", "author_url": "", "post_date": "2019-08-28T14:49:24.983000", "content": "456, efficientnetb5 lb:0,824 single model
", "votes": 6, "replies": [ { "id": 610243, "author_name": "Gary", "author_url": "", "post_date": "2019-08-28T15:20:50.643000", "content": "a nice lb score.
", "votes": 0, "replies": [] }, { "id": 610699, "author_name": "hirune924", "author_url": "", "post_date": "2019-08-29T00:19:29.260000", "content": "nice work! my best lb score of 256 single model is 0.821.
", "votes": 0, "replies": [] }, { "id": 610786, "author_name": "leixiang@AInnovation", "author_url": "", "post_date": "2019-08-29T01:15:25.363000", "content": "whats kind of network?
", "votes": 0, "replies": [] }, { "id": 610896, "author_name": "hirune924", "author_url": "", "post_date": "2019-08-29T02:44:11.283000", "content": "efficientnetb5
", "votes": 0, "replies": [] }, { "id": 611029, "author_name": "Ankit", "author_url": "", "post_date": "2019-08-29T04:21:47.390000", "content": "@hirune924 @leixiang Nice scores !!\nWhat kind of preprocessing you guys used?\nDid you first pretrain on old train and finetune on new
", "votes": 0, "replies": [] }, { "id": 611757, "author_name": "SchenbergZ", "author_url": "", "post_date": "2019-08-29T12:19:18.590000", "content": "@leixiang @hirune924 nice to see single model reach the score. May I ask for If you have any change of the output header of the efficientnet like adding BN or concat [avgpool, maxpool]? Thank you all.
", "votes": 0, "replies": [] }, { "id": 612317, "author_name": "Andrew Minh Nguyen", "author_url": "", "post_date": "2019-08-29T18:06:14.153000", "content": "Hi,
\n\nYou are asking others, but I would like to offer my experience: Using MaxPool with EfficientNet seemed to hurt the model's performance. I tried several times with different experiments and no good. I thought ConcatPool should improve performance, but it seems that the information from the MaxPool only brings the performance of the AvgPool down. Of course, this is only data from my limited experiments.
", "votes": 0, "replies": [] }, { "id": 612656, "author_name": "SchenbergZ", "author_url": "", "post_date": "2019-08-29T23:28:22.537000", "content": "This idea is borrowed from the bi-lstm model. It seems no one who has high rank in the discussion has told this feature recently... you are right and I make a mistake... I just search my log and find that using concat just decrease my CV from 0.8091 to 0.7993 because I have too many trials and i get confused... Maybe I need to figure out the difference between them.
", "votes": 1, "replies": [] } ] }, { "id": 612000, "author_name": "Chanhu", "author_url": "", "post_date": "2019-08-29T14:52:56.757000", "content": "328 B5 Classification, reach 0.825.
\n328 B5 Regression, reach 0.823.\n256 B4 Regression, reach 0.821
[Uptate 2019/08/30 13:18]\n256 B4 Classification, reach 0.820
", "votes": 1, "replies": [ { "id": 612012, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-29T15:04:45.767000", "content": "and ensemble gives you your current score 0.833 !?
", "votes": 0, "replies": [] }, { "id": 612058, "author_name": "Chanhu", "author_url": "", "post_date": "2019-08-29T15:27:09.777000", "content": "Yes.... Only ensemble B5 regression and B4 regression
", "votes": 1, "replies": [] }, { "id": 612124, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T15:58:43.767000", "content": "0.823 & 0.812 -> 0.833 ? This is a great synergy effect!!\nDo you think that the difference in size has an effect?
", "votes": 0, "replies": [] }, { "id": 612148, "author_name": "leixiang@AInnovation", "author_url": "", "post_date": "2019-08-29T16:11:55.400000", "content": "i got similar result with him just ensemble a lb 0.824 eftnet b5 456 and lb 0.814 eftnet b4 380，maybe different image size boost the score
", "votes": 1, "replies": [] }, { "id": 612182, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-29T16:34:44.243000", "content": "I tried voting between 4 classification models \nb3 300 0.808\nb3 416 0.812\nb5 384 0.812\nb5 456 0.817\nbut got 0.785, couldn't find any bug so I concluded its better to use averaging, but i haven't tried yet.
", "votes": 1, "replies": [] }, { "id": 612660, "author_name": "Chanhu", "author_url": "", "post_date": "2019-08-29T23:33:59.357000", "content": "@octpath0302 I am not sure maybe not.\nIn my experience, combining different structure model output together may boost score.\n I am still trying to improve my single model score right now. (struggling)
", "votes": 2, "replies": [] }, { "id": 612667, "author_name": "Chanhu", "author_url": "", "post_date": "2019-08-29T23:44:59.013000", "content": "@valanm, I think you use majority voting, it did not work for my.
\n\nI use this way to ensemble two classification models. \nit slightly improve my score(b4:0.815, b5:0.825 -> 0.829),
\n\nfor one model prediction\nfor _ in range(TTA):\nwith torch.no_grad():\n for i, data in tqdm(enumerate(test_loader)):\n images = data\n images = images.cuda()\n pred = model(images)\n pred = torch.softmax(pred, dim=-1)\n test_pred[i * 64:(i + 1) * 64] += pred
add two model prediction together\noutput = np.argmax(((test_pred1 / TTA) + (test_pred2 / TTA)) / 2, axis=1)
@chanhu This also happened to our case. Ensembling (averaging) B5 (0. 819) and B0 (0.812) made our current leaderboard score (0.837). Although, the training procedure for each model was completely different, which might have caused such synergy!
", "votes": 3, "replies": [] }, { "id": 613251, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-30T11:01:34.387000", "content": "@udaykamal that's just awesome synergy
", "votes": 1, "replies": [] }, { "id": 614383, "author_name": "kevin", "author_url": "", "post_date": "2019-08-31T12:21:09.477000", "content": "@chanhu @udaykamal how many tta transform do you use? flip,transpose and multi-scale?
", "votes": 0, "replies": [] }, { "id": 614397, "author_name": "Chanhu", "author_url": "", "post_date": "2019-08-31T12:41:10.700000", "content": "tta 5 or 10. only flip, and rotation.
", "votes": 0, "replies": [] }, { "id": 614420, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-08-31T13:10:01.410000", "content": "@chanhu have you checked how much boost this gives you. i tried horizontal flip but the score got worse 0.001
", "votes": 0, "replies": [] }, { "id": 614483, "author_name": "Uday Kamal", "author_url": "", "post_date": "2019-08-31T14:49:45.930000", "content": "@hustkevin1037 we have not tried with TTA yet. Out current leaderboard score is simple average of two predictions from two different model. No TTA, no K-fold.
", "votes": 0, "replies": [] }, { "id": 614515, "author_name": "Jaideep", "author_url": "", "post_date": "2019-08-31T15:54:09.760000", "content": "Hey uday did you try regression only for both ?
", "votes": 0, "replies": [] }, { "id": 614745, "author_name": "Gabriel", "author_url": "", "post_date": "2019-09-01T01:55:32.897000", "content": "@valanm do you use torch or keras?
", "votes": 0, "replies": [] }, { "id": 614748, "author_name": "kevin", "author_url": "", "post_date": "2019-09-01T02:07:41.613000", "content": "@udaykamal did you use optimized threshold or just [0.5,1,5,2,5,3,5] ?
", "votes": 0, "replies": [] }, { "id": 614903, "author_name": "Chanhu", "author_url": "", "post_date": "2019-09-01T08:03:41.890000", "content": "@valanm about +0.004.
", "votes": 0, "replies": [] }, { "id": 615011, "author_name": "Uday Kamal", "author_url": "", "post_date": "2019-09-01T11:03:34.163000", "content": "@jaideepvalani yes\n@hustkevin1037 optimized threshold resulted in lower score in our case. We are using just [0.5,1,5,2,5,3,5]
", "votes": 0, "replies": [] }, { "id": 615207, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-09-01T15:56:06.917000", "content": "@custodiogabriel i am trying fastai for this competition\nthis might be a dumb question but are you guys ensembling logits or softmax outputs, does this matter at all?
", "votes": 0, "replies": [] }, { "id": 615508, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-09-02T03:15:49.100000", "content": "@valanm If you are doing classification, softmax outputs makes more sense.
", "votes": 1, "replies": [] }, { "id": 615547, "author_name": "Jaideep", "author_url": "", "post_date": "2019-09-02T04:54:49.937000", "content": "@ uday thanks...\ncould you please tell what is distribution of each class in submission score..
", "votes": 0, "replies": [] }, { "id": 615574, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-09-02T05:44:04.953000", "content": "@naivelamb thanks. Is the reason scaling? Would ligits make sense for stacking?
", "votes": 0, "replies": [] }, { "id": 615596, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-09-02T06:23:07.523000", "content": "It depends on what stacking methods you are using. Some are obviously sensitive to scaling.
", "votes": 0, "replies": [] }, { "id": 615733, "author_name": "yu", "author_url": "", "post_date": "2019-09-02T09:58:29.877000", "content": "may I know what batch size u are using? I am training with 16 only due to limited gpu ram. Considering to rent a machine if batch size matters
", "votes": 0, "replies": [] }, { "id": 616046, "author_name": "Miroslav Valan", "author_url": "", "post_date": "2019-09-02T16:25:06.247000", "content": "Averaging softmax outputs instead of logits gave the same score. However, 12 images (0.6%) got different label.
", "votes": 0, "replies": [] }, { "id": 620049, "author_name": "Chanhu", "author_url": "", "post_date": "2019-09-07T00:08:12.497000", "content": "@valanm I hope you can get good score in private test set
", "votes": 0, "replies": [] } ] }, { "id": 616493, "author_name": "Atabay Ziyaden", "author_url": "", "post_date": "2019-09-03T07:21:07.007000", "content": "EfficientNet b5, no CV or LB improvement after 256
", "votes": 0, "replies": [] }, { "id": 612028, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2019-08-29T15:11:56.193000", "content": "Is there a reason to use img size 256x256 with efficient nets? I ask this because it's not the default size for any of its models, but people seem to get good results anyway.
", "votes": 0, "replies": [ { "id": 612076, "author_name": "oct_path", "author_url": "", "post_date": "2019-08-29T15:37:22.917000", "content": "In my understanding, the default sizes have been searched on a specific dataset. And the dataset are different from this competition dataset. So, I think it is not always necessary to resize this dataset according to the default size.\nMany people have reported that the accuracy is good even if it is different from the default size, so I think it supports this idea.
", "votes": 3, "replies": [] }, { "id": 612098, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2019-08-29T15:50:34.097000", "content": "I understand @octpath0302 , this makes sense.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "610122": "This is the last chance to rethink the image size because we have little time.\nI am currently using only 256. I've already tried a larger size but it only took a long time to train and didn't give good score. What is your best image size？", "612806": "My single models are, \n1. 300 Regression EfficientNet B3, LB 0.830\n2. 260 Regression EfficientNet B2, LB 0.820\n\nOur current place is from Model 1. ", "611126": "I use 256 preprocessed by mean subtraction.\nMy best single model (EfficientNet-B4) got 0.829.\nI've tried 320 and 512, but it was a waste of time.", "615912": "Here are some of my results. I did on image size. All 5x CV \n\n`EfficientNet B5, 224 LB: 0.807`\n`EfficientNet B5, 300 LB: 0.812`\n`EfficientNet B5, 380 LB: 0.816`\n`EfficientNet B5, 456 LB: 0.823`\n\n`EfficientNet B4, 224 LB: 0.800`\n`EfficientNet B4, 300 LB: 0.811`\n`EfficientNet B4, 380 LB: 0.816`\n\n", "610215": "456, efficientnetb5 lb:0,824 single model", "612000": "328 B5 Classification, reach 0.825. \n328 B5 Regression, reach 0.823.\n256 B4 Regression, reach 0.821\n\n[Uptate 2019/08/30 13:18]\n256 B4 Classification, reach 0.820", "616493": "EfficientNet b5, no CV or LB improvement after 256", "612028": "Is there a reason to use img size 256x256 with efficient nets? I ask this because it's not the default size for any of its models, but people seem to get good results anyway." } }