Just starting a common competition thread.
\n\nWhat is your current best single model?
\n\nI'll start.
\n\nMy current LB (0.760) is a single ResNet50, classification (not regression), single fold. But I am not relying too much on this due to the problems with test label distribution.
", "messages": [ { "id": "571795", "postDate": "07/10/2019 05:19:33", "content": "Just starting a common competition thread.
\n\nWhat is your current best single model?
\n\nI'll start.
\n\nMy current LB (0.760) is a single ResNet50, classification (not regression), single fold. But I am not relying too much on this due to the problems with test label distribution.
", "rawMarkdown": "Just starting a common competition thread.\n\nWhat is your current best single model?\n\nI'll start.\n\nMy current LB (0.760) is a single ResNet50, classification (not regression), single fold. But I am not relying too much on this due to the problems with test label distribution.", "votes": null }, { "id": "571820", "postDate": "07/10/2019 05:59:22", "content": "Are you using data from past contest too? My current best single model is 0.723 ResNet50 regression
", "rawMarkdown": "Are you using data from past contest too? My current best single model is 0.723 ResNet50 regression", "votes": null }, { "id": "571840", "postDate": "07/10/2019 06:29:00", "content": "Yes I am using past data.
", "rawMarkdown": "Yes I am using past data.", "votes": null }, { "id": "571885", "postDate": "07/10/2019 07:38:03", "content": "First, I want to appreciate your effort to create past competition dataset.
\n\nI'm using now !
\n\nPardon me, are you using cropped data? or not cropped but resized data in your past competition set or both?
\n\nMy score (0.752) is average of resnet50 and resnext50 using this competition data with regression and single fold
\n\nAnd my best single is 0.740 with EfficientB5 with regression using this competition data.
", "rawMarkdown": "First, I want to appreciate your effort to create past competition dataset.\n\nI'm using now !\n\nPardon me, are you using cropped data? or not cropped but resized data in your past competition set or both?\n\nMy score (0.752) is average of resnet50 and resnext50 using this competition data with regression and single fold\n\nAnd my best single is 0.740 with EfficientB5 with regression using this competition data.", "votes": null }, { "id": "571895", "postDate": "07/10/2019 07:56:22", "content": "My current best single model is 0.74(LB) ResNet 18, multi label classification like Lex Toumbourou's kernel.\nI do preprocessing image by referring Neuron Engineer's kernel.\nI'm learned a lot from them.
", "rawMarkdown": "My current best single model is 0.74(LB) ResNet 18, multi label classification like [Lex Toumbourou's kernel](https://www.kaggle.com/lextoumbourou/blindness-detection-resnet34-ordinal-targets).\nI do preprocessing image by referring [Neuron Engineer's kernel](https://www.kaggle.com/ratthachat/aptos-updated-preprocessing-ben-s-cropping).\nI'm learned a lot from them.", "votes": null }, { "id": "571917", "postDate": "07/10/2019 08:28:26", "content": "@youhanlee did you use past data for 0.742 single model ?
", "rawMarkdown": "youhanlee did you use past data for 0.742 single model ?", "votes": null }, { "id": "571964", "postDate": "07/10/2019 09:32:29", "content": "I've modified that. I got 0.740 with EfficientB5 with regression using this competition data :)
", "rawMarkdown": "I've modified that. I got 0.740 with EfficientB5 with regression using this competition data :)", "votes": null }, { "id": "571989", "postDate": "07/10/2019 10:00:16", "content": "Nice, can you tell what was your kappa score throughout the training on past data?
", "rawMarkdown": "Nice, can you tell what was your kappa score throughout the training on past data?", "votes": null }, { "id": "572536", "postDate": "07/11/2019 03:33:07", "content": "0.750 LB with SE-ResNeXt. 1 fold. Using both dataset.\nAlso TTA of RandomHorizontalFlip & RandomRotate of 360 degree.
Hi @tanlikesmath , how is your CV on old (previous competition) data ? I am not quite success on the old data, and got only CV 0.55-0.6
", "rawMarkdown": "Hi @tanlikesmath , how is your CV on old (previous competition) data ? I am not quite success on the old data, and got only CV 0.55-0.6", "votes": null }, { "id": "573016", "postDate": "07/11/2019 16:59:05", "content": "Hey have you modified the backbone layers of SE-ResNeXt?
", "rawMarkdown": "Hey have you modified the backbone layers of SE-ResNeXt?", "votes": null }, { "id": "573044", "postDate": "07/11/2019 17:58:35", "content": "I have a CV of 0.747 on the previous competition data. There are many different tricks to improve this score, and when I used a traditional CNN pipeline, I obtained similar scores. But once I added a couple of training tricks I was able to reach 0.65 and above. I still haven't reached around 0.84, which (I think) is the SOTA by Google (see here) and would be on par with the performance of ophthalmologists. But then again, that's Google, and they have a lot more data and processing power, so it would be hard to beat them!
\n\nRight now I don't plan to share the techniques I used to reach a high CV on the previous dataset, but maybe later in the competition I will ;)
", "rawMarkdown": "I have a CV of 0.747 on the previous competition data. There are many different tricks to improve this score, and when I used a traditional CNN pipeline, I obtained similar scores. But once I added a couple of training tricks I was able to reach 0.65 and above. I still haven't reached around 0.84, which (I think) is the SOTA by Google (see [here](https://www.aaojournal.org/article/S0161-6420(17)32698-2/fulltext)) and would be on par with the performance of ophthalmologists. But then again, that's Google, and they have a lot more data and processing power, so it would be hard to beat them!\n\nRight now I don't plan to share the techniques I used to reach a high CV on the previous dataset, but maybe later in the competition I will ;)", "votes": null }, { "id": "573072", "postDate": "07/11/2019 18:43:29", "content": "I only split the old competition training data with 0.9 training and 0.1 validation, I can get 0.804 with validation data. But I was using threshold optimizer for the metric, now I'm trying to get rid of the threshold optimizer which is highly depends on the target distribution, still running experiments.
", "rawMarkdown": "I only split the old competition training data with 0.9 training and 0.1 validation, I can get 0.804 with validation data. But I was using threshold optimizer for the metric, now I'm trying to get rid of the threshold optimizer which is highly depends on the target distribution, still running experiments.", "votes": null }, { "id": "573135", "postDate": "07/11/2019 20:25:50", "content": "My split was 0.8 train 0.2 validation, so another reason you are performing better could be due to including more training data in the dataset. In addition, you are performing regression w/ threshold optimizer, while my experiments have been using plain multi-class classification with a regular ResNet50.
", "rawMarkdown": "My split was 0.8 train 0.2 validation, so another reason you are performing better could be due to including more training data in the dataset. In addition, you are performing regression w/ threshold optimizer, while my experiments have been using plain multi-class classification with a regular ResNet50.", "votes": null }, { "id": "573194", "postDate": "07/11/2019 23:27:48", "content": "Thanks both of you!
", "rawMarkdown": "Thanks both of you!", "votes": null }, { "id": "573241", "postDate": "07/12/2019 02:33:09", "content": "0.752, no tta, no k-fold, competition data\nupdate:0.770
", "rawMarkdown": "0.752, no tta, no k-fold, competition data\nupdate:0.770", "votes": null }, { "id": "573254", "postDate": "07/12/2019 03:20:37", "content": "No. I kept it as it is. Just changed last FC layer.
", "rawMarkdown": "No. I kept it as it is. Just changed last FC layer.", "votes": null }, { "id": "573478", "postDate": "07/12/2019 10:29:01", "content": "0.739 LB & 0.92 CV with ResNet50. 1 fold. Pretrained on old contest dataset with no TTA
", "rawMarkdown": "0.739 LB & 0.92 CV with ResNet50. 1 fold. Pretrained on old contest dataset with no TTA", "votes": null }, { "id": "573481", "postDate": "07/12/2019 10:29:48", "content": "@dldmw579 With old contest data or without it?
", "rawMarkdown": "dldmw579 With old contest data or without it?", "votes": null }, { "id": "573488", "postDate": "07/12/2019 10:37:51", "content": "without it
", "rawMarkdown": "without it", "votes": null }, { "id": "573940", "postDate": "07/13/2019 01:51:07", "content": "Why does the regression model work better than the classification
", "rawMarkdown": "Why does the regression model work better than the classification", "votes": null }, { "id": "573961", "postDate": "07/13/2019 03:15:11", "content": "This thread may give some info.
", "rawMarkdown": "[This](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/98239#latest-573709) thread may give some info.", "votes": null }, { "id": "574100", "postDate": "07/13/2019 08:18:37", "content": "Which model ?
", "rawMarkdown": "Which model ?", "votes": null }, { "id": "574280", "postDate": "07/13/2019 15:02:01", "content": "Thanks，valuable information
", "rawMarkdown": "Thanks，valuable information", "votes": null }, { "id": "574486", "postDate": "07/14/2019 00:25:47", "content": "您好！您有兴趣组队吗？谢谢！
", "rawMarkdown": "您好！您有兴趣组队吗？谢谢！", "votes": null }, { "id": "574508", "postDate": "07/14/2019 02:07:41", "content": "0.777. make it to public. \nwe still have two mouth. we should do better than this.\nhttps://www.kaggle.com/chanhu/eye-efficientnet-pytorch-lb-0-777
", "rawMarkdown": "0.777. make it to public. \nwe still have two mouth. we should do better than this.\nhttps://www.kaggle.com/chanhu/eye-efficientnet-pytorch-lb-0-777", "votes": null }, { "id": "574529", "postDate": "07/14/2019 03:24:01", "content": "If someone had the machine power($$$) to do it I bet training ResNeXt-101 32×48d would prob get the best score. Otherwise it looks like efficientnet would be best for the rest of us kernels/colab users. \nI bet that getting past that would depend on hyperparams, validations strategies, data augmentation, preprocessing, TTA, ensembling, etc.
", "rawMarkdown": "If someone had the machine power($$$) to do it I bet training ResNeXt-101 32×48d would prob get the best score. Otherwise it looks like efficientnet would be best for the rest of us kernels/colab users. \nI bet that getting past that would depend on hyperparams, validations strategies, data augmentation, preprocessing, TTA, ensembling, etc.", "votes": null }, { "id": "574535", "postDate": "07/14/2019 03:44:47", "content": "I doubt efficientnet is more effective (or might I say efficient? ;) ) than a ResNet50 (What I am currently using?). I think members of the fastai/Pytorch community have observed than indeed training a ResNet50 is more effective.
\n\nI do tricks can take you a really long way. I have been able to reach impressive results for binary classification on the previous competition dataset with just a few tricks. Similarly, I think training tricks will be key for a winning solution.
", "rawMarkdown": "I doubt efficientnet is more effective (or might I say efficient? ;) ) than a ResNet50 (What I am currently using?). I think members of the fastai/Pytorch community have observed than indeed training a ResNet50 is more effective.\n\nI do tricks can take you a really long way. I have been able to reach impressive results for binary classification on the previous competition dataset with just a few tricks. Similarly, I think training tricks will be key for a winning solution.", "votes": null }, { "id": "574541", "postDate": "07/14/2019 03:58:16", "content": "嗯嗯，到最后冲刺的时候，我们讨论一下
", "rawMarkdown": "嗯嗯，到最后冲刺的时候，我们讨论一下", "votes": null }, { "id": "574563", "postDate": "07/14/2019 05:09:50", "content": "Good job
", "rawMarkdown": "Good job", "votes": null }, { "id": "574665", "postDate": "07/14/2019 09:20:04", "content": "Not sure why the -1. No secret methods being shared here.
\n\n```\nBcw93: Hello! Do you have interest in forming a group? Let me know.
\n\ncare: 嗯嗯 - acknowledged message, when we reach 'closer to the end of the competition', let's discuss.\n```
", "rawMarkdown": "Not sure why the -1. No secret methods being shared here.\n\n```\nBcw93: Hello! Do you have interest in forming a group? Let me know.\n\ncare: *嗯嗯 - acknowledged message*, when we reach 'closer to the end of the competition', let's discuss.\n```", "votes": null }, { "id": "576906", "postDate": "07/16/2019 06:53:43", "content": "0.784 with TTA, Regression, no-fold
", "rawMarkdown": "0.784 with TTA, Regression, no-fold", "votes": null }, { "id": "576908", "postDate": "07/16/2019 06:54:50", "content": "Previous dataset?
", "rawMarkdown": "Previous dataset?", "votes": null }, { "id": "576960", "postDate": "07/16/2019 07:51:45", "content": "both past comp dataset and comp dataset
", "rawMarkdown": "both past comp dataset and comp dataset", "votes": null }, { "id": "578173", "postDate": "07/17/2019 12:38:03", "content": "resnet50
", "rawMarkdown": "resnet50", "votes": null }, { "id": "578290", "postDate": "07/17/2019 14:58:19", "content": "@tanlikesmath why aren't you using this competition data as validation set when pretraining on previous competition data? are you just splitting within previous data?
", "rawMarkdown": "tanlikesmath why aren't you using this competition data as validation set when pretraining on previous competition data? are you just splitting within previous data?", "votes": null }, { "id": "578683", "postDate": "07/18/2019 02:50:33", "content": "Hello, do you use classification or regression?, Thanks
", "rawMarkdown": "Hello, do you use classification or regression?, Thanks", "votes": null }, { "id": "578685", "postDate": "07/18/2019 02:54:34", "content": "Yes I did a 0.8-0.2 split in the previous data, and got a score of 0.747 on the validation of the previous dataset, just trained on the previous training set.
", "rawMarkdown": "Yes I did a 0.8-0.2 split in the previous data, and got a score of 0.747 on the validation of the previous dataset, just trained on the previous training set.", "votes": null }, { "id": "578690", "postDate": "07/18/2019 03:10:26", "content": "regression
", "rawMarkdown": "regression", "votes": null }, { "id": "579703", "postDate": "07/19/2019 06:13:11", "content": "single fold resent50 lb0.744 tta4. cv 0.91~0.92
", "rawMarkdown": "single fold resent50 lb0.744 tta4. cv 0.91~0.92", "votes": null }, { "id": "579848", "postDate": "07/19/2019 09:58:11", "content": "@zxyu1995, which model? ResNet50?
", "rawMarkdown": "zxyu1995, which model? ResNet50?", "votes": null }, { "id": "579858", "postDate": "07/19/2019 10:15:21", "content": "@prashantkikani is your lb 0.780 result of folds ensemble?
", "rawMarkdown": "prashantkikani is your lb 0.780 result of folds ensemble?", "votes": null }, { "id": "579905", "postDate": "07/19/2019 11:36:24", "content": "res50
", "rawMarkdown": "res50", "votes": null }, { "id": "579972", "postDate": "07/19/2019 13:39:21", "content": "Its a ensemble of Efficient net & SE-ResNeXt101.\nSingle Efficient net 0.779 LB.\nSingle SE-ResNeXt101 ~0.76 LB.
", "rawMarkdown": "Its a ensemble of Efficient net & SE-ResNeXt101.\nSingle Efficient net 0.779 LB.\nSingle SE-ResNeXt101 ~0.76 LB.", "votes": null }, { "id": "580435", "postDate": "07/20/2019 06:26:04", "content": "dose past comp dataset really help? i mean: is that using both datasets better than using this comp dataset only?
", "rawMarkdown": "dose past comp dataset really help? i mean: is that using both datasets better than using this comp dataset only?", "votes": null }, { "id": "580514", "postDate": "07/20/2019 09:04:17", "content": "@frank518 Yes. Use past competition dataset for pretraining, then fine tune the model on this competition's dataset. It'll help.
", "rawMarkdown": "frank518 Yes. Use past competition dataset for pretraining, then fine tune the model on this competition's dataset. It'll help.", "votes": null }, { "id": "580548", "postDate": "07/20/2019 10:39:00", "content": "@rishabhiitbhu thank you very much. BTW, i wonder how to using regression in this problem, really appreciate if you can post some links of kernels or articles. @zxyu1995
", "rawMarkdown": "rishabhiitbhu thank you very much. BTW, i wonder how to using regression in this problem, really appreciate if you can post some links of kernels or articles. @zxyu1995", "votes": null }, { "id": "580574", "postDate": "07/20/2019 11:45:30", "content": "For regression model, you'll have a single neuron output, as the labels are [0, 1, 2, 3, 4] you'll train your model to output a value around these values, for final predictions you can use a set of thresholds say [0.5, 1.5, 2.5, 3.5]. Check this out
", "rawMarkdown": "For regression model, you'll have a single neuron output, as the labels are [0, 1, 2, 3, 4] you'll train your model to output a value around these values, for final predictions you can use a set of thresholds say [0.5, 1.5, 2.5, 3.5]. Check [this](https://www.kaggle.com/taindow/instagram-to-aptos-resnext-101-32x8d) out", "votes": null }, { "id": "585379", "postDate": "07/27/2019 10:44:15", "content": "Anybody got 0.79+ using single model?
I'm using one model.
", "rawMarkdown": "I'm using one model.", "votes": null }, { "id": "585645", "postDate": "07/27/2019 19:42:11", "content": "Can you please elaborate? Which model are you using? Are you treating it as a regression or classification task? And what sort of augmentation techniques are you using? Thanks in advance :)
", "rawMarkdown": "Can you please elaborate? Which model are you using? Are you treating it as a regression or classification task? And what sort of augmentation techniques are you using? Thanks in advance :)", "votes": null }, { "id": "585648", "postDate": "07/27/2019 19:44:56", "content": "Everything is from public kernels :)
", "rawMarkdown": "Everything is from public kernels :)", "votes": null }, { "id": "585649", "postDate": "07/27/2019 19:48:41", "content": "Thanks. :)
", "rawMarkdown": "Thanks. :)", "votes": null }, { "id": "585650", "postDate": "07/27/2019 19:49:05", "content": "Could you at least mention what kind of model you are using? I am right now using just a vanilla ResNet50, but it seems to do well, people are using larger architectures like a se-resnet152
", "rawMarkdown": "Could you at least mention what kind of model you are using? I am right now using just a vanilla ResNet50, but it seems to do well, people are using larger architectures like a se-resnet152", "votes": null }, { "id": "585720", "postDate": "07/27/2019 23:29:31", "content": "Im using efficientnet(like most of the top public kernels). Just take whatever and adjust hyperparameters(can easily give 0.01 boost).
", "rawMarkdown": "Im using efficientnet(like most of the top public kernels). Just take whatever and adjust hyperparameters(can easily give 0.01 boost).", "votes": null }, { "id": "586175", "postDate": "07/28/2019 18:16:42", "content": "Can you please elaborate? Which model are you using? Are you treating it as a regression or classification task? And what sort of augmentation techniques are you using? Thanks in advance :)
", "rawMarkdown": "Can you please elaborate? Which model are you using? Are you treating it as a regression or classification task? And what sort of augmentation techniques are you using? Thanks in advance :)", "votes": null }, { "id": "586176", "postDate": "07/28/2019 18:16:44", "content": "Thanks. :)
", "rawMarkdown": "Thanks. :)", "votes": null }, { "id": "593556", "postDate": "08/06/2019 19:39:53", "content": "LB 793 with seresnext50 encoder and some custom-designed head. Flip TTA. Regression.
", "rawMarkdown": "LB 793 with seresnext50 encoder and some custom-designed head. Flip TTA. Regression.", "votes": null }, { "id": "593589", "postDate": "08/06/2019 20:53:24", "content": "Is TTA helping? because most competitors, including myself, seem to report otherwise
", "rawMarkdown": "Is TTA helping? because most competitors, including myself, seem to report otherwise", "votes": null }, { "id": "593684", "postDate": "08/07/2019 01:23:15", "content": "seresnext50 with single fold?
", "rawMarkdown": "seresnext50 with single fold?", "votes": null }, { "id": "593720", "postDate": "08/07/2019 02:49:05", "content": "LB 0.804 with DenseNet201 on a single fold. Ordinal regression + lots of preprocessing + augs + progressive resizing + trial and error.
\n\nFunnily enough, I haven't managed to get ensembling to work at all yet.
", "rawMarkdown": "LB 0.804 with DenseNet201 on a single fold. Ordinal regression + lots of preprocessing + augs + progressive resizing + trial and error.\n\nFunnily enough, I haven't managed to get ensembling to work at all yet.", "votes": null }, { "id": "593741", "postDate": "08/07/2019 03:26:09", "content": "odd, I ensembled 4 models(0.798, 0.798, 0.799, 0.799) and was able to get 0.806
", "rawMarkdown": "odd, I ensembled 4 models(0.798, 0.798, 0.799, 0.799) and was able to get 0.806", "votes": null }, { "id": "593756", "postDate": "08/07/2019 04:17:40", "content": "Nice to hear about progressive resizing worked in this competition. I will try that next :)
", "rawMarkdown": "Nice to hear about progressive resizing worked in this competition. I will try that next :)", "votes": null }, { "id": "593805", "postDate": "08/07/2019 06:28:34", "content": "LB 0.804 with EfficientNetB5 on a single fold.\ncircle crop + rotation + flips + old data(balanced)
", "rawMarkdown": "LB 0.804 with EfficientNetB5 on a single fold.\ncircle crop + rotation + flips + old data(balanced)", "votes": null }, { "id": "594220", "postDate": "08/07/2019 18:09:53", "content": "Single fold 0.799 efficientnetb4, regression, trained on 25 epochs on old and new data with flips and rotation augmentation
", "rawMarkdown": "Single fold 0.799 efficientnetb4, regression, trained on 25 epochs on old and new data with flips and rotation augmentation", "votes": null }, { "id": "594399", "postDate": "08/08/2019 00:26:56", "content": "Wondering what is the gap of local CV and LB?
", "rawMarkdown": "Wondering what is the gap of local CV and LB?", "votes": null }, { "id": "594414", "postDate": "08/08/2019 01:29:05", "content": "trained old and new data together？
", "rawMarkdown": "trained old and new data together？", "votes": null }, { "id": "594878", "postDate": "08/08/2019 15:16:08", "content": "single model 1 fold LB 0.813 EfficientNet, no TTA.
That's nice score. Do you unfreeze layers while training on any of the datasets ?
", "rawMarkdown": "That's nice score. Do you unfreeze layers while training on any of the datasets ?", "votes": null }, { "id": "594894", "postDate": "08/08/2019 15:35:49", "content": "Yes, We have custom unfreezing strategy.
", "rawMarkdown": "Yes, We have custom unfreezing strategy.", "votes": null }, { "id": "594897", "postDate": "08/08/2019 15:38:48", "content": "Thanks :)
", "rawMarkdown": "Thanks :)", "votes": null }, { "id": "594898", "postDate": "08/08/2019 15:39:28", "content": "Interesting, thanks for reporting. If you don't mind me asking (pls ignore otherwise): This is a single fit? Only on 2019 data or also incorporating 2015 in some way?
", "rawMarkdown": "Interesting, thanks for reporting. If you don't mind me asking (pls ignore otherwise): This is a single fit? Only on 2019 data or also incorporating 2015 in some way?", "votes": null }, { "id": "594901", "postDate": "08/08/2019 15:44:28", "content": "yes
", "rawMarkdown": "yes", "votes": null }, { "id": "594936", "postDate": "08/08/2019 16:59:35", "content": "Regarding data and training it pretty much the same (with some tweeks) as I discussed here https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815#581136.
", "rawMarkdown": "Regarding data and training it pretty much the same (with some tweeks) as I discussed here https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815#581136.", "votes": null }, { "id": "595157", "postDate": "08/08/2019 22:56:10", "content": "Were those 4 models all efficientnet or were they more varied?
", "rawMarkdown": "Were those 4 models all efficientnet or were they more varied?", "votes": null }, { "id": "596030", "postDate": "08/10/2019 03:47:23", "content": "0.805 with single model. EfficientNet B3 pretrained on Imagenet, ordinal regression, soft TTA, used Ben's preprocessing. Used both 2015 & 2019 data.\nMostly followed the suggestions of DrHB & Risabh.\nEnsemble of 3 such models did not improve or degrade the score much.
", "rawMarkdown": "0.805 with single model. EfficientNet B3 pretrained on Imagenet, ordinal regression, soft TTA, used Ben's preprocessing. Used both 2015 & 2019 data.\nMostly followed the suggestions of DrHB & Risabh.\nEnsemble of 3 such models did not improve or degrade the score much.", "votes": null }, { "id": "597260", "postDate": "08/12/2019 04:48:11", "content": "Hi Ravi, did you use any augmentations while training?
", "rawMarkdown": "Hi Ravi, did you use any augmentations while training?", "votes": null }, { "id": "597842", "postDate": "08/12/2019 19:44:47", "content": "0.799 with single model. \nEfficientNet B4, use pretrained weights on Imagenet. no TTA, resolution 300*300, monitor val loss.\nAug with rotate 360, flip by probability 0.5, Contrast 0.75 to 1.5, zoom in center 1 to 1.3.\nAlmost follow the idea by DrHB.
Hi Viraj, yes, used random flips, rotations and scale in augmentation.
", "rawMarkdown": "Hi Viraj, yes, used random flips, rotations and scale in augmentation.", "votes": null } ], "comments": [ { "id": 571820, "author_name": "axel81", "author_url": "", "post_date": "07/10/2019 05:59:22", "content": "Are you using data from past contest too? My current best single model is 0.723 ResNet50 regression
", "votes": null, "replies": [ { "id": 571840, "author_name": "tanlikesmath", "author_url": "", "post_date": "07/10/2019 06:29:00", "content": "Yes I am using past data.
", "votes": null, "replies": [] }, { "id": 571885, "author_name": "youhanlee", "author_url": "", "post_date": "07/10/2019 07:38:03", "content": "First, I want to appreciate your effort to create past competition dataset.
\n\nI'm using now !
\n\nPardon me, are you using cropped data? or not cropped but resized data in your past competition set or both?
\n\nMy score (0.752) is average of resnet50 and resnext50 using this competition data with regression and single fold
\n\nAnd my best single is 0.740 with EfficientB5 with regression using this competition data.
", "votes": null, "replies": [] }, { "id": 571917, "author_name": "axel81", "author_url": "", "post_date": "07/10/2019 08:28:26", "content": "@youhanlee did you use past data for 0.742 single model ?
", "votes": null, "replies": [] }, { "id": 571964, "author_name": "youhanlee", "author_url": "", "post_date": "07/10/2019 09:32:29", "content": "I've modified that. I got 0.740 with EfficientB5 with regression using this competition data :)
", "votes": null, "replies": [] }, { "id": 571989, "author_name": "axel81", "author_url": "", "post_date": "07/10/2019 10:00:16", "content": "Nice, can you tell what was your kappa score throughout the training on past data?
", "votes": null, "replies": [] } ] }, { "id": 571895, "author_name": "takamichitoda", "author_url": "", "post_date": "07/10/2019 07:56:22", "content": "My current best single model is 0.74(LB) ResNet 18, multi label classification like Lex Toumbourou's kernel.\nI do preprocessing image by referring Neuron Engineer's kernel.\nI'm learned a lot from them.
", "votes": null, "replies": [] }, { "id": 572536, "author_name": "prashantkikani", "author_url": "", "post_date": "07/11/2019 03:33:07", "content": "0.750 LB with SE-ResNeXt. 1 fold. Using both dataset.\nAlso TTA of RandomHorizontalFlip & RandomRotate of 360 degree.
Hey have you modified the backbone layers of SE-ResNeXt?
", "votes": null, "replies": [] }, { "id": 573254, "author_name": "prashantkikani", "author_url": "", "post_date": "07/12/2019 03:20:37", "content": "No. I kept it as it is. Just changed last FC layer.
", "votes": null, "replies": [] }, { "id": 579858, "author_name": "yangsaewon", "author_url": "", "post_date": "07/19/2019 10:15:21", "content": "@prashantkikani is your lb 0.780 result of folds ensemble?
", "votes": null, "replies": [] }, { "id": 579972, "author_name": "prashantkikani", "author_url": "", "post_date": "07/19/2019 13:39:21", "content": "Its a ensemble of Efficient net & SE-ResNeXt101.\nSingle Efficient net 0.779 LB.\nSingle SE-ResNeXt101 ~0.76 LB.
", "votes": null, "replies": [] } ] }, { "id": 572968, "author_name": "ratthachat", "author_url": "", "post_date": "07/11/2019 15:48:31", "content": "Hi @tanlikesmath , how is your CV on old (previous competition) data ? I am not quite success on the old data, and got only CV 0.55-0.6
", "votes": null, "replies": [ { "id": 573044, "author_name": "tanlikesmath", "author_url": "", "post_date": "07/11/2019 17:58:35", "content": "I have a CV of 0.747 on the previous competition data. There are many different tricks to improve this score, and when I used a traditional CNN pipeline, I obtained similar scores. But once I added a couple of training tricks I was able to reach 0.65 and above. I still haven't reached around 0.84, which (I think) is the SOTA by Google (see here) and would be on par with the performance of ophthalmologists. But then again, that's Google, and they have a lot more data and processing power, so it would be hard to beat them!
\n\nRight now I don't plan to share the techniques I used to reach a high CV on the previous dataset, but maybe later in the competition I will ;)
", "votes": null, "replies": [] }, { "id": 573072, "author_name": "jionie", "author_url": "", "post_date": "07/11/2019 18:43:29", "content": "I only split the old competition training data with 0.9 training and 0.1 validation, I can get 0.804 with validation data. But I was using threshold optimizer for the metric, now I'm trying to get rid of the threshold optimizer which is highly depends on the target distribution, still running experiments.
", "votes": null, "replies": [] }, { "id": 573135, "author_name": "tanlikesmath", "author_url": "", "post_date": "07/11/2019 20:25:50", "content": "My split was 0.8 train 0.2 validation, so another reason you are performing better could be due to including more training data in the dataset. In addition, you are performing regression w/ threshold optimizer, while my experiments have been using plain multi-class classification with a regular ResNet50.
", "votes": null, "replies": [] }, { "id": 573194, "author_name": "ratthachat", "author_url": "", "post_date": "07/11/2019 23:27:48", "content": "Thanks both of you!
", "votes": null, "replies": [] }, { "id": 578290, "author_name": "yangsaewon", "author_url": "", "post_date": "07/17/2019 14:58:19", "content": "@tanlikesmath why aren't you using this competition data as validation set when pretraining on previous competition data? are you just splitting within previous data?
", "votes": null, "replies": [] }, { "id": 578685, "author_name": "tanlikesmath", "author_url": "", "post_date": "07/18/2019 02:54:34", "content": "Yes I did a 0.8-0.2 split in the previous data, and got a score of 0.747 on the validation of the previous dataset, just trained on the previous training set.
", "votes": null, "replies": [] } ] }, { "id": 573241, "author_name": "dldmw579", "author_url": "", "post_date": "07/12/2019 02:33:09", "content": "0.752, no tta, no k-fold, competition data\nupdate:0.770
", "votes": null, "replies": [ { "id": 573481, "author_name": "axel81", "author_url": "", "post_date": "07/12/2019 10:29:48", "content": "@dldmw579 With old contest data or without it?
", "votes": null, "replies": [] }, { "id": 573488, "author_name": "dldmw579", "author_url": "", "post_date": "07/12/2019 10:37:51", "content": "without it
", "votes": null, "replies": [] }, { "id": 574100, "author_name": "harshthaker", "author_url": "", "post_date": "07/13/2019 08:18:37", "content": "Which model ?
", "votes": null, "replies": [] }, { "id": 578173, "author_name": "dldmw579", "author_url": "", "post_date": "07/17/2019 12:38:03", "content": "resnet50
", "votes": null, "replies": [] }, { "id": 578683, "author_name": "zxyu1995", "author_url": "", "post_date": "07/18/2019 02:50:33", "content": "Hello, do you use classification or regression?, Thanks
", "votes": null, "replies": [] }, { "id": 578690, "author_name": "dldmw579", "author_url": "", "post_date": "07/18/2019 03:10:26", "content": "regression
", "votes": null, "replies": [] } ] }, { "id": 573478, "author_name": "axel81", "author_url": "", "post_date": "07/12/2019 10:29:01", "content": "0.739 LB & 0.92 CV with ResNet50. 1 fold. Pretrained on old contest dataset with no TTA
", "votes": null, "replies": [] }, { "id": 573940, "author_name": "", "author_url": "", "post_date": "07/13/2019 01:51:07", "content": "Why does the regression model work better than the classification
", "votes": null, "replies": [ { "id": 573961, "author_name": "prashantkikani", "author_url": "", "post_date": "07/13/2019 03:15:11", "content": "This thread may give some info.
", "votes": null, "replies": [] }, { "id": 574280, "author_name": "", "author_url": "", "post_date": "07/13/2019 15:02:01", "content": "Thanks，valuable information
", "votes": null, "replies": [] }, { "id": 574486, "author_name": "bcwang", "author_url": "", "post_date": "07/14/2019 00:25:47", "content": "您好！您有兴趣组队吗？谢谢！
", "votes": null, "replies": [] }, { "id": 574541, "author_name": "", "author_url": "", "post_date": "07/14/2019 03:58:16", "content": "嗯嗯，到最后冲刺的时候，我们讨论一下
", "votes": null, "replies": [] }, { "id": 574665, "author_name": "learnmower", "author_url": "", "post_date": "07/14/2019 09:20:04", "content": "Not sure why the -1. No secret methods being shared here.
\n\n```\nBcw93: Hello! Do you have interest in forming a group? Let me know.
\n\ncare: 嗯嗯 - acknowledged message, when we reach 'closer to the end of the competition', let's discuss.\n```
", "votes": null, "replies": [] } ] }, { "id": 574508, "author_name": "chanhu", "author_url": "", "post_date": "07/14/2019 02:07:41", "content": "0.777. make it to public. \nwe still have two mouth. we should do better than this.\nhttps://www.kaggle.com/chanhu/eye-efficientnet-pytorch-lb-0-777
", "votes": null, "replies": [] }, { "id": 574529, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "07/14/2019 03:24:01", "content": "If someone had the machine power($$$) to do it I bet training ResNeXt-101 32×48d would prob get the best score. Otherwise it looks like efficientnet would be best for the rest of us kernels/colab users. \nI bet that getting past that would depend on hyperparams, validations strategies, data augmentation, preprocessing, TTA, ensembling, etc.
", "votes": null, "replies": [ { "id": 574535, "author_name": "tanlikesmath", "author_url": "", "post_date": "07/14/2019 03:44:47", "content": "I doubt efficientnet is more effective (or might I say efficient? ;) ) than a ResNet50 (What I am currently using?). I think members of the fastai/Pytorch community have observed than indeed training a ResNet50 is more effective.
\n\nI do tricks can take you a really long way. I have been able to reach impressive results for binary classification on the previous competition dataset with just a few tricks. Similarly, I think training tricks will be key for a winning solution.
", "votes": null, "replies": [] } ] }, { "id": 574563, "author_name": "qestar0405", "author_url": "", "post_date": "07/14/2019 05:09:50", "content": "Good job
", "votes": null, "replies": [] }, { "id": 576906, "author_name": "zxyu1995", "author_url": "", "post_date": "07/16/2019 06:53:43", "content": "0.784 with TTA, Regression, no-fold
", "votes": null, "replies": [ { "id": 576908, "author_name": "tanlikesmath", "author_url": "", "post_date": "07/16/2019 06:54:50", "content": "Previous dataset?
", "votes": null, "replies": [] }, { "id": 576960, "author_name": "zxyu1995", "author_url": "", "post_date": "07/16/2019 07:51:45", "content": "both past comp dataset and comp dataset
", "votes": null, "replies": [] }, { "id": 579848, "author_name": "prashantkikani", "author_url": "", "post_date": "07/19/2019 09:58:11", "content": "@zxyu1995, which model? ResNet50?
", "votes": null, "replies": [] }, { "id": 579905, "author_name": "zxyu1995", "author_url": "", "post_date": "07/19/2019 11:36:24", "content": "res50
", "votes": null, "replies": [] }, { "id": 580435, "author_name": "frank518", "author_url": "", "post_date": "07/20/2019 06:26:04", "content": "dose past comp dataset really help? i mean: is that using both datasets better than using this comp dataset only?
", "votes": null, "replies": [] }, { "id": 580514, "author_name": "rishabhiitbhu", "author_url": "", "post_date": "07/20/2019 09:04:17", "content": "@frank518 Yes. Use past competition dataset for pretraining, then fine tune the model on this competition's dataset. It'll help.
", "votes": null, "replies": [] }, { "id": 580548, "author_name": "frank518", "author_url": "", "post_date": "07/20/2019 10:39:00", "content": "@rishabhiitbhu thank you very much. BTW, i wonder how to using regression in this problem, really appreciate if you can post some links of kernels or articles. @zxyu1995
", "votes": null, "replies": [] }, { "id": 580574, "author_name": "rishabhiitbhu", "author_url": "", "post_date": "07/20/2019 11:45:30", "content": "For regression model, you'll have a single neuron output, as the labels are [0, 1, 2, 3, 4] you'll train your model to output a value around these values, for final predictions you can use a set of thresholds say [0.5, 1.5, 2.5, 3.5]. Check this out
", "votes": null, "replies": [] } ] }, { "id": 579703, "author_name": "yangsaewon", "author_url": "", "post_date": "07/19/2019 06:13:11", "content": "single fold resent50 lb0.744 tta4. cv 0.91~0.92
", "votes": null, "replies": [] }, { "id": 585379, "author_name": "tahsin", "author_url": "", "post_date": "07/27/2019 10:44:15", "content": "Anybody got 0.79+ using single model?
I'm using one model.
", "votes": null, "replies": [] }, { "id": 585645, "author_name": "tahsin", "author_url": "", "post_date": "07/27/2019 19:42:11", "content": "Can you please elaborate? Which model are you using? Are you treating it as a regression or classification task? And what sort of augmentation techniques are you using? Thanks in advance :)
", "votes": null, "replies": [] }, { "id": 585648, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "07/27/2019 19:44:56", "content": "Everything is from public kernels :)
", "votes": null, "replies": [] }, { "id": 585649, "author_name": "tahsin", "author_url": "", "post_date": "07/27/2019 19:48:41", "content": "Thanks. :)
", "votes": null, "replies": [] }, { "id": 585650, "author_name": "tanlikesmath", "author_url": "", "post_date": "07/27/2019 19:49:05", "content": "Could you at least mention what kind of model you are using? I am right now using just a vanilla ResNet50, but it seems to do well, people are using larger architectures like a se-resnet152
", "votes": null, "replies": [] }, { "id": 585720, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "07/27/2019 23:29:31", "content": "Im using efficientnet(like most of the top public kernels). Just take whatever and adjust hyperparameters(can easily give 0.01 boost).
", "votes": null, "replies": [] }, { "id": 586175, "author_name": "tahsin", "author_url": "", "post_date": "07/28/2019 18:16:42", "content": "Can you please elaborate? Which model are you using? Are you treating it as a regression or classification task? And what sort of augmentation techniques are you using? Thanks in advance :)
", "votes": null, "replies": [] }, { "id": 586176, "author_name": "tahsin", "author_url": "", "post_date": "07/28/2019 18:16:44", "content": "Thanks. :)
", "votes": null, "replies": [] } ] }, { "id": 593556, "author_name": "bloodaxe", "author_url": "", "post_date": "08/06/2019 19:39:53", "content": "LB 793 with seresnext50 encoder and some custom-designed head. Flip TTA. Regression.
", "votes": null, "replies": [ { "id": 593589, "author_name": "tanlikesmath", "author_url": "", "post_date": "08/06/2019 20:53:24", "content": "Is TTA helping? because most competitors, including myself, seem to report otherwise
", "votes": null, "replies": [] }, { "id": 593684, "author_name": "garybios", "author_url": "", "post_date": "08/07/2019 01:23:15", "content": "seresnext50 with single fold?
", "votes": null, "replies": [] } ] }, { "id": 593720, "author_name": "lextoumbourou", "author_url": "", "post_date": "08/07/2019 02:49:05", "content": "LB 0.804 with DenseNet201 on a single fold. Ordinal regression + lots of preprocessing + augs + progressive resizing + trial and error.
\n\nFunnily enough, I haven't managed to get ensembling to work at all yet.
", "votes": null, "replies": [ { "id": 593741, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "08/07/2019 03:26:09", "content": "odd, I ensembled 4 models(0.798, 0.798, 0.799, 0.799) and was able to get 0.806
", "votes": null, "replies": [] }, { "id": 593756, "author_name": "tanlikesmath", "author_url": "", "post_date": "08/07/2019 04:17:40", "content": "Nice to hear about progressive resizing worked in this competition. I will try that next :)
", "votes": null, "replies": [] }, { "id": 595157, "author_name": "dreimd", "author_url": "", "post_date": "08/08/2019 22:56:10", "content": "Were those 4 models all efficientnet or were they more varied?
", "votes": null, "replies": [] } ] }, { "id": 593805, "author_name": "monsterspy", "author_url": "", "post_date": "08/07/2019 06:28:34", "content": "LB 0.804 with EfficientNetB5 on a single fold.\ncircle crop + rotation + flips + old data(balanced)
", "votes": null, "replies": [] }, { "id": 594220, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "08/07/2019 18:09:53", "content": "Single fold 0.799 efficientnetb4, regression, trained on 25 epochs on old and new data with flips and rotation augmentation
", "votes": null, "replies": [ { "id": 594414, "author_name": "garybios", "author_url": "", "post_date": "08/08/2019 01:29:05", "content": "trained old and new data together？
", "votes": null, "replies": [] }, { "id": 594901, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "08/08/2019 15:44:28", "content": "yes
", "votes": null, "replies": [] } ] }, { "id": 594399, "author_name": "strideradu", "author_url": "", "post_date": "08/08/2019 00:26:56", "content": "Wondering what is the gap of local CV and LB?
", "votes": null, "replies": [] }, { "id": 594878, "author_name": "drhabib", "author_url": "", "post_date": "08/08/2019 15:16:08", "content": "single model 1 fold LB 0.813 EfficientNet, no TTA.
That's nice score. Do you unfreeze layers while training on any of the datasets ?
", "votes": null, "replies": [] }, { "id": 594894, "author_name": "drhabib", "author_url": "", "post_date": "08/08/2019 15:35:49", "content": "Yes, We have custom unfreezing strategy.
", "votes": null, "replies": [] }, { "id": 594897, "author_name": "harshthaker", "author_url": "", "post_date": "08/08/2019 15:38:48", "content": "Thanks :)
", "votes": null, "replies": [] }, { "id": 594898, "author_name": "philippsinger", "author_url": "", "post_date": "08/08/2019 15:39:28", "content": "Interesting, thanks for reporting. If you don't mind me asking (pls ignore otherwise): This is a single fit? Only on 2019 data or also incorporating 2015 in some way?
", "votes": null, "replies": [] }, { "id": 594936, "author_name": "drhabib", "author_url": "", "post_date": "08/08/2019 16:59:35", "content": "Regarding data and training it pretty much the same (with some tweeks) as I discussed here https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815#581136.
", "votes": null, "replies": [] } ] }, { "id": 596030, "author_name": "ravivadapalli", "author_url": "", "post_date": "08/10/2019 03:47:23", "content": "0.805 with single model. EfficientNet B3 pretrained on Imagenet, ordinal regression, soft TTA, used Ben's preprocessing. Used both 2015 & 2019 data.\nMostly followed the suggestions of DrHB & Risabh.\nEnsemble of 3 such models did not improve or degrade the score much.
", "votes": null, "replies": [ { "id": 597260, "author_name": "virajbagal", "author_url": "", "post_date": "08/12/2019 04:48:11", "content": "Hi Ravi, did you use any augmentations while training?
", "votes": null, "replies": [] }, { "id": 597862, "author_name": "ravivadapalli", "author_url": "", "post_date": "08/12/2019 20:21:07", "content": "Hi Viraj, yes, used random flips, rotations and scale in augmentation.
", "votes": null, "replies": [] } ] }, { "id": 597842, "author_name": "haofanwang", "author_url": "", "post_date": "08/12/2019 19:44:47", "content": "0.799 with single model. \nEfficientNet B4, use pretrained weights on Imagenet. no TTA, resolution 300*300, monitor val loss.\nAug with rotate 360, flip by probability 0.5, Contrast 0.75 to 1.5, zoom in center 1 to 1.3.\nAlmost follow the idea by DrHB.