I tried to pre-train my model on past-competition data,what i noticed is that it is even more unbalanced,so i selected all the training data from of diagnosis level 3 and diagnosis level 4 and 1500 examples from category 0,1,2.While training with the same architectures i am getting a quadratic kappa of 0.68 wheareas with the same configuration i am getting 0.90 with the present competition data,What i am doing wrong?
", "messages": [ { "id": "585117", "postDate": "07/27/2019 01:45:22", "content": "I tried to pre-train my model on past-competition data,what i noticed is that it is even more unbalanced,so i selected all the training data from of diagnosis level 3 and diagnosis level 4 and 1500 examples from category 0,1,2.While training with the same architectures i am getting a quadratic kappa of 0.68 wheareas with the same configuration i am getting 0.90 with the present competition data,What i am doing wrong?
", "rawMarkdown": "I tried to pre-train my model on past-competition data,what i noticed is that it is even more unbalanced,so i selected all the training data from of diagnosis level 3 and diagnosis level 4 and 1500 examples from category 0,1,2.While training with the same architectures i am getting a quadratic kappa of 0.68 wheareas with the same configuration i am getting 0.90 with the present competition data,What i am doing wrong?", "votes": null }, { "id": "588166", "postDate": "07/30/2019 08:36:47", "content": "I use past competition data only in the first phase of the training.\nIn the second phase I use the new data only.\nThat way it greatly improves results.
\n\nWith the old data only I never achive good results either
", "rawMarkdown": "I use past competition data only in the first phase of the training.\nIn the second phase I use the new data only.\nThat way it greatly improves results.\n\nWith the old data only I never achive good results either", "votes": null }, { "id": "588328", "postDate": "07/30/2019 12:27:52", "content": "Are you using the entire train data or a part of it?training the entire data would take a lot of time I guess.
", "rawMarkdown": "Are you using the entire train data or a part of it?training the entire data would take a lot of time I guess.", "votes": null }, { "id": "589556", "postDate": "08/01/2019 04:55:10", "content": "I used 1500 images from each class from old competition data. Got only 0.61 QWK. Not yet tried that model on new competition data.
", "rawMarkdown": "I used 1500 images from each class from old competition data. Got only 0.61 QWK. Not yet tried that model on new competition data.", "votes": null }, { "id": "589605", "postDate": "08/01/2019 06:47:42", "content": "The entire data! 1 hour per epoch ;-)
\n\nI train a few epochs on the old data and then replace train dataset with the new data.\nThis improves score!
", "rawMarkdown": "The entire data! 1 hour per epoch ;-)\n\nI train a few epochs on the old data and then replace train dataset with the new data.\nThis improves score!", "votes": null }, { "id": "589665", "postDate": "08/01/2019 08:11:20", "content": "I'm having trouble improving my model too - on only the new data, I can achieve my highest LB score but when I train on old and then new my model does slightly worse. I did 15 epochs on old data and around 10 on new data, and didn't freeze any of the layers for either. Is there anything I should do differently?
", "rawMarkdown": "I'm having trouble improving my model too - on only the new data, I can achieve my highest LB score but when I train on old and then new my model does slightly worse. I did 15 epochs on old data and around 10 on new data, and didn't freeze any of the layers for either. Is there anything I should do differently?", "votes": null }, { "id": "589763", "postDate": "08/01/2019 10:42:59", "content": "I trained 5 epochs first and I froze all but the top layer.\nThen 15 eopchs full training on the old data.\nThen some more on the new data.\nProbably I'll train more epochs before the end, but it i takes too much time...
\n\nWhat model did you use to achive your current score? \nI never managed to achive that high score without the old trainingset...
", "rawMarkdown": "I trained 5 epochs first and I froze all but the top layer.\nThen 15 eopchs full training on the old data.\nThen some more on the new data.\nProbably I'll train more epochs before the end, but it i takes too much time...\n\nWhat model did you use to achive your current score? \nI never managed to achive that high score without the old trainingset...", "votes": null }, { "id": "590062", "postDate": "08/01/2019 18:51:10", "content": "I used B4 for regression. What transforms/augments did you use when pretraining on the old data? Did you resize the images?
", "rawMarkdown": "I used B4 for regression. What transforms/augments did you use when pretraining on the old data? Did you resize the images?", "votes": null }, { "id": "590332", "postDate": "08/02/2019 05:14:18", "content": "I started with this one:https://imgaug.readthedocs.io/en/latest/source/examples_basics.html#heavy-augmentations\nVery strong augmentation, but it helps a lot. Later I softened it a bit.
\n\nDo you manage to train all B4 layers?\nI'm stuggling with B5 now. I can train the top layers, but it stops improving when I train the full model. Loss doesn't improve.
", "rawMarkdown": "I started with this one:https://imgaug.readthedocs.io/en/latest/source/examples_basics.html#heavy-augmentations\nVery strong augmentation, but it helps a lot. Later I softened it a bit.\n\nDo you manage to train all B4 layers?\nI'm stuggling with B5 now. I can train the top layers, but it stops improving when I train the full model. Loss doesn't improve.", "votes": null }, { "id": "590479", "postDate": "08/02/2019 08:24:31", "content": "I do train all B4 layers. Are those heavy augments really better than Ben's preprocessing? Have you experimented with both?
", "rawMarkdown": "I do train all B4 layers. Are those heavy augments really better than Ben's preprocessing? Have you experimented with both?", "votes": null }, { "id": "590484", "postDate": "08/02/2019 08:28:52", "content": "Guys, can you please tell me what are B4 and B5 models ? Thank you.
", "rawMarkdown": "Guys, can you please tell me what are B4 and B5 models ? Thank you.", "votes": null }, { "id": "590509", "postDate": "08/02/2019 09:10:08", "content": "I found heavy augmentation to be better than Ben's preprocessing.\nBut I added his preprocessing as a custom augmentation as well for a small fraction of the images :-)
", "rawMarkdown": "I found heavy augmentation to be better than Ben's preprocessing.\nBut I added his preprocessing as a custom augmentation as well for a small fraction of the images :-)", "votes": null }, { "id": "590730", "postDate": "08/02/2019 14:19:17", "content": "@virajbagal Efficientnet-B4 and B5.
", "rawMarkdown": "virajbagal Efficientnet-B4 and B5.", "votes": null } ], "comments": [ { "id": 588166, "author_name": "nemethpeti", "author_url": "", "post_date": "07/30/2019 08:36:47", "content": "I use past competition data only in the first phase of the training.\nIn the second phase I use the new data only.\nThat way it greatly improves results.
\n\nWith the old data only I never achive good results either
", "votes": null, "replies": [ { "id": 588328, "author_name": "amardeepganguly", "author_url": "", "post_date": "07/30/2019 12:27:52", "content": "Are you using the entire train data or a part of it?training the entire data would take a lot of time I guess.
", "votes": null, "replies": [] }, { "id": 589605, "author_name": "nemethpeti", "author_url": "", "post_date": "08/01/2019 06:47:42", "content": "The entire data! 1 hour per epoch ;-)
\n\nI train a few epochs on the old data and then replace train dataset with the new data.\nThis improves score!
", "votes": null, "replies": [] }, { "id": 589665, "author_name": "dreimd", "author_url": "", "post_date": "08/01/2019 08:11:20", "content": "I'm having trouble improving my model too - on only the new data, I can achieve my highest LB score but when I train on old and then new my model does slightly worse. I did 15 epochs on old data and around 10 on new data, and didn't freeze any of the layers for either. Is there anything I should do differently?
", "votes": null, "replies": [] }, { "id": 589763, "author_name": "nemethpeti", "author_url": "", "post_date": "08/01/2019 10:42:59", "content": "I trained 5 epochs first and I froze all but the top layer.\nThen 15 eopchs full training on the old data.\nThen some more on the new data.\nProbably I'll train more epochs before the end, but it i takes too much time...
\n\nWhat model did you use to achive your current score? \nI never managed to achive that high score without the old trainingset...
", "votes": null, "replies": [] }, { "id": 590062, "author_name": "dreimd", "author_url": "", "post_date": "08/01/2019 18:51:10", "content": "I used B4 for regression. What transforms/augments did you use when pretraining on the old data? Did you resize the images?
", "votes": null, "replies": [] }, { "id": 590332, "author_name": "nemethpeti", "author_url": "", "post_date": "08/02/2019 05:14:18", "content": "I started with this one:https://imgaug.readthedocs.io/en/latest/source/examples_basics.html#heavy-augmentations\nVery strong augmentation, but it helps a lot. Later I softened it a bit.
\n\nDo you manage to train all B4 layers?\nI'm stuggling with B5 now. I can train the top layers, but it stops improving when I train the full model. Loss doesn't improve.
", "votes": null, "replies": [] }, { "id": 590479, "author_name": "dreimd", "author_url": "", "post_date": "08/02/2019 08:24:31", "content": "I do train all B4 layers. Are those heavy augments really better than Ben's preprocessing? Have you experimented with both?
", "votes": null, "replies": [] }, { "id": 590484, "author_name": "virajbagal", "author_url": "", "post_date": "08/02/2019 08:28:52", "content": "Guys, can you please tell me what are B4 and B5 models ? Thank you.
", "votes": null, "replies": [] }, { "id": 590509, "author_name": "nemethpeti", "author_url": "", "post_date": "08/02/2019 09:10:08", "content": "I found heavy augmentation to be better than Ben's preprocessing.\nBut I added his preprocessing as a custom augmentation as well for a small fraction of the images :-)
", "votes": null, "replies": [] }, { "id": 590730, "author_name": "harshthaker", "author_url": "", "post_date": "08/02/2019 14:19:17", "content": "@virajbagal Efficientnet-B4 and B5.
", "votes": null, "replies": [] } ] }, { "id": 589556, "author_name": "virajbagal", "author_url": "", "post_date": "08/01/2019 04:55:10", "content": "I used 1500 images from each class from old competition data. Got only 0.61 QWK. Not yet tried that model on new competition data.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "585117": "I tried to pre-train my model on past-competition data,what i noticed is that it is even more unbalanced,so i selected all the training data from of diagnosis level 3 and diagnosis level 4 and 1500 examples from category 0,1,2.While training with the same architectures i am getting a quadratic kappa of 0.68 wheareas with the same configuration i am getting 0.90 with the present competition data,What i am doing wrong?", "588166": "I use past competition data only in the first phase of the training.\nIn the second phase I use the new data only.\nThat way it greatly improves results.\n\nWith the old data only I never achive good results either", "588328": "Are you using the entire train data or a part of it?training the entire data would take a lot of time I guess.", "589556": "I used 1500 images from each class from old competition data. Got only 0.61 QWK. Not yet tried that model on new competition data.", "589605": "The entire data! 1 hour per epoch ;-)\n\nI train a few epochs on the old data and then replace train dataset with the new data.\nThis improves score!", "589665": "I'm having trouble improving my model too - on only the new data, I can achieve my highest LB score but when I train on old and then new my model does slightly worse. I did 15 epochs on old data and around 10 on new data, and didn't freeze any of the layers for either. Is there anything I should do differently?", "589763": "I trained 5 epochs first and I froze all but the top layer.\nThen 15 eopchs full training on the old data.\nThen some more on the new data.\nProbably I'll train more epochs before the end, but it i takes too much time...\n\nWhat model did you use to achive your current score? \nI never managed to achive that high score without the old trainingset...", "590062": "I used B4 for regression. What transforms/augments did you use when pretraining on the old data? Did you resize the images?", "590332": "I started with this one:https://imgaug.readthedocs.io/en/latest/source/examples_basics.html#heavy-augmentations\nVery strong augmentation, but it helps a lot. Later I softened it a bit.\n\nDo you manage to train all B4 layers?\nI'm stuggling with B5 now. I can train the top layers, but it stops improving when I train the full model. Loss doesn't improve.", "590479": "I do train all B4 layers. Are those heavy augments really better than Ben's preprocessing? Have you experimented with both?", "590484": "Guys, can you please tell me what are B4 and B5 models ? Thank you.", "590509": "I found heavy augmentation to be better than Ben's preprocessing.\nBut I added his preprocessing as a custom augmentation as well for a small fraction of the images :-)", "590730": "virajbagal Efficientnet-B4 and B5." }, "source": "meta" }