{ "id": 251889, "title": "Has Anyone Seen improvement from Alien's public notebook in the 2 Class Classifier by using AUX loss with BCE?", "url": "/competitions/siim-covid19-detection/discussion/251889", "author_name": "", "post_date": "2021-07-09T08:57:12.464257Z", "votes": 8, "comment_count": 17, "views": 0, "content": "

I am wondering if anyone has tried effv2 and aux loss for 2 class classification and seen a significant improvement. Somehow my models are performing worse than the public models shared by alien.

", "messages": [ { "id": "1381826", "postDate": "07/09/2021 08:57:12", "content": "

I am wondering if anyone has tried effv2 and aux loss for 2 class classification and seen a significant improvement. Somehow my models are performing worse than the public models shared by alien.

", "rawMarkdown": "I am wondering if anyone has tried effv2 and aux loss for 2 class classification and seen a significant improvement. Somehow my models are performing worse than the public models shared by alien.", "votes": null }, { "id": "1382406", "postDate": "07/09/2021 19:52:44", "content": "

I did not use the losses you mentioned, but I retrained the models of two classes - I was waiting for an improvement, but I did not get it either. I am linking this result with the fact that in alien's notebook the 2 class models are only used to set the value to none. - Of course, I may not understand something, therefore, if I'm wrong, I ask you to tell me what.

", "rawMarkdown": "I did not use the losses you mentioned, but I retrained the models of two classes - I was waiting for an improvement, but I did not get it either. I am linking this result with the fact that in alien's notebook the 2 class models are only used to set the value to none. - Of course, I may not understand something, therefore, if I'm wrong, I ask you to tell me what.", "votes": null }, { "id": "1382443", "postDate": "07/09/2021 21:27:24", "content": "

.007 improvement over public with using v2+aux…

", "rawMarkdown": ".007 improvement over public with using v2+aux...", "votes": null }, { "id": "1382444", "postDate": "07/09/2021 21:28:25", "content": "

Same here, couldn't get improvements with different technics that worked well for study level classification.

", "rawMarkdown": "Same here, couldn't get improvements with different technics that worked well for study level classification.", "votes": null }, { "id": "1382452", "postDate": "07/09/2021 21:39:00", "content": "

Did you need to make major changes compared 4 class classifier training?

", "rawMarkdown": "Did you need to make major changes compared 4 class classifier training?", "votes": null }, { "id": "1382467", "postDate": "07/09/2021 22:25:23", "content": "

I made no change…

", "rawMarkdown": "I made no change...", "votes": null }, { "id": "1382502", "postDate": "07/10/2021 00:05:18", "content": "

How did you use the v2+aux?
\nDid you change something or added anything to the code?
\nThx for answering.

", "rawMarkdown": "How did you use the v2+aux? \nDid you change something or added anything to the code?\nThx for answering.", "votes": null }, { "id": "1382516", "postDate": "07/10/2021 01:01:52", "content": "

I used a custom aux which I would not like to expose, but it is not same as heng's

", "rawMarkdown": "I used a custom aux which I would not like to expose, but it is not same as heng's", "votes": null }, { "id": "1382980", "postDate": "07/10/2021 11:44:11", "content": "

I got ~0.01 CV and 0.001 LB boost from very similar but not exact approach.

", "rawMarkdown": "I got ~0.01 CV and 0.001 LB boost from very similar but not exact approach.", "votes": null }, { "id": "1383005", "postDate": "07/10/2021 12:12:01", "content": "

@steamedsheep even i got a ~0.01 improvement in my CV but it lowered my lb by ~0.001. It is weird.

", "rawMarkdown": "steamedsheep even i got a ~0.01 improvement in my CV but it lowered my lb by ~0.001. It is weird.", "votes": null }, { "id": "1383037", "postDate": "07/10/2021 12:51:43", "content": "

Since public test set is very small, I think LB is unstable.

", "rawMarkdown": "Since public test set is very small, I think LB is unstable.", "votes": null }, { "id": "1383103", "postDate": "07/10/2021 14:18:11", "content": "

as @steamedsheep mentioned LB is way to small to be reliable so i think we should trust our CV scores

", "rawMarkdown": "as @steamedsheep mentioned LB is way to small to be reliable so i think we should trust our CV scores", "votes": null }, { "id": "1383335", "postDate": "07/10/2021 18:24:18", "content": "

i got an 5 fold accuracy cv of 0.86 ,roc_auc of 0.91 and ap of 0.96 in my torch model

", "rawMarkdown": "i got an 5 fold accuracy cv of 0.86 ,roc_auc of 0.91 and ap of 0.96 in my torch model", "votes": null }, { "id": "1384277", "postDate": "07/11/2021 16:18:20", "content": "

Dear geeks, just a not very related question but I'm planning to re-train Alien's efficient notebook with image size of 1024 (the original is 512 for test and 600 for study). So do you think that this will improve the score? Any recommendation is appreciated, thanks!

", "rawMarkdown": "Dear geeks, just a not very related question but I'm planning to re-train Alien's efficient notebook with image size of 1024 (the original is 512 for test and 600 for study). So do you think that this will improve the score? Any recommendation is appreciated, thanks!", "votes": null }, { "id": "1384321", "postDate": "07/11/2021 17:34:59", "content": "

Also, why do we need to resize to 600 for study and 512 for test image? why not 512 for both? thanks

", "rawMarkdown": "Also, why do we need to resize to 600 for study and 512 for test image? why not 512 for both? thanks", "votes": null }, { "id": "1385543", "postDate": "07/12/2021 19:07:01", "content": "

what is CV score?

", "rawMarkdown": "what is CV score?", "votes": null }, { "id": "1385766", "postDate": "07/13/2021 03:02:12", "content": "

@zaakciiru It's Cross Validation score.

", "rawMarkdown": "zaakciiru It's Cross Validation score.", "votes": null }, { "id": "1387656", "postDate": "07/14/2021 10:50:48", "content": "

I got a ~0.01 improvement in the two class classifier in my CV but it lowered my lb about 0.001 too.and i think you are right, the test dataset is too small

", "rawMarkdown": "I got a ~0.01 improvement in the two class classifier in my CV but it lowered my lb about 0.001 too.and i think you are right, the test dataset is too small", "votes": null } ], "comments": [ { "id": 1382406, "author_name": "zaakciiru", "author_url": "", "post_date": "07/09/2021 19:52:44", "content": "

I did not use the losses you mentioned, but I retrained the models of two classes - I was waiting for an improvement, but I did not get it either. I am linking this result with the fact that in alien's notebook the 2 class models are only used to set the value to none. - Of course, I may not understand something, therefore, if I'm wrong, I ask you to tell me what.

", "votes": null, "replies": [ { "id": 1383103, "author_name": "varundutt9213", "author_url": "", "post_date": "07/10/2021 14:18:11", "content": "

as @steamedsheep mentioned LB is way to small to be reliable so i think we should trust our CV scores

", "votes": null, "replies": [] }, { "id": 1385543, "author_name": "zaakciiru", "author_url": "", "post_date": "07/12/2021 19:07:01", "content": "

what is CV score?

", "votes": null, "replies": [] }, { "id": 1385766, "author_name": "pvtien96", "author_url": "", "post_date": "07/13/2021 03:02:12", "content": "

@zaakciiru It's Cross Validation score.

", "votes": null, "replies": [] } ] }, { "id": 1382443, "author_name": "harshitsheoran", "author_url": "", "post_date": "07/09/2021 21:27:24", "content": "

.007 improvement over public with using v2+aux…

", "votes": null, "replies": [ { "id": 1382452, "author_name": "varundutt9213", "author_url": "", "post_date": "07/09/2021 21:39:00", "content": "

Did you need to make major changes compared 4 class classifier training?

", "votes": null, "replies": [] }, { "id": 1382467, "author_name": "harshitsheoran", "author_url": "", "post_date": "07/09/2021 22:25:23", "content": "

I made no change…

", "votes": null, "replies": [] }, { "id": 1382502, "author_name": "lautman", "author_url": "", "post_date": "07/10/2021 00:05:18", "content": "

How did you use the v2+aux?
\nDid you change something or added anything to the code?
\nThx for answering.

", "votes": null, "replies": [] }, { "id": 1382516, "author_name": "harshitsheoran", "author_url": "", "post_date": "07/10/2021 01:01:52", "content": "

I used a custom aux which I would not like to expose, but it is not same as heng's

", "votes": null, "replies": [] } ] }, { "id": 1382444, "author_name": "josephamigo", "author_url": "", "post_date": "07/09/2021 21:28:25", "content": "

Same here, couldn't get improvements with different technics that worked well for study level classification.

", "votes": null, "replies": [] }, { "id": 1382980, "author_name": "steamedsheep", "author_url": "", "post_date": "07/10/2021 11:44:11", "content": "

I got ~0.01 CV and 0.001 LB boost from very similar but not exact approach.

", "votes": null, "replies": [ { "id": 1383005, "author_name": "varundutt9213", "author_url": "", "post_date": "07/10/2021 12:12:01", "content": "

@steamedsheep even i got a ~0.01 improvement in my CV but it lowered my lb by ~0.001. It is weird.

", "votes": null, "replies": [] }, { "id": 1383037, "author_name": "steamedsheep", "author_url": "", "post_date": "07/10/2021 12:51:43", "content": "

Since public test set is very small, I think LB is unstable.

", "votes": null, "replies": [] } ] }, { "id": 1383335, "author_name": "trooperog", "author_url": "", "post_date": "07/10/2021 18:24:18", "content": "

i got an 5 fold accuracy cv of 0.86 ,roc_auc of 0.91 and ap of 0.96 in my torch model

", "votes": null, "replies": [] }, { "id": 1384277, "author_name": "pvtien96", "author_url": "", "post_date": "07/11/2021 16:18:20", "content": "

Dear geeks, just a not very related question but I'm planning to re-train Alien's efficient notebook with image size of 1024 (the original is 512 for test and 600 for study). So do you think that this will improve the score? Any recommendation is appreciated, thanks!

", "votes": null, "replies": [ { "id": 1384321, "author_name": "pvtien96", "author_url": "", "post_date": "07/11/2021 17:34:59", "content": "

Also, why do we need to resize to 600 for study and 512 for test image? why not 512 for both? thanks

", "votes": null, "replies": [] } ] }, { "id": 1387656, "author_name": "wyyyywwy", "author_url": "", "post_date": "07/14/2021 10:50:48", "content": "

I got a ~0.01 improvement in the two class classifier in my CV but it lowered my lb about 0.001 too.and i think you are right, the test dataset is too small

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1381826": "I am wondering if anyone has tried effv2 and aux loss for 2 class classification and seen a significant improvement. Somehow my models are performing worse than the public models shared by alien.", "1382406": "I did not use the losses you mentioned, but I retrained the models of two classes - I was waiting for an improvement, but I did not get it either. I am linking this result with the fact that in alien's notebook the 2 class models are only used to set the value to none. - Of course, I may not understand something, therefore, if I'm wrong, I ask you to tell me what.", "1382443": ".007 improvement over public with using v2+aux...", "1382444": "Same here, couldn't get improvements with different technics that worked well for study level classification.", "1382452": "Did you need to make major changes compared 4 class classifier training?", "1382467": "I made no change...", "1382502": "How did you use the v2+aux? \nDid you change something or added anything to the code?\nThx for answering.", "1382516": "I used a custom aux which I would not like to expose, but it is not same as heng's", "1382980": "I got ~0.01 CV and 0.001 LB boost from very similar but not exact approach.", "1383005": "steamedsheep even i got a ~0.01 improvement in my CV but it lowered my lb by ~0.001. It is weird.", "1383037": "Since public test set is very small, I think LB is unstable.", "1383103": "as @steamedsheep mentioned LB is way to small to be reliable so i think we should trust our CV scores", "1383335": "i got an 5 fold accuracy cv of 0.86 ,roc_auc of 0.91 and ap of 0.96 in my torch model", "1384277": "Dear geeks, just a not very related question but I'm planning to re-train Alien's efficient notebook with image size of 1024 (the original is 512 for test and 600 for study). So do you think that this will improve the score? Any recommendation is appreciated, thanks!", "1384321": "Also, why do we need to resize to 600 for study and 512 for test image? why not 512 for both? thanks", "1385543": "what is CV score?", "1385766": "zaakciiru It's Cross Validation score.", "1387656": "I got a ~0.01 improvement in the two class classifier in my CV but it lowered my lb about 0.001 too.and i think you are right, the test dataset is too small" }, "source": "meta" }