{ "id": 406038, "title": "what is your LB and CV score comparison", "url": "/competitions/vesuvius-challenge-ink-detection/discussion/406038", "author_name": "", "post_date": "2023-04-30T14:42:57.968524900Z", "votes": 4, "comment_count": 13, "views": 0, "content": "

I have made 3 folds for each fold I have taken one image as validation, and two as train

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
validation imageCV score
image 10.47
image 20.35
image 30.48
\n

these are my best scores after doing thresholding
\nwhat all you guys are getting?

", "messages": [ { "id": "2240444", "postDate": "04/30/2023 14:42:57", "content": "

I have made 3 folds for each fold I have taken one image as validation, and two as train

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
validation imageCV score
image 10.47
image 20.35
image 30.48
\n

these are my best scores after doing thresholding
\nwhat all you guys are getting?

", "rawMarkdown": "I have made 3 folds for each fold I have taken one image as validation, and two as train \n\n| validation image | CV score |\n| --- | --- |\n| image 1 | 0.47 |\n| image 2 | 0.35 |\n| image 3 | 0.48 |\n\nthese are my best scores after doing thresholding \nwhat all you guys are getting?", "votes": null }, { "id": "2240452", "postDate": "04/30/2023 14:51:49", "content": "

Val score for Image-3 is 0.63+ with threshold 0.65, but LB is 0 😂

", "rawMarkdown": "Val score for Image-3 is 0.63+ with threshold 0.65, but LB is 0 😂", "votes": null }, { "id": "2241257", "postDate": "05/01/2023 10:39:06", "content": "

which model are you using, numb of epochs and image size

", "rawMarkdown": "which model are you using, numb of epochs and image size", "votes": null }, { "id": "2241269", "postDate": "05/01/2023 10:51:13", "content": "

efficientnet-b3, ~50 epochs, 256 image size

", "rawMarkdown": "efficientnet-b3, ~50 epochs, 256 image size", "votes": null }, { "id": "2242263", "postDate": "05/02/2023 06:41:19", "content": "

try resnet50, img size 224 and 15 epochs gives good results

", "rawMarkdown": "try resnet50, img size 224 and 15 epochs gives good results", "votes": null }, { "id": "2242341", "postDate": "05/02/2023 07:46:56", "content": "

Thank you! Which threshold do you apply in submission: from the best result of trained model or find out separately for lb?

", "rawMarkdown": "Thank you! Which threshold do you apply in submission: from the best result of trained model or find out separately for lb?", "votes": null }, { "id": "2242474", "postDate": "05/02/2023 10:01:08", "content": "

well from my exp I got good thresh at 0.4 from the train dataset but on lb the good one is 0.34

", "rawMarkdown": "well from my exp I got good thresh at 0.4 from the train dataset but on lb the good one is 0.34", "votes": null }, { "id": "2242788", "postDate": "05/02/2023 14:01:43", "content": "

Lucario129, your results above are from CV, not LB, right?

", "rawMarkdown": "Lucario129, your results above are from CV, not LB, right?", "votes": null }, { "id": "2242915", "postDate": "05/02/2023 15:16:25", "content": "

they are from CV I have made 5-6 models 3 folds each and used ensemble to get my LB score

", "rawMarkdown": "they are from CV I have made 5-6 models 3 folds each and used ensemble to get my LB score", "votes": null }, { "id": "2244183", "postDate": "05/03/2023 13:50:28", "content": "

Hello @lucario129, which loss do you use for training? If you are using BCE loss, what are the loss values corresponding to these fbeta CV scores?

", "rawMarkdown": "Hello @lucario129, which loss do you use for training? If you are using BCE loss, what are the loss values corresponding to these fbeta CV scores?", "votes": null }, { "id": "2253741", "postDate": "05/10/2023 12:27:14", "content": "

validation image id:1, CV:0.58, LB:0.61, I think the most important work is the data side in feature.

", "rawMarkdown": "validation image id:1, CV:0.58, LB:0.61, I think the most important work is the data side in feature.", "votes": null }, { "id": "2254548", "postDate": "05/11/2023 04:32:39", "content": "

Thanks for sharing your CV and LB result. I have two questions. Is your CV and LB correlate well? Second, what did you do on the \"data side\" to improve performance?

", "rawMarkdown": "Thanks for sharing your CV and LB result. I have two questions. Is your CV and LB correlate well? Second, what did you do on the \"data side\" to improve performance?", "votes": null }, { "id": "2254584", "postDate": "05/11/2023 05:18:43", "content": "

I only trained one model which I submited, my LB Increased is because I added TTA and I found the best threshold,
\nso I could not answer your first question. Second, I found a preprocessing method that achieved CV:0.57 with 20 minutes of training, while my previous model required 5 hours of training, in addition, I have trying to remove the noise form the original data although it seems fail…

", "rawMarkdown": "I only trained one model which I submited, my LB Increased is because I added TTA and I found the best threshold, \nso I could not answer your first question. Second, I found a preprocessing method that achieved CV:0.57 with 20 minutes of training, while my previous model required 5 hours of training, in addition, I have trying to remove the noise form the original data although it seems fail...", "votes": null }, { "id": "2254620", "postDate": "05/11/2023 05:44:57", "content": "

Thanks for answering. The preprocessing method that achieved CV: 0.57 is on preprocessing the data right? Is it some sort of data augmentation? And after the preprocessing you trained for 20 minutes with Unet and achieved CV: 0.57?

", "rawMarkdown": "Thanks for answering. The preprocessing method that achieved CV: 0.57 is on preprocessing the data right? Is it some sort of data augmentation? And after the preprocessing you trained for 20 minutes with Unet and achieved CV: 0.57?", "votes": null } ], "comments": [ { "id": 2240452, "author_name": "mknzfr", "author_url": "", "post_date": "04/30/2023 14:51:49", "content": "

Val score for Image-3 is 0.63+ with threshold 0.65, but LB is 0 😂

", "votes": null, "replies": [ { "id": 2241257, "author_name": "lucario129", "author_url": "", "post_date": "05/01/2023 10:39:06", "content": "

which model are you using, numb of epochs and image size

", "votes": null, "replies": [ { "id": 2241269, "author_name": "mknzfr", "author_url": "", "post_date": "05/01/2023 10:51:13", "content": "

efficientnet-b3, ~50 epochs, 256 image size

", "votes": null, "replies": [ { "id": 2242263, "author_name": "lucario129", "author_url": "", "post_date": "05/02/2023 06:41:19", "content": "

try resnet50, img size 224 and 15 epochs gives good results

", "votes": null, "replies": [ { "id": 2242341, "author_name": "mknzfr", "author_url": "", "post_date": "05/02/2023 07:46:56", "content": "

Thank you! Which threshold do you apply in submission: from the best result of trained model or find out separately for lb?

", "votes": null, "replies": [ { "id": 2242474, "author_name": "lucario129", "author_url": "", "post_date": "05/02/2023 10:01:08", "content": "

well from my exp I got good thresh at 0.4 from the train dataset but on lb the good one is 0.34

", "votes": null, "replies": [] } ] } ] } ] } ] } ] }, { "id": 2242788, "author_name": "mknzfr", "author_url": "", "post_date": "05/02/2023 14:01:43", "content": "

Lucario129, your results above are from CV, not LB, right?

", "votes": null, "replies": [ { "id": 2242915, "author_name": "lucario129", "author_url": "", "post_date": "05/02/2023 15:16:25", "content": "

they are from CV I have made 5-6 models 3 folds each and used ensemble to get my LB score

", "votes": null, "replies": [] } ] }, { "id": 2244183, "author_name": "turkenm", "author_url": "", "post_date": "05/03/2023 13:50:28", "content": "

Hello @lucario129, which loss do you use for training? If you are using BCE loss, what are the loss values corresponding to these fbeta CV scores?

", "votes": null, "replies": [] }, { "id": 2253741, "author_name": "yoyobar", "author_url": "", "post_date": "05/10/2023 12:27:14", "content": "

validation image id:1, CV:0.58, LB:0.61, I think the most important work is the data side in feature.

", "votes": null, "replies": [ { "id": 2254548, "author_name": "lionfishy", "author_url": "", "post_date": "05/11/2023 04:32:39", "content": "

Thanks for sharing your CV and LB result. I have two questions. Is your CV and LB correlate well? Second, what did you do on the \"data side\" to improve performance?

", "votes": null, "replies": [ { "id": 2254584, "author_name": "yoyobar", "author_url": "", "post_date": "05/11/2023 05:18:43", "content": "

I only trained one model which I submited, my LB Increased is because I added TTA and I found the best threshold,
\nso I could not answer your first question. Second, I found a preprocessing method that achieved CV:0.57 with 20 minutes of training, while my previous model required 5 hours of training, in addition, I have trying to remove the noise form the original data although it seems fail…

", "votes": null, "replies": [ { "id": 2254620, "author_name": "lionfishy", "author_url": "", "post_date": "05/11/2023 05:44:57", "content": "

Thanks for answering. The preprocessing method that achieved CV: 0.57 is on preprocessing the data right? Is it some sort of data augmentation? And after the preprocessing you trained for 20 minutes with Unet and achieved CV: 0.57?

", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "2240444": "I have made 3 folds for each fold I have taken one image as validation, and two as train \n\n| validation image | CV score |\n| --- | --- |\n| image 1 | 0.47 |\n| image 2 | 0.35 |\n| image 3 | 0.48 |\n\nthese are my best scores after doing thresholding \nwhat all you guys are getting?", "2240452": "Val score for Image-3 is 0.63+ with threshold 0.65, but LB is 0 😂", "2241257": "which model are you using, numb of epochs and image size", "2241269": "efficientnet-b3, ~50 epochs, 256 image size", "2242263": "try resnet50, img size 224 and 15 epochs gives good results", "2242341": "Thank you! Which threshold do you apply in submission: from the best result of trained model or find out separately for lb?", "2242474": "well from my exp I got good thresh at 0.4 from the train dataset but on lb the good one is 0.34", "2242788": "Lucario129, your results above are from CV, not LB, right?", "2242915": "they are from CV I have made 5-6 models 3 folds each and used ensemble to get my LB score", "2244183": "Hello @lucario129, which loss do you use for training? If you are using BCE loss, what are the loss values corresponding to these fbeta CV scores?", "2253741": "validation image id:1, CV:0.58, LB:0.61, I think the most important work is the data side in feature.", "2254548": "Thanks for sharing your CV and LB result. I have two questions. Is your CV and LB correlate well? Second, what did you do on the \"data side\" to improve performance?", "2254584": "I only trained one model which I submited, my LB Increased is because I added TTA and I found the best threshold, \nso I could not answer your first question. Second, I found a preprocessing method that achieved CV:0.57 with 20 minutes of training, while my previous model required 5 hours of training, in addition, I have trying to remove the noise form the original data although it seems fail...", "2254620": "Thanks for answering. The preprocessing method that achieved CV: 0.57 is on preprocessing the data right? Is it some sort of data augmentation? And after the preprocessing you trained for 20 minutes with Unet and achieved CV: 0.57?" }, "source": "meta" }