![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3548771%2Fc99ae303c251de297332b7808ee8a2fa%2FF1_curve.png?generation=1747773612058578&alt=media\")
Hi guys, i have used two of the public notebooks to train a yolo model, whatever i tried, i have good scores in test sets that i split from the train folder but my LB scores are very very low, like 0.14 - 0.45. I have tried several train/val splits but no luck. How the public notebook that achieves 0.77 with a single model was trained? what can be wrong in my trains?
", "messages": [ { "id": "3205992", "postDate": "05/20/2025 17:40:50", "content": "Hi guys, i have used two of the public notebooks to train a yolo model, whatever i tried, i have good scores in test sets that i split from the train folder but my LB scores are very very low, like 0.14 - 0.45. I have tried several train/val splits but no luck. How the public notebook that achieves 0.77 with a single model was trained? what can be wrong in my trains?
", "rawMarkdown": "Hi guys, i have used two of the public notebooks to train a yolo model, whatever i tried, i have good scores in test sets that i split from the train folder but my LB scores are very very low, like 0.14 - 0.45. I have tried several train/val splits but no luck. How the public notebook that achieves 0.77 with a single model was trained? what can be wrong in my trains?", "votes": null }, { "id": "3205998", "postDate": "05/20/2025 17:50:58", "content": "0.14-0.45 just insane, perhaps you can boost your score by just changing the params in inference! One can get to 0.8+ with single model trained without any external data.
", "rawMarkdown": "0.14-0.45 just insane, perhaps you can boost your score by just changing the params in inference! One can get to 0.8+ with single model trained without any external data.", "votes": null }, { "id": "3206014", "postDate": "05/20/2025 18:17:59", "content": "Start with one of those starter notebooks and expand from there. Feel free to share your solution and I am happy to help but here are a few considerations for why your solution may score so low:
\nOverall if you cannot figure out why your solution is way worse then take one of the notebooks that achieves a good score and work backwards, figure out what you did different and why your score is lower. Most of the time it is something silly like mixing up coordinates or creating incorrect labels (ie saying the target is at X=30 Y=40 instead of X=40 Y=30). Again I am happy to help further if you have more detail! :D
", "rawMarkdown": "Start with one of those starter notebooks and expand from there. Feel free to share your solution and I am happy to help but here are a few considerations for why your solution may score so low:\n- Mixing up axis on inference such as writing X as Y and Y as X\n- If you are doing a 2D solution on each slice then ensure that you are extracting the correct Z axis for your predictions (ie if you read the slices out of order and don't label which slice it is then you could end up with the wrong Z target)\n- Check if your validation metrics are similar to your LB submission to know if the issue is with a bad model or a bug in your inference notebook. For example, if you achieve a very good validation score during your cross validation experiments and then the LB score is way off then you may have a data leak or some other issue.\n- Make sure your preprocessing for your train and submission are identical (ie use the same normalization).\n\nOverall if you cannot figure out why your solution is way worse then take one of the notebooks that achieves a good score and work backwards, figure out what you did different and why your score is lower. Most of the time it is something silly like mixing up coordinates or creating incorrect labels (ie saying the target is at X=30 Y=40 instead of X=40 Y=30). Again I am happy to help further if you have more detail! :D", "votes": null }, { "id": "3206080", "postDate": "05/20/2025 19:34:38", "content": "thanks, i thought about the above but they are not it ! i use also a public inference notebook and i just substitute my model, i tried with 2 of them
", "rawMarkdown": "thanks, i thought about the above but they are not it ! i use also a public inference notebook and i just substitute my model, i tried with 2 of them", "votes": null }, { "id": "3206082", "postDate": "05/20/2025 19:35:10", "content": "yes i have heard of that so thats why i dont get why my trains go so wrong ;/
", "rawMarkdown": "yes i have heard of that so thats why i dont get why my trains go so wrong ;/", "votes": null }, { "id": "3206109", "postDate": "05/20/2025 20:40:16", "content": "Are you using YOLO? Make sure that if you plugged your model in that you are using the same preprocessing steps as they do. Also consider that they may use a different confidence threshold. Since we are using a Beta value of 2 for this competition I recommend using a lower confidence threshold on your predictions to get a higher recall score. It is possible your model makes less confident predictions and requires a lower conf threshold than what is used in the notebook you plugged into. Can you share your training and validation F1 curves? I can compare to mine but a baseline train for me got 0.85 F1 as shown below.
\n
Here are my train and validation loss curves as well throughout training if you want to compare yours. This is not for my best model but for a baseline that is scoring pretty well:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3548771%2F8a50349fdefa6d32fb2e3ad94c8b9753%2Fresults.png?generation=1747773662053780&alt=media\")
What's your local CV score? Maybe you have too much noisy data, I think trained on only one motor images would help in this stage.
", "rawMarkdown": "What's your local CV score? Maybe you have too much noisy data, I think trained on only one motor images would help in this stage.", "votes": null }, { "id": "3206691", "postDate": "05/21/2025 16:44:42", "content": "@connorjd i get lots of these train: WARNING ⚠️ C:\\Users\\chara\\PycharmProjects\\Kaggle_BYU_flagellar\\kaggle\\working\\yolo_dataset\\images\\train\\tomo_003acc_z0000_y-001_x-001.jpg: ignoring corrupt image/label: negative label values [-0.00054142 -0.00052301]. Can they be the problem? do you also have these?
", "rawMarkdown": "connorjd i get lots of these train: WARNING ⚠️ C:\\Users\\chara\\PycharmProjects\\Kaggle_BYU_flagellar\\kaggle\\working\\yolo_dataset\\images\\train\\tomo_003acc_z0000_y-001_x-001.jpg: ignoring corrupt image/label: negative label values [-0.00054142 -0.00052301]. Can they be the problem? do you also have these?", "votes": null }, { "id": "3206692", "postDate": "05/21/2025 16:45:29", "content": "when i use a test set (from the train data that i dont use for either train or val) i get big scores, like i dont know 0.82
", "rawMarkdown": "when i use a test set (from the train data that i dont use for either train or val) i get big scores, like i dont know 0.82", "votes": null }, { "id": "3206709", "postDate": "05/21/2025 17:22:03", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1108534%2F5ff43564e32a4fc2b2ef050de373ae7a%2FF1_curve.png?generation=1747848112096263&alt=media\")
why mine is so low? that one is from a model that use the extra data
", "rawMarkdown": "why mine is so low? that one is from a model that use the extra data", "votes": null }, { "id": "3206753", "postDate": "05/21/2025 18:20:40", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1108534%2Ffd58e5b4fa4cad8d629d268e09eab1a3%2FF1_curve.png?generation=1747851637142688&alt=media\")
and this is the one when i train without the extra data. They are both yolo 8 with the standard configs box 24 trust 4 etc.
", "rawMarkdown": "and this is the one when i train without the extra data. They are both yolo 8 with the standard configs box 24 trust 4 etc.", "votes": null }, { "id": "3206852", "postDate": "05/21/2025 20:43:16", "content": "I would recommend using a majority slices that have a label in them and maybe like 10% of background images. The warning is likely due to negative labels where instead of having a negative bounding box centroid it should be empty. What input resolution are you using? Also make sure that any resizing you are doing takes into account the label (ie if you resize an image by a factor of 1.5 then make sure to adjust the output too). I recommend plotting the labels like I did below to guarantee they are correct:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3548771%2Fbc4fc4bb60f91e74787f0a46d95f9b78%2FScreenshot%202025-05-21%20at%204.42.57PM.png?generation=1747860195715836&alt=media\")
Thanks for sharing! nice work!
", "rawMarkdown": "Thanks for sharing! nice work!", "votes": null } ], "comments": [ { "id": 3205998, "author_name": "shanzhong8", "author_url": "", "post_date": "05/20/2025 17:50:58", "content": "0.14-0.45 just insane, perhaps you can boost your score by just changing the params in inference! One can get to 0.8+ with single model trained without any external data.
", "votes": null, "replies": [ { "id": 3206082, "author_name": "vasileioscharatsidis", "author_url": "", "post_date": "05/20/2025 19:35:10", "content": "yes i have heard of that so thats why i dont get why my trains go so wrong ;/
", "votes": null, "replies": [ { "id": 3206219, "author_name": "shanzhong8", "author_url": "", "post_date": "05/21/2025 03:34:04", "content": "What's your local CV score? Maybe you have too much noisy data, I think trained on only one motor images would help in this stage.
", "votes": null, "replies": [ { "id": 3206692, "author_name": "vasileioscharatsidis", "author_url": "", "post_date": "05/21/2025 16:45:29", "content": "when i use a test set (from the train data that i dont use for either train or val) i get big scores, like i dont know 0.82
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3206014, "author_name": "connorjd", "author_url": "", "post_date": "05/20/2025 18:17:59", "content": "Start with one of those starter notebooks and expand from there. Feel free to share your solution and I am happy to help but here are a few considerations for why your solution may score so low:
\nOverall if you cannot figure out why your solution is way worse then take one of the notebooks that achieves a good score and work backwards, figure out what you did different and why your score is lower. Most of the time it is something silly like mixing up coordinates or creating incorrect labels (ie saying the target is at X=30 Y=40 instead of X=40 Y=30). Again I am happy to help further if you have more detail! :D
", "votes": null, "replies": [ { "id": 3206080, "author_name": "vasileioscharatsidis", "author_url": "", "post_date": "05/20/2025 19:34:38", "content": "thanks, i thought about the above but they are not it ! i use also a public inference notebook and i just substitute my model, i tried with 2 of them
", "votes": null, "replies": [ { "id": 3206109, "author_name": "connorjd", "author_url": "", "post_date": "05/20/2025 20:40:16", "content": "Are you using YOLO? Make sure that if you plugged your model in that you are using the same preprocessing steps as they do. Also consider that they may use a different confidence threshold. Since we are using a Beta value of 2 for this competition I recommend using a lower confidence threshold on your predictions to get a higher recall score. It is possible your model makes less confident predictions and requires a lower conf threshold than what is used in the notebook you plugged into. Can you share your training and validation F1 curves? I can compare to mine but a baseline train for me got 0.85 F1 as shown below.
\n
Here are my train and validation loss curves as well throughout training if you want to compare yours. This is not for my best model but for a baseline that is scoring pretty well:
\n
@connorjd i get lots of these train: WARNING ⚠️ C:\\Users\\chara\\PycharmProjects\\Kaggle_BYU_flagellar\\kaggle\\working\\yolo_dataset\\images\\train\\tomo_003acc_z0000_y-001_x-001.jpg: ignoring corrupt image/label: negative label values [-0.00054142 -0.00052301]. Can they be the problem? do you also have these?
", "votes": null, "replies": [ { "id": 3206709, "author_name": "vasileioscharatsidis", "author_url": "", "post_date": "05/21/2025 17:22:03", "content": "
why mine is so low? that one is from a model that use the extra data
", "votes": null, "replies": [ { "id": 3206753, "author_name": "vasileioscharatsidis", "author_url": "", "post_date": "05/21/2025 18:20:40", "content": "
and this is the one when i train without the extra data. They are both yolo 8 with the standard configs box 24 trust 4 etc.
", "votes": null, "replies": [ { "id": 3206852, "author_name": "connorjd", "author_url": "", "post_date": "05/21/2025 20:43:16", "content": "I would recommend using a majority slices that have a label in them and maybe like 10% of background images. The warning is likely due to negative labels where instead of having a negative bounding box centroid it should be empty. What input resolution are you using? Also make sure that any resizing you are doing takes into account the label (ie if you resize an image by a factor of 1.5 then make sure to adjust the output too). I recommend plotting the labels like I did below to guarantee they are correct:
\n
Thanks for sharing! nice work!
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "3205992": "Hi guys, i have used two of the public notebooks to train a yolo model, whatever i tried, i have good scores in test sets that i split from the train folder but my LB scores are very very low, like 0.14 - 0.45. I have tried several train/val splits but no luck. How the public notebook that achieves 0.77 with a single model was trained? what can be wrong in my trains?", "3205998": "0.14-0.45 just insane, perhaps you can boost your score by just changing the params in inference! One can get to 0.8+ with single model trained without any external data.", "3206014": "Start with one of those starter notebooks and expand from there. Feel free to share your solution and I am happy to help but here are a few considerations for why your solution may score so low:\n- Mixing up axis on inference such as writing X as Y and Y as X\n- If you are doing a 2D solution on each slice then ensure that you are extracting the correct Z axis for your predictions (ie if you read the slices out of order and don't label which slice it is then you could end up with the wrong Z target)\n- Check if your validation metrics are similar to your LB submission to know if the issue is with a bad model or a bug in your inference notebook. For example, if you achieve a very good validation score during your cross validation experiments and then the LB score is way off then you may have a data leak or some other issue.\n- Make sure your preprocessing for your train and submission are identical (ie use the same normalization).\n\nOverall if you cannot figure out why your solution is way worse then take one of the notebooks that achieves a good score and work backwards, figure out what you did different and why your score is lower. Most of the time it is something silly like mixing up coordinates or creating incorrect labels (ie saying the target is at X=30 Y=40 instead of X=40 Y=30). Again I am happy to help further if you have more detail! :D", "3206080": "thanks, i thought about the above but they are not it ! i use also a public inference notebook and i just substitute my model, i tried with 2 of them", "3206082": "yes i have heard of that so thats why i dont get why my trains go so wrong ;/", "3206109": "Are you using YOLO? Make sure that if you plugged your model in that you are using the same preprocessing steps as they do. Also consider that they may use a different confidence threshold. Since we are using a Beta value of 2 for this competition I recommend using a lower confidence threshold on your predictions to get a higher recall score. It is possible your model makes less confident predictions and requires a lower conf threshold than what is used in the notebook you plugged into. Can you share your training and validation F1 curves? I can compare to mine but a baseline train for me got 0.85 F1 as shown below.\n\n", "3206111": "Here are my train and validation loss curves as well throughout training if you want to compare yours. This is not for my best model but for a baseline that is scoring pretty well:\n", "3206219": "What's your local CV score? Maybe you have too much noisy data, I think trained on only one motor images would help in this stage.", "3206691": "connorjd i get lots of these train: WARNING ⚠️ C:\\Users\\chara\\PycharmProjects\\Kaggle_BYU_flagellar\\kaggle\\working\\yolo_dataset\\images\\train\\tomo_003acc_z0000_y-001_x-001.jpg: ignoring corrupt image/label: negative label values [-0.00054142 -0.00052301]. Can they be the problem? do you also have these?", "3206692": "when i use a test set (from the train data that i dont use for either train or val) i get big scores, like i dont know 0.82", "3206709": "", "3206710": "why mine is so low? that one is from a model that use the extra data", "3206753": "", "3206756": "and this is the one when i train without the extra data. They are both yolo 8 with the standard configs box 24 trust 4 etc.", "3206852": "I would recommend using a majority slices that have a label in them and maybe like 10% of background images. The warning is likely due to negative labels where instead of having a negative bounding box centroid it should be empty. What input resolution are you using? Also make sure that any resizing you are doing takes into account the label (ie if you resize an image by a factor of 1.5 then make sure to adjust the output too). I recommend plotting the labels like I did below to guarantee they are correct:\n\n", "3207728": "Thanks for sharing! nice work!" }, "source": "meta" }