BCE+MixUp is the popular choice rn.
\nMixup does nothing more than creating a sort of rank for the model to learn. Targets become just values between [0,1], reflecting the signal intensity thus boosting ROC. In this way I had luck with other losses e.g. MSE.
\nHave you also tried something different?
BCE+MixUp is the popular choice rn.
\nMixup does nothing more than creating a sort of rank for the model to learn. Targets become just values between [0,1], reflecting the signal intensity thus boosting ROC. In this way I had luck with other losses e.g. MSE.
\nHave you also tried something different?
Do you mix inputs like this:
\nmixed_imgs = imgs * lam.sqrt() + (1-lam).sqrt() * imgs[idxs]
\nto preserve standard deviation of 1?
You can try to use different coefficients for the beta distribution, but i think the old good uniform is enough.
", "rawMarkdown": "You can try to use different coefficients for the beta distribution, but i think the old good uniform is enough. " }, { "id": 1343724, "postDate": "2021-06-10T11:46:24.440Z", "content": "I mean, after you generated lam with whatever distribution, if you add the images together with
\n* lam + (1-lam) *
\nthen your new images won't have std of 1;
\nbut you'll have std of sqrt(lam^2+(1-lam)^2)
If you use alpha of 1.0, then you always have lam = 0.5
\nIn this case you'll always have std of sqrt(0.5) ~ 0.71
If so, you may want to try multiplying your model's inputs by 0.71 while you create your submission, perhaps it's better to have the same std as you had at training time.
", "rawMarkdown": "I mean, after you generated lam with whatever distribution, if you add the images together with\n`* lam + (1-lam) *`\nthen your new images won't have std of 1;\nbut you'll have std of `sqrt(lam^2+(1-lam)^2)`\n\nIf you use alpha of 1.0, then you always have lam = 0.5\nIn this case you'll always have std of sqrt(0.5) ~ 0.71\n\nIf so, you may want to try multiplying your model's inputs by 0.71 while you create your submission, perhaps it's better to have the same std as you had at training time.", "votes": 9 }, { "id": 1343746, "postDate": "2021-06-10T12:05:14.253Z", "content": "Wops, now i see that you were referring to the std. Yes i sqrt as you do :)
", "rawMarkdown": "Wops, now i see that you were referring to the std. Yes i sqrt as you do :)", "votes": 1 }, { "id": 1343756, "postDate": "2021-06-10T12:16:13.260Z", "content": "Thanks! :)
", "rawMarkdown": "Thanks! :)" }, { "id": 1343941, "postDate": "2021-06-10T14:24:36.337Z", "content": "PS: in my mix up i do the same for the targets (without the sqrt ofc). Then feed everything to the model and loss. It is different than the one proposed here, even simpler.
", "rawMarkdown": "PS: in my mix up i do the same for the targets (without the sqrt ofc). Then feed everything to the model and loss. It is different than the one proposed [here](https://www.kaggle.com/ttahara/seti-e-t-resnet18d-baseline), even simpler.", "votes": 1 }, { "id": 1343963, "postDate": "2021-06-10T14:39:55.013Z", "content": "Yes, adding up targets the same way and use BCELoss should lead to the same result, not regarding numerical instability.
\nBut if one uses mixed precision, then BCELoss is not even allowed by pytorch.. :D
In that case, one can just take the weighted targets
\n(targets = targets*lam+(1-lam)*targets[idxs])
\nand do:
\nbcewithlogitsloss = torch.nn.BCEWithLogitsLoss(reduction='none')
loss1 = bcewithlogitsloss(outs, torch.zeros_like(targets))\nloss2 = bcewithlogitsloss(outs, torch.ones_like(targets))\nloss = (loss1 * (1-targets) + targets * loss2).mean()\nReshape outs and/or the \"dummy\" zero and one targets as needed, also create them on outs.device and type of float.
", "rawMarkdown": "Yes, adding up targets the same way and use BCELoss should lead to the same result, not regarding numerical instability.\nBut if one uses mixed precision, then BCELoss is not even allowed by pytorch.. :D\n\nIn that case, one can just take the weighted targets\n(`targets = targets*lam+(1-lam)*targets[idxs]`)\nand do:\n`bcewithlogitsloss = torch.nn.BCEWithLogitsLoss(reduction='none')`\n\n```\nloss1 = bcewithlogitsloss(outs, torch.zeros_like(targets))\nloss2 = bcewithlogitsloss(outs, torch.ones_like(targets))\nloss = (loss1 * (1-targets) + targets * loss2).mean()\n```\n\nReshape outs and/or the \"dummy\" zero and one targets as needed, also create them on outs.device and type of float.", "votes": 1 }, { "id": 1349420, "postDate": "2021-06-14T18:32:44.897Z", "content": "\n\nBut if one uses mixed precision, then BCELoss is not even allowed by pytorch.. :D
\n
??
\nI used BCELoss with amp autocast in my last competition and there was absolutely no problem.
", "rawMarkdown": "> But if one uses mixed precision, then BCELoss is not even allowed by pytorch.. :D\n\n??\n\nI used BCELoss with amp autocast in my last competition and there was absolutely no problem.", "votes": 1 }, { "id": 1349475, "postDate": "2021-06-14T19:57:17.677Z", "rawMarkdown": "", "votes": 3, "isDeleted": true }, { "id": 1349479, "postDate": "2021-06-14T20:03:44.580Z", "content": "Right, I meant BCEWithLogitsLoss.
", "rawMarkdown": "Right, I meant BCEWithLogitsLoss. " } ] }, { "id": 3179331, "postDate": "2025-04-15T10:12:10.123Z", "content": "I think you can use thordata to use proxy IP
\nhttps://www.thordata.com/?ls=lt&lk=kaggle
", "rawMarkdown": "I think you can use thordata to use proxy IP\n\nhttps://www.thordata.com/?ls=lt&lk=kaggle" }, { "id": 2906372, "postDate": "2024-07-05T14:48:34.287Z", "content": "Choosing the best shared proxies for ROC (Republic of China) depends on your specific needs and preferences. BCE proxies are known for their reliability and performance, offering a range of features that cater to different uses. However, the best shared proxies can vary based on factors like speed, location coverage, and pricing. It's beneficial to compare BCE with other providers, considering factors such as customer support and user reviews. Ultimately, the best shared proxies for ROC are those that meet your requirements effectively, whether for browsing, security, or other specific purposes.
", "rawMarkdown": "Choosing the best shared proxies for ROC (Republic of China) depends on your specific needs and preferences. BCE proxies are known for their reliability and performance, offering a range of features that cater to different uses. However, the **[best shared proxies](https://proxies.best/shared-proxies/)** can vary based on factors like speed, location coverage, and pricing. It's beneficial to compare BCE with other providers, considering factors such as customer support and user reviews. Ultimately, the best shared proxies for ROC are those that meet your requirements effectively, whether for browsing, security, or other specific purposes." }, { "id": 1343811, "postDate": "2021-06-10T13:10:48.407Z", "content": "Can I ask your validation loss with mse?
\nI think your loss in [0.005, 0.03)
Mixup being advantageous for ROC is insightful. Never thought of that, thank you! :)
", "rawMarkdown": "Mixup being advantageous for ROC is insightful. Never thought of that, thank you! :)", "replies": [ { "id": 1343663, "postDate": "2021-06-10T11:02:58.127Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1343601, "postDate": "2021-06-10T09:54:01.970Z", "content": "I tried focal loss. I don’t understand why it’s not working. Can anyone enlighten me with some intuition on this? Any mathematical explanation is fine with me.
\nAlso do you mean you tried MSE + Mixup?
", "rawMarkdown": "I tried focal loss. I don’t understand why it’s not working. Can anyone enlighten me with some intuition on this? Any mathematical explanation is fine with me. \n\nAlso do you mean you tried MSE + Mixup?", "replies": [ { "id": 1343608, "postDate": "2021-06-10T10:08:38.393Z", "content": "I could go in the math but I wont just for sake of simplicity aahaha.
\nAnyway, focal is just a fancier way for label smoothing (don't hate me for this statement <3). In both smoothing and focal loss we try to \"penalize\" well classified samples by respectively adding an eps (smoothing) or using an exponent (gamma) to the probs (focal). Neither of these solutions alone creates a way to rank the observations.
\n\nAlso do you mean you tried MSE + Mixup?
\n
Correct
", "rawMarkdown": "I could go in the math but I wont just for sake of simplicity aahaha.\nAnyway, focal is just a fancier way for label smoothing (don't hate me for this statement <3). In both smoothing and focal loss we try to \"penalize\" well classified samples by respectively adding an eps (smoothing) or using an exponent (gamma) to the probs (focal). Neither of these solutions alone creates a way to rank the observations. \n\n> Also do you mean you tried MSE + Mixup?\n\nCorrect", "votes": 4 }, { "id": 1343625, "postDate": "2021-06-10T10:30:05.130Z", "content": "@callmeb You certainly seem to be well versed in this. and I would certainly not shy away from asking more questions. I see the word ranking quite often, and understand how roc works (in a naive way).
\nWhat I don't understand is this link here. Is ROC a ranking metric in the sense described in the link?
", "rawMarkdown": "@callmeb You certainly seem to be well versed in this. and I would certainly not shy away from asking more questions. I see the word ranking quite often, and understand how roc works (in a naive way).\n\nWhat I don't understand is this[ link here](https://stats.stackexchange.com/questions/517022/why-is-roc-a-ranking-metric-and-how-do-we-understand-it-as-a-c-statistic). Is ROC a ranking metric in the sense described in the link?" }, { "id": 1343705, "postDate": "2021-06-10T11:38:56.717Z", "content": "Don't worry, I am here to help!
\nThanks, point 1 and 3 are well understood by me. I think point 2 is the one I need to digest, as when I tried to code my own roc function, I did not \"really sort it\" and it still worked.
", "rawMarkdown": "Thanks, point 1 and 3 are well understood by me. I think point 2 is the one I need to digest, as when I tried to code my own roc function, I did not \"really sort it\" and it still worked." }, { "id": 1343802, "postDate": "2021-06-10T13:00:22.480Z", "content": "ROC AUC is very similar to label-weighted label-ranking average precision, for which here is a pretty good explanation. If this helps, then ROC AUC is quite easier to understand with the previously referenced code example written by @cpmpml
Thanks. This clears things up. I’ll try to read cpmp code again to trace out.
", "rawMarkdown": "Thanks. This clears things up. I’ll try to read cpmp code again to trace out. ", "votes": 1 } ] }, { "id": 1343683, "postDate": "2021-06-10T11:17:08.353Z", "rawMarkdown": "", "votes": 2, "isDeleted": true, "replies": [ { "id": 1343712, "postDate": "2021-06-10T11:42:19.390Z", "content": "I see we are at the same point 👍
", "rawMarkdown": "I see we are at the same point 👍", "votes": 1 }, { "id": 1343748, "postDate": "2021-06-10T12:09:52.660Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1343942, "postDate": "2021-06-10T14:25:33.620Z", "content": "Too early to team up, maybe later ;)
", "rawMarkdown": "Too early to team up, maybe later ;)" }, { "id": 1343988, "postDate": "2021-06-10T14:51:05.033Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 1343665, "author_name": "nofreewill42", "author_url": "", "post_date": "2021-06-10T11:03:31.237000", "content": "Do you mix inputs like this:
\nmixed_imgs = imgs * lam.sqrt() + (1-lam).sqrt() * imgs[idxs]
\nto preserve standard deviation of 1?
You can try to use different coefficients for the beta distribution, but i think the old good uniform is enough.
", "votes": 0, "replies": [] }, { "id": 1343724, "author_name": "nofreewill42", "author_url": "", "post_date": "2021-06-10T11:46:24.440000", "content": "I mean, after you generated lam with whatever distribution, if you add the images together with
\n* lam + (1-lam) *
\nthen your new images won't have std of 1;
\nbut you'll have std of sqrt(lam^2+(1-lam)^2)
If you use alpha of 1.0, then you always have lam = 0.5
\nIn this case you'll always have std of sqrt(0.5) ~ 0.71
If so, you may want to try multiplying your model's inputs by 0.71 while you create your submission, perhaps it's better to have the same std as you had at training time.
", "votes": 9, "replies": [] }, { "id": 1343746, "author_name": "Riccardo", "author_url": "", "post_date": "2021-06-10T12:05:14.253000", "content": "Wops, now i see that you were referring to the std. Yes i sqrt as you do :)
", "votes": 1, "replies": [] }, { "id": 1343756, "author_name": "nofreewill42", "author_url": "", "post_date": "2021-06-10T12:16:13.260000", "content": "Thanks! :)
", "votes": 0, "replies": [] }, { "id": 1343941, "author_name": "Riccardo", "author_url": "", "post_date": "2021-06-10T14:24:36.337000", "content": "PS: in my mix up i do the same for the targets (without the sqrt ofc). Then feed everything to the model and loss. It is different than the one proposed here, even simpler.
", "votes": 1, "replies": [] }, { "id": 1343963, "author_name": "nofreewill42", "author_url": "", "post_date": "2021-06-10T14:39:55.013000", "content": "Yes, adding up targets the same way and use BCELoss should lead to the same result, not regarding numerical instability.
\nBut if one uses mixed precision, then BCELoss is not even allowed by pytorch.. :D
In that case, one can just take the weighted targets
\n(targets = targets*lam+(1-lam)*targets[idxs])
\nand do:
\nbcewithlogitsloss = torch.nn.BCEWithLogitsLoss(reduction='none')
loss1 = bcewithlogitsloss(outs, torch.zeros_like(targets))\nloss2 = bcewithlogitsloss(outs, torch.ones_like(targets))\nloss = (loss1 * (1-targets) + targets * loss2).mean()\nReshape outs and/or the \"dummy\" zero and one targets as needed, also create them on outs.device and type of float.
", "votes": 1, "replies": [] }, { "id": 1349420, "author_name": "CPMP", "author_url": "", "post_date": "2021-06-14T18:32:44.897000", "content": "\n\nBut if one uses mixed precision, then BCELoss is not even allowed by pytorch.. :D
\n
??
\nI used BCELoss with amp autocast in my last competition and there was absolutely no problem.
", "votes": 1, "replies": [] }, { "id": 1349475, "author_name": "", "author_url": "", "post_date": "2021-06-14T19:57:17.677000", "content": "", "votes": 3, "replies": [] }, { "id": 1349479, "author_name": "CPMP", "author_url": "", "post_date": "2021-06-14T20:03:44.580000", "content": "Right, I meant BCEWithLogitsLoss.
", "votes": 0, "replies": [] } ] }, { "id": 3179331, "author_name": "", "author_url": "", "post_date": "2025-04-15T10:12:10.123000", "content": "I think you can use thordata to use proxy IP
\nhttps://www.thordata.com/?ls=lt&lk=kaggle
", "votes": 0, "replies": [] }, { "id": 2906372, "author_name": "Samantha Tan", "author_url": "", "post_date": "2024-07-05T14:48:34.287000", "content": "Choosing the best shared proxies for ROC (Republic of China) depends on your specific needs and preferences. BCE proxies are known for their reliability and performance, offering a range of features that cater to different uses. However, the best shared proxies can vary based on factors like speed, location coverage, and pricing. It's beneficial to compare BCE with other providers, considering factors such as customer support and user reviews. Ultimately, the best shared proxies for ROC are those that meet your requirements effectively, whether for browsing, security, or other specific purposes.
", "votes": 0, "replies": [] }, { "id": 1343811, "author_name": "assign", "author_url": "", "post_date": "2021-06-10T13:10:48.407000", "content": "Can I ask your validation loss with mse?
\nI think your loss in [0.005, 0.03)
Mixup being advantageous for ROC is insightful. Never thought of that, thank you! :)
", "votes": 0, "replies": [ { "id": 1343663, "author_name": "", "author_url": "", "post_date": "2021-06-10T11:02:58.127000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1343601, "author_name": "gao-hongnan", "author_url": "", "post_date": "2021-06-10T09:54:01.970000", "content": "I tried focal loss. I don’t understand why it’s not working. Can anyone enlighten me with some intuition on this? Any mathematical explanation is fine with me.
\nAlso do you mean you tried MSE + Mixup?
", "votes": 0, "replies": [ { "id": 1343608, "author_name": "Riccardo", "author_url": "", "post_date": "2021-06-10T10:08:38.393000", "content": "I could go in the math but I wont just for sake of simplicity aahaha.
\nAnyway, focal is just a fancier way for label smoothing (don't hate me for this statement <3). In both smoothing and focal loss we try to \"penalize\" well classified samples by respectively adding an eps (smoothing) or using an exponent (gamma) to the probs (focal). Neither of these solutions alone creates a way to rank the observations.
\n\nAlso do you mean you tried MSE + Mixup?
\n
Correct
", "votes": 4, "replies": [] }, { "id": 1343625, "author_name": "gao-hongnan", "author_url": "", "post_date": "2021-06-10T10:30:05.130000", "content": "@callmeb You certainly seem to be well versed in this. and I would certainly not shy away from asking more questions. I see the word ranking quite often, and understand how roc works (in a naive way).
\nWhat I don't understand is this link here. Is ROC a ranking metric in the sense described in the link?
", "votes": 0, "replies": [] }, { "id": 1343705, "author_name": "Riccardo", "author_url": "", "post_date": "2021-06-10T11:38:56.717000", "content": "Don't worry, I am here to help!
\nThanks, point 1 and 3 are well understood by me. I think point 2 is the one I need to digest, as when I tried to code my own roc function, I did not \"really sort it\" and it still worked.
", "votes": 0, "replies": [] }, { "id": 1343802, "author_name": "nofreewill42", "author_url": "", "post_date": "2021-06-10T13:00:22.480000", "content": "ROC AUC is very similar to label-weighted label-ranking average precision, for which here is a pretty good explanation. If this helps, then ROC AUC is quite easier to understand with the previously referenced code example written by @cpmpml
Thanks. This clears things up. I’ll try to read cpmp code again to trace out.
", "votes": 1, "replies": [] } ] }, { "id": 1343683, "author_name": "", "author_url": "", "post_date": "2021-06-10T11:17:08.353000", "content": "", "votes": 2, "replies": [ { "id": 1343712, "author_name": "Riccardo", "author_url": "", "post_date": "2021-06-10T11:42:19.390000", "content": "I see we are at the same point 👍
", "votes": 1, "replies": [] }, { "id": 1343748, "author_name": "", "author_url": "", "post_date": "2021-06-10T12:09:52.660000", "content": "", "votes": 1, "replies": [] }, { "id": 1343942, "author_name": "Riccardo", "author_url": "", "post_date": "2021-06-10T14:25:33.620000", "content": "Too early to team up, maybe later ;)
", "votes": 0, "replies": [] }, { "id": 1343988, "author_name": "", "author_url": "", "post_date": "2021-06-10T14:51:05.033000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1343566": "BCE+MixUp is the popular choice rn.\nMixup does nothing more than creating a sort of rank for the model to learn. Targets become just values between [0,1], reflecting the signal intensity thus boosting ROC. In this way I had luck with other losses e.g. MSE.\nHave you also tried something different?", "1343665": "Do you mix inputs like this:\n`mixed_imgs = imgs * lam.sqrt() + (1-lam).sqrt() * imgs[idxs]`\nto preserve standard deviation of 1?", "3179331": "I think you can use thordata to use proxy IP\n\nhttps://www.thordata.com/?ls=lt&lk=kaggle", "2906372": "Choosing the best shared proxies for ROC (Republic of China) depends on your specific needs and preferences. BCE proxies are known for their reliability and performance, offering a range of features that cater to different uses. However, the **[best shared proxies](https://proxies.best/shared-proxies/)** can vary based on factors like speed, location coverage, and pricing. It's beneficial to compare BCE with other providers, considering factors such as customer support and user reviews. Ultimately, the best shared proxies for ROC are those that meet your requirements effectively, whether for browsing, security, or other specific purposes.", "1343811": "Can I ask your validation loss with mse?\nI think your loss in [0.005, 0.03)", "1343653": "Mixup being advantageous for ROC is insightful. Never thought of that, thank you! :)", "1343601": "I tried focal loss. I don’t understand why it’s not working. Can anyone enlighten me with some intuition on this? Any mathematical explanation is fine with me. \n\nAlso do you mean you tried MSE + Mixup?", "1343683": "" } }