![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1659719%2Fd73c2016accb4eff861c893b2eae8e15%2FLoss1.png?generation=1599383049802037&alt=media\")
Laplace Loss Split
\nWhy we want to split the competition into separate FVC and Sigma prediction? There are some reasons:
\nThere is mathematical proof that splitting is the next:
\n$$
\nLoss(y_{true}, y_{pred}, \\sigma_{pred}) = Loss_1(y_{true}, y_{pred}) + Loss_2(\\Delta_{true}, \\Delta_{pred}),
\n$$
where
\n$$
\n\\sigma = \\sqrt{2} \\cdot \\Delta_{pred}, \\
\n \\Delta = \\Delta_{true}
\n$$
So it works as follows: at first you train a model for FVC prediction, and than for these predictions compute optimal Sigma and train the second model. You can use it as a custom loss for your models or as a metric for understanding what model is the best after training. In my experiments first approach with boost algorithms haven't worked so I'm going to use the second option.
\nFirst Loss
\n$$
\nLoss_1(y_{true}, y_{pred}) = \\log(|y_{true} - y_{pred}|)
\n$$
First loss equals geometric mean of the module of the errors, so behaviour is obvious:
\n
Second Loss
\n$$
\nLoss_2(\\Delta_{true}, \\Delta_{pred}) = \\frac{\\Delta_{true}}{\\Delta_{pred}} + \\log{\\frac{\\Delta_{pred}}{\\Delta_{true}}}.
\n$$
That loss is more sophisticated and has two cases that different in it's behaviour:
\n$$ \\Delta_{pred} < \\Delta_{true}, (x = \\frac{\\Delta_{true}}{\\Delta_{pred}} > 1) \\rightarrow Loss_2(\\Delta_{true}, \\Delta_{pred}) = x - \\ln x$$
\n$$\\Delta_{pred} \\geq \\Delta_{true}, (x = \\frac{\\Delta_{pred}}{\\Delta_{true}} \\geq 1) \\rightarrow Loss_2(\\Delta_{true}, \\Delta_{pred}) = \\frac{1}{x} + \\ln x \\sim \\ln x$$
For instance, penalty for underrated Sigma is stronger, than for overrated one. It's clear to see on the pictures below (even with the next fact: more true Sigma, stronger penalty for underestimation):
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1659719%2Fae6ccf733d088eedafbe4b4cbb9d82e2%2FLoss2.png?generation=1599383072353590&alt=media\")
Laplace Loss Split
\nWhy we want to split the competition into separate FVC and Sigma prediction? There are some reasons:
\nThere is mathematical proof that splitting is the next:
\n$$
\nLoss(y_{true}, y_{pred}, \\sigma_{pred}) = Loss_1(y_{true}, y_{pred}) + Loss_2(\\Delta_{true}, \\Delta_{pred}),
\n$$
where
\n$$
\n\\sigma = \\sqrt{2} \\cdot \\Delta_{pred}, \\
\n \\Delta = \\Delta_{true}
\n$$
So it works as follows: at first you train a model for FVC prediction, and than for these predictions compute optimal Sigma and train the second model. You can use it as a custom loss for your models or as a metric for understanding what model is the best after training. In my experiments first approach with boost algorithms haven't worked so I'm going to use the second option.
\nFirst Loss
\n$$
\nLoss_1(y_{true}, y_{pred}) = \\log(|y_{true} - y_{pred}|)
\n$$
First loss equals geometric mean of the module of the errors, so behaviour is obvious:
\n
Second Loss
\n$$
\nLoss_2(\\Delta_{true}, \\Delta_{pred}) = \\frac{\\Delta_{true}}{\\Delta_{pred}} + \\log{\\frac{\\Delta_{pred}}{\\Delta_{true}}}.
\n$$
That loss is more sophisticated and has two cases that different in it's behaviour:
\n$$ \\Delta_{pred} < \\Delta_{true}, (x = \\frac{\\Delta_{true}}{\\Delta_{pred}} > 1) \\rightarrow Loss_2(\\Delta_{true}, \\Delta_{pred}) = x - \\ln x$$
\n$$\\Delta_{pred} \\geq \\Delta_{true}, (x = \\frac{\\Delta_{pred}}{\\Delta_{true}} \\geq 1) \\rightarrow Loss_2(\\Delta_{true}, \\Delta_{pred}) = \\frac{1}{x} + \\ln x \\sim \\ln x$$
For instance, penalty for underrated Sigma is stronger, than for overrated one. It's clear to see on the pictures below (even with the next fact: more true Sigma, stronger penalty for underestimation):
\n
what's $\\Delta$ standards for?
\nshould the loss1 in your post be scaled by sigma?
", "rawMarkdown": "what's $\\Delta$ standards for? \n\nshould the loss1 in your post be scaled by sigma?", "votes": null }, { "id": "1000120", "postDate": "09/06/2020 09:58:29", "content": "Delta is an abbreviation from the Laplace loss:
\n$$
\n\\Delta = \\min(|FVC_{true} - FVC_{predicted}, 1000)
\n$$
at first you train a model for FVC prediction, and than for these predictions compute optimal Sigma and train the second model.\nThis is what was done in the following notebook, correct?
\nhttps://www.kaggle.com/yasufuminakama/osic-lgb-baseline
", "rawMarkdown": "```\nat first you train a model for FVC prediction, and than for these predictions compute optimal Sigma and train the second model.\n```\n\nThis is what was done in the following notebook, correct? \n\nhttps://www.kaggle.com/yasufuminakama/osic-lgb-baseline", "votes": null }, { "id": "1000210", "postDate": "09/06/2020 11:37:13", "content": "Great notebook, exactly the same idea. But this topic more about the losses: the notebook has used RMSE for each task (chances are a good practice solution indeed), but I suggest to use particular losses for each step with high correlation with Laplace Loss. For instance RMSE=0.001 means that you get good model but you still don't know what Laplace Loss you get after all. On the other hand Loss_1=4.321 means lower bound for Laplace Loss (drop some constant for better understanding).
what's $\\Delta$ standards for?
\nshould the loss1 in your post be scaled by sigma?
", "votes": null, "replies": [ { "id": 1000120, "author_name": "koza4ukdmitrij", "author_url": "", "post_date": "09/06/2020 09:58:29", "content": "Delta is an abbreviation from the Laplace loss:
\n$$
\n\\Delta = \\min(|FVC_{true} - FVC_{predicted}, 1000)
\n$$
at first you train a model for FVC prediction, and than for these predictions compute optimal Sigma and train the second model.\nThis is what was done in the following notebook, correct?
\nhttps://www.kaggle.com/yasufuminakama/osic-lgb-baseline
", "votes": null, "replies": [ { "id": 1000210, "author_name": "koza4ukdmitrij", "author_url": "", "post_date": "09/06/2020 11:37:13", "content": "Great notebook, exactly the same idea. But this topic more about the losses: the notebook has used RMSE for each task (chances are a good practice solution indeed), but I suggest to use particular losses for each step with high correlation with Laplace Loss. For instance RMSE=0.001 means that you get good model but you still don't know what Laplace Loss you get after all. On the other hand Loss_1=4.321 means lower bound for Laplace Loss (drop some constant for better understanding).