Here is my brief summary that may be different from others:
\n\nFeatures inspired by Starter Kit
\nNot only construct features between adjacent passbands as starter kit showed, like between passband 0 and 1, 1 and 2. But also between passband 0 and 2, 0 and 3, and so on. Besides, not only mean flux, but also try max flux and min flux. Finally, features like
\n\nlog((passband_1_max_flux) / (passband_3_max_flux))\n
\ncontribute a lot to score.
Features I've tried that useful
\n1) 1st quartile and 3rd quartile flux per passband. And their gap value relative to full scale, like
\n\n(passband_0_q3_flux - passband_0_q1_flux) / (passband_0_max_flux - passband_0_min_flux)\n
\n2) 3rd quartile flux per passband multiply square of hostgal_photoz, like
\n\npassband_0_q3_flux * (hostgal_photoz**2)
\n(passband_0_q3_flux - passband_0_q1_flux) * (hostgal_photoz**2)\n
\nThose features work well on my model.
Features from cesium and feets packages
\nperiod_fast feature per passband from cesium pkg and CAR feature per passband from feets pkg, made some contribution to my model.
Features referred to papers
\nHere is the very important paper I've read and referred to. In this paper, authors introduced several method to extract light curve parameters for classification. I referred to the most effective method of them -- salt2 model, in part 3.2 of the paper. Fortunately, you can use sncosmo api to fit salt2 model and get model parameters as features.
Features referred to github
\nBazin function, this method is same with what 4th solution has metioned, but I found it by this link. I've tried modify this method, but the result shows no big different.
Features referred to a public kernel
\nThis public kernel is very impressive, but only got a few upvotes. Instead of use it as hostgal_photoz == 0 ( author set in the kernel), I use it for ext-gal model with a set hostgal_photoz > 0. It really improved my model.
Features selection
\nIn order to remove some inappropriate features, I do it simple, observing some less important features' distribution on train and test. Use this method I've used before. Since there're class99 in test set, it didn't work very well, but still contribute a little.
Modify Objective Function
\nMithrillion has public a very impressive kernel. But you may find well but not as good as just setting sample weights. So I add sample weights to grad and hess, it works better than built-in objective. This result in my final model structure:
\n\n0.55 * (lgb with customized objective) + 0.45 * (lgb with built-in objective)\n\nseparated by gal/ext-gal, which means the final model contains 4 lgbs.
Tune Parameters
\nI tuned parameters to avoid overfitting, small max_depth: 3, small max_bin: 20, large min_child_weight: 10, large min_data_in_leaf: 35. These parameters' setting work well both on CV and LB.
When my CV gets better, there are three things that decreasing the gap between CV and LB: 1) adding salt2 model parameters (improve CV 0.033, but LB 0.058); 2) Remove inappropriate features; 3) Tuning Parameters. Finally, I got CV 0.411, public LB 0.794, gap 0.383, private LB 0.822, gap 0.411.
\n\nHere are links to what I've found may useful
\n1) Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
\n2) Deep Recurrent Neural Networks for Supernovae Classification
As a novice in kaggle before this competition, I really learn a lot from those who public their impressive kernels, like ogrellier, mithrillion, kyleboone, meaninglesslives ( this is my first time to use keras practicing NN). And CPMP, he gives lots of truly useful tips in discussion. Giba, his class weights discussion help us a lot . sionek released the most famous 'detected_mjd' feature. It's strange that those useful and original kernel have more fork times than upvotes, those truly helpful discussions got only a little upvotes. If you think they are helpful, just upvote.
\nBesides, there is an abnormal thing :), Silogram didn't post his famous 'a few notes...' as he did in previous competitions I joined before, like 1) Home Credit Default Risk; 2) Google Analytics Customer Revenue Prediction . We can learn a lot from his notes, as CPMP do in this competition.
\nThey are the real fundamental of kaggle community !!
I just began to learn python and data science about 2 years ago ( I learn mechanical engineer in college). I think I've got lots luck in this competition, I'm not good enough. When I received teaming up invitations, I regarded them as an honor and encouragement, Thank you !
", "messages": [ { "id": "441847", "postDate": "12/19/2018 06:48:09", "content": "Here is my brief summary that may be different from others:
\n\nFeatures inspired by Starter Kit
\nNot only construct features between adjacent passbands as starter kit showed, like between passband 0 and 1, 1 and 2. But also between passband 0 and 2, 0 and 3, and so on. Besides, not only mean flux, but also try max flux and min flux. Finally, features like
\n\nlog((passband_1_max_flux) / (passband_3_max_flux))\n
\ncontribute a lot to score.
Features I've tried that useful
\n1) 1st quartile and 3rd quartile flux per passband. And their gap value relative to full scale, like
\n\n(passband_0_q3_flux - passband_0_q1_flux) / (passband_0_max_flux - passband_0_min_flux)\n
\n2) 3rd quartile flux per passband multiply square of hostgal_photoz, like
\n\npassband_0_q3_flux * (hostgal_photoz**2)
\n(passband_0_q3_flux - passband_0_q1_flux) * (hostgal_photoz**2)\n
\nThose features work well on my model.
Features from cesium and feets packages
\nperiod_fast feature per passband from cesium pkg and CAR feature per passband from feets pkg, made some contribution to my model.
Features referred to papers
\nHere is the very important paper I've read and referred to. In this paper, authors introduced several method to extract light curve parameters for classification. I referred to the most effective method of them -- salt2 model, in part 3.2 of the paper. Fortunately, you can use sncosmo api to fit salt2 model and get model parameters as features.
Features referred to github
\nBazin function, this method is same with what 4th solution has metioned, but I found it by this link. I've tried modify this method, but the result shows no big different.
Features referred to a public kernel
\nThis public kernel is very impressive, but only got a few upvotes. Instead of use it as hostgal_photoz == 0 ( author set in the kernel), I use it for ext-gal model with a set hostgal_photoz > 0. It really improved my model.
Features selection
\nIn order to remove some inappropriate features, I do it simple, observing some less important features' distribution on train and test. Use this method I've used before. Since there're class99 in test set, it didn't work very well, but still contribute a little.
Modify Objective Function
\nMithrillion has public a very impressive kernel. But you may find well but not as good as just setting sample weights. So I add sample weights to grad and hess, it works better than built-in objective. This result in my final model structure:
\n\n0.55 * (lgb with customized objective) + 0.45 * (lgb with built-in objective)\n\nseparated by gal/ext-gal, which means the final model contains 4 lgbs.
Tune Parameters
\nI tuned parameters to avoid overfitting, small max_depth: 3, small max_bin: 20, large min_child_weight: 10, large min_data_in_leaf: 35. These parameters' setting work well both on CV and LB.
When my CV gets better, there are three things that decreasing the gap between CV and LB: 1) adding salt2 model parameters (improve CV 0.033, but LB 0.058); 2) Remove inappropriate features; 3) Tuning Parameters. Finally, I got CV 0.411, public LB 0.794, gap 0.383, private LB 0.822, gap 0.411.
\n\nHere are links to what I've found may useful
\n1) Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream
\n2) Deep Recurrent Neural Networks for Supernovae Classification
As a novice in kaggle before this competition, I really learn a lot from those who public their impressive kernels, like ogrellier, mithrillion, kyleboone, meaninglesslives ( this is my first time to use keras practicing NN). And CPMP, he gives lots of truly useful tips in discussion. Giba, his class weights discussion help us a lot . sionek released the most famous 'detected_mjd' feature. It's strange that those useful and original kernel have more fork times than upvotes, those truly helpful discussions got only a little upvotes. If you think they are helpful, just upvote.
\nBesides, there is an abnormal thing :), Silogram didn't post his famous 'a few notes...' as he did in previous competitions I joined before, like 1) Home Credit Default Risk; 2) Google Analytics Customer Revenue Prediction . We can learn a lot from his notes, as CPMP do in this competition.
\nThey are the real fundamental of kaggle community !!
I just began to learn python and data science about 2 years ago ( I learn mechanical engineer in college). I think I've got lots luck in this competition, I'm not good enough. When I received teaming up invitations, I regarded them as an honor and encouragement, Thank you !
", "rawMarkdown": "Here is my brief summary that may be different from others: \n\n--- \n\n## Feature Engineer \n\n* **Features inspired by Starter Kit** \nNot only construct features between adjacent passbands as starter kit showed, like between passband 0 and 1, 1 and 2. But also between passband 0 and 2, 0 and 3, and so on. Besides, not only mean flux, but also try max flux and min flux. Finally, features like \n```\nlog((passband_1_max_flux) / (passband_3_max_flux))\n``` \ncontribute a lot to score. \n\n* **Features I've tried that useful** \n1) 1st quartile and 3rd quartile flux per passband. And their gap value relative to full scale, like \n```\n(passband_0_q3_flux - passband_0_q1_flux) / (passband_0_max_flux - passband_0_min_flux)\n``` \n2) 3rd quartile flux per passband multiply square of hostgal_photoz, like \n```\npassband_0_q3_flux * (hostgal_photoz**2) \n(passband_0_q3_flux - passband_0_q1_flux) * (hostgal_photoz**2)\n``` \nThose features work well on my model. \n\n* **Features from cesium and feets packages** \nperiod_fast feature per passband from cesium pkg and CAR feature per passband from feets pkg, made some contribution to my model.\n\n* **Features referred to papers** \n[Here is the very important paper I've read and referred to](https://arxiv.org/abs/1603.00882). In this paper, authors introduced several method to extract light curve parameters for classification. I referred to the most effective method of them -- salt2 model, in part 3.2 of the paper. Fortunately, you can use [sncosmo api](https://sncosmo.readthedocs.io/en/v1.6.x/examples/plot_lc_fit.html#sphx-glr-examples-plot-lc-fit-py) to fit salt2 model and get model parameters as features. \n\n* **Features referred to github** \nBazin function, this method is same with what [4th solution](https://www.kaggle.com/c/PLAsTiCC-2018/discussion/75011) has metioned, but I found it by this [link](https://github.com/ramp-kits/supernovae/blob/master/PLAsTiCC_starting_kit.ipynb). I've tried modify this method, but the result shows no big different.\n\n* **Features referred to a public kernel** \n[This public kernel is very impressive](https://www.kaggle.com/manugangler/optimal-feature-extraction-for-class-6), but only got a few upvotes. Instead of use it as hostgal_photoz == 0 ( author set in the kernel), I use it for ext-gal model with a set hostgal_photoz > 0. It really improved my model. \n\n* **Features selection** \nIn order to remove some inappropriate features, I do it simple, observing some less important features' distribution on train and test. Use [this method](https://www.analyticsvidhya.com/blog/2017/07/covariate-shift-the-hidden-problem-of-real-world-data-science/) I've used before. Since there're class99 in test set, it didn't work very well, but still contribute a little.\n\n--- \n\n## LGB algorithm \n\n* **Modify Objective Function** \nMithrillion has public [a very impressive kernel](https://www.kaggle.com/mithrillion/know-your-objective). But you may find well but not as good as just setting sample weights. So I add sample weights to grad and hess, it works better than built-in objective. This result in my final model structure: \n```\n0.55 * (lgb with customized objective) + 0.45 * (lgb with built-in objective)\n```\nseparated by gal/ext-gal, which means the final model contains 4 lgbs.\n\n\n* **Tune Parameters** \nI tuned parameters to avoid overfitting, small max_depth: 3, small max_bin: 20, large min_child_weight: 10, large min_data_in_leaf: 35. These parameters' setting work well both on CV and LB.\n\n--- \n\n## Final Score \nWhen my CV gets better, there are three things that decreasing the gap between CV and LB: 1) adding salt2 model parameters (improve CV 0.033, but LB 0.058); 2) Remove inappropriate features; 3) Tuning Parameters. Finally, I got CV 0.411, public LB 0.794, gap 0.383, private LB 0.822, gap 0.411.\n\n---\n\n## What I've tried but failed, you may try it better \n\n* **Wavelet Decomposition** \nAgain in [this important paper](https://arxiv.org/abs/1603.00882), part 3.4. Wavelet Decomposition is model-independent compared to salt2 model, but also effective. Three steps: GP -> Wavelet Decomposition -> PCA. I've tried, but failed. You may do it better.\n\n--- \n\n## Useful papers I didn't tried \nHere are links to what I've found may useful \n1) [Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream](https://arxiv.org/abs/1801.07323) \n2) [Deep Recurrent Neural Networks for Supernovae Classification](https://arxiv.org/abs/1606.07442) \n\n---\n\n## Thanks \nAs a novice in kaggle before this competition, I really learn a lot from those who public their impressive kernels, like [ogrellier](https://www.kaggle.com/ogrellier), [mithrillion](https://www.kaggle.com/mithrillion), [kyleboone](https://www.kaggle.com/kyleboone), [meaninglesslives](https://www.kaggle.com/meaninglesslives) ( this is my first time to use keras practicing NN). And [CPMP](https://www.kaggle.com/cpmpml), he gives lots of truly useful tips in discussion. [Giba](https://www.kaggle.com/titericz), his class weights discussion help us a lot . [sionek](https://www.kaggle.com/sionek) released the most famous 'detected_mjd' feature. It's strange that those useful and original kernel have more fork times than upvotes, those truly helpful discussions got only a little upvotes. If you think they are helpful, just upvote. \nBesides, there is an abnormal thing :), [Silogram](https://www.kaggle.com/psilogram) didn't post his famous 'a few notes...' as he did in previous competitions I joined before, like 1) [Home Credit Default Risk](https://www.kaggle.com/c/home-credit-default-risk/discussion/58332); 2) [Google Analytics Customer Revenue Prediction](https://www.kaggle.com/c/ga-customer-revenue-prediction/discussion/67767) . We can learn a lot from his notes, as CPMP do in this competition. \nThey are the real fundamental of kaggle community !!\n \n---\n \n## PS\nI just began to learn python and data science about 2 years ago ( I learn mechanical engineer in college). I think I've got lots luck in this competition, I'm not good enough. When I received teaming up invitations, I regarded them as an honor and encouragement, Thank you !", "votes": null }, { "id": "442030", "postDate": "12/19/2018 11:43:58", "content": "Congrats, nice feature engineering and related ideas.
", "rawMarkdown": "Congrats, nice feature engineering and related ideas.", "votes": null }, { "id": "442104", "postDate": "12/19/2018 13:45:31", "content": "Thanks and congratulations! I don't think I have your first group of features. One of my team mates tried SALT2 but could not make it work. I see that we should have tried harder.
", "rawMarkdown": "Thanks and congratulations! I don't think I have your first group of features. One of my team mates tried SALT2 but could not make it work. I see that we should have tried harder.", "votes": null }, { "id": "442137", "postDate": "12/19/2018 14:20:18", "content": "Thanks for sharing and congratuations!
\n\nI also looked at SALT2 to try it out, but did not recognize comparable passband definitions. Did I just overlook it? What parameters did you use in particular for the passbands?
\n\nI also used the kernel for class 6. That helped (and I up-voted it!). I also thought of using it for extra galactic, but did not have the time any longer to try it it. Interesting that it worked
", "rawMarkdown": "Thanks for sharing and congratuations!\n\nI also looked at SALT2 to try it out, but did not recognize comparable passband definitions. Did I just overlook it? What parameters did you use in particular for the passbands?\n\nI also used the kernel for class 6. That helped (and I up-voted it!). I also thought of using it for extra galactic, but did not have the time any longer to try it it. Interesting that it worked", "votes": null }, { "id": "442139", "postDate": "12/19/2018 14:24:41", "content": "Maybe you have some correlative but more effective features than SALT2 model.
", "rawMarkdown": "Maybe you have some correlative but more effective features than SALT2 model.", "votes": null }, { "id": "442147", "postDate": "12/19/2018 14:35:44", "content": "For SALT2, I set passband map :
\n\n{0: 'lsstu', 1: 'lsstg', 2: 'lsstr', 3: 'lssti', 4: 'lsstz', 5: 'lssty' }\n
\nreferred to this doc
\nAnd set zp = 25 ( I tried 5, 25, 40), zpsys = 'ab'.
\nFor microlensing feature, I first found it's useless for gal model. Then I tried for ext-gal model, it worked.
I'm sure SALT2 would improve our result. Thanks for sharing the parameters above.
", "rawMarkdown": "I'm sure SALT2 would improve our result. Thanks for sharing the parameters above.", "votes": null }, { "id": "442165", "postDate": "12/19/2018 15:02:28", "content": "Thanks, I will try that out again. I had the parameters you describe for zp and zpsys, but do not remember if I tested this for the passbands.
", "rawMarkdown": "Thanks, I will try that out again. I had the parameters you describe for zp and zpsys, but do not remember if I tested this for the passbands.", "votes": null }, { "id": "442167", "postDate": "12/19/2018 15:05:07", "content": "Addition, set z bound:
\n\nbounds={'z':(max([hostgal_photoz - hostgal_photoz_err, 0]), hostgal_photoz + hostgal_photoz_err)}\n
I also read the paper you link concerning the Wavelet transform. This was one of the things in my list I did not try out, but wanted to check. Could you give some more details on how you did the GP and Wavelet Decomposition part?
", "rawMarkdown": "I also read the paper you link concerning the Wavelet transform. This was one of the things in my list I did not try out, but wanted to check. Could you give some more details on how you did the GP and Wavelet Decomposition part?", "votes": null }, { "id": "442187", "postDate": "12/19/2018 15:30:55", "content": "Use celerite pkg for GP, referred to this exmple. And pywavelets pkg for Wavelet Decomposition, referred to these exmples. I've never learned wavelet before, I stuck and can't find a suitable way to do it well. You may do it better !
", "rawMarkdown": "Use celerite pkg for GP, referred to [this exmple](https://celerite.readthedocs.io/en/stable/tutorials/modeling/). And pywavelets pkg for Wavelet Decomposition, referred to [these exmples](https://pywavelets.readthedocs.io/en/latest/regression/dwt-idwt.html#more-examples). I've never learned wavelet before, I stuck and can't find a suitable way to do it well. You may do it better !", "votes": null }, { "id": "442198", "postDate": "12/19/2018 15:46:30", "content": "Thanks for sharing. Interesting feature engineering.
", "rawMarkdown": "Thanks for sharing. Interesting feature engineering.", "votes": null }, { "id": "442205", "postDate": "12/19/2018 15:57:27", "content": "Now that works. I will try to use it in predictions. Thanks for sharing.
", "rawMarkdown": "Now that works. I will try to use it in predictions. Thanks for sharing.", "votes": null }, { "id": "442259", "postDate": "12/19/2018 17:36:15", "content": "Congrats! Very solid feature engineering!
", "rawMarkdown": "Congrats! Very solid feature engineering!", "votes": null }, { "id": "443294", "postDate": "12/21/2018 11:20:17", "content": "Features like\nlog((passband_1_max_flux) / (passband_3_max_flux))-- your first group of features is related to astronomical colors, they define it as g-r, r-i etc, as a difference in abs magnitudes in the corresponding bands. If you look at the formula for absolute magnitudes and recall that (log(a/b) = loga - logb) you can see -- it's colors but without k-correction. We calculated them on the last 6h of competition (better late than never) and they worked :). Just wanted to show you a physical meaning of your wonderful features (I used to work as a physics teacher :-)).
Thank you !! You makes it clear to me. You're a good teacher !
", "rawMarkdown": "Thank you !! You makes it clear to me. You're a good teacher !", "votes": null } ], "comments": [ { "id": 442030, "author_name": "longyin2", "author_url": "", "post_date": "12/19/2018 11:43:58", "content": "Congrats, nice feature engineering and related ideas.
", "votes": null, "replies": [] }, { "id": 442104, "author_name": "cpmpml", "author_url": "", "post_date": "12/19/2018 13:45:31", "content": "Thanks and congratulations! I don't think I have your first group of features. One of my team mates tried SALT2 but could not make it work. I see that we should have tried harder.
", "votes": null, "replies": [ { "id": 442139, "author_name": "chenshaomeng", "author_url": "", "post_date": "12/19/2018 14:24:41", "content": "Maybe you have some correlative but more effective features than SALT2 model.
", "votes": null, "replies": [] }, { "id": 442161, "author_name": "cpmpml", "author_url": "", "post_date": "12/19/2018 14:55:58", "content": "I'm sure SALT2 would improve our result. Thanks for sharing the parameters above.
", "votes": null, "replies": [] } ] }, { "id": 442137, "author_name": "helgith", "author_url": "", "post_date": "12/19/2018 14:20:18", "content": "Thanks for sharing and congratuations!
\n\nI also looked at SALT2 to try it out, but did not recognize comparable passband definitions. Did I just overlook it? What parameters did you use in particular for the passbands?
\n\nI also used the kernel for class 6. That helped (and I up-voted it!). I also thought of using it for extra galactic, but did not have the time any longer to try it it. Interesting that it worked
", "votes": null, "replies": [ { "id": 442147, "author_name": "chenshaomeng", "author_url": "", "post_date": "12/19/2018 14:35:44", "content": "For SALT2, I set passband map :
\n\n{0: 'lsstu', 1: 'lsstg', 2: 'lsstr', 3: 'lssti', 4: 'lsstz', 5: 'lssty' }\n
\nreferred to this doc
\nAnd set zp = 25 ( I tried 5, 25, 40), zpsys = 'ab'.
\nFor microlensing feature, I first found it's useless for gal model. Then I tried for ext-gal model, it worked.
Thanks, I will try that out again. I had the parameters you describe for zp and zpsys, but do not remember if I tested this for the passbands.
", "votes": null, "replies": [] }, { "id": 442167, "author_name": "chenshaomeng", "author_url": "", "post_date": "12/19/2018 15:05:07", "content": "Addition, set z bound:
\n\nbounds={'z':(max([hostgal_photoz - hostgal_photoz_err, 0]), hostgal_photoz + hostgal_photoz_err)}\n
Now that works. I will try to use it in predictions. Thanks for sharing.
", "votes": null, "replies": [] } ] }, { "id": 442168, "author_name": "helgith", "author_url": "", "post_date": "12/19/2018 15:05:26", "content": "I also read the paper you link concerning the Wavelet transform. This was one of the things in my list I did not try out, but wanted to check. Could you give some more details on how you did the GP and Wavelet Decomposition part?
", "votes": null, "replies": [ { "id": 442187, "author_name": "chenshaomeng", "author_url": "", "post_date": "12/19/2018 15:30:55", "content": "Use celerite pkg for GP, referred to this exmple. And pywavelets pkg for Wavelet Decomposition, referred to these exmples. I've never learned wavelet before, I stuck and can't find a suitable way to do it well. You may do it better !
", "votes": null, "replies": [] } ] }, { "id": 442198, "author_name": "andypenrose", "author_url": "", "post_date": "12/19/2018 15:46:30", "content": "Thanks for sharing. Interesting feature engineering.
", "votes": null, "replies": [] }, { "id": 442259, "author_name": "jiweiliu", "author_url": "", "post_date": "12/19/2018 17:36:15", "content": "Congrats! Very solid feature engineering!
", "votes": null, "replies": [] }, { "id": 443294, "author_name": "blondinka", "author_url": "", "post_date": "12/21/2018 11:20:17", "content": "Features like\nlog((passband_1_max_flux) / (passband_3_max_flux))-- your first group of features is related to astronomical colors, they define it as g-r, r-i etc, as a difference in abs magnitudes in the corresponding bands. If you look at the formula for absolute magnitudes and recall that (log(a/b) = loga - logb) you can see -- it's colors but without k-correction. We calculated them on the last 6h of competition (better late than never) and they worked :). Just wanted to show you a physical meaning of your wonderful features (I used to work as a physics teacher :-)).
Thank you !! You makes it clear to me. You're a good teacher !
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "441847": "Here is my brief summary that may be different from others: \n\n--- \n\n## Feature Engineer \n\n* **Features inspired by Starter Kit** \nNot only construct features between adjacent passbands as starter kit showed, like between passband 0 and 1, 1 and 2. But also between passband 0 and 2, 0 and 3, and so on. Besides, not only mean flux, but also try max flux and min flux. Finally, features like \n```\nlog((passband_1_max_flux) / (passband_3_max_flux))\n``` \ncontribute a lot to score. \n\n* **Features I've tried that useful** \n1) 1st quartile and 3rd quartile flux per passband. And their gap value relative to full scale, like \n```\n(passband_0_q3_flux - passband_0_q1_flux) / (passband_0_max_flux - passband_0_min_flux)\n``` \n2) 3rd quartile flux per passband multiply square of hostgal_photoz, like \n```\npassband_0_q3_flux * (hostgal_photoz**2) \n(passband_0_q3_flux - passband_0_q1_flux) * (hostgal_photoz**2)\n``` \nThose features work well on my model. \n\n* **Features from cesium and feets packages** \nperiod_fast feature per passband from cesium pkg and CAR feature per passband from feets pkg, made some contribution to my model.\n\n* **Features referred to papers** \n[Here is the very important paper I've read and referred to](https://arxiv.org/abs/1603.00882). In this paper, authors introduced several method to extract light curve parameters for classification. I referred to the most effective method of them -- salt2 model, in part 3.2 of the paper. Fortunately, you can use [sncosmo api](https://sncosmo.readthedocs.io/en/v1.6.x/examples/plot_lc_fit.html#sphx-glr-examples-plot-lc-fit-py) to fit salt2 model and get model parameters as features. \n\n* **Features referred to github** \nBazin function, this method is same with what [4th solution](https://www.kaggle.com/c/PLAsTiCC-2018/discussion/75011) has metioned, but I found it by this [link](https://github.com/ramp-kits/supernovae/blob/master/PLAsTiCC_starting_kit.ipynb). I've tried modify this method, but the result shows no big different.\n\n* **Features referred to a public kernel** \n[This public kernel is very impressive](https://www.kaggle.com/manugangler/optimal-feature-extraction-for-class-6), but only got a few upvotes. Instead of use it as hostgal_photoz == 0 ( author set in the kernel), I use it for ext-gal model with a set hostgal_photoz > 0. It really improved my model. \n\n* **Features selection** \nIn order to remove some inappropriate features, I do it simple, observing some less important features' distribution on train and test. Use [this method](https://www.analyticsvidhya.com/blog/2017/07/covariate-shift-the-hidden-problem-of-real-world-data-science/) I've used before. Since there're class99 in test set, it didn't work very well, but still contribute a little.\n\n--- \n\n## LGB algorithm \n\n* **Modify Objective Function** \nMithrillion has public [a very impressive kernel](https://www.kaggle.com/mithrillion/know-your-objective). But you may find well but not as good as just setting sample weights. So I add sample weights to grad and hess, it works better than built-in objective. This result in my final model structure: \n```\n0.55 * (lgb with customized objective) + 0.45 * (lgb with built-in objective)\n```\nseparated by gal/ext-gal, which means the final model contains 4 lgbs.\n\n\n* **Tune Parameters** \nI tuned parameters to avoid overfitting, small max_depth: 3, small max_bin: 20, large min_child_weight: 10, large min_data_in_leaf: 35. These parameters' setting work well both on CV and LB.\n\n--- \n\n## Final Score \nWhen my CV gets better, there are three things that decreasing the gap between CV and LB: 1) adding salt2 model parameters (improve CV 0.033, but LB 0.058); 2) Remove inappropriate features; 3) Tuning Parameters. Finally, I got CV 0.411, public LB 0.794, gap 0.383, private LB 0.822, gap 0.411.\n\n---\n\n## What I've tried but failed, you may try it better \n\n* **Wavelet Decomposition** \nAgain in [this important paper](https://arxiv.org/abs/1603.00882), part 3.4. Wavelet Decomposition is model-independent compared to salt2 model, but also effective. Three steps: GP -> Wavelet Decomposition -> PCA. I've tried, but failed. You may do it better.\n\n--- \n\n## Useful papers I didn't tried \nHere are links to what I've found may useful \n1) [Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream](https://arxiv.org/abs/1801.07323) \n2) [Deep Recurrent Neural Networks for Supernovae Classification](https://arxiv.org/abs/1606.07442) \n\n---\n\n## Thanks \nAs a novice in kaggle before this competition, I really learn a lot from those who public their impressive kernels, like [ogrellier](https://www.kaggle.com/ogrellier), [mithrillion](https://www.kaggle.com/mithrillion), [kyleboone](https://www.kaggle.com/kyleboone), [meaninglesslives](https://www.kaggle.com/meaninglesslives) ( this is my first time to use keras practicing NN). And [CPMP](https://www.kaggle.com/cpmpml), he gives lots of truly useful tips in discussion. [Giba](https://www.kaggle.com/titericz), his class weights discussion help us a lot . [sionek](https://www.kaggle.com/sionek) released the most famous 'detected_mjd' feature. It's strange that those useful and original kernel have more fork times than upvotes, those truly helpful discussions got only a little upvotes. If you think they are helpful, just upvote. \nBesides, there is an abnormal thing :), [Silogram](https://www.kaggle.com/psilogram) didn't post his famous 'a few notes...' as he did in previous competitions I joined before, like 1) [Home Credit Default Risk](https://www.kaggle.com/c/home-credit-default-risk/discussion/58332); 2) [Google Analytics Customer Revenue Prediction](https://www.kaggle.com/c/ga-customer-revenue-prediction/discussion/67767) . We can learn a lot from his notes, as CPMP do in this competition. \nThey are the real fundamental of kaggle community !!\n \n---\n \n## PS\nI just began to learn python and data science about 2 years ago ( I learn mechanical engineer in college). I think I've got lots luck in this competition, I'm not good enough. When I received teaming up invitations, I regarded them as an honor and encouragement, Thank you !", "442030": "Congrats, nice feature engineering and related ideas.", "442104": "Thanks and congratulations! I don't think I have your first group of features. One of my team mates tried SALT2 but could not make it work. I see that we should have tried harder.", "442137": "Thanks for sharing and congratuations!\n\nI also looked at SALT2 to try it out, but did not recognize comparable passband definitions. Did I just overlook it? What parameters did you use in particular for the passbands?\n\nI also used the kernel for class 6. That helped (and I up-voted it!). I also thought of using it for extra galactic, but did not have the time any longer to try it it. Interesting that it worked", "442139": "Maybe you have some correlative but more effective features than SALT2 model.", "442147": "For SALT2, I set passband map : \n```\n{0: 'lsstu', 1: 'lsstg', 2: 'lsstr', 3: 'lssti', 4: 'lsstz', 5: 'lssty' }\n``` \nreferred to [this doc](https://sncosmo.readthedocs.io/en/v1.6.x/bandpass-list.html) \nAnd set zp = 25 ( I tried 5, 25, 40), zpsys = 'ab'. \nFor microlensing feature, I first found it's useless for gal model. Then I tried for ext-gal model, it worked.", "442161": "I'm sure SALT2 would improve our result. Thanks for sharing the parameters above.", "442165": "Thanks, I will try that out again. I had the parameters you describe for zp and zpsys, but do not remember if I tested this for the passbands.", "442167": "Addition, set z bound: \n```\nbounds={'z':(max([hostgal_photoz - hostgal_photoz_err, 0]), hostgal_photoz + hostgal_photoz_err)}\n```", "442168": "I also read the paper you link concerning the Wavelet transform. This was one of the things in my list I did not try out, but wanted to check. Could you give some more details on how you did the GP and Wavelet Decomposition part?", "442187": "Use celerite pkg for GP, referred to [this exmple](https://celerite.readthedocs.io/en/stable/tutorials/modeling/). And pywavelets pkg for Wavelet Decomposition, referred to [these exmples](https://pywavelets.readthedocs.io/en/latest/regression/dwt-idwt.html#more-examples). I've never learned wavelet before, I stuck and can't find a suitable way to do it well. You may do it better !", "442198": "Thanks for sharing. Interesting feature engineering.", "442205": "Now that works. I will try to use it in predictions. Thanks for sharing.", "442259": "Congrats! Very solid feature engineering!", "443294": "```Features like\nlog((passband_1_max_flux) / (passband_3_max_flux)) ```-- your first group of features is related to astronomical colors, they define it as g-r, r-i etc, as a difference in abs magnitudes in the corresponding bands. If you look at the formula for absolute magnitudes and recall that (log(a/b) = loga - logb) you can see -- it's colors but without k-correction. We calculated them on the last 6h of competition (better late than never) and they worked :). Just wanted to show you a physical meaning of your wonderful features (I used to work as a physics teacher :-)).", "443414": "Thank you !! You makes it clear to me. You're a good teacher !" }, "source": "meta" }