![\"model](\"https://storage.googleapis.com/kaggle-forum-message-attachments/441823/10901/model_pipeline_compreessed.png\")
We used weighted multi logloss(same as kernel) and multi logloss.\nAfter predicted by weighted multi logloss, calculated weight for each class using gradient descent .\nFor multi logloss, here.
\n\n
We used weighted multi logloss(same as kernel) and multi logloss.\nAfter predicted by weighted multi logloss, calculated weight for each class using gradient descent .\nFor multi logloss, here.
\n\n
Congratulations and thanks for sharing your solution with a very nice infographic.
\n\nI see that you used XGB. I was also always using XGB along with LightGBM, and saw that XGB performed consistently bad compared to LightGBM. Given that XGB was much slower than the other, I found it very annoying that I had to wait longer and get a worse score. \nDid you also have a same sort of inferior performance with XGB? If not, could you share what was the differences in the hyperparameters of the LighGBM and XGB, or how did you tune the hyperparameters for both models.
", "rawMarkdown": "Congratulations and thanks for sharing your solution with a very nice infographic. \n\nI see that you used XGB. I was also always using XGB along with LightGBM, and saw that XGB performed consistently bad compared to LightGBM. Given that XGB was much slower than the other, I found it very annoying that I had to wait longer and get a worse score. \nDid you also have a same sort of inferior performance with XGB? If not, could you share what was the differences in the hyperparameters of the LighGBM and XGB, or how did you tune the hyperparameters for both models.", "votes": null }, { "id": "442044", "postDate": "12/19/2018 12:07:17", "content": "Your feature engineering is quite great, thanks for sharing.
", "rawMarkdown": "Your feature engineering is quite great, thanks for sharing.", "votes": null }, { "id": "442463", "postDate": "12/20/2018 01:50:42", "content": "Thanks, Vig.
\n\nIn our case, XGB performed better than LGB, the score gap was ~0.01. Certainly the learning speed was slower than LGB, but I did not feel so stressful because I tried it with a higher learning rate. (I noteced it now that I forgot to decrease the learning rate the time of submission :D) As the ensemble effect of XGB and LGB was low, it may be better to simply use XGB in this competition.
\n\nWhen I started using XGB was two days before the competition deadline and I searched parameters manually, so maybe my parameters is not the best. Below is my parameters (external galaxtic model).
\n\n\nlgb_params = {\n 'application': 'multiclass',
\n 'num_class': eg_classes.shape[0],
\n 'metric': 'multi_logloss',
\n 'learning_rate': 0.03,
\n 'max_depth': 5,
\n 'num_leaves': 63,
\n 'max_bin': 127,
\n 'min_child_weight': 10,
\n 'min_data_in_leaf': 100,
\n 'reg_lambda': 0.01,
\n 'reg_alpha': 0.01,
\n 'colsample_bytree': 0.4,
\n 'subsample': 0.7,
\n 'bagging_freq': 1,
\n}
\nxgb_params = {. \n 'objective': 'multi:softprob',
\n 'eval_metric': 'mlogloss',
\n 'num_class': eg_classes.shape[0],
\n 'tree_method': 'exact',
\n 'learning_rate' : 0.1,
\n 'max_depth' : 4,
\n 'subsample': .7,
\n 'colsample_bytree': .3,
\n 'reg_alpha': .01,
\n 'reg_lambda': .01,
\n 'min_split_loss': 0.02,
\n 'min_child_weight': 10,
\n}. \n
Thanks very much for your reply.
", "rawMarkdown": "Thanks very much for your reply.", "votes": null } ], "comments": [ { "id": 441956, "author_name": "vignam", "author_url": "", "post_date": "12/19/2018 09:39:06", "content": "Congratulations and thanks for sharing your solution with a very nice infographic.
\n\nI see that you used XGB. I was also always using XGB along with LightGBM, and saw that XGB performed consistently bad compared to LightGBM. Given that XGB was much slower than the other, I found it very annoying that I had to wait longer and get a worse score. \nDid you also have a same sort of inferior performance with XGB? If not, could you share what was the differences in the hyperparameters of the LighGBM and XGB, or how did you tune the hyperparameters for both models.
", "votes": null, "replies": [ { "id": 442463, "author_name": "mtfall", "author_url": "", "post_date": "12/20/2018 01:50:42", "content": "Thanks, Vig.
\n\nIn our case, XGB performed better than LGB, the score gap was ~0.01. Certainly the learning speed was slower than LGB, but I did not feel so stressful because I tried it with a higher learning rate. (I noteced it now that I forgot to decrease the learning rate the time of submission :D) As the ensemble effect of XGB and LGB was low, it may be better to simply use XGB in this competition.
\n\nWhen I started using XGB was two days before the competition deadline and I searched parameters manually, so maybe my parameters is not the best. Below is my parameters (external galaxtic model).
\n\n\nlgb_params = {\n 'application': 'multiclass',
\n 'num_class': eg_classes.shape[0],
\n 'metric': 'multi_logloss',
\n 'learning_rate': 0.03,
\n 'max_depth': 5,
\n 'num_leaves': 63,
\n 'max_bin': 127,
\n 'min_child_weight': 10,
\n 'min_data_in_leaf': 100,
\n 'reg_lambda': 0.01,
\n 'reg_alpha': 0.01,
\n 'colsample_bytree': 0.4,
\n 'subsample': 0.7,
\n 'bagging_freq': 1,
\n}
\nxgb_params = {. \n 'objective': 'multi:softprob',
\n 'eval_metric': 'mlogloss',
\n 'num_class': eg_classes.shape[0],
\n 'tree_method': 'exact',
\n 'learning_rate' : 0.1,
\n 'max_depth' : 4,
\n 'subsample': .7,
\n 'colsample_bytree': .3,
\n 'reg_alpha': .01,
\n 'reg_lambda': .01,
\n 'min_split_loss': 0.02,
\n 'min_child_weight': 10,
\n}. \n
Thanks very much for your reply.
", "votes": null, "replies": [] } ] }, { "id": 442044, "author_name": "longyin2", "author_url": "", "post_date": "12/19/2018 12:07:17", "content": "Your feature engineering is quite great, thanks for sharing.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "441823": "# Final stand\n- Public LB: 0.838\n- Private LB: 0.851\n\n# Features\n- aggregate\n- focused on peak\n- focused on detected\n- Luminosity\n- LC fitter\n\n# Models\n- LGB\n- XGB\n- MLP\n\n# Post process\nWe used weighted multi logloss(same as kernel) and multi logloss.\nAfter predicted by weighted multi logloss, calculated weight for each class using [gradient descent][1] .\nFor multi logloss, [here][2].\n\n# Did not work\n- AE\n- Augmentation\n\n![model pipeline][3]\n\n\n [1]: https://github.com/KazukiOnodera/PLAsTiCC-2018/blob/master/py/utils_post.py\n [2]: https://github.com/KazukiOnodera/PLAsTiCC-2018/blob/master/py/utils.py#L400-L406\n [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/441823/10901/model_pipeline_compreessed.png", "441956": "Congratulations and thanks for sharing your solution with a very nice infographic. \n\nI see that you used XGB. I was also always using XGB along with LightGBM, and saw that XGB performed consistently bad compared to LightGBM. Given that XGB was much slower than the other, I found it very annoying that I had to wait longer and get a worse score. \nDid you also have a same sort of inferior performance with XGB? If not, could you share what was the differences in the hyperparameters of the LighGBM and XGB, or how did you tune the hyperparameters for both models.", "442044": "Your feature engineering is quite great, thanks for sharing.", "442463": "Thanks, Vig. \n\nIn our case, XGB performed better than LGB, the score gap was ~0.01. Certainly the learning speed was slower than LGB, but I did not feel so stressful because I tried it with a higher learning rate. (I noteced it now that I forgot to decrease the learning rate the time of submission :D) As the ensemble effect of XGB and LGB was low, it may be better to simply use XGB in this competition. \n\nWhen I started using XGB was two days before the competition deadline and I searched parameters manually, so maybe my parameters is not the best. Below is my parameters (external galaxtic model). \n\n```\nlgb_params = {\n\t'application': 'multiclass', \n\t'num_class': eg_classes.shape[0], \n\t'metric': 'multi_logloss', \n\t'learning_rate': 0.03, \n\t'max_depth': 5, \n\t'num_leaves': 63, \n\t'max_bin': 127, \n\t'min_child_weight': 10, \n\t'min_data_in_leaf': 100, \n\t'reg_lambda': 0.01, \n\t'reg_alpha': 0.01, \n\t'colsample_bytree': 0.4, \n\t'subsample': 0.7, \n\t'bagging_freq': 1, \n} \nxgb_params = {. \n\t'objective': 'multi:softprob', \n\t'eval_metric': 'mlogloss', \n\t'num_class': eg_classes.shape[0], \n\t'tree_method': 'exact', \n\t'learning_rate' : 0.1, \n\t'max_depth' : 4, \n\t'subsample': .7, \n\t'colsample_bytree': .3, \n\t'reg_alpha': .01, \n\t'reg_lambda': .01, \n\t'min_split_loss': 0.02, \n\t'min_child_weight': 10, \n}. \n```", "442650": "Thanks very much for your reply." }, "source": "meta" }