Congrats winners!
\n\nI would like to share my features briefly before I forget it.
\n\nI used BigQuery on the almost all feature engineering parts.
\n https://gist.github.com/tkm2261/1b3c3c37753e55ed2914577c0f96d222
It takes only about 20 minutes. very fast!
\n\nThere may not be surprising features if you have seen the shared kernels. I tried to find good features by brute force way.
\n\nCPU: 16 cores, MEM: 100GB on GCP
\n\nI used validation data to determin the boosting round. Then, I used train+valid data for training.
\n\nSimply, I used LightGBM and ensembled with different seeds.
\n\nThe single best model is 0.9823 on the public LB.
\n\nThe parameter is here:
\n\n{'colsample_bytree': 0.6, 'learning_rate': 0.1, 'max_bin': 1023, 'max_depth': -1, 'metric': ['binary_logloss', 'auc'], 'min_child_weight': 30, 'min_split_gain': 0.0001, 'num_leaves': 127, 'objective': 'binary', 'reg_alpha': 0, 'scale_pos_weight': 1, 'seed': 1142, 'subsample': 0.9, 'subsample_freq': 1, 'verbose': -1}\nsource code is here: https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/train_lgb.py
\n\nI tried to earn my tuition for MS CS program starting next fall. But, there was no free lunch!
", "messages": [ { "id": "324921", "postDate": "05/08/2018 01:40:19", "content": "Congrats winners!
\n\nI would like to share my features briefly before I forget it.
\n\nI used BigQuery on the almost all feature engineering parts.
\n https://gist.github.com/tkm2261/1b3c3c37753e55ed2914577c0f96d222
It takes only about 20 minutes. very fast!
\n\nThere may not be surprising features if you have seen the shared kernels. I tried to find good features by brute force way.
\n\nCPU: 16 cores, MEM: 100GB on GCP
\n\nI used validation data to determin the boosting round. Then, I used train+valid data for training.
\n\nSimply, I used LightGBM and ensembled with different seeds.
\n\nThe single best model is 0.9823 on the public LB.
\n\nThe parameter is here:
\n\n{'colsample_bytree': 0.6, 'learning_rate': 0.1, 'max_bin': 1023, 'max_depth': -1, 'metric': ['binary_logloss', 'auc'], 'min_child_weight': 30, 'min_split_gain': 0.0001, 'num_leaves': 127, 'objective': 'binary', 'reg_alpha': 0, 'scale_pos_weight': 1, 'seed': 1142, 'subsample': 0.9, 'subsample_freq': 1, 'verbose': -1}\nsource code is here: https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/train_lgb.py
\n\nI tried to earn my tuition for MS CS program starting next fall. But, there was no free lunch!
", "rawMarkdown": "Congrats winners!\n\nI would like to share my features briefly before I forget it. \n\n### Features\n\nI used BigQuery on the almost all feature engineering parts. \n https://gist.github.com/tkm2261/1b3c3c37753e55ed2914577c0f96d222\n\nIt takes only about 20 minutes. very fast!\n\nThere may not be surprising features if you have seen the shared kernels. I tried to find good features by brute force way.\n\n### Machine\n\nCPU: 16 cores, MEM: 100GB on GCP\n\n### Data\n\n* training data: day 7 and day 8 (13,188,695,398 rows)\n* validation data: day 9 (53,016,937 rows)\n* test data: day 10 (18,790,469 rows)\n\nI used validation data to determin the boosting round. Then, I used train+valid data for training.\n\n### Training\n\nSimply, I used LightGBM and ensembled with different seeds.\n\nThe single best model is 0.9823 on the public LB.\n\nThe parameter is here: \n\n {'colsample_bytree': 0.6, 'learning_rate': 0.1, 'max_bin': 1023, 'max_depth': -1, 'metric': ['binary_logloss', 'auc'], 'min_child_weight': 30, 'min_split_gain': 0.0001, 'num_leaves': 127, 'objective': 'binary', 'reg_alpha': 0, 'scale_pos_weight': 1, 'seed': 1142, 'subsample': 0.9, 'subsample_freq': 1, 'verbose': -1}\n\nsource code is here: https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/train_lgb.py\n\nI tried to earn my tuition for MS CS program starting next fall. But, there was no free lunch!", "votes": null }, { "id": "324925", "postDate": "05/08/2018 01:42:23", "content": "Good game and well played!
", "rawMarkdown": "Good game and well played!", "votes": null }, { "id": "324929", "postDate": "05/08/2018 01:44:21", "content": "Congrats and thanks for sharing~
", "rawMarkdown": "Congrats and thanks for sharing~", "votes": null }, { "id": "324936", "postDate": "05/08/2018 01:54:36", "content": "Cool! Thanks for sharing! I hope you do well in your MS program!
", "rawMarkdown": "Cool! Thanks for sharing! I hope you do well in your MS program!", "votes": null }, { "id": "324943", "postDate": "05/08/2018 02:01:50", "content": "Congrats and thanks for sharing. so first step is make train csv file to sql dataset.
", "rawMarkdown": "Congrats and thanks for sharing. so first step is make train csv file to sql dataset.", "votes": null }, { "id": "324949", "postDate": "05/08/2018 02:06:15", "content": "Thanks for sharing.
", "rawMarkdown": "Thanks for sharing.", "votes": null }, { "id": "324957", "postDate": "05/08/2018 02:15:32", "content": "Congrats and thanks for sharing!
\nAnd may I ask here
\n\nI used validation data to determin the boosting round. Then, I used train+valid data for training.\n
\nThat means you use the same parameters and rounds you get in validation to retrain the model without using validation in retraining, is that right?
\nTHANKS.
'scale_pos_weight': 1 ? I have not seen anyone using such a small number.\nCongratulation on your accomplishment!
", "rawMarkdown": "'scale_pos_weight': 1 ? I have not seen anyone using such a small number.\nCongratulation on your accomplishment!", "votes": null }, { "id": "324966", "postDate": "05/08/2018 02:22:18", "content": "I mean the validation data is used like fold-out data.
", "rawMarkdown": "I mean the validation data is used like fold-out data.", "votes": null }, { "id": "324975", "postDate": "05/08/2018 02:29:33", "content": "I handle the imbalance by min_sample_weight, num_leaves, and other parameters. Since the eval metric is AUC, the scale is not important.
", "rawMarkdown": "I handle the imbalance by min_sample_weight, num_leaves, and other parameters. Since the eval metric is AUC, the scale is not important.", "votes": null }, { "id": "324978", "postDate": "05/08/2018 02:31:31", "content": "Thank you for sharing! I'll try BigQuery
", "rawMarkdown": "Thank you for sharing! I'll try BigQuery", "votes": null }, { "id": "325006", "postDate": "05/08/2018 03:16:02", "content": "Nice Work!
", "rawMarkdown": "Nice Work!", "votes": null }, { "id": "325018", "postDate": "05/08/2018 03:23:41", "content": "just wondering about whether using validation(or early_stopping) in retraining or not. thx
", "rawMarkdown": "just wondering about whether using validation(or early_stopping) in retraining or not. thx", "votes": null }, { "id": "325025", "postDate": "05/08/2018 03:30:07", "content": "haha, i am going for MS CS , in the coming fall ! how did you come up with the hyperparameters
", "rawMarkdown": "haha, i am going for MS CS , in the coming fall ! how did you come up with the hyperparameters", "votes": null }, { "id": "325028", "postDate": "05/08/2018 03:33:03", "content": "One quick question: do you refit on the whole set after get the best number of trees from the validation?
", "rawMarkdown": "One quick question: do you refit on the whole set after get the best number of trees from the validation?", "votes": null }, { "id": "325034", "postDate": "05/08/2018 03:42:59", "content": "Wow that's so fast to build all those features, thanks for sharing & congrats! I never knew it can be so much faster using BiqQuery, I only use VM instances for this competittion.
", "rawMarkdown": "Wow that's so fast to build all those features, thanks for sharing & congrats! I never knew it can be so much faster using BiqQuery, I only use VM instances for this competittion.", "votes": null }, { "id": "325061", "postDate": "05/08/2018 04:38:00", "content": "Brilliant, \nThanks for sharing.
", "rawMarkdown": "Brilliant, \nThanks for sharing.", "votes": null }, { "id": "325090", "postDate": "05/08/2018 05:30:14", "content": "great solution! most suitable for production
", "rawMarkdown": "great solution! most suitable for production", "votes": null }, { "id": "325094", "postDate": "05/08/2018 05:44:19", "content": "Congrats! Vote for the \"next_click\" part, you've done way more than I did. Thanks for sharing.
", "rawMarkdown": "Congrats! Vote for the \"next_click\" part, you've done way more than I did. Thanks for sharing.", "votes": null }, { "id": "325097", "postDate": "05/08/2018 05:48:38", "content": "Could you tell a bit about tuning of the hyper-parameters, if you did any?
", "rawMarkdown": "Could you tell a bit about tuning of the hyper-parameters, if you did any?", "votes": null }, { "id": "325099", "postDate": "05/08/2018 05:54:12", "content": "yes, I refit.
", "rawMarkdown": "yes, I refit.", "votes": null }, { "id": "325100", "postDate": "05/08/2018 05:54:57", "content": "I used the validation data in retraining.
", "rawMarkdown": "I used the validation data in retraining.", "votes": null }, { "id": "325123", "postDate": "05/08/2018 06:30:33", "content": "Congrats.
Did you skip day 8 from your training set?
sorry, day 8 is included in my training data.
", "rawMarkdown": "sorry, day 8 is included in my training data.", "votes": null }, { "id": "325137", "postDate": "05/08/2018 06:51:37", "content": "May I ask if you made features for each day separately? Like below
\n\nday7 = make_features(day7)\n...\n...\nday9 = make_features(day9)\n\nX_train = pd.concat([day7,day8])\nX_val = day9[hours[4,11]]\nor
\n\nall_days = make_features(all_days)\nX_train = all_days[days[7,8]]\nX_val = all_days[day[9]]\nthe latter is correct. plz see my sql gist for more detail.
", "rawMarkdown": "the latter is correct. plz see my sql gist for more detail.", "votes": null }, { "id": "325190", "postDate": "05/08/2018 07:55:43", "content": "Something still confuses me, so I try to make it clear.
\n1.You use train data and valid data to train models->find the best parameters and rounds.
\n2.You use the parameters and rounds in 1 and the whole dataset to retrain a model, in this process you won't split the whole dataset for validation, and just train the number of rounds in 1 without using early stopping.
\nAm I get the right thing here?
maybe right. if you want to learn more, please see my code: https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/train_lgb.py
", "rawMarkdown": "maybe right. if you want to learn more, please see my code: https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/train_lgb.py", "votes": null }, { "id": "325245", "postDate": "05/08/2018 08:56:16", "content": "Cool, I think I got what I want in your code, many thanks.
", "rawMarkdown": "Cool, I think I got what I want in your code, many thanks.", "votes": null }, { "id": "325248", "postDate": "05/08/2018 08:59:36", "content": "Well done, you have been the lead solo competitor till the last few days if I remember well, chasing you during the competition helped me a lot!
", "rawMarkdown": "Well done, you have been the lead solo competitor till the last few days if I remember well, chasing you during the competition helped me a lot!", "votes": null }, { "id": "325285", "postDate": "05/08/2018 09:22:47", "content": "Well done tkm2261!\nAfter reading all post, I realized I completely missed that it was valuable to split data before and after day 9 and 10. \nAnd thanks for sharing the code! tkm2261, I am surprised changing the seeds plays a role. I will try in the next competition. And good luck for your MS.
", "rawMarkdown": "Well done tkm2261!\nAfter reading all post, I realized I completely missed that it was valuable to split data before and after day 9 and 10. \nAnd thanks for sharing the code! tkm2261, I am surprised changing the seeds plays a role. I will try in the next competition. And good luck for your MS.", "votes": null }, { "id": "329250", "postDate": "05/16/2018 05:03:39", "content": "Thanks for sharing. 2 questions regarding to your solution:
\n\nThanks
", "rawMarkdown": "Thanks for sharing. 2 questions regarding to your solution:\n\n1. All data before day 9 should have 131886953 rows and all day 9 data should have 53016937 rows. In your post looks like they have ~ 2 million rows less, did you filter out some data? Duplication? \n2. In your load data code https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/load_data.py#L43 what does \"prev\" data set refer to? Looks like it is not part of your queries. \n\nThanks", "votes": null }, { "id": "329305", "postDate": "05/16/2018 07:35:48", "content": "Thanks for questioning!
\n1. You are right. I copied these numbers from the log when I deleted duplicated rows by mistake. And, deleting duplicated rows did not improve my score.
\n2. At first, I used only day 8 data as train data. Then, I added day 7 data that I called \"prev data\". So, in my query, prev and train are merged.
Gotcha! Thanks for answering these questions. I'm on the half way of reproducing your best single model.
\n\nI realized you excluded several features from BigQuery results(https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/load_data.py#L10), how did you do the feature selection? I'm also curious how you find the best param of lightGBM, from your code, looks like you did some manual tuning?
", "rawMarkdown": "Gotcha! Thanks for answering these questions. I'm on the half way of reproducing your best single model. \n\nI realized you excluded several features from BigQuery results(https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/load_data.py#L10), how did you do the feature selection? I'm also curious how you find the best param of lightGBM, from your code, looks like you did some manual tuning?", "votes": null }, { "id": "329316", "postDate": "05/16/2018 08:11:07", "content": "Thanks for sharing. Really helpful.
", "rawMarkdown": "Thanks for sharing. Really helpful.", "votes": null } ], "comments": [ { "id": 324925, "author_name": "wythhh", "author_url": "", "post_date": "05/08/2018 01:42:23", "content": "Good game and well played!
", "votes": null, "replies": [] }, { "id": 324929, "author_name": "laevatein", "author_url": "", "post_date": "05/08/2018 01:44:21", "content": "Congrats and thanks for sharing~
", "votes": null, "replies": [] }, { "id": 324936, "author_name": "kenkoooo", "author_url": "", "post_date": "05/08/2018 01:54:36", "content": "Cool! Thanks for sharing! I hope you do well in your MS program!
", "votes": null, "replies": [] }, { "id": 324943, "author_name": "liuhdsgoal", "author_url": "", "post_date": "05/08/2018 02:01:50", "content": "Congrats and thanks for sharing. so first step is make train csv file to sql dataset.
", "votes": null, "replies": [] }, { "id": 324949, "author_name": "sheriytm", "author_url": "", "post_date": "05/08/2018 02:06:15", "content": "Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 324957, "author_name": "creamiracle", "author_url": "", "post_date": "05/08/2018 02:15:32", "content": "Congrats and thanks for sharing!
\nAnd may I ask here
\n\nI used validation data to determin the boosting round. Then, I used train+valid data for training.\n
\nThat means you use the same parameters and rounds you get in validation to retrain the model without using validation in retraining, is that right?
\nTHANKS.
I mean the validation data is used like fold-out data.
", "votes": null, "replies": [] }, { "id": 325018, "author_name": "creamiracle", "author_url": "", "post_date": "05/08/2018 03:23:41", "content": "just wondering about whether using validation(or early_stopping) in retraining or not. thx
", "votes": null, "replies": [] }, { "id": 325100, "author_name": "tkm2261", "author_url": "", "post_date": "05/08/2018 05:54:57", "content": "I used the validation data in retraining.
", "votes": null, "replies": [] }, { "id": 325190, "author_name": "creamiracle", "author_url": "", "post_date": "05/08/2018 07:55:43", "content": "Something still confuses me, so I try to make it clear.
\n1.You use train data and valid data to train models->find the best parameters and rounds.
\n2.You use the parameters and rounds in 1 and the whole dataset to retrain a model, in this process you won't split the whole dataset for validation, and just train the number of rounds in 1 without using early stopping.
\nAm I get the right thing here?
maybe right. if you want to learn more, please see my code: https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/train_lgb.py
", "votes": null, "replies": [] }, { "id": 325245, "author_name": "creamiracle", "author_url": "", "post_date": "05/08/2018 08:56:16", "content": "Cool, I think I got what I want in your code, many thanks.
", "votes": null, "replies": [] } ] }, { "id": 324962, "author_name": "tapioca", "author_url": "", "post_date": "05/08/2018 02:17:55", "content": "'scale_pos_weight': 1 ? I have not seen anyone using such a small number.\nCongratulation on your accomplishment!
", "votes": null, "replies": [ { "id": 324975, "author_name": "tkm2261", "author_url": "", "post_date": "05/08/2018 02:29:33", "content": "I handle the imbalance by min_sample_weight, num_leaves, and other parameters. Since the eval metric is AUC, the scale is not important.
", "votes": null, "replies": [] } ] }, { "id": 324978, "author_name": "takpme", "author_url": "", "post_date": "05/08/2018 02:31:31", "content": "Thank you for sharing! I'll try BigQuery
", "votes": null, "replies": [] }, { "id": 325006, "author_name": "fufufukakaka", "author_url": "", "post_date": "05/08/2018 03:16:02", "content": "Nice Work!
", "votes": null, "replies": [] }, { "id": 325025, "author_name": "nick7hill", "author_url": "", "post_date": "05/08/2018 03:30:07", "content": "haha, i am going for MS CS , in the coming fall ! how did you come up with the hyperparameters
", "votes": null, "replies": [] }, { "id": 325028, "author_name": "chengju", "author_url": "", "post_date": "05/08/2018 03:33:03", "content": "One quick question: do you refit on the whole set after get the best number of trees from the validation?
", "votes": null, "replies": [ { "id": 325099, "author_name": "tkm2261", "author_url": "", "post_date": "05/08/2018 05:54:12", "content": "yes, I refit.
", "votes": null, "replies": [] } ] }, { "id": 325034, "author_name": "sdarmawan", "author_url": "", "post_date": "05/08/2018 03:42:59", "content": "Wow that's so fast to build all those features, thanks for sharing & congrats! I never knew it can be so much faster using BiqQuery, I only use VM instances for this competittion.
", "votes": null, "replies": [] }, { "id": 325061, "author_name": "a45632", "author_url": "", "post_date": "05/08/2018 04:38:00", "content": "Brilliant, \nThanks for sharing.
", "votes": null, "replies": [] }, { "id": 325090, "author_name": "izmaylov", "author_url": "", "post_date": "05/08/2018 05:30:14", "content": "great solution! most suitable for production
", "votes": null, "replies": [] }, { "id": 325094, "author_name": "niuddd", "author_url": "", "post_date": "05/08/2018 05:44:19", "content": "Congrats! Vote for the \"next_click\" part, you've done way more than I did. Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 325097, "author_name": "burhanusm", "author_url": "", "post_date": "05/08/2018 05:48:38", "content": "Could you tell a bit about tuning of the hyper-parameters, if you did any?
", "votes": null, "replies": [] }, { "id": 325123, "author_name": "sohaibomar", "author_url": "", "post_date": "05/08/2018 06:30:33", "content": "Congrats.
Did you skip day 8 from your training set?
sorry, day 8 is included in my training data.
", "votes": null, "replies": [] }, { "id": 325137, "author_name": "sohaibomar", "author_url": "", "post_date": "05/08/2018 06:51:37", "content": "May I ask if you made features for each day separately? Like below
\n\nday7 = make_features(day7)\n...\n...\nday9 = make_features(day9)\n\nX_train = pd.concat([day7,day8])\nX_val = day9[hours[4,11]]\nor
\n\nall_days = make_features(all_days)\nX_train = all_days[days[7,8]]\nX_val = all_days[day[9]]\nthe latter is correct. plz see my sql gist for more detail.
", "votes": null, "replies": [] } ] }, { "id": 325248, "author_name": "cpmpml", "author_url": "", "post_date": "05/08/2018 08:59:36", "content": "Well done, you have been the lead solo competitor till the last few days if I remember well, chasing you during the competition helped me a lot!
", "votes": null, "replies": [] }, { "id": 325285, "author_name": "ericbenhamou", "author_url": "", "post_date": "05/08/2018 09:22:47", "content": "Well done tkm2261!\nAfter reading all post, I realized I completely missed that it was valuable to split data before and after day 9 and 10. \nAnd thanks for sharing the code! tkm2261, I am surprised changing the seeds plays a role. I will try in the next competition. And good luck for your MS.
", "votes": null, "replies": [] }, { "id": 329250, "author_name": "soundwaveli00", "author_url": "", "post_date": "05/16/2018 05:03:39", "content": "Thanks for sharing. 2 questions regarding to your solution:
\n\nThanks
", "votes": null, "replies": [ { "id": 329305, "author_name": "tkm2261", "author_url": "", "post_date": "05/16/2018 07:35:48", "content": "Thanks for questioning!
\n1. You are right. I copied these numbers from the log when I deleted duplicated rows by mistake. And, deleting duplicated rows did not improve my score.
\n2. At first, I used only day 8 data as train data. Then, I added day 7 data that I called \"prev data\". So, in my query, prev and train are merged.
Gotcha! Thanks for answering these questions. I'm on the half way of reproducing your best single model.
\n\nI realized you excluded several features from BigQuery results(https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/load_data.py#L10), how did you do the feature selection? I'm also curious how you find the best param of lightGBM, from your code, looks like you did some manual tuning?
", "votes": null, "replies": [] }, { "id": 329316, "author_name": "tkm2261", "author_url": "", "post_date": "05/16/2018 08:11:07", "content": "Thanks for sharing. Really helpful.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "324921": "Congrats winners!\n\nI would like to share my features briefly before I forget it. \n\n### Features\n\nI used BigQuery on the almost all feature engineering parts. \n https://gist.github.com/tkm2261/1b3c3c37753e55ed2914577c0f96d222\n\nIt takes only about 20 minutes. very fast!\n\nThere may not be surprising features if you have seen the shared kernels. I tried to find good features by brute force way.\n\n### Machine\n\nCPU: 16 cores, MEM: 100GB on GCP\n\n### Data\n\n* training data: day 7 and day 8 (13,188,695,398 rows)\n* validation data: day 9 (53,016,937 rows)\n* test data: day 10 (18,790,469 rows)\n\nI used validation data to determin the boosting round. Then, I used train+valid data for training.\n\n### Training\n\nSimply, I used LightGBM and ensembled with different seeds.\n\nThe single best model is 0.9823 on the public LB.\n\nThe parameter is here: \n\n {'colsample_bytree': 0.6, 'learning_rate': 0.1, 'max_bin': 1023, 'max_depth': -1, 'metric': ['binary_logloss', 'auc'], 'min_child_weight': 30, 'min_split_gain': 0.0001, 'num_leaves': 127, 'objective': 'binary', 'reg_alpha': 0, 'scale_pos_weight': 1, 'seed': 1142, 'subsample': 0.9, 'subsample_freq': 1, 'verbose': -1}\n\nsource code is here: https://github.com/tkm2261/kaggle_talkingdata/blob/master/protos/train_lgb.py\n\nI tried to earn my tuition for MS CS program starting next fall. But, there was no free lunch!", "324925": "Good game and well played!", "324929": "Congrats and thanks for sharing~", "324936": "Cool! Thanks for sharing! I hope you do well in your MS program!", "324943": "Congrats and thanks for sharing. so first step is make train csv file to sql dataset.", "324949": "Thanks for sharing.", "324957": "Congrats and thanks for sharing! \nAnd may I ask here \n```\nI used validation data to determin the boosting round. Then, I used train+valid data for training.\n``` \nThat means you use the same parameters and rounds you get in validation to retrain the model without using validation in retraining, is that right? \nTHANKS.", "324962": "'scale_pos_weight': 1 ? I have not seen anyone using such a small number.\nCongratulation on your accomplishment!", "324966": "I mean the validation data is used like fold-out data.", "324975": "I handle the imbalance by min_sample_weight, num_leaves, and other parameters. Since the eval metric is AUC, the scale is not important.", "324978": "Thank you for sharing! I'll try BigQuery", "325006": "Nice Work!", "325018": "just wondering about whether using validation(or early_stopping) in retraining or not. thx", "325025": "haha, i am going for MS CS , in the coming fall ! how did you come up with the hyperparameters", "325028": "One quick question: do you refit on the whole set after get the best number of trees from the validation?", "325034": "Wow that's so fast to build all those features, thanks for sharing & congrats! I never knew it can be so much faster using BiqQuery, I only use VM instances for this competittion.", "325061": "Brilliant, \nThanks for sharing.", "325090": "great solution! most suitable for production", "325094": "Congrats! Vote for the \"next_click\" part, you've done way more than I did. Thanks for sharing.", "325097": "Could you tell a bit about tuning of the hyper-parameters, if you did any?", "325099": "yes, I refit.", "325100": "I used the validation data in retraining.", "325123": "Congrats.