Finally I have some time to wrap up my first time experience on Kaggle. I have to say it was an amazing experience. And thanks to many wonderful kernels, I learnt a lot from this competition.
\n\nI think this competition is very good and also very challenge for a Kaggle beginner. Except some common numerical and catigorical features, this competition also has lots of text and image features which make this competition very challenge. So here is my solution ;)
\n\nFeatures
\n\nGeographic features: Such as latitude, longitude, populations, etc. Learnt a clustering trick from this kernel
\n\nImage features: basic image features such as brightness, key colours, etc. Mainly from this kernel. Applied multiprocessing, but still took a very long time to extract. I also used YOLO to perform a object detection. A python wrapper can be found from here
\n\nAggregated features: from this public kernel
\n\nText features: TF-IDF and SVD. Embedding vectors using fastText
\n\nMean Encoding: There were lots of high cardinality features. So I used mean encoding to encode these high cardinality categorical features.
\n\nFeature interactions: some groupby statistics between categorical features and numerical features. I generated 42 new features from this step.
\n\nModels
\n\nI trained one XGBoost, two LightGBM, one RNN and a ridge regression. I used a 5-fold CV followed by stacking with a LightGBM. My RNN is simple. A one-layer LSTM was concatenate with other features and then passed through 2 Dense layers. I was about to train two NN models. But because of my poor machine (8GB), I could only train a baseline model on my local computer. And I was first training my models on a spot instance on AWS. But I got terminated twice in the middle of the training. So sad.... Then I switched to a regular instance and paid regular price. So I didn't have enough time train my second NN model.
\n\nFinally, thanks again to people who have shared their kernels and methods so generously on Kaggle. There are still a lot of rooms I should improve and lots of things to learn. I indeed enjoy my first Kaggle journey. At last, Happy Kaggling!
", "messages": [ { "id": "365143", "postDate": "08/02/2018 00:59:53", "content": "Finally I have some time to wrap up my first time experience on Kaggle. I have to say it was an amazing experience. And thanks to many wonderful kernels, I learnt a lot from this competition.
\n\nI think this competition is very good and also very challenge for a Kaggle beginner. Except some common numerical and catigorical features, this competition also has lots of text and image features which make this competition very challenge. So here is my solution ;)
\n\nFeatures
\n\nGeographic features: Such as latitude, longitude, populations, etc. Learnt a clustering trick from this kernel
\n\nImage features: basic image features such as brightness, key colours, etc. Mainly from this kernel. Applied multiprocessing, but still took a very long time to extract. I also used YOLO to perform a object detection. A python wrapper can be found from here
\n\nAggregated features: from this public kernel
\n\nText features: TF-IDF and SVD. Embedding vectors using fastText
\n\nMean Encoding: There were lots of high cardinality features. So I used mean encoding to encode these high cardinality categorical features.
\n\nFeature interactions: some groupby statistics between categorical features and numerical features. I generated 42 new features from this step.
\n\nModels
\n\nI trained one XGBoost, two LightGBM, one RNN and a ridge regression. I used a 5-fold CV followed by stacking with a LightGBM. My RNN is simple. A one-layer LSTM was concatenate with other features and then passed through 2 Dense layers. I was about to train two NN models. But because of my poor machine (8GB), I could only train a baseline model on my local computer. And I was first training my models on a spot instance on AWS. But I got terminated twice in the middle of the training. So sad.... Then I switched to a regular instance and paid regular price. So I didn't have enough time train my second NN model.
\n\nFinally, thanks again to people who have shared their kernels and methods so generously on Kaggle. There are still a lot of rooms I should improve and lots of things to learn. I indeed enjoy my first Kaggle journey. At last, Happy Kaggling!
", "rawMarkdown": "Finally I have some time to wrap up my first time experience on Kaggle. I have to say it was an amazing experience. And thanks to many wonderful kernels, I learnt a lot from this competition. \n\nI think this competition is very good and also very challenge for a Kaggle beginner. Except some common numerical and catigorical features, this competition also has lots of text and image features which make this competition very challenge. So here is my solution ;)\n\n**Features**\n\nGeographic features: Such as latitude, longitude, populations, etc. Learnt a clustering trick from this [kernel][1]\n\nImage features: basic image features such as brightness, key colours, etc. Mainly from this [kernel][2]. Applied multiprocessing, but still took a very long time to extract. I also used YOLO to perform a object detection. A python wrapper can be found from [here][3]\n\nAggregated features: from this [public kernel][4]\n\nText features: TF-IDF and SVD. Embedding vectors using fastText\n\nMean Encoding: There were lots of high cardinality features. So I used mean encoding to encode these high cardinality categorical features.\n\nFeature interactions: some groupby statistics between categorical features and numerical features. I generated 42 new features from this step. \n\n**Models**\n\nI trained one XGBoost, two LightGBM, one RNN and a ridge regression. I used a 5-fold CV followed by stacking with a LightGBM. My RNN is simple. A one-layer LSTM was concatenate with other features and then passed through 2 Dense layers. I was about to train two NN models. But because of my poor machine (8GB), I could only train a baseline model on my local computer. And I was first training my models on a spot instance on AWS. But I got terminated twice in the middle of the training. So sad.... Then I switched to a regular instance and paid regular price. So I didn't have enough time train my second NN model. \n\nFinally, thanks again to people who have shared their kernels and methods so generously on Kaggle. There are still a lot of rooms I should improve and lots of things to learn. I indeed enjoy my first Kaggle journey. At last, Happy Kaggling!\n\n [1]: https://www.kaggle.com/frankherfert/region-and-city-details-with-lat-lon-and-clusters/notebook\n [2]: https://www.kaggle.com/shivamb/ideas-for-image-features-and-image-quality\n [3]: https://github.com/AlexeyAB/darknet\n [4]: https://www.kaggle.com/bminixhofer/aggregated-features-lightgbm", "votes": null }, { "id": "365152", "postDate": "08/02/2018 01:49:48", "content": "Cool
", "rawMarkdown": "Cool", "votes": null } ], "comments": [ { "id": 365152, "author_name": "sajal0jain", "author_url": "", "post_date": "08/02/2018 01:49:48", "content": "Cool
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "365143": "Finally I have some time to wrap up my first time experience on Kaggle. I have to say it was an amazing experience. And thanks to many wonderful kernels, I learnt a lot from this competition. \n\nI think this competition is very good and also very challenge for a Kaggle beginner. Except some common numerical and catigorical features, this competition also has lots of text and image features which make this competition very challenge. So here is my solution ;)\n\n**Features**\n\nGeographic features: Such as latitude, longitude, populations, etc. Learnt a clustering trick from this [kernel][1]\n\nImage features: basic image features such as brightness, key colours, etc. Mainly from this [kernel][2]. Applied multiprocessing, but still took a very long time to extract. I also used YOLO to perform a object detection. A python wrapper can be found from [here][3]\n\nAggregated features: from this [public kernel][4]\n\nText features: TF-IDF and SVD. Embedding vectors using fastText\n\nMean Encoding: There were lots of high cardinality features. So I used mean encoding to encode these high cardinality categorical features.\n\nFeature interactions: some groupby statistics between categorical features and numerical features. I generated 42 new features from this step. \n\n**Models**\n\nI trained one XGBoost, two LightGBM, one RNN and a ridge regression. I used a 5-fold CV followed by stacking with a LightGBM. My RNN is simple. A one-layer LSTM was concatenate with other features and then passed through 2 Dense layers. I was about to train two NN models. But because of my poor machine (8GB), I could only train a baseline model on my local computer. And I was first training my models on a spot instance on AWS. But I got terminated twice in the middle of the training. So sad.... Then I switched to a regular instance and paid regular price. So I didn't have enough time train my second NN model. \n\nFinally, thanks again to people who have shared their kernels and methods so generously on Kaggle. There are still a lot of rooms I should improve and lots of things to learn. I indeed enjoy my first Kaggle journey. At last, Happy Kaggling!\n\n [1]: https://www.kaggle.com/frankherfert/region-and-city-details-with-lat-lon-and-clusters/notebook\n [2]: https://www.kaggle.com/shivamb/ideas-for-image-features-and-image-quality\n [3]: https://github.com/AlexeyAB/darknet\n [4]: https://www.kaggle.com/bminixhofer/aggregated-features-lightgbm", "365152": "Cool" }, "source": "meta" }