{ "id": 335892, "title": "Tabular Classification - Tips and Tricks", "url": "/competitions/amex-default-prediction/discussion/335892", "author_name": "The Devastator", "post_date": "2022-07-08T10:19:39.431000", "votes": 358, "comment_count": 41, "views": 0, "content": "

Tabular Classification - Tips and Tricks

\n

Sharing a long list of tips and tricks found on previous tabular classification competitions on Kaggle.
\nTake this as a list of ideas you can attempt to try for improving your score!

\n
\n

Credits:

\n \n
\n

Dealing with larger datasets

\n

An issue close to our hearts!

\n\n

Data exploration

\n

Data exploration is a must on all competitions, Here are some references from past competitions use them to get some ideas on what exactly to explore.

\n\n

Preprocessing

\n

Before we start to engineer features, we need to do some preprocessing.
\nThose invlolve removing missing values, converting categorical variables to numeric, and scaling the data, etc..
\nHere are some references from past competitions use them to get some ideas for preprocessing.

\n\n

Feature engineering

\n

Now we can start to engineer features, below you can find some references from past competitions use them to get some ideas to engineer features.

\n\n

Feature selection

\n

Now that we got our features engineered, we should check if they are useful or not.
\nTo do this, we need to do some feature selection.
\nHere are some references from past competitions use them to get some ideas for feature selection.

\n\n

Modeling

\n

Below you can find The best of kaggle - GBMs.
\nThese are the top voted/viewed notebooks all over kaggle for GBMs.
\nUsage rules are simple: You like it? Upvote the original!

\n

XGBoost

\n\n

LightGBM

\n\n

CatBoost

\n\n

Ensemble

\n

Now that we have our model trained, we should ensemble our predictions.

\n

Below you can find The best of kaggle - Ensembles.
\nThese are the top voted/viewed notebooks all over kaggle for GBMs.
\nUsage rules are simple: You like it? Upvote the original!

\n

Ensembling Techniques

\n\n

Blending

\n\n

Stacking

\n\n

Bagging

\n\n

Boosting

\n\n

Hyperparameters Tuning

\n", "messages": [ { "id": 1848063, "postDate": "2022-07-08T10:19:39.430Z", "content": "

Tabular Classification - Tips and Tricks

\n

Sharing a long list of tips and tricks found on previous tabular classification competitions on Kaggle.
\nTake this as a list of ideas you can attempt to try for improving your score!

\n
\n

Credits:

\n \n
\n

Dealing with larger datasets

\n

An issue close to our hearts!

\n\n

Data exploration

\n

Data exploration is a must on all competitions, Here are some references from past competitions use them to get some ideas on what exactly to explore.

\n\n

Preprocessing

\n

Before we start to engineer features, we need to do some preprocessing.
\nThose invlolve removing missing values, converting categorical variables to numeric, and scaling the data, etc..
\nHere are some references from past competitions use them to get some ideas for preprocessing.

\n\n

Feature engineering

\n

Now we can start to engineer features, below you can find some references from past competitions use them to get some ideas to engineer features.

\n\n

Feature selection

\n

Now that we got our features engineered, we should check if they are useful or not.
\nTo do this, we need to do some feature selection.
\nHere are some references from past competitions use them to get some ideas for feature selection.

\n\n

Modeling

\n

Below you can find The best of kaggle - GBMs.
\nThese are the top voted/viewed notebooks all over kaggle for GBMs.
\nUsage rules are simple: You like it? Upvote the original!

\n

XGBoost

\n\n

LightGBM

\n\n

CatBoost

\n\n

Ensemble

\n

Now that we have our model trained, we should ensemble our predictions.

\n

Below you can find The best of kaggle - Ensembles.
\nThese are the top voted/viewed notebooks all over kaggle for GBMs.
\nUsage rules are simple: You like it? Upvote the original!

\n

Ensembling Techniques

\n\n

Blending

\n\n

Stacking

\n\n

Bagging

\n\n

Boosting

\n\n

Hyperparameters Tuning

\n", "rawMarkdown": "# Tabular Classification - Tips and Tricks\n\n\nSharing a long list of tips and tricks found on previous tabular classification competitions on Kaggle.\nTake this as a list of ideas you can attempt to try for improving your score!\n\n> **Credits:** \n> - Many of the links are based on parts of [this](https://neptune.ai/blog/tabular-data-binary-classification-tips-and-tricks-from-5-kaggle-competitions) blog post (with edits and filtering).\n> - Including links from: [The best of kaggle - GBMs](https://www.kaggle.com/discussions/getting-started/277121).\n> - Also from: [The best of kaggle - Ensembles](https://www.kaggle.com/discussions/getting-started/291437).\n\n### Dealing with larger datasets\n\nAn issue close to our hearts! \n\n* Faster [data loading with pandas.](https://www.kaggle.com/c/home-credit-default-risk/discussion/59575)\n* Data compression techniques to [reduce the size of data by 70%](https://www.kaggle.com/nickycan/compress-70-of-dataset).\n* Optimize the memory by r[educing the size of some attributes.](https://www.kaggle.com/shrutimechlearn/large-data-loading-trick-with-ms-malware-data)\n* Use open-source libraries such as [Dask to read and manipulate the data](https://www.kaggle.com/yuliagm/how-to-work-with-big-datasets-on-16g-ram-dask), it performs parallel computing and saves up memory space.\n* Use [cudf](https://github.com/rapidsai/cudf).\n* Convert data to [parquet](https://arrow.apache.org/docs/python/parquet.html) format.\n* Converting data to [feather](https://medium.com/@snehotosh.banerjee/feather-a-fast-on-disk-format-for-r-and-python-data-frames-de33d0516b03) format.\n* Reducing memory usage for [optimizing RAM](https://www.kaggle.com/mjbahmani/reducing-memory-size-for-ieee).\n\n### Data exploration\n\nData exploration is a must on all competitions, Here are some references from past competitions use them to get some ideas on what exactly to explore.\n\n* EDA for microsoft [malware detection.](https://www.kaggle.com/youhanlee/my-eda-i-want-to-see-all)\n* Time Series [EDA for malware detection.](https://www.kaggle.com/cdeotte/time-split-validation-malware-0-68)\n* Complete [EDA for home credit loan prediction](https://www.kaggle.com/codename007/home-credit-complete-eda-feature-importance).\n* Complete [EDA for Santader prediction.](https://www.kaggle.com/gpreda/santander-eda-and-prediction)\n* EDA for [VSB Power Line Fault Detection.](https://www.kaggle.com/go1dfish/basic-eda)\n\n### Preprocessing\n\nBefore we start to engineer features, we need to do some preprocessing.\nThose invlolve removing missing values, converting categorical variables to numeric, and scaling the data, etc..\nHere are some references from past competitions use them to get some ideas for preprocessing.\n\n* Methods to [tackle class imbalance](https://www.kaggle.com/shahules/tackling-class-imbalance).\n* Data augmentation by [Synthetic Minority Oversampling Technique](https://machinelearningmastery.com/smote-oversampling-for-imbalanced-classification/).\n* Fast inplace [shuffle for augmentation](https://www.kaggle.com/jiweiliu/fast-inplace-shuffle-for-augmentation).\n* Finding [synthetic samples in the dataset.](https://www.kaggle.com/yag320/list-of-fake-samples-and-public-private-lb-split)\n* [Signal denoising](https://www.kaggle.com/jackvial/dwt-signal-denoising) used in signal processing competitions.\n* Finding [patterns of missing data](https://www.kaggle.com/jpmiller/patterns-of-missing-data).\n* Methods to handle [missing data](https://towardsdatascience.com/6-different-ways-to-compensate-for-missing-values-data-imputation-with-examples-6022d9ca0779).\n* An overview of various [encoding techniques for categorical data.](https://www.kaggle.com/shahules/an-overview-of-encoding-techniques)\n* Building [model to predict missing values.](https://www.kaggle.com/c/home-credit-default-risk/discussion/64598)\n* Random [shuffling of data](https://www.kaggle.com/brandenkmurray/randomly-shuffled-data-also-works) to create new synthetic training set.\n\n### Feature engineering\n\nNow we can start to engineer features, below you can find some references from past competitions use them to get some ideas to engineer features. \n\n* Target [encoding cross validation](https://medium.com/@pouryaayria/k-fold-target-encoding-dfe9a594874b/) for better encoding.\n* Entity embedding to [handle categories](https://www.kaggle.com/abhishek/entity-embeddings-to-handle-categories).\n* Encoding c[yclic features for deep learning.](https://www.kaggle.com/avanwyk/encoding-cyclical-features-for-deep-learning)\n* Manual [feature engineering methods](https://www.kaggle.com/willkoehrsen/introduction-to-manual-feature-engineering).\n* Automated feature engineering techniques [using featuretools](https://www.kaggle.com/willkoehrsen/automated-feature-engineering-basics).\n* Top hard crafted features used in [microsoft malware detection](https://www.kaggle.com/sanderf/7th-place-solution-microsoft-malware-prediction).\n* Denoising NN for [feature extraction](https://towardsdatascience.com/applied-deep-learning-part-3-autoencoders-1c083af4d798).\n* Feature engineering [using RAPIDS framework.](https://www.kaggle.com/cdeotte/rapids-feature-engineering-fraud-0-96/)\n* Things to remember while processing f[eatures using LGBM.](https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575)\n* [Lag features and moving averages.](https://www.kaggle.com/c/home-credit-default-risk/discussion/64593)\n* [Principal component analysis](https://medium.com/machine-learning-researcher/dimensionality-reduction-pca-and-lda-6be91734f567) for dimensionality reduction.\n* LDA for [dimensionality reduction](https://medium.com/machine-learning-researcher/dimensionality-reduction-pca-and-lda-6be91734f567).\n* Best hand crafted LGBM features for [microsoft malware detection](https://www.kaggle.com/c/microsoft-malware-prediction/discussion/85157).\n* Generating [frequency features.](https://www.kaggle.com/philippsinger/frequency-features-without-test-data-information)\n* Dropping variables with [different train and test distribution.](https://www.kaggle.com/bogorodvo/lightgbm-baseline-model-using-sparse-matrix)\n* [Aggregate time series features](https://www.kaggle.com/c/home-credit-default-risk/discussion/64593) for home credit competition.\n* [Time Series](https://www.kaggle.com/c/home-credit-default-risk/discussion/64593) features used in home credit default risk.\n* Scale, Standardize and n[ormalize with sklearn](https://towardsdatascience.com/scale-standardize-or-normalize-with-scikit-learn-6ccc7d176a02).\n* Handcrafted features for [Home default risk competition.](https://www.kaggle.com/c/home-credit-default-risk/discussion/57750)\n* Handcrafted [features used in Santander Transaction Prediction.](https://www.kaggle.com/c/santander-customer-transaction-prediction/discussion/89070)\n\n### Feature selection\n\nNow that we got our features engineered, we should check if they are useful or not.\nTo do this, we need to do some feature selection.\nHere are some references from past competitions use them to get some ideas for feature selection.\n\n* Six ways to do [features selection using sklearn](https://www.kaggle.com/sz8416/6-ways-for-feature-selection).\n* [Permutation feature importance](https://www.kaggle.com/c/ieee-fraud-detection/discussion/107877#latest-635386).\n* [Adversarial feature validation](https://www.kaggle.com/tunguz/adversarial-ieee/).\n* Feature selection using [null importances.](https://www.kaggle.com/ogrellier/feature-selection-with-null-importances)\n* Tree explainer using [SHAP.](https://github.com/slundberg/shap)\n* DeepNN explainer using [SHAP](https://github.com/slundberg/shap).\n\n\n### Modeling\n\nBelow you can find [The best of kaggle - GBMs](https://www.kaggle.com/discussions/getting-started/277121).\nThese are the top voted/viewed notebooks all over kaggle for GBMs. \n**Usage rules are simple: You like it? Upvote the original!**\n\n### XGBoost\n \n* [Data Analysis & XGBoost Starter (0.35460 LB)](https://www.kaggle.com/anokas/data-analysis-xgboost-starter-0-35460-lb) (137K views, 1377 Votes)*\n* [House prices: Lasso, XGBoost, and a detailed EDA](https://www.kaggle.com/erikbruin/house-prices-lasso-xgboost-and-a-detailed-eda) (183K views, 1317 Votes)*\n* [Feature engineering, xgboost](https://www.kaggle.com/dlarionov/feature-engineering-xgboost) (144K views, 942 Votes)*\n* [XGBoost](https://www.kaggle.com/dansbecker/xgboost) (173K views, 918 Votes)*\n* [XGBoost](https://www.kaggle.com/alexisbcook/xgboost) (266K views, 578 Votes)*\n* [Market Prediction: XGBoost with GPU (Fit in 1min)](https://www.kaggle.com/hamditarek/market-prediction-xgboost-with-gpu-fit-in-1min) (53K views, 411 Votes)*\n* [A Guide on XGBoost hyperparameters tuning](https://www.kaggle.com/prashant111/a-guide-on-xgboost-hyperparameters-tuning) (55K views, 358 Votes)*\n* [Titanic WCG+XGBoost [0.84688]](https://www.kaggle.com/cdeotte/titanic-wcg-xgboost-0-84688) (27K views, 314 Votes)*\n* [XGBoost & Feature Selection DSBowl πŸ₯£ πŸ₯£](https://www.kaggle.com/shahules/xgboost-feature-selection-dsbowl) (36K views, 299 Votes)*\n* [Understanding XGBoost Model on Otto Data](https://www.kaggle.com/tqchen/understanding-xgboost-model-on-otto-data) (130K views, 297 Votes)*\n\n### LightGBM\n \n* [LightGBM with Simple Features](https://www.kaggle.com/jsaguiar/lightgbm-with-simple-features) (85K views, 619 Votes)*\n* [LightGBM. Baseline Model Using Sparse Matrix](https://www.kaggle.com/bogorodvo/lightgbm-baseline-model-using-sparse-matrix) (49K views, 409 Votes)*\n* [LightGBM starter with feature engineering idea](https://www.kaggle.com/tommy1028/lightgbm-starter-with-feature-engineering-idea) (16K views, 370 Votes)*\n* [Google Analytics EDA + LightGBM + Screenshots](https://www.kaggle.com/erikbruin/google-analytics-eda-lightgbm-screenshots) (40K views, 344 Votes)*\n* [Feature Engineering & LightGBM](https://www.kaggle.com/davidcairuz/feature-engineering-lightgbm) (26K views, 272 Votes)*\n* [Simple LightGBM without blending](https://www.kaggle.com/mfjwr1/simple-lightgbm-without-blending) (18K views, 263 Votes)*\n* [LightGBM + XGBoost + Catboost](https://www.kaggle.com/samratp/lightgbm-xgboost-catboost) (37K views, 245 Votes)*\n* [ASHRAE- KFold LightGBM - without leak (1.08)](https://www.kaggle.com/aitude/ashrae-kfold-lightgbm-without-leak-1-08) (15K views, 240 Votes)*\n* [LightGBM Classifier in Python](https://www.kaggle.com/prashant111/lightgbm-classifier-in-python) (52K views, 231 Votes)*\n* [LightGBM (Fixing unbalanced data)](https://www.kaggle.com/pranav84/lightgbm-fixing-unbalanced-data-lb-0-9680) (56K views, 228 Votes)*\n\n### CatBoost\n \n* [A new baseline for DSB 2019 - Catboost model](https://www.kaggle.com/mhviraf/a-new-baseline-for-dsb-2019-catboost-model) (15K views, 316 Votes)*\n* [LightGBM + XGBoost + Catboost](https://www.kaggle.com/samratp/lightgbm-xgboost-catboost) (37K views, 245 Votes)*\n* [CatBoost: A Deeper Dive](https://www.kaggle.com/abhinand05/catboost-a-deeper-dive) (8K views, 178 Votes)*\n* [House Prices Tutorial with Catboost](https://www.kaggle.com/allunia/house-prices-tutorial-with-catboost) (12K views, 176 Votes)*\n* [Stacking Test-Sklearn, XGBoost, CatBoost, LightGBM](https://www.kaggle.com/eliotbarr/stacking-test-sklearn-xgboost-catboost-lightgbm) (25K views, 173 Votes)*\n* [WiDSDatathon2021-Catboost-Starter](https://www.kaggle.com/usharengaraju/widsdatathon2021-catboost-starter) (6K views, 167 Votes)*\n* [Tutorial: CatBoost Overview](https://www.kaggle.com/mitribunskiy/tutorial-catboost-overview) (39K views, 127 Votes)*\n* [CatBoost Classifier in Python](https://www.kaggle.com/prashant111/catboost-classifier-in-python) (28K views, 126 Votes)*\n* [Catboost - Some more features](https://www.kaggle.com/braquino/catboost-some-more-features) (7K views, 103 Votes)*\n* [mlcourse.ai. Fall 2019. Catboost starter](https://www.kaggle.com/kashnitsky/mlcourse-ai-fall-2019-catboost-starter) (18K views, 98 Votes)*\n\n\n### Ensemble\n\nNow that we have our model trained, we should ensemble our predictions.\n\nBelow you can find [The best of kaggle - Ensembles](https://www.kaggle.com/discussions/getting-started/291437).\nThese are the top voted/viewed notebooks all over kaggle for GBMs.\n**Usage rules are simple: You like it? Upvote the original!**\n\n### Ensembling Techniques\n\n* [Introduction to Ensembling/Stacking in Python](https://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python) (620K views, 5436 Votes)*\n* [Titanic Top 4% with ensemble modeling](https://www.kaggle.com/yassineghouzam/titanic-top-4-with-ensemble-modeling) (185K views, 2501 Votes)*\n* [EDA & Ensemble Model (Top 10 Percentile)](https://www.kaggle.com/viveksrinivasan/eda-ensemble-model-top-10-percentile) (98K views, 513 Votes)*\n* [Analysis of Melanoma Metadata and EffNet Ensemble](https://www.kaggle.com/datafan07/analysis-of-melanoma-metadata-and-effnet-ensemble) (24K views, 405 Votes)*\n* [Detailed Data Analysis & Ensemble Modeling](https://www.kaggle.com/tannercarbonati/detailed-data-analysis-ensemble-modeling) (55K views, 360 Votes)*\n* [Ensemble Folds with MEDIAN - [0.153]](https://www.kaggle.com/cdeotte/ensemble-folds-with-median-0-153) (9K views, 303 Votes)*\n* [WBF approach for ensemble](https://www.kaggle.com/shonenkov/wbf-approach-for-ensemble) (18K views, 282 Votes)*\n* [Ensemble: Resnext50\\_32x4d + Efficientnet = 0.903](https://www.kaggle.com/japandata509/ensemble-resnext50-32x4d-efficientnet-0-903) (16K views, 274 Votes)*\n* [Employee attrition via Ensemble tree-based methods](https://www.kaggle.com/arthurtok/employee-attrition-via-ensemble-tree-based-methods) (64K views, 253 Votes)*\n* [Top 3%. Efficient ensembling in few lines of code](https://www.kaggle.com/isaienkov/top-3-efficient-ensembling-in-few-lines-of-code) (30K views, 242 Votes)*\n* [Ensemble Learning Techniques Tutorial](https://www.kaggle.com/pavansanagapati/ensemble-learning-techniques-tutorial) (55K views, 215 Votes)*\n* [Minimal LSTM + NB-SVM baseline ensemble](https://www.kaggle.com/jhoward/minimal-lstm-nb-svm-baseline-ensemble) (30K views, 209 Votes)*\n* [Ensemble Model: Stacked Model Example](https://www.kaggle.com/jimthompson/ensemble-model-stacked-model-example) (68K views, 207 Votes)*\n* [πŸ€—πŸ€— Ensemble πŸ€—πŸ€—](https://www.kaggle.com/hamditarek/ensemble) (64K views, 196 Votes)*\n* [EDA&Modelling of the External Data Inc. Ensemble](https://www.kaggle.com/datafan07/eda-modelling-of-the-external-data-inc-ensemble) (8K views, 195 Votes)*\n\n### Blending\n \n* [Simple LightGBM without blending](https://www.kaggle.com/mfjwr1/simple-lightgbm-without-blending) (18K views, 265 Votes)*\n* [Blending tensorflow and pytorchπŸ”₯πŸ”₯πŸ”₯](https://www.kaggle.com/a763337092/blending-tensorflow-and-pytorch) (13K views, 220 Votes)*\n* [Another model for your blending](https://www.kaggle.com/speedwagon/quadratic-discriminant-analysis) (11K views, 216 Votes)*\n* [Optimise Blending Weights with Bonus :0](https://www.kaggle.com/gogo827jz/optimise-blending-weights-with-bonus-0) (7K views, 184 Votes)*\n* [Blending with Linear Regression [0.688 LB]](https://www.kaggle.com/suicaokhoailang/blending-with-linear-regression-0-688-lb) (14K views, 136 Votes)*\n* [blending blending blending](https://www.kaggle.com/abhishek/blending-blending-blending) (3K views, 128 Votes)*\n* [Lessons from Toxic : Blending is the new sexy](https://www.kaggle.com/jagangupta/lessons-from-toxic-blending-is-the-new-sexy) (10K views, 128 Votes)*\n* [non-blending lightGBM model LB: 0.977](https://www.kaggle.com/bk0000/non-blending-lightgbm-model-lb-0-977) (16K views, 127 Votes)*\n* [Bayesian Optimization Seed Blending](https://www.kaggle.com/hengzheng/bayesian-optimization-seed-blending) (9K views, 118 Votes)*\n* [elo\\_world\\_high\\_score\\_without\\_blending](https://www.kaggle.com/gpreda/elo-world-high-score-without-blending) (8K views, 112 Votes)*\n* [competition part-5: blending 101](https://www.kaggle.com/abhishek/competition-part-5-blending-101) (3K views, 105 Votes)*\n* [Beginner EDA with Feature Eng. and Blending Models](https://www.kaggle.com/datafan07/beginner-eda-with-feature-eng-and-blending-models) (3K views, 99 Votes)*\n* [Cross-validation, weighted linear blending, errors](https://www.kaggle.com/tilii7/cross-validation-weighted-linear-blending-errors) (7K views, 84 Votes)*\n* [Blending TensorFlow + PyTorch (th=0.4914)](https://www.kaggle.com/sagarjiyani/blending-tensorflow-pytorch-th-0-4914) (5K views, 82 Votes)*\n* [Blending NN and LGBM/RF](https://www.kaggle.com/gogo827jz/blending-nn-and-lgbm-rf) (6K views, 81 Votes)*\n\n### Stacking\n \n* [Stacked Regressions : Top 4% on LeaderBoard](https://www.kaggle.com/serigne/stacked-regressions-top-4-on-leaderboard) (493K views, 6236 Votes)*\n* [Introduction to Ensembling/Stacking in Python](https://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python) (620K views, 5436 Votes)*\n* [Model stacking, feature engineering and EDA](https://www.kaggle.com/dimitreoliveira/model-stacking-feature-engineering-and-eda) (43K views, 409 Votes)*\n* [Stacking House Prices - Walkthrough to Top 5%](https://www.kaggle.com/agodwinp/stacking-house-prices-walkthrough-to-top-5) (32K views, 227 Votes)*\n* [Stacking Starter](https://www.kaggle.com/mmueller/stacking-starter) (41K views, 221 Votes)*\n* [Ensemble Model: Stacked Model Example](https://www.kaggle.com/jimthompson/ensemble-model-stacked-model-example) (68K views, 207 Votes)*\n* [Titanic: Voting, Pipeline, Stack, and Guide](https://www.kaggle.com/nicapotato/titanic-voting-pipeline-stack-and-guide) (21K views, 197 Votes)*\n* [Explore Stacking (LB 0.1463)](https://www.kaggle.com/dongxu027/explore-stacking-lb-0-1463) (18K views, 180 Votes)*\n* [Stacking Test-Sklearn, XGBoost, CatBoost, LightGBM](https://www.kaggle.com/eliotbarr/stacking-test-sklearn-xgboost-catboost-lightgbm) (26K views, 176 Votes)*\n* [Top 1% Approach: EDA, New Models and Stacking](https://www.kaggle.com/datafan07/top-1-approach-eda-new-models-and-stacking) (9K views, 172 Votes)*\n* [Simple Stacker LB 0.284](https://www.kaggle.com/yekenot/simple-stacker-lb-0-284) (20K views, 172 Votes)*\n* [Stack&Blend LRs XGB LGB {House Prices K} v17](https://www.kaggle.com/itslek/stack-blend-lrs-xgb-lgb-house-prices-k-v17) (6K views, 164 Votes)*\n* [Top 10 (0.10943): stacking, MICE and brutal force](https://www.kaggle.com/agehsbarg/top-10-0-10943-stacking-mice-and-brutal-force) (21K views, 156 Votes)*\n* [OOF stacking regime](https://www.kaggle.com/hhstrand/oof-stacking-regime) (14K views, 145 Votes)*\n* [EloDA with Feature Engineering and Stacking](https://www.kaggle.com/tunguz/eloda-with-feature-engineering-and-stacking) (10K views, 141 Votes)*\n\n### Bagging\n \n* [Riiid LGBM bagging2 + SAKT =0.781](https://www.kaggle.com/ammarnassanalhajali/riiid-lgbm-bagging2-sakt-0-781) (11K views, 188 Votes)*\n* [SAKT + Riiid LGBM bagging2](https://www.kaggle.com/leadbest/sakt-riiid-lgbm-bagging2) (8K views, 139 Votes)*\n* [Topic 5. Ensembles. Part 1. Bagging](https://www.kaggle.com/kashnitsky/topic-5-ensembles-part-1-bagging) (16K views, 137 Votes)*\n* [Bagging vs Boosting](https://www.kaggle.com/prashant111/bagging-vs-boosting) (22K views, 117 Votes)*\n* [Fork of Riiid LGBM bagging2.1 471152](https://www.kaggle.com/julianguo/fork-of-riiid-lgbm-bagging2-1-471152) (8K views, 117 Votes)*\n* [Riiid! LGBM bagging2](https://www.kaggle.com/zephyrwang666/riiid-lgbm-bagging2) (6K views, 113 Votes)*\n* [Predicting House Prices [XGB/RF/Bagging-Reg Pipe]](https://www.kaggle.com/kabure/predicting-house-prices-xgb-rf-bagging-reg-pipe) (10K views, 89 Votes)*\n* [Keras starter with bagging (LB: 1120.596)](https://www.kaggle.com/danijelk/keras-starter-with-bagging-lb-1120-596) (18K views, 77 Votes)*\n* [Keras starter with bagging 1111.84364](https://www.kaggle.com/mtinti/keras-starter-with-bagging-1111-84364) (19K views, 77 Votes)*\n* [Riiid LGBM bagging2.1](https://www.kaggle.com/zephyrwang666/riiid-lgbm-bagging2-1) (4K views, 62 Votes)*\n* [BaggingRegressor + RAPIDS Ensemble](https://www.kaggle.com/andypenrose/baggingregressor-rapids-ensemble) (4K views, 59 Votes)*\n* [Credit analysis with KNN/DTree/RF/Bagging/ANN🏦](https://www.kaggle.com/fahadmehfoooz/credit-analysis-with-knn-dtree-rf-bagging-ann) (<1K views, 55 Votes)*\n* [Sentiment Analysis:SVM/NB/Bagging/Boosting/RF/LSTM](https://www.kaggle.com/fahadmehfoooz/sentiment-analysis-svm-nb-bagging-boosting-rf-lstm) (1K views, 51 Votes)*\n* [Ensemble ML Algorithms : Bagging, Boosting, Voting](https://www.kaggle.com/faressayah/ensemble-ml-algorithms-bagging-boosting-voting) (8K views, 48 Votes)*\n* [Simple XGBoost CV Bagging](https://www.kaggle.com/cttsai/simple-xgboost-cv-bagging) (3K views, 42 Votes)*\n\n### Boosting\n \n* [Boosting creativity towards feature engineering](https://www.kaggle.com/felipemello/boosting-creativity-towards-feature-engineering) (9K views, 229 Votes)*\n* [Vehicle Insurance EDA and boosting models](https://www.kaggle.com/yashvi/vehicle-insurance-eda-and-boosting-models) (19K views, 200 Votes)*\n* [Topic 10. Gradient Boosting](https://www.kaggle.com/kashnitsky/topic-10-gradient-boosting) (21K views, 188 Votes)*\n* [Gradient boosting simplified](https://www.kaggle.com/grroverpr/gradient-boosting-simplified) (48K views, 147 Votes)*\n* [Bagging vs Boosting](https://www.kaggle.com/prashant111/bagging-vs-boosting) (22K views, 117 Votes)*\n* [Diabetes Prediction (Stacking + Boosting)](https://www.kaggle.com/niteshyadav3103/diabetes-prediction-stacking-boosting) (1K views, 85 Votes)*\n* [Elo Stack With Goss Boosting](https://www.kaggle.com/roydatascience/elo-stack-with-goss-boosting) (6K views, 78 Votes)*\n* [Assignment 10. Gradient boosting and flight delays](https://www.kaggle.com/kashnitsky/assignment-10-gradient-boosting-and-flight-delays) (9K views, 71 Votes)*\n* [Battle of the Boosting Algos: LGB, XGB, Catboost](https://www.kaggle.com/lavanyashukla01/battle-of-the-boosting-algos-lgb-xgb-catboost) (9K views, 55 Votes)*\n* [Introduction to Boosting using LGBM(LB: 0.68357)](https://www.kaggle.com/vinnsvinay/introduction-to-boosting-using-lgbm-lb-0-68357) (14K views, 52 Votes)*\n* [FE-Pipeline with HistGradientBoostingRegressor](https://www.kaggle.com/tunguz/fe-pipeline-with-histgradientboostingregressor) (3K views, 51 Votes)*\n* [Sentiment Analysis:SVM/NB/Bagging/Boosting/RF/LSTM](https://www.kaggle.com/fahadmehfoooz/sentiment-analysis-svm-nb-bagging-boosting-rf-lstm) (1K views, 51 Votes)*\n* [Explainable Boosting machines for Tabular data](https://www.kaggle.com/parulpandey/explainable-boosting-machines-for-tabular-data) (2K views, 49 Votes)*\n* [Ensemble ML Algorithms : Bagging, Boosting, Voting](https://www.kaggle.com/faressayah/ensemble-ml-algorithms-bagging-boosting-voting) (8K views, 48 Votes)*\n* [House Price Prediction : GBoosting,AdaBoost etc.](https://www.kaggle.com/sid321axn/house-price-prediction-gboosting-adaboost-etc) (8K views, 42 Votes)*\n\n\n### Hyperparameters Tuning\n\n* LGBM [hyperparameter tuning](https://www.kaggle.com/mlisovyi/lightgbm-hyperparameter-optimisation-lb-0-761) methods.\n* Automated [model tuning](https://www.kaggle.com/willkoehrsen/automated-model-tuning) methods.\n* Parameter tuning with [hyper plot](https://www.kaggle.com/bigironsphere/parameter-tuning-in-one-function-with-hyperopt).\n* [Bayesian optimization](http://krasserm.github.io/2018/03/21/bayesian-optimization/) for hyperparameter tuning.\n* [Gpyopt Hyperparameter Optimisation](https://www.kaggle.com/nicapotato/gpyopt-hyperparameter-optimisation-gpu-lgbm).", "votes": 357 }, { "id": 2325446, "postDate": "2023-07-01T10:55:05.263Z", "content": "

This is really very helpful.

", "rawMarkdown": "This is really very helpful." }, { "id": 2192622, "postDate": "2023-03-22T19:27:42.413Z", "content": "

This is awesome resource , Keep up the great work.πŸ‘πŸ‘

", "rawMarkdown": "This is awesome resource , Keep up the great work.πŸ‘πŸ‘" }, { "id": 2064823, "postDate": "2022-12-14T06:23:32.787Z", "content": "

Fantastic resource for beginner, extremely grateful to you! @thedevastator

", "rawMarkdown": "Fantastic resource for beginner, extremely grateful to you! @thedevastator " }, { "id": 2050768, "postDate": "2022-12-01T01:35:48.947Z", "content": "

A fantastic resource all in one spot. Thank you

", "rawMarkdown": "A fantastic resource all in one spot. Thank you" }, { "id": 1911840, "postDate": "2022-08-24T10:53:45.570Z", "content": "

Thank you for taking the time to put this together. Very helpful!

", "rawMarkdown": "Thank you for taking the time to put this together. Very helpful!" }, { "id": 1897431, "postDate": "2022-08-13T18:02:28.747Z", "content": "

This is very useful

", "rawMarkdown": "This is very useful " }, { "id": 1879277, "postDate": "2022-08-01T01:02:11.613Z", "content": "

very helpful.

", "rawMarkdown": "very helpful." }, { "id": 1879239, "postDate": "2022-07-31T23:54:45.377Z", "content": "

Hello @thedevastator, so much knowledge, thanks for sharing ✨

", "rawMarkdown": "Hello @thedevastator, so much knowledge, thanks for sharing ✨" }, { "id": 1864977, "postDate": "2022-07-21T12:15:32.993Z", "content": "

tttttthhhhhhhhhhhhaaaaaaaaaaaannnnnnnnnnnnnnnnnk

", "rawMarkdown": "tttttthhhhhhhhhhhhaaaaaaaaaaaannnnnnnnnnnnnnnnnk" }, { "id": 1861816, "postDate": "2022-07-19T09:04:36.197Z", "content": "

Thank you so much for sharing these tips

", "rawMarkdown": "Thank you so much for sharing these tips" }, { "id": 1860925, "postDate": "2022-07-18T16:49:56.227Z", "content": "

Thank you so much for this! I'm barely starting with Kaggle competitions but I'm saving this one for later:)

", "rawMarkdown": "Thank you so much for this! I'm barely starting with Kaggle competitions but I'm saving this one for later:)" }, { "id": 1858544, "postDate": "2022-07-17T02:42:41.103Z", "content": "

Thanks so much for putting this together!

", "rawMarkdown": "Thanks so much for putting this together!" }, { "id": 1857191, "postDate": "2022-07-15T23:56:53.627Z", "content": "

Hello, @thedevastator, thanks for sharing a massive amount of information to keep improving

", "rawMarkdown": "Hello, @thedevastator, thanks for sharing a massive amount of information to keep improving" }, { "id": 1857146, "postDate": "2022-07-15T22:09:31.913Z", "content": "

Wow !!
\nThank you very much for the amount of time you have contributed to make this…

", "rawMarkdown": "Wow !!\nThank you very much for the amount of time you have contributed to make this..." }, { "id": 1855775, "postDate": "2022-07-14T22:37:56.487Z", "content": "

wow great stuff, thank you for sharing .

", "rawMarkdown": "wow great stuff, thank you for sharing ." }, { "id": 1855526, "postDate": "2022-07-14T16:56:47.500Z", "content": "

Nice post, thanks for sharing @thedevastator

", "rawMarkdown": "Nice post, thanks for sharing @thedevastator" }, { "id": 1852313, "postDate": "2022-07-12T01:52:29.773Z", "content": "

This is remarkable summary!
\nGreat!

", "rawMarkdown": "This is remarkable summary!\nGreat!" }, { "id": 1849096, "postDate": "2022-07-09T08:04:18.123Z", "content": "

very helpful.

", "rawMarkdown": "very helpful." }, { "id": 1848827, "postDate": "2022-07-08T23:57:45.223Z", "content": "

Nice post, thanks for sharing @thedevastator

", "rawMarkdown": "Nice post, thanks for sharing @thedevastator " }, { "id": 1848622, "postDate": "2022-07-08T19:02:11.103Z", "content": "

You have done a aggregate of aggregates. Thanks somethings I was just looking for found here . I think you should put this in a github repo..

", "rawMarkdown": "You have done a aggregate of aggregates. Thanks somethings I was just looking for found here . I think you should put this in a github repo.. " }, { "id": 1897691, "postDate": "2022-08-14T02:35:51.863Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1883154, "postDate": "2022-08-03T16:02:13.380Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1848917, "postDate": "2022-07-09T03:43:09.170Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1848302, "postDate": "2022-07-08T14:01:53.160Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1848077, "postDate": "2022-07-08T10:25:17.673Z", "content": "

Wow! Very useful! Thank you!!!

", "rawMarkdown": "Wow! Very useful! Thank you!!!", "votes": 1 }, { "id": 2012013, "postDate": "2022-11-01T02:37:48.443Z", "content": "

Super useful! Thank you!

", "rawMarkdown": "Super useful! Thank you! " }, { "id": 2006418, "postDate": "2022-10-27T15:23:37.077Z", "content": "

Thanks for the insight!

", "rawMarkdown": "Thanks for the insight!" }, { "id": 1992926, "postDate": "2022-10-18T02:48:47.557Z", "content": "

Thanks for the insight!

", "rawMarkdown": "Thanks for the insight!" }, { "id": 1913373, "postDate": "2022-08-25T09:54:14.937Z", "content": "

Thanks for the insight!

", "rawMarkdown": "Thanks for the insight!" }, { "id": 1910876, "postDate": "2022-08-23T18:43:31.537Z", "content": "

This is excellent. Thanks for sharing

", "rawMarkdown": "This is excellent. Thanks for sharing" }, { "id": 1888314, "postDate": "2022-08-07T13:40:11.140Z", "content": "

Thank you. This is very useful πŸ‘

", "rawMarkdown": "Thank you. This is very useful πŸ‘" }, { "id": 1885318, "postDate": "2022-08-05T05:08:07.880Z", "content": "

Thank you. This is very useful πŸ‘

", "rawMarkdown": "Thank you. This is very useful πŸ‘" }, { "id": 1881383, "postDate": "2022-08-02T13:17:48.850Z", "content": "

Very helpful, Thank you!

", "rawMarkdown": "Very helpful, Thank you!" }, { "id": 1880525, "postDate": "2022-08-01T19:37:07.977Z", "content": "

Thanks for sharing @thedevastator

", "rawMarkdown": "Thanks for sharing @thedevastator " }, { "id": 1864373, "postDate": "2022-07-21T02:33:00.050Z", "content": "

very good,thanks for sharing

", "rawMarkdown": "very good,thanks for sharing" }, { "id": 1862027, "postDate": "2022-07-19T12:06:02.943Z", "content": "

Nice post, thanks for sharing

", "rawMarkdown": "Nice post, thanks for sharing" }, { "id": 1858651, "postDate": "2022-07-17T05:34:01.190Z", "content": "

Thanks for the tips

", "rawMarkdown": "Thanks for the tips" }, { "id": 1856808, "postDate": "2022-07-15T16:02:23.300Z", "content": "

Wow! Very useful! Thank you!!!

", "rawMarkdown": "Wow! Very useful! Thank you!!!" }, { "id": 1855356, "postDate": "2022-07-14T14:05:09.993Z", "content": "

Great summary, thanks for sharing!!

", "rawMarkdown": "Great summary, thanks for sharing!!" }, { "id": 1851762, "postDate": "2022-07-11T14:05:58.470Z", "content": "

Thanks for the information

", "rawMarkdown": "Thanks for the information" }, { "id": 1851541, "postDate": "2022-07-11T10:51:22.347Z", "content": "

Thank a lost. marked.

", "rawMarkdown": "Thank a lost. marked." } ], "comments": [ { "id": 2325446, "author_name": "Koushikphy", "author_url": "", "post_date": "2023-07-01T10:55:05.263000", "content": "

This is really very helpful.

", "votes": 0, "replies": [] }, { "id": 2192622, "author_name": "ShantamVijayputra", "author_url": "", "post_date": "2023-03-22T19:27:42.413000", "content": "

This is awesome resource , Keep up the great work.πŸ‘πŸ‘

", "votes": 0, "replies": [] }, { "id": 2064823, "author_name": "Steve Chen", "author_url": "", "post_date": "2022-12-14T06:23:32.787000", "content": "

Fantastic resource for beginner, extremely grateful to you! @thedevastator

", "votes": 0, "replies": [] }, { "id": 2050768, "author_name": "Russell Healy", "author_url": "", "post_date": "2022-12-01T01:35:48.947000", "content": "

A fantastic resource all in one spot. Thank you

", "votes": 0, "replies": [] }, { "id": 1911840, "author_name": "jeannief", "author_url": "", "post_date": "2022-08-24T10:53:45.570000", "content": "

Thank you for taking the time to put this together. Very helpful!

", "votes": 0, "replies": [] }, { "id": 1897431, "author_name": "", "author_url": "", "post_date": "2022-08-13T18:02:28.747000", "content": "

This is very useful

", "votes": 0, "replies": [] }, { "id": 1879277, "author_name": "dean1977", "author_url": "", "post_date": "2022-08-01T01:02:11.613000", "content": "

very helpful.

", "votes": 0, "replies": [] }, { "id": 1879239, "author_name": "C4rl05/V", "author_url": "", "post_date": "2022-07-31T23:54:45.377000", "content": "

Hello @thedevastator, so much knowledge, thanks for sharing ✨

", "votes": 0, "replies": [] }, { "id": 1864977, "author_name": "LIPING11", "author_url": "", "post_date": "2022-07-21T12:15:32.993000", "content": "

tttttthhhhhhhhhhhhaaaaaaaaaaaannnnnnnnnnnnnnnnnk

", "votes": 0, "replies": [] }, { "id": 1861816, "author_name": "pluto_21", "author_url": "", "post_date": "2022-07-19T09:04:36.197000", "content": "

Thank you so much for sharing these tips

", "votes": 0, "replies": [] }, { "id": 1860925, "author_name": "Lucia Constantino", "author_url": "", "post_date": "2022-07-18T16:49:56.227000", "content": "

Thank you so much for this! I'm barely starting with Kaggle competitions but I'm saving this one for later:)

", "votes": 0, "replies": [] }, { "id": 1858544, "author_name": "Jonathan Bown", "author_url": "", "post_date": "2022-07-17T02:42:41.103000", "content": "

Thanks so much for putting this together!

", "votes": 0, "replies": [] }, { "id": 1857191, "author_name": "C4rl05/V", "author_url": "", "post_date": "2022-07-15T23:56:53.627000", "content": "

Hello, @thedevastator, thanks for sharing a massive amount of information to keep improving

", "votes": 0, "replies": [] }, { "id": 1857146, "author_name": "SHANKAR T", "author_url": "", "post_date": "2022-07-15T22:09:31.913000", "content": "

Wow !!
\nThank you very much for the amount of time you have contributed to make this…

", "votes": 0, "replies": [] }, { "id": 1855775, "author_name": "Professor Kgwadi", "author_url": "", "post_date": "2022-07-14T22:37:56.487000", "content": "

wow great stuff, thank you for sharing .

", "votes": 0, "replies": [] }, { "id": 1855526, "author_name": "Bilal Suppal", "author_url": "", "post_date": "2022-07-14T16:56:47.500000", "content": "

Nice post, thanks for sharing @thedevastator

", "votes": 0, "replies": [] }, { "id": 1852313, "author_name": "Surdarla", "author_url": "", "post_date": "2022-07-12T01:52:29.773000", "content": "

This is remarkable summary!
\nGreat!

", "votes": 0, "replies": [] }, { "id": 1849096, "author_name": "jxlijunhao", "author_url": "", "post_date": "2022-07-09T08:04:18.123000", "content": "

very helpful.

", "votes": 0, "replies": [] }, { "id": 1848827, "author_name": "Muhammed Tausif", "author_url": "", "post_date": "2022-07-08T23:57:45.223000", "content": "

Nice post, thanks for sharing @thedevastator

", "votes": 0, "replies": [] }, { "id": 1848622, "author_name": "Gaurav Rawat", "author_url": "", "post_date": "2022-07-08T19:02:11.103000", "content": "

You have done a aggregate of aggregates. Thanks somethings I was just looking for found here . I think you should put this in a github repo..

", "votes": 0, "replies": [] }, { "id": 1897691, "author_name": "", "author_url": "", "post_date": "2022-08-14T02:35:51.863000", "content": "", "votes": 0, "replies": [] }, { "id": 1883154, "author_name": "", "author_url": "", "post_date": "2022-08-03T16:02:13.380000", "content": "", "votes": 0, "replies": [] }, { "id": 1848917, "author_name": "", "author_url": "", "post_date": "2022-07-09T03:43:09.170000", "content": "", "votes": 0, "replies": [] }, { "id": 1848302, "author_name": "", "author_url": "", "post_date": "2022-07-08T14:01:53.160000", "content": "", "votes": 0, "replies": [] }, { "id": 1848077, "author_name": "Hongbin Na", "author_url": "", "post_date": "2022-07-08T10:25:17.673000", "content": "

Wow! Very useful! Thank you!!!

", "votes": 1, "replies": [] }, { "id": 2012013, "author_name": "Darrylkwok", "author_url": "", "post_date": "2022-11-01T02:37:48.443000", "content": "

Super useful! Thank you!

", "votes": 0, "replies": [] }, { "id": 2006418, "author_name": "wcq_glhf", "author_url": "", "post_date": "2022-10-27T15:23:37.077000", "content": "

Thanks for the insight!

", "votes": 0, "replies": [] }, { "id": 1992926, "author_name": "kt0011", "author_url": "", "post_date": "2022-10-18T02:48:47.557000", "content": "

Thanks for the insight!

", "votes": 0, "replies": [] }, { "id": 1913373, "author_name": "Sam Forrest", "author_url": "", "post_date": "2022-08-25T09:54:14.937000", "content": "

Thanks for the insight!

", "votes": 0, "replies": [] }, { "id": 1910876, "author_name": "", "author_url": "", "post_date": "2022-08-23T18:43:31.537000", "content": "", "votes": 0, "replies": [] }, { "id": 1888314, "author_name": "", "author_url": "", "post_date": "2022-08-07T13:40:11.140000", "content": "", "votes": 0, "replies": [] }, { "id": 1885318, "author_name": "", "author_url": "", "post_date": "2022-08-05T05:08:07.880000", "content": "", "votes": 0, "replies": [] }, { "id": 1881383, "author_name": "", "author_url": "", "post_date": "2022-08-02T13:17:48.850000", "content": "", "votes": 0, "replies": [] }, { "id": 1880525, "author_name": "", "author_url": "", "post_date": "2022-08-01T19:37:07.977000", "content": "", "votes": 0, "replies": [] }, { "id": 1864373, "author_name": "", "author_url": "", "post_date": "2022-07-21T02:33:00.050000", "content": "", "votes": 0, "replies": [] }, { "id": 1862027, "author_name": "", "author_url": "", "post_date": "2022-07-19T12:06:02.943000", "content": "", "votes": 0, "replies": [] }, { "id": 1858651, "author_name": "", "author_url": "", "post_date": "2022-07-17T05:34:01.190000", "content": "", "votes": 0, "replies": [] }, { "id": 1856808, "author_name": "", "author_url": "", "post_date": "2022-07-15T16:02:23.300000", "content": "", "votes": 0, "replies": [] }, { "id": 1855356, "author_name": "", "author_url": "", "post_date": "2022-07-14T14:05:09.993000", "content": "", "votes": 0, "replies": [] }, { "id": 1851762, "author_name": "", "author_url": "", "post_date": "2022-07-11T14:05:58.470000", "content": "", "votes": 0, "replies": [] }, { "id": 1851541, "author_name": "", "author_url": "", "post_date": "2022-07-11T10:51:22.347000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1848063": "# Tabular Classification - Tips and Tricks\n\n\nSharing a long list of tips and tricks found on previous tabular classification competitions on Kaggle.\nTake this as a list of ideas you can attempt to try for improving your score!\n\n> **Credits:** \n> - Many of the links are based on parts of [this](https://neptune.ai/blog/tabular-data-binary-classification-tips-and-tricks-from-5-kaggle-competitions) blog post (with edits and filtering).\n> - Including links from: [The best of kaggle - GBMs](https://www.kaggle.com/discussions/getting-started/277121).\n> - Also from: [The best of kaggle - Ensembles](https://www.kaggle.com/discussions/getting-started/291437).\n\n### Dealing with larger datasets\n\nAn issue close to our hearts! \n\n* Faster [data loading with pandas.](https://www.kaggle.com/c/home-credit-default-risk/discussion/59575)\n* Data compression techniques to [reduce the size of data by 70%](https://www.kaggle.com/nickycan/compress-70-of-dataset).\n* Optimize the memory by r[educing the size of some attributes.](https://www.kaggle.com/shrutimechlearn/large-data-loading-trick-with-ms-malware-data)\n* Use open-source libraries such as [Dask to read and manipulate the data](https://www.kaggle.com/yuliagm/how-to-work-with-big-datasets-on-16g-ram-dask), it performs parallel computing and saves up memory space.\n* Use [cudf](https://github.com/rapidsai/cudf).\n* Convert data to [parquet](https://arrow.apache.org/docs/python/parquet.html) format.\n* Converting data to [feather](https://medium.com/@snehotosh.banerjee/feather-a-fast-on-disk-format-for-r-and-python-data-frames-de33d0516b03) format.\n* Reducing memory usage for [optimizing RAM](https://www.kaggle.com/mjbahmani/reducing-memory-size-for-ieee).\n\n### Data exploration\n\nData exploration is a must on all competitions, Here are some references from past competitions use them to get some ideas on what exactly to explore.\n\n* EDA for microsoft [malware detection.](https://www.kaggle.com/youhanlee/my-eda-i-want-to-see-all)\n* Time Series [EDA for malware detection.](https://www.kaggle.com/cdeotte/time-split-validation-malware-0-68)\n* Complete [EDA for home credit loan prediction](https://www.kaggle.com/codename007/home-credit-complete-eda-feature-importance).\n* Complete [EDA for Santader prediction.](https://www.kaggle.com/gpreda/santander-eda-and-prediction)\n* EDA for [VSB Power Line Fault Detection.](https://www.kaggle.com/go1dfish/basic-eda)\n\n### Preprocessing\n\nBefore we start to engineer features, we need to do some preprocessing.\nThose invlolve removing missing values, converting categorical variables to numeric, and scaling the data, etc..\nHere are some references from past competitions use them to get some ideas for preprocessing.\n\n* Methods to [tackle class imbalance](https://www.kaggle.com/shahules/tackling-class-imbalance).\n* Data augmentation by [Synthetic Minority Oversampling Technique](https://machinelearningmastery.com/smote-oversampling-for-imbalanced-classification/).\n* Fast inplace [shuffle for augmentation](https://www.kaggle.com/jiweiliu/fast-inplace-shuffle-for-augmentation).\n* Finding [synthetic samples in the dataset.](https://www.kaggle.com/yag320/list-of-fake-samples-and-public-private-lb-split)\n* [Signal denoising](https://www.kaggle.com/jackvial/dwt-signal-denoising) used in signal processing competitions.\n* Finding [patterns of missing data](https://www.kaggle.com/jpmiller/patterns-of-missing-data).\n* Methods to handle [missing data](https://towardsdatascience.com/6-different-ways-to-compensate-for-missing-values-data-imputation-with-examples-6022d9ca0779).\n* An overview of various [encoding techniques for categorical data.](https://www.kaggle.com/shahules/an-overview-of-encoding-techniques)\n* Building [model to predict missing values.](https://www.kaggle.com/c/home-credit-default-risk/discussion/64598)\n* Random [shuffling of data](https://www.kaggle.com/brandenkmurray/randomly-shuffled-data-also-works) to create new synthetic training set.\n\n### Feature engineering\n\nNow we can start to engineer features, below you can find some references from past competitions use them to get some ideas to engineer features. \n\n* Target [encoding cross validation](https://medium.com/@pouryaayria/k-fold-target-encoding-dfe9a594874b/) for better encoding.\n* Entity embedding to [handle categories](https://www.kaggle.com/abhishek/entity-embeddings-to-handle-categories).\n* Encoding c[yclic features for deep learning.](https://www.kaggle.com/avanwyk/encoding-cyclical-features-for-deep-learning)\n* Manual [feature engineering methods](https://www.kaggle.com/willkoehrsen/introduction-to-manual-feature-engineering).\n* Automated feature engineering techniques [using featuretools](https://www.kaggle.com/willkoehrsen/automated-feature-engineering-basics).\n* Top hard crafted features used in [microsoft malware detection](https://www.kaggle.com/sanderf/7th-place-solution-microsoft-malware-prediction).\n* Denoising NN for [feature extraction](https://towardsdatascience.com/applied-deep-learning-part-3-autoencoders-1c083af4d798).\n* Feature engineering [using RAPIDS framework.](https://www.kaggle.com/cdeotte/rapids-feature-engineering-fraud-0-96/)\n* Things to remember while processing f[eatures using LGBM.](https://www.kaggle.com/c/ieee-fraud-detection/discussion/108575)\n* [Lag features and moving averages.](https://www.kaggle.com/c/home-credit-default-risk/discussion/64593)\n* [Principal component analysis](https://medium.com/machine-learning-researcher/dimensionality-reduction-pca-and-lda-6be91734f567) for dimensionality reduction.\n* LDA for [dimensionality reduction](https://medium.com/machine-learning-researcher/dimensionality-reduction-pca-and-lda-6be91734f567).\n* Best hand crafted LGBM features for [microsoft malware detection](https://www.kaggle.com/c/microsoft-malware-prediction/discussion/85157).\n* Generating [frequency features.](https://www.kaggle.com/philippsinger/frequency-features-without-test-data-information)\n* Dropping variables with [different train and test distribution.](https://www.kaggle.com/bogorodvo/lightgbm-baseline-model-using-sparse-matrix)\n* [Aggregate time series features](https://www.kaggle.com/c/home-credit-default-risk/discussion/64593) for home credit competition.\n* [Time Series](https://www.kaggle.com/c/home-credit-default-risk/discussion/64593) features used in home credit default risk.\n* Scale, Standardize and n[ormalize with sklearn](https://towardsdatascience.com/scale-standardize-or-normalize-with-scikit-learn-6ccc7d176a02).\n* Handcrafted features for [Home default risk competition.](https://www.kaggle.com/c/home-credit-default-risk/discussion/57750)\n* Handcrafted [features used in Santander Transaction Prediction.](https://www.kaggle.com/c/santander-customer-transaction-prediction/discussion/89070)\n\n### Feature selection\n\nNow that we got our features engineered, we should check if they are useful or not.\nTo do this, we need to do some feature selection.\nHere are some references from past competitions use them to get some ideas for feature selection.\n\n* Six ways to do [features selection using sklearn](https://www.kaggle.com/sz8416/6-ways-for-feature-selection).\n* [Permutation feature importance](https://www.kaggle.com/c/ieee-fraud-detection/discussion/107877#latest-635386).\n* [Adversarial feature validation](https://www.kaggle.com/tunguz/adversarial-ieee/).\n* Feature selection using [null importances.](https://www.kaggle.com/ogrellier/feature-selection-with-null-importances)\n* Tree explainer using [SHAP.](https://github.com/slundberg/shap)\n* DeepNN explainer using [SHAP](https://github.com/slundberg/shap).\n\n\n### Modeling\n\nBelow you can find [The best of kaggle - GBMs](https://www.kaggle.com/discussions/getting-started/277121).\nThese are the top voted/viewed notebooks all over kaggle for GBMs. \n**Usage rules are simple: You like it? Upvote the original!**\n\n### XGBoost\n \n* [Data Analysis & XGBoost Starter (0.35460 LB)](https://www.kaggle.com/anokas/data-analysis-xgboost-starter-0-35460-lb) (137K views, 1377 Votes)*\n* [House prices: Lasso, XGBoost, and a detailed EDA](https://www.kaggle.com/erikbruin/house-prices-lasso-xgboost-and-a-detailed-eda) (183K views, 1317 Votes)*\n* [Feature engineering, xgboost](https://www.kaggle.com/dlarionov/feature-engineering-xgboost) (144K views, 942 Votes)*\n* [XGBoost](https://www.kaggle.com/dansbecker/xgboost) (173K views, 918 Votes)*\n* [XGBoost](https://www.kaggle.com/alexisbcook/xgboost) (266K views, 578 Votes)*\n* [Market Prediction: XGBoost with GPU (Fit in 1min)](https://www.kaggle.com/hamditarek/market-prediction-xgboost-with-gpu-fit-in-1min) (53K views, 411 Votes)*\n* [A Guide on XGBoost hyperparameters tuning](https://www.kaggle.com/prashant111/a-guide-on-xgboost-hyperparameters-tuning) (55K views, 358 Votes)*\n* [Titanic WCG+XGBoost [0.84688]](https://www.kaggle.com/cdeotte/titanic-wcg-xgboost-0-84688) (27K views, 314 Votes)*\n* [XGBoost & Feature Selection DSBowl πŸ₯£ πŸ₯£](https://www.kaggle.com/shahules/xgboost-feature-selection-dsbowl) (36K views, 299 Votes)*\n* [Understanding XGBoost Model on Otto Data](https://www.kaggle.com/tqchen/understanding-xgboost-model-on-otto-data) (130K views, 297 Votes)*\n\n### LightGBM\n \n* [LightGBM with Simple Features](https://www.kaggle.com/jsaguiar/lightgbm-with-simple-features) (85K views, 619 Votes)*\n* [LightGBM. Baseline Model Using Sparse Matrix](https://www.kaggle.com/bogorodvo/lightgbm-baseline-model-using-sparse-matrix) (49K views, 409 Votes)*\n* [LightGBM starter with feature engineering idea](https://www.kaggle.com/tommy1028/lightgbm-starter-with-feature-engineering-idea) (16K views, 370 Votes)*\n* [Google Analytics EDA + LightGBM + Screenshots](https://www.kaggle.com/erikbruin/google-analytics-eda-lightgbm-screenshots) (40K views, 344 Votes)*\n* [Feature Engineering & LightGBM](https://www.kaggle.com/davidcairuz/feature-engineering-lightgbm) (26K views, 272 Votes)*\n* [Simple LightGBM without blending](https://www.kaggle.com/mfjwr1/simple-lightgbm-without-blending) (18K views, 263 Votes)*\n* [LightGBM + XGBoost + Catboost](https://www.kaggle.com/samratp/lightgbm-xgboost-catboost) (37K views, 245 Votes)*\n* [ASHRAE- KFold LightGBM - without leak (1.08)](https://www.kaggle.com/aitude/ashrae-kfold-lightgbm-without-leak-1-08) (15K views, 240 Votes)*\n* [LightGBM Classifier in Python](https://www.kaggle.com/prashant111/lightgbm-classifier-in-python) (52K views, 231 Votes)*\n* [LightGBM (Fixing unbalanced data)](https://www.kaggle.com/pranav84/lightgbm-fixing-unbalanced-data-lb-0-9680) (56K views, 228 Votes)*\n\n### CatBoost\n \n* [A new baseline for DSB 2019 - Catboost model](https://www.kaggle.com/mhviraf/a-new-baseline-for-dsb-2019-catboost-model) (15K views, 316 Votes)*\n* [LightGBM + XGBoost + Catboost](https://www.kaggle.com/samratp/lightgbm-xgboost-catboost) (37K views, 245 Votes)*\n* [CatBoost: A Deeper Dive](https://www.kaggle.com/abhinand05/catboost-a-deeper-dive) (8K views, 178 Votes)*\n* [House Prices Tutorial with Catboost](https://www.kaggle.com/allunia/house-prices-tutorial-with-catboost) (12K views, 176 Votes)*\n* [Stacking Test-Sklearn, XGBoost, CatBoost, LightGBM](https://www.kaggle.com/eliotbarr/stacking-test-sklearn-xgboost-catboost-lightgbm) (25K views, 173 Votes)*\n* [WiDSDatathon2021-Catboost-Starter](https://www.kaggle.com/usharengaraju/widsdatathon2021-catboost-starter) (6K views, 167 Votes)*\n* [Tutorial: CatBoost Overview](https://www.kaggle.com/mitribunskiy/tutorial-catboost-overview) (39K views, 127 Votes)*\n* [CatBoost Classifier in Python](https://www.kaggle.com/prashant111/catboost-classifier-in-python) (28K views, 126 Votes)*\n* [Catboost - Some more features](https://www.kaggle.com/braquino/catboost-some-more-features) (7K views, 103 Votes)*\n* [mlcourse.ai. Fall 2019. Catboost starter](https://www.kaggle.com/kashnitsky/mlcourse-ai-fall-2019-catboost-starter) (18K views, 98 Votes)*\n\n\n### Ensemble\n\nNow that we have our model trained, we should ensemble our predictions.\n\nBelow you can find [The best of kaggle - Ensembles](https://www.kaggle.com/discussions/getting-started/291437).\nThese are the top voted/viewed notebooks all over kaggle for GBMs.\n**Usage rules are simple: You like it? Upvote the original!**\n\n### Ensembling Techniques\n\n* [Introduction to Ensembling/Stacking in Python](https://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python) (620K views, 5436 Votes)*\n* [Titanic Top 4% with ensemble modeling](https://www.kaggle.com/yassineghouzam/titanic-top-4-with-ensemble-modeling) (185K views, 2501 Votes)*\n* [EDA & Ensemble Model (Top 10 Percentile)](https://www.kaggle.com/viveksrinivasan/eda-ensemble-model-top-10-percentile) (98K views, 513 Votes)*\n* [Analysis of Melanoma Metadata and EffNet Ensemble](https://www.kaggle.com/datafan07/analysis-of-melanoma-metadata-and-effnet-ensemble) (24K views, 405 Votes)*\n* [Detailed Data Analysis & Ensemble Modeling](https://www.kaggle.com/tannercarbonati/detailed-data-analysis-ensemble-modeling) (55K views, 360 Votes)*\n* [Ensemble Folds with MEDIAN - [0.153]](https://www.kaggle.com/cdeotte/ensemble-folds-with-median-0-153) (9K views, 303 Votes)*\n* [WBF approach for ensemble](https://www.kaggle.com/shonenkov/wbf-approach-for-ensemble) (18K views, 282 Votes)*\n* [Ensemble: Resnext50\\_32x4d + Efficientnet = 0.903](https://www.kaggle.com/japandata509/ensemble-resnext50-32x4d-efficientnet-0-903) (16K views, 274 Votes)*\n* [Employee attrition via Ensemble tree-based methods](https://www.kaggle.com/arthurtok/employee-attrition-via-ensemble-tree-based-methods) (64K views, 253 Votes)*\n* [Top 3%. Efficient ensembling in few lines of code](https://www.kaggle.com/isaienkov/top-3-efficient-ensembling-in-few-lines-of-code) (30K views, 242 Votes)*\n* [Ensemble Learning Techniques Tutorial](https://www.kaggle.com/pavansanagapati/ensemble-learning-techniques-tutorial) (55K views, 215 Votes)*\n* [Minimal LSTM + NB-SVM baseline ensemble](https://www.kaggle.com/jhoward/minimal-lstm-nb-svm-baseline-ensemble) (30K views, 209 Votes)*\n* [Ensemble Model: Stacked Model Example](https://www.kaggle.com/jimthompson/ensemble-model-stacked-model-example) (68K views, 207 Votes)*\n* [πŸ€—πŸ€— Ensemble πŸ€—πŸ€—](https://www.kaggle.com/hamditarek/ensemble) (64K views, 196 Votes)*\n* [EDA&Modelling of the External Data Inc. Ensemble](https://www.kaggle.com/datafan07/eda-modelling-of-the-external-data-inc-ensemble) (8K views, 195 Votes)*\n\n### Blending\n \n* [Simple LightGBM without blending](https://www.kaggle.com/mfjwr1/simple-lightgbm-without-blending) (18K views, 265 Votes)*\n* [Blending tensorflow and pytorchπŸ”₯πŸ”₯πŸ”₯](https://www.kaggle.com/a763337092/blending-tensorflow-and-pytorch) (13K views, 220 Votes)*\n* [Another model for your blending](https://www.kaggle.com/speedwagon/quadratic-discriminant-analysis) (11K views, 216 Votes)*\n* [Optimise Blending Weights with Bonus :0](https://www.kaggle.com/gogo827jz/optimise-blending-weights-with-bonus-0) (7K views, 184 Votes)*\n* [Blending with Linear Regression [0.688 LB]](https://www.kaggle.com/suicaokhoailang/blending-with-linear-regression-0-688-lb) (14K views, 136 Votes)*\n* [blending blending blending](https://www.kaggle.com/abhishek/blending-blending-blending) (3K views, 128 Votes)*\n* [Lessons from Toxic : Blending is the new sexy](https://www.kaggle.com/jagangupta/lessons-from-toxic-blending-is-the-new-sexy) (10K views, 128 Votes)*\n* [non-blending lightGBM model LB: 0.977](https://www.kaggle.com/bk0000/non-blending-lightgbm-model-lb-0-977) (16K views, 127 Votes)*\n* [Bayesian Optimization Seed Blending](https://www.kaggle.com/hengzheng/bayesian-optimization-seed-blending) (9K views, 118 Votes)*\n* [elo\\_world\\_high\\_score\\_without\\_blending](https://www.kaggle.com/gpreda/elo-world-high-score-without-blending) (8K views, 112 Votes)*\n* [competition part-5: blending 101](https://www.kaggle.com/abhishek/competition-part-5-blending-101) (3K views, 105 Votes)*\n* [Beginner EDA with Feature Eng. and Blending Models](https://www.kaggle.com/datafan07/beginner-eda-with-feature-eng-and-blending-models) (3K views, 99 Votes)*\n* [Cross-validation, weighted linear blending, errors](https://www.kaggle.com/tilii7/cross-validation-weighted-linear-blending-errors) (7K views, 84 Votes)*\n* [Blending TensorFlow + PyTorch (th=0.4914)](https://www.kaggle.com/sagarjiyani/blending-tensorflow-pytorch-th-0-4914) (5K views, 82 Votes)*\n* [Blending NN and LGBM/RF](https://www.kaggle.com/gogo827jz/blending-nn-and-lgbm-rf) (6K views, 81 Votes)*\n\n### Stacking\n \n* [Stacked Regressions : Top 4% on LeaderBoard](https://www.kaggle.com/serigne/stacked-regressions-top-4-on-leaderboard) (493K views, 6236 Votes)*\n* [Introduction to Ensembling/Stacking in Python](https://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python) (620K views, 5436 Votes)*\n* [Model stacking, feature engineering and EDA](https://www.kaggle.com/dimitreoliveira/model-stacking-feature-engineering-and-eda) (43K views, 409 Votes)*\n* [Stacking House Prices - Walkthrough to Top 5%](https://www.kaggle.com/agodwinp/stacking-house-prices-walkthrough-to-top-5) (32K views, 227 Votes)*\n* [Stacking Starter](https://www.kaggle.com/mmueller/stacking-starter) (41K views, 221 Votes)*\n* [Ensemble Model: Stacked Model Example](https://www.kaggle.com/jimthompson/ensemble-model-stacked-model-example) (68K views, 207 Votes)*\n* [Titanic: Voting, Pipeline, Stack, and Guide](https://www.kaggle.com/nicapotato/titanic-voting-pipeline-stack-and-guide) (21K views, 197 Votes)*\n* [Explore Stacking (LB 0.1463)](https://www.kaggle.com/dongxu027/explore-stacking-lb-0-1463) (18K views, 180 Votes)*\n* [Stacking Test-Sklearn, XGBoost, CatBoost, LightGBM](https://www.kaggle.com/eliotbarr/stacking-test-sklearn-xgboost-catboost-lightgbm) (26K views, 176 Votes)*\n* [Top 1% Approach: EDA, New Models and Stacking](https://www.kaggle.com/datafan07/top-1-approach-eda-new-models-and-stacking) (9K views, 172 Votes)*\n* [Simple Stacker LB 0.284](https://www.kaggle.com/yekenot/simple-stacker-lb-0-284) (20K views, 172 Votes)*\n* [Stack&Blend LRs XGB LGB {House Prices K} v17](https://www.kaggle.com/itslek/stack-blend-lrs-xgb-lgb-house-prices-k-v17) (6K views, 164 Votes)*\n* [Top 10 (0.10943): stacking, MICE and brutal force](https://www.kaggle.com/agehsbarg/top-10-0-10943-stacking-mice-and-brutal-force) (21K views, 156 Votes)*\n* [OOF stacking regime](https://www.kaggle.com/hhstrand/oof-stacking-regime) (14K views, 145 Votes)*\n* [EloDA with Feature Engineering and Stacking](https://www.kaggle.com/tunguz/eloda-with-feature-engineering-and-stacking) (10K views, 141 Votes)*\n\n### Bagging\n \n* [Riiid LGBM bagging2 + SAKT =0.781](https://www.kaggle.com/ammarnassanalhajali/riiid-lgbm-bagging2-sakt-0-781) (11K views, 188 Votes)*\n* [SAKT + Riiid LGBM bagging2](https://www.kaggle.com/leadbest/sakt-riiid-lgbm-bagging2) (8K views, 139 Votes)*\n* [Topic 5. Ensembles. Part 1. Bagging](https://www.kaggle.com/kashnitsky/topic-5-ensembles-part-1-bagging) (16K views, 137 Votes)*\n* [Bagging vs Boosting](https://www.kaggle.com/prashant111/bagging-vs-boosting) (22K views, 117 Votes)*\n* [Fork of Riiid LGBM bagging2.1 471152](https://www.kaggle.com/julianguo/fork-of-riiid-lgbm-bagging2-1-471152) (8K views, 117 Votes)*\n* [Riiid! LGBM bagging2](https://www.kaggle.com/zephyrwang666/riiid-lgbm-bagging2) (6K views, 113 Votes)*\n* [Predicting House Prices [XGB/RF/Bagging-Reg Pipe]](https://www.kaggle.com/kabure/predicting-house-prices-xgb-rf-bagging-reg-pipe) (10K views, 89 Votes)*\n* [Keras starter with bagging (LB: 1120.596)](https://www.kaggle.com/danijelk/keras-starter-with-bagging-lb-1120-596) (18K views, 77 Votes)*\n* [Keras starter with bagging 1111.84364](https://www.kaggle.com/mtinti/keras-starter-with-bagging-1111-84364) (19K views, 77 Votes)*\n* [Riiid LGBM bagging2.1](https://www.kaggle.com/zephyrwang666/riiid-lgbm-bagging2-1) (4K views, 62 Votes)*\n* [BaggingRegressor + RAPIDS Ensemble](https://www.kaggle.com/andypenrose/baggingregressor-rapids-ensemble) (4K views, 59 Votes)*\n* [Credit analysis with KNN/DTree/RF/Bagging/ANN🏦](https://www.kaggle.com/fahadmehfoooz/credit-analysis-with-knn-dtree-rf-bagging-ann) (<1K views, 55 Votes)*\n* [Sentiment Analysis:SVM/NB/Bagging/Boosting/RF/LSTM](https://www.kaggle.com/fahadmehfoooz/sentiment-analysis-svm-nb-bagging-boosting-rf-lstm) (1K views, 51 Votes)*\n* [Ensemble ML Algorithms : Bagging, Boosting, Voting](https://www.kaggle.com/faressayah/ensemble-ml-algorithms-bagging-boosting-voting) (8K views, 48 Votes)*\n* [Simple XGBoost CV Bagging](https://www.kaggle.com/cttsai/simple-xgboost-cv-bagging) (3K views, 42 Votes)*\n\n### Boosting\n \n* [Boosting creativity towards feature engineering](https://www.kaggle.com/felipemello/boosting-creativity-towards-feature-engineering) (9K views, 229 Votes)*\n* [Vehicle Insurance EDA and boosting models](https://www.kaggle.com/yashvi/vehicle-insurance-eda-and-boosting-models) (19K views, 200 Votes)*\n* [Topic 10. Gradient Boosting](https://www.kaggle.com/kashnitsky/topic-10-gradient-boosting) (21K views, 188 Votes)*\n* [Gradient boosting simplified](https://www.kaggle.com/grroverpr/gradient-boosting-simplified) (48K views, 147 Votes)*\n* [Bagging vs Boosting](https://www.kaggle.com/prashant111/bagging-vs-boosting) (22K views, 117 Votes)*\n* [Diabetes Prediction (Stacking + Boosting)](https://www.kaggle.com/niteshyadav3103/diabetes-prediction-stacking-boosting) (1K views, 85 Votes)*\n* [Elo Stack With Goss Boosting](https://www.kaggle.com/roydatascience/elo-stack-with-goss-boosting) (6K views, 78 Votes)*\n* [Assignment 10. Gradient boosting and flight delays](https://www.kaggle.com/kashnitsky/assignment-10-gradient-boosting-and-flight-delays) (9K views, 71 Votes)*\n* [Battle of the Boosting Algos: LGB, XGB, Catboost](https://www.kaggle.com/lavanyashukla01/battle-of-the-boosting-algos-lgb-xgb-catboost) (9K views, 55 Votes)*\n* [Introduction to Boosting using LGBM(LB: 0.68357)](https://www.kaggle.com/vinnsvinay/introduction-to-boosting-using-lgbm-lb-0-68357) (14K views, 52 Votes)*\n* [FE-Pipeline with HistGradientBoostingRegressor](https://www.kaggle.com/tunguz/fe-pipeline-with-histgradientboostingregressor) (3K views, 51 Votes)*\n* [Sentiment Analysis:SVM/NB/Bagging/Boosting/RF/LSTM](https://www.kaggle.com/fahadmehfoooz/sentiment-analysis-svm-nb-bagging-boosting-rf-lstm) (1K views, 51 Votes)*\n* [Explainable Boosting machines for Tabular data](https://www.kaggle.com/parulpandey/explainable-boosting-machines-for-tabular-data) (2K views, 49 Votes)*\n* [Ensemble ML Algorithms : Bagging, Boosting, Voting](https://www.kaggle.com/faressayah/ensemble-ml-algorithms-bagging-boosting-voting) (8K views, 48 Votes)*\n* [House Price Prediction : GBoosting,AdaBoost etc.](https://www.kaggle.com/sid321axn/house-price-prediction-gboosting-adaboost-etc) (8K views, 42 Votes)*\n\n\n### Hyperparameters Tuning\n\n* LGBM [hyperparameter tuning](https://www.kaggle.com/mlisovyi/lightgbm-hyperparameter-optimisation-lb-0-761) methods.\n* Automated [model tuning](https://www.kaggle.com/willkoehrsen/automated-model-tuning) methods.\n* Parameter tuning with [hyper plot](https://www.kaggle.com/bigironsphere/parameter-tuning-in-one-function-with-hyperopt).\n* [Bayesian optimization](http://krasserm.github.io/2018/03/21/bayesian-optimization/) for hyperparameter tuning.\n* [Gpyopt Hyperparameter Optimisation](https://www.kaggle.com/nicapotato/gpyopt-hyperparameter-optimisation-gpu-lgbm).", "2325446": "This is really very helpful.", "2192622": "This is awesome resource , Keep up the great work.πŸ‘πŸ‘", "2064823": "Fantastic resource for beginner, extremely grateful to you! @thedevastator ", "2050768": "A fantastic resource all in one spot. Thank you", "1911840": "Thank you for taking the time to put this together. Very helpful!", "1897431": "This is very useful ", "1879277": "very helpful.", "1879239": "Hello @thedevastator, so much knowledge, thanks for sharing ✨", "1864977": "tttttthhhhhhhhhhhhaaaaaaaaaaaannnnnnnnnnnnnnnnnk", "1861816": "Thank you so much for sharing these tips", "1860925": "Thank you so much for this! I'm barely starting with Kaggle competitions but I'm saving this one for later:)", "1858544": "Thanks so much for putting this together!", "1857191": "Hello, @thedevastator, thanks for sharing a massive amount of information to keep improving", "1857146": "Wow !!\nThank you very much for the amount of time you have contributed to make this...", "1855775": "wow great stuff, thank you for sharing .", "1855526": "Nice post, thanks for sharing @thedevastator", "1852313": "This is remarkable summary!\nGreat!", "1849096": "very helpful.", "1848827": "Nice post, thanks for sharing @thedevastator ", "1848622": "You have done a aggregate of aggregates. Thanks somethings I was just looking for found here . I think you should put this in a github repo.. ", "1897691": "", "1883154": "", "1848917": "", "1848302": "", "1848077": "Wow! Very useful! Thank you!!!", "2012013": "Super useful! Thank you! ", "2006418": "Thanks for the insight!", "1992926": "Thanks for the insight!", "1913373": "Thanks for the insight!", "1910876": "This is excellent. Thanks for sharing", "1888314": "Thank you. This is very useful πŸ‘", "1885318": "Thank you. This is very useful πŸ‘", "1881383": "Very helpful, Thank you!", "1880525": "Thanks for sharing @thedevastator ", "1864373": "very good,thanks for sharing", "1862027": "Nice post, thanks for sharing", "1858651": "Thanks for the tips", "1856808": "Wow! Very useful! Thank you!!!", "1855356": "Great summary, thanks for sharing!!", "1851762": "Thanks for the information", "1851541": "Thank a lost. marked." } }