{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Dealing With Class Imbalance For Classification Problems !","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Method 1: Oversampling Underrepresented Class ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# SMOTE Method [Synthetic Minority Oversampling Technique]","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Imbalanced classification involves developing predictive models on classification datasets that have a severe class imbalance.\n\nThe challenge of working with imbalanced datasets is that most machine learning techniques will ignore, and in turn have poor performance on, the minority class, although typically it is performance on the minority class that is most important.\n\nOne approach to addressing imbalanced datasets is to oversample the minority class. The simplest approach involves duplicating examples in the minority class, although these examples don’t add any new information to the model. Instead, new examples can be synthesized from the existing examples. This is a type of data augmentation for the minority class and is referred to as the Synthetic Minority Oversampling Technique, or SMOTE for short.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Method 2: Underrepresenting Overrepresented Classes","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Random Under-sampling\nRandom under-sampling is a technique that randomly selects a number of images from the majority class and removes them from the dataset. This reduces the number of observations from the majority class, which may help the data to get balanced. \n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# METHOD 3: Class Weighting the loss function. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Poor predictions of under weighted classes are penalised more heavily in the loss function. Something along the lines of\nweights = [w1, w2, w3, ...]\nclass_weights = torch.FloatTensor(weights)\nlearn.crit = nn.CrossEntropyLoss(weight=class_weights)\nwhere you weight w1, w2, w3 in whatever way you wish. I’ve worked with sample size / (num classes * class frequency) but have to admit I’ve not had much luck getting class weighting to work well.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Use the right evaluation metrics \n\nApplying inappropriate evaluation metrics for model generated using imbalanced data can be dangerous. Imagine our training data is the one illustrated in graph above. If accuracy is used to measure the goodness of a model, a model which classifies all testing samples into “0” will have an excellent accuracy (99.8%), but obviously, this model won’t provide any valuable information for us.\n\nIn this case, other alternative evaluation metrics can be applied such as:\n\nPrecision/Specificity: how many selected instances are relevant.\n\nRecall/Sensitivity: how many relevant instances are selected.\n\nF1 score: harmonic mean of precision and recall.\n\nMCC: correlation coefficient between the observed and predicted binary classifications.\n\nAUC: relation between true-positive rate and false positive rate.\n\n# Use K-fold Cross-Validation in the right way\n\nIt is noteworthy that cross-validation should be applied properly while using over-sampling method to address imbalance problems.\n\nIf cross-validation is applied after over-sampling, basically what we are doing is overfitting our model to a specific artificial bootstrapping result. That is why cross-validation should always be done before over-sampling the data,  Only by resampling the data repeatedly, randomness can be introduced into the dataset to make sure that there won’t be an overfitting problem.\n\n# Ensemble different resampled datasets\n \nThe easiest way to successfully generalize a model is by using more data. The problem is that out-of-the-box classifiers like logistic regression or random forest tend to generalize by discarding the rare class. One easy best practice is building n models that use all the samples of the rare class and n-differing samples of the abundant class. Given that you want to ensemble 10 models, you would keep e.g. the 1.000 cases of the rare class and randomly sample 10.000 cases of the abundant class. Then you just split the 10.000 cases in 10 chunks and train 10 different models.\n\n\n","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}