{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# We are going to use AutoViML and PyCaret to build better models for Housing Prices\n## Please turn on the GPU on this kernel to the right in Accelerator => GPU\n###  Please \n### This is a modified version of a fantastic original notebook here:\nhttps://www.kaggle.com/pavansanagapati/6-useful-automated-ml-tools-for-data-scientists\n","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:04:42.132887Z","iopub.execute_input":"2021-06-19T17:04:42.133267Z","iopub.status.idle":"2021-06-19T17:04:42.138195Z","shell.execute_reply.started":"2021-06-19T17:04:42.133231Z","shell.execute_reply":"2021-06-19T17:04:42.136674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4.1 Load Dataset<a id=\"41\"></a> <br>\n\nTo demonstrate the pycaret capability we will use a dataset from UCI called **Default of Credit Card Clients Dataset**. This dataset contains information on default payments, demographic factors, credit data, payment history, and billing statements of credit card clients in Taiwan from April 2005 to September 2005. There are 24,000 samples and 25 features. Short descriptions of each column are as follows:\n\n- **ID:** ID of each client\n- **LIMIT_BAL:** Amount of given credit in NT dollars (includes individual and family/supplementary credit)\n- **SEX:** Gender (1=male, 2=female)\n- **EDUCATION:** (1=graduate school, 2=university, 3=high school, 4=others, 5=unknown, 6=unknown)\n- **MARRIAGE:** Marital status (1=married, 2=single, 3=others)\n- **AGE:** Age in years\n- **PAY_0 to PAY_6:** Repayment status by n months ago (PAY_0 = last month ... PAY_6 = 6 months ago) (Labels: -1=pay duly, 1=payment delay for one month, 2=payment delay for two months, ... 8=payment delay for eight months, 9=payment delay for nine months and above)\n- **BILL_AMT1 to BILL_AMT6:** Amount of bill statement by n months ago ( BILL_AMT1 = last_month .. BILL_AMT6 = 6 months ago)\n- **PAY_AMT1 to PAY_AMT6:** Amount of payment by n months ago ( BILL_AMT1 = last_month .. BILL_AMT6 = 6 months ago)\n- **default.payment.next.month:** Default payment (1=yes, 0=no) `Target Column`","metadata":{}},{"cell_type":"code","source":"import pandas as pd\n#data=pd.read_csv('../input/default-of-credit-card-clients-dataset/UCI_Credit_Card.csv')\ndata = pd.read_csv('../input/house-prices-advanced-regression-techniques/train.csv')\nprint(data.shape)\ndata.head()","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:04:43.514926Z","iopub.execute_input":"2021-06-19T17:04:43.515254Z","iopub.status.idle":"2021-06-19T17:04:43.602417Z","shell.execute_reply.started":"2021-06-19T17:04:43.515225Z","shell.execute_reply":"2021-06-19T17:04:43.601171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In order to demonstrate the predict_model() function on unseen data, a sample of 1500 records has been withheld from the original dataset to be used for predictions. This should not be confused with a train/test split as this particular split is performed to simulate a real life scenario. Another way to think about this is that these 1500 records are not available at the time when the machine learning experiment was performed.","metadata":{}},{"cell_type":"code","source":"dataset = data.sample(frac=0.95, random_state=786).reset_index(drop=True)\ndata_unseen = data.drop(dataset.index).reset_index(drop=True)\n\nprint('Data for Modeling: ' + str(data.shape))\nprint('Unseen Data For Predictions: ' + str(data_unseen.shape))","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:04:44.917307Z","iopub.execute_input":"2021-06-19T17:04:44.917706Z","iopub.status.idle":"2021-06-19T17:04:44.933664Z","shell.execute_reply.started":"2021-06-19T17:04:44.917662Z","shell.execute_reply":"2021-06-19T17:04:44.932668Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#target = 'default.payment.next.month'\ntarget = 'SalePrice'\ndataset.head(1)","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:04:45.678693Z","iopub.execute_input":"2021-06-19T17:04:45.679131Z","iopub.status.idle":"2021-06-19T17:04:45.717399Z","shell.execute_reply.started":"2021-06-19T17:04:45.679100Z","shell.execute_reply":"2021-06-19T17:04:45.716353Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Let's build a model using Auto_ViML first","metadata":{}},{"cell_type":"code","source":"!pip install autoviml --upgrade","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:04:46.980976Z","iopub.execute_input":"2021-06-19T17:04:46.981529Z","iopub.status.idle":"2021-06-19T17:04:59.299132Z","shell.execute_reply.started":"2021-06-19T17:04:46.981486Z","shell.execute_reply":"2021-06-19T17:04:59.297494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from autoviml.Auto_ViML import Auto_ViML","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:04:59.306727Z","iopub.execute_input":"2021-06-19T17:04:59.309709Z","iopub.status.idle":"2021-06-19T17:05:05.825578Z","shell.execute_reply.started":"2021-06-19T17:04:59.309662Z","shell.execute_reply":"2021-06-19T17:05:05.824568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## You have to just give the dataset, data_unseen and target variable. That's all!","metadata":{}},{"cell_type":"code","source":" m, feats, trainm, testm = Auto_ViML(dataset, target, data_unseen,\n                            sample_submission='',\n                            scoring_parameter='', KMeans_Featurizer=False,\n                            hyper_param='RS',feature_reduction=True,\n                             Boosting_Flag=True, Binning_Flag=False,\n                            Add_Poly=0, Stacking_Flag=False,Imbalanced_Flag=True,\n                            verbose=2)","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:05:47.950997Z","iopub.execute_input":"2021-06-19T17:05:47.951375Z","iopub.status.idle":"2021-06-19T17:06:19.080817Z","shell.execute_reply.started":"2021-06-19T17:05:47.951343Z","shell.execute_reply":"2021-06-19T17:06:19.079521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let's Compare it to PyCaret","metadata":{}},{"cell_type":"code","source":"!pip install pycaret","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:06:30.813909Z","iopub.execute_input":"2021-06-19T17:06:30.814281Z","iopub.status.idle":"2021-06-19T17:07:24.785537Z","shell.execute_reply.started":"2021-06-19T17:06:30.814249Z","shell.execute_reply":"2021-06-19T17:07:24.784388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#import regression and classification modules from pycaret\n#from pycaret.classification import *\nfrom pycaret.regression import *","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:07:49.727652Z","iopub.execute_input":"2021-06-19T17:07:49.728060Z","iopub.status.idle":"2021-06-19T17:07:51.475942Z","shell.execute_reply.started":"2021-06-19T17:07:49.728011Z","shell.execute_reply":"2021-06-19T17:07:51.474754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"help(setup)","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:10:09.603266Z","iopub.execute_input":"2021-06-19T17:10:09.603682Z","iopub.status.idle":"2021-06-19T17:10:09.615607Z","shell.execute_reply.started":"2021-06-19T17:10:09.603651Z","shell.execute_reply":"2021-06-19T17:10:09.614119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" reg = setup(data = data, target = target, train_size=0.8,\n                ignore_features=['Id'], session_id=21, imputation_type='iterative',\n                normalize=True, pca=True, pca_method='kernel', \n                transform_target=False, ignore_low_variance = True, \n                combine_rare_levels = True, remove_outliers=True)","metadata":{"execution":{"iopub.status.busy":"2021-06-19T17:15:56.818687Z","iopub.execute_input":"2021-06-19T17:15:56.819201Z","iopub.status.idle":"2021-06-19T17:18:35.216649Z","shell.execute_reply.started":"2021-06-19T17:15:56.819158Z","shell.execute_reply":"2021-06-19T17:18:35.212802Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"compare_models()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There you go created over 15 models using 10 fold stratified cross validation and evaluated the 6 most commonly used classification metrics (Accuracy, AUC, Recall, Precision, F1, Kappa). The score grid printed above highlights the highest performing metric for comparison purposes only. The grid by default is sorted using 'Accuracy' (highest to lowest) which can be changed by passing the sort parameter. For example **compare_models(sort = 'Recall')** will sort the grid by Recall instead of Accuracy. If you want to change the fold parameter from the default value of 10 to a different value then you can use the fold parameter. For example **compare_models(fold = 5)** will compare all models on 5 fold cross validation. Reducing the number of folds will improve the training time.\n## 4.3 Create model<a id=\"43\"></a> <br>\nWhile compare_models() is a powerful function and often a starting point in any experiment, it does not return any trained models. PyCaret's recommended experiment workflow is to use compare_models() right after setup to evaluate top performing models and finalize a few candidates for continued experimentation. As such, the function that actually allows to you create a model is unimaginatively called **create_model()**.\n\nThere are 18 classifiers available in the model library of PyCaret. \n\nFor illustration purposes only we will be considering the following Classifiers .\n\n* Logistic Regression('lr')\n* Decision Tree Classifier ('dt')\n* K Neighbors Classifier ('knn')\n* Random Forest Classifier ('rf')","metadata":{}},{"cell_type":"code","source":"### we can remove a few models \nlr = compare_models(exclude = ['en','dt','omp', 'gbr','ada', 'par'], n_select=2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr[1]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rf = create_model('rf')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Notice that the mean score of all models matches with the score printed in compare_models(). This is because the metrics printed in the compare_models() score grid are the average scores across all CV folds. Similar to compare_models(), if you want to change the fold parameter from the default value of 10 to a different value then you can use the fold parameter. For Example: create_model('dt', fold = 5) will create a Decision Tree Classifier using 5 fold stratified CV.\n## 4.4 Tune model<a id=\"44\"></a> <br>\nWhen a model is created using the create_model() function it uses the default hyperparameters. In order to tune hyperparameters, the tune_model() function is used. This function automatically tunes the hyperparameters of a model on a pre-defined search space and scores it using stratified cross validation. The output prints a score grid that shows Accuracy, AUC, Recall, Precision, F1 and Kappa by fold.\n\nNow let us tune the below models \n* Logistic Regression('lr')\n* Decision Tree Classifier ('dt')\n* K Neighbors Classifier ('knn')\n* Random Forest Classifier ('rf')","metadata":{}},{"cell_type":"code","source":"# Tune the Logistic regression model\ntuned_lr = tune_model('lr')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tune the Decision Tree Classifier model\ntuned_dt = tune_model('dt')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tune the K Neighbors Classifier model\ntuned_knn = tune_model('knn')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tune the Random Forest Classifier model\ntuned_rf = tune_model('rf')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:**\n\nNotice how the results after tuning have been improved:\n\n* Logistic Regression(Before: 0.7786 , After: 0.7786)\n* Decision Tree Classifier (Before: 0.7216 , After: 0.7413)\n* K Neighbors Classifier (Before: 0.7355 , After: 0.7772)\n* Random Forest Classifier (Before: 0.8015 , After: 0.8103)\n\n## 4.5 Plot Model<a id=\"45\"></a> <br>\n\nBefore model finalization, the `plot_model()` function can be used to analyze the performance across different aspects such as AUC, confusion_matrix, decision boundary etc. This function takes a trained model object and returns a plot based on the test / hold-out set. \n\nThere are 15 different plots available.","metadata":{}},{"cell_type":"code","source":"#Plot LR model: ROC-AUC curve\nplot_model(lr)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Plot LR model: ROC-AUC curve\nplot_model(tuned_lr)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Plot Decision Tree model: ROC-AUC curve\nplot_model(dt)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Plot KNN model: ROC-AUC curve\nplot_model(knn)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To analyze the performance of models is to use the **evaluate_model()** function which displays a user interface for all of the available plots for a given model. It internally uses the plot_model() function.","metadata":{}},{"cell_type":"code","source":"evaluate_model(lr)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#create a tree base model to interpret model and check feature importance\ndt = create_model('dt')\n#interpret a model\ninterpret_model(dt)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#optimize threshold for trained LR model\noptimize_threshold(lr)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 6.AutoViz<a id=\"6\"></a> <br>\n![](https://github.com/AutoViML/AutoViz/raw/master/logo.png)\nAutomatically Visualize any dataset, any size with a single line of code.\n\nAutoViz performs automatic visualization of any dataset with one line. Give any input file (CSV, txt or json) and AutoViz will visualize it.","metadata":{}},{"cell_type":"code","source":"!pip install autoviz","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nfrom autoviz.AutoViz_Class import AutoViz_Class\nAV = AutoViz_Class()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sep = ','\ntarget = 'medv'\ndatapath = ''\nfilename = 'https://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/csv/MASS/Boston.csv'\ndft = AV.AutoViz(datapath+filename, sep=sep, depVar=target, dfte='', header=0, verbose=2,\n                            lowess=False,chart_format='svg',max_rows_analyzed=1500,max_cols_analyzed=30)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion <a id=\"8\"></a> <br>\n\nHence Automated ML tools is enabling data scientists to improve their productivity and realize their true potential quickly and time to market with quicker insights. I hope you find this kernel useful and will use the above tools to good effect in your day to day data science career path.\n\n# If you like this kernel greatly appreciate to <font color='red'>UPVOTE ","metadata":{"_uuid":"2ee8624d8356a00b939c3eb92e2ecd399d9f0db5"}}]}