{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Starting Words\n\nThis notebook is intended to be prepared as a marathon with constant updates over time. The goal of this marathon is to include as many details as possible in the simplest form possible. Hopefully, the notebook will be able to provide the in-depth knowledge on following aspects:\n* Exploratory Data Analysis, \n* Data Visualization,\n* Feature Selection, \n* Sampling, \n* Machine Learning Algorithms - Supervised Learning Regression Analysis","metadata":{}},{"cell_type":"markdown","source":"## Data Analysis Process\n\nWe will follow following simple steps fod data analysis:\n\n* Define the problem\n* Analyze and prepare the data\n    * Exploratory Data Analyiss\n    * Data Visualization\n* Develop and evaluate the models\n    * Feature Selection\n    * Samping\n    * Spot-check Algorithms\n* Improve results\n* Present results","metadata":{}},{"cell_type":"markdown","source":"# Define the Problem\n\nReal estate is a big industry in the US with a rapid number of purchases and sales of properties such as houses taking place in regular succession. With a large number of factors to be considered prior to make a purchase decision, an appropriate model is required to predict the sale price of the house. The model should take account of historic sales data that includes the selling price of the house along with its features such as basement quality, garage condition, lot area, neighborhood, and other factors. The task is to develop a suitable system that predicts the price for a new house given that the values of features are provided to it. The performance quality is measured based on the degree of accuracy of the predicted selling price in comparison to the actual selling price.","metadata":{}},{"cell_type":"code","source":"### Import Necessary Libraries\n\nimport pandas as pd\nimport numpy as np\nfrom sklearn.model_selection import cross_val_score, train_test_split\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.linear_model import LinearRegression, RidgeCV, LassoCV, ElasticNetCV\nfrom sklearn.metrics import mean_squared_error, make_scorer\nfrom scipy.stats import skew\nfrom IPython.display import display\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\n### Get Data\ntrain = pd.read_csv(\"../input/house-prices-advanced-regression-techniques/train.csv\")\nprint(\"train : \" + str(train.shape))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:11:11.273039Z","iopub.execute_input":"2022-07-29T04:11:11.273511Z","iopub.status.idle":"2022-07-29T04:11:12.901773Z","shell.execute_reply.started":"2022-07-29T04:11:11.273412Z","shell.execute_reply":"2022-07-29T04:11:12.900479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Analyze and Prepare Data\n### EDA: Target Variable - SalePrice","metadata":{}},{"cell_type":"code","source":"### Checking presence of null value\n\ntrain.SalePrice.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:13:29.971656Z","iopub.execute_input":"2022-07-29T04:13:29.972083Z","iopub.status.idle":"2022-07-29T04:13:29.988791Z","shell.execute_reply.started":"2022-07-29T04:13:29.972048Z","shell.execute_reply":"2022-07-29T04:13:29.987505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.SalePrice.describe()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:13:33.215930Z","iopub.execute_input":"2022-07-29T04:13:33.216341Z","iopub.status.idle":"2022-07-29T04:13:33.230544Z","shell.execute_reply.started":"2022-07-29T04:13:33.216307Z","shell.execute_reply":"2022-07-29T04:13:33.229470Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Check the number of entries with SalePrice greater than 400000\n\ntrain[train['SalePrice'] > 400000]['Id'].count()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:13:36.158806Z","iopub.execute_input":"2022-07-29T04:13:36.159618Z","iopub.status.idle":"2022-07-29T04:13:36.176492Z","shell.execute_reply.started":"2022-07-29T04:13:36.159569Z","shell.execute_reply":"2022-07-29T04:13:36.175138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Filter the data with SalePrice less than 400000\n\ntrain = train[train['SalePrice'] <= 400000]\ntrain.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:13:39.215802Z","iopub.execute_input":"2022-07-29T04:13:39.216331Z","iopub.status.idle":"2022-07-29T04:13:39.227662Z","shell.execute_reply.started":"2022-07-29T04:13:39.216284Z","shell.execute_reply":"2022-07-29T04:13:39.226285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Findings**\n\n* There is no null value in the target variable\n* Target variable distribution is positively skewed (Mean > Meadian)\n* Entries with SalePrice greater than 400,000 represents less than 1% of total entries. Hence, they are omitted to correct the skewness of data","metadata":{}},{"cell_type":"code","source":"### Data Visualization\n\nplt.subplots(figsize=(12, 6))\nax = sns.histplot(data = train, x = 'SalePrice', kde = True, binwidth = 20000)\nax.set_title(\"Number of houses by Sale Price\")\nax.set(xlabel='Sale Price', ylabel='Number of Houses')\nax.bar_label(ax.containers[0])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-28T16:10:56.999332Z","iopub.execute_input":"2022-07-28T16:10:57.000402Z","iopub.status.idle":"2022-07-28T16:10:57.351630Z","shell.execute_reply.started":"2022-07-28T16:10:57.000364Z","shell.execute_reply":"2022-07-28T16:10:57.350442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.columns","metadata":{"execution":{"iopub.status.busy":"2022-07-28T14:31:52.804253Z","iopub.execute_input":"2022-07-28T14:31:52.804750Z","iopub.status.idle":"2022-07-28T14:31:52.814974Z","shell.execute_reply.started":"2022-07-28T14:31:52.804697Z","shell.execute_reply":"2022-07-28T14:31:52.813023Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### EDA - Independent Variables\n\nHere, we can observe that some of the variables have high possibility of correlation. For instance, 'BsmtQual', 'BsmtCond', 'BsmtExposure' are likely to signal the overall quality of basement of house. While considering the SalePrice, the overall quality of basement is more relevant than individual component. Hence, we will group the independent variables and conduct the EDA.\n\n","metadata":{}},{"cell_type":"markdown","source":"### Variable - Basement","metadata":{}},{"cell_type":"code","source":"### Grouping the data on basement in single dataframe\n\nbasement = train.filter(like = 'Bsmt', axis = 1)\nbasement","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:15:37.815517Z","iopub.execute_input":"2022-07-29T04:15:37.815919Z","iopub.status.idle":"2022-07-29T04:15:37.843812Z","shell.execute_reply.started":"2022-07-29T04:15:37.815887Z","shell.execute_reply":"2022-07-29T04:15:37.842480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Checking missing values\nbasement.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:15:41.522347Z","iopub.execute_input":"2022-07-29T04:15:41.522800Z","iopub.status.idle":"2022-07-29T04:15:41.534447Z","shell.execute_reply.started":"2022-07-29T04:15:41.522758Z","shell.execute_reply":"2022-07-29T04:15:41.533246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Observation**\n* NA in basement value refers to absence of basement\n* NA is replaced as 'None' indicating that house does not have basement","metadata":{}},{"cell_type":"code","source":"### Repalcing NA value with 'None' description\n\nbasement = basement.fillna('None')\nbasement.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:17:15.026165Z","iopub.execute_input":"2022-07-29T04:17:15.026637Z","iopub.status.idle":"2022-07-29T04:17:15.041024Z","shell.execute_reply.started":"2022-07-29T04:17:15.026596Z","shell.execute_reply":"2022-07-29T04:17:15.039853Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"basement.dtypes","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:18:05.507361Z","iopub.execute_input":"2022-07-29T04:18:05.507802Z","iopub.status.idle":"2022-07-29T04:18:05.516708Z","shell.execute_reply.started":"2022-07-29T04:18:05.507764Z","shell.execute_reply":"2022-07-29T04:18:05.515774Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Observation**\n* Here 'BsmtCond' refers to the current condition of the Basement\n* We will check whether the 'BsmtCond' can represent for overall indicators for basement","metadata":{}},{"cell_type":"code","source":"basement.BsmtQual.describe()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:22:34.084563Z","iopub.execute_input":"2022-07-29T04:22:34.084971Z","iopub.status.idle":"2022-07-29T04:22:34.096184Z","shell.execute_reply.started":"2022-07-29T04:22:34.084941Z","shell.execute_reply":"2022-07-29T04:22:34.094896Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Visualization of basement numeric variables\nsns.pairplot(basement)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:22:51.844074Z","iopub.execute_input":"2022-07-29T04:22:51.844475Z","iopub.status.idle":"2022-07-29T04:22:58.813749Z","shell.execute_reply.started":"2022-07-29T04:22:51.844444Z","shell.execute_reply":"2022-07-29T04:22:58.812481Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Prepare data for logistic model\n\nx = basement.drop(columns = ['BsmtCond'])\nx = pd.get_dummies(x)\n\n# Import label encoder \nfrom sklearn import preprocessing\n\n# label_encoder object knows how to understand word labels. \nlabel_encoder = preprocessing.LabelEncoder()\n\n# Encode labels in column 'Country'. \ny = label_encoder.fit_transform(basement['BsmtCond']) \n\nx_train, x_test, y_train, y_test = train_test_split(x, y, random_state=0, train_size = .75)\n\nprint('Train-test data set prepared')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:24:34.101063Z","iopub.execute_input":"2022-07-29T04:24:34.101497Z","iopub.status.idle":"2022-07-29T04:24:34.120791Z","shell.execute_reply.started":"2022-07-29T04:24:34.101459Z","shell.execute_reply":"2022-07-29T04:24:34.119427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Prepare the ML algorithm\n\nfrom sklearn.linear_model import LogisticRegression\nmodel = LogisticRegression()\nmodel.fit(x_train, y_train)\n\nprint('Model Prepared')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:25:06.779596Z","iopub.execute_input":"2022-07-29T04:25:06.779983Z","iopub.status.idle":"2022-07-29T04:25:06.883673Z","shell.execute_reply.started":"2022-07-29T04:25:06.779952Z","shell.execute_reply":"2022-07-29T04:25:06.882318Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### R square for train data\n\nprint('R square for train data: ' + str(model.score(x_train, y_train)))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:26:42.717929Z","iopub.execute_input":"2022-07-29T04:26:42.718359Z","iopub.status.idle":"2022-07-29T04:26:42.727469Z","shell.execute_reply.started":"2022-07-29T04:26:42.718325Z","shell.execute_reply":"2022-07-29T04:26:42.726578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### R square for test data\n\nprint('R square for test data: ' + str(model.score(x_test, y_test)))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:27:16.812377Z","iopub.execute_input":"2022-07-29T04:27:16.812830Z","iopub.status.idle":"2022-07-29T04:27:16.823405Z","shell.execute_reply.started":"2022-07-29T04:27:16.812795Z","shell.execute_reply":"2022-07-29T04:27:16.822363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Visualization of Basement Condition with SalePrice in train data\n\ntrain.BsmtCond = train.BsmtCond.fillna('None')\nsns.boxplot(x = 'BsmtCond', y = 'SalePrice', data = train, \n           order = ['Ex', 'Gd', 'TA', 'Fa', 'Po', 'None'])","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:29:19.319608Z","iopub.execute_input":"2022-07-29T04:29:19.320017Z","iopub.status.idle":"2022-07-29T04:29:19.573608Z","shell.execute_reply.started":"2022-07-29T04:29:19.319983Z","shell.execute_reply":"2022-07-29T04:29:19.572461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Finding**\n* With R square well above 90%, we can conclude that 'BsmtCond' can represent the overall indicators for train dataset.\n* Boxplot shows a decreasing SalePrice with the nature of basement condition.","metadata":{}},{"cell_type":"markdown","source":"### Variable - Garage","metadata":{}},{"cell_type":"code","source":"### Grouping the data on Garage in single dataframe\n\nGarage = train.filter(like='Garage', axis=1)\nGarage.dtypes","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:33:20.264475Z","iopub.execute_input":"2022-07-29T04:33:20.264871Z","iopub.status.idle":"2022-07-29T04:33:20.276010Z","shell.execute_reply.started":"2022-07-29T04:33:20.264838Z","shell.execute_reply":"2022-07-29T04:33:20.274433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Calculating age of Garage based on years built\nGarage['age'] = 2015 - Garage.GarageYrBlt\nGarage.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:34:40.088343Z","iopub.execute_input":"2022-07-29T04:34:40.088795Z","iopub.status.idle":"2022-07-29T04:34:40.107941Z","shell.execute_reply.started":"2022-07-29T04:34:40.088759Z","shell.execute_reply":"2022-07-29T04:34:40.106827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Checking missing values\nGarage.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:34:25.290617Z","iopub.execute_input":"2022-07-29T04:34:25.290983Z","iopub.status.idle":"2022-07-29T04:34:25.301872Z","shell.execute_reply.started":"2022-07-29T04:34:25.290954Z","shell.execute_reply":"2022-07-29T04:34:25.300675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Replacing NA value with 'None'\n\nGarage['age'] = Garage['age'].fillna(0) \nGarage = Garage.fillna('None')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:34:47.265183Z","iopub.execute_input":"2022-07-29T04:34:47.265588Z","iopub.status.idle":"2022-07-29T04:34:47.276551Z","shell.execute_reply.started":"2022-07-29T04:34:47.265552Z","shell.execute_reply":"2022-07-29T04:34:47.275448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Observation**\n* 'GarageQual' represents the overall quality of garage\n* We will check whether the 'GarageQual' can represent for overall indicators for Garage","metadata":{}},{"cell_type":"code","source":"### Visualization of numeric variables of Garage\n\nsns.pairplot(Garage)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:37:57.393988Z","iopub.execute_input":"2022-07-29T04:37:57.394452Z","iopub.status.idle":"2022-07-29T04:37:59.309565Z","shell.execute_reply.started":"2022-07-29T04:37:57.394413Z","shell.execute_reply":"2022-07-29T04:37:59.308205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Visualization of relation between age and GarageQual\n\nsns.boxplot(x = 'GarageQual', y = 'age', data = Garage,\n           order = ['Ex', 'Gd', 'TA', 'Fa', 'Po', 'None'])","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:38:43.298642Z","iopub.execute_input":"2022-07-29T04:38:43.299126Z","iopub.status.idle":"2022-07-29T04:38:43.569250Z","shell.execute_reply.started":"2022-07-29T04:38:43.299090Z","shell.execute_reply":"2022-07-29T04:38:43.567770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Visualization of relation between GarageArea and GarageQual\n\nsns.boxplot(x = 'GarageQual', y = 'GarageArea', data = Garage,\n           order = ['Ex', 'Gd', 'TA', 'Fa', 'Po', 'None'])","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:39:11.459074Z","iopub.execute_input":"2022-07-29T04:39:11.459483Z","iopub.status.idle":"2022-07-29T04:39:11.721075Z","shell.execute_reply.started":"2022-07-29T04:39:11.459451Z","shell.execute_reply":"2022-07-29T04:39:11.719649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Finding**\n* Visual pattern can be observed between GarageQual and other variables\n* GarageQual and GarageCond has same data and hence, GaragCond can be removed from data\n* GarageYrBlt is removed as age parameter has better meaning in the model","metadata":{}},{"cell_type":"code","source":"### Prepare data for logistic model\n\nx = Garage.drop(columns = ['GarageYrBlt', 'GarageCond', 'GarageQual'])\nx = pd.get_dummies(x)\n\n# Import label encoder \nfrom sklearn import preprocessing\n\n# label_encoder object knows how to understand word labels. \nlabel_encoder = preprocessing.LabelEncoder()\n\n# Encode labels in column 'Country'. \ny = label_encoder.fit_transform(Garage['GarageCond']) \n\n\nx_train, x_test, y_train, y_test = train_test_split(x, y, random_state=0, train_size = .75)\n\nprint('train-test data prepared')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:44:34.424658Z","iopub.execute_input":"2022-07-29T04:44:34.425029Z","iopub.status.idle":"2022-07-29T04:44:34.443764Z","shell.execute_reply.started":"2022-07-29T04:44:34.424999Z","shell.execute_reply":"2022-07-29T04:44:34.442378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Model Preparation\n\nmodel.fit(x_train, y_train)\n\nprint('Model prepared')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:45:05.109022Z","iopub.execute_input":"2022-07-29T04:45:05.109448Z","iopub.status.idle":"2022-07-29T04:45:05.213123Z","shell.execute_reply.started":"2022-07-29T04:45:05.109411Z","shell.execute_reply":"2022-07-29T04:45:05.211901Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Check R square for train data\n\nprint('R square for train data: ' + str(model.score(x_train, y_train)))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:45:53.310493Z","iopub.execute_input":"2022-07-29T04:45:53.310917Z","iopub.status.idle":"2022-07-29T04:45:53.321122Z","shell.execute_reply.started":"2022-07-29T04:45:53.310879Z","shell.execute_reply":"2022-07-29T04:45:53.319681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('R square for test data: ' + str(model.score(x_test, y_test)))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:46:18.301721Z","iopub.execute_input":"2022-07-29T04:46:18.302143Z","iopub.status.idle":"2022-07-29T04:46:18.311626Z","shell.execute_reply.started":"2022-07-29T04:46:18.302108Z","shell.execute_reply":"2022-07-29T04:46:18.310637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Visualization of Garage with SalePrice in train data\n\nsns.boxplot(x = 'GarageQual', y = 'SalePrice', data = train,\n           order = ['Ex', 'Gd', 'TA', 'Fa','Po'])","metadata":{"execution":{"iopub.status.busy":"2022-07-29T04:46:57.495065Z","iopub.execute_input":"2022-07-29T04:46:57.495473Z","iopub.status.idle":"2022-07-29T04:46:57.757358Z","shell.execute_reply.started":"2022-07-29T04:46:57.495442Z","shell.execute_reply":"2022-07-29T04:46:57.756165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Finding**\n* With R square well above 90%, we can conclude that 'GarageQual' can represent the overall indicators for train dataset.\n* Boxplot shows a decreasing SalePrice with the nature of Garage condition.","metadata":{}},{"cell_type":"markdown","source":"# Work in Progress\n\nAs I have mentioned prior, this will be a marathon and updated over the period.\nMeanwhile, if you have any queries or any suggestions, kindly provide a comment. I will be replying to it to my best knowledge.","metadata":{}}]}