{"cells":[{"metadata":{"_uuid":"8dbdeb381152ba7076e2165cd1cfd91cbf63d4e6"},"cell_type":"markdown","source":"*I don't know...*\n\n**Features:**\ndate, bedrooms, bathrooms, sqft_living, sqft_lot, floors, waterfront, condition, grade, sqft_above, sqft_basement, yr_built, yr_renovated, zipcode, lat, long\n\n**Redundant Features**\n* date (too much variation) **!important**: can be split into a month and year component to add complexity to the model)\n* waterfront (too low variation: FalseCount=14888, TrueCount=112)\n* lat and long (zipcode gives a simpler estimate for location)\n\n**Numeric (and ordinal) Features**\n* bedrooms\n* bathrooms\n* sqft_living\n* sqft_lot\n* floors\n* condition\n* grade\n* sqft_above\n* sqft_basement\n* yr_built\n* yr_renovated // if yr_renovated == 0, yr_renovated = yr_built\n\n**Categorical Features** // convert to numeric\n* zipcode (nominal)"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"# packages\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# load data\nmodelling_data_feat = np.load('../input/x_train.npy')\nmodelling_data_target = np.load('../input/y_train.npy')\nsubmit_data_feat = np.load('../input/x_test.npy')\n\n# convert ndarray to dataframe\ndf_mod_feat = pd.DataFrame(data=modelling_data_feat)\ndf_sub_feat = pd.DataFrame(data=submit_data_feat)\n\nrs = 4319","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"20cb55cea25fba1c689dc77ee9b7381dd5e26eb0"},"cell_type":"code","source":"df_mod_feat.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ee2632142bd6c6bb34dea085b0d49e7d8c3cb098"},"cell_type":"code","source":"df_sub_feat.describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c26b3de17f44e8778f5cc60cd20e8bf1412b28be"},"cell_type":"markdown","source":"Preprocessing"},{"metadata":{"trusted":true,"_uuid":"b5420c0db0c980d0d8670a5f820f5cf1e190deb2"},"cell_type":"code","source":"# packages\nfrom sklearn.preprocessing import OneHotEncoder\n\n# drop date, waterfront, lat, and long\ndf_mod_feat = df_mod_feat.drop(columns=['waterfront','lat','long'])\ndf_sub_feat = df_sub_feat.drop(columns=['waterfront','lat','long'])\n\n# split feature list into categorical and numerical\ndf_mod_feat_cat = df_mod_feat[['zipcode']].copy()\ndf_mod_feat_num = df_mod_feat.drop(columns=['zipcode'])\ndf_sub_feat_cat = df_sub_feat[['zipcode']].copy()\ndf_sub_feat_num = df_sub_feat.drop(columns=['zipcode'])\n\n# convert categorical feature(s) into nominal numeric features\n# merge train set with test set first to convert to associative values\ncat_allfeat = pd.concat([df_mod_feat_cat, df_sub_feat_cat])\nenc = OneHotEncoder()\ntemp = enc.fit_transform(cat_allfeat)\ntemp = pd.DataFrame(temp.todense())\ndf_mod_feat_cat_conv = temp.iloc[:df_mod_feat_cat.shape[0]]\ndf_sub_feat_cat_conv = temp.iloc[df_mod_feat_cat.shape[0]:cat_allfeat.shape[0]]\ndf_sub_feat_cat_conv = df_sub_feat_cat_conv.reset_index(drop=True)\n\n# if yr_renovated == 0, yr_renovated = yr_built\nfor i in range (0, df_mod_feat_num.shape[0]):\n    df_mod_feat_num.loc[i,'date'] = int(df_mod_feat_num.loc[i,'date'][0:4])\n    if df_mod_feat_num.loc[i,'yr_renovated'] == 0:\n        df_mod_feat_num.loc[i,'yr_renovated'] = df_mod_feat_num.loc[i,'yr_built']\n        \nfor i in range (0, df_sub_feat_num.shape[0]):\n    df_sub_feat_num.loc[i,'date'] = int(df_sub_feat_num.loc[i,'date'][0:4])\n    if df_sub_feat_num.loc[i,'yr_renovated'] == 0:\n        df_sub_feat_num.loc[i,'yr_renovated'] = df_sub_feat_num.loc[i,'yr_built']\n\n# merge converted categorical and numerical features\ndf_mod_feat = pd.concat([df_mod_feat_cat_conv, df_mod_feat_num], axis=1)\ndf_sub_feat = pd.concat([df_sub_feat_cat_conv, df_sub_feat_num], axis=1)\n\n# scaling\n\n# normalize\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b477a6063b89917027bf9b077f097e24a4da60ce"},"cell_type":"markdown","source":"Feature Selection and Modelling"},{"metadata":{"trusted":true,"_uuid":"92e21040c1a5157bbc551fc82e90f39f3d4834de"},"cell_type":"code","source":"# packages\nfrom sklearn.model_selection import train_test_split, StratifiedKFold\nfrom sklearn.feature_selection import SelectKBest, RFE, RFECV, SelectFromModel, f_regression\nfrom sklearn.linear_model import LinearRegression, BayesianRidge\n\ndef mean_absolute_percentage_error(y_true, y_pred): \n    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100\n\n# Split train set and test set for modelling\nX_train, X_test, y_train, y_test = train_test_split(df_mod_feat, modelling_data_target, test_size=0.1, random_state=rs)\n\n# Select a regression model\n# LinearRegression\nmodel = LinearRegression(fit_intercept=False)\n\n# Feature selection via SelectKBest\n\n# Feature selection via RFE (Recursive Feature Elimination)\n\n# Feature selection via RFECV (Recursive Feature Elmination with Cross Validation)\nselector = RFECV(estimator=model, step=1, min_features_to_select=1, cv=StratifiedKFold(n_splits=5, random_state=rs),scoring='r2')\nfeat_res = selector.fit(X_train, y_train)\nprint(\"Optimal number of features : %d\" % selector.n_features_)\n\n# Feature selection via SelectFromModel\n\n# The selected features\nsel_X_train = feat_res.transform(X_train)\nsel_X_test = feat_res.transform(X_test)\n\ny_true, y_pred = y_test, feat_res.estimator_.predict(sel_X_test)\nmean_absolute_percentage_error(y_true, y_pred)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bfd10acdba46ea5a45b8cf9d64deec8e2a747d53"},"cell_type":"markdown","source":"Submission"},{"metadata":{"trusted":true,"_uuid":"29b9b4e92023783567c9b1c239978027c9af1859"},"cell_type":"code","source":"submission_features = feat_res.transform(df_sub_feat)\ntest_predictions = feat_res.estimator_.predict(submission_features)\nsubmission = pd.DataFrame({'Id': range(1, test_predictions.shape[0]+1), 'Price': test_predictions})\nsubmission = submission.reset_index(drop=True)\nsubmission.to_csv('submission.csv', index=False)\nsubmission","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}