{"cells":[{"metadata":{"_uuid":"9a4689a609232d4a2a6654086e32e484519f6367"},"cell_type":"markdown","source":"## Recap\nYou've built a model. In this exercise you will test how good your model is.\n\nRun the cell below to set up your coding environment where the previous exercise left off."},{"metadata":{"_uuid":"afb61d0c912ed5af6b06c209557059f1b6d1c037","trusted":true},"cell_type":"code","source":"# Code you have previously used to load data\nimport pandas as pd\nfrom sklearn.tree import DecisionTreeRegressor\n\n# Path of the file to read\niowa_file_path = '../input/home-data-for-ml-course/train.csv'\n\nhome_data = pd.read_csv(iowa_file_path)\ny = home_data.SalePrice\nfeature_columns = ['LotArea', 'YearBuilt', '1stFlrSF', '2ndFlrSF', 'FullBath', 'BedroomAbvGr', 'TotRmsAbvGrd']\nX = home_data[feature_columns]\n\n# Specify Model\niowa_model = DecisionTreeRegressor()\n# Fit Model\niowa_model.fit(X, y)\n\nprint(\"First in-sample predictions:\", iowa_model.predict(X.head()))\nprint(\"Actual target values for those homes:\", y.head().tolist())\n\n# Set up code checking\nfrom learntools.core import binder\nbinder.bind(globals())\nfrom learntools.machine_learning.ex4 import *\nprint(\"Setup Complete\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"402fd8b479210e0edfe5a782447fdf6b3cfdabce"},"cell_type":"markdown","source":"# Exercises\n\n## Step 1: Split Your Data\nUse the `train_test_split` function to split up your data.\n\nGive it the argument `random_state=1` so the `check` functions know what to expect when verifying your code.\n\nRecall, your features are loaded in the DataFrame **X** and your target is loaded in **y**.\n"},{"metadata":{"_uuid":"f43aee7375b8b99054266ef7050a23d273ceecae","trusted":true},"cell_type":"code","source":"# Import the train_test_split function\nfrom sklearn.model_selection import train_test_split\n\ntrain_X, val_X, train_y, val_y = train_test_split(X, y, random_state = 1)\n\nstep_1.check()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"94756f25a6679f179011a9f7b588bd1d1add0734","trusted":true},"cell_type":"code","source":"# The lines below will show you a hint or the solution.\n# step_1.hint() \n# step_1.solution()\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5d12cccfce6eb6f8ed9353112a52b3e6da0f4809"},"cell_type":"markdown","source":"## Step 2: Specify and Fit the Model\n\nCreate a `DecisionTreeRegressor` model and fit it to the relevant data.\nSet `random_state` to 1 again when creating the model."},{"metadata":{"_uuid":"1f5b82cd38f1dbe4e0573d2c0643ac334ae1b353","trusted":true},"cell_type":"code","source":"# You imported DecisionTreeRegressor in your last exercise\n# and that code has been copied to the setup code above. So, no need to\n# import it again\n\n# Specify the model\niowa_model = DecisionTreeRegressor(random_state=1)\n\n# Fit iowa_model with the training data.\niowa_model.fit(train_X,train_y)\nstep_2.check()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c73d68eac7b78d88b75173f1fc1f29db6d0a9a49","trusted":true},"cell_type":"code","source":"# step_2.hint()\n# step_2.solution()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"575ffa26ed689ab634a47443626464201df489f7"},"cell_type":"markdown","source":"## Step 3: Make Predictions with Validation data\n"},{"metadata":{"_uuid":"363d3544f3a6546f8cd0ef6181e0d9a377c44fe5","trusted":true},"cell_type":"code","source":"# Predict with all validation observations\nval_predictions = iowa_model.predict(val_X)\n\nstep_3.check()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2b85919572568f1a74b086ee6776e7abcf452bdc","trusted":true},"cell_type":"code","source":"# step_3.hint()\n# step_3.solution()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c135d6085d88971ab6853bac5d8be4b7a5d199d9"},"cell_type":"markdown","source":"Inspect your predictions and actual values from validation data."},{"metadata":{"_uuid":"b3a62c4c289dccf1335ec90c4a3f8cf4755e220b","trusted":true},"cell_type":"code","source":"# print the top few validation predictions\nprint(val_predictions[:5])\n# print the top few actual prices from validation data\nprint(type(val_y))\nprint(val_y)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"372b8178f1c393741008f748192f61ec74f61417"},"cell_type":"markdown","source":"What do you notice that is different from what you saw with in-sample predictions (which are printed after the top code cell in this page).\n\nDo you remember why validation predictions differ from in-sample (or training) predictions? This is an important idea from the last lesson.\n\n## Step 4: Calculate the Mean Absolute Error in Validation Data\n"},{"metadata":{"_uuid":"c36564ca1da675a8398062b0f97228f49e3f220d","trusted":true},"cell_type":"code","source":"from sklearn.metrics import mean_absolute_error\nval_mae = mean_absolute_error(val_y,val_predictions)\n\n# uncomment following line to see the validation_mae\nprint(val_mae)\nstep_4.check()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"344444d6544dc5344cdcc7709c69302038220963","trusted":true},"cell_type":"code","source":"# step_4.hint()\n# step_4.solution()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ff710d9c072418a22f6a3b04c3442d5e1f333764"},"cell_type":"markdown","source":"Is that MAE good?  There isn't a general rule for what values are good that applies across applications. But you'll see how to use this number in the next step."},{"metadata":{"_uuid":"2f523a1c7bc9d6aef6b2309a3fd1d17f8da734a4"},"cell_type":"markdown","source":"# Keep Going\n\nNow that you can measure model performance, you are ready to run some experiments comparing different models. The key is to understand **[Underfitting and Overfitting](https://www.kaggle.com/dansbecker/underfitting-and-overfitting)**. It's an especially fun part of machine learning. \n\n---\n**[Course Home Page](https://www.kaggle.com/learn/machine-learning)**\n\n\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}