{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "a961734d-6f96-8c90-3897-11082fbf56d7"
      },
      "source": [
        "RentHop has a feature that shows when the rent of a certain listing is much cheaper than others with a similar number of bathrooms and bedrooms in the area. Can we use KNearestRegression to determine what the predicted price of a listing would be and take the ratio of the actual price versus the predicted price to make a new feature? Let's see.\n",
        "\n",
        "First we'll load up the training set and train KNeighborsRegressor on other listing with a similar number of bedrooms, bathrooms, latitude, and longitude."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "169b7933-5fb8-6592-de49-4f845bf50e3e"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "with open(\"../input/train.json\") as train_json:\n",
        "    raw_train = pd.read_json(train_json.read()).reset_index()\n",
        "    \n",
        "from sklearn.neighbors import KNeighborsRegressor\n",
        "model = KNeighborsRegressor(n_neighbors=300)\n",
        "price_df = pd.concat([raw_train['bedrooms'],raw_train['bathrooms'],raw_train['latitude'],raw_train['longitude'],raw_train['price']], axis=1)\n",
        "model.fit(price_df.drop(['price'], axis=1), price_df['price'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "a4539919-1768-9142-12e3-1c3b21c897b6"
      },
      "source": [
        "Now, let's look at an example of the n nearest neighbors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "8ab5855c-1ec4-0145-f33f-c4302bc915f7"
      },
      "outputs": [],
      "source": [
        "print(model.kneighbors(price_df.drop(['price'], axis=1).loc[2].reshape(1,-1), n_neighbors=300))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "dc3bf9e8-c6af-650a-f222-ca2b23c832a4"
      },
      "outputs": [],
      "source": [
        "print(price_df.drop(['price'], axis=1).loc[2])\n",
        "print(price_df.drop(['price'], axis=1).loc[311])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "9ac149f2-fe0e-e893-a3e9-fed7017088cf"
      },
      "source": [
        "The two points are basically two blocks away. It's pretty clear there could be some useful data for us."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "1ded5e2c-0cc3-b9d5-250a-b1605f0d0ec1"
      },
      "outputs": [],
      "source": [
        "pred_price = model.predict(price_df.drop(['price'], axis=1))\n",
        "\n",
        "price_df['predicted_price'] = pd.DataFrame(pred_price, columns=['predicted_price'])\n",
        "\n",
        "\n",
        "price_df['pred_price_ratio'] = price_df['price'] / price_df['predicted_price']\n",
        "\n",
        "price_df['interest_level'] = raw_train['interest_level']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "32aeec6d-2ccb-9302-72d8-7cf9f43d1b81"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "new_price_df = price_df[price_df['pred_price_ratio'] < 4]\n",
        "%matplotlib inline\n",
        "plt.figure(figsize=(10,20))\n",
        "sns.boxplot(x='interest_level', y='pred_price_ratio', data=new_price_df)\n",
        "plt.title(\"Interest Level and Price / Predicted Price Ratio\", fontsize=32)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "64abdb4f-0318-2478-4466-d03b66ec8b53"
      },
      "source": [
        "Generally, listings with a lower price to predicted price ratio have a higher interest.\n",
        "\n",
        "Let me know if this was helpful to you, it should be pretty simple to implement at the feature extraction step.\n",
        "\n",
        "Thanks to all of the awesome notebooks people have posted on here, I've learned a lot!"
      ]
    }
  ],
  "metadata": {
    "_change_revision": 0,
    "_is_fork": false,
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}