{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "375efc21-8783-57ca-6550-fe85fc5a5bcb"
      },
      "source": [
        "The dataset contains numerous sets of content recommendations served to a specific user in a specific context. Each context (i.e. a set of recommendations) is given a display_id. In each such set, the user has clicked on at least one recommendation. The identities of the clicked recommendations in the test set are not revealed. Your task is to rank the recommendations in each group by decreasing predicted likelihood of being clicked.\n",
        "As a warning, this is a very large relational dataset. While most of the tables are small enough to fit in memory, the page views log (page_views.csv) is over 2 billion rows and 100GB uncompressed. We have also uploaded a sample version of this file with the first 10,000,000 rows. The MD5 checksum of page_views.csv.zip is 3742c116bab4030e0a7ea1c0be623bd9."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "54371809-17c8-e846-a38f-514026e1f353"
      },
      "outputs": [],
      "source": [
        "# This Python 3 environment comes with many helpful analytics libraries installed\n",
        "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
        "# For example, here's several helpful packages to load in \n",
        "\n",
        "import numpy as np # linear algebra\n",
        "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
        "\n",
        "# Input data files are available in the \"../input/\" directory.\n",
        "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n",
        "\n",
        "from subprocess import check_output\n",
        "print(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))\n",
        "\n",
        "# Any results you write to the current directory are saved as output."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "d6f3eec8-6475-9d87-cf74-4ca26647261d"
      },
      "outputs": [],
      "source": [
        "import pandas as pd \n",
        "import numpy as np"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "9cc62073-99cd-586b-2135-fb7096ed9abe"
      },
      "outputs": [],
      "source": [
        "ClicksTest=pd.read_csv(\"../input/clicks_test.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "82c88a5f-7cf8-7054-f3aa-3d1fad24c1e4"
      },
      "outputs": [],
      "source": [
        "ClicksTest.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "89686c73-e580-afde-95c3-0a5772e56872"
      },
      "outputs": [],
      "source": [
        "ClicksTest.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "47bdd43a-b748-5c20-9be8-cab1d6404570"
      },
      "outputs": [],
      "source": [
        "ClicksTrain=pd.read_csv(\"../input/clicks_train.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "28ef95eb-c58c-b9ce-73d1-fc04ae41af0c"
      },
      "outputs": [],
      "source": [
        "ClicksTrain.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "c94abfcb-eadd-362b-80b6-92a881ac77c0"
      },
      "outputs": [],
      "source": [
        "ClicksTrain.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "7b0285b4-9a83-ad7e-9616-1f2fcaa3f8e6"
      },
      "outputs": [],
      "source": [
        "DocumentsCategories=pd.read_csv(\"../input/documents_categories.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "98cdc22d-0f35-bab9-39e9-4d1ec2c3cefe"
      },
      "outputs": [],
      "source": [
        "DocumentsCategories.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "ac1aed51-1c5e-d258-3200-d29014699dec"
      },
      "outputs": [],
      "source": [
        "DocumentsCategories.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "718c5741-b7b1-33a1-c4f0-720c4156a969"
      },
      "outputs": [],
      "source": [
        "DocumentsEntities=pd.read_csv(\"../input/documents_entities.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "08e4aaab-6496-1fb1-9d7b-08303dbff596"
      },
      "outputs": [],
      "source": [
        "DocumentsEntities.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "a2e50ebf-982e-a890-16fa-35a72fa893a8"
      },
      "outputs": [],
      "source": [
        "DocumentsEntities.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "025b768a-aef9-8008-b268-426f25d98853"
      },
      "outputs": [],
      "source": [
        "DocumentsMeta=pd.read_csv(\"../input/documents_meta.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "63ff9f33-a87c-8459-d9cb-b6bb899d1055"
      },
      "outputs": [],
      "source": [
        "DocumentsMeta.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "dca681ca-df15-8045-2e82-e85cbe02f1ef"
      },
      "outputs": [],
      "source": [
        "DocumentsMeta.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "47551714-2957-3a36-380b-0fcf08335d55"
      },
      "outputs": [],
      "source": [
        "DocumentsTopics=pd.read_csv(\"../input/documents_topics.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "ea025e10-c220-a43d-2171-2bad3d697db2"
      },
      "outputs": [],
      "source": [
        "DocumentsTopics.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "3ab0b8c0-7057-60d8-3a90-9734769cb371"
      },
      "outputs": [],
      "source": [
        "DocumentsTopics.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "d6f01cd6-0ccd-66ba-499c-03c2136e9a64"
      },
      "outputs": [],
      "source": [
        "Events=pd.read_csv(\"../input/events.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "ba4bb5bd-6647-f472-7bf6-c67cb9ffc4ab"
      },
      "outputs": [],
      "source": [
        "Events.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "2f0e969e-aba1-a344-eac3-8a861ff9c160"
      },
      "outputs": [],
      "source": [
        "Events.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "e43877af-b2c4-2d83-b16e-2c5943c9ced6"
      },
      "outputs": [],
      "source": [
        "PageViewsSample=pd.read_csv(\"../input/page_views_sample.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "cfae42fb-a66c-2a13-9136-5e1298aa1818"
      },
      "outputs": [],
      "source": [
        "PageViewsSample.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "aaf64485-00c0-ef93-6640-f21aebf742e2"
      },
      "outputs": [],
      "source": [
        "PageViewsSample.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "1f5fa503-083d-d62c-b186-c7f367f5b11d"
      },
      "outputs": [],
      "source": [
        "PromotedContent=pd.read_csv(\"../input/promoted_content.csv\",encoding=\"utf8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "0f72ded2-36b0-632f-37fb-0d468289b80a"
      },
      "outputs": [],
      "source": [
        "PromotedContent.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "d5ff310d-029b-4d96-d567-ccf910b0f7f4"
      },
      "outputs": [],
      "source": [
        "PromotedContent.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "55be92db-9c73-7452-54ee-c21970badf9d"
      },
      "source": [
        "We will use Merge to reduce the number of tables . the ideal is to get all the information in one table but we will try to perform this in sections"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "2a78d08f-13a5-3688-5785-1d97c3927743"
      },
      "source": [
        "\n",
        "\n",
        " 1. Document Table"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "cb5cf044-3df7-ed0d-660d-b144295522ef"
      },
      "outputs": [],
      "source": ""
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "549cb3e2-4fad-973b-427d-29e68b9bacee"
      },
      "outputs": [],
      "source": ""
    }
  ],
  "metadata": {
    "_change_revision": 0,
    "_is_fork": false,
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}