{"cells":[{"metadata":{"_kg_hide-input":true,"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))","execution_count":21,"outputs":[]},{"metadata":{"_cell_guid":"c7135348-304e-4a31-8ff5-603269736172","_uuid":"5517b87713fda135df42d92f5f83c5effd756132"},"cell_type":"markdown","source":"It was pointed out that over half of the IPs that appear in test data are new (not in the training data). In this notebook i will show how this translate if we compare 2 days from the training data. in this case we will compare day 7 and day 8 of November. \nWe could see that the same pattern exist in the training set. 77802 of the unique IPs in day 8 are new (didnt exist in day 7). However, if we check the amount of transactions in day 8 with new IPs, we notice that it only make a small percentage of ~ 2%. \n"},{"metadata":{"_cell_guid":"9e0d92ad-73fc-4408-b5ad-30e8bd741b5f","_uuid":"25f7c3f509a5d315c5724f429b6dad4babe04131"},"cell_type":"markdown","source":"**Load train data from 2017-11-07"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"collapsed":true},"cell_type":"code","source":"train_day_7 = pd.read_csv('../input/train.csv', skiprows=range(1,9308568), nrows=59633310, usecols=['click_time', 'ip', 'is_attributed'])","execution_count":22,"outputs":[]},{"metadata":{"_cell_guid":"3e2d4bd7-d1a7-486c-831a-e3bef393d41e","_uuid":"9fcdb0a180c248bd837f068a0b6a35c8a5d25797"},"cell_type":"markdown","source":"**Load train data from 2017-11-08"},{"metadata":{"_cell_guid":"96042977-5c98-4483-aa7b-26cfa861796e","collapsed":true,"_uuid":"08b2ad78d0e4137071c304e2564fa7e4c8afabe9","trusted":true},"cell_type":"code","source":"train_day_8 = pd.read_csv('../input/train.csv', skiprows=range(1,68941878), nrows=62945075, usecols=['click_time', 'ip', 'is_attributed'])","execution_count":23,"outputs":[]},{"metadata":{"_cell_guid":"36545631-8161-4530-92c4-2116d73e8656","_uuid":"cb2aef64632004e106f323f399338fccaad593eb","trusted":true},"cell_type":"code","source":"print (train_day_7.shape, train_day_7.is_attributed.mean())","execution_count":24,"outputs":[]},{"metadata":{"_cell_guid":"be2b40c6-d935-48e7-a8c5-d95bb9687b08","_uuid":"2c41a671c4662be4eb155ff1c810dfb9b4d7e31f","trusted":true},"cell_type":"code","source":"print (train_day_8.shape, train_day_8.is_attributed.mean())","execution_count":25,"outputs":[]},{"metadata":{"_cell_guid":"2f4d5a55-8111-4a07-b2e6-f0b4e4db15f3","_uuid":"972d368298afcf790bada7c9e4b0f9ab1f6feb7c","trusted":true},"cell_type":"code","source":"train_day_7.ip.nunique()","execution_count":26,"outputs":[]},{"metadata":{"_cell_guid":"11339063-b1c2-40f9-9383-b55f471b37ea","_uuid":"d90049a487d7fbcaf7572573fa8e015315615f51","trusted":true},"cell_type":"code","source":"train_day_8.ip.nunique()","execution_count":27,"outputs":[]},{"metadata":{"_uuid":"36ef5917d05d71c09c2d259672301624adb089a2"},"cell_type":"markdown","source":"How many unique IPs are new in day8 (didnt appear in day7)"},{"metadata":{"_cell_guid":"df1ae096-5e43-4f25-976a-c8313a75cf18","_uuid":"5932694c1911d8ece4cc7fcc193a4b2921c47106","trusted":true},"cell_type":"code","source":"train_day_8[~train_day_8.ip.isin(train_day_7.ip.unique())].ip.nunique()","execution_count":28,"outputs":[]},{"metadata":{"_uuid":"3f70614bb61c749c6aca4df0ccb3c522d20532e2"},"cell_type":"markdown","source":"so about half of the unique IPs in day 8 are new. Now lets see how many transactions in day8 are from new IPs"},{"metadata":{"_cell_guid":"eebf16e3-18ed-4383-9d89-eec20f4d3621","_uuid":"e6872dd0f48a86598246c2694ffb31692f0b533d","trusted":true},"cell_type":"code","source":"train_day_8[~train_day_8.ip.isin(train_day_7.ip.unique())].shape[0] / float(train_day_8.shape[0])","execution_count":29,"outputs":[]},{"metadata":{"_uuid":"24800994beba347b2791fefceea43404732015fd"},"cell_type":"markdown","source":"As can be seen only 2.5 % of transactions in day 8 are from new IPs. What is the mean is_attributed for the transactions in day8 that are from new IPs "},{"metadata":{"_cell_guid":"8eedb07b-75d6-4f30-9e3a-d6f6636b6edd","_uuid":"e0eec3f6b873c587ac3f7c9104dabd45de45c0b2","trusted":true},"cell_type":"code","source":"train_day_8[~train_day_8.ip.isin(train_day_7.ip.unique())].is_attributed.mean()","execution_count":30,"outputs":[]},{"metadata":{"_uuid":"9ec79a2d1b7b96673ecaa1ab654acee92e3c8e92"},"cell_type":"markdown","source":"As can be seen transactions from new IPs are about 15 times more likely to end in the app download. 0.037 compared to 0.0025"}],"metadata":{"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"}},"nbformat":4,"nbformat_minor":1}