{"cells":[{"metadata":{"_cell_guid":"97544821-94e5-4092-bdcd-88b7f21df7d2","_uuid":"7a6d935c76af0c3b530e2cc8b540ce953d8a930b"},"cell_type":"markdown","source":"### Importing libraries"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pylab as plt\nplt.rcParams[\"figure.figsize\"] = [8,5]\nimport seaborn as sns\nsns.set()\nimport re\nfrom sklearn.dummy import DummyClassifier\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.model_selection import GridSearchCV, train_test_split, StratifiedKFold\nfrom sklearn.metrics import (roc_auc_score, confusion_matrix)\nfrom sklearn.cluster import KMeans\nimport xgboost as xgb\n\n\nimport os\nprint(os.listdir(\"../input\"))","execution_count":62,"outputs":[]},{"metadata":{"_cell_guid":"87a4462d-3027-41a9-8852-0ab546359159","_uuid":"0f105946f5ee13ead45da87aaba206ac568d456f"},"cell_type":"markdown","source":"### Importing dataset + light preprocessing"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"infile = \"../input/train_sample.csv\"\ndf = pd.read_csv(infile)\ndf[\"attributed_time\"] = df[\"attributed_time\"].fillna(\"0\")\ndf[\"click_time\"] = pd.to_datetime( df[\"click_time\"])\ndf[\"ct_month\"] = df[\"click_time\"].map( lambda x: x.month)\ndf[\"ct_year\"] = df[\"click_time\"].map( lambda x: x.year)\ndf[\"ct_day\"] = df[\"click_time\"].map( lambda x: x.day)\ndf[\"ct_timeofday\"] = df[\"click_time\"].map( lambda x: x.hour*60 + x.minute)\n\ndf_click_span = df.groupby(\"ip\").agg({\"click_time\": lambda x : (x.max() - x.min()).seconds }).reset_index(level=0).rename(columns={\"click_time\":\"click_time_span\"})\ndf = df.merge( df_click_span, how=\"left\", on=\"ip\")\ndf_click_count = df.groupby(\"ip\").count()[[\"channel\"]].reset_index(level=0).rename(columns={\"channel\":\"click_count\"})\ndf = df.merge( df_click_count, how=\"inner\", on=\"ip\")\n\ndel df[\"click_time\"]\ndel df[\"attributed_time\"]","execution_count":28,"outputs":[]},{"metadata":{"_uuid":"60af4c855a53077daa36196a00d13b258f5d955a"},"cell_type":"markdown","source":"We simply added two features: one counting the number of clicks per ip address, one counting the time span between the first and the last one."},{"metadata":{"_cell_guid":"e6f5189e-ae09-4b6d-baea-401fde5f79ed","_uuid":"f8540fb3d65d6a0537a1772ca7a9680030696b2c","trusted":true},"cell_type":"code","source":"df.head(10)","execution_count":71,"outputs":[]},{"metadata":{"_uuid":"1b627d5c03d20870a68d75ffd7141b2fd4e67031"},"cell_type":"markdown","source":"### Class balance of the data"},{"metadata":{"_cell_guid":"0346c2d0-67f9-4e53-80d9-899100f07990","_uuid":"0306c230a932720fc7c6a415ceb293d42a69b2f9","trusted":true},"cell_type":"code","source":"df[\"is_attributed\"].plot(kind=\"hist\",normed=True,bins=2)\nax = plt.gca()\nax.set_yticks(())\nax.set_xticks( (.25,.75))\nax.set_xticklabels( [\"attributed\", \"not attributed\"])\ndf[\"is_attributed\"].value_counts()","execution_count":63,"outputs":[]},{"metadata":{"_uuid":"5dd9c554fd7cb502fae7c2b03a363d56ee22479d"},"cell_type":"markdown","source":"The training set is very imbalanced with ~0.25% of the data labelised \"not attributed\"."},{"metadata":{"_uuid":"455009d8fb58a74586ae8654037f044cc38b321b"},"cell_type":"markdown","source":"### Constructing features and label vectors"},{"metadata":{"_cell_guid":"e78bf9c6-7186-43ac-9608-a68db1b0a43e","_uuid":"83b1c68204caec6fa31e9004dc12c0b325f5cf22","scrolled":true,"trusted":true},"cell_type":"code","source":"features = [\"app\", \"device\", \"os\", \"channel\", \"ct_year\", \"ct_month\", \"ct_day\", \"ct_timeofday\",\"click_time_span\",\"click_count\"]\nX = df[features].values\ny = df[\"is_attributed\"].values","execution_count":68,"outputs":[]},{"metadata":{"_uuid":"79e730eb7cecd4bfaa9fc4f17a79644ef8cea31c"},"cell_type":"markdown","source":"## Predictive models"},{"metadata":{"_uuid":"261a38cbbca326a8fd94b3542ef626cabd1a18ad"},"cell_type":"markdown","source":"This first trial will use different classifiers (logistic regression, random forests and boosted trees to be be compared to a baseline given by the dummy classifier).  The train/test split is done with no regard to the class balance between the two, and the classifiers are not passed any arguments to penalize misclassification of the minority class."},{"metadata":{"trusted":true,"_uuid":"0ed544c882c6c1c070b1b2ec6d8e37a6cd3a3f72"},"cell_type":"code","source":"print( \"-\"*80)\nprint( \"-\"*80)\nprint( \"\\t\\tNO REBALANCE\")\nprint( \"-\"*80)\nprint( \"-\"*80)\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n\ndc = DummyClassifier(strategy=\"most_frequent\")\nlr = LogisticRegression(C=0.035)\nrf = RandomForestClassifier( n_estimators=11, max_depth=9)\nbt = xgb.XGBClassifier(max_depth=10, n_estimators=11, eta=0.6)\n\nfor classifier in (dc,lr,rf,bt):\n    classifier.fit(X_train,y_train)\n    y_pred_proba = classifier.predict_proba( X_test)[:,1]\n    y_pred = y_pred_proba > 0.5\n    print( \"-\"*80)\n    print( \"%s\" % type(classifier).__name__)\n    print( \"\\tconfusion matrix: \", confusion_matrix(y_test, y_pred).tolist())\n    print( \"\\troc_auc score: %.3f\" % roc_auc_score(y_test, y_pred_proba))","execution_count":69,"outputs":[]},{"metadata":{"_uuid":"92e8241e6f9c276c93e4756539fd1acc8d4249ab"},"cell_type":"markdown","source":"The tree classifier dominate over the logistic regression which is very normal since we have not done any preprocessing work for scaling or one-hot encoding the features. It is almost surprising that the logistic regression is closer to the trees than to the baseline.\n\nMost classification mistakes are false negatives, which is a normal consequence of the extreme minority of positive examples.\n\nNote that if you run this block several times, the number of the non-dummy classifiers will fluctuate. This is because the number of positives in test/train will change significantly. This is because of using pure randomness in splitting train/test."},{"metadata":{"trusted":true,"_uuid":"556405f627d73da2bace5bc8a0b111054479b19d"},"cell_type":"code","source":"print( \"-\"*80)\nprint( \"-\"*80)\nprint( \"\\t\\tWITH REBALANCE\")\nprint( \"-\"*80)\nprint( \"-\"*80)\ns = StratifiedKFold(n_splits=10,shuffle=True)\nindices_train, indices_test = list(s.split(X,y))[0]\nX_train, X_test, y_train, y_test = X[indices_train], X[indices_test], y[indices_train], y[indices_test]\nc_weights = { val:1./np.mean(y==val) for val in np.unique(y)}\nscale_pos = np.sum(y==0)/np.sum(y==1)\n\ndc = DummyClassifier(strategy=\"most_frequent\")\nlr = LogisticRegression(C=0.035, class_weight=c_weights)\nrf = RandomForestClassifier( n_estimators=11, max_depth=9, class_weight=c_weights)\nbt = xgb.XGBClassifier(max_depth=10, n_estimators=11, eta=0.6, scale_pos_weight=scale_pos)\n\nfor classifier in (dc,lr,rf,bt):\n    classifier.fit(X_train,y_train)\n    y_pred_proba = classifier.predict_proba( X_test)[:,1]\n    y_pred = y_pred_proba > 0.5\n    print( \"-\"*80)\n    print( \"%s\" % type(classifier).__name__)\n    print( \"\\tconfusion matrix: \", confusion_matrix(y_test, y_pred).tolist())\n    print( \"\\troc_auc score: %.3f\" % roc_auc_score(y_test, y_pred_proba))","execution_count":70,"outputs":[]},{"metadata":{"_uuid":"2e9dd8b98c9b21c101699e62b1c9ab2d4e48b50b"},"cell_type":"markdown","source":"Here the splitting is done with stratification, implying that the ratio of positives in test and train are very similar. Second, the classifiers were passed an argument to be aware of the class imbalance (`c_weights` or `scale_pos`). These increases the false negative penality over the false positive one. The consequence is that the rate of false negative has diminished significantly and the roc_auc score also took a jump in the right direction."},{"metadata":{"_uuid":"3a091d095b721b46874df52b00d395a548e84749"},"cell_type":"markdown","source":"TO DO:\n    - more preprocessing\n    - look into feature engineering\n    - hyperparameter selection\n    - possibly stack multiple models"},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"7cc39b5de3e291affa87ec843f1d4ea746ba0ba9"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}