{"cells":[{"metadata":{"_uuid":"dbcb548c192b5e9e99a309b1bd1f03e0803a2cb0"},"cell_type":"markdown","source":"I'm seeing a lot of models using variables like week of year and month to train their models, and so I build this kernel to show what variables of date can be used and what variables don't add to the model."},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import numpy\nimport pandas \n\nimport os\nprint(os.listdir(\"../input\"))\n\n\nimport matplotlib.pyplot as plt\n%matplotlib inline\n","execution_count":8,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_kg_hide-input":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"collapsed":true},"cell_type":"code","source":"train = pandas.read_csv('../input/train.csv')\ntest = pandas.read_csv('../input/test.csv')","execution_count":4,"outputs":[]},{"metadata":{"_kg_hide-input":true,"collapsed":true,"trusted":true,"_uuid":"104d752cd5a202cab1bb24074a98f7003d9a2018"},"cell_type":"code","source":"for i in [train,test]:\n    i['activation_date'] = pandas.to_datetime(i['activation_date'])\n    i['month'] = i['activation_date'].dt.month\n    i['day'] = i['activation_date'].dt.day\n    i['day_of_week'] = i['activation_date'].dt.dayofweek\n    i['weekofyear'] = i['activation_date'].dt.weekofyear","execution_count":7,"outputs":[]},{"metadata":{"_uuid":"b74110b6b23c58c911ac9291f6185f010d284b9b"},"cell_type":"markdown","source":"We can see that we have the same month in train and test, so there is no need of using this variable in the model.\n\nThe day variable have a small overlap, but we have to be cautious when using it because this diference will not apear on our cross-validation.\n\nDay of week is the only variable that I see using in models, cause we have a full overlap in train and test.\n\nIn week of year we have 0 overlap. If we build our models with week of year, we may see improvment on our cross-validation, but not on the test set\n"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"393439c5816127bbb27184cb76d81b66bee4fcc0"},"cell_type":"code","source":"### Thanks SRK for this script for venn plots https://www.kaggle.com/sudalairajkumar/simple-exploration-baseline-notebook-avito\n\nfrom matplotlib_venn import venn2\n\nfor i in ['month','day','day_of_week','weekofyear']:\n\n    plt.figure(figsize=(10,7))\n    venn2([set(train[i].unique()), set(test[i].unique())], set_labels = ('Train set', 'Test set') )\n    plt.title(\"Number of \" + i + \" in train and test\", fontsize=15)\n    plt.show()\n","execution_count":10,"outputs":[]},{"metadata":{"collapsed":true,"trusted":true,"_uuid":"5fbe351313233cfd12d22ac11b82219154339209"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.5","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}