{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import glob\nimport plotly.graph_objects as go\nimport numpy as np","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<div class=\"alert alert-block alert-info\", style=\"font-size:110%\">\n\n<center><H1> WHERE ARE WE ???? 🧐</H1></center>\n\n\n    \n<H2> Introduction </H2>\n    \n<p>  \nHey Guys !!!.<br>\n\nIn real life, we always want to know  where we are, it is the first reflex that we have when we wake up 😴( to know if everything is still in the right place) <br> <br>\nWhen we are invited in a new building for a interview for instance 💼, our brain tempt to search for information such as.<br>\n</p>\n\n<p>\n      <ul>\n          <li>Where is the Site ? 🌏🌏 </li>\n          <li>How big is the building ? 🏙🏙</li>\n          <li>How many people are in ? 👥👥</li>\n      </ul> \n</p>\n\n<p>\nThrough these questions we are going to analysis the type of building that we have and how many peoples with smartphone 📲 are in.\n</p>"},{"metadata":{},"cell_type":"markdown","source":"![](https://cdn.facilityexecutive.com/wp-content/uploads/2019/09/38391858_ml-800x418-1.jpg)"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"# information\ntrain_paths = glob.glob('../input/indoor-location-navigation/train/*/*/*')\n\ninfo = []\n\nfor i in train_paths:\n    info.append([i.split('/')[-3],i.split('/')[-2],i.split('/')[-1]])\n\ninfo = pd.DataFrame(info, columns=['Sites','Floor','Mobile'])\ninfo","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n<div class=\"alert alert-block alert-info\", style=\"font-size:110%\">\n    <h2><u> Train Dataset Organisation </u>🎓</h2>\n\n <p>The train dataset is organised by SITES 🏢 containing FLOORS↕️ where people use DEVICES📱</p>   \n    \n<p>\n      <ul>\n          <li>Sites ID:   5cd56c0ce2acfd2d33b6ab27 </li>\n              <ul> \n                  <li> Floor: B1</li>\n                      <ul>\n                          <li> Device: 5d09a625bd54340008acddb9.txt</li>\n                          <li> Device: 5d09a625bd54340008acddb7.txt</li>\n                      </ul>\n                  <li> Floor: F1</li>\n                  <li> Floor: F2</li>\n                  <li> Floor: F3</li>\n                  <li> Floor: F4</li>\n              </ul>\n          \n</p>\n<p> The size of the Site are not homogenous in termes of floors and the number of devices are also differents.<br>\n        Lets' evaluate thta below\n    </p>"},{"metadata":{},"cell_type":"markdown","source":"<div class=\"alert alert-block alert-info\", style=\"font-size:110%\">\n    <h2><u> Sites Type descriptions </u>🏢</h2>\n\n <p>In a city there is differents types of type of building, some of them are higher, larger... than others. <br>\n    lets check below the kinnd of building that we have</p>   \n    \n    \n"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"fig = go.Figure()\n\nfig.add_trace(go.Indicator(\n    title='Number of Sites',\n    mode = \"number\",\n    value = len(info['Sites'].unique()),\n    domain = {'row': 0, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title = \"Number of Devices\",\n    mode = \"number\",\n    value = len(info['Mobile'].unique()),\n    domain = {'row': 0, 'column': 1}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Maximum Floor <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info.groupby('Sites').Floor.nunique().idxmax())},\n    mode = \"number\",\n    value = max(info.groupby('Sites').Floor.nunique()),\n    domain = {'row': 1, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Minimum Floor <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info.groupby('Sites').Floor.nunique().idxmin())},\n    mode = \"number\",\n    value = min(info.groupby('Sites').Floor.nunique()),\n    domain = {'row': 1, 'column': 1}))\n\nfig.update_layout(title=\"Types of building\",\n    grid = {'rows': 2, 'columns': 2, 'pattern': \"independent\"}\n)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"# Graph vizualisation\n\nfig = go.Figure([go.Bar(x=info.groupby('Sites').Floor.nunique().sort_values(ascending=False).index,\n                        y=info.groupby('Sites').Floor.nunique().sort_values(ascending=False).values\n                       )\n                ]\n               )\nfig.update_layout(height=800,\n    xaxis_tickangle=-90,\n    title='Highest to lowest Sites ( For sure we are not in The Wall Street Center)',\n    xaxis_tickfont_size=10,\n    yaxis=dict(\n        title='Number of floors',\n        titlefont_size=16,\n        tickfont_size=14,\n    ),\n    xaxis=dict(\n        title='Sites',\n        titlefont_size=16,\n        tickfont_size=14,\n    )\n)\n\nfig.update_layout(height=600)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"fig = go.Figure()\n\nfig.add_trace(go.Indicator(\n    title='Number of Sites',\n    mode = \"number\",\n    value = len(info['Sites'].unique()),\n    domain = {'row': 0, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title = \"Number of Devices\",\n    mode = \"number\",\n    value = len(info['Mobile'].unique()),\n    domain = {'row': 0, 'column': 1}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Most populated Site <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info.groupby('Sites').Mobile.nunique().idxmax())},\n    mode = \"number\",\n    value = max(info.groupby('Sites').Mobile.nunique()),\n    domain = {'row': 1, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Less populated Site <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info.groupby('Sites').Mobile.nunique().idxmin())},\n    mode = \"number\",\n    value = min(info.groupby('Sites').Mobile.nunique()),\n    domain = {'row': 1, 'column': 1}))\n\nfig.update_layout(title=\"Devices in Building\",\n    grid = {'rows': 2, 'columns': 2, 'pattern': \"independent\"},\n)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"# Graph vizualisation\n\nfig = go.Figure([go.Bar(x=info.groupby('Sites').Mobile.nunique().sort_values(ascending=False).index,\n                        y=info.groupby('Sites').Mobile.nunique().sort_values(ascending=False).values\n                       )\n                ]\n               )\nfig.update_layout(height=600,\n    xaxis_tickangle=-90,\n    title='Sites by population',\n    xaxis_tickfont_size=10,\n    yaxis=dict(\n        title='Number of devices',\n        titlefont_size=16,\n        tickfont_size=14,\n    ),\n    xaxis=dict(\n        title='Sites',\n        titlefont_size=16,\n        tickfont_size=14,\n    )\n)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<div class=\"alert alert-block alert-info\", style=\"font-size:110%\">\n    <h2><u> Sites type and Device effectif </u>📱</h2>\n\n <p>Is the highest Sites the most populated one ? We tend to directly correlate the size of building with the population. <br>\n    But this assumtions is not always true, lets check that below</p>   \n    "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"import numpy as np\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\n\nfig=make_subplots(\n        specs=[[{\"secondary_y\": True}]])\n\nfig.update_layout(xaxis2= {'anchor': 'y', 'overlaying': 'x', 'side': 'top'},\n                  yaxis_domain=[0, 0.94]);\n\n#Population\nfig.add_trace(\n    go.Bar(x=info.groupby('Sites').Mobile.nunique().index,\n           y=info.groupby('Sites').Mobile.nunique().values,\n           name=\"Population\",\n          ), secondary_y=False)\n\n#Floor\nfig.add_trace(\n    go.Scatter(x=info.groupby('Sites').Floor.nunique().index,\n               y=info.groupby('Sites').Floor.nunique().values,\n               name=\"Floor\",\n               #line_color=\"#ee0000\"\n              ), secondary_y=True)\n\nfig.data[1].update(xaxis='x2')\nfig.update_layout(xaxis = go.layout.XAxis(\n        title = '',\n        showticklabels=False))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<div class=\"alert alert-block alert-info\", style=\"font-size:110%\">\n    <h2><u> Floors and Population With Outlier</u>🚨</h2>\n\n <p>\nThere is 43 different types of floors and the maximum evelation of a sites in 12.<br><br>\nThis means there are some duplicated (OUTLIER). In fact, the way nominate a building level can differ from countries, region and languages <br><br>\nSome call it floor1 (F1,1F), floor2, floor3,.... and others level1 (F1, 1F), level2 .......... but the reference is the same\n     <br>\n\nLets see below how data looks with outlier\n</p>   \n    \n"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = go.Figure()\n\nfig.add_trace(go.Indicator(\n    title='Type of Floor',\n    mode = \"number\",\n    value = len(info['Floor'].unique()),\n    domain = {'row': 0, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title = \"Number of Devices\",\n    mode = \"number\",\n    value = len(info['Mobile'].unique()),\n    domain = {'row': 0, 'column': 1}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Highest Populated Floor <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info.groupby('Floor').Mobile.count().idxmax())},\n    mode = \"number\",\n    value = max(info.groupby('Floor').Mobile.count()),\n    domain = {'row': 1, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Fewest populated Floor <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info.groupby('Floor').Mobile.count().idxmin())},\n    mode = \"number\",\n    value = min(info.groupby('Floor').Mobile.count()),\n    domain = {'row': 1, 'column': 1}))\n\nfig.update_layout(\n    grid = {'rows': 2, 'columns': 2, 'pattern': \"independent\"},\n)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Graph vizualisation\n\nfig = go.Figure([go.Bar(x=info.groupby('Floor').Mobile.count().sort_values(ascending=False).index,\n                        y=info.groupby('Floor').Mobile.count().sort_values(ascending=False).values\n                       )\n                ]\n               )\nfig.update_layout(height=600,\n    xaxis_tickangle=-90,\n    title='Floor by population',\n    xaxis_tickfont_size=10,\n    yaxis=dict(\n        title='Number of Devices',\n        titlefont_size=16,\n        tickfont_size=14,\n    ),\n    xaxis=dict(\n        title='Floor',\n        titlefont_size=16,\n        tickfont_size=14,\n    )\n)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<div class=\"alert alert-block alert-info\", style=\"font-size:110%\">\n    <h2><u> Floors Mapping ( Removing Outlier )</u>💊</h2>\n\n <p>The competition let us the charge of mapping the floor distribution\n    </p>\n\n<h3>Competition Quote</h3>\n    <p>\nIMPORTANT: The integer floor used in the submission must be mapped from the char/int floors used in the dataset. The mapping is as follows:<br>\n\n* F1, 1F = 0\n* F2, 2F  =1\n* etc.\n* B1, 1B  -1\n* B2, 2B  -2\n        \nThere are other floor names in the training data, e.g., LG2, LM, etc., which you may decide to use for training, but none of these non-standard floors are found in the test set.    \n\n</p>   \n \n <h3>My proposition of mapping </h3>\n    \n* Not important floor (outlier): 'LG1', 'BM', 'BF', 'LG2', 'M', 'G', 'B', 'LM', 'P1', 'P2'\n\n* F1 = 1F = L1 = 0 \n* F2 = 2F = L2 = 1\n* F3 = L3 = 3F = 2\n* F4 = 4F = L4 = 3\n* F5 = 5F = L5 = 4\n* F6 = 6F = L6 = 5\n* F7 = 7F = L7 = 6\n* 8F = F8 = L8 = 7\n* 9F = F9 = L9 = 8\n* L10 = F10 = 9\n* L11 = 10\n* B1 = -1\n* B2 =-2\n* B3 = -3\n    \nLet's discuss about that mapping 🗣️\n    \nNevertheless we can see that more people are in quite in groundfloor \n\nWhy the -1 is  empty ?"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"info2 = info[info['Floor'].isin(['LG1', 'BM', 'BF', 'LG2', 'M', 'G', 'B', 'LM', 'P1', 'P2']) == False]\n\n\nmapping = {}\n\nmapping[0] = [\"F1\", \"1F\", \"L1\"]\nmapping[1] = [\"F2\", \"2F\", \"L2\"]\nmapping[2] = ['F3', 'L3', \"3F\"]\nmapping[3] = ['F4', \"4F\", 'L4']\nmapping[4] = ['F5', \"5F\", \"L5\"]\nmapping[5] = [\"F6\", \"6F\", 'L6']\nmapping[6] = [\"F7\", \"7F\", 'L7']\nmapping[7] = [\"8F\", \"F8\", 'L8']\nmapping[8] = ['9F', 'F9', 'L9']\nmapping[9]= ['L10', 'F10']\nmapping[10] = ['L11']\nmapping[-1] = [\"B1\"]\nmapping[-2] = [\"B2\"]\nmapping[-3] = [\"B3\"]\n\nfor i in mapping.keys():\n    for j in mapping.get(i):\n        info2 = info2.replace(j,i)\n\ninfo2","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = go.Figure()\n\nfig.add_trace(go.Indicator(\n    title='Type of Floor',\n    mode = \"number\",\n    value = len(info2['Floor'].unique()),\n    domain = {'row': 0, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title = \"Number of Devices\",\n    mode = \"number\",\n    value = len(info2['Mobile'].unique()),\n    domain = {'row': 0, 'column': 1}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Highest Populated Floor <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info2.groupby('Floor').Mobile.count().idxmax())},\n    mode = \"number\",\n    value = max(info2.groupby('Floor').Mobile.count()),\n    domain = {'row': 1, 'column': 0}))\n\nfig.add_trace(go.Indicator(\n    title= {\"text\": \"Fewest populated Floor <br><span style='font-size:0.7em;color:gray'>ID: {}</span>\".format(info2.groupby('Floor').Mobile.count().idxmin())},\n    mode = \"number\",\n    value = min(info2.groupby('Floor').Mobile.count()),\n    domain = {'row': 1, 'column': 1}))\n\nfig.update_layout(\n    grid = {'rows': 2, 'columns': 2, 'pattern': \"independent\"},\n)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Graph vizualisation\n\nfig = go.Figure([go.Bar(x=info2.groupby('Floor').Mobile.count().sort_values(ascending=False).index,\n                        y=info2.groupby('Floor').Mobile.count().sort_values(ascending=False).values\n                       )\n                ]\n               )\nfig.update_layout(height=600,\n    xaxis_tickangle=-90,\n    title='Floor by population',\n    xaxis_tickfont_size=10,\n    yaxis=dict(\n        title='Number of devices',\n        titlefont_size=16,\n        tickfont_size=14,\n    ),\n    xaxis=dict(\n        title='Floor',\n        titlefont_size=16,\n        tickfont_size=14,\n    )\n)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 🔨 Device records progressing in progress........"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}