{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":22559,"databundleVersionId":1923081,"sourceType":"competition"},{"sourceId":1982495,"sourceType":"datasetVersion","datasetId":1177370},{"sourceId":54990230,"sourceType":"kernelVersion"}],"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Thanks for Kouki's sharing[LSTM by Keras with Unified Wi-Fi Feats](http://www.kaggle.com/kokitanisaka/lstm-by-keras-with-unified-wi-fi-feats), I just make a little change to it, I am used to writing codes with pytorch, So as you can see, I changed keras code to pytorch, socre 7.53**","metadata":{}},{"cell_type":"markdown","source":"## Overview\n\nIt demonstrats how to utilize [the unified Wi-Fi dataset](https://www.kaggle.com/kokitanisaka/indoorunifiedwifids).
\nThe Neural Net model is not optimized, there's much space to improve the score. \n\nIn this notebook, I refer these two excellent notebooks.\n* [wifi features with lightgbm/KFold](https://www.kaggle.com/hiro5299834/wifi-features-with-lightgbm-kfold) by [@hiro5299834](https://www.kaggle.com/hiro5299834/)
\n I took some code fragments from his notebook.\n* [Simple ๐ 99% Accurate Floor Model ๐ฏ](https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model) by [@nigelhenry](https://www.kaggle.com/nigelhenry/)
\n I use his excellent work, the \"floor\" prediction.\n\nIt takes much much time to finish learning.
\nAnd even though I enable the GPU, it doesn't help.
\nIf anybody knows how to make it better, can you please make a comment?
\n\nThank you!","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport scipy.stats as stats\nfrom pathlib import Path\nimport glob\nimport pickle\nfrom tqdm import tqdm\nimport random\nimport os\nimport copy\nfrom sklearn.model_selection import StratifiedKFold\nfrom sklearn.preprocessing import StandardScaler, LabelEncoder\nimport torch\nfrom torch import nn\nfrom torch.utils.data import Dataset, DataLoader","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:08.032248Z","iopub.execute_input":"2025-04-07T06:54:08.032666Z","iopub.status.idle":"2025-04-07T06:54:08.037566Z","shell.execute_reply.started":"2025-04-07T06:54:08.032629Z","shell.execute_reply":"2025-04-07T06:54:08.03665Z"}},"outputs":[],"execution_count":35},{"cell_type":"markdown","source":"### options\nWe can change the way it learns with these options.
\nEspecialy **NUM_FEATS** is one of the most important options.
\nIt determines how many features are used in the training.
\nWe have 100 Wi-Fi features in the dataset, but 100th Wi-Fi signal sounds not important, right?
\nSo we can use top Wi-Fi signals if we think we need to. ","metadata":{}},{"cell_type":"code","source":"# options\n\nN_SPLITS = 4\n\nSEED = 2021\n\nNUM_FEATS = 20 # number of features that we use. there are 100 feats but we don't need to use all of them\n\nbase_path = '/kaggle'","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:08.038609Z","iopub.execute_input":"2025-04-07T06:54:08.038813Z","iopub.status.idle":"2025-04-07T06:54:08.05482Z","shell.execute_reply.started":"2025-04-07T06:54:08.038795Z","shell.execute_reply":"2025-04-07T06:54:08.054021Z"}},"outputs":[],"execution_count":36},{"cell_type":"code","source":"def set_seed(seed=42):\n random.seed(seed)\n os.environ['PYTHONHASHSEED'] = str(seed)\n np.random.seed(seed)\n torch.backends.cudnn.deterministic = True\n torch.backends.cudnn.benchmark = False\n torch.manual_seed(seed)\n torch.cuda.manual_seed_all(seed)\ndef get_timestamp():\n import time\n timestamp = ''\n for i, d in enumerate(time.localtime()):\n if i == 3:\n d += 8\n timestamp += str(d) + '-'\n if i == 4:\n break\n return timestamp[:-1]\n#ๅฎไนๆๅคฑๅฝๆฐ\ndef comp_metric(xhat, yhat, fhat, x, y, f):\n intermediate = np.sqrt((xhat-x)**2 + (yhat-y)**2) + 15 * np.abs(fhat-f)\n# intermediate = np.sqrt((xhat-x)**2 + (yhat-y)**2)\n return intermediate.sum()/xhat.shape[0]","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:08.102274Z","iopub.execute_input":"2025-04-07T06:54:08.102538Z","iopub.status.idle":"2025-04-07T06:54:08.108306Z","shell.execute_reply.started":"2025-04-07T06:54:08.102516Z","shell.execute_reply":"2025-04-07T06:54:08.107553Z"}},"outputs":[],"execution_count":37},{"cell_type":"code","source":"feature_dir = f\"{base_path}/input/indoorunifiedwifids\"\ntrain_files = sorted(glob.glob(os.path.join(feature_dir, '*_train.csv')))\ntest_files = sorted(glob.glob(os.path.join(feature_dir, '*_test.csv')))\nsubm = pd.read_csv(f'{base_path}/input/indoor-location-navigation/sample_submission.csv', index_col=0)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:08.109379Z","iopub.execute_input":"2025-04-07T06:54:08.109616Z","iopub.status.idle":"2025-04-07T06:54:08.144802Z","shell.execute_reply.started":"2025-04-07T06:54:08.109597Z","shell.execute_reply":"2025-04-07T06:54:08.144026Z"}},"outputs":[],"execution_count":38},{"cell_type":"code","source":"with open(f'{feature_dir}/train_all.pkl', 'rb') as f:\n data = pickle.load( f)\n\nwith open(f'{feature_dir}/test_all.pkl', 'rb') as f:\n test_data = pickle.load(f)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:08.146139Z","iopub.execute_input":"2025-04-07T06:54:08.146382Z","iopub.status.idle":"2025-04-07T06:54:16.486798Z","shell.execute_reply.started":"2025-04-07T06:54:08.146353Z","shell.execute_reply":"2025-04-07T06:54:16.485836Z"}},"outputs":[],"execution_count":39},{"cell_type":"code","source":"# training target features\nBSSID_FEATS = [f'bssid_{i}' for i in range(NUM_FEATS)]\nRSSI_FEATS = [f'rssi_{i}' for i in range(NUM_FEATS)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:16.487958Z","iopub.execute_input":"2025-04-07T06:54:16.488195Z","iopub.status.idle":"2025-04-07T06:54:16.492266Z","shell.execute_reply.started":"2025-04-07T06:54:16.488175Z","shell.execute_reply":"2025-04-07T06:54:16.491369Z"}},"outputs":[],"execution_count":40},{"cell_type":"code","source":"# get numbers of bssids to embed them in a layer\n\nwifi_bssids = []\nfor i in range(100):\n wifi_bssids.extend(data.iloc[:,i].values.tolist())\nwifi_bssids = list(set(wifi_bssids))\n\nwifi_bssids_size = len(wifi_bssids)\nprint(f'BSSID TYPES: {wifi_bssids_size}')\n\nwifi_bssids_test = []\nfor i in range(100):\n wifi_bssids_test.extend(test_data.iloc[:,i].values.tolist())\nwifi_bssids_test = list(set(wifi_bssids_test))\n\nwifi_bssids_size = len(wifi_bssids_test)\nprint(f'BSSID TYPES: {wifi_bssids_size}')\n\nwifi_bssids.extend(wifi_bssids_test)\nwifi_bssids_size = len(wifi_bssids)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:16.494114Z","iopub.execute_input":"2025-04-07T06:54:16.494372Z","iopub.status.idle":"2025-04-07T06:54:20.094632Z","shell.execute_reply.started":"2025-04-07T06:54:16.494321Z","shell.execute_reply":"2025-04-07T06:54:20.093874Z"}},"outputs":[{"name":"stdout","text":"BSSID TYPES: 61206\nBSSID TYPES: 33042\n","output_type":"stream"}],"execution_count":41},{"cell_type":"code","source":"# preprocess\n#ๅฏนbssid ่ฟ่ก็ผ็ \nle = LabelEncoder()\nle.fit(wifi_bssids)\n#ๅฏนsite ่ฟ่ก็ผ็ \nle_site = LabelEncoder()\nle_site.fit(data['site_id'])\n\n\n#่ฎก็ฎไธ่ก๏ผไธๆฌก่งๆต็ๆๆwifi๏ผ็ๆๆwifi ็ๅๅผๅๆนๅทฎไธ้ข่ฟ่กๅฝไธๅ\nss = StandardScaler()\nss.fit(data.loc[:,RSSI_FEATS])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:20.0957Z","iopub.execute_input":"2025-04-07T06:54:20.095972Z","iopub.status.idle":"2025-04-07T06:54:20.284359Z","shell.execute_reply.started":"2025-04-07T06:54:20.095952Z","shell.execute_reply":"2025-04-07T06:54:20.283568Z"}},"outputs":[{"execution_count":42,"output_type":"execute_result","data":{"text/plain":"StandardScaler()","text/html":"StandardScaler()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
\nThanks to the layer, we can make sense of these BSSID features.
\n
\nWe concatenate all the features and put them into LSTM.
\n
\nIf something is theoritically wrong, please correct me. Thank you in advance. ","metadata":{}},{"cell_type":"code","source":"class IndoorDataset(Dataset):\n def __init__(self, data, flag='TRAIN'):\n self.data = data\n self.flag = flag\n def __len__(self):\n return self.data.shape[0]\n def __getitem__(self, index):\n tmp_data = self.data.iloc[index]\n if self.flag == 'TRAIN':\n ## ๅ ่ฝฝๆฐๆฎไน่ฎธ่ฑ่ดน่ฎธไน
็ๆถ้ด\n return {\n 'BSSID_FEATS':tmp_data[BSSID_FEATS].values.astype(float),\n 'RSSI_FEATS':tmp_data[RSSI_FEATS].values.astype(float),\n 'site_id':tmp_data['site_id'],\n 'x':tmp_data['x'],\n 'y':tmp_data['y'],\n 'floor':tmp_data['floor'],\n }\n else:\n return {\n 'BSSID_FEATS':tmp_data[BSSID_FEATS].values.astype(float),\n 'RSSI_FEATS':tmp_data[RSSI_FEATS].values.astype(float),\n 'site_id':tmp_data['site_id']\n }\nclass simpleLSTM(nn.Module):\n def __init__(self, embedding_dim = 64, seq_len=20):\n super(simpleLSTM, self).__init__()\n self.emb_BSSID_FEATS = nn.Embedding(wifi_bssids_size, embedding_dim)\n self.emb_site_id = nn.Embedding(site_count, 2)\n self.lstm1 = nn.LSTM(input_size=256,hidden_size=128, dropout=0.3, bidirectional=False)\n self.lstm2 = nn.LSTM(input_size=128,hidden_size=16, dropout=0.1, bidirectional=False)\n self.lr = nn.Linear(NUM_FEATS, NUM_FEATS * embedding_dim)\n self.lr1 = nn.Linear(2562, 256)\n self.lr_xy = nn.Linear(16, 2)\n self.lr_floor = nn.Linear(16, 1)\n self.batch_norm1 = nn.BatchNorm1d(NUM_FEATS)\n self.batch_norm2 = nn.BatchNorm1d(2562)\n self.batch_norm3 = nn.BatchNorm1d(1)\n self.dropout = nn.Dropout(0.3)\n def forward(self, x):\n \n x_bssid = self.emb_BSSID_FEATS(x['BSSID_FEATS'])\n x_bssid = torch.flatten(x_bssid, start_dim=-2)\n \n x_site_id = self.emb_site_id(x['site_id'])\n x_site_id = torch.flatten(x_site_id, start_dim=-1)\n x_rssi = self.batch_norm1(x['RSSI_FEATS'])\n x_rssi = self.lr(x_rssi)\n x_rssi = torch.relu(x_rssi)\n \n x = torch.cat([x_bssid, x_site_id, x_rssi], dim=-1)\n x = self.batch_norm2(x)\n x = self.dropout(x)\n x = torch.relu(self.lr1(x))\n\n x = x.unsqueeze(-2)\n x = self.batch_norm3(x)\n x = x.transpose(0, 1)\n x, _ = self.lstm1(x)\n x = x.transpose(0, 1)\n x = torch.relu(x)\n x = x.transpose(0, 1)\n x, _ = self.lstm2(x)\n x = x.transpose(0, 1)\n x = torch.relu(x)\n xy = self.lr_xy(x)\n floor = self.lr_floor(x)\n floor = torch.relu(floor)\n return xy.squeeze(-2), floor.squeeze(-2)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:26.75773Z","iopub.execute_input":"2025-04-07T06:54:26.757918Z","iopub.status.idle":"2025-04-07T06:54:26.769219Z","shell.execute_reply.started":"2025-04-07T06:54:26.757901Z","shell.execute_reply":"2025-04-07T06:54:26.768412Z"}},"outputs":[],"execution_count":47},{"cell_type":"code","source":"def evaluate(model, data_loader, device='cuda'):\n model.to(device)\n model.eval()\n x_list = []\n y_list = []\n floor_list = []\n prexs_list = []\n preys_list = []\n prefloors_list = []\n for d in tqdm(data_loader):\n data_dict['BSSID_FEATS'] = d['BSSID_FEATS'].to(device).long()\n data_dict['RSSI_FEATS'] = d['RSSI_FEATS'].to(device).float()\n data_dict['site_id'] = d['site_id'].to(device).long()\n x = d['x'].to(device).float()\n y = d['y'].to(device).float()\n floor = d['floor'].to(device).long()\n x_list.append(x.cpu().detach().numpy())\n y_list.append(y.cpu().detach().numpy())\n floor_list.append(floor.cpu().detach().numpy())\n xy, floor = model(data_dict)\n prexs_list.append(xy[:, 0].cpu().detach().numpy())\n preys_list.append(xy[:, 1].cpu().detach().numpy())\n prefloors_list.append(floor.squeeze().cpu().detach().numpy())\n x = np.concatenate(x_list)\n y = np.concatenate(y_list)\n floor = np.concatenate(floor_list)\n prexs = np.concatenate(prexs_list)\n preys =np.concatenate(preys_list)\n prefloors = np.concatenate(prefloors_list)\n eval_score = comp_metric(x, y, floor, prexs, preys, prefloors)\n return eval_score\ndef get_result(model, data_loader, device='cuda'):\n model.eval()\n model.to(device)\n prexs_list = []\n preys_list = []\n prefloors_list = []\n data_dict = {}\n for d in tqdm(data_loader):\n data_dict['BSSID_FEATS'] = d['BSSID_FEATS'].to(device).long()\n data_dict['RSSI_FEATS'] = d['RSSI_FEATS'].to(device).float()\n data_dict['site_id'] = d['site_id'].to(device).long()\n xy, floor = model(data_dict)\n prexs_list.append(xy[:, 0].cpu().detach().numpy())\n preys_list.append(xy[:, 1].cpu().detach().numpy())\n prefloors_list.append(floor.squeeze(-1).cpu().detach().numpy())\n prexs = np.concatenate(prexs_list)\n preys =np.concatenate(preys_list)\n prefloors = np.concatenate(prefloors_list)\n return prexs, preys, prefloors","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T06:54:26.77Z","iopub.execute_input":"2025-04-07T06:54:26.770281Z","iopub.status.idle":"2025-04-07T06:54:26.796884Z","shell.execute_reply.started":"2025-04-07T06:54:26.770253Z","shell.execute_reply":"2025-04-07T06:54:26.79599Z"}},"outputs":[],"execution_count":48},{"cell_type":"code","source":"score_df = pd.DataFrame()\noof = list()\npredictions = list()\n\noof_x, oof_y, oof_f = np.zeros(data.shape[0]), np.zeros(data.shape[0]), np.zeros(data.shape[0])\npreds_x, preds_y = 0, 0\npreds_f_arr = np.zeros((test_data.shape[0], N_SPLITS))\ndata=data.iloc[:40000]\n# print(data['path'].unique())\n\nfor fold, (trn_idx, val_idx) in enumerate(StratifiedKFold(n_splits=N_SPLITS, shuffle=True, random_state=SEED).split(data.loc[:, 'path'], data.loc[:, 'path'])):\n \n train_data = data.loc[trn_idx]\n valid_data = data.loc[val_idx]\n #valid_data=train_data\n train_dataset = IndoorDataset(train_data)\n train_dataloader = DataLoader(train_dataset, batch_size=128, shuffle=True, num_workers=6)\n valid_dataset = IndoorDataset(valid_data)\n valid_dataloader = DataLoader(valid_dataset, batch_size=128, shuffle=True, num_workers=6)\n test_dataset = IndoorDataset(test_data, 'TEST')\n test_dataloader = DataLoader(test_dataset, batch_size=128, shuffle=False, num_workers=6)\n device = 'cuda' if torch.cuda.is_available() else 'cpu'\n model = simpleLSTM()\n model = model.to(device)\n \n mse = nn.MSELoss()\n mse = mse.to(device)\n optim = torch.optim.Adam(model.parameters(), lr=5e-3)\n \n data_dict ={}\n best_loss = 1000\n num_epochs = 100\n best_epoch = 0\n for epoch in range(num_epochs):\n model.train()\n losses = []\n pbar = tqdm(train_dataloader)\n for d in pbar:\n data_dict['BSSID_FEATS'] = d['BSSID_FEATS'].to(device).long()\n data_dict['RSSI_FEATS'] = d['RSSI_FEATS'].to(device).float()\n data_dict['site_id'] = d['site_id'].to(device).long()\n x = d['x'].to(device).float().unsqueeze(-1)\n y = d['y'].to(device).float().unsqueeze(-1)\n floor = d['floor'].to(device).long()\n xy, floor = model(data_dict)\n label = torch.cat([x, y], dim=-1)\n loss = mse(xy, label)\n loss.backward()\n optim.step()\n optim.zero_grad()\n losses.append(loss.cpu().detach().numpy())\n pbar.set_description(f'loss:{np.mean(losses)}')\n score = evaluate(model, valid_dataloader, device)\n if score < best_loss:\n best_loss = score\n best_epoch = epoch\n best_model = copy.deepcopy(model)\n if best_epoch + 2\nTo make it right, we can incorporate [@nigelhenry](https://www.kaggle.com/nigelhenry/)'s [excellent work](https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model).
","metadata":{}},{"cell_type":"code","source":"# simple_accurate_99 = pd.read_csv('../input/simple-99-accurate-floor-model/submission.csv')\n\n# all_preds['floor'] = simple_accurate_99['floor'].values","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T07:02:17.303167Z","iopub.status.idle":"2025-04-07T07:02:17.303567Z","shell.execute_reply":"2025-04-07T07:02:17.303398Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# all_preds.to_csv('submission.csv')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-07T07:02:17.304181Z","iopub.status.idle":"2025-04-07T07:02:17.304565Z","shell.execute_reply":"2025-04-07T07:02:17.304404Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"That's it. \n\nThank you for reading all of it.\n\nI hope it helps!\n\nPlease make comments if you found something to point out, insights or suggestions. ","metadata":{}}]}