{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"}],"dockerImageVersionId":30635,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"[Discussion - is eeg_ids missing in train.csv but available in train_eegs folder? [Solved]](https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/467058)\n\n## Those are files without paired spectrograms where I missed some cleanup. by @sohier\n\n## Ignore these files or use it for pseudolabels","metadata":{}},{"cell_type":"markdown","source":"## Brain activity notebook series\n\n### [EEGS 10–20 system](https://www.kaggle.com/code/seshurajup/eegs-10-20-system)\nBetter understanding eegs 10-20 system\n### [Missing Eeg_ids Train.csv vs train_eegs [Resolved]](https://www.kaggle.com/code/seshurajup/missing-eeg-ids-in-train-csv-vs-train-eegs-parquet)\nExtra training eggs [Resolved] as we can ignore it\n### [EDA train.csv](https://www.kaggle.com/code/seshurajup/eda-train-csv)\nDetailed analysis of the train.csv\n### [Eegs Pairing Analysis & Features](https://www.kaggle.com/code/seshurajup/eegs-pairing-analysis-features)\nPairing features analysis and build features\n### [Eegs Target Analysis - Correct way to merge target](https://www.kaggle.com/code/seshurajup/eegs-target-analysis-correct-way-to-merge-target)\nHow to choice the target votes for training\n### [Eegs Train Split (CV)](https://www.kaggle.com/seshurajup/eegs-train-splits-cv)\ngenerate better train split without patient_id overlap\n\n#### **Upvote my work if it is useful**","metadata":{}},{"cell_type":"code","source":"from glob import glob\nefiles = glob(\"/kaggle/input/hms-harmful-brain-activity-classification/train_eegs/*.parquet\")","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(efiles), efiles[0]","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:23.136230Z","iopub.execute_input":"2024-01-11T01:58:23.136516Z","iopub.status.idle":"2024-01-11T01:58:23.143433Z","shell.execute_reply.started":"2024-01-11T01:58:23.136488Z","shell.execute_reply":"2024-01-11T01:58:23.142282Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"efile_ids = [int(x.split(\"/\")[-1].split(\".\")[0]) for x in efiles]","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:23.144810Z","iopub.execute_input":"2024-01-11T01:58:23.145117Z","iopub.status.idle":"2024-01-11T01:58:23.164341Z","shell.execute_reply.started":"2024-01-11T01:58:23.145093Z","shell.execute_reply":"2024-01-11T01:58:23.163061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\ntrain = pd.read_csv(\"/kaggle/input/hms-harmful-brain-activity-classification/train.csv\")\neeg_ids = [int(x) for x in list(set(train['eeg_id']))]\nlen(eeg_ids)","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:23.166294Z","iopub.execute_input":"2024-01-11T01:58:23.166790Z","iopub.status.idle":"2024-01-11T01:58:23.671195Z","shell.execute_reply.started":"2024-01-11T01:58:23.166758Z","shell.execute_reply":"2024-01-11T01:58:23.669909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(set(eeg_ids) - set(efile_ids))","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:23.672367Z","iopub.execute_input":"2024-01-11T01:58:23.672632Z","iopub.status.idle":"2024-01-11T01:58:23.687009Z","shell.execute_reply.started":"2024-01-11T01:58:23.672610Z","shell.execute_reply":"2024-01-11T01:58:23.685619Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(set(efile_ids) - set(eeg_ids))","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:23.688976Z","iopub.execute_input":"2024-01-11T01:58:23.689241Z","iopub.status.idle":"2024-01-11T01:58:23.699301Z","shell.execute_reply.started":"2024-01-11T01:58:23.689218Z","shell.execute_reply":"2024-01-11T01:58:23.697753Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\", \".join(sorted([str(x) for x in list(set(efile_ids) - set(eeg_ids))]))","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:27.933815Z","iopub.execute_input":"2024-01-11T01:58:27.934112Z","iopub.status.idle":"2024-01-11T01:58:27.943514Z","shell.execute_reply.started":"2024-01-11T01:58:27.934088Z","shell.execute_reply":"2024-01-11T01:58:27.942757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from glob import glob\nsfiles = glob(\"/kaggle/input/hms-harmful-brain-activity-classification/train_spectrograms/*.parquet\")","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:32.191794Z","iopub.execute_input":"2024-01-11T01:58:32.192124Z","iopub.status.idle":"2024-01-11T01:58:32.522868Z","shell.execute_reply.started":"2024-01-11T01:58:32.192094Z","shell.execute_reply":"2024-01-11T01:58:32.521031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(sfiles), sfiles[0]","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:37.116487Z","iopub.execute_input":"2024-01-11T01:58:37.116899Z","iopub.status.idle":"2024-01-11T01:58:37.124640Z","shell.execute_reply.started":"2024-01-11T01:58:37.116866Z","shell.execute_reply":"2024-01-11T01:58:37.123361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sfile_ids = [int(x.split(\"/\")[-1].split(\".\")[0]) for x in sfiles]","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:43.294974Z","iopub.execute_input":"2024-01-11T01:58:43.295324Z","iopub.status.idle":"2024-01-11T01:58:43.308039Z","shell.execute_reply.started":"2024-01-11T01:58:43.295294Z","shell.execute_reply":"2024-01-11T01:58:43.306866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\ntrain = pd.read_csv(\"/kaggle/input/hms-harmful-brain-activity-classification/train.csv\")\nspec_ids = [int(x) for x in list(set(train['spectrogram_id']))]\nlen(spec_ids)","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:58:52.633907Z","iopub.execute_input":"2024-01-11T01:58:52.634261Z","iopub.status.idle":"2024-01-11T01:58:52.745830Z","shell.execute_reply.started":"2024-01-11T01:58:52.634236Z","shell.execute_reply":"2024-01-11T01:58:52.744545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(set(spec_ids) - set(sfile_ids))","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:57:35.615290Z","iopub.execute_input":"2024-01-11T01:57:35.615716Z","iopub.status.idle":"2024-01-11T01:57:35.626635Z","shell.execute_reply.started":"2024-01-11T01:57:35.615683Z","shell.execute_reply":"2024-01-11T01:57:35.625406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(set(sfile_ids) - set(spec_ids))","metadata":{"execution":{"iopub.status.busy":"2024-01-11T01:59:02.065971Z","iopub.execute_input":"2024-01-11T01:59:02.066803Z","iopub.status.idle":"2024-01-11T01:59:02.079354Z","shell.execute_reply.started":"2024-01-11T01:59:02.066742Z","shell.execute_reply":"2024-01-11T01:59:02.077430Z"},"trusted":true},"execution_count":null,"outputs":[]}]}