{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"}],"dockerImageVersionId":30664,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Harmful Brain Activity Classification - EDA / Training and Testing data**\n\n## **Project by:** [Aarish Asif Khan](https://www.kaggle.com/aarishasifkhan)\n\n## **Date:** 3rd February 2024\n\n## **Dataset:** [HMS - Harmful Brain Activity Dataset](https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification)","metadata":{}},{"cell_type":"markdown","source":"# **Exploratory Data-Analysis with `train.csv` and `test.csv` datasets**","metadata":{}},{"cell_type":"markdown","source":"### **What is EDA Analysis?**\n\n\nExploratory Data Analysis (EDA) is a critical step in data analysis that involves examining and visualizing data to uncover patterns and gain insights. EDA helps understand data characteristics, identify outliers, explore distributions, and assess relationships between variables. \n\nThrough techniques like histograms, scatter plots, and correlation analyses, EDA guides subsequent modeling decisions, informs feature engineering, and ensures data quality. It is a key component in forming hypotheses, detecting anomalies, and communicating findings to stakeholders, providing a foundational understanding of the dataset for effective decision-making and modeling.","metadata":{}},{"cell_type":"markdown","source":"### **Import Libraries:**\n\n1. `pandas:` A library for data manipulation and analysis.\n\n2. `matplotlib.pyplot:` A library for creating static, animated, and interactive visualizations in Python.\n\n3. `seaborn:` A statistical data visualization library based on Matplotlib, providing a high-level interface for drawing attractive and informative statistical graphics.","metadata":{}},{"cell_type":"code","source":"# Import necessary libraries\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.087617Z","iopub.execute_input":"2024-03-18T08:01:58.088193Z","iopub.status.idle":"2024-03-18T08:01:58.094137Z","shell.execute_reply.started":"2024-03-18T08:01:58.088156Z","shell.execute_reply":"2024-03-18T08:01:58.092650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Load Data:**\n\nLoad the training and testing datasets (`train.csv and test.csv`) into pandas DataFrames (`train_data and test_data`).","metadata":{}},{"cell_type":"code","source":"# Load train.csv and test.csv\ntrain_data = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\ntest_data = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/test.csv')","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.096187Z","iopub.execute_input":"2024-03-18T08:01:58.096682Z","iopub.status.idle":"2024-03-18T08:01:58.339202Z","shell.execute_reply.started":"2024-03-18T08:01:58.096650Z","shell.execute_reply":"2024-03-18T08:01:58.337835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Display Data Information:**\n\nPrint basic information about the datasets, such as column names, data types, and non-null counts.","metadata":{}},{"cell_type":"code","source":"# Display basic information about the datasets\nprint(\"Training Data Info:\")\nprint(train_data.info())\n\nprint(\"\\nTesting Data Info:\")\nprint(test_data.info())\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.341268Z","iopub.execute_input":"2024-03-18T08:01:58.341764Z","iopub.status.idle":"2024-03-18T08:01:58.377264Z","shell.execute_reply.started":"2024-03-18T08:01:58.341698Z","shell.execute_reply":"2024-03-18T08:01:58.375938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Summary Statistics:**\n\nDisplay summary statistics for numerical columns in the training dataset, including measures like mean, standard deviation, minimum, maximum, etc.","metadata":{}},{"cell_type":"code","source":"# Summary statistics for numerical columns in train.csv\nprint(\"\\nSummary Statistics - Training Data:\")\nprint(train_data.describe())\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.378500Z","iopub.execute_input":"2024-03-18T08:01:58.379427Z","iopub.status.idle":"2024-03-18T08:01:58.461924Z","shell.execute_reply.started":"2024-03-18T08:01:58.379377Z","shell.execute_reply":"2024-03-18T08:01:58.460532Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Check Missing Values:**\n\nIdentify and print the number of missing values in each column of the training dataset.","metadata":{}},{"cell_type":"code","source":"# Check for missing values in train.csv\nprint(\"\\nMissing Values - Training Data:\")\nprint(train_data.isnull().sum())\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.465429Z","iopub.execute_input":"2024-03-18T08:01:58.465950Z","iopub.status.idle":"2024-03-18T08:01:58.487977Z","shell.execute_reply.started":"2024-03-18T08:01:58.465907Z","shell.execute_reply":"2024-03-18T08:01:58.486604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Distribution of Classes:**\n\nSum the votes for each brain activity class in the training dataset to understand the distribution.","metadata":{}},{"cell_type":"code","source":"# Distribution of classes in train.csv\nclass_columns = ['seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']\nclass_distribution = train_data[class_columns].sum()\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.490525Z","iopub.execute_input":"2024-03-18T08:01:58.490995Z","iopub.status.idle":"2024-03-18T08:01:58.501612Z","shell.execute_reply.started":"2024-03-18T08:01:58.490961Z","shell.execute_reply":"2024-03-18T08:01:58.500075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Visualize Class Distribution:**\n\nPlot a bar chart to visualize the distribution of votes for each brain activity class in the training dataset.","metadata":{}},{"cell_type":"code","source":"# Plot the distribution of classes\nplt.figure(figsize=(10, 6))\nsns.barplot(x=class_distribution.index, y=class_distribution.values)\nplt.title('Distribution of Classes in Training Data')\nplt.xlabel('Brain Activity Classes')\nplt.ylabel('Number of Votes')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.503199Z","iopub.execute_input":"2024-03-18T08:01:58.503559Z","iopub.status.idle":"2024-03-18T08:01:58.870022Z","shell.execute_reply.started":"2024-03-18T08:01:58.503528Z","shell.execute_reply":"2024-03-18T08:01:58.868699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Visualize Relationship:**\n\nPlot a boxplot to visualize the relationship between seizure votes and patient IDs in the training dataset.","metadata":{}},{"cell_type":"code","source":"# Relationship between classes and patient_id in train.csv\nplt.figure(figsize=(12, 8))\nsns.boxplot(x='patient_id', y='seizure_vote', data=train_data)\nplt.title('Relationship between Seizure Votes and Patient ID in Training Data')\nplt.xlabel('Patient ID')\nplt.ylabel('Seizure Votes')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:01:58.871856Z","iopub.execute_input":"2024-03-18T08:01:58.872581Z","iopub.status.idle":"2024-03-18T08:02:35.106297Z","shell.execute_reply.started":"2024-03-18T08:01:58.872527Z","shell.execute_reply":"2024-03-18T08:02:35.105064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Correlation Heatmap:**\n\nGenerate a heatmap to visualize the correlations between numerical features in the training dataset.","metadata":{}},{"cell_type":"code","source":"# Select only numeric columns for correlation\nnumeric_columns = train_data.select_dtypes(include=['number'])\n\n# Plot the correlation heatmap\nplt.figure(figsize=(12, 8))\nsns.heatmap(numeric_columns.corr(), annot=True, cmap='coolwarm', linewidths=.5)\nplt.title('Correlation Heatmap - Training Data')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:02:35.107782Z","iopub.execute_input":"2024-03-18T08:02:35.108117Z","iopub.status.idle":"2024-03-18T08:02:36.200520Z","shell.execute_reply.started":"2024-03-18T08:02:35.108087Z","shell.execute_reply":"2024-03-18T08:02:36.199461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check for NaN values in numeric columns\nprint(\"NaN Values in Numeric Columns:\")\nprint(numeric_columns.isnull().sum())\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:02:36.201922Z","iopub.execute_input":"2024-03-18T08:02:36.202471Z","iopub.status.idle":"2024-03-18T08:02:36.212446Z","shell.execute_reply.started":"2024-03-18T08:02:36.202438Z","shell.execute_reply":"2024-03-18T08:02:36.211068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Visualize Distribution:**\n\nPlot a histogram to visualize the distribution of EEG label offset seconds in the training dataset.","metadata":{}},{"cell_type":"code","source":"# Visualize distribution of eeg_label_offset_seconds in train.csv\nplt.figure(figsize=(10, 6))\nsns.histplot(train_data['eeg_label_offset_seconds'], bins=30, kde=True)\nplt.title('Distribution of EEG Label Offset Seconds in Training Data')\nplt.xlabel('EEG Label Offset Seconds')\nplt.ylabel('Count')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-03-18T08:02:36.214260Z","iopub.execute_input":"2024-03-18T08:02:36.215015Z","iopub.status.idle":"2024-03-18T08:02:37.247339Z","shell.execute_reply.started":"2024-03-18T08:02:36.214968Z","shell.execute_reply":"2024-03-18T08:02:37.245826Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Conclusion:**\n\nThese steps that I have provided, collectively perform exploratory data analysis (EDA) on the training and testing datasets.\n\nThey help in understanding the data, identifying patterns, and making informed decisions about feature engineering and modeling. The visualizations provide insights into the distribution of classes, relationships between variables, and key characteristics of the data.","metadata":{}}]}