{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":97984,"databundleVersionId":14096757,"sourceType":"competition"}],"dockerImageVersionId":31153,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# ⚕️ PhysioNet ECG Image Digitization - In-Depth EDA\n\n
\n

📊 Comprehensive Exploratory Data Analysis

\n

Converting Decades of ECG Images to Digital Time Series

\n
\n\n---\n\n## 🎯 Competition Overview\n\n**Challenge**: Extract time series data from ECG images to enable modern ML software to process billions of historical ECG recordings.\n\n**Why This Matters**: \n- 🏥 Billions of ECG images exist globally as paper printouts, scans, and photos\n- 🤖 Modern diagnostic software requires digital time series data\n- 💡 Converting these images unlocks decades of medical data for AI training\n- ❤️ Better models = improved cardiovascular diagnosis and treatment\n\n**Key Challenge**: Physical printouts, scanning, and photography introduce artifacts (rotations, blurring, stains, damage) that make digitization difficult.\n\n---\n\n## 📚 What We'll Explore\n\n1. **Dataset Structure** - Files, formats, and organization\n2. **Image Properties** - Dimensions, quality, and variants\n3. **Signal Analysis** - ECG waveforms and their characteristics\n4. **Lead Configuration** - Understanding the 12-lead ECG layout\n5. **Image Degradation** - Types and impact of artifacts\n6. **Modeling Insights** - Key findings for building solutions\n\nLet's dive in! 🚀","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport cv2\nfrom pathlib import Path\nfrom PIL import Image\nimport warnings\nfrom scipy import signal\nfrom collections import Counter\n\n!pip install --upgrade plotly\nimport plotly.express as px\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\n\nwarnings.filterwarnings('ignore')\n\n# Set style for beautiful plots\nplt.style.use('seaborn-v0_8-darkgrid')\nsns.set_palette(\"husl\")\nplt.rcParams['figure.figsize'] = (16, 8)\nplt.rcParams['font.size'] = 11\nplt.rcParams['axes.titlesize'] = 14\nplt.rcParams['axes.labelsize'] = 12\n\n# Define paths\nDATA_PATH = Path('/kaggle/input/physionet-ecg-image-digitization')\nTRAIN_PATH = DATA_PATH / 'train'\nTEST_PATH = DATA_PATH / 'test'\n\nprint(\"✅ Libraries imported successfully!\")\nprint(f\"📁 Data path: {DATA_PATH}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:51:57.368966Z","iopub.execute_input":"2025-11-04T12:51:57.369229Z","iopub.status.idle":"2025-11-04T12:52:23.137581Z","shell.execute_reply.started":"2025-11-04T12:51:57.369207Z","shell.execute_reply":"2025-11-04T12:52:23.136770Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Load training and test metadata\ntrain_df = pd.read_csv(DATA_PATH / 'train.csv')\ntest_df = pd.read_csv(DATA_PATH / 'test.csv')\n\nprint(f\"📊 Training samples: {len(train_df):,}\")\nprint(f\"📊 Test samples: {len(test_df['id'].unique()):,} unique IDs\")\nprint(f\"📊 Total test predictions needed: {len(test_df):,} (12 leads per ID)\")\n\n# Display first few rows\nprint(\"\\n\" + \"=\"*80)\nprint(\"TRAINING DATA SAMPLE\")\nprint(\"=\"*80)\ndisplay(train_df.head(10))\n\nprint(\"\\n\" + \"=\"*80)\nprint(\"TEST DATA SAMPLE\")\nprint(\"=\"*80)\ndisplay(test_df.head(15))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:23.138472Z","iopub.execute_input":"2025-11-04T12:52:23.138956Z","iopub.status.idle":"2025-11-04T12:52:23.197478Z","shell.execute_reply.started":"2025-11-04T12:52:23.138931Z","shell.execute_reply":"2025-11-04T12:52:23.196738Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 📈 1. Dataset Statistics\n\nUnderstanding the distribution of sampling frequencies, signal lengths, and data characteristics.\n","metadata":{}},{"cell_type":"code","source":"# Analyze sampling frequency distribution\nfs_counts = train_df['fs'].value_counts().sort_index()\n\nfig, axes = plt.subplots(1, 2, figsize=(18, 6))\n\n# Bar plot\ncolors = sns.color_palette(\"rocket\", len(fs_counts))\naxes[0].bar(fs_counts.index.astype(str), fs_counts.values, color=colors, edgecolor='black', linewidth=1.5)\naxes[0].set_xlabel('Sampling Frequency (Hz)', fontsize=13, fontweight='bold')\naxes[0].set_ylabel('Count', fontsize=13, fontweight='bold')\naxes[0].set_title('📊 Distribution of Sampling Frequencies', fontsize=15, fontweight='bold', pad=20)\naxes[0].grid(axis='y', alpha=0.3, linestyle='--')\n\n# Add value labels on bars\nfor i, (fs, count) in enumerate(zip(fs_counts.index, fs_counts.values)):\n axes[0].text(i, count + 10, f'{count}\\n({count/len(train_df)*100:.1f}%)', \n ha='center', va='bottom', fontweight='bold', fontsize=11)\n\n# Pie chart\ncolors_pie = sns.color_palette(\"Set2\", len(fs_counts))\nwedges, texts, autotexts = axes[1].pie(fs_counts.values, labels=[f'{fs} Hz' for fs in fs_counts.index], \n autopct='%1.1f%%', startangle=90, colors=colors_pie,\n explode=[0.05]*len(fs_counts), shadow=True)\naxes[1].set_title('📊 Sampling Frequency Proportions', fontsize=15, fontweight='bold', pad=20)\n\nfor autotext in autotexts:\n autotext.set_color('white')\n autotext.set_fontweight('bold')\n autotext.set_fontsize(12)\n\nplt.tight_layout()\nplt.show()\n\nprint(\"\\n\" + \"=\"*60)\nprint(\"SAMPLING FREQUENCY STATISTICS\")\nprint(\"=\"*60)\nfor fs, count in fs_counts.items():\n print(f\" {fs:4d} Hz: {count:3d} samples ({count/len(train_df)*100:5.2f}%)\")\nprint(\"=\"*60)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:23.199198Z","iopub.execute_input":"2025-11-04T12:52:23.199450Z","iopub.status.idle":"2025-11-04T12:52:23.729901Z","shell.execute_reply.started":"2025-11-04T12:52:23.199431Z","shell.execute_reply":"2025-11-04T12:52:23.728532Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Analyze signal lengths\nsig_len_stats = train_df.groupby('fs')['sig_len'].agg(['min', 'max', 'mean', 'std', 'count'])\n\nprint(\"\\n\" + \"=\"*80)\nprint(\"SIGNAL LENGTH STATISTICS BY SAMPLING FREQUENCY\")\nprint(\"=\"*80)\ndisplay(sig_len_stats)\n\n# Verify the relationship: sig_len = fs * 10 seconds\ntrain_df['calculated_sig_len'] = train_df['fs'] * 10\ntrain_df['sig_len_match'] = train_df['sig_len'] == train_df['calculated_sig_len']\n\nprint(f\"\\n✅ All signal lengths match fs × 10 seconds: {train_df['sig_len_match'].all()}\")\n\n# Visualize\nfig, ax = plt.subplots(figsize=(16, 6))\n\nfor fs in sorted(train_df['fs'].unique()):\n subset = train_df[train_df['fs'] == fs]\n ax.scatter([fs]*len(subset), subset['sig_len'], alpha=0.6, s=100, \n label=f'{fs} Hz (n={len(subset)})', edgecolors='black', linewidth=0.5)\n\nax.set_xlabel('Sampling Frequency (Hz)', fontsize=13, fontweight='bold')\nax.set_ylabel('Signal Length (samples)', fontsize=13, fontweight='bold')\nax.set_title('📏 Signal Length vs Sampling Frequency (All = fs × 10 seconds)', \n fontsize=15, fontweight='bold', pad=20)\nax.legend(fontsize=11, loc='best')\nax.grid(True, alpha=0.3, linestyle='--')\n\n# Add reference line\nfs_range = sorted(train_df['fs'].unique())\nsig_len_range = [fs * 10 for fs in fs_range]\nax.plot(fs_range, sig_len_range, 'r--', linewidth=2, label='Theoretical (fs × 10)', alpha=0.7)\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:23.730887Z","iopub.execute_input":"2025-11-04T12:52:23.731312Z","iopub.status.idle":"2025-11-04T12:52:24.188689Z","shell.execute_reply.started":"2025-11-04T12:52:23.731284Z","shell.execute_reply":"2025-11-04T12:52:24.187780Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 📊 2. Understanding ECG Structure\n\n## What is a 12-Lead ECG?\n\nAn ECG (Electrocardiogram) measures the electrical activity of the heart from 12 different perspectives:\n\n**Limb Leads (6 leads):**\n- **I, II, III**: Bipolar limb leads (measure voltage between two electrodes)\n- **aVR, aVL, aVF**: Augmented unipolar limb leads\n\n**Chest Leads (6 leads):**\n- **V1-V6**: Precordial leads placed across the chest\n\n## Standard Layout:\n- **Top 3 rows**: Each contains 4 leads (2.5 seconds each)\n- **Bottom row**: Lead II \"rhythm strip\" (10 seconds - full cardiac rhythm)\n\n## ECG Waveform Components:\n- **P wave**: Atrial depolarization\n- **QRS complex**: Ventricular depolarization (the heartbeat)\n- **T wave**: Ventricular repolarization\n- **Grid**: Time (horizontal) vs Voltage (vertical)\n - Standard: 1mm = 0.04s (time), 1mm = 0.1mV (voltage)","metadata":{}},{"cell_type":"code","source":"# Load a sample ECG for detailed analysis\nsample_id = str(train_df['id'].iloc[0])\nsample_fs = train_df[train_df['id'] == int(sample_id)]['fs'].values[0]\nsample_sig_len = train_df[train_df['id'] == int(sample_id)]['sig_len'].values[0]\n\n# Load time series data\necg_data = pd.read_csv(TRAIN_PATH / sample_id / f'{sample_id}.csv')\n\nprint(f\"📋 Sample ECG ID: {sample_id}\")\nprint(f\"📊 Sampling Frequency: {sample_fs} Hz\")\nprint(f\"📏 Signal Length: {sample_sig_len} samples ({sample_sig_len/sample_fs:.1f} seconds)\")\nprint(f\"📌 ECG Data Shape: {ecg_data.shape}\")\nprint(f\"\\n🔍 Available Leads: {list(ecg_data.columns)}\")\n\ndisplay(ecg_data.head())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:24.189864Z","iopub.execute_input":"2025-11-04T12:52:24.190148Z","iopub.status.idle":"2025-11-04T12:52:24.224293Z","shell.execute_reply.started":"2025-11-04T12:52:24.190127Z","shell.execute_reply":"2025-11-04T12:52:24.223514Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Create beautiful 12-lead ECG visualization\nlead_names = ['I', 'II', 'III', 'aVR', 'aVL', 'aVF', 'V1', 'V2', 'V3', 'V4', 'V5', 'V6']\ncolors = plt.cm.tab20(np.linspace(0, 1, 12))\n\nfig, axes = plt.subplots(12, 1, figsize=(20, 18), sharex=True)\nfig.suptitle(f'🫀 12-Lead ECG Waveform - Sample ID: {sample_id} (fs={sample_fs} Hz)', \n fontsize=18, fontweight='bold', y=0.995)\n\ntime_axis = np.arange(len(ecg_data)) / sample_fs\n\nfor idx, (ax, lead, color) in enumerate(zip(axes, lead_names, colors)):\n # Plot the waveform\n ax.plot(time_axis, ecg_data[lead], color=color, linewidth=1.5, alpha=0.9)\n ax.fill_between(time_axis, ecg_data[lead], alpha=0.2, color=color)\n \n # Styling\n ax.set_ylabel(lead, fontsize=13, fontweight='bold', rotation=0, ha='right', va='center')\n ax.grid(True, alpha=0.3, linestyle='--', linewidth=0.5)\n ax.set_xlim(0, len(ecg_data) / sample_fs)\n \n # Add horizontal line at zero\n ax.axhline(y=0, color='black', linestyle='-', linewidth=0.5, alpha=0.5)\n \n # Remove spines for cleaner look\n ax.spines['top'].set_visible(False)\n ax.spines['right'].set_visible(False)\n \n # Calculate and display basic stats\n mean_val = ecg_data[lead].mean()\n std_val = ecg_data[lead].std()\n ax.text(0.02, 0.95, f'μ={mean_val:.3f}, σ={std_val:.3f}', \n transform=ax.transAxes, fontsize=9, verticalalignment='top',\n bbox=dict(boxstyle='round', facecolor='wheat', alpha=0.5))\n\naxes[-1].set_xlabel('Time (seconds)', fontsize=14, fontweight='bold')\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:24.225099Z","iopub.execute_input":"2025-11-04T12:52:24.225321Z","iopub.status.idle":"2025-11-04T12:52:26.393838Z","shell.execute_reply.started":"2025-11-04T12:52:24.225303Z","shell.execute_reply":"2025-11-04T12:52:26.392831Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Demonstrate lead duration differences\n# In clinical ECGs: Lead II is 10 seconds, others are 2.5 seconds\n\nprint(\"=\" * 80)\nprint(\"LEAD DURATION IN CLINICAL ECG IMAGES\")\nprint(\"=\" * 80)\nprint(\"📏 Lead II (Rhythm Strip): 10.0 seconds ← Full cardiac rhythm\")\nprint(\"📏 Other 11 leads: 2.5 seconds ← Shorter synchronized segments\")\nprint(\"=\" * 80)\nprint(\"\\n💡 Note: In our CSV files, all leads show the full 10-second recording.\")\nprint(\" However, in the ECG IMAGES, only the first 2.5 seconds of leads I, III,\")\nprint(\" aVR, aVL, aVF, V1-V6 are displayed. Lead II shows all 10 seconds.\")\nprint(\"=\" * 80)\n\n# Visualize this concept\nfig, axes = plt.subplots(3, 1, figsize=(20, 10))\n\n# Full 10-second view\ntime_full = np.arange(len(ecg_data)) / sample_fs\naxes[0].plot(time_full, ecg_data['II'], color='crimson', linewidth=2, label='Lead II')\naxes[0].set_title('🔴 Lead II - Full 10-Second Rhythm Strip (as shown in ECG images)', \n fontsize=14, fontweight='bold', pad=15)\naxes[0].set_ylabel('Amplitude (mV)', fontsize=12, fontweight='bold')\naxes[0].legend(fontsize=12, loc='upper right')\naxes[0].grid(True, alpha=0.3)\naxes[0].set_xlim(0, 10)\n\n# First 2.5 seconds - other leads\ntime_25s = time_full[time_full <= 2.5]\naxes[1].plot(time_25s, ecg_data['I'][:len(time_25s)], color='blue', linewidth=2, label='Lead I')\naxes[1].plot(time_25s, ecg_data['V1'][:len(time_25s)], color='green', linewidth=2, label='Lead V1', alpha=0.7)\naxes[1].plot(time_25s, ecg_data['V5'][:len(time_25s)], color='orange', linewidth=2, label='Lead V5', alpha=0.7)\naxes[1].axvline(x=2.5, color='red', linestyle='--', linewidth=2, label='2.5s cutoff')\naxes[1].set_title('🔵 Other Leads - First 2.5 Seconds Only (as shown in ECG images)', \n fontsize=14, fontweight='bold', pad=15)\naxes[1].set_ylabel('Amplitude (mV)', fontsize=12, fontweight='bold')\naxes[1].legend(fontsize=12, loc='upper right')\naxes[1].grid(True, alpha=0.3)\naxes[1].set_xlim(0, 10)\n\n# Comparison\naxes[2].plot(time_full, ecg_data['II'], color='crimson', linewidth=2, alpha=0.8, label='Lead II (10s shown)')\naxes[2].plot(time_25s, ecg_data['I'][:len(time_25s)], color='blue', linewidth=2, label='Lead I (2.5s shown)')\naxes[2].axvspan(0, 2.5, alpha=0.2, color='blue', label='Other leads visible region')\naxes[2].axvspan(2.5, 10, alpha=0.1, color='gray', label='Lead II only region')\naxes[2].set_title('📊 Comparison: Lead II vs Other Leads Display Duration', \n fontsize=14, fontweight='bold', pad=15)\naxes[2].set_xlabel('Time (seconds)', fontsize=12, fontweight='bold')\naxes[2].set_ylabel('Amplitude (mV)', fontsize=12, fontweight='bold')\naxes[2].legend(fontsize=11, loc='upper right')\naxes[2].grid(True, alpha=0.3)\naxes[2].set_xlim(0, 10)\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:26.395022Z","iopub.execute_input":"2025-11-04T12:52:26.395289Z","iopub.status.idle":"2025-11-04T12:52:27.149778Z","shell.execute_reply.started":"2025-11-04T12:52:26.395268Z","shell.execute_reply":"2025-11-04T12:52:27.148538Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 🖼️ 3. Image Analysis\n\nNow let's examine the ECG images themselves - their properties, quality, and challenges.","metadata":{}},{"cell_type":"code","source":"# Analyze image properties across multiple samples\nsample_ids = train_df['id'].head(20).tolist()\nimage_properties = []\n\nfor sample_id in sample_ids:\n sample_dir = TRAIN_PATH / str(sample_id)\n img_path = sample_dir / f\"{sample_id}-0001.png\" # Original image\n \n if img_path.exists():\n img = Image.open(img_path)\n img_array = np.array(img)\n \n image_properties.append({\n 'id': sample_id,\n 'width': img.size[0],\n 'height': img.size[1],\n 'aspect_ratio': img.size[0] / img.size[1],\n 'mode': img.mode,\n 'channels': img_array.shape[2] if len(img_array.shape) == 3 else 1,\n 'dtype': str(img_array.dtype),\n 'file_size_kb': img_path.stat().st_size / 1024,\n 'mean_intensity': img_array.mean(),\n 'std_intensity': img_array.std()\n })\n\nprops_df = pd.DataFrame(image_properties)\n\nprint(\"=\" * 80)\nprint(\"IMAGE PROPERTIES ANALYSIS\")\nprint(\"=\" * 80)\ndisplay(props_df.head(10))\n\nprint(\"\\n\" + \"=\" * 80)\nprint(\"STATISTICAL SUMMARY\")\nprint(\"=\" * 80)\ndisplay(props_df.describe())\n\n# Visualize distributions\nfig, axes = plt.subplots(2, 3, figsize=(20, 12))\n\n# Width distribution\naxes[0, 0].hist(props_df['width'], bins=20, color='skyblue', edgecolor='black', alpha=0.7)\naxes[0, 0].set_title('📏 Image Width Distribution', fontsize=13, fontweight='bold')\naxes[0, 0].set_xlabel('Width (pixels)')\naxes[0, 0].set_ylabel('Frequency')\naxes[0, 0].grid(axis='y', alpha=0.3)\n\n# Height distribution\naxes[0, 1].hist(props_df['height'], bins=20, color='lightcoral', edgecolor='black', alpha=0.7)\naxes[0, 1].set_title('📏 Image Height Distribution', fontsize=13, fontweight='bold')\naxes[0, 1].set_xlabel('Height (pixels)')\naxes[0, 1].set_ylabel('Frequency')\naxes[0, 1].grid(axis='y', alpha=0.3)\n\n# Aspect ratio\naxes[0, 2].hist(props_df['aspect_ratio'], bins=20, color='lightgreen', edgecolor='black', alpha=0.7)\naxes[0, 2].set_title('📐 Aspect Ratio Distribution', fontsize=13, fontweight='bold')\naxes[0, 2].set_xlabel('Width / Height')\naxes[0, 2].set_ylabel('Frequency')\naxes[0, 2].grid(axis='y', alpha=0.3)\n\n# File size\naxes[1, 0].hist(props_df['file_size_kb'], bins=20, color='plum', edgecolor='black', alpha=0.7)\naxes[1, 0].set_title('💾 File Size Distribution', fontsize=13, fontweight='bold')\naxes[1, 0].set_xlabel('File Size (KB)')\naxes[1, 0].set_ylabel('Frequency')\naxes[1, 0].grid(axis='y', alpha=0.3)\n\n# Mean intensity\naxes[1, 1].hist(props_df['mean_intensity'], bins=20, color='gold', edgecolor='black', alpha=0.7)\naxes[1, 1].set_title('💡 Mean Pixel Intensity', fontsize=13, fontweight='bold')\naxes[1, 1].set_xlabel('Mean Intensity (0-255)')\naxes[1, 1].set_ylabel('Frequency')\naxes[1, 1].grid(axis='y', alpha=0.3)\n\n# Channels\nchannel_counts = props_df['channels'].value_counts()\naxes[1, 2].bar(channel_counts.index.astype(str), channel_counts.values, \n color='coral', edgecolor='black', alpha=0.7)\naxes[1, 2].set_title('🎨 Color Channels Distribution', fontsize=13, fontweight='bold')\naxes[1, 2].set_xlabel('Number of Channels')\naxes[1, 2].set_ylabel('Frequency')\naxes[1, 2].grid(axis='y', alpha=0.3)\n\nplt.tight_layout()\nplt.show()\n\nprint(f\"\\n✅ Most common dimensions: {props_df['width'].mode().values[0]:.0f} × {props_df['height'].mode().values[0]:.0f} pixels\")\nprint(f\"✅ Most common format: {props_df['mode'].mode().values[0]} ({props_df['channels'].mode().values[0]:.0f} channels)\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:27.150876Z","iopub.execute_input":"2025-11-04T12:52:27.151262Z","iopub.status.idle":"2025-11-04T12:52:32.120445Z","shell.execute_reply.started":"2025-11-04T12:52:27.151233Z","shell.execute_reply":"2025-11-04T12:52:32.119560Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Load and display a sample ECG image\nimg_path = TRAIN_PATH / str(sample_id) / f'{sample_id}-0001.png'\nimg = Image.open(img_path)\nimg_array = np.array(img)\n\nfig, axes = plt.subplots(1, 2, figsize=(22, 10))\n\n# Original image\naxes[0].imshow(img_array)\naxes[0].set_title(f'🖼️ Original ECG Image - ID: {sample_id}\\n(2200 × 1700 pixels, {img.mode})', \n fontsize=14, fontweight='bold', pad=15)\naxes[0].axis('off')\n\n# Add annotations\naxes[0].text(100, 100, '📊 ECG Grid', fontsize=12, color='red', \n bbox=dict(boxstyle='round', facecolor='yellow', alpha=0.7), fontweight='bold')\n\n# Grayscale version\nimg_gray = cv2.cvtColor(img_array[:,:,:3], cv2.COLOR_RGB2GRAY) if len(img_array.shape) == 3 else img_array\naxes[1].imshow(img_gray, cmap='gray')\naxes[1].set_title(f'⚫ Grayscale Version\\n(Useful for signal extraction)', \n fontsize=14, fontweight='bold', pad=15)\naxes[1].axis('off')\n\nplt.tight_layout()\nplt.show()\n\nprint(f\"📊 Image shape: {img_array.shape}\")\nprint(f\"📊 Data type: {img_array.dtype}\")\nprint(f\"📊 Value range: [{img_array.min()}, {img_array.max()}]\")\nprint(f\"📊 Mean intensity: {img_array.mean():.2f}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:32.123059Z","iopub.execute_input":"2025-11-04T12:52:32.123394Z","iopub.status.idle":"2025-11-04T12:52:33.774975Z","shell.execute_reply.started":"2025-11-04T12:52:32.123373Z","shell.execute_reply":"2025-11-04T12:52:33.774066Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 📸 4. Image Quality Variants\n\nThe dataset includes multiple versions of each ECG, simulating real-world degradation:\n\n| Code | Description | Challenge |\n|------|-------------|-----------|\n| **0001** | Original synthetic image | ✅ Clean baseline |\n| **0003** | Printed & scanned (color) | 🔸 Scanning artifacts |\n| **0004** | Printed & scanned (B&W) | 🔸 Loss of color info |\n| **0005** | Mobile photos (color print) | 🔶 Camera distortion, lighting |\n| **0006** | Mobile photos (screen) | 🔶 Moiré patterns, glare |\n| **0009** | Stained & soaked prints | 🔴 Physical damage, stains |\n| **0010** | Extensively damaged | 🔴 Severe degradation |\n| **0011** | Mold (color) | 🔴 Biological damage |\n| **0012** | Mold (B&W) | 🔴 Severe quality loss |\n\nLet's visualize these differences!","metadata":{}},{"cell_type":"code","source":"# Compare different image quality variants\nimage_variants = {\n '0001': 'Original Clean',\n '0003': 'Color Scanned',\n '0004': 'B&W Scanned',\n '0005': 'Mobile Photo',\n '0010': 'Extensively Damaged'\n}\n\nfig, axes = plt.subplots(len(image_variants), 2, figsize=(20, 5*len(image_variants)))\nfig.suptitle(f'🔍 Image Quality Comparison - Sample ID: {sample_id}', \n fontsize=18, fontweight='bold', y=0.995)\n\nfor idx, (variant_code, variant_name) in enumerate(image_variants.items()):\n img_path = TRAIN_PATH / str(sample_id) / f'{sample_id}-{variant_code}.png'\n \n if img_path.exists():\n img = cv2.imread(str(img_path))\n img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n \n # RGB version\n axes[idx, 0].imshow(img_rgb)\n axes[idx, 0].set_title(f'{variant_name} - Color', fontsize=13, fontweight='bold')\n axes[idx, 0].axis('off')\n \n # Intensity histogram\n axes[idx, 1].hist(img_gray.flatten(), bins=50, color='steelblue', \n edgecolor='black', alpha=0.7)\n axes[idx, 1].set_title(f'{variant_name} - Intensity Distribution', \n fontsize=13, fontweight='bold')\n axes[idx, 1].set_xlabel('Pixel Intensity (0-255)')\n axes[idx, 1].set_ylabel('Frequency')\n axes[idx, 1].grid(axis='y', alpha=0.3)\n \n # Add statistics\n mean_int = img_gray.mean()\n std_int = img_gray.std()\n axes[idx, 1].axvline(mean_int, color='red', linestyle='--', linewidth=2, \n label=f'Mean: {mean_int:.1f}')\n axes[idx, 1].legend()\n else:\n axes[idx, 0].text(0.5, 0.5, 'Image not found', ha='center', va='center', fontsize=14)\n axes[idx, 0].axis('off')\n axes[idx, 1].text(0.5, 0.5, 'Image not found', ha='center', va='center', fontsize=14)\n axes[idx, 1].axis('off')\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:33.776111Z","iopub.execute_input":"2025-11-04T12:52:33.776355Z","iopub.status.idle":"2025-11-04T12:52:42.464713Z","shell.execute_reply.started":"2025-11-04T12:52:33.776336Z","shell.execute_reply":"2025-11-04T12:52:42.463544Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Zoom into a specific region to see degradation details\ncrop_height = 400\ncrop_width = 600\nstart_y = 600\nstart_x = 800\n\nvariants_to_compare = ['0001', '0003', '0005', '0010']\nvariant_names_short = ['Original', 'Scanned', 'Photo', 'Damaged']\n\nfig, axes = plt.subplots(2, len(variants_to_compare), figsize=(24, 12))\nfig.suptitle('🔬 Zoomed Detail Comparison - Signal Quality Analysis', \n fontsize=16, fontweight='bold', y=0.995)\n\nfor idx, (variant_code, variant_name) in enumerate(zip(variants_to_compare, variant_names_short)):\n img_path = TRAIN_PATH / str(sample_id) / f'{sample_id}-{variant_code}.png'\n \n if img_path.exists():\n img = cv2.imread(str(img_path))\n img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n \n # Crop the region\n cropped_rgb = img_rgb[start_y:start_y+crop_height, start_x:start_x+crop_width]\n cropped_gray = img_gray[start_y:start_y+crop_height, start_x:start_x+crop_width]\n \n # Show full image with crop location\n axes[0, idx].imshow(img_gray, cmap='gray')\n rect = plt.Rectangle((start_x, start_y), crop_width, crop_height, \n fill=False, edgecolor='red', linewidth=3)\n axes[0, idx].add_patch(rect)\n axes[0, idx].set_title(f'{variant_name}\\nFull Image', fontsize=12, fontweight='bold')\n axes[0, idx].axis('off')\n \n # Show cropped detail\n axes[1, idx].imshow(cropped_gray, cmap='gray')\n axes[1, idx].set_title(f'{variant_name}\\nZoomed Detail', fontsize=12, fontweight='bold')\n axes[1, idx].axis('off')\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:42.465945Z","iopub.execute_input":"2025-11-04T12:52:42.466491Z","iopub.status.idle":"2025-11-04T12:52:47.160308Z","shell.execute_reply.started":"2025-11-04T12:52:42.466453Z","shell.execute_reply":"2025-11-04T12:52:47.159169Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 📐 5. Grid Structure Analysis\n\nECG images contain a calibrated grid that defines:\n- **Horizontal axis**: Time (standard: 1mm = 0.04 seconds)\n- **Vertical axis**: Voltage (standard: 1mm = 0.1 mV)\n\nUnderstanding this grid is crucial for accurate signal extraction!","metadata":{}},{"cell_type":"code","source":"# Detect and visualize grid structure\nimg_path = TRAIN_PATH / str(sample_id) / f'{sample_id}-0001.png'\nimg = cv2.imread(str(img_path))\nimg_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\nimg_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n\n# Edge detection\nedges = cv2.Canny(img_gray, 30, 100)\n\n# Detect lines using Hough Transform\nlines = cv2.HoughLinesP(edges, 1, np.pi/180, threshold=100, minLineLength=100, maxLineGap=10)\n\n# Separate horizontal and vertical lines\nhorizontal_lines = []\nvertical_lines = []\n\nimg_with_lines = img_rgb.copy()\n\nif lines is not None:\n for line in lines:\n x1, y1, x2, y2 = line[0]\n angle = np.abs(np.arctan2(y2 - y1, x2 - x1) * 180 / np.pi)\n \n if angle < 10 or angle > 170: # Horizontal\n horizontal_lines.append(line)\n cv2.line(img_with_lines, (x1, y1), (x2, y2), (255, 0, 0), 2)\n elif 80 < angle < 100: # Vertical\n vertical_lines.append(line)\n cv2.line(img_with_lines, (x1, y1), (x2, y2), (0, 255, 0), 2)\n\nprint(\"=\" * 80)\nprint(\"GRID STRUCTURE ANALYSIS\")\nprint(\"=\" * 80)\nprint(f\"✅ Total lines detected: {len(lines) if lines is not None else 0}\")\nprint(f\"📊 Horizontal lines: {len(horizontal_lines)} (shown in RED)\")\nprint(f\"📊 Vertical lines: {len(vertical_lines)} (shown in GREEN)\")\nprint(\"=\" * 80)\n\n# Visualize\nfig, axes = plt.subplots(1, 3, figsize=(24, 8))\n\naxes[0].imshow(img_gray, cmap='gray')\naxes[0].set_title('📄 Original Grayscale Image', fontsize=14, fontweight='bold')\naxes[0].axis('off')\n\naxes[1].imshow(edges, cmap='gray')\naxes[1].set_title('🔍 Edge Detection (Canny)', fontsize=14, fontweight='bold')\naxes[1].axis('off')\n\naxes[2].imshow(img_with_lines)\naxes[2].set_title(f'📐 Detected Grid Lines\\n🔴 Horizontal ({len(horizontal_lines)}) | 🟢 Vertical ({len(vertical_lines)})', \n fontsize=14, fontweight='bold')\naxes[2].axis('off')\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:47.161310Z","iopub.execute_input":"2025-11-04T12:52:47.161585Z","iopub.status.idle":"2025-11-04T12:52:49.753700Z","shell.execute_reply.started":"2025-11-04T12:52:47.161563Z","shell.execute_reply":"2025-11-04T12:52:49.752709Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Analyze horizontal and vertical projections to understand structure\nhorizontal_proj = np.sum(img_gray, axis=1)\nvertical_proj = np.sum(img_gray, axis=0)\n\nfig = plt.figure(figsize=(20, 12))\ngs = fig.add_gridspec(3, 3, hspace=0.3, wspace=0.3)\n\n# Main image\nax_main = fig.add_subplot(gs[1:, :2])\nax_main.imshow(img_gray, cmap='gray', aspect='auto')\nax_main.set_title('🖼️ ECG Image', fontsize=14, fontweight='bold')\nax_main.set_xlabel('Column Index (Width)', fontsize=12)\nax_main.set_ylabel('Row Index (Height)', fontsize=12)\n\n# Horizontal projection (top)\nax_top = fig.add_subplot(gs[0, :2], sharex=ax_main)\nax_top.plot(vertical_proj, color='blue', linewidth=2)\nax_top.fill_between(range(len(vertical_proj)), vertical_proj, alpha=0.3, color='blue')\nax_top.set_title('📊 Vertical Projection (Sum across rows)', fontsize=12, fontweight='bold')\nax_top.set_ylabel('Intensity Sum', fontsize=10)\nax_top.grid(True, alpha=0.3)\nax_top.tick_params(labelbottom=False)\n\n# Vertical projection (right)\nax_right = fig.add_subplot(gs[1:, 2], sharey=ax_main)\nax_right.plot(horizontal_proj, range(len(horizontal_proj)), color='red', linewidth=2)\nax_right.fill_betweenx(range(len(horizontal_proj)), horizontal_proj, alpha=0.3, color='red')\nax_right.set_title('📊 Horizontal\\nProjection\\n(Sum across columns)', fontsize=11, fontweight='bold')\nax_right.set_xlabel('Intensity Sum', fontsize=10)\nax_right.grid(True, alpha=0.3)\nax_right.tick_params(labelleft=False)\nax_right.invert_xaxis()\nax_right.invert_yaxis()\n\nplt.suptitle('📐 Image Projection Analysis - Understanding ECG Layout', \n fontsize=16, fontweight='bold', y=0.995)\nplt.show()\n\nprint(\"\\n💡 Interpretation:\")\nprint(\" - Vertical projection shows the overall horizontal distribution of signals\")\nprint(\" - Horizontal projection reveals the vertical layout (rows of leads)\")\nprint(\" - Peaks and valleys help identify lead boundaries and grid structure\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:49.754845Z","iopub.execute_input":"2025-11-04T12:52:49.755409Z","iopub.status.idle":"2025-11-04T12:52:50.798012Z","shell.execute_reply.started":"2025-11-04T12:52:49.755380Z","shell.execute_reply":"2025-11-04T12:52:50.797059Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 📊 6. Signal Characteristics\n\nLet's analyze the actual ECG signal properties and their relationships.","metadata":{}},{"cell_type":"code","source":"# Comprehensive signal statistics\nlead_stats = []\n\nfor lead in lead_names:\n lead_data = ecg_data[lead].values\n \n lead_stats.append({\n 'Lead': lead,\n 'Mean (mV)': lead_data.mean(),\n 'Std Dev (mV)': lead_data.std(),\n 'Min (mV)': lead_data.min(),\n 'Max (mV)': lead_data.max(),\n 'Range (mV)': lead_data.max() - lead_data.min(),\n 'Median (mV)': np.median(lead_data),\n 'RMS': np.sqrt(np.mean(lead_data**2))\n })\n\nstats_df = pd.DataFrame(lead_stats)\n\nprint(\"=\" * 100)\nprint(\"SIGNAL STATISTICS BY LEAD\")\nprint(\"=\" * 100)\ndisplay(stats_df.style.background_gradient(cmap='RdYlGn', subset=['Range (mV)', 'Std Dev (mV)']))\n\n# Visualize distributions\nfig, axes = plt.subplots(2, 2, figsize=(20, 12))\n\n# Mean values\naxes[0, 0].barh(stats_df['Lead'], stats_df['Mean (mV)'], color='skyblue', edgecolor='black')\naxes[0, 0].set_title('📊 Mean Amplitude by Lead', fontsize=14, fontweight='bold')\naxes[0, 0].set_xlabel('Mean (mV)', fontsize=12)\naxes[0, 0].axvline(x=0, color='red', linestyle='--', linewidth=1)\naxes[0, 0].grid(axis='x', alpha=0.3)\n\n# Standard deviation\naxes[0, 1].barh(stats_df['Lead'], stats_df['Std Dev (mV)'], color='lightcoral', edgecolor='black')\naxes[0, 1].set_title('📊 Signal Variability (Std Dev) by Lead', fontsize=14, fontweight='bold')\naxes[0, 1].set_xlabel('Standard Deviation (mV)', fontsize=12)\naxes[0, 1].grid(axis='x', alpha=0.3)\n\n# Range\naxes[1, 0].barh(stats_df['Lead'], stats_df['Range (mV)'], color='lightgreen', edgecolor='black')\naxes[1, 0].set_title('📊 Signal Range (Max - Min) by Lead', fontsize=14, fontweight='bold')\naxes[1, 0].set_xlabel('Range (mV)', fontsize=12)\naxes[1, 0].grid(axis='x', alpha=0.3)\n\n# RMS\naxes[1, 1].barh(stats_df['Lead'], stats_df['RMS'], color='plum', edgecolor='black')\naxes[1, 1].set_title('📊 RMS (Root Mean Square) by Lead', fontsize=14, fontweight='bold')\naxes[1, 1].set_xlabel('RMS', fontsize=12)\naxes[1, 1].grid(axis='x', alpha=0.3)\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:50.798940Z","iopub.execute_input":"2025-11-04T12:52:50.799240Z","iopub.status.idle":"2025-11-04T12:52:51.558648Z","shell.execute_reply.started":"2025-11-04T12:52:50.799219Z","shell.execute_reply":"2025-11-04T12:52:51.557766Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Lead correlation analysis\ncorrelation_matrix = ecg_data[lead_names].corr()\n\nfig, ax = plt.subplots(figsize=(14, 12))\n\n# Create heatmap\nsns.heatmap(correlation_matrix, annot=True, fmt='.2f', cmap='coolwarm', \n center=0, square=True, linewidths=1, cbar_kws={\"shrink\": 0.8},\n vmin=-1, vmax=1, ax=ax)\n\nax.set_title('🔗 Inter-Lead Correlation Matrix\\n(Understanding relationships between ECG leads)', \n fontsize=16, fontweight='bold', pad=20)\n\nplt.tight_layout()\nplt.show()\n\nprint(\"\\n💡 Key Observations:\")\nprint(\" - Some leads show strong positive correlations (similar waveforms)\")\nprint(\" - Some leads show negative correlations (inverted waveforms)\")\nprint(\" - These relationships are based on cardiac electrical vectors\")\nprint(\" - aVR typically shows inverse patterns compared to other leads\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:51.559569Z","iopub.execute_input":"2025-11-04T12:52:51.560175Z","iopub.status.idle":"2025-11-04T12:52:52.240527Z","shell.execute_reply.started":"2025-11-04T12:52:51.560152Z","shell.execute_reply":"2025-11-04T12:52:52.239667Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Analyze frequency content of ECG signals\nfrom scipy.fft import fft, fftfreq\n\n# Select a few representative leads\nselected_leads = ['II', 'V1', 'V5']\ncolors_freq = ['crimson', 'blue', 'green']\n\nfig, axes = plt.subplots(len(selected_leads), 2, figsize=(20, 12))\nfig.suptitle('🌊 Time vs Frequency Domain Analysis', fontsize=16, fontweight='bold', y=0.995)\n\nfor idx, (lead, color) in enumerate(zip(selected_leads, colors_freq)):\n signal_data = ecg_data[lead].values\n n = len(signal_data)\n \n # Time domain\n time = np.arange(n) / sample_fs\n axes[idx, 0].plot(time, signal_data, color=color, linewidth=1.5)\n axes[idx, 0].set_title(f'Lead {lead} - Time Domain', fontsize=13, fontweight='bold')\n axes[idx, 0].set_xlabel('Time (s)', fontsize=11)\n axes[idx, 0].set_ylabel('Amplitude (mV)', fontsize=11)\n axes[idx, 0].grid(True, alpha=0.3)\n \n # Frequency domain (FFT)\n yf = fft(signal_data)\n xf = fftfreq(n, 1/sample_fs)\n \n # Only positive frequencies\n positive_freqs = xf > 0\n axes[idx, 1].plot(xf[positive_freqs], np.abs(yf[positive_freqs]), color=color, linewidth=1.5)\n axes[idx, 1].set_title(f'Lead {lead} - Frequency Domain', fontsize=13, fontweight='bold')\n axes[idx, 1].set_xlabel('Frequency (Hz)', fontsize=11)\n axes[idx, 1].set_ylabel('Magnitude', fontsize=11)\n axes[idx, 1].set_xlim(0, 50) # Focus on physiologically relevant frequencies\n axes[idx, 1].grid(True, alpha=0.3)\n\nplt.tight_layout()\nplt.show()\n\nprint(\"\\n💡 Frequency Domain Insights:\")\nprint(\" - ECG signals contain frequencies mainly < 50 Hz\")\nprint(\" - Dominant frequencies correspond to heart rate (1-3 Hz)\")\nprint(\" - Higher frequencies capture fine details of QRS complex\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:52.241574Z","iopub.execute_input":"2025-11-04T12:52:52.241918Z","iopub.status.idle":"2025-11-04T12:52:53.352593Z","shell.execute_reply.started":"2025-11-04T12:52:52.241888Z","shell.execute_reply":"2025-11-04T12:52:53.351582Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 🎯 7. Test Data Analysis\n\nUnderstanding what we need to predict.","metadata":{}},{"cell_type":"code","source":"# Analyze test data structure\nprint(\"=\" * 80)\nprint(\"TEST DATA STRUCTURE\")\nprint(\"=\" * 80)\n\ntest_summary = test_df.groupby('id').agg({\n 'lead': 'count',\n 'fs': 'first',\n 'number_of_rows': 'sum'\n}).reset_index()\ntest_summary.columns = ['id', 'num_leads', 'fs', 'total_rows_to_predict']\n\nprint(f\"\\n📊 Total unique test IDs: {len(test_summary)}\")\nprint(f\"📊 Total predictions required: {test_df.shape[0]:,} rows\")\nprint(f\"\\n{test_summary.to_string(index=False)}\")\n\n# Analyze predictions per lead\nlead_predictions = test_df.groupby('lead')['number_of_rows'].agg(['min', 'max', 'mean', 'count'])\nprint(\"\\n\" + \"=\" * 80)\nprint(\"PREDICTIONS REQUIRED PER LEAD\")\nprint(\"=\" * 80)\nprint(f\"\\n{lead_predictions.to_string()}\")\n\n# Visualize\nfig, axes = plt.subplots(1, 2, figsize=(20, 6))\n\n# Predictions per lead\nlead_counts = test_df.groupby('lead')['number_of_rows'].mean()\ncolors_bar = plt.cm.tab10(np.arange(len(lead_counts)))\n\naxes[0].bar(lead_counts.index, lead_counts.values, color=colors_bar, edgecolor='black', linewidth=1.5)\naxes[0].set_title('📊 Average Predictions Required per Lead', fontsize=14, fontweight='bold')\naxes[0].set_xlabel('Lead', fontsize=12)\naxes[0].set_ylabel('Number of Rows to Predict', fontsize=12)\naxes[0].grid(axis='y', alpha=0.3)\naxes[0].tick_params(axis='x', rotation=0)\n\n# Add value labels\nfor i, (lead, count) in enumerate(zip(lead_counts.index, lead_counts.values)):\n axes[0].text(i, count + 50, f'{count:.0f}', ha='center', va='bottom', fontweight='bold')\n\n# Sampling frequency distribution in test\ntest_fs_dist = test_summary['fs'].value_counts().sort_index()\naxes[1].bar(test_fs_dist.index.astype(str), test_fs_dist.values, \n color='steelblue', edgecolor='black', linewidth=1.5)\naxes[1].set_title('📊 Test Data Sampling Frequency Distribution', fontsize=14, fontweight='bold')\naxes[1].set_xlabel('Sampling Frequency (Hz)', fontsize=12)\naxes[1].set_ylabel('Count', fontsize=12)\naxes[1].grid(axis='y', alpha=0.3)\n\n# Add value labels\nfor i, (fs, count) in enumerate(zip(test_fs_dist.index, test_fs_dist.values)):\n axes[1].text(i, count + 0.05, f'{count}', ha='center', va='bottom', fontweight='bold')\n\nplt.tight_layout()\nplt.show()\n\nprint(\"\\n💡 Key Points:\")\nprint(\" - Lead II requires ~2.5× more predictions (10s vs 2.5s)\")\nprint(\" - All 12 leads must be predicted for each test image\")\nprint(\" - Output format: {base_id}_{row_id}_{lead}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:53.353734Z","iopub.execute_input":"2025-11-04T12:52:53.354144Z","iopub.status.idle":"2025-11-04T12:52:53.801677Z","shell.execute_reply.started":"2025-11-04T12:52:53.354119Z","shell.execute_reply":"2025-11-04T12:52:53.800829Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Display a sample test image\ntest_image_files = list(TEST_PATH.glob('*.png'))\nif test_image_files:\n sample_test_img_path = test_image_files[0]\n sample_test_id = sample_test_img_path.stem\n \n test_img = Image.open(sample_test_img_path)\n test_img_array = np.array(test_img)\n \n fig, axes = plt.subplots(1, 2, figsize=(22, 10))\n \n axes[0].imshow(test_img_array)\n axes[0].set_title(f'🎯 Sample Test Image\\nID: {sample_test_id}', \n fontsize=14, fontweight='bold', pad=15)\n axes[0].axis('off')\n \n # Grayscale\n if len(test_img_array.shape) == 3:\n test_img_gray = cv2.cvtColor(test_img_array[:,:,:3], cv2.COLOR_RGB2GRAY)\n else:\n test_img_gray = test_img_array\n \n axes[1].imshow(test_img_gray, cmap='gray')\n axes[1].set_title(f'⚫ Grayscale Version', fontsize=14, fontweight='bold', pad=15)\n axes[1].axis('off')\n \n plt.tight_layout()\n plt.show()\n \n print(f\"📊 Test Image ID: {sample_test_id}\")\n print(f\"📊 Image Shape: {test_img_array.shape}\")\n print(f\"📊 Image Size: {test_img.size}\")\nelse:\n print(\"⚠️ No test images found in the test directory\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:53.802706Z","iopub.execute_input":"2025-11-04T12:52:53.803288Z","iopub.status.idle":"2025-11-04T12:52:55.409467Z","shell.execute_reply.started":"2025-11-04T12:52:53.803259Z","shell.execute_reply":"2025-11-04T12:52:55.408299Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# 💡 8. Key Insights & Modeling Recommendations\n\n## 🎯 Main Challenges Identified:\n\n### 1. **Image Variability**\n- ✅ Consistent dimensions (2200×1700) - Good for modeling!\n- ⚠️ Multiple quality levels (clean → severely damaged)\n- ⚠️ Color vs B&W, scanned vs photographed\n\n### 2. **Signal Extraction**\n- 📐 Grid structure must be detected and calibrated\n- 🎯 12 different lead regions must be identified\n- 📏 Lead II: 10 seconds, Others: 2.5 seconds\n- 🔄 Variable sampling rates (250/500/1000 Hz)\n\n### 3. **Quality Degradation**\n- Stains, mold, physical damage\n- Scanning artifacts, misalignments\n- Mobile photo distortions (lighting, angle, blur)\n\n---\n\n## 🛠️ Recommended Modeling Approach:\n\n### **Pipeline Components:**\n\n1. **Image Preprocessing**\n - Grid detection and alignment\n - Rotation/skew correction\n - Contrast enhancement\n - Denoising for damaged images\n\n2. **Lead Segmentation**\n - Identify 12 lead regions\n - Handle different lead durations\n - Account for standard ECG layout\n\n3. **Signal Extraction**\n - Pixel-to-voltage conversion\n - Time axis calibration\n - Waveform smoothing\n\n4. **Model Architecture Options**\n - **U-Net / Segmentation**: Identify signal pixels\n - **CNN + LSTM**: Image → Time series\n - **Transformer**: Attention-based extraction\n - **Ensemble**: Combine multiple approaches\n\n5. **Post-Processing**\n - Temporal alignment\n - Baseline correction\n - Interpolation to match sampling rate\n\n---\n\n## 📊 Success Metrics:\n\n- **Modified SNR**: Accounts for small alignment errors\n- Higher score = better reconstruction\n- Focus on signal fidelity over perfect alignment\n\n---\n\n## 🚀 Next Steps:\n\n1. Build baseline model (start simple!)\n2. Implement robust preprocessing\n3. Handle multiple image quality levels\n4. Optimize for SNR metric\n5. Ensemble different approaches\n\nGood luck! 🍀","metadata":{}},{"cell_type":"code","source":"# Final comprehensive summary\nprint(\"=\" * 100)\nprint(\"🎯 COMPREHENSIVE DATASET SUMMARY\")\nprint(\"=\" * 100)\n\nsummary_stats = {\n 'Category': [\n 'Training Samples',\n 'Test Images',\n 'Total Test Predictions',\n 'ECG Leads',\n 'Sampling Frequencies',\n 'Image Dimensions (typical)',\n 'Image Variants per Sample',\n 'Lead II Duration',\n 'Other Leads Duration',\n 'Evaluation Metric'\n ],\n 'Value': [\n f'{len(train_df):,}',\n f'{len(test_df[\"id\"].unique()):,}',\n f'{len(test_df):,}',\n '12 (I, II, III, aVR, aVL, aVF, V1-V6)',\n '250, 500, 1000 Hz',\n '2200 × 1700 pixels',\n '9 quality variants (0001-0012)',\n '10 seconds (full rhythm)',\n '2.5 seconds',\n 'Modified SNR (dB)'\n ]\n}\n\nsummary_df = pd.DataFrame(summary_stats)\ndisplay(summary_df.style.set_properties(**{\n 'text-align': 'left',\n 'font-size': '12pt',\n}).hide(axis='index'))\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-04T12:52:55.410509Z","iopub.execute_input":"2025-11-04T12:52:55.410774Z","iopub.status.idle":"2025-11-04T12:52:55.422852Z","shell.execute_reply.started":"2025-11-04T12:52:55.410751Z","shell.execute_reply":"2025-11-04T12:52:55.421979Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n
\n

🙏 Thank You!

\n

If you found this EDA helpful, please:

\n

⭐ Upvote this notebook!

\n

Your support helps others discover useful content.

\n
\n

💬 Questions or suggestions? Leave a comment!

\n

🤝 Let's collaborate and learn together!

\n
\n\n---\n\n\n## 🔗 Connect & Collaborate:\n\nGood luck with your models! Let's digitize those ECGs!","metadata":{}}]}