{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":97984,"databundleVersionId":14096757,"sourceType":"competition"}],"dockerImageVersionId":31153,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## 1. Load and Explore Metadata Files (train.csv, test.csv, sample_submission.parquet)\nFirst, we load the metadata files using pandas and inspect their contents. The training and test CSV files contain metadata for each ECG record, and the sample submission file shows the required output format for the competition. We will examine the number of records and the columns provided in each:","metadata":{}},{"cell_type":"code","source":"import pandas as pd\n\n# Load metadata files\ntrain_df = pd.read_csv(\"/kaggle/input/physionet-ecg-image-digitization/train.csv\")\ntest_df = pd.read_csv(\"/kaggle/input/physionet-ecg-image-digitization/test.csv\")\nsubmission_df = pd.read_parquet(\"/kaggle/input/physionet-ecg-image-digitization/sample_submission.parquet\")\n\n# Print basic information\nprint(f\"Training records: {len(train_df)}\")\nprint(f\"Test records: {len(test_df)}\")\nprint(\"Train columns:\", train_df.columns.tolist())\nprint(\"Test columns:\", test_df.columns.tolist())\nprint(\"Sample submission columns:\", submission_df.columns.tolist())\n\n# Peek at the first few rows of each\nprint(\"\\nTrain.csv sample:\")\nprint(train_df.head(3))  # first 3 rows of train metadata\nprint(\"\\nTest.csv sample:\")\nprint(test_df.head(3))\nprint(\"\\nSample_submission.parquet sample:\")\nprint(submission_df.head(5))","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-11-14T10:38:31.666300Z","iopub.execute_input":"2025-11-14T10:38:31.666670Z","iopub.status.idle":"2025-11-14T10:38:34.280642Z","shell.execute_reply.started":"2025-11-14T10:38:31.666635Z","shell.execute_reply":"2025-11-14T10:38:34.279273Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2. Load and Visualize Sample ECG Time-Series (12-Lead Signals)\n\nNow, let’s load a few example ECG time-series from the training set and plot their 12-lead signals. Each training ECG has a corresponding CSV file (train/<id>/<id>.csv) containing the raw waveform data. This CSV has 12 columns, one for each ECG lead (typically labeled I, II, III, aVR, aVL, aVF, V1, V2, V3, V4, V5, V6 – the standard 12 leads ). We will:\n\t•\tSelect a few sample record IDs from train_df.\n\t•\tFor each example ID, read its CSV file into a DataFrame.\n\t•\tPrint the sampling frequency and signal length from the metadata (to know the time span of the recording).\n\t•\tPlot all 12 lead signals using matplotlib, arranging subplots for clarity.","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# Select a few example ECG IDs from the training set (e.g., first 2 records)\nexample_ids = train_df['id'].iloc[:2].tolist()\n\nfor ecg_id in example_ids:\n    # Load the raw time-series data for this ECG ID\n    filepath = f\"/kaggle/input/physionet-ecg-image-digitization/train/{ecg_id}/{ecg_id}.csv\"\n    ecg_signal = pd.read_csv(filepath)\n    \n    # Get sampling frequency (fs) and signal length from metadata\n    fs = train_df.loc[train_df['id'] == ecg_id, 'fs'].iloc[0]\n    sig_len = train_df.loc[train_df['id'] == ecg_id, 'sig_len'].iloc[0]\n    duration = sig_len / fs if fs else 0\n    print(f\"\\nECG ID: {ecg_id} -> Sampling frequency = {fs} Hz, Signal length = {sig_len} samples (~{duration:.1f} sec)\")\n    print(\"Lead columns:\", list(ecg_signal.columns))\n    \n    # Plot the 12-lead ECG signals\n    fig, axes = plt.subplots(nrows=6, ncols=2, figsize=(12, 8))\n    axes = axes.ravel()  # flatten the 6x2 grid to a 1D array for easy iteration\n    for i, lead in enumerate(ecg_signal.columns):\n        axes[i].plot(ecg_signal[lead], color='black')\n        axes[i].set_title(lead)\n        axes[i].set_xlabel(\"Sample index\")\n        axes[i].set_ylabel(\"Amplitude\")\n        # Optional: add grid or adjust y-axis for clarity\n        axes[i].grid(True, which='both', linestyle='--', linewidth=0.5)\n    plt.tight_layout()\n    plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-14T10:38:34.281383Z","iopub.execute_input":"2025-11-14T10:38:34.281691Z","iopub.status.idle":"2025-11-14T10:38:38.364735Z","shell.execute_reply.started":"2025-11-14T10:38:34.281665Z","shell.execute_reply":"2025-11-14T10:38:38.363479Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **3. Load and Visualize Sample ECG Images**\n\nIn addition to numeric signals, the dataset provides **ECG images** for each record. These are scans or renderings of the ECG paper printouts. Each training record’s folder contains one or more PNG images of the ECG (multiple images if the ECG spans more than one page) , and each test record has at least one ECG image (since our task is to digitize these images). We can use Matplotlib to load and display these images to understand how the ECGs look in image form.\n\n*Example of a 12-lead ECG image from the dataset (synthetic data). The 12 leads (I, II, III, aVR, aVL, aVF, V1–V6) are arranged on a standard grid. Time runs along the horizontal axis and voltage along the vertical axis; each small grid box typically represents 40 ms (horizontal) and 0.1 mV (vertical). The printout includes metadata (patient details, recording info) at the top, which in this example is synthetically generated or anonymized. This image illustrates the kind of input (paper ECG format) that needs to be converted back to digital signals.*\n\nIn code, let’s read and display one training ECG image and one test ECG image:","metadata":{}},{"cell_type":"code","source":"# Choose an example train ID and test ID to visualize images\ntrain_example_id = example_ids[0]            # use one of the earlier example IDs\ntest_example_id = test_df['id'].iloc[0]      # first test ID (for example)\n\n# Load and display the first page of the train ECG image\ntrain_img_path = f\"/kaggle/input/physionet-ecg-image-digitization/train/{train_example_id}/{train_example_id}-0001.png\"\ntrain_img = plt.imread(train_img_path)\nplt.figure(figsize=(6, 4))\nplt.imshow(train_img, cmap='gray')\nplt.title(f\"Train ECG Image - ID: {train_example_id} (page 1)\")\nplt.axis('off')  # hide axis ticks\nplt.show()\n\n# Load and display the test ECG image\ntest_img_path = f\"/kaggle/input/physionet-ecg-image-digitization/test/{test_example_id}.png\"\ntest_img = plt.imread(test_img_path)\nplt.figure(figsize=(6, 4))\nplt.imshow(test_img, cmap='gray')\nplt.title(f\"Test ECG Image - ID: {test_example_id}\")\nplt.axis('off')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-14T10:38:38.367225Z","iopub.execute_input":"2025-11-14T10:38:38.367652Z","iopub.status.idle":"2025-11-14T10:38:40.354310Z","shell.execute_reply.started":"2025-11-14T10:38:38.367616Z","shell.execute_reply":"2025-11-14T10:38:40.353218Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **4. Visualize 12-Lead Signals in a Stacked (ECG Paper) Layout**\n\nThe 6x2 plot is good for inspecting individual leads, but it's not how they are presented on the ECG paper. A stacked plot, where each lead is drawn one above the other in a single column, provides a much better visual comparison to the ECG images. This helps in spotting artifacts or patterns across all leads simultaneously, just as one would on a clinical printout.","metadata":{}},{"cell_type":"code","source":"import numpy as np\n\n\n# Use the first example ID again\necg_id_stacked = example_ids[0]\nfilepath_stacked = f\"/kaggle/input/physionet-ecg-image-digitization/train/{ecg_id_stacked}/{ecg_id_stacked}.csv\"\necg_signal_stacked = pd.read_csv(filepath_stacked)\n\n# Get metadata\nfs_stacked = train_df.loc[train_df['id'] == ecg_id_stacked, 'fs'].iloc[0].item()\nsig_len_stacked = train_df.loc[train_df['id'] == ecg_id_stacked, 'sig_len'].iloc[0].item()\n\n# Create a time axis in seconds\ntime_axis = np.arange(sig_len_stacked) / fs_stacked\n\n# Get lead names\nleads = ecg_signal_stacked.columns\n\n# Create a 12-row, 1-column subplot, sharing the x-axis\nfig, axes = plt.subplots(12, 1, figsize=(16, 22), sharex=True)\n\nfig.suptitle(f'12-Lead ECG (Stacked Layout) - ID: {ecg_id_stacked}', fontsize=20, y=1.02)\n\nfor i, lead in enumerate(leads):\n    # Plot the signal\n    axes[i].plot(time_axis, ecg_signal_stacked[lead], color='black', linewidth=0.7)\n    \n    # Add the lead name as a y-label (simulates the label on the paper)\n    axes[i].set_ylabel(lead, rotation=0, labelpad=30, ha='right', va='center', fontsize=12, fontweight='bold')\n    \n    # Turn off y-ticks and frame\n    axes[i].set_yticks([])\n    axes[i].spines['left'].set_visible(False)\n    axes[i].spines['top'].set_visible(False)\n    axes[i].spines['right'].set_visible(False)\n    \n    # Add a light grid (like ECG paper)\n    axes[i].grid(True, which='both', linestyle='--', linewidth=0.5, color='gray', alpha=0.5)\n\n# Set the x-axis label only on the last plot\naxes[-1].set_xlabel(\"Time (seconds)\", fontsize=14)\n\n# Adjust spacing to prevent overlap\nplt.subplots_adjust(hspace=0)\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-14T10:55:10.515261Z","iopub.execute_input":"2025-11-14T10:55:10.515845Z","iopub.status.idle":"2025-11-14T10:55:11.903432Z","shell.execute_reply.started":"2025-11-14T10:55:10.515820Z","shell.execute_reply":"2025-11-14T10:55:11.902159Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **5.  FFT(Fast Fourier Transform) Spectrum Analysis and Visualization**\nOk. Actullay I hought that main thing about competition. In my opinion: A core challenge in this competition is separating the **ECG signal** from the **background grid lines** in the image. These two components have different characteristics in the frequency domain:\n\n1.  **ECG Signals:** These are generally smooth, continuous lines, representing *low-frequency* components.\n2.  **Grid Lines:** These are sharp, periodic vertical and horizontal lines, which create strong peaks at specific *high frequencies*.\n\nBy applying a 2D Fast Fourier Transform (FFT) to the image, we can move from the spatial (pixel) domain to the frequency domain. The resulting magnitude spectrum helps us visualize this separation. The center of the spectrum represents low frequencies (the signal), while the bright, star-like spikes represent the high frequencies of the grid.\n\nThis analysis is the first step toward building a filter (e.g., a notch filter or mask in the frequency domain) to remove the grid lines, isolating the ECG signal for digitization.\n","metadata":{}},{"cell_type":"code","source":"import cv2\nimport numpy as np\nimport matplotlib.pyplot as plt\n\ndef analyze_fft_spectrum(image_path):\n    \"\"\"\n    Loads an ECG image, calculates its FFT, and\n    returns the log magnitude spectrum for visualization.\n    \"\"\"\n    # Load the image in grayscale\n    img = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)\n    if img is None:\n        print(\"Image could not be loaded.\")\n        return\n\n    # Expand to optimal size for FFT (optional but recommended)\n    rows, cols = img.shape\n    nrows = cv2.getOptimalDFTSize(rows)\n    ncols = cv2.getOptimalDFTSize(cols)\n    right, bottom = ncols - cols, nrows - rows\n    img_padded = cv2.copyMakeBorder(img, 0, bottom, 0, right, \n                                      cv2.BORDER_CONSTANT, value=0)\n\n    # Apply 2D DFT (Fast Fourier Transform)\n    # cv2.dft returns a two-channel float32 image for complex number output\n    dft = cv2.dft(np.float32(img_padded), flags=cv2.DFT_COMPLEX_OUTPUT)\n    \n    # Shift low frequencies to the center (standard step for visualization)\n    dft_shift = np.fft.fftshift(dft)\n\n    # Calculate magnitude spectrum (on a logarithmic scale)\n    # Magnitude = sqrt(Re^2 + Im^2)\n    # For visualization: 20 * log(Magnitude) \n    magnitude_spectrum = 20 * np.log(cv2.magnitude(dft_shift[:, :, 0], \n                                                dft_shift[:, :, 1]) + 1e-6) # 1e-6 prevents log(0) error\n\n    # Plot Original and Spectrum Images\n    plt.figure(figsize=(12, 6))\n    plt.subplot(121), plt.imshow(img, cmap='gray')\n    plt.title('Original Image'), plt.xticks(), plt.yticks()\n    plt.subplot(122), plt.imshow(magnitude_spectrum, cmap='gray')\n    plt.title('FFT Magnitude Spectrum'), plt.xticks(), plt.yticks()\n    plt.show()\n\n    return dft_shift, img_padded\n\n# Test with an example aligned image (e.g., '...-0001.png')\n# dft_shift, img_padded = analyze_fft_spectrum('/kaggle/input/physionet.../id-0001.png')\ndft_shift, img_padded = analyze_fft_spectrum('/kaggle/input/physionet-ecg-image-digitization/train/1006427285/1006427285-0005.png')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-14T11:15:19.566391Z","iopub.execute_input":"2025-11-14T11:15:19.566740Z","iopub.status.idle":"2025-11-14T11:15:22.841862Z","shell.execute_reply.started":"2025-11-14T11:15:19.566718Z","shell.execute_reply":"2025-11-14T11:15:22.840697Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def remove_grid_fft(dft_shift, img_padded):\n    \"\"\"\n    Takes the shifted DFT spectrum, masks the periodic grid noise,\n    and returns the cleaned image.\n    \"\"\"\n    rows, cols, _ = dft_shift.shape\n    crow, ccol = rows // 2, cols // 2\n\n    # --- Masking Grid Frequencies (Notch Filter) ---\n    # This step should be manually adjusted by looking at the \n    # 'analyze_fft_spectrum' output or done with an auto 'spike' detection algorithm.\n    # Grid lines appear as bright spots (spikes) radiating from the center (DC component).\n    # Example: Horizontal lines create spikes on the vertical axis, vertical lines on the horizontal axis.\n    \n    mask = np.ones((rows, cols, 2), np.float32)\n    \n    # Example masking: 'blacking out' the central vertical and horizontal axes\n    # This should be adjusted based on the grid strength and frequency.\n    # A more sophisticated approach would mask only the spike locations (points).\n    # This simple example targets the main axes with a width of +/- 5 pixels from the center.\n    # CAUTION: This can also damage the signal itself.\n    # The ideal way is to target only the bright spots belonging to the grid.\n    # An approach similar to [18]:\n    \n    # Mask the central DC component and its surroundings (general brightness)\n    cv2.circle(mask, (ccol, crow), radius=10, color=(0,0), thickness=-1)\n    \n    # Example: mask vertical spikes (horizontal grid lines)\n    # These coordinates should be derived from spectrum analysis (Cell A).\n    # Hypothetical spike coordinates:\n    spike_coords = [(crow - 50, ccol), (crow + 50, ccol)] \n    for r, c in spike_coords:\n        cv2.circle(mask, (c, r), radius=5, color=(0,0), thickness=-1)\n\n    # Apply the mask\n    fshift_masked = dft_shift * mask\n\n    # --- Inverse FFT (iFFT) ---\n    # Undo the shift\n    f_ishift = np.fft.ifftshift(fshift_masked)\n    \n    # Apply Inverse DFT\n    img_back = cv2.idft(f_ishift, flags=cv2.DFT_SCALE | cv2.DFT_REAL_OUTPUT)\n    \n    # Crop the image (remove padding)\n    original_rows, original_cols = img_padded.shape[:2]\n    img_back = img_back[0:original_rows, 0:original_cols]\n\n    # Show the results\n    plt.figure(figsize=(12, 6))\n    plt.subplot(121), plt.imshow(img_padded, cmap='gray')\n    plt.title('Original (Padded)'), plt.xticks(), plt.yticks()\n    plt.subplot(122), plt.imshow(img_back, cmap='gray')\n    plt.title('Grid Removed Image'), plt.xticks(), plt.yticks()\n    plt.show()\n\n    return img_back\n\n# Usage:\n# (With dft_shift and img_padded from Cell A)\ncleaned_image = remove_grid_fft(dft_shift, img_padded)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-14T11:15:27.968857Z","iopub.execute_input":"2025-11-14T11:15:27.969183Z","iopub.status.idle":"2025-11-14T11:15:30.666755Z","shell.execute_reply.started":"2025-11-14T11:15:27.969159Z","shell.execute_reply":"2025-11-14T11:15:30.665670Z"}},"outputs":[],"execution_count":null}]}