Two weeks into the challenge, the leaderboard scores have exceeded 7 dB, which is very promising. As teams continue to improve their models, a few points may be helpful for building more efficient and generalizable models:
\nMany articles have been published in this field with valuable ideas. Whether you are participating as a student, researcher, or just for fun, it is worth spending some time reviewing the literature before deciding on your models and methods. Domain knowledge can take you a long way. If you are not already familiar with ECGs, learn more about them. For beginners, start with tutorials on clinical 12-lead ECGs, how the leads are related and collected, their typical amplitude ranges, signal bandwidths, and how they are printed on standard paper ECG grids. Read the articles shared under this post and Chapters 15-19 of this book.
The ECG grid is probably the most reliable (or perhaps the only) reference for accurately converting pixel units to physical units (mV and time). However, as you may have noticed, the grid can be deformed due to perspective and imaging artifacts that result in affine and non-affine transformations of the ECG and its grid. Make the best use of the grid to calibrate the images and retrieve the physical units.
It's important to distinguish between the original time-series sampling frequency, the image-based sampling frequency (related to image size and DPI), and the recovered sampling frequency. Read the appendix of the ECG-Image-Database to understand the relationship between these sampling rates. Also, check out this post.
Read about lead names and the calibration pulse here and here.
The challenge score is a measure of average SNR. We have a post on the expected SNR range. Considering that the ECG is a nonstationary signal, with higher signal power across the QRS complex and lower power in the T-wave, P-wave, and ST segment, a first and quick performance boost would be to correctly recover the QRS complexes. However, as you refine your models, the accuracy of the lower-power segments of the ECG also becomes very important and can make a significant difference between models. The challenge score automatically corrects minor horizontal and vertical offsets (which are irrelevant to ECG diagnosis) per lead (see here); so you are not penalized for minor offsets, but scaling (both horizontal and vertical) is very important.
The challenge score is in decibels (dB). So, for example, going from 0 dB (equivalent to reporting all zeros) to 6 dB is much easier than going from 6 dB to 12 dB. So, don't get discouraged if you don't see similar jumps in SNR improvement over the next few weeks. Considering the inevitable impact of digital-to-analog and analog-to-digital conversion effects required for creating images and converting them back to digital samples, reaching a score of over 15–20 dB would already be close to what human eyes can distinguish in ECG printouts. Importantly, the SNR score is an average across all leads and records; therefore, minor systematic issues in your models, missing low-amplitude segments such as the Q-wave or ST segment, or even edge cases can negatively impact your overall performance. Visual inspection of your model outputs, and overlapping the raw and recovered time series on top of the training data images, can be used to diagnose your models.
Although not very common, ECG leads can sometimes overlap or cross one another on printed ECGs (due to baseline wander or differences in ECG amplitude). Separating overlapping leads is one of the biggest bottlenecks for many open-source models.
Make sure not to hard-code items such as signal length and sampling frequency. Read them from the metadata files (train.csv and test.csv). This will make your code more generalizable to unseen test data and more applicable beyond the challenge.
Deep learning–based methods are powerful, but do not underestimate the value of classical computer vision and signal processing methods, or the potential of combining them with deep learning approaches. There are open-access solutions for the different components of the task, including lead segmentation, grid detection, estimating physical units from an ECG image, connecting pixels to recover a waveform, etc. Read the main references of the challenge (and the references cited there), as well as the functions we've shared in ECG-Image-Kit for digitization.
ECG-Image-Kit has many other features for changing the grid, perspective, color, etc. There are also many 12-lead ECG datasets available publicly, which can be printed on ECG paper-like backgrounds using ECG-Image-Kit or similar tools. Tools such as the open-source electrophysiological toolbox (OSET) also have functions that can be used to generate synthetic ECG time series with full control over ECG morphology and heart rate, which, combined with ECG-Image-Kit, provide an infinite resource for creating more training data.
If you have created publicly available real ECG image datasets (with access to the ECG time series and the images), please list them under this post so that the community can use them to improve their models.
Two weeks into the challenge, the leaderboard scores have exceeded 7 dB, which is very promising. As teams continue to improve their models, a few points may be helpful for building more efficient and generalizable models:
\nMany articles have been published in this field with valuable ideas. Whether you are participating as a student, researcher, or just for fun, it is worth spending some time reviewing the literature before deciding on your models and methods. Domain knowledge can take you a long way. If you are not already familiar with ECGs, learn more about them. For beginners, start with tutorials on clinical 12-lead ECGs, how the leads are related and collected, their typical amplitude ranges, signal bandwidths, and how they are printed on standard paper ECG grids. Read the articles shared under this post and Chapters 15-19 of this book.
The ECG grid is probably the most reliable (or perhaps the only) reference for accurately converting pixel units to physical units (mV and time). However, as you may have noticed, the grid can be deformed due to perspective and imaging artifacts that result in affine and non-affine transformations of the ECG and its grid. Make the best use of the grid to calibrate the images and retrieve the physical units.
It's important to distinguish between the original time-series sampling frequency, the image-based sampling frequency (related to image size and DPI), and the recovered sampling frequency. Read the appendix of the ECG-Image-Database to understand the relationship between these sampling rates. Also, check out this post.
Read about lead names and the calibration pulse here and here.
The challenge score is a measure of average SNR. We have a post on the expected SNR range. Considering that the ECG is a nonstationary signal, with higher signal power across the QRS complex and lower power in the T-wave, P-wave, and ST segment, a first and quick performance boost would be to correctly recover the QRS complexes. However, as you refine your models, the accuracy of the lower-power segments of the ECG also becomes very important and can make a significant difference between models. The challenge score automatically corrects minor horizontal and vertical offsets (which are irrelevant to ECG diagnosis) per lead (see here); so you are not penalized for minor offsets, but scaling (both horizontal and vertical) is very important.
The challenge score is in decibels (dB). So, for example, going from 0 dB (equivalent to reporting all zeros) to 6 dB is much easier than going from 6 dB to 12 dB. So, don't get discouraged if you don't see similar jumps in SNR improvement over the next few weeks. Considering the inevitable impact of digital-to-analog and analog-to-digital conversion effects required for creating images and converting them back to digital samples, reaching a score of over 15–20 dB would already be close to what human eyes can distinguish in ECG printouts. Importantly, the SNR score is an average across all leads and records; therefore, minor systematic issues in your models, missing low-amplitude segments such as the Q-wave or ST segment, or even edge cases can negatively impact your overall performance. Visual inspection of your model outputs, and overlapping the raw and recovered time series on top of the training data images, can be used to diagnose your models.
Although not very common, ECG leads can sometimes overlap or cross one another on printed ECGs (due to baseline wander or differences in ECG amplitude). Separating overlapping leads is one of the biggest bottlenecks for many open-source models.
Make sure not to hard-code items such as signal length and sampling frequency. Read them from the metadata files (train.csv and test.csv). This will make your code more generalizable to unseen test data and more applicable beyond the challenge.
Deep learning–based methods are powerful, but do not underestimate the value of classical computer vision and signal processing methods, or the potential of combining them with deep learning approaches. There are open-access solutions for the different components of the task, including lead segmentation, grid detection, estimating physical units from an ECG image, connecting pixels to recover a waveform, etc. Read the main references of the challenge (and the references cited there), as well as the functions we've shared in ECG-Image-Kit for digitization.
ECG-Image-Kit has many other features for changing the grid, perspective, color, etc. There are also many 12-lead ECG datasets available publicly, which can be printed on ECG paper-like backgrounds using ECG-Image-Kit or similar tools. Tools such as the open-source electrophysiological toolbox (OSET) also have functions that can be used to generate synthetic ECG time series with full control over ECG morphology and heart rate, which, combined with ECG-Image-Kit, provide an infinite resource for creating more training data.
If you have created publicly available real ECG image datasets (with access to the ECG time series and the images), please list them under this post so that the community can use them to improve their models.
This is my first PhysioNet competition, and although I don’t have a medical background, I’ve learned a lot already especially how ECG domain knowledge connects to model performance. Some of the insights here also relate to a few medical-related courses I’ve taken in the past, which helped me follow along better.
\nThank you for this detailed explanation
", "rawMarkdown": "This is my first PhysioNet competition, and although I don’t have a medical background, I’ve learned a lot already especially how ECG domain knowledge connects to model performance. Some of the insights here also relate to a few medical-related courses I’ve taken in the past, which helped me follow along better.\n\nThank you for this detailed explanation", "votes": null }, { "id": "3351997", "postDate": "11/28/2025 19:44:07", "content": "Midway through the challenge, the leaderboard has reached 20dB, which is great! Some thoughts on how models can be improved and generalized:
\nWhile the challenge score is an average across all records, the model standard deviations can be very different. Check and diagnose the outliers of your models on the training set. This will give you ideas about the aspects and corner cases where your models are failing.
For real-world practical use (and within the context of this challenge), not reporting is better than returning erroneous values that do not match the ECG. As you may have noticed, a trivial all-zeros solution has an SNR of 0dB (for any specific record and lead) simply because the signal and noise power become the same at 0dB. If you have internal mechanisms for assessing your model confidence and measuring the quality of the extracted time series, you may take advantage of this fact and return an all-zeros vector for any specific lead or record that you strongly suspect the model is failing on (which could result in negative SNR, worse than an all-zeros entry). Of course, you should use this technique only in very limited cases because the current leaderboard is already performing very well on most records.
After converting the image into time series, post-processing can help. As we know, the time series must be resampled to a target sampling frequency to match the length of the expected time series. Unless you have better solutions, make sure to use customized linear-phase resampling filters with the same impulse response across all leads to avoid phase distortion. Nonlinear-phase filters may introduce post-extraction distortions that negatively impact your scores.
Organize your codes into task-specific modules. Unless you have an end-to-end solution, most teams should have components for lead segmentation, fixing image deformations, grid detection and removal, normalizing colors and contrast, physical unit recovery, and image-to–time-series conversion. Using a modular design helps generalize your codes in the future and makes it easier to team up with others toward the end of the challenge.
Thanks for the comprehensive guidance. The discussion around grid deformation, multiple sampling frequencies, and how the SNR metric treats offsets versus scaling is extremely useful. I’ve been experimenting with classical CV methods as a baseline and agree that combining them with learned models—especially for difficult cases like overlapping leads and low-amplitude segments—seems to be a promising direction.
", "rawMarkdown": "Thanks for the comprehensive guidance. The discussion around grid deformation, multiple sampling frequencies, and how the SNR metric treats offsets versus scaling is extremely useful. I’ve been experimenting with classical CV methods as a baseline and agree that combining them with learned models—especially for difficult cases like overlapping leads and low-amplitude segments—seems to be a promising direction.", "votes": null } ], "comments": [ { "id": 3311070, "author_name": "ladiposamson", "author_url": "", "post_date": "11/04/2025 06:48:03", "content": "This is my first PhysioNet competition, and although I don’t have a medical background, I’ve learned a lot already especially how ECG domain knowledge connects to model performance. Some of the insights here also relate to a few medical-related courses I’ve taken in the past, which helped me follow along better.
\nThank you for this detailed explanation
", "votes": null, "replies": [] }, { "id": 3351997, "author_name": "r2241272", "author_url": "", "post_date": "11/28/2025 19:44:07", "content": "Midway through the challenge, the leaderboard has reached 20dB, which is great! Some thoughts on how models can be improved and generalized:
\nWhile the challenge score is an average across all records, the model standard deviations can be very different. Check and diagnose the outliers of your models on the training set. This will give you ideas about the aspects and corner cases where your models are failing.
For real-world practical use (and within the context of this challenge), not reporting is better than returning erroneous values that do not match the ECG. As you may have noticed, a trivial all-zeros solution has an SNR of 0dB (for any specific record and lead) simply because the signal and noise power become the same at 0dB. If you have internal mechanisms for assessing your model confidence and measuring the quality of the extracted time series, you may take advantage of this fact and return an all-zeros vector for any specific lead or record that you strongly suspect the model is failing on (which could result in negative SNR, worse than an all-zeros entry). Of course, you should use this technique only in very limited cases because the current leaderboard is already performing very well on most records.
After converting the image into time series, post-processing can help. As we know, the time series must be resampled to a target sampling frequency to match the length of the expected time series. Unless you have better solutions, make sure to use customized linear-phase resampling filters with the same impulse response across all leads to avoid phase distortion. Nonlinear-phase filters may introduce post-extraction distortions that negatively impact your scores.
Organize your codes into task-specific modules. Unless you have an end-to-end solution, most teams should have components for lead segmentation, fixing image deformations, grid detection and removal, normalizing colors and contrast, physical unit recovery, and image-to–time-series conversion. Using a modular design helps generalize your codes in the future and makes it easier to team up with others toward the end of the challenge.
Thanks for the comprehensive guidance. The discussion around grid deformation, multiple sampling frequencies, and how the SNR metric treats offsets versus scaling is extremely useful. I’ve been experimenting with classical CV methods as a baseline and agree that combining them with learned models—especially for difficult cases like overlapping leads and low-amplitude segments—seems to be a promising direction.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3310897": "Two weeks into the challenge, the leaderboard scores have exceeded 7 dB, which is very promising. As teams continue to improve their models, a few points may be helpful for building more efficient and generalizable models:\n\n* Many articles have been published in this field with valuable ideas. Whether you are participating as a student, researcher, or just for fun, it is worth spending some time reviewing the literature before deciding on your models and methods. Domain knowledge can take you a long way. If you are not already familiar with ECGs, learn more about them. For beginners, start with tutorials on clinical 12-lead ECGs, how the leads are related and collected, their typical amplitude ranges, signal bandwidths, and how they are printed on standard paper ECG grids. Read the articles shared under this [post](https://www.kaggle.com/competitions/physionet-ecg-image-digitization/discussion/612729) and Chapters 15-19 of [this](https://www.bem.fi/book/) book.\n\n* The ECG grid is probably the most reliable (or perhaps the only) reference for accurately converting pixel units to physical units (mV and time). However, as you may have noticed, the grid can be deformed due to perspective and imaging artifacts that result in affine and non-affine transformations of the ECG and its grid. Make the best use of the grid to calibrate the images and retrieve the physical units.\n\n* It's important to distinguish between the original time-series sampling frequency, the image-based sampling frequency (related to image size and DPI), and the recovered sampling frequency. Read the appendix of the ECG-Image-Database to understand the relationship between these sampling rates. Also, check out [this](https://www.kaggle.com/competitions/physionet-ecg-image-digitization/discussion/613179#3306633) post.\n\n* Read about lead names and the calibration pulse [here](https://www.kaggle.com/competitions/physionet-ecg-image-digitization/discussion/613040#3306418) and [here](https://www.kaggle.com/competitions/physionet-ecg-image-digitization/discussion/613040#3305990). \n\n* The challenge score is a measure of average SNR. We have a [post](https://www.kaggle.com/competitions/physionet-ecg-image-digitization/discussion/612891#3305787) on the expected SNR range. Considering that the ECG is a nonstationary signal, with higher signal power across the QRS complex and lower power in the T-wave, P-wave, and ST segment, a first and quick performance boost would be to correctly recover the QRS complexes. However, as you refine your models, the accuracy of the lower-power segments of the ECG also becomes very important and can make a significant difference between models. The challenge score automatically corrects minor horizontal and vertical offsets (which are irrelevant to ECG diagnosis) per lead (see [here](https://www.kaggle.com/competitions/physionet-ecg-image-digitization/discussion/612773#3305098)); so you are not penalized for minor offsets, but scaling (both horizontal and vertical) is very important.\n\n* The challenge score is in decibels (dB). So, for example, going from 0 dB (equivalent to reporting all zeros) to 6 dB is much easier than going from 6 dB to 12 dB. So, don't get discouraged if you don't see similar jumps in SNR improvement over the next few weeks. Considering the inevitable impact of digital-to-analog and analog-to-digital conversion effects required for creating images and converting them back to digital samples, reaching a score of over 15–20 dB would already be close to what human eyes can distinguish in ECG printouts. Importantly, the SNR score is an average across all leads and records; therefore, minor systematic issues in your models, missing low-amplitude segments such as the Q-wave or ST segment, or even edge cases can negatively impact your overall performance. Visual inspection of your model outputs, and overlapping the raw and recovered time series on top of the training data images, can be used to diagnose your models.\n\n* Although not very common, ECG leads can sometimes overlap or cross one another on printed ECGs (due to baseline wander or differences in ECG amplitude). Separating overlapping leads is one of the biggest bottlenecks for many open-source models.\n\n* Make sure not to hard-code items such as signal length and sampling frequency. Read them from the metadata files (*train.csv* and *test.csv*). This will make your code more generalizable to unseen test data and more applicable beyond the challenge.\n\n* Deep learning–based methods are powerful, but do not underestimate the value of classical computer vision and signal processing methods, or the potential of combining them with deep learning approaches. There are open-access solutions for the different components of the task, including lead segmentation, grid detection, estimating physical units from an ECG image, connecting pixels to recover a waveform, etc. Read the main references of the challenge (and the references cited there), as well as the functions we've shared in ECG-Image-Kit for digitization. \n\n* ECG-Image-Kit has many other features for changing the grid, perspective, color, etc. There are also many 12-lead ECG datasets available publicly, which can be printed on ECG paper-like backgrounds using ECG-Image-Kit or similar tools. Tools such as the [open-source electrophysiological toolbox (OSET)](https://github.com/alphanumericslab/OSET/tree/master/matlab/tools/modeling) also have functions that can be used to generate synthetic ECG time series with full control over ECG morphology and heart rate, which, combined with ECG-Image-Kit, provide an infinite resource for creating more training data.\n\n* If you have created publicly available real ECG image datasets (with access to the ECG time series and the images), please list them under this post so that the community can use them to improve their models.", "3311070": "This is my first PhysioNet competition, and although I don’t have a medical background, I’ve learned a lot already especially how ECG domain knowledge connects to model performance. Some of the insights here also relate to a few medical-related courses I’ve taken in the past, which helped me follow along better.\n\nThank you for this detailed explanation", "3351997": "Midway through the challenge, the leaderboard has reached 20dB, which is great! Some thoughts on how models can be improved and generalized:\n\n1. While the challenge score is an average across all records, the model standard deviations can be very different. Check and diagnose the outliers of your models on the training set. This will give you ideas about the aspects and corner cases where your models are failing.\n\n2. For real-world practical use (and within the context of this challenge), not reporting is better than returning erroneous values that do not match the ECG. As you may have noticed, a trivial all-zeros solution has an SNR of 0dB (for any specific record and lead) simply because the signal and noise power become the same at 0dB. If you have internal mechanisms for assessing your model confidence and measuring the quality of the extracted time series, you may take advantage of this fact and return an all-zeros vector for any specific lead or record that you strongly suspect the model is failing on (which could result in negative SNR, worse than an all-zeros entry). Of course, you should use this technique only in very limited cases because the current leaderboard is already performing very well on most records.\n\n3. After converting the image into time series, post-processing can help. As we know, the time series must be resampled to a target sampling frequency to match the length of the expected time series. Unless you have better solutions, make sure to use customized linear-phase resampling filters with the same impulse response across all leads to avoid phase distortion. Nonlinear-phase filters may introduce post-extraction distortions that negatively impact your scores.\n\n4. Organize your codes into task-specific modules. Unless you have an end-to-end solution, most teams should have components for lead segmentation, fixing image deformations, grid detection and removal, normalizing colors and contrast, physical unit recovery, and image-to–time-series conversion. Using a modular design helps generalize your codes in the future and makes it easier to team up with others toward the end of the challenge.", "3385610": "Thanks for the comprehensive guidance. The discussion around grid deformation, multiple sampling frequencies, and how the SNR metric treats offsets versus scaling is extremely useful. I’ve been experimenting with classical CV methods as a baseline and agree that combining them with learned models—especially for difficult cases like overlapping leads and low-amplitude segments—seems to be a promising direction." }, "source": "meta" }