This challenge is a continuation of the \"PhysioNet 2024 Competition on ECG Digitization\"
\nI have listed the winning solution and their published papers:
\n1st Place Winners: SignalSavant
\nTeam Members: Felix Krones, Ben Walker, Terry Lyons, Adam Mahdi
\nPaper: \"Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Printouts\"
\nScore: SNR of 12.15 on hidden test set
\nReference: CinC2024-097.pdf, arXiv:2410.14185
\nApproach: U-Net based segmentation model with Hough transform for rotation, mask vectorisation for signal reconstruction
2nd Place: BAPORLab
\nScore: SNR of 5.493 on hidden test set
\nReference: CinC2024-227.pdf
3rd Place: wavie ABI
\nPaper: \"WAVIE: A Modular and Open-Source Python…\"
\nScore: SNR of 5.469
\nReference: CinC2024-229.pdf
5th Place: USST Med (University of Shanghai for Science and Technology)
\nTeam Members: Xiankai Yu, Yangcheng Huang, Jian Wu, Jiahao Wang, Wenjie Cai
\nPaper:\"From Paper to Digital: ECG Processing with U-Net Digitization and ResNet Classification\"
\nScore:SNR of 2.202
\nReference: CinC2024-134.pdf
\nApproach: YOLOv8 Tiny for image correction, ResUNet with CBAM module for digitization
6th Place: mins-eth (ETH Zurich)
\nTeam Members: Haoliang Shang, Clemens Hutter, Yani Zhang
\nPaper: \"Automated Digitization of Paper ECG Records Using Convolutional Networks: a Faster R-CNN and U-Net Approach\"
\nScore:SNR of 0.893
\nReference:CinC2024-199.pdf
\nApproach: Faster R-CNN for signal detection, U-Net for pixel-level segmentation
Other Published approaches:
\nHackathon Winner: Easy Geese (University of Oxford/Leeds)
\nTeam Members: S. Summerton, N. Dinsdale, T. Leinonen, et al.
\nPaper: \"A Modular Framework for the Interpretation of Paper ECGs\"
\nScore: SNR of −5.272 (digitization), F-measure of 0.082 (classification)
\nApproach: YOLO for area extraction, ResUnet for segmentation, SEresnet classifier
\nReference: WhiteRose Online Research
VinDigitizer
\nPaper: \"An Image Processing Approach to Digitize Paper ECG Records\"
\n**Reference: **CinC2024-135.pdf
\nApproach: Image processing-based digitization with lead segmentation, signal extraction, and scale conversion
PapyrusECG
\nPaper: \"Fusion of Deep Learning and Rule-Based Techniques for Enhanced Paper-Based ECG Digitization\"
\nReference: CinC2024-495.pdf
\nApproach: Combined deep learning and rule-based pipeline for paper ECG digitization
Paper: Automated Conversion and Analysis of Printed ECG Using Random Signal Pretrained Digitizer
\nReference: https://cinc.org/archives/2024/pdf/CinC2024-004.pdf
These are not the test score results, and we have concerns that the information posted in this thread may have been automatically generated. Please note that some of the papers report their leaderboard scores on the validation set rather than the final unseen test set of the challenge. Please see the official paper for this event and note that the scores reported by the teams are the validation scores, with much lower scores on the hidden test data. Please also note that the current Kaggle-based challenge has significant differences with the 2024 PhysioNet Challenge. In particular, the test data are completely new (and much more realistic), and the evaluation metric is slightly modified. You should therefore not expect that the scores or rankings will map to this new challenge.
", "rawMarkdown": "These are not the test score results, and we have concerns that the information posted in this thread may have been automatically generated. Please note that some of the papers report their leaderboard scores on the validation set rather than the final unseen test set of the challenge. Please see the official paper for [this event](https://cinc.org/archives/2024/pdf/CinC2024-011.pdf) and note that the scores reported by the teams are the validation scores, with much lower scores on the hidden test data. Please also note that the current Kaggle-based challenge has significant differences with the 2024 PhysioNet Challenge. In particular, the test data are completely new (and much more realistic), and the evaluation metric is slightly modified. You should therefore not expect that the scores or rankings will map to this new challenge.", "votes": 8, "replies": [ { "id": 3305654, "postDate": "2025-10-23T07:29:38.643Z", "content": "Is the perfect score expected to be 384 (PERFECT_SCORE = 384) in 2024 PhysioNet Challenge and in this year's competition?
", "rawMarkdown": "Is the perfect score expected to be 384 (PERFECT_SCORE = 384) in 2024 PhysioNet Challenge and in this year's competition?", "votes": 1, "replies": [ { "id": 3305787, "postDate": "2025-10-23T13:05:43.613Z", "content": "Good question! The perfect score of 384 and the minimum score of -384 are simply upper and lower limits, which exceed the 64-bit floating-point noise floor in dB (required for Kaggle’s scoring purposes). No model is expected to come anywhere near the perfect score. In practice, even an excellent model would achieve a score of around 20 to 30 dB because converting an ECG from a time series to an image and then back to a time series involves unavoidable filtering and resampling, which limit reconstruction quality. We have addressed some of these aspects in the ECG-Image-Database and ECG-Image-Kit papers.
\nA trivial baseline performance is 0 dB (when an entry returns all zeros). Of course, this doesn't mean entries can't perform worse than that; negative scores indicate that a submission performed even worse than returning all zeros (on average).
\nAlso note that the score reflects the average performance across all test records. As a rule of thumb, as teams develop their models, once the scores rise noticeably above zero (even by a few decibels), they're moving in the right direction. We'll share more details about this as we go.
", "rawMarkdown": "Good question! The perfect score of 384 and the minimum score of -384 are simply upper and lower limits, which exceed the 64-bit floating-point noise floor in dB (required for Kaggle’s scoring purposes). No model is expected to come anywhere near the perfect score. In practice, even an excellent model would achieve a score of around 20 to 30 dB because converting an ECG from a time series to an image and then back to a time series involves unavoidable filtering and resampling, which limit reconstruction quality. We have addressed some of these aspects in the [ECG-Image-Database](https://doi.org/10.48550/arXiv.2409.16612) and [ECG-Image-Kit](https://doi.org/10.1088/1361-6579/ad4954) papers.\n\nA trivial baseline performance is 0 dB (when an entry returns all zeros). Of course, this doesn't mean entries can't perform worse than that; negative scores indicate that a submission performed even worse than returning all zeros (on average).\n\nAlso note that the score reflects the average performance across all test records. As a rule of thumb, as teams develop their models, once the scores rise noticeably above zero (even by a few decibels), they're moving in the right direction. We'll share more details about this as we go.", "votes": 4 }, { "id": 3311018, "postDate": "2025-11-04T03:33:21.593Z", "content": "Looking at the evaluation script code, I see that the result from compute_snr(), which maxes out at 384, is subjected to:
(( * .log10(image_scores.())), -PERFECT_SCORE)\nin score() function. This makes the maximum score equal to
(( * np.(PERFECT_SCORE)), -PERFECT_SCORE)\nwhich is ~25.8433. Is my interpretation accurate?
", "rawMarkdown": "Looking at the evaluation script code, I see that the result from `compute_snr()`, which maxes out at 384, is subjected to:\n```\nreturn max(float(10 * np.log10(image_scores.mean())), -PERFECT_SCORE)\n```\nin `score()` function. This makes the maximum score equal to\n```\nmax(float(10 * np.log10(PERFECT_SCORE)), -PERFECT_SCORE)\n```\nwhich is ~25.8433. Is my interpretation accurate?\n" }, { "id": 3311216, "postDate": "2025-11-04T13:59:50.177Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 3305473, "postDate": "2025-10-22T19:49:36.060Z", "content": "This challenge is a continuation of the \"PhysioNet 2024 Competition on ECG Digitization\"
\nI have listed the winning solution and their published papers:
\n1st Place Winners: SignalSavant
\nTeam Members: Felix Krones, Ben Walker, Terry Lyons, Adam Mahdi
\nPaper: \"Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Printouts\"
\nScore: SNR of 12.15 on hidden test set
\nReference: CinC2024-097.pdf, arXiv:2410.14185
\nApproach: U-Net based segmentation model with Hough transform for rotation, mask vectorisation for signal reconstruction
2nd Place: BAPORLab
\nScore: SNR of 5.493 on hidden test set
\nReference: CinC2024-227.pdf
3rd Place: wavie ABI
\nPaper: \"WAVIE: A Modular and Open-Source Python…\"
\nScore: SNR of 5.469
\nReference: CinC2024-229.pdf
5th Place: USST Med (University of Shanghai for Science and Technology)
\nTeam Members: Xiankai Yu, Yangcheng Huang, Jian Wu, Jiahao Wang, Wenjie Cai
\nPaper:\"From Paper to Digital: ECG Processing with U-Net Digitization and ResNet Classification\"
\nScore:SNR of 2.202
\nReference: CinC2024-134.pdf
\nApproach: YOLOv8 Tiny for image correction, ResUNet with CBAM module for digitization
6th Place: mins-eth (ETH Zurich)
\nTeam Members: Haoliang Shang, Clemens Hutter, Yani Zhang
\nPaper: \"Automated Digitization of Paper ECG Records Using Convolutional Networks: a Faster R-CNN and U-Net Approach\"
\nScore:SNR of 0.893
\nReference:CinC2024-199.pdf
\nApproach: Faster R-CNN for signal detection, U-Net for pixel-level segmentation
Other Published approaches:
\nHackathon Winner: Easy Geese (University of Oxford/Leeds)
\nTeam Members: S. Summerton, N. Dinsdale, T. Leinonen, et al.
\nPaper: \"A Modular Framework for the Interpretation of Paper ECGs\"
\nScore: SNR of −5.272 (digitization), F-measure of 0.082 (classification)
\nApproach: YOLO for area extraction, ResUnet for segmentation, SEresnet classifier
\nReference: WhiteRose Online Research
VinDigitizer
\nPaper: \"An Image Processing Approach to Digitize Paper ECG Records\"
\n**Reference: **CinC2024-135.pdf
\nApproach: Image processing-based digitization with lead segmentation, signal extraction, and scale conversion
PapyrusECG
\nPaper: \"Fusion of Deep Learning and Rule-Based Techniques for Enhanced Paper-Based ECG Digitization\"
\nReference: CinC2024-495.pdf
\nApproach: Combined deep learning and rule-based pipeline for paper ECG digitization
Paper: Automated Conversion and Analysis of Printed ECG Using Random Signal Pretrained Digitizer
\nReference: https://cinc.org/archives/2024/pdf/CinC2024-004.pdf
These are not the test score results, and we have concerns that the information posted in this thread may have been automatically generated. Please note that some of the papers report their leaderboard scores on the validation set rather than the final unseen test set of the challenge. Please see the official paper for this event and note that the scores reported by the teams are the validation scores, with much lower scores on the hidden test data. Please also note that the current Kaggle-based challenge has significant differences with the 2024 PhysioNet Challenge. In particular, the test data are completely new (and much more realistic), and the evaluation metric is slightly modified. You should therefore not expect that the scores or rankings will map to this new challenge.
", "votes": 8, "replies": [ { "id": 3305654, "author_name": "Paul Jurczak", "author_url": "", "post_date": "2025-10-23T07:29:38.643000", "content": "Is the perfect score expected to be 384 (PERFECT_SCORE = 384) in 2024 PhysioNet Challenge and in this year's competition?
", "votes": 1, "replies": [ { "id": 3305787, "author_name": "Reza Sameni", "author_url": "", "post_date": "2025-10-23T13:05:43.613000", "content": "Good question! The perfect score of 384 and the minimum score of -384 are simply upper and lower limits, which exceed the 64-bit floating-point noise floor in dB (required for Kaggle’s scoring purposes). No model is expected to come anywhere near the perfect score. In practice, even an excellent model would achieve a score of around 20 to 30 dB because converting an ECG from a time series to an image and then back to a time series involves unavoidable filtering and resampling, which limit reconstruction quality. We have addressed some of these aspects in the ECG-Image-Database and ECG-Image-Kit papers.
\nA trivial baseline performance is 0 dB (when an entry returns all zeros). Of course, this doesn't mean entries can't perform worse than that; negative scores indicate that a submission performed even worse than returning all zeros (on average).
\nAlso note that the score reflects the average performance across all test records. As a rule of thumb, as teams develop their models, once the scores rise noticeably above zero (even by a few decibels), they're moving in the right direction. We'll share more details about this as we go.
", "votes": 4, "replies": [] }, { "id": 3311018, "author_name": "Paul Jurczak", "author_url": "", "post_date": "2025-11-04T03:33:21.593000", "content": "Looking at the evaluation script code, I see that the result from compute_snr(), which maxes out at 384, is subjected to:
(( * .log10(image_scores.())), -PERFECT_SCORE)\nin score() function. This makes the maximum score equal to
(( * np.(PERFECT_SCORE)), -PERFECT_SCORE)\nwhich is ~25.8433. Is my interpretation accurate?
", "votes": 0, "replies": [] }, { "id": 3311216, "author_name": "", "author_url": "", "post_date": "2025-11-04T13:59:50.177000", "content": "", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3305488": "These are not the test score results, and we have concerns that the information posted in this thread may have been automatically generated. Please note that some of the papers report their leaderboard scores on the validation set rather than the final unseen test set of the challenge. Please see the official paper for [this event](https://cinc.org/archives/2024/pdf/CinC2024-011.pdf) and note that the scores reported by the teams are the validation scores, with much lower scores on the hidden test data. Please also note that the current Kaggle-based challenge has significant differences with the 2024 PhysioNet Challenge. In particular, the test data are completely new (and much more realistic), and the evaluation metric is slightly modified. You should therefore not expect that the scores or rankings will map to this new challenge.", "3305473": "This challenge is a continuation of the \"**PhysioNet 2024 Competition on ECG Digitization**\"\n\nI have listed the winning solution and their published papers:\n\n***1st Place Winners: SignalSavant***\n**Team Members:** Felix Krones, Ben Walker, Terry Lyons, Adam Mahdi\n**Paper:** \"Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Printouts\"\n**Score:** SNR of 12.15 on hidden test set\n**Reference:** [CinC2024-097.pdf](https://cinc.org/archives/2024/pdf/CinC2024-097.pdf), [arXiv:2410.14185](https://arxiv.org/abs/2410.14185)\n**Approach:** U-Net based segmentation model with Hough transform for rotation, mask vectorisation for signal reconstruction \n\n\n***2nd Place: BAPORLab***\n**Score:** SNR of 5.493 on hidden test set\n**Reference:** [CinC2024-227.pdf](https://cinc.org/archives/2024/pdf/CinC2024-227.pdf)\n\n***3rd Place: wavie ABI***\nPaper: \"WAVIE: A Modular and Open-Source Python...\"\nScore: SNR of 5.469\nReference: [CinC2024-229.pdf](https://cinc.org/archives/2024/pdf/CinC2024-229.pdf)\n\n***5th Place: USST Med (University of Shanghai for Science and Technology)***\n**Team Members:** Xiankai Yu, Yangcheng Huang, Jian Wu, Jiahao Wang, Wenjie Cai\n**Paper:**\"From Paper to Digital: ECG Processing with U-Net Digitization and ResNet Classification\"\n**Score:**SNR of 2.202\n**Reference:** [CinC2024-134.pdf](https://cinc.org/archives/2024/pdf/CinC2024-134.pdf)\n**Approach:** YOLOv8 Tiny for image correction, ResUNet with CBAM module for digitization\n\n***6th Place: mins-eth (ETH Zurich)***\n**Team Members:** Haoliang Shang, Clemens Hutter, Yani Zhang\n**Paper:** \"Automated Digitization of Paper ECG Records Using Convolutional Networks: a Faster R-CNN and U-Net Approach\"\n**Score:**SNR of 0.893\n**Reference:**[CinC2024-199.pdf](https://cinc.org/archives/2024/pdf/CinC2024-199.pdf)\n**Approach:** Faster R-CNN for signal detection, U-Net for pixel-level segmentation\n\n\nOther Published approaches:\n\n**Hackathon Winner:** Easy Geese (University of Oxford/Leeds)\n**Team Members:** S. Summerton, N. Dinsdale, T. Leinonen, et al.\n**Paper:** \"A Modular Framework for the Interpretation of Paper ECGs\"\n**Score:** SNR of −5.272 (digitization), F-measure of 0.082 (classification)\n**Approach:** YOLO for area extraction, ResUnet for segmentation, SEresnet classifier\n**Reference:** [WhiteRose Online Research](https://eprints.whiterose.ac.uk/id/eprint/222013/)\n\n***VinDigitizer***\n**Paper:** \"An Image Processing Approach to Digitize Paper ECG Records\"\n**Reference: **[CinC2024-135.pdf](https://cinc.org/archives/2024/pdf/CinC2024-135.pdf)\n**Approach:** Image processing-based digitization with lead segmentation, signal extraction, and scale conversion\n\n***PapyrusECG***\n**Paper:** \"Fusion of Deep Learning and Rule-Based Techniques for Enhanced Paper-Based ECG Digitization\"\n**Reference:** [CinC2024-495.pdf](https://cinc.org/archives/2024/pdf/CinC2024-495.pdf)\n**Approach:** Combined deep learning and rule-based pipeline for paper ECG digitization\n\n**Paper:** Automated Conversion and Analysis of Printed ECG Using Random Signal Pretrained Digitizer\n**Reference:** https://cinc.org/archives/2024/pdf/CinC2024-004.pdf\n\n\n" } }