Is it allowed to probe the test set, e.g. to get the distribution of different image types? The rationale is to prioritize work on fine-tuning specific image types. I searched the rules and can't find any reference to \"probe\" or \"probing\".
", "messages": [ { "id": "3308235", "postDate": "10/28/2025 23:39:56", "content": "Is it allowed to probe the test set, e.g. to get the distribution of different image types? The rationale is to prioritize work on fine-tuning specific image types. I searched the rules and can't find any reference to \"probe\" or \"probing\".
", "rawMarkdown": "Is it allowed to probe the test set, e.g. to get the distribution of different image types? The rationale is to prioritize work on fine-tuning specific image types. I searched the rules and can't find any reference to \"probe\" or \"probing\".", "votes": null }, { "id": "3308240", "postDate": "10/29/2025 00:00:41", "content": "No - deliberately probing the test set not allowed. We want you to try to solve the general a problem, not a toy problem. If you work out the test set composition, and then tune your entry on that data, then the result isn't a test score, and it will not reflect how your algorithm is likely to perform in the real world.
", "rawMarkdown": "No - deliberately probing the test set not allowed. We want you to try to solve the general a problem, not a toy problem. If you work out the test set composition, and then tune your entry on that data, then the result isn't a test score, and it will not reflect how your algorithm is likely to perform in the real world.", "votes": null }, { "id": "3308243", "postDate": "10/29/2025 00:08:58", "content": "Thank you for the clarification.
", "rawMarkdown": "Thank you for the clarification.", "votes": null }, { "id": "3308262", "postDate": "10/29/2025 01:25:59", "content": "I think maybe need not restore to probing to get good results. With synthetic generator, we have infinite data and can get good generation. I am more concern with the consistent qrcode,4x3 lead signal layout, same dc pulse, etc This is present in all train. I wonder if we put the model in the wild outside kaggle,eg a new image without qrcode, results will differs.
", "rawMarkdown": "I think maybe need not restore to probing to get good results. With synthetic generator, we have infinite data and can get good generation. I am more concern with the consistent qrcode,4x3 lead signal layout, same dc pulse, etc This is present in all train. I wonder if we put the model in the wild outside kaggle,eg a new image without qrcode, results will differs.", "votes": null }, { "id": "3308440", "postDate": "10/29/2025 12:35:45", "content": "Adding to Gari's response, incorporating contextual knowledge regarding the ECG, its physical units, how the leads are related, and how they are printed on standard ECG grid is a more promising direction than probing the test set.
", "rawMarkdown": "Adding to Gari's response, incorporating contextual knowledge regarding the ECG, its physical units, how the leads are related, and how they are printed on standard ECG grid is a more promising direction than probing the test set.", "votes": null }, { "id": "3308671", "postDate": "10/30/2025 00:54:43", "content": "I haven't stated my intent clearly enough. I meant prioritization of work items, i.e. a list of ideas, which is too long to implement in its entirety before the end of this competition. Knowing the image type distribution would allow prioritizing ideas with the highest impact.
\nI'm going to assume a uniform distribution, to put this issue to rest.
", "rawMarkdown": "I haven't stated my intent clearly enough. I meant prioritization of work items, i.e. a list of ideas, which is too long to implement in its entirety before the end of this competition. Knowing the image type distribution would allow prioritizing ideas with the highest impact.\n\nI'm going to assume a uniform distribution, to put this issue to rest.", "votes": null } ], "comments": [ { "id": 3308240, "author_name": "gdclifford", "author_url": "", "post_date": "10/29/2025 00:00:41", "content": "No - deliberately probing the test set not allowed. We want you to try to solve the general a problem, not a toy problem. If you work out the test set composition, and then tune your entry on that data, then the result isn't a test score, and it will not reflect how your algorithm is likely to perform in the real world.
", "votes": null, "replies": [ { "id": 3308243, "author_name": "pauljurczak", "author_url": "", "post_date": "10/29/2025 00:08:58", "content": "Thank you for the clarification.
", "votes": null, "replies": [ { "id": 3308440, "author_name": "r2241272", "author_url": "", "post_date": "10/29/2025 12:35:45", "content": "Adding to Gari's response, incorporating contextual knowledge regarding the ECG, its physical units, how the leads are related, and how they are printed on standard ECG grid is a more promising direction than probing the test set.
", "votes": null, "replies": [ { "id": 3308671, "author_name": "pauljurczak", "author_url": "", "post_date": "10/30/2025 00:54:43", "content": "I haven't stated my intent clearly enough. I meant prioritization of work items, i.e. a list of ideas, which is too long to implement in its entirety before the end of this competition. Knowing the image type distribution would allow prioritizing ideas with the highest impact.
\nI'm going to assume a uniform distribution, to put this issue to rest.
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3308262, "author_name": "hengck23", "author_url": "", "post_date": "10/29/2025 01:25:59", "content": "I think maybe need not restore to probing to get good results. With synthetic generator, we have infinite data and can get good generation. I am more concern with the consistent qrcode,4x3 lead signal layout, same dc pulse, etc This is present in all train. I wonder if we put the model in the wild outside kaggle,eg a new image without qrcode, results will differs.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3308235": "Is it allowed to probe the test set, e.g. to get the distribution of different image types? The rationale is to prioritize work on fine-tuning specific image types. I searched the rules and can't find any reference to \"probe\" or \"probing\".", "3308240": "No - deliberately probing the test set not allowed. We want you to try to solve the general a problem, not a toy problem. If you work out the test set composition, and then tune your entry on that data, then the result isn't a test score, and it will not reflect how your algorithm is likely to perform in the real world.", "3308243": "Thank you for the clarification.", "3308262": "I think maybe need not restore to probing to get good results. With synthetic generator, we have infinite data and can get good generation. I am more concern with the consistent qrcode,4x3 lead signal layout, same dc pulse, etc This is present in all train. I wonder if we put the model in the wild outside kaggle,eg a new image without qrcode, results will differs.", "3308440": "Adding to Gari's response, incorporating contextual knowledge regarding the ECG, its physical units, how the leads are related, and how they are printed on standard ECG grid is a more promising direction than probing the test set.", "3308671": "I haven't stated my intent clearly enough. I meant prioritization of work items, i.e. a list of ideas, which is too long to implement in its entirety before the end of this competition. Knowing the image type distribution would allow prioritizing ideas with the highest impact.\n\nI'm going to assume a uniform distribution, to put this issue to rest." }, "source": "meta" }