![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4001300%2Fc8e52c59343b2049477785cf4531f0f0%2Fsenneterror.jpg?generation=1701937579895633&alt=media\")
Hello everyone.
\nI just finished the Child Mind Institute (CMI) competition and was looking at this competition here.
\nI noticed that there are a lot of people making Scoring errors in the CMI competition, and I noticed that the mystery is probably the specification that the error occurs if the Scoring takes longer than 30 minutes.
\nThis is written in my topic in the CMI competition here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617).
\nYou can see \"Scoring error if it takes 30 minutes for scoring\" by the fact that when you COPY AND EDIT the METRIC NOTEBOOK, you will see this in the upper right corner. I just did the same thing with this competition and got the same display.
\nmetric notebook url from overview in this competition : https://www.kaggle.com/code/metric/surface-dice-metric/notebook
\n
It says here that an error will occur if the scoring takes 30 minutes.
\nI saw the topics like these in this competition:
\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/455220
\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/456761
Several people seemed to have errors even though the code had been corrected (but @ryanholbrook may be able to fix it and make it work.) .
\nPerhaps there is an OOM issue, but is it also affected by the 30 minute scoring time?
If so, this specification is very confusing to find it and I think it should be written on the EVALUATION page of the OVERVIEW, and I wrote it in the topic here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617) yesterday as well.
\nIn the CMI competition, this specification was not very well known, and it was mentioned that the long number of lines in the SUBMISSION file would cause an error. But some people were confused. I was concerned that we would be wasting our time in thinking about this if the same phenomenon exists in this competition.
\nAnd is 30 minutes too short a time for grading in this competition? (Sorry. I just joined the competition and I don't know.)
\nSorry if I am wrong.
\nThank you in advance.
@ryanholbrook Can you confirm this specification ??
", "messages": [ { "id": "2552156", "postDate": "12/07/2023 08:51:24", "content": "Hello everyone.
\nI just finished the Child Mind Institute (CMI) competition and was looking at this competition here.
\nI noticed that there are a lot of people making Scoring errors in the CMI competition, and I noticed that the mystery is probably the specification that the error occurs if the Scoring takes longer than 30 minutes.
\nThis is written in my topic in the CMI competition here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617).
\nYou can see \"Scoring error if it takes 30 minutes for scoring\" by the fact that when you COPY AND EDIT the METRIC NOTEBOOK, you will see this in the upper right corner. I just did the same thing with this competition and got the same display.
\nmetric notebook url from overview in this competition : https://www.kaggle.com/code/metric/surface-dice-metric/notebook
\n
It says here that an error will occur if the scoring takes 30 minutes.
\nI saw the topics like these in this competition:
\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/455220
\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/456761
Several people seemed to have errors even though the code had been corrected (but @ryanholbrook may be able to fix it and make it work.) .
\nPerhaps there is an OOM issue, but is it also affected by the 30 minute scoring time?
If so, this specification is very confusing to find it and I think it should be written on the EVALUATION page of the OVERVIEW, and I wrote it in the topic here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617) yesterday as well.
\nIn the CMI competition, this specification was not very well known, and it was mentioned that the long number of lines in the SUBMISSION file would cause an error. But some people were confused. I was concerned that we would be wasting our time in thinking about this if the same phenomenon exists in this competition.
\nAnd is 30 minutes too short a time for grading in this competition? (Sorry. I just joined the competition and I don't know.)
\nSorry if I am wrong.
\nThank you in advance.
@ryanholbrook Can you confirm this specification ??
", "rawMarkdown": "Hello everyone.\n\nI just finished the [Child Mind Institute (CMI) competition](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states) and was looking at this competition here.\n\nI noticed that there are a lot of people making Scoring errors in the CMI competition, and I noticed that the mystery is probably the specification that the error occurs if the Scoring takes longer than 30 minutes.\n\nThis is written in my topic in the CMI competition here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617). \n\nYou can see \"Scoring error if it takes 30 minutes for scoring\" by the fact that when you COPY AND EDIT the METRIC NOTEBOOK, you will see this in the upper right corner. I just did the same thing with this competition and got the same display.\n\nmetric notebook url from overview in this competition : https://www.kaggle.com/code/metric/surface-dice-metric/notebook\n\n\n\nIt says here that an error will occur if the scoring takes 30 minutes.\n\nI saw the topics like these in this competition:\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/455220\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/456761\n\nSeveral people seemed to have errors even though the code had been corrected (but @ryanholbrook may be able to fix it and make it work.) .\nPerhaps there is an OOM issue, but is it also affected by the 30 minute scoring time?\n\nIf so, this specification is very confusing to find it and I think it should be written on the EVALUATION page of the OVERVIEW, and I wrote it in the topic here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617) yesterday as well.\n\nIn the CMI competition, this specification was not very well known, and it was mentioned that the long number of lines in the SUBMISSION file would cause an error. But some people were confused. I was concerned that we would be wasting our time in thinking about this if the same phenomenon exists in this competition.\n\nAnd is 30 minutes too short a time for grading in this competition? (Sorry. I just joined the competition and I don't know.)\n\nSorry if I am wrong.\nThank you in advance.\n\n@ryanholbrook Can you confirm this specification ??", "votes": null }, { "id": "2552591", "postDate": "12/07/2023 15:41:19", "content": "Hi @chumajin, That's a great question. For this metric, I extended the scoring time to 90 minutes. I haven't seen any submissions that need longer than that, but I can extend it further if necessary. Let me know.
", "rawMarkdown": "Hi @chumajin, That's a great question. For this metric, I extended the scoring time to 90 minutes. I haven't seen any submissions that need longer than that, but I can extend it further if necessary. Let me know.", "votes": null } ], "comments": [ { "id": 2552591, "author_name": "ryanholbrook", "author_url": "", "post_date": "12/07/2023 15:41:19", "content": "Hi @chumajin, That's a great question. For this metric, I extended the scoring time to 90 minutes. I haven't seen any submissions that need longer than that, but I can extend it further if necessary. Let me know.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2552156": "Hello everyone.\n\nI just finished the [Child Mind Institute (CMI) competition](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states) and was looking at this competition here.\n\nI noticed that there are a lot of people making Scoring errors in the CMI competition, and I noticed that the mystery is probably the specification that the error occurs if the Scoring takes longer than 30 minutes.\n\nThis is written in my topic in the CMI competition here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617). \n\nYou can see \"Scoring error if it takes 30 minutes for scoring\" by the fact that when you COPY AND EDIT the METRIC NOTEBOOK, you will see this in the upper right corner. I just did the same thing with this competition and got the same display.\n\nmetric notebook url from overview in this competition : https://www.kaggle.com/code/metric/surface-dice-metric/notebook\n\n\n\nIt says here that an error will occur if the scoring takes 30 minutes.\n\nI saw the topics like these in this competition:\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/455220\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/456761\n\nSeveral people seemed to have errors even though the code had been corrected (but @ryanholbrook may be able to fix it and make it work.) .\nPerhaps there is an OOM issue, but is it also affected by the 30 minute scoring time?\n\nIf so, this specification is very confusing to find it and I think it should be written on the EVALUATION page of the OVERVIEW, and I wrote it in the topic here(https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459617) yesterday as well.\n\nIn the CMI competition, this specification was not very well known, and it was mentioned that the long number of lines in the SUBMISSION file would cause an error. But some people were confused. I was concerned that we would be wasting our time in thinking about this if the same phenomenon exists in this competition.\n\nAnd is 30 minutes too short a time for grading in this competition? (Sorry. I just joined the competition and I don't know.)\n\nSorry if I am wrong.\nThank you in advance.\n\n@ryanholbrook Can you confirm this specification ??", "2552591": "Hi @chumajin, That's a great question. For this metric, I extended the scoring time to 90 minutes. I haven't seen any submissions that need longer than that, but I can extend it further if necessary. Let me know." }, "source": "meta" }