Hi all!
\nWe have updated the metric from F1 to mAP and are in the process of re-running all submissions. That may take some time, depending on how much time the submissions require to run. We'll let you know when the process is complete. Expect scores to drop slightly in most cases. [The update should now be complete for all submissions.]
\nWe have added the 16-bit training images here. We are leaving the 8-bit images as the default images in the competition dataset because they are 3x smaller, easier to download, and easier to iterate on.
", "messages": [ { "id": 1186595, "postDate": "2021-02-04T23:18:17.590Z", "content": "Hi all!
\nWe have updated the metric from F1 to mAP and are in the process of re-running all submissions. That may take some time, depending on how much time the submissions require to run. We'll let you know when the process is complete. Expect scores to drop slightly in most cases. [The update should now be complete for all submissions.]
\nWe have added the 16-bit training images here. We are leaving the 8-bit images as the default images in the competition dataset because they are 3x smaller, easier to download, and easier to iterate on.
", "rawMarkdown": "Hi all!\n\nWe have updated the metric from F1 to mAP and are in the process of re-running all submissions. That may take some time, depending on how much time the submissions require to run. We'll let you know when the process is complete. Expect scores to drop slightly in most cases. [The update should now be complete for all submissions.]\n\nWe have added the 16-bit training images [here](https://www.kaggle.com/philculliton/hpa-2020-16bit-training-set). We are leaving the 8-bit images as the default images in the competition dataset because they are 3x smaller, easier to download, and easier to iterate on.", "votes": 7 }, { "id": 1213038, "postDate": "2021-02-21T19:36:33.200Z", "content": "In the \"updates coming\" thread you mentioned:
\n\n\n\"Updating the public dataset to make it easier / faster to download.\"
\n
Is this still happening? Should we assume that no further updates regarding this will be forthcoming? Not trying to be difficult… I simply want to know if it's safe for me to invest time in speeding things up myself or if I should wait.
", "rawMarkdown": "In the \"updates coming\" thread you mentioned:\n\n> **\"Updating the public dataset to make it easier / faster to download.\"**\n\nIs this still happening? Should we assume that no further updates regarding this will be forthcoming? Not trying to be difficult... I simply want to know if it's safe for me to invest time in speeding things up myself or if I should wait.", "votes": 1, "replies": [ { "id": 1219554, "postDate": "2021-02-27T00:19:32.563Z", "content": "Hi! Thanks for your question. Yes - this is still in progress. We'll be making a trimmed version of the public data available as a Kaggle dataset, which should be quicker to access via Kaggle notebooks (and also be a smaller download elsewhere).
", "rawMarkdown": "Hi! Thanks for your question. Yes - this is still in progress. We'll be making a trimmed version of the public data available as a Kaggle dataset, which should be quicker to access via Kaggle notebooks (and also be a smaller download elsewhere).", "votes": 3 }, { "id": 1220045, "postDate": "2021-02-27T13:40:31.497Z", "content": "Thanks for the update. That will be marvellous.
", "rawMarkdown": "Thanks for the update. That will be marvellous. " }, { "id": 1252223, "postDate": "2021-03-25T14:07:29.303Z", "content": "@philculliton - Hi there! Is this still happening?
", "rawMarkdown": "@philculliton - Hi there! Is this still happening?", "votes": 2 }, { "id": 1263096, "postDate": "2021-04-05T04:55:58.790Z", "content": "@philculliton - it's probably the last moment to publish the public data as a Kaggle dataset… I got stuck waiting for it unfortunately :(
", "rawMarkdown": "@philculliton - it's probably the last moment to publish the public data as a Kaggle dataset... I got stuck waiting for it unfortunately :( ", "votes": 1 }, { "id": 1263548, "postDate": "2021-04-05T14:00:45.393Z", "content": "Hi - sorry, there were a number of issues to solve to make this available, but a large subset of the public data is available now as a Kaggle dataset.
", "rawMarkdown": "Hi - sorry, there were a number of issues to solve to make this available, but a large subset of the public data is available now as a [Kaggle dataset](https://www.kaggle.com/philculliton/hpa-challenge-2021-extra-train-images).", "votes": 1 }, { "id": 1263559, "postDate": "2021-04-05T14:06:52.650Z", "content": "Hi . Thank you so much for doing this. I know how hard you’ve all worked and it is greatly appreciated!
", "rawMarkdown": "Hi . Thank you so much for doing this. I know how hard you’ve all worked and it is greatly appreciated!" }, { "id": 1263621, "postDate": "2021-04-05T14:43:03.570Z", "content": "Awesome, thank you!
", "rawMarkdown": "Awesome, thank you!" } ] }, { "id": 1189374, "postDate": "2021-02-07T00:30:29.803Z", "content": "@philculliton, I can only sympathize with those who invested a lot, got an edge, only to see it evaporate because the competition changes.
\nEdit. The above can be seen as harsh and unfair, but it is just an observation about when metrics reset occur. I'm sure they are done with good will. But the effect can be devastating Here we lose a top contestant.
Thanks. Yes, I appreciate that. We felt it was a better fit for the competition, and since it was so early - we announced the proposed change a few days into the competition - I did not feel as though people would lose an excessive amount of work. Also, we plan on extending the deadline by a week to account for the lost time.
\nSo: I definitely considered these things when I made the change. I think the change will allow more people to compete more effectively.
", "rawMarkdown": "Thanks. Yes, I appreciate that. We felt it was a better fit for the competition, and since it was so early - we announced the proposed change a few days into the competition - I did not feel as though people would lose an excessive amount of work. Also, we plan on extending the deadline by a week to account for the lost time.\n\nSo: I definitely considered these things when I made the change. I think the change will allow more people to compete more effectively.", "votes": 2 } ] }, { "id": 1187629, "postDate": "2021-02-05T15:33:59.113Z", "content": "@philculliton
\nWhy you change metric?
\nyou said,
\nShifting from F1 to mAP, which should be easier to use / more readily available.
\nSorry, I can't agree with you on this information alone.
\nPlease explain why we should evaluate prediction as mAP instead of macro f1 in this task.
Hi! mAP is a much more common metric for tasks like these. More reference code (including the reference code provided) is available and it is easier to compare to similar tasks. Overall, the change should make it easier to participate. Basically, we proposed a new metric for the task, and realized a well-worn and -understood metric was probably the better choice based on early feedback from competitors.
", "rawMarkdown": "Hi! mAP is a much more common metric for tasks like these. More reference code (including the reference code provided) is available and it is easier to compare to similar tasks. Overall, the change should make it easier to participate. Basically, we proposed a new metric for the task, and realized a well-worn and -understood metric was probably the better choice based on early feedback from competitors." } ] }, { "id": 1189399, "postDate": "2021-02-07T01:56:48.970Z", "content": "Has this update already completed for all submissions?
", "rawMarkdown": "Has this update already completed for all submissions?", "replies": [ { "id": 1192029, "postDate": "2021-02-08T21:26:05.667Z", "content": "We are still working on the updates. We will update this post when we are done.
", "rawMarkdown": "We are still working on the updates. We will update this post when we are done. ", "votes": 1 } ] }, { "id": 1187394, "postDate": "2021-02-05T12:01:33.733Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1213038, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-02-21T19:36:33.200000", "content": "In the \"updates coming\" thread you mentioned:
\n\n\n\"Updating the public dataset to make it easier / faster to download.\"
\n
Is this still happening? Should we assume that no further updates regarding this will be forthcoming? Not trying to be difficult… I simply want to know if it's safe for me to invest time in speeding things up myself or if I should wait.
", "votes": 1, "replies": [ { "id": 1219554, "author_name": "Phil Culliton", "author_url": "", "post_date": "2021-02-27T00:19:32.563000", "content": "Hi! Thanks for your question. Yes - this is still in progress. We'll be making a trimmed version of the public data available as a Kaggle dataset, which should be quicker to access via Kaggle notebooks (and also be a smaller download elsewhere).
", "votes": 3, "replies": [] }, { "id": 1220045, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-02-27T13:40:31.497000", "content": "Thanks for the update. That will be marvellous.
", "votes": 0, "replies": [] }, { "id": 1252223, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-03-25T14:07:29.303000", "content": "@philculliton - Hi there! Is this still happening?
", "votes": 2, "replies": [] }, { "id": 1263096, "author_name": "Darek Kłeczek", "author_url": "", "post_date": "2021-04-05T04:55:58.790000", "content": "@philculliton - it's probably the last moment to publish the public data as a Kaggle dataset… I got stuck waiting for it unfortunately :(
", "votes": 1, "replies": [] }, { "id": 1263548, "author_name": "Phil Culliton", "author_url": "", "post_date": "2021-04-05T14:00:45.393000", "content": "Hi - sorry, there were a number of issues to solve to make this available, but a large subset of the public data is available now as a Kaggle dataset.
", "votes": 1, "replies": [] }, { "id": 1263559, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-04-05T14:06:52.650000", "content": "Hi . Thank you so much for doing this. I know how hard you’ve all worked and it is greatly appreciated!
", "votes": 0, "replies": [] }, { "id": 1263621, "author_name": "Darek Kłeczek", "author_url": "", "post_date": "2021-04-05T14:43:03.570000", "content": "Awesome, thank you!
", "votes": 0, "replies": [] } ] }, { "id": 1189374, "author_name": "CPMP", "author_url": "", "post_date": "2021-02-07T00:30:29.803000", "content": "@philculliton, I can only sympathize with those who invested a lot, got an edge, only to see it evaporate because the competition changes.
\nEdit. The above can be seen as harsh and unfair, but it is just an observation about when metrics reset occur. I'm sure they are done with good will. But the effect can be devastating Here we lose a top contestant.
Thanks. Yes, I appreciate that. We felt it was a better fit for the competition, and since it was so early - we announced the proposed change a few days into the competition - I did not feel as though people would lose an excessive amount of work. Also, we plan on extending the deadline by a week to account for the lost time.
\nSo: I definitely considered these things when I made the change. I think the change will allow more people to compete more effectively.
", "votes": 2, "replies": [] } ] }, { "id": 1187629, "author_name": "phalanx", "author_url": "", "post_date": "2021-02-05T15:33:59.113000", "content": "@philculliton
\nWhy you change metric?
\nyou said,
\nShifting from F1 to mAP, which should be easier to use / more readily available.
\nSorry, I can't agree with you on this information alone.
\nPlease explain why we should evaluate prediction as mAP instead of macro f1 in this task.
Hi! mAP is a much more common metric for tasks like these. More reference code (including the reference code provided) is available and it is easier to compare to similar tasks. Overall, the change should make it easier to participate. Basically, we proposed a new metric for the task, and realized a well-worn and -understood metric was probably the better choice based on early feedback from competitors.
", "votes": 0, "replies": [] } ] }, { "id": 1189399, "author_name": "ryches", "author_url": "", "post_date": "2021-02-07T01:56:48.970000", "content": "Has this update already completed for all submissions?
", "votes": 0, "replies": [ { "id": 1192029, "author_name": "Maggie", "author_url": "", "post_date": "2021-02-08T21:26:05.667000", "content": "We are still working on the updates. We will update this post when we are done.
", "votes": 1, "replies": [] } ] }, { "id": 1187394, "author_name": "", "author_url": "", "post_date": "2021-02-05T12:01:33.733000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1186595": "Hi all!\n\nWe have updated the metric from F1 to mAP and are in the process of re-running all submissions. That may take some time, depending on how much time the submissions require to run. We'll let you know when the process is complete. Expect scores to drop slightly in most cases. [The update should now be complete for all submissions.]\n\nWe have added the 16-bit training images [here](https://www.kaggle.com/philculliton/hpa-2020-16bit-training-set). We are leaving the 8-bit images as the default images in the competition dataset because they are 3x smaller, easier to download, and easier to iterate on.", "1213038": "In the \"updates coming\" thread you mentioned:\n\n> **\"Updating the public dataset to make it easier / faster to download.\"**\n\nIs this still happening? Should we assume that no further updates regarding this will be forthcoming? Not trying to be difficult... I simply want to know if it's safe for me to invest time in speeding things up myself or if I should wait.", "1189374": " @philculliton, I can only sympathize with those who invested a lot, got an edge, only to see it evaporate because the competition changes.\n\nEdit. The above can be seen as harsh and unfair, but it is just an observation about when metrics reset occur. I'm sure they are done with good will. But the effect can be devastating Here we lose a top contestant.", "1187629": "@philculliton \nWhy you change metric?\nyou said, \n`Shifting from F1 to mAP, which should be easier to use / more readily available.`\nSorry, I can't agree with you on this information alone.\nPlease explain why we should evaluate prediction as mAP instead of macro f1 in this task.", "1189399": "Has this update already completed for all submissions?", "1187394": "" } }