{ "id": 422250, "title": "can we learn the expert annotations?", "url": "/competitions/hubmap-hacking-the-human-vasculature/discussion/422250", "author_name": "", "post_date": "2023-07-09T02:47:18.912064600Z", "votes": 1, "comment_count": 3, "views": 0, "content": "

can we learn the expert annotations?

\n

\"\"

\n

score submission:
\nhow to combine sureness score and iou prediction score in submission …. ????

\n

ideally,

\n
unknow_function = f(sureness,iou) = metric \n\nwhere unknow_function can learned gbt, catboost, MLP hand-coded\n\nof using metric as ground truth, it is easier to use metric rank in training.\ni.e.  rank lower than tp in the unknow_function f\n
\n

i am think of a ranking head in model and learn directly during end-2-end training

", "messages": [ { "id": "2335987", "postDate": "07/09/2023 02:47:18", "content": "

can we learn the expert annotations?

\n

\"\"

\n

score submission:
\nhow to combine sureness score and iou prediction score in submission …. ????

\n

ideally,

\n
unknow_function = f(sureness,iou) = metric \n\nwhere unknow_function can learned gbt, catboost, MLP hand-coded\n\nof using metric as ground truth, it is easier to use metric rank in training.\ni.e.  rank lower than tp in the unknow_function f\n
\n

i am think of a ranking head in model and learn directly during end-2-end training

", "rawMarkdown": "can we learn the expert annotations?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe11dd11411398ec6ff4987d9f46b39f7%2FSelection_999(2633).png?generation=1688870796501761&alt=media)\n\nscore submission:\nhow to combine sureness score and iou prediction score in submission .... ????\n\nideally,\n```\nunknow_function = f(sureness,iou) = metric score \n\nwhere unknow_function can be learned gbt, catboost, or MLP or hand-coded\n\ninstead of using metric score as ground truth, it is easier to use metric rank in training.\ni.e. fp should rank lower than tp in the unknow_function f\n\n```\n\ni am think of a ranking head in model and learn directly during end-2-end training", "votes": null }, { "id": "2335996", "postDate": "07/09/2023 02:57:20", "content": "

extending the unknown ranking function:

\n
 = f(sureness, iou, ... others ...) \n\n = area of object, textureness (e.g. unconfidence prediction are blurry. so we can use std of pixel prediction), ...\n
", "rawMarkdown": "extending the unknown ranking function:\n\n```\nunknow_function = f(sureness, iou, ... others ...) \n\nothers = area of object, textureness (e.g. unconfidence prediction are blurry. so we can use std of pixel prediction), ...\n\n```", "votes": null }, { "id": "2357889", "postDate": "07/25/2023 07:31:14", "content": "

I would like to take a stab at the idea you propose, but I am new to this. The ranking function you propose I can understand as a simple classification function (e.g. MLP) by using the predicted instance confidence, iou, area, texture, etc. information as variables, but how to get the labels

", "rawMarkdown": "I would like to take a stab at the idea you propose, but I am new to this. The ranking function you propose I can understand as a simple classification function (e.g. MLP) by using the predicted instance confidence, iou, area, texture, etc. information as variables, but how to get the labels", "votes": null }, { "id": "2358014", "postDate": "07/25/2023 09:04:10", "content": "

Perhaps I can see it as a regression problem, predicting the true confidence level

", "rawMarkdown": "Perhaps I can see it as a regression problem, predicting the true confidence level", "votes": null } ], "comments": [ { "id": 2335996, "author_name": "hengck23", "author_url": "", "post_date": "07/09/2023 02:57:20", "content": "

extending the unknown ranking function:

\n
 = f(sureness, iou, ... others ...) \n\n = area of object, textureness (e.g. unconfidence prediction are blurry. so we can use std of pixel prediction), ...\n
", "votes": null, "replies": [ { "id": 2357889, "author_name": "bent1e", "author_url": "", "post_date": "07/25/2023 07:31:14", "content": "

I would like to take a stab at the idea you propose, but I am new to this. The ranking function you propose I can understand as a simple classification function (e.g. MLP) by using the predicted instance confidence, iou, area, texture, etc. information as variables, but how to get the labels

", "votes": null, "replies": [ { "id": 2358014, "author_name": "bent1e", "author_url": "", "post_date": "07/25/2023 09:04:10", "content": "

Perhaps I can see it as a regression problem, predicting the true confidence level

", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2335987": "can we learn the expert annotations?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe11dd11411398ec6ff4987d9f46b39f7%2FSelection_999(2633).png?generation=1688870796501761&alt=media)\n\nscore submission:\nhow to combine sureness score and iou prediction score in submission .... ????\n\nideally,\n```\nunknow_function = f(sureness,iou) = metric score \n\nwhere unknow_function can be learned gbt, catboost, or MLP or hand-coded\n\ninstead of using metric score as ground truth, it is easier to use metric rank in training.\ni.e. fp should rank lower than tp in the unknow_function f\n\n```\n\ni am think of a ranking head in model and learn directly during end-2-end training", "2335996": "extending the unknown ranking function:\n\n```\nunknow_function = f(sureness, iou, ... others ...) \n\nothers = area of object, textureness (e.g. unconfidence prediction are blurry. so we can use std of pixel prediction), ...\n\n```", "2357889": "I would like to take a stab at the idea you propose, but I am new to this. The ranking function you propose I can understand as a simple classification function (e.g. MLP) by using the predicted instance confidence, iou, area, texture, etc. information as variables, but how to get the labels", "2358014": "Perhaps I can see it as a regression problem, predicting the true confidence level" }, "source": "meta" }