{ "id": 286935, "title": "Help shed a light on challenge participation characteristics", "url": "/competitions/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/286935", "author_name": "Matthias Eisenmann", "post_date": "2021-11-11T11:56:33.869000", "votes": 3, "comment_count": 0, "views": 0, "content": "

Dear participants,

\n

Today we would like to inform you about a new initiative with respect to biomedical challenges. This initiative involves many research institutions and is led by the MICCAI Special Interest Group on Biomedical Image Analysis Challenges (MICCAI SIG-BIAC). We thank the challenge organizers for supporting the SIG-BIAC efforts and in particular this initiative.

\n

What is it about?
\nIn the past few years, the initiative has been working on bringing biomedical image analysis to the next level of quality [1, 2, 3, 4, 5]. While the focus was on the meta-research question \"Is the winner really the best?\", the goal now is to go one step further and analyze challenge participation characteristics (e.g., expertise of team, algorithm design, computational infrastructure used). To this end, we are performing a meta-analysis of the challenges conducted in 2021 (ISBI and MICCAI) by means of a survey.

\n

To shed a maximum of light on these important aspects, we now rely on YOU. We kindly ask you to fill out the survey, such that we have a large collection of challenge strategies. We hope and assume that the corresponding arXiv paper, on which you will be co-author, will obtain a lot of interest by the research community. As an additional 'thank you', you can choose to be considered for prizes that will be raffled among the pool of ISBI, MICCAI and Kaggle challenge participants that submit the questionnaire. Specifically, NVIDIA is offering 20 vouchers for their self-paced online courses.

\n

We would really appreciate your time and effort, and we kindly ask you to complete this survey by December 10, 2021.

\n

To participate, please follow this link:
\nhttps://onlinesurvey.dkfz.de/index.php/825947?lang=en

\n

We look forward to working with you!

\n

Best regards,
\nMatthias Eisenmann, Annika Reinke, Annette Kopp-Schneider and Lena Maier-Hein on behalf of all contributors

\n
\n

References:
\n[1] Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M., Scholz, P., Arbel, T., Bogunovic, H., Bradley, A.P., Carass, A., Feldmann, C., Frangi, A.F., Full, P.M., van Ginneken, B., Hanbury, A., Honauer, K., Kozubek, M., Landman, B.A., März, K., Maier, O., Maier-Hein, K., Menze, B.H., Müller, H., Neher, P.F., Niessen, W., Rajpoot, N., Sharp, G.C., Sirinukunwattana, K., Speidel, S., Stock, C., Stoyanov, D., Taha, A.A., van der Sommen, F., Wang, C.-W., Weber, M.-A., Zheng, G., Jannin, P., Kopp-Schneider, A., 2018. Why rankings of biomedical image analysis competitions should be interpreted with care. Nat. Commun. 9, 5217. https://doi.org/10.1038/s41467-018-07619-7

\n

[2] Reinke, A., Eisenmann, M., Onogur, S., Stankovic, M., Scholz, P., Full, P.M., Bogunovic, H., Landman, B.A., Maier, O., Menze, B., Sharp, G.C., Sirinukunwattana, K., Speidel, S., van der Sommen, F., Zheng, G., Müller, H., Kozubek, M., Arbel, T., Bradley, A.P., Jannin, P., Kopp-Schneider, A., Maier-Hein, L., 2018. How to Exploit Weaknesses in Biomedical Challenge Design and Organization, in: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Springer International Publishing, Cham, pp. 388–395.

\n

[3] Maier-Hein, L., Reinke, A., Kozubek, M., Martel, A.L., Arbel, T., Eisenmann, M., Hanbury, A., Jannin, P., Müller, H., Onogur, S., Saez-Rodriguez, J., van Ginneken, B., Kopp-Schneider, A., Landman, B.A., 2020. BIAS: Transparent reporting of biomedical image analysis challenges. Med. Image Anal. 66, 101796. https://doi.org/10.1016/j.media.2020.101796

\n

[4] Wiesenfarth, M., Reinke, A., Landman, B.A., Eisenmann, M., Aguilera Saiz, L., Cardoso, M.J., Maier-Hein, L., Kopp-Schneider, A., 2021. Methods and open-source toolkit for analyzing and visualizing challenge results. Sci. Rep. 11, 1–15. https://doi.org/10.1038/s41598-021-82017-6

\n

[5] Roß, T., Bruno, P., Reinke, A., Wiesenfarth, M., Koeppel, L., Full, P.M., Pekdemir, B., Godau, P., Trofimova, D., Isensee, F., Moccia, S., Calimeri, F., Müller-Stich, B.P., Kopp-Schneider, A., Maier-Hein, L., 2021. How can we learn (more) from challenges? A statistical approach to driving future algorithm development. ArXiv210609302 Cs.

", "messages": [ { "id": 1578889, "postDate": "2021-11-11T11:56:33.870Z", "content": "

Dear participants,

\n

Today we would like to inform you about a new initiative with respect to biomedical challenges. This initiative involves many research institutions and is led by the MICCAI Special Interest Group on Biomedical Image Analysis Challenges (MICCAI SIG-BIAC). We thank the challenge organizers for supporting the SIG-BIAC efforts and in particular this initiative.

\n

What is it about?
\nIn the past few years, the initiative has been working on bringing biomedical image analysis to the next level of quality [1, 2, 3, 4, 5]. While the focus was on the meta-research question \"Is the winner really the best?\", the goal now is to go one step further and analyze challenge participation characteristics (e.g., expertise of team, algorithm design, computational infrastructure used). To this end, we are performing a meta-analysis of the challenges conducted in 2021 (ISBI and MICCAI) by means of a survey.

\n

To shed a maximum of light on these important aspects, we now rely on YOU. We kindly ask you to fill out the survey, such that we have a large collection of challenge strategies. We hope and assume that the corresponding arXiv paper, on which you will be co-author, will obtain a lot of interest by the research community. As an additional 'thank you', you can choose to be considered for prizes that will be raffled among the pool of ISBI, MICCAI and Kaggle challenge participants that submit the questionnaire. Specifically, NVIDIA is offering 20 vouchers for their self-paced online courses.

\n

We would really appreciate your time and effort, and we kindly ask you to complete this survey by December 10, 2021.

\n

To participate, please follow this link:
\nhttps://onlinesurvey.dkfz.de/index.php/825947?lang=en

\n

We look forward to working with you!

\n

Best regards,
\nMatthias Eisenmann, Annika Reinke, Annette Kopp-Schneider and Lena Maier-Hein on behalf of all contributors

\n
\n

References:
\n[1] Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M., Scholz, P., Arbel, T., Bogunovic, H., Bradley, A.P., Carass, A., Feldmann, C., Frangi, A.F., Full, P.M., van Ginneken, B., Hanbury, A., Honauer, K., Kozubek, M., Landman, B.A., März, K., Maier, O., Maier-Hein, K., Menze, B.H., Müller, H., Neher, P.F., Niessen, W., Rajpoot, N., Sharp, G.C., Sirinukunwattana, K., Speidel, S., Stock, C., Stoyanov, D., Taha, A.A., van der Sommen, F., Wang, C.-W., Weber, M.-A., Zheng, G., Jannin, P., Kopp-Schneider, A., 2018. Why rankings of biomedical image analysis competitions should be interpreted with care. Nat. Commun. 9, 5217. https://doi.org/10.1038/s41467-018-07619-7

\n

[2] Reinke, A., Eisenmann, M., Onogur, S., Stankovic, M., Scholz, P., Full, P.M., Bogunovic, H., Landman, B.A., Maier, O., Menze, B., Sharp, G.C., Sirinukunwattana, K., Speidel, S., van der Sommen, F., Zheng, G., Müller, H., Kozubek, M., Arbel, T., Bradley, A.P., Jannin, P., Kopp-Schneider, A., Maier-Hein, L., 2018. How to Exploit Weaknesses in Biomedical Challenge Design and Organization, in: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Springer International Publishing, Cham, pp. 388–395.

\n

[3] Maier-Hein, L., Reinke, A., Kozubek, M., Martel, A.L., Arbel, T., Eisenmann, M., Hanbury, A., Jannin, P., Müller, H., Onogur, S., Saez-Rodriguez, J., van Ginneken, B., Kopp-Schneider, A., Landman, B.A., 2020. BIAS: Transparent reporting of biomedical image analysis challenges. Med. Image Anal. 66, 101796. https://doi.org/10.1016/j.media.2020.101796

\n

[4] Wiesenfarth, M., Reinke, A., Landman, B.A., Eisenmann, M., Aguilera Saiz, L., Cardoso, M.J., Maier-Hein, L., Kopp-Schneider, A., 2021. Methods and open-source toolkit for analyzing and visualizing challenge results. Sci. Rep. 11, 1–15. https://doi.org/10.1038/s41598-021-82017-6

\n

[5] Roß, T., Bruno, P., Reinke, A., Wiesenfarth, M., Koeppel, L., Full, P.M., Pekdemir, B., Godau, P., Trofimova, D., Isensee, F., Moccia, S., Calimeri, F., Müller-Stich, B.P., Kopp-Schneider, A., Maier-Hein, L., 2021. How can we learn (more) from challenges? A statistical approach to driving future algorithm development. ArXiv210609302 Cs.

", "rawMarkdown": "Dear participants,\n\nToday we would like to inform you about a new initiative with respect to biomedical challenges. This initiative involves many research institutions and is led by the MICCAI Special Interest Group on Biomedical Image Analysis Challenges ([MICCAI SIG-BIAC](http://www.miccai.org/special-interest-groups/challenges/)). We thank the challenge organizers for supporting the SIG-BIAC efforts and in particular this initiative.\n\nWhat is it about?\nIn the past few years, the initiative has been working on bringing biomedical image analysis to the next level of quality [1, 2, 3, 4, 5]. While the focus was on the meta-research question \"Is the winner really the best?\", the goal now is to go one step further and analyze challenge participation characteristics (e.g., expertise of team, algorithm design, computational infrastructure used). To this end, we are performing a meta-analysis of the challenges conducted in 2021 (ISBI and MICCAI) by means of a survey.\n \nTo shed a maximum of light on these important aspects, we now rely on YOU. We kindly ask you to fill out the survey, such that we have a large collection of challenge strategies. We hope and assume that the corresponding arXiv paper, on which you will be **co-author**, will obtain a lot of interest by the research community. As an additional 'thank you', you can choose to be considered for **prizes** that will be raffled among the pool of ISBI, MICCAI and Kaggle challenge participants that submit the questionnaire. Specifically, NVIDIA is offering 20 vouchers for their self-paced online courses.\n\nWe would really appreciate your time and effort, and we kindly ask you to complete this survey by December 10, 2021.\n\nTo participate, please follow this link:\n[https://onlinesurvey.dkfz.de/index.php/825947?lang=en](https://onlinesurvey.dkfz.de/index.php/825947?lang=en)\n\nWe look forward to working with you!\n\nBest regards,\nMatthias Eisenmann, Annika Reinke, Annette Kopp-Schneider and Lena Maier-Hein on behalf of all contributors\n\n---\nReferences:\n[1] Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M., Scholz, P., Arbel, T., Bogunovic, H., Bradley, A.P., Carass, A., Feldmann, C., Frangi, A.F., Full, P.M., van Ginneken, B., Hanbury, A., Honauer, K., Kozubek, M., Landman, B.A., März, K., Maier, O., Maier-Hein, K., Menze, B.H., Müller, H., Neher, P.F., Niessen, W., Rajpoot, N., Sharp, G.C., Sirinukunwattana, K., Speidel, S., Stock, C., Stoyanov, D., Taha, A.A., van der Sommen, F., Wang, C.-W., Weber, M.-A., Zheng, G., Jannin, P., Kopp-Schneider, A., 2018. *Why rankings of biomedical image analysis competitions should be interpreted with care*. Nat. Commun. 9, 5217. https://doi.org/10.1038/s41467-018-07619-7\n\n[2] Reinke, A., Eisenmann, M., Onogur, S., Stankovic, M., Scholz, P., Full, P.M., Bogunovic, H., Landman, B.A., Maier, O., Menze, B., Sharp, G.C., Sirinukunwattana, K., Speidel, S., van der Sommen, F., Zheng, G., Müller, H., Kozubek, M., Arbel, T., Bradley, A.P., Jannin, P., Kopp-Schneider, A., Maier-Hein, L., 2018. *How to Exploit Weaknesses in Biomedical Challenge Design and Organization*, in: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Springer International Publishing, Cham, pp. 388–395.\n\n[3] Maier-Hein, L., Reinke, A., Kozubek, M., Martel, A.L., Arbel, T., Eisenmann, M., Hanbury, A., Jannin, P., Müller, H., Onogur, S., Saez-Rodriguez, J., van Ginneken, B., Kopp-Schneider, A., Landman, B.A., 2020. *BIAS: Transparent reporting of biomedical image analysis challenges*. Med. Image Anal. 66, 101796. https://doi.org/10.1016/j.media.2020.101796\n\n[4] Wiesenfarth, M., Reinke, A., Landman, B.A., Eisenmann, M., Aguilera Saiz, L., Cardoso, M.J., Maier-Hein, L., Kopp-Schneider, A., 2021. *Methods and open-source toolkit for analyzing and visualizing challenge results*. Sci. Rep. 11, 1–15. https://doi.org/10.1038/s41598-021-82017-6\n\n[5] Roß, T., Bruno, P., Reinke, A., Wiesenfarth, M., Koeppel, L., Full, P.M., Pekdemir, B., Godau, P., Trofimova, D., Isensee, F., Moccia, S., Calimeri, F., Müller-Stich, B.P., Kopp-Schneider, A., Maier-Hein, L., 2021. *How can we learn (more) from challenges? A statistical approach to driving future algorithm development*. ArXiv210609302 Cs.", "votes": 3 } ], "comments": [], "raw_markdown_by_id": { "1578889": "Dear participants,\n\nToday we would like to inform you about a new initiative with respect to biomedical challenges. This initiative involves many research institutions and is led by the MICCAI Special Interest Group on Biomedical Image Analysis Challenges ([MICCAI SIG-BIAC](http://www.miccai.org/special-interest-groups/challenges/)). We thank the challenge organizers for supporting the SIG-BIAC efforts and in particular this initiative.\n\nWhat is it about?\nIn the past few years, the initiative has been working on bringing biomedical image analysis to the next level of quality [1, 2, 3, 4, 5]. While the focus was on the meta-research question \"Is the winner really the best?\", the goal now is to go one step further and analyze challenge participation characteristics (e.g., expertise of team, algorithm design, computational infrastructure used). To this end, we are performing a meta-analysis of the challenges conducted in 2021 (ISBI and MICCAI) by means of a survey.\n \nTo shed a maximum of light on these important aspects, we now rely on YOU. We kindly ask you to fill out the survey, such that we have a large collection of challenge strategies. We hope and assume that the corresponding arXiv paper, on which you will be **co-author**, will obtain a lot of interest by the research community. As an additional 'thank you', you can choose to be considered for **prizes** that will be raffled among the pool of ISBI, MICCAI and Kaggle challenge participants that submit the questionnaire. Specifically, NVIDIA is offering 20 vouchers for their self-paced online courses.\n\nWe would really appreciate your time and effort, and we kindly ask you to complete this survey by December 10, 2021.\n\nTo participate, please follow this link:\n[https://onlinesurvey.dkfz.de/index.php/825947?lang=en](https://onlinesurvey.dkfz.de/index.php/825947?lang=en)\n\nWe look forward to working with you!\n\nBest regards,\nMatthias Eisenmann, Annika Reinke, Annette Kopp-Schneider and Lena Maier-Hein on behalf of all contributors\n\n---\nReferences:\n[1] Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M., Scholz, P., Arbel, T., Bogunovic, H., Bradley, A.P., Carass, A., Feldmann, C., Frangi, A.F., Full, P.M., van Ginneken, B., Hanbury, A., Honauer, K., Kozubek, M., Landman, B.A., März, K., Maier, O., Maier-Hein, K., Menze, B.H., Müller, H., Neher, P.F., Niessen, W., Rajpoot, N., Sharp, G.C., Sirinukunwattana, K., Speidel, S., Stock, C., Stoyanov, D., Taha, A.A., van der Sommen, F., Wang, C.-W., Weber, M.-A., Zheng, G., Jannin, P., Kopp-Schneider, A., 2018. *Why rankings of biomedical image analysis competitions should be interpreted with care*. Nat. Commun. 9, 5217. https://doi.org/10.1038/s41467-018-07619-7\n\n[2] Reinke, A., Eisenmann, M., Onogur, S., Stankovic, M., Scholz, P., Full, P.M., Bogunovic, H., Landman, B.A., Maier, O., Menze, B., Sharp, G.C., Sirinukunwattana, K., Speidel, S., van der Sommen, F., Zheng, G., Müller, H., Kozubek, M., Arbel, T., Bradley, A.P., Jannin, P., Kopp-Schneider, A., Maier-Hein, L., 2018. *How to Exploit Weaknesses in Biomedical Challenge Design and Organization*, in: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Springer International Publishing, Cham, pp. 388–395.\n\n[3] Maier-Hein, L., Reinke, A., Kozubek, M., Martel, A.L., Arbel, T., Eisenmann, M., Hanbury, A., Jannin, P., Müller, H., Onogur, S., Saez-Rodriguez, J., van Ginneken, B., Kopp-Schneider, A., Landman, B.A., 2020. *BIAS: Transparent reporting of biomedical image analysis challenges*. Med. Image Anal. 66, 101796. https://doi.org/10.1016/j.media.2020.101796\n\n[4] Wiesenfarth, M., Reinke, A., Landman, B.A., Eisenmann, M., Aguilera Saiz, L., Cardoso, M.J., Maier-Hein, L., Kopp-Schneider, A., 2021. *Methods and open-source toolkit for analyzing and visualizing challenge results*. Sci. Rep. 11, 1–15. https://doi.org/10.1038/s41598-021-82017-6\n\n[5] Roß, T., Bruno, P., Reinke, A., Wiesenfarth, M., Koeppel, L., Full, P.M., Pekdemir, B., Godau, P., Trofimova, D., Isensee, F., Moccia, S., Calimeri, F., Müller-Stich, B.P., Kopp-Schneider, A., Maier-Hein, L., 2021. *How can we learn (more) from challenges? A statistical approach to driving future algorithm development*. ArXiv210609302 Cs." } }