{ "id": 325590, "title": "Final Leaderboard, Next Steps, Working Notes Papers, ", "url": "/competitions/snakeclef2022/discussion/325590", "author_name": "", "post_date": "2022-05-17T09:38:11.374132700Z", "votes": 3, "comment_count": 7, "views": 0, "content": "

Dear All,

\n

As the competition deadline has already passed, I would like to thank you for your submissions. We believe that you have utilized various interesting approaches based on your submission description. Thus, we look forward to reviewing your working-note papers describing your method/approach.

\n

But first, let me talk about TROLLS.
\nAs you might know, we were targeted by, let's say, \"trolls\" that did not respect competition rules and used publicly available labels to submit the nearly perfect submission. I was very disappointed seeing that there still exists - excuse my language - such scumbags that do not value publicly available data. In our eyes, publicly available data are important asset for the community and should not be exploited under any circumstances.

\n

BASED ON THIS EXPERIENCE, WE HAVE STARTED AN INVESTIGATION.
\nWe will review every submission, WN papers and Code, and everyone who will be identified as a potential fraud will have to defend his behaviour. If you behaved on the line, do not hesitate to contact us first. You might avoid disqualification.

\n

NEXT STEPS:
\nAs you might know, this competition takes part in two workshops [FGVC + LifeCLEF] / conferences [CVPR + CLEF], and it is expected that:

\n\n

IMPORTANT: This competition is not about exploiting the leaderboard but about presenting/evaluating new bright ideas in the area of Fine-Grained Recognition.

\n

Final Leaderboard and Post competition evaluation
\nAs some concerns about the Kaggle Metric were raised, we plan to export your selected submissions and validate the results on our back-end.
\nFurthermore, any team caught cheating will be disqualified.
\nWe are very sorry about that, but we have no other choice but to wait for the first version of the WN papers before publishing the final standing.

\n

The final standing might change a bit and will be published on the https://www.imageclef.org page.

\n

For a more straightforward evaluation of your methods or \"ablation studies, we will keep the post-competition submissions active.

\n

Appendix
\nFor the best paper/papers, we planned to award prices sponsored by Microsoft. Sadly, Microsoft discontinued their grant and won't provide any credits as a price for the selected publications. Thus, we plan to offer different prices based on the team's preference. What we currently have in mind:

\n\n

Official ranking of LifeCLEF - SnakeCLEF 2022
\nIn order to be part of the official ranking of SnakeCLEF 2022 lab which will be published and used in scientific communication, the teams are required to submit a working note paper to LifeCLEF workshop describing the methods implemented which will be peer reviewed.
\nThis working note must provide sufficient details to allow to reproduce all final submissions.
\nAny run that could not be reproduced thanks to its description in the working notes might be removed from the official publication of the results.
\nWorking notes are published within CEUR-WS proceedings, resulting in an assignment of an individual DOI (URN) and an indexing by many bibliography systems including DBLP.
\nAccording to the CEUR-WS policies, a single-blind, two stage review of the working notes will be conducted by LifeCLEF organizing committee and external reviewers to ensure quality. As an illustration, LifeCLEF 2021 working notes (task overviews and participant working notes) can be found within CLEF 2021 CEUR-WS proceedings.

\n

Below the link gives more information about the templates (latex, doc) to use, the mandatory information to report (about the names, affiliations, etc):

\n\n

IN CASE ANYTHING IS UNCLEAR, DO NOT HESITATE TO CONTACT ME.
\nLukas

", "messages": [ { "id": "1792773", "postDate": "05/17/2022 09:38:11", "content": "

Dear All,

\n

As the competition deadline has already passed, I would like to thank you for your submissions. We believe that you have utilized various interesting approaches based on your submission description. Thus, we look forward to reviewing your working-note papers describing your method/approach.

\n

But first, let me talk about TROLLS.
\nAs you might know, we were targeted by, let's say, \"trolls\" that did not respect competition rules and used publicly available labels to submit the nearly perfect submission. I was very disappointed seeing that there still exists - excuse my language - such scumbags that do not value publicly available data. In our eyes, publicly available data are important asset for the community and should not be exploited under any circumstances.

\n

BASED ON THIS EXPERIENCE, WE HAVE STARTED AN INVESTIGATION.
\nWe will review every submission, WN papers and Code, and everyone who will be identified as a potential fraud will have to defend his behaviour. If you behaved on the line, do not hesitate to contact us first. You might avoid disqualification.

\n

NEXT STEPS:
\nAs you might know, this competition takes part in two workshops [FGVC + LifeCLEF] / conferences [CVPR + CLEF], and it is expected that:

\n\n

IMPORTANT: This competition is not about exploiting the leaderboard but about presenting/evaluating new bright ideas in the area of Fine-Grained Recognition.

\n

Final Leaderboard and Post competition evaluation
\nAs some concerns about the Kaggle Metric were raised, we plan to export your selected submissions and validate the results on our back-end.
\nFurthermore, any team caught cheating will be disqualified.
\nWe are very sorry about that, but we have no other choice but to wait for the first version of the WN papers before publishing the final standing.

\n

The final standing might change a bit and will be published on the https://www.imageclef.org page.

\n

For a more straightforward evaluation of your methods or \"ablation studies, we will keep the post-competition submissions active.

\n

Appendix
\nFor the best paper/papers, we planned to award prices sponsored by Microsoft. Sadly, Microsoft discontinued their grant and won't provide any credits as a price for the selected publications. Thus, we plan to offer different prices based on the team's preference. What we currently have in mind:

\n\n

Official ranking of LifeCLEF - SnakeCLEF 2022
\nIn order to be part of the official ranking of SnakeCLEF 2022 lab which will be published and used in scientific communication, the teams are required to submit a working note paper to LifeCLEF workshop describing the methods implemented which will be peer reviewed.
\nThis working note must provide sufficient details to allow to reproduce all final submissions.
\nAny run that could not be reproduced thanks to its description in the working notes might be removed from the official publication of the results.
\nWorking notes are published within CEUR-WS proceedings, resulting in an assignment of an individual DOI (URN) and an indexing by many bibliography systems including DBLP.
\nAccording to the CEUR-WS policies, a single-blind, two stage review of the working notes will be conducted by LifeCLEF organizing committee and external reviewers to ensure quality. As an illustration, LifeCLEF 2021 working notes (task overviews and participant working notes) can be found within CLEF 2021 CEUR-WS proceedings.

\n

Below the link gives more information about the templates (latex, doc) to use, the mandatory information to report (about the names, affiliations, etc):

\n\n

IN CASE ANYTHING IS UNCLEAR, DO NOT HESITATE TO CONTACT ME.
\nLukas

", "rawMarkdown": "Dear All,\n\nAs the competition deadline has already passed, I would like to thank you for your submissions. We believe that you have utilized various interesting approaches based on your submission description. Thus, we look forward to reviewing your working-note papers describing your method/approach. \n\nBut first, let me talk about TROLLS. \nAs you might know, we were targeted by, let's say, \"trolls\" that did not respect competition rules and used publicly available labels to submit the nearly perfect submission. I was very disappointed seeing that there still exists - excuse my language - such scumbags that do not value publicly available data. In our eyes, publicly available data are important asset for the community and should not be exploited under any circumstances. \n\nBASED ON THIS EXPERIENCE, WE HAVE STARTED AN INVESTIGATION.\nWe will review every submission, WN papers and Code, and everyone who will be identified as a potential fraud will have to defend his behaviour. If you behaved on the line, do not hesitate to contact us first. You might avoid disqualification.\n\nNEXT STEPS:\nAs you might know, this competition takes part in two workshops [FGVC + LifeCLEF] / conferences [CVPR + CLEF], and it is expected that:\n- Participants write a scientific report / working notes(WN) article about their work to sustain reproducibility.\n- Selected participants will present their work at the CVPR and CLEF conferences.\nPlease refer to the [LINK](https://www.kaggle.com/competitions/snakeclef2022/overview/clef2022-lifeclef-workshop) and follow the provided guidelines if you would like to submit your WN paper. If you run into any problems, do not hesitate to contact me. Preferable here in the discussion so everyone can read it :)\n\nIMPORTANT: This competition is not about exploiting the leaderboard but about presenting/evaluating new bright ideas in the area of Fine-Grained Recognition.\n\n**Final Leaderboard and Post competition evaluation**\nAs some concerns about the Kaggle Metric were raised, we plan to export your selected submissions and validate the results on our back-end. \nFurthermore, any team caught cheating will be disqualified. \nWe are very sorry about that, but we have no other choice but to wait for the first version of the WN papers before publishing the final standing. \n\nThe final standing might change a bit and will be published on the https://www.imageclef.org page.\n\nFor a more straightforward evaluation of your methods or \"ablation studies, we will keep the post-competition submissions active.\n\n**Appendix**\nFor the best paper/papers, we planned to award prices sponsored by Microsoft. Sadly, Microsoft discontinued their [grant](https://www.microsoft.com/en-us/ai/ai-for-earth) and won't provide any credits as a price for the selected publications. Thus, we plan to offer different prices based on the team's preference. What we currently have in mind:\n- CLEF Registration Fee\n- CVPR Registration Fee\n- [Summer Scholl Registration Fee](https://cmp.felk.cvut.cz/summerschool2022/)\n- Internship under [prof. Jiri Matas](https://scholar.google.com/citations?user=EJCNY6QAAAAJ&hl=cs&oi=ao) at the Czech Technical Faculty\n- Joint Journal Publication + Open Access fee\n- \"Amenities\".\n\n**Official ranking of LifeCLEF - SnakeCLEF 2022**\nIn order to be part of the official ranking of SnakeCLEF 2022 lab which will be published and used in scientific communication, the teams are required to submit a working note paper to LifeCLEF workshop describing the methods implemented which will be peer reviewed.\nThis working note must provide sufficient details to allow to reproduce all final submissions.\nAny run that could not be reproduced thanks to its description in the working notes might be removed from the official publication of the results.\nWorking notes are published within CEUR-WS proceedings, resulting in an assignment of an individual DOI (URN) and an indexing by many bibliography systems including DBLP.\nAccording to the CEUR-WS policies, a single-blind, two stage review of the working notes will be conducted by LifeCLEF organizing committee and external reviewers to ensure quality. As an illustration, LifeCLEF 2021 working notes (task overviews and participant working notes) can be found within [CLEF 2021 CEUR-WS proceedings] (http://ceur-ws.org/Vol-2936/).\n\nBelow the link gives more information about the templates (latex, doc) to use, the mandatory information to report (about the names, affiliations, etc):\n\n- [CLEF instructions for authors](https://clef2022.clef-initiative.eu/index.php?page=Pages/instructions_for_authors.html)\n- [The direct link to submit your working note through easychair](https://easychair.org/my/conference?conf=clef2022)\n\nIN CASE ANYTHING IS UNCLEAR, DO NOT HESITATE TO CONTACT ME.\nLukas", "votes": null }, { "id": "1802695", "postDate": "05/27/2022 03:44:50", "content": "

Dear Lukas：
\n I trained my model with extra SnakeCLEF 2021 datasets, and I notice that the rules updated some days ago: Using external observation data (such as data from GBIF, etc.) is forbidden. The rules are not listed when the competition started, and I have not noticed the new rules until recent days. I want to know if my submission is invalid accroding to the lasted rules.

", "rawMarkdown": "Dear Lukas：\n I trained my model with extra SnakeCLEF 2021 datasets, and I notice that the rules updated some days ago: Using external observation data (such as data from GBIF, etc.) is forbidden. The rules are not listed when the competition started, and I have not noticed the new rules until recent days. I want to know if my submission is invalid accroding to the lasted rules.", "votes": null }, { "id": "1805288", "postDate": "05/30/2022 03:38:03", "content": "

Dear @picekl ,

\n

I have submitted the Working Notes Paper, but there still exist some questions:

\n

1) Where and how to provide reproducible training code?

\n

2) I found that the following required registration had been closed, so do we have to register to LifeCLEF lab?

\n
\n

Being part of scientific research, the participants are encouraged to participate to the LifeCLEF lab by registering to it using this form (and checking \"Task 4 - SnakeCLEF\" of LifeCLEF).

\n
\n

Thanks.

", "rawMarkdown": "Dear @picekl ,\n\nI have submitted the Working Notes Paper, but there still exist some questions:\n\n1) Where and how to provide reproducible training code?\n\n2) I found that the following required registration had been closed, so do we have to register to LifeCLEF lab? \n> Being part of scientific research, the participants are encouraged to participate to the LifeCLEF lab by registering to it using this form (and checking \"Task 4 - SnakeCLEF\" of LifeCLEF).\n\nThanks.", "votes": null }, { "id": "1806796", "postDate": "05/31/2022 13:20:50", "content": "

Hi @wuyouzc,

\n

You can use Kaggle notebook or GitHub to provide the code.
\nAbout the registration. Please send me an email with the personal information you added to EasyChair.

\n

Lukas

", "rawMarkdown": "Hi @wuyouzc,\n\nYou can use Kaggle notebook or GitHub to provide the code.\nAbout the registration. Please send me an email with the personal information you added to EasyChair.\n\nLukas", "votes": null }, { "id": "1806811", "postDate": "05/31/2022 13:31:14", "content": "

Dear @dhuziyu,

\n

Using external data was forbidden since the beginning. No changes were made to the Rules Section.
\nThe general rule is that participants should only use the provided training and validation images for training models to classify the test images.

\n

Besides that, the Rules section includes the following sentence \"In case of doubt, please contact us!\".
\nI'm really sorry but you had a plenty of time to step further and ask here on Forum or via e-mail.

\n

Best,
\nLukas

", "rawMarkdown": "Dear @dhuziyu,\n\nUsing external data was forbidden since the beginning. No changes were made to the Rules Section.\nThe general rule is that participants should only use the provided training and validation images for training models to classify the test images.\n\nBesides that, the Rules section includes the following sentence \"In case of doubt, please contact us!\".\nI'm really sorry but you had a plenty of time to step further and ask here on Forum or via e-mail.\n\nBest,\nLukas", "votes": null }, { "id": "1814996", "postDate": "06/08/2022 14:43:53", "content": "

Please refer to my topic in discussion, I need your help, your upvote, thanks. @dhuziyu @wuyouzc

", "rawMarkdown": "Please refer to my topic in discussion, I need your help, your upvote, thanks. @dhuziyu @wuyouzc", "votes": null }, { "id": "1842492", "postDate": "07/04/2022 04:18:41", "content": "

Dear @picekl ,

\n

When will the camera ready Working Note Papers be available ?

", "rawMarkdown": "Dear @picekl ,\n\nWhen will the camera ready Working Note Papers be available ?", "votes": null }, { "id": "1859407", "postDate": "07/17/2022 15:50:00", "content": "

Dear @picekl ，
\nWhere could I find the final leaderboard and working notes?

", "rawMarkdown": "Dear @picekl ，\nWhere could I find the final leaderboard and working notes?", "votes": null } ], "comments": [ { "id": 1802695, "author_name": "dhuziyu", "author_url": "", "post_date": "05/27/2022 03:44:50", "content": "

Dear Lukas：
\n I trained my model with extra SnakeCLEF 2021 datasets, and I notice that the rules updated some days ago: Using external observation data (such as data from GBIF, etc.) is forbidden. The rules are not listed when the competition started, and I have not noticed the new rules until recent days. I want to know if my submission is invalid accroding to the lasted rules.

", "votes": null, "replies": [ { "id": 1806811, "author_name": "picekl", "author_url": "", "post_date": "05/31/2022 13:31:14", "content": "

Dear @dhuziyu,

\n

Using external data was forbidden since the beginning. No changes were made to the Rules Section.
\nThe general rule is that participants should only use the provided training and validation images for training models to classify the test images.

\n

Besides that, the Rules section includes the following sentence \"In case of doubt, please contact us!\".
\nI'm really sorry but you had a plenty of time to step further and ask here on Forum or via e-mail.

\n

Best,
\nLukas

", "votes": null, "replies": [] } ] }, { "id": 1805288, "author_name": "wuyouzc", "author_url": "", "post_date": "05/30/2022 03:38:03", "content": "

Dear @picekl ,

\n

I have submitted the Working Notes Paper, but there still exist some questions:

\n

1) Where and how to provide reproducible training code?

\n

2) I found that the following required registration had been closed, so do we have to register to LifeCLEF lab?

\n
\n

Being part of scientific research, the participants are encouraged to participate to the LifeCLEF lab by registering to it using this form (and checking \"Task 4 - SnakeCLEF\" of LifeCLEF).

\n
\n

Thanks.

", "votes": null, "replies": [ { "id": 1806796, "author_name": "picekl", "author_url": "", "post_date": "05/31/2022 13:20:50", "content": "

Hi @wuyouzc,

\n

You can use Kaggle notebook or GitHub to provide the code.
\nAbout the registration. Please send me an email with the personal information you added to EasyChair.

\n

Lukas

", "votes": null, "replies": [] } ] }, { "id": 1814996, "author_name": "abuduori", "author_url": "", "post_date": "06/08/2022 14:43:53", "content": "

Please refer to my topic in discussion, I need your help, your upvote, thanks. @dhuziyu @wuyouzc

", "votes": null, "replies": [] }, { "id": 1842492, "author_name": "wuyouzc", "author_url": "", "post_date": "07/04/2022 04:18:41", "content": "

Dear @picekl ,

\n

When will the camera ready Working Note Papers be available ?

", "votes": null, "replies": [] }, { "id": 1859407, "author_name": "plcrtkqrm", "author_url": "", "post_date": "07/17/2022 15:50:00", "content": "

Dear @picekl ，
\nWhere could I find the final leaderboard and working notes?

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1792773": "Dear All,\n\nAs the competition deadline has already passed, I would like to thank you for your submissions. We believe that you have utilized various interesting approaches based on your submission description. Thus, we look forward to reviewing your working-note papers describing your method/approach. \n\nBut first, let me talk about TROLLS. \nAs you might know, we were targeted by, let's say, \"trolls\" that did not respect competition rules and used publicly available labels to submit the nearly perfect submission. I was very disappointed seeing that there still exists - excuse my language - such scumbags that do not value publicly available data. In our eyes, publicly available data are important asset for the community and should not be exploited under any circumstances. \n\nBASED ON THIS EXPERIENCE, WE HAVE STARTED AN INVESTIGATION.\nWe will review every submission, WN papers and Code, and everyone who will be identified as a potential fraud will have to defend his behaviour. If you behaved on the line, do not hesitate to contact us first. You might avoid disqualification.\n\nNEXT STEPS:\nAs you might know, this competition takes part in two workshops [FGVC + LifeCLEF] / conferences [CVPR + CLEF], and it is expected that:\n- Participants write a scientific report / working notes(WN) article about their work to sustain reproducibility.\n- Selected participants will present their work at the CVPR and CLEF conferences.\nPlease refer to the [LINK](https://www.kaggle.com/competitions/snakeclef2022/overview/clef2022-lifeclef-workshop) and follow the provided guidelines if you would like to submit your WN paper. If you run into any problems, do not hesitate to contact me. Preferable here in the discussion so everyone can read it :)\n\nIMPORTANT: This competition is not about exploiting the leaderboard but about presenting/evaluating new bright ideas in the area of Fine-Grained Recognition.\n\n**Final Leaderboard and Post competition evaluation**\nAs some concerns about the Kaggle Metric were raised, we plan to export your selected submissions and validate the results on our back-end. \nFurthermore, any team caught cheating will be disqualified. \nWe are very sorry about that, but we have no other choice but to wait for the first version of the WN papers before publishing the final standing. \n\nThe final standing might change a bit and will be published on the https://www.imageclef.org page.\n\nFor a more straightforward evaluation of your methods or \"ablation studies, we will keep the post-competition submissions active.\n\n**Appendix**\nFor the best paper/papers, we planned to award prices sponsored by Microsoft. Sadly, Microsoft discontinued their [grant](https://www.microsoft.com/en-us/ai/ai-for-earth) and won't provide any credits as a price for the selected publications. Thus, we plan to offer different prices based on the team's preference. What we currently have in mind:\n- CLEF Registration Fee\n- CVPR Registration Fee\n- [Summer Scholl Registration Fee](https://cmp.felk.cvut.cz/summerschool2022/)\n- Internship under [prof. Jiri Matas](https://scholar.google.com/citations?user=EJCNY6QAAAAJ&hl=cs&oi=ao) at the Czech Technical Faculty\n- Joint Journal Publication + Open Access fee\n- \"Amenities\".\n\n**Official ranking of LifeCLEF - SnakeCLEF 2022**\nIn order to be part of the official ranking of SnakeCLEF 2022 lab which will be published and used in scientific communication, the teams are required to submit a working note paper to LifeCLEF workshop describing the methods implemented which will be peer reviewed.\nThis working note must provide sufficient details to allow to reproduce all final submissions.\nAny run that could not be reproduced thanks to its description in the working notes might be removed from the official publication of the results.\nWorking notes are published within CEUR-WS proceedings, resulting in an assignment of an individual DOI (URN) and an indexing by many bibliography systems including DBLP.\nAccording to the CEUR-WS policies, a single-blind, two stage review of the working notes will be conducted by LifeCLEF organizing committee and external reviewers to ensure quality. As an illustration, LifeCLEF 2021 working notes (task overviews and participant working notes) can be found within [CLEF 2021 CEUR-WS proceedings] (http://ceur-ws.org/Vol-2936/).\n\nBelow the link gives more information about the templates (latex, doc) to use, the mandatory information to report (about the names, affiliations, etc):\n\n- [CLEF instructions for authors](https://clef2022.clef-initiative.eu/index.php?page=Pages/instructions_for_authors.html)\n- [The direct link to submit your working note through easychair](https://easychair.org/my/conference?conf=clef2022)\n\nIN CASE ANYTHING IS UNCLEAR, DO NOT HESITATE TO CONTACT ME.\nLukas", "1802695": "Dear Lukas：\n I trained my model with extra SnakeCLEF 2021 datasets, and I notice that the rules updated some days ago: Using external observation data (such as data from GBIF, etc.) is forbidden. The rules are not listed when the competition started, and I have not noticed the new rules until recent days. I want to know if my submission is invalid accroding to the lasted rules.", "1805288": "Dear @picekl ,\n\nI have submitted the Working Notes Paper, but there still exist some questions:\n\n1) Where and how to provide reproducible training code?\n\n2) I found that the following required registration had been closed, so do we have to register to LifeCLEF lab? \n> Being part of scientific research, the participants are encouraged to participate to the LifeCLEF lab by registering to it using this form (and checking \"Task 4 - SnakeCLEF\" of LifeCLEF).\n\nThanks.", "1806796": "Hi @wuyouzc,\n\nYou can use Kaggle notebook or GitHub to provide the code.\nAbout the registration. Please send me an email with the personal information you added to EasyChair.\n\nLukas", "1806811": "Dear @dhuziyu,\n\nUsing external data was forbidden since the beginning. No changes were made to the Rules Section.\nThe general rule is that participants should only use the provided training and validation images for training models to classify the test images.\n\nBesides that, the Rules section includes the following sentence \"In case of doubt, please contact us!\".\nI'm really sorry but you had a plenty of time to step further and ask here on Forum or via e-mail.\n\nBest,\nLukas", "1814996": "Please refer to my topic in discussion, I need your help, your upvote, thanks. @dhuziyu @wuyouzc", "1842492": "Dear @picekl ,\n\nWhen will the camera ready Working Note Papers be available ?", "1859407": "Dear @picekl ，\nWhere could I find the final leaderboard and working notes?" }, "source": "meta" }