As we approach the home stretch of the challenge, we would like to announce a Judges’ Award.
\n\nThe goal of the Judges’ Award is to encourage contestants to use novel and problem-specific approaches which leverage knowledge of the audio domain. Another key factor is computational efficiency: we want to promote models that can fit within reasonable resource constraints, as opposed to very large models that may merely memorize a dataset and may not fit within typical deployment environments on mobile devices, for example.
\n\nRules:
\n\nFor the Judges’ Award, submissions will be evaluated according to:
\n\nThe following strict rules apply for the Judges’ Award:
\n\nRequirements:
\n\nPrize:
\n\nAt the close of the competition, submissions meeting the Requirements will be considered for the Judges’ Award. The “prize” consists of the public announcement of the winner in this thread by the organizers, along with a brief description of the most interesting aspects of the system. There is no monetary prize.
\n\n
\nPlease feel free to post your questions in this thread. And please check this other thread for further clarification on the different type of submissions in the Freesound General-Purpose Audio Tagging Challenge.
Good luck!
\n\nFrederic Font, Eduardo Fonseca (Freesound, MTG-UPF)
\nDan Ellis, Manoj Plakal (Google Machine Perception)
As we approach the home stretch of the challenge, we would like to announce a Judges’ Award.
\n\nThe goal of the Judges’ Award is to encourage contestants to use novel and problem-specific approaches which leverage knowledge of the audio domain. Another key factor is computational efficiency: we want to promote models that can fit within reasonable resource constraints, as opposed to very large models that may merely memorize a dataset and may not fit within typical deployment environments on mobile devices, for example.
\n\nRules:
\n\nFor the Judges’ Award, submissions will be evaluated according to:
\n\nThe following strict rules apply for the Judges’ Award:
\n\nRequirements:
\n\nPrize:
\n\nAt the close of the competition, submissions meeting the Requirements will be considered for the Judges’ Award. The “prize” consists of the public announcement of the winner in this thread by the organizers, along with a brief description of the most interesting aspects of the system. There is no monetary prize.
\n\n
\nPlease feel free to post your questions in this thread. And please check this other thread for further clarification on the different type of submissions in the Freesound General-Purpose Audio Tagging Challenge.
Good luck!
\n\nFrederic Font, Eduardo Fonseca (Freesound, MTG-UPF)
\nDan Ellis, Manoj Plakal (Google Machine Perception)
Same question here, is it possible to use the ImageNet-based pretrained model? It is based on the Imagenet 2014 image dataset.
", "rawMarkdown": "Same question here, is it possible to use the ImageNet-based pretrained model? It is based on the Imagenet 2014 image dataset.", "votes": null }, { "id": "356238", "postDate": "07/13/2018 06:55:00", "content": "Hello, thanks for asking!
\n\nI am afraid pre-trained models are not allowed for the Judges' Award, since they have used external data during its training (Imagenet in this case). One of the goals of the Judges' Award is to see how far we can get using the provided FSDKaggle2018 dataset without using any additional external data in any form.
\n\nHowever, doing data augmentation of FSDKaggle2018 without the use of external data is allowed (e.g. by using techniques such as pitch shifting or time stretching).
\n\nhope that helps!
", "rawMarkdown": "Hello, thanks for asking!\n\nI am afraid pre-trained models are not allowed for the Judges' Award, since they have used external data during its training (Imagenet in this case). One of the goals of the Judges' Award is to see how far we can get using the provided **FSDKaggle2018** dataset without using any additional external data in any form. \n\nHowever, doing data augmentation of FSDKaggle2018 without the use of external data is allowed (e.g. by using techniques such as pitch shifting or time stretching).\n\nhope that helps!", "votes": null }, { "id": "356263", "postDate": "07/13/2018 08:00:05", "content": "Could you please clarify: what can be used for the competition? In your previous reply (https://www.kaggle.com/c/freesound-audio-tagging/discussion/53340), it seems that it is OK to employ the pre-trained models (such as ResNet, Inception).
\n\nFor me, it seems that, presently, it is OK to use pre-trained models for the competition, but it is not eligible for the award?
", "rawMarkdown": "Could you please clarify: what can be used for the competition? In your previous reply (https://www.kaggle.com/c/freesound-audio-tagging/discussion/53340), it seems that it is OK to employ the pre-trained models (such as ResNet, Inception). \n\nFor me, it seems that, presently, it is OK to use pre-trained models for the competition, but it is not eligible for the award?", "votes": null }, { "id": "356484", "postDate": "07/13/2018 17:16:06", "content": "Hi again,
\n\nyou are right.
\n\nIn this competition, usage of pre-trained models is allowed.
\n\nHowever, for the Judges' Award (which is a special prize within the competition), a slightly stricter set of Rules apply (which are specified at the top of this thread). One of them is that external data (including pre-trained models) is not allowed in any form. Therefore, when considering submissions for the Judges' award, those using pre-trained models will not be considered.
\n\nNow, if you are interested in using pre-trained models for your systems, but you would also like to be eligible for the Judges' Award, one possibility is to make 2 final submissions. According to the competition Rules, the maximum number of submissions eligible for the final private leaderboard is 2. And participants can hand-select the eligible submissions.
\n\nSo, you could select:
\ni) one final submission that complies with the Rules of the Judges' Award, and
\nii) another submission that uses pre-trained models. This one will only be considered for being the competition winner.
If you consider this double option, please specify it clearly in the technical report as the the judging of the Judges' award will be based primarily upon it.
\n\nIf you have any further questions, do not hesitate to ask them here. Thanks!
", "rawMarkdown": "Hi again,\n\nyou are right.\n\nIn this competition, usage of pre-trained models is allowed.\n\nHowever, **for the Judges' Award** (which is a special prize within the competition), a slightly stricter set of Rules apply (which are specified at the top of this thread). One of them is that external data (including pre-trained models) is not allowed in any form. Therefore, when considering submissions **for the Judges' award**, those using pre-trained models will not be considered.\n\nNow, if you are interested in using pre-trained models for your systems, but you would also like to be eligible for the Judges' Award, one possibility is to make 2 final submissions. According to the competition Rules, the maximum number of submissions eligible for the final private leaderboard is 2. And participants can hand-select the eligible submissions.\n\nSo, you could select: \ni) one final submission that complies with the Rules of the Judges' Award, and \nii) another submission that uses pre-trained models. This one will only be considered for being the competition winner.\n\nIf you consider this *double* option, please specify it clearly in the technical report as the the judging of the Judges' award will be based primarily upon it.\n\nIf you have any further questions, do not hesitate to ask them here. Thanks!", "votes": null }, { "id": "356602", "postDate": "07/14/2018 01:01:32", "content": "Thanks for the clarification, and it sounds great!~\nI will use one submission without any pretrained models, and another one with the pretrained ImageNet-Based model.
", "rawMarkdown": "Thanks for the clarification, and it sounds great!~\nI will use one submission without any pretrained models, and another one with the pretrained ImageNet-Based model.", "votes": null }, { "id": "357307", "postDate": "07/15/2018 17:56:03", "content": "Hi again,
\n\nAfter discussing with other organizers, we think that there is a more flexible possibility.
\n\nRegarding the two final submissions that each participant can select in this competition, there is no need to select one of each (i.e. one for the private leaderboard, and another for the Judges' Award). The two allowed final submissions can be used to achieve the best possible score on the private leaderboard (using any kind of method according to the Competition Rules, for example pre-trained models).
\n\nAnd then, additionally, another submission can be designated to be considered for the Judges' Award (using the approaches allowed according to the Judges' Award Rules , which are specified at the top of this thread). In order to designate a submission to be considered for the Judges' Award, participants must specify in the technical report the submission file (filename.csv) corresponding to the candidate model.
\n\nHope this clarifies! As usual, please ask any doubt.
", "rawMarkdown": "Hi again, \n\nAfter discussing with other organizers, we think that there is a more flexible possibility.\n\nRegarding the two final submissions that each participant can select in this competition, there is no need to select one of each (i.e. one for the private leaderboard, and another for the Judges' Award). **The two allowed final submissions can be used to achieve the best possible score on the private leaderboard (using any kind of method according to the [Competition Rules][1], for example pre-trained models).** \n\nAnd then, **additionally, another submission can be designated to be considered for the Judges' Award (using the approaches allowed according to the Judges' Award Rules** , which are specified at the top of this thread). In order to designate a submission to be considered for the Judges' Award, participants must specify in the technical report the submission file (*filename.csv*) corresponding to the candidate model.\n\nHope this clarifies! As usual, please ask any doubt.\n\n\n [1]: https://www.kaggle.com/c/freesound-audio-tagging/rules", "votes": null }, { "id": "357918", "postDate": "07/17/2018 06:29:22", "content": "This is a very good initiative! One question: When is the deadline for the write-up and technical report submission? Both for the judges award and the research track?
", "rawMarkdown": "This is a very good initiative! One question: When is the deadline for the write-up and technical report submission? Both for the judges award and the research track?", "votes": null }, { "id": "358025", "postDate": "07/17/2018 11:38:32", "content": "Glad to hear that you like it!
\n\nShort answer: deadline is July 31st, as specified in this thread:
\nhttps://www.kaggle.com/c/freesound-audio-tagging/discussion/59933
\n(I will include the deadline in the current thread too for clarity).
Long Answer: For the Judges' Award, participants must submit a technical report and a metainformation file through the DCASE submission system using the DCASE template. In addition, participants must submit the code and the (same) technical report to Kaggle. The deadline for all of this is July 31st.
\nMore info here: http://dcase.community/challenge2018/submission
\nAdditionally, those participants interested in submitting a paper to the DCASE Workshop can re-use the technical report. In this case, the paper must respect the structure of a scientific publication. Submitting a workshop paper is not required for the Judges' Award. We just give the info here for completeness. The deadline for workshop paper submission is July 31st.
\nMore info here: http://dcase.community/workshop2018/call-for-papers
Hope this helps!
\n\nP.D. By the way, Gyat, what do you mean exactly by 'research track'? Thanks!
", "rawMarkdown": "Glad to hear that you like it!\n\n**Short answer:** deadline is July 31st, as specified in this thread: \nhttps://www.kaggle.com/c/freesound-audio-tagging/discussion/59933 \n(I will include the deadline in the current thread too for clarity).\n\n**Long Answer:** For the Judges' Award, participants must submit a **technical report and a metainformation file** through the DCASE submission system using the DCASE template. In addition, participants must submit the code and the (same) technical report to Kaggle. The deadline for all of this is July 31st. \nMore info here: http://dcase.community/challenge2018/submission\n\nThank you for your reply. I thought the deadlines for Judge's award and usual research paper submission is different.
", "rawMarkdown": "Thank you for your reply. I thought the deadlines for Judge's award and usual research paper submission is different.", "votes": null }, { "id": "362970", "postDate": "07/27/2018 13:59:10", "content": "@Eduardo There's something I am a little confused about, and need your help. So the Kaggle system allows us to select a maximum of two submissions. However, the DCASE system allows up to four submissions.
\n\nNow, if I understand correctly, we can describe all the methods used to generate the four submissions, in a single technical report, correct?
In that case, in order for a submission to qualify for the judges award, where do we mark it? Do we mark it in the same report, saying something like: This is the submission for the judge's consideration? Or do we need to create a separate report for that and notify you all for the same?
Requesting your clarifications on this.
", "rawMarkdown": "Eduardo There's something I am a little confused about, and need your help. So the Kaggle system allows us to select a maximum of two submissions. However, the DCASE system allows up to four submissions. \n\n1. Now, if I understand correctly, we can describe all the methods used to generate the four submissions, in a single technical report, correct? \n\n2. In that case, in order for a submission to qualify for the judges award, where do we mark it? Do we mark it in the same report, saying something like: This is the submission for the judge's consideration? Or do we need to create a separate report for that and notify you all for the same?\n\nRequesting your clarifications on this.", "votes": null }, { "id": "363004", "postDate": "07/27/2018 15:21:56", "content": "Hei Gyat!
\n\nthe DCASE system allows up to four submissions, but in our competition, only 2 are allowed, as per the Rules https://www.kaggle.com/c/freesound-audio-tagging/rules
\n\nThen, answering to your questions:
\n\nFinally, let me emphasize what was explained a couple of posts above:
\n\nhope that helps
", "rawMarkdown": "Hei Gyat!\n\nthe DCASE system allows up to four submissions, but **in our competition, only 2 are allowed**, as per the Rules https://www.kaggle.com/c/freesound-audio-tagging/rules \n\nThen, answering to your questions:\n\n1. All the methods used to generate the **two** submissions can be described in a single technical report.\n2. For a submission to qualify for the Judges' Award, you must describe the candidate model and specify the corresponding submission file (*filename.csv*) in the technical report, as specified at the very top of this thread (please read those rules carefully). No additional report is needed. You can use a single report for everything. And yes, as you suggested, please mention that \"this model/submission is for the Judges' Award\".\n\nFinally, let me emphasize what was explained a couple of posts above: \n\n- The two allowed final submissions can be used to achieve the best possible score on the private leaderboard (using any kind of method according to the Competition Rules)\n- additionally, another submission can be designated to be considered for the Judges' Award (using the approaches allowed according to the Judges' Award Rules)\n\nhope that helps", "votes": null }, { "id": "364588", "postDate": "07/31/2018 20:22:40", "content": "Hi all, just a gentle reminder that submissions to the Judges' Award must include in the technical report:
\n\nin order to facilitate the review process.
\n\nthanks!
", "rawMarkdown": "Hi all, just a gentle reminder that submissions to the Judges' Award must include in the technical report:\n\n- the submission filename (filename.csv) corresponding to the candidate model \n- and the Team Name\n\nin order to facilitate the review process.\n\nthanks!", "votes": null }, { "id": "364740", "postDate": "08/01/2018 06:35:01", "content": "I'm confused where I submit a submission file for Judges’ Award. There are only 2 slots for final submission and you said that we can submit best performance sets for Kaggle submission.
\n\nWhich one is a right way to submit external submission for Judges’ Award?
\n\nI just noticed that the first one is correct.
\n\nTherefore, we should submit files below,
\n\nAm I right?
", "rawMarkdown": "I just noticed that the first one is correct. \n\nTherefore, we should submit files below,\n\n- technical report, 3 submissions csv files, and their corresponding yaml files for DCASE submission system\n- technical report, model & codes for Kaggle team tab\n\nAm I right?", "votes": null }, { "id": "364827", "postDate": "08/01/2018 10:13:55", "content": "yes, you're right
", "rawMarkdown": "yes, you're right", "votes": null }, { "id": "364914", "postDate": "08/01/2018 14:22:29", "content": "@Eduardo,
\n\nWould the judges consider a submission file which had already been submitted to kaggle before the competition end (but not mentioned in the submitted technical report) with the same model and same features, minus one small change in pre-processing step?
\n\nUnder the same model and same features, the single model submission scores 0.915 as opposed to 0.907 (which was selected based on the public LB performance). :'(
", "rawMarkdown": "Eduardo, \n\nWould the judges consider a submission file which had already been submitted to kaggle before the competition end (but not mentioned in the submitted technical report) with the same model and same features, minus one small change in pre-processing step? \n\nUnder the same model and same features, the single model submission scores 0.915 as opposed to 0.907 (which was selected based on the public LB performance). :'(", "votes": null }, { "id": "365538", "postDate": "08/02/2018 21:19:20", "content": "Hi Gyat,
\n\nI'm afraid that we cannot accept that, as it could be somewhat unfair to other participants. Besides, taking into account the implications of being ranked live on the public LB but eventually being ranked on the private LB is also part of the competition.
\n\nLastly, if you say that the approaches and scores are so similar, it is likely that our judgment will be very similar too.
\n\nThanks!
", "rawMarkdown": "Hi Gyat, \n\nI'm afraid that we cannot accept that, as it could be somewhat unfair to other participants. Besides, taking into account the implications of being ranked live on the public LB but eventually being ranked on the private LB is also part of the competition.\n\nLastly, if you say that the approaches and scores are so similar, it is likely that our judgment will be very similar too.\n\nThanks!", "votes": null }, { "id": "388903", "postDate": "09/17/2018 19:48:14", "content": "Hello, are the results in for the judge's award?
", "rawMarkdown": "Hello, are the results in for the judge's award?", "votes": null }, { "id": "388929", "postDate": "09/17/2018 20:46:12", "content": "coming soon!
", "rawMarkdown": "coming soon!", "votes": null } ], "comments": [ { "id": 356200, "author_name": "kelexu", "author_url": "", "post_date": "07/13/2018 05:10:57", "content": "Same question here, is it possible to use the ImageNet-based pretrained model? It is based on the Imagenet 2014 image dataset.
", "votes": null, "replies": [ { "id": 356238, "author_name": "eduardofonseca", "author_url": "", "post_date": "07/13/2018 06:55:00", "content": "Hello, thanks for asking!
\n\nI am afraid pre-trained models are not allowed for the Judges' Award, since they have used external data during its training (Imagenet in this case). One of the goals of the Judges' Award is to see how far we can get using the provided FSDKaggle2018 dataset without using any additional external data in any form.
\n\nHowever, doing data augmentation of FSDKaggle2018 without the use of external data is allowed (e.g. by using techniques such as pitch shifting or time stretching).
\n\nhope that helps!
", "votes": null, "replies": [] }, { "id": 356263, "author_name": "kelexu", "author_url": "", "post_date": "07/13/2018 08:00:05", "content": "Could you please clarify: what can be used for the competition? In your previous reply (https://www.kaggle.com/c/freesound-audio-tagging/discussion/53340), it seems that it is OK to employ the pre-trained models (such as ResNet, Inception).
\n\nFor me, it seems that, presently, it is OK to use pre-trained models for the competition, but it is not eligible for the award?
", "votes": null, "replies": [] }, { "id": 356484, "author_name": "eduardofonseca", "author_url": "", "post_date": "07/13/2018 17:16:06", "content": "Hi again,
\n\nyou are right.
\n\nIn this competition, usage of pre-trained models is allowed.
\n\nHowever, for the Judges' Award (which is a special prize within the competition), a slightly stricter set of Rules apply (which are specified at the top of this thread). One of them is that external data (including pre-trained models) is not allowed in any form. Therefore, when considering submissions for the Judges' award, those using pre-trained models will not be considered.
\n\nNow, if you are interested in using pre-trained models for your systems, but you would also like to be eligible for the Judges' Award, one possibility is to make 2 final submissions. According to the competition Rules, the maximum number of submissions eligible for the final private leaderboard is 2. And participants can hand-select the eligible submissions.
\n\nSo, you could select:
\ni) one final submission that complies with the Rules of the Judges' Award, and
\nii) another submission that uses pre-trained models. This one will only be considered for being the competition winner.
If you consider this double option, please specify it clearly in the technical report as the the judging of the Judges' award will be based primarily upon it.
\n\nIf you have any further questions, do not hesitate to ask them here. Thanks!
", "votes": null, "replies": [] }, { "id": 356602, "author_name": "kelexu", "author_url": "", "post_date": "07/14/2018 01:01:32", "content": "Thanks for the clarification, and it sounds great!~\nI will use one submission without any pretrained models, and another one with the pretrained ImageNet-Based model.
", "votes": null, "replies": [] }, { "id": 357307, "author_name": "eduardofonseca", "author_url": "", "post_date": "07/15/2018 17:56:03", "content": "Hi again,
\n\nAfter discussing with other organizers, we think that there is a more flexible possibility.
\n\nRegarding the two final submissions that each participant can select in this competition, there is no need to select one of each (i.e. one for the private leaderboard, and another for the Judges' Award). The two allowed final submissions can be used to achieve the best possible score on the private leaderboard (using any kind of method according to the Competition Rules, for example pre-trained models).
\n\nAnd then, additionally, another submission can be designated to be considered for the Judges' Award (using the approaches allowed according to the Judges' Award Rules , which are specified at the top of this thread). In order to designate a submission to be considered for the Judges' Award, participants must specify in the technical report the submission file (filename.csv) corresponding to the candidate model.
\n\nHope this clarifies! As usual, please ask any doubt.
", "votes": null, "replies": [] } ] }, { "id": 357918, "author_name": "gyat2017", "author_url": "", "post_date": "07/17/2018 06:29:22", "content": "This is a very good initiative! One question: When is the deadline for the write-up and technical report submission? Both for the judges award and the research track?
", "votes": null, "replies": [ { "id": 358025, "author_name": "eduardofonseca", "author_url": "", "post_date": "07/17/2018 11:38:32", "content": "Glad to hear that you like it!
\n\nShort answer: deadline is July 31st, as specified in this thread:
\nhttps://www.kaggle.com/c/freesound-audio-tagging/discussion/59933
\n(I will include the deadline in the current thread too for clarity).
Long Answer: For the Judges' Award, participants must submit a technical report and a metainformation file through the DCASE submission system using the DCASE template. In addition, participants must submit the code and the (same) technical report to Kaggle. The deadline for all of this is July 31st.
\nMore info here: http://dcase.community/challenge2018/submission
\nAdditionally, those participants interested in submitting a paper to the DCASE Workshop can re-use the technical report. In this case, the paper must respect the structure of a scientific publication. Submitting a workshop paper is not required for the Judges' Award. We just give the info here for completeness. The deadline for workshop paper submission is July 31st.
\nMore info here: http://dcase.community/workshop2018/call-for-papers
Hope this helps!
\n\nP.D. By the way, Gyat, what do you mean exactly by 'research track'? Thanks!
", "votes": null, "replies": [] }, { "id": 358909, "author_name": "gyat2017", "author_url": "", "post_date": "07/19/2018 07:19:50", "content": "Thank you for your reply. I thought the deadlines for Judge's award and usual research paper submission is different.
", "votes": null, "replies": [] } ] }, { "id": 362970, "author_name": "gyat2017", "author_url": "", "post_date": "07/27/2018 13:59:10", "content": "@Eduardo There's something I am a little confused about, and need your help. So the Kaggle system allows us to select a maximum of two submissions. However, the DCASE system allows up to four submissions.
\n\nNow, if I understand correctly, we can describe all the methods used to generate the four submissions, in a single technical report, correct?
In that case, in order for a submission to qualify for the judges award, where do we mark it? Do we mark it in the same report, saying something like: This is the submission for the judge's consideration? Or do we need to create a separate report for that and notify you all for the same?
Requesting your clarifications on this.
", "votes": null, "replies": [ { "id": 363004, "author_name": "eduardofonseca", "author_url": "", "post_date": "07/27/2018 15:21:56", "content": "Hei Gyat!
\n\nthe DCASE system allows up to four submissions, but in our competition, only 2 are allowed, as per the Rules https://www.kaggle.com/c/freesound-audio-tagging/rules
\n\nThen, answering to your questions:
\n\nFinally, let me emphasize what was explained a couple of posts above:
\n\nhope that helps
", "votes": null, "replies": [] } ] }, { "id": 364588, "author_name": "eduardofonseca", "author_url": "", "post_date": "07/31/2018 20:22:40", "content": "Hi all, just a gentle reminder that submissions to the Judges' Award must include in the technical report:
\n\nin order to facilitate the review process.
\n\nthanks!
", "votes": null, "replies": [] }, { "id": 364740, "author_name": "hyunguilim", "author_url": "", "post_date": "08/01/2018 06:35:01", "content": "I'm confused where I submit a submission file for Judges’ Award. There are only 2 slots for final submission and you said that we can submit best performance sets for Kaggle submission.
\n\nWhich one is a right way to submit external submission for Judges’ Award?
\n\nI just noticed that the first one is correct.
\n\nTherefore, we should submit files below,
\n\nAm I right?
", "votes": null, "replies": [] }, { "id": 364827, "author_name": "eduardofonseca", "author_url": "", "post_date": "08/01/2018 10:13:55", "content": "yes, you're right
", "votes": null, "replies": [] } ] }, { "id": 364914, "author_name": "gyat2017", "author_url": "", "post_date": "08/01/2018 14:22:29", "content": "@Eduardo,
\n\nWould the judges consider a submission file which had already been submitted to kaggle before the competition end (but not mentioned in the submitted technical report) with the same model and same features, minus one small change in pre-processing step?
\n\nUnder the same model and same features, the single model submission scores 0.915 as opposed to 0.907 (which was selected based on the public LB performance). :'(
", "votes": null, "replies": [ { "id": 365538, "author_name": "eduardofonseca", "author_url": "", "post_date": "08/02/2018 21:19:20", "content": "Hi Gyat,
\n\nI'm afraid that we cannot accept that, as it could be somewhat unfair to other participants. Besides, taking into account the implications of being ranked live on the public LB but eventually being ranked on the private LB is also part of the competition.
\n\nLastly, if you say that the approaches and scores are so similar, it is likely that our judgment will be very similar too.
\n\nThanks!
", "votes": null, "replies": [] } ] }, { "id": 388903, "author_name": "gyat2017", "author_url": "", "post_date": "09/17/2018 19:48:14", "content": "Hello, are the results in for the judge's award?
", "votes": null, "replies": [ { "id": 388929, "author_name": "eduardofonseca", "author_url": "", "post_date": "09/17/2018 20:46:12", "content": "coming soon!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "349636": "As we approach the home stretch of the challenge, we would like to announce a **Judges’ Award**.\n\nThe goal of the Judges’ Award is to encourage contestants to use **novel and problem-specific approaches** which leverage knowledge of the audio domain. Another key factor is **computational efficiency**: we want to promote models that can fit within reasonable resource constraints, as opposed to very large models that may merely memorize a dataset and may not fit within typical deployment environments on mobile devices, for example.\n\n**Rules:**\n\nFor the Judges’ Award, submissions will be evaluated according to:\n\n- Innovation and novelty\n- Consideration of domain-specific properties of audio\n- Consideration of issues specific to the **FSDKaggle2018** dataset, including different reliability of annotations, variable length of audio clips, etc. \n- Computational efficiency\n- Classification performance\n\nThe following strict rules apply for the Judges’ Award:\n\n- External data is not allowed, in any form (including pre-trained models)\n- Data augmentation is allowed\n- For the sake of efficiency, single models are preferred over ensembles\n\n**Requirements:**\n\n- **Deliver write-up & code**: Participants must submit a technical report and a metainformation file through the DCASE submission system using the DCASE template. The technical report will be made publicly available through the DCASE website. There is more info about DCASE here: https://www.kaggle.com/c/freesound-audio-tagging#DCASE . In addition, participants must submit the code and the technical report to Kaggle. The model upload occurs through the Team tab of the Kaggle competition page. The judging of the award will be based primarily upon the technical report. \n***Submission Deadline***: July 31st\n- Please **report on the computational efficiency** of your model by at least including the number of model parameters. Additional measures or information to satisfy the motivation of computational efficiency will be welcome.\n- Please **specify in the technical report the submission file (*filename.csv*) corresponding to the candidate model** to be considered for the Judges' Award.\n- As with the Kaggle competition winners, the **Judges’ Award Winner will be asked to publish their code as open-source**. \n\n**Prize:** \n\nAt the close of the competition, submissions meeting the Requirements will be considered for the Judges’ Award. The “prize” consists of the **public announcement of the winner in this thread by the organizers**, along with a brief description of the most interesting aspects of the system. There is no monetary prize.\n\n