![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F532347%2F279a69103b30647366bf9853d6826351%2FScreenshot%20from%202019-06-28%2018-34-57.png?generation=1561736404577322&alt=media\")
EDIT, 15:22 UTC, Friday June 28: The leaderboard has been re-released!
\n\nEDIT, 23:45pm UTC, Monday June 17: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.
\n\nPrevious Post: Kernels have been re-run for the second stage, and the private leaderboard has been released! Thank you all for your hard work in this competition and congratulations to our winners! Note: the \"change\" in leaderboard score from Public to Private leaderboards is unreliable, as the public leaderboard will not reflect the Stage 1 scores.
\n\nAnd a continued congratulation to the many of you may reach a new tier because of this competition, as well as far more of you for which this was your first competitions.
\n\nWe're also continually pleased to see how Kaggle Competitions serve as a great medium and vehicle through which the greater data science community can learn and develop their machine learning abilities and skills. Beginners and experts can come together to start, grow, and succeed.
\n\nTo those of you for whom this was your first Kaggle competition - great work! We hope to see you again in a future competition! To those for whom this competition resulted in a Kaggle tier promotion - congratulations! Thanks to the many of you who have shared your solutions in the forums for others to learn from, and have shared valuable insights.
\n\nWe also would like to give special thanks to Eduardo and Frederic from UPF, and Manoj and Dan from Google for allowing us to run this competition as a part of the DCASE challenge - it's especially great to have a unique audio dataset like this! We hope to be able to work with them in the future.
\n\nAs noted in the forums, the competition hosts reserve the right to review any team's Kernel submission for adherence to the posted rules. Should it be determined that any of these teams have violated the rules, we will remove them from the leaderboard. If you think you were removed by mistake, please contact compliance. Please fill in all the fields honestly.
\n\nThe top potential winning teams will be contacted in the next few days via email to provide their winning solutions for sponsor review. Should we need to move down the leaderboard for any reason, we will do so and continue to make contact with subsequent teams.
\n\nYou should see your points and medals awarded soon, however it may take a few hours to propagate through.
\n\nWe hope to see you in our next competitions!
\n\nHappy Modeling!
\n\ncc/ @fredericfont @plakal @eduardofonseca
", "messages": [ { "id": 560520, "postDate": "2019-06-25T14:48:01.890Z", "content": "Hi All,
\n\nKaggle has performed it's kernel re-runs and was able to reinstate some teams that had been mistakenly removed (utility scripts, released mid-competition, was the culprit!). The competition sponsors have asked that they have some time to perform some of their own checks prior to releasing the private leaderboard.
\n\nThanks for your patience!
", "rawMarkdown": "Hi All,\n\nKaggle has performed it's kernel re-runs and was able to reinstate some teams that had been mistakenly removed (utility scripts, released mid-competition, was the culprit!). The competition sponsors have asked that they have some time to perform some of their own checks prior to releasing the private leaderboard.\n\nThanks for your patience!", "votes": 16, "replies": [ { "id": 560526, "postDate": "2019-06-25T14:51:36.553Z", "content": "Great job! Thank you for sharing!
", "rawMarkdown": "Great job! Thank you for sharing!", "votes": 1 }, { "id": 560540, "postDate": "2019-06-25T14:59:19.527Z", "content": "thank you for your reply π
", "rawMarkdown": "thank you for your reply π", "votes": 2 }, { "id": 560586, "postDate": "2019-06-25T15:49:01.227Z", "content": "Thanks. I hope no more accident happen.
", "rawMarkdown": "Thanks. I hope no more accident happen.", "votes": 1 }, { "id": 560609, "postDate": "2019-06-25T16:04:19.470Z", "content": "I hope the sponsors don't take another week for their checks ππ
", "rawMarkdown": "I hope the sponsors don't take another week for their checks ππ", "votes": 2 }, { "id": 561025, "postDate": "2019-06-26T04:16:31.480Z", "content": "finally, we can see the results, it has been a long waiting, thanks
", "rawMarkdown": "finally, we can see the results, it has been a long waiting, thanks", "votes": 1 }, { "id": 561034, "postDate": "2019-06-26T04:26:51.040Z", "content": "the final LB is yet to be published as the sponsors are also doing some validations at their end. Not sure how much time are they going to take to validate at their end.
", "rawMarkdown": "the final LB is yet to be published as the sponsors are also doing some validations at their end. Not sure how much time are they going to take to validate at their end.", "votes": 2 }, { "id": 562169, "postDate": "2019-06-27T03:22:18.010Z", "content": "How long does the sponsor need to checkοΌ
", "rawMarkdown": "How long does the sponsor need to checkοΌ", "votes": 2 }, { "id": 563660, "postDate": "2019-06-28T14:39:37.697Z", "content": "@addisonhoward - any chance we will see the LB today before everyone goes off for the weekend? or are we now looking for sometime next week now?
", "rawMarkdown": "@addisonhoward - any chance we will see the LB today before everyone goes off for the weekend? or are we now looking for sometime next week now?", "votes": 1 } ] }, { "id": 554687, "postDate": "2019-06-17T22:54:19.610Z", "content": "EDIT, 15:22 UTC, Friday June 28: The leaderboard has been re-released!
\n\nEDIT, 23:45pm UTC, Monday June 17: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.
\n\nPrevious Post: Kernels have been re-run for the second stage, and the private leaderboard has been released! Thank you all for your hard work in this competition and congratulations to our winners! Note: the \"change\" in leaderboard score from Public to Private leaderboards is unreliable, as the public leaderboard will not reflect the Stage 1 scores.
\n\nAnd a continued congratulation to the many of you may reach a new tier because of this competition, as well as far more of you for which this was your first competitions.
\n\nWe're also continually pleased to see how Kaggle Competitions serve as a great medium and vehicle through which the greater data science community can learn and develop their machine learning abilities and skills. Beginners and experts can come together to start, grow, and succeed.
\n\nTo those of you for whom this was your first Kaggle competition - great work! We hope to see you again in a future competition! To those for whom this competition resulted in a Kaggle tier promotion - congratulations! Thanks to the many of you who have shared your solutions in the forums for others to learn from, and have shared valuable insights.
\n\nWe also would like to give special thanks to Eduardo and Frederic from UPF, and Manoj and Dan from Google for allowing us to run this competition as a part of the DCASE challenge - it's especially great to have a unique audio dataset like this! We hope to be able to work with them in the future.
\n\nAs noted in the forums, the competition hosts reserve the right to review any team's Kernel submission for adherence to the posted rules. Should it be determined that any of these teams have violated the rules, we will remove them from the leaderboard. If you think you were removed by mistake, please contact compliance. Please fill in all the fields honestly.
\n\nThe top potential winning teams will be contacted in the next few days via email to provide their winning solutions for sponsor review. Should we need to move down the leaderboard for any reason, we will do so and continue to make contact with subsequent teams.
\n\nYou should see your points and medals awarded soon, however it may take a few hours to propagate through.
\n\nWe hope to see you in our next competitions!
\n\nHappy Modeling!
\n\ncc/ @fredericfont @plakal @eduardofonseca
", "rawMarkdown": "EDIT, 15:22 UTC, Friday June 28: The leaderboard has been re-released!\n\nEDIT, 23:45pm UTC, Monday June 17: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.\n\nPrevious Post: Kernels have been re-run for the second stage, and the private leaderboard has been released! Thank you all for your hard work in this competition and congratulations to our winners! Note: the \"change\" in leaderboard score from Public to Private leaderboards is unreliable, as the public leaderboard will not reflect the Stage 1 scores.\n\nAnd a continued congratulation to the many of you may reach a new tier because of this competition, as well as far more of you for which this was your first competitions.\n\nWe're also continually pleased to see how Kaggle Competitions serve as a great medium and vehicle through which the greater data science community can learn and develop their machine learning abilities and skills. Beginners and experts can come together to start, grow, and succeed. \n\nTo those of you for whom this was your first Kaggle competition - great work! We hope to see you again in a future competition! To those for whom this competition resulted in a Kaggle tier promotion - congratulations! Thanks to the many of you who have shared your solutions in the forums for others to learn from, and have shared valuable insights.\n\nWe also would like to give special thanks to Eduardo and Frederic from UPF, and Manoj and Dan from Google for allowing us to run this competition as a part of the DCASE challenge - it's especially great to have a unique audio dataset like this! We hope to be able to work with them in the future.\n\nAs noted in the forums, the competition hosts reserve the right to review any team's Kernel submission for adherence to the posted rules. Should it be determined that any of these teams have violated the rules, we will remove them from the leaderboard. If you think you were removed by mistake, please contact [compliance][1]. Please fill in all the fields honestly.\n\nThe top potential winning teams will be contacted in the next few days via email to provide their winning solutions for sponsor review. Should we need to move down the leaderboard for any reason, we will do so and continue to make contact with subsequent teams.\n\nYou should see your points and medals awarded soon, however it may take a few hours to propagate through.\n\nWe hope to see you in our next competitions!\n\nHappy Modeling!\n\ncc/ @fredericfont @plakal @eduardofonseca \n\n [1]: https://www.kaggle.com/compliance", "votes": 12 }, { "id": 562145, "postDate": "2019-06-27T02:44:30.820Z", "content": "The workshop paper deadline is getting close without the final result nor late submission. lol.
", "rawMarkdown": "The workshop paper deadline is getting close without the final result nor late submission. lol.", "votes": 9, "replies": [ { "id": 564071, "postDate": "2019-06-29T00:10:01.030Z", "content": "EDIT Monday July 1st: the DCASE workshop paper deadline has been extended to July 16. More info in this post.\nhttps://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/98192#latest-566202
\n\nworkshop paper deadline is specified here: http://dcase.community/workshop2019/
\n\nJuly 5th: abstract submission\nJuly 12th: final paper submission
\n\nThe results publication date was planned by the DCASE organization (and has been publicly specified all along), so that there is enough time before the workshop deadline(s).
\n\nWe encourage you to submit the the DCASE Workshop, regardless whether you submitted to the DCASE Challenge or not. What matters is the methodology, the approach, the ideas, rather than the tyranny of the leaderboard. The community will benefit from any ideas that you want to share :)
", "rawMarkdown": "EDIT Monday July 1st: the DCASE workshop paper deadline has been extended to July 16. More info in this post.\n[https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/98192#latest-566202](https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/98192#latest-566202)\n\n---\n\nworkshop paper deadline is specified here: [http://dcase.community/workshop2019/](http://dcase.community/workshop2019/)\n\nJuly 5th: abstract submission\nJuly 12th: final paper submission\n\nThe results publication date was planned by the DCASE organization (and has been publicly specified all along), so that there is enough time before the workshop deadline(s).\n\nWe encourage you to submit the the DCASE Workshop, regardless whether you submitted to the DCASE Challenge or not. What matters is the methodology, the approach, the ideas, rather than the tyranny of the leaderboard. The community will benefit from any ideas that you want to share :)\n\n", "votes": 1 } ] }, { "id": 557056, "postDate": "2019-06-21T00:25:09.550Z", "content": "How I wish this competition were synchronous kernel-only competition.
", "rawMarkdown": "How I wish this competition were synchronous kernel-only competition. ", "votes": 8, "replies": [ { "id": 560502, "postDate": "2019-06-25T14:24:15.913Z", "content": "indeed...the instant gratification competition finished after this one and the Private LB got finalized before this :(
", "rawMarkdown": "indeed...the instant gratification competition finished after this one and the Private LB got finalized before this :(", "votes": 1 } ] }, { "id": 563711, "postDate": "2019-06-28T15:40:09.303Z", "content": "@addisonhoward, it looks like my second selected kernel was not executed and evaluated. Meanwhile, I just run it manually and submit via late submission and get correct results which may affect the private leaderboard.\n
Standby - we're investigating.
", "rawMarkdown": "Standby - we're investigating." }, { "id": 563729, "postDate": "2019-06-28T16:00:30.507Z", "content": "Ruslan,
\n\nOur records show that you selected the following for scoring:
\n\n> Freesound Predict Dataset version 50/52\n>Stacking 008\nThis kernel errored due to an IndexError: list index out of range
\n\n>Freesound Predict Dataset version 52/52 \n>Stacking 007\nThis one ran successfully and was on the LB
\n\nKernel Freesound Predict 2 (version 8/8) Stacking 008 was not selected for your final two selections.
", "rawMarkdown": "Ruslan,\n\nOur records show that you selected the following for scoring:\n\n> Freesound Predict Dataset version 50/52\n>Stacking 008\nThis kernel errored due to an IndexError: list index out of range\n\n>Freesound Predict Dataset version 52/52 \n>Stacking 007\nThis one ran successfully and was on the LB\n\nKernel Freesound Predict 2 (version 8/8) Stacking 008 was not selected for your final two selections." }, { "id": 563755, "postDate": "2019-06-28T16:30:53.810Z", "content": "\"Kernel Freesound Predict 2 (version 8/8) Stacking 008\" is copy of \"Freesound Predict Dataset version 50/52\nStacking 008\"
", "rawMarkdown": "\"Kernel Freesound Predict 2 (version 8/8) Stacking 008\" is copy of \"Freesound Predict Dataset version 50/52\nStacking 008\"", "votes": 2 }, { "id": 563767, "postDate": "2019-06-28T16:46:57.397Z", "content": "Download .zip dataset from kaggle, upload to new dataset and copy paste code.
", "rawMarkdown": "Download .zip dataset from kaggle, upload to new dataset and copy paste code.", "votes": 2 }, { "id": 564040, "postDate": "2019-06-28T23:20:01.827Z", "content": "Now I forked \"Freesound Predict Dataset version 50/52\", kernel failed with IndexError error because my code can't find model weights.
\nI checked the 'input' folder through the console and found that files in running kernel do not match the files from data in \"Draft Environment\". I will report more information to compliance.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F532347%2Fee5b064710e6c4e843d7e14110a0d25b%2FScreenshot%20from%202019-06-29%2002-04-02.png?generation=1561763961342052&alt=media\")
While competing I experienced similar problems with datasets containing models as Ruslan. It can be reproduced with following steps:\n1. Add dataset to kernel.\n2. Commit version.\n3. Update dataset.\n4. Fork or start editing.\n5. Sometimes old version of dataset were available in kernel while new one is shown in interface.
", "rawMarkdown": "While competing I experienced similar problems with datasets containing models as Ruslan. It can be reproduced with following steps:\n1. Add dataset to kernel.\n2. Commit version.\n3. Update dataset.\n4. Fork or start editing.\n5. Sometimes old version of dataset were available in kernel while new one is shown in interface.\n", "votes": 2 }, { "id": 564275, "postDate": "2019-06-29T08:15:58.253Z", "content": "I have done several tests and now I am quite sure that one of my selected for the final submission kernels was not executed because of a kaggle bug.\nFailed kernel: https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383
\n\n> Freesound Predict Dataset version 50/52\nStacking 008\nThis kernel errored due to an IndexError: list index out of range
\n\nI have made the kernel and dataset public. Anyone can run the following tests:
\n\nTest 1: Successful execution with the code and data from kernel \n1. Go to https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383\n2. Download experiments.zip from data sources.\n3. Create a new dataset with the name \"Freesound train dataset\", upload experiments.zip from 2.
\n4. Create a new kernel, add dataset from 3. and competition data.\n5. Commit.
The kernel will successfully run with a private score of 0.75980 which now would be the first place.
\n\nTest 2: Error reason is kaggle dataset versioning bug \n1. Go to https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383\n2. Fork the kernel.\n3. Run command in console \"ls ../input/freesound-train-dataset/experiments\"
\n\nYou will see that that files in running kernel do not match the files from data in \"Draft Environment\" and data in experiments.zip from Test 1.\nIf you commit the kernel, it will fail with IndexError because my code can't find models weight. On stage 1 kernel was run successfully. Means that there were was correct data. But in stage 2 data replaced with a different version of the dataset due the kaggle bug.
", "rawMarkdown": "I have done several tests and now I am quite sure that one of my selected for the final submission kernels was not executed because of a kaggle bug.\nFailed kernel: https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383\n\n> Freesound Predict Dataset version 50/52\nStacking 008\nThis kernel errored due to an IndexError: list index out of range\n\nI have made the kernel and dataset public. Anyone can run the following tests: \n\n**Test 1:** Successful execution with the code and data from kernel \n1. Go to https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383\n2. Download experiments.zip from data sources.\n3. Create a new dataset with the name \"Freesound train dataset\", upload experiments.zip from 2. \n4. Create a new kernel, add dataset from 3. and competition data.\n5. Commit. \n\nThe kernel will successfully run with a private score of 0.75980 which now would be the first place.\n \n\n**Test 2:** Error reason is kaggle dataset versioning bug \n1. Go to https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383\n2. Fork the kernel.\n3. Run command in console \"ls ../input/freesound-train-dataset/experiments\"\n\nYou will see that that files in running kernel do not match the files from data in \"Draft Environment\" and data in experiments.zip from Test 1.\nIf you commit the kernel, it will fail with IndexError because my code can't find models weight. On stage 1 kernel was run successfully. Means that there were was correct data. But in stage 2 data replaced with a different version of the dataset due the kaggle bug. ", "votes": 3 }, { "id": 564535, "postDate": "2019-06-29T15:11:18.453Z", "content": "Hi - we're continuing to look into this.
\n\nThanks,\nKaggle Team
", "rawMarkdown": "Hi - we're continuing to look into this.\n\nThanks,\nKaggle Team", "votes": 3 } ] }, { "id": 559978, "postDate": "2019-06-24T22:56:14.590Z", "content": "When can we expect an update on this? It has been 7 days since the flawed leaderboard was taken down.
", "rawMarkdown": "When can we expect an update on this? It has been 7 days since the flawed leaderboard was taken down.", "votes": 4 }, { "id": 565324, "postDate": "2019-06-30T19:00:28.223Z", "content": "@addisonhoward For our team it shows only 1 entry and there is only 1 rerun but we've submitted two entries and both are running fine on private dataset. Please let me know if any other information is needed from my side.
", "rawMarkdown": "@addisonhoward For our team it shows only 1 entry and there is only 1 rerun but we've submitted two entries and both are running fine on private dataset. Please let me know if any other information is needed from my side.", "votes": 1 }, { "id": 563984, "postDate": "2019-06-28T21:05:45.753Z", "content": "Will you release audio data and true labels of stage 1? It will help us to verify our solution.
", "rawMarkdown": "Will you release audio data and true labels of stage 1? It will help us to verify our solution.", "votes": 1, "replies": [ { "id": 564062, "postDate": "2019-06-28T23:50:59.123Z", "content": "We will do that, but it'll take some time; probably at the end of summer. We 'll be sure to post here when the data are available. thanks.
", "rawMarkdown": "We will do that, but it'll take some time; probably at the end of summer. We 'll be sure to post here when the data are available. thanks.", "votes": 1 }, { "id": 564351, "postDate": "2019-06-29T10:01:29.943Z", "content": "Thanks for organizing the competition! Congrats to all the winners! When can we start submitting again for knowledge? I just tried, and it says 0.000 and doesn't show up in the lb.
", "rawMarkdown": "Thanks for organizing the competition! Congrats to all the winners! When can we start submitting again for knowledge? I just tried, and it says 0.000 and doesn't show up in the lb." }, { "id": 564536, "postDate": "2019-06-29T15:13:00.930Z", "content": "Hi Chi,
\n\nYou can continue to submit using a Late Submission, however it will not appear on the leaderboard, as the competition has concluded.
", "rawMarkdown": "Hi Chi,\n\nYou can continue to submit using a Late Submission, however it will not appear on the leaderboard, as the competition has concluded." }, { "id": 564545, "postDate": "2019-06-29T15:30:07.783Z", "content": "Thanks @addisonhoward! I see. I found the score in \"my submissions\" page. I can get the score, but not the ranking. There's no way to know where my late submission score stands in comparison to other people's late submissions then? Is that correct?
", "rawMarkdown": "Thanks @addisonhoward! I see. I found the score in \"my submissions\" page. I can get the score, but not the ranking. There's no way to know where my late submission score stands in comparison to other people's late submissions then? Is that correct?" }, { "id": 564552, "postDate": "2019-06-29T15:37:11.433Z", "content": "That's correct - you can go to the Leaderboard to compare where you would have squeezed in, but there is no formalized tracking of rankings after the competition has concluded
", "rawMarkdown": "That's correct - you can go to the Leaderboard to compare where you would have squeezed in, but there is no formalized tracking of rankings after the competition has concluded" }, { "id": 564617, "postDate": "2019-06-29T17:53:14.310Z", "content": "Hello @addisonhoward May I know what happened to my submissions? Will you let me know about my submissions and ranking for the competition?
", "rawMarkdown": "Hello @addisonhoward May I know what happened to my submissions? Will you let me know about my submissions and ranking for the competition?", "votes": -3 }, { "id": 564689, "postDate": "2019-06-29T20:18:53.953Z", "content": "Hi Arunkumar,
\n\nWe cannot provide individual critique on why code failed in a two-stage Kernels-only competition. You can re-run your kernel on the Stage 2 data and see what errors may have come from it.
\n\nWe are currently investigating a potential bug that's the result of dataset versioning, and will notify you if that was the result of your kernel failing.
", "rawMarkdown": "Hi Arunkumar,\n\nWe cannot provide individual critique on why code failed in a two-stage Kernels-only competition. You can re-run your kernel on the Stage 2 data and see what errors may have come from it.\n\nWe are currently investigating a potential bug that's the result of dataset versioning, and will notify you if that was the result of your kernel failing." }, { "id": 564952, "postDate": "2019-06-30T08:23:55.417Z", "content": "@addisonhoward The number of submissions per day is still limited to 2. Now that the competition is over can't this limit be removed so that we can perform the ablation studies faster?
", "rawMarkdown": "@addisonhoward The number of submissions per day is still limited to 2. Now that the competition is over can't this limit be removed so that we can perform the ablation studies faster?" }, { "id": 565287, "postDate": "2019-06-30T17:52:29.583Z", "content": "Miguel,
\n\nWe generally don't change the submission limit after the competition has concluded for the same reason we don't release the Private LB labels - we want to keep the competition open, without the labels revealed, so people can continue to make submissions and not have the dataset available to them.
\n\nHowever, we ultimately allow the competition sponsors to make that judgement.
", "rawMarkdown": "Miguel,\n\nWe generally don't change the submission limit after the competition has concluded for the same reason we don't release the Private LB labels - we want to keep the competition open, without the labels revealed, so people can continue to make submissions and not have the dataset available to them.\n\nHowever, we ultimately allow the competition sponsors to make that judgement.", "votes": 1 } ] }, { "id": 560516, "postDate": "2019-06-25T14:45:25.800Z", "content": "the instant gratification competition finished... \nHow long do we have to wait?
", "rawMarkdown": "the instant gratification competition finished... \nHow long do we have to wait?", "votes": 1 }, { "id": 554711, "postDate": "2019-06-17T23:44:00.330Z", "content": "Update: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.
", "rawMarkdown": "Update: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.", "votes": 1, "replies": [ { "id": 554717, "postDate": "2019-06-18T00:16:06.503Z", "content": "Well as long as it wont take another week Β―\\_(γ)_/Β―
", "rawMarkdown": "Well as long as it wont take another week Β―\\\\_(γ)_/Β―", "votes": 7 }, { "id": 554817, "postDate": "2019-06-18T03:43:15.650Z", "content": "@suicaokhoailang - your statement makes me remember the massive delay in the recent Santander competition. I seriously hope it's not that long.
", "rawMarkdown": "@suicaokhoailang - your statement makes me remember the massive delay in the recent Santander competition. I seriously hope it's not that long." }, { "id": 554947, "postDate": "2019-06-18T08:12:52.370Z", "content": "For the latest run by Kaggle, it says wait time is -506356 seconds, execution time is 1 seconds , and score is 0.00000. :(\nI just copied my kernel and ran, and it successfully completed in 1328.2 seconds.\nHopefully it's not my script's fault.
", "rawMarkdown": "For the latest run by Kaggle, it says wait time is -506356 seconds, execution time is 1 seconds , and score is 0.00000. :(\nI just copied my kernel and ran, and it successfully completed in 1328.2 seconds.\nHopefully it's not my script's fault.", "votes": 2 }, { "id": 554956, "postDate": "2019-06-18T08:27:32.887Z", "content": "Hello,
\n\nIt seems my kernels passed the kaggle rerun, as I saw my new position on the temporary private leaderboard before its withdrawal, and I have similar display as you about the latest run I did. I didn't save the leaderboard result, so I can't help if you passed it or not.\nI wish you the best.
", "rawMarkdown": "Hello,\n\nIt seems my kernels passed the kaggle rerun, as I saw my new position on the temporary private leaderboard before its withdrawal, and I have similar display as you about the latest run I did. I didn't save the leaderboard result, so I can't help if you passed it or not.\nI wish you the best." }, { "id": 555765, "postDate": "2019-06-19T11:58:11.900Z", "content": "@addisonhoward So are there any updates on this?
", "rawMarkdown": "@addisonhoward So are there any updates on this? ", "votes": 1 }, { "id": 555836, "postDate": "2019-06-19T13:58:39.863Z", "content": "I'm guessing a source of some of the errors was that some of us (including me!) assumed that the sample_submission.csv file would be updated for the stage 2 data. I know a few of the public kernels did this as well. It just occurred to me that my kernel uses that sample submission file to create the final submission file.
", "rawMarkdown": "I'm guessing a source of some of the errors was that some of us (including me!) assumed that the sample_submission.csv file would be updated for the stage 2 data. I know a few of the public kernels did this as well. It just occurred to me that my kernel uses that sample submission file to create the final submission file.", "votes": 1 }, { "id": 555842, "postDate": "2019-06-19T14:04:16.050Z", "content": "If that's the case then even I also made the same mistake :( and a costly mistake it seems :(
", "rawMarkdown": "If that's the case then even I also made the same mistake :( and a costly mistake it seems :(" }, { "id": 555847, "postDate": "2019-06-19T14:14:29.493Z", "content": "I think sample_submission.csv did get updated? If you look at it now, it has 3361 rows. Before it had 1120 rows.
", "rawMarkdown": "I think sample_submission.csv did get updated? If you look at it now, it has 3361 rows. Before it had 1120 rows.", "votes": 2 }, { "id": 555851, "postDate": "2019-06-19T14:18:36.490Z", "content": "I used sample_submission.csv to make an inference csv and it did not make an error.
\nI guess the reason would be another.
I guess we will know the source of errors only when organizers share an update...till then it's agonizing wait
", "rawMarkdown": "I guess we will know the source of errors only when organizers share an update...till then it's agonizing wait", "votes": 1 }, { "id": 555871, "postDate": "2019-06-19T14:49:47.513Z", "content": "@suicaokhoailang - not yet, thanks for your patience.
", "rawMarkdown": "@suicaokhoailang - not yet, thanks for your patience." }, { "id": 555898, "postDate": "2019-06-19T15:28:15.537Z", "content": "I think some of you don't understand: if you present in leaderboard - your submission was valid and you don't have to worry :) But for others (like me) there was some sort of mistake, so we didn't pass to the second stage - as you can see, I currently don't have my place on leaderboard near my nickname. But in my case I can run kernel without error and do a submission (and some other people can do this aswell) - so we can only hope that organizers will find error and bring us back to competition.
", "rawMarkdown": "I think some of you don't understand: if you present in leaderboard - your submission was valid and you don't have to worry :) But for others (like me) there was some sort of mistake, so we didn't pass to the second stage - as you can see, I currently don't have my place on leaderboard near my nickname. But in my case I can run kernel without error and do a submission (and some other people can do this aswell) - so we can only hope that organizers will find error and bring us back to competition.", "votes": 7 }, { "id": 556059, "postDate": "2019-06-19T19:33:46.433Z", "content": "@koba35 Do you have any external utility scripts in your kernel? I am thinking whether that causes the error since that's a new feature introduced recently.
", "rawMarkdown": "@koba35 Do you have any external utility scripts in your kernel? I am thinking whether that causes the error since that's a new feature introduced recently.", "votes": 1 }, { "id": 556061, "postDate": "2019-06-19T19:40:03.947Z", "content": "Yep, I think this is the case.
", "rawMarkdown": "Yep, I think this is the case." }, { "id": 556650, "postDate": "2019-06-20T13:46:32.863Z", "content": "@addisonhoward Would you tell me the order of magnitude of time until publish \nthe leaderboard? a few days? a week? a few weeks?
", "rawMarkdown": "@addisonhoward Would you tell me the order of magnitude of time until publish \nthe leaderboard? a few days? a week? a few weeks?", "votes": 6 }, { "id": 557245, "postDate": "2019-06-21T02:56:30.243Z", "content": "The DCASE papers submission deadline will be due on 12th July so there is that.
", "rawMarkdown": "The DCASE papers submission deadline will be due on 12th July so there is that." }, { "id": 557312, "postDate": "2019-06-21T04:11:09.887Z", "content": "@hattan0523 - We're doing our best to get them out sooner rather than later. I know that's not the answer you're hoping for, but we're re-running kernels, doing some error checking, and diagnosing what is user error (as noted here), and what may have been a different error.
\n\nOne that we've specifically uncovered are errors thrown for utility scripts, which were released mid-competition. We've identified those and are looking into others.
\n\nWe certainly hope to have the results re-released early next week, but we can't make any guarantees, as we keep troubleshooting.
\n\nOn the bright side, and as I stall for time, our synchronous kernels-only competition just concluded, and should address a majority of these issues. We're excited to see what that brings for the future state!
", "rawMarkdown": "@hattan0523 - We're doing our best to get them out sooner rather than later. I know that's not the answer you're hoping for, but we're re-running kernels, doing some error checking, and diagnosing what is user error (as noted [here](https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/96082#554697)), and what may have been a different error.\n\nOne that we've specifically uncovered are errors thrown for utility scripts, which were released mid-competition. We've identified those and are looking into others.\n\nWe certainly hope to have the results re-released early next week, but we can't make any guarantees, as we keep troubleshooting. \n\nOn the bright side, and as I stall for time, our synchronous kernels-only competition just concluded, and should address a majority of these issues. We're excited to see what that brings for the future state!", "votes": 6 }, { "id": 557345, "postDate": "2019-06-21T04:43:15.647Z", "content": "@addisonhoward - thanks for sharing the some sort of tentative timeline. I can personally take a break from checking Kaggle every few hours in the hope of an update. I will come back sometime next week. Hope everyone has a good weekend!!
\n\nCheers!!
", "rawMarkdown": "@addisonhoward - thanks for sharing the some sort of tentative timeline. I can personally take a break from checking Kaggle every few hours in the hope of an update. I will come back sometime next week. Hope everyone has a good weekend!! \n\nCheers!!", "votes": 2 }, { "id": 557373, "postDate": "2019-06-21T05:14:01.907Z", "content": "Thank you @addisonhoward for the update. Sync KO sounds like a nice evolution as it will find these problems earlier and allow immediate private leaderboard results.
", "rawMarkdown": "Thank you @addisonhoward for the update. Sync KO sounds like a nice evolution as it will find these problems earlier and allow immediate private leaderboard results.", "votes": 1 }, { "id": 559698, "postDate": "2019-06-24T13:15:14.193Z", "content": "Thank you for sharing it. I'm looking forward to opening leaderboard!
", "rawMarkdown": "Thank you for sharing it. I'm looking forward to opening leaderboard!" }, { "id": 560066, "postDate": "2019-06-25T02:17:12.507Z", "content": "Sometimes dreams can come true\n> Khoi Nguyen wrote\n> \n> > Well as long as it wont take another week Β―(γ)/Β―
", "rawMarkdown": "\nSometimes dreams can come true\n> **Khoi Nguyen wrote**\n> \n> > Well as long as it wont take another week Β―\\(γ)/Β―\n\n", "votes": 1 }, { "id": 560093, "postDate": "2019-06-25T03:14:54.867Z", "content": "@suicaokhoailang I knew it's gonna be another week because they re-ran all submissions again... First re-ran took a week, second re-ran should be the same
", "rawMarkdown": "@suicaokhoailang I knew it's gonna be another week because they re-ran all submissions again... First re-ran took a week, second re-ran should be the same" }, { "id": 560098, "postDate": "2019-06-25T03:25:03.567Z", "content": "There is no reasons to re-run every team on the leaderboard. Beside, the Quora competitions had 4000+ teams, 1400+ were valid, the allowed run time were 2 hours and it took less than 2 weeks. In this case I think it was just lots of manual checking and debugging.
", "rawMarkdown": "There is no reasons to re-run every team on the leaderboard. Beside, the Quora competitions had 4000+ teams, 1400+ were valid, the allowed run time were 2 hours and it took less than 2 weeks. In this case I think it was just lots of manual checking and debugging. ", "votes": 3 }, { "id": 560101, "postDate": "2019-06-25T03:36:55.170Z", "content": "I agree with you, but I remember someone from Kaggle (probably Addison?) saying they \"re-ran all kernels\".
", "rawMarkdown": "I agree with you, but I remember someone from Kaggle (probably Addison?) saying they \"re-ran all kernels\"." } ] }, { "id": 554695, "postDate": "2019-06-17T23:16:34.010Z", "content": "Hi Addison. I checked my leaderboard page and it showed \"Invalidated for Stage 2 Reruns\". I tested my kernel on 2nd stage test data couple days ago and it completed in 1 hour without any problem. Can I have an explanation about this? I made another post in the thread but I think I can get a quicker response here. Thanks!
", "rawMarkdown": "Hi Addison. I checked my leaderboard page and it showed \"Invalidated for Stage 2 Reruns\". I tested my kernel on 2nd stage test data couple days ago and it completed in 1 hour without any problem. Can I have an explanation about this? I made another post in the thread but I think I can get a quicker response here. Thanks!", "votes": 1 }, { "id": 562933, "postDate": "2019-06-27T17:25:53.483Z", "content": "@addisonhoward news?\nbtw, the burner account NovaPulsar got a discussion bronze medal having 6 downvotes lol
", "rawMarkdown": "@addisonhoward **news?**\nbtw, the burner account **NovaPulsar** got a discussion bronze medal having 6 downvotes lol", "votes": 2, "replies": [ { "id": 562941, "postDate": "2019-06-27T17:30:43.500Z", "content": "I guess the bronze medal has to do with how soon the bronze is awarded for a upvote. I was the first one to reply to him and when I replied to him there was one upvote on his comment at that time. Guess the bronze medal got awarded at that hour...post that 7 downvotes resulted in net of 6 downvotes. Maybe it's time Kaggle should revoke the discussion bronze medal for the cases when the final number of votes is less than 0.
", "rawMarkdown": "I guess the bronze medal has to do with how soon the bronze is awarded for a upvote. I was the first one to reply to him and when I replied to him there was one upvote on his comment at that time. Guess the bronze medal got awarded at that hour...post that 7 downvotes resulted in net of 6 downvotes. Maybe it's time Kaggle should revoke the discussion bronze medal for the cases when the final number of votes is less than 0. " }, { "id": 562945, "postDate": "2019-06-27T17:36:23.867Z", "content": "The Freesound team is working on validating some of the teams that submitted to DCASE - they have not provided us with an update on releasing the competition LB.
\n\nAs for medals, they typically propagate every few hours or so, but are based on the last static version. I've seen some discussions go from gold medal to silver after downvotes :)
", "rawMarkdown": "The Freesound team is working on validating some of the teams that submitted to DCASE - they have not provided us with an update on releasing the competition LB.\n\nAs for medals, they typically propagate every few hours or so, but are based on the last static version. I've seen some discussions go from gold medal to silver after downvotes :)" }, { "id": 562956, "postDate": "2019-06-27T17:42:29.283Z", "content": "@addisonhoward - thanks for sharing the update. Although it's very disappointing to hear that we still don't have a concrete timeline regarding when the final LB would be published :(
\n\nit's an agonizing wait everyday :(
", "rawMarkdown": "@addisonhoward - thanks for sharing the update. Although it's very disappointing to hear that we still don't have a concrete timeline regarding when the final LB would be published :(\n\nit's an agonizing wait everyday :(" }, { "id": 563694, "postDate": "2019-06-28T15:23:33.850Z", "content": "Just got permission to release! Take a look!
", "rawMarkdown": "Just got permission to release! Take a look!", "votes": 6 }, { "id": 563778, "postDate": "2019-06-28T17:02:06.570Z", "content": "@addisonhoward Kindly please look into my kernel submissions? Please let me know what happened to them as I forgot to select my kernels submissions...
", "rawMarkdown": "@addisonhoward Kindly please look into my kernel submissions? Please let me know what happened to them as I forgot to select my kernels submissions..." }, { "id": 564065, "postDate": "2019-06-28T23:58:13.297Z", "content": "Hi,
\n\nI'm sorry you were checking every day for results. Nonetheless, there has been indeed an expected timeline that we (organizers) thought you were aware of. As you know, this evaluation is part of the DCASE Challenge, whose results notification date has been specified in the DCASE Challenge website all along (see left bar Schedule).
\n\nhttp://dcase.community/challenge2019/
", "rawMarkdown": "Hi, \n\nI'm sorry you were checking every day for results. Nonetheless, there has been indeed an expected timeline that we (organizers) thought you were aware of. As you know, this evaluation is part of the DCASE Challenge, whose results notification date has been specified in the DCASE Challenge website all along (see left bar Schedule).\n\n[http://dcase.community/challenge2019/](http://dcase.community/challenge2019/)" } ] }, { "id": 557075, "postDate": "2019-06-21T00:35:32.927Z", "content": "Ten days have passed since the competition ended...
", "rawMarkdown": "Ten days have passed since the competition ended...", "votes": 2, "replies": [ { "id": 557471, "postDate": "2019-06-21T07:47:21.160Z", "content": "What is the cost of bugs? Itβs not that we will mistake them for a feature. The real danger is that, if we have enough bugs, then we no longer can receive the competition result as soon as the competition ended.
", "rawMarkdown": "What is the cost of bugs? Itβs not that we will mistake them for a feature. The real danger is that, if we have enough bugs, then we no longer can receive the competition result as soon as the competition ended.", "votes": 1 }, { "id": 560242, "postDate": "2019-06-25T07:45:47.387Z", "content": "Two weeks passed...
", "rawMarkdown": "Two weeks passed...", "votes": 1 }, { "id": 560253, "postDate": "2019-06-25T07:59:33.830Z", "content": "and still waiting.....
", "rawMarkdown": "and still waiting....." } ] }, { "id": 562534, "postDate": "2019-06-27T10:27:42.983Z", "content": "One of the worst competitons ever. A huge disrespect to all participants
", "rawMarkdown": "One of the worst competitons ever. A huge disrespect to all participants", "votes": -16, "replies": [ { "id": 562569, "postDate": "2019-06-27T10:55:33.533Z", "content": "I respectfully disagree with you. The learning opportunities in this competition has been tremendous (atleast for me). The delay in LB finalization is frustrating, but saying that the competition is a huge disrespect to all competitors is a bit too harsh. I am sure Kaggle team will learn from the issues in this competition and probably try to avoid them in future. I thank the organizers for arranging a competition with such a unique topic that pushed us to try so many new things.
\n\nCheers!! πβ
", "rawMarkdown": "I respectfully disagree with you. The learning opportunities in this competition has been tremendous (atleast for me). The delay in LB finalization is frustrating, but saying that the competition is a huge disrespect to all competitors is a bit too harsh. I am sure Kaggle team will learn from the issues in this competition and probably try to avoid them in future. I thank the organizers for arranging a competition with such a unique topic that pushed us to try so many new things.\n\nCheers!! πβ", "votes": 4 }, { "id": 562578, "postDate": "2019-06-27T11:04:26.450Z", "content": "I don't think so. There are so many worse competitions with fatal leakage or a meaningless target setting. This competition has some problems but still has so many things to learn.
", "rawMarkdown": "I don't think so. There are so many worse competitions with fatal leakage or a meaningless target setting. This competition has some problems but still has so many things to learn.", "votes": 7 }, { "id": 562588, "postDate": "2019-06-27T11:19:15.187Z", "content": "Disrespect would be to ignore the teams affected by the error. I know the long waiting for the results can be frustrating but I like to see that they are checking the results to make sure everything is okay. If anything it is a sign of respect for all participants and for the fairness of the competition.
\n\nEdit: And I find weird that you just created an account to write this comment.
", "rawMarkdown": "Disrespect would be to ignore the teams affected by the error. I know the long waiting for the results can be frustrating but I like to see that they are checking the results to make sure everything is okay. If anything it is a sign of respect for all participants and for the fairness of the competition. \n\nEdit: And I find weird that you just created an account to write this comment.", "votes": 13 }, { "id": 562866, "postDate": "2019-06-27T16:05:29.860Z", "content": "@mnpinto Good catch. The account indeed got created few hours ago! Should be reported as spam.
", "rawMarkdown": "@mnpinto Good catch. The account indeed got created few hours ago! Should be reported as spam.", "votes": 3 }, { "id": 564078, "postDate": "2019-06-29T00:29:21.880Z", "content": "Hi NovaPulsar,
\n\nWe're sorry you feel that way. I just want to stress that all the decisions taken by the organizing team have always been taken thinking on the participants' benefit. If it's not up to your standards, we welcome you to organize a competition yourself and share with us any recommendation, so that we can make it better next time.
\n\nAlso, we're a bit puzzled as it seems you just created a new account to post this, and you're not giving any reasons to argument your claim. Anyway, we welcome you to the competition!
\n\nPS thanks @mnpinto @osciiart @sandy1112 and everyone who downvoted NovaPulsar's post for your support. Organizing a competition indeed involves a lot of work behind the scenes, and it is not an easy task.
", "rawMarkdown": "Hi NovaPulsar, \n\nWe're sorry you feel that way. I just want to stress that all the decisions taken by the organizing team have always been taken thinking on the participants' benefit. If it's not up to your standards, we welcome you to organize a competition yourself and share with us any recommendation, so that we can make it better next time.\n\nAlso, we're a bit puzzled as it seems you just created a new account to post this, and you're not giving any reasons to argument your claim. Anyway, we welcome you to the competition!\n\nPS thanks @mnpinto @osciiart @sandy1112 and everyone who downvoted NovaPulsar's post for your support. Organizing a competition indeed involves a lot of work behind the scenes, and it is not an easy task.", "votes": 3 } ] }, { "id": 562104, "postDate": "2019-06-27T01:36:53.043Z", "content": "Happy Modeling!
", "rawMarkdown": "Happy Modeling!", "replies": [ { "id": 562120, "postDate": "2019-06-27T02:20:12.227Z", "content": "For the last 2 weeks it's more like happy waiting or get frustrated while waiting based on how patient one is π€―π΄π΄
", "rawMarkdown": "For the last 2 weeks it's more like happy waiting or get frustrated while waiting based on how patient one is π€―π΄π΄", "votes": 1 } ] }, { "id": 555249, "postDate": "2019-06-18T16:19:25.157Z", "content": "emm
", "rawMarkdown": "emm" }, { "id": 554969, "postDate": "2019-06-18T08:53:02.730Z", "content": "I also faced error in kernel and dropped off leaderboard - but I successfully ran kernel on new test data in 45 minutes and got a valid submission file. I even can do a late submission without error :) So it definitely some sort of error. Maybe cos I am using utility scripts?
", "rawMarkdown": "I also faced error in kernel and dropped off leaderboard - but I successfully ran kernel on new test data in 45 minutes and got a valid submission file. I even can do a late submission without error :) So it definitely some sort of error. Maybe cos I am using utility scripts?", "replies": [ { "id": 556788, "postDate": "2019-06-20T16:49:51.230Z", "content": "Using utility scripts is the only reason my team (with @GeSnow) can imagine, because we have run the full data set twice and finished in about 3300s without any errors.
", "rawMarkdown": "Using utility scripts is the only reason my team (with @GeSnow) can imagine, because we have run the full data set twice and finished in about 3300s without any errors." }, { "id": 557305, "postDate": "2019-06-21T04:06:08.387Z", "content": "@lftuwujie @jihangz - this is one of the items we've uncovered so far as unintentionally erroring out a kernel. However only a few teams were impacted. Standby.
", "rawMarkdown": "@lftuwujie @jihangz - this is one of the items we've uncovered so far as unintentionally erroring out a kernel. However only a few teams were impacted. Standby.", "votes": 3 }, { "id": 557544, "postDate": "2019-06-21T08:50:39.773Z", "content": "Thanks @addisonhoward for the updating and time of investigation on this.
", "rawMarkdown": "Thanks @addisonhoward for the updating and time of investigation on this.", "votes": 1 } ] }, { "id": 554885, "postDate": "2019-06-18T06:25:46.130Z", "content": "@addisonhoward @fredericfont @plakal @eduardofonseca We are also facing the \"Invalidated for Stage 2 Reruns\" error. But we successfuly ran a fork of our submission on stage2 data. We didn't hardcode test data size. We also didn't load any pickled test data. We had code in the kernel to process the test data. Hope the issue is fixed soon.
", "rawMarkdown": "@addisonhoward @fredericfont @plakal @eduardofonseca We are also facing the \"Invalidated for Stage 2 Reruns\" error. But we successfuly ran a fork of our submission on stage2 data. We didn't hardcode test data size. We also didn't load any pickled test data. We had code in the kernel to process the test data. Hope the issue is fixed soon." }, { "id": 554801, "postDate": "2019-06-18T03:16:07.450Z", "content": "Thanks for all your effort on this!\nHope we all can get good result.
", "rawMarkdown": "Thanks for all your effort on this!\nHope we all can get good result." }, { "id": 554697, "postDate": "2019-06-17T23:17:19.270Z", "content": "One further note: As many of you have noted in the forums, not all submitted Kernels successfully ran on the Stage 2 dataset. The predominant reasons were either: A) a hard coded number of test samples, which didn't translate to the Stage 2 dataset size, or B) Using a pickled, precomputed feature file for the test set that was the wrong shape.
\n\nLess than 8% of Kernels didn't finish due to constraint violations.
\n\nKaggle does not provide feedback on individual error messages for Kernels that did not complete the Stage 2 re-run.
", "rawMarkdown": "One further note: As many of you have noted in the forums, not all submitted Kernels successfully ran on the Stage 2 dataset. The predominant reasons were either: A) a hard coded number of test samples, which didn't translate to the Stage 2 dataset size, or B) Using a pickled, precomputed feature file for the test set that was the wrong shape.\n\nLess than 8% of Kernels didn't finish due to constraint violations.\n\nKaggle does not provide feedback on individual error messages for Kernels that did not complete the Stage 2 re-run.", "replies": [ { "id": 554700, "postDate": "2019-06-17T23:25:18.273Z", "content": "Thanks for your explanation. Since my new kernel ran perfectly on the Stage 2 dataset, I don't think (A) or (B) caused my problem. And my new kernel finished within 1 hour, so I don't think I am in that 8% either. Anyway, I filed a ticket to the Compliance team. Hope they can help me with this issue.
", "rawMarkdown": "Thanks for your explanation. Since my new kernel ran perfectly on the Stage 2 dataset, I don't think (A) or (B) caused my problem. And my new kernel finished within 1 hour, so I don't think I am in that 8% either. Anyway, I filed a ticket to the Compliance team. Hope they can help me with this issue.", "votes": 1 }, { "id": 554810, "postDate": "2019-06-18T03:35:36.203Z", "content": "Same here π
", "rawMarkdown": "Same here π" } ] }, { "id": 564615, "postDate": "2019-06-29T17:51:03.863Z", "rawMarkdown": "", "isDeleted": true }, { "id": 554882, "postDate": "2019-06-18T06:21:20.333Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 560520, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-25T14:48:01.890000", "content": "Hi All,
\n\nKaggle has performed it's kernel re-runs and was able to reinstate some teams that had been mistakenly removed (utility scripts, released mid-competition, was the culprit!). The competition sponsors have asked that they have some time to perform some of their own checks prior to releasing the private leaderboard.
\n\nThanks for your patience!
", "votes": 16, "replies": [ { "id": 560526, "author_name": "Leo0523", "author_url": "", "post_date": "2019-06-25T14:51:36.553000", "content": "Great job! Thank you for sharing!
", "votes": 1, "replies": [] }, { "id": 560540, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-25T14:59:19.527000", "content": "thank you for your reply π
", "votes": 2, "replies": [] }, { "id": 560586, "author_name": "cfiken", "author_url": "", "post_date": "2019-06-25T15:49:01.227000", "content": "Thanks. I hope no more accident happen.
", "votes": 1, "replies": [] }, { "id": 560609, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-25T16:04:19.470000", "content": "I hope the sponsors don't take another week for their checks ππ
", "votes": 2, "replies": [] }, { "id": 561025, "author_name": "good good study", "author_url": "", "post_date": "2019-06-26T04:16:31.480000", "content": "finally, we can see the results, it has been a long waiting, thanks
", "votes": 1, "replies": [] }, { "id": 561034, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-26T04:26:51.040000", "content": "the final LB is yet to be published as the sponsors are also doing some validations at their end. Not sure how much time are they going to take to validate at their end.
", "votes": 2, "replies": [] }, { "id": 562169, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-27T03:22:18.010000", "content": "How long does the sponsor need to checkοΌ
", "votes": 2, "replies": [] }, { "id": 563660, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-28T14:39:37.697000", "content": "@addisonhoward - any chance we will see the LB today before everyone goes off for the weekend? or are we now looking for sometime next week now?
", "votes": 1, "replies": [] } ] }, { "id": 562145, "author_name": "OsciiArt", "author_url": "", "post_date": "2019-06-27T02:44:30.820000", "content": "The workshop paper deadline is getting close without the final result nor late submission. lol.
", "votes": 9, "replies": [ { "id": 564071, "author_name": "Eduardo Fonseca", "author_url": "", "post_date": "2019-06-29T00:10:01.030000", "content": "EDIT Monday July 1st: the DCASE workshop paper deadline has been extended to July 16. More info in this post.\nhttps://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/98192#latest-566202
\n\nworkshop paper deadline is specified here: http://dcase.community/workshop2019/
\n\nJuly 5th: abstract submission\nJuly 12th: final paper submission
\n\nThe results publication date was planned by the DCASE organization (and has been publicly specified all along), so that there is enough time before the workshop deadline(s).
\n\nWe encourage you to submit the the DCASE Workshop, regardless whether you submitted to the DCASE Challenge or not. What matters is the methodology, the approach, the ideas, rather than the tyranny of the leaderboard. The community will benefit from any ideas that you want to share :)
", "votes": 1, "replies": [] } ] }, { "id": 557056, "author_name": "OsciiArt", "author_url": "", "post_date": "2019-06-21T00:25:09.550000", "content": "How I wish this competition were synchronous kernel-only competition.
", "votes": 8, "replies": [ { "id": 560502, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-25T14:24:15.913000", "content": "indeed...the instant gratification competition finished after this one and the Private LB got finalized before this :(
", "votes": 1, "replies": [] } ] }, { "id": 563711, "author_name": "Ruslan Baikulov", "author_url": "", "post_date": "2019-06-28T15:40:09.303000", "content": "@addisonhoward, it looks like my second selected kernel was not executed and evaluated. Meanwhile, I just run it manually and submit via late submission and get correct results which may affect the private leaderboard.\n
Standby - we're investigating.
", "votes": 0, "replies": [] }, { "id": 563729, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-28T16:00:30.507000", "content": "Ruslan,
\n\nOur records show that you selected the following for scoring:
\n\n> Freesound Predict Dataset version 50/52\n>Stacking 008\nThis kernel errored due to an IndexError: list index out of range
\n\n>Freesound Predict Dataset version 52/52 \n>Stacking 007\nThis one ran successfully and was on the LB
\n\nKernel Freesound Predict 2 (version 8/8) Stacking 008 was not selected for your final two selections.
", "votes": 0, "replies": [] }, { "id": 563755, "author_name": "Ruslan Baikulov", "author_url": "", "post_date": "2019-06-28T16:30:53.810000", "content": "\"Kernel Freesound Predict 2 (version 8/8) Stacking 008\" is copy of \"Freesound Predict Dataset version 50/52\nStacking 008\"
", "votes": 2, "replies": [] }, { "id": 563767, "author_name": "Ruslan Baikulov", "author_url": "", "post_date": "2019-06-28T16:46:57.397000", "content": "Download .zip dataset from kaggle, upload to new dataset and copy paste code.
", "votes": 2, "replies": [] }, { "id": 564040, "author_name": "Ruslan Baikulov", "author_url": "", "post_date": "2019-06-28T23:20:01.827000", "content": "Now I forked \"Freesound Predict Dataset version 50/52\", kernel failed with IndexError error because my code can't find model weights.
\nI checked the 'input' folder through the console and found that files in running kernel do not match the files from data in \"Draft Environment\". I will report more information to compliance.\n
While competing I experienced similar problems with datasets containing models as Ruslan. It can be reproduced with following steps:\n1. Add dataset to kernel.\n2. Commit version.\n3. Update dataset.\n4. Fork or start editing.\n5. Sometimes old version of dataset were available in kernel while new one is shown in interface.
", "votes": 2, "replies": [] }, { "id": 564275, "author_name": "Ruslan Baikulov", "author_url": "", "post_date": "2019-06-29T08:15:58.253000", "content": "I have done several tests and now I am quite sure that one of my selected for the final submission kernels was not executed because of a kaggle bug.\nFailed kernel: https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383
\n\n> Freesound Predict Dataset version 50/52\nStacking 008\nThis kernel errored due to an IndexError: list index out of range
\n\nI have made the kernel and dataset public. Anyone can run the following tests:
\n\nTest 1: Successful execution with the code and data from kernel \n1. Go to https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383\n2. Download experiments.zip from data sources.\n3. Create a new dataset with the name \"Freesound train dataset\", upload experiments.zip from 2.
\n4. Create a new kernel, add dataset from 3. and competition data.\n5. Commit.
The kernel will successfully run with a private score of 0.75980 which now would be the first place.
\n\nTest 2: Error reason is kaggle dataset versioning bug \n1. Go to https://www.kaggle.com/romul0212/freesound-predict-dataset?scriptVersionId=15465383\n2. Fork the kernel.\n3. Run command in console \"ls ../input/freesound-train-dataset/experiments\"
\n\nYou will see that that files in running kernel do not match the files from data in \"Draft Environment\" and data in experiments.zip from Test 1.\nIf you commit the kernel, it will fail with IndexError because my code can't find models weight. On stage 1 kernel was run successfully. Means that there were was correct data. But in stage 2 data replaced with a different version of the dataset due the kaggle bug.
", "votes": 3, "replies": [] }, { "id": 564535, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-29T15:11:18.453000", "content": "Hi - we're continuing to look into this.
\n\nThanks,\nKaggle Team
", "votes": 3, "replies": [] } ] }, { "id": 559978, "author_name": "ilovescience", "author_url": "", "post_date": "2019-06-24T22:56:14.590000", "content": "When can we expect an update on this? It has been 7 days since the flawed leaderboard was taken down.
", "votes": 4, "replies": [] }, { "id": 565324, "author_name": "Acku Chauhan", "author_url": "", "post_date": "2019-06-30T19:00:28.223000", "content": "@addisonhoward For our team it shows only 1 entry and there is only 1 rerun but we've submitted two entries and both are running fine on private dataset. Please let me know if any other information is needed from my side.
", "votes": 1, "replies": [] }, { "id": 563984, "author_name": "OsciiArt", "author_url": "", "post_date": "2019-06-28T21:05:45.753000", "content": "Will you release audio data and true labels of stage 1? It will help us to verify our solution.
", "votes": 1, "replies": [ { "id": 564062, "author_name": "Eduardo Fonseca", "author_url": "", "post_date": "2019-06-28T23:50:59.123000", "content": "We will do that, but it'll take some time; probably at the end of summer. We 'll be sure to post here when the data are available. thanks.
", "votes": 1, "replies": [] }, { "id": 564351, "author_name": "Chi Kim", "author_url": "", "post_date": "2019-06-29T10:01:29.943000", "content": "Thanks for organizing the competition! Congrats to all the winners! When can we start submitting again for knowledge? I just tried, and it says 0.000 and doesn't show up in the lb.
", "votes": 0, "replies": [] }, { "id": 564536, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-29T15:13:00.930000", "content": "Hi Chi,
\n\nYou can continue to submit using a Late Submission, however it will not appear on the leaderboard, as the competition has concluded.
", "votes": 0, "replies": [] }, { "id": 564545, "author_name": "Chi Kim", "author_url": "", "post_date": "2019-06-29T15:30:07.783000", "content": "Thanks @addisonhoward! I see. I found the score in \"my submissions\" page. I can get the score, but not the ranking. There's no way to know where my late submission score stands in comparison to other people's late submissions then? Is that correct?
", "votes": 0, "replies": [] }, { "id": 564552, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-29T15:37:11.433000", "content": "That's correct - you can go to the Leaderboard to compare where you would have squeezed in, but there is no formalized tracking of rankings after the competition has concluded
", "votes": 0, "replies": [] }, { "id": 564617, "author_name": "Arunkumar Venkataramanan", "author_url": "", "post_date": "2019-06-29T17:53:14.310000", "content": "Hello @addisonhoward May I know what happened to my submissions? Will you let me know about my submissions and ranking for the competition?
", "votes": -3, "replies": [] }, { "id": 564689, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-29T20:18:53.953000", "content": "Hi Arunkumar,
\n\nWe cannot provide individual critique on why code failed in a two-stage Kernels-only competition. You can re-run your kernel on the Stage 2 data and see what errors may have come from it.
\n\nWe are currently investigating a potential bug that's the result of dataset versioning, and will notify you if that was the result of your kernel failing.
", "votes": 0, "replies": [] }, { "id": 564952, "author_name": "Miguel Pinto", "author_url": "", "post_date": "2019-06-30T08:23:55.417000", "content": "@addisonhoward The number of submissions per day is still limited to 2. Now that the competition is over can't this limit be removed so that we can perform the ablation studies faster?
", "votes": 0, "replies": [] }, { "id": 565287, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-30T17:52:29.583000", "content": "Miguel,
\n\nWe generally don't change the submission limit after the competition has concluded for the same reason we don't release the Private LB labels - we want to keep the competition open, without the labels revealed, so people can continue to make submissions and not have the dataset available to them.
\n\nHowever, we ultimately allow the competition sponsors to make that judgement.
", "votes": 1, "replies": [] } ] }, { "id": 560516, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-25T14:45:25.800000", "content": "the instant gratification competition finished... \nHow long do we have to wait?
", "votes": 1, "replies": [] }, { "id": 554711, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-17T23:44:00.330000", "content": "Update: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.
", "votes": 1, "replies": [ { "id": 554717, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2019-06-18T00:16:06.503000", "content": "Well as long as it wont take another week Β―\\_(γ)_/Β―
", "votes": 7, "replies": [] }, { "id": 554817, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-18T03:43:15.650000", "content": "@suicaokhoailang - your statement makes me remember the massive delay in the recent Santander competition. I seriously hope it's not that long.
", "votes": 0, "replies": [] }, { "id": 554947, "author_name": "Chi Kim", "author_url": "", "post_date": "2019-06-18T08:12:52.370000", "content": "For the latest run by Kaggle, it says wait time is -506356 seconds, execution time is 1 seconds , and score is 0.00000. :(\nI just copied my kernel and ran, and it successfully completed in 1328.2 seconds.\nHopefully it's not my script's fault.
", "votes": 2, "replies": [] }, { "id": 554956, "author_name": "Eric Bouteillon", "author_url": "", "post_date": "2019-06-18T08:27:32.887000", "content": "Hello,
\n\nIt seems my kernels passed the kaggle rerun, as I saw my new position on the temporary private leaderboard before its withdrawal, and I have similar display as you about the latest run I did. I didn't save the leaderboard result, so I can't help if you passed it or not.\nI wish you the best.
", "votes": 0, "replies": [] }, { "id": 555765, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2019-06-19T11:58:11.900000", "content": "@addisonhoward So are there any updates on this?
", "votes": 1, "replies": [] }, { "id": 555836, "author_name": "RS Turley", "author_url": "", "post_date": "2019-06-19T13:58:39.863000", "content": "I'm guessing a source of some of the errors was that some of us (including me!) assumed that the sample_submission.csv file would be updated for the stage 2 data. I know a few of the public kernels did this as well. It just occurred to me that my kernel uses that sample submission file to create the final submission file.
", "votes": 1, "replies": [] }, { "id": 555842, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-19T14:04:16.050000", "content": "If that's the case then even I also made the same mistake :( and a costly mistake it seems :(
", "votes": 0, "replies": [] }, { "id": 555847, "author_name": "Chi Kim", "author_url": "", "post_date": "2019-06-19T14:14:29.493000", "content": "I think sample_submission.csv did get updated? If you look at it now, it has 3361 rows. Before it had 1120 rows.
", "votes": 2, "replies": [] }, { "id": 555851, "author_name": "Maxwell", "author_url": "", "post_date": "2019-06-19T14:18:36.490000", "content": "I used sample_submission.csv to make an inference csv and it did not make an error.
\nI guess the reason would be another.
I guess we will know the source of errors only when organizers share an update...till then it's agonizing wait
", "votes": 1, "replies": [] }, { "id": 555871, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-19T14:49:47.513000", "content": "@suicaokhoailang - not yet, thanks for your patience.
", "votes": 0, "replies": [] }, { "id": 555898, "author_name": "Dmitrii Krasnov", "author_url": "", "post_date": "2019-06-19T15:28:15.537000", "content": "I think some of you don't understand: if you present in leaderboard - your submission was valid and you don't have to worry :) But for others (like me) there was some sort of mistake, so we didn't pass to the second stage - as you can see, I currently don't have my place on leaderboard near my nickname. But in my case I can run kernel without error and do a submission (and some other people can do this aswell) - so we can only hope that organizers will find error and bring us back to competition.
", "votes": 7, "replies": [] }, { "id": 556059, "author_name": "Darkate", "author_url": "", "post_date": "2019-06-19T19:33:46.433000", "content": "@koba35 Do you have any external utility scripts in your kernel? I am thinking whether that causes the error since that's a new feature introduced recently.
", "votes": 1, "replies": [] }, { "id": 556061, "author_name": "Dmitrii Krasnov", "author_url": "", "post_date": "2019-06-19T19:40:03.947000", "content": "Yep, I think this is the case.
", "votes": 0, "replies": [] }, { "id": 556650, "author_name": "Leo0523", "author_url": "", "post_date": "2019-06-20T13:46:32.863000", "content": "@addisonhoward Would you tell me the order of magnitude of time until publish \nthe leaderboard? a few days? a week? a few weeks?
", "votes": 6, "replies": [] }, { "id": 557245, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2019-06-21T02:56:30.243000", "content": "The DCASE papers submission deadline will be due on 12th July so there is that.
", "votes": 0, "replies": [] }, { "id": 557312, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-21T04:11:09.887000", "content": "@hattan0523 - We're doing our best to get them out sooner rather than later. I know that's not the answer you're hoping for, but we're re-running kernels, doing some error checking, and diagnosing what is user error (as noted here), and what may have been a different error.
\n\nOne that we've specifically uncovered are errors thrown for utility scripts, which were released mid-competition. We've identified those and are looking into others.
\n\nWe certainly hope to have the results re-released early next week, but we can't make any guarantees, as we keep troubleshooting.
\n\nOn the bright side, and as I stall for time, our synchronous kernels-only competition just concluded, and should address a majority of these issues. We're excited to see what that brings for the future state!
", "votes": 6, "replies": [] }, { "id": 557345, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-21T04:43:15.647000", "content": "@addisonhoward - thanks for sharing the some sort of tentative timeline. I can personally take a break from checking Kaggle every few hours in the hope of an update. I will come back sometime next week. Hope everyone has a good weekend!!
\n\nCheers!!
", "votes": 2, "replies": [] }, { "id": 557373, "author_name": "Eric Bouteillon", "author_url": "", "post_date": "2019-06-21T05:14:01.907000", "content": "Thank you @addisonhoward for the update. Sync KO sounds like a nice evolution as it will find these problems earlier and allow immediate private leaderboard results.
", "votes": 1, "replies": [] }, { "id": 559698, "author_name": "Leo0523", "author_url": "", "post_date": "2019-06-24T13:15:14.193000", "content": "Thank you for sharing it. I'm looking forward to opening leaderboard!
", "votes": 0, "replies": [] }, { "id": 560066, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2019-06-25T02:17:12.507000", "content": "Sometimes dreams can come true\n> Khoi Nguyen wrote\n> \n> > Well as long as it wont take another week Β―(γ)/Β―
", "votes": 1, "replies": [] }, { "id": 560093, "author_name": "Darkate", "author_url": "", "post_date": "2019-06-25T03:14:54.867000", "content": "@suicaokhoailang I knew it's gonna be another week because they re-ran all submissions again... First re-ran took a week, second re-ran should be the same
", "votes": 0, "replies": [] }, { "id": 560098, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2019-06-25T03:25:03.567000", "content": "There is no reasons to re-run every team on the leaderboard. Beside, the Quora competitions had 4000+ teams, 1400+ were valid, the allowed run time were 2 hours and it took less than 2 weeks. In this case I think it was just lots of manual checking and debugging.
", "votes": 3, "replies": [] }, { "id": 560101, "author_name": "Darkate", "author_url": "", "post_date": "2019-06-25T03:36:55.170000", "content": "I agree with you, but I remember someone from Kaggle (probably Addison?) saying they \"re-ran all kernels\".
", "votes": 0, "replies": [] } ] }, { "id": 554695, "author_name": "Darkate", "author_url": "", "post_date": "2019-06-17T23:16:34.010000", "content": "Hi Addison. I checked my leaderboard page and it showed \"Invalidated for Stage 2 Reruns\". I tested my kernel on 2nd stage test data couple days ago and it completed in 1 hour without any problem. Can I have an explanation about this? I made another post in the thread but I think I can get a quicker response here. Thanks!
", "votes": 1, "replies": [] }, { "id": 562933, "author_name": "Nanashi", "author_url": "", "post_date": "2019-06-27T17:25:53.483000", "content": "@addisonhoward news?\nbtw, the burner account NovaPulsar got a discussion bronze medal having 6 downvotes lol
", "votes": 2, "replies": [ { "id": 562941, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-27T17:30:43.500000", "content": "I guess the bronze medal has to do with how soon the bronze is awarded for a upvote. I was the first one to reply to him and when I replied to him there was one upvote on his comment at that time. Guess the bronze medal got awarded at that hour...post that 7 downvotes resulted in net of 6 downvotes. Maybe it's time Kaggle should revoke the discussion bronze medal for the cases when the final number of votes is less than 0.
", "votes": 0, "replies": [] }, { "id": 562945, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-27T17:36:23.867000", "content": "The Freesound team is working on validating some of the teams that submitted to DCASE - they have not provided us with an update on releasing the competition LB.
\n\nAs for medals, they typically propagate every few hours or so, but are based on the last static version. I've seen some discussions go from gold medal to silver after downvotes :)
", "votes": 0, "replies": [] }, { "id": 562956, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-27T17:42:29.283000", "content": "@addisonhoward - thanks for sharing the update. Although it's very disappointing to hear that we still don't have a concrete timeline regarding when the final LB would be published :(
\n\nit's an agonizing wait everyday :(
", "votes": 0, "replies": [] }, { "id": 563694, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-28T15:23:33.850000", "content": "Just got permission to release! Take a look!
", "votes": 6, "replies": [] }, { "id": 563778, "author_name": "Arunkumar Venkataramanan", "author_url": "", "post_date": "2019-06-28T17:02:06.570000", "content": "@addisonhoward Kindly please look into my kernel submissions? Please let me know what happened to them as I forgot to select my kernels submissions...
", "votes": 0, "replies": [] }, { "id": 564065, "author_name": "Eduardo Fonseca", "author_url": "", "post_date": "2019-06-28T23:58:13.297000", "content": "Hi,
\n\nI'm sorry you were checking every day for results. Nonetheless, there has been indeed an expected timeline that we (organizers) thought you were aware of. As you know, this evaluation is part of the DCASE Challenge, whose results notification date has been specified in the DCASE Challenge website all along (see left bar Schedule).
\n\nhttp://dcase.community/challenge2019/
", "votes": 0, "replies": [] } ] }, { "id": 557075, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-21T00:35:32.927000", "content": "Ten days have passed since the competition ended...
", "votes": 2, "replies": [ { "id": 557471, "author_name": "Ilham Firdausi Putra", "author_url": "", "post_date": "2019-06-21T07:47:21.160000", "content": "What is the cost of bugs? Itβs not that we will mistake them for a feature. The real danger is that, if we have enough bugs, then we no longer can receive the competition result as soon as the competition ended.
", "votes": 1, "replies": [] }, { "id": 560242, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-25T07:45:47.387000", "content": "Two weeks passed...
", "votes": 1, "replies": [] }, { "id": 560253, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-25T07:59:33.830000", "content": "and still waiting.....
", "votes": 0, "replies": [] } ] }, { "id": 562534, "author_name": "NovaPulsar", "author_url": "", "post_date": "2019-06-27T10:27:42.983000", "content": "One of the worst competitons ever. A huge disrespect to all participants
", "votes": -16, "replies": [ { "id": 562569, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-27T10:55:33.533000", "content": "I respectfully disagree with you. The learning opportunities in this competition has been tremendous (atleast for me). The delay in LB finalization is frustrating, but saying that the competition is a huge disrespect to all competitors is a bit too harsh. I am sure Kaggle team will learn from the issues in this competition and probably try to avoid them in future. I thank the organizers for arranging a competition with such a unique topic that pushed us to try so many new things.
\n\nCheers!! πβ
", "votes": 4, "replies": [] }, { "id": 562578, "author_name": "OsciiArt", "author_url": "", "post_date": "2019-06-27T11:04:26.450000", "content": "I don't think so. There are so many worse competitions with fatal leakage or a meaningless target setting. This competition has some problems but still has so many things to learn.
", "votes": 7, "replies": [] }, { "id": 562588, "author_name": "Miguel Pinto", "author_url": "", "post_date": "2019-06-27T11:19:15.187000", "content": "Disrespect would be to ignore the teams affected by the error. I know the long waiting for the results can be frustrating but I like to see that they are checking the results to make sure everything is okay. If anything it is a sign of respect for all participants and for the fairness of the competition.
\n\nEdit: And I find weird that you just created an account to write this comment.
", "votes": 13, "replies": [] }, { "id": 562866, "author_name": "Acku Chauhan", "author_url": "", "post_date": "2019-06-27T16:05:29.860000", "content": "@mnpinto Good catch. The account indeed got created few hours ago! Should be reported as spam.
", "votes": 3, "replies": [] }, { "id": 564078, "author_name": "Eduardo Fonseca", "author_url": "", "post_date": "2019-06-29T00:29:21.880000", "content": "Hi NovaPulsar,
\n\nWe're sorry you feel that way. I just want to stress that all the decisions taken by the organizing team have always been taken thinking on the participants' benefit. If it's not up to your standards, we welcome you to organize a competition yourself and share with us any recommendation, so that we can make it better next time.
\n\nAlso, we're a bit puzzled as it seems you just created a new account to post this, and you're not giving any reasons to argument your claim. Anyway, we welcome you to the competition!
\n\nPS thanks @mnpinto @osciiart @sandy1112 and everyone who downvoted NovaPulsar's post for your support. Organizing a competition indeed involves a lot of work behind the scenes, and it is not an easy task.
", "votes": 3, "replies": [] } ] }, { "id": 562104, "author_name": "Naruhiko Nakanishi", "author_url": "", "post_date": "2019-06-27T01:36:53.043000", "content": "Happy Modeling!
", "votes": 0, "replies": [ { "id": 562120, "author_name": "sandy1112", "author_url": "", "post_date": "2019-06-27T02:20:12.227000", "content": "For the last 2 weeks it's more like happy waiting or get frustrated while waiting based on how patient one is π€―π΄π΄
", "votes": 1, "replies": [] } ] }, { "id": 555249, "author_name": "wanglilin", "author_url": "", "post_date": "2019-06-18T16:19:25.157000", "content": "emm
", "votes": 0, "replies": [] }, { "id": 554969, "author_name": "Dmitrii Krasnov", "author_url": "", "post_date": "2019-06-18T08:53:02.730000", "content": "I also faced error in kernel and dropped off leaderboard - but I successfully ran kernel on new test data in 45 minutes and got a valid submission file. I even can do a late submission without error :) So it definitely some sort of error. Maybe cos I am using utility scripts?
", "votes": 0, "replies": [ { "id": 556788, "author_name": "Jie Wu", "author_url": "", "post_date": "2019-06-20T16:49:51.230000", "content": "Using utility scripts is the only reason my team (with @GeSnow) can imagine, because we have run the full data set twice and finished in about 3300s without any errors.
", "votes": 0, "replies": [] }, { "id": 557305, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-21T04:06:08.387000", "content": "@lftuwujie @jihangz - this is one of the items we've uncovered so far as unintentionally erroring out a kernel. However only a few teams were impacted. Standby.
", "votes": 3, "replies": [] }, { "id": 557544, "author_name": "Jie Wu", "author_url": "", "post_date": "2019-06-21T08:50:39.773000", "content": "Thanks @addisonhoward for the updating and time of investigation on this.
", "votes": 1, "replies": [] } ] }, { "id": 554885, "author_name": "Deepak Singh Rawat", "author_url": "", "post_date": "2019-06-18T06:25:46.130000", "content": "@addisonhoward @fredericfont @plakal @eduardofonseca We are also facing the \"Invalidated for Stage 2 Reruns\" error. But we successfuly ran a fork of our submission on stage2 data. We didn't hardcode test data size. We also didn't load any pickled test data. We had code in the kernel to process the test data. Hope the issue is fixed soon.
", "votes": 0, "replies": [] }, { "id": 554801, "author_name": "0xFunky", "author_url": "", "post_date": "2019-06-18T03:16:07.450000", "content": "Thanks for all your effort on this!\nHope we all can get good result.
", "votes": 0, "replies": [] }, { "id": 554697, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-06-17T23:17:19.270000", "content": "One further note: As many of you have noted in the forums, not all submitted Kernels successfully ran on the Stage 2 dataset. The predominant reasons were either: A) a hard coded number of test samples, which didn't translate to the Stage 2 dataset size, or B) Using a pickled, precomputed feature file for the test set that was the wrong shape.
\n\nLess than 8% of Kernels didn't finish due to constraint violations.
\n\nKaggle does not provide feedback on individual error messages for Kernels that did not complete the Stage 2 re-run.
", "votes": 0, "replies": [ { "id": 554700, "author_name": "Darkate", "author_url": "", "post_date": "2019-06-17T23:25:18.273000", "content": "Thanks for your explanation. Since my new kernel ran perfectly on the Stage 2 dataset, I don't think (A) or (B) caused my problem. And my new kernel finished within 1 hour, so I don't think I am in that 8% either. Anyway, I filed a ticket to the Compliance team. Hope they can help me with this issue.
", "votes": 1, "replies": [] }, { "id": 554810, "author_name": "cfiken", "author_url": "", "post_date": "2019-06-18T03:35:36.203000", "content": "Same here π
", "votes": 0, "replies": [] } ] }, { "id": 564615, "author_name": "", "author_url": "", "post_date": "2019-06-29T17:51:03.863000", "content": "", "votes": 0, "replies": [] }, { "id": 554882, "author_name": "", "author_url": "", "post_date": "2019-06-18T06:21:20.333000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "560520": "Hi All,\n\nKaggle has performed it's kernel re-runs and was able to reinstate some teams that had been mistakenly removed (utility scripts, released mid-competition, was the culprit!). The competition sponsors have asked that they have some time to perform some of their own checks prior to releasing the private leaderboard.\n\nThanks for your patience!", "554687": "EDIT, 15:22 UTC, Friday June 28: The leaderboard has been re-released!\n\nEDIT, 23:45pm UTC, Monday June 17: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.\n\nPrevious Post: Kernels have been re-run for the second stage, and the private leaderboard has been released! Thank you all for your hard work in this competition and congratulations to our winners! Note: the \"change\" in leaderboard score from Public to Private leaderboards is unreliable, as the public leaderboard will not reflect the Stage 1 scores.\n\nAnd a continued congratulation to the many of you may reach a new tier because of this competition, as well as far more of you for which this was your first competitions.\n\nWe're also continually pleased to see how Kaggle Competitions serve as a great medium and vehicle through which the greater data science community can learn and develop their machine learning abilities and skills. Beginners and experts can come together to start, grow, and succeed. \n\nTo those of you for whom this was your first Kaggle competition - great work! We hope to see you again in a future competition! To those for whom this competition resulted in a Kaggle tier promotion - congratulations! Thanks to the many of you who have shared your solutions in the forums for others to learn from, and have shared valuable insights.\n\nWe also would like to give special thanks to Eduardo and Frederic from UPF, and Manoj and Dan from Google for allowing us to run this competition as a part of the DCASE challenge - it's especially great to have a unique audio dataset like this! We hope to be able to work with them in the future.\n\nAs noted in the forums, the competition hosts reserve the right to review any team's Kernel submission for adherence to the posted rules. Should it be determined that any of these teams have violated the rules, we will remove them from the leaderboard. If you think you were removed by mistake, please contact [compliance][1]. Please fill in all the fields honestly.\n\nThe top potential winning teams will be contacted in the next few days via email to provide their winning solutions for sponsor review. Should we need to move down the leaderboard for any reason, we will do so and continue to make contact with subsequent teams.\n\nYou should see your points and medals awarded soon, however it may take a few hours to propagate through.\n\nWe hope to see you in our next competitions!\n\nHappy Modeling!\n\ncc/ @fredericfont @plakal @eduardofonseca \n\n [1]: https://www.kaggle.com/compliance", "562145": "The workshop paper deadline is getting close without the final result nor late submission. lol.", "557056": "How I wish this competition were synchronous kernel-only competition. ", "563711": "@addisonhoward, it looks like my second selected kernel was not executed and evaluated. Meanwhile, I just run it manually and submit via late submission and get correct results which may affect the private leaderboard.\n\n", "559978": "When can we expect an update on this? It has been 7 days since the flawed leaderboard was taken down.", "565324": "@addisonhoward For our team it shows only 1 entry and there is only 1 rerun but we've submitted two entries and both are running fine on private dataset. Please let me know if any other information is needed from my side.", "563984": "Will you release audio data and true labels of stage 1? It will help us to verify our solution.", "560516": "the instant gratification competition finished... \nHow long do we have to wait?", "554711": "Update: Kaggle has withdrawn the private leaderboard to investigate a few of the Kernel errors a bit further. Standby. We will update this thread when we have more information.", "554695": "Hi Addison. I checked my leaderboard page and it showed \"Invalidated for Stage 2 Reruns\". I tested my kernel on 2nd stage test data couple days ago and it completed in 1 hour without any problem. Can I have an explanation about this? I made another post in the thread but I think I can get a quicker response here. Thanks!", "562933": "@addisonhoward **news?**\nbtw, the burner account **NovaPulsar** got a discussion bronze medal having 6 downvotes lol", "557075": "Ten days have passed since the competition ended...", "562534": "One of the worst competitons ever. A huge disrespect to all participants", "562104": "Happy Modeling!", "555249": "emm", "554969": "I also faced error in kernel and dropped off leaderboard - but I successfully ran kernel on new test data in 45 minutes and got a valid submission file. I even can do a late submission without error :) So it definitely some sort of error. Maybe cos I am using utility scripts?", "554885": "@addisonhoward @fredericfont @plakal @eduardofonseca We are also facing the \"Invalidated for Stage 2 Reruns\" error. But we successfuly ran a fork of our submission on stage2 data. We didn't hardcode test data size. We also didn't load any pickled test data. We had code in the kernel to process the test data. Hope the issue is fixed soon.", "554801": "Thanks for all your effort on this!\nHope we all can get good result.", "554697": "One further note: As many of you have noted in the forums, not all submitted Kernels successfully ran on the Stage 2 dataset. The predominant reasons were either: A) a hard coded number of test samples, which didn't translate to the Stage 2 dataset size, or B) Using a pickled, precomputed feature file for the test set that was the wrong shape.\n\nLess than 8% of Kernels didn't finish due to constraint violations.\n\nKaggle does not provide feedback on individual error messages for Kernels that did not complete the Stage 2 re-run.", "564615": "", "554882": "" } }