Judging from my own experience i assume many of us do not go into details of every post. Just see what happened to deep fake where No2 team got removed because they did not see that one external dataset they used was actually banned by Kaggle in the discussion thread.
\n\nTo avoid such situations in the future and to make it easier to find relevant information I suggest KAGGLE includes the list of allowed external data in the Official External Data Thread description.
\n\nWhat do you guys think?
", "messages": [ { "id": "886811", "postDate": "06/15/2020 09:43:55", "content": "Judging from my own experience i assume many of us do not go into details of every post. Just see what happened to deep fake where No2 team got removed because they did not see that one external dataset they used was actually banned by Kaggle in the discussion thread.
\n\nTo avoid such situations in the future and to make it easier to find relevant information I suggest KAGGLE includes the list of allowed external data in the Official External Data Thread description.
\n\nWhat do you guys think?
", "rawMarkdown": "Judging from my own experience i assume many of us do not go into details of every post. Just see what happened to [deep fake](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/158506) where No2 team got removed because they did not see that one external dataset they used was actually banned by Kaggle in the discussion thread.\n\nTo avoid such situations in the future and to make it easier to find relevant information I suggest KAGGLE includes the list of allowed external data in the **Official External Data Thread** description.\n\nWhat do you guys think?", "votes": null }, { "id": "886824", "postDate": "06/15/2020 09:51:41", "content": "Of course no one want to repeat what happened there, there should be steps taken to prevent such misadventures.
\n\nI believe your suggestion may help in a long way. Nice initiative! :)
", "rawMarkdown": "Of course no one want to repeat what happened there, there should be steps taken to prevent such misadventures. \n\nI believe your suggestion may help in a long way. Nice initiative! :)", "votes": null }, { "id": "886873", "postDate": "06/15/2020 10:30:19", "content": "Yes, especially now that apparently we are obliged to use external data in order to be eligible for prizes!
\n\nAlso, what amount of external data is sufficient to be eligible for prizes? If I use 1 external image of a dog or cat, does that count? We are (again!) in for a good time post-competition if these subjective details are not cleared up soon.
", "rawMarkdown": "Yes, especially now that apparently we are obliged to use external data in order to be eligible for prizes! \n\nAlso, what amount of external data is sufficient to be eligible for prizes? If I use 1 external image of a dog or cat, does that count? We are (again!) in for a good time post-competition if these subjective details are not cleared up soon.", "votes": null }, { "id": "886923", "postDate": "06/15/2020 11:13:36", "content": "thanks
", "rawMarkdown": "thanks", "votes": null }, { "id": "887053", "postDate": "06/15/2020 13:05:51", "content": "@group16 good point.\nperhaps @juliaelliott can help us to clarify this?
", "rawMarkdown": "group16 good point.\nperhaps @juliaelliott can help us to clarify this?", "votes": null }, { "id": "887067", "postDate": "06/15/2020 13:14:27", "content": "Last year it was allowed to use any external data you want as long as you disclose it in the submission and paper.\nProbably it's the same rule, but your concern is totally fair!
", "rawMarkdown": "Last year it was allowed to use any external data you want as long as you disclose it in the submission and paper.\nProbably it's the same rule, but your concern is totally fair!", "votes": null }, { "id": "887206", "postDate": "06/15/2020 14:37:02", "content": "@group16 Can you explain the following comment
\n\n\n\n\nYes, especially now that apparently we are obliged to use external data in order to be eligible for prizes!
\n
Are you saying we are required to use external data? I don't understand.
", "rawMarkdown": "group16 Can you explain the following comment\n> Yes, especially now that apparently we are obliged to use external data in order to be eligible for prizes!\n\nAre you saying we are required to use external data? I don't understand.", "votes": null }, { "id": "887209", "postDate": "06/15/2020 14:41:04", "content": "According to the current rules, you are obliged to use external data in order to be eligible for prize money:
\n\nThe following provision supersedes General Rules Section 7.C. below: \"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will post any External Data used and the provenance of the data to the official competition forum prior to the Entry Deadline, and if the data is publicly available, include the link to such access. To receive a Prize, a potential winner must have used External Data available to use by all participants of the competition for purposes of the competition, for use that includes research or academic purposes, at no cost to the other participants (\"Public External Data\"). Participants who do not use Public External Data are not eligible to receive a Prize, but may retain their non-monetary winning features in accordance with Section 10 of the General Rules.\"
", "rawMarkdown": "According to the current rules, you are **obliged** to use external data in order to be eligible for prize money:\n\n1. EXTERNAL DATA\n\nThe following provision supersedes General Rules Section 7.C. below: \"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will post any External Data used and the provenance of the data to the official competition forum prior to the Entry Deadline, and if the data is publicly available, include the link to such access. To receive a Prize, a potential winner must have used External Data available to use by all participants of the competition for purposes of the competition, for use that includes research or academic purposes, at no cost to the other participants (\"Public External Data\"). **Participants who do not use Public External Data are not eligible to receive a Prize, but may retain their non-monetary winning features in accordance with Section 10 of the General Rules.\"**", "votes": null }, { "id": "887350", "postDate": "06/15/2020 16:15:55", "content": "You are not obligated to use external data to win a prize, but you are obligated to use Public external data (and post on the official thread) if you do choose to use external data for prize eligibility.
", "rawMarkdown": "You are not obligated to use external data to win a prize, but you are obligated to use Public external data (and post on the official thread) if you do choose to use external data for prize eligibility.", "votes": null }, { "id": "887368", "postDate": "06/15/2020 16:23:44", "content": "Ok, and what's the difference between \"external data\" and \"Public external data\"? 😓 stupid question?
", "rawMarkdown": "Ok, and what's the difference between \"external data\" and \"Public external data\"? 😓 stupid question?", "votes": null }, { "id": "888312", "postDate": "06/16/2020 09:12:05", "content": "\"Public External Data\" -> Everyone else can also use the data.\n\"External Data\" -> Only you can use the data.
\n\nmakes sense. Thank you for clarifying.
", "rawMarkdown": "makes sense. Thank you for clarifying.", "votes": null }, { "id": "890292", "postDate": "06/17/2020 12:37:46", "content": "No problem! :)
", "rawMarkdown": "No problem! :)", "votes": null }, { "id": "891719", "postDate": "06/18/2020 12:14:41", "content": "I till hope kaggle considers this...
", "rawMarkdown": "I till hope kaggle considers this...", "votes": null }, { "id": "891726", "postDate": "06/18/2020 12:19:58", "content": "I'm afraid they may not be on this. :/
", "rawMarkdown": "I'm afraid they may not be on this. :/", "votes": null }, { "id": "891876", "postDate": "06/18/2020 13:59:24", "content": "Why do you think so. It would make the platform better, not?
", "rawMarkdown": "Why do you think so. It would make the platform better, not?", "votes": null }, { "id": "893704", "postDate": "06/19/2020 20:10:27", "content": "The rules considering external datasets where very strict in \"DFDC\" unlike any other competition. In the \"External Data thread\" of that competition there was a lot of discussion about which external datasets you can use, and Kaggle personal were vary active in answering and clarifying the rules. It was obvious from their answers that the only external data which is allowed is \"the usual\" pre-tained models.
\n\nThis competition is not DFDC, and the main source of external data - previous ISIC competition is legal, so we don't need to be too defensive. (and if you still worry about something, just ask about it directly, they'll answer)
", "rawMarkdown": "The rules considering external datasets where very strict in \"DFDC\" unlike any other competition. In the \"External Data thread\" of that competition there was a lot of discussion about which external datasets you can use, and Kaggle personal were vary active in answering and clarifying the rules. It was obvious from their answers that the only external data which is allowed is \"the usual\" pre-tained models.\n\nThis competition is not DFDC, and the main source of external data - previous ISIC competition is legal, so we don't need to be too defensive. (and if you still worry about something, just ask about it directly, they'll answer)", "votes": null }, { "id": "897479", "postDate": "06/22/2020 22:03:58", "content": "@valanm I appreciate your recommendation, but realistically we (Kaggle) are not in a position to be vetting every external dataset. The evaluation and enforcement of external dataset eligibility is ultimately in the hands of the host for each competition. The extent of scrutiny of external data or any competition-specific rules a host opts to add is very host-specific (some, frankly, don't care), so the proposed approach will not be scalable or practical for all hosts.
\n\nSo what does that mean for this competition? In this competition, the host has opened up the opportunity for any external data to be used (including private/non-public datasets), due to their interests in inclusivity of all approaches. However, for the purposes of prize-eligibility, if you opt to use external datasets (which I must emphasize is not a requirement), then those datasets must be public (with licensing that minimally includes academic/research use) with dataset provenance clearly shared on the external data thread.
\n\nIn the interest of mitigating any issues around uncertainty in external data use, we are urging hosts (this competition included) to monitor closely the external data thread, in the event questions or concerns are raised around the use of certain datasets. If you check that thread, you'll see many of these detailed questions have already been addressed, so I encourage that to be the method for effective escalation.
", "rawMarkdown": "valanm I appreciate your recommendation, but realistically we (Kaggle) are not in a position to be vetting every external dataset. The evaluation and enforcement of external dataset eligibility is ultimately in the hands of the host for each competition. The extent of scrutiny of external data or any competition-specific rules a host opts to add is very host-specific (some, frankly, don't care), so the proposed approach will not be scalable or practical for all hosts.\n\nSo what does that mean for this competition? In this competition, the host has opened up the opportunity for any external data to be used (including private/non-public datasets), due to their interests in inclusivity of all approaches. However, for the purposes of prize-eligibility, if you opt to use external datasets (which I must emphasize is **not** a requirement), then those datasets must be public (with licensing that minimally includes academic/research use) with dataset provenance clearly shared on the external data thread. \n\nIn the interest of mitigating any issues around uncertainty in external data use, we are urging hosts (this competition included) to monitor closely the external data thread, in the event questions or concerns are raised around the use of certain datasets. If you check that thread, you'll see many of these detailed questions have already been addressed, so I encourage that to be the method for effective escalation.", "votes": null } ], "comments": [ { "id": 886824, "author_name": "sarques", "author_url": "", "post_date": "06/15/2020 09:51:41", "content": "Of course no one want to repeat what happened there, there should be steps taken to prevent such misadventures.
\n\nI believe your suggestion may help in a long way. Nice initiative! :)
", "votes": null, "replies": [ { "id": 886923, "author_name": "valanm", "author_url": "", "post_date": "06/15/2020 11:13:36", "content": "thanks
", "votes": null, "replies": [] } ] }, { "id": 886873, "author_name": "group16", "author_url": "", "post_date": "06/15/2020 10:30:19", "content": "Yes, especially now that apparently we are obliged to use external data in order to be eligible for prizes!
\n\nAlso, what amount of external data is sufficient to be eligible for prizes? If I use 1 external image of a dog or cat, does that count? We are (again!) in for a good time post-competition if these subjective details are not cleared up soon.
", "votes": null, "replies": [ { "id": 887053, "author_name": "valanm", "author_url": "", "post_date": "06/15/2020 13:05:51", "content": "@group16 good point.\nperhaps @juliaelliott can help us to clarify this?
", "votes": null, "replies": [] }, { "id": 887206, "author_name": "cdeotte", "author_url": "", "post_date": "06/15/2020 14:37:02", "content": "@group16 Can you explain the following comment
\n\n\n\n\nYes, especially now that apparently we are obliged to use external data in order to be eligible for prizes!
\n
Are you saying we are required to use external data? I don't understand.
", "votes": null, "replies": [] }, { "id": 887209, "author_name": "group16", "author_url": "", "post_date": "06/15/2020 14:41:04", "content": "According to the current rules, you are obliged to use external data in order to be eligible for prize money:
\n\nThe following provision supersedes General Rules Section 7.C. below: \"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will post any External Data used and the provenance of the data to the official competition forum prior to the Entry Deadline, and if the data is publicly available, include the link to such access. To receive a Prize, a potential winner must have used External Data available to use by all participants of the competition for purposes of the competition, for use that includes research or academic purposes, at no cost to the other participants (\"Public External Data\"). Participants who do not use Public External Data are not eligible to receive a Prize, but may retain their non-monetary winning features in accordance with Section 10 of the General Rules.\"
", "votes": null, "replies": [] }, { "id": 887350, "author_name": "veronicarotemberg", "author_url": "", "post_date": "06/15/2020 16:15:55", "content": "You are not obligated to use external data to win a prize, but you are obligated to use Public external data (and post on the official thread) if you do choose to use external data for prize eligibility.
", "votes": null, "replies": [] }, { "id": 887368, "author_name": "agentauers", "author_url": "", "post_date": "06/15/2020 16:23:44", "content": "Ok, and what's the difference between \"external data\" and \"Public external data\"? 😓 stupid question?
", "votes": null, "replies": [] }, { "id": 888312, "author_name": "danoozy44", "author_url": "", "post_date": "06/16/2020 09:12:05", "content": "\"Public External Data\" -> Everyone else can also use the data.\n\"External Data\" -> Only you can use the data.
\n\nmakes sense. Thank you for clarifying.
", "votes": null, "replies": [] }, { "id": 890292, "author_name": "danoozy44", "author_url": "", "post_date": "06/17/2020 12:37:46", "content": "No problem! :)
", "votes": null, "replies": [] } ] }, { "id": 887067, "author_name": "apacheco", "author_url": "", "post_date": "06/15/2020 13:14:27", "content": "Last year it was allowed to use any external data you want as long as you disclose it in the submission and paper.\nProbably it's the same rule, but your concern is totally fair!
", "votes": null, "replies": [] }, { "id": 891719, "author_name": "valanm", "author_url": "", "post_date": "06/18/2020 12:14:41", "content": "I till hope kaggle considers this...
", "votes": null, "replies": [ { "id": 891726, "author_name": "sarques", "author_url": "", "post_date": "06/18/2020 12:19:58", "content": "I'm afraid they may not be on this. :/
", "votes": null, "replies": [] }, { "id": 891876, "author_name": "valanm", "author_url": "", "post_date": "06/18/2020 13:59:24", "content": "Why do you think so. It would make the platform better, not?
", "votes": null, "replies": [] }, { "id": 893704, "author_name": "yuval6967", "author_url": "", "post_date": "06/19/2020 20:10:27", "content": "The rules considering external datasets where very strict in \"DFDC\" unlike any other competition. In the \"External Data thread\" of that competition there was a lot of discussion about which external datasets you can use, and Kaggle personal were vary active in answering and clarifying the rules. It was obvious from their answers that the only external data which is allowed is \"the usual\" pre-tained models.
\n\nThis competition is not DFDC, and the main source of external data - previous ISIC competition is legal, so we don't need to be too defensive. (and if you still worry about something, just ask about it directly, they'll answer)
", "votes": null, "replies": [] } ] }, { "id": 897479, "author_name": "juliaelliott", "author_url": "", "post_date": "06/22/2020 22:03:58", "content": "@valanm I appreciate your recommendation, but realistically we (Kaggle) are not in a position to be vetting every external dataset. The evaluation and enforcement of external dataset eligibility is ultimately in the hands of the host for each competition. The extent of scrutiny of external data or any competition-specific rules a host opts to add is very host-specific (some, frankly, don't care), so the proposed approach will not be scalable or practical for all hosts.
\n\nSo what does that mean for this competition? In this competition, the host has opened up the opportunity for any external data to be used (including private/non-public datasets), due to their interests in inclusivity of all approaches. However, for the purposes of prize-eligibility, if you opt to use external datasets (which I must emphasize is not a requirement), then those datasets must be public (with licensing that minimally includes academic/research use) with dataset provenance clearly shared on the external data thread.
\n\nIn the interest of mitigating any issues around uncertainty in external data use, we are urging hosts (this competition included) to monitor closely the external data thread, in the event questions or concerns are raised around the use of certain datasets. If you check that thread, you'll see many of these detailed questions have already been addressed, so I encourage that to be the method for effective escalation.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "886811": "Judging from my own experience i assume many of us do not go into details of every post. Just see what happened to [deep fake](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/158506) where No2 team got removed because they did not see that one external dataset they used was actually banned by Kaggle in the discussion thread.\n\nTo avoid such situations in the future and to make it easier to find relevant information I suggest KAGGLE includes the list of allowed external data in the **Official External Data Thread** description.\n\nWhat do you guys think?", "886824": "Of course no one want to repeat what happened there, there should be steps taken to prevent such misadventures. \n\nI believe your suggestion may help in a long way. Nice initiative! :)", "886873": "Yes, especially now that apparently we are obliged to use external data in order to be eligible for prizes! \n\nAlso, what amount of external data is sufficient to be eligible for prizes? If I use 1 external image of a dog or cat, does that count? We are (again!) in for a good time post-competition if these subjective details are not cleared up soon.", "886923": "thanks", "887053": "group16 good point.\nperhaps @juliaelliott can help us to clarify this?", "887067": "Last year it was allowed to use any external data you want as long as you disclose it in the submission and paper.\nProbably it's the same rule, but your concern is totally fair!", "887206": "group16 Can you explain the following comment\n> Yes, especially now that apparently we are obliged to use external data in order to be eligible for prizes!\n\nAre you saying we are required to use external data? I don't understand.", "887209": "According to the current rules, you are **obliged** to use external data in order to be eligible for prize money:\n\n1. EXTERNAL DATA\n\nThe following provision supersedes General Rules Section 7.C. below: \"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will post any External Data used and the provenance of the data to the official competition forum prior to the Entry Deadline, and if the data is publicly available, include the link to such access. To receive a Prize, a potential winner must have used External Data available to use by all participants of the competition for purposes of the competition, for use that includes research or academic purposes, at no cost to the other participants (\"Public External Data\"). **Participants who do not use Public External Data are not eligible to receive a Prize, but may retain their non-monetary winning features in accordance with Section 10 of the General Rules.\"**", "887350": "You are not obligated to use external data to win a prize, but you are obligated to use Public external data (and post on the official thread) if you do choose to use external data for prize eligibility.", "887368": "Ok, and what's the difference between \"external data\" and \"Public external data\"? 😓 stupid question?", "888312": "\"Public External Data\" -> Everyone else can also use the data.\n\"External Data\" -> Only you can use the data.\n\n- If you use public external data, then you're eligible for a prize.\n- If you used external data **without making it public**, then you cannot receive prizes.", "888330": "makes sense. Thank you for clarifying.", "890292": "No problem! :)", "891719": "I till hope kaggle considers this...", "891726": "I'm afraid they may not be on this. :/", "891876": "Why do you think so. It would make the platform better, not?", "893704": "The rules considering external datasets where very strict in \"DFDC\" unlike any other competition. In the \"External Data thread\" of that competition there was a lot of discussion about which external datasets you can use, and Kaggle personal were vary active in answering and clarifying the rules. It was obvious from their answers that the only external data which is allowed is \"the usual\" pre-tained models.\n\nThis competition is not DFDC, and the main source of external data - previous ISIC competition is legal, so we don't need to be too defensive. (and if you still worry about something, just ask about it directly, they'll answer)", "897479": "valanm I appreciate your recommendation, but realistically we (Kaggle) are not in a position to be vetting every external dataset. The evaluation and enforcement of external dataset eligibility is ultimately in the hands of the host for each competition. The extent of scrutiny of external data or any competition-specific rules a host opts to add is very host-specific (some, frankly, don't care), so the proposed approach will not be scalable or practical for all hosts.\n\nSo what does that mean for this competition? In this competition, the host has opened up the opportunity for any external data to be used (including private/non-public datasets), due to their interests in inclusivity of all approaches. However, for the purposes of prize-eligibility, if you opt to use external datasets (which I must emphasize is **not** a requirement), then those datasets must be public (with licensing that minimally includes academic/research use) with dataset provenance clearly shared on the external data thread. \n\nIn the interest of mitigating any issues around uncertainty in external data use, we are urging hosts (this competition included) to monitor closely the external data thread, in the event questions or concerns are raised around the use of certain datasets. If you check that thread, you'll see many of these detailed questions have already been addressed, so I encourage that to be the method for effective escalation." }, "source": "meta" }