{ "id": 312841, "title": "Change of dataset - removal of restrictions", "url": "/competitions/ultra-mnist/discussion/312841", "author_name": "Deepak Gupta", "post_date": "2022-03-14T10:17:02.306000", "votes": 18, "comment_count": 23, "views": 0, "content": "

[UPDATE: DATASET IS NOW CHANGED, AND THE COMPETITIVE IS LIVE.]

\n

Dear All,

\n

There has been a lot of confusion about the rules on what is allowed to use and what not. We understand your concern that such regulations are affecting the Kaggle spirit of the participants. However, when we chose the simplicity of UltraMNIST, these constraints were an integral part of it. Nevertheless, it seems we cannot control the submissions on the leaderboard, and this makes the leaderboard itself a false representative of the competition progress. There is no way we can control if the submissions comply with our regulations. This has upset some of the experienced Kagglers which was never our intention.

\n

In this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself. We hope the community approves our change of data at an intermediate stage, and to account for the time loss, we will extend the competition deadline by 4 days.

\n

For the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method. This constraint comes from our intentions to take forward the learning from Kaggle to a broader set of problems from the real-world.

\n

Finally, we request all Kagglers to hold their submissions till tomorrow. We will update the dataset, remove the confusing posts from the discussion forum (posted by us on rules) reset the leaderboard and start again - this time with high spirits and positive energy!

\n

Best regards
\nCompetition Hosts

", "messages": [ { "id": 1722234, "postDate": "2022-03-14T10:17:02.307Z", "content": "

[UPDATE: DATASET IS NOW CHANGED, AND THE COMPETITIVE IS LIVE.]

\n

Dear All,

\n

There has been a lot of confusion about the rules on what is allowed to use and what not. We understand your concern that such regulations are affecting the Kaggle spirit of the participants. However, when we chose the simplicity of UltraMNIST, these constraints were an integral part of it. Nevertheless, it seems we cannot control the submissions on the leaderboard, and this makes the leaderboard itself a false representative of the competition progress. There is no way we can control if the submissions comply with our regulations. This has upset some of the experienced Kagglers which was never our intention.

\n

In this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself. We hope the community approves our change of data at an intermediate stage, and to account for the time loss, we will extend the competition deadline by 4 days.

\n

For the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method. This constraint comes from our intentions to take forward the learning from Kaggle to a broader set of problems from the real-world.

\n

Finally, we request all Kagglers to hold their submissions till tomorrow. We will update the dataset, remove the confusing posts from the discussion forum (posted by us on rules) reset the leaderboard and start again - this time with high spirits and positive energy!

\n

Best regards
\nCompetition Hosts

", "rawMarkdown": "[UPDATE: DATASET IS NOW CHANGED, AND THE COMPETITIVE IS LIVE.]\n\nDear All,\n\nThere has been a lot of confusion about the rules on what is allowed to use and what not. We understand your concern that such regulations are affecting the Kaggle spirit of the participants. However, when we chose the simplicity of UltraMNIST, these constraints were an integral part of it. Nevertheless, it seems we cannot control the submissions on the leaderboard, and this makes the leaderboard itself a false representative of the competition progress. There is no way we can control if the submissions comply with our regulations. This has upset some of the experienced Kagglers which was never our intention.\n\nIn this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself. We hope the community approves our change of data at an intermediate stage, and to account for the time loss, we will extend the competition deadline by 4 days.\n\nFor the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method. This constraint comes from our intentions to take forward the learning from Kaggle to a broader set of problems from the real-world.\n\nFinally, we request all Kagglers to hold their submissions till tomorrow. We will update the dataset, remove the confusing posts from the discussion forum (posted by us on rules) reset the leaderboard and start again - this time with high spirits and positive energy!\n\nBest regards\nCompetition Hosts", "votes": 18 }, { "id": 1722361, "postDate": "2022-03-14T13:17:26.267Z", "content": "

I liked the original constraints. By restricting the use of MNIST data and pretrained models, it becomes challenging for a model to a) learn what are the useful patterns (we know them as digits) and what is noise b) learn that there only 10 different actual patterns but they can occur in n- different ways (a handwritten digit have infinite patterns) c) To associate with each digit a value and d) easiest of all, learn to sum those values. (these are insanely difficult task, even for humans)
\nI understand that the rules and how these competitions work prohibits these kind of challenges, but then if that is the intent, why not use arbitrary shapes with some multi-facet randomness instead of digits. Nobody, including humans would know the meaning and everybody have to resort to find a novel way to handle 4K images along with a tough learning task, (which I suppose is the original motive of the organizers). Even if one can remove the background, finding the meaning of those shapes would remain a complicated task.

", "rawMarkdown": "I liked the original constraints. By restricting the use of MNIST data and pretrained models, it becomes challenging for a model to a) learn what are the useful patterns (we know them as digits) and what is noise b) learn that there only 10 different actual patterns but they can occur in n- different ways (a handwritten digit have infinite patterns) c) To associate with each digit a value and d) easiest of all, learn to sum those values. (these are insanely difficult task, even for humans)\nI understand that the rules and how these competitions work prohibits these kind of challenges, but then if that is the intent, why not use arbitrary shapes with some multi-facet randomness instead of digits. Nobody, including humans would know the meaning and everybody have to resort to find a novel way to handle 4K images along with a tough learning task, (which I suppose is the original motive of the organizers). Even if one can remove the background, finding the meaning of those shapes would remain a complicated task.\n", "votes": 7, "replies": [ { "id": 1722384, "postDate": "2022-03-14T13:43:33.740Z", "content": "

I am really happy to see that people liked the constraints and I will probably do similar contests on a different and more scientific platform.

\n

We are doing something along a similar direction now, adding a more complicated background, adding some random shapes into the data. However, removing the MNIST digits might require a whole new competition now. Moreover it will be hard to think of a way to divide tha randomly sampled shapes/patterns into a set of classes (possible but hard to do it in the time frame that we promised - tomorrow!). But we are trying our best and hope to incorporate parts of your suggestion to come up with a better dataset.

", "rawMarkdown": "I am really happy to see that people liked the constraints and I will probably do similar contests on a different and more scientific platform.\n\nWe are doing something along a similar direction now, adding a more complicated background, adding some random shapes into the data. However, removing the MNIST digits might require a whole new competition now. Moreover it will be hard to think of a way to divide tha randomly sampled shapes/patterns into a set of classes (possible but hard to do it in the time frame that we promised - tomorrow!). But we are trying our best and hope to incorporate parts of your suggestion to come up with a better dataset.", "votes": 8 }, { "id": 1722427, "postDate": "2022-03-14T14:27:34.227Z", "content": "

Yeah i think the constraints are pretty good as it encourages new techniques to learn generalizability. Now the competition has become some bounding box detector and passing it to some mnist classifier.

", "rawMarkdown": "Yeah i think the constraints are pretty good as it encourages new techniques to learn generalizability. Now the competition has become some bounding box detector and passing it to some mnist classifier.", "votes": 3 }, { "id": 1722639, "postDate": "2022-03-14T18:36:31.940Z", "content": "

I hope the background of the new dataset is complex enough to make even the techniques of bounding box detector to append some innovation beyond fine tuning. Fingers crossed :)

", "rawMarkdown": "I hope the background of the new dataset is complex enough to make even the techniques of bounding box detector to append some innovation beyond fine tuning. Fingers crossed :)", "votes": 1 } ] }, { "id": 1722486, "postDate": "2022-03-14T15:19:45.973Z", "content": "

I like original challenge. :panda_sad:

", "rawMarkdown": "I like original challenge. :panda_sad:", "votes": 6, "replies": [ { "id": 1722637, "postDate": "2022-03-14T18:34:42.877Z", "content": "

@haqishen I wish many more people had shown support and we could work with all the constraints :)
\nThe whole reason to try it on Kaggle was to see if we could find a bridge between scientific innovation and the skills of Kagglers. Nevertheless, feel free to participate in the innovation track for the modified version of the dataset that will come soon :)

", "rawMarkdown": "@haqishen I wish many more people had shown support and we could work with all the constraints :)\nThe whole reason to try it on Kaggle was to see if we could find a bridge between scientific innovation and the skills of Kagglers. Nevertheless, feel free to participate in the innovation track for the modified version of the dataset that will come soon :)", "votes": 2 }, { "id": 1722732, "postDate": "2022-03-14T20:14:47.747Z", "content": "

Me too @haqishen. I'll miss the original challenge. The MNIST usage restriction sounded plausible to me. But in fact, most ppl got confused and focussed on transforming the ultraMNIST into MNIST problem

", "rawMarkdown": "Me too @haqishen. I'll miss the original challenge. The MNIST usage restriction sounded plausible to me. But in fact, most ppl got confused and focussed on transforming the ultraMNIST into MNIST problem", "votes": 1 } ] }, { "id": 1730289, "postDate": "2022-03-21T05:06:21.810Z", "content": "

My understanding :-

\n
    \n
  1. External Datasets and Models are not allowed .(Except for MNIST-pre-trained models, MNIST data)
    2. \n
  2. Also there is some confusion regarding
    3. \n
\n
\n

background should not be removed by a preprocessing method

\n
\n

and at the same time there is written

\n
\n

thereby permitting the use of all possible tricks including removing the background.
\n So what is the removing the background and is it allowed or not.

\n
", "rawMarkdown": "My understanding :- \n1. External Datasets and Models are not allowed .(Except for MNIST-pre-trained models, MNIST data)\n2. Also there is some confusion regarding \n> background should not be removed by a preprocessing method \n\nand at the same time there is written \n> thereby permitting the use of all possible tricks including removing the background.\nSo what is the removing the background and is it allowed or not.", "votes": 1 }, { "id": 1730251, "postDate": "2022-03-21T04:09:00.277Z", "content": "
\n

In this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself.

\n

For the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method.

\n
\n

@dkgupta90 Can you clear what do you mean by background should not be removed by preprocessing method?

", "rawMarkdown": "> In this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself. \n\n> For the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method.\n\n@dkgupta90 Can you clear what do you mean by background should not be removed by preprocessing method?", "votes": 1 }, { "id": 1722261, "postDate": "2022-03-14T11:23:30.207Z", "content": "

I'm glad to hear it! Now, I think there will be much less confusion about the rules.

", "rawMarkdown": "I'm glad to hear it! Now, I think there will be much less confusion about the rules. ", "votes": 1 }, { "id": 1732404, "postDate": "2022-03-23T11:04:36.910Z", "content": "

Is the old dataset still available somewhere to download?

", "rawMarkdown": "Is the old dataset still available somewhere to download?" }, { "id": 1727332, "postDate": "2022-03-17T21:36:26.847Z", "content": "

all possible tricks allowed? except for hand labeling? i wonder why? if someone wants to label the train set and then use such a model on prediction, what's wrong with it?
\nAlso, I wonder how can you enforce this rule and keep the leaderboard clean?

\n
\n

Submissions may not use or incorporate information from hand labeling or human prediction of the training or test data records.

\n
", "rawMarkdown": "all possible tricks allowed? except for hand labeling? i wonder why? if someone wants to label the train set and then use such a model on prediction, what's wrong with it? \nAlso, I wonder how can you enforce this rule and keep the leaderboard clean?\n\n> Submissions may not use or incorporate information from hand labeling or human prediction of the training or test data records.", "replies": [ { "id": 1727362, "postDate": "2022-03-17T22:58:02.897Z", "content": "

We will verify the final submissions as the winners are supposed to open source their solutions within 3 days. Hand labelling cant be verified and that's because its not allowed. I dont want to get further involved in this conversation. Ill let other co-hosts answer if they want to. From my side, these are the rules and in my honest opinion, they are very very relaxed now. :)

", "rawMarkdown": "We will verify the final submissions as the winners are supposed to open source their solutions within 3 days. Hand labelling cant be verified and that's because its not allowed. I dont want to get further involved in this conversation. Ill let other co-hosts answer if they want to. From my side, these are the rules and in my honest opinion, they are very very relaxed now. :)", "votes": 3 }, { "id": 1727561, "postDate": "2022-03-18T04:04:11.450Z", "content": "

We can definitely not enforce these rules to keep the leaderboard clean, and we really wish there was a way. Nevertheless, we have put effort in planning this competition with a vision mind, and hand-labelling does not comply with it. Of course, if someone has to really go on top of the leaderboard, they can choose to label the entire test set manually and we cannot do anything about it.

\n

However, we will look at each solution on the leaderboard one by one from the top to make sure the winning 3 comply with the rules - so even if someone is on the top of the board with wrong strategies, such solutions will be removed at the end.

", "rawMarkdown": "We can definitely not enforce these rules to keep the leaderboard clean, and we really wish there was a way. Nevertheless, we have put effort in planning this competition with a vision mind, and hand-labelling does not comply with it. Of course, if someone has to really go on top of the leaderboard, they can choose to label the entire test set manually and we cannot do anything about it.\n\nHowever, we will look at each solution on the leaderboard one by one from the top to make sure the winning 3 comply with the rules - so even if someone is on the top of the board with wrong strategies, such solutions will be removed at the end.", "votes": 1 }, { "id": 1727636, "postDate": "2022-03-18T06:15:49.783Z", "content": "

sure i understand.
\nso one would need to take external mnist dataset, transform it so it looks similar to competition data, hand label it and train model with that. hand labeling of external data is allowed. right?

", "rawMarkdown": "sure i understand.\nso one would need to take external mnist dataset, transform it so it looks similar to competition data, hand label it and train model with that. hand labeling of external data is allowed. right? ", "votes": 1, "replies": [ { "id": 1727711, "postDate": "2022-03-18T07:38:31.140Z", "content": "
\n

sure i understand.
\n so one would need to take external mnist dataset, transform it so it looks similar to competition data, hand label it and train model with that. hand labeling of external data is allowed. right?

\n
\n

any sort of hand labeling is not allowed at all for train, test, and external data

", "rawMarkdown": "> sure i understand.\n> so one would need to take external mnist dataset, transform it so it looks similar to competition data, hand label it and train model with that. hand labeling of external data is allowed. right?\n\nany sort of hand labeling is not allowed at all for train, test, and external data", "votes": 1 }, { "id": 1727851, "postDate": "2022-03-18T11:41:27.317Z", "content": "
\n

any sort of hand labeling is not allowed at all for train, test, and external data

\n
\n

I am fine with that. just wonder how did you arrive at this conclusion? thanks!

", "rawMarkdown": "> any sort of hand labeling is not allowed at all for train, test, and external data\n\nI am fine with that. just wonder how did you arrive at this conclusion? thanks!" } ] }, { "id": 1727644, "postDate": "2022-03-18T06:26:06.690Z", "content": "

I am not asking to change the rules. i think they are fine. just trying to understand that's why the question.

", "rawMarkdown": "I am not asking to change the rules. i think they are fine. just trying to understand that's why the question." }, { "id": 1727710, "postDate": "2022-03-18T07:37:56.417Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1727853, "postDate": "2022-03-18T11:45:51.427Z", "content": "

\"Image\"

", "rawMarkdown": "![Image](https://i.imgur.com/e914glA.png)", "replies": [ { "id": 1727859, "postDate": "2022-03-18T11:56:31.763Z", "content": "
\n

\"Image\"

\n
\n

does it mention external data?

", "rawMarkdown": "> ![Image](https://i.imgur.com/e914glA.png)\n\ndoes it mention external data?" } ] }, { "id": 1730206, "postDate": "2022-03-21T02:48:44.360Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1733457, "postDate": "2022-03-24T09:46:16.707Z", "content": "

C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your Submissions. However, you will ensure the External Data is publicly available and equally accessible to use by all participants of the Competition for purposes of the competition at no cost to the other participants.

", "rawMarkdown": "C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your Submissions. However, you will ensure the External Data is publicly available and equally accessible to use by all participants of the Competition for purposes of the competition at no cost to the other participants. " } ] } ], "comments": [ { "id": 1722361, "author_name": "trancenoid", "author_url": "", "post_date": "2022-03-14T13:17:26.267000", "content": "

I liked the original constraints. By restricting the use of MNIST data and pretrained models, it becomes challenging for a model to a) learn what are the useful patterns (we know them as digits) and what is noise b) learn that there only 10 different actual patterns but they can occur in n- different ways (a handwritten digit have infinite patterns) c) To associate with each digit a value and d) easiest of all, learn to sum those values. (these are insanely difficult task, even for humans)
\nI understand that the rules and how these competitions work prohibits these kind of challenges, but then if that is the intent, why not use arbitrary shapes with some multi-facet randomness instead of digits. Nobody, including humans would know the meaning and everybody have to resort to find a novel way to handle 4K images along with a tough learning task, (which I suppose is the original motive of the organizers). Even if one can remove the background, finding the meaning of those shapes would remain a complicated task.

", "votes": 7, "replies": [ { "id": 1722384, "author_name": "Deepak Gupta", "author_url": "", "post_date": "2022-03-14T13:43:33.740000", "content": "

I am really happy to see that people liked the constraints and I will probably do similar contests on a different and more scientific platform.

\n

We are doing something along a similar direction now, adding a more complicated background, adding some random shapes into the data. However, removing the MNIST digits might require a whole new competition now. Moreover it will be hard to think of a way to divide tha randomly sampled shapes/patterns into a set of classes (possible but hard to do it in the time frame that we promised - tomorrow!). But we are trying our best and hope to incorporate parts of your suggestion to come up with a better dataset.

", "votes": 8, "replies": [] }, { "id": 1722427, "author_name": "garyong", "author_url": "", "post_date": "2022-03-14T14:27:34.227000", "content": "

Yeah i think the constraints are pretty good as it encourages new techniques to learn generalizability. Now the competition has become some bounding box detector and passing it to some mnist classifier.

", "votes": 3, "replies": [] }, { "id": 1722639, "author_name": "Deepak Gupta", "author_url": "", "post_date": "2022-03-14T18:36:31.940000", "content": "

I hope the background of the new dataset is complex enough to make even the techniques of bounding box detector to append some innovation beyond fine tuning. Fingers crossed :)

", "votes": 1, "replies": [] } ] }, { "id": 1722486, "author_name": "Qishen Ha", "author_url": "", "post_date": "2022-03-14T15:19:45.973000", "content": "

I like original challenge. :panda_sad:

", "votes": 6, "replies": [ { "id": 1722637, "author_name": "Deepak Gupta", "author_url": "", "post_date": "2022-03-14T18:34:42.877000", "content": "

@haqishen I wish many more people had shown support and we could work with all the constraints :)
\nThe whole reason to try it on Kaggle was to see if we could find a bridge between scientific innovation and the skills of Kagglers. Nevertheless, feel free to participate in the innovation track for the modified version of the dataset that will come soon :)

", "votes": 2, "replies": [] }, { "id": 1722732, "author_name": "Jonatas Grosman", "author_url": "", "post_date": "2022-03-14T20:14:47.747000", "content": "

Me too @haqishen. I'll miss the original challenge. The MNIST usage restriction sounded plausible to me. But in fact, most ppl got confused and focussed on transforming the ultraMNIST into MNIST problem

", "votes": 1, "replies": [] } ] }, { "id": 1730289, "author_name": "Devansh Chowdhury", "author_url": "", "post_date": "2022-03-21T05:06:21.810000", "content": "

My understanding :-

\n
    \n
  1. External Datasets and Models are not allowed .(Except for MNIST-pre-trained models, MNIST data)
    2. \n
  2. Also there is some confusion regarding
    3. \n
\n
\n

background should not be removed by a preprocessing method

\n
\n

and at the same time there is written

\n
\n

thereby permitting the use of all possible tricks including removing the background.
\n So what is the removing the background and is it allowed or not.

\n
", "votes": 1, "replies": [] }, { "id": 1730251, "author_name": "Devansh Chowdhury", "author_url": "", "post_date": "2022-03-21T04:09:00.277000", "content": "
\n

In this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself.

\n

For the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method.

\n
\n

@dkgupta90 Can you clear what do you mean by background should not be removed by preprocessing method?

", "votes": 1, "replies": [] }, { "id": 1722261, "author_name": "dmitry_lessy", "author_url": "", "post_date": "2022-03-14T11:23:30.207000", "content": "

I'm glad to hear it! Now, I think there will be much less confusion about the rules.

", "votes": 1, "replies": [] }, { "id": 1732404, "author_name": "Daniele Acquaviva", "author_url": "", "post_date": "2022-03-23T11:04:36.910000", "content": "

Is the old dataset still available somewhere to download?

", "votes": 0, "replies": [] }, { "id": 1727332, "author_name": "miwojc", "author_url": "", "post_date": "2022-03-17T21:36:26.847000", "content": "

all possible tricks allowed? except for hand labeling? i wonder why? if someone wants to label the train set and then use such a model on prediction, what's wrong with it?
\nAlso, I wonder how can you enforce this rule and keep the leaderboard clean?

\n
\n

Submissions may not use or incorporate information from hand labeling or human prediction of the training or test data records.

\n
", "votes": 0, "replies": [ { "id": 1727362, "author_name": "Abhishek Thakur", "author_url": "", "post_date": "2022-03-17T22:58:02.897000", "content": "

We will verify the final submissions as the winners are supposed to open source their solutions within 3 days. Hand labelling cant be verified and that's because its not allowed. I dont want to get further involved in this conversation. Ill let other co-hosts answer if they want to. From my side, these are the rules and in my honest opinion, they are very very relaxed now. :)

", "votes": 3, "replies": [] }, { "id": 1727561, "author_name": "Deepak Gupta", "author_url": "", "post_date": "2022-03-18T04:04:11.450000", "content": "

We can definitely not enforce these rules to keep the leaderboard clean, and we really wish there was a way. Nevertheless, we have put effort in planning this competition with a vision mind, and hand-labelling does not comply with it. Of course, if someone has to really go on top of the leaderboard, they can choose to label the entire test set manually and we cannot do anything about it.

\n

However, we will look at each solution on the leaderboard one by one from the top to make sure the winning 3 comply with the rules - so even if someone is on the top of the board with wrong strategies, such solutions will be removed at the end.

", "votes": 1, "replies": [] }, { "id": 1727636, "author_name": "miwojc", "author_url": "", "post_date": "2022-03-18T06:15:49.783000", "content": "

sure i understand.
\nso one would need to take external mnist dataset, transform it so it looks similar to competition data, hand label it and train model with that. hand labeling of external data is allowed. right?

", "votes": 1, "replies": [ { "id": 1727711, "author_name": "Amit Bera", "author_url": "", "post_date": "2022-03-18T07:38:31.140000", "content": "
\n

sure i understand.
\n so one would need to take external mnist dataset, transform it so it looks similar to competition data, hand label it and train model with that. hand labeling of external data is allowed. right?

\n
\n

any sort of hand labeling is not allowed at all for train, test, and external data

", "votes": 1, "replies": [] }, { "id": 1727851, "author_name": "miwojc", "author_url": "", "post_date": "2022-03-18T11:41:27.317000", "content": "
\n

any sort of hand labeling is not allowed at all for train, test, and external data

\n
\n

I am fine with that. just wonder how did you arrive at this conclusion? thanks!

", "votes": 0, "replies": [] } ] }, { "id": 1727644, "author_name": "miwojc", "author_url": "", "post_date": "2022-03-18T06:26:06.690000", "content": "

I am not asking to change the rules. i think they are fine. just trying to understand that's why the question.

", "votes": 0, "replies": [] }, { "id": 1727710, "author_name": "", "author_url": "", "post_date": "2022-03-18T07:37:56.417000", "content": "", "votes": 0, "replies": [] }, { "id": 1727853, "author_name": "Amit Bera", "author_url": "", "post_date": "2022-03-18T11:45:51.427000", "content": "

\"Image\"

", "votes": 0, "replies": [ { "id": 1727859, "author_name": "miwojc", "author_url": "", "post_date": "2022-03-18T11:56:31.763000", "content": "
\n

\"Image\"

\n
\n

does it mention external data?

", "votes": 0, "replies": [] } ] }, { "id": 1730206, "author_name": "", "author_url": "", "post_date": "2022-03-21T02:48:44.360000", "content": "", "votes": 0, "replies": [] }, { "id": 1733457, "author_name": "miwojc", "author_url": "", "post_date": "2022-03-24T09:46:16.707000", "content": "

C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your Submissions. However, you will ensure the External Data is publicly available and equally accessible to use by all participants of the Competition for purposes of the competition at no cost to the other participants.

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1722234": "[UPDATE: DATASET IS NOW CHANGED, AND THE COMPETITIVE IS LIVE.]\n\nDear All,\n\nThere has been a lot of confusion about the rules on what is allowed to use and what not. We understand your concern that such regulations are affecting the Kaggle spirit of the participants. However, when we chose the simplicity of UltraMNIST, these constraints were an integral part of it. Nevertheless, it seems we cannot control the submissions on the leaderboard, and this makes the leaderboard itself a false representative of the competition progress. There is no way we can control if the submissions comply with our regulations. This has upset some of the experienced Kagglers which was never our intention.\n\nIn this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself. We hope the community approves our change of data at an intermediate stage, and to account for the time loss, we will extend the competition deadline by 4 days.\n\nFor the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method. This constraint comes from our intentions to take forward the learning from Kaggle to a broader set of problems from the real-world.\n\nFinally, we request all Kagglers to hold their submissions till tomorrow. We will update the dataset, remove the confusing posts from the discussion forum (posted by us on rules) reset the leaderboard and start again - this time with high spirits and positive energy!\n\nBest regards\nCompetition Hosts", "1722361": "I liked the original constraints. By restricting the use of MNIST data and pretrained models, it becomes challenging for a model to a) learn what are the useful patterns (we know them as digits) and what is noise b) learn that there only 10 different actual patterns but they can occur in n- different ways (a handwritten digit have infinite patterns) c) To associate with each digit a value and d) easiest of all, learn to sum those values. (these are insanely difficult task, even for humans)\nI understand that the rules and how these competitions work prohibits these kind of challenges, but then if that is the intent, why not use arbitrary shapes with some multi-facet randomness instead of digits. Nobody, including humans would know the meaning and everybody have to resort to find a novel way to handle 4K images along with a tough learning task, (which I suppose is the original motive of the organizers). Even if one can remove the background, finding the meaning of those shapes would remain a complicated task.\n", "1722486": "I like original challenge. :panda_sad:", "1730289": "My understanding :- \n1. External Datasets and Models are not allowed .(Except for MNIST-pre-trained models, MNIST data)\n2. Also there is some confusion regarding \n> background should not be removed by a preprocessing method \n\nand at the same time there is written \n> thereby permitting the use of all possible tricks including removing the background.\nSo what is the removing the background and is it allowed or not.", "1730251": "> In this regard, we have decided to increase the complexity of the dataset and alleviate all the restrictions, thereby permitting the use of all possible tricks including removing the background, using MNIST-pretrained models as well as the MNIST data itself. \n\n> For the innovation track, we will keep one constraint - the background should not be removed by a preprocessing method.\n\n@dkgupta90 Can you clear what do you mean by background should not be removed by preprocessing method?", "1722261": "I'm glad to hear it! Now, I think there will be much less confusion about the rules. ", "1732404": "Is the old dataset still available somewhere to download?", "1727332": "all possible tricks allowed? except for hand labeling? i wonder why? if someone wants to label the train set and then use such a model on prediction, what's wrong with it? \nAlso, I wonder how can you enforce this rule and keep the leaderboard clean?\n\n> Submissions may not use or incorporate information from hand labeling or human prediction of the training or test data records." } }