The challenge for this competition isn't the lack of information about the data but just the lack of training data. Not sure why it's called the "black box learning challenge" - would be more deserved to be called "Lack of training data" challenge.
\r\n\r\n
The winners would have proved that they are the best at working with minimal training data. Lack of data knowledge seems to be irrelevant (which defeats the whole object of the competition)
", "messages": [ { "id": "22834", "postDate": "04/15/2013 13:45:24", "content": "The challenge for this competition isn't the lack of information about the data but just the lack of training data. Not sure why it's called the "black box learning challenge" - would be more deserved to be called "Lack of training data" challenge.
\r\n\r\n
The winners would have proved that they are the best at working with minimal training data. Lack of data knowledge seems to be irrelevant (which defeats the whole object of the competition)
", "rawMarkdown": "", "votes": null }, { "id": "22835", "postDate": "04/15/2013 14:07:48", "content": "You have plenty of training data--over 100,000 examples. What you don't have a lot of is *labeled* training data.
\r\nIn a competition with few labels and a known data domain, a human beings would be able to design good features for a problem with few labels, based on their knowledge of the data domain. Here, you don't have knowledge of the data domain, so it's up to your\r\n algorithm to build that knowledge from the unlabeled data.
\r\nWith the right features for a task, it's entirely possible to generalize well to new data using only a few examples. Consider for example the popular Iris dataset. It's based on classifying species of flowers from measurements of parts of the plant. These\r\n measurements are a good feature space in which to classify the plant so even a simple model like softmax regression can learn to classify essentially perfectly, with fewer than 150 training cases.\r\n
", "rawMarkdown": "", "votes": null }, { "id": "22842", "postDate": "04/15/2013 15:06:27", "content": "I don't consider unlabelled data as training data. Most of the competitions on Kaggle don't require an expert so I'm not sure why this one is any different. (apart from the lack of training data) So the challenge is semi-supervised learning rather than black\r\n box
\r\nEDIT: Sorry - just was excited about not having to spend intellectual effort on this competition (ie just data mining) but now realise I have to spend more intellectual effort learning semi-supervised methods. No time for this effort :(\r\n
", "rawMarkdown": "", "votes": null }, { "id": "22848", "postDate": "04/15/2013 16:40:11", "content": "Yes, this contest is for a scientific conference, so some intellectual effort is do expected. Specifically, it's for a workshop on representation learning, so we do expect semi-supervised methods to perform well.
", "rawMarkdown": "", "votes": null }, { "id": "22850", "postDate": "04/15/2013 17:02:41", "content": "I'd just like to add that, though I'm not sure I'll have time to compete in this competition, I'm very glad to see something covering un/semi-supervised learning on Kaggle as I find that problems with a high unlabeld/labeld data ratio are extremely common\r\n in the real world. This is especially true for problems that require human expertise for labeling as human experts tend to take a long time to label data and are also very costly. Not only is this an interesting research question, it's also extremely applicable\r\n to a wide range of intesting data challenges out there.
", "rawMarkdown": "", "votes": null }, { "id": "23753", "postDate": "05/01/2013 15:45:15", "content": "Class count
\r\n| 9 | \r\n75 | \r\n
| 8 | \r\n73 | \r\n
| 7 | \r\n81 | \r\n
| 6 | \r\n82 | \r\n
| 5 | \r\n80 | \r\n
| 4 | \r\n111 | \r\n
| 3 | \r\n130 | \r\n
| 2 | \r\n170 | \r\n
| 1 | \r\n197 | \r\n
Wish the count of objects in each class were consistent.
", "rawMarkdown": "", "votes": null }, { "id": "23754", "postDate": "05/01/2013 15:48:33", "content": "The real world isn't always sorted into nicely balanced classes.
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 22835, "author_name": "iangoodfellow", "author_url": "", "post_date": "04/15/2013 14:07:48", "content": "You have plenty of training data--over 100,000 examples. What you don't have a lot of is *labeled* training data.
\r\nIn a competition with few labels and a known data domain, a human beings would be able to design good features for a problem with few labels, based on their knowledge of the data domain. Here, you don't have knowledge of the data domain, so it's up to your\r\n algorithm to build that knowledge from the unlabeled data.
\r\nWith the right features for a task, it's entirely possible to generalize well to new data using only a few examples. Consider for example the popular Iris dataset. It's based on classifying species of flowers from measurements of parts of the plant. These\r\n measurements are a good feature space in which to classify the plant so even a simple model like softmax regression can learn to classify essentially perfectly, with fewer than 150 training cases.\r\n
", "votes": null, "replies": [] }, { "id": 22842, "author_name": "domcastro", "author_url": "", "post_date": "04/15/2013 15:06:27", "content": "I don't consider unlabelled data as training data. Most of the competitions on Kaggle don't require an expert so I'm not sure why this one is any different. (apart from the lack of training data) So the challenge is semi-supervised learning rather than black\r\n box
\r\nEDIT: Sorry - just was excited about not having to spend intellectual effort on this competition (ie just data mining) but now realise I have to spend more intellectual effort learning semi-supervised methods. No time for this effort :(\r\n
", "votes": null, "replies": [] }, { "id": 22848, "author_name": "iangoodfellow", "author_url": "", "post_date": "04/15/2013 16:40:11", "content": "Yes, this contest is for a scientific conference, so some intellectual effort is do expected. Specifically, it's for a workshop on representation learning, so we do expect semi-supervised methods to perform well.
", "votes": null, "replies": [] }, { "id": 22850, "author_name": "willkurt", "author_url": "", "post_date": "04/15/2013 17:02:41", "content": "I'd just like to add that, though I'm not sure I'll have time to compete in this competition, I'm very glad to see something covering un/semi-supervised learning on Kaggle as I find that problems with a high unlabeld/labeld data ratio are extremely common\r\n in the real world. This is especially true for problems that require human expertise for labeling as human experts tend to take a long time to label data and are also very costly. Not only is this an interesting research question, it's also extremely applicable\r\n to a wide range of intesting data challenges out there.
", "votes": null, "replies": [] }, { "id": 23753, "author_name": "kosmos", "author_url": "", "post_date": "05/01/2013 15:45:15", "content": "Class count
\r\n| 9 | \r\n75 | \r\n
| 8 | \r\n73 | \r\n
| 7 | \r\n81 | \r\n
| 6 | \r\n82 | \r\n
| 5 | \r\n80 | \r\n
| 4 | \r\n111 | \r\n
| 3 | \r\n130 | \r\n
| 2 | \r\n170 | \r\n
| 1 | \r\n197 | \r\n
Wish the count of objects in each class were consistent.
", "votes": null, "replies": [] }, { "id": 23754, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/01/2013 15:48:33", "content": "The real world isn't always sorted into nicely balanced classes.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "22834": "", "22835": "", "22842": "", "22848": "", "22850": "", "23753": "", "23754": "" }, "source": "meta" }