{
  "id": 4673,
  "title": "How much did the extra data help you?",
  "url": "/competitions/challenges-in-representation-learning-the-black-box-learning-challenge/discussion/4673",
  "author_name": "",
  "post_date": "2013-05-25T00:28:57.077Z",
  "votes": null,
  "comment_count": 2,
  "views": 1809,
  "content": "<p>First of, I must say I really enjoyed this competition. It was a great way to put to practice all these papers about deep learning although I still need to learn a lot. Thanks for Pylearn too. Great piece of software. I had used Matlab in the past but Pylearn\r\n &#43; Theano really made my life easy.</p>\r\n<p>I was wondering how much unsupervised learning improved your results. I got in a little late but tried a few deep architectures (stacked auto-encoders, deep belief nets, conv nets ...) but my model barely improved with the extra data (I &nbsp;used lots of it).\r\n I was never able to pass the .56 bar. Could you please share a bit about how much the extra data helped you? Also if hyperparam selection does impact the usefulness of that data please do share how you came up with them (grid search or simply luck?)</p>\r\n<p>Thank you.</p>",
  "messages": [
    {
      "id": "24771",
      "postDate": "05/25/2013 00:28:57",
      "content": "<p>First of, I must say I really enjoyed this competition. It was a great way to put to practice all these papers about deep learning although I still need to learn a lot. Thanks for Pylearn too. Great piece of software. I had used Matlab in the past but Pylearn\r\n &#43; Theano really made my life easy.</p>\r\n<p>I was wondering how much unsupervised learning improved your results. I got in a little late but tried a few deep architectures (stacked auto-encoders, deep belief nets, conv nets ...) but my model barely improved with the extra data (I &nbsp;used lots of it).\r\n I was never able to pass the .56 bar. Could you please share a bit about how much the extra data helped you? Also if hyperparam selection does impact the usefulness of that data please do share how you came up with them (grid search or simply luck?)</p>\r\n<p>Thank you.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24772",
      "postDate": "05/25/2013 00:33:50",
      "content": "<p>I definitely had trouble getting the extra data to do anything useful when I was making the baseline submissions, but I didn't have much time to spend on that. I'm curious what the winners did.</p>\r\n<p>For selecting hyperparameters, random search is usually much better than grid search:&nbsp;http://jmlr.org/papers/v13/bergstra12a.html</p>\r\n<p>It's also possible to develop a fairly good understanding of the hyperparameters and use that knowledge to help you set them. For example, if you monitor the training cost and you see it go up from one epoch to the next, that's a very strong sign that your\r\n learning rate is too high.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24780",
      "postDate": "05/25/2013 01:40:14",
      "content": "<p>In the baselines that I ran, the extra data helped only a bit. But part of the point of this contest was to see what others can do with it!</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 24772,
      "author_name": "iangoodfellow",
      "author_url": "",
      "post_date": "05/25/2013 00:33:50",
      "content": "<p>I definitely had trouble getting the extra data to do anything useful when I was making the baseline submissions, but I didn't have much time to spend on that. I'm curious what the winners did.</p>\r\n<p>For selecting hyperparameters, random search is usually much better than grid search:&nbsp;http://jmlr.org/papers/v13/bergstra12a.html</p>\r\n<p>It's also possible to develop a fairly good understanding of the hyperparameters and use that knowledge to help you set them. For example, if you monitor the training cost and you see it go up from one epoch to the next, that's a very strong sign that your\r\n learning rate is too high.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 24780,
      "author_name": "dumitru0",
      "author_url": "",
      "post_date": "05/25/2013 01:40:14",
      "content": "<p>In the baselines that I ran, the extra data helped only a bit. But part of the point of this contest was to see what others can do with it!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "24771": "",
    "24772": "",
    "24780": ""
  },
  "source": "meta"
}