{
  "id": 4629,
  "title": "yaml and unsupervised + supervised training",
  "url": "/competitions/challenges-in-representation-learning-the-black-box-learning-challenge/discussion/4629",
  "author_name": "",
  "post_date": "2013-05-19T20:16:51.630Z",
  "votes": null,
  "comment_count": 7,
  "views": 3363,
  "content": "<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>",
  "messages": [
    {
      "id": "24498",
      "postDate": "05/19/2013 20:16:51",
      "content": "<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24507",
      "postDate": "05/20/2013 02:50:14",
      "content": "<p></p>\r\n<p>Yes. You can.&nbsp;</p>\r\n<p>Use write a yaml to do pretraining, then save it in a pkl file.</p>\r\n<p>When do supervised learning, you can load the pkl file in a pretraining_layer.</p>\r\n<p>[quote=seylom;24498]</p>\r\n<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>\r\n<p>[/quote]</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24515",
      "postDate": "05/20/2013 10:31:04",
      "content": "<p>[quote=binghsu;24507]</p>\r\n<p></p>\r\n<p>Yes. You can.&nbsp;</p>\r\n<p>Use write a yaml to do pretraining, then save it in a pkl file.</p>\r\n<p>When do supervised learning, you can load the pkl file in a pretraining_layer.</p>\r\n<p>[quote=seylom;24498]</p>\r\n<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>\r\n<p>[/quote]</p>\r\n<p>[/quote]</p>\r\n<p></p>\r\n<p>Thanks for your reply. I spent hours reading and modifying YAML files in order to create a deep net with 2 stacked AutoEncoders &nbsp;and a sigmoid layer. I trained the first AutoEncoder using the unsupervised data, but then I am unable to figure out how to pass\r\n the results to the second AutoEncoder. I thought I needed to use the&nbsp;scripts.train.FeatureDump class but I get an error when running it &nbsp;on the GPU (It looks like my data needs to be casted to float32 for Theano).&nbsp;</p>\r\n<p>Where you &nbsp;(or anyone for that matter) able to successfully train first layers in an unsupervised way before fine tuning the resulting model?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24525",
      "postDate": "05/20/2013 17:01:22",
      "content": "<p></p>\r\n<p>You should read the tutorial of deep_trainer if you can't get everything right in YAML file.</p>\r\n<p></p>\r\n<p>[quote=seylom;24515]</p>\r\n<p>[quote=binghsu;24507]</p>\r\n<p></p>\r\n<p>Yes. You can.&nbsp;</p>\r\n<p>Use write a yaml to do pretraining, then save it in a pkl file.</p>\r\n<p>When do supervised learning, you can load the pkl file in a pretraining_layer.</p>\r\n<p>[quote=seylom;24498]</p>\r\n<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>\r\n<p>[/quote]</p>\r\n<p>[/quote]</p>\r\n<p></p>\r\n<p>Thanks for your reply. I spent hours reading and modifying YAML files in order to create a deep net with 2 stacked AutoEncoders &nbsp;and a sigmoid layer. I trained the first AutoEncoder using the unsupervised data, but then I am unable to figure out how to pass\r\n the results to the second AutoEncoder. I thought I needed to use the&nbsp;scripts.train.FeatureDump class but I get an error when running it &nbsp;on the GPU (It looks like my data needs to be casted to float32 for Theano).&nbsp;</p>\r\n<p>Where you &nbsp;(or anyone for that matter) able to successfully train first layers in an unsupervised way before fine tuning the resulting model?</p>\r\n<p>[/quote]</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24526",
      "postDate": "05/20/2013 17:03:22",
      "content": "<p>I will try that instead. Thanks!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24550",
      "postDate": "05/20/2013 20:51:34",
      "content": "<p>Seylom,&nbsp;<br>\r\nRegarding 32bit gpu issue. Did you install theano from github or just pip install?<br>\r\nI'm not sure for this problem, but i think that some sp/dp dtypes are fixed in github master &nbsp;<br>\r\n<br>\r\n</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24553",
      "postDate": "05/20/2013 20:59:40",
      "content": "<p>I used pip install. Great to hear it has probably been fixed. Hopefully it will clear a big hurdle for me.</p>\r\n<p>Thank you!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "24575",
      "postDate": "05/21/2013 04:48:06",
      "content": "<p>After looking a bit more into the issue it looks like it was in the loading of the 'extra' unsupervised dataset.</p>\r\n<p>I replaced line 59 of <strong>black_box_dataset.py</strong></p>\r\n<p>X = serial.load(path).T</p>\r\n<p>with</p>\r\n<p>X = serial.load(path).astype('float32').T</p>\r\n<p>This is just a temporary hack as I know I am loading an numpy array</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 24507,
      "author_name": "binghsu",
      "author_url": "",
      "post_date": "05/20/2013 02:50:14",
      "content": "<p></p>\r\n<p>Yes. You can.&nbsp;</p>\r\n<p>Use write a yaml to do pretraining, then save it in a pkl file.</p>\r\n<p>When do supervised learning, you can load the pkl file in a pretraining_layer.</p>\r\n<p>[quote=seylom;24498]</p>\r\n<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>\r\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 24515,
      "author_name": "seylom",
      "author_url": "",
      "post_date": "05/20/2013 10:31:04",
      "content": "<p>[quote=binghsu;24507]</p>\r\n<p></p>\r\n<p>Yes. You can.&nbsp;</p>\r\n<p>Use write a yaml to do pretraining, then save it in a pkl file.</p>\r\n<p>When do supervised learning, you can load the pkl file in a pretraining_layer.</p>\r\n<p>[quote=seylom;24498]</p>\r\n<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>\r\n<p>[/quote]</p>\r\n<p>[/quote]</p>\r\n<p></p>\r\n<p>Thanks for your reply. I spent hours reading and modifying YAML files in order to create a deep net with 2 stacked AutoEncoders &nbsp;and a sigmoid layer. I trained the first AutoEncoder using the unsupervised data, but then I am unable to figure out how to pass\r\n the results to the second AutoEncoder. I thought I needed to use the&nbsp;scripts.train.FeatureDump class but I get an error when running it &nbsp;on the GPU (It looks like my data needs to be casted to float32 for Theano).&nbsp;</p>\r\n<p>Where you &nbsp;(or anyone for that matter) able to successfully train first layers in an unsupervised way before fine tuning the resulting model?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 24525,
      "author_name": "binghsu",
      "author_url": "",
      "post_date": "05/20/2013 17:01:22",
      "content": "<p></p>\r\n<p>You should read the tutorial of deep_trainer if you can't get everything right in YAML file.</p>\r\n<p></p>\r\n<p>[quote=seylom;24515]</p>\r\n<p>[quote=binghsu;24507]</p>\r\n<p></p>\r\n<p>Yes. You can.&nbsp;</p>\r\n<p>Use write a yaml to do pretraining, then save it in a pkl file.</p>\r\n<p>When do supervised learning, you can load the pkl file in a pretraining_layer.</p>\r\n<p>[quote=seylom;24498]</p>\r\n<p>Is it possible to use the yaml file to perform unsupervised training on all but the last layer and then used derived weights to train the last layer with the supervised data? I couldn't find doc on how to use the yaml file to achieve it. I could get it done\r\n with a regular python script though. Any help is appreciated.</p>\r\n<p>Thanks</p>\r\n<p>[/quote]</p>\r\n<p>[/quote]</p>\r\n<p></p>\r\n<p>Thanks for your reply. I spent hours reading and modifying YAML files in order to create a deep net with 2 stacked AutoEncoders &nbsp;and a sigmoid layer. I trained the first AutoEncoder using the unsupervised data, but then I am unable to figure out how to pass\r\n the results to the second AutoEncoder. I thought I needed to use the&nbsp;scripts.train.FeatureDump class but I get an error when running it &nbsp;on the GPU (It looks like my data needs to be casted to float32 for Theano).&nbsp;</p>\r\n<p>Where you &nbsp;(or anyone for that matter) able to successfully train first layers in an unsupervised way before fine tuning the resulting model?</p>\r\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 24526,
      "author_name": "seylom",
      "author_url": "",
      "post_date": "05/20/2013 17:03:22",
      "content": "<p>I will try that instead. Thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 24550,
      "author_name": "djajetic",
      "author_url": "",
      "post_date": "05/20/2013 20:51:34",
      "content": "<p>Seylom,&nbsp;<br>\r\nRegarding 32bit gpu issue. Did you install theano from github or just pip install?<br>\r\nI'm not sure for this problem, but i think that some sp/dp dtypes are fixed in github master &nbsp;<br>\r\n<br>\r\n</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 24553,
      "author_name": "seylom",
      "author_url": "",
      "post_date": "05/20/2013 20:59:40",
      "content": "<p>I used pip install. Great to hear it has probably been fixed. Hopefully it will clear a big hurdle for me.</p>\r\n<p>Thank you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 24575,
      "author_name": "seylom",
      "author_url": "",
      "post_date": "05/21/2013 04:48:06",
      "content": "<p>After looking a bit more into the issue it looks like it was in the loading of the 'extra' unsupervised dataset.</p>\r\n<p>I replaced line 59 of <strong>black_box_dataset.py</strong></p>\r\n<p>X = serial.load(path).T</p>\r\n<p>with</p>\r\n<p>X = serial.load(path).astype('float32').T</p>\r\n<p>This is just a temporary hack as I know I am loading an numpy array</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "24498": "",
    "24507": "",
    "24515": "",
    "24525": "",
    "24526": "",
    "24550": "",
    "24553": "",
    "24575": ""
  },
  "source": "meta"
}