{ "id": 4478, "title": "GaussianBinaryRBM Question", "url": "/competitions/challenges-in-representation-learning-the-black-box-learning-challenge/discussion/4478", "author_name": "", "post_date": "2013-05-01T19:29:53.483Z", "votes": null, "comment_count": 14, "views": 4592, "content": "

Hi,

\r\n

Which of the following methods in GaussianBinaryRBM class is used to get the output probability of the classes?

\r\n

a) score

\r\n

b) P_H_given_V

\r\n

c) mean_h_given_V

\r\n

d) mean_v_given_h

\r\n

e) free_energy_given_v

\r\n

f) free_energy

\r\n

I am trying to find the equivalent of fprop in mlp.py so that I can use similar code to make a submission

", "messages": [ { "id": "23766", "postDate": "05/01/2013 19:29:53", "content": "

Hi,

\r\n

Which of the following methods in GaussianBinaryRBM class is used to get the output probability of the classes?

\r\n

a) score

\r\n

b) P_H_given_V

\r\n

c) mean_h_given_V

\r\n

d) mean_v_given_h

\r\n

e) free_energy_given_v

\r\n

f) free_energy

\r\n

I am trying to find the equivalent of fprop in mlp.py so that I can use similar code to make a submission

", "rawMarkdown": "", "votes": null }, { "id": "23767", "postDate": "05/01/2013 19:34:40", "content": "

The GaussianBinaryRBM does not model classes. It models input pixels using latent random variables that help explain the pixels.

", "rawMarkdown": "", "votes": null }, { "id": "23768", "postDate": "05/01/2013 19:42:07", "content": "

Thanks -

\r\n

I tried:

\r\n

X = model.get_input_space().make_batch_theano()
\r\nY = model.score (X)

\r\n

 

\r\n

and Y was indeed a 10000 * 9 matrix - indicating there were predictions -

\r\n

Is there a way I can tweak this piece to get multi-class predictions (Sorry to bother - if you can even point me to useful literature, it would help)

", "rawMarkdown": "", "votes": null }, { "id": "23769", "postDate": "05/01/2013 19:44:07", "content": "

Maybe you set the RBM to have 9 hidden units? Here is my lab's tutorial on RBMs: http://deeplearning.net/tutorial/rbm.html

", "rawMarkdown": "", "votes": null }, { "id": "23770", "postDate": "05/01/2013 19:52:08", "content": "

Thanks a ton - Yes, I had number of hidden units = number of outputs.

\r\n

This just models the pixels as binaries correct - no way to model classes using these. I will give up this approach.

\r\n

 

\r\n

Thanks

", "rawMarkdown": "", "votes": null }, { "id": "23771", "postDate": "05/01/2013 19:56:14", "content": "

It models the pixels as gaussian, and tries to explain them with binary latent factors. You can use the binary latent factors as features for predicting the class labels. Here's an example paper: http://jmlr.csail.mit.edu/proceedings/papers/v15/coates11a.html

", "rawMarkdown": "", "votes": null }, { "id": "23772", "postDate": "05/01/2013 19:59:44", "content": "

Thanks - last question for the day.

\r\n

What method in GaussianBinaryRBM class returns the latent factors?

\r\n

I will try out the method in Andrew's paper

\r\n

Thanks a ton

\r\n

Kiran

", "rawMarkdown": "", "votes": null }, { "id": "23776", "postDate": "05/01/2013 21:23:46", "content": "

You can compute the conditional expectation of them using mean_H_given_V.

", "rawMarkdown": "", "votes": null }, { "id": "23813", "postDate": "05/02/2013 15:23:31", "content": "

Thanks a ton as usual Ian!

\r\n

I have got mean_h_given_v -> for 1000 hidden nodes. These are the same as latent factors. Correct?

\r\n


\r\nThanks
\r\n
\r\n

", "rawMarkdown": "", "votes": null }, { "id": "23817", "postDate": "05/02/2013 15:45:49", "content": "

Well, there are lots of things you can do with the latent variables. But taking their conditional expectation is what people usually do for feature extraction. So yeah, you're doing the right thing.

", "rawMarkdown": "", "votes": null }, { "id": "23822", "postDate": "05/02/2013 16:36:42", "content": "

It works.

\r\n

but in deep_trainer.py, changing n_hid in the 3 functions (get_denoising_autoencoder,  get_grbm,get_autoencoder ) to any value other than n_output results in an error

", "rawMarkdown": "", "votes": null }, { "id": "23823", "postDate": "05/02/2013 16:37:44", "content": "

What is n_output, the number of classes? What is the error?

", "rawMarkdown": "", "votes": null }, { "id": "23824", "postDate": "05/02/2013 16:39:36", "content": "

It shows the follwoing error when we make nhid to 1000 or any arbitrary value

\r\n

only value that works is the default nhid = n_output

\r\n

 

\r\n

Traceback (most recent call last):
\r\n  File "/home/kirana/pylearn2/pylearn2/scripts/tutorials/deep_trainer/run_deep_trainer.py", line 282, in <module>
\r\n    main()
\r\n  File "/home/kirana/pylearn2/pylearn2/scripts/tutorials/deep_trainer/run_deep_trainer.py", line 268, in main
\r\n    layer_trainer.main_loop()
\r\n  File "/usr/local/lib/python2.7/dist-packages/pylearn2-0.1dev-py2.7.egg/pylearn2/train.py", line 116, in main_loop
\r\n    self.run_callbacks_and_monitoring()
\r\n  File "/usr/local/lib/python2.7/dist-packages/pylearn2-0.1dev-py2.7.egg/pylearn2/train.py", line 137, in run_callbacks_and_monitoring
\r\n    self.model.monitor()
\r\n  File "/usr/local/lib/python2.7/dist-packages/pylearn2-0.1dev-py2.7.egg/pylearn2/monitor.py", line 195, in __call__
\r\n    a(X)
\r\n  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 586, in __call__
\r\n    gof.vm.raise_with_op(self.fn.nodes[self.fn.position_of_error])
\r\n  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 580, in __call__
\r\n    outputs = self.fn()
\r\nValueError: Shape mismatch: x has 1000 cols (and 10 rows) but y has 10 rows (and 1000 cols)
\r\n
\r\n
\r\n

", "rawMarkdown": "", "votes": null }, { "id": "23825", "postDate": "05/02/2013 16:41:22", "content": "

I've asked the author of the tutorial to look into it.

", "rawMarkdown": "", "votes": null }, { "id": "23828", "postDate": "05/02/2013 17:52:19", "content": "

When trying to stack models, you need to make sure the shape of inputs and outputs from subsequent layers are consistent. That means, the outputs from layer i should have the same shape of inputs for layer i+1. It looks like you have a problem over there.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 23767, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/01/2013 19:34:40", "content": "

The GaussianBinaryRBM does not model classes. It models input pixels using latent random variables that help explain the pixels.

", "votes": null, "replies": [] }, { "id": 23768, "author_name": "rkirana", "author_url": "", "post_date": "05/01/2013 19:42:07", "content": "

Thanks -

\r\n

I tried:

\r\n

X = model.get_input_space().make_batch_theano()
\r\nY = model.score (X)

\r\n

 

\r\n

and Y was indeed a 10000 * 9 matrix - indicating there were predictions -

\r\n

Is there a way I can tweak this piece to get multi-class predictions (Sorry to bother - if you can even point me to useful literature, it would help)

", "votes": null, "replies": [] }, { "id": 23769, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/01/2013 19:44:07", "content": "

Maybe you set the RBM to have 9 hidden units? Here is my lab's tutorial on RBMs: http://deeplearning.net/tutorial/rbm.html

", "votes": null, "replies": [] }, { "id": 23770, "author_name": "rkirana", "author_url": "", "post_date": "05/01/2013 19:52:08", "content": "

Thanks a ton - Yes, I had number of hidden units = number of outputs.

\r\n

This just models the pixels as binaries correct - no way to model classes using these. I will give up this approach.

\r\n

 

\r\n

Thanks

", "votes": null, "replies": [] }, { "id": 23771, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/01/2013 19:56:14", "content": "

It models the pixels as gaussian, and tries to explain them with binary latent factors. You can use the binary latent factors as features for predicting the class labels. Here's an example paper: http://jmlr.csail.mit.edu/proceedings/papers/v15/coates11a.html

", "votes": null, "replies": [] }, { "id": 23772, "author_name": "rkirana", "author_url": "", "post_date": "05/01/2013 19:59:44", "content": "

Thanks - last question for the day.

\r\n

What method in GaussianBinaryRBM class returns the latent factors?

\r\n

I will try out the method in Andrew's paper

\r\n

Thanks a ton

\r\n

Kiran

", "votes": null, "replies": [] }, { "id": 23776, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/01/2013 21:23:46", "content": "

You can compute the conditional expectation of them using mean_H_given_V.

", "votes": null, "replies": [] }, { "id": 23813, "author_name": "rkirana", "author_url": "", "post_date": "05/02/2013 15:23:31", "content": "

Thanks a ton as usual Ian!

\r\n

I have got mean_h_given_v -> for 1000 hidden nodes. These are the same as latent factors. Correct?

\r\n


\r\nThanks
\r\n
\r\n

", "votes": null, "replies": [] }, { "id": 23817, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/02/2013 15:45:49", "content": "

Well, there are lots of things you can do with the latent variables. But taking their conditional expectation is what people usually do for feature extraction. So yeah, you're doing the right thing.

", "votes": null, "replies": [] }, { "id": 23822, "author_name": "rkirana", "author_url": "", "post_date": "05/02/2013 16:36:42", "content": "

It works.

\r\n

but in deep_trainer.py, changing n_hid in the 3 functions (get_denoising_autoencoder,  get_grbm,get_autoencoder ) to any value other than n_output results in an error

", "votes": null, "replies": [] }, { "id": 23823, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/02/2013 16:37:44", "content": "

What is n_output, the number of classes? What is the error?

", "votes": null, "replies": [] }, { "id": 23824, "author_name": "rkirana", "author_url": "", "post_date": "05/02/2013 16:39:36", "content": "

It shows the follwoing error when we make nhid to 1000 or any arbitrary value

\r\n

only value that works is the default nhid = n_output

\r\n

 

\r\n

Traceback (most recent call last):
\r\n  File "/home/kirana/pylearn2/pylearn2/scripts/tutorials/deep_trainer/run_deep_trainer.py", line 282, in <module>
\r\n    main()
\r\n  File "/home/kirana/pylearn2/pylearn2/scripts/tutorials/deep_trainer/run_deep_trainer.py", line 268, in main
\r\n    layer_trainer.main_loop()
\r\n  File "/usr/local/lib/python2.7/dist-packages/pylearn2-0.1dev-py2.7.egg/pylearn2/train.py", line 116, in main_loop
\r\n    self.run_callbacks_and_monitoring()
\r\n  File "/usr/local/lib/python2.7/dist-packages/pylearn2-0.1dev-py2.7.egg/pylearn2/train.py", line 137, in run_callbacks_and_monitoring
\r\n    self.model.monitor()
\r\n  File "/usr/local/lib/python2.7/dist-packages/pylearn2-0.1dev-py2.7.egg/pylearn2/monitor.py", line 195, in __call__
\r\n    a(X)
\r\n  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 586, in __call__
\r\n    gof.vm.raise_with_op(self.fn.nodes[self.fn.position_of_error])
\r\n  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 580, in __call__
\r\n    outputs = self.fn()
\r\nValueError: Shape mismatch: x has 1000 cols (and 10 rows) but y has 10 rows (and 1000 cols)
\r\n
\r\n
\r\n

", "votes": null, "replies": [] }, { "id": 23825, "author_name": "iangoodfellow", "author_url": "", "post_date": "05/02/2013 16:41:22", "content": "

I've asked the author of the tutorial to look into it.

", "votes": null, "replies": [] }, { "id": 23828, "author_name": "macmadman", "author_url": "", "post_date": "05/02/2013 17:52:19", "content": "

When trying to stack models, you need to make sure the shape of inputs and outputs from subsequent layers are consistent. That means, the outputs from layer i should have the same shape of inputs for layer i+1. It looks like you have a problem over there.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "23766": "", "23767": "", "23768": "", "23769": "", "23770": "", "23771": "", "23772": "", "23776": "", "23813": "", "23817": "", "23822": "", "23823": "", "23824": "", "23825": "", "23828": "" }, "source": "meta" }