{
  "id": 16901,
  "title": "CNN problem: stop after some epochs",
  "url": "/competitions/challenges-in-representation-learning-the-black-box-learning-challenge/discussion/16901",
  "author_name": "pierre",
  "post_date": "2015-10-08T09:09:42.417000",
  "votes": 0,
  "comment_count": 0,
  "views": 910,
  "content": "<p>I have just run the CNN code of the tutorial without yaml file, it stopped after 11 epochs. I have checked it is not because of the theano flag float32. My machine is 64bit. Does anybody know whats wrong with the code?\nThese are the warnings I got:</p>\n\n<p>/pylearn2/model_extensions/norm_constraint.py:96: UserWarning: MaxL2FilterNorm is deprecated and may be removed on or after 2016-01-31. Use ConstrainFilterL2Norm.\n  warnings.warn(&quot;MaxL2FilterNorm is deprecated and may be removed on or&quot;</p>\n\n<p>/pylearn2/train.py:85: UserWarning: dataset has no yaml src, model won't know what data it was trained on\n  &quot;data it was trained on&quot;)</p>\n\n<p>/pylearn2/costs/mlp/<strong>init</strong>.py:105: UserWarning: Coefficients should be given as a dictionary with layer names as key. The support of coefficients as list would be deprecated from 03/06/2015\n  warnings.warn(&quot;Coefficients should be given as a dictionary &quot;</p>\n\n<p>/pylearn2/monitor.py:572: UserWarning: Trained model saved without indicating yaml_src\n  'indicating yaml_src')</p>\n\n<p>and this is my code : </p>\n\n<pre><code>import numpy as np\nfrom pylearn2.utils import serial\nfrom pylearn2.models.mlp import MLP\nfrom pylearn2.datasets.dense_design_matrix import DenseDesignMatrix\nfrom pylearn2.format.target_format import convert_to_one_hot\nfrom pylearn2.space import Conv2DSpace\nfrom pylearn2.models.mlp import ConvRectifiedLinear\nfrom pylearn2.models.mlp import Softmax\nfrom pylearn2.costs.cost import SumOfCosts\nfrom pylearn2.costs.cost import MethodCost\nfrom pylearn2.costs.mlp import WeightDecay\nfrom pylearn2.training_algorithms.sgd import SGD\nfrom pylearn2.training_algorithms.learning_rule import Momentum\nfrom pylearn2.termination_criteria import And\nfrom pylearn2.train import Train\nfrom pylearn2.train_extensions.best_params import MonitorBasedSaveBest\nfrom pylearn2.training_algorithms.learning_rule import MomentumAdjustor\nfrom pylearn2.termination_criteria import MonitorBased\nfrom pylearn2.termination_criteria import EpochCounter\nfilepath = '/veu4/usuaris18/pierre/data/MNIST/mnist.pkl'\ntrain_set,valid_set,test_set=serial.load(filepath)\ntrain_set_x , train_set_y = train_set\nvalid_set_x , valid_set_y = valid_set\ntest_set_x , test_set_y = test_set\ntrain_set_y = convert_to_one_hot(integer_vector=train_set_y,max_labels=10)\nvalid_set_y = convert_to_one_hot(integer_vector=valid_set_y,max_labels=10)\ntest_set_y = convert_to_one_hot(integer_vector=test_set_y,max_labels=10)\nprint np.shape(test_set_y)\nCNN_model = MLP(batch_size = 100,\n                input_space = Conv2DSpace(shape = [28,28],num_channels=1),\n                layers = [ConvRectifiedLinear(layer_name='h2',\n                     output_channels=64,\n                     irange= .05,\n                     kernel_shape=[5, 5],\n                     pool_shape=[4, 4],\n                     pool_stride=[2, 2],\n                     max_kernel_norm=1.9365),\n                        ConvRectifiedLinear(layer_name='h3',\n                     output_channels=64,\n                     irange= .05,\n                     kernel_shape=[5, 5],\n                     pool_shape=[4, 4],\n                     pool_stride=[2, 2],\n                     max_kernel_norm=1.9365),\n                        Softmax(max_col_norm=1.9365,\n                     layer_name='y',\n                     n_classes=10,\n                     istdev=.05)]\n)\ncost_function = SumOfCosts(costs = [MethodCost(method='cost_from_X'),WeightDecay(coeffs=[.00005,.00005,.00005])])\ntrainer = SGD(batch_size=100,\n        learning_rate=.01,\n        learning_rule=Momentum(init_momentum=0.5),\n        monitoring_dataset= {'valid':DenseDesignMatrix(X=valid_set_x,y=valid_set_y),'test':DenseDesignMatrix(X=test_set_x,y=test_set_y)},\n        cost=cost_function,\n        termination_criterion =And(criteria=[MonitorBased(channel_name=&quot;valid_y_misclass&quot;,prop_decrease=0.50,N=10),EpochCounter(max_epochs=500)])\n)\ntrained_model=Train(dataset=DenseDesignMatrix(X=train_set_x,y=train_set_y),model=CNN_model,\n                        algorithm=trainer,\n                        extensions=[MonitorBasedSaveBest(channel_name='valid_y_misclass',\n                                        save_path=&quot;./convolutional_network_best.pkl&quot;),\n                                MomentumAdjustor(start=1,\n                                                saturate=10,\n                                                final_momentum=.99)],\n                        save_path='/CNN/MNIST_CNN.pkl',\n                        save_freq=1)\ntrained_model.main_loop()\n</code></pre>",
  "messages": [
    {
      "id": 95474,
      "postDate": "2015-10-08T09:09:42.417Z",
      "content": "<p>I have just run the CNN code of the tutorial without yaml file, it stopped after 11 epochs. I have checked it is not because of the theano flag float32. My machine is 64bit. Does anybody know whats wrong with the code?\nThese are the warnings I got:</p>\n\n<p>/pylearn2/model_extensions/norm_constraint.py:96: UserWarning: MaxL2FilterNorm is deprecated and may be removed on or after 2016-01-31. Use ConstrainFilterL2Norm.\n  warnings.warn(&quot;MaxL2FilterNorm is deprecated and may be removed on or&quot;</p>\n\n<p>/pylearn2/train.py:85: UserWarning: dataset has no yaml src, model won't know what data it was trained on\n  &quot;data it was trained on&quot;)</p>\n\n<p>/pylearn2/costs/mlp/<strong>init</strong>.py:105: UserWarning: Coefficients should be given as a dictionary with layer names as key. The support of coefficients as list would be deprecated from 03/06/2015\n  warnings.warn(&quot;Coefficients should be given as a dictionary &quot;</p>\n\n<p>/pylearn2/monitor.py:572: UserWarning: Trained model saved without indicating yaml_src\n  'indicating yaml_src')</p>\n\n<p>and this is my code : </p>\n\n<pre><code>import numpy as np\nfrom pylearn2.utils import serial\nfrom pylearn2.models.mlp import MLP\nfrom pylearn2.datasets.dense_design_matrix import DenseDesignMatrix\nfrom pylearn2.format.target_format import convert_to_one_hot\nfrom pylearn2.space import Conv2DSpace\nfrom pylearn2.models.mlp import ConvRectifiedLinear\nfrom pylearn2.models.mlp import Softmax\nfrom pylearn2.costs.cost import SumOfCosts\nfrom pylearn2.costs.cost import MethodCost\nfrom pylearn2.costs.mlp import WeightDecay\nfrom pylearn2.training_algorithms.sgd import SGD\nfrom pylearn2.training_algorithms.learning_rule import Momentum\nfrom pylearn2.termination_criteria import And\nfrom pylearn2.train import Train\nfrom pylearn2.train_extensions.best_params import MonitorBasedSaveBest\nfrom pylearn2.training_algorithms.learning_rule import MomentumAdjustor\nfrom pylearn2.termination_criteria import MonitorBased\nfrom pylearn2.termination_criteria import EpochCounter\nfilepath = '/veu4/usuaris18/pierre/data/MNIST/mnist.pkl'\ntrain_set,valid_set,test_set=serial.load(filepath)\ntrain_set_x , train_set_y = train_set\nvalid_set_x , valid_set_y = valid_set\ntest_set_x , test_set_y = test_set\ntrain_set_y = convert_to_one_hot(integer_vector=train_set_y,max_labels=10)\nvalid_set_y = convert_to_one_hot(integer_vector=valid_set_y,max_labels=10)\ntest_set_y = convert_to_one_hot(integer_vector=test_set_y,max_labels=10)\nprint np.shape(test_set_y)\nCNN_model = MLP(batch_size = 100,\n                input_space = Conv2DSpace(shape = [28,28],num_channels=1),\n                layers = [ConvRectifiedLinear(layer_name='h2',\n                     output_channels=64,\n                     irange= .05,\n                     kernel_shape=[5, 5],\n                     pool_shape=[4, 4],\n                     pool_stride=[2, 2],\n                     max_kernel_norm=1.9365),\n                        ConvRectifiedLinear(layer_name='h3',\n                     output_channels=64,\n                     irange= .05,\n                     kernel_shape=[5, 5],\n                     pool_shape=[4, 4],\n                     pool_stride=[2, 2],\n                     max_kernel_norm=1.9365),\n                        Softmax(max_col_norm=1.9365,\n                     layer_name='y',\n                     n_classes=10,\n                     istdev=.05)]\n)\ncost_function = SumOfCosts(costs = [MethodCost(method='cost_from_X'),WeightDecay(coeffs=[.00005,.00005,.00005])])\ntrainer = SGD(batch_size=100,\n        learning_rate=.01,\n        learning_rule=Momentum(init_momentum=0.5),\n        monitoring_dataset= {'valid':DenseDesignMatrix(X=valid_set_x,y=valid_set_y),'test':DenseDesignMatrix(X=test_set_x,y=test_set_y)},\n        cost=cost_function,\n        termination_criterion =And(criteria=[MonitorBased(channel_name=&quot;valid_y_misclass&quot;,prop_decrease=0.50,N=10),EpochCounter(max_epochs=500)])\n)\ntrained_model=Train(dataset=DenseDesignMatrix(X=train_set_x,y=train_set_y),model=CNN_model,\n                        algorithm=trainer,\n                        extensions=[MonitorBasedSaveBest(channel_name='valid_y_misclass',\n                                        save_path=&quot;./convolutional_network_best.pkl&quot;),\n                                MomentumAdjustor(start=1,\n                                                saturate=10,\n                                                final_momentum=.99)],\n                        save_path='/CNN/MNIST_CNN.pkl',\n                        save_freq=1)\ntrained_model.main_loop()\n</code></pre>",
      "rawMarkdown": "I have just run the CNN code of the tutorial without yaml file, it stopped after 11 epochs. I have checked it is not because of the theano flag float32. My machine is 64bit. Does anybody know whats wrong with the code?\r\nThese are the warnings I got:\r\n\r\n/pylearn2/model_extensions/norm_constraint.py:96: UserWarning: MaxL2FilterNorm is deprecated and may be removed on or after 2016-01-31. Use ConstrainFilterL2Norm.\r\n  warnings.warn(\"MaxL2FilterNorm is deprecated and may be removed on or\"\r\n\r\n/pylearn2/train.py:85: UserWarning: dataset has no yaml src, model won't know what data it was trained on\r\n  \"data it was trained on\")\r\n\r\n/pylearn2/costs/mlp/__init__.py:105: UserWarning: Coefficients should be given as a dictionary with layer names as key. The support of coefficients as list would be deprecated from 03/06/2015\r\n  warnings.warn(\"Coefficients should be given as a dictionary \"\r\n\r\n/pylearn2/monitor.py:572: UserWarning: Trained model saved without indicating yaml_src\r\n  'indicating yaml_src')\r\n\r\nand this is my code : \r\n\r\n    import numpy as np\r\n    from pylearn2.utils import serial\r\n    from pylearn2.models.mlp import MLP\r\n    from pylearn2.datasets.dense_design_matrix import DenseDesignMatrix\r\n    from pylearn2.format.target_format import convert_to_one_hot\r\n    from pylearn2.space import Conv2DSpace\r\n    from pylearn2.models.mlp import ConvRectifiedLinear\r\n    from pylearn2.models.mlp import Softmax\r\n    from pylearn2.costs.cost import SumOfCosts\r\n    from pylearn2.costs.cost import MethodCost\r\n    from pylearn2.costs.mlp import WeightDecay\r\n    from pylearn2.training_algorithms.sgd import SGD\r\n    from pylearn2.training_algorithms.learning_rule import Momentum\r\n    from pylearn2.termination_criteria import And\r\n    from pylearn2.train import Train\r\n    from pylearn2.train_extensions.best_params import MonitorBasedSaveBest\r\n    from pylearn2.training_algorithms.learning_rule import MomentumAdjustor\r\n    from pylearn2.termination_criteria import MonitorBased\r\n    from pylearn2.termination_criteria import EpochCounter\r\n    filepath = '/veu4/usuaris18/pierre/data/MNIST/mnist.pkl'\r\n    train_set,valid_set,test_set=serial.load(filepath)\r\n    train_set_x , train_set_y = train_set\r\n    valid_set_x , valid_set_y = valid_set\r\n    test_set_x , test_set_y = test_set\r\n    train_set_y = convert_to_one_hot(integer_vector=train_set_y,max_labels=10)\r\n    valid_set_y = convert_to_one_hot(integer_vector=valid_set_y,max_labels=10)\r\n    test_set_y = convert_to_one_hot(integer_vector=test_set_y,max_labels=10)\r\n    print np.shape(test_set_y)\r\n    CNN_model = MLP(batch_size = 100,\r\n                    input_space = Conv2DSpace(shape = [28,28],num_channels=1),\r\n                    layers = [ConvRectifiedLinear(layer_name='h2',\r\n                         output_channels=64,\r\n                         irange= .05,\r\n                         kernel_shape=[5, 5],\r\n                         pool_shape=[4, 4],\r\n                         pool_stride=[2, 2],\r\n                         max_kernel_norm=1.9365),\r\n                            ConvRectifiedLinear(layer_name='h3',\r\n                         output_channels=64,\r\n                         irange= .05,\r\n                         kernel_shape=[5, 5],\r\n                         pool_shape=[4, 4],\r\n                         pool_stride=[2, 2],\r\n                         max_kernel_norm=1.9365),\r\n                            Softmax(max_col_norm=1.9365,\r\n                         layer_name='y',\r\n                         n_classes=10,\r\n                         istdev=.05)]\r\n    )\r\n    cost_function = SumOfCosts(costs = [MethodCost(method='cost_from_X'),WeightDecay(coeffs=[.00005,.00005,.00005])])\r\n    trainer = SGD(batch_size=100,\r\n            learning_rate=.01,\r\n            learning_rule=Momentum(init_momentum=0.5),\r\n            monitoring_dataset= {'valid':DenseDesignMatrix(X=valid_set_x,y=valid_set_y),'test':DenseDesignMatrix(X=test_set_x,y=test_set_y)},\r\n            cost=cost_function,\r\n            termination_criterion =And(criteria=[MonitorBased(channel_name=\"valid_y_misclass\",prop_decrease=0.50,N=10),EpochCounter(max_epochs=500)])\r\n    )\r\n    trained_model=Train(dataset=DenseDesignMatrix(X=train_set_x,y=train_set_y),model=CNN_model,\r\n                            algorithm=trainer,\r\n                            extensions=[MonitorBasedSaveBest(channel_name='valid_y_misclass',\r\n                                            save_path=\"./convolutional_network_best.pkl\"),\r\n                                    MomentumAdjustor(start=1,\r\n                                                    saturate=10,\r\n                                                    final_momentum=.99)],\r\n                            save_path='/CNN/MNIST_CNN.pkl',\r\n                            save_freq=1)\r\n    trained_model.main_loop()\r\n\r\n\r\n\r\n"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "95474": "I have just run the CNN code of the tutorial without yaml file, it stopped after 11 epochs. I have checked it is not because of the theano flag float32. My machine is 64bit. Does anybody know whats wrong with the code?\r\nThese are the warnings I got:\r\n\r\n/pylearn2/model_extensions/norm_constraint.py:96: UserWarning: MaxL2FilterNorm is deprecated and may be removed on or after 2016-01-31. Use ConstrainFilterL2Norm.\r\n  warnings.warn(\"MaxL2FilterNorm is deprecated and may be removed on or\"\r\n\r\n/pylearn2/train.py:85: UserWarning: dataset has no yaml src, model won't know what data it was trained on\r\n  \"data it was trained on\")\r\n\r\n/pylearn2/costs/mlp/__init__.py:105: UserWarning: Coefficients should be given as a dictionary with layer names as key. The support of coefficients as list would be deprecated from 03/06/2015\r\n  warnings.warn(\"Coefficients should be given as a dictionary \"\r\n\r\n/pylearn2/monitor.py:572: UserWarning: Trained model saved without indicating yaml_src\r\n  'indicating yaml_src')\r\n\r\nand this is my code : \r\n\r\n    import numpy as np\r\n    from pylearn2.utils import serial\r\n    from pylearn2.models.mlp import MLP\r\n    from pylearn2.datasets.dense_design_matrix import DenseDesignMatrix\r\n    from pylearn2.format.target_format import convert_to_one_hot\r\n    from pylearn2.space import Conv2DSpace\r\n    from pylearn2.models.mlp import ConvRectifiedLinear\r\n    from pylearn2.models.mlp import Softmax\r\n    from pylearn2.costs.cost import SumOfCosts\r\n    from pylearn2.costs.cost import MethodCost\r\n    from pylearn2.costs.mlp import WeightDecay\r\n    from pylearn2.training_algorithms.sgd import SGD\r\n    from pylearn2.training_algorithms.learning_rule import Momentum\r\n    from pylearn2.termination_criteria import And\r\n    from pylearn2.train import Train\r\n    from pylearn2.train_extensions.best_params import MonitorBasedSaveBest\r\n    from pylearn2.training_algorithms.learning_rule import MomentumAdjustor\r\n    from pylearn2.termination_criteria import MonitorBased\r\n    from pylearn2.termination_criteria import EpochCounter\r\n    filepath = '/veu4/usuaris18/pierre/data/MNIST/mnist.pkl'\r\n    train_set,valid_set,test_set=serial.load(filepath)\r\n    train_set_x , train_set_y = train_set\r\n    valid_set_x , valid_set_y = valid_set\r\n    test_set_x , test_set_y = test_set\r\n    train_set_y = convert_to_one_hot(integer_vector=train_set_y,max_labels=10)\r\n    valid_set_y = convert_to_one_hot(integer_vector=valid_set_y,max_labels=10)\r\n    test_set_y = convert_to_one_hot(integer_vector=test_set_y,max_labels=10)\r\n    print np.shape(test_set_y)\r\n    CNN_model = MLP(batch_size = 100,\r\n                    input_space = Conv2DSpace(shape = [28,28],num_channels=1),\r\n                    layers = [ConvRectifiedLinear(layer_name='h2',\r\n                         output_channels=64,\r\n                         irange= .05,\r\n                         kernel_shape=[5, 5],\r\n                         pool_shape=[4, 4],\r\n                         pool_stride=[2, 2],\r\n                         max_kernel_norm=1.9365),\r\n                            ConvRectifiedLinear(layer_name='h3',\r\n                         output_channels=64,\r\n                         irange= .05,\r\n                         kernel_shape=[5, 5],\r\n                         pool_shape=[4, 4],\r\n                         pool_stride=[2, 2],\r\n                         max_kernel_norm=1.9365),\r\n                            Softmax(max_col_norm=1.9365,\r\n                         layer_name='y',\r\n                         n_classes=10,\r\n                         istdev=.05)]\r\n    )\r\n    cost_function = SumOfCosts(costs = [MethodCost(method='cost_from_X'),WeightDecay(coeffs=[.00005,.00005,.00005])])\r\n    trainer = SGD(batch_size=100,\r\n            learning_rate=.01,\r\n            learning_rule=Momentum(init_momentum=0.5),\r\n            monitoring_dataset= {'valid':DenseDesignMatrix(X=valid_set_x,y=valid_set_y),'test':DenseDesignMatrix(X=test_set_x,y=test_set_y)},\r\n            cost=cost_function,\r\n            termination_criterion =And(criteria=[MonitorBased(channel_name=\"valid_y_misclass\",prop_decrease=0.50,N=10),EpochCounter(max_epochs=500)])\r\n    )\r\n    trained_model=Train(dataset=DenseDesignMatrix(X=train_set_x,y=train_set_y),model=CNN_model,\r\n                            algorithm=trainer,\r\n                            extensions=[MonitorBasedSaveBest(channel_name='valid_y_misclass',\r\n                                            save_path=\"./convolutional_network_best.pkl\"),\r\n                                    MomentumAdjustor(start=1,\r\n                                                    saturate=10,\r\n                                                    final_momentum=.99)],\r\n                            save_path='/CNN/MNIST_CNN.pkl',\r\n                            save_freq=1)\r\n    trained_model.main_loop()\r\n\r\n\r\n\r\n"
  }
}