I just fall into some problems， I use the pretrained VGG_16 net and set the Convolutional layer's params non-trainable, Then followed the keras scrips to create the local train_val set(20 drivers for train and 6 for val), the loss(categorical_crossentropy) is down to almost 0.3 after one epoch and the val_loss almost unchange even raise. whatever I do to change the model's capacity, the same thing happened except let all the model's params trainable.
", "messages": [ { "id": "116000", "postDate": "04/21/2016 10:22:07", "content": "I just fall into some problems， I use the pretrained VGG_16 net and set the Convolutional layer's params non-trainable, Then followed the keras scrips to create the local train_val set(20 drivers for train and 6 for val), the loss(categorical_crossentropy) is down to almost 0.3 after one epoch and the val_loss almost unchange even raise. whatever I do to change the model's capacity, the same thing happened except let all the model's params trainable.
", "rawMarkdown": "I just fall into some problems， I use the pretrained VGG_16 net and set the Convolutional layer's params non-trainable, Then followed the keras scrips to create the local train_val set(20 drivers for train and 6 for val), the loss(categorical_crossentropy) is down to almost 0.3 after one epoch and the val_loss almost unchange even raise. whatever I do to change the model's capacity, the same thing happened except let all the model's params trainable.", "votes": null }, { "id": "116009", "postDate": "04/21/2016 12:46:12", "content": "Maybe your learning rate is too large.\nOr you do not modify the train_val.prototxt to make the final output to be 10.
", "rawMarkdown": "Maybe your learning rate is too large.\r\nOr you do not modify the train_val.prototxt to make the final output to be 10.", "votes": null }, { "id": "116020", "postDate": "04/21/2016 14:00:39", "content": "[quote=Xingzhong Du;116009]
\n\nMaybe your learning rate is too large.\nOr you do not modifu the train_val.prototxt to make the final output to be 10.
\n\n[/quote]\nI use Adam optimizer and the output layer with softmax is defenitly 10 units.
", "rawMarkdown": "[quote=Xingzhong Du;116009]\r\n\r\nMaybe your learning rate is too large.\r\nOr you do not modifu the train_val.prototxt to make the final output to be 10.\r\n\r\n[/quote]\r\nI use Adam optimizer and the output layer with softmax is defenitly 10 units.", "votes": null }, { "id": "116128", "postDate": "04/22/2016 06:43:44", "content": "There are many things could let your problem happen.\nI think you'd better provide more details.\nSuch as the framework you use, the model configure file and other things.
", "rawMarkdown": "There are many things could let your problem happen.\r\nI think you'd better provide more details.\r\nSuch as the framework you use, the model configure file and other things.", "votes": null }, { "id": "116183", "postDate": "04/22/2016 12:56:39", "content": "These are are my codes, notices I am not split the train_val across the driver any more, but still seems overfitting. the json file is the the recording of the performence
", "rawMarkdown": "These are are my codes, notices I am not split the train_val across the driver any more, but still seems overfitting. the json file is the the recording of the performence", "votes": null }, { "id": "116202", "postDate": "04/22/2016 14:00:10", "content": "Currently, I think you learning rate is too large.\nMaybe 0.01 is better.
", "rawMarkdown": "Currently, I think you learning rate is too large.\r\nMaybe 0.01 is better.", "votes": null }, { "id": "116547", "postDate": "04/25/2016 04:39:40", "content": "I have the same problem.\nI think my neurons are fitting per user and not as general features.\nI reduced my neuron numbers and increased the drop rate in order to minimize it.
", "rawMarkdown": "I have the same problem.\r\nI think my neurons are fitting per user and not as general features.\r\nI reduced my neuron numbers and increased the drop rate in order to minimize it.", "votes": null } ], "comments": [ { "id": 116009, "author_name": "redflack", "author_url": "", "post_date": "04/21/2016 12:46:12", "content": "Maybe your learning rate is too large.\nOr you do not modify the train_val.prototxt to make the final output to be 10.
", "votes": null, "replies": [] }, { "id": 116020, "author_name": "meanku", "author_url": "", "post_date": "04/21/2016 14:00:39", "content": "[quote=Xingzhong Du;116009]
\n\nMaybe your learning rate is too large.\nOr you do not modifu the train_val.prototxt to make the final output to be 10.
\n\n[/quote]\nI use Adam optimizer and the output layer with softmax is defenitly 10 units.
", "votes": null, "replies": [] }, { "id": 116128, "author_name": "redflack", "author_url": "", "post_date": "04/22/2016 06:43:44", "content": "There are many things could let your problem happen.\nI think you'd better provide more details.\nSuch as the framework you use, the model configure file and other things.
", "votes": null, "replies": [] }, { "id": 116183, "author_name": "meanku", "author_url": "", "post_date": "04/22/2016 12:56:39", "content": "These are are my codes, notices I am not split the train_val across the driver any more, but still seems overfitting. the json file is the the recording of the performence
", "votes": null, "replies": [] }, { "id": 116202, "author_name": "redflack", "author_url": "", "post_date": "04/22/2016 14:00:10", "content": "Currently, I think you learning rate is too large.\nMaybe 0.01 is better.
", "votes": null, "replies": [] }, { "id": 116547, "author_name": "mrbeer", "author_url": "", "post_date": "04/25/2016 04:39:40", "content": "I have the same problem.\nI think my neurons are fitting per user and not as general features.\nI reduced my neuron numbers and increased the drop rate in order to minimize it.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "116000": "I just fall into some problems， I use the pretrained VGG_16 net and set the Convolutional layer's params non-trainable, Then followed the keras scrips to create the local train_val set(20 drivers for train and 6 for val), the loss(categorical_crossentropy) is down to almost 0.3 after one epoch and the val_loss almost unchange even raise. whatever I do to change the model's capacity, the same thing happened except let all the model's params trainable.", "116009": "Maybe your learning rate is too large.\r\nOr you do not modify the train_val.prototxt to make the final output to be 10.", "116020": "[quote=Xingzhong Du;116009]\r\n\r\nMaybe your learning rate is too large.\r\nOr you do not modifu the train_val.prototxt to make the final output to be 10.\r\n\r\n[/quote]\r\nI use Adam optimizer and the output layer with softmax is defenitly 10 units.", "116128": "There are many things could let your problem happen.\r\nI think you'd better provide more details.\r\nSuch as the framework you use, the model configure file and other things.", "116183": "These are are my codes, notices I am not split the train_val across the driver any more, but still seems overfitting. the json file is the the recording of the performence", "116202": "Currently, I think you learning rate is too large.\r\nMaybe 0.01 is better.", "116547": "I have the same problem.\r\nI think my neurons are fitting per user and not as general features.\r\nI reduced my neuron numbers and increased the drop rate in order to minimize it." }, "source": "meta" }