I know that there are another approaches, but I wonder should simple regression on tile level work? I have trained model only on tiles with objects and get not very good results.
\n\ndef get_vgg16_model(include_top=False, input_shape=(256,256,3)):\n vgg16= keras.applications.vgg16.VGG16(include_top=False, weights='imagenet', input_shape=(256,256,3))\n\n x= Conv2D(256, (3, 3), activation='relu')(vgg16.output)\n x= Conv2D(n_classes, (1, 1), activation='relu', name='output_conv')(x) # Maybe use activation='linear' ?\n x= GlobalAveragePooling2D()(x)\n\n model = Model(vgg16.input, x)\n\n #Freeze vgg16 layers\n for layer in model.layers[:19]: #Note: number of layers is model specific\n layer.trainable = False\n\n model.compile(loss=root_mean_squared_error, optimizer= keras.optimizers.Adadelta())\n\n print model.summary()\n\n return model\nAlso in attachment is output of 'output_conv' blob, map is just 6x6.
", "messages": [ { "id": "194906", "postDate": "06/22/2017 07:08:27", "content": "I know that there are another approaches, but I wonder should simple regression on tile level work? I have trained model only on tiles with objects and get not very good results.
\n\ndef get_vgg16_model(include_top=False, input_shape=(256,256,3)):\n vgg16= keras.applications.vgg16.VGG16(include_top=False, weights='imagenet', input_shape=(256,256,3))\n\n x= Conv2D(256, (3, 3), activation='relu')(vgg16.output)\n x= Conv2D(n_classes, (1, 1), activation='relu', name='output_conv')(x) # Maybe use activation='linear' ?\n x= GlobalAveragePooling2D()(x)\n\n model = Model(vgg16.input, x)\n\n #Freeze vgg16 layers\n for layer in model.layers[:19]: #Note: number of layers is model specific\n layer.trainable = False\n\n model.compile(loss=root_mean_squared_error, optimizer= keras.optimizers.Adadelta())\n\n print model.summary()\n\n return model\nAlso in attachment is output of 'output_conv' blob, map is just 6x6.
", "rawMarkdown": "I know that there are another approaches, but I wonder should simple regression on tile level work? I have trained model only on tiles with objects and get not very good results.\n\n def get_vgg16_model(include_top=False, input_shape=(256,256,3)):\n \tvgg16= keras.applications.vgg16.VGG16(include_top=False, weights='imagenet', input_shape=(256,256,3))\n \t\n \tx= Conv2D(256, (3, 3), activation='relu')(vgg16.output)\n \tx= Conv2D(n_classes, (1, 1), activation='relu', name='output_conv')(x) # Maybe use activation='linear' ?\n \tx= GlobalAveragePooling2D()(x)\n \t\n \tmodel = Model(vgg16.input, x)\n \t\n \t#Freeze vgg16 layers\n \tfor layer in model.layers[:19]: #Note: number of layers is model specific\n \t\tlayer.trainable = False\n \t\n \tmodel.compile(loss=root_mean_squared_error, optimizer= keras.optimizers.Adadelta())\n \n \tprint model.summary()\n \n \treturn model\n\nAlso in attachment is output of 'output_conv' blob, map is just 6x6.", "votes": null }, { "id": "194932", "postDate": "06/22/2017 08:57:12", "content": "I think this approach don't guarantee location. Just look at output value. When training is stable, try to fine tune the whole model.
", "rawMarkdown": "I think this approach don't guarantee location. Just look at output value. When training is stable, try to fine tune the whole model.", "votes": null }, { "id": "194994", "postDate": "06/22/2017 14:06:46", "content": "This is what I have now. The overall result seems like performing much worse on blue(juveniles) and green(pups) comparing to the other 3 classes. Not really sure why, maybe pups are just too small(but shouldn't 3x3 filters pick that up anyway)? I also thought maybe due to class imbalance(small number of males) but that should have shown better performance on pups which have larger number.
", "rawMarkdown": "This is what I have now. The overall result seems like performing much worse on blue(juveniles) and green(pups) comparing to the other 3 classes. Not really sure why, maybe pups are just too small(but shouldn't 3x3 filters pick that up anyway)? I also thought maybe due to class imbalance(small number of males) but that should have shown better performance on pups which have larger number.", "votes": null }, { "id": "195098", "postDate": "06/22/2017 18:35:57", "content": "Yes, this approach can be considered as weakly-supervised.
", "rawMarkdown": "Yes, this approach can be considered as weakly-supervised.", "votes": null }, { "id": "195207", "postDate": "06/22/2017 23:54:12", "content": "Also I have tried density map regression only on tiles that contain objects, but seems because of large portion of background or maybe because U-net is hard to train from scratch I can't get any good results.
", "rawMarkdown": "Also I have tried density map regression only on tiles that contain objects, but seems because of large portion of background or maybe because U-net is hard to train from scratch I can't get any good results.", "votes": null }, { "id": "195692", "postDate": "06/24/2017 15:11:19", "content": "Did you try freezing less layers? I think the far more upper conv blocks are trainied to \"hard\" for classification tasks. For example, I have frozen only [:15] thus keeping one conv block to fine-tune.
\n\nOverall:\ninput(1024x1024 tiles) - vgg[:15]_frozen - vgg[:15]_trainabled - conv2d(5, 3x3, relu) - gap, adam(lr=1e-5), given me an LB of 24.05.
", "rawMarkdown": "Did you try freezing less layers? I think the far more upper conv blocks are trainied to \"hard\" for classification tasks. For example, I have frozen only [:15] thus keeping one conv block to fine-tune. \n\nOverall:\ninput(1024x1024 tiles) - vgg[:15]_frozen - vgg[:15]_trainabled - conv2d(5, 3x3, relu) - gap, adam(lr=1e-5), given me an LB of 24.05.", "votes": null } ], "comments": [ { "id": 194932, "author_name": "outrunner", "author_url": "", "post_date": "06/22/2017 08:57:12", "content": "I think this approach don't guarantee location. Just look at output value. When training is stable, try to fine tune the whole model.
", "votes": null, "replies": [ { "id": 195098, "author_name": "mrgloom", "author_url": "", "post_date": "06/22/2017 18:35:57", "content": "Yes, this approach can be considered as weakly-supervised.
", "votes": null, "replies": [] } ] }, { "id": 194994, "author_name": "", "author_url": "", "post_date": "06/22/2017 14:06:46", "content": "This is what I have now. The overall result seems like performing much worse on blue(juveniles) and green(pups) comparing to the other 3 classes. Not really sure why, maybe pups are just too small(but shouldn't 3x3 filters pick that up anyway)? I also thought maybe due to class imbalance(small number of males) but that should have shown better performance on pups which have larger number.
", "votes": null, "replies": [] }, { "id": 195207, "author_name": "mrgloom", "author_url": "", "post_date": "06/22/2017 23:54:12", "content": "Also I have tried density map regression only on tiles that contain objects, but seems because of large portion of background or maybe because U-net is hard to train from scratch I can't get any good results.
", "votes": null, "replies": [] }, { "id": 195692, "author_name": "firolino", "author_url": "", "post_date": "06/24/2017 15:11:19", "content": "Did you try freezing less layers? I think the far more upper conv blocks are trainied to \"hard\" for classification tasks. For example, I have frozen only [:15] thus keeping one conv block to fine-tune.
\n\nOverall:\ninput(1024x1024 tiles) - vgg[:15]_frozen - vgg[:15]_trainabled - conv2d(5, 3x3, relu) - gap, adam(lr=1e-5), given me an LB of 24.05.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "194906": "I know that there are another approaches, but I wonder should simple regression on tile level work? I have trained model only on tiles with objects and get not very good results.\n\n def get_vgg16_model(include_top=False, input_shape=(256,256,3)):\n \tvgg16= keras.applications.vgg16.VGG16(include_top=False, weights='imagenet', input_shape=(256,256,3))\n \t\n \tx= Conv2D(256, (3, 3), activation='relu')(vgg16.output)\n \tx= Conv2D(n_classes, (1, 1), activation='relu', name='output_conv')(x) # Maybe use activation='linear' ?\n \tx= GlobalAveragePooling2D()(x)\n \t\n \tmodel = Model(vgg16.input, x)\n \t\n \t#Freeze vgg16 layers\n \tfor layer in model.layers[:19]: #Note: number of layers is model specific\n \t\tlayer.trainable = False\n \t\n \tmodel.compile(loss=root_mean_squared_error, optimizer= keras.optimizers.Adadelta())\n \n \tprint model.summary()\n \n \treturn model\n\nAlso in attachment is output of 'output_conv' blob, map is just 6x6.", "194932": "I think this approach don't guarantee location. Just look at output value. When training is stable, try to fine tune the whole model.", "194994": "This is what I have now. The overall result seems like performing much worse on blue(juveniles) and green(pups) comparing to the other 3 classes. Not really sure why, maybe pups are just too small(but shouldn't 3x3 filters pick that up anyway)? I also thought maybe due to class imbalance(small number of males) but that should have shown better performance on pups which have larger number.", "195098": "Yes, this approach can be considered as weakly-supervised.", "195207": "Also I have tried density map regression only on tiles that contain objects, but seems because of large portion of background or maybe because U-net is hard to train from scratch I can't get any good results.", "195692": "Did you try freezing less layers? I think the far more upper conv blocks are trainied to \"hard\" for classification tasks. For example, I have frozen only [:15] thus keeping one conv block to fine-tune. \n\nOverall:\ninput(1024x1024 tiles) - vgg[:15]_frozen - vgg[:15]_trainabled - conv2d(5, 3x3, relu) - gap, adam(lr=1e-5), given me an LB of 24.05." }, "source": "meta" }