My model always predict probability under 0.5 for all pixels.
\nI have tried focal loss,iou loss,weighted loss.
\nBut the result is same.
\nAfter few batches the masks i predicted gradually became all zeros.
\nHere is my notebook: enter link description here
In the notebook , basically what i did is :
\n（1）discard all samples where there is no ship
\n（2）build a plain u-net
\n（3）define three custom loss function(iouloss,focal_binarycrossentropy,biased_crossentropy), all of which i have tried.
\n（4）train and submit
#define different losses to try\ndef iouloss(y_true,y_pred):\n intersection = K.sum(y_true * y_pred, axis=-1)\n sum_ = K.sum(y_true + y_pred, axis=-1)\n jac = intersection / (sum_ - intersection)\n return 1 - jac\ndef focal_binarycrossentropy(y_true,y_pred):\n #focal loss with gamma 8\n t1=K.binary_crossentropy(y_true, y_pred)\n t2=tf.where(tf.equal(y_true,0),t1*(y_pred**8),t1*((1-y_pred)**8))\n return t2\ndef biased_crossentropy(y_true,y_pred):\n #apply 1000 times heavier punishment to ship pixels\n t1=K.binary_crossentropy(y_true, y_pred)\n t2=tf.where(tf.equal(y_true,0),t1*1000,t1)\n return t2\n...\n#try different loss function\nunet.compile(loss=iouloss, optimizer=\"adam\", metrics=[ioumetric])\nor\nunet.compile(loss=focal_binarycrossentropy, optimizer=\"adam\", metrics=[ioumetric])\nor\nunet.compile(loss=biased_crossentropy, optimizer=\"adam\", metrics=[ioumetric])\n...\n#start training\nunet.train_on_batch(x=image_batch,y=mask_batch)\nMy model always predict probability under 0.5 for all pixels.
\nI have tried focal loss,iou loss,weighted loss.
\nBut the result is same.
\nAfter few batches the masks i predicted gradually became all zeros.
\nHere is my notebook: enter link description here
In the notebook , basically what i did is :
\n（1）discard all samples where there is no ship
\n（2）build a plain u-net
\n（3）define three custom loss function(iouloss,focal_binarycrossentropy,biased_crossentropy), all of which i have tried.
\n（4）train and submit
#define different losses to try\ndef iouloss(y_true,y_pred):\n intersection = K.sum(y_true * y_pred, axis=-1)\n sum_ = K.sum(y_true + y_pred, axis=-1)\n jac = intersection / (sum_ - intersection)\n return 1 - jac\ndef focal_binarycrossentropy(y_true,y_pred):\n #focal loss with gamma 8\n t1=K.binary_crossentropy(y_true, y_pred)\n t2=tf.where(tf.equal(y_true,0),t1*(y_pred**8),t1*((1-y_pred)**8))\n return t2\ndef biased_crossentropy(y_true,y_pred):\n #apply 1000 times heavier punishment to ship pixels\n t1=K.binary_crossentropy(y_true, y_pred)\n t2=tf.where(tf.equal(y_true,0),t1*1000,t1)\n return t2\n...\n#try different loss function\nunet.compile(loss=iouloss, optimizer=\"adam\", metrics=[ioumetric])\nor\nunet.compile(loss=focal_binarycrossentropy, optimizer=\"adam\", metrics=[ioumetric])\nor\nunet.compile(loss=biased_crossentropy, optimizer=\"adam\", metrics=[ioumetric])\n...\n#start training\nunet.train_on_batch(x=image_batch,y=mask_batch)\nYou probably need to tweak your learning rate from the default of 0.01. Another thing to consider, I've done a lot of testing to train a model from scratch, I have yet to successfully train a full resolution from scratch on this dataset, even with some exotic loss functions. Normally I have to bootstrap training at a lower resolution first either by downscaling, or doing some smart crops.
", "rawMarkdown": "You probably need to tweak your learning rate from the default of 0.01. Another thing to consider, I've done a lot of testing to train a model from scratch, I have yet to successfully train a full resolution from scratch on this dataset, even with some exotic loss functions. Normally I have to bootstrap training at a lower resolution first either by downscaling, or doing some smart crops.", "votes": null }, { "id": "395681", "postDate": "09/29/2018 06:09:08", "content": "Can you check the mask that are created by your getImage. What will happen if you have several ships in one image? I may be wrong, but in this case self.masks.EncodedPixels.loc[id] will not be of type str. Let me know if it is the case.
", "rawMarkdown": "Can you check the mask that are created by your getImage. What will happen if you have several ships in one image? I may be wrong, but in this case self.masks.EncodedPixels.loc[id] will not be of type str. Let me know if it is the case.", "votes": null } ], "comments": [ { "id": 395632, "author_name": "oewyn000", "author_url": "", "post_date": "09/29/2018 02:45:06", "content": "You probably need to tweak your learning rate from the default of 0.01. Another thing to consider, I've done a lot of testing to train a model from scratch, I have yet to successfully train a full resolution from scratch on this dataset, even with some exotic loss functions. Normally I have to bootstrap training at a lower resolution first either by downscaling, or doing some smart crops.
", "votes": null, "replies": [] }, { "id": 395681, "author_name": "iafoss", "author_url": "", "post_date": "09/29/2018 06:09:08", "content": "Can you check the mask that are created by your getImage. What will happen if you have several ships in one image? I may be wrong, but in this case self.masks.EncodedPixels.loc[id] will not be of type str. Let me know if it is the case.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "395387": "My model always predict probability under 0.5 for all pixels. \nI have tried focal loss,iou loss,weighted loss. \nBut the result is same. \nAfter few batches the masks i predicted gradually became all zeros. \nHere is my notebook: [enter link description here][1]\n\nIn the notebook , basically what i did is : \n（1）discard all samples where there is no ship \n（2）build a plain u-net \n（3）define three custom loss function(iouloss,focal_binarycrossentropy,biased_crossentropy), all of which i have tried. \n（4）train and submit\n\n #define different losses to try\n def iouloss(y_true,y_pred):\n intersection = K.sum(y_true * y_pred, axis=-1)\n sum_ = K.sum(y_true + y_pred, axis=-1)\n jac = intersection / (sum_ - intersection)\n return 1 - jac\n def focal_binarycrossentropy(y_true,y_pred):\n #focal loss with gamma 8\n t1=K.binary_crossentropy(y_true, y_pred)\n t2=tf.where(tf.equal(y_true,0),t1*(y_pred**8),t1*((1-y_pred)**8))\n return t2\n def biased_crossentropy(y_true,y_pred):\n #apply 1000 times heavier punishment to ship pixels\n t1=K.binary_crossentropy(y_true, y_pred)\n t2=tf.where(tf.equal(y_true,0),t1*1000,t1)\n return t2\n ...\n #try different loss function\n unet.compile(loss=iouloss, optimizer=\"adam\", metrics=[ioumetric])\n or\n unet.compile(loss=focal_binarycrossentropy, optimizer=\"adam\", metrics=[ioumetric])\n or\n unet.compile(loss=biased_crossentropy, optimizer=\"adam\", metrics=[ioumetric])\n ...\n #start training\n unet.train_on_batch(x=image_batch,y=mask_batch)\n\n [1]: https://www.kaggle.com/abbracadabra/fork-of-fork-of-kernel95a936c8c5?scriptVersionId=6057276", "395632": "You probably need to tweak your learning rate from the default of 0.01. Another thing to consider, I've done a lot of testing to train a model from scratch, I have yet to successfully train a full resolution from scratch on this dataset, even with some exotic loss functions. Normally I have to bootstrap training at a lower resolution first either by downscaling, or doing some smart crops.", "395681": "Can you check the mask that are created by your getImage. What will happen if you have several ships in one image? I may be wrong, but in this case self.masks.EncodedPixels.loc[id] will not be of type str. Let me know if it is the case." }, "source": "meta" }