![\"image\"](\"https://gist.githubusercontent.com/jakubczakon/cac72983726a970690ba7c33708e100b/raw/c1028519e3242c76e9646bbcedc6adfdf165816c/ships_progress.png\")
Hi,
\n\nWe would like to start sharing our results in this competition.
\n\nIt means that we are going to open:
\n\nThese are pretty straightforward (and similar to other competitions):
\n\nWe want this topic to be our journal or project diary, where we discuss our approach , techniques used, network architectures and all other deep learning related stuff! We want this place to be good address for people who want to share knowledge or gain knowledge :)
\n\nHappy Training!
\n\nKamil & Kuba
", "messages": [ { "id": 368405, "postDate": "2018-08-09T20:38:24.683Z", "content": "Hi,
\n\nWe would like to start sharing our results in this competition.
\n\nIt means that we are going to open:
\n\nThese are pretty straightforward (and similar to other competitions):
\n\nWe want this topic to be our journal or project diary, where we discuss our approach , techniques used, network architectures and all other deep learning related stuff! We want this place to be good address for people who want to share knowledge or gain knowledge :)
\n\nHappy Training!
\n\nKamil & Kuba
", "rawMarkdown": "Hi,\n\nWe would like to start sharing our results in this competition.\n\n### The Open Solution approach\nIt means that we are going to open:\n\n1. the [code on GitHub](https://github.com/neptune-ml/open-solution-ship-detection) - *(no worries competitive guys -> we only publish code that scores below bronze medal)*\n2. our [experiments results](https://app.neptune.ml/neptune-ml/Ships)\n3. our approach, that is what we have tried, what worked well, etc.\n\n### Goals\nThese are pretty straightforward (and similar to other competitions):\n\n1. **Learning from the process**.\n2. Encourage more Kagglers to start working on this competition.\n3. Share open source solution with no strings attached, so that less experienced Kagglers can join competition.\n\n### What can you find here?\nWe want this topic to be our *journal* or *project diary*, where we discuss our approach , techniques used, network architectures and all other deep learning related stuff! We want this place to be good address for people who want to share knowledge or gain knowledge :)\n\nHappy Training!\n\nKamil & Kuba\n", "votes": 21 }, { "id": 411567, "postDate": "2018-10-28T13:29:43.053Z", "content": "It pushed the score to CV 719 LB 725.
Fine grained look at the local validation:
\n\nEmpty | f2: 0.994 | gain: 0.003\nNon Empty f2: 0.422 | gain: 0.277\n1 ship f2: 0.445 | gain: 0.178\n2-5 ships f2: 0.393 | gain: 0.083\n5-10 ships f2: 0.308 | gain: 0.012\n10+ ships f2: 0.205 | gain: 0.005\nWe have added:
\n\ntraining in stages:
\n\nfocal(alpha=1.0, gamma=2.0) and lovasz hinge neptune experimentadded intensity based test time augmentation:
\n\n iaa.ContrastNormalization((0.75, 1.25))\nSince we are doing flips (x2x2) and rotations (x4) and random contrast (x8) the inference is very time-consuming. On the flip side, it pushes the score by quite a lot CV +0.09 LB +0.09 so I guess it's here to stay.
Right now we are working on tweaking the loss function to improve the results on the small-one ship images.
", "rawMarkdown": "# Solution 5 (not open yet)\n\nIt pushed the score to `CV 719 LB 725`.\n\nFine grained look at the local validation:\n\n Empty | f2: 0.994 | gain: 0.003\n Non Empty f2: 0.422 | gain: 0.277\n 1 ship f2: 0.445 | gain: 0.178\n 2-5 ships f2: 0.393 | gain: 0.083\n 5-10 ships f2: 0.308 | gain: 0.012\n 10+ ships f2: 0.205 | gain: 0.005\n\nWe have added:\n\n- fixed local validation based on [this post](https://www.kaggle.com/c/airbus-ship-detection/discussion/69322)\n- training in stages:\n - 256x256 some 100 epochs with weighted loss BCE + 0.25 DICE [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/d525f719-ead5-44a2-a59a-558ffbde73d6)\n - 256x256 some 100 epochs with lovasz hinge [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/554c52ad-742d-4aac-adeb-1a3d1da13e61)\n - 512x512 another 100-150 epochs with `focal(alpha=1.0, gamma=2.0)` and lovasz hinge [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/95b36a94-a80f-4020-9fc9-eecf3b5a4d7d)\n - 768x768 another 100 epochs with focal + lovasz hinge [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/66f015b8-5145-48d9-a71a-83dfe52459f4) \n\n- added intensity based test time augmentation:\n\n iaa.ContrastNormalization((0.75, 1.25))\n\n Since we are doing flips (x2x2) and rotations (x4) and random contrast (x8) the inference is very time-consuming. On the flip side, it pushes the score by quite a lot `CV +0.09 LB +0.09` so I guess it's here to stay.\n\nRight now we are working on tweaking the loss function to improve the results on the small-one ship images.", "votes": 3, "replies": [ { "id": 411661, "postDate": "2018-10-28T18:01:45.257Z", "rawMarkdown": "", "isDeleted": true }, { "id": 411917, "postDate": "2018-10-29T07:52:00.577Z", "content": "Yeah, you are right @train2018, it is quite cryptic.
\n\nSo what I did was:\n- I took a subset of validation images where nr_ships=0 (or nr_ships.between(2,5) etc)\n- I calculated the f2 score, which as I learned in this competition, is
5*(precision*recall)/(4*precision+recall)\nThe important part in my view, is that it puts more emphasis on recall. One should try and tweak the model (loss/postprocessing) so that it doesn't lose too many ships :)
", "rawMarkdown": "Yeah, you are right @train2018, it is quite cryptic.\n\nSo what I did was:\n- I took a subset of validation images where `nr_ships=0` (or `nr_ships.between(2,5)` etc)\n- I calculated the f2 score, which as I learned in this competition, is \n\n 5*(precision*recall)/(4*precision+recall)\n\nThe important part in my view, is that it puts more emphasis on recall. One should try and tweak the model (loss/postprocessing) so that it doesn't lose too many ships :)", "votes": 1 } ] }, { "id": 402397, "postDate": "2018-10-11T16:30:37.637Z", "content": "Hi all,\nWe would like to announce that:
\n\nCV 0.541 LB 0.573empty_fraction is set to be 0.0. You can tweak it however you like.You can play around with training/validation parameters by changing stuff in neptune.yaml:
training_sampler_size: 2000\n training_sampler_empty_fraction: 0.0\n evaluation_size: 10000\n evaluation_empty_fraction: 0.52\n in_train_evaluation_size: 1000\nIn order to choose an architecture you need to specify it in the neptune.yaml:
architecture: UNetSeResNetXt\nYou can also combine the losses however you like, so do experiment with those.
\n\nWe added Reduce on plateau callback that automatically reduces LR whenever your model is not improving for a while. You can set the params for it in neptune.yaml:
lr: 0.0007\n momentum: 0.9\n gamma: 0.95\n patience: 10\n validation_metric_name: 'f2'\n minimize_validation_metric: 0\n reduce_factor: 0.5\n reduce_patience: 5\n min_lr: 0\nWe added neptune image channel that visualizes some validation predictions
We created Initial learning rate finder that will help you choose your… initial lr :)\nTo do that you need to uncomment it in the models.py :
\n\n def callbacks_network(callbacks_config):\n experiment_timing = cbk.ExperimentTiming(**callbacks_config['experiment_timing'])\n model_checkpoints = cbk.ModelCheckpoint(**callbacks_config['model_checkpoint'])\n lr_scheduler = cbk.ReduceLROnPlateauScheduler(**callbacks_config['reduce_lr_on_plateau_scheduler'])\n training_monitor = cbk.TrainingMonitor(**callbacks_config['training_monitor'])\n validation_monitor = cbk.ValidationMonitor(**callbacks_config['validation_monitor'])\n neptune_monitor = cbk.NeptuneMonitor(**callbacks_config['neptune_monitor'])\n early_stopping = cbk.EarlyStopping(**callbacks_config['early_stopping'])\n init_lr_finder = cbk.InitialLearningRateFinder()\n return cbk.CallbackList(\n callbacks=[experiment_timing, training_monitor, validation_monitor,\n model_checkpoints, lr_scheduler, neptune_monitor, early_stopping,\n # init_lr_finder\n ])\nThen, based on the charts like this
\n\n\n\npng)
\n\nYou can select the learning rate that will bring the fastest returns as explained in this post
\n\nmain.pyUSE_TTA to True in main.pyBest\nKamil & Kuba
", "rawMarkdown": "Hi all,\nWe would like to announce that:\n\n# Solution 1 is now open!\n\n## Basic information\n- It should get you around `CV 0.541` `LB 0.573`\n- It contains all the boilerplate to get you started quickly\n- All the experiments can be found [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=1bc4da1e-6e47-4a26-a50e-3e55cbc052a7)\n- How-to instructions can be found either in the [neptune project](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=about)\n- If you have any questions regarding the solution please drop a comment either in this post or in [project discussion](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=discussion) (likely faster response)\n\n## What we have developed so far\n\n### Training and Validation Scheme\n- Model is evaluated on the same distribution as test set (0.52 of empty images).\n- You can select the size of the validation set and the size of the in-train validation set that is used in callbacks at the end of each epoch\n- We created a [sampler](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/loaders.py#L48-L79) that selects images from the train set with a specified fraction of empty images. By default we are training on non-empty images and so the `empty_fraction` is set to be 0.0. You can tweak it however you like.\n\nYou can play around with training/validation parameters by changing stuff in `neptune.yaml`:\n\n training_sampler_size: 2000\n training_sampler_empty_fraction: 0.0\n evaluation_size: 10000\n evaluation_empty_fraction: 0.52\n in_train_evaluation_size: 1000\n\n### Architectures\n- [Unet](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/unet.py) with a ton of encoder options \n - Resnet 34/50/101/152\n - SERresnet 50/101/152\n - SEResnetXT 50/101\n - Densenet 121/161/169/201\n[PSPNet](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/pspnet.py) with encoder of choice. You can read more about it in this [paper](https://arxiv.org/pdf/1612.01105.pdf) with encoder of choice\n- LargeKernelMatters](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/large_kernel_matters.py) with Resnet 34/50/101/152 encoder. You can read more about it in this architecture in this [paper](https://arxiv.org/pdf/1703.02719.pdf) \n- Decoders are equipped with both [channel and spatial squeeze and excitation blocks](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/base.py#L65-L117). You can read about them in this [paper](https://arxiv.org/pdf/1808.08127.pdf)\n\nIn order to choose an architecture you need to specify it in the `neptune.yaml`:\n\n architecture: UNetSeResNetXt\n\n### Losses\n- Lovash loss which took Salt Identification by storm. It is a surrogate loss of IOU and you should take a look at the [original paper](https://arxiv.org/pdf/1512.07797.pdf).\n- Focal Loss\n- Dice Loss\n- BCE\nYou can easily choose/change losses [here](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py#L161-L218) by uncommenting:\n loss_function = lovasz_loss\n # loss_function = DiceLoss()\n # loss_function = FocalWithLogitsLoss()\n # loss_function = nn.BCEWithLogitsLoss()\n\nYou can also combine the losses however you like, so do experiment with those.\n\n### Callbacks\n- We created callbacks that calculate both validation loss and the competition metric at the end of each epoch\n- We added Reduce on plateau callback that automatically reduces LR whenever your model is not improving for a while. You can set the params for it in `neptune.yaml`:\n\n lr: 0.0007\n momentum: 0.9\n gamma: 0.95\n patience: 10\n validation_metric_name: 'f2'\n minimize_validation_metric: 0\n reduce_factor: 0.5\n reduce_patience: 5\n min_lr: 0\n\n- We added neptune image channel that visualizes some validation predictions\n\n\n\n- We created Initial learning rate finder that will help you choose your… initial lr :)\nTo do that you need to uncomment it in the [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py#L232-L245) :\n\n def callbacks_network(callbacks_config):\n experiment_timing = cbk.ExperimentTiming(**callbacks_config['experiment_timing'])\n model_checkpoints = cbk.ModelCheckpoint(**callbacks_config['model_checkpoint'])\n lr_scheduler = cbk.ReduceLROnPlateauScheduler(**callbacks_config['reduce_lr_on_plateau_scheduler'])\n training_monitor = cbk.TrainingMonitor(**callbacks_config['training_monitor'])\n validation_monitor = cbk.ValidationMonitor(**callbacks_config['validation_monitor'])\n neptune_monitor = cbk.NeptuneMonitor(**callbacks_config['neptune_monitor'])\n early_stopping = cbk.EarlyStopping(**callbacks_config['early_stopping'])\n init_lr_finder = cbk.InitialLearningRateFinder()\n return cbk.CallbackList(\n callbacks=[experiment_timing, training_monitor, validation_monitor,\n model_checkpoints, lr_scheduler, neptune_monitor, early_stopping,\n # init_lr_finder\n ])\n\nThen, based on the charts like this\n\n\n\nYou can select the learning rate that will bring the fastest returns as explained in this [post](https://www.jeremyjordan.me/nn-learning-rate/)\n\n## Misc\n- We have implemented the [competition metric](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/metrics.py#L80-L87) for you to use it however you like\n- Since the dataset, and images are large we have developed evaluation/prediction in chunks so that you can easily work. - Just choose a chunk size that fits in memory in `main.py`\n- Test-time augmentation with flips (up-down, left-right) and rotations (0,90,180,270) are implemented and can be used by changing `USE_TTA` to `True` in `main.py`\n\n \nBest\nKamil & Kuba", "votes": 3 }, { "id": 375612, "postDate": "2018-08-25T16:02:20.090Z", "content": "Traceback (most recent call last):\nFile \"main.py\", line 89, in <module>\nmain()\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 722, in __call__\n return self.main(*args, **kwargs)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 697, in main\n rv = self.invoke(ctx)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 1066, in invoke\n return _process_result(sub_ctx.command.invoke(sub_ctx))\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 895, in invoke\n return ctx.invoke(self.callback, **ctx.params)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 535, in invoke\n return callback(*args, **kwargs)\nFile \"main.py\", line 27, in train\n pipeline_manager.train(pipeline_name, dev_mode)\nFile \"/home/ary_dhatt/osship/src/pipeline_manager.py\", line 28, in train\n train(pipeline_name, dev_mode)\nFile \"/home/ary_dhatt/osship/src/pipeline_manager.py\", line 77, in train\n pipeline.fit_transform(data)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/steppy/base.py\", line 323, in fit_transform\n step_output_data = self._cached_fit_transform(step_inputs)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/steppy/base.py\", line 443, in _cached_fit_transform\n step_output_data = self.transformer.fit_transform(**step_inputs)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/steppy/base.py\", line 605, in fit_transform\n self.fit(*args, **kwargs)\nFile \"/home/ary_dhatt/osship/src/models.py\", line 68, in fit\n for batch_id, data in enumerate(batch_gen):\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 267, in __next__\n return self._process_next_batch(batch)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 301, in _process_next_batch\n raise batch.exc_type(batch.exc_msg)\nOSError: Traceback (most recent call last):\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in <listcomp>\n samples = collate_fn([dataset[i] for i in batch_indices])\nFile \"/home/ary_dhatt/osship/src/loaders.py\", line 127, in __getitem__\n Xi = load_func(self.X, index, filetype='png', grayscale=False)\nFile \"/home/ary_dhatt/osship/src/loaders.py\", line 159, in load_from_disk\n return self.load_image(img_filepath, grayscale=grayscale)\nFile \"/home/ary_dhatt/osship/src/loaders.py\", line 172, in load_image\n image = image.convert('RGB')\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/PIL/Image.py\", line 879, in convert\n self.load()\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/PIL/ImageFile.py\", line 228, in load\n\"(%d bytes not processed)\" % len(b))\nOSError: image file is truncated (55 bytes not processed)\nI get this error a while after the program logs 2018-08-25 15-23-23 ships-detection >>> epoch 0 .... Do you have an idea of how I can fix this?
Ok, I found the fix. I just had to remove 6384c3e78.jpg from the the csv and the train directory.
", "rawMarkdown": "Ok, I found the fix. I just had to remove 6384c3e78.jpg from the the csv and the train directory.", "votes": 1 }, { "id": 375716, "postDate": "2018-08-25T21:29:59.333Z", "content": "Hmm but it should be excluded as specified in the pipeline_config.py .
\n\nAre you working on the latest master branch?
", "rawMarkdown": "Hmm but it should be excluded as specified in the pipeline_config.py .\n\nAre you working on the latest master branch?" }, { "id": 375723, "postDate": "2018-08-25T21:57:37.590Z", "content": "Yes, I cloned the repository to my GCP instance yesterday evening.
", "rawMarkdown": "Yes, I cloned the repository to my GCP instance yesterday evening.", "votes": 1 }, { "id": 375728, "postDate": "2018-08-25T22:11:23.817Z", "content": "Have you generated metadata.csv after that?
\n\nSorry for nagging but I am only asking to figure out what should be fixed.
", "rawMarkdown": "Have you generated metadata.csv after that? \n\nSorry for nagging but I am only asking to figure out what should be fixed.\n\n" }, { "id": 375733, "postDate": "2018-08-25T22:20:45.227Z", "content": "Yes, after I figured out what was wrong, I deleted the overlayed masks folder AND metadata.csv. Then I removed all references to the file from train_ship_segmentations.csv and the train folder
", "rawMarkdown": "Yes, after I figured out what was wrong, I deleted the overlayed masks folder AND metadata.csv. Then I removed all references to the file from train_ship_segmentations.csv and the train folder", "votes": 1 } ] }, { "id": 375589, "postDate": "2018-08-25T15:16:32.213Z", "content": "If I want to change the image size to 512x512 and change it in the config, do I have to run prepare masks again, or can I just start the training?
", "rawMarkdown": "If I want to change the image size to 512x512 and change it in the config, do I have to run prepare masks again, or can I just start the training?", "votes": 1, "replies": [ { "id": 375597, "postDate": "2018-08-25T15:25:53.460Z", "content": "Just run train with the modified config. Mind that with larger size you may want to run it on a smaller batch or multiple gpus.
", "rawMarkdown": "Just run train with the modified config. Mind that with larger size you may want to run it on a smaller batch or multiple gpus." }, { "id": 375619, "postDate": "2018-08-25T16:25:26.343Z", "content": "Okay, thanks for the quick reply!
", "rawMarkdown": "Okay, thanks for the quick reply!", "votes": 1 } ] }, { "id": 368956, "postDate": "2018-08-11T11:56:09.430Z", "content": "Hello,
\n\nQuick info about our recent work and thoughts... :)
\n\nBest,
\n\nKamil
", "rawMarkdown": "Hello,\n\nQuick info about our recent work and thoughts... :)\n\n1. U-Net -> this architecture is excellent for such competitions. It proved to work well, for example in DSB'18, where [winning solution](https://www.kaggle.com/c/data-science-bowl-2018/discussion/54741) was based on it. U-Net gives you possibility to apply modifications to the architecture, both simple like depth or convolutional block thickness, as well as more sophisticated ones such as custom encoders.\n1. We stick to PyTorch :)\n1. We know that [84% of test images are empty](https://www.kaggle.com/c/airbus-ship-detection/discussion/62376) -> zero ships. In this context, it is recommended to try to balance the signal via sampling (give more positive signal during training).\n1. Next, we will try to use information about the [corrupted images](https://www.kaggle.com/c/airbus-ship-detection/discussion/62921) as reported by @abnerzhang -> thanks!\n1. Also, we will experiment with loss function.\n\nBest,\n\nKamil\n", "votes": 1, "replies": [ { "id": 369582, "postDate": "2018-08-13T12:04:33.760Z", "content": "Hi, I have some question since I'm quite new to image segmentation.
\n\nSince you use U-Net, I assume the output are single image mask, which then will be separated to it's own mask for each ship that are predicted. How do you that?
\n\nThank you.
", "rawMarkdown": "Hi, I have some question since I'm quite new to image segmentation. \n\nSince you use U-Net, I assume the output are single image mask, which then will be separated to it's own mask for each ship that are predicted. How do you that? \n\nThank you.", "votes": 1 }, { "id": 369600, "postDate": "2018-08-13T12:26:14.037Z", "content": "Hi @ivanachlaqullah !
\n\nYes, you are right - output is a single mask, where 0 means background and 1 means ship. Later we apply ndi.label transformation (code). This function looks for all not connected instances of class 1 and puts unique label to each instance. Later we just run RLE on this labeled mask (code)
\n\nCheers and good luck in the competition!
\n\nAndrzej
", "rawMarkdown": "Hi @ivanachlaqullah !\n\nYes, you are right - output is a single mask, where 0 means background and 1 means ship. Later we apply ndi.label transformation ([code][1]). This function looks for all not connected instances of class 1 and puts unique label to each instance. Later we just run RLE on this labeled mask ([code][2])\n\nCheers and good luck in the competition!\n\nAndrzej\n\n [1]: https://github.com/neptune-ml/open-solution-ship-detection/blob/master/src/pipelines.py#L217\n [2]: https://github.com/neptune-ml/open-solution-ship-detection/blob/master/src/utils.py#L109", "votes": 1 }, { "id": 374856, "postDate": "2018-08-24T01:57:51.017Z", "content": "Can you update use weekly on different ideas that you found worked well or not? Also, your code is really well organized. And thanks for keeping it below bronze(otherwise I would have probably just used the code and never coded at all).
", "rawMarkdown": "Can you update use weekly on different ideas that you found worked well or not? Also, your code is really well organized. And thanks for keeping it below bronze(otherwise I would have probably just used the code and never coded at all).", "votes": 1 } ] }, { "id": 368411, "postDate": "2018-08-09T20:43:59.717Z", "content": "By the end of the week, I will add some info about techniques that worked well in Ships detection :)
\n\nAlso, I want to share what you can find in our starter code (of course, it is below bronze medal, as we discussed with the community :) ).
", "rawMarkdown": "By the end of the week, I will add some info about techniques that worked well in Ships detection :)\n\nAlso, I want to share what you can find in our starter code (of course, it is below bronze medal, as we discussed with the community :) ).\n", "votes": 1 }, { "id": 371803, "postDate": "2018-08-17T16:22:03.907Z", "content": "I still believe you it could be helpful to fix your metric :)
", "rawMarkdown": "I still believe you it could be helpful to [fix your metric](https://github.com/neptune-ml/open-solution-ship-detection/pull/2#issuecomment-413595209) :)", "votes": 2, "replies": [ { "id": 371839, "postDate": "2018-08-17T17:08:59.550Z", "content": "Thank you @Varal7 for dropping that PR. I just wanted to think about it and was really swamped with other stuff.
\n\nI will look into that first thing next week.
\n\nThanks a lot for that fix!
", "rawMarkdown": "Thank you @Varal7 for dropping that PR. I just wanted to think about it and was really swamped with other stuff. \n\nI will look into that first thing next week.\n\nThanks a lot for that fix!", "votes": 2 } ] }, { "id": 413341, "postDate": "2018-10-31T18:44:56.827Z", "content": "I cannot find the neptune.yaml file in the repository
", "rawMarkdown": "I cannot find the neptune.yaml file in the repository" }, { "id": 417049, "postDate": "2018-11-07T16:46:16.820Z", "content": "Done @Chandan Verma.
\n\nSorry about that.
", "rawMarkdown": "Done @Chandan Verma.\n\nSorry about that.", "votes": 1 } ] }, { "id": 412206, "postDate": "2018-10-29T19:30:16.167Z", "content": "Neptune.ml is pretty neat!
\n\nIt would be cool to see the confusion matrix or drill down into the false positives / negatives as part of the summary in an experiment.
\n\nBasically this:
\n\n\nEmpty | f2: 0.994 | gain: 0.003\nNon Empty f2: 0.422 | gain: 0.277\n1 ship f2: 0.445 | gain: 0.178\n2-5 ships f2: 0.393 | gain: 0.083\n5-10 ships f2: 0.308 | gain: 0.012\n10+ ships f2: 0.205 | gain: 0.005\n
As a configurable part of the summary of each experiment.
", "rawMarkdown": "Neptune.ml is pretty neat!\n\nIt would be cool to see the confusion matrix or drill down into the false positives / negatives as part of the summary in an experiment.\n\nBasically this:\n\n```\nEmpty | f2: 0.994 | gain: 0.003\nNon Empty f2: 0.422 | gain: 0.277\n1 ship f2: 0.445 | gain: 0.178\n2-5 ships f2: 0.393 | gain: 0.083\n5-10 ships f2: 0.308 | gain: 0.012\n10+ ships f2: 0.205 | gain: 0.005\n```\n\nAs a configurable part of the summary of each experiment.", "replies": [ { "id": 412421, "postDate": "2018-10-30T07:42:27.340Z", "content": "Hi @phun, I am glad you like it.
\n\nWe are working on improving charts/image channel funcionality as we speak, but in the meantime \nyou can do that by sending the matplotlib chart to Neptune via image_channel.
\n\nYou need to convert it first to PIL by running something like this:
\n\nimport numpy as np\nfrom PIL import Image\n\ndef fig2pil(fig):\n fig.canvas.draw()\n\n w,h = fig.canvas.get_width_height()\n buf = np.fromstring(fig.canvas.tostring_argb(), dtype=np.uint8)\n buf.shape = (w, h, 4)\n buf = np.roll(buf, 3, axis=2)\n\n buf = fig2numpy(fig)\n w, h, d = buf.shape\n return Image.frombytes(\"RGBA\", (w , h), buf.tostring())\nand then you can send it to Neptune, even after every epoch:
\n\nimport neptune\nimport matplotlib.pyplot as plt\n\nctx = neptune.Context()\n\n for i in range(epoch_nr): \n ...\n fig = plt.figure()\n plot_confusion_matrix(results)\n\n pil_image = fig2pil(fig)\n ctx.channel_send('confusion_matrix', \n neptune.Image(name='cm iter {}'.format(i),\n description='Confusion matrix',\n data=pil_image))\nYour charts will be presented in the confusion_matrix channel, and will look similiar to this:
![\"image\"](\"https://gist.githubusercontent.com/jakubczakon/cac72983726a970690ba7c33708e100b/raw/23bbf33e17fdcabb6f9ae46e41226faa3626bd88/matplotlib_chart.png\")
It pushed the score to CV 722 LB 703.
We have added:
\n\nWe have encountered heavy overfitting on our local validation. We are testing the idea from this kernel
\n\nThat's it
", "rawMarkdown": "# Solution 4 (not open yet)\n\nIt pushed the score to `CV 722 LB 703`.\n\nWe have added:\n\n- cyclic learning rates\n- squeeze and excitation (spatial and channel wise) to the deconv layer of large kernel matters.\n\nWe have encountered heavy overfitting on our local validation. We are testing the idea from this [kernel](https://www.kaggle.com/manuscrits/create-a-validation-dataset-correcting-the-leak)\n\nThat's it" }, { "id": 406496, "postDate": "2018-10-19T10:56:14.337Z", "content": "Hi all,\nWe would like to announce that:
\n\nCV 0.694 LB 0.696f2 0.31 for ship masks.We are training in 2 stages:
\n\nAround 180 epochs with BCE + 0.25 * DICE and reduce on plateau callback. Check the experiment here. You can select your loss and weights in models.py
\n\n def set_loss(self):\n if self.activation_func == 'softmax':\n raise NotImplementedError('No softmax loss defined')\n elif self.activation_func == 'sigmoid':\n\n loss_function = weighted_sum_loss\n # loss_function = nn.BCEWithLogitsLoss()\n # loss_function = DiceWithLogitsLoss()\n # loss_function = lovasz_loss\n # loss_function = FocalWithLogitsLoss()\nYou can play with those values in the neptune.yaml:
postpro__drop_size: 50\n postpro__mid_min_size: 50\n postpro__mid_max_size: 1000\nBest\nKamil & Kuba
", "rawMarkdown": "Hi all,\nWe would like to announce that:\n\n# Solution 3 is now open!\n\n## Basic information\n- It should get you around `CV 0.694` `LB 0.696`\n- All the experiments can be found [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=be842434-7c8b-4ab9-afa5-f9c00816d3c3)\n- How-to instructions can be found either in the [neptune project](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=about)\n- If you have any questions regarding the solution please drop a comment either in this post or in [project discussion](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=discussion) (likely faster response)\n\n\n## What have we improved\n\n### Architectures\n- We experimented with different flavours and what works best is **Large Kernel Matters** with ** Densenet 201** encoder . That gets `f2 0.31` for ship masks.\n- We also chose **Densenet 201** for the ship/no ship model. Gets 0.98+ accuracy and f2 0.996 for no ship images.\n\n### Training\n- We are sampling 7500 images every epoch\n- We are training in 2 stages:\n - Around 180 epochs with BCE + 0.25 * DICE and reduce on plateau callback. Check the [experiment here](https://app.neptune.ml/-/dashboard/experiment/6e866735-db5b-4b8c-ad79-a71e1224377c). You can select your loss and weights in [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py)\n\n def set_loss(self):\n if self.activation_func == 'softmax':\n raise NotImplementedError('No softmax loss defined')\n elif self.activation_func == 'sigmoid':\n\n loss_function = weighted_sum_loss\n # loss_function = nn.BCEWithLogitsLoss()\n # loss_function = DiceWithLogitsLoss()\n # loss_function = lovasz_loss\n # loss_function = FocalWithLogitsLoss()\n\n - Train for another 130 epoch with Lovash loss. Check the [experiment here](https://app.neptune.ml/-/dashboard/experiment/b77a708e-3410-4932-8550-e61ca72c33b8). Again you can select your loss in [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py)\n\n### Post-processing\n- We are dropping predicted object masks if they are smaller than 50 pixels\n- For objects between 50 and 1000 pixels, we apply the mask to bbox function which draws the minimal rectangle over those objects\n\nYou can play with those values in the `neptune.yaml`:\n\n postpro__drop_size: 50\n postpro__mid_min_size: 50\n postpro__mid_max_size: 1000\n\n### Misc\n- Parallel apply of mask resize and other post-processing functions speed up inference by a lot (~10x).\n- Added prediction [exploration notebook](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/prediction_exploration.ipynb) where you can inspect your model in more detail and figure out where it is not doing so well.\n \nBest\nKamil & Kuba\n" }, { "id": 404169, "postDate": "2018-10-15T11:16:50.477Z", "content": "We have just made the model weights for solution-1 and solution-2 available in here.
\n\n![\"image\"](\"https://gist.githubusercontent.com/jakubczakon/cac72983726a970690ba7c33708e100b/raw/435e7bbdb567882eae0825d34818549438d0b7cc/neptune_download.png\")
Feel free to use them however you like.
", "rawMarkdown": "We have just made the model weights for solution-1 and solution-2 available in [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=browseFiles).\n\n[](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=browseFiles)\n\nFeel free to use them however you like." }, { "id": 404121, "postDate": "2018-10-15T09:13:35.460Z", "content": "Hi all,\nWe would like to announce that:
\n\nCV 0.661 LB 0.679![\"image\"](\"https://gist.githubusercontent.com/jakubczakon/cac72983726a970690ba7c33708e100b/raw/f25c8b4f6a16d370588541fc6b0f25ba3e3c4c75/ships_exp_images_1.png\")
![\"image\"](\"https://gist.githubusercontent.com/jakubczakon/cac72983726a970690ba7c33708e100b/raw/f25c8b4f6a16d370588541fc6b0f25ba3e3c4c75/ships_exp_images_2.png\")
![\"image\"](\"https://gist.githubusercontent.com/jakubczakon/cac72983726a970690ba7c33708e100b/raw/f25c8b4f6a16d370588541fc6b0f25ba3e3c4c75/ships_exp_images_3.png\")
INFERENCE_WITH_SHIP_NO_SHIP to True/FalseWe implemented encoder freezing and batchnorm freezing during training and an interface to play with that easily. You simply need to go to models.py and input your logic:
\n\n def freeze_weights(self):\n # freeze encoder\n if isinstance(self.model, nn.DataParallel):\n encoder_params = self.model.module.encoder.parameters()\n else:\n encoder_params = self.model.encoder.parameters()\n\n for parameter in encoder_params:\n parameter.requires_grad = False\n\n # freeze batchnorm\n for m in self.model.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n m.weight.requires_grad = False\n m.bias.requires_grad = False\n pass\nWe dropped mask to oriented bounding box in postprocessing. Basically the idea was to make all masks oriented rectangular objects since that is the target but for some reason it made results a bit worse. The function may come in handy later:
\n\n import numpy as np\n import cv2\n\ndef masks_to_bounding_boxes(labeled_mask):\n if labeled_mask.max() == 0:\n return labeled_mask\n else:\n img_box = np.zeros_like(labeled_mask)\n for label_id in range(1, labeled_mask.max() + 1, 1):\n label = np.where(labeled_mask == label_id, 1, 0).astype(np.uint8)\n _, cnt, _ = cv2.findContours(label, 1, 2)\n rect = cv2.minAreaRect(cnt[0])\n box = cv2.boxPoints(rect)\n box = np.int0(box)\n cv2.drawContours(img_box, [box], 0, label_id, -1)\n return img_box\nBest\nKamil & Kuba
", "rawMarkdown": "Hi all,\nWe would like to announce that:\n\n# Solution 2 is now open!\n\n## Basic information\n- It should get you around `CV 0.661` `LB 0.679`\n- All the experiments can be found [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=8ad61fcb-f0ac-4aaf-aa9c-9db47e0aa222)\n- How-to instructions can be found either in the [neptune project](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=about)\n- If you have any questions regarding the solution please drop a comment either in this post or in [project discussion](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=discussion) (likely faster response)\n\n## Some example outputs from the best model\n\n[](https://app.neptune.ml/-/dashboard/experiment/97f7385e-a6cd-4302-bc35-2863f364e9ed)\n\n[](https://app.neptune.ml/-/dashboard/experiment/97f7385e-a6cd-4302-bc35-2863f364e9ed)\n\n[](https://app.neptune.ml/-/dashboard/experiment/97f7385e-a6cd-4302-bc35-2863f364e9ed)\n\n\n## What have we improved\n\n### Architectures\n- We decoupled Encoders from Architectures so now you can combine Resnet/SeResNet/SeResNeXt/DenseNet with Unet/LargeKernelMatters/PSPNet however you like\nThe best model so far is actually LargeKernelMatters with SeResNeXT encoder. What is important is that this model is less resource heavy. I can train on 1 GPU with 16 image batch\n\n\n### Misc\n- We implemented two stage ship/no_ship + segmentation pipeline. You train both binary and segmentation model separately and then you can run inference with one or two stage model. Simply change global setup in the [main.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/main.py) by changing the `INFERENCE_WITH_SHIP_NO_SHIP` to True/False\n- We implemented encoder freezing and batchnorm freezing during training and an interface to play with that easily. You simply need to go to [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py#L194-L210) and input your logic:\n\n def freeze_weights(self):\n # freeze encoder\n if isinstance(self.model, nn.DataParallel):\n encoder_params = self.model.module.encoder.parameters()\n else:\n encoder_params = self.model.encoder.parameters()\n \n for parameter in encoder_params:\n parameter.requires_grad = False\n \n # freeze batchnorm\n for m in self.model.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n m.weight.requires_grad = False\n m.bias.requires_grad = False\n pass\n\n- We dropped mask to oriented bounding box in postprocessing. Basically the idea was to make all masks oriented rectangular objects since that is the target but for some reason it made results a bit worse. The function may come in handy later:\n\n import numpy as np\n import cv2\n\n def masks_to_bounding_boxes(labeled_mask):\n if labeled_mask.max() == 0:\n return labeled_mask\n else:\n img_box = np.zeros_like(labeled_mask)\n for label_id in range(1, labeled_mask.max() + 1, 1):\n label = np.where(labeled_mask == label_id, 1, 0).astype(np.uint8)\n _, cnt, _ = cv2.findContours(label, 1, 2)\n rect = cv2.minAreaRect(cnt[0])\n box = cv2.boxPoints(rect)\n box = np.int0(box)\n cv2.drawContours(img_box, [box], 0, label_id, -1)\n return img_box\n \nBest\nKamil & Kuba" }, { "id": 381979, "postDate": "2018-09-05T13:29:09.063Z", "content": "\n\n\nTraceback (most recent call last):
\n \nFile \"main.py\", line 89, in
\n \nmain()
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 722, in call
\n \nreturn self.main(*args, **kwargs)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 697, in main
\n \nrv = self.invoke(ctx)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 1066, in invoke
\n \nreturn _process_result(sub_ctx.command.invoke(sub_ctx))
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 895, in invoke
\n \nreturn ctx.invoke(self.callback, **ctx.params)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 535, in invoke
\n \nreturn callback(*args, **kwargs)
\n \nFile \"main.py\", line 27, in train
\n \npipeline_manager.train(pipeline_name, dev_mode)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/pipeline_manager.py\", line 28, in train
\n \ntrain(pipeline_name, dev_mode)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/pipeline_manager.py\", line 77, in train
\n \npipeline.fit_transform(data)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/steppy/base.py\", line 323, in fit_transform
\n \nstep_output_data = self._cached_fit_transform(step_inputs)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/steppy/base.py\", line 443, in _cached_fit_transform
\n \nstep_output_data = self.transformer.fit_transform(**step_inputs)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/steppy/base.py\", line 605, in fit_transform
\n \nself.fit(*args, **kwargs)\n File \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/models.py\", line 68, in fit
\n \nfor batch_id, data in enumerate(batch_gen):
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 281, in next
\n \nreturn self._process_next_batch(batch)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 301, in _process_next_batch
\n \nraise batch.exc_type(batch.exc_msg)
\n \nFileNotFoundError: Traceback (most recent call last):
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in _worker_loop
\n \nsamples = collate_fn([dataset[i] for i in batch_indices])
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in
\n \nsamples = collate_fn([dataset[i] for i in batch_indices])
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 132, in getitem
\n \nMi = self.load_target(self.y, index, load_func)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 211, in load_target
\n \nMi = load_func(data_source, index, filetype='joblib')
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 167, in load_from_disk
\n \nreturn self.load_joblib(img_filepath)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 180, in load_joblib
\n \ntarget = joblib.load(img_filepath)\n File \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/sklearn/externals/joblib/numpy_pickle.py\", line 570, in load
\n \nwith open(filename, 'rb') as f:
\n \nFileNotFoundError: [Errno 2] No such file or directory: '/home/kwu/data/kaggle/AirbusShipDetection/masks/6384c3e78'
\n
when I use
\n\npython main.py -- prepare_masks
python main.py -- prepare_metadata
python main.py -- train --pipeline_name unet
I don't have find this /home/kwu/data/kaggle/AirbusShipDetection/masks/6384c3e78 in my path. I don't know how this tag came from.
Hi,
\n\nJust one quick question: did you configure all paths correctly in the neptune.yaml - especially path to masks? This might be the root cause of your error.
\n\nBest,
\n\nKamil
", "rawMarkdown": "Hi,\n\nJust one quick question: did you configure all paths correctly in the neptune.yaml - especially path to masks? This might be the root cause of your error.\n\nBest,\n\nKamil" } ] }, { "id": 381196, "postDate": "2018-09-04T08:57:10.623Z", "content": "In the segmentation file, a label of a picture is divided into a several parts , is it necessary to fuse them before train?
", "rawMarkdown": "In the segmentation file, a label of a picture is divided into a several parts , is it necessary to fuse them before train?", "replies": [ { "id": 381229, "postDate": "2018-09-04T10:22:10.140Z", "content": "Since we are training unets we do need to have one mask.\nHowever we could fuse them at train time. The problem with this (when training many epochs) is that it is quite slow and can be done beforehand to speed up training.
", "rawMarkdown": "Since we are training unets we do need to have one mask.\nHowever we could fuse them at train time. The problem with this (when training many epochs) is that it is quite slow and can be done beforehand to speed up training." } ] }, { "id": 380244, "postDate": "2018-09-02T06:03:35.520Z", "content": "getting the following error:\nTypeError: init() got an unexpected keyword argument 'is_trainable'\nwhile running : python main.py -- evaluate_predict --pipeline_name unet
", "rawMarkdown": "getting the following error:\nTypeError: __init__() got an unexpected keyword argument 'is_trainable'\nwhile running : python main.py -- evaluate_predict --pipeline_name unet", "replies": [ { "id": 380278, "postDate": "2018-09-02T07:55:09.157Z", "content": "Are you using the exact same requirements.txt ? What are your steppy and steppy-toolkit versions ?
", "rawMarkdown": "Are you using the exact same requirements.txt ? What are your steppy and steppy-toolkit versions ?", "votes": -1 }, { "id": 380293, "postDate": "2018-09-02T08:48:52.043Z", "content": "steppy==0.1.5\nsteppy-toolkit==0.1.8
\n\nif i use steppy 1.6 i get the following
\n\nFile \"E:\\toolkits.win\\anaconda3-5.2.0\\envs\\dlwin36\\lib\\site-packages\\steppy\\base.py\", line 477, in _cached_transform\n raise ValueError('No transformer cached {}'.format(self.name))\nValueError: No transformer cached unet
", "rawMarkdown": "steppy==0.1.5\nsteppy-toolkit==0.1.8\n\nif i use steppy 1.6 i get the following\n\n File \"E:\\toolkits.win\\anaconda3-5.2.0\\envs\\dlwin36\\lib\\site-packages\\steppy\\base.py\", line 477, in _cached_transform\n raise ValueError('No transformer cached {}'.format(self.name))\nValueError: No transformer cached unet" }, { "id": 380644, "postDate": "2018-09-03T06:23:37.420Z", "content": "Ok, I see.
\n\nI will try to reproduce and get back to you.
\n\nBTW, please drop problems with the code in the github repo issues section. I feel it's just a more natural way of talking about bugs.
", "rawMarkdown": "Ok, I see.\n\nI will try to reproduce and get back to you.\n\nBTW, please drop problems with the code in the github repo issues section. I feel it's just a more natural way of talking about bugs." } ] }, { "id": 371437, "postDate": "2018-08-16T19:56:28.917Z", "content": "Is this the correct reproduce_results.md file for open-solution-ship-detection?
", "rawMarkdown": "Is this the correct [reproduce_results.md][1] file for open-solution-ship-detection? \n\n\n [1]: https://github.com/neptune-ml/open-solution-ship-detection/blob/master/REPRODUCE_RESULTS.md", "replies": [ { "id": 371584, "postDate": "2018-08-17T06:57:21.450Z", "content": "Hi @William Green.
\n\nThat file was pasted from another project by accident.
\n\nI will update it today (hopefully).
", "rawMarkdown": "Hi @William Green. \n\nThat file was pasted from another project by accident. \n\nI will update it today (hopefully)." } ] }, { "id": 369994, "postDate": "2018-08-14T04:09:21.383Z", "content": "HI, got this error on training. Could you please shed some lights? Thanks.
\n\nneptune run main.py -- train --pipeline_name unet
\n\n2018-08-14 14-00-49 ships-detection >>> epoch 0 ...\nTraceback (most recent call last):\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/deepsense/neptune/job_wrapper.py\", line 107, in \n execute()\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/deepsense/neptune/job_wrapper.py\", line 103, in execute\n execfile(job_filepath, job_globals)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/past/builtins/misc.py\", line 82, in execfile\n exec_(code, myglobals, mylocals)\n File \"main.py\", line 89, in \n main()\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 722, in call\n return self.main(*args, **kwargs)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 697, in main\n rv = self.invoke(ctx)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 1066, in invoke\n return _process_result(sub_ctx.command.invoke(sub_ctx))\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 895, in invoke\n return ctx.invoke(self.callback, **ctx.params)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 535, in invoke\n return callback(*args, **kwargs)\n File \"main.py\", line 27, in train\n pipeline_manager.train(pipeline_name, dev_mode)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/pipeline_manager.py\", line 27, in train\n train(pipeline_name, dev_mode)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/pipeline_manager.py\", line 76, in train\n pipeline.fit_transform(data)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/steppy/base.py\", line 323, in fit_transform\n step_output_data = self._cached_fit_transform(step_inputs)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/steppy/base.py\", line 443, in _cached_fit_transform\n step_output_data = self.transformer.fit_transform(**step_inputs)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/steppy/base.py\", line 605, in fit_transform\n self.fit(*args, **kwargs)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/models.py\", line 71, in fit\n self.callbacks.on_batch_end(metrics=metrics)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/callbacks.py\", line 118, in on_batch_end\n callback.on_batch_end(*args, **kwargs)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/callbacks.py\", line 149, in on_batch_end\n loss = loss.data.cpu().numpy()[0]\nIndexError: too many indices for array\nCalculated experiment snapshot size: 0 Bytes
\nProcess exited with return code 1.
Have you installed pytorch from the requirements. It seems like it could be some sort of pytorch 0.3.1 vs 0.4 .
", "rawMarkdown": "Have you installed pytorch from the requirements. It seems like it could be some sort of pytorch 0.3.1 vs 0.4 . " }, { "id": 370048, "postDate": "2018-08-14T06:25:50.820Z", "content": "Indeed. my torch version is 0.4. Thanks.
", "rawMarkdown": "Indeed. my torch version is 0.4. Thanks. " } ] }, { "id": 369802, "postDate": "2018-08-13T19:21:26.167Z", "content": "I get a error when I run python main.py -- prepare_masks
neptune: Executing in Offline Mode.\nTraceback (most recent call last):\nFile \"main.py\", line 2, in <module>\nfrom src.pipeline_manager import PipelineManager\nFile \"/floyd/home/src/pipeline_manager.py\", line 8, in <module>\nfrom .pipelines import PIPELINES\nFile \"/floyd/home/src/pipelines.py\", line 7, in <module>\nfrom .models import PyTorchUNet\nFile \"/floyd/home/src/models.py\", line 8, in <module>\nfrom toolkit.pytorch_transformers.architectures.unet import UNet\n File \"/usr/local/lib/python3.6/site-packages/toolkit/pytorch_transformers/architectures/unet.py\", line 6, in <module>\nfrom toolkit.pytorch_transformers.utils import get_downsample_pad, get_upsample_pad\nImportError: cannot import name 'get_downsample_pad'\nHi @William Green,
\n\nMake sure to have the exact requirements.txt from the master branch.
I've updated that a moment ago and tested on a fresh environment.
\n\nI hope you will have no problem running it this time.
", "rawMarkdown": "Hi @William Green,\n\nMake sure to have the exact `requirements.txt` from the master branch.\n\nI've updated that a moment ago and tested on a fresh environment.\n\nI hope you will have no problem running it this time.", "votes": 1 }, { "id": 370290, "postDate": "2018-08-14T15:18:17.137Z", "content": "Thanks you @Jakub.
\n\nWhen I run !python main.py -- prepare_metadata --train_data --valid_data --test_data
I get the following error:
\n\nneptune: Executing in Offline Mode.\nneptune: Executing in Offline Mode.\nError: no such option: --train_data\nAlso, if I just run:
\n\n!python main.py -- prepare_metadata\nI get the following error:
\n\nneptune: Executing in Offline Mode.\nneptune: Executing in Offline Mode.\n2018-08-14 14-56-59 ships-detection >>> creating metadata\n100%|██████████████████████████████████| 104070/104070 [07:11<00:00, 241.13it/s]\n100%|█████████████████████████████████| 88500/88500 [00:00<00:00, 237279.13it/s]\nTraceback (most recent call last):\nFile \"main.py\", line 89, in <module>\nmain()\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 722, in __call__\nreturn self.main(*args, **kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 697, in main\nrv = self.invoke(ctx)\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 1066, in invoke\nreturn _process_result(sub_ctx.command.invoke(sub_ctx))\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 895, in invoke\nreturn ctx.invoke(self.callback, **ctx.params)\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 535, in invoke\nreturn callback(*args, **kwargs)\nFile \"main.py\", line 20, in prepare_metadata\npipeline_manager.prepare_metadata()\nFile \"/floyd/home/src/pipeline_manager.py\", line 25, in prepare_metadata\nprepare_metadata()\nFile \"/floyd/home/src/pipeline_manager.py\", line 52, in prepare_metadata\nmeta.to_csv(os.path.join(PARAMS.meta_dir, 'metadata.csv'), index=None)\nFile \"/usr/local/lib/python3.6/site-packages/pandas/core/frame.py\", line 1524, in to_csv\nformatter.save()\nFile \"/usr/local/lib/python3.6/site-packages/pandas/io/formats/format.py\", line 1637, in save\ncompression=self.compression)\nFile \"/usr/local/lib/python3.6/site-packages/pandas/io/common.py\", line 390, in _get_handle\nf = open(path_or_buf, mode, encoding=encoding)\nFileNotFoundError: [Errno 2] No such file or directory: 'meta/metadata.csv'\nIt seems not be creating the metadata folder, so it can save the file.
", "rawMarkdown": "Thanks you @Jakub. \n\nWhen I run `!python main.py -- prepare_metadata --train_data --valid_data --test_data`\n\nI get the following error: \n\n neptune: Executing in Offline Mode.\n neptune: Executing in Offline Mode.\n Error: no such option: --train_data\n\n\nAlso, if I just run: \n\n !python main.py -- prepare_metadata\n\nI get the following error: \n\n neptune: Executing in Offline Mode.\n neptune: Executing in Offline Mode.\n 2018-08-14 14-56-59 ships-detection >>> creating metadata\n 100%|██████████████████████████████████| 104070/104070 [07:11<00:00, 241.13it/s]\n 100%|█████████████████████████████████| 88500/88500 [00:00<00:00, 237279.13it/s]\n Traceback (most recent call last):\n File \"main.py\", line 89, in " }, { "id": 370364, "postDate": "2018-08-14T17:35:43.610Z", "content": "I guess you need to create that folder. I usually create folders data, files experiments for every project so I never run into this problem.
what about ?
\n\n!python main.py -- prepare_metadata --train_data --valid_data --test_data
I get the following error:
\n\nneptune: Executing in Offline Mode.\nneptune: Executing in Offline Mode.\nError: no such option: --train_data\nI get the a error when I try to run python main.py -- prepare_masks :
neptune: Executing in Offline Mode.\nTraceback (most recent call last):\nFile \"main.py\", line 2, in <module>\nfrom src.pipeline_manager import PipelineManager\nFile \"/floyd/home/src/pipeline_manager.py\", line 6, in <module>\nfrom .metrics import f_beta_metric\nFile \"/floyd/home/src/metrics.py\", line 8, in <module>\nfrom .pipeline_config import ORIGINAL_SIZE\nFile \"/floyd/home/src/pipeline_config.py\", line 106, in <module>\n'annotation_file': PARAMS.annotation_file,\nFile \"/usr/local/lib/python3.6/site-packages/attrdict/mixins.py\", line 82, in __getattr__\ncls=self.__class__.__name__, name=key\nAttributeError: 'AttrDict' instance has no attribute 'annotation_file'\nSorry about that @William Green .
\n\nneptune.yaml was missing params. It is fixed already.
@Jakub, thank you
", "rawMarkdown": "@Jakub, thank you" } ] }, { "id": 369457, "postDate": "2018-08-13T04:58:59.457Z", "content": "Hi @abnerzhang. Could you elaborate?
", "rawMarkdown": "Hi @abnerzhang. Could you elaborate?", "votes": -1 }, { "id": 369534, "postDate": "2018-08-13T09:47:46.083Z", "content": "Hi @abnerzhang,
\n\nNot at this point. However, for sure we need to treat it somehow...
\n\nBest,
\n\nKamil
", "rawMarkdown": "Hi @abnerzhang,\n\nNot at this point. However, for sure we need to treat it somehow...\n\nBest,\n\nKamil\n" } ] } ], "comments": [ { "id": 411567, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-28T13:29:43.053000", "content": "It pushed the score to CV 719 LB 725.
Fine grained look at the local validation:
\n\nEmpty | f2: 0.994 | gain: 0.003\nNon Empty f2: 0.422 | gain: 0.277\n1 ship f2: 0.445 | gain: 0.178\n2-5 ships f2: 0.393 | gain: 0.083\n5-10 ships f2: 0.308 | gain: 0.012\n10+ ships f2: 0.205 | gain: 0.005\nWe have added:
\n\ntraining in stages:
\n\nfocal(alpha=1.0, gamma=2.0) and lovasz hinge neptune experimentadded intensity based test time augmentation:
\n\n iaa.ContrastNormalization((0.75, 1.25))\nSince we are doing flips (x2x2) and rotations (x4) and random contrast (x8) the inference is very time-consuming. On the flip side, it pushes the score by quite a lot CV +0.09 LB +0.09 so I guess it's here to stay.
Right now we are working on tweaking the loss function to improve the results on the small-one ship images.
", "votes": 3, "replies": [ { "id": 411661, "author_name": "", "author_url": "", "post_date": "2018-10-28T18:01:45.257000", "content": "", "votes": 0, "replies": [] }, { "id": 411917, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-29T07:52:00.577000", "content": "Yeah, you are right @train2018, it is quite cryptic.
\n\nSo what I did was:\n- I took a subset of validation images where nr_ships=0 (or nr_ships.between(2,5) etc)\n- I calculated the f2 score, which as I learned in this competition, is
5*(precision*recall)/(4*precision+recall)\nThe important part in my view, is that it puts more emphasis on recall. One should try and tweak the model (loss/postprocessing) so that it doesn't lose too many ships :)
", "votes": 1, "replies": [] } ] }, { "id": 402397, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-11T16:30:37.637000", "content": "Hi all,\nWe would like to announce that:
\n\nCV 0.541 LB 0.573empty_fraction is set to be 0.0. You can tweak it however you like.You can play around with training/validation parameters by changing stuff in neptune.yaml:
training_sampler_size: 2000\n training_sampler_empty_fraction: 0.0\n evaluation_size: 10000\n evaluation_empty_fraction: 0.52\n in_train_evaluation_size: 1000\nIn order to choose an architecture you need to specify it in the neptune.yaml:
architecture: UNetSeResNetXt\nYou can also combine the losses however you like, so do experiment with those.
\n\nWe added Reduce on plateau callback that automatically reduces LR whenever your model is not improving for a while. You can set the params for it in neptune.yaml:
lr: 0.0007\n momentum: 0.9\n gamma: 0.95\n patience: 10\n validation_metric_name: 'f2'\n minimize_validation_metric: 0\n reduce_factor: 0.5\n reduce_patience: 5\n min_lr: 0\nWe added neptune image channel that visualizes some validation predictions
We created Initial learning rate finder that will help you choose your… initial lr :)\nTo do that you need to uncomment it in the models.py :
\n\n def callbacks_network(callbacks_config):\n experiment_timing = cbk.ExperimentTiming(**callbacks_config['experiment_timing'])\n model_checkpoints = cbk.ModelCheckpoint(**callbacks_config['model_checkpoint'])\n lr_scheduler = cbk.ReduceLROnPlateauScheduler(**callbacks_config['reduce_lr_on_plateau_scheduler'])\n training_monitor = cbk.TrainingMonitor(**callbacks_config['training_monitor'])\n validation_monitor = cbk.ValidationMonitor(**callbacks_config['validation_monitor'])\n neptune_monitor = cbk.NeptuneMonitor(**callbacks_config['neptune_monitor'])\n early_stopping = cbk.EarlyStopping(**callbacks_config['early_stopping'])\n init_lr_finder = cbk.InitialLearningRateFinder()\n return cbk.CallbackList(\n callbacks=[experiment_timing, training_monitor, validation_monitor,\n model_checkpoints, lr_scheduler, neptune_monitor, early_stopping,\n # init_lr_finder\n ])\nThen, based on the charts like this
\n\n\n\npng)
\n\nYou can select the learning rate that will bring the fastest returns as explained in this post
\n\nmain.pyUSE_TTA to True in main.pyBest\nKamil & Kuba
", "votes": 3, "replies": [] }, { "id": 375612, "author_name": "Arpan Dhatt", "author_url": "", "post_date": "2018-08-25T16:02:20.090000", "content": "Traceback (most recent call last):\nFile \"main.py\", line 89, in <module>\nmain()\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 722, in __call__\n return self.main(*args, **kwargs)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 697, in main\n rv = self.invoke(ctx)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 1066, in invoke\n return _process_result(sub_ctx.command.invoke(sub_ctx))\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 895, in invoke\n return ctx.invoke(self.callback, **ctx.params)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/click/core.py\", line 535, in invoke\n return callback(*args, **kwargs)\nFile \"main.py\", line 27, in train\n pipeline_manager.train(pipeline_name, dev_mode)\nFile \"/home/ary_dhatt/osship/src/pipeline_manager.py\", line 28, in train\n train(pipeline_name, dev_mode)\nFile \"/home/ary_dhatt/osship/src/pipeline_manager.py\", line 77, in train\n pipeline.fit_transform(data)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/steppy/base.py\", line 323, in fit_transform\n step_output_data = self._cached_fit_transform(step_inputs)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/steppy/base.py\", line 443, in _cached_fit_transform\n step_output_data = self.transformer.fit_transform(**step_inputs)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/steppy/base.py\", line 605, in fit_transform\n self.fit(*args, **kwargs)\nFile \"/home/ary_dhatt/osship/src/models.py\", line 68, in fit\n for batch_id, data in enumerate(batch_gen):\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 267, in __next__\n return self._process_next_batch(batch)\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 301, in _process_next_batch\n raise batch.exc_type(batch.exc_msg)\nOSError: Traceback (most recent call last):\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in <listcomp>\n samples = collate_fn([dataset[i] for i in batch_indices])\nFile \"/home/ary_dhatt/osship/src/loaders.py\", line 127, in __getitem__\n Xi = load_func(self.X, index, filetype='png', grayscale=False)\nFile \"/home/ary_dhatt/osship/src/loaders.py\", line 159, in load_from_disk\n return self.load_image(img_filepath, grayscale=grayscale)\nFile \"/home/ary_dhatt/osship/src/loaders.py\", line 172, in load_image\n image = image.convert('RGB')\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/PIL/Image.py\", line 879, in convert\n self.load()\nFile \"/home/ary_dhatt/anaconda3/envs/shippy35/lib/python3.5/site-packages/PIL/ImageFile.py\", line 228, in load\n\"(%d bytes not processed)\" % len(b))\nOSError: image file is truncated (55 bytes not processed)\nI get this error a while after the program logs 2018-08-25 15-23-23 ships-detection >>> epoch 0 .... Do you have an idea of how I can fix this?
Ok, I found the fix. I just had to remove 6384c3e78.jpg from the the csv and the train directory.
", "votes": 1, "replies": [] }, { "id": 375716, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-08-25T21:29:59.333000", "content": "Hmm but it should be excluded as specified in the pipeline_config.py .
\n\nAre you working on the latest master branch?
", "votes": 0, "replies": [] }, { "id": 375723, "author_name": "Arpan Dhatt", "author_url": "", "post_date": "2018-08-25T21:57:37.590000", "content": "Yes, I cloned the repository to my GCP instance yesterday evening.
", "votes": 1, "replies": [] }, { "id": 375728, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-08-25T22:11:23.817000", "content": "Have you generated metadata.csv after that?
\n\nSorry for nagging but I am only asking to figure out what should be fixed.
", "votes": 0, "replies": [] }, { "id": 375733, "author_name": "Arpan Dhatt", "author_url": "", "post_date": "2018-08-25T22:20:45.227000", "content": "Yes, after I figured out what was wrong, I deleted the overlayed masks folder AND metadata.csv. Then I removed all references to the file from train_ship_segmentations.csv and the train folder
", "votes": 1, "replies": [] } ] }, { "id": 375589, "author_name": "Arpan Dhatt", "author_url": "", "post_date": "2018-08-25T15:16:32.213000", "content": "If I want to change the image size to 512x512 and change it in the config, do I have to run prepare masks again, or can I just start the training?
", "votes": 1, "replies": [ { "id": 375597, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-08-25T15:25:53.460000", "content": "Just run train with the modified config. Mind that with larger size you may want to run it on a smaller batch or multiple gpus.
", "votes": 0, "replies": [] }, { "id": 375619, "author_name": "Arpan Dhatt", "author_url": "", "post_date": "2018-08-25T16:25:26.343000", "content": "Okay, thanks for the quick reply!
", "votes": 1, "replies": [] } ] }, { "id": 368956, "author_name": "kamil", "author_url": "", "post_date": "2018-08-11T11:56:09.430000", "content": "Hello,
\n\nQuick info about our recent work and thoughts... :)
\n\nBest,
\n\nKamil
", "votes": 1, "replies": [ { "id": 369582, "author_name": "Ivan Achlaqullah", "author_url": "", "post_date": "2018-08-13T12:04:33.760000", "content": "Hi, I have some question since I'm quite new to image segmentation.
\n\nSince you use U-Net, I assume the output are single image mask, which then will be separated to it's own mask for each ship that are predicted. How do you that?
\n\nThank you.
", "votes": 1, "replies": [] }, { "id": 369600, "author_name": "dzem90", "author_url": "", "post_date": "2018-08-13T12:26:14.037000", "content": "Hi @ivanachlaqullah !
\n\nYes, you are right - output is a single mask, where 0 means background and 1 means ship. Later we apply ndi.label transformation (code). This function looks for all not connected instances of class 1 and puts unique label to each instance. Later we just run RLE on this labeled mask (code)
\n\nCheers and good luck in the competition!
\n\nAndrzej
", "votes": 1, "replies": [] }, { "id": 374856, "author_name": "Arpan Dhatt", "author_url": "", "post_date": "2018-08-24T01:57:51.017000", "content": "Can you update use weekly on different ideas that you found worked well or not? Also, your code is really well organized. And thanks for keeping it below bronze(otherwise I would have probably just used the code and never coded at all).
", "votes": 1, "replies": [] } ] }, { "id": 368411, "author_name": "kamil", "author_url": "", "post_date": "2018-08-09T20:43:59.717000", "content": "By the end of the week, I will add some info about techniques that worked well in Ships detection :)
\n\nAlso, I want to share what you can find in our starter code (of course, it is below bronze medal, as we discussed with the community :) ).
", "votes": 1, "replies": [] }, { "id": 371803, "author_name": "Varal7", "author_url": "", "post_date": "2018-08-17T16:22:03.907000", "content": "I still believe you it could be helpful to fix your metric :)
", "votes": 2, "replies": [ { "id": 371839, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-08-17T17:08:59.550000", "content": "Thank you @Varal7 for dropping that PR. I just wanted to think about it and was really swamped with other stuff.
\n\nI will look into that first thing next week.
\n\nThanks a lot for that fix!
", "votes": 2, "replies": [] } ] }, { "id": 413341, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-31T18:44:56.827000", "content": "I cannot find the neptune.yaml file in the repository
", "votes": 0, "replies": [] }, { "id": 417049, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-11-07T16:46:16.820000", "content": "Done @Chandan Verma.
\n\nSorry about that.
", "votes": 1, "replies": [] } ] }, { "id": 412206, "author_name": "phun", "author_url": "", "post_date": "2018-10-29T19:30:16.167000", "content": "Neptune.ml is pretty neat!
\n\nIt would be cool to see the confusion matrix or drill down into the false positives / negatives as part of the summary in an experiment.
\n\nBasically this:
\n\n\nEmpty | f2: 0.994 | gain: 0.003\nNon Empty f2: 0.422 | gain: 0.277\n1 ship f2: 0.445 | gain: 0.178\n2-5 ships f2: 0.393 | gain: 0.083\n5-10 ships f2: 0.308 | gain: 0.012\n10+ ships f2: 0.205 | gain: 0.005\n
As a configurable part of the summary of each experiment.
", "votes": 0, "replies": [ { "id": 412421, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-30T07:42:27.340000", "content": "Hi @phun, I am glad you like it.
\n\nWe are working on improving charts/image channel funcionality as we speak, but in the meantime \nyou can do that by sending the matplotlib chart to Neptune via image_channel.
\n\nYou need to convert it first to PIL by running something like this:
\n\nimport numpy as np\nfrom PIL import Image\n\ndef fig2pil(fig):\n fig.canvas.draw()\n\n w,h = fig.canvas.get_width_height()\n buf = np.fromstring(fig.canvas.tostring_argb(), dtype=np.uint8)\n buf.shape = (w, h, 4)\n buf = np.roll(buf, 3, axis=2)\n\n buf = fig2numpy(fig)\n w, h, d = buf.shape\n return Image.frombytes(\"RGBA\", (w , h), buf.tostring())\nand then you can send it to Neptune, even after every epoch:
\n\nimport neptune\nimport matplotlib.pyplot as plt\n\nctx = neptune.Context()\n\n for i in range(epoch_nr): \n ...\n fig = plt.figure()\n plot_confusion_matrix(results)\n\n pil_image = fig2pil(fig)\n ctx.channel_send('confusion_matrix', \n neptune.Image(name='cm iter {}'.format(i),\n description='Confusion matrix',\n data=pil_image))\nYour charts will be presented in the confusion_matrix channel, and will look similiar to this:
It pushed the score to CV 722 LB 703.
We have added:
\n\nWe have encountered heavy overfitting on our local validation. We are testing the idea from this kernel
\n\nThat's it
", "votes": 0, "replies": [] }, { "id": 406496, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-19T10:56:14.337000", "content": "Hi all,\nWe would like to announce that:
\n\nCV 0.694 LB 0.696f2 0.31 for ship masks.We are training in 2 stages:
\n\nAround 180 epochs with BCE + 0.25 * DICE and reduce on plateau callback. Check the experiment here. You can select your loss and weights in models.py
\n\n def set_loss(self):\n if self.activation_func == 'softmax':\n raise NotImplementedError('No softmax loss defined')\n elif self.activation_func == 'sigmoid':\n\n loss_function = weighted_sum_loss\n # loss_function = nn.BCEWithLogitsLoss()\n # loss_function = DiceWithLogitsLoss()\n # loss_function = lovasz_loss\n # loss_function = FocalWithLogitsLoss()\nYou can play with those values in the neptune.yaml:
postpro__drop_size: 50\n postpro__mid_min_size: 50\n postpro__mid_max_size: 1000\nBest\nKamil & Kuba
", "votes": 0, "replies": [] }, { "id": 404169, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-15T11:16:50.477000", "content": "We have just made the model weights for solution-1 and solution-2 available in here.
\n\n
Feel free to use them however you like.
", "votes": 0, "replies": [] }, { "id": 404121, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-10-15T09:13:35.460000", "content": "Hi all,\nWe would like to announce that:
\n\nCV 0.661 LB 0.679
INFERENCE_WITH_SHIP_NO_SHIP to True/FalseWe implemented encoder freezing and batchnorm freezing during training and an interface to play with that easily. You simply need to go to models.py and input your logic:
\n\n def freeze_weights(self):\n # freeze encoder\n if isinstance(self.model, nn.DataParallel):\n encoder_params = self.model.module.encoder.parameters()\n else:\n encoder_params = self.model.encoder.parameters()\n\n for parameter in encoder_params:\n parameter.requires_grad = False\n\n # freeze batchnorm\n for m in self.model.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n m.weight.requires_grad = False\n m.bias.requires_grad = False\n pass\nWe dropped mask to oriented bounding box in postprocessing. Basically the idea was to make all masks oriented rectangular objects since that is the target but for some reason it made results a bit worse. The function may come in handy later:
\n\n import numpy as np\n import cv2\n\ndef masks_to_bounding_boxes(labeled_mask):\n if labeled_mask.max() == 0:\n return labeled_mask\n else:\n img_box = np.zeros_like(labeled_mask)\n for label_id in range(1, labeled_mask.max() + 1, 1):\n label = np.where(labeled_mask == label_id, 1, 0).astype(np.uint8)\n _, cnt, _ = cv2.findContours(label, 1, 2)\n rect = cv2.minAreaRect(cnt[0])\n box = cv2.boxPoints(rect)\n box = np.int0(box)\n cv2.drawContours(img_box, [box], 0, label_id, -1)\n return img_box\nBest\nKamil & Kuba
", "votes": 0, "replies": [] }, { "id": 381979, "author_name": "quinwu", "author_url": "", "post_date": "2018-09-05T13:29:09.063000", "content": "\n\n\nTraceback (most recent call last):
\n \nFile \"main.py\", line 89, in
\n \nmain()
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 722, in call
\n \nreturn self.main(*args, **kwargs)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 697, in main
\n \nrv = self.invoke(ctx)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 1066, in invoke
\n \nreturn _process_result(sub_ctx.command.invoke(sub_ctx))
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 895, in invoke
\n \nreturn ctx.invoke(self.callback, **ctx.params)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/click/core.py\", line 535, in invoke
\n \nreturn callback(*args, **kwargs)
\n \nFile \"main.py\", line 27, in train
\n \npipeline_manager.train(pipeline_name, dev_mode)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/pipeline_manager.py\", line 28, in train
\n \ntrain(pipeline_name, dev_mode)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/pipeline_manager.py\", line 77, in train
\n \npipeline.fit_transform(data)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/steppy/base.py\", line 323, in fit_transform
\n \nstep_output_data = self._cached_fit_transform(step_inputs)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/steppy/base.py\", line 443, in _cached_fit_transform
\n \nstep_output_data = self.transformer.fit_transform(**step_inputs)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/steppy/base.py\", line 605, in fit_transform
\n \nself.fit(*args, **kwargs)\n File \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/models.py\", line 68, in fit
\n \nfor batch_id, data in enumerate(batch_gen):
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 281, in next
\n \nreturn self._process_next_batch(batch)
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 301, in _process_next_batch
\n \nraise batch.exc_type(batch.exc_msg)
\n \nFileNotFoundError: Traceback (most recent call last):
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in _worker_loop
\n \nsamples = collate_fn([dataset[i] for i in batch_indices])
\n \nFile \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/torch/utils/data/dataloader.py\", line 55, in
\n \nsamples = collate_fn([dataset[i] for i in batch_indices])
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 132, in getitem
\n \nMi = self.load_target(self.y, index, load_func)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 211, in load_target
\n \nMi = load_func(data_source, index, filetype='joblib')
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 167, in load_from_disk
\n \nreturn self.load_joblib(img_filepath)
\n \nFile \"/home/kwu/Project/kaggle/open-solution-ship-detection/src/loaders.py\", line 180, in load_joblib
\n \ntarget = joblib.load(img_filepath)\n File \"/home/kwu/anaconda3/envs/ship-pytorch/lib/python3.5/site-packages/sklearn/externals/joblib/numpy_pickle.py\", line 570, in load
\n \nwith open(filename, 'rb') as f:
\n \nFileNotFoundError: [Errno 2] No such file or directory: '/home/kwu/data/kaggle/AirbusShipDetection/masks/6384c3e78'
\n
when I use
\n\npython main.py -- prepare_masks
python main.py -- prepare_metadata
python main.py -- train --pipeline_name unet
I don't have find this /home/kwu/data/kaggle/AirbusShipDetection/masks/6384c3e78 in my path. I don't know how this tag came from.
Hi,
\n\nJust one quick question: did you configure all paths correctly in the neptune.yaml - especially path to masks? This might be the root cause of your error.
\n\nBest,
\n\nKamil
", "votes": 0, "replies": [] } ] }, { "id": 381196, "author_name": "Kim Jin", "author_url": "", "post_date": "2018-09-04T08:57:10.623000", "content": "In the segmentation file, a label of a picture is divided into a several parts , is it necessary to fuse them before train?
", "votes": 0, "replies": [ { "id": 381229, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-09-04T10:22:10.140000", "content": "Since we are training unets we do need to have one mask.\nHowever we could fuse them at train time. The problem with this (when training many epochs) is that it is quite slow and can be done beforehand to speed up training.
", "votes": 0, "replies": [] } ] }, { "id": 380244, "author_name": "Chandan Verma", "author_url": "", "post_date": "2018-09-02T06:03:35.520000", "content": "getting the following error:\nTypeError: init() got an unexpected keyword argument 'is_trainable'\nwhile running : python main.py -- evaluate_predict --pipeline_name unet
", "votes": 0, "replies": [ { "id": 380278, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-09-02T07:55:09.157000", "content": "Are you using the exact same requirements.txt ? What are your steppy and steppy-toolkit versions ?
", "votes": -1, "replies": [] }, { "id": 380293, "author_name": "Chandan Verma", "author_url": "", "post_date": "2018-09-02T08:48:52.043000", "content": "steppy==0.1.5\nsteppy-toolkit==0.1.8
\n\nif i use steppy 1.6 i get the following
\n\nFile \"E:\\toolkits.win\\anaconda3-5.2.0\\envs\\dlwin36\\lib\\site-packages\\steppy\\base.py\", line 477, in _cached_transform\n raise ValueError('No transformer cached {}'.format(self.name))\nValueError: No transformer cached unet
", "votes": 0, "replies": [] }, { "id": 380644, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-09-03T06:23:37.420000", "content": "Ok, I see.
\n\nI will try to reproduce and get back to you.
\n\nBTW, please drop problems with the code in the github repo issues section. I feel it's just a more natural way of talking about bugs.
", "votes": 0, "replies": [] } ] }, { "id": 371437, "author_name": "William Green", "author_url": "", "post_date": "2018-08-16T19:56:28.917000", "content": "Is this the correct reproduce_results.md file for open-solution-ship-detection?
", "votes": 0, "replies": [ { "id": 371584, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-08-17T06:57:21.450000", "content": "Hi @William Green.
\n\nThat file was pasted from another project by accident.
\n\nI will update it today (hopefully).
", "votes": 0, "replies": [] } ] }, { "id": 369994, "author_name": "Mingtao Sun", "author_url": "", "post_date": "2018-08-14T04:09:21.383000", "content": "HI, got this error on training. Could you please shed some lights? Thanks.
\n\nneptune run main.py -- train --pipeline_name unet
\n\n2018-08-14 14-00-49 ships-detection >>> epoch 0 ...\nTraceback (most recent call last):\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/deepsense/neptune/job_wrapper.py\", line 107, in \n execute()\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/deepsense/neptune/job_wrapper.py\", line 103, in execute\n execfile(job_filepath, job_globals)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/past/builtins/misc.py\", line 82, in execfile\n exec_(code, myglobals, mylocals)\n File \"main.py\", line 89, in \n main()\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 722, in call\n return self.main(*args, **kwargs)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 697, in main\n rv = self.invoke(ctx)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 1066, in invoke\n return _process_result(sub_ctx.command.invoke(sub_ctx))\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 895, in invoke\n return ctx.invoke(self.callback, **ctx.params)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/click/core.py\", line 535, in invoke\n return callback(*args, **kwargs)\n File \"main.py\", line 27, in train\n pipeline_manager.train(pipeline_name, dev_mode)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/pipeline_manager.py\", line 27, in train\n train(pipeline_name, dev_mode)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/pipeline_manager.py\", line 76, in train\n pipeline.fit_transform(data)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/steppy/base.py\", line 323, in fit_transform\n step_output_data = self._cached_fit_transform(step_inputs)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/steppy/base.py\", line 443, in _cached_fit_transform\n step_output_data = self.transformer.fit_transform(**step_inputs)\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/steppy/base.py\", line 605, in fit_transform\n self.fit(*args, **kwargs)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/models.py\", line 71, in fit\n self.callbacks.on_batch_end(metrics=metrics)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/callbacks.py\", line 118, in on_batch_end\n callback.on_batch_end(*args, **kwargs)\n File \"/Users/msun/Downloads/dev/open-solution-ship-detection/src/callbacks.py\", line 149, in on_batch_end\n loss = loss.data.cpu().numpy()[0]\nIndexError: too many indices for array\nCalculated experiment snapshot size: 0 Bytes
\nProcess exited with return code 1.
Have you installed pytorch from the requirements. It seems like it could be some sort of pytorch 0.3.1 vs 0.4 .
", "votes": 0, "replies": [] }, { "id": 370048, "author_name": "Mingtao Sun", "author_url": "", "post_date": "2018-08-14T06:25:50.820000", "content": "Indeed. my torch version is 0.4. Thanks.
", "votes": 0, "replies": [] } ] }, { "id": 369802, "author_name": "William Green", "author_url": "", "post_date": "2018-08-13T19:21:26.167000", "content": "I get a error when I run python main.py -- prepare_masks
neptune: Executing in Offline Mode.\nTraceback (most recent call last):\nFile \"main.py\", line 2, in <module>\nfrom src.pipeline_manager import PipelineManager\nFile \"/floyd/home/src/pipeline_manager.py\", line 8, in <module>\nfrom .pipelines import PIPELINES\nFile \"/floyd/home/src/pipelines.py\", line 7, in <module>\nfrom .models import PyTorchUNet\nFile \"/floyd/home/src/models.py\", line 8, in <module>\nfrom toolkit.pytorch_transformers.architectures.unet import UNet\n File \"/usr/local/lib/python3.6/site-packages/toolkit/pytorch_transformers/architectures/unet.py\", line 6, in <module>\nfrom toolkit.pytorch_transformers.utils import get_downsample_pad, get_upsample_pad\nImportError: cannot import name 'get_downsample_pad'\nHi @William Green,
\n\nMake sure to have the exact requirements.txt from the master branch.
I've updated that a moment ago and tested on a fresh environment.
\n\nI hope you will have no problem running it this time.
", "votes": 1, "replies": [] }, { "id": 370290, "author_name": "William Green", "author_url": "", "post_date": "2018-08-14T15:18:17.137000", "content": "Thanks you @Jakub.
\n\nWhen I run !python main.py -- prepare_metadata --train_data --valid_data --test_data
I get the following error:
\n\nneptune: Executing in Offline Mode.\nneptune: Executing in Offline Mode.\nError: no such option: --train_data\nAlso, if I just run:
\n\n!python main.py -- prepare_metadata\nI get the following error:
\n\nneptune: Executing in Offline Mode.\nneptune: Executing in Offline Mode.\n2018-08-14 14-56-59 ships-detection >>> creating metadata\n100%|██████████████████████████████████| 104070/104070 [07:11<00:00, 241.13it/s]\n100%|█████████████████████████████████| 88500/88500 [00:00<00:00, 237279.13it/s]\nTraceback (most recent call last):\nFile \"main.py\", line 89, in <module>\nmain()\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 722, in __call__\nreturn self.main(*args, **kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 697, in main\nrv = self.invoke(ctx)\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 1066, in invoke\nreturn _process_result(sub_ctx.command.invoke(sub_ctx))\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 895, in invoke\nreturn ctx.invoke(self.callback, **ctx.params)\nFile \"/usr/local/lib/python3.6/site-packages/click/core.py\", line 535, in invoke\nreturn callback(*args, **kwargs)\nFile \"main.py\", line 20, in prepare_metadata\npipeline_manager.prepare_metadata()\nFile \"/floyd/home/src/pipeline_manager.py\", line 25, in prepare_metadata\nprepare_metadata()\nFile \"/floyd/home/src/pipeline_manager.py\", line 52, in prepare_metadata\nmeta.to_csv(os.path.join(PARAMS.meta_dir, 'metadata.csv'), index=None)\nFile \"/usr/local/lib/python3.6/site-packages/pandas/core/frame.py\", line 1524, in to_csv\nformatter.save()\nFile \"/usr/local/lib/python3.6/site-packages/pandas/io/formats/format.py\", line 1637, in save\ncompression=self.compression)\nFile \"/usr/local/lib/python3.6/site-packages/pandas/io/common.py\", line 390, in _get_handle\nf = open(path_or_buf, mode, encoding=encoding)\nFileNotFoundError: [Errno 2] No such file or directory: 'meta/metadata.csv'\nIt seems not be creating the metadata folder, so it can save the file.
", "votes": 0, "replies": [] }, { "id": 370364, "author_name": "Jakub Czakon", "author_url": "", "post_date": "2018-08-14T17:35:43.610000", "content": "I guess you need to create that folder. I usually create folders data, files experiments for every project so I never run into this problem.
what about ?
\n\n!python main.py -- prepare_metadata --train_data --valid_data --test_data
I get the following error:
\n\nneptune: Executing in Offline Mode.\nneptune: Executing in Offline Mode.\nError: no such option: --train_data\nI get the a error when I try to run python main.py -- prepare_masks :
neptune: Executing in Offline Mode.\nTraceback (most recent call last):\nFile \"main.py\", line 2, in <module>\nfrom src.pipeline_manager import PipelineManager\nFile \"/floyd/home/src/pipeline_manager.py\", line 6, in <module>\nfrom .metrics import f_beta_metric\nFile \"/floyd/home/src/metrics.py\", line 8, in <module>\nfrom .pipeline_config import ORIGINAL_SIZE\nFile \"/floyd/home/src/pipeline_config.py\", line 106, in <module>\n'annotation_file': PARAMS.annotation_file,\nFile \"/usr/local/lib/python3.6/site-packages/attrdict/mixins.py\", line 82, in __getattr__\ncls=self.__class__.__name__, name=key\nAttributeError: 'AttrDict' instance has no attribute 'annotation_file'\nSorry about that @William Green .
\n\nneptune.yaml was missing params. It is fixed already.
@Jakub, thank you
", "votes": 0, "replies": [] } ] }, { "id": 369457, "author_name": "abnerzhang", "author_url": "", "post_date": "2018-08-13T04:58:59.457000", "content": "Hi @abnerzhang. Could you elaborate?
", "votes": -1, "replies": [] }, { "id": 369534, "author_name": "kamil", "author_url": "", "post_date": "2018-08-13T09:47:46.083000", "content": "Hi @abnerzhang,
\n\nNot at this point. However, for sure we need to treat it somehow...
\n\nBest,
\n\nKamil
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "368405": "Hi,\n\nWe would like to start sharing our results in this competition.\n\n### The Open Solution approach\nIt means that we are going to open:\n\n1. the [code on GitHub](https://github.com/neptune-ml/open-solution-ship-detection) - *(no worries competitive guys -> we only publish code that scores below bronze medal)*\n2. our [experiments results](https://app.neptune.ml/neptune-ml/Ships)\n3. our approach, that is what we have tried, what worked well, etc.\n\n### Goals\nThese are pretty straightforward (and similar to other competitions):\n\n1. **Learning from the process**.\n2. Encourage more Kagglers to start working on this competition.\n3. Share open source solution with no strings attached, so that less experienced Kagglers can join competition.\n\n### What can you find here?\nWe want this topic to be our *journal* or *project diary*, where we discuss our approach , techniques used, network architectures and all other deep learning related stuff! We want this place to be good address for people who want to share knowledge or gain knowledge :)\n\nHappy Training!\n\nKamil & Kuba\n", "411567": "# Solution 5 (not open yet)\n\nIt pushed the score to `CV 719 LB 725`.\n\nFine grained look at the local validation:\n\n Empty | f2: 0.994 | gain: 0.003\n Non Empty f2: 0.422 | gain: 0.277\n 1 ship f2: 0.445 | gain: 0.178\n 2-5 ships f2: 0.393 | gain: 0.083\n 5-10 ships f2: 0.308 | gain: 0.012\n 10+ ships f2: 0.205 | gain: 0.005\n\nWe have added:\n\n- fixed local validation based on [this post](https://www.kaggle.com/c/airbus-ship-detection/discussion/69322)\n- training in stages:\n - 256x256 some 100 epochs with weighted loss BCE + 0.25 DICE [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/d525f719-ead5-44a2-a59a-558ffbde73d6)\n - 256x256 some 100 epochs with lovasz hinge [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/554c52ad-742d-4aac-adeb-1a3d1da13e61)\n - 512x512 another 100-150 epochs with `focal(alpha=1.0, gamma=2.0)` and lovasz hinge [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/95b36a94-a80f-4020-9fc9-eecf3b5a4d7d)\n - 768x768 another 100 epochs with focal + lovasz hinge [neptune experiment](https://app.neptune.ml/-/dashboard/experiment/66f015b8-5145-48d9-a71a-83dfe52459f4) \n\n- added intensity based test time augmentation:\n\n iaa.ContrastNormalization((0.75, 1.25))\n\n Since we are doing flips (x2x2) and rotations (x4) and random contrast (x8) the inference is very time-consuming. On the flip side, it pushes the score by quite a lot `CV +0.09 LB +0.09` so I guess it's here to stay.\n\nRight now we are working on tweaking the loss function to improve the results on the small-one ship images.", "402397": "Hi all,\nWe would like to announce that:\n\n# Solution 1 is now open!\n\n## Basic information\n- It should get you around `CV 0.541` `LB 0.573`\n- It contains all the boilerplate to get you started quickly\n- All the experiments can be found [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=1bc4da1e-6e47-4a26-a50e-3e55cbc052a7)\n- How-to instructions can be found either in the [neptune project](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=about)\n- If you have any questions regarding the solution please drop a comment either in this post or in [project discussion](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=discussion) (likely faster response)\n\n## What we have developed so far\n\n### Training and Validation Scheme\n- Model is evaluated on the same distribution as test set (0.52 of empty images).\n- You can select the size of the validation set and the size of the in-train validation set that is used in callbacks at the end of each epoch\n- We created a [sampler](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/loaders.py#L48-L79) that selects images from the train set with a specified fraction of empty images. By default we are training on non-empty images and so the `empty_fraction` is set to be 0.0. You can tweak it however you like.\n\nYou can play around with training/validation parameters by changing stuff in `neptune.yaml`:\n\n training_sampler_size: 2000\n training_sampler_empty_fraction: 0.0\n evaluation_size: 10000\n evaluation_empty_fraction: 0.52\n in_train_evaluation_size: 1000\n\n### Architectures\n- [Unet](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/unet.py) with a ton of encoder options \n - Resnet 34/50/101/152\n - SERresnet 50/101/152\n - SEResnetXT 50/101\n - Densenet 121/161/169/201\n[PSPNet](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/pspnet.py) with encoder of choice. You can read more about it in this [paper](https://arxiv.org/pdf/1612.01105.pdf) with encoder of choice\n- LargeKernelMatters](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/large_kernel_matters.py) with Resnet 34/50/101/152 encoder. You can read more about it in this architecture in this [paper](https://arxiv.org/pdf/1703.02719.pdf) \n- Decoders are equipped with both [channel and spatial squeeze and excitation blocks](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/architectures/base.py#L65-L117). You can read about them in this [paper](https://arxiv.org/pdf/1808.08127.pdf)\n\nIn order to choose an architecture you need to specify it in the `neptune.yaml`:\n\n architecture: UNetSeResNetXt\n\n### Losses\n- Lovash loss which took Salt Identification by storm. It is a surrogate loss of IOU and you should take a look at the [original paper](https://arxiv.org/pdf/1512.07797.pdf).\n- Focal Loss\n- Dice Loss\n- BCE\nYou can easily choose/change losses [here](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py#L161-L218) by uncommenting:\n loss_function = lovasz_loss\n # loss_function = DiceLoss()\n # loss_function = FocalWithLogitsLoss()\n # loss_function = nn.BCEWithLogitsLoss()\n\nYou can also combine the losses however you like, so do experiment with those.\n\n### Callbacks\n- We created callbacks that calculate both validation loss and the competition metric at the end of each epoch\n- We added Reduce on plateau callback that automatically reduces LR whenever your model is not improving for a while. You can set the params for it in `neptune.yaml`:\n\n lr: 0.0007\n momentum: 0.9\n gamma: 0.95\n patience: 10\n validation_metric_name: 'f2'\n minimize_validation_metric: 0\n reduce_factor: 0.5\n reduce_patience: 5\n min_lr: 0\n\n- We added neptune image channel that visualizes some validation predictions\n\n\n\n- We created Initial learning rate finder that will help you choose your… initial lr :)\nTo do that you need to uncomment it in the [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py#L232-L245) :\n\n def callbacks_network(callbacks_config):\n experiment_timing = cbk.ExperimentTiming(**callbacks_config['experiment_timing'])\n model_checkpoints = cbk.ModelCheckpoint(**callbacks_config['model_checkpoint'])\n lr_scheduler = cbk.ReduceLROnPlateauScheduler(**callbacks_config['reduce_lr_on_plateau_scheduler'])\n training_monitor = cbk.TrainingMonitor(**callbacks_config['training_monitor'])\n validation_monitor = cbk.ValidationMonitor(**callbacks_config['validation_monitor'])\n neptune_monitor = cbk.NeptuneMonitor(**callbacks_config['neptune_monitor'])\n early_stopping = cbk.EarlyStopping(**callbacks_config['early_stopping'])\n init_lr_finder = cbk.InitialLearningRateFinder()\n return cbk.CallbackList(\n callbacks=[experiment_timing, training_monitor, validation_monitor,\n model_checkpoints, lr_scheduler, neptune_monitor, early_stopping,\n # init_lr_finder\n ])\n\nThen, based on the charts like this\n\n\n\nYou can select the learning rate that will bring the fastest returns as explained in this [post](https://www.jeremyjordan.me/nn-learning-rate/)\n\n## Misc\n- We have implemented the [competition metric](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/metrics.py#L80-L87) for you to use it however you like\n- Since the dataset, and images are large we have developed evaluation/prediction in chunks so that you can easily work. - Just choose a chunk size that fits in memory in `main.py`\n- Test-time augmentation with flips (up-down, left-right) and rotations (0,90,180,270) are implemented and can be used by changing `USE_TTA` to `True` in `main.py`\n\n \nBest\nKamil & Kuba", "375612": " Traceback (most recent call last):\n File \"main.py\", line 89, in ", "375589": "If I want to change the image size to 512x512 and change it in the config, do I have to run prepare masks again, or can I just start the training?", "368956": "Hello,\n\nQuick info about our recent work and thoughts... :)\n\n1. U-Net -> this architecture is excellent for such competitions. It proved to work well, for example in DSB'18, where [winning solution](https://www.kaggle.com/c/data-science-bowl-2018/discussion/54741) was based on it. U-Net gives you possibility to apply modifications to the architecture, both simple like depth or convolutional block thickness, as well as more sophisticated ones such as custom encoders.\n1. We stick to PyTorch :)\n1. We know that [84% of test images are empty](https://www.kaggle.com/c/airbus-ship-detection/discussion/62376) -> zero ships. In this context, it is recommended to try to balance the signal via sampling (give more positive signal during training).\n1. Next, we will try to use information about the [corrupted images](https://www.kaggle.com/c/airbus-ship-detection/discussion/62921) as reported by @abnerzhang -> thanks!\n1. Also, we will experiment with loss function.\n\nBest,\n\nKamil\n", "368411": "By the end of the week, I will add some info about techniques that worked well in Ships detection :)\n\nAlso, I want to share what you can find in our starter code (of course, it is below bronze medal, as we discussed with the community :) ).\n", "371803": "I still believe you it could be helpful to [fix your metric](https://github.com/neptune-ml/open-solution-ship-detection/pull/2#issuecomment-413595209) :)", "413341": "# Solution 4 is now open!\n\n- All the experiments can be found [here](https://app.neptune.ml/neptune-ml/Ships/experiments/e43c10b9-6a3d-4f0b-80e7-8d74eb86ff62)\n- How-to instructions can be found either in the [neptune project](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=about) or in our [project repo](https://github.com/neptune-ml/open-solution-ship-detection)\n- If you have any questions regarding the solution please drop a comment either in this post or in [project discussion](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=discussion) (likely faster response)", "412206": "Neptune.ml is pretty neat!\n\nIt would be cool to see the confusion matrix or drill down into the false positives / negatives as part of the summary in an experiment.\n\nBasically this:\n\n```\nEmpty | f2: 0.994 | gain: 0.003\nNon Empty f2: 0.422 | gain: 0.277\n1 ship f2: 0.445 | gain: 0.178\n2-5 ships f2: 0.393 | gain: 0.083\n5-10 ships f2: 0.308 | gain: 0.012\n10+ ships f2: 0.205 | gain: 0.005\n```\n\nAs a configurable part of the summary of each experiment.", "407773": "# Solution 4 (not open yet)\n\nIt pushed the score to `CV 722 LB 703`.\n\nWe have added:\n\n- cyclic learning rates\n- squeeze and excitation (spatial and channel wise) to the deconv layer of large kernel matters.\n\nWe have encountered heavy overfitting on our local validation. We are testing the idea from this [kernel](https://www.kaggle.com/manuscrits/create-a-validation-dataset-correcting-the-leak)\n\nThat's it", "406496": "Hi all,\nWe would like to announce that:\n\n# Solution 3 is now open!\n\n## Basic information\n- It should get you around `CV 0.694` `LB 0.696`\n- All the experiments can be found [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=be842434-7c8b-4ab9-afa5-f9c00816d3c3)\n- How-to instructions can be found either in the [neptune project](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=about)\n- If you have any questions regarding the solution please drop a comment either in this post or in [project discussion](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=discussion) (likely faster response)\n\n\n## What have we improved\n\n### Architectures\n- We experimented with different flavours and what works best is **Large Kernel Matters** with ** Densenet 201** encoder . That gets `f2 0.31` for ship masks.\n- We also chose **Densenet 201** for the ship/no ship model. Gets 0.98+ accuracy and f2 0.996 for no ship images.\n\n### Training\n- We are sampling 7500 images every epoch\n- We are training in 2 stages:\n - Around 180 epochs with BCE + 0.25 * DICE and reduce on plateau callback. Check the [experiment here](https://app.neptune.ml/-/dashboard/experiment/6e866735-db5b-4b8c-ad79-a71e1224377c). You can select your loss and weights in [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py)\n\n def set_loss(self):\n if self.activation_func == 'softmax':\n raise NotImplementedError('No softmax loss defined')\n elif self.activation_func == 'sigmoid':\n\n loss_function = weighted_sum_loss\n # loss_function = nn.BCEWithLogitsLoss()\n # loss_function = DiceWithLogitsLoss()\n # loss_function = lovasz_loss\n # loss_function = FocalWithLogitsLoss()\n\n - Train for another 130 epoch with Lovash loss. Check the [experiment here](https://app.neptune.ml/-/dashboard/experiment/b77a708e-3410-4932-8550-e61ca72c33b8). Again you can select your loss in [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py)\n\n### Post-processing\n- We are dropping predicted object masks if they are smaller than 50 pixels\n- For objects between 50 and 1000 pixels, we apply the mask to bbox function which draws the minimal rectangle over those objects\n\nYou can play with those values in the `neptune.yaml`:\n\n postpro__drop_size: 50\n postpro__mid_min_size: 50\n postpro__mid_max_size: 1000\n\n### Misc\n- Parallel apply of mask resize and other post-processing functions speed up inference by a lot (~10x).\n- Added prediction [exploration notebook](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/prediction_exploration.ipynb) where you can inspect your model in more detail and figure out where it is not doing so well.\n \nBest\nKamil & Kuba\n", "404169": "We have just made the model weights for solution-1 and solution-2 available in [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=browseFiles).\n\n[](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=browseFiles)\n\nFeel free to use them however you like.", "404121": "Hi all,\nWe would like to announce that:\n\n# Solution 2 is now open!\n\n## Basic information\n- It should get you around `CV 0.661` `LB 0.679`\n- All the experiments can be found [here](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=8ad61fcb-f0ac-4aaf-aa9c-9db47e0aa222)\n- How-to instructions can be found either in the [neptune project](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=about)\n- If you have any questions regarding the solution please drop a comment either in this post or in [project discussion](https://app.neptune.ml/neptune-ml/Ships?namedFilterId=discussion) (likely faster response)\n\n## Some example outputs from the best model\n\n[](https://app.neptune.ml/-/dashboard/experiment/97f7385e-a6cd-4302-bc35-2863f364e9ed)\n\n[](https://app.neptune.ml/-/dashboard/experiment/97f7385e-a6cd-4302-bc35-2863f364e9ed)\n\n[](https://app.neptune.ml/-/dashboard/experiment/97f7385e-a6cd-4302-bc35-2863f364e9ed)\n\n\n## What have we improved\n\n### Architectures\n- We decoupled Encoders from Architectures so now you can combine Resnet/SeResNet/SeResNeXt/DenseNet with Unet/LargeKernelMatters/PSPNet however you like\nThe best model so far is actually LargeKernelMatters with SeResNeXT encoder. What is important is that this model is less resource heavy. I can train on 1 GPU with 16 image batch\n\n\n### Misc\n- We implemented two stage ship/no_ship + segmentation pipeline. You train both binary and segmentation model separately and then you can run inference with one or two stage model. Simply change global setup in the [main.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/main.py) by changing the `INFERENCE_WITH_SHIP_NO_SHIP` to True/False\n- We implemented encoder freezing and batchnorm freezing during training and an interface to play with that easily. You simply need to go to [models.py](https://github.com/neptune-ml/open-solution-ship-detection/blob/master/common_blocks/models.py#L194-L210) and input your logic:\n\n def freeze_weights(self):\n # freeze encoder\n if isinstance(self.model, nn.DataParallel):\n encoder_params = self.model.module.encoder.parameters()\n else:\n encoder_params = self.model.encoder.parameters()\n \n for parameter in encoder_params:\n parameter.requires_grad = False\n \n # freeze batchnorm\n for m in self.model.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n m.weight.requires_grad = False\n m.bias.requires_grad = False\n pass\n\n- We dropped mask to oriented bounding box in postprocessing. Basically the idea was to make all masks oriented rectangular objects since that is the target but for some reason it made results a bit worse. The function may come in handy later:\n\n import numpy as np\n import cv2\n\n def masks_to_bounding_boxes(labeled_mask):\n if labeled_mask.max() == 0:\n return labeled_mask\n else:\n img_box = np.zeros_like(labeled_mask)\n for label_id in range(1, labeled_mask.max() + 1, 1):\n label = np.where(labeled_mask == label_id, 1, 0).astype(np.uint8)\n _, cnt, _ = cv2.findContours(label, 1, 2)\n rect = cv2.minAreaRect(cnt[0])\n box = cv2.boxPoints(rect)\n box = np.int0(box)\n cv2.drawContours(img_box, [box], 0, label_id, -1)\n return img_box\n \nBest\nKamil & Kuba", "381979": "> Traceback (most recent call last):\n> \n> File \"main.py\", line 89, in ", "381196": "In the segmentation file, a label of a picture is divided into a several parts , is it necessary to fuse them before train?", "380244": "getting the following error:\nTypeError: __init__() got an unexpected keyword argument 'is_trainable'\nwhile running : python main.py -- evaluate_predict --pipeline_name unet", "371437": "Is this the correct [reproduce_results.md][1] file for open-solution-ship-detection? \n\n\n [1]: https://github.com/neptune-ml/open-solution-ship-detection/blob/master/REPRODUCE_RESULTS.md", "369994": "HI, got this error on training. Could you please shed some lights? Thanks. \n\nneptune run main.py -- train --pipeline_name unet\n\n2018-08-14 14-00-49 ships-detection >>> epoch 0 ...\nTraceback (most recent call last):\n File \"/Users/msun/anaconda3/lib/python3.6/site-packages/deepsense/neptune/job_wrapper.py\", line 107, in ", "369802": "I get a error when I run `python main.py -- prepare_masks`\n\n neptune: Executing in Offline Mode.\n Traceback (most recent call last):\n File \"main.py\", line 2, in ", "369653": "I get the a error when I try to run `python main.py -- prepare_masks` :\n\n neptune: Executing in Offline Mode.\n Traceback (most recent call last):\n File \"main.py\", line 2, in ", "369457": "## Hi, is there some difference between normal images and corrupted images? ##" } }