{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 4. Exploring public models\n### Airbus Ship Detection Challenge - A quick overview for computer vision noobs\n\n \n\n\nHi, and welcome! This is the fourth kernel of the series `Airbus Ship Detection Challenge - A quick overview for computer vision noobs.` This short kernel has two goals: first, to do a quick research of the currently trending public models and, second, to analyze the main approaches those models take.\n\n\nThe full series consist of the following notebooks:\n1. [Loading and visualizing the images](https://www.kaggle.com/julian3833/1-loading-and-visualizing-the-images)\n2. [Understanding and plotting rle bounding boxes](https://www.kaggle.com/julian3833/2-understanding-and-plotting-rle-bounding-boxes) \n3. [Basic exploratory analysis](https://www.kaggle.com/julian3833/3-basic-exploratory-analysis)\n4. *[Exploring public models](https://www.kaggle.com/julian3833/4-exploring-models-shared-by-the-community)*\n5. [1.0 submission: submitting the test file](https://www.kaggle.com/julian3833/5-1-0-submission-submitting-the-test-file)\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true}},{"cell_type":"markdown","source":"## 1. Searching for public models","metadata":{"_uuid":"68eb3cbdc800013def2f45f4d24c698a09cf5818"}},{"cell_type":"markdown","source":"We can access the most relevant models sorting by best score in the [Kernel's tab](https://www.kaggle.com/c/airbus-ship-detection/kernels?sortBy=scoreDescending&group=everyone&pageSize=20&competitionId=9988) of the competition.\n\n\n\nNote that some kernels with models don't have a submission associated with them and do not appear in the top of the list. For example, the one below reports a 0.89 on it's title, higher than the top kernel in the above list. Be aware of this, you may find that the hottests models don't have a submission associated. Typically, the title will suggest there is a model in that kernel.\n\n\n\nOn the other hand, the models are not independent: most of them are forks or improvements of previously shared models. \n\nIn the rest of this notebook we will review the approaches, models and technologies proposed in some of the currently trending kernels, as well as their evolution trees.","metadata":{"_uuid":"217350aaff553685b1e385b75b2f22e5b85329f5"}},{"cell_type":"markdown","source":"## 2. Currently trending models","metadata":{"_uuid":"085dd8f72cd885914d0d015ee6f15d9fd08e69e1"}},{"cell_type":"markdown","source":"While reading these kernels, as we expected, we found a lot of keywords and concepts which we don't know. We took notes of the most relevant keywords for further research on the relevant topics of the Airbus Challenge. We plan to go deeper in the following topics in future notebooks: `Image segmentation`, `Pytorch`, `U-net`. ","metadata":{"_uuid":"359f21a08adfa995e3f0907b4a5c44d95c1ab0c9"}},{"cell_type":"markdown","source":"### 2.1. **0.89** - ResNet34 + U-net model by [iafoss](https://www.kaggle.com/iafoss)\n\n \n\nThe most relevant public model at stake right now is the ResNet34 + U-net ensemble proposed by the user [iafoss](https://www.kaggle.com/iafoss) in his series of three kernels. All this work is based on the `fastai` package which, in turn, is fully based on `pytorch`. \n\n\nGive a quick look to the following table with a summary of the notebooks; we will go into some extra details right below.\n\n\n| Kernel | Creation date | Description | Keywords |\n| ----- |:----|: ---|: ---|\n| [Fine-tuning ResNet34 on ship detection](https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook) | August 26th | `Creates a ship detection (classification) model using transfer learning and ResNet34.` | `ResNet34, object detection, learning rate annealing, class imbalance, different input sizes for same network`|\n| [U-net34 (dice 0.87+)](https://www.kaggle.com/iafoss/unet34-dice-0-87/notebook) | September 1st | `Builds a U-net for image segmentation inspired by a fast.ai lesson using the ship detection ResNet34 from previous notebook for the encoder part. Dice loss and Focal loss are presented and used too.` | `SSD, rotating bounding boxes, U-net, image segmentation, Carvana Challenge, dice loss, focal loss` |\n|[U-net34 submission (0.89 public LB)](https://www.kaggle.com/iafoss/unet34-submission-0-89-public-lb) | September 4th | `Ensembles the ship/no-ship ResNet34 classifier from the first notebook with the U-net image segmentation from the second one.` | `Ensemble`\n\n.\n\n\nIn the [first kernel, `Fine-tuning ResNet34 on ship detection`](https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook), the author trains a model for object detection retraining just some layers of a pretrained ResNet34. This is a ship detection model (this is, given a picture, it determines whether there is a ship in it or not). The reported precision for the detection task is of ~98%. On the other hand, iafoss does a very focused training on some specific layers using learning rate annealing over images of different resolutions using the same network. Note that this model doesn't fully address the Challenge's problem, which is `image segmentation`.\n\n \n\nThe [second kernel, `U-net34 (dice 0.87+)`](https://www.kaggle.com/iafoss/unet34-dice-0-87/notebook), addresses the problem of `image segmentation` itself. Image segmentation is the most evolved of a series of related deep learning / computer vision tasks as the picture below shows and we will cover in a *future notebook*. \n\n\n\nThe author mentions 2 possible approaches - `U-net` and `SSD` - and explains pros and cons of each of them. After that, he proposes a customized `U-net` model, importing the `ResNet34` of the previous kernel as the encoder part. This model is trained very selectively only over images with at least one ship, dropping almost %80 of the dataset, based on the class imbalance observation which is discussed in [this previous notebook](https://www.kaggle.com/julian3833/3-basic-exploratory-analysis) of the series. \n\n`U-net`, as we will see in *another future notebook*, is a fully convolutional network with a symmetric encoding-decoding architecture which works pretty well for `image segmentation`:\n\n\n\n\nFinally, the [third kernel, `U-net34 submission (0.89 public LB)`](https://www.kaggle.com/iafoss/unet34-submission-0-89-public-lb) stacks these two models achieving a 0.889 score. So, first, the classifier determines whether a picture has ships or not and, if the picture has ships, the U-net determines where. \n\nThere are few other kernels which are forks of some kernel of these series. In fact, the current top-ranked kernel is just a fork of this work without any addition.\n \n ### 2.2. **0.847** - Baseline trivial empty submission\n \n  \n \n [Early](https://www.kaggle.com/c/airbus-ship-detection/discussion/62376) on the competition it was publicly known that an empty submission reported an 84.7 score, because of the class imbalance which was discussed on [this kernel](https://www.kaggle.com/julian3833/3-basic-exploratory-analysis). This model can be considered a baseline although 25% of the competitors scored lower than the one it achieves. The empty submission can be constructed with literally one line of code:\n\nThe kernels [One line Base Model LB:0.847](https://www.kaggle.com/paulorzp/one-line-base-model-lb-0-847) and [Naive Model](https://www.kaggle.com/npatta01/naive-model) contain this solution, this code is taken from the first one:\n\n","metadata":{"_uuid":"717ec3c47b51fed833015aa2b44ba00601839a86"}},{"cell_type":"code","source":"#https://www.kaggle.com/paulorzp/one-line-base-model-lb-0-847\nimport os; \nimport pandas as pd\n\npd.read_csv('../input/sample_submission_v2.csv', \n converters = {'EncodedPixels': lambda p: None}).to_csv('submission_paulorzp.csv', index=False)","metadata":{"_uuid":"2ad30f0ef1e8c5af845b569ff7478b08be98ec92","execution":{"iopub.status.busy":"2021-08-14T16:34:56.905940Z","iopub.execute_input":"2021-08-14T16:34:56.906299Z","iopub.status.idle":"2021-08-14T16:34:56.991518Z","shell.execute_reply.started":"2021-08-14T16:34:56.906245Z","shell.execute_reply":"2021-08-14T16:34:56.990527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.3. **1.0** - trivial submission: submitting the test file\n\n \n\nAs we will discuss in a *future notebook* of the series, the competition had a [data leakage](https://www.kaggle.com/c/airbus-ship-detection/discussion/64355) (reported on August 28th), trivializing the full competition. The organizers [took action](https://www.kaggle.com/c/airbus-ship-detection/discussion/64388) about this and will change the evaluation set. Meanwhile, they shared the segmentations for the test set. We wrote a [7-lines notebook](https://www.kaggle.com/julian3833/5-1-0-submission-submitting-the-test-file) which creates a submission with this data, reaching a 1.0 score on the leaderboard (until the new test set is released and the scores are resetted). \n\n\n","metadata":{"_uuid":"636e77940881b39efa23bcedf8c48b945affa91b"}},{"cell_type":"markdown","source":"### 2.4. More public models\n \n\nThe most relevant of the rest of the kernels are strongly based on [kmader](https://www.kaggle.com/kmader/)'s early work. As iafoss, he has built a ship detector using transfer learning and a U-net for image segmentation. kmader didn't publish an ensemble of these two models as iafoss did, but [hmendonca](https://www.kaggle.com/hmendonca) did, obtaining a 85.3 score. \n\nReturning to kmader's work, the ship detector can be found in the `Transfer Learning for Boat or No-Boat` kernel. It's a `DenseNet169` pretrained net with few extra trained layers around it. It's implemented with `Keras`, and the kernel uses `skimage` and `keras' data augumentation mechanisms`. On the other hand, the U-net image segmentation model presented in [Baseline U-net model](https://www.kaggle.com/kmader/baseline-u-net-model-part-1/notebook) is a hand-crafted implementation of a basic `U-net` model, also in `Keras`. This model is splitted in two kernels in fact and, as far as I could understand, it predicts no-ship for every pixels (generating the 0.847 score, as the trivial baseline). \n\n\n| Kernel | Description | Keywords |\n| ----- |:----- |: ---|\n| [Baseline U-net model - Part 1](https://www.kaggle.com/kmader/baseline-u-net-model-part-1/notebook) and [Part 2](https://www.kaggle.com/kmader/from-trained-u-net-to-submission-part-2) | `U-net model implemented with Keras, splitted in 2 notebooks. As far as I understand, the model outputs the empty submission.` | `U-net`, `Keras`\n| [Transfer Learning for Boat or No-Boat](https://www.kaggle.com/kmader/transfer-learning-for-boat-or-no-boat) | `Transfer learning for ship/no-ship classification`| `DenseNet169`, `ImageDataGenerator`, `Keras' callbacks`\n| [Classification and Segmentation (-FP)](https://www.kaggle.com/hmendonca/classification-and-segmentation-fp) | `An ensemble of the above transfer leaning ship/no-ship classifier and the tweaked U-net model` |\n\n \n\n\n ","metadata":{"_uuid":"1d9c6e2f6eb117b92e734ed32d8fae71c1dc3232"}},{"cell_type":"markdown","source":"## 3. Conclusion and next steps","metadata":{"_uuid":"8a31fd8cff06234ec9fbd689ff4dbe2f3ec59548"}},{"cell_type":"markdown","source":"### Conclusion\n\nAs we can see, the general approach of a `ship detection + image segmentation ensemble` is the most prominent in the public models right now, one using `pytorch` and one using `keras`. This is a common way for approaching the strong imbalance with respect to the pixel labels. The image segmentation model is trained dropping all the images without ships by one author and dropping most of those images by the other. Also, in both cases, the ship detection task is addressed with transfer learning while the image segmentation task is addressed with a `U-net` model. Currently, iafoss' is working on an `SSD`, a more complex but also more promising model, at least from his point of view.\n\n\n### References\n#### Kernels\n* [Fine-tuning ResNet34 on ship detection](https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook)\n* [U-net34 (dice 0.87+)](https://www.kaggle.com/iafoss/unet34-dice-0-87/notebook) \n* [U-net34 submission (0.89 public LB)](https://www.kaggle.com/iafoss/unet34-submission-0-89-public-lb)\n* [Baseline U-net model - Part 1](https://www.kaggle.com/kmader/baseline-u-net-model-part-1/notebook) and [Part 2](https://www.kaggle.com/kmader/from-trained-u-net-to-submission-part-2) \n* [Transfer Learning for Boat or No-Boat](https://www.kaggle.com/kmader/transfer-learning-for-boat-or-no-boat)\n* [Classification and Segmentation (-FP)](https://www.kaggle.com/hmendonca/classification-and-segmentation-fp) \n* [One line Base Model LB:0.847](https://www.kaggle.com/paulorzp/one-line-base-model-lb-0-847) \n* [Naive Model](https://www.kaggle.com/npatta01/naive-model)\n\n\n#### Other resources\n* [Deeplearning.net's U-net explanation](http://deeplearning.net/tutorial/unet.html)\n* [Deeplearning.net's FCN for 2D segmentation explanation](http://deeplearning.net/tutorial/fcn_2D_segm.html)\n* [A 2017 Guide to Semantic Segmentation with Deep Learning](http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review#dilation)\n* [How to do Semantic Segmentation using Deep learning](https://medium.com/nanonets/how-to-do-image-segmentation-using-deep-learning-c673cc5862ef)\n* [Deep Learning book](https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/image_segmentation.html)\n\n### What's next?\nYou can check the [next kernel](https://www.kaggle.com/julian3833/5-1-0-submission-submitting-the-test-file) of the series, where we create a trivial 1.0 scoring submission using the test segmentation csv shared by the organizers.\n","metadata":{"_uuid":"a9a678acb9e6869c200b24cd2e7549d619702275"}}]}