{"cells":[{"metadata":{},"cell_type":"markdown","source":"# XSE_Resnext with R"},{"metadata":{},"cell_type":"markdown","source":"Since I did not find convenient way to implement a resnext and all the derivative using keras, I will switch to fastai. Fastai v2 has [a lot of state of the art model](https://docs.fast.ai/vision.models.xresnet.html) directly implemented. There is a R wrapper for fastai (links below), so we can use it to implement easily all this resnext from R :)"},{"metadata":{},"cell_type":"markdown","source":"Source material for this notebook is https://henry090.github.io/fastai and https://www.kaggle.com/henry090/fast-ai-from-r-timm-learner, as well as the fastai book. This notebook is forked from one where I just traind a xresnet, mistaken it with a resnext :-) "},{"metadata":{},"cell_type":"markdown","source":"As a reminder :\n* X stand for Resnet from bags of tricks paper (Xresnet).\n* SE stand for SENet — Squeeze-and-Excitation Network. It is an independant type of CNN, but you can add SE unit inside a Resnet, giving a Serenet.\n* The last X in Resnext stand for ResNeXt Block, a type of block added to Resnets to create a more efficient Resnet (Aggregated Residual Transformations for Deep Neural Networks).\n\nMix everything and you got a XSEResnext."},{"metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":true},"cell_type":"code","source":"# This R environment comes with many helpful analytics packages installed\n# It is defined by the kaggle/rstats Docker image: https://github.com/kaggle/docker-rstats\n# For example, here's a helpful package to load\n\nlibrary(tidyverse) # metapackage of all tidyverse packages\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nlist.files(path = \"../input\")\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"devtools::install_github(\"rstudio/reticulate\", dependencies = FALSE) #required to not have to specify num_worker=0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"devtools::install_github(\"henry090/fastai\",dependencies=FALSE)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fastai::install_fastai(gpu = TRUE)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"library(fastai)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data loader"},{"metadata":{},"cell_type":"markdown","source":"I am using this two ressources : [the documentation of fastai](https://docs.fast.ai/vision.data.html#ImageDataLoaders.from_df) and the [tutorial of the wrapper](https://henry090.github.io/fastai/articles/basic_img_class.html)."},{"metadata":{"trusted":true},"cell_type":"code","source":"path_img = '../input/cassava-leaf-disease-classification/train_images/'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels<-read_csv('/kaggle/input/cassava-leaf-disease-classification//train.csv')\n#labels = data.table::fread('/kaggle/input/cassava-leaf-disease-classification//train.csv')\nhead(labels)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataloader <- fastai::ImageDataLoaders_from_df(df=labels, path=path_img, bs=32, seed=6, \n                                               item_tfms = Resize(448),\n                                               batch_tfms = aug_transforms(size=224, min_scale=0.75))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataloader %>% show_batch(dpi = 200 ,figsize = c(6,6))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In this notebook I use a xse_resnext18, as the xse_resnext50 has a tendency to explode the memory of the GPU on kaggle."},{"metadata":{"trusted":true},"cell_type":"code","source":"learnR <- dataloader %>% cnn_learner(xse_resnext18(), metrics = accuracy,  model_dir=\"fastai_model/\") #prettier","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To save computation power :"},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"learnR$to_fp16()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Fastai introduced something to quickly test the fine-tuning a model without thinking it too much :"},{"metadata":{"trusted":true},"cell_type":"code","source":"learnR %>% fine_tune(epochs = 6, freeze_epochs = 6)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Quick and easy."},{"metadata":{},"cell_type":"markdown","source":"**Side note** : it seems to me that fine_tuning process seems to consume more and more gpu as the time pass. Running fine_tune(epochs = 1, freeze_epochs = 1) works fine with the XSE_Resnext50 but goes out of memory for some reason when adding more epoch."},{"metadata":{"trusted":true},"cell_type":"code","source":"learnR %>% plot_loss(dpi = 200)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"interp <- ClassificationInterpretation_from_learner(learnR)\n\ninterp %>% plot_confusion_matrix(dpi = 200 ,figsize = c(6,6))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"3.6.3"}},"nbformat":4,"nbformat_minor":4}