{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Contents\n### YOLO v3 for image detection\n0. <a href='#0.-Introduction'>Introduction</a>\n1. <a href='#1.-Clone-and-Build-YOLOv3'>Clone and Build YOLOv3</a>\n2. <a href='#2.-Data-Migration-for-YOLOv3'>Data Migration</a>\n3. <a href='#3.-Prepare-Configuration-Files-for-Using-YOLOv3'>Prepare Configuration Files for training</a>\n4. <a href='#4.-Training-YOLOv3'>Training model</a>\n5. <a href='#5.-How-to-use-trainined-YOLOv3-for-test-images-(command-line)'>How to use trained model for test images (command line)</a>\n6. <a href='#6.-Generate-Submission-Files-with-YOLOv3-Python-Wrapper'>Generate Submission Files Using YOLOv3 Python Wrapper</a>\n7. <a href='#7.-Future-works-&-Etc'>Future works & Etc</a>","metadata":{"_uuid":"cc075cd8b1f3912c3e65651871a4ce263f772750"}},{"cell_type":"markdown","source":"## 0. Introduction\n* I'll introduce super easy and quick way to train [YOLOv3](https://pjreddie.com/darknet/yolo/) on RSNA and to generate submission file (to be honest, not super easy ...!).\n\n\n* The purpose of this competition is 'object detection'. Generally, object detection algorithms with deep learning take a long time to train model and require a lot of gpu resources. Most individual participants use one or two gpu (... or zero). Therefore, there is a need for **algorithms that works quickly with less gpu resources.**\n\n\n* I tried to use Mask R-CNN, UNet, Fast R-CNN and FCN algorithms, But eventually switched to YOLOv3.\n\n\n* In comparison to YOLOv3, Other algorithms(Mask R-CNN, UNet, FCN, ...) which contain instance/sementic segmentation tasks are very slow, and require more gpu resources, redundant parameter tunning and post-processes. Therefore, if you try to use these algorithms, you may experience difficulties in terms of training time and gpu resources. (Please see [YOLOv3 paper](https://pjreddie.com/media/files/papers/YOLOv3.pdf) for details)\n\n\n* In addition, **YOLOv3 was able to obtain high score (LB: 0.141) without additional processes(data augmentation, parameter tunning, etc...)** compared to other algorithms. So I think YOLOv3 has sufficient potential for this competition.\n\n\n* In this notebook, I'll introduce how to simply apply YOLOv3 on RSNA data. I hope this notebook would be helpful for everyone.","metadata":{"_uuid":"c828e6dd0b8dc2e80806ef2d3f4137ae4a3ea670"}},{"cell_type":"code","source":"import math\nimport os\nimport shutil\nimport sys\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport glob\nimport pydicom\nimport cv2\nfrom sklearn.model_selection import train_test_split\nfrom tqdm import tqdm","metadata":{"_uuid":"ce28f29df7b9f64725998f8133be50068c53cb69","execution":{"iopub.status.busy":"2022-08-24T03:42:24.432721Z","iopub.execute_input":"2022-08-24T03:42:24.433132Z","iopub.status.idle":"2022-08-24T03:42:25.800368Z","shell.execute_reply.started":"2022-08-24T03:42:24.433066Z","shell.execute_reply":"2022-08-24T03:42:25.799357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"random_stat = 123\nnp.random.seed(random_stat)","metadata":{"_uuid":"c4b33affdc049734daa58852c4ed06e37c344063","execution":{"iopub.status.busy":"2022-08-24T03:42:25.805117Z","iopub.execute_input":"2022-08-24T03:42:25.807534Z","iopub.status.idle":"2022-08-24T03:42:25.813803Z","shell.execute_reply.started":"2022-08-24T03:42:25.80537Z","shell.execute_reply":"2022-08-24T03:42:25.813184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1. Clone and Build YOLOv3","metadata":{"_uuid":"97d22b2e1c828e655e7df04474169cd0d7a5a031"}},{"cell_type":"code","source":"!git clone https://github.com/pjreddie/darknet.git\n\n# Build gpu version darknet\n!cd darknet && sed '1 s/^.*$/GPU=1/; 2 s/^.*$/CUDNN=1/' -i Makefile\n\n# -j <The # of cpu cores to use>. Chang 999 to fit your environment. Actually i used '-j 50'.\n!cd darknet && make -j 999 -s\n!cp darknet/darknet darknet_gpu","metadata":{"_uuid":"558964006de8b4e454d76c91a5d0a605e397a93d","execution":{"iopub.status.busy":"2022-08-24T03:42:25.818444Z","iopub.execute_input":"2022-08-24T03:42:25.820918Z","iopub.status.idle":"2022-08-24T03:43:01.798351Z","shell.execute_reply.started":"2022-08-24T03:42:25.82084Z","shell.execute_reply":"2022-08-24T03:43:01.797395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Data Migration for YOLOv3\nIt might take a while.","metadata":{"_uuid":"9f77f087a15c73fae7a710fd560244b8f7884197"}},{"cell_type":"markdown","source":"### 2.0. Make subdirectories","metadata":{"_uuid":"5702330ac23995099c3f184ef0424731d91a303c"}},{"cell_type":"code","source":"DATA_DIR = \"../input\"\n\ntrain_dcm_dir = os.path.join(DATA_DIR, \"stage_1_train_images\")\ntest_dcm_dir = os.path.join(DATA_DIR, \"stage_1_test_images\")\n\nimg_dir = os.path.join(os.getcwd(), \"images\")  # .jpg\nlabel_dir = os.path.join(os.getcwd(), \"labels\")  # .txt\nmetadata_dir = os.path.join(os.getcwd(), \"metadata\") # .txt\n\n# YOLOv3 config file directory\ncfg_dir = os.path.join(os.getcwd(), \"cfg\")\n# YOLOv3 training checkpoints will be saved here\nbackup_dir = os.path.join(os.getcwd(), \"backup\")\n\nfor directory in [img_dir, label_dir, metadata_dir, cfg_dir, backup_dir]:\n    if os.path.isdir(directory):\n        continue\n    os.mkdir(directory)","metadata":{"_uuid":"55f5d67ac085afcc445db23227ba60748eb269fa","execution":{"iopub.status.busy":"2022-08-24T03:43:01.799996Z","iopub.execute_input":"2022-08-24T03:43:01.800289Z","iopub.status.idle":"2022-08-24T03:43:01.807205Z","shell.execute_reply.started":"2022-08-24T03:43:01.800239Z","shell.execute_reply":"2022-08-24T03:43:01.806427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls -shtl","metadata":{"_uuid":"f0b2173fa0ff6a1bfdf304310466d01052920fe6","execution":{"iopub.status.busy":"2022-08-24T03:43:01.80836Z","iopub.execute_input":"2022-08-24T03:43:01.809104Z","iopub.status.idle":"2022-08-24T03:43:02.792378Z","shell.execute_reply.started":"2022-08-24T03:43:01.809048Z","shell.execute_reply":"2022-08-24T03:43:02.791416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.1. Load stage_1_train_labels.csv","metadata":{"_uuid":"ba6a35b97d0a42060e6fdd68a6da23454aa40568"}},{"cell_type":"code","source":"annots = pd.read_csv(os.path.join(DATA_DIR, \"stage_1_train_labels.csv\"))\nannots.head()","metadata":{"_uuid":"e242b29961c22433703e442a84572783b9459de9","execution":{"iopub.status.busy":"2022-08-24T03:43:02.794085Z","iopub.execute_input":"2022-08-24T03:43:02.794366Z","iopub.status.idle":"2022-08-24T03:43:03.163587Z","shell.execute_reply.started":"2022-08-24T03:43:02.794316Z","shell.execute_reply":"2022-08-24T03:43:03.064011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.2. Generate images and labels for training YOLOv3\n* YOLOv3 needs .txt file for each image, which contains ground truth object in the image that looks like:\n```\n<object-class_1> <x_1> <y_1> <width_1> <height_1>\n<object-class_2> <x_2> <y_2> <width_2> <height_2>\n```\n* <object-class\\>: Since RSNA task is binary classification basically, <object-class\\> is 0.\n* <x\\>, <y\\>: Those are float values of bbox center coordinate, divided by image width and height respectively.\n* <w\\>, <h\\>: Those are width and height of bbox, divided by image width and height respectively.\n\n* So it is different from the format of label data provided by kaggle. We should change it.","metadata":{"_uuid":"9b26d929bb119deca558ac983a5ce4e95fe9b7cc"}},{"cell_type":"code","source":"def save_img_from_dcm(dcm_dir, img_dir, patient_id):\n    img_fp = os.path.join(img_dir, \"{}.jpg\".format(patient_id))\n    if os.path.exists(img_fp):\n        return\n    dcm_fp = os.path.join(dcm_dir, \"{}.dcm\".format(patient_id))\n    img_1ch = pydicom.read_file(dcm_fp).pixel_array\n    img_3ch = np.stack([img_1ch]*3, -1)\n\n    img_fp = os.path.join(img_dir, \"{}.jpg\".format(patient_id))\n    cv2.imwrite(img_fp, img_3ch)\n    \ndef save_label_from_dcm(label_dir, patient_id, row=None):\n    # rsna defualt image size\n    img_size = 1024\n    label_fp = os.path.join(label_dir, \"{}.txt\".format(patient_id))\n    \n    f = open(label_fp, \"a\")\n    if row is None:\n        f.close()\n        return\n\n    top_left_x = row[1]\n    top_left_y = row[2]\n    w = row[3]\n    h = row[4]\n    \n    # 'r' means relative. 'c' means center.\n    rx = top_left_x/img_size\n    ry = top_left_y/img_size\n    rw = w/img_size\n    rh = h/img_size\n    rcx = rx+rw/2\n    rcy = ry+rh/2\n    \n    line = \"{} {} {} {} {}\\n\".format(0, rcx, rcy, rw, rh)\n    \n    f.write(line)\n    f.close()\n        \ndef save_yolov3_data_from_rsna(dcm_dir, img_dir, label_dir, annots):\n    for row in tqdm(annots.values):\n        patient_id = row[0]\n\n        img_fp = os.path.join(img_dir, \"{}.jpg\".format(patient_id))\n        if os.path.exists(img_fp):\n            save_label_from_dcm(label_dir, patient_id, row)\n            continue\n\n        target = row[5]\n        # Since kaggle kernel have samll volume (5GB ?), I didn't contain files with no bbox here.\n        if target == 0:\n            continue\n        save_label_from_dcm(label_dir, patient_id, row)\n        save_img_from_dcm(dcm_dir, img_dir, patient_id)","metadata":{"_uuid":"5204d3b142e17927c06e9e992dd4e4f4a3d0c8ab","execution":{"iopub.status.busy":"2022-08-24T03:43:03.06468Z","iopub.status.idle":"2022-08-24T03:43:03.065031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_yolov3_data_from_rsna(train_dcm_dir, img_dir, label_dir, annots)","metadata":{"_uuid":"1e907f30b11bf6956884061138d615bee60c2a15","execution":{"iopub.status.busy":"2022-08-24T03:43:03.065785Z","iopub.status.idle":"2022-08-24T03:43:03.066217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!du -sh images labels","metadata":{"_uuid":"19d2c4ab8448c0572ac57b49eff52c2230c9f079","execution":{"iopub.status.busy":"2022-08-24T03:43:03.066773Z","iopub.status.idle":"2022-08-24T03:43:03.067269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.3. Plot a sample train image and label","metadata":{"_uuid":"17868a4d327c2e22adae76cc7ce163c5546e1743"}},{"cell_type":"code","source":"ex_patient_id = annots[annots.Target == 1].patientId.values[0]\nex_img_path = os.path.join(img_dir, \"{}.jpg\".format(ex_patient_id))\nex_label_path = os.path.join(label_dir, \"{}.txt\".format(ex_patient_id))\n\nplt.imshow(cv2.imread(ex_img_path))\n\nimg_size = 1014\nwith open(ex_label_path, \"r\") as f:\n    for line in f:\n        print(line)\n        class_id, rcx, rcy, rw, rh = list(map(float, line.strip().split()))\n        x = (rcx-rw/2)*img_size\n        y = (rcy-rh/2)*img_size\n        w = rw*img_size\n        h = rh*img_size\n        plt.plot([x, x, x+w, x+w, x], [y, y+h, y+h, y, y])","metadata":{"_uuid":"8ebf845a4f3b51555991292d10ddc272b226bf7c","execution":{"iopub.status.busy":"2022-08-24T03:43:03.068122Z","iopub.status.idle":"2022-08-24T03:43:03.068762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.4. Generate train/val file path list (.txt)\n* We should give the list of image paths to YOLO. two seperate list textfiles for training images and validation images.","metadata":{"_uuid":"42551c1e11208ee88f9b93bea40c23dcad295b8a"}},{"cell_type":"code","source":"def write_train_list(metadata_dir, img_dir, name, series):\n    list_fp = os.path.join(metadata_dir, name)\n    with open(list_fp, \"w\") as f:\n        for patient_id in series:\n            line = \"{}\\n\".format(os.path.join(img_dir, \"{}.jpg\".format(patient_id)))\n            f.write(line)","metadata":{"_uuid":"d62f98b287e73a66c234c4c9f9e41503fc0d4bca","execution":{"iopub.status.busy":"2022-08-24T03:43:03.069618Z","iopub.status.idle":"2022-08-24T03:43:03.070483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Following lines do not contain data with no bbox\npatient_id_series = annots[annots.Target == 1].patientId.drop_duplicates()\n\ntr_series, val_series = train_test_split(patient_id_series, test_size=0.1, random_state=random_stat)\nprint(\"The # of train set: {}, The # of validation set: {}\".format(tr_series.shape[0], val_series.shape[0]))\n\n# train image path list\nwrite_train_list(metadata_dir, img_dir, \"tr_list.txt\", tr_series)\n# validation image path list\nwrite_train_list(metadata_dir, img_dir, \"val_list.txt\", val_series)","metadata":{"_uuid":"4c3982b42711fe4a20600b90ed3d123384da7096","execution":{"iopub.status.busy":"2022-08-24T03:43:03.071306Z","iopub.status.idle":"2022-08-24T03:43:03.073099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.5. Create test image and labels for YOLOv3","metadata":{"_uuid":"482fc4b015279312504c910b6c910a5d58960159"}},{"cell_type":"code","source":"def save_yolov3_test_data(test_dcm_dir, img_dir, metadata_dir, name, series):\n    list_fp = os.path.join(metadata_dir, name)\n    with open(list_fp, \"w\") as f:\n        for patient_id in series:\n            save_img_from_dcm(test_dcm_dir, img_dir, patient_id)\n            line = \"{}\\n\".format(os.path.join(img_dir, \"{}.jpg\".format(patient_id)))\n            f.write(line)","metadata":{"_uuid":"7394207f5f23778aee12d5f4dadae632b340ce62","execution":{"iopub.status.busy":"2022-08-24T03:43:03.073961Z","iopub.status.idle":"2022-08-24T03:43:03.074684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_dcm_fps = list(set(glob.glob(os.path.join(test_dcm_dir, '*.dcm'))))\ntest_dcm_fps = pd.Series(test_dcm_fps).apply(lambda dcm_fp: dcm_fp.strip().split(\"/\")[-1].replace(\".dcm\",\"\"))\n\nsave_yolov3_test_data(test_dcm_dir, img_dir, metadata_dir, \"te_list.txt\", test_dcm_fps)","metadata":{"_uuid":"e99463093b202e2aabfcd3979701b7e47ba91c3e","execution":{"iopub.status.busy":"2022-08-24T03:43:03.075537Z","iopub.status.idle":"2022-08-24T03:43:03.07625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.6. Plot a sample test Image","metadata":{"_uuid":"42c007da9fd2009310dcf697bca943c4dd7a33f7"}},{"cell_type":"code","source":"ex_patient_id = test_dcm_fps[0]\nex_img_path = os.path.join(img_dir, \"{}.jpg\".format(ex_patient_id))\n\nplt.imshow(cv2.imread(ex_img_path))","metadata":{"_uuid":"f057a0fc9938308f69a5d402cf51ba340af592c0","execution":{"iopub.status.busy":"2022-08-24T03:43:03.077088Z","iopub.status.idle":"2022-08-24T03:43:03.077759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. Prepare Configuration Files for Using YOLOv3\nWe should prepare and modify config files, and bring pre-trained weights necessary for training. This proceeds with following four steps.\n```\n cfg/rsna.data\n cfg/rsna.names\n darknet53.conv.74\n cfg/rsna_yolov3.cfg_train\n```","metadata":{"_uuid":"8ac8f43aae2bc82599fd1bb4a66d1dd444b66a28"}},{"cell_type":"markdown","source":"### - cfg/rsna.data\nThis file point to RSNA data path\n  * train: Path to training image list textfile\n  * val: Path to validation image list textfile\n  * names: RSNA class name list (see <a href='#3.1.-cfg/rsna.names'>3.1</a>)\n  * backup: A directory where trained weights(checkpoints) will be stored as training progresses.","metadata":{"_uuid":"72fb38e755495fca3bf456c4494c05b5300d3411"}},{"cell_type":"code","source":"data_extention_file_path = os.path.join(cfg_dir, 'rsna.data')\nwith open(data_extention_file_path, 'w') as f:\n    contents = \"\"\"classes= 1\ntrain  = {}\nvalid  = {}\nnames  = {}\nbackup = {}\n    \"\"\".format(os.path.join(metadata_dir, \"tr_list.txt\"),\n               os.path.join(metadata_dir, \"val_list.txt\"),\n               os.path.join(cfg_dir, 'rsna.names'),\n               backup_dir)\n    f.write(contents)","metadata":{"_uuid":"f78dcf8c107c67e49f8bba8f483ab6a5b409ec0b","execution":{"iopub.status.busy":"2022-08-24T03:43:03.078571Z","iopub.status.idle":"2022-08-24T03:43:03.079241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat cfg/rsna.data","metadata":{"_uuid":"684b3e9dde58f56a746952d2934661712e417161","execution":{"iopub.status.busy":"2022-08-24T03:43:03.080057Z","iopub.status.idle":"2022-08-24T03:43:03.080724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### - cfg/rsna.names","metadata":{"_uuid":"46cd3f02af89917b9715f51f188a47d6030bd758"}},{"cell_type":"code","source":"# Label list of bounding box.\n!echo \"pneumonia\" > cfg/rsna.names","metadata":{"_uuid":"917144c1cffd160a997db563cf9506db13e69414","execution":{"iopub.status.busy":"2022-08-24T03:43:03.081531Z","iopub.status.idle":"2022-08-24T03:43:03.082192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### - darknet53.conv.74  (Download Pre-trained Model)\nFor training, we would download the pre-trained model weights(darknet53.conv.74) using following wget command. I recommend you to use this pre-trained weight too. Author of darknet also uses this pre-trained weights in different fields of image recognition.","metadata":{"_uuid":"6415f6efcae6fca1812cb402af668428d2918e46"}},{"cell_type":"code","source":"!wget -q https://pjreddie.com/media/files/darknet53.conv.74","metadata":{"_uuid":"ec2d5fbdfe6f3eebd553495d10f9adcdae29f16f","execution":{"iopub.status.busy":"2022-08-24T03:43:03.082985Z","iopub.status.idle":"2022-08-24T03:43:03.083643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### - cfg/rsna_yolov3.cfg_train\n* Basically, you can use darknet/cfg/yolov3.cfg files. However it won't work for RSNA. you need to edit for RSNA.\n* You can just download a cfg file I edited for RSNA with following wget command.\n\n\n* I refer to the following articles for editing cfg files.\n  * [YOLOv3 blog](https://pjreddie.com/darknet/yolo/)\n  * [YOLOv3 paper](https://pjreddie.com/media/files/papers/YOLOv3.pdf)\n  * [how to train yolov2 blog](https://medium.com/@manivannan_data/how-to-train-yolov2-to-detect-custom-objects-9010df784f36)\n  * [darknet github issues/236](https://github.com/pjreddie/darknet/issues/236)","metadata":{"_uuid":"978655a61edee55f635c8bf8c5befae03c838652"}},{"cell_type":"code","source":"!wget --no-check-certificate -q \"https://docs.google.com/uc?export=download&id=18ptTK4Vbeokqpux8Onr0OmwUP9ipmcYO\" -O cfg/rsna_yolov3.cfg_train","metadata":{"_uuid":"f64517332ee89f69af1b013ef6376b19d639f0a6","execution":{"iopub.status.busy":"2022-08-24T03:43:03.084458Z","iopub.status.idle":"2022-08-24T03:43:03.085115Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Training YOLOv3","metadata":{"_uuid":"f03592ee228f9ae16edac192071955d01233a147"}},{"cell_type":"markdown","source":"### 4.0. Command for training with Pre-trained CNN Weights (darknet53.conv.74)\n* I didn't run following command on kaggle kernel becuase of the long output.\n* If you crash with  'CUDA Error: out of memory', Solve it by Editing 'batch' and 'subdivisions' in 'cfg/rsna_yolov3.cfg_train'\n* If 'batch' and 'subdivisions' are 64 and 64 respectively, for every iteration only one image will be loaded on GPU memory. So it will use less GPU memory.","metadata":{"_uuid":"69f52f31dea66f3da1952b537c63dc5376d64492"}},{"cell_type":"code","source":"# !./darknet_gpu detector train cfg/rsna.data cfg/rsna_yolov3.cfg_train darknet53.conv.74 -i 0 | tee train_log.txt","metadata":{"_uuid":"0be53a581ab12776c1c276f72acbf08b2db16d77","execution":{"iopub.status.busy":"2022-08-24T03:43:03.085939Z","iopub.status.idle":"2022-08-24T03:43:03.086592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 4.1. Command for training with Multi-gpu after 1000 iteration\n\nIf you are trying to train with multi-gpu, there are three things to watch out.\n* (The # of gpus)x('learning rate' in 'cfg/rsna_yolov3.cfg_train') is the real learning rate for training\n* I don't recommend you to use multi-gpu for first 1000 iterations. with multi-gpu, training would not be stable. Use single gpu before 1000 and after 1000, continue with more gpus.\n* By the way, If the # of gpus is over 5, training is not stable.\n\n```\nAbove things will depend on your environment. The best way to find the most appropriate method is to just give it a try :)\n```","metadata":{"_uuid":"1c43f25ac894192e9c0e03cbcb4c36072210be12"}},{"cell_type":"code","source":"# !./darknet_gpu detector train cfg/rsna.data cfg/rsna_yolov3.cfg_train backup/rsna_yolov3_1000.weights -gpus 0,1,2,3 | tee train_log.txt","metadata":{"_uuid":"a3694016ec0d7372b131837b8e6279a6f59a12a2","execution":{"iopub.status.busy":"2022-08-24T03:43:03.087487Z","iopub.status.idle":"2022-08-24T03:43:03.088142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 4.2. My Plot of Training Loss\nIt's a loss graph up to about 2000 iteration. Since it tooks too long on kaggle kernel, I brought it. When learning, don't be surprised of big loss values at the beginning. Stay calm and It'll go down. Please See the following loss graph.","metadata":{"_uuid":"3cfd26f1c657bd30952f04b4ebbe6ca6ff7dfea8"}},{"cell_type":"code","source":"!wget --no-check-certificate -q \"https://docs.google.com/uc?export=download&id=1OhnlV3s7r6xsEme6DKkNYjcYjsl-C_Av\" -O train_log.txt","metadata":{"_uuid":"417c9157d98d5a552f519a3935670fd4bd437a4b","execution":{"iopub.status.busy":"2022-08-24T03:43:03.088946Z","iopub.status.idle":"2022-08-24T03:43:03.089621Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"iters = []\nlosses = []\ntotal_losses = []\nwith open(\"train_log.txt\", 'r') as f:\n    for i,line in enumerate(f):\n        if \"images\" in line:\n            iters.append(int(line.strip().split()[0].split(\":\")[0]))\n            losses.append(float(line.strip().split()[2]))        \n            total_losses.append(float(line.strip().split()[1].split(',')[0]))\n\nplt.figure(figsize=(20, 5))\nplt.subplot(1,2,1)\nsns.lineplot(iters, total_losses, label=\"totla loss\")\nsns.lineplot(iters, losses, label=\"avg loss\")\nplt.xlabel(\"Iteration\")\nplt.ylabel(\"Loss\")\n\nplt.subplot(1,2,2)\nsns.lineplot(iters, total_losses, label=\"totla loss\")\nsns.lineplot(iters, losses, label=\"avg loss\")\nplt.xlabel(\"Iteration\")\nplt.ylabel(\"Loss\")\nplt.ylim([0, 4.05])","metadata":{"_uuid":"f761373c3c46f5c7875d78cc3023becc5924f6a4","execution":{"iopub.status.busy":"2022-08-24T03:43:03.090453Z","iopub.status.idle":"2022-08-24T03:43:03.091124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5. How to use trainined YOLOv3 for test images (command line)","metadata":{"_uuid":"ef7409909b9403f43b2c92760220b23a38d42029"}},{"cell_type":"markdown","source":"### 5.0. Copy sample test image","metadata":{"_uuid":"c97da5c5f52e0257a3b1a2568e6260a9560700a0"}},{"cell_type":"code","source":"ex_patient_id = annots[annots.Target == 1].patientId.values[2]\nshutil.copy(ex_img_path, \"test.jpg\")\nprint(ex_patient_id)","metadata":{"_uuid":"688344d91c7f7fb87418486b9b6b5f91a8b26e9a","execution":{"iopub.status.busy":"2022-08-24T03:43:03.091921Z","iopub.status.idle":"2022-08-24T03:43:03.092588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 5.1. Load trained model (at 15300 iteration)\nSince i uploaded the weights file (large big file) on my google drive, the command is very very long ...\n* It's a weight file at 15300 iteration, which I made submission file with. If you use this weight, you'll get a score of 0.141LB.\n  * Up to 15300 iteration, It takes about 8 hours.\n    * In .cfg file, I set 'batch' and 'subdivisions' as 64 and 8 respectively.\n    * Up to 1000 iteration from 0, it takes about 1h with **one** Tesla P100 GPU.      **(1000 iter/h)**\n    * Up to 15300 iteration from 1000, it takes about 7h with **four** Tesla P100 GPU. **(2043 iter/h)**","metadata":{"_uuid":"245c607d7753b947a91fe4d7ce8daf983afdfd0e"}},{"cell_type":"code","source":"!wget --load-cookies /tmp/cookies.txt -q \"https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1FDzMN-kGVYCvBeDKwemAazldSVkAEFyd' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\\1\\n/p')&id=1FDzMN-kGVYCvBeDKwemAazldSVkAEFyd\" -O backup/rsna_yolov3_15300.weights && rm -rf /tmp/cookies.txt","metadata":{"_uuid":"91d60e964d4cbd5397f6094fd811d115977abc3e","execution":{"iopub.status.busy":"2022-08-24T03:43:03.093389Z","iopub.status.idle":"2022-08-24T03:43:03.094072Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls -alsth backup","metadata":{"_uuid":"d745c3d07102f9f1f27bf57a0c009311f9e0fd3b","execution":{"iopub.status.busy":"2022-08-24T03:43:03.094841Z","iopub.status.idle":"2022-08-24T03:43:03.095544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 5.2. cfg file for test (not for training)","metadata":{"_uuid":"41b4ca297869af6808b758824960e9dad12d00c0"}},{"cell_type":"code","source":"!wget --no-check-certificate -q \"https://docs.google.com/uc?export=download&id=10Yk6ZMAKGz5LeBbikciALy82aK3lX-57\" -O cfg/rsna_yolov3.cfg_test","metadata":{"_uuid":"17708ffdc9b834060ccb2b8cb4aefaa3c00b96b6","execution":{"iopub.status.busy":"2022-08-24T03:43:03.09633Z","iopub.status.idle":"2022-08-24T03:43:03.097017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cd darknet && ./darknet detector test ../cfg/rsna.data ../cfg/rsna_yolov3.cfg_test ../backup/rsna_yolov3_15300.weights ../test.jpg -thresh 0.005","metadata":{"_uuid":"de9f7f2f30feeeaa3417ebd338c402e57a9b1f38","execution":{"iopub.status.busy":"2022-08-24T03:43:03.0978Z","iopub.status.idle":"2022-08-24T03:43:03.098532Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ![](predictions.jpg)\nplt.imshow(cv2.imread(\"./darknet/predictions.jpg\"))","metadata":{"_uuid":"0dec1cc6f5bd1f8c854697128091ec7bd69d6e3e","execution":{"iopub.status.busy":"2022-08-24T03:43:03.099319Z","iopub.status.idle":"2022-08-24T03:43:03.099996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 6. Generate Submission Files with YOLOv3 Python Wrapper","metadata":{"_uuid":"064029258018a3aa0ee22c59d74e6f321b767bf8"}},{"cell_type":"markdown","source":"### 6.0. Download darknet python wrapper (darknet.py)\n* Basically, you can use darknet/python/darknet.py files. However it'll show error.\n* So, I edited the darknet.py. There are two main modifications.\n  * Change print statement to print function for python3\n  * Edit dynamic library('libdarknet.so') file path\n* I leaved '# ===' marks where i edited in darknet.py. For example,\n```\n# ==============================================================================\n#lib = CDLL(\"/home/pjreddie/documents/darknet/libdarknet.so\", RTLD_GLOBAL)\ndarknet_lib_path = os.path.join(os.getcwd(), \"darknet\", \"libdarknet.so\")\nlib = CDLL(darknet_lib_path, RTLD_GLOBAL)\n# ==============================================================================\n```","metadata":{"_uuid":"a4848803d5f41f1082de0b0a1e348a26f5d38d58"}},{"cell_type":"code","source":"!wget --no-check-certificate -q \"https://docs.google.com/uc?export=download&id=1-KTV7K9G1bl3SmnLnzmpkDyNt6tDmH7j\" -O darknet.py","metadata":{"_uuid":"6cef00c59783cf8ff052b3c8b79903a9dfa24fc5","execution":{"iopub.status.busy":"2022-08-24T03:43:03.100763Z","iopub.status.idle":"2022-08-24T03:43:03.10146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 6.1. Load darknet python wrapper module","metadata":{"_uuid":"91eafd163f9e78557c7acb1be7176845cc15d4c1"}},{"cell_type":"code","source":"from darknet import *","metadata":{"_uuid":"238a70858c290db54f72700fe57215bd7934f6d8","execution":{"iopub.status.busy":"2022-08-24T03:43:03.102278Z","iopub.status.idle":"2022-08-24T03:43:03.103018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 6.2. Generate submission files\n* When making submission files, be aware of label format which is different in yolo.","metadata":{"_uuid":"1f6193268a5bd5b27d3ba855d6d36715a63d4ee0"}},{"cell_type":"code","source":"threshold = 0.2","metadata":{"_uuid":"877c1dc79f4a4cb0dc3c322a05282d3d902ed615","execution":{"iopub.status.busy":"2022-08-24T03:43:03.103817Z","iopub.status.idle":"2022-08-24T03:43:03.10452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submit_file_path = \"submission.csv\"\ncfg_path = os.path.join(cfg_dir, \"rsna_yolov3.cfg_test\")\nweight_path = os.path.join(backup_dir, \"rsna_yolov3_15300.weights\")\n\ntest_img_list_path = os.path.join(metadata_dir, \"te_list.txt\")","metadata":{"_uuid":"b776bab5a37d180d2e5185555b47cafb1a964a71","execution":{"iopub.status.busy":"2022-08-24T03:43:03.105316Z","iopub.status.idle":"2022-08-24T03:43:03.106016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gpu_index = 0\nnet = load_net(cfg_path.encode(),\n               weight_path.encode(), \n               gpu_index)\nmeta = load_meta(data_extention_file_path.encode())","metadata":{"_uuid":"d82a944502827338091976c4912f5eb6af6e886e","execution":{"iopub.status.busy":"2022-08-24T03:43:03.106802Z","iopub.status.idle":"2022-08-24T03:43:03.107506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submit_dict = {\"patientId\": [], \"PredictionString\": []}\n\nwith open(test_img_list_path, \"r\") as test_img_list_f:\n    # tqdm run up to 1000(The # of test set)\n    for line in tqdm(test_img_list_f):\n        patient_id = line.strip().split('/')[-1].strip().split('.')[0]\n\n        infer_result = detect(net, meta, line.strip().encode(), thresh=threshold)\n\n        submit_line = \"\"\n        for e in infer_result:\n            confi = e[1]\n            w = e[2][2]\n            h = e[2][3]\n            x = e[2][0]-w/2\n            y = e[2][1]-h/2\n            submit_line += \"{} {} {} {} {} \".format(confi, x, y, w, h)\n\n        submit_dict[\"patientId\"].append(patient_id)\n        submit_dict[\"PredictionString\"].append(submit_line)\n\npd.DataFrame(submit_dict).to_csv(submit_file_path, index=False)","metadata":{"_uuid":"c11fa319a577beff0a0d53f107489ee79771bc22","execution":{"iopub.status.busy":"2022-08-24T03:43:03.108323Z","iopub.status.idle":"2022-08-24T03:43:03.109012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !ls -lsht\n!rm -rf darknet images labels metadata backup cfg\n!rm -rf train_log.txt darknet53.conv.74 darknet.py darknet_gpu\n!rm -rf test.jpg\n!rm -rf __pycache__ .ipynb_checkpoints","metadata":{"_uuid":"7ffbb3e3205f76f5b8f216266539469c65d56cc1","execution":{"iopub.status.busy":"2022-08-24T03:43:03.109802Z","iopub.status.idle":"2022-08-24T03:43:03.110492Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls -alsht","metadata":{"_uuid":"74cd76e2b853568a7f2962f0c220a64fe87e67c1","execution":{"iopub.status.busy":"2022-08-24T03:43:03.111268Z","iopub.status.idle":"2022-08-24T03:43:03.111948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7. Future works & Etc\n\n### Future works (Things to try)\n* Image augmentation\n* More training\n* Utilizing the not labeled images because we got rid of not labeled images above\n\n### ETC\n* For a private matter, i can not proceed RSNA task after 09/27. If you have any ideas, questions and problems with this kernel after 09/27, Please leave those things anyway~! Collaborator '@John Byun' will reply to your comments.","metadata":{"_uuid":"2fbf050ee377907eb544c04382d07e1449ef577a","trusted":true}}]}