{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## I. Introduction\n\nIn YOLOV3, bounding boxes for objects in training images are a key input. Anchor boxes are a key part YOLOV3 model configuration, one that can sets YOLOV3 up for efficent training. Anchor boxes are defined at the resolution chosen for training inputs. As an example, RSNA images are of dimensions 1024x1024, but possible YOLOV3 training dimensions are 416x416, 512x512, and 608x608 (notice these are all multiples of 32). During training, YOLOV3 figures out offsets from the closest anchor box that provides the lowest loss for a ground truth bounding box. So, while any reasonable set of anchor boxes can be adequate for model convergence using YOLOV3, this kernel analyzes RSNA Stage 2 training inputs with the goal of potentially choosing anchor boxes that lead to more efficient training convergence.\n\nV2 was a stable version with input image size of (defined by YOLOV3_SIZE) of 608x608. V3 analyzes anchor boxes at YOLOV3_SIZE of 512.\n\nComments welcome!\n\nReferences: YOLOV3 Paper, YOLOV3 Web Site","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nfrom sklearn.cluster import MiniBatchKMeans\nimport matplotlib.pyplot as plt\n\nimport seaborn as sns\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"# clone darknet\n!git clone https://github.com/pjreddie/darknet","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets look at the default anchor boxes in yolov3.cfg (9 anchor boxes) and the associated input image sizes\n\n\n!cp darknet/cfg/yolov3.cfg .\n!grep -E 'width|height|anchors' yolov3.cfg","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# cleanup darknet download\n!rm -rf darknet","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# global variables\nTRAIN_LABELS_CSV_FILE=\"/kaggle/input/rsna-pneumonia-detection-challenge/stage_2_train_labels.csv\"\n# pedantic nit: we are changing 'Target' to 'label' on the way in\nTRAIN_LABELS_CSV_COLUMN_NAMES=['patientId', 'x1', 'y1', 'bw', 'bh', 'label']\n\nDICOM_IMAGE_SIZE=1024\nYOLOV3_SIZE=512","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# read RSNA TRAIN_LABELS_CSV_FILE into a pandas dataframe\nlabelsbboxdf = pd.read_csv(TRAIN_LABELS_CSV_FILE,\n                           names=TRAIN_LABELS_CSV_COLUMN_NAMES,\n                           # skip the header line\n                           header=0,\n                           # index the dataframe on patientId\n                           index_col='patientId')\n\nlabelsbboxdf.head( )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We Can See there are a lot of Null Data in Bounding Box Pneumonia Patient Disease Detector Dataset Dataset. Then We Delete Null Data in Bounding Box Pneumonia Patient Disease Detector Dataset Dataset","metadata":{}},{"cell_type":"code","source":"labelsbboxdf = labelsbboxdf.dropna()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labelsbboxdf.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# drop all fields except the bounding box dimensions and\n# all row except the Lung Opacity ones\nyolov3bboxesdf=labelsbboxdf[['bw', 'bh']].dropna()\nyolov3bboxesdf.head(10)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# resize bounding boxes for YOLOV3_SIZE\nyolov3bboxesdf=yolov3bboxesdf*(YOLOV3_SIZE/DICOM_IMAGE_SIZE)\nyolov3bboxesdf.head(10)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# as reference, below are the vitals on bounding boxes at DICOM_IMAGE_SIZE\nlabelsbboxdf[['bw', 'bh']].describe(percentiles=[0.25, 0.5, 0.75, .95])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# below are the vitals on bounding boxes at chosen input size of YOLOV3_SIZE\nyolov3bboxesdf.describe(percentiles=[0.25, 0.5, 0.75, 0.85, .95])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# we could hand-craft the following anchor boxes :\n# ~<min, ~<25%, ~<50%, ~<75%, ~<85%, ~<95% and have a 6 anchor box set (for yolov3 tiny)\n!printf '10,15, 75,75 100,125, 100,175 125,225, 150,275\\n' > rsna-yolov3-manual-anchors.txt","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## II. Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering\n\nThen We Can Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering. Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering Can Be Seen As Below.","metadata":{}},{"cell_type":"code","source":"# convert to numpy array\nbboxarray=np.array(yolov3bboxesdf)\n\nprint (bboxarray.shape)\nprint (bboxarray)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First We Have to Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering. Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering can be seen as below.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nfrom sklearn.datasets import load_iris\nfrom sklearn.cluster import KMeans\nimport matplotlib.pyplot as plt\n\nsns.set_theme()\n\n\"\"\"\niris = load_iris()\nX = pd.DataFrame(iris.data, columns=iris['feature_names'])\n#print(X)\ndata = X[['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)']]\n\"\"\"\n\ndata = bboxarray\n\nsse = {}\nfor k in range(1, 15):\n    kmeans = KMeans(n_clusters=k, max_iter=1000).fit(data)\n    #data[\"clusters\"] = kmeans.labels_\n    #print(data[\"clusters\"])\n    sse[k] = kmeans.inertia_ # Inertia: Sum of distances of samples to their closest cluster center\n    \n    \nplt.figure( figsize = ( 20 , 13 ))\nplt.plot(list(sse.keys()), list(sse.values()))\nplt.xlabel(\"Number of cluster\")\nplt.ylabel(\"SSE\")\nplt.show()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**From Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering We Can See that Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering with Machine Learning K- Means Clustering Class 8 have the most Nearest Machine Learning K- Means Clustering distance to the K- Means Clustering Class. Then We will use Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering using Machine Learning K- Means Clustering Class 8. Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering with Machine Learning K- Means Clustering Class 8 Can be Seen As Below.**","metadata":{}},{"cell_type":"code","source":"# fit to 8 kmeans clusters (for yolov3 tiny)\nkmeans= KMeans(n_clusters=8, verbose=1)\ncolors=['b.', 'g.', 'r.', 'c.', 'm.', 'y.',  'k.' , 'tab:orange' , 'tab:purple']\nkmeans.fit(bboxarray)\ncentroids=kmeans.cluster_centers_\nlabels=kmeans.labels_\n\nprint (centroids.shape)\nprint (centroids)\nprint (labels.shape)\nprint (labels)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# view computed centroids to bounding box dimensions' scatterplot\nplt.figure(figsize=(10,10))\nfor i in range(len(bboxarray)):\n    plt.plot(bboxarray[i][0], bboxarray[i][1], colors[labels[i]], markersize=10)   \nplt.scatter(centroids[:,0], centroids[:,1], marker=\"x\", s=150, linewidth=5, zorder=10)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# post process centroids\nanchors=np.around(centroids)\nprint (len(anchors))\nprint (anchors)\nprint (\"---------\")\nind = np.lexsort((anchors[:,1], anchors[:,0])) # lexsort uses the second argument first, followed by the first argument\n#print (ind)\nsortedanchors=np.array([anchors[i] for i in ind])\nprint(sortedanchors)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# write anchor boxes to file\n# organize anchor boxes in YOLOV3 format\nfor i in range (len(sortedanchors)):\n    anchorbox=\"{},{}\".format(int(sortedanchors[i][0]), int(sortedanchors[i][1]))\n    if i==0:\n        anchorrecord=anchorbox\n    else:\n        anchorrecord=\"{},  {}\".format(anchorrecord, anchorbox)\nanchorrecord=\"{}\\n\".format(anchorrecord)\n\nprint (anchorrecord)\n\n# save anchor box specification to file\nsavedanchorsfilename='rsna-yolov3-KMeans-anchors.txt'\nwith open(savedanchorsfilename,'w') as file:\n    file.write(anchorrecord)\nfile.close()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## III. Training Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning YOLO V3\n\nAfter We've Got Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning K- Means Clustering We Can Training Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning YOLO V3. Training Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning YOLO V3 Can Be Seen As Below.","metadata":{"execution":{"iopub.status.busy":"2023-05-17T05:19:43.756399Z","iopub.execute_input":"2023-05-17T05:19:43.756734Z","iopub.status.idle":"2023-05-17T05:19:43.763166Z","shell.execute_reply.started":"2023-05-17T05:19:43.756711Z","shell.execute_reply":"2023-05-17T05:19:43.761515Z"}}},{"cell_type":"code","source":"import math\nimport os\nimport shutil\nimport sys\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport glob\nimport pydicom\nimport cv2\nfrom sklearn.model_selection import train_test_split\nfrom tqdm import tqdm","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"random_stat = 123\nnp.random.seed(random_stat)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!rm -r darknet/\n\n!git clone https://github.com/pjreddie/darknet.git\n\n# Build gpu version darknet\n#!cd darknet && sed '1 s/^.*$/GPU=1/; 2 s/^.*$/CUDNN=1/' -i Makefile\n\n# -j <The # of cpu cores to use>. Chang 999 to fit your environment. Actually i used '-j 50'.\n!cd darknet && make -j 50 -s\n!cp darknet/darknet darknet_gpu","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"DATA_DIR = \"/kaggle/input/rsna-pneumonia-detection-challenge\"\n\ntrain_dcm_dir = os.path.join(DATA_DIR, \"stage_2_train_images\")\ntest_dcm_dir = os.path.join(DATA_DIR, \"stage_2_test_images\")\n\nimg_dir = os.path.join(os.getcwd(), \"images\")  # .jpg\nlabel_dir = os.path.join(os.getcwd(), \"labels\")  # .txt\nmetadata_dir = os.path.join(os.getcwd(), \"metadata\") # .txt\n\n# YOLOv3 config file directory\ncfg_dir = os.path.join(os.getcwd(), \"cfg\")\n# YOLOv3 training checkpoints will be saved here\nbackup_dir = os.path.join(os.getcwd(), \"backup\")\n\nfor directory in [img_dir, label_dir, metadata_dir, cfg_dir, backup_dir]:\n    if os.path.isdir(directory):\n        continue\n    os.mkdir(directory)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls -shtl","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"annots = pd.read_csv(os.path.join(\"/kaggle/input/rsna-pneumonia-detection-challenge\", \"stage_2_train_labels.csv\"))\nannots.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"YOLOv3 needs .txt file for each image, which contains ground truth object in the image that looks like:\n```\n<object-class_1> <x_1> <y_1> <width_1> <height_1>\n<object-class_2> <x_2> <y_2> <width_2> <height_2>\n```\n\n- <object-class>: Since RSNA task is binary classification basically, <object-class> is 0.\n- <x>, <y>: Those are float values of bbox center coordinate, divided by image width and height respectively.\n- <w>, <h>: Those are width and height of bbox, divided by image width and height respectively.\n\nSo it is different from the format of label data provided by kaggle. We should change it.","metadata":{}},{"cell_type":"code","source":"def save_img_from_dcm(dcm_dir, img_dir, patient_id):\n    img_fp = os.path.join(img_dir, \"{}.jpg\".format(patient_id))\n    if os.path.exists(img_fp):\n        return\n    dcm_fp = os.path.join(dcm_dir, \"{}.dcm\".format(patient_id))\n    img_1ch = pydicom.read_file(dcm_fp).pixel_array\n    img_3ch = np.stack([img_1ch]*3, -1)\n\n    img_fp = os.path.join(img_dir, \"{}.jpg\".format(patient_id))\n    cv2.imwrite(img_fp, img_3ch)\n    \ndef save_label_from_dcm(label_dir, patient_id, row=None):\n    # rsna defualt image size\n    img_size = 1024\n    label_fp = os.path.join(label_dir, \"{}.txt\".format(patient_id))\n    \n    f = open(label_fp, \"a\")\n    if row is None:\n        f.close()\n        return\n\n    top_left_x = row[1]\n    top_left_y = row[2]\n    w = row[3]\n    h = row[4]\n    \n    # 'r' means relative. 'c' means center.\n    rx = top_left_x/img_size\n    ry = top_left_y/img_size\n    rw = w/img_size\n    rh = h/img_size\n    rcx = rx+rw/2\n    rcy = ry+rh/2\n    \n    line = \"{} {} {} {} {}\\n\".format(0, rcx, rcy, rw, rh)\n    \n    f.write(line)\n    f.close()\n        \ndef save_yolov3_data_from_rsna(dcm_dir, img_dir, label_dir, annots):\n    for row in tqdm(annots.values):\n        patient_id = row[0]\n\n        img_fp = os.path.join(img_dir, \"{}.jpg\".format(patient_id))\n        if os.path.exists(img_fp):\n            save_label_from_dcm(label_dir, patient_id, row)\n            continue\n\n        target = row[5]\n        # Since kaggle kernel have samll volume (5GB ?), I didn't contain files with no bbox here.\n        if target == 0:\n            continue\n        save_label_from_dcm(label_dir, patient_id, row)\n        save_img_from_dcm(dcm_dir, img_dir, patient_id)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_yolov3_data_from_rsna(train_dcm_dir, img_dir, label_dir, annots)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!du -sh images labels","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ex_patient_id = annots[annots.Target == 1].patientId.values[0]\nex_img_path = os.path.join(img_dir, \"{}.jpg\".format(ex_patient_id))\nex_label_path = os.path.join(label_dir, \"{}.txt\".format(ex_patient_id))\n\nplt.imshow(cv2.imread(ex_img_path))\n\nimg_size = 1014\nwith open(ex_label_path, \"r\") as f:\n    for line in f:\n        print(line)\n        class_id, rcx, rcy, rw, rh = list(map(float, line.strip().split()))\n        x = (rcx-rw/2)*img_size\n        y = (rcy-rh/2)*img_size\n        w = rw*img_size\n        h = rh*img_size\n        plt.plot([x, x, x+w, x+w, x], [y, y+h, y+h, y, y])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def write_train_list(metadata_dir, img_dir, name, series):\n    list_fp = os.path.join(metadata_dir, name)\n    with open(list_fp, \"w\") as f:\n        for patient_id in series:\n            line = \"{}\\n\".format(os.path.join(img_dir, \"{}.jpg\".format(patient_id)))\n            f.write(line)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Following lines do not contain data with no bbox\npatient_id_series = annots[annots.Target == 1].patientId.drop_duplicates()\n\ntr_series, val_series = train_test_split(patient_id_series, test_size=0.1, random_state=random_stat)\nprint(\"The # of train set: {}, The # of validation set: {}\".format(tr_series.shape[0], val_series.shape[0]))\n\n# train image path list\nwrite_train_list(metadata_dir, img_dir, \"tr_list.txt\", tr_series)\n# validation image path list\nwrite_train_list(metadata_dir, img_dir, \"val_list.txt\", val_series)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def save_yolov3_test_data(test_dcm_dir, img_dir, metadata_dir, name, series):\n    list_fp = os.path.join(metadata_dir, name)\n    with open(list_fp, \"w\") as f:\n        for patient_id in series:\n            save_img_from_dcm(test_dcm_dir, img_dir, patient_id)\n            line = \"{}\\n\".format(os.path.join(img_dir, \"{}.jpg\".format(patient_id)))\n            f.write(line)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_dcm_fps = list(set(glob.glob(os.path.join(test_dcm_dir, '*.dcm'))))\ntest_dcm_fps = pd.Series(test_dcm_fps).apply(lambda dcm_fp: dcm_fp.strip().split(\"/\")[-1].replace(\".dcm\",\"\"))\n\nsave_yolov3_test_data(test_dcm_dir, img_dir, metadata_dir, \"te_list.txt\", test_dcm_fps)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ex_patient_id = test_dcm_fps[ 1 ]\nex_img_path = os.path.join(img_dir, \"{}.jpg\".format(ex_patient_id))\n\nplt.imshow(cv2.imread(ex_img_path))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We should prepare and modify config files, and bring pre-trained weights necessary for training. This proceeds with following four steps.\n\n```\ncfg/rsna.data\ncfg/rsna.names\ndarknet53.conv.74\ncfg/rsna_yolov3.cfg_train\n```\n\nFile Training Tuning Anchor Box Pneumonia Patient Disease Detector using Machine Learning YOLO V 3 `cfg/rsna.data` point to RSNA data path\n\n- train: Path to training image list textfile\n- val: Path to validation image list textfile\n- names: RSNA class name list (see 3.1)\n- backup: A directory where trained weights(checkpoints) will be stored as training progresses.","metadata":{"execution":{"iopub.status.busy":"2023-05-17T05:51:44.607581Z","iopub.execute_input":"2023-05-17T05:51:44.607949Z","iopub.status.idle":"2023-05-17T05:51:44.616561Z","shell.execute_reply.started":"2023-05-17T05:51:44.607927Z","shell.execute_reply":"2023-05-17T05:51:44.614338Z"}}},{"cell_type":"code","source":"data_extention_file_path = os.path.join(cfg_dir, 'rsna.data')\nwith open(data_extention_file_path, 'w') as f:\n    contents = \"\"\"classes= 1\ntrain  = {}\nvalid  = {}\nnames  = {}\nbackup = {}\n    \"\"\".format(os.path.join(metadata_dir, \"tr_list.txt\"),\n               os.path.join(metadata_dir, \"val_list.txt\"),\n               os.path.join(cfg_dir, 'rsna.names'),\n               backup_dir)\n    f.write(contents)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat cfg/rsna.data","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Label list of bounding box.\n!echo \"pneumonia\" > cfg/rsna.names","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For training, we would download the pre-trained model weights(darknet53.conv.74) using following wget command. I recommend you to use this pre-trained weight too. Author of darknet also uses this pre-trained weights in different fields of image recognition.","metadata":{}},{"cell_type":"code","source":"!wget -q https://pjreddie.com/media/files/darknet53.conv.74","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Basically, you can use `darknet/cfg/yolov3.cfg` files. However it won't work for RSNA. you need to edit for RSNA. You can just download a cfg file I edited for RSNA with following wget command. I refer to the following articles for editing cfg files","metadata":{}},{"cell_type":"code","source":"!wget --no-check-certificate -q \"https://docs.google.com/uc?export=download&id=18ptTK4Vbeokqpux8Onr0OmwUP9ipmcYO\" -O cfg/rsna_yolov3.cfg_train","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!sed -i 's/learning_rate=0.001/learning_rate=0.01/g' cfg/rsna_yolov3.cfg_train","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!sed -i 's/max_batches = 500000/max_batches = 1000/g' cfg/rsna_yolov3.cfg_train","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!grep -E 'max_batches|learning_rate' cfg/rsna_yolov3.cfg_train","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!./darknet_gpu detector train cfg/rsna.data cfg/rsna_yolov3.cfg_train darknet53.conv.74 -i 0 | tee train_log.txt","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"iters = []\nlosses = []\ntotal_losses = []\nwith open(\"train_log.txt\", 'r') as f:\n    for i,line in enumerate(f):\n        if \"images\" in line:\n            iters.append(int(line.strip().split()[0].split(\":\")[0]))\n            losses.append(float(lzine.strip().split()[2]))        \n            total_losses.append(float(line.strip().split()[1].split(',')[0]))\n\nplt.figure(figsize=(20, 5))\nplt.subplot(1,2,1)\nsns.lineplot(iters, total_losses, label=\"totla loss\")\nsns.lineplot(iters, losses, label=\"avg loss\")\nplt.xlabel(\"Iteration\")\nplt.ylabel(\"Loss\")\n\nplt.subplot(1,2,2)\nsns.lineplot(iters, total_losses, label=\"totla loss\")\nsns.lineplot(iters, losses, label=\"avg loss\")\nplt.xlabel(\"Iteration\")\nplt.ylabel(\"Loss\")\nplt.ylim([0, 4.05])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}],"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}}