{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# EfficientDet\n\nARXIV [https://arxiv.org/pdf/1911.09070.pdf](https://arxiv.org/pdf/1911.09070.pdf)  \nGithub [https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch](https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch)  \n\nEfficientDet employs EfficientNet as the backbone network, BiFPN as the feature network, and shared class/box prediction network. Both BiFPN layers and class/box net layers are repeated multiple times based on different resource constraints.\n\n[Object Detection SOTA model](https://paperswithcode.com/sota/object-detection-on-coco)  \nThis page shows object detection models' score on COCO test-dev. I focus AP50 score, because this competitions' metric is AP50.    \n1. DyHead (Based Swin-L) : 78.5\n2. DetectoRS (Based ResNeXt) : 74.2\n3. YOLOv4-P7 (Based Scaled-YOLO) : 73.3\n4. EfficientDet-D7 (Based EfficientNet) : 72.4\n5. YOLOv4-608 (Based YOLO) : 65.7\n\n<figure>\n<img src=\"https://blog.roboflow.com/content/images/2020/06/yolov5-performance.png\" style=\"width:700px\">\n    <figcaption>EfficientDet is better model than YOLOv5 on AP.</figcaption>\n</figure>\n\n\nEveryone used model based YOLOv4 or YOLOv5, but this model isn't SOTA model. I'll try EfficientDet first and then Scaled-YOLOv4, DyHead.  \n  \n### Three notebooks summarize how to use this model.\n#### [Preprocessing](https://www.kaggle.com/adldotori/efficientdet-preprocessing-better-than-yolov5/)\n#### [Training](https://www.kaggle.com/adldotori/efficientdet-training-better-than-yolov5/)\n#### [Inference](https://www.kaggle.com/adldotori/efficientdet-inference-better-than-yolov5/)\n\nThis notebook is third notebook which includes how to inference EfficientDet.  \nLet's start!  \n\nThis picture shows the rough structure of efficientdet.\n![image](https://aihub-storage.s3.ap-northeast-2.amazonaws.com/file/efficientdet.png)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"markdown","source":"# Environment","metadata":{}},{"cell_type":"code","source":"import os\n!cp -r '/kaggle/input/cocoapi' '/kaggle/'\nos.chdir('/kaggle/cocoapi/cocoapi/PythonAPI')\n!make\n!make install\n!python setup.py install","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-25T12:04:43.022139Z","iopub.execute_input":"2021-07-25T12:04:43.022821Z","iopub.status.idle":"2021-07-25T12:05:05.025211Z","shell.execute_reply.started":"2021-07-25T12:04:43.022698Z","shell.execute_reply":"2021-07-25T12:05:05.024242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!conda install '/kaggle/input/pydicom-conda-helper/libjpeg-turbo-2.1.0-h7f98852_0.tar.bz2' -c conda-forge -y\n!conda install '/kaggle/input/pydicom-conda-helper/libgcc-ng-9.3.0-h2828fa1_19.tar.bz2' -c conda-forge -y\n!conda install '/kaggle/input/pydicom-conda-helper/gdcm-2.8.9-py37h500ead1_1.tar.bz2' -c conda-forge -y\n!conda install '/kaggle/input/pydicom-conda-helper/conda-4.10.1-py37h89c1867_0.tar.bz2' -c conda-forge -y\n!conda install '/kaggle/input/pydicom-conda-helper/certifi-2020.12.5-py37h89c1867_1.tar.bz2' -c conda-forge -y\n!conda install '/kaggle/input/pydicom-conda-helper/openssl-1.1.1k-h7f98852_0.tar.bz2' -c conda-forge -y\n!conda install '/kaggle/input/notebook6765ef7aa0/webcolors-1.11.1-pyhd8ed1ab_0.tar.bz2' -c conda-forge -y","metadata":{"execution":{"iopub.status.busy":"2021-07-25T12:05:05.027055Z","iopub.execute_input":"2021-07-25T12:05:05.027383Z","iopub.status.idle":"2021-07-25T12:06:23.214993Z","shell.execute_reply.started":"2021-07-25T12:05:05.027345Z","shell.execute_reply":"2021-07-25T12:06:23.21401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Move Directory","metadata":{}},{"cell_type":"code","source":"!cp -r /kaggle/input/efficientdet-training-better-than-yolov5/Yet-Another-EfficientDet-Pytorch /kaggle/","metadata":{"execution":{"iopub.status.busy":"2021-07-25T12:06:23.216819Z","iopub.execute_input":"2021-07-25T12:06:23.217125Z","iopub.status.idle":"2021-07-25T12:07:45.510009Z","shell.execute_reply.started":"2021-07-25T12:06:23.21709Z","shell.execute_reply":"2021-07-25T12:07:45.508806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir('/kaggle/Yet-Another-EfficientDet-Pytorch/')","metadata":{"execution":{"iopub.status.busy":"2021-07-25T12:07:45.51187Z","iopub.execute_input":"2021-07-25T12:07:45.512218Z","iopub.status.idle":"2021-07-25T12:07:45.520728Z","shell.execute_reply.started":"2021-07-25T12:07:45.512187Z","shell.execute_reply":"2021-07-25T12:07:45.519857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preprocessing Test Image","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nfrom glob import glob\n\nif pd.read_csv('/kaggle/input/siim-covid19-detection/sample_submission.csv').shape[0] == 2477:\n    fast_sub = True\n    \n    INPUT_PATH = 'datasets/siim/tmp/'\n    \n    # move just 5 folders\n    sample = os.listdir('/kaggle/input/siim-covid19-detection/test/')[:5]\n    os.makedirs(INPUT_PATH, exist_ok=True)\n    for i in sample:\n        !cp -r '/kaggle/input/siim-covid19-detection/test/'{i} {INPUT_PATH}\n    \n    tmp = []\n    for path in glob(INPUT_PATH + '*'):\n        study_name = path.split('/')[-1][:-4]\n        tmp.append([f'{study_name}_study', 'negative 1 0 0 1 1'])\n\n    for path in glob(INPUT_PATH + '*/*/*'):\n        image_name = path.split('/')[-1][:-4]\n        tmp.append([f'{image_name}_image', 'none 1 0 0 1 1'])\n\n    submission = pd.DataFrame(tmp, columns=['id', 'PredictionString'])\n\nelse:\n    fast_sub = False\n    INPUT_PATH = '/kaggle/input/siim-covid19-detection/test/'\n    submission = pd.read_csv('/kaggle/input/siim-covid19-detection/sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2021-07-25T12:07:45.521883Z","iopub.execute_input":"2021-07-25T12:07:45.52235Z","iopub.status.idle":"2021-07-25T12:07:50.332199Z","shell.execute_reply.started":"2021-07-25T12:07:45.522313Z","shell.execute_reply":"2021-07-25T12:07:50.331155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nfrom PIL import Image\nimport pandas as pd\nfrom tqdm.auto import tqdm\nimport numpy as np\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\n\nimport torch\n\ndef read_xray(path, voi_lut=False, fix_monochrome=True):\n    # Original from: https://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way\n    dicom = pydicom.read_file(path)\n    # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to\n    # \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n\n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n\n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n\n    return data\n\n    \ndef resize(array, size, keep_ratio=False, resample=Image.LANCZOS):\n    # Original from: https://www.kaggle.com/xhlulu/vinbigdata-process-and-resize-to-image\n    im = Image.fromarray(array)\n\n    if keep_ratio:\n        im.thumbnail((size, size), resample)\n    else:\n        im = im.resize((size, size), resample)\n\n    return im","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:20:16.590799Z","iopub.execute_input":"2021-07-24T11:20:16.591476Z","iopub.status.idle":"2021-07-24T11:20:17.988384Z","shell.execute_reply.started":"2021-07-24T11:20:16.59143Z","shell.execute_reply":"2021-07-24T11:20:17.987531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from glob import glob\n\nsave_dir = \"datasets/siim/test\"\n!rm -rf {save_dir}\nos.makedirs(save_dir, exist_ok=True)\n\nimage_size_dict = {}\nfor path in tqdm(glob(INPUT_PATH + '*/*/*')):\n    # set keep_ratio=True to have original aspect ratio\n    xray = read_xray(path)\n    image_size_dict[path.split('/')[-1][:-4]] = xray.shape\n    im = resize(xray, size=256)\n    im.save(os.path.join(save_dir, path.split('/')[-1][:-3]+'jpg'))","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:20:17.989646Z","iopub.execute_input":"2021-07-24T11:20:17.990013Z","iopub.status.idle":"2021-07-24T11:20:21.032619Z","shell.execute_reply.started":"2021-07-24T11:20:17.989977Z","shell.execute_reply":"2021-07-24T11:20:21.031581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission[submission.id.str.contains('image')]","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:20:21.035326Z","iopub.execute_input":"2021-07-24T11:20:21.035688Z","iopub.status.idle":"2021-07-24T11:20:21.063131Z","shell.execute_reply.started":"2021-07-24T11:20:21.035646Z","shell.execute_reply":"2021-07-24T11:20:21.062143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Evaluation","metadata":{"execution":{"iopub.status.busy":"2021-07-24T09:21:08.861635Z","iopub.execute_input":"2021-07-24T09:21:08.861965Z","iopub.status.idle":"2021-07-24T09:21:08.882134Z","shell.execute_reply.started":"2021-07-24T09:21:08.861931Z","shell.execute_reply":"2021-07-24T09:21:08.88119Z"}}},{"cell_type":"code","source":"weight_file = !ls logs/siim/\n! python coco_eval.py -c 0 -p siim -w \"logs/siim/{weight_file[-2]}\"","metadata":{"execution":{"iopub.status.busy":"2021-07-24T09:21:11.031731Z","iopub.execute_input":"2021-07-24T09:21:11.032078Z","iopub.status.idle":"2021-07-24T09:23:00.487263Z","shell.execute_reply.started":"2021-07-24T09:21:11.032043Z","shell.execute_reply":"2021-07-24T09:23:00.486347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# TEST\n## Setting","metadata":{"execution":{"iopub.status.busy":"2021-07-24T09:37:27.267059Z","iopub.execute_input":"2021-07-24T09:37:27.26742Z","iopub.status.idle":"2021-07-24T09:37:27.944411Z","shell.execute_reply.started":"2021-07-24T09:37:27.267383Z","shell.execute_reply":"2021-07-24T09:37:27.943457Z"}}},{"cell_type":"code","source":"name2color = {\n    'Negative' : [0, 0, 0], # Typical Appearance\n    'Typical': [66,9,255], # Typical Appearance\n    'Indeterminate': [255,186,8], # Indeterminate Appearance\n    'Atypical': [247,37,69], # Atypical Appearance\n}\n\n\"\"\"\nSimple Inference Script of EfficientDet-Pytorch\n\"\"\"\nimport time\nimport torch\nfrom torch.backends import cudnn\nfrom matplotlib import colors\n\nfrom backbone import EfficientDetBackbone\nimport cv2\nimport numpy as np\nimport matplotlib.pyplot as plt\n\nfrom efficientdet.utils import BBoxTransform, ClipBoxes\nfrom utils.utils import preprocess, invert_affine, postprocess, STANDARD_COLORS, standard_to_bgr, get_index_label, plot_one_box\n","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:20:21.064587Z","iopub.execute_input":"2021-07-24T11:20:21.064945Z","iopub.status.idle":"2021-07-24T11:20:21.377866Z","shell.execute_reply.started":"2021-07-24T11:20:21.064894Z","shell.execute_reply":"2021-07-24T11:20:21.377029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model Upload","metadata":{}},{"cell_type":"code","source":"compound_coef = 0\nforce_input_size = None  # set None to use default size\nimg_path = 'datasets/siim/test/'\nimg_list = glob(img_path + '*')\n\n# replace this part with your project's anchor config\nanchor_ratios = [(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]\nanchor_scales = [2 ** 0, 2 ** (1.0 / 3.0), 2 ** (2.0 / 3.0)]\n\nthreshold = 0.2\niou_threshold = 0.2\n\nuse_cuda = True\nuse_float16 = False\ncudnn.fastest = True\ncudnn.benchmark = True\n\nobj_list = ['Typical', 'Indeterminate', 'Atypical']\n\n\ncolor_list = standard_to_bgr(STANDARD_COLORS)\n# tf bilinear interpolation is different from any other's, just make do\ninput_sizes = [512, 640, 768, 896, 1024, 1280, 1280, 1536, 1536]\ninput_size = input_sizes[compound_coef] if force_input_size is None else force_input_size\n\nmodel = EfficientDetBackbone(compound_coef=compound_coef, num_classes=len(obj_list),\n                             ratios=anchor_ratios, scales=anchor_scales)\nmodel.load_state_dict(torch.load('logs/siim/efficientdet-d0_39_12600.pth', map_location='cpu'))\nmodel.requires_grad_(False)\nmodel.eval()\n\nif use_cuda:\n    model = model.cuda()\nif use_float16:\n    model = model.half()","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:24:37.617431Z","iopub.execute_input":"2021-07-24T11:24:37.617804Z","iopub.status.idle":"2021-07-24T11:24:37.887327Z","shell.execute_reply.started":"2021-07-24T11:24:37.61777Z","shell.execute_reply":"2021-07-24T11:24:37.886381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Display","metadata":{}},{"cell_type":"code","source":"def display(preds, imgs):\n    for i in range(len(imgs)):\n        if len(preds[i]['rois']) == 0:\n            continue\n\n        imgs[i] = imgs[i].copy()\n\n        for j in range(len(preds[i]['rois'])):\n            x1, y1, x2, y2 = preds[i]['rois'][j].astype(np.int)\n            obj = obj_list[preds[i]['class_ids'][j]]\n            cv2.rectangle(ori_imgs[i], (x1, y1), (x2, y2), name2color[obj], 2)\n            score = float(out[i]['scores'][j])\n\n            cv2.putText(ori_imgs[i], '{:.3f}'.format(score),\n                        (x1, y1 + 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5,\n                        (255,255,255), 1)\n        return imgs[i]\n    \nimgs = []\nfor k, img_path in enumerate(img_list[:5]):\n    ori_imgs, framed_imgs, framed_metas = preprocess(img_path, max_size=input_size)\n\n    if use_cuda:\n        x = torch.stack([torch.from_numpy(fi).cuda() for fi in framed_imgs], 0)\n    else:\n        x = torch.stack([torch.from_numpy(fi) for fi in framed_imgs], 0)\n\n    x = x.to(torch.float32 if not use_float16 else torch.float16).permute(0, 3, 1, 2)\n\n    with torch.no_grad():\n        features, regression, classification, anchors = model(x)\n\n        regressBoxes = BBoxTransform()\n        clipBoxes = ClipBoxes()\n\n        out = postprocess(x,\n                          anchors, regression, classification,\n                          regressBoxes, clipBoxes,\n                          threshold, iou_threshold)\n\n    out = invert_affine(framed_metas, out)\n    imgs.append(display(out, ori_imgs))\n    \nfig = plt.figure(figsize=(5, len(imgs)*5))\nfor i, img in enumerate(imgs[:30]):\n    img = cv2.resize(img, (300,300))\n    ax = fig.add_subplot(len(imgs), 1, i+1)\n    ax.get_xaxis().set_visible(False)\n    ax.get_yaxis().set_visible(False)\n    plt.imshow(img)","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:24:51.331646Z","iopub.execute_input":"2021-07-24T11:24:51.332124Z","iopub.status.idle":"2021-07-24T11:24:52.256851Z","shell.execute_reply.started":"2021-07-24T11:24:51.332079Z","shell.execute_reply":"2021-07-24T11:24:52.255893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"blue box : typical  \nyellow box : indeterminate  \nred box : atypical  ","metadata":{}},{"cell_type":"markdown","source":"![image](https://aihub-storage.s3.ap-northeast-2.amazonaws.com/file/%E1%84%89%E1%85%B3%E1%84%8F%E1%85%B3%E1%84%85%E1%85%B5%E1%86%AB%E1%84%89%E1%85%A3%E1%86%BA_2021-07-18_%E1%84%8B%E1%85%A9%E1%84%92%E1%85%AE_10.53.32.png)","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:23:27.965725Z","iopub.execute_input":"2021-07-24T11:23:27.966097Z","iopub.status.idle":"2021-07-24T11:23:28.633718Z","shell.execute_reply.started":"2021-07-24T11:23:27.966066Z","shell.execute_reply":"2021-07-24T11:23:28.632758Z"}}},{"cell_type":"markdown","source":"## Inference","metadata":{}},{"cell_type":"code","source":"pcs_imgs = []\nfor k, img_path in enumerate(img_list):\n    ori_imgs, framed_imgs, framed_metas = preprocess(img_path, max_size=input_size)\n\n    if use_cuda:\n        x = torch.stack([torch.from_numpy(fi).cuda() for fi in framed_imgs], 0)\n    else:\n        x = torch.stack([torch.from_numpy(fi) for fi in framed_imgs], 0)\n\n    x = x.to(torch.float32 if not use_float16 else torch.float16).permute(0, 3, 1, 2)\n    \n    pcs_imgs.append(x)\n    with torch.no_grad():\n        features, regression, classification, anchors = model(x)\n\n        regressBoxes = BBoxTransform()\n        clipBoxes = ClipBoxes()\n\n        out = postprocess(x,\n                          anchors, regression, classification,\n                          regressBoxes, clipBoxes,\n                          threshold, iou_threshold)\n\n    out = invert_affine(framed_metas, out)\n    imgs.append(display(out, ori_imgs))","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:38:02.147806Z","iopub.execute_input":"2021-07-24T11:38:02.148142Z","iopub.status.idle":"2021-07-24T11:38:02.425888Z","shell.execute_reply.started":"2021-07-24T11:38:02.148111Z","shell.execute_reply":"2021-07-24T11:38:02.425035Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"batch size is 32.","metadata":{}},{"cell_type":"code","source":"outs = []\nbatch = 32\nwith torch.no_grad():\n    for i in range(len(pcs_imgs)//batch + 1):\n        features, regression, classification, anchors = model(torch.cat(pcs_imgs[batch*i:batch*(i+1)]))\n\n        regressBoxes = BBoxTransform()\n        clipBoxes = ClipBoxes()\n\n        out = postprocess(torch.cat(pcs_imgs[batch*i:batch*(i+1)]),\n                          anchors, regression, classification,\n                          regressBoxes, clipBoxes,\n                          threshold, iou_threshold)\n        outs += out","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:40:24.946192Z","iopub.execute_input":"2021-07-24T11:40:24.946524Z","iopub.status.idle":"2021-07-24T11:40:25.006389Z","shell.execute_reply.started":"2021-07-24T11:40:24.946493Z","shell.execute_reply":"2021-07-24T11:40:25.005573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outs[0]","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:56:34.778321Z","iopub.execute_input":"2021-07-24T11:56:34.778667Z","iopub.status.idle":"2021-07-24T11:56:34.788566Z","shell.execute_reply.started":"2021-07-24T11:56:34.778634Z","shell.execute_reply":"2021-07-24T11:56:34.785957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Result looks like this. \n  \n**rois** : box position  \n**class_ids** : 0 => typical, 1 => indeterminate, 2=> atypical  \n**scores** : confidence  ","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:57:35.831004Z","iopub.execute_input":"2021-07-24T11:57:35.83132Z","iopub.status.idle":"2021-07-24T11:57:35.837491Z","shell.execute_reply.started":"2021-07-24T11:57:35.831293Z","shell.execute_reply":"2021-07-24T11:57:35.836076Z"}}},{"cell_type":"code","source":"# for idx, name in enumerate(tqdm(img_list)):\n#     name = name.split('/')[-1][:-4]\n#     img_size = image_size_dict[name]\n#     data_id = submission[submission.id==name+'_image'].index\n#     ans = \"\"\n#     for label in range(len(outs[idx]['rois'])):\n#         xmin, ymin, xmax, ymax = outs[idx]['rois'][label]\n#         confidence = outs[idx]['scores'][label]\n#         xmin = int(xmin * img_size[1]/512)\n#         xmax = int(xmax * img_size[1]/512)\n#         ymin = int(ymin * img_size[0]/512)\n#         ymax = int(ymax * img_size[0]/512)\n#         ans += f\"opacity {confidence} {xmin} {ymin} {xmax} {ymax} \"\n#     submission.loc[data_id, 'PredictionString'] = ans","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:52:11.679195Z","iopub.execute_input":"2021-07-24T11:52:11.679518Z","iopub.status.idle":"2021-07-24T11:52:11.730509Z","shell.execute_reply.started":"2021-07-24T11:52:11.679488Z","shell.execute_reply":"2021-07-24T11:52:11.729579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv('/kaggle/working/submission.csv', index=False)  ","metadata":{"execution":{"iopub.status.busy":"2021-07-24T11:56:24.805938Z","iopub.execute_input":"2021-07-24T11:56:24.806244Z","iopub.status.idle":"2021-07-24T11:56:24.814019Z","shell.execute_reply.started":"2021-07-24T11:56:24.806217Z","shell.execute_reply":"2021-07-24T11:56:24.813118Z"},"trusted":true},"execution_count":null,"outputs":[]}]}