{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### thanks reference\n\n- [Sartorius - Cell Instance_Starter](https://www.kaggle.com/drcapa/sartorius-cell-instance-starter)","metadata":{}},{"cell_type":"markdown","source":"## Library","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport time\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport cv2\n","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.242055Z","iopub.execute_input":"2021-10-18T12:19:31.242344Z","iopub.status.idle":"2021-10-18T12:19:31.247278Z","shell.execute_reply.started":"2021-10-18T12:19:31.242313Z","shell.execute_reply":"2021-10-18T12:19:31.246298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path = \"../input/sartorius-cell-instance-segmentation/\"\nos.listdir(path)","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.249119Z","iopub.execute_input":"2021-10-18T12:19:31.249725Z","iopub.status.idle":"2021-10-18T12:19:31.266445Z","shell.execute_reply.started":"2021-10-18T12:19:31.249692Z","shell.execute_reply":"2021-10-18T12:19:31.265462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The training annotations are provided as run length encoded masks, and the images are in PNG format. The number of images is small, but the number of annotated objects is quite high. The hidden test set is roughly 240 images.","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv(path + 'train.csv')\nsubmission = pd.read_csv(path + 'sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.268217Z","iopub.execute_input":"2021-10-18T12:19:31.268573Z","iopub.status.idle":"2021-10-18T12:19:31.568837Z","shell.execute_reply.started":"2021-10-18T12:19:31.268523Z","shell.execute_reply":"2021-10-18T12:19:31.568074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### train\n- id - unique identifier for object\n- annotation - run length encoded pixels for the identified neuronal cell\n- width - source image width\n- height - source image height\n- cell_type - the cell line\n- plate_time - time plate was created\n- sample_date - date sample was created\n- sample_id - sample identifier\n- elapsed_timedelta - time since first image taken of sample","metadata":{}},{"cell_type":"code","source":"train","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.570683Z","iopub.execute_input":"2021-10-18T12:19:31.571491Z","iopub.status.idle":"2021-10-18T12:19:31.592061Z","shell.execute_reply.started":"2021-10-18T12:19:31.571448Z","shell.execute_reply":"2021-10-18T12:19:31.590940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.593191Z","iopub.execute_input":"2021-10-18T12:19:31.593419Z","iopub.status.idle":"2021-10-18T12:19:31.604864Z","shell.execute_reply.started":"2021-10-18T12:19:31.593388Z","shell.execute_reply":"2021-10-18T12:19:31.603809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Domain knowledge\n\ncell line\n\nAtrocycle\n- Astrocyte, star-shaped cell that is a type of neuroglia found in the nervous system in both invertebrates and vertebrates.Astrocyte fix neuron and supply nutrition to nueron.\n\nGlioblastoma\n- Glioblastoma is an aggressive type of cancer that can occur in the brain or spinal cord","metadata":{}},{"cell_type":"markdown","source":"## EDA","metadata":{}},{"cell_type":"markdown","source":"### train data","metadata":{}},{"cell_type":"markdown","source":"##### show height ( all 520) / width(all 704)","metadata":{}},{"cell_type":"code","source":"print(train['height'].value_counts())\nprint(train['width'].value_counts())","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.606167Z","iopub.execute_input":"2021-10-18T12:19:31.606400Z","iopub.status.idle":"2021-10-18T12:19:31.621304Z","shell.execute_reply.started":"2021-10-18T12:19:31.606369Z","shell.execute_reply":"2021-10-18T12:19:31.620361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### utils functoin(countplot)","metadata":{}},{"cell_type":"code","source":"def util_conutplot(columns, input_data):\n    fig, ax = plt.subplots(figsize = (20, 8))\n    plt.subplot(2, 2, 1)\n    sns.countplot(x=columns, data=input_data)\n    train[columns].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.622773Z","iopub.execute_input":"2021-10-18T12:19:31.623007Z","iopub.status.idle":"2021-10-18T12:19:31.634905Z","shell.execute_reply.started":"2021-10-18T12:19:31.622978Z","shell.execute_reply":"2021-10-18T12:19:31.634095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### distribution cell type / id / plate_time / sample_data . elapsed_timedelta","metadata":{}},{"cell_type":"code","source":"column_list = ['cell_type','id' ,'plate_time' , 'sample_date', 'elapsed_timedelta']\nfor _, column in enumerate(column_list):\n    util_conutplot(column, train)","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:31.636518Z","iopub.execute_input":"2021-10-18T12:19:31.636769Z","iopub.status.idle":"2021-10-18T12:19:47.180610Z","shell.execute_reply.started":"2021-10-18T12:19:31.636738Z","shell.execute_reply":"2021-10-18T12:19:47.179631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"row = 0\nid_ = train.loc[row, 'id']\nfile = id_+'.png'\nprint(file)\nprint(\"id {} 1 id contain {} images\".format(file,len(train[train['id']==id_])))","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:47.182909Z","iopub.execute_input":"2021-10-18T12:19:47.183136Z","iopub.status.idle":"2021-10-18T12:19:47.205092Z","shell.execute_reply.started":"2021-10-18T12:19:47.183107Z","shell.execute_reply":"2021-10-18T12:19:47.204157Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"if change row ex)400 id (0140b3c8f445) has 108 images , so each file hold different numbers of images.","metadata":{}},{"cell_type":"markdown","source":"#### Look image file","metadata":{}},{"cell_type":"code","source":"img = cv2.imread(path+'train/'+file)\nprint('Image shape:', img.shape)","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:47.206319Z","iopub.execute_input":"2021-10-18T12:19:47.206604Z","iopub.status.idle":"2021-10-18T12:19:47.223870Z","shell.execute_reply.started":"2021-10-18T12:19:47.206571Z","shell.execute_reply":"2021-10-18T12:19:47.222945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(1, 1, figsize=(7, 7))\naxs.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))\naxs.set_xticklabels([])\naxs.set_yticklabels([])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:47.225095Z","iopub.execute_input":"2021-10-18T12:19:47.225352Z","iopub.status.idle":"2021-10-18T12:19:47.518184Z","shell.execute_reply.started":"2021-10-18T12:19:47.225323Z","shell.execute_reply":"2021-10-18T12:19:47.517201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ref https://www.kaggle.com/ihelon/cell-segmentation-run-length-decoding\ndef rle_decode(mask_rle, shape, color=1):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height, width, channels) of array to return \n    color: color for the mask\n    Returns numpy array (mask)\n\n    '''\n    s = mask_rle.split()\n    \n    starts =  list(map(lambda x: int(x) - 1, s[0::2]))\n    lengths = list(map(int, s[1::2]))\n    ends = [x + y for x, y in zip(starts, lengths)]\n    \n    img = np.zeros((shape[0] * shape[1], shape[2]), dtype=np.float32)\n    \n    for start, end in zip(starts, ends):\n        img[start : end] = color\n    \n    return img.reshape(shape)","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:19:47.519522Z","iopub.execute_input":"2021-10-18T12:19:47.520372Z","iopub.status.idle":"2021-10-18T12:19:47.531400Z","shell.execute_reply.started":"2021-10-18T12:19:47.520329Z","shell.execute_reply":"2021-10-18T12:19:47.530201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# detail rle_decode(mask_rle, shape, color=1):\nshape=(520, 704, 3)\nimg = np.zeros((shape[0] * shape[1], shape[2]), dtype=np.float32)\nprint(img, img.shape)\n\ncolor = 1\nlabels = train[train['id'] == '0030fd0e6378']['annotation'].tolist()\ns = labels[0].split()\nprint(\"s is \\n{}\".format(s))\n\nstarts =  list(map(lambda x: int(x) - 1, s[0::2]))\nprint(\"start is {}\".format(starts))\n\nlengths = list(map(int, s[1::2]))\nprint(\"length is {}\".format(lengths))\n\nends = [x + y for x, y in zip(starts, lengths)]\nprint(\"ends is {}\".format(ends))\n\nfor start, end in zip(starts, ends):\n    img[start : end] = color\n\nprint(\"img(masked) is \\n{}\".format(img))","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:20:50.081330Z","iopub.execute_input":"2021-10-18T12:20:50.081931Z","iopub.status.idle":"2021-10-18T12:20:50.107252Z","shell.execute_reply.started":"2021-10-18T12:20:50.081888Z","shell.execute_reply":"2021-10-18T12:20:50.106358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_masks(image_id, colors=True):\n    labels = train[train['id'] == image_id]['annotation'].tolist()\n    \n    if colors:\n        mask = np.zeros((520, 704, 3))\n        for label in labels:\n            mask += rle_decode(label, shape=(520, 704, 3), color=np.random.rand(3))\n    else:\n        mask = np.zeros((520, 704, 3))\n        for label in labels:\n            mask += rle_decode(label, shape=(520, 704, 3))\n    \n    mask = mask.clip(0, 1)\n    \n    image = cv2.imread(f\"../input/sartorius-cell-instance-segmentation/train/{image_id}.png\")\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    \n    plt.figure(figsize=(16, 32))\n    plt.subplot(3, 1 , 1)\n    plt.imshow(image)\n    plt.axis(\"off\")\n    plt.figure(figsize=(16, 32))\n    plt.subplot(3, 1 , 2)\n    plt.imshow(image)\n    plt.imshow(mask, alpha=0.4)\n    plt.axis(\"off\")\n    plt.subplot(3, 1 , 3)\n    plt.imshow(image)\n    plt.imshow(mask)\n    plt.axis(\"off\")\n\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:20:50.243080Z","iopub.execute_input":"2021-10-18T12:20:50.243973Z","iopub.status.idle":"2021-10-18T12:20:50.257163Z","shell.execute_reply.started":"2021-10-18T12:20:50.243933Z","shell.execute_reply":"2021-10-18T12:20:50.256032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_masks('0030fd0e6378', colors=True)","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:20:50.399660Z","iopub.execute_input":"2021-10-18T12:20:50.399927Z","iopub.status.idle":"2021-10-18T12:20:52.293965Z","shell.execute_reply.started":"2021-10-18T12:20:50.399897Z","shell.execute_reply":"2021-10-18T12:20:52.292988Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_masks('0140b3c8f445', colors=True)","metadata":{"execution":{"iopub.status.busy":"2021-10-18T12:20:52.296147Z","iopub.execute_input":"2021-10-18T12:20:52.296475Z","iopub.status.idle":"2021-10-18T12:20:53.536325Z","shell.execute_reply.started":"2021-10-18T12:20:52.296429Z","shell.execute_reply":"2021-10-18T12:20:53.535247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### train","metadata":{}},{"cell_type":"markdown","source":"#### LIVECell_dataset","metadata":{}},{"cell_type":"markdown","source":"LIVECell_dataset_2021 - A mirror of the data from the LIVECell dataset. LIVECell is the predecessor dataset to this competition. You will find extra data for the SH-SHY5Y cell line, plus several other cell lines not covered in the competition dataset that may be of interest for transfer learning.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}