{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Custom U-Net model","metadata":{}},{"cell_type":"markdown","source":"Using a custom u-net model to extract the required features from dataset to draw out accurate masking image. My idea is to create a modified encoder insipired by mobile-net V2 as it is light weight and will be faster yet accurate to extract required features from data. As for decoding part is will be similar to u-net with some minor changes to optimize the outcome.","metadata":{"execution":{"iopub.status.busy":"2023-08-19T15:46:49.226840Z","iopub.execute_input":"2023-08-19T15:46:49.227676Z","iopub.status.idle":"2023-08-19T15:46:49.236637Z","shell.execute_reply.started":"2023-08-19T15:46:49.227631Z","shell.execute_reply":"2023-08-19T15:46:49.234892Z"}}},{"cell_type":"markdown","source":"## 1. Importing Libraries\n\n> In this section, we are importing various Python libraries that will be utilized throughout the code. These libraries provide powerful tools and functions for tasks such as:\n\n- Image processing\n- Data manipulation\n- Visualization\n- Deep learning","metadata":{}},{"cell_type":"code","source":"pip install monai","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:21:36.676644Z","iopub.execute_input":"2023-09-09T14:21:36.676965Z","iopub.status.idle":"2023-09-09T14:21:54.567531Z","shell.execute_reply.started":"2023-09-09T14:21:36.676937Z","shell.execute_reply":"2023-09-09T14:21:54.566345Z"},"_kg_hide-input":true,"_kg_hide-output":true,"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pip install torch.summary","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:21:54.570622Z","iopub.execute_input":"2023-09-09T14:21:54.571246Z","iopub.status.idle":"2023-09-09T14:22:07.079676Z","shell.execute_reply.started":"2023-09-09T14:21:54.571205Z","shell.execute_reply":"2023-09-09T14:22:07.078282Z"},"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import zipfile\nfrom PIL import Image\nimport io\nimport csv\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport torch\nfrom torch import nn\nfrom torch import tensor\nfrom torchinfo import summary\nimport torch.nn.functional as F\n","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:07.081888Z","iopub.execute_input":"2023-09-09T14:22:07.082358Z","iopub.status.idle":"2023-09-09T14:22:12.363854Z","shell.execute_reply.started":"2023-09-09T14:22:07.082314Z","shell.execute_reply":"2023-09-09T14:22:12.362841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Listing Files in a Directory\n\n> In this code segment, we are working with the `os` library to interact with the file system and list files within a specified directory. The purpose of this code is to traverse through a directory and its subdirectories, printing out the paths of all the files encountered.","metadata":{}},{"cell_type":"code","source":"import os\ndirectory = r'/kaggle/input'\nfor dirname, _, filenames in os.walk(directory):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-09-09T14:22:12.366732Z","iopub.execute_input":"2023-09-09T14:22:12.367338Z","iopub.status.idle":"2023-09-09T14:22:12.382575Z","shell.execute_reply.started":"2023-09-09T14:22:12.367303Z","shell.execute_reply":"2023-09-09T14:22:12.380949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Contained within the ZIP file are images that pique our curiosity. Let's unveil the contents within and embrace the anticipation of discovering the intriguing data that lies in wait.","metadata":{}},{"cell_type":"markdown","source":"## 3. Visualize the Data\n\n> This code snippet illustrates the process of extracting and displaying image and CSV data from either ZIP archives or standalone image files within the specified directory. Images are displayed in a layout, while CSV data is read and printed. The code showcases efficient handling of diverse data formats.","metadata":{}},{"cell_type":"code","source":"image_list = []  \ncsv_list = []\n\nfor dirname, _, filenames in os.walk(directory):\n    for filename in filenames:\n        if filename.endswith(\".pth\"):\n            pass\n        elif filename.endswith(\".zip\"):\n            zipfile_path = os.path.join(dirname, filename)\n            with zipfile.ZipFile(zipfile_path, 'r') as zip_ref:\n                file_list = zip_ref.infolist()\n                #print(\"No. of files inside \",filename,\" - \",len(file_list))\n                for item in file_list:\n                    if item.filename.lower().endswith((\".jpg\", \".jpeg\", \".png\", \".gif\")):\n                        #print(\"-----\",item.filename,\"-----\")\n                        image_data = zip_ref.read(item.filename)\n                        img = Image.open(io.BytesIO(image_data))\n                        #print(img.size)\n                        length = len(file_list)\n                        image_list.append((item.filename, img, length))\n                        #plt.imshow(img)\n                        #plt.title(item.filename)\n                        #plt.axis('off')\n                        #plt.show()\n                        break\n                        \n                    elif item.filename.lower().endswith(\".csv\"):\n                        #print(\"-----\",item.filename,\"-----\")\n                        csv_data_bytes = zip_ref.read(item.filename)\n                        csv_data_text = csv_data_bytes.decode(\"utf-8\")\n                        csv_file_obj = io.StringIO(csv_data_text)\n                        df = pd.read_csv(csv_file_obj)\n                        #print(df.head())\n                        csv_list.append((item.filename, csv_data_text))\n                        break\n                            \n        else:\n            zipfile_path = os.path.join(dirname, filename)\n            image = Image.open(zipfile_path)\n            image_list.append((filename, image, 1))\n            #plt.imshow(image)\n            #plt.title(filename)\n            #plt.axis('off')\n            #plt.show()\n            \nnum_rows = 2 \nnum_cols = (len(image_list) + 1) // num_rows\n\nplt.figure(figsize=(15, 10))\n\nfor i, (img_filename, img, length) in enumerate(image_list, 1):\n    plt.subplot(num_rows, num_cols, i)\n    plt.imshow(img)\n    plt.title(img_filename)\n    plt.axis('off')\n    img_size = img.size\n    plt.annotate(f\"Image Size: {img_size[0]} x {img_size[1]} \\n 1 out of {length}\", xy=(0.5, -0.2), ha='center', fontsize=10, xycoords='axes fraction')\n\nplt.tight_layout()\nplt.show()\n\n# Display CSV data\nfor csv_filename, csv_data_text in csv_list:\n    print(\"\\n -----\", csv_filename, \"----- \\n\")\n    csv_file_obj = io.StringIO(csv_data_text)\n    df = pd.read_csv(csv_file_obj)\n    print(df.head())\n    print(f\"Data Size: {len(df)}\")","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:12.384303Z","iopub.execute_input":"2023-09-09T14:22:12.384697Z","iopub.status.idle":"2023-09-09T14:22:21.172873Z","shell.execute_reply.started":"2023-09-09T14:22:12.384662Z","shell.execute_reply":"2023-09-09T14:22:21.171356Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Examining the contents within the ZIP archives, we find a wealth of data, yet our focus narrows to \"train.zip\" and \"train_mask.zip.\" These specific archives contain corresponding images, forming a valuable set for training. Our intent is to extract and store these ZIP files on the hard drive. This approach optimizes memory usage, as loading images directly from the ZIP files diminishes strain on RAM and GPU resources.","metadata":{}},{"cell_type":"markdown","source":"## 4. Selective Extraction of Training Data\n> This code segment focuses on extracting specific ZIP files, namely \"train.zip\" and \"train_masks.zip,\" from the given directory. Extracted contents are placed at the specified extract_path, optimizing access and facilitating data usage without overloading memory resources.","metadata":{}},{"cell_type":"code","source":"extract_path = '/kaggle/working/'  # Replace this with the path where you want to extract the contents\n\nfor dirname, _, filenames in os.walk(directory):\n    for filename in filenames:\n        if filename in [\"train.zip\", \"train_masks.zip\"]:\n            print(filename)\n            zipfile_path = os.path.join(dirname, filename)\n            with zipfile.ZipFile(zipfile_path, 'r') as zip_ref:\n                zip_ref.extractall(extract_path)\n                print(\"Extraction complete.\")","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:21.174677Z","iopub.execute_input":"2023-09-09T14:22:21.175756Z","iopub.status.idle":"2023-09-09T14:22:30.121069Z","shell.execute_reply.started":"2023-09-09T14:22:21.175719Z","shell.execute_reply":"2023-09-09T14:22:30.119975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5. Visualizing Paired Data\n\n> The code's purpose is to create a visualization that pairs original images with their corresponding masks. This enables a quick and clear understanding of how masks align with their respective images, aiding in data comprehension and analysis for potential applications like image segmentation tasks.","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(nrows=5, ncols=2, figsize=(6, 6))\nfor root, dirs, filenames in os.walk(extract_path):\n    c = 0\n    for filename in filenames:\n        if c >= 5:\n            break\n        if 'train_masks' in root:\n            \n            image_path = os.path.join(root.replace('train_masks', 'train'), filename.split(\"_\")[0]+\"_\"+filename.split(\"_\")[1]+\".jpg\")\n            mask_path = os.path.join(root, filename)\n            \n            img = Image.open(image_path)\n            img_mask = Image.open(mask_path)\n            \n            axes[c][0].imshow(img)\n            axes[c][0].set_title(f\"Non-Masked {filename.split('_')[0]}\")\n            axes[c][0].axis('off')\n            \n            axes[c][1].imshow(img_mask)\n            axes[c][1].set_title(f\"Masked {filename.split('_')[0]}\")\n            axes[c][1].axis('off')\n            \n            c += 1\n\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:30.122686Z","iopub.execute_input":"2023-09-09T14:22:30.123116Z","iopub.status.idle":"2023-09-09T14:22:33.440749Z","shell.execute_reply.started":"2023-09-09T14:22:30.123082Z","shell.execute_reply":"2023-09-09T14:22:33.439854Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> The data appears suitable for loading into the data loader to commence training.Data looks quite fine to load into the data loader to train it.","metadata":{}},{"cell_type":"markdown","source":"## 6. Custom Dataset Class for Paired Images\n\n> This custom dataset class serves as a foundation for efficiently loading paired image data for training and other deep learning tasks. It offers flexibility in handling different types of image data and their relationships.","metadata":{}},{"cell_type":"code","source":"from torch.utils.data import Dataset, DataLoader, Subset\nfrom torchvision import transforms\n\n\npreprocess_X = transforms.Compose([\n            transforms.Resize((512, 512)),\n            transforms.ToTensor(),\n            transforms.Normalize(mean=(0,0,0),std=(1,1,1))])\n\npreprocess_y = transforms.Compose([\n            transforms.Resize((258, 258)),\n            transforms.ToTensor(),\n            transforms.Normalize(mean=(0),std=(1))])\n\nclass CustomDataset(Dataset):\n    def __init__(self, image_paths, mask_paths, transform_X=None, transform_y=None):\n        self.image_paths = image_paths\n        self.mask_paths = mask_paths\n        self.transform_X = transform_X\n        self.transform_y = transform_y\n\n    def __len__(self):\n        return len(self.image_paths)\n\n    def __getitem__(self, idx):\n        image = Image.open(self.image_paths[idx])\n        mask = Image.open(self.mask_paths[idx])\n        \n        if self.transform_X:\n            image = self.transform_X(image)\n        if self.transform_y:\n            mask = self.transform_y(mask)\n        \n        return image, mask","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:33.441974Z","iopub.execute_input":"2023-09-09T14:22:33.443804Z","iopub.status.idle":"2023-09-09T14:22:33.857938Z","shell.execute_reply.started":"2023-09-09T14:22:33.443770Z","shell.execute_reply":"2023-09-09T14:22:33.856931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 6.1 Dataset Preparation with Image Preprocessing and Progress Tracking\n\n> This code efficiently processes paired images and their corresponding masks, applying resizing and transformations. The tqdm library is utilized for progress visualization. The prepared dataset is encapsulated using a custom class for streamlined usage.","metadata":{}},{"cell_type":"code","source":"from tqdm import tqdm\n\ntrain_img_X = []\ntrain_img_y = []\n\nfor root, dirs, filenames in os.walk(extract_path):\n    for filename in tqdm(filenames, desc=\"Processing images\"):\n        if 'train_masks' in root:\n            image_path = os.path.join(root.replace('train_masks', 'train'), filename.split(\"_\")[0] + \"_\" + filename.split(\"_\")[1] + \".jpg\")\n            mask_path = os.path.join(root, filename)\n            \n            train_img_X.append(image_path)\n            train_img_y.append(mask_path)\n\ndataset = CustomDataset(train_img_X, train_img_y, transform_X=preprocess_X, transform_y=preprocess_y)\nprint(\"Dataset processing complete.\")","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-09-09T14:22:33.859475Z","iopub.execute_input":"2023-09-09T14:22:33.859843Z","iopub.status.idle":"2023-09-09T14:22:33.938675Z","shell.execute_reply.started":"2023-09-09T14:22:33.859810Z","shell.execute_reply":"2023-09-09T14:22:33.937794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7. Dataset for Training and Testing\n\n> This section of code divides the dataset into training and testing subsets. It calculates the sizes for both subsets and creates two subsets using the `Subset` class. The training subset contains 90% of the data, while the testing subset holds the remaining 10%. This partitioning allows for effective model training and subsequent evaluation.","metadata":{"execution":{"iopub.status.busy":"2023-08-24T16:46:25.842657Z","iopub.execute_input":"2023-08-24T16:46:25.843968Z","iopub.status.idle":"2023-08-24T16:46:25.851721Z","shell.execute_reply.started":"2023-08-24T16:46:25.843928Z","shell.execute_reply":"2023-08-24T16:46:25.849950Z"}}},{"cell_type":"code","source":"train_size = int(0.9 * len(dataset))\ntest_size = len(dataset) - train_size  \n\ntrain_subset = Subset(dataset, range(train_size))\ntest_subset = Subset(dataset, range(train_size, len(dataset)))","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:33.942931Z","iopub.execute_input":"2023-09-09T14:22:33.943890Z","iopub.status.idle":"2023-09-09T14:22:33.949648Z","shell.execute_reply.started":"2023-09-09T14:22:33.943856Z","shell.execute_reply":"2023-09-09T14:22:33.948719Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 7.1 Data Loading and Batching for Model Training\n\n> This code snippet focuses on loading data for training and testing. It creates data loaders using the `DataLoader` class, ensuring efficient handling of batches during model training. The batch size is set to 16 for both loaders. The snippet showcases the data loader structure and the shapes of input images and output masks, demonstrating the successful configuration of data inputs for training and subsequent usage.","metadata":{}},{"cell_type":"code","source":"batch_size = 4\n\ntrain_loader = DataLoader(train_subset, batch_size=batch_size, shuffle=True)\ntest_loader = DataLoader(test_subset, batch_size=batch_size, shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:33.951636Z","iopub.execute_input":"2023-09-09T14:22:33.952465Z","iopub.status.idle":"2023-09-09T14:22:33.959357Z","shell.execute_reply.started":"2023-09-09T14:22:33.952431Z","shell.execute_reply":"2023-09-09T14:22:33.958721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_X_batch, train_y_batch = next(iter(train_loader))\nprint(train_X_batch.shape, train_y_batch.shape)\nprint(f\"Length of train dataloader: {len(train_loader)}\")","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:33.960897Z","iopub.execute_input":"2023-09-09T14:22:33.961522Z","iopub.status.idle":"2023-09-09T14:22:34.290044Z","shell.execute_reply.started":"2023-09-09T14:22:33.961491Z","shell.execute_reply":"2023-09-09T14:22:34.288723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for batch in train_loader:\n    image_x_batch, image_y_batch = batch\n    break\n    \nprint(\"Input Image shape: \",image_x_batch.shape)\nprint(\"Output Mask shape: \",image_y_batch.shape)","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:34.291683Z","iopub.execute_input":"2023-09-09T14:22:34.292187Z","iopub.status.idle":"2023-09-09T14:22:34.467469Z","shell.execute_reply.started":"2023-09-09T14:22:34.292146Z","shell.execute_reply":"2023-09-09T14:22:34.466413Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 7.2 Image and Mask Visualization\n\n> This part of the code focuses on visualizing a randomly selected pair of images and their corresponding masks from the training dataset. It demonstrates the actual content of the data. ","metadata":{}},{"cell_type":"code","source":"random_idx = torch.randint(0, len(train_X_batch), size=[1]).item()\n#print(random_idx)\nimg_X, img_y = train_X_batch[random_idx], train_y_batch[random_idx]\nfig,axes = plt.subplots(1, 2)\nimg_array = img_X.permute(1, 2, 0).cpu().numpy()\naxes[0].imshow(img_array)\naxes[0].set_title('Original Image')\naxes[0].axis(\"Off\")\nmask_array = img_y.permute(1, 2, 0).cpu().numpy()\naxes[1].imshow(mask_array, cmap=\"gray\")\naxes[1].set_title('Masked Image')\naxes[1].axis(\"Off\");\nprint(f\"Image size: {img_X.shape}\")\nprint(f\"Masked image size: {img_y.shape}\")","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:34.469134Z","iopub.execute_input":"2023-09-09T14:22:34.469796Z","iopub.status.idle":"2023-09-09T14:22:34.839443Z","shell.execute_reply.started":"2023-09-09T14:22:34.469758Z","shell.execute_reply":"2023-09-09T14:22:34.838257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### Device agnostic code","metadata":{}},{"cell_type":"code","source":"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:34.841122Z","iopub.execute_input":"2023-09-09T14:22:34.841551Z","iopub.status.idle":"2023-09-09T14:22:34.911457Z","shell.execute_reply.started":"2023-09-09T14:22:34.841515Z","shell.execute_reply":"2023-09-09T14:22:34.910377Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 8. Custom MobileNetV2 U-Net Model\n\n> This custom model, named \"MobileNetV2 U-Net,\" combines the architecture of MobileNetV2 with the U-Net framework to facilitate semantic segmentation tasks. The model is designed for input images of size 224x224 pixels.\n\n#### Encoder (MobileNetV2):\n\n> The model employs the MobileNetV2 architecture as its encoder. The encoder's initial convolutional layer reduces the input image's dimensions, transforming it into a feature map. It further employs bottlenecks and depthwise separable convolutions for efficient feature extraction. One of the bottlenecks, namely \"Bottleneck_6,\" downsamples the feature map to facilitate learning of lower-level features.\n\n#### Decoder (U-net):\n\n> The decoder section of the \"MobileNetV2 U-Net\" model is responsible for transforming the encoded feature maps into meaningful output. This process involves two main steps: upsampling and refining.\n\n**1. Upsampling:**\n> During the upsampling phase, the decoder increases the resolution of the feature maps obtained from the encoder. This step aims to recover the finer details of the original input image. Transpose convolutional layers are used to perform this operation, effectively \"blowing up\" the feature maps while retaining important spatial relationships.\n\n**2. Refining and Merging:**\n> After upsampling, the decoder focuses on refining the upscaled feature maps. It achieves this by incorporating skip connections, which connect corresponding levels from the encoder and decoder. These connections enable the decoder to access both low-level and high-level features, combining detailed information with context.\n\n#### Overall Purpose:\n> The decoder serves as the counterpart to the encoder, restoring spatial information and detail to the feature maps. By utilizing skip connections, the decoder effectively merges multi-level features, leading to a more comprehensive and precise understanding of the input image. This combined process of upsampling, refining, and merging enables the model to generate high-quality segmentation masks that accurately highlight objects of interest in the image.","metadata":{}},{"cell_type":"code","source":"class Conv2dNormActivation(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size, stride, padding, groups):\n        super(Conv2dNormActivation, self).__init__()\n        \n        if in_channels == out_channels:\n            self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=kernel_size, stride=stride, padding=padding, groups=groups, bias=False)\n        else:\n            self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=kernel_size, stride=stride, padding=padding, bias=False)\n        self.bn = nn.BatchNorm2d(out_channels)\n        self.relu6 = nn.ReLU6(inplace=True)\n\n    def forward(self, x):\n        x = self.conv(x)\n        x = self.bn(x)\n        x = self.relu6(x)\n        return x\n\nclass InvertedResidual(nn.Module):\n    def __init__(self, in_channels, out_channels, stride, exp):\n        super(InvertedResidual, self).__init__()\n        \n        self.residual_connection = True if stride == 1 and in_channels == out_channels else False\n        \n        if exp == 1:\n            self.conv = nn.Sequential(\n                Conv2dNormActivation(in_channels*exp, in_channels*exp, kernel_size=3, stride=stride, padding=1, groups=in_channels*exp),\n                nn.Conv2d(in_channels*exp, out_channels, kernel_size=1, stride=1, bias=False),\n                nn.BatchNorm2d(out_channels)\n            )\n        else:\n            self.conv = nn.Sequential(\n                    Conv2dNormActivation(in_channels, in_channels*exp, kernel_size=1, stride=1, padding=0, groups=in_channels*exp),\n                    Conv2dNormActivation(in_channels*exp, in_channels*exp, kernel_size=3, stride=stride, padding=1, groups=in_channels*exp),\n                    nn.Conv2d(in_channels*exp, out_channels, kernel_size=1, stride=1, bias=False),\n                    nn.BatchNorm2d(out_channels)\n                )\n\n    def forward(self, x):\n        output = self.conv(x)\n        if self.residual_connection:\n            output = x + self.conv(x)\n        return output\n\nclass MobileNetV2(nn.Module):\n    def __init__(self, input_ch = 3, hidden = 32):\n        super(MobileNetV2, self).__init__()\n        self.features = self._make_layers(input_ch, hidden)\n\n    def _make_layers(self, input_ch, hidden):\n        ###          channels, iteration, stride, expansion\n        net_info = [[16, 1, 1, 1],\n                    [24, 2, 2, 6],\n                    [32, 3, 2, 6],\n                    [64, 4, 2, 6],\n                    [96, 3, 1, 6],\n                    [160, 3, 2, 6],\n                    [320, 1, 1, 6]]\n\n        input_channel = hidden\n        layers = []\n        layers.append(Conv2dNormActivation(input_ch, input_channel, kernel_size=3, stride=2, padding=1, groups=0))\n\n        for channel, itr, stride, expansion in net_info:\n            for i in range(itr):\n                layers.append(InvertedResidual(in_channels=input_channel, out_channels=channel, exp=expansion, stride=stride))\n                input_channel = channel\n                stride=1\n                \n        layers.append(Conv2dNormActivation(320, 1024, kernel_size=1, stride=2, padding=0, groups=0))\n\n        return nn.Sequential(*layers)\n\n    def forward(self, x):\n        x1 = self.features[:2](x)\n        x2 = self.features[2:4](x1)\n        x3 = self.features[4:7](x2)\n        x4 = self.features[7:11](x3)\n        x5 = self.features[11:14](x4)\n        x6 = self.features[14:17](x5)\n        x7 = self.features[17:18](x6)\n        x8 = self.features[18:](x7)\n        return x1, x2, x3, x4, x5, x6, x7, x8\n    \nclass Upsample_Block(nn.Module):\n    def __init__(self,in_ch,out_ch,skip_ch,scale):\n        super().__init__()\n        \n        #skip_ch.shape[-1]/in_ch.shape[-1] \n        self.upsample = nn.Upsample(scale_factor=scale, mode='bilinear', align_corners=True)\n        self.bn = nn.BatchNorm2d(num_features=in_ch+skip_ch)\n        self.conv1 = nn.Conv2d(in_channels=in_ch+skip_ch, out_channels=out_ch, kernel_size=3, padding=1)\n        self.conv2 = nn.Conv2d(in_channels=out_ch, out_channels=out_ch, kernel_size=3, padding=1)\n   \n\n    def forward(self,x,skip):\n        #print(x.shape)\n        up = self.upsample(x)\n        #print(up.shape, skip.shape)\n        out = torch.cat((skip,up),axis=1)\n        out = self.bn(out)\n        out = F.relu(self.conv1(out))\n        out = F.relu(self.conv2(out))\n        return out\n\nclass Modified_Unet(nn.Module):\n    def __init__(self):\n        super().__init__()\n        \n        self.encoder = MobileNetV2(input_ch=3, hidden=24)\n\n        self.dblock1 = Upsample_Block(1024,320,320,2)\n        self.dblock2 = Upsample_Block(320,160,160,1)\n        self.dblock3 = Upsample_Block(160,96,96,2)\n        self.dblock4 = Upsample_Block(96,64,64,1)\n        self.dblock5 = Upsample_Block(64,32,32,2)\n        self.dblock6 = Upsample_Block(32,24,24,2)\n        self.dblock7 = Upsample_Block(24,16,16,2)\n        \n        self.conv = nn.Conv2d(16,1,kernel_size=1,padding=1)\n\n    def forward(self,x):\n        \n        d1, d2, d3, d4, d5, d6, d7, d8 = self.encoder(x)\n        \n        u1 = self.dblock1(d8,d7)\n        u2 = self.dblock2(u1,d6) \n        u3 = self.dblock3(u2,d5)\n        u4 = self.dblock4(u3,d4)\n        u5 = self.dblock5(u4,d3)\n        u6 = self.dblock6(u5,d2)\n        u7 = self.dblock7(u6,d1)\n\n        out = self.conv(u7)\n\n        return out","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:34.913265Z","iopub.execute_input":"2023-09-09T14:22:34.913690Z","iopub.status.idle":"2023-09-09T14:22:34.946689Z","shell.execute_reply.started":"2023-09-09T14:22:34.913656Z","shell.execute_reply":"2023-09-09T14:22:34.945693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 9. Quantifying Segmentation Accuracy\n\n> The provided code defines two functions related to the Dice coefficient and Dice loss, which are commonly used metrics for evaluating the accuracy of binary segmentation models, like the U-Net.","metadata":{}},{"cell_type":"code","source":"### Extras\ndef dice_metric(inputs, target):\n    intersection = 2.0 * (target * inputs).sum()\n    union = target.sum() + inputs.sum()\n    if target.sum() == 0 and inputs.sum() == 0:\n        return 1.0\n\n    return intersection / union\n\ndef dice_loss(inputs, target):\n    #print(target)\n    num = target.size\n    #print(num)\n    inputs = inputs.reshape(num, -1)\n    target = target.reshape(num, -1)\n    smooth = 1.0\n    intersection = (inputs * target)\n    dice = (2. * intersection.sum(1) + smooth) / (inputs.sum(1) + target.sum(1) + smooth)\n    dice = 1 - dice.sum() / num\n    return dice\n\ndef bce_dice_loss(inputs, target):\n    dicescore = dice_loss(inputs, target)\n    return dicescore","metadata":{"execution":{"iopub.status.busy":"2023-09-09T14:22:34.948340Z","iopub.execute_input":"2023-09-09T14:22:34.948904Z","iopub.status.idle":"2023-09-09T14:22:34.964779Z","shell.execute_reply.started":"2023-09-09T14:22:34.948871Z","shell.execute_reply":"2023-09-09T14:22:34.963734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 10. Model Initialization and Training Setup\n\n> This code snippet initializes a \"MobileNetV2 U-Net\" model, moves it to the specified device (like GPU), and sets up the training process by defining an optimizer.","metadata":{}},{"cell_type":"code","source":"model_1 = Modified_Unet()\nmodel_1 = model_1.to(device)\nmodel_weights_path = \"/kaggle/input/no-data/MobileNetV2_Unet_wts2023-09-09_15-33-29_30 epoch.pth\"\nmodel_1.load_state_dict(torch.load(model_weights_path, map_location=device))\n#loss_fn = monai.losses.FocalLoss()\n#loss_fn = monai.losses.DiceLoss(sigmoid=True, squared_pred=True, reduction='mean')\nloss_fn = torch.nn.BCEWithLogitsLoss()\noptimizer = torch.optim.Adam(params=model_1.parameters(),\n                        lr=0.001)","metadata":{"execution":{"iopub.status.busy":"2023-09-09T15:39:18.934932Z","iopub.execute_input":"2023-09-09T15:39:18.935391Z","iopub.status.idle":"2023-09-09T15:39:19.521128Z","shell.execute_reply.started":"2023-09-09T15:39:18.935354Z","shell.execute_reply":"2023-09-09T15:39:19.520096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 10.1 Model Architecture Overview\n\n> This code generates a summary of the \"MobileNetV2 U-Net\" model, providing insights into its architecture, layers, and parameters. The summary function is used to analyze the model's structure and visualize its layers along with the expected input size.","metadata":{}},{"cell_type":"code","source":"from torchsummary import summary\nsummary(model_1, input_size=(8, 3, 512, 512))","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-09-09T15:39:20.666072Z","iopub.execute_input":"2023-09-09T15:39:20.667234Z","iopub.status.idle":"2023-09-09T15:39:20.702812Z","shell.execute_reply.started":"2023-09-09T15:39:20.667195Z","shell.execute_reply":"2023-09-09T15:39:20.701642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 11. Model Training Loop and Evaluation\n\n> This code snippet demonstrates the training loop for the \"MobileNetV2 U-Net\" model. The loop spans a predefined number of epochs, during which the model is optimized using training data.\n\n> `torch.manual_seed(42)`: Seeds the random number generator for reproducibility.\n> \n> `Epochs`: Specifies the number of training epochs.\n> \n\nWithin each epoch:\n\nTraining Loop:\n> The loop iterates through the training data in train_loader. For each batch, the model predicts segmentation masks, calculates the Dice loss using the dice_loss function, and updates the model's parameters using the Adam optimizer.\n\nTesting Loop:\n> After each epoch's training, the model is switched to evaluation mode using model_1.eval(). The loop iterates through the testing data in test_loader, computes predicted masks, calculates the Dice loss using dice_loss, and records the average test loss.\n\nOutput:\n> The loop outputs the train and test loss values for each epoch.","metadata":{}},{"cell_type":"code","source":"torch.manual_seed(42)\n\nEpochs = 15\n\nfor epoch in tqdm(range(Epochs)):\n    print(f\"Epoch: {epoch}\\n-------\")\n    train_loss = 0\n    for batch, (X_train_image, y_train_image) in enumerate(train_loader):\n        model_1.train()\n        y_train_pred = model_1(X_train_image.to(device))\n        loss = loss_fn(y_train_pred.to('cpu'), y_train_image)\n        train_loss += loss \n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n        if batch % 20 == 0:\n            print(f\"Looked at {batch * len(X_train_image)}/{len(train_loader.dataset)} samples\")\n        train_loss /= len(train_loader)\n    test_loss = 0\n    model_1.eval()\n    with torch.inference_mode():\n        for X_test_image, y_test_image in test_loader:\n            test_pred = model_1(X_test_image.to(device))       \n            test_loss += loss_fn(test_pred.to('cpu'), y_test_image)\n        test_loss /= len(test_loader)\n    print(f\"\\nTrain loss: {train_loss:.10f} | Test loss: {test_loss:.5f}\")","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-09-09T15:39:23.795407Z","iopub.execute_input":"2023-09-09T15:39:23.796174Z","iopub.status.idle":"2023-09-09T16:50:23.570475Z","shell.execute_reply.started":"2023-09-09T15:39:23.796137Z","shell.execute_reply":"2023-09-09T16:50:23.569468Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 12. Model Saving\n\n> The provided code segment saves the trained \"MobileNetV2 x U-Net\" model's parameters and the entire model for future use:\n \n> torch.save(model_1.state_dict(), 'MobileNetV2_Unet_wts.pth'): This line saves the model's state dictionary, which contains the learned parameters of the model, into a file named \"MobileNetV2_Unet_wts.pth\". This is useful for loading the model's parameters to continue training or for inference later.\n\n> torch.save(model_1,'MobileNetV2_Unet_model.pth'): This line saves the entire model, including its architecture and learned parameters, into a file named \"MobileNetV2_Unet_model.pth\". This is useful if you want to load and use the entire model for inference without needing to reconstruct the architecture separately.\n\n> Both of these save operations facilitate storing the trained model in a serialized format, enabling easy retrieval and utilization in subsequent tasks or applications.","metadata":{}},{"cell_type":"code","source":"import time \n\ncurrent_time = time.strftime(\"%Y-%m-%d_%H-%M-%S\", time.localtime())\ntorch.save(model_1.state_dict(), f'MobileNetV2_Unet_wts{current_time}_withDice.pth')\ntorch.save(model_1,f'MobileNetV2_Unet_model{current_time}_withDice.pth')","metadata":{"execution":{"iopub.status.busy":"2023-09-09T16:50:23.572714Z","iopub.execute_input":"2023-09-09T16:50:23.573330Z","iopub.status.idle":"2023-09-09T16:50:23.812273Z","shell.execute_reply.started":"2023-09-09T16:50:23.573293Z","shell.execute_reply":"2023-09-09T16:50:23.811045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 13. Inference and Visualization\n\n> The provided code segment demonstrates the inference process of the trained \"MobileNetV2 U-Net\" model on a randomly selected sample from the training dataset. It also visualizes the original image, the model's predicted segmentation, and the expected mask:\n\n> Random Sample Selection:\n> A random index (random_idx) is chosen to select a sample from the training batch.\n\n> Prediction:\n> The selected sample (img_X) is passed through the trained model (model_1) for segmentation prediction.\n\n> Visualization:\n> The original image, predicted segmentation, and expected mask are displayed side by side using subplots.\n\nThe code snippet provides insights into the model's performance by showcasing its ability to generate segmentation predictions. The visual comparison between the predicted segmentation and the expected mask helps assess the model's accuracy and effectiveness.","metadata":{}},{"cell_type":"code","source":"random_idx = torch.randint(0, len(train_X_batch), size=[1]).item()\n#print(random_idx)\nimg_X, img_y = train_X_batch[random_idx], train_y_batch[random_idx]\npredicted_image = model_1(img_X.unsqueeze(0).to(device))\n\nfig,axes = plt.subplots(1, 3)\nimg_array = img_X.permute(1, 2, 0).cpu().numpy()\naxes[0].imshow(img_array)\naxes[0].set_title('Original Image')\naxes[0].axis(\"Off\")\n\nimg_array = predicted_image.squeeze().detach().cpu().numpy()\naxes[1].imshow(img_array, cmap='gray')\naxes[1].set_title('Prediceted Image')\naxes[1].axis(\"Off\")\n\nmask_array = img_y.permute(1, 2, 0).cpu().numpy()\naxes[2].imshow(mask_array, cmap=\"gray\")\naxes[2].set_title('Expected Image')\naxes[2].axis(\"Off\")\n\nprint(f\"Image size: {img_X.shape}\")\nprint(f\"Masked image size: {img_y.shape}\")","metadata":{"execution":{"iopub.status.busy":"2023-09-09T16:50:23.814288Z","iopub.execute_input":"2023-09-09T16:50:23.814728Z","iopub.status.idle":"2023-09-09T16:50:24.245929Z","shell.execute_reply.started":"2023-09-09T16:50:23.814687Z","shell.execute_reply":"2023-09-09T16:50:24.244301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" *Feel free for any suggestions.*\n\n> # **Thanks** !!!","metadata":{}}],"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}}