{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Image Segementation\n\n$Image$ $segmentation$ is a process of `partitioning a digital image` into `multiple image segments`, also known as $image$ $regions$ or $image$ $objects$ (sets of pixels). The `goal` of segmentation is to `simplify` and/or `change the representation` of an image into something that is `more meaningful` and `easier to analyze`.\n\n$Image$ $segmentation$ is typically used to `locate objects` and `boundaries` (lines, curves, etc.) in images. More precisely, $image$ $segmentation$ is the process of `assigning a label to every pixel` in an image such that `pixels with the same label share certain characteristics`.\n\n\n\nToday we will try to understand some of the well-known `Image Segemtation CNN Architechtures` and try to `make up them from scratch`\n\n# 1 | Basic Terminologies 🏫\n\nBut before that, lets understand some of the `basic terminilogies` in the world of `Image Pprocessing`\n\n* $Convolution$ $Layer$\n* $Pooling$ $Layer$\n* $DropOut$ $Layer$\n* $Flatten$ $Layer$\n* $Concatenation$ $Layer$\n* $Sigmoid$ $Activation$ $Function$\n* $Rectified$ $Linear$ $Unit$ $Activation$ $Function$ $(ReLU)$\n* $TanH$\n* $SoftMax$\n\n## 1.1 | Convotlution\n\n\n\n$$Y_i = B_i + \\sum\\limits_{i = 1}^{n}(x_{ij} * k_{ij})$$\n\n[Image Credits](https://www.youtube.com/watch?v=KuXjwB4LzSA)\n","metadata":{}},{"cell_type":"code","source":"import scipy\nimport numpy as np\n\nfrom matplotlib import pyplot as plt\nfrom IPython.display import IFrame","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T16:12:00.262488Z","iopub.execute_input":"2023-06-05T16:12:00.262939Z","iopub.status.idle":"2023-06-05T16:12:00.332667Z","shell.execute_reply.started":"2023-06-05T16:12:00.262908Z","shell.execute_reply":"2023-06-05T16:12:00.331258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Okay se we all know if we do $$(x+y)(x+y) = (x+y)^2 = x^2 + 2xy + y^2$$\n\nand $$(x + y + z)(x + y + z) = (x + y + z)^2 = x^2 + y^2 + z^2 + 2(xy + yz+ zx)$$\n\nThis was preety simple, but what if we have higher degrees like $$(x+ y + z....100terms)^2$$ or $$(a+b...89_-terms)(y+g...45_-terms)(t+y...69_-terms)$$ \n\nLets assume we want the coeffiecents of the eqution $(x+y)^2 =? $","metadata":{}},{"cell_type":"code","source":"np.convolve([1 , 1] , [1 , 1])","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:04.639117Z","iopub.execute_input":"2023-05-30T09:01:04.640427Z","iopub.status.idle":"2023-05-30T09:01:04.651811Z","shell.execute_reply.started":"2023-05-30T09:01:04.640382Z","shell.execute_reply":"2023-05-30T09:01:04.650480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The coeffiecients will be $(1 , 2 , 1)$ or $(1x^2 + 2xy + 1y^2)$\n\nWhat if we want of more ","metadata":{}},{"cell_type":"code","source":"np.convolve([1 , 1 , 1] , \n [1 , 1 , 1])","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:05.545507Z","iopub.execute_input":"2023-05-30T09:01:05.545912Z","iopub.status.idle":"2023-05-30T09:01:05.554286Z","shell.execute_reply.started":"2023-05-30T09:01:05.545883Z","shell.execute_reply":"2023-05-30T09:01:05.553107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The coeffiecients will be $(1 , 2, 3 , 2 , 1)$ or $(1x^4 + 2x^3 + 3x^2 + 2x + 1)$\n\nAnd furthermore\n\n```\nnp.convolve([1 , 2 , 3] , \n [4 , 5 , 6] , \n [7 , 8 , 9])\n```\nThis will throw an error ","metadata":{}},{"cell_type":"markdown","source":"```\n---------------------------------------------------------------------------\nTypeError Traceback (most recent call last)\n in ()\n----> 1 np.convolve([1 , 2 , 3] , \n 2 [4 , 5 , 6] ,\n 3 [7 , 8 , 9])\n\n1 frames\n/usr/local/lib/python3.10/dist-packages/numpy/core/overrides.py in convolve(*args, **kwargs)\n\n/usr/local/lib/python3.10/dist-packages/numpy/core/numeric.py in convolve(a, v, mode)\n 842 if len(v) == 0:\n 843 raise ValueError('v cannot be empty')\n--> 844 return multiarray.correlate(a, v[::-1], mode)\n 845 \n 846 \n\nTypeError: convolve/correlate mode not understood\n```","metadata":{"_kg_hide-input":true}},{"cell_type":"markdown","source":"So we cannot do this directly, what we can rather do is","metadata":{}},{"cell_type":"code","source":"np.convolve(np.convolve([1 , 2 , 3] , \n [4 , 5 , 6]) , \n [7 , 8 , 9])","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:07.290191Z","iopub.execute_input":"2023-05-30T09:01:07.290579Z","iopub.status.idle":"2023-05-30T09:01:07.298997Z","shell.execute_reply.started":"2023-05-30T09:01:07.290548Z","shell.execute_reply":"2023-05-30T09:01:07.297756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Or","metadata":{}},{"cell_type":"code","source":"np.convolve(np.convolve([4 , 5 , 6] , \n [7 , 8 , 9]) , \n [1 , 2 ,3])","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:08.640016Z","iopub.execute_input":"2023-05-30T09:01:08.640958Z","iopub.status.idle":"2023-05-30T09:01:08.650917Z","shell.execute_reply.started":"2023-05-30T09:01:08.640914Z","shell.execute_reply":"2023-05-30T09:01:08.649478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And we got the same arrays \n\nBut this gets tricky when we have high level of arrays ","metadata":{}},{"cell_type":"code","source":"array_1 = np.arange(1000000)\narray_2 = np.arange(1000000)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:09.551735Z","iopub.execute_input":"2023-05-30T09:01:09.552217Z","iopub.status.idle":"2023-05-30T09:01:09.568488Z","shell.execute_reply.started":"2023-05-30T09:01:09.552184Z","shell.execute_reply":"2023-05-30T09:01:09.567435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# np.convolve(array_1 , array_2) # Thats gonna take a lot of time, I dont want to loose my cores, I have already lost 15 :( , and thus i am commenting this out","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:10.321122Z","iopub.execute_input":"2023-05-30T09:01:10.321627Z","iopub.status.idle":"2023-05-30T09:01:10.327015Z","shell.execute_reply.started":"2023-05-30T09:01:10.321591Z","shell.execute_reply":"2023-05-30T09:01:10.325807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So how can we do large functions, for that we use","metadata":{}},{"cell_type":"code","source":"scipy.signal.fftconvolve(array_1 , array_2)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:11.127181Z","iopub.execute_input":"2023-05-30T09:01:11.127648Z","iopub.status.idle":"2023-05-30T09:01:12.069643Z","shell.execute_reply.started":"2023-05-30T09:01:11.127616Z","shell.execute_reply":"2023-05-30T09:01:12.068292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"One more intution you can get of convolve is from the rolling of the dice\n\n\n\n|Terms|No of occrences, if both the dice are added|\n|---|---|\n|1|0|\n|2|1|\n|3|2|\n|4|3|\n|5|4|\n|6|5|\n|7|6|\n|8|5|\n|9|4|\n|10|3|\n|11|2|\n|12|1|\n\nAnd the convolve of $(1 , 1 , 1 , 1 , 1 , 1)*((1 , 1 , 1 , 1 , 1 , 1))$ is ","metadata":{}},{"cell_type":"code","source":"np.convolve([1 , 1 , 1 , 1 , 1 , 1] , \n [1 , 1 , 1 , 1 , 1 , 1])","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:01:12.278781Z","iopub.execute_input":"2023-05-30T09:01:12.279180Z","iopub.status.idle":"2023-05-30T09:01:12.288826Z","shell.execute_reply.started":"2023-05-30T09:01:12.279151Z","shell.execute_reply":"2023-05-30T09:01:12.287390Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Notice something??\n\nConvolve is basically aslo teeling the number of occurence of the number in a particular order.\n\nSo now we have a basic understanding of the convolve. \n\nViewung it with the respect of images.\n\nLets assume we have an `image` of $1$ channel only, means the image is `black and white`.\n\nWe can take the image is a $2D$ matrix of numbers in the range $(0,1)$.\n\nIf we take a small matrix of dimensions, say $(3,3)$, and try to place it over a subset of the orginal big matrix of image. Then we multiply the corresponding numbers.\n\nWe say we `convolved a subset` of the image with the small matrix.\n\nIf we try to do this `making a stride over the image` matrix and try to cover the whole image.\n\nWe say we `convolved the whole image` with the small matrix.\n\nIn technincal terms we say the small matrix, `the kernel or filter`.\n\nIf we take the image of say $3$ channels. We do the same process for each channel perticularly.\n\n**`So what is the use of doing this...?`**\n\nThe use majorly `depends on the type of kernel` we use. We can use different kernels to perform different image processing tasks like\n\n* Bluring\n* Sharpening\n* Detecting Edges\n\nWe should always choose a `kernel size of odd number`, as at that point we can find the `middle element of the kernel`\n\nSometimes when we convolve an image. We find a `sort of border on the resultant image`. That is because convolution at the corners and the edges basically means that we are `only taking` $1$ or $2$ `pixels into account`. To counter that, we sometimes create a `padding at the edges of the image`. \n\n**Padding is basically placing $0s$ over the border**\n\nWhen we convolve an image of pixels $(x,x)$ with a kernel of size $(a,a)$ . The `resultant image` gets a `bit shorter`, making its `size smaller`. We can `caluclate the relsutant size` by the formula\n\n$$\\frac{I_s + K+2P}{S}+1$$\n \nwhere\n\n* $I_s=Input$ $Size$\n \n* $K=Kernel$\n \n* $P=Padding$\n \n* $S=Stride$\n \nSo now we have a basic idea of the Convolution.\n\n## 1.2 | Pooling \n\nHere we take a `image matrix` and `only focus on a defined part` of the matrix. Like the image is of the size $(500,500)$ . But we only focus on the first $(3,3)$\n. Pooling is usually doing `different operations on this small subset of the image`.\n\nTechnically we say the whole matrix as a `Pool`, and we do operations on a defined subset.\n\nIf we do this only one time, we call this `Pooling a subset of the image`\n\nIf we make a stride and try to move it over the whole image, we cal that `Pooling the whole image`\n\nBy operations we mean different mathemmatical actions with the subset. The major ones we use are\n\n* Max Pooling\n* Min Pooling\n* Mean/Average Pooling\n\n**`So what is the use of doing this...?`**\n\nPooling is basically used to `reduce the dimensions of the image` and still `retaining the conceptual` or the core information of the image\n\nFor caluclating the size of the resultant image, the same formula can be used as in the `Convolution Layer`\n\n```\ntensorflow.keras.layers.MaxPooling1D()\ntensorflow.keras.layers.MaxPooling2D()\ntensorflow.keras.layers.MaxPooling3D()\n\ntensorflow.keras.layers.AveragePooling1D()\ntensorflow.keras.layers.AveragePooling2D()\ntensorflow.keras.layers.AveragePooling3D()\n```\n\n\n\n## 1.3 | DropOut Layer\n\nThe `Dropout layer` is a `regularization technique` commonly used in neural networks, including convolutional neural networks $CNNs$. Its purpose is to `prevent overfitting`, which occurs when the model performs `well on the training data but fails to generalize to new, unseen data`.\n\nThe Dropout layer works by randomly `dropping out` or `setting to zero` a `certain proportion` of the `input units` or neurons during training. This means that during each training `iteration`, a different `subset of neurons is deactivated`. By doing so, the network is `forced to learn redundant representations`, as it cannot `rely on specific neurons` or `combinations of neurons to make predictions`. This encourages the network to become `more robust` and prevents it from `relying too heavily on specific features`.\n\n```\ntensorflow.keras.layers.DropOut()\n```\n\n\n\n## 1.4 | Flatten Layer \n\nLets assume we have an array like this","metadata":{}},{"cell_type":"code","source":"sample_array = np.array([[x for x in range(100)] , \n [x for x in range(100)]])\nsample_array , sample_array.shape","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-30T09:01:13.839591Z","iopub.execute_input":"2023-05-30T09:01:13.840030Z","iopub.status.idle":"2023-05-30T09:01:13.850215Z","shell.execute_reply.started":"2023-05-30T09:01:13.839990Z","shell.execute_reply":"2023-05-30T09:01:13.848652Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If we use the function `flatten()`","metadata":{}},{"cell_type":"code","source":"sample_array = sample_array.flatten()\nsample_array , sample_array.shape","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-30T09:01:14.711419Z","iopub.execute_input":"2023-05-30T09:01:14.711914Z","iopub.status.idle":"2023-05-30T09:01:14.721575Z","shell.execute_reply.started":"2023-05-30T09:01:14.711880Z","shell.execute_reply":"2023-05-30T09:01:14.720302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So flatten means basically making a $1βˆ’D$ array\n\n\n\n## 1.5 | Concatination Layer \n\n$Concatenation$ $Layer$ is a layer that `concatenates a list of inputs`. It takes as `input a list of tensors`, all of the `same shape` except for the concatenation axis, and `returns a single tensor that is the concatenation of all inputs`.\n\nThe $Concatenate$ $Layer$ is often used to `combine the outputs of different layers in a neural network`. For example, we might use a `concatenate layer to combine the outputs of two convolutional layers` in order to create a deeper convolutional layer.\n\nThe concatenate layer can also be used to `combine the outputs of different models`. For example, we might use a `concatenate layer to combine the outputs of a model that predicts the presence of an object` in an image with the `outputs of a model that predicts the location of the object in the image`.\n\n```\ntensorflow.keras.layers.Concatenate()\n```\n\n\n\n## 1.6 | Sigmoid Function\n\nWe can see this is \n$$f(x)=\\Bigg[\\frac{1}{0}\\frac{...}{...}\\frac{if}{if}\\frac{x>=1}{x<=0}$$\n \nsome people also write this as Formula $$f(x)=\\frac{1}{1+e^{-x}}$$\n\nAnd actually they ar pretty both the same\n\nLets call this bitch `Sigmoid`","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/zgzwi3v5vx\" , 1000 , 400)","metadata":{"_kg_hide-output":false,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:28:03.426556Z","iopub.execute_input":"2023-06-05T09:28:03.427052Z","iopub.status.idle":"2023-06-05T09:28:03.437487Z","shell.execute_reply.started":"2023-06-05T09:28:03.427012Z","shell.execute_reply":"2023-06-05T09:28:03.435975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So how this function knows that $1$ is the limit...?\n\nOne way to find out is to change some values\n$$f(x)=\\frac{5}{1+e^{βˆ’x}}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/8kpchslmim\" , 1000 , 400)","metadata":{"_kg_hide-output":false,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:28:26.259083Z","iopub.execute_input":"2023-06-05T09:28:26.259537Z","iopub.status.idle":"2023-06-05T09:28:26.269246Z","shell.execute_reply.started":"2023-06-05T09:28:26.259502Z","shell.execute_reply":"2023-06-05T09:28:26.267727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"What if we take $x$ as postive\n$$f(x)=\\frac{1}{1+e^x}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/tilthcejmx\" , 1000 , 400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:28:32.737949Z","iopub.execute_input":"2023-06-05T09:28:32.738400Z","iopub.status.idle":"2023-06-05T09:28:32.745547Z","shell.execute_reply.started":"2023-06-05T09:28:32.738367Z","shell.execute_reply":"2023-06-05T09:28:32.744654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"What if we change the value of $x$\n \n$$f(x)=\\frac{5}{1+e^{(βˆ’x+10)}}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/ictkwtquez\" , 1000 , 400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:28:39.946679Z","iopub.execute_input":"2023-06-05T09:28:39.947111Z","iopub.status.idle":"2023-06-05T09:28:39.955065Z","shell.execute_reply.started":"2023-06-05T09:28:39.947078Z","shell.execute_reply":"2023-06-05T09:28:39.953810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A visulaization how math can be perfect sometimes $:)$\n$$f(x)=\\frac{5}{5+e^{βˆ’x}}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/xz7jwzs2v9\" , 1000 , 400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:28:46.207703Z","iopub.execute_input":"2023-06-05T09:28:46.208163Z","iopub.status.idle":"2023-06-05T09:28:46.217684Z","shell.execute_reply.started":"2023-06-05T09:28:46.208129Z","shell.execute_reply":"2023-06-05T09:28:46.215707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"One thing we can note is that the function not merely depends on the numeriator to find the value of the upper limit, but rather the dividend of both the numerator and the independent term in the dimoninator\n\n## 1.7 | ReLU \n\nThere are some cases where we do not need the negative values and only need the positive ones. If you havent find any case like that, soldier world is very small, the case will find its way to you soon\n\nOne way to counter this problem is to use a function like Formula\n$$f(x)=\\Bigg [\\frac{x}{0}\\frac{if}{if}\\frac{x>0}{x<0}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/wptxy1wqa1\" , 1000, 400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:28:53.830356Z","iopub.execute_input":"2023-06-05T09:28:53.830778Z","iopub.status.idle":"2023-06-05T09:28:53.838862Z","shell.execute_reply.started":"2023-06-05T09:28:53.830744Z","shell.execute_reply":"2023-06-05T09:28:53.837553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"You can see this as the amount of sadness you got on y-axix and your years of life on the x-axis. You have not born yet before the $0$\n \nLets assume you were not able to feel emotions till $5$ years, we can see that as\n$$f(x)=\\Bigg[\\frac{x+5}{0}\\frac{if}{if}\\frac{x>0}{x<0}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/tto4l1ylqo\" , 1000 , 400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:29:01.677064Z","iopub.execute_input":"2023-06-05T09:29:01.677701Z","iopub.status.idle":"2023-06-05T09:29:01.686816Z","shell.execute_reply.started":"2023-06-05T09:29:01.677653Z","shell.execute_reply":"2023-06-05T09:29:01.685396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Lets assume your are a straight person and thats why you get less privelages and thus your sadness did not started from $0$ but rather than $2$\n \n$$f(x)=\\Bigg[\\frac{x + 2}{2}\\frac{if}{if}\\frac{x>0}{x<0}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/yoowzap3y1\" , 1000 , 400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:29:08.061791Z","iopub.execute_input":"2023-06-05T09:29:08.062193Z","iopub.status.idle":"2023-06-05T09:29:08.069314Z","shell.execute_reply.started":"2023-06-05T09:29:08.062163Z","shell.execute_reply":"2023-06-05T09:29:08.068163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.8 Tanh Activation Function\n\nSo `trigonometry` got discovered in the $3^{rd}Century$ (I just googled it $:)$). And from that point many people have tried to `burn the original books` but have failed many times. Then came another person who `invented hyperbolic functions` in $1760$ and till now people have tried to burn other books too.\n\nSo what is really a hyperbolic function $...?$\n \nYou see there is a very deep difference in them.\n\nA trignometric function is used to describe a cricle.\n\nA hyperbolic function is used to describe a hyperbola.","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/g72ymsc9tt\" , 1000 , 400)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T10:16:59.878536Z","iopub.execute_input":"2023-06-05T10:16:59.879033Z","iopub.status.idle":"2023-06-05T10:16:59.888128Z","shell.execute_reply.started":"2023-06-05T10:16:59.878999Z","shell.execute_reply":"2023-06-05T10:16:59.886713Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is a hyperbola. We will not get deep into this. But a function that defines hyperbola is tanh or hyperbolic tangent. wiht the formula\n\n$$f(x)=\\frac{e^xβˆ’e^{βˆ’x}}{e^x+e^{βˆ’x}}$$","metadata":{}},{"cell_type":"code","source":"IFrame(\"https://www.desmos.com/calculator/sovpkgmwon\" , 1000 , 400)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T10:18:09.108924Z","iopub.execute_input":"2023-06-05T10:18:09.109401Z","iopub.status.idle":"2023-06-05T10:18:09.117923Z","shell.execute_reply.started":"2023-06-05T10:18:09.109368Z","shell.execute_reply":"2023-06-05T10:18:09.116428Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.9 | Softmax \n\nSometimes we get in a situation where we want the probablities of some classes. one wat is to obviously divide occurence of every class by the total occurrence. and thats pretty straight forward. But there is a problem with that.\n\nTry to imagine that we have particular positions of all the classes in a $2D$ plane. how can we do that then...?\n\nOne way to counter this one is to use a graph of $e^x$ and porject that points on the graph and then get the distance.\n\nAnd that why we come up with the formula\n\n$$f(x)=\\frac{e^x}{\\sum\\limits e^x}$$\n\n# 2 | HuBMAP Data πŸš€\n\nThe $HuBMAP$ $-$ $Hacking$ $the$ $Human$ $Vasculature$ $competition$ is a $Kaggle$ $competition$ that challenges participants to `develop machine learning models to segment microvascular structures` in $2D$ `PAS-stained histology images` from healthy human kidney tissue slides. The goal of the competition is to `improve researchers' understanding of how the blood vessels are arranged in human tissues`.\n\nThe competition is hosted by the $Human$ $BioMolecular$ $Atlas$ $Program$ $HuBMAP$, which is a `global effort to create a comprehensive and open-access atlas of human cells`. $HuBMAP$ researchers are using the `latest molecular and cellular biology technologies` to `study the connections that cells have with each other` throughout the body.\n\nThe `microvascular structures` that are being segmented in this competition `include` \n* $Capillaries$\n* $Arterioles$\n* $Venules$\n\nThese structures are `very small` and `difficult to see with the naked eye`, so `automated segmentation methods are essential` for researchers to study them.\n\n**[National effort to focus on mapping human body on cellular level](https://www.purdue.edu/newsroom/releases/2019/Q4/national-effort-to-focus-on-mapping-human-body-on-cellular-level.html)**\n\n\n\nThe images are $256x256$ pixels in `size` and are in the `.tif` format. The images are a `diverse set` of images from `different patients` and `different tissue slides`.\n\nThe `ground truth segmentation` for the `images` in the data set was created by a `team of experts`. The experts used a `variety of techniques` to create the ground truth segmentation\n* Manual segmentation\n* Semi-automatic segmentation. \n\nThe ground truth segmentation is accurate and reliable, and it is essential for training machine learning models to segment microvascular structures.","metadata":{}},{"cell_type":"code","source":"import os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:29:25.637025Z","iopub.execute_input":"2023-06-05T09:29:25.637481Z","iopub.status.idle":"2023-06-05T09:29:31.690788Z","shell.execute_reply.started":"2023-06-05T09:29:25.637447Z","shell.execute_reply":"2023-06-05T09:29:31.689541Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Thanks to **[YASSINE ALOUINI](https://www.kaggle.com/yassinealouini)=>[Working with TIFF files](https://www.kaggle.com/code/yassinealouini/working-with-tiff-files)** for providing a simple way to work wit the `tif` files withing python environments\n\nI have change the way of input for the image from `rasterio` to `opencv`, though it was concluded in **[YASSINE ALOUINI](https://www.kaggle.com/yassinealouini)=>[Some Insights](https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/333389)** that `rasterio` is the fastest way ","metadata":{}},{"cell_type":"code","source":"train_dir = \"/kaggle/input/hubmap-hacking-the-human-vasculature/train\"\ntest_dir = \"/kaggle/input/hubmap-hacking-the-human-vasculature/test\"","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:29:36.319529Z","iopub.execute_input":"2023-06-05T09:29:36.320001Z","iopub.status.idle":"2023-06-05T09:29:36.325363Z","shell.execute_reply.started":"2023-06-05T09:29:36.319960Z","shell.execute_reply":"2023-06-05T09:29:36.324004Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(\"/kaggle/input/hubmap-hacking-the-human-vasculature/sample_submission.csv\" , \"r\") as f:\n k = list(f)\nprint(k[0])","metadata":{"execution":{"iopub.status.busy":"2023-06-04T08:56:24.573390Z","iopub.execute_input":"2023-06-04T08:56:24.574365Z","iopub.status.idle":"2023-06-04T08:56:24.585014Z","shell.execute_reply.started":"2023-06-04T08:56:24.574325Z","shell.execute_reply":"2023-06-04T08:56:24.583869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3 | Visualizing The Data 😎\n\nVisualizing the data is an integral part of image models. It gives us information on what we aare actually working, and also we do this for fun. Yayyyyyyyy\n\nLets asusme we take this iamge as sample \n\nLets assume we want to train a model on the dataset we have. But it is not a good idea to directly train a big model on this data. \n\nWhen we are working with the images, it is advised to first preproces the images, before sending them to any model \n\nBy preporcessing we mostly mean that \n\n|_____|______\n|---|---\n|Resize|To ensure that all images are the same size.\n||To reduce the amount of data that needs to be processed.\n||To improve the accuracy of the model.\n|Mean|It helps to improve the stability of the model. \n||It helps to improve the performance of the model.\n||It helps to make the model more robust to changes in lighting.\n|STD|It helps to improve the stability of the training process.\n||It helps to improve the performance of the model. \n||It makes the model more interpretable.\n\nWe could have actually made a proper function from scratch to do these things, like this \n```\ndef preprocess(image):\n \n image = np.clip(image , width , height)\n\n image = (image - image.mean()) / image.std()\n\n return image\n```\n(this might be not correct, but just a example)\n\nBut we rather use a specialized library that is faster. The `code we just wrote` can be `very slow to work`. But `albumnetnation library` can be `fast enough`","metadata":{}},{"cell_type":"code","source":"import cv2\n\nimport tensorflow as tf\n\nimport albumentations as A\nfrom albumentations.pytorch import ToTensorV2","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:29:50.861802Z","iopub.execute_input":"2023-06-05T09:29:50.862256Z","iopub.status.idle":"2023-06-05T09:30:07.258635Z","shell.execute_reply.started":"2023-06-05T09:29:50.862218Z","shell.execute_reply":"2023-06-05T09:30:07.257233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"A.Compose([\n A.Resize(width = 512 , height = 512) , \n A.Normalize(\n mean = [0 , 0] , \n std = [1 , 1] , \n max_pixel_value = 255\n ) , \n ToTensorV2()\n ])","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:07.260553Z","iopub.execute_input":"2023-06-05T09:30:07.261429Z","iopub.status.idle":"2023-06-05T09:30:07.271334Z","shell.execute_reply.started":"2023-06-05T09:30:07.261384Z","shell.execute_reply":"2023-06-05T09:30:07.270010Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is our `compose` block\n\nNow lets make a proper function `display` for viewing the image","metadata":{}},{"cell_type":"code","source":"sample_image = cv2.imread(\"/kaggle/input/hubmap-hacking-the-human-vasculature/test/72e40acccadf.tif\")","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:16.549682Z","iopub.execute_input":"2023-06-05T09:30:16.550163Z","iopub.status.idle":"2023-06-05T09:30:16.604427Z","shell.execute_reply.started":"2023-06-05T09:30:16.550125Z","shell.execute_reply":"2023-06-05T09:30:16.603169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display(im , augments = False):\n\n img = im\n \n if augments :\n \n img = A.Compose([\n A.Resize(width = 512 , height = 512) , \n A.Normalize(\n mean = [0 , 0 , 0] , \n std = [1 , 1 , 1] , \n max_pixel_value = 255\n ) , \n ToTensorV2()\n ])(image = im)[\"image\"]\n\n # return image\n \n plt.imshow(tf.reshape(img , (512 , 512 , 3)))","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:18.791502Z","iopub.execute_input":"2023-06-05T09:30:18.792010Z","iopub.status.idle":"2023-06-05T09:30:18.799866Z","shell.execute_reply.started":"2023-06-05T09:30:18.791967Z","shell.execute_reply":"2023-06-05T09:30:18.799025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Test Image before preprocessing : \")\ndisplay(sample_image)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-30T08:58:56.620840Z","iopub.execute_input":"2023-05-30T08:58:56.621481Z","iopub.status.idle":"2023-05-30T08:58:57.117879Z","shell.execute_reply.started":"2023-05-30T08:58:56.621449Z","shell.execute_reply":"2023-05-30T08:58:57.116757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Test Image before preprocessing : \")\ndisplay(sample_image , augments = True)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T05:01:30.687955Z","iopub.execute_input":"2023-05-30T05:01:30.688332Z","iopub.status.idle":"2023-05-30T05:01:31.096601Z","shell.execute_reply.started":"2023-05-30T05:01:30.688303Z","shell.execute_reply":"2023-05-30T05:01:31.095631Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4 | Pytorch DataLoader πŸ“–\n\nA data loader is a tool that helps to load data into a deep learning model. It is used to break down the data into smaller batches, which can then be processed by the model more efficiently. Data loaders can also be used to shuffle the data, which can help to improve the performance of the model.\n\nThe `DataLoader` we are going to create here, is highly inspired by **[Thomas Rochefort-Beaudoin](https://www.kaggle.com/thomasrochefort)=>[HuBMAP : Simple PyTorch DataLoade](https://www.kaggle.com/code/thomasrochefort/hubmap-simple-pytorch-dataloader/notebook)**\n\n* We simply first read the `json files`\n* Then we get the images from the set\n* Then according to the coordinates in the `json_file`. We mask the blood vessels. \n* The we return the image and the mask\n\nBlood vessels are often masked in medical imaging competitions to make the task of identifying other objects in the image more challenging. This is because blood vessels can be very similar in appearance to other objects, such as tumors or lesions. Masking the blood vessels forces the model to focus on the other objects in the image, and to learn to distinguish them from the blood vessels. This can lead to more accurate and reliable detection of other objects in the image.","metadata":{}},{"cell_type":"code","source":"import json\n\nfrom torch.utils.data import DataLoader, Dataset","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:30:23.552172Z","iopub.execute_input":"2023-06-05T09:30:23.552611Z","iopub.status.idle":"2023-06-05T09:30:23.558350Z","shell.execute_reply.started":"2023-06-05T09:30:23.552577Z","shell.execute_reply":"2023-06-05T09:30:23.557020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"a = A.Compose([\n A.Resize(width = 512 , height = 512) , \n A.Normalize(\n mean = [0 , 0 , 0] , \n std = [1 , 1 , 1] , \n max_pixel_value = 255\n ) , \n ToTensorV2()\n ])","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:23.998960Z","iopub.execute_input":"2023-06-05T09:30:24.000185Z","iopub.status.idle":"2023-06-05T09:30:24.006364Z","shell.execute_reply.started":"2023-06-05T09:30:24.000140Z","shell.execute_reply":"2023-06-05T09:30:24.005103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class hubmapDataset(Dataset):\n \n def __init__(self, image_dir, labels_file , augments = False):\n \n with open(labels_file, 'r') as json_file:\n self.json_labels = [json.loads(line) for line in json_file]\n\n self.image_dir = image_dir\n# self.transform = transform\n self.augments = augments\n\n __len__ = lambda self : len(self.json_labels) \n \n def __getitem__(self, idx):\n \n image_path = os.path.join(self.image_dir, f\"{self.json_labels[idx]['id']}.tif\")\n image = Image.open(image_path)\n \n if self.augments:\n \n image = a(image = image)[\"image\"]\n \n mask = np.zeros((512, 512), dtype=np.float32)\n\n for annot in self.json_labels[idx]['annotations']:\n\n cords = annot['coordinates']\n \n if annot['type'] == \"blood_vessel\":\n \n for cord in cords:\n \n rr, cc = np.array([i[1] for i in cord]), np.asarray([i[0] for i in cord])\n \n mask[rr, cc] = 1\n\n image = torch.tensor(np.array(image), dtype=torch.float32).permute(2, 0, 1) # Shape: [C, H, W]\n mask = torch.tensor(mask, dtype=torch.float32)\n\n# if self.transform:\n# image = self.transform(image)\n\n return image, mask","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:24.735119Z","iopub.execute_input":"2023-06-05T09:30:24.735603Z","iopub.status.idle":"2023-06-05T09:30:24.749099Z","shell.execute_reply.started":"2023-06-05T09:30:24.735560Z","shell.execute_reply":"2023-06-05T09:30:24.747751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dataset = hubmapDataset(image_dir = train_dir, labels_file = '../input/hubmap-hacking-the-human-vasculature/polygons.jsonl')","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:27.032689Z","iopub.execute_input":"2023-06-05T09:30:27.033128Z","iopub.status.idle":"2023-06-05T09:30:32.552665Z","shell.execute_reply.started":"2023-06-05T09:30:27.033093Z","shell.execute_reply":"2023-06-05T09:30:32.551398Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 5 | Seperating The Files πŸ“‚\n\nLets make a seperete direcotry for these `images` and their corresponding `masks`","metadata":{}},{"cell_type":"code","source":"import torchvision.transforms as T\nimport torch\n\nfrom tqdm import tqdm\nfrom PIL import Image","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:30:37.191618Z","iopub.execute_input":"2023-06-05T09:30:37.192195Z","iopub.status.idle":"2023-06-05T09:30:37.199165Z","shell.execute_reply.started":"2023-06-05T09:30:37.192144Z","shell.execute_reply":"2023-06-05T09:30:37.197679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path = \"/kaggle/working/\"\n\nos.makedirs('/kaggle/working/Sample_data/Image')\nos.makedirs('/kaggle/working/Sample_data/Mask')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-06-05T09:30:37.806547Z","iopub.execute_input":"2023-06-05T09:30:37.808360Z","iopub.status.idle":"2023-06-05T09:30:37.813972Z","shell.execute_reply.started":"2023-06-05T09:30:37.808298Z","shell.execute_reply":"2023-06-05T09:30:37.812976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We got `torch.tensor()` data type from our `dataloader`, and thus we will convert that into `PILImage` format and then save the corresponding into the `file`","metadata":{}},{"cell_type":"code","source":"transform = T.ToPILImage()","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:40.059261Z","iopub.execute_input":"2023-06-05T09:30:40.059677Z","iopub.status.idle":"2023-06-05T09:30:40.065287Z","shell.execute_reply.started":"2023-06-05T09:30:40.059647Z","shell.execute_reply":"2023-06-05T09:30:40.064331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for index , data in tqdm(enumerate(train_dataset) , total = len(train_dataset)):\n \n img , mask = data\n \n img = transform(img)\n mask = transform(mask)\n \n img.save(path + \"Sample_data/Image/img_\" + str(index) + \".jpg\")\n mask.save(path + \"Sample_data/Mask/mas_\" + str(index) + \".jpg\")","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:30:41.500448Z","iopub.execute_input":"2023-06-05T09:30:41.501525Z","iopub.status.idle":"2023-06-05T09:31:24.597197Z","shell.execute_reply.started":"2023-06-05T09:30:41.501484Z","shell.execute_reply":"2023-06-05T09:31:24.595579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And we have succesfully loaded our seperate files into the `working directory `\n\n# 6 | Tensorflow DataLoader πŸ“…\nSo now this is our `train` and `test` directories","metadata":{}},{"cell_type":"code","source":"image_dir = \"/kaggle/working/Sample_data/Image/\"\nmask_dir = \"/kaggle/working/Sample_data/Mask/\"","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:31:29.579195Z","iopub.execute_input":"2023-06-05T09:31:29.579633Z","iopub.status.idle":"2023-06-05T09:31:29.585197Z","shell.execute_reply.started":"2023-06-05T09:31:29.579595Z","shell.execute_reply":"2023-06-05T09:31:29.583915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"These our data files ","metadata":{}},{"cell_type":"code","source":"train = os.listdir(image_dir)\ntest = os.listdir(mask_dir)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:31:31.246858Z","iopub.execute_input":"2023-06-05T09:31:31.247709Z","iopub.status.idle":"2023-06-05T09:31:31.257737Z","shell.execute_reply.started":"2023-06-05T09:31:31.247642Z","shell.execute_reply":"2023-06-05T09:31:31.255974Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is our actual paths to `train` and `test`","metadata":{}},{"cell_type":"code","source":"X_train = []\nY_train = []","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:31:33.273859Z","iopub.execute_input":"2023-06-05T09:31:33.274264Z","iopub.status.idle":"2023-06-05T09:31:33.280043Z","shell.execute_reply.started":"2023-06-05T09:31:33.274232Z","shell.execute_reply":"2023-06-05T09:31:33.278729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we will load data for `tensorflow`","metadata":{}},{"cell_type":"code","source":"image_height = 128\nimage_width = 128","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:22:00.565105Z","iopub.execute_input":"2023-06-05T16:22:00.565601Z","iopub.status.idle":"2023-06-05T16:22:00.571887Z","shell.execute_reply.started":"2023-06-05T16:22:00.565566Z","shell.execute_reply":"2023-06-05T16:22:00.570200Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for path in tqdm(train , total = len(train)):\n train_path = str(image_dir) + str(path)\n \n img = cv2.imread(train_path)\n img = cv2.resize(img , (image_height , image_width))\n img = cv2.cvtColor(img , cv2.COLOR_BGR2GRAY)\n\n X_train.append(img)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:31:35.766420Z","iopub.execute_input":"2023-06-05T09:31:35.766820Z","iopub.status.idle":"2023-06-05T09:31:42.767687Z","shell.execute_reply.started":"2023-06-05T09:31:35.766790Z","shell.execute_reply":"2023-06-05T09:31:42.766334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for path in tqdm(test , total = len(test)):\n test_path = str(mask_dir) + str(path)\n\n img = cv2.imread(test_path)\n img = cv2.resize(img , (image_height , image_width))\n img = cv2.cvtColor(img , cv2.COLOR_BGR2GRAY)\n \n Y_train.append(img)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:31:44.175067Z","iopub.execute_input":"2023-06-05T09:31:44.175476Z","iopub.status.idle":"2023-06-05T09:31:46.961202Z","shell.execute_reply.started":"2023-06-05T09:31:44.175444Z","shell.execute_reply":"2023-06-05T09:31:46.960091Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is our `train` and `test` data ","metadata":{}},{"cell_type":"code","source":"len(X_train) , len(Y_train)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:31:48.182741Z","iopub.execute_input":"2023-06-05T09:31:48.184606Z","iopub.status.idle":"2023-06-05T09:31:48.191191Z","shell.execute_reply.started":"2023-06-05T09:31:48.184537Z","shell.execute_reply":"2023-06-05T09:31:48.190191Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train = np.array(X_train)\nY_train = np.array(Y_train)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:31:49.337786Z","iopub.execute_input":"2023-06-05T09:31:49.338871Z","iopub.status.idle":"2023-06-05T09:31:49.368270Z","shell.execute_reply.started":"2023-06-05T09:31:49.338824Z","shell.execute_reply":"2023-06-05T09:31:49.366518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 7 | Image Segmentation Architechtures πŸ—οΈ\n\n* $LENET-5$\n* $U-NET$\n* $V-NET$","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras import Sequential\nfrom tensorflow.keras.layers import (\n Input , Conv2D , \n Dropout , MaxPooling2D , \n AveragePooling2D , Conv2DTranspose , \n concatenate , Lambda , \n Flatten , Dense\n)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:20:32.770942Z","iopub.execute_input":"2023-06-05T16:20:32.771336Z","iopub.status.idle":"2023-06-05T16:20:43.271947Z","shell.execute_reply.started":"2023-06-05T16:20:32.771307Z","shell.execute_reply":"2023-06-05T16:20:43.270683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 7.1 | LENET-5 ✨\n\nThough $LeNet$ is an `image-classification` architechture, But with small tweekings, we can modify it for `image-segemntation`. As $LENET-5$ is one of the first $CNN$ that was made, I thought it would be great if we started the journey of `image-segmentation` with $LENET-5$ \n\n$LeNet$ is a `convolutional neural network` architecture developed by `Yann LeCun` for `handwritten digit recognition`. It consists of `several convolutional` and `pooling layers`. $LeNet$ was one of the `first successful deep learning models` and `paved the way for modern convolutional neural networks`. It has since been adapted for various image recognition tasks and remains a fundamental building block of computer vision systems.\n\n\n\nBelow is the code for the original Lenet Architechture","metadata":{}},{"cell_type":"markdown","source":"```\nsample_lenet = Sequential()\n\nsample_lenet.add(Conv2D(6 , kernel_size = (5, 5) , activation = \"tanh\" , input_shape = (28 , 28 , 1)))\nsample_lenet.add(AveragePooling2D(pool_size = (2 , 2) , strides = 2))\n\nsample_lenet.add(Conv2D(16 , kernel_size = (5 , 5), activation = \"tanh\"))\nsample_lenet.add(AveragePooling2D(pool_size = (2 , 2) , strides = 2))\n\nsample_lenet.add(Flatten())\n\nsample_lenet.add(Dense(120 , activation = \"tanh\"))\nsample_lenet.add(Dense(84 , activation = \"tanh\"))\nsample_lenet.add(Dense(10 , activation = \"softmax\"))\n```","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-05T09:43:32.160443Z","iopub.execute_input":"2023-06-05T09:43:32.160850Z","iopub.status.idle":"2023-06-05T09:43:32.288611Z","shell.execute_reply.started":"2023-06-05T09:43:32.160819Z","shell.execute_reply":"2023-06-05T09:43:32.287038Z"}}},{"cell_type":"markdown","source":"Now here is the code for the `LeNet` for `image-segmentation`\n\n## 7.2.1 | Lenet-5 Architechture","metadata":{}},{"cell_type":"code","source":"inputs = Input((image_width, image_width, 1))\n\ns = Lambda(lambda x: x / 255)(inputs)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:45:32.355462Z","iopub.execute_input":"2023-06-05T09:45:32.356059Z","iopub.status.idle":"2023-06-05T09:45:32.370763Z","shell.execute_reply.started":"2023-06-05T09:45:32.356022Z","shell.execute_reply":"2023-06-05T09:45:32.369691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_1 = Conv2D(1 , (1 , 1) , activation = \"tanh\")(s)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:46:19.249174Z","iopub.execute_input":"2023-06-05T09:46:19.249709Z","iopub.status.idle":"2023-06-05T09:46:19.278313Z","shell.execute_reply.started":"2023-06-05T09:46:19.249670Z","shell.execute_reply":"2023-06-05T09:46:19.276855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_2 = Conv2D(6 , (5 , 5) , activation = \"tanh\")(conv_1)\n\navg_1 = AveragePooling2D((2 , 2) , strides = 2)(conv_2)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:46:19.410198Z","iopub.execute_input":"2023-06-05T09:46:19.410615Z","iopub.status.idle":"2023-06-05T09:46:19.437831Z","shell.execute_reply.started":"2023-06-05T09:46:19.410583Z","shell.execute_reply":"2023-06-05T09:46:19.436240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_3 = Conv2D(6 , (5 , 5) , activation = \"tanh\")(avg_1)\n\navg_2 = AveragePooling2D((2 , 2) , strides = 2)(conv_1)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:46:19.596722Z","iopub.execute_input":"2023-06-05T09:46:19.597214Z","iopub.status.idle":"2023-06-05T09:46:19.625953Z","shell.execute_reply.started":"2023-06-05T09:46:19.597173Z","shell.execute_reply":"2023-06-05T09:46:19.624023Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_1 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_3)\nunion_1 = concatenate([union_1 , conv_3])\n\nconv_3 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_1)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:46:19.793017Z","iopub.execute_input":"2023-06-05T09:46:19.793437Z","iopub.status.idle":"2023-06-05T09:46:19.847727Z","shell.execute_reply.started":"2023-06-05T09:46:19.793400Z","shell.execute_reply":"2023-06-05T09:46:19.846333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_2 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_1)\nunion_2 = concatenate([union_2 , conv_1])","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:46:20.207536Z","iopub.execute_input":"2023-06-05T09:46:20.208008Z","iopub.status.idle":"2023-06-05T09:46:20.247647Z","shell.execute_reply.started":"2023-06-05T09:46:20.207960Z","shell.execute_reply":"2023-06-05T09:46:20.246361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outputs = Conv2D(1 , (1 , 1) , activation = \"sigmoid\")(union_2)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:46:51.828801Z","iopub.execute_input":"2023-06-05T09:46:51.829225Z","iopub.status.idle":"2023-06-05T09:46:51.853426Z","shell.execute_reply.started":"2023-06-05T09:46:51.829189Z","shell.execute_reply":"2023-06-05T09:46:51.851790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lenet = tf.keras.Model(inputs=[inputs], outputs=[outputs])\nlenet.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\nlenet.summary()","metadata":{"execution":{"iopub.status.busy":"2023-06-05T09:46:52.188781Z","iopub.execute_input":"2023-06-05T09:46:52.189912Z","iopub.status.idle":"2023-06-05T09:46:52.248196Z","shell.execute_reply.started":"2023-06-05T09:46:52.189866Z","shell.execute_reply":"2023-06-05T09:46:52.246837Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7.1.2 | LENET-5 Training\n\nI dont the exact reason, but everytime I try to access `GPU` for some training in `Kaggle`. `CUDA goes out of memory`. Thus I have trained the model on `Colab` and will imported the results to `Wandb`. \n\n```\nlenet.fit(X_train , Y_train , epochs = 100)\n```","metadata":{}},{"cell_type":"markdown","source":"```\n---------------------------------------------------------------------------\nOutOfMemoryError Traceback (most recent call last)\nCell In[31], line 15\n 12 img = img.to(\"cuda\")\n 13 mask = mask.to(\"cuda\")\n---> 15 outputs = model(img) \n 17 loss = loss_func(outputs , mask)\n 19 loss.backward()\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/base/model.py:29, in SegmentationModel.forward(self, x)\n 25 \"\"\"Sequentially pass `x` trough model`s encoder, decoder and heads\"\"\"\n 27 self.check_input_shape(x)\n---> 29 features = self.encoder(x)\n 30 decoder_output = self.decoder(*features)\n 32 masks = self.segmentation_head(decoder_output)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/encoders/efficientnet.py:73, in EfficientNetEncoder.forward(self, x)\n 71 drop_connect = drop_connect_rate * block_number / len(self._blocks)\n 72 block_number += 1.0\n---> 73 x = module(x, drop_connect)\n 75 features.append(x)\n 77 return features\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/model.py:111, in MBConvBlock.forward(self, inputs, drop_connect_rate)\n 109 x = self._depthwise_conv(x)\n 110 x = self._bn1(x)\n--> 111 x = self._swish(x)\n 113 # Squeeze and Excitation\n 114 if self.has_se:\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:80, in MemoryEfficientSwish.forward(self, x)\n 79 def forward(self, x):\n---> 80 return SwishImplementation.apply(x)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/autograd/function.py:506, in Function.apply(cls, *args, **kwargs)\n 503 if not torch._C._are_functorch_transforms_active():\n 504 # See NOTE: [functorch vjp and autograd interaction]\n 505 args = _functorch.utils.unwrap_dead_wrappers(args)\n--> 506 return super().apply(*args, **kwargs) # type: ignore[misc]\n 508 if cls.setup_context == _SingleLevelFunction.setup_context:\n 509 raise RuntimeError(\n 510 'In order to use an autograd.Function with functorch transforms '\n 511 '(vmap, grad, jvp, jacrev, ...), it must override the setup_context '\n 512 'staticmethod. For more details, please see '\n 513 'https://pytorch.org/docs/master/notes/extending.func.html style=\"color:rgb(175,0,0)\">')\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:67, in SwishImplementation.forward(ctx, i)\n 65 @staticmethod\n 66 def forward(ctx, i):\n---> 67 result = i * torch.sigmoid(i)\n 68 ctx.save_for_backward(i)\n 69 return result\n\nOutOfMemoryError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 15.90 GiB total capacity; 319.37 MiB already allocated; 7.75 MiB free; 326.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF\n```","metadata":{"_kg_hide-input":true}},{"cell_type":"code","source":"IFrame(\"https://wandb.ai/ayushsinghal659/HuBMAP/reports/LENET-5-HuBMAP--Vmlldzo0NTYxMDc4\" , 1300 , 400)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T10:10:19.317913Z","iopub.execute_input":"2023-06-05T10:10:19.318384Z","iopub.status.idle":"2023-06-05T10:10:19.326669Z","shell.execute_reply.started":"2023-06-05T10:10:19.318352Z","shell.execute_reply":"2023-06-05T10:10:19.325492Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 7.2 | Alex Net ✨ \n\nThough $AlexNet$ is an `image-classification` architechture, But with small tweekings, we can modify it for `image-segemntation`.\n\n$AlexNet$ is a `convolutional neural network` $CNN$ architectur introduced by `Alex Krizhevsky` in $2012$. It was a `breakthrough model` that `won the ImageNet Large Scale Visual Recognition Challenge` $ILSVRC$ that year. Its `pioneering design`, along with the `utilization of GPUs` for training, contributed to the `resurgence of deep learning` and paved the way for subsequent advancements in computer vision.\n\nThis code represents the orgitnal `ALex Net`","metadata":{}},{"cell_type":"markdown","source":"```\nalex = Sequential()\n\nalex.add(Conv2D(96 , kernel_size = (11 , 11) , strides = 4 , activation = \"relu\" , input_shape = (227 , 227 , 3)))\nalex.add(MaxPooling2D(pool_size = (3 , 3) , strides = 2))\n\nalex.add(Conv2D(256 , kernel_size = (5 , 5) , padding = \"same\" , activation = \"relu\"))\nalex.add(MaxPooling2D(pool_size = (3 , 3) , strides = 2))\n\nalex.add(Conv2D(384 , kernel_size = (3 , 3) , padding = \"same\" , activation = \"relu\"))\nalex.add(Conv2D(384 , kernel_size = (3 , 3) , padding = \"same\" , activation = \"relu\"))\nalex.add(Conv2D(384 , kernel_size = (3 , 3) , padding = \"same\" , activation = \"relu\"))\nalex.add(MaxPooling2D(pool_size = (3 , 3) , strides = 2))\n\nalex.add(Flatten())\n\nalex.add(Dropout(rate = 0.5))\n \nalex.add(Dense(4096 , activation = \"relu\"))\nalex.add(Dropout(rate = 0.5))\n\nalex.add(Dense(4096 , activation = \"relu\"))\nalex.add(Dense(4096 , activation = \"relu\"))\n\nalex.add(Dense(1 , activation = \"sigmoid\"))\n```","metadata":{"_kg_hide-input":true}},{"cell_type":"markdown","source":"And here is the code for `image-segmentation` with `AlexNet`\n\n## 7.2.1 | AlexNet Architechture","metadata":{}},{"cell_type":"code","source":"inputs = Input((image_width, image_width, 1))\n\ns = Lambda(lambda x: x / 255)(inputs)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:31.169333Z","iopub.execute_input":"2023-06-05T16:23:31.169833Z","iopub.status.idle":"2023-06-05T16:23:31.184680Z","shell.execute_reply.started":"2023-06-05T16:23:31.169797Z","shell.execute_reply":"2023-06-05T16:23:31.183564Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_1 = Conv2D(1 , (1 , 1) , activation = \"tanh\")(s)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:31.369533Z","iopub.execute_input":"2023-06-05T16:23:31.370818Z","iopub.status.idle":"2023-06-05T16:23:31.518041Z","shell.execute_reply.started":"2023-06-05T16:23:31.370779Z","shell.execute_reply":"2023-06-05T16:23:31.516799Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_2 = Conv2D(96 , (11 , 11) , activation = \"relu\")(conv_1)\n\navg_1 = MaxPooling2D((3 , 3) , strides = 2)(conv_2)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:31.594249Z","iopub.execute_input":"2023-06-05T16:23:31.594761Z","iopub.status.idle":"2023-06-05T16:23:31.626922Z","shell.execute_reply.started":"2023-06-05T16:23:31.594726Z","shell.execute_reply":"2023-06-05T16:23:31.625971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_3 = Conv2D(256 , (5 , 5) , padding = \"same\" , activation = \"relu\")(avg_1)\n\navg_2 = MaxPooling2D((3 , 3) , strides = 2)(conv_3)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:31.890230Z","iopub.execute_input":"2023-06-05T16:23:31.890952Z","iopub.status.idle":"2023-06-05T16:23:31.930769Z","shell.execute_reply.started":"2023-06-05T16:23:31.890905Z","shell.execute_reply":"2023-06-05T16:23:31.929204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_4 = Conv2D(384 , (3 , 3) , padding = \"same\" , activation = \"relu\")(avg_2)\nconv_5 = Conv2D(384 , (3 , 3) , padding = \"same\" , activation = \"relu\")(conv_4)\nconv_6 = Conv2D(384 , (3 , 3) , padding = \"same\" , activation = \"relu\")(conv_5)\n\navg_2 = MaxPooling2D((3 , 3) , strides = 2)(conv_6)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:32.133357Z","iopub.execute_input":"2023-06-05T16:23:32.133748Z","iopub.status.idle":"2023-06-05T16:23:32.231720Z","shell.execute_reply.started":"2023-06-05T16:23:32.133720Z","shell.execute_reply":"2023-06-05T16:23:32.230405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_1 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_6)\nunion_1 = concatenate([union_1 , conv_6])\n\nconv_7 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_1)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:32.411038Z","iopub.execute_input":"2023-06-05T16:23:32.411955Z","iopub.status.idle":"2023-06-05T16:23:32.476099Z","shell.execute_reply.started":"2023-06-05T16:23:32.411915Z","shell.execute_reply":"2023-06-05T16:23:32.474840Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_2 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_3)\nunion_2 = concatenate([union_2 , conv_3])\n\nconv_8 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_2)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:32.770192Z","iopub.execute_input":"2023-06-05T16:23:32.771172Z","iopub.status.idle":"2023-06-05T16:23:32.825874Z","shell.execute_reply.started":"2023-06-05T16:23:32.771125Z","shell.execute_reply":"2023-06-05T16:23:32.824828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_3 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_2)\nunion_3 = concatenate([union_3 , conv_2])\n\nconv_9 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_3)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:33.110717Z","iopub.execute_input":"2023-06-05T16:23:33.111189Z","iopub.status.idle":"2023-06-05T16:23:33.170313Z","shell.execute_reply.started":"2023-06-05T16:23:33.111154Z","shell.execute_reply":"2023-06-05T16:23:33.168929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_4 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_1)\nunion_4 = concatenate([union_4 , conv_1])\n\nconv_10 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_4)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:33.809944Z","iopub.execute_input":"2023-06-05T16:23:33.810432Z","iopub.status.idle":"2023-06-05T16:23:33.870008Z","shell.execute_reply.started":"2023-06-05T16:23:33.810396Z","shell.execute_reply":"2023-06-05T16:23:33.868407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outputs = Conv2D(1 , (1 , 1) , activation = \"sigmoid\")(conv_10)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:35.020167Z","iopub.execute_input":"2023-06-05T16:23:35.020646Z","iopub.status.idle":"2023-06-05T16:23:35.046758Z","shell.execute_reply.started":"2023-06-05T16:23:35.020612Z","shell.execute_reply":"2023-06-05T16:23:35.045403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"AlexNet = tf.keras.Model(inputs=[inputs], outputs=[outputs])\nAlexNet.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\nAlexNet.summary()","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:23:35.948880Z","iopub.execute_input":"2023-06-05T16:23:35.949258Z","iopub.status.idle":"2023-06-05T16:23:36.005183Z","shell.execute_reply.started":"2023-06-05T16:23:35.949230Z","shell.execute_reply":"2023-06-05T16:23:36.000993Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7.2.2 | AlexNet Training\n\nI dont the exact reason, but everytime I try to access `GPU` for some training in `Kaggle`. `CUDA goes out of memory`. Thus I have trained the model on `Colab` and will imported the results to `Wandb`. \n\n```\nAlexNet.fit(X_train , Y_train , epochs = 100 , validation_split = 0.2)\n```","metadata":{}},{"cell_type":"markdown","source":"```\n---------------------------------------------------------------------------\nOutOfMemoryError Traceback (most recent call last)\nCell In[31], line 15\n 12 img = img.to(\"cuda\")\n 13 mask = mask.to(\"cuda\")\n---> 15 outputs = model(img) \n 17 loss = loss_func(outputs , mask)\n 19 loss.backward()\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/base/model.py:29, in SegmentationModel.forward(self, x)\n 25 \"\"\"Sequentially pass `x` trough model`s encoder, decoder and heads\"\"\"\n 27 self.check_input_shape(x)\n---> 29 features = self.encoder(x)\n 30 decoder_output = self.decoder(*features)\n 32 masks = self.segmentation_head(decoder_output)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/encoders/efficientnet.py:73, in EfficientNetEncoder.forward(self, x)\n 71 drop_connect = drop_connect_rate * block_number / len(self._blocks)\n 72 block_number += 1.0\n---> 73 x = module(x, drop_connect)\n 75 features.append(x)\n 77 return features\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/model.py:111, in MBConvBlock.forward(self, inputs, drop_connect_rate)\n 109 x = self._depthwise_conv(x)\n 110 x = self._bn1(x)\n--> 111 x = self._swish(x)\n 113 # Squeeze and Excitation\n 114 if self.has_se:\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:80, in MemoryEfficientSwish.forward(self, x)\n 79 def forward(self, x):\n---> 80 return SwishImplementation.apply(x)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/autograd/function.py:506, in Function.apply(cls, *args, **kwargs)\n 503 if not torch._C._are_functorch_transforms_active():\n 504 # See NOTE: [functorch vjp and autograd interaction]\n 505 args = _functorch.utils.unwrap_dead_wrappers(args)\n--> 506 return super().apply(*args, **kwargs) # type: ignore[misc]\n 508 if cls.setup_context == _SingleLevelFunction.setup_context:\n 509 raise RuntimeError(\n 510 'In order to use an autograd.Function with functorch transforms '\n 511 '(vmap, grad, jvp, jacrev, ...), it must override the setup_context '\n 512 'staticmethod. For more details, please see '\n 513 'https://pytorch.org/docs/master/notes/extending.func.html style=\"color:rgb(175,0,0)\">')\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:67, in SwishImplementation.forward(ctx, i)\n 65 @staticmethod\n 66 def forward(ctx, i):\n---> 67 result = i * torch.sigmoid(i)\n 68 ctx.save_for_backward(i)\n 69 return result\n\nOutOfMemoryError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 15.90 GiB total capacity; 319.37 MiB already allocated; 7.75 MiB free; 326.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF\n```","metadata":{"_kg_hide-input":true}},{"cell_type":"code","source":"IFrame(\"https://wandb.ai/ayushsinghal659/HuBMAP/reports/AlexNet-HuBMAP--Vmlldzo0NTY0Mzk4\" , 1300 , 400)","metadata":{"execution":{"iopub.status.busy":"2023-06-05T16:29:58.485351Z","iopub.execute_input":"2023-06-05T16:29:58.486787Z","iopub.status.idle":"2023-06-05T16:29:58.496670Z","shell.execute_reply.started":"2023-06-05T16:29:58.486713Z","shell.execute_reply":"2023-06-05T16:29:58.494958Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 7.3 | Z-F Net ✨\n\nThough $Z-FNet$ is an `image-classification` architechture, But with small tweekings, we can modify it for `image-segemntation`.\n\n$Z-FNet$ is a `convolutional neural network` $CNN$ `architecture` introduced in $2013$. It was designed to improve `object recognition` in images by `increasing the depth of the network` while maintaining computational efficiency. $ZFβˆ’Net$ achieved `state-of-the-art performance on the ImageNet\n dataset and served as a precursor to more advanced CNN\n models like VGGNet\n and ResNet\n .","metadata":{}},{"cell_type":"markdown","source":"```\nzf = Sequential()\n\nzf.add(Conv2D(96 , kernel_size = (7 , 7) , strides = 2 , activation = \"relu\" , input_shape = (224 , 224 , 3)))\nzf.add(MaxPooling2D(pool_size = (3 , 3) , strides = 2))\n\nzf.add(Conv2D(256 , kernel_size = (3 , 3) , activation = \"relu\"))\nzf.add(MaxPooling2D(pool_size = (3 , 3) , strides = 2))\n\nzf.add(Conv2D(384 , kernel_size = (3 , 3) , padding = \"same\" , activation = \"relu\"))\nzf.add(Conv2D(384 , kernel_size = (3 , 3) , padding = \"same\" , activation = \"relu\"))\nzf.add(Conv2D(256 , kernel_size = (3 , 3) , padding = \"same\" , activation = \"relu\"))\n\nzf.add(MaxPooling2D(pool_size = (3 , 3) , strides = 2))\n\nzf.add(Flatten())\n\nzf.add(Dense(4096 , activation = \"relu\"))\nzf.add(Dense(4096 , activation = \"relu\"))\n\nzf.add(Dense(1 , activation = \"sigmoid\"))\n```","metadata":{"_kg_hide-input":true}},{"cell_type":"markdown","source":"And here is the code for `image-segmentation` with `AlexNet`\n\n## 7.2.1 | AlexNet Architechture\n\nIf we look closely `Z-F Net` is very much similar to `AlexNet`. The only difference is `one of the kernel sizes `","metadata":{}},{"cell_type":"code","source":"inputs = Input((image_width, image_width, 1))\n\ns = Lambda(lambda x: x / 255)(inputs)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_1 = Conv2D(1 , (1 , 1) , activation = \"tanh\")(s)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_2 = Conv2D(96 , (11 , 11) , activation = \"relu\")(conv_1)\n\navg_1 = MaxPooling2D((3 , 3) , strides = 2)(conv_2)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_3 = Conv2D(256 , (5 , 5) , padding = \"same\" , activation = \"relu\")(avg_1)\n\navg_2 = MaxPooling2D((3 , 3) , strides = 2)(conv_3)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_4 = Conv2D(384 , (3 , 3) , padding = \"same\" , activation = \"relu\")(avg_2)\nconv_5 = Conv2D(384 , (3 , 3) , padding = \"same\" , activation = \"relu\")(conv_4)\nconv_6 = Conv2D(384 , (3 , 3) , padding = \"same\" , activation = \"relu\")(conv_5)\n\navg_2 = MaxPooling2D((3 , 3) , strides = 2)(conv_6)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_1 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_6)\nunion_1 = concatenate([union_1 , conv_6])\n\nconv_7 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_1)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_2 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_3)\nunion_2 = concatenate([union_2 , conv_3])\n\nconv_8 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_2)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_3 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_2)\nunion_3 = concatenate([union_3 , conv_2])\n\nconv_9 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_3)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_4 = Conv2DTranspose(6 , (5 , 5) , padding = \"same\")(conv_1)\nunion_4 = concatenate([union_4 , conv_1])\n\nconv_10 = Conv2D(6 , (3 , 3) , padding = \"same\")(union_4)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outputs = Conv2D(1 , (1 , 1) , activation = \"sigmoid\")(conv_10)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ZF_Net = tf.keras.Model(inputs=[inputs], outputs=[outputs])\nZF_Net.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\nZF_Net.summary()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7.3.2 | Z-F Net Training\nI dont the exact reason, but everytime I try to access `GPU` for some training in Kaggle. `CUDA goes out of memory`. Thus I have trained the model on `Colab` and imported the results to `Wandb`.\n```\nZF_Net.fit(X_train , Y_train , epochs = 100 , validation_split = 0.2)\n```","metadata":{}},{"cell_type":"markdown","source":"```\n---------------------------------------------------------------------------\nOutOfMemoryError Traceback (most recent call last)\nCell In[31], line 15\n 12 img = img.to(\"cuda\")\n 13 mask = mask.to(\"cuda\")\n---> 15 outputs = model(img) \n 17 loss = loss_func(outputs , mask)\n 19 loss.backward()\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/base/model.py:29, in SegmentationModel.forward(self, x)\n 25 \"\"\"Sequentially pass `x` trough model`s encoder, decoder and heads\"\"\"\n 27 self.check_input_shape(x)\n---> 29 features = self.encoder(x)\n 30 decoder_output = self.decoder(*features)\n 32 masks = self.segmentation_head(decoder_output)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/encoders/efficientnet.py:73, in EfficientNetEncoder.forward(self, x)\n 71 drop_connect = drop_connect_rate * block_number / len(self._blocks)\n 72 block_number += 1.0\n---> 73 x = module(x, drop_connect)\n 75 features.append(x)\n 77 return features\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/model.py:111, in MBConvBlock.forward(self, inputs, drop_connect_rate)\n 109 x = self._depthwise_conv(x)\n 110 x = self._bn1(x)\n--> 111 x = self._swish(x)\n 113 # Squeeze and Excitation\n 114 if self.has_se:\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:80, in MemoryEfficientSwish.forward(self, x)\n 79 def forward(self, x):\n---> 80 return SwishImplementation.apply(x)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/autograd/function.py:506, in Function.apply(cls, *args, **kwargs)\n 503 if not torch._C._are_functorch_transforms_active():\n 504 # See NOTE: [functorch vjp and autograd interaction]\n 505 args = _functorch.utils.unwrap_dead_wrappers(args)\n--> 506 return super().apply(*args, **kwargs) # type: ignore[misc]\n 508 if cls.setup_context == _SingleLevelFunction.setup_context:\n 509 raise RuntimeError(\n 510 'In order to use an autograd.Function with functorch transforms '\n 511 '(vmap, grad, jvp, jacrev, ...), it must override the setup_context '\n 512 'staticmethod. For more details, please see '\n 513 'https://pytorch.org/docs/master/notes/extending.func.html style=\"color:rgb(175,0,0)\">')\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:67, in SwishImplementation.forward(ctx, i)\n 65 @staticmethod\n 66 def forward(ctx, i):\n---> 67 result = i * torch.sigmoid(i)\n 68 ctx.save_for_backward(i)\n 69 return result\n\nOutOfMemoryError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 15.90 GiB total capacity; 319.37 MiB already allocated; 7.75 MiB free; 326.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF\n```","metadata":{"_kg_hide-output":true}},{"cell_type":"code","source":"IFrame(\"https://wandb.ai/ayushsinghal659/HuBMAP/reports/Z-F-Net-HuBMAP---Vmlldzo0NTczNTYz\" , 1300 , 400)","metadata":{"_kg_hide-input":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 7.4 | U-NET ✨\n\n\n\n$U-Net$ is a `convolutional neural network` $CNN$ architecture that is commonly used for `image segmentation` tasks. It has a `symmetric \"U\"-shaped architecture`, with a `contracting path` and an `expansive path`. The `contracting path learns to extract features` from the input image, while the `expansive path learns to upsample these features` and generate the output segmentation map. $U-Net$ has been shown to be very `effective` for a variety of `image segmentation tasks`, including `medical image segmentation` and `natural image segmentation`.\n\n## 7.4.1 | U-NET Architechture\n\n\n\n$U-NET$ only considers $(128 , 128)$ images, We have already resized our iamges according to this size ","metadata":{}},{"cell_type":"code","source":"X_train[0].shape","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:04:21.040899Z","iopub.execute_input":"2023-05-30T09:04:21.041369Z","iopub.status.idle":"2023-05-30T09:04:21.050896Z","shell.execute_reply.started":"2023-05-30T09:04:21.041337Z","shell.execute_reply":"2023-05-30T09:04:21.049434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_dim = 128\nimage_channels = 1","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:04:22.344175Z","iopub.execute_input":"2023-05-30T09:04:22.344613Z","iopub.status.idle":"2023-05-30T09:04:22.350388Z","shell.execute_reply.started":"2023-05-30T09:04:22.344583Z","shell.execute_reply":"2023-05-30T09:04:22.349124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"inputs = Input((image_dim, image_dim, image_channels))\nfunc = Lambda(lambda x: x / 255)(inputs)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:04:32.767810Z","iopub.execute_input":"2023-05-30T09:04:32.768257Z","iopub.status.idle":"2023-05-30T09:04:32.781974Z","shell.execute_reply.started":"2023-05-30T09:04:32.768225Z","shell.execute_reply":"2023-05-30T09:04:32.780606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_1 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(func)\nconv_1 = Dropout(0.1)(conv_1)\nconv_1 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_1)\n\npool_1 = MaxPooling2D((2, 2))(conv_1)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:05:37.720587Z","iopub.execute_input":"2023-05-30T09:05:37.721081Z","iopub.status.idle":"2023-05-30T09:05:37.765570Z","shell.execute_reply.started":"2023-05-30T09:05:37.721048Z","shell.execute_reply":"2023-05-30T09:05:37.764338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_2 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(pool_1)\nconv_2 = Dropout(0.1)(conv_2)\nconv_2 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_2)\n\npool_2 = MaxPooling2D((2, 2))(conv_2)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:06:11.422359Z","iopub.execute_input":"2023-05-30T09:06:11.422802Z","iopub.status.idle":"2023-05-30T09:06:11.473629Z","shell.execute_reply.started":"2023-05-30T09:06:11.422770Z","shell.execute_reply":"2023-05-30T09:06:11.472389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_3 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(pool_2)\nconv_3 = Dropout(0.2)(conv_3)\nconv_3 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_3)\n\npool_3 = MaxPooling2D((2, 2))(conv_3)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:07:56.414268Z","iopub.execute_input":"2023-05-30T09:07:56.414748Z","iopub.status.idle":"2023-05-30T09:07:56.460507Z","shell.execute_reply.started":"2023-05-30T09:07:56.414713Z","shell.execute_reply":"2023-05-30T09:07:56.459103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_4 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(pool_3)\nconv_4 = Dropout(0.2)(conv_4)\nconv_4 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_4)\n\npool_4 = MaxPooling2D(pool_size=(2, 2))(conv_4) ","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:07:57.984546Z","iopub.execute_input":"2023-05-30T09:07:57.984997Z","iopub.status.idle":"2023-05-30T09:07:58.041900Z","shell.execute_reply.started":"2023-05-30T09:07:57.984938Z","shell.execute_reply":"2023-05-30T09:07:58.040556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_5 = Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(pool_4)\nconv_5 = Dropout(0.3)(conv_5)\nconv_5 = Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_5)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:08:00.156555Z","iopub.execute_input":"2023-05-30T09:08:00.156957Z","iopub.status.idle":"2023-05-30T09:08:00.212949Z","shell.execute_reply.started":"2023-05-30T09:08:00.156926Z","shell.execute_reply":"2023-05-30T09:08:00.212063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_6 = Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(conv_5)\nunion_6 = concatenate([union_6, conv_4])\n\nconv_6 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(union_6)\nconv_6 = Dropout(0.2)(conv_6)\nconv_6 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_6)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:09:14.385918Z","iopub.execute_input":"2023-05-30T09:09:14.386354Z","iopub.status.idle":"2023-05-30T09:09:14.467998Z","shell.execute_reply.started":"2023-05-30T09:09:14.386326Z","shell.execute_reply":"2023-05-30T09:09:14.466851Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_7 = Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(conv_6)\nunion_7 = concatenate([union_7, conv_3])\n\nconv_7 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(union_7)\nconv_7 = Dropout(0.2)(conv_7)\nconv_7 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_7)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:10:31.596309Z","iopub.execute_input":"2023-05-30T09:10:31.596736Z","iopub.status.idle":"2023-05-30T09:10:31.665661Z","shell.execute_reply.started":"2023-05-30T09:10:31.596708Z","shell.execute_reply":"2023-05-30T09:10:31.664264Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_8 = Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(conv_7)\nunion_8 = concatenate([union_8, conv_2])\n\nconv_8 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(union_8)\nconv_8 = Dropout(0.1)(conv_8)\nconv_8 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_8)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:11:13.274918Z","iopub.execute_input":"2023-05-30T09:11:13.275388Z","iopub.status.idle":"2023-05-30T09:11:13.349174Z","shell.execute_reply.started":"2023-05-30T09:11:13.275348Z","shell.execute_reply":"2023-05-30T09:11:13.348054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_9 = Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same')(conv_8)\nunion_9 = concatenate([union_9, conv_1], axis=3)\n\nconv_9 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(union_9)\nconv_9 = Dropout(0.1)(conv_9)\nconv_9 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_9)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:11:53.350519Z","iopub.execute_input":"2023-05-30T09:11:53.350937Z","iopub.status.idle":"2023-05-30T09:11:53.419620Z","shell.execute_reply.started":"2023-05-30T09:11:53.350905Z","shell.execute_reply":"2023-05-30T09:11:53.418775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outputs = Conv2D(1 , (1 , 1) , activation = 'sigmoid')(conv_9)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:12:29.918490Z","iopub.execute_input":"2023-05-30T09:12:29.918954Z","iopub.status.idle":"2023-05-30T09:12:29.940343Z","shell.execute_reply.started":"2023-05-30T09:12:29.918923Z","shell.execute_reply":"2023-05-30T09:12:29.938712Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"u_net = tf.keras.Model(inputs=[inputs], outputs=[outputs])\nu_net.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\nu_net.summary()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-30T09:13:06.162357Z","iopub.execute_input":"2023-05-30T09:13:06.162794Z","iopub.status.idle":"2023-05-30T09:13:06.325784Z","shell.execute_reply.started":"2023-05-30T09:13:06.162763Z","shell.execute_reply":"2023-05-30T09:13:06.324234Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7.3.2 | U-NET Training\n\nI dont the exact reason, but everytime I try to access `GPU` for some training in `Kaggle`. `CUDA goes out of memory`. Thus I have trained the model on `Colab` and will imported the results to `Wandb`. ","metadata":{}},{"cell_type":"markdown","source":"```\nu_net.fit(X_train , Y_train , epochs = 100)\n```","metadata":{}},{"cell_type":"markdown","source":"```\n---------------------------------------------------------------------------\nOutOfMemoryError Traceback (most recent call last)\nCell In[31], line 15\n 12 img = img.to(\"cuda\")\n 13 mask = mask.to(\"cuda\")\n---> 15 outputs = model(img) \n 17 loss = loss_func(outputs , mask)\n 19 loss.backward()\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/base/model.py:29, in SegmentationModel.forward(self, x)\n 25 \"\"\"Sequentially pass `x` trough model`s encoder, decoder and heads\"\"\"\n 27 self.check_input_shape(x)\n---> 29 features = self.encoder(x)\n 30 decoder_output = self.decoder(*features)\n 32 masks = self.segmentation_head(decoder_output)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/encoders/efficientnet.py:73, in EfficientNetEncoder.forward(self, x)\n 71 drop_connect = drop_connect_rate * block_number / len(self._blocks)\n 72 block_number += 1.0\n---> 73 x = module(x, drop_connect)\n 75 features.append(x)\n 77 return features\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/model.py:111, in MBConvBlock.forward(self, inputs, drop_connect_rate)\n 109 x = self._depthwise_conv(x)\n 110 x = self._bn1(x)\n--> 111 x = self._swish(x)\n 113 # Squeeze and Excitation\n 114 if self.has_se:\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:80, in MemoryEfficientSwish.forward(self, x)\n 79 def forward(self, x):\n---> 80 return SwishImplementation.apply(x)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/autograd/function.py:506, in Function.apply(cls, *args, **kwargs)\n 503 if not torch._C._are_functorch_transforms_active():\n 504 # See NOTE: [functorch vjp and autograd interaction]\n 505 args = _functorch.utils.unwrap_dead_wrappers(args)\n--> 506 return super().apply(*args, **kwargs) # type: ignore[misc]\n 508 if cls.setup_context == _SingleLevelFunction.setup_context:\n 509 raise RuntimeError(\n 510 'In order to use an autograd.Function with functorch transforms '\n 511 '(vmap, grad, jvp, jacrev, ...), it must override the setup_context '\n 512 'staticmethod. For more details, please see '\n 513 'https://pytorch.org/docs/master/notes/extending.func.html style=\"color:rgb(175,0,0)\">')\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:67, in SwishImplementation.forward(ctx, i)\n 65 @staticmethod\n 66 def forward(ctx, i):\n---> 67 result = i * torch.sigmoid(i)\n 68 ctx.save_for_backward(i)\n 69 return result\n\nOutOfMemoryError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 15.90 GiB total capacity; 319.37 MiB already allocated; 7.75 MiB free; 326.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF\n```","metadata":{"_kg_hide-output":true,"_kg_hide-input":true}},{"cell_type":"code","source":"IFrame(\"https://wandb.ai/ayushsinghal659/HuBMAP/reports/HuBMAP-U-NET--Vmlldzo0NDk2MjEz\" , 1300 , 400)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T05:04:04.450496Z","iopub.execute_input":"2023-05-30T05:04:04.450882Z","iopub.status.idle":"2023-05-30T05:04:04.457889Z","shell.execute_reply.started":"2023-05-30T05:04:04.450854Z","shell.execute_reply":"2023-05-30T05:04:04.456714Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I knwo the training didnt go actuall well. We will try to figure out what we did wrong in the upcoming version\n\n# 7.4 | V-NET ✨\n\n\n\n$V-NET$ is a `convolutional neural network architecture` for $3D$ medical `image segmentation`. It is a fully `convolutional network` that uses a `combination of convolution` and `pooling operations` in the `encoder path`. $V-NET$ also uses `upsampling operations` in the `decoder path`. $V-NET$ uses `skip connections between the encoder and decoder` paths. $V-NET$ has been shown to be `effective` for a variety of $3D$ medical `image segmentation` tasks. $V-NET$ is a `promising architecture` for $3D$ medical `image segmentation`.\n\n## 7.4.1 | V-NET Architechture","metadata":{}},{"cell_type":"code","source":"inputs = Input((image_dim , image_dim , image_channels))\nfunc = Lambda(lambda x: x / 255)(inputs)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:21:08.531138Z","iopub.execute_input":"2023-05-30T09:21:08.531540Z","iopub.status.idle":"2023-05-30T09:21:08.546477Z","shell.execute_reply.started":"2023-05-30T09:21:08.531512Z","shell.execute_reply":"2023-05-30T09:21:08.545158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_1 = Conv2D(64 , (3 , 3) , activation = 'relu' , kernel_initializer = 'he_normal' , padding='same')(func)\nconv_1 = Dropout(0.1)(conv_1)\nconv_1 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_1)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:24:59.516567Z","iopub.execute_input":"2023-05-30T09:24:59.517105Z","iopub.status.idle":"2023-05-30T09:24:59.561381Z","shell.execute_reply.started":"2023-05-30T09:24:59.517069Z","shell.execute_reply":"2023-05-30T09:24:59.560022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_2 = Conv2D(64 , (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_1)\nconv_2 = Dropout(0.1)(conv_2)\nconv_2 = Conv2D(64 , (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_2)\n\npool_2 = MaxPooling2D((2, 2))(conv_2)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:25:07.269203Z","iopub.execute_input":"2023-05-30T09:25:07.269628Z","iopub.status.idle":"2023-05-30T09:25:07.315635Z","shell.execute_reply.started":"2023-05-30T09:25:07.269596Z","shell.execute_reply":"2023-05-30T09:25:07.314344Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_3 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(pool_2)\nconv_3 = Dropout(0.2)(conv_3)\nconv_3 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_3)\n\npool_3 = MaxPooling2D((2, 2))(conv_3)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:25:14.542227Z","iopub.execute_input":"2023-05-30T09:25:14.542638Z","iopub.status.idle":"2023-05-30T09:25:14.591841Z","shell.execute_reply.started":"2023-05-30T09:25:14.542609Z","shell.execute_reply":"2023-05-30T09:25:14.590375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_4 = Conv2D(16 , (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(pool_3)\nconv_4 = Dropout(0.2)(conv_4)\nconv_4 = Conv2D(16 , (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_4)\n\npool_4 = MaxPooling2D(pool_size=(2, 2))(conv_4)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:25:21.886151Z","iopub.execute_input":"2023-05-30T09:25:21.886607Z","iopub.status.idle":"2023-05-30T09:25:21.931885Z","shell.execute_reply.started":"2023-05-30T09:25:21.886576Z","shell.execute_reply":"2023-05-30T09:25:21.930564Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv_5 = Conv2D(8 , (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(pool_4)\nconv_5 = Dropout(0.3)(conv_5)\nconv_5 = Conv2D(8 , (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_5)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:25:30.362730Z","iopub.execute_input":"2023-05-30T09:25:30.363163Z","iopub.status.idle":"2023-05-30T09:25:30.407179Z","shell.execute_reply.started":"2023-05-30T09:25:30.363133Z","shell.execute_reply":"2023-05-30T09:25:30.405838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_6 = Conv2DTranspose(8, (2, 2), strides=(2, 2), padding='same')(conv_5)\nunion_6 = concatenate([union_6, conv_4])\n\nconv_6 = Conv2D(8, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(union_6)\nconv_6 = Dropout(0.2)(conv_6)\nconv_6 = Conv2D(8, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_6)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:25:37.887671Z","iopub.execute_input":"2023-05-30T09:25:37.888102Z","iopub.status.idle":"2023-05-30T09:25:37.969536Z","shell.execute_reply.started":"2023-05-30T09:25:37.888071Z","shell.execute_reply":"2023-05-30T09:25:37.968242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_7 = Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same')(conv_6)\nunion_7 = concatenate([union_7, conv_3])\n\nconv_7 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(union_7)\nconv_7 = Dropout(0.2)(conv_7)\nconv_7 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_7)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:25:46.299671Z","iopub.execute_input":"2023-05-30T09:25:46.300098Z","iopub.status.idle":"2023-05-30T09:25:46.385068Z","shell.execute_reply.started":"2023-05-30T09:25:46.300069Z","shell.execute_reply":"2023-05-30T09:25:46.383886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"union_8 = Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(conv_7)\nunion_8 = concatenate([union_8, conv_2])\n\nconv_8 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(union_8)\nconv_8 = Dropout(0.1)(conv_8)\nconv_8 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(conv_8)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:25:53.645240Z","iopub.execute_input":"2023-05-30T09:25:53.645644Z","iopub.status.idle":"2023-05-30T09:25:53.721575Z","shell.execute_reply.started":"2023-05-30T09:25:53.645615Z","shell.execute_reply":"2023-05-30T09:25:53.720326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outputs = Conv2D(1, (1, 1), activation='sigmoid')(conv_8)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:26:02.762766Z","iopub.execute_input":"2023-05-30T09:26:02.763250Z","iopub.status.idle":"2023-05-30T09:26:02.790227Z","shell.execute_reply.started":"2023-05-30T09:26:02.763214Z","shell.execute_reply":"2023-05-30T09:26:02.789081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"v_net = tf.keras.Model(inputs=[inputs], outputs=[outputs])\nv_net.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\nv_net.summary()","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:26:18.707646Z","iopub.execute_input":"2023-05-30T09:26:18.708146Z","iopub.status.idle":"2023-05-30T09:26:18.842588Z","shell.execute_reply.started":"2023-05-30T09:26:18.708111Z","shell.execute_reply":"2023-05-30T09:26:18.840194Z"},"_kg_hide-output":true,"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7.4.2 | V-NET Training\n\nI dont the exact reason, but everytime I try to access `GPU` for some training in `Kaggle`. `CUDA goes out of memory`. Thus I have trained the model on `Colab` and will imported the results to `Wandb`. ","metadata":{}},{"cell_type":"markdown","source":"```\nv_net.fit(X_train , Y_train , epochs = 100)\n```","metadata":{}},{"cell_type":"markdown","source":"```\n---------------------------------------------------------------------------\nOutOfMemoryError Traceback (most recent call last)\nCell In[31], line 15\n 12 img = img.to(\"cuda\")\n 13 mask = mask.to(\"cuda\")\n---> 15 outputs = model(img) \n 17 loss = loss_func(outputs , mask)\n 19 loss.backward()\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/base/model.py:29, in SegmentationModel.forward(self, x)\n 25 \"\"\"Sequentially pass `x` trough model`s encoder, decoder and heads\"\"\"\n 27 self.check_input_shape(x)\n---> 29 features = self.encoder(x)\n 30 decoder_output = self.decoder(*features)\n 32 masks = self.segmentation_head(decoder_output)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/segmentation_models_pytorch/encoders/efficientnet.py:73, in EfficientNetEncoder.forward(self, x)\n 71 drop_connect = drop_connect_rate * block_number / len(self._blocks)\n 72 block_number += 1.0\n---> 73 x = module(x, drop_connect)\n 75 features.append(x)\n 77 return features\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/model.py:111, in MBConvBlock.forward(self, inputs, drop_connect_rate)\n 109 x = self._depthwise_conv(x)\n 110 x = self._bn1(x)\n--> 111 x = self._swish(x)\n 113 # Squeeze and Excitation\n 114 if self.has_se:\n\nFile /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)\n 1496 # If we don't have any hooks, we want to skip the rest of the logic in\n 1497 # this function, and just call forward.\n 1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks\n 1499 or _global_backward_pre_hooks or _global_backward_hooks\n 1500 or _global_forward_hooks or _global_forward_pre_hooks):\n-> 1501 return forward_call(*args, **kwargs)\n 1502 # Do not call functions when jit is used\n 1503 full_backward_hooks, non_full_backward_hooks = [], []\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:80, in MemoryEfficientSwish.forward(self, x)\n 79 def forward(self, x):\n---> 80 return SwishImplementation.apply(x)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/autograd/function.py:506, in Function.apply(cls, *args, **kwargs)\n 503 if not torch._C._are_functorch_transforms_active():\n 504 # See NOTE: [functorch vjp and autograd interaction]\n 505 args = _functorch.utils.unwrap_dead_wrappers(args)\n--> 506 return super().apply(*args, **kwargs) # type: ignore[misc]\n 508 if cls.setup_context == _SingleLevelFunction.setup_context:\n 509 raise RuntimeError(\n 510 'In order to use an autograd.Function with functorch transforms '\n 511 '(vmap, grad, jvp, jacrev, ...), it must override the setup_context '\n 512 'staticmethod. For more details, please see '\n 513 'https://pytorch.org/docs/master/notes/extending.func.html style=\"color:rgb(175,0,0)\">')\n\nFile /opt/conda/lib/python3.10/site-packages/efficientnet_pytorch/utils.py:67, in SwishImplementation.forward(ctx, i)\n 65 @staticmethod\n 66 def forward(ctx, i):\n---> 67 result = i * torch.sigmoid(i)\n 68 ctx.save_for_backward(i)\n 69 return result\n\nOutOfMemoryError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 15.90 GiB total capacity; 319.37 MiB already allocated; 7.75 MiB free; 326.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF\n```","metadata":{"_kg_hide-input":true,"_kg_hide-output":true}},{"cell_type":"code","source":"IFrame(\"https://wandb.ai/ayushsinghal659/V-Net%20HuBMAP/reports/V-NET-HuBMAP--Vmlldzo0NTAyMTMy\" , 1300 , 400)","metadata":{"execution":{"iopub.status.busy":"2023-05-30T09:37:26.290685Z","iopub.execute_input":"2023-05-30T09:37:26.291203Z","iopub.status.idle":"2023-05-30T09:37:26.300882Z","shell.execute_reply.started":"2023-05-30T09:37:26.291168Z","shell.execute_reply":"2023-05-30T09:37:26.299199Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 8 | TO DO LIST πŸ“ƒ\n\n```\nTO DO 1 : IMPORVE THE TRAINING\n\nTO DO 2 : VISUALIZE THE DATA \n\nTO DO 3 : DIG IN THE DATA MORE\n\nTO DO 4 : DANCE ON \"I LIKE TO MOVE IT MOVE IT\"\n```\n\n\n\n# 9 | Ending 🫑\n\n**THAT IT FOR TODAY GUYS**\n\n**WE WILL GO DEEPER INTO THE DATA IN THE UPCOMING VERSIONS**\n\n**PLEASE COMMENT YOUR THOUGHTS, HIHGLY APPRICIATED**\n\n**DONT FORGET TO MAKE AN UPVOTE, IF YOU LIKED MY WORK :)**\n\n\n\n**PEACE OUT !!!! :)**","metadata":{}}]}