{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":71549,"databundleVersionId":8561470,"sourceType":"competition"},{"sourceId":9275623,"sourceType":"datasetVersion","datasetId":5613828}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.12"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport cv2\nimport pydicom\nimport numpy as np\nimport os\nimport glob\nfrom tqdm import tqdm\nimport gc\nimport pickle\n\nimport torchvision\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset\nfrom fastai.vision.all import *\nimport segmentation_models_pytorch as smp\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2024-09-02T15:02:54.904972Z","iopub.status.busy":"2024-09-02T15:02:54.904583Z","iopub.status.idle":"2024-09-02T15:03:25.419018Z","shell.execute_reply":"2024-09-02T15:03:25.417922Z","shell.execute_reply.started":"2024-09-02T15:02:54.904933Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"FOLDS = [1,2,3,4,5]\nPATCH_SIZE = 512\npatch_size = 64\nLmax = 5\nBS = 64\nEPOCHS = 5\nSEED = 777\nlevel_swap = .1\nSWAPS = [[1,2],[2,3]]","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:25.421385Z","iopub.status.busy":"2024-09-02T15:03:25.421014Z","iopub.status.idle":"2024-09-02T15:03:25.427045Z","shell.execute_reply":"2024-09-02T15:03:25.425814Z","shell.execute_reply.started":"2024-09-02T15:03:25.421342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![image.png](attachment:image.png)","metadata":{},"attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAjkAAAGdCAIAAAB/7PVfAAAAAXNSR0IArs4c6QAAIABJREFUeAHtnQ9sVFW+x08t7bS1tKVLKX9Kq0hhNzQU3kMEF928IH8EpCtZ8RWsLGxEVt4KUWnTADbRtFYluBulpiaI2YjlgSmQlQLxARLgyZ+GUigFQapYETVbnLGrtEB7ns8bfnecMz3cmc6c++87Mct3zjn3/r7n87tzv9tyRxnHCwRAAARAAASsTYBZ2x7cgQAIgAAIgABHVuEiAAEQAAEQsDoBZJXVOwR/IAACIAACyCpcAyAAAiAAAlYngKyyeofgDwRAAARAAFmFawAEQAAEQMDqBJBVVu8Q/IEACIAACCCrcA2AAAiAAAhYnYC9s6qrq6u1tdXr9frwAgEQAAEQsDMBr9fb2tra1dUVNDbtnVWtra0MLxAAARAAAacQaG1tdWBWeb1exlhra6ud/88EvIMACIAACPi0nz28Xq8Ds8rn8zHGfD5f0L1hEARAAARAwC4E5Pdze/8OUL43u3QIPkEABEAABOT3c2QVrhAQAAEQAAHzCSCrzO8BHIAACIAACMgJIKvkfDALAiAAAiBgPgFklfk9gAMQAAEQAAE5AWSVnA9mQQAEQAAEzCeArDK/B3AAAiAAAiAgJ4CskvPBLAiAAAiAgPkEkFXm9wAOQAAEQAAE5ASQVXI+mAUBEAABEDCfALLK/B7AAQiAAAiAgJwAskrOB7MgAAIgAALmE0BWmd8DOAABEAABEJATQFbJ+WAWBEAABEDAfALIKvN7AAcOIJBT8oHKfxxADFsAgZAIIKtCwoXFIBCcgMqgyin5ILgJjIKAcwkgq5zbW+xMIQFklULYKOVGAsgqN3Yde444AWRVxJHihCDgTwBZ5U8DGgTCJICsChMcDgMBYwSQVcY4YRUISAkgq6R4MAkCvSWArOotQRwPApxzZBUuAxCIKgFkVVTx4uRuIYCsckunsU+TCCCrTAKPss4igKxyVj+xG8sRiEBW7d+/f9asWYMGDWKMbd261X+Lzc3NDz30UEpKSlJS0rhx4y5evKjNXr169amnnkpPT7/99tvnzJnz9ddf01EXL16cMWNGYmJiRkbGc889d/36dZrat2/f2LFj4+Pj77rrrg0bNtB4T0K+t56OwjgIhEEAWRUGNBwCAsYJyO/nzMiJ6urqVq5cWVtbG5BVn376aXp6+ooVK44fP/7pp59u3779m2++0U64ZMmSoUOH7tmzp76+fsKECffee682fuPGjby8vAceeKChoaGurq5///6lpaXaVEtLS1JS0jPPPNPc3Pz666/Hxsbu2rVLbk++N/mxmAWBkAggq0LChcUgECoB+f3cUFZRyYCsevTRRx977DGaJeH1euPi4rZs2aKNnDlzhjH28ccfc87r6upuu+02+jHrzTffTElJ6ezs5JwXFxePGjWKTvLoo49OmzaN3gYV8r0FPQSDIBAeAWRVeNxwFAgYJCC/n4efVV1dXcnJyS+88MLUqVMzMjLGjx9Pvx7cs2cPY+y7774ji9nZ2WvXruWcr169Oj8/n8ZbWloYY8ePH+ec33fffcuWLaOpt99+OyUlhd6S6Ojo8N18tba2MsZ8Ph/NQoBAlAggq6IEFqcFAY1AtLLq8uXLjLGkpKS1a9c2NDS89NJLMTExH330Eed848aN8fHx/g24++67i4uLOedPPPHE1KlTaeqHH35gjNXV1f10VG5ubkVFBU3t2LGDMfbjjz/SiCbKysrYL1/IqgBEeBsNAsiqaFDFOUGACEQrqy5dusQYKywspEoPPfTQf/7nf0Y7q/BzFQGHUEkAWaWSNmq5kEC0sqqzs7NPnz4vvvgiMS0uLtaeoYjq7wCpHOdcvjf/ldAg0EsCyKpeAsThICAnIL+fh//3VZzziRMn+j9b8fvf/177MUt7tuL999/XnJ09ezbg2Qp6XLC6ujolJaWjo0N7tiIvL482U1hYiGcriAaE6QSQVaa3AAacTSACWdXe3t7w84sxpv3tlPY9qtra2ri4uLfeeuv8+fPaU+YHDhzQaC5ZsiQ7O3vv3r319fUTf35p49oz61OnTj1x4sSuXbsyMjICnllfsWLFmTNn1q1bh2fWnX1d2m53yCrbtQyG7UUgAlm1b9++Xz7NwBYsWKBRWL9+/fDhwxMSEvLz87dt20ZotO8C9+vXLykp6eGHH758+TJNff755w8++GBiYmL//v2fffbZgO8CjxkzJj4+ftiwYfguMBGDsAIBZJUVugAPDiYQgayyLB353ixrG8bsSABZZceuwbONCMjv56H9fZXVti3fm9Xcwo+tCSCrbN0+mLc+Afn9HFll/Q7CoSUIIKss0QaYcC4BZJVze4udKSSArFIIG6XcSABZ5cauY88RJ4CsijhSnBAE/Akgq/xpQINAmASQVWGCw2EgYIwAssoYJ6wCASkBZJUUDyZBoLcEkFW9JYjjQYBzjqzCZQACUSWArIoqXpzcLQSQVW7pNPZpEgFklUngUdZZBJBVzuondmM5Asgqy7UEhuxIAFllx67Bs40IIKts1CxYtS4BZJV1ewNnjiCArHJEG7EJswkgq8zuAOo7nACyyuENxvbUEEBWqeGMKq4lgKxybeux8UgSQFZFkibOBQICAWSVgAQDIBA6AWRV6MxwBAiEQABZFQIsLAWBngggq3oig3EQiAgBZFVEMOIkbieArHL7FYD9R5kAsirKgHF6dxBAVrmjz9ilaQSQVaahR2EnEUBWOamb2IsFCSCrLNgUWLIfAWSV/XoGx7YigKyyVbtg1qoEkFVW7Qx8OYQAssohjcQ2zCWArDKXP6o7ngCyyvEtxgZVEEBWqaCMGi4mgKxycfOx9cgRQFZFjiXOBAJBCCCrgkDBEAiESgBZFSoxrAeBkAggq0LChcUgEJwAsio4F4yCQIQIIKsiBBKncTcBZJW7+4/dR50AsirqiFHADQSQVW7oMvZoIgFklYnwUdo5BJBVzukldmJJAsgqS7YFpuxGAFllt47Br80IIKts1jDYtSYBZJU1+wJXjiGArHJMK7ERMwkgq8ykj9ouIBCBrNq/f/+sWbMGDRrEGNu6dasI7cknn2SMvfbaazTV1tY2b968vn37pqamLlq0qL29naYaGxsnTZrk8XiysrJefvllGuecb968eeTIkR6PJy8vb8eOHf5TQbV8b0EPwSAIhEcAWRUeNxwFAgYJyO/nzMhZ6urqVq5cWVtbGzSramtr8/PzBw8e7J9V06dPz8/PP3z48IEDB4YPH15YWKgV8vl8mZmZ8+fPb2pqqqmpSUxMrK6u1qYOHToUGxv7yiuvNDc3r1q1Ki4u7tSpU3J78r3Jj8UsCIREAFkVEi4sBoFQCcjv54ayikqKWfXll18OGTKkqakpJyeHsqq5uZkxduzYMe3AnTt3xsTEXLp0iXNeVVXVr1+/zs5ObaqkpGTkyJGanjt37syZM6nWPffc8+STT9LboEK+t6CHYBAEwiOArAqPG44CAYME5PfzXmVVV1fXf/zHf/z1r3/lnPtn1fr169PS0sjf9evXY2Nja2trOedFRUUFBQU0tXfvXsbYlStXOOdDhw6ltOOcP//886NHj6aVJDo6Onw3X62trYwxn89HsxAgECUCyKoogcVpQUAjEMWsqqiomDJlSnd3d0BWlZeXjxgxwr8BGRkZVVVVnPMpU6YsXryYpk6fPs0Ya25u5pzHxcW99957NLVu3boBAwbQWxJlZWXsly9kFcGBiB4BZFX02OLMIMA5j1ZW1dfXZ2Zmar/ZU5lV+LkKl7UpBJBVpmBHUfcQiFZWvfbaazExMbE3X4yx2267LScn56ff5kX1d4D+nZPvzX8lNAj0kgCyqpcAcTgIyAnI7+fh/33VP//5z1N+r8GDB5eUlJw9e5Zzrj1bUV9frznbvXt3wLMV165d06ZKS0v9n62YNWsWbWbixIl4toJoQJhOAFllegtgwNkEIpBV7e3tDT+/GGNr165taGi4ePFiADX/Zys459OnTx87duyRI0cOHjyYm5tLz6x7vd7MzMyioqKmpqZNmzYlJSX5P7Pep0+fNWvWnDlzpqysDM+sBxDGW3MJIKvM5Y/qjicQgazat2/fL59mYAsWLAgAF5BVbW1thYWFycnJKSkpCxcuDPpd4CFDhlRWVvqfZ/PmzSNGjIiPjx81ahS+C+xPBtp0Asgq01sAA84mEIGssiwg+d4saxvG7EgAWWXHrsGzjQjI7+eh/X2V1bYt35vV3MKPrQkgq2zdPpi3PgH5/RxZZf0OwqElCCCrLNEGmHAuAWSVc3uLnSkkgKxSCBul3EgAWeXGrmPPESeArIo4UpwQBPwJIKv8aUCDQJgEkFVhgsNhIGCMALLKGCesAgEpAWSVFA8mQaC3BJBVvSWI40Hg//+NlyUfqPwHzEHAbQSQVW7rOPYbFQIqgyqn5IOo7AEnBQELE0BWWbg5sGYfAsgq+/QKTm1JAFlly7bBtNUIIKus1hH4cRgBZJXDGortmEMAWWUOd1R1DQFklWtajY1GkwCyKpp0cW4QiOh/F9hqOOU5bDW38GNrAsgqW7cP5q1PQH4/x78P0PodhENLEEBWWaINMOFcAsgq5/YWO1NIAFmlEDZKuZEAssqNXceeI04AWRVxpDghCPgTQFb504AGgTAJIKvCBIfDQMAYAWSVMU5YBQJSAsgqKR5MgkBvCSCreksQx4MA/n2AuAZAINoEkFXRJozzu4IAfq5yRZuxSfMIIKvMY4/KDiKArHJQM7EVKxJAVlmxK/BkOwLIKtu1DIbtRQBZZa9+wa1FCSCrLNoY2HIKAWSVUzqJfZhKAFllKn4Udz4BZJXze4wdKiCArFIAGSXcTABZ5ebuY+8RI4CsihhKnAgEghFAVgWjgjEQCJEAsipEYFgOAqERQFaFxgurQSAoAWRVUCwYBIFIEUBWRYokzuNqAsgqV7cfm48+AWRV9BmjggsIIKtc0GRs0UwCyCoz6aO2YwggqxzTSmzEmgQikFX79++fNWvWoEGDGGNbt27V9nnt2rXi4uK8vLykpKRBgwYVFRVdunSJELS1tc2bN69v376pqamLFi1qb2+nqcbGxkmTJnk8nqysrJdffpnGOeebN28eOXKkx+PJy8vbsWOH/1RQLd9b0EMwCALhEUBWhccNR4GAQQLy+7mh/4Z9XV3dypUra2tr/bPK6/U+8MAD//3f/3327NmPP/54/Pjx//7v/06epk+fnp+ff/jw4QMHDgwfPrywsFCb8vl8mZmZ8+fPb2pqqqmpSUxMrK6u1qYOHToUGxv7yiuvNDc3r1q1Ki4u7tSpU3TCoEK+t6CHYBAEwiOArAqPG44CAYME5PdzQ1lFlfyzigY1cfToUcbYxYsXOefNzc2MsWPHjmlTO3fujImJ0X7qqqqq6tevX2dnpzZVUlIycuRITc+dO3fmzJma5pzfc889Tz75JL0NKuR7C3oIBkEgPALIqvC44SgQMEhAfj+PWFZ9+OGHMTExPp+Pc75+/fq0tDTyd/369djY2NraWs55UVFRQUEBTe3du5cxduXKlZ+OGjp06GuvvUZTzz///OjRo+ktiY6ODt/NV2trK2NMK0oLIEAgGgSQVdGginOCABFQkVVXr179t3/7t3nz5mlVy8vLR4wYQQ445xkZGVVVVZzzKVOmLF68mKZOnz7NGGtubuacx8XFvffeezS1bt26AQMG0FsSZWVl7JcvZBXBgYgeAWRV9NjizCDAOY96Vl27du2hhx4aO3YsZUZUswo/V+GyNoUAssoU7CjqHgLRzapr1679/ve/Hz169D//+U9iGtXfAVKVW+aw/0poEOglAWRVLwHicBCQE4hiVmlBNWrUqG+//dbfhPZsRX19vTa4e/fugGcrrl27pk2Vlpb6P1sxa9YsOs/EiRPxbAXRgDCdALLK9BbAgLMJRCCr2tvbG35+McbWrl3b0NBw8eLFa9euzZ49Oysr68SJE5dvvugBv+nTp48dO/bIkSMHDx7Mzc2lZ9a9Xm9mZmZRUVFTU9OmTZt++m6W/zPrffr0WbNmzZkzZ8rKyvDMurOvS9vtDlllu5bBsL0IRCCr9u3b98unGdiCBQs+++yzgEHG2L59+zQ6bW1thYWFycnJKSkpCxcuDPpd4CFDhlRWVvrT3Lx584gRI+Lj40eNGoXvAvuTgTadALLK9BbAgLMJRCCrLAtIvjfL2oYxOxJAVtmxa/BsIwLy+3lo36+y2rble7OaW/ixNQFkla3bB/PWJyC/nyOrrN9BOLQEAWSVJdoAE84lgKxybm+xM4UEkFUKYaOUGwkgq9zYdew54gSQVRFHihOCgD8BZJU/DWgQCJMAsipMcDgMBIwRQFYZ44RVICAlgKyS4sEkCPSWALKqtwRxPAhwzpFVuAxAIKoEkFVRxYuTu4UAssotncY+TSKArDIJPMo6iwCyyln9xG4sRwBZZbmWwJAdCSCr7Ng1eLYRAWSVjZoFq9YlgKyybm/gzBEEkFWOaCM2YTYBZJXZHUB9hxNAVjm8wdieGgLIKjWcUcW1BJBVrm09Nh5JAsiqSNLEuUBAIICsEpBgAARCJ4CsCp0ZjgCBEAggq0KAhaUg0BMBZFVPZDAOAhEhgKyKCEacxO0EkFVuvwKw/ygTQFZFGTBO7w4CyCp39Bm7NI0Asso09CjsJALIKid1E3uxIAFklQWbAkv2I4Cssl/P4NhWBJBVtmoXzFqVALLKqp2BL4cQQFY5pJHYhrkEkFXm8kd1xxNAVjm+xdigCgLIKhWUUcPFBJBVLm4+th45AsiqyLHEmUAgCAFkVRAoGAKBUAkgq0IlhvUgEBIBZFVIuLAYBIITQFYF54JREIgQAWRVhEDiNO4mgKxyd/+x+6gTQFZFHTEKuIEAssoNXcYeTSSArDIRPko7hwCyyjm9xE4sSQBZZcm2wJTdCCCr7NYx+LUZAWSVzRoGu9YkgKyyZl/gyjEEIpBV+/fvnzVr1qBBgxhjW7duJTTd3d2rV68eOHBgQkLC5MmTz507R1NtbW3z5s3r27dvamrqokWL2tvbaaqxsXHSpEkejycrK+vll1+mcc755s2bR44c6fF48vLyduzY4T8VVMv3FvQQDIJAeASQVeFxw1EgYJCA/H7OjJylrq5u5cqVtbW1AVlVWVmZmpq6bdu2xsbG2bNn33nnnVevXtVOOH369Pz8/MOHDx84cGD48OGFhYXauM/ny8zMnD9/flNTU01NTWJiYnV1tTZ16NCh2NjYV155pbm5edWqVXFxcadOnZLbk+9NfixmQSAkAsiqkHBhMQiESkB+PzeUVVTSP6u6u7sHDhz46quvarNer9fj8dTU1HDOm5ubGWPHjh3Tpnbu3BkTE3Pp0iXOeVVVVb9+/To7O7WpkpKSkSNHanru3LkzZ87UNOf8nnvuefLJJ+ltUCHfW9BDMAgC4RFAVoXHDUeBgEEC8vt5+Fl14cIFxlhDQwP5uP/++59++mnO+fr169PS0mj8+vXrsbGxtbW1nPOioqKCggKa2rt3L2PsypUrPx01dOjQ1157jaaef/750aNH01sSHR0dvpuv1tZWxpjP56NZCBCIEgFkVZTA4rQgoBGIVlYdOnSIMfbVV18R6EceeWTu3Lmc8/Ly8hEjRtA45zwjI6OqqopzPmXKlMWLF9PU6dOnGWPNzc0/HRUXF/fee+/R1Lp16wYMGEBvSZSVlbFfvpBVBAciegSQVdFjizODAOfcaVmFn6twWZtCAFllCnYUdQ+BaGWVWb8D9O+cfG/+K6FBoJcEkFW9BIjDQUBOQH4/D//vq7RnK9asWaOV9/l8Ac9W1NfXa1O7d+8OeLbi2rVr2lRpaan/sxWzZs2izUycOBHPVhANCNMJIKtMbwEMOJtABLKqvb294ecXY2zt2rUNDQ0XL17knFdWVqalpW3fvv3kyZMFBQUBz6yPHTv2yJEjBw8ezM3NpWfWvV5vZmZmUVFRU1PTpk2bkpKS/J9Z79Onz5o1a86cOVNWVoZn1p19Xdpud8gq27UMhu1FIAJZtW/fvl8+zcAWLFjAOde+C5yZmenxeCZPnvzJJ58Qmra2tsLCwuTk5JSUlIULFwb9LvCQIUMqKyvpEO27wCNGjIiPjx81ahS+C+xPBtp0Asgq01sAA84mEIGssiwg+d4saxvG7EgAWWXHrsGzjQjI7+eh/X2V1bYt35vV3MKPrQkgq2zdPpi3PgH5/RxZZf0OwqElCCCrLNEGmHAuAWSVc3uLnSkkgKxSCBul3EgAWeXGrmPPESeArIo4UpwQBPwJIKv8aUCDQJgEkFVhgsNhIGCMALLKGCesAgEpAWSVFA8mQaC3BJBVvSWI40GAc46swmUAAlElgKyKKl6c3C0EkFVu6TT2aRIBZJVJ4FHWWQSQVc7qJ3ZjOQLIKsu1BIbsSABZZceuwbONCCCrbNQsWLUuAWSVdXsDZ44ggKxyRBuxCbMJIKvM7gDqO5wAssrhDcb21BBAVqnhjCquJYCscm3rsfFIEkBWRZImzgUCAgFklYAEAyAQOgFkVejMcAQIhEAAWRUCLCwFgZ4IIKt6IoNxEIgIAWRVRDDiJG4ngKxy+xWA/UeZALIqyoBxencQQFa5o8/YpWkEkFWmoUdhJxFAVjmpm9iLBQkgqyzYFFiyHwFklf16Bse2IoCsslW7YNaqBJBVVu0MfDmEALLKIY3ENswlgKwylz+qO54AssrxLcYGVRBAVqmgjBouJoCscnHzsfXIEUBWRY4lzgQCQQggq4JAwRAIhEoAWRUqMawHgZAIIKtCwoXFIBCcALIqOBeMgkCECCCrIgQSp3E3AWSVu/uP3UedALIq6ohRwA0EkFVu6DL2aCIBZJWJ8FHaOQSQVc7pJXZiSQLIKku2BabsRgBZZbeOwa/NCCCrbNYw2LUmAWSVNfsCV44hgKxyTCuxETMJIKvMpI/aLiAQxay6cePGqlWr7rjjjoSEhGHDhr3wwgvd3d0a0u7u7tWrVw8cODAhIWHy5Mnnzp0j1G1tbfPmzevbt29qauqiRYva29tpqrGxcdKkSR6PJysr6+WXX6bxnoR8bz0dhXEQCIMAsioMaDgEBIwTkN/PmfETiSvLy8t/9atfffDBB5999tmWLVuSk5P/9re/acsqKytTU1O3bdvW2Ng4e/bsO++88+rVq9rU9OnT8/PzDx8+fODAgeHDhxcWFmrjPp8vMzNz/vz5TU1NNTU1iYmJ1dXVYlH/Efne/FdCg0AvCSCregkQh4OAnID8ft6rrJo5c+aiRYuo/Jw5c+bPn8857+7uHjhw4KuvvqpNeb1ej8dTU1PDOW9ubmaMHTt2TJvauXNnTEzMpUuXOOdVVVX9+vXr7OzUpkpKSkaOHEknDyrkewt6CAZBIDwCyKrwuOEoEDBIQH4/71VWlZeX5+TkfPLJJ5zzEydODBgw4N133+WcX7hwgTHW0NBAFu+///6nn36ac75+/fq0tDQav379emxsbG1t7U9HFRUVFRQU0NTevXsZY1euXKERTXR0dPhuvlpbWxljPp8vYA3egkDECSCrIo4UJwQBfwJRzKqurq6SkpKYmJg+ffrExMRUVFRohQ8dOsQY++qrr8jHI488MnfuXM55eXn5iBEjaJxznpGRUVVVxTmfMmXK4sWLaer06dOMsebmZhrRRFlZGfvlC1kVgAhvo0EAWRUNqjgnCBCBKGZVTU1NVlZWTU3NyZMn//73v6enp7/zzjs/pU5Uswo/V1FrIVQSQFappI1aLiQQxazKysp64403iOmLL76o/Q1TVH8HSOU45/K9+a+EBoFeEkBW9RIgDgcBOQH5/bxXf1+Vnp6u/fpOc1BRUZGbm0vPVqxZs0Yb9/l8Ac9W1NfXa1O7d+8OeLbi2rVr2lRpaSmerdBQ4H+tQABZZYUuwIODCUQxqxYsWDBkyBDtmfXa2tr+/fsXFxdrKCsrK9PS0rZv337y5MmCgoKAZ9bHjh175MiRgwcP5ubm0jPrXq83MzOzqKioqalp06ZNSUlJeGbdwdel7baGrLJdy2DYXgSimFXff//9smXLsrOzte8Cr1y5kp44174LnJmZ6fF4Jk+erD0rqIFra2srLCxMTk5OSUlZuHBh0O8CDxkypLKy8pag5Xu75eFYAALGCSCrjLPCShAIg4D8ft6r3wGG4Sayh8j3FtlaOJvLCSCrXH4BYPvRJiC/nyOros0f53cIAWSVQxqJbViVALLKqp2BL1sRQFbZql0waz8CyCr79QyOLUgAWWXBpsCSkwggq5zUTezFNALIKtPQo7A7CCCr3NFn7DLKBJBVUQaM07udALLK7VcA9h8RAsiqiGDESUCgJwLIqp7IYBwEQiCArAoBFpaCQOgEkFWhM8MRICAQQFYJSDAAApEkgKyKJE2cy7UEkFWubT02roYAskoNZ1RxOAFklcMbjO2ZTQBZZXYHUN8RBJBVjmgjNmFdAsgq6/YGzmxEAFllo2bBqh0JIKvs2DV4thwBZJXlWgJDziKArHJWP7Ebkwggq0wCj7JuIYCsckunsc+oEkBWRRUvTg4CyCpcAyAQAQLIqghAxClAoGcCyKqe2WAGBAwTQFYZRoWFIBAOAWRVONRwDAgEEEBWBQDBWxCILAFkVWR54mwuJYCscmnjsW1VBJBVqkijjqMJIKsc3V5sznwCyCrzewAHDiCArHJAE7EFKxNAVlm5O/BmGwLIKtu0CkbtSQBZZc++wbXFCCCrLNYQ2HEaAWSV0zqK/ZhCAFllCnYUdQ8BZJV7eo2dRpEAsiqKcHFqEOAcWYWrAAQiQABZFQGIOAUI9EwAWdUzG8yAgGECyCrDqLAQBMIhgKwKhxqOAYEAAsiqACB4CwKRJYCsiixPnM2lBJBVLm08tq2KALJKFWnUcTQBZJWj24vNmU8AWWV+D+DAAQSQVQ5oIrZgZQLRzaovv/xy/vz56enpCQkJeXl5x44d01h0d3evXr164MCBCQnoYh4QAAAf7UlEQVQJkydPPnfuHDFqa2ubN29e3759U1NTFy1a1N7eTlONjY2TJk3yeDxZWVkvv/wyjfck5Hvr6SiMg0AYBJBVYUDDISBgnID8fs6Mn0hceeXKlZycnD/+8Y9HjhxpaWnZvXv3p59+qi2rrKxMTU3dtm1bY2Pj7Nmz77zzzqtXr2pT06dPz8/PP3z48IEDB4YPH15YWKiN+3y+zMzM+fPnNzU11dTUJCYmVldXi0X9R+R7818JDQK9JICs6iVAHA4CcgLy+3mvsqqkpGTSpEli+e7u7oEDB7766qvalNfr9Xg8NTU1nPPm5mbGGP34tXPnzpiYmEuXLnHOq6qq+vXr19nZqR1VUlIycuRI8eT+I/K9+a+EBoFeEkBW9RIgDgcBOQH5/bxXWfWb3/xm+fLlf/jDHzIyMsaMGfPWW29pVi5cuMAYa2hoIGf333//008/zTlfv359WloajV+/fj02Nra2tpZzXlRUVFBQQFN79+5ljF25coVGNNHR0eG7+WptbWWM+Xy+gDV4CwIRJ4CsijhSnBAE/AlEMas8P79KS0uPHz9eXV2dkJDwzjvvcM4PHTrEGPvqq6/IxyOPPDJ37lzOeXl5+YgRI2icc56RkVFVVfXTUVOmTFm8eDFNnT59mjHW3NxMI5ooKytjv3whqwIQ4W00CCCrokEV5wQBIhDFrIqLi5s4cSJV+stf/jJhwoRoZxV+riLgECoJIKtU0kYtFxKIYlZlZ2f/6U9/IqZVVVWDBw/+6bd5Uf0dIJXjt/p3HfqvhAaBXhJAVvUSIA4HATmBKGZVYWGh/7MVy5cv137M0p6tWLNmjebM5/MFPFtRX1+vTe3evTvg2Ypr165pU6WlpXi2Qt5azKokgKxSSRu1XEggill19OjRPn36lJeXnz9/fuPGjUlJSe+++66GuLKyMi0tbfv27SdPniwoKAh4Zn3s2LFHjhw5ePBgbm4uPbPu9XozMzOLioqampo2bdqUlJSEZ9ZdeL1adsvWySrrOLFss2DMjgSimFWc83/84x95eXkej+fXv/41PQfIOde+C5yZmenxeCZPnvzJJ58Qu7a2tsLCwuTk5JSUlIULFwb9LvCQIUMqKyvpkJ6EfG89HYVxEAiDgHUSwjpOwsCIQ0CgJwLy+3mvnlnvqaSycfnelNlAITcQsE5CWMeJG/qOPSojIL+fI6uUNQKF7E3AOglhHSf27ijcW4wAsspiDYEdexKwTkJYx4k9OwnXFiWArLJoY2DLXgSskxDWcWKvDsKtxQkgqyzeINizBwHrJIR1nNijc3BpEwLIKps0CjatTcA6CWEdJ9buGNzZjACyymYNg11rErBOQljHiTU7BVc2JYCssmnjYNtaBKyTENZxYq0OwY3NCSCrbN5A2LcGAeskhHWcWKMzcOEQAsgqhzQS2zCXgHUSwjpOzO2If3Uw8adhU42ssmnjYNtaBKxzN7SOE+t0CEys04uwnSCrwkaHA80nYJ17EJyYfzX07MA63enZI2ZuQQBZdQtAmLYyAevcg+BEvE7ARGSCkbAJIKvCRocDzSeAu6HYAzCxMhPRG0YMEkBWGQSFZVYkgPuy2BUwsTIT0RtGDBJAVhkEhWVWJID7stgVMLEyE9EbRgwSQFYZBIVlViSA+7LYFTCxMhPRG0YMEkBWGQSFZVYkgPuy2BUwsTIT0RtGDBJAVhkEhWVWJID7stgVMLEyE9EbRgwSQFYZBIVlViSA+7LYFTCxMhPRG0YMEkBWGQSFZVYkgPuy2BUwsTIT0RtGDBJAVhkEhWVWJID7stgVMLEyE9EbRgwSQFYZBIVlViSA+7LYFTCxMhPRG0YMEkBWGQSFZVYkgPuy2BUwsTIT0RtGDBJAVhkEhWVWJID7stgVMLEyE9EbRgwSQFYZBIVlViSA+7LYFTCxMhPRG0YMEkBWGQSFZVYkgPuy2BUwsTIT0RtGDBJAVhkEhWVWJID7stgVMLEyE9EbRgwSQFYZBIVlViSA+7LYFTCxMhPRG0YMEkBWGQSFZVYkgPuy2BUwsTIT0RtGDBJAVhkEhWVWJID7stgVMLEyE9EbRgwSQFYZBIVlViSA+7LYFTCxMhPRG0YMElCUVS+99BJjbNmyZZqtq1evPvXUU+np6bfffvucOXO+/vprsnvx4sUZM2YkJiZmZGQ899xz169fp6l9+/aNHTs2Pj7+rrvu2rBhA433JOR76+kojNuIAO7LYrPAxMpMRG8YMUhAfj9nBs8iX3b06NE77rhj9OjRlFVLliwZOnTonj176uvrJ0yYcO+992pnuHHjRl5e3gMPPNDQ0FBXV9e/f//S0lJtqqWlJSkp6Zlnnmlubn799ddjY2N37dolryvfm/xYzNqCAO7LYpvAxMpMRG8YMUhAfj+PQFa1t7fn5uZ++OGHv/vd77Ss8nq9cXFxW7Zs0SyeOXOGMfbxxx9zzuvq6m677Tb6MevNN99MSUnp7OzknBcXF48aNYp29eijj06bNo3eBhXyvQU9BIP2IoD7stgvMLEyE9EbRgwSkN/PI5BVjz/++PLlyznnlFV79uz56feB3333HVnMzs5eu3Yt53z16tX5+fk03tLSwhg7fvw45/y+++6jH8s452+//XZKSgqtDCrkewt6CAbtRQD3ZbFfYGJlJqI3jBgkIL+f9zarampq8vLyrl696p9VGzdujI+P9/d39913FxcXc86feOKJqVOn0tQPP/zAGKurq+Oc5+bmVlRU0NSOHTsYYz/++CONaKKjo8N389Xa2soY8/l8AWvw1jEEcF8WWwkmVmYiesOIQQJRzKovvvhiwIABjY2NmhX6uSqqWVVWVsZ++UJWGbwU7LgM92Wxa2BiZSaiN4wYJBDFrNq6dStjLPbmizEWExMTGxv7P//zP9H7HSB+rjLYeGcsw31Z7COYWJmJ6A0jBglEMau+//77U36vcePGPfbYY6dOndKerXj//fc1i2fPng14tuKbb77Rpqqrq1NSUjo6OrRnK/Ly8mhXhYWFeLaCaLhW4L4sth5MrMxE9IYRgwSimFUBDuh3gJzzJUuWZGdn7927t76+fuLPL22x9sz61KlTT5w4sWvXroyMjIBn1lesWHHmzJl169bhmfUAvO58i/uy2HcwsTIT0RtGDBIwJ6u07wL369cvKSnp4Ycfvnz5Mtn9/PPPH3zwwcTExP79+z/77LMB3wUeM2ZMfHz8sGHD8F1gIuZmgfuy2H0wsTIT0RtGDBJQl1UGDUVwmXxvESyEU5lFAPdlkTyYgAldAyIK+47I7+e9fWbdXC7yvZnrDdUjQoA+k2qExLMaA1QFTggFCTAhFCQkTGw3Jb+fI6ts11B3GabPpBohgavGAFWBE0JBAkwIBQkJE9tNIats1zIY1gnQZ1KN0AsLSo0BqiLU1wdojRqhFxaUGgNURaivD9AaNUIvLCg1BqiKUN/GA8gqGzcP1ukzqUZIgKsxQFXghFCQABNCQULCxHZTyCrbtQyGdQL0mVQj9MKCUmOAqgj19QFao0bohQWlxgBVEerrA7RGjdALC0qNAaoi1LfxALLKxs2DdfpMqhES4GoMUBU4IRQkwIRQkJAwsd0Ussp2LYNhnQB9JtUIvbCg1BigKkJ9fYDWqBF6YUGpMUBVhPr6AK1RI/TCglJjgKoI9W08gKyycfNgnT6TaoQEuBoDVAVOCAUJMCEUJCRMbDeFrLJdy2BYJ0CfSTVCLywoNQaoilBfH6A1aoReWFBqDFAVob4+QGvUCL2woNQYoCpCfRsPIKts3DxYp8+kGiEBrsYAVYETQkECTAgFCQkT200hq2zXMhjWCdBnUo3QCwtKjQGqItTXB2iNGqEXFpQaA1RFqK8P0Bo1Qi8sKDUGqIpQ38YDyCobNw/W6TOpRkiAqzFAVeCEUJAAE0JBQsLEdlPIKtu1DIZ1AvSZVCP0woJSY4CqCPX1AVqjRuiFBaXGAFUR6usDtEaN0AsLSo0BqiLUt/EAssrGzYN1+kyqERLgagxQFTghFCTAhFCQkDCx3RSyynYtg2GdAH0m1Qi9sKDUGKAqQn19gNaoEXphQakxQFWE+voArVEj9MKCUmOAqgj1bTyArLJx82CdPpNqhAS4GgNUBU4IBQkwIRQkJExsN4Wssl3LYFgnQJ9JNUIvLCg1BqiKUF8foDVqhF5YUGoMUBWhvj5Aa9QIvbCg1BigKkJ9Gw8gq2zcPFinz6QaIQGuxgBVgRNCQQJMCAUJCRPbTSGrbNcyGNYJ0GdSjdALC0qNAaoi1NcHaI0aoRcWlBoDVEWorw/QGjVCLywoNQaoilDfxgPIKhs3D9bpM6lGSICrMUBV4IRQkAATQkFCwsR2U8gq27UMhnUC9JlUI/TCglJjgKoI9fUBWqNG6IUFpcYAVRHq6wO0Ro3QCwtKjQGqItS38QCyysbNg3X6TKoREuBqDFAVOCEUJMCEUJCQMLHdFLLKdi2DYZ0AfSbVCL2woNQYoCpCfX2A1qgRemFBqTFAVYT6+gCtUSP0woJSY4CqCPVtPICssnHzYJ0+k2qEBLgaA1QFTggFCTAhFCQkTGw3hayyXctgWCdAn0k1Qi8sKDUGqIpQXx+gNWqEXlhQagxQFaG+PkBr1Ai9sKDUGKAqQn0bDyCrbNw8WKfPpBohAa7GAFWBE0JBAkwIBQkJE9tNIats1zIY1gnQZ1KN0AsLSo0BqiLU1wdojRqhFxaUGgNURaivD9AaNUIvLCg1BqiKUN/GA8gqGzcP1ukzqUZIgKsxQFXghFCQABNCQULCxHZTyCrbtQyGdQL0mVQj9MKCUmOAqgj19QFao0bohQWlxgBVEerrA7RGjdALC0qNAaoi1LfxALLKxs2DdfpMqhES4GoMUBU4IRQkwIRQkJAwsd0Ussp2LYNhnQB9JtUIvbCg1BigKkJ9fYDWqBF6YUGpMUBVhPr6AK1RI/TCglJjgKoI9W08gKyycfNgnT6TaoQEuBoDVAVOCAUJMCEUJCRMbDcVxayqqKgYN25ccnJyRkZGQUHB2bNnic7Vq1efeuqp9PT022+/fc6cOV9//TVNXbx4ccaMGYmJiRkZGc8999z169dpat++fWPHjo2Pj7/rrrs2bNhA4z0J+d56OgrjNiJAn0k1QkJGjQGqAieEggSYEAoSEia2m5Lfz1lv9jNt2rQNGzY0NTWdOHFixowZ2dnZ//rXv7QTLlmyZOjQoXv27Kmvr58wYcK9996rjd+4cSMvL++BBx5oaGioq6vr379/aWmpNtXS0pKUlPTMM880Nze//vrrsbGxu3btktuT701+LGZtQYA+k2qEhIkaA1QFTggFCTAhFCQkTGw3Jb+f9yqr/Fl8++23jLH9+/dzzr1eb1xc3JYtW7QFZ86cYYx9/PHHnPO6urrbbruNfsx68803U1JSOjs7OefFxcWjRo2icz766KPTpk2jt0GFfG9BD8GgvQjQZ1KNkMBRY4CqwAmhIAEmhIKEhIntpuT384hl1fnz5xljp06d4pzv2bOHMfbdd98RrOzs7LVr13LOV69enZ+fT+MtLS2MsePHj/901H333bds2TKaevvtt1NSUugtiY6ODt/NV2trK2PM5/PRLITDCNBnUo2Q0FNjgKrACaEgASaEgoSEie2mVGRVV1fXzJkzf/vb32p0Nm7cGB8f70/q7rvvLi4u5pw/8cQTU6dOpakffviBMVZXV8c5z83NraiooKkdO3Ywxn788Uca0URZWRn75QtZFYDISW/pM6lGSNCpMUBV4IRQkAATQkFCwsR2UyqyasmSJTk5Oa2trRqdqGYVfq6y3SXYG8P0mVQjJFbVGKAqcEIoSIAJoSAhYWK7qahn1dKlS7OyslpaWghNVH8HSFU45/K9+a+EtikB+kyqERJKagxQFTghFCTAhFCQkDCx3ZT8ft6rv6/q7u5eunTp4MGDz507589Fe7bi/fff1wbPnj0b8GzFN998o01VV1enpKR0dHRoz1bk5eXReQoLC/FsBdFwraDPpBoh4azGAFWBE0JBAkwIBQkJE9tNRTGr/vznP6empn700UeXb77or5eWLFmSnZ29d+/e+vr6iT+/NHDaM+tTp049ceLErl27MjIyAp5ZX7FixZkzZ9atW4dn1m13qUXDMH0m1QjJFtQYoCpwQihIgAmhICFhYrupKGbVLx9x+P939AVe7bvA/fr1S0pKevjhhy9fvkzgPv/88wcffDAxMbF///7PPvtswHeBx4wZEx8fP2zYMDoVHSgK+d7E9RixHQH6TKoREj5qDFAVOCEUJMCEUJCQMLHdlPx+3qvfAZrOQr430+3BQO8J0GdSjZAYVmOAqsAJoSABJoSChC2YSEz6T8nv58gqf1bQliNAn0k1QrJ/NQaoCpwQChJgQihI2IKJxKT/FLLKnwa0zQjQZ1KNkNBRY4CqwAmhIAEmhIKELZhITPpPIav8aUDbjAB9JtUICR01BqgKnBAKEmBCKEjYgonEpP8UssqfBrQhAvRJUCMkntQYoCpwQihIgAmhIAEmhIKEhInBKWSVQVBYphOg60+N0AsLSo0BqiLU1wdojRqhFxaUGgNURaivD9AaNUIvLCg1BqiKUF8foDVqhF5YUGoMUBWhfsgDyKqQkeEAuv7UCAlwNQaoCpwQChJgQihIgAmhICFhYnAKWWUQFJbpBOj6UyP0woJSY4CqCPX1AVqjRuiFBaXGAFUR6usDtEaN0AsLSo0BqiLU1wdojRqhFxaUGgNURagf8gCyKmRkOICuPzVCAlyNAaoCJ4SCBJgQChJgQihISJgYnEJWGQSFZToBuv7UCL2woNQYoCpCfX2A1qgRemFBqTFAVYT6+gCtUSP0woJSY4CqCPX1AVqjRuiFBaXGAFUR6oc8gKwKGRkOoOtPjZAAV2OAqsAJoSABJoSCBJgQChISJgankFUGQWGZToCuPzVCLywoNQaoilBfH6A1aoReWFBqDFAVob4+QGvUCL2woNQYoCpCfX2A1qgRemFBqTFAVYT6IQ8gq0JGhgPo+lMjJMDVGKAqcEIoSIAJoSABJoSChISJwSlklUFQWKYToOtPjdALC0qNAaoi1NcHaI0aoRcWlBoDVEWorw/QGjVCLywoNQaoilBfH6A1aoReWFBqDFAVoX7IA8iqkJHhALr+1AgJcDUGqAqcEAoSYEIoSIAJoSAhYWJwClllEBSW6QTo+lMj9MKCUmOAqgj19QFao0bohQWlxgBVEerrA7RGjdALC0qNAaoi1NcHaI0aoRcWlBoDVEWoH/IAsipkZDiArj81QgJcjQGqAieEggSYEAoSYEIoSEiYGJxCVhkEhWU6Abr+1Ai9sKDUGKAqQn19gNaoEXphQakxQFWE+voArVEj9MKCUmOAqgj19QFao0bohQWlxgBVEeqHPICsChkZDqDrT42QAFdjgKrACaEgASaEggSYEAoSEiYGp5BVBkFhmU6Arj81Qi8sKDUGqIpQXx+gNWqEXlhQagxQFaG+PkBr1Ai9sKDUGKAqQn19gNaoEXphQakxQFWE+iEPIKtCRoYD6PpTIyTA1RigKnBCKEiACaEgASaEgoSEicEpZJVBUFimE6DrT43QCwtKjQGqItTXB2iNGqEXFpQaA1RFqK8P0Bo1Qi8sKDUGqIpQXx+gNWqEXlhQagxQFaF+yAPIqpCR4QC6/tQICXA1BqgKnBAKEmBCKEiACaEgIWFicApZZRAUlukE6PpTI/TCglJjgKoI9fUBWqNG6IUFpcYAVRHq6wO0Ro3QCwtKjQGqItTXB2iNGqEXFpQaA1RFqB/yALIqZGQ4gK4/NUICXI0BqgInhIIEmBAKEmBCKEhImBicQlYZBGX+Muq6GiHZsBoDVAVOCAUJMCEUJMCEUJCwBROJSf8pZJU/jSCauq5GBHFwc0iNAapys2yQP2mNGhHEwc0hNQaoys2yQf6kNWpEEAc3h9QYoCo3ywb5k9aoEUEc3BxSY4Cq3Cwb5E9ao0YEcXBzSI0BqnKzbPh/IqtuwY5YqxESN2oMUBU4IRQkwIRQkAATQkECTAgFCQkTg1PIqluAItZqhMSNGgNUBU4IBQkwIRQkwIRQkAATQkFCwsTgFLLqFqCItRohcaPGAFWBE0JBAkwIBQkwIRQkwIRQkJAwMTiFrLoFKGKtRkjcqDFAVeCEUJAAE0JBAkwIBQkwIRQkJEwMTiGrbgGKWKsREjdqDFAVOCEUJMCEUJAAE0JBAkwIBQkJE4NTNsuqN954Iycnx+PxjB8//siRI/JNyvcmP5ZmibUaQXVFocYAVREN0AitUSOorijUGKAqogEaoTVqBNUVhRoDVEU0QCO0Ro2guqJQY4CqiAZohNaoEVRXFGoMUBXRQKgj8vs5C/V0UV2/adOm+Pj4t99++/Tp00888URaWto333wjqSjfm+RA/ylirUb4lw7QagxQlYDq/m9pjRrhXzpAqzFAVQKq+7+lNWqEf+kArcYAVQmo7v+W1qgR/qUDtBoDVCWguv9bWqNG+JcO0GoMUJWA6mG8ld/PrZVV48ePX7p0qbbJrq6uwYMHv/TSS5I9y/cmOdB/ilirEf6lA7QaA1QloLr/W1qjRviXDtBqDFCVgOr+b2mNGuFfOkCrMUBVAqr7v6U1aoR/6QCtxgBVCaju/5bWqBH+pQO0GgNUJaB6GG/l93MLZVVnZ2dsbOzWrVtpk48//vjs2bPprSY6Ojp8N19ffPEFY6y1tfXmQDh/Dl2+WeU/EosqbQxdvhlOROBgAiYiAXEE10lITCS4/KdaW1sZY16vN+Cer721UFZdunSJMfa///u/ZHTFihXjx4+nt5ooKytjeIEACIAACDiRQGtra8A9X3trv6zy/7nqu+++u3Dhgtfr9Q9nt2nt/4z08odLh0EDE7GhYAImIgFxxMTrxOv1tra2dnV1WT2rDP4OMOg23Dwo/yWvO8mAidh3MAETkYA4YtnrxEI/V3HOx48f/1//9V8avq6uriFDhsifrRBBu3DEsteWib0AExE+mICJSEAcsex1Yq2s2rRpk8fjeeedd5qbmxcvXpyWlvb111+LNDHiT8Cy15a/ScUaTETgYAImIgFxxLLXibWyinP++uuvZ2dnx8fHjx8//vDhwyJKjAQQ6OjoKCsr++l/A8bd/BZMxO6DCZiIBMQRy14nlssqkR1GQAAEQAAEXE4AWeXyCwDbBwEQAAEbEEBW2aBJsAgCIAACLieArHL5BYDtgwAIgIANCCCrbNAkWAQBEAABlxNAVtn1AqioqBg3blxycnJGRkZBQcHZs2ftupPo+H7ppZcYY8uWLYvO6e101i+//HL+/Pnp6ekJCQl5eXnHjh2zk/soeL1x48aqVavuuOOOhISEYcOGvfDCC93d3VGoY4NT7t+/f9asWYMGDWKM+f+7WLu7u1evXj1w4MCEhITJkyefO3fO9M0gq0xvQZgGpk2btmHDhqamphMnTsyYMSM7O/tf//pXmOdy3GFHjx694447Ro8ejay6cuVKTk7OH//4xyNHjrS0tOzevfvTTz91XMND21B5efmvfvWrDz744LPPPtuyZUtycvLf/va30E7hlNV1dXUrV66sra0NyKrKysrU1NRt27Y1NjbOnj37zjvvvHr1qrmbRlaZyz8y1b/99lvG2P79+yNzOpufpb29PTc398MPP/zd736HrCopKZk0aZLNWxph+zNnzly0aBGddM6cOfPnz6e37hT+WdXd3T1w4MBXX31VQ+H1ej0eT01NjblkkFXm8o9M9fPnzzPGTp06FZnT2fwsjz/++PLlyznnyCrO+W9+85vly5f/4Q9/yMjIGDNmzFtvvWXz9kbAfnl5eU5OzieffMI5P3HixIABA959990InNfOp/DPqgsXLjDGGhoaaEP333//008/TW9NEcgqU7BHsmhXV9fMmTN/+9vfRvKktj1XTU1NXl6e9vsKZBXn3PPzq7S09Pjx49XV1QkJCe+8845t2xsZ411dXSUlJTExMX369ImJiamoqIjMee18Fv+sOnToEGPsq6++og098sgjc+fOpbemCGSVKdgjWXTJkiU5OTk9/UdfIlnJ8uf64osvBgwY0NjYqDlFVnHO4+LiJk6cSK37y1/+MmHCBHrrTlFTU5OVlVVTU3Py5Mm///3v6enpyG9klTs/C+p2vXTp0qysrJaWFnUlLVxp69atjLHYmy/GWExMTGxs7I0bNyzsOrrWsrOz//SnP1GNqqqqwYMH01t3iqysrDfeeIP2/uKLL44cOZLeulP4ZxV+B+jOayBau+7u7l66dOngwYOt8DhptDYZ4nm///77U36vcePGPfbYYy7/a7zCwkL/ZyuWL1/u/2NWiIAdsjw9Pb2qqoo2U1FRkZubS2/dKfyzSnu2Ys2aNRoKn8+HZyvceVVEZtd//vOfU1NTP/roo8s3Xz/++GNkTu2Us+B3gJzzo0eP9unTp7y8/Pz58xs3bkxKSsJzBAsWLBgyZIj2zHptbW3//v2Li4udctWHto/29vaGn1+MsbVr1zY0NFy8eJFzXllZmZaWtn379pMnTxYUFOCZ9dCwYrU/ASa8NmzY4L8AGlmlXQP/+Mc/8vLyPB7Pr3/9azwHyDn//vvvly1blp2drX0XeOXKlZ2dne78vOzbty/gRrJgwQLOufZd4MzMTI/HM3nyZO2ZSXMR4dkKc/mjOgiAAAiAwK0JIKtuzQgrQAAEQAAEzCWArDKXP6qDAAiAAAjcmgCy6taMsAIEQAAEQMBcAsgqc/mjOgiAAAiAwK0JIKtuzQgrQAAEQAAEzCWArDKXP6qDAAiAAAjcmgCy6taMsAIEQAAEQMBcAsgqc/mjOgiAAAiAwK0JIKtuzQgrQAAEQAAEzCWArDKXP6qDAAiAAAjcmgCy6taMsAIEQAAEQMBcAsgqc/mjOgiAAAiAwK0J/B8e+B6wo55rNQAAAABJRU5ErkJggg=="}}},{"cell_type":"code","source":"def seed_everything(seed):\n    random.seed(seed)\n    np.random.seed(seed)\n    os.environ[\"PYTHONHASHSEED\"] = str(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n    \n# CosineAnnealingAlpha\ndef nt(nmin,nmax,tcur,tmax):\n    return (nmax - .5*(nmax-nmin)*(1+np.cos(tcur*np.pi/tmax))).astype(np.float32)\n\nplt.plot(nt(.25,1,np.arange(EPOCHS),EPOCHS))\nplt.show()\n\n# callback to update alpha during training\ndef cb(self):\n    alpha = torch.as_tensor(nt(.25,1,learn.train_iter,EPOCHS*n_iter))\n    learn.dls.train_ds.alpha = alpha\nalpha_cb = Callback(before_batch=cb)\n\ndef augment_image(image,alpha):\n#   Randomly rotate the image.\n    angle = torch.as_tensor(random.uniform(-180, 180)*alpha)\n    image = torchvision.transforms.functional.rotate(image,angle.item())#,interpolation=torchvision.transforms.InterpolationMode.BILINEAR)\n\n    return image\n# To better center spine I consider L and R as the two superior vertices of an inverted equilateral triangle\n# and I find the 3rd vertex adding to one of them the 60 degree rotated vector that unites them\n# With those 3 vertices the Icenter is just (A + B + C)/3\nangle = -60\ntheta = (angle/180.) * np.pi\n\nrotMatrix = torch.as_tensor([\n    [np.cos(theta), -np.sin(theta)],\n    [np.sin(theta),  np.cos(theta)]\n]).float().to(device)","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:25.428649Z","iopub.status.busy":"2024-09-02T15:03:25.428302Z","iopub.status.idle":"2024-09-02T15:03:25.857104Z","shell.execute_reply":"2024-09-02T15:03:25.856293Z","shell.execute_reply.started":"2024-09-02T15:03:25.428612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spinal = [\n    'spinal_canal_stenosis_l1_l2',\n    'spinal_canal_stenosis_l2_l3',\n    'spinal_canal_stenosis_l3_l4',\n    'spinal_canal_stenosis_l4_l5',\n    'spinal_canal_stenosis_l5_s1'\n]\n\nlabels = {\n    'Normal/Mild':0,\n    'Moderate':1,\n    'Severe':2,\n    'UNK':-100\n}\n\navailable_flips = [[True],[False]]\nfor _ in range(4): available_flips = [v+[True] for v in available_flips]+[v+[False] for v in available_flips]","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:25.859741Z","iopub.status.busy":"2024-09-02T15:03:25.859366Z","iopub.status.idle":"2024-09-02T15:03:25.865532Z","shell.execute_reply":"2024-09-02T15:03:25.864670Z","shell.execute_reply.started":"2024-09-02T15:03:25.859691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv('train_split.csv')\ntrain = train[['study_id','fold'] + spinal]\ntrain.tail()","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:25.867421Z","iopub.status.busy":"2024-09-02T15:03:25.866992Z","iopub.status.idle":"2024-09-02T15:03:25.936123Z","shell.execute_reply":"2024-09-02T15:03:25.935212Z","shell.execute_reply.started":"2024-09-02T15:03:25.867380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = train.fillna('UNK')\ntrain[(train[spinal] == 'UNK').sum(1)>0].reset_index(drop=True).tail()","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:25.937889Z","iopub.status.busy":"2024-09-02T15:03:25.937244Z","iopub.status.idle":"2024-09-02T15:03:25.956866Z","shell.execute_reply":"2024-09-02T15:03:25.955935Z","shell.execute_reply.started":"2024-09-02T15:03:25.937827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open('axial_centers.pkl', 'rb') as f:\n    coord = pickle.load(f)\n\nwith open('level_assignments.pkl', 'rb') as f:\n    assignments = pickle.load(f)","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:25.958160Z","iopub.status.busy":"2024-09-02T15:03:25.957884Z","iopub.status.idle":"2024-09-02T15:03:28.624944Z","shell.execute_reply":"2024-09-02T15:03:28.624101Z","shell.execute_reply.started":"2024-09-02T15:03:25.958131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame({'key':assignments.keys()})\ndf['study_id'] = df['key'].apply(lambda v:int(v.split('_')[0]))\ndf['series_id'] = df['key'].apply(lambda v:int(v.split('_')[-1]))\ndf.tail()","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:28.626465Z","iopub.status.busy":"2024-09-02T15:03:28.626124Z","iopub.status.idle":"2024-09-02T15:03:28.645004Z","shell.execute_reply":"2024-09-02T15:03:28.644068Z","shell.execute_reply.started":"2024-09-02T15:03:28.626428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df.merge(train,left_on='study_id',right_on='study_id')\ndf.tail()","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:28.646558Z","iopub.status.busy":"2024-09-02T15:03:28.646261Z","iopub.status.idle":"2024-09-02T15:03:28.675106Z","shell.execute_reply":"2024-09-02T15:03:28.674265Z","shell.execute_reply.started":"2024-09-02T15:03:28.646526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df[df[spinal].isna().sum(1) < 5]","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:28.678356Z","iopub.status.busy":"2024-09-02T15:03:28.678071Z","iopub.status.idle":"2024-09-02T15:03:28.686727Z","shell.execute_reply":"2024-09-02T15:03:28.685645Z","shell.execute_reply.started":"2024-09-02T15:03:28.678325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['flip'] = False\nfdf = df.copy()\nfdf['flip'] = True\ndf = pd.concat([df,fdf]).reset_index(drop=True)\ndf.tail()","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:03:28.688536Z","iopub.status.busy":"2024-09-02T15:03:28.688052Z","iopub.status.idle":"2024-09-02T15:03:28.708568Z","shell.execute_reply":"2024-09-02T15:03:28.707751Z","shell.execute_reply.started":"2024-09-02T15:03:28.688468Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class subarticular_Dataset(Dataset):\n    def __init__(self, df, f, VALID=False, P=patch_size, alpha=0):\n        self.data = df\n        self.VALID = VALID\n        self.f = f\n        self.P = P\n        self.alpha = alpha\n        self.resize = torchvision.transforms.Resize((PATCH_SIZE,PATCH_SIZE),antialias=True)\n        self.indices = torch.arange(Lmax).float()\n\n    def __len__(self):\n        return len(self.data)\n\n    def __getitem__(self, index):\n\n        row = self.data.iloc[index]\n        \n        sample = 'C:/Users/Angel/kaggle/train/'\n        sample = sample+str(int(row['study_id']))+'/'+str(int(row['series_id']))\n\n        images = [x.replace('\\\\','/') for x in glob.glob(sample+'/*.dcm')]\n        images.sort(key=lambda k:int(k.split('/')[-1].replace('.dcm','')))\n        instance_numbers = [int(k.split('/')[-1].replace('.dcm','')) for k in images]\n        images = [torch.as_tensor(pydicom.dcmread(img).pixel_array.astype('float32')) for img in images]\n        shapes = [img.shape for img in images]\n        H,W = np.array(shapes).max(0)\n\n        images = torch.concat([torch.nn.functional.pad(\n            images[k].unsqueeze(0),(\n                (W - shapes[k][-1])//2,\n                (W - shapes[k][-1]) - (W - shapes[k][-1])//2,\n                (H - shapes[k][-2])//2,\n                (H - shapes[k][-2]) - (H - shapes[k][-2])//2\n            ),\n        mode='reflect') for k in range(len(images))]).float()\n\n        if H > W:\n            d = W\n            h = (H - d)//2\n            images = images[:,h:h+d]\n            H = W\n        elif H < W:\n            d = H\n            w = (W - d)//2\n            images = images[:,:,w:w+d]\n            W = H\n\n        images = self.resize(images/images.max()).float().to(device)\n\n        c = coord[self.f][row['study_id']][row['series_id']].clone()\n\n        c[c < 64] = torch.nan\n        c[c > 512 - 64] = torch.nan\n        c = torch.cat([\n            c,\n            (c[:,1] + torch.matmul(c[:,0] - c[:,1],rotMatrix)).unsqueeze(1)\n        ],1)\n        c_mean = torch.nanmean(c, dim=0)\n        instance_to_k = {instance_numbers[k]:k for k in range(len(c))}\n        \n        img = torch.zeros(5,Lmax,128,128)\n        slices_mask = torch.ones(5,Lmax).bool()\n        for k in [1,2,3,4,5]:\n            instance_numbers = assignments[row['key']][k]['instance_numbers']\n            if len(instance_numbers) == 0: continue\n            distances = assignments[row['key']][k]['dis']\n            dis_sign = np.sign(distances)\n            if dis_sign[0] != dis_sign[-1]:\n                c_k = torch.stack([c[instance_to_k[i]] for i in instance_numbers])\n                c_k_mean = torch.nanmean(c_k, dim=0)\n                mask =  torch.isnan(c_k_mean)\n                c_k_mean[mask] = c_mean[mask]\n                c_k_mean = c_k_mean.unsqueeze(0).tile(len(c_k),1,1)\n                mask = torch.isnan(c_k)\n                c_k[mask] = c_k_mean[mask]\n                    \n                c_spine = c_k.mean(1)\n                '''print(c_mean)\n                print(c_k_mean)\n                print(c_k)\n                print(c_spine)'''\n        \n                images_k = torch.stack([\n                        images[\n                            instance_to_k[instance_numbers[i]],\n                            c_spine[i,1].long()-self.P:c_spine[i,1].long()+self.P,\n                            c_spine[i,0].long()-self.P:c_spine[i,0].long()+self.P\n                        ] for i in range(len(distances))#instance_numbers\n                ])\n\n                if len(distances) > Lmax:\n#                   abs_dist = [abs(v) for v in distances]\n#                   abs_dist.sort()\n#                   dis_th = abs_dist[Lmax]\n#                   img[k-1] = images_k[:,abs(distances) < dis_th]\n#                   slices_mask[k-1] = False\n                    indices = [i for i in range(len(distances))]\n                    indices.sort(key=lambda i:abs(distances[i]))\n                    indices = indices[:Lmax]\n                    indices.sort()\n                    img[k-1] = images_k[indices]\n                    slices_mask[k-1] = False\n\n                else:\n                    d = (Lmax - len(distances))//2\n                    img[k-1,d:d+len(distances)] = images_k\n                    slices_mask[k-1,d:d+len(distances)] = False\n\n        images = img.view(-1,2*self.P,2*self.P)\n        if not self.VALID:\n            images = augment_image(images,self.alpha)[...,self.P//2:self.P//2+self.P,self.P//2:self.P//2+self.P]\n        else:\n            images = images[...,self.P//2:self.P//2+self.P,self.P//2:self.P//2+self.P]\n\n        images = images.view(5,Lmax,self.P,self.P)\n        \n        label = torch.as_tensor([labels[x] for x in row[spinal]])\n        label[slices_mask.sum(-1) == Lmax] = -100\n        \n        if row['flip']:\n#            Flip L to R\n            images = images.flip(-1)\n\n        return [images.to(device),slices_mask.to(device)],label.to(device)","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:04:05.080102Z","iopub.status.busy":"2024-09-02T15:04:05.079679Z","iopub.status.idle":"2024-09-02T15:04:05.111220Z","shell.execute_reply":"2024-09-02T15:04:05.110150Z","shell.execute_reply.started":"2024-09-02T15:04:05.080065Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds = subarticular_Dataset(df,1)","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:04:08.904113Z","iopub.status.busy":"2024-09-02T15:04:08.903034Z","iopub.status.idle":"2024-09-02T15:04:08.909618Z","shell.execute_reply":"2024-09-02T15:04:08.908273Z","shell.execute_reply.started":"2024-09-02T15:04:08.904060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample = ds.__getitem__(np.random.randint(len(ds)))\nprint(sample[1])\nfor i in range(5):\n    fig, axes = plt.subplots(1, 5, figsize=(10,10))\n    for k in range(5):\n        axes[k].imshow(sample[0][0][k][i].cpu())\n    plt.show()\nplt.imshow(sample[0][1].cpu().view(-1,Lmax))\nplt.show()","metadata":{"execution":{"iopub.execute_input":"2024-09-02T15:08:34.097088Z","iopub.status.busy":"2024-09-02T15:08:34.096648Z","iopub.status.idle":"2024-09-02T15:08:38.304145Z","shell.execute_reply":"2024-09-02T15:08:38.303242Z","shell.execute_reply.started":"2024-09-02T15:08:34.097048Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class myUNet(nn.Module):\n    def __init__(self):\n        super(myUNet, self).__init__()\n\n        self.UNet = smp.Unet(\n            encoder_name=\"resnet18\",\n            classes=2,\n            in_channels=1\n        ).to(device)\n\n    def forward(self,X):\n        x = self.UNet(X)\n#       MinMaxScaling along the class plane to generate a heatmap\n        min_values = x.view(-1,2,PATCH_SIZE*PATCH_SIZE).min(-1)[0].view(-1,2,1,1) # Bug, I've been MinMaxScaling with the wrong values\n        max_values = x.view(-1,2,PATCH_SIZE*PATCH_SIZE).max(-1)[0].view(-1,2,1,1)\n        x = (x - min_values)/(max_values - min_values)\n        \n        return x","metadata":{"execution":{"iopub.status.busy":"2024-09-02T15:03:34.973488Z","iopub.status.idle":"2024-09-02T15:03:34.973994Z","shell.execute_reply":"2024-09-02T15:03:34.973749Z","shell.execute_reply.started":"2024-09-02T15:03:34.973724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class SinusoidalPosEmb(nn.Module):\n    def __init__(self, dim=16, M=10000):\n        super().__init__()\n        self.dim = dim\n        self.M = M\n\n    def forward(self, x):\n        device = x.device\n        half_dim = self.dim // 2\n        emb = math.log(self.M) / half_dim\n        emb = torch.exp(torch.arange(half_dim, device=device) * (-emb))\n        emb = x[...,None] * emb[None,...]\n        emb = torch.cat((emb.sin(), emb.cos()), dim=-1)\n        return emb\n\nclass myViT(nn.Module):\n    def __init__(self, ENCODER, dim=512, depth=24, head_size=64, **kwargs):\n        super().__init__()\n        self.ENCODER = ENCODER\n        self.AvgPool = nn.AdaptiveAvgPool2d(output_size=1).to(device)\n        self.slices_pos_enc = nn.Parameter(SinusoidalPosEmb(dim)(torch.arange(Lmax, device=device).unsqueeze(0)))\n        self.side_pos_enc = nn.Parameter(SinusoidalPosEmb(dim)(torch.arange(2, device=device).unsqueeze(0)))\n        self.level_pos_enc = nn.Parameter(SinusoidalPosEmb(dim)(torch.arange(5, device=device).unsqueeze(0)))\n        self.slices_transformer = nn.TransformerEncoder(\n                nn.TransformerEncoderLayer(d_model=dim, nhead=dim//head_size, dim_feedforward=4*dim,\n                dropout=0.1, activation=nn.GELU(), batch_first=True, norm_first=True, device=device), 24)\n        self.side_transformer = nn.TransformerEncoder(\n                nn.TransformerEncoderLayer(d_model=dim, nhead=dim//head_size, dim_feedforward=4*dim,\n                dropout=0.1, activation=nn.GELU(), batch_first=True, norm_first=True, device=device), 12)\n        self.level_transformer = nn.TransformerEncoder(\n                nn.TransformerEncoderLayer(d_model=dim, nhead=dim//head_size, dim_feedforward=4*dim,\n                dropout=0.1, activation=nn.GELU(), batch_first=True, norm_first=True, device=device), 12)\n        self.proj_out = nn.Linear(dim,3).to(device)\n    \n    def forward(self, x):\n        x,slices_mask = x\n        \n        '''for kk in range(BS):\n            fig, axesL = plt.subplots(1, 5, figsize=(10,10))\n            fig, axesR = plt.subplots(1, 5, figsize=(10,10))\n            for k in range(5):\n                axesL[k].imshow(x[kk,k,0].sum(0).cpu())\n                axesR[k].imshow(x[kk,k,1].sum(0).cpu())\n            plt.show()'''\n        \n        x = self.ENCODER(x.view(-1,1,patch_size,patch_size))[-1]\n        x = self.AvgPool(x)\n        slices_mask = slices_mask.view(-1,Lmax)\n        mask = slices_mask.sum(-1) < Lmax\n        x = x.view(-1,Lmax,512) + self.slices_pos_enc\n        x[mask] = self.slices_transformer(x[mask],src_key_padding_mask=slices_mask[mask])\n        x[slices_mask] = 0\n        d = (~slices_mask).sum(1).unsqueeze(-1).tile(1,512)\n        x = x.sum(1)\n        x[d > 0] = x[d > 0]/d[d > 0]\n\n        side_mask = slices_mask.view(-1,2,Lmax).sum(-1) == Lmax\n        mask = side_mask.sum(-1) < 2\n        x = x.view(-1,2,512) + self.side_pos_enc\n        x[mask] = self.side_transformer(x[mask],src_key_padding_mask=side_mask[mask])\n\n        level_mask = side_mask.view(-1,5,2).permute(0,2,1).reshape(-1,5)\n        mask = level_mask.sum(-1) < 5\n        x = x.view(-1,5,2,512).permute(0,2,1,3).reshape(-1,5,512)\n        x = x + self.level_pos_enc\n        x[mask] = self.level_transformer(x[mask],src_key_padding_mask=level_mask[mask])\n\n        x = self.proj_out(x.view(-1,512)).view(-1,2,5,3)\n\n        return x\n    \nclass AxialSpinalViT(nn.Module):\n    def __init__(\n        self,\n        ENCODER,\n        slices_pos_enc,\n        level_pos_enc,\n        slices_transformer,\n        level_transformer,\n        proj_out,\n        dim=512, depth=24, head_size=64, **kwargs\n    ):\n        super().__init__()\n        self.ENCODER = ENCODER\n        self.AvgPool = nn.AdaptiveAvgPool2d(output_size=1).to(device)\n        self.slices_pos_enc = slices_pos_enc\n        self.level_pos_enc = level_pos_enc\n        self.slices_transformer = slices_transformer\n        self.level_transformer = level_transformer\n        self.proj_out = proj_out\n    \n    def forward(self, x):\n        x,slices_mask = x\n        \n        '''for kk in range(BS):\n            fig, axes = plt.subplots(1, 5, figsize=(10,10))\n            for k in range(5):\n                axes[k].imshow(x[kk,k].sum(0).cpu())\n            plt.show()'''\n        \n        x = self.ENCODER(x.view(-1,1,patch_size,patch_size))[-1]\n        x = self.AvgPool(x)\n        slices_mask = slices_mask.view(-1,Lmax)\n        mask = slices_mask.sum(-1) < Lmax\n        x = x.view(-1,Lmax,512) + self.slices_pos_enc\n        x[mask] = self.slices_transformer(x[mask],src_key_padding_mask=slices_mask[mask])\n        x[slices_mask] = 0\n        d = (~slices_mask).sum(1).unsqueeze(-1).tile(1,512)\n        x = x.sum(1)\n        x[d > 0] = x[d > 0]/d[d > 0]\n\n        level_mask = slices_mask.view(-1,5,Lmax).sum(-1) == Lmax\n        mask = level_mask.sum(-1) < 5\n        x = x.view(-1,5,512) + self.level_pos_enc\n        x[mask] = self.level_transformer(x[mask],src_key_padding_mask=level_mask[mask])\n\n        x = self.proj_out(x.view(-1,512)).view(-1,5,3)\n\n        return x","metadata":{"execution":{"iopub.status.busy":"2024-09-02T15:03:34.976083Z","iopub.status.idle":"2024-09-02T15:03:34.976735Z","shell.execute_reply":"2024-09-02T15:03:34.976492Z","shell.execute_reply.started":"2024-09-02T15:03:34.976466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def myLoss(preds,target):\n    target = target.view(-1)\n    preds = preds.view(-1,3)\n    Loss = nn.CrossEntropyLoss(weight=torch.as_tensor([1.,2.,4.]).to(device))(preds,target.long())\n    return Loss","metadata":{"execution":{"iopub.status.busy":"2024-09-02T15:03:34.977992Z","iopub.status.idle":"2024-09-02T15:03:34.978453Z","shell.execute_reply":"2024-09-02T15:03:34.978235Z","shell.execute_reply.started":"2024-09-02T15:03:34.978211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for f in FOLDS:\n    seed_everything(SEED)\n    seg_model = myUNet()\n    sub_model = torch.load('subarticular_DICOM_V2_ViT_'+str(f))#sub_model = myViT(seg_model.UNet.encoder)#\n    model = AxialSpinalViT(\n        sub_model.ENCODER,\n        sub_model.slices_pos_enc,\n        sub_model.level_pos_enc,\n        sub_model.slices_transformer,\n        sub_model.level_transformer,\n        sub_model.proj_out\n    )\n    \n    tdf = df[df.fold != f]\n    vdf = df[df.fold == f]\n    tds = subarticular_Dataset(tdf,f)\n    vds = subarticular_Dataset(vdf,f,VALID=True)\n    tdl = torch.utils.data.DataLoader(tds, batch_size=BS, shuffle=True, drop_last=True)\n    vdl = torch.utils.data.DataLoader(vds, batch_size=BS, shuffle=False)\n    dls = DataLoaders(tdl,vdl)\n\n    n_iter = len(tds)//BS\n\n    learn = Learner(\n        dls,\n        model,\n        loss_func=myLoss,\n        cbs=[\n            ShowGraphCallback(),\n            GradientClip(3.0),\n            SaveModelCallback (fname='spinal_DICOM_ViT_'+str(f)),\n            EarlyStoppingCallback (patience=0),\n            alpha_cb\n        ]\n    )\n    learn.fit_one_cycle(EPOCHS, lr_max=5e-4\n                        , wd=0.05, pct_start=0.02)\n#   torch.save(model,'spinal_DICOM_ViT_'+str(f))\n    del model,seg_model,sub_model,tdf,vdf,tds,vds,tdl,vdl,dls,learn\n    gc.collect()","metadata":{"execution":{"iopub.status.busy":"2024-09-02T15:03:34.980484Z","iopub.status.idle":"2024-09-02T15:03:34.980967Z","shell.execute_reply":"2024-09-02T15:03:34.980727Z","shell.execute_reply.started":"2024-09-02T15:03:34.980706Z"},"trusted":true},"execution_count":null,"outputs":[]}]}