{"cells":[{"metadata":{},"cell_type":"markdown","source":"## My best submission Private LB 0.902/Public LB 0.901 - would have been in the money/gold medal zone\n![image.png](attachment:image.png)\n\n### General approach\n* Final models were trained on the FULL data.\n* Still used CV for validation/evaluation purposes only.\n\n### Models\neffnet b3/b4, vit, resnext50 architectures\nLabel Smoothing\nVarious loss functions: CE, Symmetric CE, Taylor CE\nVarious schedulers: Gradual Warmup/CosineAnnealingWarmRestarts\nAugmentation: played mostly around crops/cutmix apart from the standard techniques.\nData:\nonly 2020\npretrain on 2019, train on 2020\ntrain on both (one pass)\n\n### Ensemble:\nHierarchical blend:\n* Blended different epoch/best of kind models first\n* Then blended the already blended “super” models\n* Used CV to optimise the weights\n\nYou can find more details on my approach here: https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220752\n\nInference code taken with modifications from this wonderful notebook (please upvote it): https://www.kaggle.com/khyeh0719/pytorch-efficientnet-baseline-inference-tta","attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAA24AAABzCAYAAADg8VG/AAAgAElEQVR4Ae2dz08bybr+82dkO8uznGW2WY56leVIvUJZRJHYgO7CiTRolINQIEexMjMoSkBnfJLJMffakzuBG65DkCFjuIwDTHMIRiQmA4hgvsyYBHWG+AT0fFXVv6rbbWywCZA8kSz3j6q33vpUNanHb1X1KfAfCZAACZAACZAACZAACZAACZDAsSZw6lh7R+dIgARIgARIgARIgARIgASOnEBx8w0yE5N4PDJ2oI/IK2zw38EJULgdnB1zkgAJkAAJkAAJkAAJkMAnQaAe0eaIPWGD/w5OgMLt4OyYkwRIgARIgARIgARIgAQ+CQKO+DpoZevNf9ByP6Z8FG4fU2uyLiRAAiRAAiRAAiRAAiRwCATqFV715j+EKp04kxRuJ67J6DAJkAAJkAAJkAAJkAAJfFgC9QqvevN/2Noez9Io3I5nu9ArEiABEiABEiABEiABEjg2BOoVXvXmPzYgjtARCrcjhM+iSYAESIAESIAESIAESOAkEKhXeNWb/yQwOmwfKdwOmzDtkwAJkAAJkAAJkAAJkMAJJ1Cv8Ko3/wnH1xD3GyDc3mP79RY23+1Wdqj0JzZfm9jeI0nlzMf4zm4Jm6+3sF06LB//xOL4DLoSk+gaXbUL2cXm7Cy+EdcGXmBNXH1nYvP1nzU4sYvtrXVMTa5gsab0NZh0kpjrGBiclL4mXry3rwb9F+VvYdN07juZ9/q2+9dWnZB3X2NscBoDL+u0s5erh3FPPjtbsp+JvuZ8yvvce2yvr2FsZg1rtfKt5bk0t7BZL/tqXGrxQ9qw+4/CQfI4bP+q+c/7exIoFZeRM/IomCfs2atUq50STNNEXbVphI1K/n1q17eLWJ7PYbm4jxYpmSjkDeTX92rHEsz1PIx8AXt1Xdm/55dR3CvRp9Ymx6W+R9w3JIZtE+ZH1DfqFV715j8uXeso/WiAcHuJK1eHcG6oWLkeRhanro7irlQZlZM16s726hIeT65hs1EGK9lZm8W5q0O4YlRKUN/1zScZnBZs70yia3zdMjY/iTNXh3C2dwKXUkvYBrA2NIpTV7MY26s4cxW3bw/h1FXv89ntWSy+2ytTrfe2MNA7hFNdo2hJTGJgycpX7n8Rd28O4VTfy1oNw7Fx6uasJVJrzhlIuDGHL68O4cwDRwBvYW7yBaZW9/GffcDkBzmVz47XZk77+fpcSNt+8d8rsm/s6WMNz+VY3xDqZr+nEwBq8MMyYfcfpQ9LHvX2jWr+8f7BCKxn0RPRoWma+2nq6Ede/NE6yf/m49C0dqQ36qhEI2zUUfzHkbWE/E/taFL71/UUlqv0r8JEN1qVPNr5KDL2f68ul2Df1SOITZvubXmwvYzU9Sa3b2uajsgdA4FU/jw8+0AEjrhvyFqaWB5x+locuQ9U88Mupl7hVW/+w67fSbD/UQq3moRMI1rnkIWbHDR3z2JR8dWq2wTGlOhl9fruYurHYZy6Po6BdTvatTSLL7uG8MVg8H8spbCaD23xnvrdl6Pc/30KN3MBLV1DOBMdbox4MNWobw0/OPhqczQnQrieujqBgUCUyYu4lTCVGJai+e6SaNv3WBsbx+dXh3ApW0WU1iCYjpdwW0HX9SGce7jqRh4ZcTuaflm91DySLRr0ayks2yPZ0kYWMXHtZvZkD24bIboaYaN6I3zUKUpGDLrWjJ4J+0fj9TQ6dQ3N93KVo6H5JFo1HZ2Dy1Yf3Cki+30rtJYk8i6tAlJfadA6+u2+ayJ3r7VMrOcTzdCae5C1izd/60e7piE6Runmojyig6PuG8Ay+iM69EgP4r0RaBqFm9MVKNwcEgf/Phzh9noVicQ4znWPyyl+2zUMEN0qvF7CbRG1WTQxNzqJi92juJiYw9xrW6n88VLevzuvDkq3MCam6aVXkRubxKVbwzh1NY2LiUncntlyTUP49WACTTeEzVmMOSJGpHDKfVHE4wfjOHdjGlMy53usTc7g0t9Hcfb2BG6Pr2PTEU2OcJuu4KtXcvhRqYjHgxNWHftmMLDiTHe06nPx2yGcuj6KS3Y9FgN16xqzRJcr3F4v4e6djPTzm9FVz0+UsDb7Ao8XFRZ4j8c/DOHUDy+qR2YkH4ddBi0PZjG1YUOQ3MbxxdUhfH5rAl2JGYy9Dvcf8ISbmO7ZcjuNpjvTSCyE/Uf3J8b+OYzTt+YwJiKK1aIqVfrFNtYx4PSHMp/nfeI4tLEq9o9dbC7M4RuH+9CSwl1Yeo+1mVnrvnge1PuOzUUxjdPuB6KvKzgWU6M4FRDvPv/eLaDlalCAbyEhoqu983tHnZ3nckmwsZ7XS4MvsKY8WuXCzXsezv3deh68H7gdxr9jbnTabt8ZjDl9xXH8oH8fdl/ikhCk084D6Bjk97EjsJGWg9j4vN8zcz6FZCKLgrhs5pBKJJF9paR5lUUykUJOeQbM3zLo740i0tGN5EgOxR0lvTBT6/3rPegfyZeLRnMZmZ96EL0cRc9PaeTf+O1D3E90o12UP7aMUpjoaoSNQLE83YuAiex3GrTrGV97FkfaoWk9MJS/YaqV/H0htlSRBkD2VR0xJ1M+iWatGUlPyQHIIa5riAzKngvxf1n2VgSRn9RERaQ7NGj3PpbYikruJB0fdd8QrPJI9Rnyb5XVJyncnB5E4eaQOPh344Wb+QKXuoZw+kZGCo5LvWk03RJT+WqcKmmLoXM3R+V0wK6EEFpDOBXNwvohax23vx3CaVVwvJ7HxatDaBrdQlDcuMJt1YowfXZTiItJXLotohRpfDNvDwLtcj+/nsYXf59AV98c5vAnxvqsdE19k+jqy+CsiAD1vbTEjp3ni29HremMiXF80SV8ncRUtbGl+RJXohanrqFZ3L4zis+uDuOKIcRbuPAJ1s0v3IZx5ltrqmKXtKX4GdY/NuZxsdaIm8Lu9tAsunrTON01iruru7bg3adwiw7jC7sdWsTUSbfenqNC7J8RfWZ115oKWk24Ye9+ASgRtoMIt9D+sYvFoVF7OusM7g5Nyr56+tYsFp1ulR7F6a40mh7M4/HoNJpEmzuCyrZ5Jpq2+o/dv1SRKoXT7RmZV651fKD8iCFwzU/i86vD6AoMkKXgqzZ9Vgq3YZy5kYbav1X//cJNeR4Ss7g7OIFz6vNgM/48msYXveI5s+6fup7FmDMlt56/D5LXMK6MvZA/3ggetyf3mKLtdScefWgCJQMx3Yq4lQkhx5cwcRcQRYWRTuhaE9p7k0gmetB+XthMW8IPQO33U0iPJBEVv4Ir+WFHaZo6epAaSSN5IwJd70TamYiwbSAmyoxEEUskEbvRhug1IQ6UqZKNsOEw4XeNBHKIaxrahx0hZWd7lUJE0xFfCDeT79OhXU65/UemMrPoFrZGrL8l1kA7FpjaVoLRq0G7lkHFvzilHOLNnp1wD3j18Akcr75B4eZvcQo3P4+DnDVcuC0+TONU1zgeu7+YWpGT/Qq3zx1xJGplT5lzpvXJCFNX1p0uaE0ny2DgtYXAjUC5REpWdOnbGXdALcSRjErcnrOiEo4Ie6gsxFuakZGklgknEgZsZ7M4e2MCA3+IxWXWGrcvlDzb2QmcvprG7SrLuBYH0zh1XRV49nRGRaD4B82V6uascRtG15ynFjdHxRS7crG89mQcZ6PDOH11GBeHVgLRIReYcmCzczjJOxa70+5aNUUUKTnL/bcjbr562+2g1BvvlqSodbjK9lTvK2Woh1a/mMBjWyT4+0XQx+C5ainkOKSt8c76kUL8YOD++2MOTXLdo2iL11Yf++GFexsbq3g8b//XH2Jze3xcaTd77aCIqIkfExIT+PJv1lrChNNNnaiZc26XVP4MeC64RzKv9YOHe21OCEGxbtPqS742lM+Dv59Bpnf6mcX09J0FL4r7YhpnlXWgdf19kCJ1CKf/Zv1AcenvaXwm1i0mrLWebh14cCwIlFZSiJ631rc1tbSjO5FGTh31VhNutvhrc6McAF6lEb3chn4R6Kjxvm/q2psMomJgL3/oKMH4Xg9EbUxkrmvQ7ahJYTACTe+B4YaVS8jdaVaEWyNsHIvmOmFO2INzW2y5zof1Kfem6DJiemUr4s+cAUrJngbpRcoqDbRz98T0yXQF4WbbaYkjXyHap7jBw0MlcLz6RqX+dKgIDtF4vcKr3vyHWLUTY7rBwq2IxK0hnFIHbgLF9IQyGK3Cxh7M+jZfQAmP7yhTv3yDY2twq0bgygetS7jSNYSzisASXlgD+3FroB9S7lpaRAr968l83ofkccSc339fLjnNQnLqncbjyRfe54EQW94mI75Bs22ivG6OcPPyyaT2ZhzBaWXWxi0LuNsnInxDaEr716UFPQVsdn1znp+TL3D3n+q6s3ARVO6/N1VSLcfPeRdT/z3si1rKOtcg3GD3C2ttV7BfBH0MnqsehRyHtfVsVor0S6NKG07O4VLU26xH1M3aYGYad8dXsKju+Bhms0yI7WL7nbILp/kSl64PwRXNZekt361+skffFclC81rtfc5er6i2odVOGXyj9tnstC1UhcEQpr46NuDvQ6mk7E67Cyuy6AjHkHbjpSMmUIK5YiD9Uw86L4qNSpoQfWJHScIG2WrEbSEOXUS2nOhXsCY13Y8g9sSAYTifNGL/4URF8tb0tzsZ5b6BtFjzJAfoRWSuadB6Df+aqWcxRbg1wkawYjyvTuBgg3PAhCHaV9NxIRJB5KKO1t4Ukh1e9K7SQLuycCsh1xeI1FavAFMcGoHj1DeASv3p0Kp/yIbrFV715j/k6p0I8w0XbqG7BoYOECvw8Q30nDS7GPunurudEvGx1/ioGzGUi5uQAaUwbQvKhNgdLKTccjuOP/Z3SJ4wO4FcUrhJTt+OW1v9i6393Y+33kodNDs2wnyS10QUy0kkvm0uF58o0SD1PnYx90BER70Ile+2e2Kxs9avqX5OomtwyV5DFc633H9LuH3+3yuudXGwPSGiTHbE1I5yXsq+djegmHso1nnNYK7qKyUUsWZPn/X6RdDH4LnPpfKTsLaW/XoYX/4zwMVZSyetWGvcusQaSREtEzuCJvxTbX0iv4Znxce10lRJGfnO+vtEsFayLC9Sbd1ew+0bQ3DaSC3L6nvW2lGvv1p1t3YSDWHq4xYu3MMFZNDZCuc++xXS8PIxIWBHq/SYtQapmnBTRVxYDWq634zO78U0S/8nNS8iLtYAr/larOx+8lEOJiqsWfKV2wgbYZXjtb0J2IJZjcaKDKHr08otmSs5KdZzK1Y/iGkauiesKFzxSWfIOjl73dR3wY11Slge7ISutyHJUFs56CO5clz6hlV5Cjd/J6Bw8/M4yFmDhdsu5EDvxoxvswc5LTBk2l6ow/ZAzP96AWsweTph7zPvRMu6sngsppYp0yaFzXJxYw0Y1agc4Oy0aAuesAGgYUVUbnvFAsU1PJ5cwpqYjheWJ+xaWUVtTr7ph2LqjxJZASyWgUhTed2c+maQENM3nX9yCpuz9sneNMLwizgrWhEcuDsGnG9bbP3oF1t+X0MG7KH+2wN3X70D7SDFRPn299Y2+NUjK1IEdmUxIKaK+vpF0MfguVPfCt9h7SqvBaYO4j38P8+/x7b7jsNAhCjMpk+4hbWbPbXUWSe3+9KKJruvORD+W+v9yiLfwarJsgL+21FLR/Crwk0IrNNXA/1s9714lOx/IUx9dazv78PmjHin4TyUGcGwpmo6/dzxg99HTaA0HUPkcjcygS3zzbEoNK0TGTFl0hZusWeet6Wn3V40S95XNo0QyUpF5A0Dy2IDkZruBzeZAOBubmIJs+aEusGEeh/I3StfE1V41Ob5aIu7+mx49edRrQTsKapf+derFQbboDk/DISYcjbHWVbuic1mmrUoMs6mNK9SaNMC/W7bQI+uoe2Rf01dYViswWxF3LdZmmKch0dA4Hj0DafiH5twG8lMwBFfB/3OTEw6ePh9AAINFm6A2FRCrJH58uEK1l5vYW1+Bk1iw459CrfTXRncni9i83URUw/FdDMxOHNHiHDWF53uGsLnAVFhTYEcxTcivz1oFteEjUvj69g2/5R+ye3wnemTvgGmTXJ3BV1iM4lbM5ha38Lm+gpui6mgzuYjYXnCroU1jHwfm73O7N17bP9h21ZEjW/QbNuoKNzEVNCY2LhiC5srL3ClW/ET9lqprlF8M7MuI1mLM9OyXdyNMsJ8tK9Jdl1pXJlcx3bpPbZXFuR0QI97yIB9D+EmNuu4OLpm+TGZlRu6nAlE4VR3ZJ0DAla97zu2152V94ugj9bW8p/fma/tZeSh7WpH+LqzeCx2BC2JF45P4Izoq1Jd2BumiM1KRD8sbWHsR2sNqFyPGWbTJ9xK9uY4o96zMJiRU1y9SOIurHVjVvtsvl7H40S6/HnxQbJPZFlD+OzmJMbs/n03JjbjyVhrOINtaD8PZ2KzmPqjBLx7bT+bjpgLMi7/caOuvw82rzM/LGBR9vMFXBIbF4m1q2H147WjI/Ami26xOcnXcWRXivKF1YX5frnmTXejFnkkmzVoLTFkVoqQ9+V0SmfjDxHl0OX97Lp4iW3B2rpd70ZWDrJruy98MNZLwE4JhacxtCo7BpoT3TJaEjcKKO0ApXXrlQWOELMG9WL7+DyKpoliXqzbE1M+HR+BRtg4uoY6wSXbW/u39RkomCYKRhxtuobW+54QL451I9IRh+GIMvvHgtbvM1gumijMp9DdoqG1L6/83ub0q26k80WYxTzSN1uhuf3OYmZtjKOhrS/rm2prLBQVWyeY70l2/Yj7horuYxNuxc03EMKrHtEmbPDfwQk0XLgBf2JuMIO/SLEmdk3MYmx8Hy/gdgazQwtoEYMy8bLdrjRangTXYtmbZriDZAXCuxV8I3cr9KZ8Bf0Sds89ULZud8oNvkx7YwFXxLb89kt/T98Yx4DYTVH8C8sTdk1xzTvcxeaEJVoc25/dnMaU89qD4KDZzlhRuN2cxVjaGtQLez4/RV5zFXd/sDZzcMr74odZ39bznm/BI3+bivxCJC66i7BDBuyh/lsRt3Opl9amHTbTs7F5ZdOYYNl2RLFW4QY7qlPWL8p9XBsfl7uEutM0y4v2rlRqV/Hy616x2YvSV0eVRTmr814/ln15FN/M2ZvdhNn0CTcRYVhHQm038Syo9oWHu6/xuE9p29DnxauKeyTLymLAyNocrI1PXP/C2jDwPIiXrn8z6yz0L2dc/oz4+9L+/j7sYnN22tqgxeb9l1v+Z8atGw+OnEBpJY2erwIvKL6V9r0guZTvlztFipd0N32VRO5p4OXW4iXHNy5Ad16YfL4d/eqUtP3e1yPoGVOjJiUsP4rigu68JFxH5FYGBTcq57+vR+LlPsKfRr6Ied82jry5TqQDhYkeRNy2a0LbPf8LsJd/EoKrHWnllRPmgtfnxJrLtl779RQqge08+ju8vqtfjCK14v6HJ5c7yK3/nX6pflfcwEQtgMeHTeDo+oa/Zh+bcPPXjmdHQaABwq2C26U/sfna243RTWUPVh3xoH7LtT6+wex7bFdc12RPsQtMy3TLqXSwW8JmRZuVMgnhs4XNLfUP9x5p7VsyYuYM6NVvnwgRddxyI4PVrVZJIbjv5Wdo/a0Bt9oWzrFvyqrMuwXv5c9VfKl2WzB1pxFWS2zfr9Z/ajRTlqxeu+/M8P7uFHSA/uNkld+Vnic1kUxT/pJxpy3Vb1+7ChuybUOeV9W+enxI9bF+mPB+KPF8Vjff2cX21gH6juo/jz8cgW0RLTNlROvAhZaEjT3+/tZ7H6W9fdwR9/coX1asETYOTOgTzliFewUypVr6pOy71dq9QgG8fAwIsG8cg0agCw0mcHjCrZKj9uB/U4iVwEcKAp9wCzey+fIlHo9a0aqyAWh4lqO5Kga3gTrK872E1ZF4aovHMF/3K6wO2/9q/eeg5R+W3YP605B8J6hdnfpKARz23OxDVDq2+E0CJEACJEACJEACHxGBDy/cqsHbEFPL0vhmrnLCqYE0zt4YxUWxq6Gy7K1yDt4hARIgARIgARIgARIgARIggZNL4PgJt5PLkp6TAAmQAAmQAAmQAAmQAAmQwKEQoHA7FKw0SgIkQAIkQAIkQAIkQAIkQAKNI0Dh1jiWNVoykRtMot8QLzLiPxIgARIgARIgARIgARIgARKoTqCBwq2E/HASqWfO1uDVC29sChO5R/3Ivjr8HaDEblSVdhhbnkgi+VTdbhqQ6beV2uaTaA68E0a5y0MSIAESIAESIAESIAESIAES8BFonHCTLzyMInNU79UrZtCpaWj9adlXwYae7BRh9LVZ7xQKfVeL9ULZnmlbPJrL1os7xTte7uUUVwpIXdbQ9sgv8JQEPCQBEiABEiABEiABEiABEiABl0CDhJuJzHUN+vcGDj/e5fpefrBtHl75poFYi4amjiTiNzRoYcLtVQoRrRtZEXR8lUKbLl7mGkfP5aBwA/L3m6E1J5EvrwWvkAAJkAAJkAAJkAAJkAAJkICPQGOEm5lFt6bBjTS9ySGVSCKdV2WcmMqYRPLJsiuuzN8y6O+NInK9B/0jeaiTLAtiyuGjHAq/ZRC/HkHPhLUmzHyVRepON9ovR9HzKIuCOwWxgGwiidS8YkVEyB7F0N0RQbQ3hewr5R6c9AUsj/Wj57qVJlcpYriRRf/IsvQxdy9cuJkT3dD+moaMoy2kEJfr2IpId5QLt9J0DzStExkudfN1SJ6QAAmQAAmQAAmQAAmQAAmUE2iMcHsWg6a1I73uFFBA6qtABM4Wd9ExSzwVRjqha01o700hPZJENKJDv2aLHgBSHDU3o/ViJ3oS1mYepYU4WkWee2lkn6YR+1qH1uJErXKIaxraR2wltJ1DvEWDHokilkgieUtMcWxC91NHvFnpm1siaLsRQzLRg/bzGrTmOHKq3nSqpHyHC7cSjF4NzfeDMbRw4Qaxzk3TEJ9XDPOQBEiABEiABEiABEiABEiABEIINES4FUfaoWkxqKu45DU9BsNZ7iWiUc40wpKBmK7BEXHSrzcZRDXdFTJSHOk9MNyIGpBPNEH7OmVFtESm7QJy8wU7UucXboVHbdDK8qvTE630eq8yvXOlH5EaxFS4cMshrnv+e6wrCLeNNNprKMuzwyMSIAESIAESIAESIAESIIFPlUADhVvcJ9xgC7GYVG4mst8pEbiFOHQtgtgTA4bhfNKI/YcXMQsTR1bETUfkRgypsRyW36ihMVW4FZG5pkFTRZloYRnliiD1Spyo6e3mr1FMhflm2e5xharXoSjcPBY8IgESIAESIAESIAESIAESOAiBwxNuUMSanCapwxJxAObj0LRmdH6fRFJMY1Q+zhq1UHEEQK5xE+viWpqgaRqarmfsCJwqxCqIpXUR5dIRXxCo1PQ2ujqEW2EwAu27rG+dnmW1gi+SgSMiD9J0zEMCJEACJEACJEACJEACJPCpEGiMcHvS6U2DVMjJDTj0OLJj3dB0JSInBVIzksHlYDte5krCraQE2UrPYso6Mb8Qy93ToX2lTKsUom8sqvjpTy9LPrBws8SZu77OqwaAcOFWlMzCInS+zDwhARIgARIgARIgARIgARIgATREuFnTBEOEmFzLpkPXNTQnVJUmonE69K/jMNZLwE4JhacxtGqejXLhVoLxvdiMJA5D7vxYQvFJN3StFf2/iZYMCDH5XjkdnYN5mNslmOtZuZ2/t6YtkF6YOKhwkxHFStGzMOFmbWQSHqFjryQBEiABEiABEiABEiABEiABP4HGCDdYL54Oe6G0jHwpgswtfnsZqRsXrJdZixdU6xH0jHkvpC4XbmIzkjz6O6wpkmKapMwz4eQpF2LmdBxtYqdIkVYT71TLoOBG9crTH1i4zceh6/EK72QLEW4lAz2aMnXUhcIDEiABEiABEiABEiABEiABEign0CDhBhQG28qmJpYXF3KlZMI0lfmPIUnKLu0zT8k0UXIFW5m1ui/Il2kHN0LZw6pgpV/LgK9w2wMSb5EACZAACZAACZAACZAACbgEGibcsG2gR29GfH6fIsx15aQeWC8WdzZVqaUWQkjuV6vWYpdpSIAESIAESIAESIAESIAEPk4CjRNugs+2CVN579rHiYy1IgESIAESIAESIAESIAESIIEPS6Cxwu3D+s7SSIAESIAESIAESIAESIAESOCTIEDh9kk0MytJAiRAAiRAAiRAAiRAAiRwkglQuJ3k1qPvJEACJEACJEACJEACJEACnwQBCrdPoplZSRIgARIgARIgARIgARIggZNMgMLtJLcefScBEiABEiABEiABEiABEvgkCFC4fRLNzEqSAAmQAAmQAAmQAAmQAAmcZAIUbie59eg7CZAACZAACZAACZAACZDAJ0GAwu2TaGZWkgRIgARIgARIgARIgARI4CQTaJhw23o5jtTALDYkjQ3MDjzAg+DnX9bdvYG9xdLYOJbe7p3qRNx9u4TxsSU0oirbKwv45u+jOHsjjbO3s0gsmB8cwfbEOE5HpzEXLHljHi3CL/tz7u+TuDv7Opjq8M5F+bF5rB1eCbRMAiRAAiRAAiRAAiRAAkdKoH7h9u8NzD66j9SvUxi5OYWCXZ2dUgkl5VOYuY+H81s1VPYtnj9K4fmH1yU1+LbPJOZzpB49r1u4bRtZnImO4/bMOjZfb2FtcR6XokNoGi3u06E6k5eKmFsKaZi1WZy7OoGB11vSv82VF7jSPYSWiT/rLLDG7KL8m7MUbjXiYjISIAESIAESIAESIIGTR6B+4fbHEp7/UQJQwJQi3Hwodlbxy71xrP7bd9U72X2L1XkDxq85LG1tBoTbDrZWn2P2VwPG/BK2RFHYwWbewPPfdzwb2MTSr4vYtC+9XVfyVCr331tYfT4L41cDueUtSNOORZ9PO9j6LYeCkqD0Zgk54dO/nqNQKaRmC7et7YKV1vUfeLs6i9lVNWMJhdwsfJekL+u4/W0at1/uOp5Z3y9n8MX1SUw5l3dNLI7PoCsxiRi/7lsAAB7nSURBVG+GlrCm+IqK99YxMLiExRdz+CYxgzEZJNvF5oI4t+xs/rGE22PrVpmvl3B7cAmbfk8AKdyyGFOvG1mc7p230wqb87idmETXg1mMrb93Uy6OzWBso4jHg+LeHOZe7wLrL6y0gy+8egTLVs8Dwm17bhZdQytW2aF1f4+59CTuziuQdtcwkJjDnHLJdZIHJEACJEACJEACJEACJHDEBOoXbm4FKgu3rfmHe0TbNmD8eB8/zxew9XYTi9kU7v+XE3Hbwcb0fdzP5FDYKmFrdQoPfzTkdMyd1V/ww8QqXOm2PuWel377GQ8yz7HxtoSt5V9w/39yKIv17Wxgyim3tIXVyYe4P+NM5Qz6NIKH/+P4BOysT+HB4BSWNt9iaz2Hnx88xvOyAgAI4fZfDzEyKfx/i83lKTx8MIWCEJJvcnio+iXSqucOVyFKumex6JyHfv+Jsb40vkgsYG59HWOD4zh7a86OQIXdm8WiFHwvceX6MM7FZjAwuYS1d8Dmkww+685iYLGIxZlptNxM4/O+l1apAYHkuhIm3OYn8bkt3NbSo67NtflZXIxmMPCHlXusbxhneycxsLiOqVQGn3+bRtM/5zG3sobED8P4/MeV8LJVX5RjEZ384tssxmRgsHLdt7MT+OyHF9h2KmFk/efOdX6TAAmQAAmQAAmQAAmQwDEgcPjC7d+rGL/3C1ZdheWv9c7KOP4x6UywFPc2YLjCDcDujifOsIXn/+sIqAKmXLs7WJ34Ab/YhWxM/wM//6aETpyolL9oYEdxasub1ri3T1vI/c9D5N54xspEpHNLiLGbP2NJifht5R4g9VxE2sSUUM9ORXFrZHHqn7ZwEnbf/Y6pyRd4LD+W2MLSDL64PadEwv7EQGwYXfOAvPftjCL8Snj8wxAuZQWfl7hydQJjLp813L4xirvKYrHN0QxO1STclKmS6yu4fWsIF5/Yana3hG23OXYx9s8hXDEsSGN9Q7g07Tgg/BnH43c2QEWQyaieOh0y5N7c3CTOuaKtSt13X+JK14RdlsrEaTx+kwAJkAAJkAAJkAAJkMDxIXDowm0rt1e0DXj7PGULGQdKYI3b9gae/2scI2Kjkwf3Ef+HI9yAjV8TGF/ZAcRUTDsSJ61sLWH8wQ+IPxjB+L8WsemKBqcM67v0/55j9v9GrE1UfozjH/Z6tL19Coks2lMi1YmPsgRx/X+f+6N961O4aQvV0ssRPMgJcSPE4AgW3fCP4ufLGZxVBYuYIiimHCYm8OV1W2QJcdc17G4OIjYJOdNliyNxzxFettm1oVGcGxLr44RQUqc4Bs8BqPlVsaS4aE2VHMIZZ4OS7nF8M6msvzPFlMwJXOxO42x0GH9xfAMghJsj4sr8UctTj0XZ6rk4vj4s69w0qoQ+Vd/L6r6LqR+H0TJRAt69wKWoMu1UrRuPSYAESIAESIAESIAESOAYEDhc4VYl2ibqL0XSgjLY9kXVNjH7Ywqzv7/FjgzKBETd7wYS/7eK0uovuP+rM81Rofpve3riP0Iifn/M4v6jWWy8taNuivh6m39sCyrHViDSd/MXdxMWmUJE64YXyzchETb/y5ra6VgS0byEMyVzexEjAzlsiWmT6UX/Gjsng4wMZZCwpxY6lyGv29EpIVDUqJybyBZegXuLqVF8mQ4Tbku40qVEvISd6QlP+KliSS1DXPcJQPXmloz+tYytY/OdFVlTxZp6XJdwuzGNudcvcUVEDFftCF4IF6/uAMR0zjsL2MxO4IwzJVN1ncckQAIkQAIkQAIkQAIkcEwIHKpwqxZtkwyEaPnRXvclLojzm05UTUS3FJH0dhE/KxE3GakaSCH16IEydXEHG/8ax3N3KuMmZgceYzEYDhORr6w3RfPty5/diBvMRTz+0XslQWl1Cg/ccq1pmY/dRW07KExW2DFTCLebCfyy5kzJ3MLzRwkYvzutv4PV/0tI/2Xk0Lns+97F4tAoPrs1Y23cIe6VtvC4bxhnEkvWGi0RMeoax4DY2EP82/0dib4sHos9ReQ9RcyYL3ElmsbtJZEwGGErYSoxjHMPVrEpTL1bw93bQ3UKtyLu3hTlW66htIKuqBdlq1m4bczhy+sTeCzr+B6Lg6M45UQiFUFp7cBpr3Hbs+7CH7HxyyjO3RSbv9j+8YsESIAESIAESIAESIAEjiGBwxNuNUTbHB5b+RHE//M+Hgzcx8PMEp5POsINsO7Z74QbmcIv7ho3K7dYG3YzsKnHzu+zSP1n3J4CeR8jC1vKOjm3VCym44g/EFMwH2Bk+hff1v07m88x/kiUm8L48yXMqq8okK9AsPP+GMeDsSX/dEinCBnFm8Xz6YeWL/8ZR+pfGz5fxPq4f4RFBB0b8vtPzA2N42zXEP4SHcZnV4fwxQ/z9gYjVsLtuWmc+5s9XfJvQnytuBtvqPfO/C2NlieOcgwKNyH6XmMskbHfFzeJsXS9ETdAiKmzjm/fZnEldgDhhj8x9WBU1v2zv6VxZWjaewWAItwAR+haG7BUrrvFbXEwjVO+NYA+8DwhARIgARIgARIgARIggWNBoIHCrc767O6gpGzi4bMm7pWcqJXvDoRwszb78F8XZ+JdcuG5lLT/LiHUtB28slKqUyVryKskcQ//XbKne7pX5IHc2MS3OYv/vv9sF9tbJrZ9vqkp9rr/Htuv98pr29l9L7SP90+dKuld3f/Rbgmbrz/Qe93KvKux7mX5eIEESIAESIAESIAESIAEjgeB4yPc9sujtIHV51N4uNf74fZr00m/vYSf//Mhpl4WsPF7AYu/phAPW8PmpD/Q9xYKz3P4+UdvZ8kDmWlopveYepDGWfF6gPkVTI3PoOVGGldmK+zu0tCyaYwESIAESIAESIAESIAESKASgZMr3LY3sbpasF/IXal6dVwX73ZzXs79cgNv1ShUHWbdrDtvsbG6io3g2js3wVEdvMfavPey7MdL8oVoR+UMyyUBEiABEiABEiABEiABEgBwcoUbm48ESIAESIAESIAESIAESIAEPhECFG6fSEOzmiRAAiRAAiRAAiRAAiRAAieXAIXbyW07ek4CJEACJEACJEACJEACJPCJEKBw+0QamtUkARIgARIgARIgARIgARI4uQQo3E5u29FzEiABEiABEiABEiABEiCBT4QAhdsn0tCsJgmQAAmQAAmQAAmQAAmQwMklQOF2ctuOnpMACZAACZAACZAACZAACXwiBCjcPpGGZjVJgARIgARIgARIgARIgAROLgEKt5PbdvScBEiABEiABEiABEiABEjgEyFA4faJNDSrSQIkQAIkQAIkQAIkQAIkcHIJULid3Laj5yRAAiRAAiRAAiRAAiRAAp8IAQq3T6ShWU0SIAESIAESIAESIAESIIGTS4DC7eS2HT0nARIgARIgARIgARIgARL4RAhQuH0iDc1qkgAJkAAJkAAJkAAJkAAJnFwCDRFuJdOEGfIp7RwhmJ0STLMU7sBe98Jz7P/qhygDgGB/pJz3T4Y5SIAESIAESIAESIAESIAE9kmgAcKtiHSHBk0r/8Tn9+lNI5PPx6FpOnqmQ8SbuNeRRrHm8kooLhhYflNzBmDfZezDtps0h7im4Ug5u77wgARIgARIgARIgARIgARI4LAINEy4tY/ULoMOqzI+u1K4adD0bmSDgmvfosoSp/sSSPsuw+d9jScUbjWCYjISIAESIAESIAESIAESONEEPohwK0wkkZovYHksie6OHmRtjWf+lkF/bxSR6z3oH1uGqaA051NITixDpIlfjyDam0JOCLA3eaQT3Wi/3oOUsYdYlMIthuT9VujXMv7oWoioquxLAdlEDJ3NGtq+SyL5KOf5uVOE8SiG7o52dN9JweeOXUbhTQ6pCnWU1TWXkfmpB9HLUfT8lMGyCsHmUTRSiH3XjvbvYoE6lws381kKyUEDxZBAo4KXhyRAAiRAAiRAAiRAAiRAAieIwAcRbrl7GiJftaP9+zSyRl6KiuJYFE0t3Ug9NWAYWaRu+gVWcaQdWkcneu6kYRgG0t+3QvsqiuiNODKGAWMsjja9GfH5CgpFCrc4cigg9ZWOzpGC1ywB4ba3LyaWjQxilzVEBwwYC0XIEkt5JCNNaOvLSP+yj3oQOd+O1Cu7GFHG5U50djh1zCD+tY7Wezkrv0i2kUH0fCu6H2VtG91o1TuR2XBcLSGfiKDpa7vOhm2jL2/b8As381kcrS1x5ELEn2OR3yRAAiRAAiRAAiRAAiRAAiePQMOEW/O1GJKJpPdRIlNCuGk3s55gQQ5xvRnJvArMElgxwxJiUrhdTsGTWznENH+efJ+OyKCXQrUm15hpQrgBWE+jXe9Eet1O4RNu1X0ByqdKmmNR6L2GUidAXvveviaFY5sn5ETR2wZ6dO9a7p6O5oQPAgqDbdAdGxtptGtRZNSpnj4bnnCzRFsMhprWB4QnJEACJEACJEACJEACJEACJ5VAw4Rb5I4VeRLRMflxIlMApHC7JyWUxelVChE9Dr9kAfL3m9F837pqRdzUDUSESGlH2o1GATKNaldtBSmcbOEGoDDSCb0jbQlBVbjV4Eu5cCvB6NXQnrAiZW6dB6LQHLEpymhOBupoIvudhu4JERIrIHVZR3xBdRpAPolmO19pugdacJonVBuWcOu+F0erFkV6j5mjgVJ4SgIkQAIkQAIkQAIkQAIkcIIINEy47bU5SZlwW4hD/6stohRYheF2aLYQa7RwE0IpfU1Hm4jQqcKtBl/KhVsRmWv2mjc1yiiPs544DKlj7o4Gi1Uecb3diwI6HER00I4UFp90QrujCF47jWfDEm761zHErgWmYTr2+E0CJEACJEACJEACJEACJHDiCRyNcDOz6Na6kfWtxbKiWJ1PrLBR44WbtaasU29FMhHzXgdQgy/lwg3Yc5qm6BYy4tcDe+an3VHUKJsaOfP6kS/KpkTfvBSqDW+qJLZziLcE1vJ5mXhEAiRAAiRAAiRAAiRAAiRwggkcjXCDEGlWhMjRbnKNlrJ1/6EINwDmRDd08c459z1u1X1xhJs1xdFu7VcptKnr5mAid68VF/rsCJkQbrqOyD1vF8rCcCf0Fm/6pBBpurqZiGmJL68ce2OVYW8dn9+GItzEy7gX4nJzE3ct3wnumHSdBEiABEiABEiABEiABEjAI3BEwk1s1LGM9K0IdK0JTec16BejSK14O0QelnCDLRo94VbdF4GrNB9HRNd869bMZ0m0S98voEnTceFqP/LbNlx7OmbuSRRN+gVcEOkiPcg6G6TIZCUsj/RIu03nm6DpFxB9tOzb8ARvckh2iHtNaNI1NH0VVzYg8Qs3YVKu5VPFoO0Ov0iABEiABEiABEiABEiABE4ugQYItzorXzJhmp5gq9NafdkP6EvJNLFnFXZKVeoo7pt+wRasybYJ0xGFwXs8JwESIAESIAESIAESIAES+KgJHL1w+6jxsnIkQAIkQAIkQAIkQAIkQAIkUD8BCrf6GdICCZAACZAACZAACZAACZAACRwqAQq3Q8VL4yRAAiRAAiRAAiRAAiRAAiRQPwEKt/oZ0gIJkAAJkAAJkAAJkAAJkAAJHCoBCrdDxUvjJEACJEACJEACJEACJEACJFA/AQq3+hnSAgmQAAmQAAmQAAmQAAmQAAkcKgEKt0PFS+MkQAIfioA5n0IykZSf1Lz5oYplOSRAAiRAAiRAAiTwQQg0Rri9yroDpuREQTou3m1W2tmrDtXebbZX3uN6r4b3sdXj+k4RxiMxMO2HUbQNBa9VfWec4kDJRCFvIL9e5R1ySpZaD4uGNYjudx0FgtdkH6nVoEgn61atX9Vg8E0e2YUTNrAX7/ET7wtUPyHv9SuZBeTnl1EMuRdKpob+Uv1ZDrW8r4s1l7EHh9JGHoZhoP+GhvYR5wHZlxtMTAIkQAIkQAIkQALHlkBjhNt8HNrlGDKGAWNFDIhziGsa4vN71HsjjXYtjtweSRp3q4TigoHlN42zGGpJ1qkd6Y3Qu3VeNJG9qaP1ZgrZpzkU5DvLQ66JtuhIo9qwtTDRg4jehNbLEURamqBHepBdr9NFO7v5tBt6SzdST7PIvbJerl5+TfSR/bAqId/XCq1av6qhCss/tUJr6ceynVYO+GW/rSHzkSQpwejVZN1F/d3PPfXpKSB7sxXa+VZELkdwQW9C9In1I8qeLlftLzU8y3sWUMvNItIdVf5eSDO1cABy9yjcaqHONCRAAiRAAiRAAieLQOOEm08s1DDY+6DCrdaBYZ2Nd6jCLawOIdeqDsQBvEqhTe9EWhFqywNt0L83YMms+jgUR9qh+UQFUH5tn8Itn0RrSyc6L9cywN+f/+W+7S//4acOaedAobIOHSkUnCj3ehqdehtSrwIJg6dV+0sNz3LQ5r7Pq9fPMllbOgq3fTcAM5AACZAACZAACZwAAh9OuJkFZB/1IHo5ip5HOZjrtUfc5NqViQJKKxkkv2tH+3cxpHxT8PqRnHDiJzb1NzmkEhkslwrIJmLobNbQ9l0SSVG20zBymmEM3R3t6L6T8qYfyvsiXwq59WVkEt1ov5V1o1jmbxn090YRud6D/rFlz54j3FbsPE5dnfKqfZvLyPxkM/opg2XHUVPURalDIotC2DVh3x6IF0T9w3wEICNMC0W/SKs6gFecr8jNRO5RErFrzdC+6pbTZ7Ovwq4JW7ZwWxf3Q+qsFAcUkPqqFfGFQk2RmaKxV38AZH+y+0FhQvU3hZzD3Fe+ciK5Z7Hs8P0p794UU0FjIf1TJtgpIjeSRHdHBNHefmRWPIksfBBrssxnKfRcF/eDfbGayC0gdVlHzPBsijLziWY03/f8cx1VD6r2lxDhVrH9bba+ZzWJzG9BqCYKT7265t7U1q5un6kS0aZwUxuYxyRAAiRAAiRAAh8LgQ8j3LZziLfoaOvLyDUo2UfdaL/WiUiNUyWtaEInoh1xazrm0xS6W3R0jthTwfJJNDcnoQ5RC4+cCJKJZSOD2GUN0QEDhiNYSnkkI02uT8ZYHG26EAfO4FcMWCNo+2s7YiNZN19xLIqm8+1IjhkwjCz6rzehyYlUSeHWjNaOHqSeivsZxL/Wa4tkbWQQPd+K7kdZySiTaEfT+RgMsVapVERerYOxDDPsmuiVYiB+uROdHWKqoudD672cX6j5enAJuXutaK02yBd5AtyEGI+cb7cjO9aU1MydCLQb/bIey2/CrglDFt/Oa+1enfvaoLfEkQuszyoMt6P5jvC/togL9uwPdvTPjhCbKwY8f/MoOs3v46OcyDZuQ9v1KPpFH5BTLEvIJyJo+trrnz2RJrQ/cqYqmsh+Z01zNVYKWH4aR9t5L+IphEbkWhRR+/nICA56N7LO1F4zi26tG+n5DPrF5hs/pZFT58LK+53IqNeEy89i1afNVu0vAeG2Z/s7bDvRcyuFrGFAPOutWiuS7sNp9TVdYdXdUWMktRoHu5ko3JT+ykMSIAESIAESIIGPhsAHEW5SRPX6p+EVBtug7Ue46T2WiHHQy+l+MVhBBivi4K2pExEaNQJRPuCXYvB6xouWCU0y3QP9qxSs4bY1YO1+qo7k80g2NyuDUEvIpBNp5IXYkIP6CPpXHCetaYm1CFQRHWkbdAb6Vn55zR38l9cBYUJGDMS1wBS5bQM9YdPmiln0iDVuF5vRds/wsVBq4Ds0x6LQA20przniFWHTIsOuWXz9dRZrmHS0uXUGICKzbtuHMfC5Z59Y7V+pP1g/BHjrAOV5YGpnmFV5LayN32QQdX20c8prTv8UdW33TU01iwV38x4hNLSbWUVYW/XsnrAjVWLa4/kmtPf2I/00i/S9djRprYjP231T+hSyXlSKMqc/V6hR1f7iF27V2l+yvOytHxSl5u7pXuRPPrchz3ItaxercbCrSOFWoa15mQRIgARIgARI4EQT+ADCzdpQwB2EOrhepfYXcbuWcacqWiaEWPNElBSHzuBbDA59EbjggN/yqfNJIERh/6KfleNl/4BVlil89tl1KmN/ywF0YMONSoNqX1ZRlwvoeSQiZN5HRoJckRSsgzAQck0MxMt8FBEfDWVtIKN2BgwxbU1EH+/nFfHgc9A+sbi1J6yooOvrQBTaZU8ghAmh8muCr9d+TmnmRDe077K2iCwic61J8Tukvk7GwPde/UH6oqzJLPctYEw9DWljIfi1jqSMMLlMjH5EtYgdibQibnpEROlyWA6E9aRwc/quXVY18SE4uT8yVOpjoi/8NW3/EKFWQjmu2l/U56B6+wfZipJUvpKV276OH6L/H2ztoo+Dba4aO6dUfpMACZAACZAACZDASSLwAYSbGHxr6JlWI1d2JGU/ETdXwDh4A4N4MXjVrQiHGLQ3J9y5WSECx/Ip9syx5XznEHN3OlQHrPb9hTj0vQbCIYN6KwoXEg1xipTfecT1ZnR+b72DynkXlfx21+QF6ivzhVyrMFjP3amy056IELl19zmnnFjc5FpB+31Znq9ZVyCoA3Unc/k1wdcfhZJplel9clB+NYVldwv8ZaT+qiH21IQZmE7plON+79EfguKi3DfXSvlBSBsXn3S6a/o8HlZbZt3NQUwUjLS1NvKiDj3iTQk9iHCz+lU35I8MJQM9IW0nRVLZDx6BKlXtL+pzUL39g2xFaSpfyarsWS4g/deDCTcfB7tqFG6BNuYpCZAACZAACZDAR0HgAwg3IN+nTJWysclB5X6EmxLRkSbkYFVd12Mic11Mj8wjddmJdDhtVC5w8vdDNm4QETU9bq+VUwesth0Zkeuxp2c6tgH3fWQhg/rahFsFcesVESI+xc3yesk1blrQR3sq6YJlsPA0Ced9e14Ror7lETDvvnUk2jISmNIZTKMO1J175dcsvkFBXxiMQO+zRLcUNOr29+pxIELllON9q/3BX6+guCj3zbNSdhTWxkLQB/tnWUb1gniNg+ZOH6wm3Eq/Zfyb6ghTvoi11Q+iY+omICUY35c/d6oX8lgItz37i/85qNb+QbaiDB9fwSoYEZbPcnXhVp2DVTsKt7JW5gUSIAESIAESIIGPgMAHEW5is4hWdft5M4d4RIe+H+Gm6egcdtaAmXIzDT0w5UpGaFpa0eyuU3NayBrY+qYKBrfE3ykgfU1XNujwD1gtS2KqmA6x0YczRDaNmLcxStigXl6rFnGz19eJjTkcw9KfJnS6LxIOEWmVhJuuI6L4WBjuhN7ibd5SMmLQ9U6k3N3+TCwP+tM45Mq+g9xgtcWFPu+dYr6Bum2g/Jrga0eenDrLLezVjSyCpYcxCKbxziv1h6C4kOc3nemZXv7Qo7A2lrteqv0TMJ/F0XoxjpzYnl/kUfv/TgGpDi8CWk24WeK/FXHDBlWy+2qfN7VV/BCit8SQtWf/yvL1TmSq7MAohf6e/SXwHFRp/yBbwdDf9nkkxcZCvmc5Al2vLtxq4SDKo3AL7bm8SAIkQAIkQAIkcMIJfBjhBsB64bOGposXoJ+PIvOs9tcBWIPBfqT7ItDPX8AFXUNTR7+1IYjaAPYv9+2u2PFulubjiOiab/2X+SyJ9vMa9PNN0LWmwAYdgQGrY2p7GakbF2T6JpH3YhQpZ2v3sEF9jcINKGF5RLwU22Yk/On1ph+GRtcqCbeONHJPomjSL+CC8DHk5dqFsR60nfde5tzUkUTO2cXQqWuFb5fbxQto0nRcuOpvC/9A3TJSfk3wbUf6mdhNU8eFi03Q9Ah6JhxxHlb4/oQbSgZiuubf7MQREsoaN4hdTyM6tBoijpZ4CKxjFK6+ySHZIepgM7/Yif68Nz3Y6v86LkRaJbPILa9tqwo3AOZCv+yr1su3g31DOFBC/qdO+WzINOfbfeWH0ZTXRMRtz/5S/hzs1f7Ws+pt/CLKKGv79Sx6BG/5LIsXhedqes2DsFWdA4VbxbbmDRIgARIgARIggRNN4JCEWyUmJZimWbYBhhzYqdPgnGN7cO0bDJb2WN8kB+pRZGoUII6Xcqqj8+Ji52K1b+GH6Q3MqyW3tr/3hJI1ALfOvd0PhRWb0X79CXNgR9ja20dR92ASKSScNlC/A9MTw/KGuVHLNXe6aS2J7TTV+s0+THlJpdAOa6cQsebl8o629+oX9bdt9b4abHNLeKn9zTn29zsANfQXr6LWFOFg31HvVzsO7z+1+RvGQe0PYT/eVPOH90mABEiABEiABEjgOBNonHBzBviBwX0jKu8TbqEGSzCLReQH2mt7Z1qoDV4kARIgARIgARIgARIgARIggeNJoDHC7ZDrVpzogZhaFti83yt120DscgTtvWksV9tt0MvFIxIgARIgARIgARIgARIgARI4EQROhHA7ESTpJAmQAAmQAAmQAAmQAAmQAAkcEgEKt0MCS7MkQAIkQAIkQAIkQAIkQAIk0CgCFG6NIkk7JEACJEACJEACJEACJEACJHBIBCjcDgkszZIACZAACZAACZAACZAACZBAowhQuDWKJO2QAAmQAAmQAAmQAAmQAAmQwCERoHA7JLA0SwIkQAIkQAIkQAIkQAIkQAKNIkDh1iiStEMCJEACJEACJEACJEACJEACh0SAwu2QwNIsCZAACZAACZAACZAACZAACTSKwP8HdwrH/wid5WUAAAAASUVORK5CYII="}}},{"metadata":{"trusted":true},"cell_type":"code","source":"package_path = '../input/pytorch-image-models/pytorch-image-models-master' #'../input/efficientnet-pytorch-07/efficientnet_pytorch-0.7.0'\nimport sys; sys.path.append(package_path)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"from glob import glob\nfrom sklearn.model_selection import GroupKFold, StratifiedKFold\nimport cv2\nfrom skimage import io\nimport torch\nfrom torch import nn\nimport os\nfrom datetime import datetime\nimport time\nimport random\nimport cv2\nimport torchvision\nfrom torchvision import transforms\nimport pandas as pd\nimport numpy as np\nfrom tqdm import tqdm\n\nimport matplotlib.pyplot as plt\nfrom torch.utils.data import Dataset,DataLoader\nfrom torch.utils.data.sampler import SequentialSampler, RandomSampler\nfrom  torch.cuda.amp import autocast, GradScaler\n\nimport sklearn\nimport warnings\nimport joblib\nfrom sklearn.metrics import roc_auc_score, log_loss\nfrom sklearn import metrics\nimport warnings\nimport cv2\nimport pydicom\nimport timm #from efficientnet_pytorch import EfficientNet\nfrom scipy.ndimage.interpolation import zoom\nfrom sklearn.metrics import log_loss","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/cassava-leaf-disease-merged/merged.csv')\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.label.value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.read_csv('../input/cassava-leaf-disease-classification/sample_submission.csv')\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Helper Functions"},{"metadata":{"trusted":true},"cell_type":"code","source":"def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = True\n    \ndef get_img(path):\n    im_bgr = cv2.imread(path)\n    im_rgb = im_bgr[:, :, ::-1]\n    #print(im_rgb)\n    return im_rgb\n\nimg = get_img('../input/cassava-leaf-disease-classification/train_images/1000015157.jpg')\nplt.imshow(img)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"class CassavaDataset(Dataset):\n    def __init__(\n        self, df, data_root, transforms=None, output_label=True\n    ):\n        \n        super().__init__()\n        self.df = df.reset_index(drop=True).copy()\n        self.transforms = transforms\n        self.data_root = data_root\n        self.output_label = output_label\n    \n    def __len__(self):\n        return self.df.shape[0]\n    \n    def __getitem__(self, index: int):\n        \n        # get labels\n        if self.output_label:\n            target = self.df.iloc[index]['label']\n          \n        path = \"{}/{}\".format(self.data_root, self.df.iloc[index]['image_id'])\n        \n        img  = get_img(path)\n        \n        if self.transforms:\n            img = self.transforms(image=img)['image']\n            \n        # do label smoothing\n        if self.output_label == True:\n            return img, target\n        else:\n            return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Define Train\\Validation Image Augmentations"},{"metadata":{"trusted":true},"cell_type":"code","source":"from albumentations import (\n    HorizontalFlip, VerticalFlip, IAAPerspective, ShiftScaleRotate, CLAHE, RandomRotate90,\n    Transpose, ShiftScaleRotate, Blur, OpticalDistortion, GridDistortion, HueSaturationValue,\n    IAAAdditiveGaussianNoise, GaussNoise, MotionBlur, MedianBlur, IAAPiecewiseAffine, RandomResizedCrop,\n    IAASharpen, IAAEmboss, RandomBrightnessContrast, Flip, OneOf, Compose, Normalize, Cutout, CoarseDropout, ShiftScaleRotate, CenterCrop, Resize\n)\n\nfrom albumentations.pytorch import ToTensorV2\n\nfrom albumentations import (\n    HorizontalFlip, VerticalFlip, IAAPerspective, ShiftScaleRotate, CLAHE, RandomRotate90,\n    Transpose, ShiftScaleRotate, Blur, OpticalDistortion, GridDistortion, HueSaturationValue,\n    IAAAdditiveGaussianNoise, GaussNoise, MotionBlur, MedianBlur, IAAPiecewiseAffine, RandomResizedCrop,\n    IAASharpen, IAAEmboss, RandomBrightnessContrast, Flip, OneOf, Compose, Normalize, Cutout, CoarseDropout, ShiftScaleRotate, CenterCrop, Resize\n)\n\nfrom albumentations.pytorch import ToTensorV2\n\ndef get_train_transforms():\n    return Compose([\n            RandomResizedCrop(CFG['img_size'], CFG['img_size']),\n            Transpose(p=0.5),\n            HorizontalFlip(p=0.5),\n            VerticalFlip(p=0.5),\n            ShiftScaleRotate(p=0.5),\n            HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n            RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n            Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n            CoarseDropout(p=0.5),\n            Cutout(p=0.5),\n            ToTensorV2(p=1.0),\n        ], p=1.)\n  \n        \ndef get_valid_transforms():\n    return Compose([\n            CenterCrop(CFG['img_size'], CFG['img_size'], p=1.),\n            Resize(CFG['img_size'], CFG['img_size']),\n            Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n            ToTensorV2(p=1.0),\n        ], p=1.)\n\ndef get_inference_transforms():\n    return Compose([\n            RandomResizedCrop(CFG['img_size'], CFG['img_size']),\n            Transpose(p=0.5),\n            HorizontalFlip(p=0.5),\n            VerticalFlip(p=0.5),\n            HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n            RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n            Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n            ToTensorV2(p=1.0),\n        ], p=1.)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def inference_one_epoch(model, data_loader, device):\n    model.eval()\n\n    image_preds_all = []\n    \n    pbar = tqdm(enumerate(data_loader), total=len(data_loader))\n    for step, (imgs) in pbar:\n        imgs = imgs.to(device).float()\n        \n        image_preds = model(imgs)   #output = model(input)\n        image_preds_all += [torch.softmax(image_preds, 1).detach().cpu().numpy()]\n        \n    \n    image_preds_all = np.concatenate(image_preds_all, axis=0)\n    return image_preds_all","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# ld-efficientnet-b4-best\n\nCFG = {\n    'fold_num': 6,\n    'n_class':5,\n    'seed': 42,\n    'model_arch': 'tf_efficientnet_b4_ns',\n    'img_size': 512,\n    'epochs': 32,\n    'train_bs': 8,\n    'valid_bs': 8,\n    'lr': 1e-4,\n    'num_workers': 4,\n    'accum_iter': 1,\n    'verbose_step': 1,\n    'device': 'cuda:0',\n    'tta': 3,\n    'used_epochs': [0,1,2,3],\n    'weights': [1,1,1,1]\n}\n\nclass CustomEfficientNet(nn.Module):\n    def __init__(self, model_name=CFG['model_arch'], pretrained=False):\n        super().__init__()\n        self.model = timm.create_model(CFG['model_arch'], pretrained=pretrained)\n        n_features = self.model.classifier.in_features\n        self.model.classifier = nn.Linear(n_features, CFG['n_class'])\n\n    def forward(self, x):\n        x = self.model(x)\n        return x\n\n\nseed_everything(CFG['seed'])\n\nfolds = StratifiedKFold(n_splits=CFG['fold_num'], shuffle=True, random_state=CFG['seed']).split(np.arange(train.shape[0]), train.label.values)\n\nfor fold, (trn_idx, val_idx) in enumerate(folds):\n    # we'll train fold 0 first\n    if fold > 0:\n        break \n\n    print('Inference fold {} started'.format(fold))\n    \n    test = pd.DataFrame()\n    test['image_id'] = list(os.listdir('../input/cassava-leaf-disease-classification/test_images/'))\n    test_ds = CassavaDataset(test, '../input/cassava-leaf-disease-classification/test_images/', transforms=get_inference_transforms(), output_label=False)\n    \n    tst_loader = torch.utils.data.DataLoader(\n        test_ds, \n        batch_size=CFG['valid_bs'],\n        num_workers=CFG['num_workers'],\n        shuffle=False,\n        pin_memory=False,\n    )\n\n    device = torch.device(CFG['device'])\n    model = CustomEfficientNet(CFG['model_arch']).to(device)\n    \n    tst_preds = []\n    \n    for i, epoch in enumerate(CFG['used_epochs']):    \n        model.load_state_dict(torch.load('../input/ld-efficientnet-b4-best/{}_fold{}_{}'.format(CFG['model_arch'], fold, epoch)))\n        \n        with torch.no_grad():\n            for _ in range(CFG['tta']):\n                tst_preds += [CFG['weights'][i]/sum(CFG['weights'])/CFG['tta']*inference_one_epoch(model, tst_loader, device)]\n\n    tst_preds = np.mean(tst_preds, axis=0) \n    \n    del model\n    torch.cuda.empty_cache()\n    \ntst_preds_0 = tst_preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ld-efficientnet-b3-best\n\nCFG = {\n    'fold_num': 6,\n    'n_class':5,\n    'seed': 42,\n    'model_arch': 'tf_efficientnet_b3_ns',\n    'img_size': 512,\n    'epochs': 32,\n    'train_bs': 8,\n    'valid_bs': 8,\n    'lr': 1e-4,\n    'num_workers': 4,\n    'accum_iter': 1,\n    'verbose_step': 1,\n    'device': 'cuda:0',\n    'tta': 3,\n    'used_epochs': [0,1,2,3],\n    'weights': [1,1,1,1]\n}\n\nclass CustomEfficientNet(nn.Module):\n    def __init__(self, model_name=CFG['model_arch'], pretrained=False):\n        super().__init__()\n        self.model = timm.create_model(CFG['model_arch'], pretrained=pretrained)\n        n_features = self.model.classifier.in_features\n        self.model.classifier = nn.Linear(n_features, CFG['n_class'])\n\n    def forward(self, x):\n        x = self.model(x)\n        return x\n\n\nseed_everything(CFG['seed'])\n\nfolds = StratifiedKFold(n_splits=CFG['fold_num'], shuffle=True, random_state=CFG['seed']).split(np.arange(train.shape[0]), train.label.values)\n\nfor fold, (trn_idx, val_idx) in enumerate(folds):\n    # we'll train fold 0 first\n    if fold > 0:\n        break \n\n    print('Inference fold {} started'.format(fold))\n    \n    test = pd.DataFrame()\n    test['image_id'] = list(os.listdir('../input/cassava-leaf-disease-classification/test_images/'))\n    test_ds = CassavaDataset(test, '../input/cassava-leaf-disease-classification/test_images/', transforms=get_inference_transforms(), output_label=False)\n    \n    tst_loader = torch.utils.data.DataLoader(\n        test_ds, \n        batch_size=CFG['valid_bs'],\n        num_workers=CFG['num_workers'],\n        shuffle=False,\n        pin_memory=False,\n    )\n\n    device = torch.device(CFG['device'])\n    model = CustomEfficientNet(CFG['model_arch']).to(device)\n    \n    tst_preds = []\n    \n    for i, epoch in enumerate(CFG['used_epochs']):    \n        model.load_state_dict(torch.load('../input/ld-efficientnet-b3-best/{}_fold{}_{}'.format(CFG['model_arch'], fold, epoch)))\n        \n        with torch.no_grad():\n            for _ in range(CFG['tta']):\n                tst_preds += [CFG['weights'][i]/sum(CFG['weights'])/CFG['tta']*inference_one_epoch(model, tst_loader, device)]\n\n    tst_preds = np.mean(tst_preds, axis=0) \n    \n    del model\n    torch.cuda.empty_cache()\n    \n    \ntst_preds_1 = tst_preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ld-pytrch-effnet-b3-lr-lbs-valprv\n\nCFG = {\n    'fold_num': 6,\n    'n_class':5,\n    'seed': 42,\n    'model_arch': 'tf_efficientnet_b3_ns',\n    'img_size': 512,\n    'epochs': 32,\n    'train_bs': 8,\n    'valid_bs': 8,\n    'lr': 1e-4,\n    'num_workers': 4,\n    'accum_iter': 1,\n    'verbose_step': 1,\n    'device': 'cuda:0',\n    'tta': 3,\n    'used_epochs': [5,6,8,9],\n    'weights': [1,1,1,1]\n}\n\nclass CassvaImgClassifier(nn.Module):\n    def __init__(self, model_arch, n_class, pretrained=False):\n        super().__init__()\n        self.model = timm.create_model(model_arch, pretrained=pretrained)\n        n_features = self.model.classifier.in_features\n        self.model.classifier = nn.Linear(n_features, n_class)\n        \n    def forward(self, x):\n        x = self.model(x)\n        return x\n\n\nseed_everything(CFG['seed'])\n\nfolds = StratifiedKFold(n_splits=CFG['fold_num'], shuffle=True, random_state=CFG['seed']).split(np.arange(train.shape[0]), train.label.values)\n\nfor fold, (trn_idx, val_idx) in enumerate(folds):\n    # we'll train fold 0 first\n    if fold > 0:\n        break \n\n    print('Inference fold {} started'.format(fold))\n    \n    test = pd.DataFrame()\n    test['image_id'] = list(os.listdir('../input/cassava-leaf-disease-classification/test_images/'))\n    test_ds = CassavaDataset(test, '../input/cassava-leaf-disease-classification/test_images/', transforms=get_inference_transforms(), output_label=False)\n    \n    tst_loader = torch.utils.data.DataLoader(\n        test_ds, \n        batch_size=CFG['valid_bs'],\n        num_workers=CFG['num_workers'],\n        shuffle=False,\n        pin_memory=False,\n    )\n\n    device = torch.device(CFG['device'])\n    model = CassvaImgClassifier(CFG['model_arch'], CFG['n_class']).to(device)\n    \n    tst_preds = []\n    \n    for i, epoch in enumerate(CFG['used_epochs']):    \n        model.load_state_dict(torch.load('../input/ld-pytrch-effnet-b3-lr-lbs-valprv-train/{}_fold_{}_{}'.format(CFG['model_arch'], fold, epoch)))\n        \n        with torch.no_grad():\n            for _ in range(CFG['tta']):\n                tst_preds += [CFG['weights'][i]/sum(CFG['weights'])/CFG['tta']*inference_one_epoch(model, tst_loader, device)]\n\n    tst_preds = np.mean(tst_preds, axis=0) \n    \n    del model\n    torch.cuda.empty_cache()\n    \ntst_preds_2 = tst_preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ld-pytorch-tpu-vit-base-patch16-tcel-vldprv-train\n\nCFG = {\n    'fold_num': 6,\n    'n_class':5,\n    'seed': 42,\n    'model_arch': 'vit_base_patch16_384',\n    'img_size': 384,\n    'epochs': 32,\n    'train_bs': 8,\n    'valid_bs': 8,\n    'lr': 1e-4,\n    'num_workers': 4,\n    'accum_iter': 1,\n    'verbose_step': 1,\n    'device': 'cuda:0',\n    'tta': 3,\n    'used_epochs': [3,5,7,11],\n    'weights': [1,1,1,1]\n}\n\nclass CustomViT(nn.Module):\n    def __init__(self, model_name=CFG['model_arch'], pretrained=False):\n        super().__init__()\n        self.model = timm.create_model(model_name, pretrained=pretrained)\n        n_features = self.model.head.in_features\n        self.model.head = nn.Linear(n_features, CFG['n_class'])\n\n    def forward(self, x):\n        x = self.model(x)\n        return x\n\n\nseed_everything(CFG['seed'])\n\nfolds = StratifiedKFold(n_splits=CFG['fold_num'], shuffle=True, random_state=CFG['seed']).split(np.arange(train.shape[0]), train.label.values)\n\nfor fold, (trn_idx, val_idx) in enumerate(folds):\n    # we'll train fold 0 first\n    if fold > 0:\n        break \n\n    print('Inference fold {} started'.format(fold))\n    \n    test = pd.DataFrame()\n    test['image_id'] = list(os.listdir('../input/cassava-leaf-disease-classification/test_images/'))\n    test_ds = CassavaDataset(test, '../input/cassava-leaf-disease-classification/test_images/', transforms=get_inference_transforms(), output_label=False)\n    \n    tst_loader = torch.utils.data.DataLoader(\n        test_ds, \n        batch_size=CFG['valid_bs'],\n        num_workers=CFG['num_workers'],\n        shuffle=False,\n        pin_memory=False,\n    )\n\n    device = torch.device(CFG['device'])\n    model = CustomViT(CFG['model_arch']).to(device)\n    \n    tst_preds = []\n    \n    for i, epoch in enumerate(CFG['used_epochs']):    \n        model.load_state_dict(torch.load('../input/ld-pytorch-tpu-vit-base-patch16-tcel-vldprv-train/{}_fold{}_{}'.format(CFG['model_arch'], fold, epoch)))\n        \n        with torch.no_grad():\n            for _ in range(CFG['tta']):\n                tst_preds += [CFG['weights'][i]/sum(CFG['weights'])/CFG['tta']*inference_one_epoch(model, tst_loader, device)]\n\n    tst_preds = np.mean(tst_preds, axis=0) \n        \n    del model\n    torch.cuda.empty_cache()\n    \ntst_preds_3 = tst_preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ld-pytorch-tpu-resnext50-tcel-1-vldprv-train\n\nCFG = {\n    'fold_num': 6,\n    'n_class':5,\n    'seed': 42,\n    'model_arch': 'resnext50_32x4d',\n    'img_size': 512,\n    'epochs': 32,\n    'train_bs': 8,\n    'valid_bs': 8,\n    'lr': 1e-4,\n    'num_workers': 4,\n    'accum_iter': 1,\n    'verbose_step': 1,\n    'device': 'cuda:0',\n    'tta': 3,\n    'used_epochs': [3,4,6,10],\n    'weights': [1,1,1,1]\n}\n\nclass CustomResNext(nn.Module):\n    def __init__(self, model_name=CFG['model_arch'], pretrained=False):\n        super().__init__()\n        self.model = timm.create_model(CFG['model_arch'], pretrained=pretrained)\n        n_features = self.model.fc.in_features\n        self.model.fc = nn.Linear(n_features, CFG['n_class'])\n\n    def forward(self, x):\n        x = self.model(x)\n        return x\n\n\nseed_everything(CFG['seed'])\n\nfolds = StratifiedKFold(n_splits=CFG['fold_num'], shuffle=True, random_state=CFG['seed']).split(np.arange(train.shape[0]), train.label.values)\n\nfor fold, (trn_idx, val_idx) in enumerate(folds):\n    # we'll train fold 0 first\n    if fold > 0:\n        break \n\n    print('Inference fold {} started'.format(fold))\n    \n    # overwrite to point to prev data\n    val_idx = train[train[\"source\"]==2019].index\n    \n    test = pd.DataFrame()\n    test['image_id'] = list(os.listdir('../input/cassava-leaf-disease-classification/test_images/'))\n    test_ds = CassavaDataset(test, '../input/cassava-leaf-disease-classification/test_images/', transforms=get_inference_transforms(), output_label=False)\n    \n    tst_loader = torch.utils.data.DataLoader(\n        test_ds, \n        batch_size=CFG['valid_bs'],\n        num_workers=CFG['num_workers'],\n        shuffle=False,\n        pin_memory=False,\n    )\n\n    device = torch.device(CFG['device'])\n    model = CustomResNext(CFG['model_arch']).to(device)\n    \n    tst_preds = []\n    \n    for i, epoch in enumerate(CFG['used_epochs']):    \n        model.load_state_dict(torch.load('../input/ld-pytorch-tpu-resnext50-tcel-vldprv-train/{}_fold{}_{}'.format(CFG['model_arch'], fold, epoch)))\n        \n        with torch.no_grad():\n            for _ in range(CFG['tta']):\n                tst_preds += [CFG['weights'][i]/sum(CFG['weights'])/CFG['tta']*inference_one_epoch(model, tst_loader, device)]\n\n    tst_preds = np.mean(tst_preds, axis=0) \n    \n    del model\n    torch.cuda.empty_cache()\n    \ntst_preds_4 = tst_preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ld-pytorch-tpu-effnet-b4-gws-prtr-vldprv\n\nCFG = {\n    'fold_num': 6,\n    'n_class':5,\n    'seed': 42,\n    'model_arch': 'tf_efficientnet_b4_ns',\n    'img_size': 512,\n    'epochs': 32,\n    'train_bs': 8,\n    'valid_bs': 8,\n    'lr': 1e-4,\n    'num_workers': 4,\n    'accum_iter': 1,\n    'verbose_step': 1,\n    'device': 'cuda:0',\n    'tta': 3,\n    'used_epochs': [2,4,10,13],\n    'weights': [1,1,1,1]\n}\n\nclass CustomEfficientNet(nn.Module):\n    def __init__(self, model_name=CFG['model_arch'], pretrained=False):\n        super().__init__()\n        self.model = timm.create_model(CFG['model_arch'], pretrained=pretrained)\n        n_features = self.model.classifier.in_features\n        self.model.classifier = nn.Linear(n_features, CFG['n_class'])\n\n    def forward(self, x):\n        x = self.model(x)\n        return x\n\n\nseed_everything(CFG['seed'])\n\nfolds = StratifiedKFold(n_splits=CFG['fold_num'], shuffle=True, random_state=CFG['seed']).split(np.arange(train.shape[0]), train.label.values)\n\nfor fold, (trn_idx, val_idx) in enumerate(folds):\n    # we'll train fold 0 first\n    if fold > 0:\n        break \n\n    print('Inference fold {} started'.format(fold))\n    \n    test = pd.DataFrame()\n    test['image_id'] = list(os.listdir('../input/cassava-leaf-disease-classification/test_images/'))\n    test_ds = CassavaDataset(test, '../input/cassava-leaf-disease-classification/test_images/', transforms=get_inference_transforms(), output_label=False)\n    \n    tst_loader = torch.utils.data.DataLoader(\n        test_ds, \n        batch_size=CFG['valid_bs'],\n        num_workers=CFG['num_workers'],\n        shuffle=False,\n        pin_memory=False,\n    )\n\n    device = torch.device(CFG['device'])\n    model = CustomEfficientNet(CFG['model_arch']).to(device)\n    \n    tst_preds = []\n    \n    for i, epoch in enumerate(CFG['used_epochs']):    \n        model.load_state_dict(torch.load('../input/ld-pytorch-tpu-effnet-b4-gws-prtr-vldprv/{}_fold{}_{}'.format(CFG['model_arch'], fold, epoch)))\n        \n        with torch.no_grad():\n            for _ in range(CFG['tta']):\n                tst_preds += [CFG['weights'][i]/sum(CFG['weights'])/CFG['tta']*inference_one_epoch(model, tst_loader, device)]\n\n    tst_preds = np.mean(tst_preds, axis=0) \n    \n    del model\n    torch.cuda.empty_cache()\n    \ntst_preds_5 = tst_preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# blend - CV scores used to otimise the weights =============================================================\ntst_preds = (tst_preds_0*0.7 + tst_preds_1*0.3)*0.25 + tst_preds_2*0.25 + tst_preds_3*0.25 + tst_preds_4*0.1 + tst_preds_5*0.15","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test['label'] = np.argmax(tst_preds, axis=1)\ntest.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}