{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":14774,"databundleVersionId":875431,"sourceType":"competition"},{"sourceId":418031,"sourceType":"datasetVersion","datasetId":131128}],"dockerImageVersionId":27938,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Versions:\n* v3: first scoring kernel (LB 0.676)\n* v5: Longer epochs, export model, and optimize kappa (LB 0.705)\n* v8: Converted to regression problem, as required by the QWK optimizer(LB 0.722)\n* v11: Added TTA and seed","metadata":{}},{"cell_type":"markdown","source":"# Diabetic Retinopathy \n\nDiabetic retinopathy (DR), also known as diabetic eye disease, is a medical condition in which damage occurs to the retina due to diabetes mellitus. It is a leading cause of blindness. Diabetic retinopathy affects up to 80 percent of those who have had diabetes for 20 years or more. Diabetic retinopathy often has no early warning signs. **Retinal (fundus) photography with manual interpretation is a widely accepted screening tool for diabetic retinopathy**, with performance that can exceed that of in-person dilated eye examinations. \n\nThe below figure shows an example of a healthy patient and a patient with diabetic retinopathy as viewed by fundus photography ([source](https://www.biorxiv.org/content/biorxiv/early/2018/06/19/225508.full.pdf)):\n\n![image.png](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBxITEhUUExIVFRUXGBcXFhgYFxcYGRoaGRoXFhcYGRUYHiggGRolGxUWITElJSkrLi4uFx8zODMtNygtLisBCgoKDg0OGxAQGy8lICYtLTU1Ly0tLS0vMDUtLS0tLS0vLS0tLS0vLS8tLS0tLy8tLS01NS0tLS0tLS0tNS0tLf/AABEIAKUBMgMBIgACEQEDEQH/xAAcAAABBQEBAQAAAAAAAAAAAAAAAgMEBQYBBwj/xABFEAACAQIDBQUECAMFBwUAAAABAgMAEQQSIQUGMUFREyJhcYEykaGxByNCUnLB0fAUYoIzkrLh8RVDU1RzosIWNGODs//EABoBAQACAwEAAAAAAAAAAAAAAAACAwEEBQb/xAAwEQACAQMDAgMHBAMBAAAAAAAAAQIDBBESITEFQVGx8BMiYXGBkdEyocHhIzNCFP/aAAwDAQACEQMRAD8A9xooooAooooDM764nFp/C/wtyxn762urosUrmNjY5c2UKG5MVqh2PvRjcsAaJiWjzMHil7U5klkLl1sihCioVIuSeIuoPodNYnEogzOyqOpNqN4MpZ4PPBvVtURs5ghLKrPlEE/ey4eHFZQc/E9q0V7e0pNtMtStq7yYwzTQxoyqrw5XWJwwtiIEkUklg6skjG9l0BPjV9id78MpspaQ/wAq6e9rCoEm+33YD/UwHyBrXldUY8yNqFhcS3UH9dvMg7QxWOG0WKtKMOMRBHfigVogzL2PZ6qznL2gfusdRYGqxt7NpSRhzH2Qy4nMiwyZi4wxkjizXJV0cEZvtGwsCLG4bfx1YBsOMp0BEnPoQVqwh30i+3HIvlZvlrWFeUX/ANE5dOuYrOnyKqfenGpmvEMp7VUbsZWKmOaKJWk7wDB1lLX7tshOovbWbt4558LDNInZySRqzpYjKxGq2bUWPWjA7bgl0SVSfunun3GrGr4yUllPJpyhKDxJYZ2iiipEQooooAooooAooooArK764pkfDDtp4YmM3aNCpZriMmO9kb7XAW1Nh4VqqKA8zw29e0IzGssDmRnwwkDROVs0WD7bKykCMhppGIs3sNwsbdj3rx8auvZ9rZ3HaNDKOyHbTKgkBZRLdFSxUiwIJvpf0quM4AuTYDmaA87k3kx6yylo2OVS6RCOSyg4aJ7FgLygSM/RrowHQWO2Mfi5cJhmhYtJJiAjmNZMOHQdqCfrEdokORTcgjUWNiDV1i96cLHp2mc9EBb48PjVbLvsn2YWPmwHyvVErmlHmRtQsq891B+XmUmy95No/VQmMljC5Z3jbOJPr9LiysYjEiHu2fMGBW4BTBvNtARllQSN2AlLPDLlZkwySsqotirM5Zeet9OVWWI36kUXGHBHP6w3t19mpWH34QgFoXAP3SG/SoK8ov8A68yx9NuUs6f3X5KrFb54xO1ZoUUKyqA0ct4r4qHDJ2jXtKZI5GlXJa2S2t71xN7sczpGIdXSYFxBKBdVxBhnQs3sP2MfcIuO1AzcK00e3sHiAEdl4q2WVbd5WDqe9pcMoI6ECrxWB51sQnGazF5NSdOdN4mmvmea7N3m2giPK9pkvmOaGRCFjw0E0mQkgAH65QCPb5/Zrd7AxUk2HjlkAVpBnCi4srEmMG/2shW/jepWLwscqFJEV0bRlYAqeeoPGngKkQO0UUUAUUUUAUUUUAUUUUAUUUUAU3NKFBZiABqSTYDzNN43FLGhdzZVFyf3xNeX7wbblxbkXKwDgnC55FiOJ+A+Na1xcxorfnwN2zsp3MttkuX67l7tjfcsSmGAsNDK3M9EX8z7qzU0zyNmdi7dWN/d09KZWK2lrCnFFcKtczqv3ment7SlQXuL69xQ01pCknjoPnQq3PhypwVTqNjgR2CnxHmaZYsnVl5g8QPA8xUi3SuG/wC/0pqMpnTYjSrHZ28OJgIs3aIOKOT8H4qfeKpmw33WK9NdPceVLikPsvo3wI6j9KshVlB5iyupRhUjiSyviem7B3hhxQ7hyuPajawdfTmPEaVcCvGmVlIeNisi6qw0I/UVv90951xKhHsswFyOAcDiVHXqK7Nreqo9MufM85f9NdFe0p7x/df0aaiuXrtb5yQooooAooooArhNBNefb370O5MOGfKODuOJHMKeQ8eflVNavGlHMjZtrWdxPTD7+Bcbwb4pE3ZwgSy89e4n4iOJ8B8KxmP2nNMbyOza8Bog/p4fOoccAUd0Wrt7X/WuFXup1eePA9Ra2NKgtll+L5/oczcLWvcVztDmsD5n/L0pOUtyt1/Q1x1YW/L9ePrWvk3MIWFF7fmdeFcMRFyhI6qeHp0pMLkknnp5jTh8Kcs1tD+tYyMioZQwvax4EHkRUrBbSnh1ikI/lPeQ/wBH6WNGEkgVHzDvtz4VAVimjG68m5+TfrVqk44lF7lTjGpmLjt8e56BsDe+OZhHKBFKeAJ7j/gY8/A6+daYGvHJoQwsa026e9RQiDEve1lSQ+PAOenj766ltf6npqfc4d90rSnUo8d1+DfUVwGu10zhhRRRQBRRRQBRRRQBXCa7WZ362t2MORTZ5e6PBftH8vWoVKipxcn2LKNJ1ZqEe5lN79ufxEmRT9SjWH854Fj4ch69apl0PgfnTRcAeooYE25C/rb8q81VqOpJykeyoUo0oKEeCQ01tPd1pLO1uHxHypK6X+P+tKUc6oZsIcjm5HQ9DToamreF66E6G3nqKGNhw0CkC9tb+mtdAY8PjagBpADXHjVx8iOI8qcWO3j86bcDy8uNEE/ARFLY5WPe5H7w/WoJJR45FbKVY5SOTA6VJnXMLEnwJHyYVFxcfcK34d5W8RrrVsHuYnhrHievbtbZXExBtA47rr0P6HjVvXkm6G1TDOjm4jkCq48+Deh+BNetLXobWt7WG/KPH3tv7GphcPj8HaKKK2TTCiioW2cesELyt9kaDqToo9SRWG0llmYxcmkjMb9bdKj+HjNiR9Yw+yDwXzPy86w6aAEctT+dSYF7YuzyAMxLG/MnU00MM5UkWAA49fKvO3FSVWWp8dj11pShQhoXPf4nTMBz0NJMjch79P8AOmoxbre1tfy6CljXyrUZvoewaZiQzhONr8OvEHjQ4AJAIPXn6611Jbcr3o7IcRofC1Zclgj33IsuEs4mF8wUhgDoy8SCOZHEevWpq2axGt+FufSkqG56+X6UzC7RuAtwCTl0Fg3ErryOpHqOlZzq5IY0vK7kmU246eFN51Olrg8f8xUqVmaxa2YUxKBzFv3yqL2exKL23J8WFiEeQXJGubp4Xqh2pDfODyAJ8RzqX/ENlsrG3W1/86gxtqbnMDob6n/TWrtSeNiEYyjnLzk9B3C292iiB2u6i6MeLILaE8yt/dWyrwrZWKeJlZD34n7p6/5WJB9a9r2bjFmiSReDqCPDqPQ6V2rKvrjpfK8jzPU7ZUqmqPD8yVRRRW8c0KKKKAKKKKA4a8k322l2mLcC5Edox001b/uJ91esTyZVLHgAT7hevBllLkueLEsfU3PzrndRniCidbpNPM5Tfb+SVGL2vqb+7yp8jUeR/f76UzEKeYmuHI9GmdKUvMBpeo5auFredRSMvJKEgpyNhVcJPGpET1lowmTlFKNNI9KBqOSQvJ1oEfhRnqVglBOtZISlpWREeH0uRVftaIFSemvxGlaAgE25VU7bh4hTcAi46g+NTSK6VTMsMhxyi1rXH70r1PdrGGXDRsfatla/VdPjYH1ryuO4+ySeOugrcfR9iWImRgBlZWFjf2gR/wCNdDp82quPE5/V6SdLUuz/AKNhRRRXcPNhWB+k3aVuygF9frG9O6vxzH0rfV45vxie0x8vRMkY9FBP/czVp30sUseJ0Om09VfL7blYhuNeHT9amO5yBbmwtUKIeNSS1hxtyvXAkepXYWqXvx6+6nImC63193xpqxIPnw9CfdpSQ1r3sfOq+STeR0tc24+puSfKlo+oBHEfpxqEZCG1uDyA0/YrhxRWVoipy2DI/Xk6W8NDf+bwrOM8EHLDSLTXmD+/CuyxKy2N/TQ9QR4ggH0rkE1jc/DjQZCT4fvWoptMzzsJhkJBDWzLoRwv0YeB+Go5V2RDY90f3hTeIXUMASwHD7y8189LjxHiae9sKVIIup1GhHG9T2e5HLWw3FDcqOAIJ58efCkDDakE68j15MflVoAL2sGBs3Ai3RgDxHlXdoRgAAHUjjpobHQdKnjOyK41cyKDB6AW4i4Pv1NehbgY0skkZFgpDL5Nx+I+NeeICLAob6c9Lak69b1qNy8U64pFIAV1ZfaJPDMOPlW1Zz01l8SjqdNToy+G/wBj0miuCu16A8mFFFFAFFFFAV+8D2w05HKKT/Ca8Pwz8K9z23Fmw8y9Y5B71NeC4fgPKuX1FbxO50j9Mvmi1QinA96gpyqReuRJHcQ63GmJDenQ3Go+JQMcg8C34eFv6tR5XrEUJPBGwUjNdiuUFjkHMroAxPibkeFqsIr0gRG9OoLVlvJGLeB8NUhG0piJbU7a2o4dKqZZEcvXYnsa5SawSayiXisUVUsp15eZ0FU22oXOHl75zlGu/EjQ3sKeDFnFjoup6X5Vza09opPwN8QQKuhlSRRJYi0SI5VZbk3sLedqvvo2mJxE4J+wh18GP61h53KvppmF/UaGtv8ARcCZJ2I4JGAfMsfyrds1/mj67Gj1H/TL13PRaKKK7x5gK8N29LfF4gn/AI0nwYj8q9yrwzeWPLjcSP8A5WP97vD51odQXuL5nV6T/sl8v5OYfUgHhcVdbTwKRxAjVidOnX8qzkY08eXSpZnYqASbDhfhXG2SeUd9puSaeB0P3DwDHj4rzH76UxIelOxvccLgc/Dnf4VDjTKzZnJRzdbj2DzUniVY8OnDmKrjHktlLT2Bgb3v/pXZQbZhxU5vP7wHmt/W1L7M08iWotiDeR2Nja4N9Lj1qVCw14nyqBhBYstuFio/lb9DcegqcCfsnUfvWoTXZEovKFtwvb/WoN2VrFbKxup07rnUjwvqR436ipqsSNeV6bmiDKVPA8fLw6Hx5WpF45JOOV8RazcWW4tlUak8eP6+lRJMb9YgkJvLn7P7oKAadcxGvoaQXJ+r4NfW3Mff8Li/qCK5jswOYfZykW/l1+IzD1q6K3w/XgVPaOV68SVh5QWOb1vxU+XTXjUvYs1sdh7cO0t7waocZPe0g4X49VOg/I1a7m3bGQC17PfysrGrqK9+PzRRdv8Axy+T8j2QV2uCu16M8iFFFFAFFFFAJcXFeB4zDGKWSI/Ydl9x0+Fq99NeU/SZswx4kTD2Zl1/GgsfeuU+hrRvoZgpeB1OlVdNVxfdeRmYwalR2NRIn0p+9u9r0PlXEkj0S3HZ5Aouwvy08dALdb6UQwiw1GZu8x8fDwAsBUcuWa4tlU2Hi3An04ed6eYWN7kdeFvOotYWCSWdyUFFDGwNhf8Ad7DxplFPU0qJjwIvUTOk7s2ZnijZlszKGZehI1FTD5VAwbkRpofZHTp51J7Runx/SkuWZgvdWRwMbcKTMWsLaEn9mmJcUV0IuTwA/elP4drDMzAt8AKwkWDqRhQAPWq3ahzJJ0CsT520qZJIW0XhzP6VF2gAInAH2W+RqUX7yIyjiLGNpRnRhxX5c69D+jHDWw7ycpH7v4VAHzLVg5QWIAFySAo6k6AV7BsHZ4w+HihH2FAJ6n7R9Teun06DlNyfb+TkdYqKNJQ7t+RYUUUV2TzQGvIPpGwpTHM3KVEf1HcP+Ee+vX6xP0obMz4dZgNYm1/A2h9xyn31rXcNVJ/A3en1NFdZ77evqebxj1qRGeRqLC1SOPW/EVwJI9SsEmMkDTlw/Slo0ZBuAQQQRyueAI6XqOJLi4+R+VdeIgA3GouCOB8P8jVTRPCYQDIcp1+6eoHI/wAwHvGvI1JU2qKyZhqT100II4eRFEEjAlWsSLcNLjky+B6ciLdCZNZ3MaexIlAXK9yMp72n2T7VvKwP9NSwdDpqT16VFZ/A+tj6caRhZGtltqump5cVPu08wai1lGVHDJakjlpRc20FqYkmZRc2tXcPIXN2ICjgvP1rCRaAhIAfjJ8xzXw8PECkYhg1ipve3uHy41cWR4mK2zCqML2bdQ51PRzy8A3z/FVjWEimLy2yOIB2Zj+7oPLivwIHoa0n0YQF8Sz2/s0Ib8RNl+Gas9O9nB5MMp8xdl+Gb4V6T9HWy+ygaQ6NM2f+kDKn5n1rcsoudVP6+vqaXUpqnbyXd7evoawUUUV3jygUUUUAUUUUAVTb17HGKw7x/a9qM9HXh6HUHzq5oNYlFSWGShNwkpLlHz8LqxVgQwJUg8QRoQfWlSSnRV0Y8+g5t6fMit39I27PHFxDgCZ16qP94PEAa+Avyrz/AAhv3jxPAdByHnzPifCuDWoulLc9TbXEa0E4/X4E1O6LAadOlKje+h5UkGuGx5XrVaN1eA8j208dKdR9fdUPIOGvvpcYYcx63qLRPA/gW+rXyFcMxYlUPm3IeXjUDBSM6AXso004nU1OimCjKBcjkPz6VmSw2YhwhSYcXsLn7xPE+tKkjUA6VxTbnqeJomewB8dag+S5McMltKibRf6t/wALfKlO16VDgJJxIijRUZnbkqgHW/U8hU6cHKSSKq1SMINvY1O4WyDJN27DuRkhfF+v9IPvPhXo4qPs7CJFGqILKo0HzJ8TUmvR29FUoKJ468uXcVXPt2+QUUUVeaoU1iYFdWRhdWBUjqDoadooDwnbezWws7wtewPcP3kPsn3aHxBqfj9nBIkkU+1a4Neh767tjFxd2wmS5jbr1QnobDyOteQQyzd5Jrhkd1yk6qAbAEcjXFuKCpt7bdj0tpde2Ud91yTATy9fLjXVlsbEaHr+tWcCwNhzmv2o9kCqyNGYHu3txrSlDB0Yz1J9sCyQCenh86XIwIGgzAkq1z7iOangR+YFRgnmPWlIhGot43v+VV4LHHKwyVBMGvbQ/aHRuYvzHQ9DTbOBKnAZu6RxsD7JPgGt/eNQppJM3ctmGhtzHTz6Hr5mn4WSxvchhY/ePIi3Ig+4ipKKW5jDe2dy7xGwwiBzJmkJsOg8hUEwKOVMYfFOy99j3TYjx+95kEH1pXbXzAHW1YqYb2WCVFTx7zyLjksvhSZ+8pBFwRY+tNmS49KsdlY0KBGsRklZrKBre/y0/WkIZfJmrPTFtLJC2Vs2TEv2I1ZBnY9VU3B8CxGXzJr2fBMpRSnslQV8rC3wqi2ZsE4ePOtjPcu1tA1+MWv2bWt4gHrVhsKdSrIvBTdfwPdlFuVjmW3LJXoLSh7KG/LPJdQu/wD0VNuFx+S0oooraNAKKKKAKKKKAKKzu098sNBM0D586mAWCg37Ziq5ddbWu3QVKXeXCtos6X74sc2hQXbMLXAsQdbXvpQD2JPayCP7CWaXxOhSP/yPgFH2qx+925Ju02FW5Ny8XDXm0fifu+7pV8m8OEw4CPMD3ZJHksSt17IuWI4aTxkDgF8BWiVgRcG4OoNV1aUakcSLqFedGWqB8/5jcjgRoQRYg8wQeBpwSWr1/eDdXD4rvMuSTlIlg3ryYedeebb3JxsP9mgxC9UsGHiYyfkTXJq2c48bo9Db9RpVNpPD+P5KGSYDiQKYbGFhYcOF7anwApM0YiNpFIfo6kH3MKVBbieJrVccHQUsisECFAvYa+ftHif0qYrAW0qLhSMvHm3+I080ijiR4VGS3EJPCHsxodtKlbP2Ripz9VA5H3iMi/3m/K9bLY30fKLNiXzn/hron9TcW+Aq2la1KnCKK9/Sord7+C3Zkt39hzYsjs9EBs8hHdHgPvNbkPWvSpdlR4bBTRxjTs3JJ1ZjlOrHrVzBCqKFVQqjQACwA8AKi7f/APbT/wDSf/Ca7Fvaxpb8vxPO3l9O4eOI+H5JqcB5UqkpwHlSq2TRCiiigCiiigOGsXtndRcUryJZJxJKAx4MAxsr2+B4jxra1B2T7L/9WX/G1QnCM1pkWU6kqctUXueJ4yGSFzHKpRxxU9OoI0ZfEaUiLEMoNiRfQ+Ne27X2NBiUyTRhhyPBlPVWGqnyrzzbu4E8QZ8O4lUXOVyEcAa+17Letq5daylHeG6O/bdTpzWKmz/b18zJtLzvUrZW8CPHLFbIFFlkIvnOt8o5jhrzvoNKqcRgnXvYiN0U6gMpAbpqdMvz8uPYnDG+lhwHLztWsl7POVubzxUSw9iZsmcxuHtfW+vXwHIcal7ZdRKZVsQ1i4HuuOpAGvUeIqEtutKzAcxaq9WFgscU5as7jYezAg6Pp1F/snw0LD1WpSNYn41HhwcjjLFG7IxABCmyMSMvePdCliCLnQ+B02O7248kyh8RIIxchkTV8ymzKW4LqDwvU4286n6UV1L2lRypvBhX2oolaIggjUnllCGR29LDT+YV6HurtHAYVQzO7SMcjyGNgqNqxiv9khUZyde6ua9rVrsPuxg0XIMNERct3lDEkoYyxZtSShKk9DanBu7hP+Wi/szFqgPca4ZDfiCGYf1Hqa61C0hTxLueeu+oVK2Yr9Prkr8RvphFtcyHNF2q5Yycy8rAai471zYW1vTGA21hu0eeNpOzbs0b6s5DJK8YUBx9q8q3HDvsetriXd/CM2ZsPEWyhL5RfKBYAHlYaCkru1gwCBhoQGUIbIouotZf+1f7o6Cts55X4ffTDtL2feGaQRo1tGJUHUcVGY5fS/CkYXffDuFcLKUdkWMiNiTnjEqsUtcKUOa+tlBJtVnHu3g1ZXXDQhltlIRQRa1tfCwt0pM27GCcKGwsJCjKAY1sFCrHlt93IirbooFAVZ36w5yFVfI18zsLKmWQRMGtc3ubi2h61JO+OGDKjdqrFlUgxtdMzQqpf7oJxENvx+BtOXdzCDLbDRdy5XuDQsQWI8yoPpSotgYVQAMPHYajujSzI49zRxn+hegoCyorlqKAp8fuvhZpe1kjzSad7Mw4BRpY6aItMYTc7CR+wri9w31snfBAWz97viwGh8TzNaCigM3NuPgnUKyObZtTLJchxGrAnNqCsMY14ZdNa0MMQVQo4AADyAsKXRQBXK7WT3oxuNTEIIM3ZBUaSyBhrKEYkZCzgJclVZSOOtAaXEYRJBZ0Vx0ZQ3zqql3QwLccLF6Ll+VqzcG9W0JCt8KYlIk0COz5lfC2W5UqCFlmB4huyYi1rB8b240LEWwRDO8VwEmICSJEzXJXR0MjX0P9mdBraLinyicako/pbRP2TujgSrXwyG0ko1zHQOwHE1dYTYuGj/s4I18Qi399qzMu3Magwf1ZcywkyjsnB7UvCosVUiNgryNZrA5T5iLhN7scI1D4U5hHDncxTjIWEGaV0VNVYySWRLleyOa3eyFCK4Rl1ZvZyf3N+BUHbm0Rh8PLMVzdmpa17A26tyHU8hc1n9k7fxk2JhR8O0CFM0ilHJuYo3BMhUKozs6WvmvGbgcBrWUEWIuDxFSKzKjfERl0mjBdHKsYGV0sEhkZ7uVOgnW6gE6aA3FMYbfCPE5YHhdTPCrlQyXRHJjZmdmAOpSwW7d7hoa0P+wsNeMiCMdlm7MBAFUsVYkIBYNdFN7XFqe/2XB3fqYu62Zfq17ranMNNGuza/zHrQFHtTfWDDyPG0cpKMIwVCENJliYILtcG0yG5AHHWupvvAXVezmGcxLGxVQGklWJ1iAzZg4SUMbgABXN9Kvp8BE4IeJGDXzBkU3uADe410UD0FKGEj07i6EEd0aEDKCPELpfppQFdu9vDHjA7RK4CEC7i2YEXVlsToehsRzAq4pnD4VI82RFTMxZsqhbseLG3EnrT1AFFFFAFQdkew3/AFJf/wBGqdVZgJlSFmYhVEkpJ/8Asf4+FAWEsqqCzEAAXJJsAOpNVyxmchnBWLiqEWL9GkHJeYX1PQdihaUh5QVUEGOM9eTyfzdF4Lx48LICgEvECLEAjoRcfGqrEbr4JzdsLET+AA/C1XFcNYcU+SUZyj+l4M6N0Nnf8vHrw7zeWmvWpuG3dwiG6YaIH8APzrDbP+jnFwgdnio1ZUEcbAMezGeOZrAjX63tj5MvSrufdvGNFHaUqyJl7MYvE5WvJmZWxFu0N00z2uOQqKpwXZE3WqPZyf3ZrmgUqVKjKQQVtoQdCLdKr9mYWSKR19qJgGVr3NxZcrA6k5QuvPLrre+Xm2LjYi0mI2giRZcOjsZZI75JcOXJubRsyJOl1IzdoL2peB3d2ijF/wCM7UWhyAyyZWCFLq3dNtFfvi5btO8NL1MqN1ei9YfBbv46OSDPiJJMzr/EMJGKCKOOMgBWIs7TIdQDdZHueFNYPdrGt3Wx1xHKcwWeYvZpMJIyyOLd4wpiABYAduv4qA3t6L1jt39k46GQSPOuIRo4UN5XOoEatIlwFAADt9osWGo1pnau7GOlknIxV45G0jMsiKUIYAHIt48lwe6bPYhqA296L1g8TuljyzhcYUQrZAssigWgaNBlUdwJLkbunvC9xe1TYd38as+HYYm8UUkrMDLMWZXaQhGDXDizLa+oK6G1gANhRRRQBRRRQBRRRQBRRRQBXK7RQFPt3EYpGi/h41cO+SS/+7BsRKdRdVCuCOJLLwsaz2C2xtYtCZMMoDSlZFCNdVsl+8SBlUl+99oLprodzRQGAj2ttd1CnDhbiUFwjLche7lBJKWubFhqRbUam83axmMaWWPERkIqx9m+Ui5IswJJ7zaXJAtrx5DR0UBy1doooAooooAooooAooooCFtbaKwR52DNqqqqi7M7sERVGguWI1JAHEkCoE+9eGjQtMWiYRtI0Tqe0AXOSCq3BJEbkAE5gpIuBerLaWAjnQpILrdToSpDKQysrLqrBgCCOBFVUm6OEb2kc90qbyyHNcSLna7avaaQZjr3vAWAdw+9OEdiolAYMVswZTddeY4cgeZBA1FQMFtDChO3kxMZjWSUx6lUU2eck5uL9mS1/u8ONzUDFbJu0paY99xIT2+W8VizuvAIhfPmOgLE05iMfsox9m0cjxFhiGOSUi0EcZjkt7RRkiUAgENYg8TQGm/9R4TNl/iI81mPtadzPmu3AW7OQ2vwRjyNL2ftyGZ8kTZu5nuAQLZihBvqGBHAiqnZm7Wz5YYnjjYxdnkRWaTKVyyR3ZCdXyyyLmOve48Kttm7DhgYugbOQQzM7OzXOa7Fjqb8+gAoCzooooAooooCj3v2G2Mg7ESdn3lY3BIOW9gcrKws2VhZhqgBuCQYGI3Wlf8AhC2KJOHWzdzIrkFCGCIwCGyZSOGV2HA2rV0UBh03JnWNYxjWFi2Z8r5yGkhlGU9poV7IoCb2QgcjdqHcKRRGBiVXLOs75YmF8q4dCATISMywOG1N+2bpY72igPPcF9HciQvEcVcNHHGtlkUAIYzYgSar9WbDgO0fiCQdxszDtHFHGzZ2RFUta2YqAC1uV7XtUqigCiiigCiiigCiiigCiiigCiiigCiiigCiiigCiiigCiiigCiiigCiiigCiiigCiiigKKXdLBs4cxDQsxFzlYtl9oHiBlsBw1OmtPru3hALCBRqx0uPaXKQCDouU2A4DlaiigJ2AwMcKCOJAiC9gOpJZj4kkkknUkk1IoooAooooAooooAooooAooooAooooAooooAooooAooooD//2Q==)\n\nAn automated tool for grading severity of diabetic retinopathy would be very useful for accerelating detection and treatment. Recently, there have been a number of attempts to utilize deep learning to diagnose DR and automatically grade diabetic retinopathy. This includes a previous [competition](https://kaggle.com/c/diabetic-retinopathy-detection) and [work by Google](https://ai.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html). Even one deep-learning based system is [FDA approved](https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm604357.htm). \n\nClearly, this dataset and deep learning problem is quite important. \n\n# A look at the data:\n\nData description from the competition:\n\n>You are provided with a large set of high-resolution retina images taken under a variety of imaging conditions. A left and right field is provided for every subject. >Images are labeled with a subject id as well as either left or right (e.g. 1_left.jpeg is the left eye of patient id 1).\n>\n>A clinician has rated the presence of diabetic retinopathy in each image on a scale of 0 to 4, according to the following scale:\n>\n>0 - No DR\n>\n>1 - Mild\n>\n>2 - Moderate\n>\n>3 - Severe\n>\n>4 - Proliferative DR\n>\n>Your task is to create an automated analysis system capable of assigning a score based on this scale.\n\n...\n\n> Like any real-world data set, you will encounter noise in both the images and labels. Images may contain artifacts, be out of focus, underexposed, or overexposed. A major aim of this competition is to develop robust algorithms that can function in the presence of noise and variation.\n\n# Diabetic Retinopathy: Custom Lightweight Model\n\n* **Model:** `MyCustomDRModel` (inspired by paper 108548)\n* **Fusion:** Attention-based (Squeeze-and-Excitation)\n* **Training:** 2-GPU (DataParallel)\n","metadata":{}},{"cell_type":"code","source":"%reload_ext autoreload\n%autoreload 2\n%matplotlib inline","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:56.292590Z","iopub.execute_input":"2025-11-05T05:29:56.292855Z","iopub.status.idle":"2025-11-05T05:29:56.317062Z","shell.execute_reply.started":"2025-11-05T05:29:56.292816Z","shell.execute_reply":"2025-11-05T05:29:56.316526Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from fastai import *\nfrom fastai.vision import *\nimport pandas as pd\nimport matplotlib.pyplot as plt","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:56.318705Z","iopub.execute_input":"2025-11-05T05:29:56.318887Z","iopub.status.idle":"2025-11-05T05:29:58.089018Z","shell.execute_reply.started":"2025-11-05T05:29:56.318855Z","shell.execute_reply":"2025-11-05T05:29:58.088395Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Making pretrained weights work without needing to find the default filename\nif not os.path.exists('/tmp/.cache/torch/checkpoints/'):\n        os.makedirs('/tmp/.cache/torch/checkpoints/')\n!cp '../input/resnet50/resnet50.pth' '/tmp/.cache/torch/checkpoints/resnet50-19c8e357.pth'","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:58.090385Z","iopub.execute_input":"2025-11-05T05:29:58.090642Z","iopub.status.idle":"2025-11-05T05:29:59.112645Z","shell.execute_reply.started":"2025-11-05T05:29:58.090604Z","shell.execute_reply":"2025-11-05T05:29:59.111942Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nprint('Make sure cudnn is enabled:', torch.backends.cudnn.enabled)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.114537Z","iopub.execute_input":"2025-11-05T05:29:59.114859Z","iopub.status.idle":"2025-11-05T05:29:59.142449Z","shell.execute_reply.started":"2025-11-05T05:29:59.114800Z","shell.execute_reply":"2025-11-05T05:29:59.141767Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nos.listdir('../input')","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.146072Z","iopub.execute_input":"2025-11-05T05:29:59.146270Z","iopub.status.idle":"2025-11-05T05:29:59.172068Z","shell.execute_reply.started":"2025-11-05T05:29:59.146233Z","shell.execute_reply":"2025-11-05T05:29:59.171419Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print('Make sure cudnn is enabled:', torch.backends.cudnn.enabled)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.175182Z","iopub.execute_input":"2025-11-05T05:29:59.175404Z","iopub.status.idle":"2025-11-05T05:29:59.200201Z","shell.execute_reply.started":"2025-11-05T05:29:59.175366Z","shell.execute_reply":"2025-11-05T05:29:59.199540Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n\nSEED = 999\nseed_everything(SEED)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.201402Z","iopub.execute_input":"2025-11-05T05:29:59.201697Z","iopub.status.idle":"2025-11-05T05:29:59.228653Z","shell.execute_reply.started":"2025-11-05T05:29:59.201640Z","shell.execute_reply":"2025-11-05T05:29:59.227926Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Reading data and Basic EDA\n\nHere I am going to open the dataset with pandas, check distribution of labels.","metadata":{}},{"cell_type":"code","source":"base_image_dir = os.path.join('..', 'input/aptos2019-blindness-detection/')\ntrain_dir = os.path.join(base_image_dir,'train_images/')\ndf = pd.read_csv(os.path.join(base_image_dir, 'train.csv'))\ndf['path'] = df['id_code'].map(lambda x: os.path.join(train_dir,'{}.png'.format(x)))\ndf = df.drop(columns=['id_code'])\ndf = df.sample(frac=1).reset_index(drop=True) #shuffle dataframe\ndf.head(10)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.230053Z","iopub.execute_input":"2025-11-05T05:29:59.230341Z","iopub.status.idle":"2025-11-05T05:29:59.378195Z","shell.execute_reply.started":"2025-11-05T05:29:59.230294Z","shell.execute_reply":"2025-11-05T05:29:59.377529Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"len_df = len(df)\nprint(f\"There are {len_df} images\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.379706Z","iopub.execute_input":"2025-11-05T05:29:59.380012Z","iopub.status.idle":"2025-11-05T05:29:59.406254Z","shell.execute_reply.started":"2025-11-05T05:29:59.379949Z","shell.execute_reply":"2025-11-05T05:29:59.405618Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This is actually very small. The [previous competition](https://kaggle.com/c/diabetic-retinopathy-detection) had ~35k images, which supports the idea that pretraining on that dataset may be quite beneficial.","metadata":{}},{"cell_type":"markdown","source":"The dataset is highly imbalanced, with many samples for level 0, and very little for the rest of the levels.","metadata":{}},{"cell_type":"code","source":"df['diagnosis'].hist(figsize = (10, 5))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.407537Z","iopub.execute_input":"2025-11-05T05:29:59.407761Z","iopub.status.idle":"2025-11-05T05:29:59.825548Z","shell.execute_reply.started":"2025-11-05T05:29:59.407724Z","shell.execute_reply":"2025-11-05T05:29:59.824375Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Let's look at an example image:","metadata":{}},{"cell_type":"code","source":"from PIL import Image\n\nim = Image.open(df['path'][1])\nwidth, height = im.size\nprint(width,height) \nim.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:29:59.827239Z","iopub.execute_input":"2025-11-05T05:29:59.827804Z","iopub.status.idle":"2025-11-05T05:30:00.465320Z","shell.execute_reply.started":"2025-11-05T05:29:59.827745Z","shell.execute_reply":"2025-11-05T05:30:00.464348Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.imshow(np.asarray(im))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:30:00.466979Z","iopub.execute_input":"2025-11-05T05:30:00.467257Z","iopub.status.idle":"2025-11-05T05:30:01.246565Z","shell.execute_reply.started":"2025-11-05T05:30:00.467202Z","shell.execute_reply":"2025-11-05T05:30:01.245579Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"The images are actually quite big. We will resize to a much smaller size.","metadata":{}},{"cell_type":"code","source":"bs = 128 #smaller batch size is better for training, but may take longer\nsz=224","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:30:01.248585Z","iopub.execute_input":"2025-11-05T05:30:01.248967Z","iopub.status.idle":"2025-11-05T05:30:01.289854Z","shell.execute_reply.started":"2025-11-05T05:30:01.248909Z","shell.execute_reply":"2025-11-05T05:30:01.289036Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"tfms = get_transforms(do_flip=True,flip_vert=True,max_rotate=360,max_warp=0,max_zoom=1.1,max_lighting=0.1,p_lighting=0.5)\nsrc = (ImageList.from_df(df=df,path='./',cols='path') #get dataset from dataset\n        .split_by_rand_pct(0.2) #Splitting the dataset\n        .label_from_df(cols='diagnosis',label_cls=FloatList) #obtain labels from the level column\n      )\ndata= (src.transform(tfms,size=sz,resize_method=ResizeMethod.SQUISH,padding_mode='zeros') #Data augmentation\n        .databunch(bs=bs,num_workers=4) #DataBunch\n        .normalize(imagenet_stats) #Normalize     \n       )","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:30:01.291019Z","iopub.execute_input":"2025-11-05T05:30:01.291246Z","iopub.status.idle":"2025-11-05T05:30:30.182682Z","shell.execute_reply.started":"2025-11-05T05:30:01.291200Z","shell.execute_reply":"2025-11-05T05:30:30.182061Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"data.show_batch(rows=3, figsize=(7,6))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:30:30.183887Z","iopub.execute_input":"2025-11-05T05:30:30.184089Z","iopub.status.idle":"2025-11-05T05:31:58.532888Z","shell.execute_reply.started":"2025-11-05T05:30:30.184055Z","shell.execute_reply":"2025-11-05T05:31:58.532085Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Training (Transfer learning)","metadata":{}},{"cell_type":"markdown","source":"The Kaggle competition used the Cohen's quadratically weighted kappa so I have that here to compare. This is a better metric when dealing with imbalanced datasets like this one, and for measuring inter-rater agreement for categorical classification (the raters being the human-labeled dataset and the neural network predictions). Here is an implementation based on the scikit-learn's implementation, but converted to a pytorch tensor, as that is what fastai uses.","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import cohen_kappa_score\n\ndef quadratic_kappa(y_hat, y):\n    \"Calculates the Quadratic Weighted Kappa metric for fastai\"\n    \n    # Detach, move to CPU, and convert to numpy\n    y_hat_preds = torch.round(y_hat).cpu().numpy()\n    y_true = y.cpu().numpy()\n    \n    # Calculate sklearn's cohen_kappa_score\n    score = cohen_kappa_score(y_true, y_hat_preds, weights='quadratic')\n    \n    # Return as a tensor (fastai expects this)\n    return torch.tensor(score)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:31:58.534784Z","iopub.execute_input":"2025-11-05T05:31:58.535090Z","iopub.status.idle":"2025-11-05T05:31:58.837678Z","shell.execute_reply.started":"2025-11-05T05:31:58.535042Z","shell.execute_reply":"2025-11-05T05:31:58.836622Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Training:**\n\nWe use transfer learning, where we retrain the last layers of a pretrained neural network. I use the ResNet50 architecture trained on the ImageNet dataset, which has been commonly used for pre-training applications in computer vision. Fastai makes it quite simple to create a model and train:","metadata":{}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n#--- 1. The \"Better\" Fusion Block (Squeeze-and-Excitation) ---\nclass SEBlock(nn.Module):\n    def __init__(self, c, r=16):\n        super().__init__()\n        self.squeeze = nn.AdaptiveAvgPool2d(1)\n        self.excitation = nn.Sequential(\n            nn.Linear(c, c // r, bias=False),\n            nn.ReLU(inplace=True),\n            nn.Linear(c // r, c, bias=False),\n            nn.Sigmoid()\n        )\n\n    def forward(self, x):\n        bs, c, _, _ = x.shape\n        y = self.squeeze(x).view(bs, c)\n        y = self.excitation(y).view(bs, c, 1, 1)\n        return x * y.expand_as(x)\n\n#--- 2. The Multi-Scale Block (Our version of MSFE) ---\nclass AttentionMSFEBlock(nn.Module):\n    def __init__(self, c_in, f_out, f_fused):\n        # c_in: input channels\n        # f_out: filters for parallel 3x3 and 5x5 convs\n        # f_fused: filters for the final fusion conv\n        super().__init__()\n        \n        # Branch 1: 3x3 convolution (for fine features)\n        self.conv1 = nn.Sequential(\n            nn.Conv2d(c_in, f_out, kernel_size=3, padding=1),\n            nn.BatchNorm2d(f_out),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2)\n        )\n        \n        # Branch 2: 5x5 convolution (for coarse features)\n        self.conv2 = nn.Sequential(\n            nn.Conv2d(c_in, f_out, kernel_size=5, padding=2),\n            nn.BatchNorm2d(f_out),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2)\n        )\n        \n        c_fused = f_out * 2 \n        \n        # Our \"Better\" Fusion:\n        self.fusion = nn.Sequential(\n            SEBlock(c_fused), # Attention-based fusion\n            nn.Conv2d(c_fused, f_fused, kernel_size=3, padding=1),\n            nn.BatchNorm2d(f_fused),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2)\n        )\n\n    def forward(self, x):\n        x1 = self.conv1(x)\n        x2 = self.conv2(x)\n        x_cat = torch.cat([x1, x2], dim=1)\n        x_fused = self.fusion(x_cat)\n        return x_fused\n\n#--- 3. The Final Custom Model ---\nclass MyCustomDRModel(nn.Module):\n    def __init__(self, n_out=1):\n        super().__init__()\n        # Input 224x224\n        self.stem = nn.Sequential(\n            nn.Conv2d(3, 16, kernel_size=3, padding=1),\n            nn.BatchNorm2d(16),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2)\n        ) # Output 112x112\n        \n        # Block 1 (f=32, f^=64)\n        self.msfe1 = AttentionMSFEBlock(c_in=16, f_out=32, f_fused=64)\n        # Output 28x28\n        \n        # Block 2 (f=128, f^=256)\n        self.msfe2 = AttentionMSFEBlock(c_in=64, f_out=128, f_fused=256)\n        # Output 7x7\n        \n        # Final Head\n        self.head = nn.Sequential(\n            nn.AdaptiveAvgPool2d(1), # Global Avg Pooling\n            Flatten(),\n            nn.Dropout(0.5),\n            nn.Linear(256, n_out)\n        )\n\n    def forward(self, x):\n        x = self.stem(x)\n        x = self.msfe1(x)\n        x = self.msfe2(x)\n        x = self.head(x)\n        return x","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:31:58.839281Z","iopub.execute_input":"2025-11-05T05:31:58.839577Z","iopub.status.idle":"2025-11-05T05:31:58.888111Z","shell.execute_reply.started":"2025-11-05T05:31:58.839533Z","shell.execute_reply":"2025-11-05T05:31:58.887400Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 1. Create an instance of our new model ---\nmy_model = MyCustomDRModel(n_out=1)\n\n# --- 2. Wrap the model for 2 GPUs *first* ---\nparallel_model = torch.nn.DataParallel(my_model)\n\n# --- 3. Create the fastai Learner ---\nlearn_custom = Learner(data, \n                         parallel_model, # <-- Pass the parallel_model here\n                         metrics=[quadratic_kappa],\n                         loss_func=MSELossFlat())\n\n# --- 4. Check the parameters ---\ntotal_params = sum(p.numel() for p in my_model.parameters())\ntrainable_params = sum(p.numel() for p in my_model.parameters() if p.requires_grad)\n\nprint(f\"--- My Custom Model Parameters ---\")\nprint(f\"Total parameters:     {total_params:,}\")\nprint(f\"Trainable parameters: {trainable_params:,}\")\nprint(f\"(Paper's model had 2.7M parameters)\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:31:58.889714Z","iopub.execute_input":"2025-11-05T05:31:58.890080Z","iopub.status.idle":"2025-11-05T05:31:58.939960Z","shell.execute_reply.started":"2025-11-05T05:31:58.890013Z","shell.execute_reply":"2025-11-05T05:31:58.939103Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"learn_custom.lr_find()\nlearn_custom.recorder.plot(suggestion=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:31:58.941448Z","iopub.execute_input":"2025-11-05T05:31:58.941753Z","iopub.status.idle":"2025-11-05T05:50:45.159311Z","shell.execute_reply.started":"2025-11-05T05:31:58.941696Z","shell.execute_reply":"2025-11-05T05:50:45.158618Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Here we can see that the loss decreases fastest around `lr=1e-2` so that is what we will use to train:","metadata":{}},{"cell_type":"code","source":"# ❗️ CHOOSE A LEARNING RATE FROM THE PLOT ABOVE\nlearn_custom.fit_one_cycle(8, max_lr = 1e-2)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T05:50:45.160632Z","iopub.execute_input":"2025-11-05T05:50:45.160833Z","iopub.status.idle":"2025-11-05T06:30:41.673586Z","shell.execute_reply.started":"2025-11-05T05:50:45.160798Z","shell.execute_reply":"2025-11-05T06:30:41.672543Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"learn_custom.recorder.plot_losses()\nlearn_custom.recorder.plot_metrics()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T06:30:41.675449Z","iopub.execute_input":"2025-11-05T06:30:41.675769Z","iopub.status.idle":"2025-11-05T06:30:42.304949Z","shell.execute_reply.started":"2025-11-05T06:30:41.675712Z","shell.execute_reply":"2025-11-05T06:30:42.303925Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"learn_custom.save('stage-1-custom')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T06:30:42.306630Z","iopub.execute_input":"2025-11-05T06:30:42.307211Z","iopub.status.idle":"2025-11-05T06:30:42.388150Z","shell.execute_reply.started":"2025-11-05T06:30:42.307139Z","shell.execute_reply":"2025-11-05T06:30:42.387465Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Now we find a new, lower learning rate for fine-tuning\nlearn_custom.lr_find()\nlearn_custom.recorder.plot(suggestion=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T06:30:42.389471Z","iopub.execute_input":"2025-11-05T06:30:42.389721Z","iopub.status.idle":"2025-11-05T06:48:27.884345Z","shell.execute_reply.started":"2025-11-05T06:30:42.389672Z","shell.execute_reply":"2025-11-05T06:48:27.883586Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# ❗️ CHOOSE A NEW, LOWER LEARNING RATE\nlearn_custom.fit_one_cycle(6, max_lr=1e-5)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T06:48:27.885627Z","iopub.execute_input":"2025-11-05T06:48:27.886102Z","iopub.status.idle":"2025-11-05T07:19:24.121777Z","shell.execute_reply.started":"2025-11-05T06:48:27.886051Z","shell.execute_reply":"2025-11-05T07:19:24.120916Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"learn_custom.recorder.plot_losses()\nlearn_custom.recorder.plot_metrics()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:19:24.123424Z","iopub.execute_input":"2025-11-05T07:19:24.123713Z","iopub.status.idle":"2025-11-05T07:19:24.699475Z","shell.execute_reply.started":"2025-11-05T07:19:24.123662Z","shell.execute_reply":"2025-11-05T07:19:24.698412Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"learn_custom.export('diabetic_retinopathy_model.pkl')\nlearn_custom.save('stage-2-custom')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:19:24.701202Z","iopub.execute_input":"2025-11-05T07:19:24.701795Z","iopub.status.idle":"2025-11-05T07:19:24.832261Z","shell.execute_reply.started":"2025-11-05T07:19:24.701733Z","shell.execute_reply":"2025-11-05T07:19:24.831406Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Let's evaluate our model:","metadata":{}},{"cell_type":"markdown","source":"## Optimize the Metric\n\nOptimizing the quadratic kappa metric was an important part of the top solutions in the previous competition. Thankfully, @abhishek has already provided code to do this for us. We will use this to improve the score.","metadata":{}},{"cell_type":"code","source":"valid_preds, y_true = learn_custom.get_preds(ds_type=DatasetType.Valid)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:19:24.833618Z","iopub.execute_input":"2025-11-05T07:19:24.834048Z","iopub.status.idle":"2025-11-05T07:20:32.264494Z","shell.execute_reply.started":"2025-11-05T07:19:24.833979Z","shell.execute_reply":"2025-11-05T07:20:32.263812Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport scipy as sp\nfrom functools import partial\nfrom sklearn import metrics\nfrom collections import Counter\nimport json","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:32.266361Z","iopub.execute_input":"2025-11-05T07:20:32.266704Z","iopub.status.idle":"2025-11-05T07:20:32.297281Z","shell.execute_reply.started":"2025-11-05T07:20:32.266645Z","shell.execute_reply":"2025-11-05T07:20:32.296733Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class OptimizedRounder(object):\n    def __init__(self):\n        self.coef_ = 0\n\n    def _kappa_loss(self, coef, X, y):\n        X_p = np.copy(X)\n        for i, pred in enumerate(X_p):\n            if pred < coef[0]:\n                X_p[i] = 0\n            elif pred >= coef[0] and pred < coef[1]:\n                X_p[i] = 1\n            elif pred >= coef[1] and pred < coef[2]:\n                X_p[i] = 2\n            elif pred >= coef[2] and pred < coef[3]:\n                X_p[i] = 3\n            else:\n                X_p[i] = 4\n\n        ll = metrics.cohen_kappa_score(y, X_p, weights='quadratic')\n        return -ll\n\n    def fit(self, X, y):\n        loss_partial = partial(self._kappa_loss, X=X, y=y)\n        initial_coef = [0.5, 1.5, 2.5, 3.5]\n        self.coef_ = sp.optimize.minimize(loss_partial, initial_coef, method='nelder-mead')\n        print(-loss_partial(self.coef_['x']))\n\n    def predict(self, X, coef):\n        X_p = np.copy(X)\n        for i, pred in enumerate(X_p):\n            if pred < coef[0]:\n                X_p[i] = 0\n            elif pred >= coef[0] and pred < coef[1]:\n                X_p[i] = 1\n            elif pred >= coef[1] and pred < coef[2]:\n                X_p[i] = 2\n            elif pred >= coef[2] and pred < coef[3]:\n                X_p[i] = 3\n            else:\n                X_p[i] = 4\n        return X_p\n\n    def coefficients(self):\n        return self.coef_['x']","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:32.298459Z","iopub.execute_input":"2025-11-05T07:20:32.298667Z","iopub.status.idle":"2025-11-05T07:20:32.332429Z","shell.execute_reply.started":"2025-11-05T07:20:32.298631Z","shell.execute_reply":"2025-11-05T07:20:32.331643Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"optR = OptimizedRounder()\noptR.fit(valid_preds, y_true)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:32.333686Z","iopub.execute_input":"2025-11-05T07:20:32.333989Z","iopub.status.idle":"2025-11-05T07:20:32.920169Z","shell.execute_reply.started":"2025-11-05T07:20:32.333933Z","shell.execute_reply":"2025-11-05T07:20:32.919330Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"coefficients = optR.coefficients()\nprint(coefficients)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:32.921578Z","iopub.execute_input":"2025-11-05T07:20:32.921896Z","iopub.status.idle":"2025-11-05T07:20:32.949678Z","shell.execute_reply.started":"2025-11-05T07:20:32.921836Z","shell.execute_reply":"2025-11-05T07:20:32.948855Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from sklearn.metrics import confusion_matrix\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n# --- Step 1: Apply the Optimized Rounder ---\nprint(\"Applying optimized coefficients to validation predictions...\")\ny_pred = optR.predict(valid_preds, coefficients)\n\n# --- Step 2: Calculate and Print the Quadratic Kappa Score ---\n# We use the y_true and y_pred from the validation set\nkappa_score = cohen_kappa_score(y_true.numpy(), y_pred, weights='quadratic')\nprint(f\"\\n--- Final Validation Accuracy ---\")\nprint(f\"Quadratic Kappa Score: {kappa_score:.6f}\")\n\n# --- Step 3: Plot the Confusion Matrix ---\nprint(\"\\nPlotting Confusion Matrix...\")\ncm = confusion_matrix(y_true.numpy(), y_pred)\nclass_names = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative']\n\nfig, ax = plt.subplots(figsize=(10, 8))\nsns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=class_names, yticklabels=class_names, ax=ax)\nplt.ylabel('Actual')\nplt.xlabel('Predicted')\nplt.title('Confusion Matrix (Validation Set)')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:32.954617Z","iopub.execute_input":"2025-11-05T07:20:32.955054Z","iopub.status.idle":"2025-11-05T07:20:33.556902Z","shell.execute_reply.started":"2025-11-05T07:20:32.954783Z","shell.execute_reply":"2025-11-05T07:20:33.555868Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## TTA\n\nTest-time augmentation, or TTA, is a commonly-used technique to provide a boost in your score, and is very simple to implement. Fastai already has TTA implemented, but it is not the best for all purposes, so I am redefining the fastai function and using my custom version.","metadata":{}},{"cell_type":"code","source":"from fastai.core import *\nfrom fastai.basic_data import *\nfrom fastai.basic_train import *\nfrom fastai.torch_core import *\n\ndef _tta_only(learn:Learner, ds_type:DatasetType=DatasetType.Valid, num_pred:int=10) -> Iterator[List[Tensor]]:\n    \"Computes the outputs for several augmented inputs for TTA\"\n    dl = learn.dl(ds_type)\n    ds = dl.dataset\n    old = ds.tfms\n    aug_tfms = [o for o in learn.data.train_ds.tfms]\n    try:\n        pbar = master_bar(range(num_pred))\n        for i in pbar:\n            ds.tfms = aug_tfms\n            yield get_preds(learn.model, dl, pbar=pbar)[0]\n    finally: ds.tfms = old\n\nLearner.tta_only = _tta_only\n\ndef _TTA(learn:Learner, beta:float=0, ds_type:DatasetType=DatasetType.Valid, num_pred:int=10, with_loss:bool=False) -> Tensors:\n    \"Applies TTA to predict on `ds_type` dataset.\"\n    preds,y = learn.get_preds(ds_type)\n    all_preds = list(learn.tta_only(ds_type=ds_type, num_pred=num_pred))\n    avg_preds = torch.stack(all_preds).mean(0)\n    if beta is None: return preds,avg_preds,y\n    else:            \n        final_preds = preds*beta + avg_preds*(1-beta)\n        if with_loss: \n            with NoneReduceOnCPU(learn.loss_func) as lf: loss = lf(final_preds, y)\n            return final_preds, y, loss\n        return final_preds, y\n\nLearner.TTA = _TTA","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:33.558602Z","iopub.execute_input":"2025-11-05T07:20:33.558983Z","iopub.status.idle":"2025-11-05T07:20:33.633781Z","shell.execute_reply.started":"2025-11-05T07:20:33.558912Z","shell.execute_reply":"2025-11-05T07:20:33.632564Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Submission\nLet's now create a submission","metadata":{}},{"cell_type":"code","source":"sample_df = pd.read_csv('../input/aptos2019-blindness-detection/sample_submission.csv')\nsample_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:33.635288Z","iopub.execute_input":"2025-11-05T07:20:33.635536Z","iopub.status.idle":"2025-11-05T07:20:33.691190Z","shell.execute_reply.started":"2025-11-05T07:20:33.635486Z","shell.execute_reply":"2025-11-05T07:20:33.690419Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"learn_custom.data.add_test(ImageList.from_df(sample_df,'../input/aptos2019-blindness-detection',folder='test_images',suffix='.png'))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:33.692359Z","iopub.execute_input":"2025-11-05T07:20:33.692689Z","iopub.status.idle":"2025-11-05T07:20:33.766903Z","shell.execute_reply.started":"2025-11-05T07:20:33.692630Z","shell.execute_reply":"2025-11-05T07:20:33.766317Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"preds,y = learn_custom.TTA(ds_type=DatasetType.Test)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:20:33.768247Z","iopub.execute_input":"2025-11-05T07:20:33.768557Z","iopub.status.idle":"2025-11-05T07:31:41.650278Z","shell.execute_reply.started":"2025-11-05T07:20:33.768502Z","shell.execute_reply":"2025-11-05T07:31:41.649554Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_predictions = optR.predict(preds, coefficients)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:41.651964Z","iopub.execute_input":"2025-11-05T07:31:41.652387Z","iopub.status.idle":"2025-11-05T07:31:41.699417Z","shell.execute_reply.started":"2025-11-05T07:31:41.652214Z","shell.execute_reply":"2025-11-05T07:31:41.698596Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sample_df.diagnosis = test_predictions.astype(int)\nsample_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:41.700448Z","iopub.execute_input":"2025-11-05T07:31:41.700702Z","iopub.status.idle":"2025-11-05T07:31:41.738541Z","shell.execute_reply.started":"2025-11-05T07:31:41.700655Z","shell.execute_reply":"2025-11-05T07:31:41.737846Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sample_df.to_csv('submission.csv',index=False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:41.739852Z","iopub.execute_input":"2025-11-05T07:31:41.740291Z","iopub.status.idle":"2025-11-05T07:31:42.036485Z","shell.execute_reply.started":"2025-11-05T07:31:41.740109Z","shell.execute_reply":"2025-11-05T07:31:42.035881Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# .h5 is not a native format, but we can save the state_dict\n# This saves the weights from the *parallel model*\ntorch.save(learn_custom.model.state_dict(), 'model_weights.h5')\nprint(\"Model weights saved as 'model_weights.h5'\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:42.037729Z","iopub.execute_input":"2025-11-05T07:31:42.037964Z","iopub.status.idle":"2025-11-05T07:31:42.075136Z","shell.execute_reply.started":"2025-11-05T07:31:42.037918Z","shell.execute_reply":"2025-11-05T07:31:42.074352Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from fastai.vision import *\nfrom sklearn.metrics import cohen_kappa_score\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom functools import partial\nimport scipy as sp\n\n# --- Step 0: RE-DEFINE ALL CUSTOM OBJECTS ---\n# This is *required* for `load_learner` to find these classes.\n\ndef quadratic_kappa(y_hat, y):\n    \"Calculates the Quadratic Weighted Kappa metric for fastai\"\n    y_hat_preds = torch.round(y_hat).cpu().numpy()\n    y_true = y.cpu().numpy()\n    score = cohen_kappa_score(y_true, y_hat_preds, weights='quadratic')\n    return torch.tensor(score)\n\nclass SEBlock(nn.Module):\n    def __init__(self, c, r=16):\n        super().__init__()\n        self.squeeze = nn.AdaptiveAvgPool2d(1)\n        self.excitation = nn.Sequential(\n            nn.Linear(c, c // r, bias=False), nn.ReLU(inplace=True),\n            nn.Linear(c // r, c, bias=False), nn.Sigmoid())\n    def forward(self, x):\n        bs, c, _, _ = x.shape\n        y = self.squeeze(x).view(bs, c)\n        y = self.excitation(y).view(bs, c, 1, 1)\n        return x * y.expand_as(x)\n\nclass AttentionMSFEBlock(nn.Module):\n    def __init__(self, c_in, f_out, f_fused):\n        super().__init__()\n        self.conv1 = nn.Sequential(\n            nn.Conv2d(c_in, f_out, kernel_size=3, padding=1), nn.BatchNorm2d(f_out),\n            nn.ReLU(inplace=True), nn.MaxPool2d(2))\n        self.conv2 = nn.Sequential(\n            nn.Conv2d(c_in, f_out, kernel_size=5, padding=2), nn.BatchNorm2d(f_out),\n            nn.ReLU(inplace=True), nn.MaxPool2d(2))\n        c_fused = f_out * 2 \n        self.fusion = nn.Sequential(\n            SEBlock(c_fused),\n            nn.Conv2d(c_fused, f_fused, kernel_size=3, padding=1),\n            nn.BatchNorm2d(f_fused), nn.ReLU(inplace=True), nn.MaxPool2d(2))\n    def forward(self, x):\n        x1 = self.conv1(x); x2 = self.conv2(x)\n        x_cat = torch.cat([x1, x2], dim=1); x_fused = self.fusion(x_cat)\n        return x_fused\n\nclass MyCustomDRModel(nn.Module):\n    def __init__(self, n_out=1):\n        super().__init__()\n        self.stem = nn.Sequential(\n            nn.Conv2d(3, 16, kernel_size=3, padding=1), nn.BatchNorm2d(16),\n            nn.ReLU(inplace=True), nn.MaxPool2d(2))\n        self.msfe1 = AttentionMSFEBlock(c_in=16, f_out=32, f_fused=64)\n        self.msfe2 = AttentionMSFEBlock(c_in=64, f_out=128, f_fused=256)\n        self.head = nn.Sequential(\n            nn.AdaptiveAvgPool2d(1), Flatten(), nn.Dropout(0.5),\n            nn.Linear(256, n_out))\n    def forward(self, x):\n        x = self.stem(x); x = self.msfe1(x); x = self.msfe2(x); x = self.head(x)\n        return x\n\nclass OptimizedRounder(object):\n    def __init__(self): self.coef_ = 0\n    def _kappa_loss(self, coef, X, y):\n        X_p = np.copy(X); X_p[X_p < coef[0]] = 0; X_p[(X_p >= coef[0]) & (X_p < coef[1])] = 1\n        X_p[(X_p >= coef[1]) & (X_p < coef[2])] = 2; X_p[(X_p >= coef[2]) & (X_p < coef[3])] = 3\n        X_p[X_p >= coef[3]] = 4; return -cohen_kappa_score(y, X_p, weights='quadratic')\n    def fit(self, X, y):\n        loss_partial = partial(self._kappa_loss, X=X, y=y); initial_coef = [0.5, 1.5, 2.5, 3.5]\n        self.coef_ = sp.optimize.minimize(loss_partial, initial_coef, method='nelder-mead')['x']\n    def predict(self, X, coef):\n        X_p = np.copy(X); X_p[X_p < coef[0]] = 0; X_p[(X_p >= coef[0]) & (X_p < coef[1])] = 1\n        X_p[(X_p >= coef[1]) & (X_p < coef[2])] = 2; X_p[(X_p >= coef[2]) & (X_p < coef[3])] = 3\n        X_p[X_p >= coef[3]] = 4; return X_p.astype(int)\n    def coefficients(self): return self.coef_\n# --- END OF REQUIRED DEFINITIONS ---\n\n\n# --- Step 1: Load the saved learner object ---\nprint(\"Loading learner from .pkl...\")\ninfer_learn = load_learner(path='.', file='diabetic_retinopathy_model.pkl')\nprint(\"Learner loaded.\")\n\n# --- Step 2: Define your OptimizedRounder logic ---\n# ❗️PASTE YOUR NEW COEFFICIENTS HERE (from cell 34)\ncoefficients = [0.575233, 1.551249, 2.130471, 3.557537]\noptR = OptimizedRounder()\nclass_names = ['No DR', 'Mild', 'Moderate', 'Severe', 'Proliferative']\n\n# --- Step 3: Get an image path and make a prediction ---\nimage_path = '/kaggle/input/aptos2019-blindness-detection/train_images/00b74780d31d.png'\nimg = open_image(image_path)\n\npred_class, pred_idx, outputs = infer_learn.predict(img)\nraw_prediction = outputs.item()\nfinal_prediction = optR.predict([raw_prediction], coefficients)[0]\n\n# --- Step 4: Display the results ---\nimg.show(title=f'Prediction: {final_prediction}')\nprint(f\"--- PREDICTION USING LOADED .pkl LEARNER ---\")\nprint(f\"Raw Model Prediction: \\t{raw_prediction:.4f}\")\nprint(f\"Final Predicted Class: \\t{final_prediction} ({class_names[final_prediction]})\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:42.076180Z","iopub.execute_input":"2025-11-05T07:31:42.076382Z","iopub.status.idle":"2025-11-05T07:31:43.001015Z","shell.execute_reply.started":"2025-11-05T07:31:42.076347Z","shell.execute_reply":"2025-11-05T07:31:43.000114Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import time\n\n# --- Warm up the GPU ---\nprint(\"\\nWarming up GPU for speed test...\")\n_ = infer_learn.predict(img)\n\n# --- Run the speed test ---\nnum_predictions = 100\nstart_time = time.time()\n\nfor _ in range(num_predictions):\n    _ = infer_learn.predict(img)\n\nend_time = time.time()\n\n# --- Calculate and Print Results ---\ntotal_time = end_time - start_time\navg_time_per_image = (total_time / num_predictions) * 1000 # in milliseconds\n\nprint(f\"\\n--- Inference Speed Test (on GPU) ---\")\nprint(f\"Total time for {num_predictions} predictions: {total_time:.4f} seconds\")\nprint(f\"Average time per image: {avg_time_per_image:.2f} milliseconds\")\nprint(f\"(Paper's inference time (on Raspberry Pi): 30,000 milliseconds)\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:43.002477Z","iopub.execute_input":"2025-11-05T07:31:43.002887Z","iopub.status.idle":"2025-11-05T07:31:50.688331Z","shell.execute_reply.started":"2025-11-05T07:31:43.002716Z","shell.execute_reply":"2025-11-05T07:31:50.687561Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from fastai.vision import *\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom functools import partial\nimport scipy as sp\n\n# --- Step 1: RE-DEFINE THE MODEL ARCHITECTURE ---\n# This is *required* to create a model to load weights into.\nclass SEBlock(nn.Module):\n    def __init__(self, c, r=16):\n        super().__init__()\n        self.squeeze = nn.AdaptiveAvgPool2d(1)\n        self.excitation = nn.Sequential(\n            nn.Linear(c, c // r, bias=False), nn.ReLU(inplace=True),\n            nn.Linear(c // r, c, bias=False), nn.Sigmoid())\n    def forward(self, x):\n        bs, c, _, _ = x.shape\n        y = self.squeeze(x).view(bs, c)\n        y = self.excitation(y).view(bs, c, 1, 1)\n        return x * y.expand_as(x)\n\nclass AttentionMSFEBlock(nn.Module):\n    def __init__(self, c_in, f_out, f_fused):\n        super().__init__()\n        self.conv1 = nn.Sequential(\n            nn.Conv2d(c_in, f_out, kernel_size=3, padding=1), nn.BatchNorm2d(f_out),\n            nn.ReLU(inplace=True), nn.MaxPool2d(2))\n        self.conv2 = nn.Sequential(\n            nn.Conv2d(c_in, f_out, kernel_size=5, padding=2), nn.BatchNorm2d(f_out),\n            nn.ReLU(inplace=True), nn.MaxPool2d(2))\n        c_fused = f_out * 2 \n        self.fusion = nn.Sequential(\n            SEBlock(c_fused),\n            nn.Conv2d(c_fused, f_fused, kernel_size=3, padding=1),\n            nn.BatchNorm2d(f_fused), nn.ReLU(inplace=True), nn.MaxPool2d(2))\n    def forward(self, x):\n        x1 = self.conv1(x); x2 = self.conv2(x)\n        x_cat = torch.cat([x1, x2], dim=1); x_fused = self.fusion(x_cat)\n        return x_fused\n\nclass MyCustomDRModel(nn.Module):\n    def __init__(self, n_out=1):\n        super().__init__()\n        self.stem = nn.Sequential(\n            nn.Conv2d(3, 16, kernel_size=3, padding=1), nn.BatchNorm2d(16),\n            nn.ReLU(inplace=True), nn.MaxPool2d(2))\n        self.msfe1 = AttentionMSFEBlock(c_in=16, f_out=32, f_fused=64)\n        self.msfe2 = AttentionMSFEBlock(c_in=64, f_out=128, f_fused=256)\n        self.head = nn.Sequential(\n            nn.AdaptiveAvgPool2d(1), Flatten(), nn.Dropout(0.5),\n            nn.Linear(256, n_out))\n    def forward(self, x):\n        x = self.stem(x); x = self.msfe1(x); x = self.msfe2(x); x = self.head(x)\n        return x\n# --- END OF MODEL DEFINITIONS ---\n\n\n# --- Step 2: Create a dummy DataBunch ---\n# We need this to create a Learner object\nempty_df = pd.DataFrame(columns=['path', 'diagnosis'])\nrecreated_data = (ImageList.from_df(df=empty_df, path='.')\n                  .split_none()\n                  .label_from_df(cols='diagnosis', label_cls=FloatList)\n                  .transform(get_transforms(), size=224)\n                  .databunch()\n                  .normalize(imagenet_stats)\n                 )\n\n# --- Step 3: Create a Learner with our custom model ---\nrecreated_model = MyCustomDRModel(n_out=1)\nrecreated_learn = Learner(recreated_data, recreated_model)\n\n# --- Step 4: Load your saved weights ---\n# Note: We must strip the \"module.\" prefix that .to_parallel() adds\nprint(\"Loading weights from .h5...\")\nstate_dict = torch.load('model_weights.h5')\nfrom collections import OrderedDict\nnew_state_dict = OrderedDict()\nfor k, v in state_dict.items():\n    name = k[7:] if k.startswith('module.') else k # strip `module.`\n    new_state_dict[name] = v\n\nrecreated_learn.model.load_state_dict(new_state_dict)\nprint(\"Weights loaded.\")\n\n# --- Step 5: Define your OptimizedRounder logic ---\nclass OptimizedRounder(object):\n    def __init__(self): self.coef_ = 0\n    def _kappa_loss(self, coef, X, y):\n        X_p = np.copy(X); X_p[X_p < coef[0]] = 0; X_p[(X_p >= coef[0]) & (X_p < coef[1])] = 1\n        X_p[(X_p >= coef[1]) & (X_p < coef[2])] = 2; X_p[(X_p >= coef[2]) & (X_p < coef[3])] = 3\n        X_p[X_p >= coef[3]] = 4; return -cohen_kappa_score(y, X_p, weights='quadratic')\n    def fit(self, X, y):\n        loss_partial = partial(self._kappa_loss, X=X, y=y); initial_coef = [0.5, 1.5, 2.5, 3.5]\n        self.coef_ = sp.optimize.minimize(loss_partial, initial_coef, method='nelder-mead')['x']\n    def predict(self, X, coef):\n        X_p = np.copy(X); X_p[X_p < coef[0]] = 0; X_p[(X_p >= coef[0]) & (X_p < coef[1])] = 1\n        X_p[(X_p >= coef[1]) & (X_p < coef[2])] = 2; X_p[(X_p >= coef[2]) & (X_p < coef[3])] = 3\n        X_p[X_p >= coef[3]] = 4; return X_p.astype(int)\n    def coefficients(self): return self.coef_\n\n# ❗️PASTE YOUR NEW COEFFICIENTS HERE (from cell 34)\ncoefficients = [0.5, 1.5, 2.5, 3.5] \noptR = OptimizedRounder()\nclass_names = ['No DR', 'Mild', 'Moderate', 'Severe', 'Proliferative']\n\n# --- Step 6: Run inference on an image ---\nimage_path = '/kaggle/input/aptos2019-blindness-detection/train_images/00b74780d31d.png'\nimg = open_image(image_path)\n\npred_class, pred_idx, outputs = recreated_learn.predict(img)\nraw_prediction = outputs.item()\nfinal_prediction = optR.predict([raw_prediction], coefficients)[0]\n\n# --- Step 7: Display the results ---\nimg.show(figsize=(6,6))\nprint(f\"--- PREDICTION USING LOADED .h5 WEIGHTS ---\")\nprint(f\"Raw Model Prediction: \\t{raw_prediction:.4f}\")\nprint(f\"Final Predicted Class: \\t{final_prediction} ({class_names[final_prediction]})\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:50.689879Z","iopub.execute_input":"2025-11-05T07:31:50.690198Z","iopub.status.idle":"2025-11-05T07:31:51.638846Z","shell.execute_reply.started":"2025-11-05T07:31:50.690131Z","shell.execute_reply":"2025-11-05T07:31:51.637990Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- Check Parameters from Saved .pkl Model ---\n# This cell assumes 'infer_learn' is already loaded.\n\ntotal_params = sum(p.numel() for p in infer_learn.model.parameters())\ntrainable_params = sum(p.numel() for p in infer_learn.model.parameters() if p.requires_grad)\n\nprint(f\"--- Parameters from infer_learn (loaded from .pkl) ---\")\nprint(f\"Total parameters:     {total_params:,}\")\nprint(f\"Trainable parameters: {trainable_params:,}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-05T07:31:51.640064Z","iopub.execute_input":"2025-11-05T07:31:51.640280Z","iopub.status.idle":"2025-11-05T07:31:51.673725Z","shell.execute_reply.started":"2025-11-05T07:31:51.640244Z","shell.execute_reply":"2025-11-05T07:31:51.672522Z"}},"outputs":[],"execution_count":null}]}