{"cells":[{"metadata":{"papermill":{"duration":0.035562,"end_time":"2020-12-18T20:11:48.255998","exception":false,"start_time":"2020-12-18T20:11:48.220436","status":"completed"},"tags":[]},"cell_type":"markdown","source":"In this kernel I would use tf.data plus Keras to build a baseline, this type of baseline can be helpful to you in solving similar problems as well."},{"metadata":{"papermill":{"duration":0.033988,"end_time":"2020-12-18T20:11:48.324137","exception":false,"start_time":"2020-12-18T20:11:48.290149","status":"completed"},"tags":[]},"cell_type":"markdown","source":"![dia%20%281%29.png](attachment:dia%20%281%29.png)","attachments":{"dia%20%281%29.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAmMAAABvCAYAAACglccvAAAE23RFWHRteGZpbGUAJTNDbXhmaWxlJTIwaG9zdCUzRCUyMmFwcC5kaWFncmFtcy5uZXQlMjIlMjBtb2RpZmllZCUzRCUyMjIwMjAtMTItMTFUMTklM0E1NSUzQTE0LjAxNVolMjIlMjBhZ2VudCUzRCUyMjUuMCUyMChYMTElM0IlMjBMaW51eCUyMHg4Nl82NCklMjBBcHBsZVdlYktpdCUyRjUzNy4zNiUyMChLSFRNTCUyQyUyMGxpa2UlMjBHZWNrbyklMjBDaHJvbWUlMkY4Ny4wLjQyODAuNjYlMjBTYWZhcmklMkY1MzcuMzYlMjIlMjBldGFnJTNEJTIyVWFSbEhGZkxDNUkxMXN0YkNTVk8lMjIlMjB2ZXJzaW9uJTNEJTIyMTQuMC4xJTIyJTIwdHlwZSUzRCUyMmRldmljZSUyMiUzRSUzQ2RpYWdyYW0lMjBpZCUzRCUyMlUwMkMzcTl0ZUdRNEVVTVlzVDZkJTIyJTIwbmFtZSUzRCUyMlBhZ2UtMSUyMiUzRTVaYmJqdE13RUlhZnBwZXMwamc5WGJMZHNpQXRyRVNGS25HRDNIaVNlT3RrZ2p2cGdhZkhicHltSm0wRjBsWUNjUlg3OXppZSUyQlQlMkZiU1k5Tjg5Mmo1bVgyRVFXb1hoaUlYWTg5OU1Ld3p3YUJlVmhsWHl0anhtb2gxVks0b0ZhWXl4JTJGZ1JEY3ZyYVNBdFJkSWlJcGs2WXN4RmdYRTVHbGNhOXo2WVFrcWY5V1NwOUFSNWpGWFhYVWhCV1d1aXFZc3E3OEhtV2JOeXYzQWplUzhDWGJDT3VNQ3R5Y1NtJTJGWFlWQ05TM2NwM1UxRFd2TWFYZXQ2N0M2UEh4RFFVOURzVFBuJTJCSjhSdDclMkJickF4ZVMlMkJsSiUyQiUyQlI2UG96Y0RsUnZ1bVlCQ21mdGRGVFJtbVdIQTFhOVY3alZVaHdMNDFNTDAyNWdteE5HTGZpQzlBdEhjd2VVVm9wSXh5NVVick5lMUNGMHR4MGhvckhjT1YlMkZFTzNKYmhPZ2E3RVJVZkR6VTRGeklIMDNzelRvRGpKalo4SGQxc21QY2ExcnBxR00lMkZZUFRIWkpicmlxM0VvZmNydnR3dUNCRXplUFJ5aEFjMExkb1dFMlRXbWJjYlcwMW04elNUQXYlMkJjR1RyVGx4dnJITG1zelQ4aWp3ZUpVZWVEMVhwR1FCVGhkY3I1N05heVRaWXhuY0JRTmZEQSUyQnFqVXl3SUFleVA3WjlxZFFVbFVuVjVzY0VoM0VTRzMxTkdsZHdNaktNeDdCTWpyUTNvQWwyMTNsMyUyQlRRVG1rTzFiNjRWMTklMkIyUnpKc3RNdyUyRmpqZGlHbldZVW5JbkRqaiUyRk1ZUkpBc1A0TEVJeG1peUQ0SFVRUnVPJTJGRHVHd2czQ3c2OUF6RlpOUHlMZXBRRXZFODlSSlhNbTBzTnlOUTJEMGUlMkJ1Zk5GJTJCV3QyNGdsMEtvUzF2Q3YyUlBFVWF2QTRTTmZnRXk3QUtKenZBSWI0VmpkT2FXTERWdUlMY1YlMkZiOWNKamZqWXJydDc4ZGg3T1FuanMxJTJCQWclM0QlM0QlM0MlMkZkaWFncmFtJTNFJTNDJTJGbXhmaWxlJTNFBeL15QAAIABJREFUeF7tnQd0VcUWhn+kFwGlIy0EIXSQ3qQEDIQmVboUqdJCDSAQpEnvoD6Q3nvvRelIEQhdepcivWPe2oP3SsINnJtzbzjn3H/WestncmfO3t8eyOfMnEmUwHGbQ+49fomQkBCwkQAJRIyAV4r48C/sjaxeiSI2AHuRQCQQODIlFUKeXUbIP5HwMD6CBCxKIE7SQkiWtwfipSnvsgyjtBi0JiRa9Jj4okB6pE4W32UDcyAS8CQCF67+jTW7TuPbarkpZJ5UeJPl+seEKIgVHUiUfRBifVzIZNEzXBIwBoGHN3bixqGu8PJf4TIhi9Js4MqQ0vm98dvBqxQyY9SZUZiUAIXMpIXzoLD3jY2CJNn74NGp3hQyD6o7U3U9AVcLmZKxBuWy48qtRxQy19eLI3oYAQqZhxXcZOmKjKUouhov7u6hkJmsdgzXeARcKWR2GZM0KWTGKzYjMh8BCpn5auYpEdtkTPKlkHlK1ZmnOwm4SshCyRiFzJ0l49ieRIBC5knVNk+ur8sYhcw8dWOkxibgCiF7Q8YoZMYuOqMzDwEKmXlq5SmRhpUxCpmnVJ55upuAXcjKr0C81M6/ZelQxihk7i4bx/cUAhQyT6m0OfJ0JGMUMnPUjlEan4AeIQtXxihkxi88IzQHAQqZOerkCVGGJ2MUMk+oPnOMDAIRFbK3yhiFLDJKx2d4AgEKmSdU2fg5vk3GKGTGrx8jNAeBiAjZO2WMQmaO4jNK4xOgkBm/RlaP8F0yRiGz+gxgfpFF4OFfO3HjcFd4aTxDpknGKGSRVT4+x+oEKGRWr7Cx89MiYxQyY9eQ0ZmHgDNCplnGKGTmmQCM1NgEKGTGro+Vo9MqYxQyK88C5haZBLQKmVMyRiGLzBLyWVYmQCGzcnWNm5szMkYhM24dGZm5CGgRMqdljEJmrknAaI1LgEJm3NpYNTJnZYxCZtWZwLwim8C7hCxCMkYhi+wy8nlWJUAhs2pljZlXRGSMQmbMWjIq8xEQIbsZHIh0/svfuBg2wjJGITPfRGDExiRAITNmXawYVURljEJmxdnAnN4HAbVCFhwIrzBCpkvGKGTvo5R8phUJUMisWFXj5aRHxihkxqsnIzInAUdCplvGKGTmnAyM2ngEKGTGq4nVItIrYxQyq80I5vO+CIQVMpfIGIXsfZWTz7UaAQqZ1SpqrHxcIWMUMmPVlNGYl4BNyNL7L4fLZMwmZPtP3cSDR8/NS4eRk4ABCCSIGx39mhUxQCQMwUoEXCVjNiF7cmEC/nl6zUqImAsJRDqBD2Kndq2MSQZzNp1G2zrFIj0ZPpAErERg9KytGBNQwkopMRcDEHCljEk69/aWg0/5vQbIjCGQgHkJHF+ZlzJm3vIxcisToIxZubrvLzfK2PtjzyeTQHgEKGOcGyRgUAKUMYMWxuRhUcZMXkCGb0kClDFLlpVJWYEAZcwKVTReDpQx49WEEZEAZYxzgAQMSoAyZtDCmDwsypjJC8jwLUmAMmbJsjIpKxCgjFmhisbLgTJmvJowIhKgjHEOkIBBCVDGDFoYk4dFGTN5ARm+JQlQxixZViZlBQKUMStU0Xg5UMaMVxNGRAKUMc4BEjAoAcqYQQtj8rAoYyYvIMO3JAHKmCXLyqSsQIAyZoUqGi8HypjxasKISIAyxjlAAgYlQBkzaGFMHhZlzOQFZPiWJEAZs2RZmZQVCFDGrFBF4+VAGTNeTRgRCVDGOAdIwKAEKGMGLYzJw6KMmbyADN+SBChjliwrk7ICAcqYFapovBwoY8arCSMiAcoY5wAJGJQAZcyghTF5WJQxkxeQ4VuSAGXMkmVlUlYgYEUZmzx5Mho3bvxGeU6dOoUMGTJYoWyGz4EyZvgSMUAPJEAZ88CiM2VzELCijI0YMQLz58/HvHnzQhUhRYoUiBo1qjkKY/IoKWMmLyDDtyQBypgly8qkrEDAijLWu3dvHD58GIsWLXqjRI8fP0b27NnRpk0btGvXTn2/X79+mDVrFv744w/EiBHDCmV97zlQxt57CRgACbxBgDLGSUECBiVgRRlr27Yt9u3bh3jx4iE4OBjp06dHUFAQfH19VRU2b96MqlWr4sSJE3j69CmyZs2KNWvWoHDhwgatkvnCooyZr2aM2PoEKGPWrzEzNCkBK8pYQEAANm3aBFkh+/TTTzF79mwMGTIEBw4cQLZs2VSlmjVrhufPn+PRo0dIliwZRo8ebdIKGjNsypgx68KoPJsAZcyz68/sDUzAijLmCHeRIkWQP39+yHkyaXfv3kWWLFnwwQcf4NixY2oVjc11BChjrmPJkUjAVQQoY64iyXFIwMUEPEXGmjRpggcPHmDu3LmK4JkzZ5A3b178888/asXMy8vLxWQ9ezjKmGfXn9kbkwBlzJh1YVQkACvKWN++fZVolStXzl7hQoUKoXjx4vjhhx/U10qXLq3OiMmB/oMHD2LdunWcDS4kQBlzIUwORQIuIkAZcxFIDkMCriZgRRnr0qWLutZixowZSJcuHaZOnQoRtCNHjsDb2xtyD5m8QSn//uLFC/j4+Kh/b9iwoavxeux4lDGPLT0TNzABypiBi8PQPJuAFWVMDuYHBgZizpw5uHfvnrrKYtCgQShWrBiuX7+OzJkzK1Hz9/dXxZc7yVq0aIGjR4+qw/xs+glQxvQz5Agk4GoClDFXE+V4JOAiAlaUMReh4TA6CFDGdMBjVxJwEwHKmJvAclgS0EuAMqaXIPs7IkAZ47wgAeMRoIwZryaMiAQUAcoYJ4I7CFDG3EGVY5KAPgKUMX382JsE3EaAMuY2tB49MGXMo8vP5A1KgDJm0MIwLBKgjHEOuIMAZcwdVDkmCegjQBnTx4+9ScBtBChjbkNr6YGHDRumfqXUhx9+6DBPypily8/kTEqAMmbSwjFs6xOgjFm/xu7IMHbs2Hj58iU6duyI7t27vyFllDF3UOeYJKCPAGVMHz/2JgG3EaCMuQ2tpQceNWqUustNfp1UlChRIL+c/XUpo4xZuvxMzqQEKGMmLRzDtj4Bypj1a+yuDBMnToxbt26p4WPEiBFKyk5OjY8URVe77NH39paDT/m9LhuPA5GAJxKgjHli1ZmzKQiIjI3tUNIUsTJI4xOIHj06KlWqhG4lFlLGjF8uRuhhBChjHlZwpmseAlwZM0+tjBYpV8aMVhHGQwJvJ0AZ4wwhAYMSoIwZtDAGD0vOjHXr1k0d4ueZMYMXi+GRwL8EDCtjfkWyqhDXbj9iqWK9fPECmVPFCpVTjBgxkThJMuTInQ/V6zTC56XKGipnOQi8etl8LJ43DcEH9+PevTv46KNESOPljYpVaqNm3SaIFj26oWK2QjCUMStUMfJz4NuUkc/cHU98/vwF6rXpidWbtuODDz7A9UPrEDNGDM2PKlihIV68eIm9a6Zr7sMPvj8ClLFIZm+Tsey58qJW/abq6U+ePMalC+ewYc0yXDx/BuUqVsfQ8dMQPbr2P3i2NL6pUx6FipZCk1YdXZLZ06dP0KZJTWzZsAoZM2eDr18lJEqcBLdv3cSmdctx/MghFCxSAj/NWIbYseO45JnvcxBX89OTC2VMDz3P7ct7xsxX+937g+FbsyUe/LnVHvzi1ZtRtUlntGhQDf6+RVDet6iSMq0tojLmKBatz+TnIk6AMhZxdhHqaZMx/8o1MfKnWaHGkG2FEQN74uexg1G3USv0HjjaqWfIClY+nyRo1b67y2SsT2AbzJwyAd92+A5tO/dW2x62FhISguEDvsNPYwaheZuu6Nijv1PxuvvDwhMhIYgaLZqmR7mSn7PPdhQgZUxT2fghJwnwagsngUXCx4f/NBO9hvwYSsZ+mr4ILboOwKGNc5A9cwano4iojDmKxemHs4PTBEwjY9t/3YBGX5XFtIUbsGrpPPW/Fy9eoHCxUhg4ahL27tqGEQO/w/mzp5E6bXoEBg0Otd137eoljPyhN3b8thF/376JpMlTooz/l2jTsSfixvvvpurgQ/vRv2cAgv/Yh/gJEqJG3SYoWaY8avgXxrhfFqg+0h49fIBRQ/pgzbIFuHHjGhIlTgpfv4oI6NYXCRJ8FG4h3iZjtk6yOiP5rt95HKnSeKkvvyv+fbu3o3bl4qGeO2/lduTKU+CdfcML9vrVyyiRzxt58hXGjMWbHH5MBGbi+KFqNU5W+2xNCx9bTWct3YIt61dh8fzpeHDvLrw/9UFgn6HIX+hzp8b7deNqNK1bEdMWrMeEUQPx+66tmLdiG7LlzPNOBm/jd+fvWxjxQy9sWrcCt27+pbZoi5X0Q8fu/ZAkWQoV49ue7fSfyn87UMYiSo793kaAMmas+VG6Zits3LbHHpRfiUJ48vQpft25P1Sgj8/uQKyYb+6WyN/B3w2agMlzl+HO3ftK3Ib0bI/AAWMgW52vb1NOm78SY36Zi5NnziPqB1GRzccbfTo1R8kir/7udhTLmllj1Pfe1ddYVM0XjWlkbNf2LWhQrTSyZM+F6rUbKUESserRsTmKlfwCz54+Rbc+wxA9Rgx0bdsIp08dx45Dl5RoySpF+eI5cffObbTrEoSUqdLi8B+/Y+ywvqhQpRaGjJ2qKieS9kWhzGolpdN3A5A8RSpM/d9oPH70EHt2/oafpi9Vz5Xx6lf1xbEjB9VqUdYcn+HU8WCMHBSE1Gm9MH/l9nBXY7TImPzQb9HgS7UyJitkWuK3xSgy0qh5e3xZoz7SeX8KOY+mJXdHU3fhnCno1v4bxady9bqaZ7dWPraaZs2eG5/7lkOVmvXx9MkTdG3XSEn11gPnEe/D+Jp5y3xoWNMPBYuWRMpP0qBI8dIoWqIM4if46J0MwuMXLVo0VCtbUMXTvmsfZMmeG3+ePKpWBBMlSYZlG/chVqzYai46evZHHyfWzC3sByljEUbHjm8hQBkz1vQ4fe4SAnoPx4atu7Fj+WTEjxcX/4T8gylzl6P/qF+w4H+D4Z0uFXJm+TTUzoQti74jJqpVtSa1K6N2FT9cvHwdA8dMxtNnz5H444R2GZu5aDXqte6J2l/6oU6Vsnjy9BkGj5+KQ0f/xP51M5AlY3o4iiV92k+gpa+xqJovGtPI2O4dvyoBEimwyZPg9i2QEZcvnsOGXSfsq0hL5k1Hl7aNMHPJZuQrWEydwwoKbI3K1euhUrU69iq1a1pLndM6dO4+okaNqsSrf88OmDR7lRI8aSJPlXw/w6kTR+0ytnbFIrT5piaGT5ihZM7WZLWuffM6GDNxHvwqVHU4G7TI2NUrF1H8My/Ub9IaPfuP1By/cCiZLwO69hpk36bUmrujYIf26662TFdsOYiMPq9eqNDStPKx1bR0ucoYP3mhfeilC2aic+uvMWXeWhT+3BfOjlegcHFMX7TRPp5WBo74LZk/A13aNMTQcdNCzZ11KxejdZMaGDDif+o/Dmy5hH22Fl7hfYYypoce+4ZHgDJmvLnRsH0QFqzYGGqb8sdpC9EycCAOb5qrVrAcNVkVS57TDyJMu1ZMsX9k/+HjyONXD3lyZLbLmIjd1t0HsGrGKPvZs6MnzyBriZr4vnML9Az4RvV3FIvWvsYja56ITCdj3w8ej1oNmtkJ16r0Oa5cvIDfDpyzf822SjFh6mK1dRhek62nCSMHYNsfF9S2ZadvG2Dl0nk4fPZ+qLcDRUhETGwrYz07t8T8mZOw+9h1tSpia08eP0LBrClQq34z9P7h1dJu2KZFxmQFL59PUvVDXn7Ya43fkUxo7evoc317tMf0SWOxee9pfJIqrf0jsvJ17crFN7okS/6J4qaVj01ghFXdhi3t4+3athkNqpfByJ9nw79SDafHC+w9GI1bdnjnn8Kw9XfET1YGZYXwwJ9/h9rOlpcucnrFR5WaDfDDqEl2GdP67HcGB4AypoUSP+MsAcqYs8Tc//mIytiZ85fhXagyurVphAHdvg0V6Ce5yyFFssRvfZtS3raMnqYAmtatgp+H9AhXxhwRcNTX/aSs+wTTydjoiXNRtkI1e0XqVSmlrlpYtvG//XXbD3lZbZFVF2nrVy1RYnHy+BE8fHAfcvj85csXagtMRE62JOVM2omjh7Hj8OVQFbetgthkTLYC5YxQeK102UoYP2VRhGXs7OkTkKs9Xj8UryX+8GRMS19HwY4b3g+jBgdhyfq9anvY1m78dQ1FcqR6o8uabcFIn8FHndvSwsdWp7A1tX19xI8zUf7Lr5web9j46ahYtXao+LQwcMSvca1yOLh/D/adfPWrZV5vIsw5cudVK6m2mB09O6J/fVDGIkqO/d5GgDJmvPkRURnbsfcQilRqjBF9OqB90/92fSTDvGXrq0RtZ8Zu3r6DoGE/Y92WXbh24yaePXuhvv/02TO1xTlxWM9wZUxrX+ORNU9EHiFjO7duUud5ChUrhcYtOiB5ylSIFi0qpk8aj5mTx9tlTD5z8ljwGzK2YfVStGpUzb4y1qxeJXVGKLxD7fETJFBS4qhpWRmTVTc5C2c7q6U1fkcyobWvo1hXLZuP9s1qo8+gcaj9dXP7R549e4rfNq6x//v23zYqjjYZ08pHq4zpHU8rA0f8mtT2x54dv+Hw+QdvylimJMiZJz8mzlppl7GwYqnnrwJXy9jQoUMxceJEHD9+XE9YHtd31apVyJAhAzJmzGiJ3CljxitjRGVs++8HUbRyE4cylrtMHXX8xiZjxas2w96DR9GvaysUzZ8LH8Z7dRVR5s+rv1PGtPY1HlnzROQRMhbYrglWLJ6D34//hdhx4tqr06tLK8yZ9rNdxuS8l6yg2M6Q2T44afwwDPq+q13Genf9FrOn/oQ9x64j4UeJnKr2u2RMvl+1bAGcOXUCO4OvqAPsWuN3JBNa+zpKQt6IlG1Xudx16fq94b6UYJNHm4xp5aNVxvSOp5WBI349OjTD/Fm/vFFrOfCfyzuh2kruP/xnythrE0jecpYXH6zSihYtisDAQFSoUMESKVHGjFfGiMrYyTMXkKloVQS2boiB3VvbE5Odn8RZfeGV5hMlY+cvXUW6/BXRo11jJWO2du2vW0iRy++tMuZMX+ORNU9EHiFjctBaVkde32q6cf0q/P99w9J2JkruyxrWvwdmL/0VeQoUUVUUOapcJq9aMbNtU9q2LbsFDUGjFgH/TeyrlzB6cB80bd0JXt6ZnF4Zk2eJIMoPf7k2oXnbQDWG1vgvXzqPknm91ZugzVp3capveFPWdl5OpKPP4HFvXEQrwtY9oClkFW3d9qNI550RWvlolTG94+nhZ3spI+zq4MolcxHQoq79JY7wctHzV4E7V8YePnyIePHiYd68eRgyZAhOnz6NQoUKYeTIkWjVqhWOHj2KlClTYsGCBUiTJo36Z+fOndGlSxeMGTMGt27dQtmyZfHjjz9Cbnzv0KED7t69i9u3b2Pr1q24efMmHj9+rD6/dOlSPHjwAHny5FF9fXx8kD9/fvj5+aFv3752RN27d8fmzZuxc+dOnD9/XsXx66+/IkGCBPD394dcZho/fnysXLkSbdu2VYIk8V69elV9tkyZMujUqRMuXLiA4sWLY9asWXYpHDt2LGRlUOKSFa5+/fqpMaXly5cP9erVw5YtW3D48GE8f/5cMalZsyZKlSqlYooVKxaqV6+O6dPNf5s5ZUzPn0r39G0U0AdzlqyFXF9ha1oO8L98+Y+SrozeabB75atbAaTJVRlyTYXtAP/hY38ih28tDA/qgIBm/21ndhswFj+MnYKGX1XE5BG9Vd+wsTjT1z10PGNUj5AxOSsmh9GbfttZ3RMmZ7L+N3YoSn1RQb0tKPIib1nKf034Fc6CjxMlQYfu/dQ/5Q1LOVu2bcv6UFdb1P2yBA79sVed68qTvwiuX7uMn0YPwsOHD7D6t8PqjjJHzdEN/PKX/+WL55XQyOqM3G0mLyrIErM0rfEnSPgxPsvwkbpqQ3LNlCU7tm5eqyl3OTPnqMnbOn27t1MXv8qdZ/KWaMpPUuPx40dKUDevX6muo5ALX+VKDSWwL19CCx+tMqZ3PD38UqfxQo3yRXD65DF1tUWmLDlw4ughjBwchPTeGTF/1Q710oLZZOzJkydKomrUqIGZM2fi3r17SJ8+PVKnTo1169YhefLkKF26NHLmzIkRI0ZgyZIl+Oqrr9CmTRslKiJeBQsWRJUqVTBw4EB07doVM2bMUP8UiZH+rVu3xq5du7Bw4ULIL67u3bu3etaZM2cwbtw4TJ06VcmPrWXOnBktW7ZUolWgQAElh/3791dSV79+fSRLlgxTpkzBmjVr1HNlPBGy9evXK7GTXESWRDQlF/ls5cqVIduMTZo0wYoVK1Q+q1evVjHKs2X7UfK4du0a1q5di0yZMqnYevbsqYRTLjlOmDChyo0rY45/KN7bWw4+5fd6xk9MN2XZfeA4dR3F0F7t8Wn6NKj0xedwJGOzFq9Bg7a9MG309+p6Cmld+43G4PHT1JUV1Sv44tzFqxg1cTbixI6FuHFiq5UxORfmVaCS+trY/l0QK2ZMzFm6Fg8fPcHOfYcQLWpU/DKiNwrnzYGwsfiVKKi5r5vweMSwHiFjL54/x+C+gZDVjAf37yFztpzo0nMQMmTKgq+rl8G5s3+iU4/+6k4vWUEbGNQJp08eR9JkKVC3cUtk9MmGb+pUwM8zlqFE6Vf/NS0vAYwe0kddu/DXX1fVRaByJk0ufX39zcOws8jR76aUX3GROGlyZMmWS70pKpL4enMmfjl0P3H8MMSJExd9h/6Iz0v6ac79bTNeuMiW7qEDv0MO8MuvPkqbPoO6/LR2g2bqbdTXmxY+WmVMK+/wxtPDT2px9+7f6jcjrF+9FLdv3VC/R7S0XyW07/a9/YJfs8qYCEr58uVV6URKZMVq9OhXv/nhu+++w8GDB7F8+XIlYyJAIi0iRdJkdWnu3LlKakSK5P+fPXtWfU/+wyZu3LhKvqSfNFkdS5QoERYtWoTs2bMjXbp0OHXqFLy9vdVKnHzt0qVL6n+yNXj//n3E+Pd38e3evVt9TcRsw4YNKFeunPq+rO7ZxFKkr2rVV1fKyGfluR07dlT5yepXUFCQfYpKfxE++ZrkLd+XVTtpIosSky1XytjbfxZSxvS7wrmLV1CpYQd1z1ehPDmwYd54hzI2Y+Eq1G/TC9PHfI961V79LJKLXbv2H40ZC1fj3v2H6tLX4UEBStAuXLqGgxtnq8+JdLXrORTBx0/jowTx1Z1k8gbmpFlLVX+5k+zU9iW4eOXaG7Fo7Rs1qvZf16SfmrVGMKyMGQlz2HvLjBQbY7EuAXduU9oEZv/+/cidO7eCWKJECbXC1K1bN7tsydadyI/ImKxOiQDZmqw8yfakbE2KjO3bt0+tUkmTrUPZ5gwODkbWrP/dUScrb7J1KStsIkGymiXCJGIn24EbN27E7NmzUadO6DfDbM8U2ZMXEGSVTlbnbE1WsHbs2KFW06TJqp7kI0Ipq10nT558Y6J8/fXXavVM4qhWrZrahpUmMihxyrNEGCljlDHr/i3DzIxCgDL2WiXkclA5M1aham3I9RS2Jpd+Ll88B7uCr0C2AtlIIDIIRIaMHThwALlyvbq2RORFzoGJWEkTQXpdxkSQHj16ZE990qRJajvvypUrqo+Il6y0vS5jR44cQZYsWex9UqVKpT4rW5jDhw/H4sWL1Rmzzz77TG1RNm3aVK2wNW/eHHfu3HGIWbYpa9WqFer7ImNy1kzEKqyMyfanjNe+/att9LBN+sh5MDlvRhlzfmZzZcx5ZuxBAmEJUMZeIyLbWf4lcuLu37fV+aAUn6RWvyNy2sQx6hxXv6E/cgaRQKQRMJqMybbf5cuX1YqXtF69eqnzW3v27HlDxmSb8sMPP1RnuGzblLKSlSRJEixbtkxJn6xAeXl5QbYgZUVLVtM+/vhjtcKWN29edRBfVqikyTkwEUHp76yMVaxYEUmTJoXIo63J2CKGckSAMqZvSlPG9PFjbxIQApSxMPNAfhWRrI7JPWJ37txWl8HKr2D6NqBHqFv5OX1IwN0EjCZjtWvXVluV8lbijRs34OvrixYtWigRC7syJmwCAgKwbds2tfolb0TK4X55E1LOidnOghUuXBgxY8ZU58tsq2rSV86uiSzJvWjyIku7du2UvMmWqbMyJgf4ZTtUzpTJG5eynSmCJs+Ts2XvkrEUKVKordVGjRqpLUuzN75NafYKMn4rEqCMWbGqzMkSBIwmY7KFKG9SytakHMaXFa8JEyYomXIkY7KaJW9GyoF9WYGS1S95OUDedLS1UaNGqe1DWUGT6yVsTc5ryVamXG0hd5aJ+I0fP169POCsjMmY8lzZFpVD+WnTpoVcoyFnxqS9S8bkrU3J+4svvlBn58zeKGNmryDjtyIBypgVq8qcLEHA1TKmB4pIyDfffKPu6WIzNwHKmLnrx+itSYAyZs26MisLEKCMWaCIBkyBMmbAojAkjydAGfP4KUAARiVAGTNqZcwdF2XM3PVj9NYkQBmzZl2ZlQUIGEnGLICTKfxLgDLGqUACxiNAGTNeTRgRCSgClDFOBHcQoIy5gyrHJAF9BChj+vixNwm4jQBlzG1oPXpgyphHl5/JG5QAZcyghWFYJEAZ4xxwBwHKmDuockwS0EeAMqaPH3uTgNsIUMbchtajB6aMeXT5mbxBCVDGDFoYhkUClDHOAXcQoIy5gyrHJAF9BChj+vixNwm4jQBlzG1oPXpgyphHl5/JG5QAZcyghWFYJEAZ4xxwBwHKmDuockwS0EeAMqaPH3uTgNsIUMbchtajB6aMeXT5mbxBCVDGDFoYhkUClDHOAXcQoIy5gyrHJAF9BChj+vixNwm4jQBlzG1oPXpgyphHl5/JG5SAkrEBk7eGeKVMgNTJErokzDmbTqNtnWIuGYuDkICnEqCMeWrl3Zv3sbkFES2RP+Imzu+SB93bWw4+5fe6ZCwOQgKeSkDJWPCZmyGj5+5Gyc9Su0TIKGOeOp2YtysJUMZcSZMPWxMbAAACsElEQVRj2Qg8uLgSJ5ZWQHyfPi4RMsoY5xYJ6CegZCwkJCTkyNlbcJWQUcb0F4YjkABljHPAXQRcKWSUMXdVieN6EgG7jEnSrhIyypgnTSHm6i4ClDF3keW4QsBVQkYZ43wiAf0EQsmYq4SMMqa/MByBBChjnAPuJuAKIaOMubtKHN8TCLwhY64QMsqYJ0wd5uhuApQxdxPm+K5YIaOMcR6RgH4CDmVMr5BRxvQXhiOQAGWMcyCyCOhZIaOMRVaV+BwrEwhXxvQIGWXMylOGuUUWAcpYZJHmc/SskFHGOH9IQD+Bt8pYRIWMMqa/MByBBChjnAORTSAiK2SUsciuEp9nRQLvlLGICBllzIpThTlFNgHKWGQT5/MiskJGGeO8IQH9BDTJmLNCRhnTXxiOQAKUMc6B90XAmRUyytj7qhKfayUCmmXMGSGjjFlpijCX90WAMva+yPO5zqyQUcY4X0hAPwGnZEyrkFHG9BeGI5AAZYxz4H0T0LJCRhl731Xi861AwGkZ0yJklDErTA3m8L4JUMbedwX4fC0rZJQxzhMS0E8gQjL2LiGjjOkvDEcgAcoY54BRCLxthYwyZpQqMQ4zE4iwjL1NyChjZp4SjN0oBChjRqkE43jbChlljPODBPQT0CVj4QkZZUx/YTgCCVDGOAeMRsDRChllzGhVYjxmJKBbxhwJGWXMjFOBMRuNAGXMaBVhPI5WyChjnBckoJ+AS2QsrJBtP3ILbesU0x8dRyABDyZAGfPg4hs89ddXyF6e6w2f8nsNHjHDIwFjE3CZjL0uZDFixqGMGbvujM4EBChjJiiSB4doE7K4sUAZ8+B5wNRdQ8ClMmYTsplrj+D+439cEyFHIQEPJeCVMgE6fJXbQ7Nn2mYgIEJ2cXMLRHl2yQzhMkYSMCyBOEkL4/9VcTbCngSYIgAAAABJRU5ErkJggg=="}}},{"metadata":{},"cell_type":"markdown","source":"## Update: 24/11\n\n* Using EfficientNetB4\n* Using TTA * 5"},{"metadata":{"papermill":{"duration":0.034684,"end_time":"2020-12-18T20:11:48.394056","exception":false,"start_time":"2020-12-18T20:11:48.359372","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Update: 15/11\n\n* Using Mixed Precision Training\n* Storing data cache in /kaggle folder as it has more memory available"},{"metadata":{"papermill":{"duration":0.033927,"end_time":"2020-12-18T20:11:48.497151","exception":false,"start_time":"2020-12-18T20:11:48.463224","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Update: 14/11\n\n* I was shuffling the validation dataset too, hence that was a bug. \n* Trying out NS."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","execution":{"iopub.execute_input":"2020-12-18T20:11:48.574939Z","iopub.status.busy":"2020-12-18T20:11:48.574376Z","iopub.status.idle":"2020-12-18T20:11:53.427827Z","shell.execute_reply":"2020-12-18T20:11:53.426827Z"},"papermill":{"duration":4.896066,"end_time":"2020-12-18T20:11:53.42794","exception":false,"start_time":"2020-12-18T20:11:48.531874","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport tensorflow as tf\nfrom tensorflow.keras.models import Sequential, Model\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\nfrom tensorflow.keras.layers import Dense, Dropout, Flatten,GlobalAveragePooling2D,BatchNormalization, Activation\nimport glob\n\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow import keras\n\n\nimport os\n\n# from tensorflow.compat.v1 import ConfigProto\n# from tensorflow.compat.v1 import InteractiveSession\n# config = ConfigProto()\n# config.gpu_options.allow_growth = True\n# session = InteractiveSession(config=config)\n","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.034948,"end_time":"2020-12-18T20:11:53.4982","exception":false,"start_time":"2020-12-18T20:11:53.463252","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Mixed Precision Training"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:53.571984Z","iopub.status.busy":"2020-12-18T20:11:53.571342Z","iopub.status.idle":"2020-12-18T20:11:56.218991Z","shell.execute_reply":"2020-12-18T20:11:56.219493Z"},"papermill":{"duration":2.686751,"end_time":"2020-12-18T20:11:56.219662","exception":false,"start_time":"2020-12-18T20:11:53.532911","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"from tensorflow.keras.mixed_precision import experimental as mixed_precision\npolicy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')\nmixed_precision.set_policy(policy)","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.034715,"end_time":"2020-12-18T20:11:56.290186","exception":false,"start_time":"2020-12-18T20:11:56.255471","status":"completed"},"tags":[]},"cell_type":"markdown","source":"Adding Seed helps to reproduce results. Setting Debug Parameter will run the model on smaller number of epochs to validate the architecture."},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:56.36727Z","iopub.status.busy":"2020-12-18T20:11:56.366722Z","iopub.status.idle":"2020-12-18T20:11:56.370561Z","shell.execute_reply":"2020-12-18T20:11:56.370133Z"},"papermill":{"duration":0.045563,"end_time":"2020-12-18T20:11:56.370694","exception":false,"start_time":"2020-12-18T20:11:56.325131","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"SEED = 42\nDEBUG = False\n\nos.environ['PYTHONHASHSEED'] = str(SEED)\nnp.random.seed(SEED)\ntf.random.set_seed(SEED)","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.034535,"end_time":"2020-12-18T20:11:56.439888","exception":false,"start_time":"2020-12-18T20:11:56.405353","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Prepare Data"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:56.516247Z","iopub.status.busy":"2020-12-18T20:11:56.515688Z","iopub.status.idle":"2020-12-18T20:11:56.552856Z","shell.execute_reply":"2020-12-18T20:11:56.553272Z"},"papermill":{"duration":0.0785,"end_time":"2020-12-18T20:11:56.553384","exception":false,"start_time":"2020-12-18T20:11:56.474884","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"df = pd.read_csv('../input/cassava-leaf-disease-classification/train.csv')\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.label.unique()","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.035169,"end_time":"2020-12-18T20:11:56.624196","exception":false,"start_time":"2020-12-18T20:11:56.589027","status":"completed"},"tags":[]},"cell_type":"markdown","source":"Distribution of dataset:"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:56.700337Z","iopub.status.busy":"2020-12-18T20:11:56.69982Z","iopub.status.idle":"2020-12-18T20:11:56.728972Z","shell.execute_reply":"2020-12-18T20:11:56.72809Z"},"papermill":{"duration":0.069212,"end_time":"2020-12-18T20:11:56.729058","exception":false,"start_time":"2020-12-18T20:11:56.659846","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"df['path'] = '../input/cassava-leaf-disease-classification/train_images/' + df['image_id']\ndf.label.value_counts(normalize=True) * 100\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"so there is a high imbalance in the dataset as we can see."},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:56.806422Z","iopub.status.busy":"2020-12-18T20:11:56.804637Z","iopub.status.idle":"2020-12-18T20:11:56.807439Z","shell.execute_reply":"2020-12-18T20:11:56.807923Z"},"papermill":{"duration":0.043158,"end_time":"2020-12-18T20:11:56.808038","exception":false,"start_time":"2020-12-18T20:11:56.76488","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"if DEBUG:\n    _, df = train_test_split(df, test_size = 0.1, random_state=SEED, shuffle=True, stratify=df['label'])\n","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.035425,"end_time":"2020-12-18T20:11:56.879204","exception":false,"start_time":"2020-12-18T20:11:56.843779","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Data Loader using tf.Data"},{"metadata":{"papermill":{"duration":0.03553,"end_time":"2020-12-18T20:11:56.950118","exception":false,"start_time":"2020-12-18T20:11:56.914588","status":"completed"},"tags":[]},"cell_type":"markdown","source":"### Spliting Dataset"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.028074Z","iopub.status.busy":"2020-12-18T20:11:57.027212Z","iopub.status.idle":"2020-12-18T20:11:57.032689Z","shell.execute_reply":"2020-12-18T20:11:57.033145Z"},"papermill":{"duration":0.047359,"end_time":"2020-12-18T20:11:57.033246","exception":false,"start_time":"2020-12-18T20:11:56.985887","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"X_train, X_valid = train_test_split(df, test_size = 0.1, random_state=SEED, shuffle=True)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.115668Z","iopub.status.busy":"2020-12-18T20:11:57.115121Z","iopub.status.idle":"2020-12-18T20:11:57.123659Z","shell.execute_reply":"2020-12-18T20:11:57.123139Z"},"papermill":{"duration":0.054853,"end_time":"2020-12-18T20:11:57.123742","exception":false,"start_time":"2020-12-18T20:11:57.068889","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"train_ds = tf.data.Dataset.from_tensor_slices((X_train.path.values, X_train.label.values))\nvalid_ds = tf.data.Dataset.from_tensor_slices((X_valid.path.values, X_valid.label.values))","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.200373Z","iopub.status.busy":"2020-12-18T20:11:57.199857Z","iopub.status.idle":"2020-12-18T20:11:57.244925Z","shell.execute_reply":"2020-12-18T20:11:57.24536Z"},"papermill":{"duration":0.085685,"end_time":"2020-12-18T20:11:57.245464","exception":false,"start_time":"2020-12-18T20:11:57.159779","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"for path, label in train_ds.take(5):\n    print ('Path: {}, Label: {}'.format(path, label))","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.32294Z","iopub.status.busy":"2020-12-18T20:11:57.322056Z","iopub.status.idle":"2020-12-18T20:11:57.331196Z","shell.execute_reply":"2020-12-18T20:11:57.330785Z"},"papermill":{"duration":0.04937,"end_time":"2020-12-18T20:11:57.331281","exception":false,"start_time":"2020-12-18T20:11:57.281911","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"for path, label in valid_ds.take(5):\n    print ('Path: {}, Label: {}'.format(path, label))","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.036468,"end_time":"2020-12-18T20:11:57.404669","exception":false,"start_time":"2020-12-18T20:11:57.368201","status":"completed"},"tags":[]},"cell_type":"markdown","source":"### Data Generator"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.481395Z","iopub.status.busy":"2020-12-18T20:11:57.480828Z","iopub.status.idle":"2020-12-18T20:11:57.484907Z","shell.execute_reply":"2020-12-18T20:11:57.484443Z"},"papermill":{"duration":0.043927,"end_time":"2020-12-18T20:11:57.484989","exception":false,"start_time":"2020-12-18T20:11:57.441062","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"AUTOTUNE = tf.data.experimental.AUTOTUNE\n\ntarget_size_dim = 512","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.568542Z","iopub.status.busy":"2020-12-18T20:11:57.566981Z","iopub.status.idle":"2020-12-18T20:11:57.569895Z","shell.execute_reply":"2020-12-18T20:11:57.56947Z"},"papermill":{"duration":0.048197,"end_time":"2020-12-18T20:11:57.569989","exception":false,"start_time":"2020-12-18T20:11:57.521792","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"def process_data_train(image_path, label):\n    # load the raw data from the file as a string\n    img = tf.io.read_file(image_path)\n    img = tf.image.decode_jpeg(img, channels=3)\n    img = tf.image.random_brightness(img, 0.3)\n    img = tf.image.random_flip_left_right(img, seed=None)\n    img = tf.image.random_flip_up_down(img)\n    #img = tf.image.random_crop(img, size=[target_size_dim, target_size_dim, 3])\n    return img, label\n\ndef process_data_valid(image_path, label):\n    # load the raw data from the file as a string\n    img = tf.io.read_file(image_path)\n    img = tf.image.decode_jpeg(img, channels=3)\n    img = tf.image.resize(img, [target_size_dim,target_size_dim])\n    return img, label\n","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.668981Z","iopub.status.busy":"2020-12-18T20:11:57.668056Z","iopub.status.idle":"2020-12-18T20:11:57.902469Z","shell.execute_reply":"2020-12-18T20:11:57.90191Z"},"papermill":{"duration":0.281784,"end_time":"2020-12-18T20:11:57.902579","exception":false,"start_time":"2020-12-18T20:11:57.620795","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"# Set `num_parallel_calls` so multiple images are loaded/processed in parallel.\ntrain_ds = train_ds.map(process_data_train, num_parallel_calls=AUTOTUNE)\nvalid_ds = valid_ds.map(process_data_valid, num_parallel_calls=AUTOTUNE)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:57.983059Z","iopub.status.busy":"2020-12-18T20:11:57.982147Z","iopub.status.idle":"2020-12-18T20:11:58.352212Z","shell.execute_reply":"2020-12-18T20:11:58.352707Z"},"papermill":{"duration":0.413083,"end_time":"2020-12-18T20:11:58.352856","exception":false,"start_time":"2020-12-18T20:11:57.939773","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"for image, label in train_ds.take(1):\n    plt.imshow(image.numpy().astype('uint8'))\n    plt.show()\n    print(\"Image shape: \", image.numpy().shape)\n    print(\"Label: \", label.numpy())","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.040648,"end_time":"2020-12-18T20:11:58.435857","exception":false,"start_time":"2020-12-18T20:11:58.395209","status":"completed"},"tags":[]},"cell_type":"markdown","source":"### Improving Performance"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:58.52524Z","iopub.status.busy":"2020-12-18T20:11:58.524726Z","iopub.status.idle":"2020-12-18T20:11:58.545412Z","shell.execute_reply":"2020-12-18T20:11:58.544978Z"},"papermill":{"duration":0.068582,"end_time":"2020-12-18T20:11:58.545497","exception":false,"start_time":"2020-12-18T20:11:58.476915","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"def configure_for_performance(ds, batch_size = 32):\n    ds = ds.cache('/kaggle/dump.tfcache') \n    \n    ds = ds.shuffle(buffer_size=1024)\n    ds = ds.batch(batch_size)\n    ds = ds.prefetch(buffer_size=AUTOTUNE)\n    return ds\n\nbatch_size = 8\n\ntrain_ds_batch = configure_for_performance(train_ds, batch_size)\nvalid_ds_batch = valid_ds.batch(batch_size)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:11:58.631547Z","iopub.status.busy":"2020-12-18T20:11:58.630709Z","iopub.status.idle":"2020-12-18T20:12:09.116576Z","shell.execute_reply":"2020-12-18T20:12:09.115845Z"},"papermill":{"duration":10.530394,"end_time":"2020-12-18T20:12:09.116686","exception":false,"start_time":"2020-12-18T20:11:58.586292","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"image_batch, label_batch = next(iter(train_ds_batch))","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:09.213573Z","iopub.status.busy":"2020-12-18T20:12:09.211054Z","iopub.status.idle":"2020-12-18T20:12:09.65153Z","shell.execute_reply":"2020-12-18T20:12:09.651954Z"},"papermill":{"duration":0.493683,"end_time":"2020-12-18T20:12:09.652074","exception":false,"start_time":"2020-12-18T20:12:09.158391","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"\nplt.figure(figsize=(10, 10))\nfor i in range(8):\n    ax = plt.subplot(4, 4, i + 1)\n    plt.imshow(image_batch[i].numpy().astype(\"uint8\"))\n    label = label_batch[i].numpy()\n    plt.title(label)\n    plt.axis(\"off\")","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.047222,"end_time":"2020-12-18T20:12:09.747573","exception":false,"start_time":"2020-12-18T20:12:09.700351","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Data Augmentation"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:09.848038Z","iopub.status.busy":"2020-12-18T20:12:09.847411Z","iopub.status.idle":"2020-12-18T20:12:09.861771Z","shell.execute_reply":"2020-12-18T20:12:09.862166Z"},"papermill":{"duration":0.067088,"end_time":"2020-12-18T20:12:09.862278","exception":false,"start_time":"2020-12-18T20:12:09.79519","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"data_augmentation = keras.Sequential(\n    [\n        tf.keras.layers.experimental.preprocessing.RandomRotation(0.2, interpolation='nearest'),\n        tf.keras.layers.experimental.preprocessing.RandomContrast((0.2 ))\n    ]\n)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:09.965291Z","iopub.status.busy":"2020-12-18T20:12:09.964242Z","iopub.status.idle":"2020-12-18T20:12:10.561203Z","shell.execute_reply":"2020-12-18T20:12:10.561633Z"},"papermill":{"duration":0.652486,"end_time":"2020-12-18T20:12:10.561751","exception":false,"start_time":"2020-12-18T20:12:09.909265","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"\nplt.figure(figsize=(10, 10))\nfor i in range(8):\n    augmented_images = data_augmentation(image_batch)\n    ax = plt.subplot(4, 4, i + 1)\n    plt.imshow(augmented_images[i].numpy().astype(\"uint8\"))\n    label = label_batch[i].numpy()\n    plt.title(label)\n    plt.axis(\"off\")","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.055579,"end_time":"2020-12-18T20:12:10.672938","exception":false,"start_time":"2020-12-18T20:12:10.617359","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Creating Model"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:10.786081Z","iopub.status.busy":"2020-12-18T20:12:10.785508Z","iopub.status.idle":"2020-12-18T20:12:10.789501Z","shell.execute_reply":"2020-12-18T20:12:10.78909Z"},"papermill":{"duration":0.062569,"end_time":"2020-12-18T20:12:10.789605","exception":false,"start_time":"2020-12-18T20:12:10.727036","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"## Only available in tf2.3+\n\nfrom tensorflow.keras.applications import EfficientNetB5\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Need to try out [ohem loss](https://github.com/GXYM/OHEM-loss)."},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:10.910057Z","iopub.status.busy":"2020-12-18T20:12:10.909376Z","iopub.status.idle":"2020-12-18T20:12:10.912558Z","shell.execute_reply":"2020-12-18T20:12:10.912936Z"},"papermill":{"duration":0.069502,"end_time":"2020-12-18T20:12:10.91304","exception":false,"start_time":"2020-12-18T20:12:10.843538","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"def load_pretrained_model(weights_path, drop_connect, target_size_dim, layers_to_unfreeze=5):\n    model = EfficientNetB5(\n            weights=None, \n            include_top=False, \n            drop_connect_rate=0.2\n        )\n    \n    model.load_weights(weights_path)\n    \n    model.trainable = True\n\n    # for layer in model.layers[-layers_to_unfreeze:]:\n    #     if not isinstance(layer, tf.keras.layers.BatchNormalization): \n    #         layer.trainable = True\n\n    if DEBUG:\n        for layer in model.layers:\n            #print(layer.name, layer.trainable)\n            pass\n\n    return model\n\ndef build_my_model(base_model, optimizer, loss='sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy']):\n    \n    inputs = tf.keras.layers.Input(shape=(target_size_dim, target_size_dim, 3))\n    x = data_augmentation(inputs)\n    outputs_eff = base_model(x)\n    global_avg_pooling = GlobalAveragePooling2D()(outputs_eff)\n    dense_1= Dense(256)(global_avg_pooling)\n    bn_1 = BatchNormalization()(dense_1)\n    activation = Activation('relu')(bn_1)\n    dropout = Dropout(0.3)(activation)\n    dense_2 = Dense(5)(dropout)\n    outputs = Activation('softmax', dtype='float32', name='predictions')(dense_2)\n\n    my_model = tf.keras.Model(inputs, outputs)\n    \n    my_model.compile(\n        optimizer=optimizer,\n        loss=loss,\n        metrics=metrics\n        \n    )\n    return my_model\n\n","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:11.025361Z","iopub.status.busy":"2020-12-18T20:12:11.024845Z","iopub.status.idle":"2020-12-18T20:12:11.028831Z","shell.execute_reply":"2020-12-18T20:12:11.028391Z"},"papermill":{"duration":0.0621,"end_time":"2020-12-18T20:12:11.028918","exception":false,"start_time":"2020-12-18T20:12:10.966818","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"#!wget https://storage.googleapis.com/keras-applications/efficientnetb3_notop.h5\n## to get model weights","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:11.142743Z","iopub.status.busy":"2020-12-18T20:12:11.142098Z","iopub.status.idle":"2020-12-18T20:12:11.145351Z","shell.execute_reply":"2020-12-18T20:12:11.145771Z"},"papermill":{"duration":0.063193,"end_time":"2020-12-18T20:12:11.145871","exception":false,"start_time":"2020-12-18T20:12:11.082678","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"#model_weights_path = '../input/effnetb4-ns/effnetb4_ns.h5'\nmodel_weights_path = '../input/tfkerasefficientnetimagenetnotop/efficientnetb5_notop.h5'\nmodel_weights_path","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"drop connect rate has been changed to 0.2 from 0.4 and now changing drop_rate also to 0.2.<br/>\nusing b5 instead of b4.<br/>\nincreased lr from 1e-4 to 1e-3.<br/>"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:11.263607Z","iopub.status.busy":"2020-12-18T20:12:11.262786Z","iopub.status.idle":"2020-12-18T20:12:19.808388Z","shell.execute_reply":"2020-12-18T20:12:19.807622Z"},"papermill":{"duration":8.608496,"end_time":"2020-12-18T20:12:19.80849","exception":false,"start_time":"2020-12-18T20:12:11.199994","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"drop_rate = 0.2 ## value of dropout to be used in loaded network\nbase_model = load_pretrained_model( model_weights_path, drop_rate, target_size_dim )\n\noptimizer = tf.keras.optimizers.Adam(lr = 1e-3)\nmy_model = build_my_model(base_model, optimizer)\nmy_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.05647,"end_time":"2020-12-18T20:12:19.924089","exception":false,"start_time":"2020-12-18T20:12:19.867619","status":"completed"},"tags":[]},"cell_type":"markdown","source":"### Callbacks"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:20.047416Z","iopub.status.busy":"2020-12-18T20:12:20.046583Z","iopub.status.idle":"2020-12-18T20:12:20.050123Z","shell.execute_reply":"2020-12-18T20:12:20.049686Z"},"papermill":{"duration":0.069273,"end_time":"2020-12-18T20:12:20.050212","exception":false,"start_time":"2020-12-18T20:12:19.980939","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"weight_path_save = 'best_model.hdf5'\nlast_weight_path = 'last_model.hdf5'\n\ncheckpoint = ModelCheckpoint(weight_path_save, \n                             monitor= 'val_loss', \n                             verbose=1, \n                             save_best_only=True, \n                             mode= 'min', \n                             save_weights_only = False)\ncheckpoint_last = ModelCheckpoint(last_weight_path, \n                             monitor= 'val_loss', \n                             verbose=1, \n                             save_best_only=False, \n                             mode= 'min', \n                             save_weights_only = False)\n\n\nearly = EarlyStopping(monitor= 'val_loss', \n                      mode= 'min', \n                      patience=5)\n\nreduceLROnPlat = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=2, verbose=1, mode='auto', epsilon=0.0001, cooldown=5, min_lr=0.00001)\ncallbacks_list = [checkpoint, checkpoint_last, early, reduceLROnPlat]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"changed reducelronplateau from facto 0.8 to 0.2."},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:20.168278Z","iopub.status.busy":"2020-12-18T20:12:20.167776Z","iopub.status.idle":"2020-12-18T20:12:20.172317Z","shell.execute_reply":"2020-12-18T20:12:20.172925Z"},"papermill":{"duration":0.065636,"end_time":"2020-12-18T20:12:20.173166","exception":false,"start_time":"2020-12-18T20:12:20.10753","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"print('Compute dtype: %s' % policy.compute_dtype)\nprint('Variable dtype: %s' % policy.variable_dtype)","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":0.056638,"end_time":"2020-12-18T20:12:20.286718","exception":false,"start_time":"2020-12-18T20:12:20.23008","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Train Model"},{"metadata":{},"cell_type":"markdown","source":"increasing epoch number from 12 to 20. so finally this failed at 13th epoch. Sounds about right why it had 12 epochs!<br/>"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:20.405684Z","iopub.status.busy":"2020-12-18T20:12:20.404906Z","iopub.status.idle":"2020-12-18T20:12:20.410272Z","shell.execute_reply":"2020-12-18T20:12:20.409844Z"},"papermill":{"duration":0.06715,"end_time":"2020-12-18T20:12:20.410354","exception":false,"start_time":"2020-12-18T20:12:20.343204","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"if DEBUG:\n    epochs = 3\nelse:\n    epochs = 12\n    \nprint(f\"Model will train for {epochs} epochs\")","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T20:12:21.463546Z","iopub.status.busy":"2020-12-18T20:12:21.462862Z","iopub.status.idle":"2020-12-18T23:00:16.584221Z","shell.execute_reply":"2020-12-18T23:00:16.58491Z"},"papermill":{"duration":10075.190828,"end_time":"2020-12-18T23:00:16.585124","exception":false,"start_time":"2020-12-18T20:12:21.394296","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"if DEBUG:\n    history = my_model.fit(train_ds_batch, \n                              validation_data = valid_ds_batch, \n                              epochs = epochs, \n                              callbacks = callbacks_list,\n                               steps_per_epoch = 1,\n                               \n                           \n                              )\nelse:\n    history = my_model.fit(train_ds_batch, \n                              validation_data = valid_ds_batch, \n                              epochs = epochs, \n                              callbacks = callbacks_list\n                               \n                            \n                              )","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:00:30.403421Z","iopub.status.busy":"2020-12-18T23:00:30.402494Z","iopub.status.idle":"2020-12-18T23:00:30.408007Z","shell.execute_reply":"2020-12-18T23:00:30.408382Z"},"papermill":{"duration":7.007458,"end_time":"2020-12-18T23:00:30.4085","exception":false,"start_time":"2020-12-18T23:00:23.401042","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"\ndef plot_hist(hist):\n    plt.figure(figsize=(15,5))\n    local_epochs = len(hist.history[\"sparse_categorical_accuracy\"])\n    plt.plot(np.arange(local_epochs, step=1), hist.history[\"sparse_categorical_accuracy\"], '-o', label='Train Accuracy',color='#ff7f0e')\n    plt.plot(np.arange(local_epochs, step=1), hist.history[\"val_sparse_categorical_accuracy\"], '-o',label='Val Accuracy',color='#1f77b4')\n    plt.xlabel('Epoch',size=14)\n    plt.ylabel('Accuracy',size=14)\n    plt.legend(loc=2)\n    \n    plt2 = plt.gca().twinx()\n    plt2.plot(np.arange(local_epochs, step=1) ,history.history['loss'],'-o',label='Train Loss',color='#2ca02c')\n    plt2.plot(np.arange(local_epochs, step=1) ,history.history['val_loss'],'-o',label='Val Loss',color='#d62728')\n    plt.legend(loc=3)\n    plt.ylabel('Loss',size=14)\n    plt.title(\"Model Accuracy and loss\")\n    \n    plt.savefig('loss.png')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:00:45.737601Z","iopub.status.busy":"2020-12-18T23:00:45.736784Z","iopub.status.idle":"2020-12-18T23:00:46.094219Z","shell.execute_reply":"2020-12-18T23:00:46.094648Z"},"papermill":{"duration":8.096141,"end_time":"2020-12-18T23:00:46.094769","exception":false,"start_time":"2020-12-18T23:00:37.998628","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"plot_hist(history)","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":6.991721,"end_time":"2020-12-18T23:01:00.184574","exception":false,"start_time":"2020-12-18T23:00:53.192853","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Evaluating Model on Validation Set"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:01:14.522175Z","iopub.status.busy":"2020-12-18T23:01:14.520416Z","iopub.status.idle":"2020-12-18T23:01:14.522851Z","shell.execute_reply":"2020-12-18T23:01:14.523269Z"},"papermill":{"duration":7.535469,"end_time":"2020-12-18T23:01:14.523378","exception":false,"start_time":"2020-12-18T23:01:06.987909","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"from sklearn.metrics import confusion_matrix, classification_report","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:01:28.287548Z","iopub.status.busy":"2020-12-18T23:01:28.277414Z","iopub.status.idle":"2020-12-18T23:01:28.698412Z","shell.execute_reply":"2020-12-18T23:01:28.697407Z"},"papermill":{"duration":7.127589,"end_time":"2020-12-18T23:01:28.698544","exception":false,"start_time":"2020-12-18T23:01:21.570955","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"my_model.load_weights(weight_path_save) ## load the best model or all your metrics would be on the last run not on the best one","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:01:44.198321Z","iopub.status.busy":"2020-12-18T23:01:44.197486Z","iopub.status.idle":"2020-12-18T23:02:10.971179Z","shell.execute_reply":"2020-12-18T23:02:10.970265Z"},"papermill":{"duration":35.274458,"end_time":"2020-12-18T23:02:10.971284","exception":false,"start_time":"2020-12-18T23:01:35.696826","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"pred_valid_y = my_model.predict(valid_ds_batch, workers=4, verbose = True)\npred_valid_y_labels = np.argmax(pred_valid_y, axis=-1)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:02:25.906907Z","iopub.status.busy":"2020-12-18T23:02:25.906055Z","iopub.status.idle":"2020-12-18T23:02:34.935633Z","shell.execute_reply":"2020-12-18T23:02:34.93483Z"},"papermill":{"duration":16.216108,"end_time":"2020-12-18T23:02:34.935758","exception":false,"start_time":"2020-12-18T23:02:18.71965","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"valid_labels = np.concatenate([y.numpy() for x, y in valid_ds_batch], axis=0)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:02:50.607477Z","iopub.status.busy":"2020-12-18T23:02:50.606537Z","iopub.status.idle":"2020-12-18T23:02:50.618715Z","shell.execute_reply":"2020-12-18T23:02:50.619125Z"},"papermill":{"duration":7.830964,"end_time":"2020-12-18T23:02:50.619238","exception":false,"start_time":"2020-12-18T23:02:42.788274","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"\nprint(classification_report(valid_labels, pred_valid_y_labels ))","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:03:04.503679Z","iopub.status.busy":"2020-12-18T23:03:04.502864Z","iopub.status.idle":"2020-12-18T23:03:04.517682Z","shell.execute_reply":"2020-12-18T23:03:04.51809Z"},"papermill":{"duration":7.084478,"end_time":"2020-12-18T23:03:04.518211","exception":false,"start_time":"2020-12-18T23:02:57.433733","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"print(confusion_matrix(valid_labels, pred_valid_y_labels ))","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":6.826988,"end_time":"2020-12-18T23:03:18.369482","exception":false,"start_time":"2020-12-18T23:03:11.542494","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Predictions + Test Time Augmentation"},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:03:33.138347Z","iopub.status.busy":"2020-12-18T23:03:33.137453Z","iopub.status.idle":"2020-12-18T23:03:33.140315Z","shell.execute_reply":"2020-12-18T23:03:33.139895Z"},"papermill":{"duration":7.059159,"end_time":"2020-12-18T23:03:33.140408","exception":false,"start_time":"2020-12-18T23:03:26.081249","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"import glob","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:03:48.098366Z","iopub.status.busy":"2020-12-18T23:03:48.097831Z","iopub.status.idle":"2020-12-18T23:03:48.107518Z","shell.execute_reply":"2020-12-18T23:03:48.108047Z"},"papermill":{"duration":7.716775,"end_time":"2020-12-18T23:03:48.108166","exception":false,"start_time":"2020-12-18T23:03:40.391391","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"test_images = glob.glob('../input/cassava-leaf-disease-classification/test_images/*.jpg')\n#test_images = test_images * 5\nprint(test_images)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:04:02.554417Z","iopub.status.busy":"2020-12-18T23:04:02.553644Z","iopub.status.idle":"2020-12-18T23:04:02.556799Z","shell.execute_reply":"2020-12-18T23:04:02.557231Z"},"papermill":{"duration":6.780232,"end_time":"2020-12-18T23:04:02.557344","exception":false,"start_time":"2020-12-18T23:03:55.777112","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"df_test = pd.DataFrame(np.array(test_images), columns=['Path'])\ndf_test.head()","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:04:16.769435Z","iopub.status.busy":"2020-12-18T23:04:16.76858Z","iopub.status.idle":"2020-12-18T23:04:16.826713Z","shell.execute_reply":"2020-12-18T23:04:16.827097Z"},"papermill":{"duration":7.182393,"end_time":"2020-12-18T23:04:16.827219","exception":false,"start_time":"2020-12-18T23:04:09.644826","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"test_ds = tf.data.Dataset.from_tensor_slices((df_test.Path.values))\n\n\ndef process_test(image_path):\n    # load the raw data from the file as a string\n    img = tf.io.read_file(image_path)\n    img = tf.image.decode_jpeg(img, channels=3)\n    img = tf.image.random_brightness(img, 0.3)\n    img = tf.image.random_flip_left_right(img, seed=None)\n    img = tf.image.random_flip_up_down(img)\n    img = tf.image.random_crop(img, size=[target_size_dim, target_size_dim, 3])\n    return img\n    \ntest_ds = test_ds.map(process_test, num_parallel_calls=AUTOTUNE).batch(batch_size*2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds = []\nfor i in range(5):\n    \n    pred_test = my_model.predict(test_ds, workers=16, verbose=1)\n    preds.append(pred_test)\n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pred_y = np.mean(preds, axis=0)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:04:31.234224Z","iopub.status.busy":"2020-12-18T23:04:31.233354Z","iopub.status.idle":"2020-12-18T23:04:33.594039Z","shell.execute_reply":"2020-12-18T23:04:33.592937Z"},"papermill":{"duration":9.858497,"end_time":"2020-12-18T23:04:33.594159","exception":false,"start_time":"2020-12-18T23:04:23.735662","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"#pred_y = my_model.predict(test_ds, workers=4)\npred_y_argmax = np.argmax(pred_y, axis=-1)","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:04:48.817348Z","iopub.status.busy":"2020-12-18T23:04:48.81614Z","iopub.status.idle":"2020-12-18T23:04:48.819732Z","shell.execute_reply":"2020-12-18T23:04:48.820137Z"},"papermill":{"duration":7.269358,"end_time":"2020-12-18T23:04:48.820249","exception":false,"start_time":"2020-12-18T23:04:41.550891","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"df_test['image_id'] = df_test.Path.str.split('/').str[-1]\ndf_test['label'] = pred_y_argmax\ndf_test= df_test[['image_id','label']]\ndf_test.head()","execution_count":null,"outputs":[]},{"metadata":{"execution":{"iopub.execute_input":"2020-12-18T23:05:03.488732Z","iopub.status.busy":"2020-12-18T23:05:03.488151Z","iopub.status.idle":"2020-12-18T23:05:04.034643Z","shell.execute_reply":"2020-12-18T23:05:04.033567Z"},"papermill":{"duration":8.207917,"end_time":"2020-12-18T23:05:04.034758","exception":false,"start_time":"2020-12-18T23:04:55.826841","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"df_test.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"papermill":{"duration":7.694666,"end_time":"2020-12-18T23:05:32.83502","exception":false,"start_time":"2020-12-18T23:05:25.140354","status":"completed"},"tags":[]},"cell_type":"markdown","source":"\n\n### If you learnt something from this kernel kindly upvote :) This keeps me motivated to produce more kernels."},{"metadata":{"papermill":{"duration":7.603867,"end_time":"2020-12-18T23:05:47.764568","exception":false,"start_time":"2020-12-18T23:05:40.160701","status":"completed"},"tags":[]},"cell_type":"markdown","source":"My other Notebooks in this competition:\n\n1. EfficientNetB3 Training with Pure Keras/tf2 ImageDataGenerator Method: [Link](https://www.kaggle.com/harveenchadha/efficientnetb3-keras-tf2-baseline-training)\n2. EfficientNetB3 Inference with Pure Keras/tf2 ImageDataGenerator Method: [Link](https://www.kaggle.com/harveenchadha/efficientnetb3-baseline-inference-keras-tf2)"},{"metadata":{"papermill":{"duration":7.328539,"end_time":"2020-12-18T23:06:02.233885","exception":false,"start_time":"2020-12-18T23:05:54.905346","status":"completed"},"tags":[]},"cell_type":"markdown","source":"## Important Points:\n\n1. ~~Due to Kaggle's space and RAM limitation, I could not make use of cache~~\n2. ~~With the use of Cache, I can confirm I have achieved a 5x speed than ImageDataGenerator. Maybe I will do a kernel will small subset of the data confirming the same.~~\n\nI am using cache now!"},{"metadata":{"papermill":{"duration":7.097324,"end_time":"2020-12-18T23:06:16.454911","exception":false,"start_time":"2020-12-18T23:06:09.357587","status":"completed"},"tags":[],"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}