{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Bengali.Ai : Multi_Output_DenseNet121 (keras)"},{"metadata":{},"cell_type":"markdown","source":"
\n
\n I referred to [kaushl Shah : Starter EDA+ Multi Output CNN](https://www.kaggle.com/kaushal2896/bengali-graphemes-starter-eda-multi-output-cnn). Thanks.
\nThis kernel has changed the Model from my [previous kernel](https://www.kaggle.com/amanooo/bengali-ai-multi-output-densenet-keras). \n- Change a small DenseNet built by myself to **DenseNet121**.\n- And I used locally trained model to avoid GPU time limits.","attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAjMAAAC3CAYAAADn7N8lAAAgAElEQVR4Ae2dP6gfR7bnO5nBoUMzkUMzgXDg4HKDQaHDASUOBGO0cHHoRGCYwBZjVn7yG+u3rGY9zM5YYmcudjCeq+c3XJndAT2PQeZhFoEceEGBAgc3UKBACAcKajlVdapPVVf/r+6u/vX3wqW7q6tOVZ+qrvP5nTrdXfz9739Xt27dGv1/dnamUvyRnBTtYRlff/11imZpGX/961+Tte0vf/mLev78ebK2QRA0AA1AA9AANLBVDRS73U5999136tGjR6P+UypwbFtk+adPnyZr2uPHj0fpSLYrFfwluzgIggagAWgAGoAGVqqBgjwYP/7440qbj2ZDA9AANAANQAPQwNY1UGxdAbh+aAAagAagAWgAGli3BgAz6+6/RVtPy2bvvfce/jPUwZ07Xy46NoZU/tH1HcZSorH0/vvvq5RL7EP6E2WggTk1AJiZU9t7Vtf9+/fVrc8+V/fPnuE/Ix3823/8p/rbye3VjTYCY4ylNPfSv350XT158mR1YwANhgaGagAwM1RzKKcAM2kMT2oDDpjJs19S93OTPMAMJuitaQAws7UeT3i9gJk8jSZgJs9+aYKP1OcAMwknOohahQYAM6vopjwbCZjJ02gCZrhfHqjLrxWqKCL/F0/2ekkLMBPOmafqqCjU0WmYjmOngdOjVesHMON6Ejt9NQCYYaOZ1xYww/1hYObclQcBuFjIee2a+iKjWKeU3hnATDibAWZCjfjH69cPYMbvURz10ABgho1mXlvADPdHHczQ+RN1oSjUhWPOu19bwEw4ka3fWIdXlPZ4/foBzKQdEZuSBpjJ0wACZrhfmmDmmfriyoEqvOUm67Fxy1KX1A323Ny7ps4VB+ryMW3LZSsfhoLyXWVzHQm3ucLM6VGpu6I4VLvdkdLbhzR1kkHlNMp3pHhVyC8XLBc93KlDKndK21L+4U4LtXOyNdY7P09s2enh7tBbmvTk6LqofiOPlzB1HnuO00LZjXI7WY56/ejip6TLuusn9dacD9pdHO6U1FynpmWQCTCTQSestQmAGTaaeW0BM9wfzTBz//iSKtxSkwURASAadvi8hplC5LcwVDDwhOWl5yc8Z8uy7IQQw0tVOcKMAZISUErjeqgMd1hACIypLifIwEABl1FKCWPsslnD7Y41KJGhL8tV5GhRBDJlHpbtgMbVVeYpIaW8NpMWHpdlKnI7GYG4fnTRuutlBbSdt/rh7J2ak1kmwExmHbKm5gBm2GjmtQXMcH90gBmGEQIb3ndwIYDEwozniWFvzb1n6r7cd+VtO9pkh/kTHGcHMxYCQmNpAIeNvDHWDhz0ZGi9EdJVEMqyx345YpxDVXoZIrJDOdagh3IMdFkwidXVkGauN1I3XZsGjBJ42uf+GjnqododFirebtJt23ndmNUHSANm2kcQctRoADDDRjOvLWCG+6MDzFjviPbCCBe9dNfrAOIYrMi0KLCYdrTKTgAv7JHhbXYwU2e4dboPMyHweNOP9TBQ/7h8FSixJVplGzhwcrz8olYpX+5zlra0LnJZVuM2aC/njdVP5zjdLq2564yWq5HNeVewBcxk0knffvutoq9yr+kPMMNGM68tYIb7oxlmblwsXMyMt6QUgwsJLnxeprXBzIRLSgwwcpsbzITLLm6e8wx93KCWyzg2joaNNAfU6GMGIifZej44PSY7SNNtKWNOJNA6eArrpura0rrIFc2u3w3ayxlj9etzNn8dzHhLSzWyuY4VbAEzmXTS7du39Rt1M2lOp2YAZtho5rUFzHB/NMGMWEIiONEwcqAu05IRw4rcSnDhdJkm9/k8b9tkc76E29xgpnZJpQ1mYoY6TNPHDC1i6mqT7RlzXvaJyBEiW8GF88o2eu3gDEO2NcAh65JiOb0OZvi8hsIa2VJe5vuAmUw6CDBTY0QSTvBRI7WH8gEzPJbqYMakl8G/lD+SJiEkBitemi3vAojlcYvsCcZgdjDjGc5y0jVeFwaIiEHVIBDElei06jJTGDOi43FcMHFEdggzleOynW4vdh2tabG6ncQeO3Vy2mJi2s5TE+pk92jewlkBMwt3AFcPmGEDhO1Y6ALM8BiyEBGJham+SE8AjcsvPDUeuFj5lbSgPgc2LbK3ADPa8WGXiXjSs1BSPj0UMagVUDB5aNnHwYvNU8phL4sAnqixrtbnw5VpqBekXGlPh2Wm2FNSTh8McqyUpm21vS53CHj2egsOlGk7b/M7nTrB69kBzGTSV4AZNkDYAmYwBsaOgew8M3aeNWDAcSlH6lQbWfa8xI21HzNjjH8UMBrfIROTHUuzT0E5oA0AbCDM0OX71xHI7WSH4u11RR0cGv1WwKTlvOsb581yklexA5jJpJsAMzBgYw0Yl4dnBmMpV5ipTLcezFTOdkuIAUa3ksi1Rxoovv76a/X8+fM9uqR1XgpgBgaIYWTsFjCDsZQdzESBw3gaKh6EvlN4VHZfIci/dg0Uu91OPXnyJMl10KPFjx49WvX/Uo9HA2ZggMZCDJcHzGAsZQczZGGCZQ73uPNY67N2mLHtDx8F94/7xNaMVeg6yyeFmffff1/dunVr1f8ffPCBevr06ey9CZiBAWIYGbsFzGAsZQkzs8+qqHBLGkgKM++9997qdZfSU9VHGYAZGKCxEMPlATNDxxK9e0Y8wSSeMtIv2OOg0MoL8IKnmCrnh7ZneDnATJ/ZF3n3QQOAmaAXATOBQhoO8dK84caGwWOKLWBmWL8YYKnCDKXLR7l1Pgcs1XfI+OeHtWXsuFgCZt588011cnLSMGPgFDQwnQYmhZlvvvlG3b17d1X/tMx0584d12aKAZrjD56ZZSb9sUYjx/KAmb5jybwN2MQoBDBTeZdM8FHJtvPCuzPnWFkCZjjG48UXX1QAmzmsBuqQGpgMZggCyMuxNpiR7f3yyy/1NUiFTbUPmOlrgJC/zjgCZnqODXrTL73gLgYmdM55YSJyY2ViaTNDzZIww1BDW4DNVBYDckMNTAozFAy85j96youAbI4/wEzEUMxsAOrgYG3pgJmBYykCIfoDlAQ6+tMG/LK3S+qGG5tYZuL5UUJMbB9gw5rCdgoNAGYatAqYaVCOUvrDmLc++zz+YT432Q80LCg/WK+AmYFjLgIzJo6m/Lq2BtuIt8bloyDhJk/OTOM6F89MDGrgsWmeV3F2mAZmg5mzs7PVPbL9pz/9SX344Ydeux8+fDhM0y2l4JkZaIBmMg5r8s4AZgaOpTqYqcCJ/OK2jbdx32HiJ5uk92Zge0aM7asfXFOvv/66On/+/Gz/deDSlg6PTYtxwOlOGpgNZigWhQz2ml+q949//ENfQyfN9swEmJl/wl8ToPRp6xph5ubNm4pe7dDnOpPnjcCMW2bywELAjF5+CsAlIid5W732VO+da//6W+9BBhkLONV+G7TUnf/lL3+pqP9Tvby159SL7HuigVlhhm6iNf/Ro8gEHVP8AWaqE/LcBmBf6lsTzJDHljwIL7zwQpYwY2JlAlg5I5ixTz1lCjO5LzMBYKawItuWCZjp0f+AGV9ZeM9MngC2FpihVyDQEgP9Yn/jjTfyhJkzs2wk3zOjvTVu6SlcZnqmtvqemTrPC6cDYPz5E0dpNbAYzNBjz+RWXvM/fb7hhx9+SNIj8MzkCQZr9NbkDjO0nEDvIWEjR9tPP/00U5ihcclxMPZpJgcyPGYt0NS+IZjzzbfNxTMDgEliHiCkgwYWgxl6bHuuF9J10MOgLCkBhL4Htbavl8MzM59x6gNVOcMMLTW/9NJLHsjQEtOPP/64PMy0xKH06YOl8y4JMwCYQeYEhUZqADAzQoEpYWZEMxYrCpgBzHQdfAQr77zzjgcx7JmhJSb6WzwAGDDTtTuj+RDEG1ULEmfSAGBmhKIBM/cV3jOTH9Dk5pmhz5q88sorUZDhJSa6DQEz6cbSEp6ZEVMpikIDozUAmBmhQsAMYGbp5YRY/TnBzNWrV/WTSuyFCbe8xES3IWAGMDNiOkbRjWsAMDNiAABmADMxmFg6LQeY+f7779XBwUGtN4ahhpeY6DYEzABmRkzHKLpxDQBmRgwAwAxgZmlwidW/NMx8/PHHjd4YBhna0lNM/AeYAczwWMAWGuirgYJeWpXqjyYj/qMnleSHJvmtk3weTzOxJta7RQBwOuMTg5KhaUvCDAX6vvXWW60eGQIZucREd8EwmDGPTV84Ht8X+v0x+vFq+0I8Cgi+d6Ju3Bsve1hfipfz9QxORszMeudVtHyYBophxeKlADNxvexrKmBmKSPXXO+SMMNjnTwu/EI86YmR+3KJicoNgZn45waa9RMHC/OuGA+KMvgsgX4DceW9Nu3XB5jhkYjtVjQAmBnR01hmwjJT3DC2G5spy+UAM3RbUdzMq6++WuulkUtMlL8/zAz3XFT0HwOXWFpPD0mlnt7lh3meADMjJnYUXaUGADMjug0wA5gZb6zSg08uMEO31ldffaV+9rOfVYAmXGKivH1hxv+sgNWjBhD7pl77Rt7S2xK8yZe/dB2WofQgTX/OgOHm+Jo6x2/7LQpVnivrdXUGcsgzxZ9GMMtaYklLf/OpPO/GFn3/qad3BjAzYmJH0VVqADAzotsAM4AZZ3B6/+JODzHcllxghj5bQJ4Zip8Ll53CJSa6DfvBjAETBgN97QwbMsbFwsSFYwsyDDD8iQI+ri0rYIPBRICFi7OppNHHKatLVyZ/+eFK/R0n2wa5z31Ze10t4w0wM2JiR9FVagAwM6LbADOAGc/otBiYufLmAjP0WvuTkxN3h8llp3CJiTL1g5kqKBhvSgkKnr5jX7eWaT1gxnldqL8dLAk4ZVnHD9QX4ZjgcwxcfKy9PTVtj0CRd21hHWfPFGDGDTvsbEQDgJkRHQ2YAcy0GZUlzucAM9evX1dvv/125e6ip50onbbh32iYObNfrOYlIPa6kLGPLdUwSBBYyH2GgzAtPKZ8XdI0NJVLUEUhvD1nzxR7dzxI4jbobcQL5Z0XIGXTATPh6MLxvmsAMDOihwEzgJklYKWtzqVhhj5fcP78+SiwNN1uKWCm1I3x3Jinpy6pG4vADLdBwEsEfvTyEgGYhC8PVgAzTeMG56AB0gBgZsQ4AMwAZkrjWf11vNS5JWFGxsn0vbXSwgz3h12OunhJhR4R7a0p7NJOBDIqXpcueQhCON+VSOCu9tIIuNHH1IbIspkDmqZzfJ3+Fp6ZvqMP+deuAcDMiB4EzABmlgKWpnqXhJkwTqbP7dUPZsySkhcA7MBAGHYGi3sDAoBDwHCyYvIjaQQzcklJl6flJoaZAFJi7ZdwxHE2DnJEnUEaYKbPyEPefdAAYGZELwJmADNNULHUuaVgpi5Opust1hdmonEwGgjq4lMs0MRiamKgImNwaAkolqclzS0h6ToJYkqA0efEU1D3wyesLKDomBovXz3E8JgDzHQddci3LxoAzIzoScAMYIaNR07bJWBmaJyMvP16w4z2nLCXo93A59RH3dtiAKw+ODh+3YAZObKwvwUNAGZG9HJKmHn8+LF6/vz5iNbMXxSfM4gbku6Gapryc8PMmDgZOWr7w4x9Eqg2cHYa/c7av+Rp6umVofYBZuTIwv4WNLAimDlVR0Whjk7z6ZaUMJNS1lwaAszkaSznhpkxcTJyrA6BGV6a6eu5mBVIgniW7nXTktQwzxNgRo4s7G9BAyuCmfy6IyWApJQ1l6YAM4CZsXEycqwOg5k8+6A7sEzTfsCMHFnY34IGADMjejklgKSUNeKSehUFzExjiMYawrk8MyniZOSAI5j5P//3/+E/gQ7+5cMPFS3/4Q8a2IoGVgQzwTLTw5061MtOJt28HKtQh7uHStlznBYuTZ0eyacdDtVud6SK4lBR0T5/KQEkpaw+1zAmL2BmuzCTKk5Gjr+/ndxWf/rkJv4T6OB//fnPvV9aKPsC+9DA2jSwepiREPJwd2i/znukOLTGpJXHBmTKY3VKIENwA5jpO3gBM9uFmVRxMn3HHPJDA9AANBDTQJHSFUluYv6jL+XeunWLD9Xdu3f1PyfQOcrT/S/umdGeGBZiPTKxNO2dcd4cLmC2BnAAM75W2o8AM9uEmZRxMu2jDDmgAWgAGmjXQLHb7ZKtrS4BM94SUgxWZJr2wgivDOtHpwNmWB1dt4CZ7cFM6jiZrmMN+aABaAAaaNLApmAmXHJyigHMOFX02QHMbAtmpoiT6TPekBcagAaggToNbApmTHwMPDN1g6FvOmBmWzCDOJm+dwjyQwPQwFwa2BbMyCUnoWHjscEyk1BJp13AzHZgBnEynW4JZIIGoIGFNLAtmFFK4WmmdCMNMLMNmEGcTLp7BpKgAWhgGg1sDmZIjf57Zo7UaV1gcIvOU74bJqWslmYnOw2Y2X+YQZxMstsFgqABaGBCDawIZibUAmBmkHIBM/sPM4iTGXRroBA0AA3MrIFtwUw0Zsa8v8Z7N03HTkjpTUkpq2PzR2cDzOw3zCBOZvQtAgHQADQwkwa2BTOkVO2FkZ8zGP4l7pQAklLWTGNHAWb2F2aWiJP5Hx9/rOhdVfgfr4OPru/wOYO5JkLUk4UGtgczCdWeEkBSykp4iY2iADP7CTNLxckQxIz9yCbKmzGJr2Y3Tl04uYcaAMyM6NSUAJJS1ohL6lUUMLOfMLNUnAxgJt14Asz0msqQeQ80AJgZ0YkpASSlrBGX1KsoYCad8UnpUfi3//hPRV+gHvK3ZJwMYCbdeALMdBn9wff+uhSpy+O9RT6h3Lr6VKSO0yNVft4ncr5W1n6cAMyM6MeUAJJS1ohL6lUUMJPO+OQAM0vEycgBlx5mHqjLr/nxcUVhjy+e7PWSFmBGjqy6/YQG34OZuvqmTE94LVM2c0LZgJkRyk0JIN999516/PjxiNbMXxQwsz8ws1ScjBy1U8HMuSsPAnCxkPPaNfXFWZ59OBZuATNyZNXtJwQAwEydkmdLB8yMUHVKmBnRjMWKAmbyNIRDlpmWipORg3c+mKF+O1EXikJdOM6zD/cKZqKvxOAnS4PPyARPm5avzHiodoeFKsp1FPdkqp9UqOJwpx7agWU+VVN650p5lGE4zPhyD9Vud6SKgq+lKtd/UWv8CVpfprxWksd10LXQ9wVFHVa/zuuor1+cFzdZfR0i00p3k8PMo0ePFP2Ty/qTTz5xarl7966if/67deuWzsfHa9wCZu6rW599Hvzq3U/jMNa4zFm+L8wsGScj7/t5YeaZ+uLKgSq85aZwWeqSusGem3vX1LniQF0+pm1pHH0YCsp3lc11JNzm5ZmJgAi/iV2AB782o4QTY5AZYLQhFvmdYXYFTD0MLOY8A4ZSyhp9Pj8UZto/ieODhM7v2kjNOBTgY+6AapqRYdpq9SCuvdp2v87q+Vi9sg55J65zPynMkHEnSKH/3//+9+oPf/iD0wpgxqlib3bgmckT3PrAzNJxMvJmmBtm7h9fUoVbarIgIgBEww6f1zBTiPwWhgoGnrC89PyE52xZlp0QYhia84IZNqTkUeC/0JD6IMK5DOBYIAmWcggSjo6OSk+MhhWGl1C+lahlcDtCAHC11u/UeJkM4Ph1G36hOjjdiq3IqGmra0XsfNj2bsclyDnhe7OTFGakVsg7Q1DDf4AZ1sT+bAEz64aZHOJk5N2wCMwwjBDY8L6DCwEkFmY8Twx7a+49U/flvitvx0eb7DB/guPcYKbiKQjAhL0mwoFhhoZn+KXBpv1DtTvdqUO97MLLVhZUQvk80GrlcYaWrQdDIq9Xn2ynyEO7Op/x7Llr9doU5NeHMXlhWstxax2xeteVBpgZ0V9YZsIyE/8Szmnb1TOTQ5yMvP0WgRnrHdFeGLF85OIPikLpAOIYrMi0KLAYmGmVnQBewvGXH8zYD/xaC649GXLZpNbY+kaaymnvAuXX5em88XxomUwIAhpkX/K+yebLlmOxbt8sB7FnR+RqgBlThpcmj9RpeK36OPDeCNEVENTnwra3HLfW4VW4yoPi66+/Vs+fP0/eePLM7HY7HSdDXpmbN28iZia5lpcVCM/Mej0zucTJyBE8N8zcuFi4mBlvSSkGFxJc+LxMa4OZCZeUQpCh4xxhxnglCARCw1vGszCLuHERGH4NBoc7dUpxJwKMDnenOkDYlffgwkkLdiLtCHJUDrXcHjATtF/LC9PC42ql6qgIg4bDtrcct9ZRqXR1CcVULSZAIlDi5SXAzFSaXk4uYGadMJNTnIwcvfPCjFhCIjjRMHKgLtOSEcOK3Epw4XSZJvf5PG/bZHO+hNssYYYhhuJceGnIDYAOMTOUVxvlI3XEHhqddKiKw0N16MWmhMbdVSR2uuQR2V39IVhwTBB7V4TcGPzoNCnD5K+PZxHyXHPCtG7H9XU4wavdmQxmQo0w1HA6nmZiTax3C5ipMXwJjVLUsLbIb1pmyi1ORo7++WDGBuR63pJImoSQGKx4aWGQrzxukd3Sn0PGQJ4wY5eaaDnPuVDECKgx8n5eY7TLx6BFHIpctmLI8QCH649Ah2hG227lySPb7rJNAiwqHhFuv10us5VVZCr5BJiQ5xoXppnjElbC8yFwkSBZhxO82h3AzIiuQ8wMYmaGGJqpyzTBTG5xMvL2mwpmOE5Cbqsv0iMwtdDhYmeEp8YDFwuxlbSgvHgyqlH2hmCGA2BjLKPHggMDE2NSGudypJgnh8RSjwWGWF4/XoVkinLsKSofsSoradnz5fI7YOKQVMn7MIQqU5mfTwJfFUxicTRGL/yenVgZBhqO35F1tFzwCk7PBjO05EQxNPzo9kcffYT3zKxggDQ1EZ6ZdXlmcoyTkeMrPczk2T9TwyzJz9UzI/sb+9BASg3MBjMUQ8Mv1KMtvYOGtmv+g2cGnpk5DFPfOmKemVzjZOT9D5hJB1+AGTmysL8FDcwGM6EyETMTamR9x/DMpDM+fYGlKX8IM3PEydAPE6pnzB9gJt14WgJmaD5Y7Z9dqpJLkdV9XkZa7VXudcMBMyO6F56Z8Z4Z/Xisi1EQa7lZfzeHYyP47a++EdLX5AWX+udrQeT4UpJvBYUwM0eczLvvvqto8qe66MnFIWADmOk4TjrE2CwBM9T/L7/8snr77bfVqsFmhE1A0eU0sBjMEAjQ5LX2/zUvlZ2cnAwyOjxcU3hmag2/Dq4s3wNSCwAdJvb0ZRlm4u2rvabGtgaPCjfmbTZ6EmbmipNhmJG/ZvuCDWCmuV/7jOOlYEb2P8CGZ0ps59DAYjAzx8WhjmYN0GQz5tf0pDBDxlw+GjvCuPcxAt3yWph57cB8fDB4N0kuMDNnnEwMZqRh6wI2ecIMQaZ4qskbh03n0oFJtzHp15cDzMj+B9g0z8U4O14DgJnxOlytBIYZOel0MTp8wZPDDD8q6z3iyh/4K5ekysds2bthtnxd3vd0yBix10csb4V5wlfQl3WQ0bAwc/Ek8uXlZ6oOZmplhu0ZskQljCx5Zv78l2P16quvzhZk3wYz3Be0rRtjOcKM7ssamGk6NwRAUpbJDWZk/wNseAbFNqUGADMptbkyWTGYkZNOndHhy5weZqpgYIBA/FK2IGBggyGmPF+X34MX7QEqXLxKXZkSaEqYuX/GAFX+Mo7BTLvMqpyhxo1g5uc//7n6zW9+497AzS+tnGr7q1/9Snv55Pjpsi/HWF4ww2OJoLkcT6ZPms6V42Bo/6Uo9+FvP1J37tyZrf9pXHXp7zAPwIZnU2zHagAwM1aDKy7fBjNy4pFGhy95DpgxEMCBtsaIlFBhDYeGEcoTOW9hx8GLy1tndCIyyOvhlZMwYz1FwptShZkuMk0e107haelr3N69tlM//elP1S9+8Qt1/vz5Wf77jCU5rmj/hRdeUG+88YaOn+t7rZPlp/4mj6AePwHMNJ0b0W8pr+XqB9fU66+/Pkvf8xgL+7XPMXkRKYYPf9DAUA0AZoZqLnG5zz//XN/MU/1yjsl96aWXBv2aYrD55z//qW599nn8WzYdJ/Wq4fchw4MZDRSBYaF6HLDEgCBMM8dFwYDk11cbp+PqoPw+zPAxQ1blmlrbTTLDdgbt6qhPMojkmbl06b/op0oSD9NacX2WmSTAfPrpp+rHH3/UcvPyzFj9x2CG+6LpHOdZaJvzMhNDDgEMBaiv+SGK2hsCJ2bXAGBmdpXHK6Sbmn6d8q+cObb0i5gnlr5bat+vf/3ryWHGAwMNBWWsTNjmC8cxIIilWRiJxcy01hGDGd9z47WZjFknmbF2DgMafpqJoHOuX7tdYIY9MBJg5N0AmBnW3zGPTq4wA4CRIx77KTUAmEmpzRGylnhnTd+lAQKYjz/+WJ2dnekrnX6ZKfCA1Hk43K/fGBDE0nyjoeGD4yJa66iBmTMT30PemTjMRDxKrt0ks72dMaMVS2OYmeNleTzk62DmxRdfVG+++WYnqALM+OMy1rdd03KCGQAM3yXYTqkBwMyU2u0hO1eYCQFGXtLkMFMBizaDHzsfSwuNhswj98N8fBxAFkOJXXY491qhChFD0w1UutTL9TdvGWaor+Z6PFvCTB+AkeMJMNPcr11BhvItDTMAGDmysT+HBgAzc2i5Qx05wQzHxLS9xXVSmLExKhyHwhN55akg6xExT5zEgCBIqwASLwOVnpPmOsjg1MCMa0sIM/w4eVkHXY/nEYoFLzMk9dxKmKGhN8eL86iOrh6YutsBMLNumAHA1I1spM+hAcDMHFruUMfSMNMVYOSlJIMZEbsi42DqnuwxsCFjZziYNwAXDQGRNA00srwPGQQa9XU0wwx7YXzPjDFSzTIZbqogxCDXdRvCDPXZnPEzcoz02QfMrBtm+vQ18kIDqTVQ0Acf+WmC1MIhr7sGloCZt956a/B3dOjKUsBMVwONfN0NXQxm5oyf6T7q/ZyAme593HY/LLHM5PcmjqCBeTVQ7Ha7Ud/nmbe5+1vbEjAzVpuAmXTGp8049Tkfgxnq67niZ4aOqyxhpucSX59+mjIvYGboKLFW/YEAABm3SURBVES5tWoAMJNJzwFm8gSDKQ3OVLLrYIaG+hzxM0NvKcBMunsAMDN0FKLcWjUAmMmk5wAz6SbyqSBhLXKbYIaGe67xM4CZdPcAYCaTiR3NmE0DgJnZVN1cEWAm3US+FuiYqp1tMJNr/AxgJt09AJhpnm9xdv80AJjJpE8BM+km8qkgYS1y22CGhnyO8TOAmXT3AGAmk4kdzZhNA4CZ2VTdXBFgJt1EvhbomKqdXWCGRmNu8TOAmXT3AGCmeb7F2f3TAGAmkz4FzKSbyKeChLXI7QozNPRzip8ZBjPmBYZ17yTq02fle4DEe4funagb95Yam/SOJNGWHk9WAWYymdjRjNk0AJiZTdXNFQFmljIY+1dvH5jJKX5mCMxoALl4MurL7QZ4Ii9XzOGr2PSCR+/TGN3GK2Cmeb7F2f3TAGAmkz4FzHSbpPv80t5q3j4wQ8Of3hd0cHCw+Msz+8PMcM9FZWzEwCWW1sM7UqljUNlhnifATCYTO5oxmwYAM7OpurkiwAxgJo3xe6b6wgyNzJs3b+pvKzWP0mnP9oUZ7ZUJvRb2m17xz2LYb2rx5zPYoxOWofQgTX8jjOHm+Jo6xzKKQpXnyk9kuGWvQA61i783Zpa15DKS8Q7xeTceBnhnADPTjlVIz08Dk8HM6VF5Y8uJpSgO1e5hfoowLTpVR0Whjk65feExp6ffAmYAM854DfoFX+pvCMzQiKYPRRLULPXXD2YMmHiGn2FDxrhYmLhwbEGGASb8WGhtWQEbDCYCoFycTSWNvhdWXboy+flbYvZ7XLZN+sOjrn1lfxqwEu3oMD4AM0uNYtS7lAamhZnDnQq55eHuUOULNPPBS9jhgBkxeXeYrFMZ/n2UMxRm6Btt9OXj77//Phyesxz3g5kqKBijX4KC17f646LBOZnWA2ac14XGqYMlMX5Z1vED9UU4lvkcAxcfa29P0D5XNnKt7pyoV6QBZmYZsqgkIw3MDjNKGWA4zNI9sxzMfPnll4sZkaHjEd9mihsSz4gKAzNX+lCYoXFAIENAs8THZ0fDzJn48jgtA0kvR2yphkGCwELuc5+FaeEx5euSpqFJeqp9Lwt7dzxI4jbobcQL5Z2vjkPAzNBZDeXWqoHFYKZcyjGqMx6b8ob3YYchw2x52SqUQZL85a1DtdsdeZ4g/7xcUtKlF1tmWuMAAsxUjchcwNJUzxiYoXG4VPxMCpgp9WK8GWauuKRuLAIz3AYBLxH40ctLIXx5sAKYWeP8iDbPq4HZYUZDS0AhlaWnhzt1WBSqBBqGmDLeplLGgcyRKkNeCGQIkEw5DTKi7qoMhibuhPCY07ElDQBm9hNmqG+XiJ9JCzPcN3aJ5uIlVYTvbBm0zCTAhIAjAicu7UrksWpdp5Dh2tC0lNR0jq/T38Izgzl6axqYFmZExD97U/TWi6UxwFCCi+2CUwIRBpNIHgs8jk3CYyeGYYZklDCkT1fKhPASHm9teDRfL2DGNyClV2DZ9LGeGer1JeJn+sGMWVLyAoAdGAj9O9gYEAAcBvA6WTH5kTSCGQlQujzNRwwzAaTE2l8HTJ7nRtRt0wEzzXMXzu6fBqaFGQ9arPI0pBSqYArRxwFkUFYPNGJQEaR58CM6qk4+t6Px6aWgDiEWu/DM5AIvYTtSwAyN77njZ/rCzH0y/uIpIq0HDQTlcnUJDmTwLdDwjywZUxMDFRmDQ3ljeVrS3BKSrpMgpgQYfc5rfwhcBlJ0TI2Xrwov4RgAzGCG3poG5oeZcDlIQIXnvbETjmGeGFT4aWbJiD05ohsFzJg8PNEdqVMPmKiML7N6LORiF8tMLb+OQwMz13EqmKEhPmf8TG+Y0Z4T9nK0G/i59J+2HgM49cHB8esGzGCC3poGij/+8Y/q6dOnya9bx6fEPDPa6UKPZ1vwELBR34gQMihnkKblNMDMqYnDYYeQrgswU6/yDmewzBQ3JGmNWf86UsIMDYO54mf6w8wzpb0W0sOSKWAOHhMx71OHawTMdJjAkGWvNFBMdTVNMOOfC6Ak2qBYniCtAiZGkAvy1U81BbBjvUIl4AQyQ2CKtm27iYCZ/qAx2Kh1MGAsOzXMzBU/MwRmeOmor+eCdZX3lpakhnmeADPbnZe3euXzw0wFICg8pvoiPQ08LmA3hAzqrmqaKSOAxdaln2aqeGZMeVraKoOPQ5nh8VaHSfy6ATPbgBnq/TniZ4bBTJ59sDQkAWbicxZS91cD08IMB9p520iwr1t64ngW2gooiYBLDGaomwzQsJwjdSqWn/yYGfG4di00AWaahj5gJk9Dmtozw2Ng6vgZwEy68QSY4VGL7VY0MBnMZKNAATPZtGlPGgKYSWd8Uv6SnwpmaNhOGT/z0fWdIqDB/3gdXL36wSJvcd6TqQ2XsUIN7A/MRGNmjGelXEZaYQ9l3GTAzPZgZq74mYyHPZoGDUADGWpgf2CGlOtiZHiZKfxcQYY9sOImAWa2BzM0XOeIn1nxbYGmQwPQwAIa2C+YWUCBW64SMLNNmKExP3X8zJbvK1w7NAAN9NcAYKa/zlDCagAws12YoSEwZfwMbjJoABqABvpoADDTR1vI62kAMLNtmEH8jHc74AAagAYW1MDKYAaPSi84VipVA2a2DTM0IBA/U7ktkAANQAMLaAAws4DS96VKwAxghsZyqvgZAqO7d+/iP4EOvvnmm32ZZnAd0EAnDQBmOqkJmWIaAMwAZnhcpIifoffLfPrv/xv/CXTwX69+oJ48ecLdg22uGjg9UuXndEY0Uj/Jyy+knWsFI6inci3B+RGX16XoIJjx37J7qHb6u0dSkZzmv8nXLxc8Nq3fE3OodvazA/wFbf8dMVY5O/PRSM4TGwz+237lJwv064bVYUH1G3ksR9dl31fDaaHsRrldNF6T57vvvlOPHz+uOZtnMmAGMMMjM0X8DN4AnG484Q3APDJz3iY09h7MLHHNCa9lYPN7w0zj948eUissIARfzNblBBlUvsckIMJls++Ncccs231+IP5dpzrZDoxcXQxgLMeHLyOn/KxCq9yBnUDFbt++rQgO1vQHmElnfNbyBuCm8Tk2fgYwk248AWaaRmou5xICAGBG9YMZCwElXJhBYQCHwcB0kAMHnYXS+LwdSKEse+yXs5DhwCgiO5RjgSeUY16oZ8EkVldDmrneSN10KXoQlcBjr673BjCTbiJPCQZrlDXl5wzaBvaY+Jn0MPNAXX6tfIEme1v19uKJWmPfdm1zjjBjfgyK/qgaEiX7yJ/DzfwbetNDEcrO41JOJY/9kcx5BtVT+Q5gsNLgvido2s11ubaE7XQ2ru0OM+d9XfJKCNtY1pUvq3FlxGb15RaqcA0mmVwP/+i39QQrJYW7lmo76uX7bR1y1A9m6gy3R4XVC6g0TAwmpyvbue6YC7XKDurz8rMQubQU7HOWWP0yrYtcljVgC5gBzHQ1VG35loQZGvpD42emgplzVx4E4GIh57Vr6ouz/Rx3ucGMMWJsbGmUmHnbgYS1CeX8b877xpSMaCmjIlPO1zwHh3LDY/b2u4ptvU31MMi4MuzZL9vG19fYXgc83Nhu266rI6J55gPMIqGiO/ex5+o1mD6yenGgUvahEWvOiypcH3NatU5Txo2Bbpdfm6sXzJjGRLwQnqGPXRR3NlP5kToNB54+loq0bW6VHdRnByuTcLjVig3rpqra0rrIrVVz+wnAzH4alTbwmOL80jDTFj9D52N/88EMjbUTdaEo1IXj/Rx3ecFMm9F6qHaHQVwjDZDI3O8ZvnDO1vkj9skNtkT1aAgJbFXYlhDWqA01edjYu2Y27VRkmMyx1ZFSLul/QHu9dsT6UNpeuc8FZVqsPOdLs+0FM7VLKpFBVyoy1omRNN1JgcLpGttkh3Tr5a9RUmxAtKV1kVtTXZdkwMx+GpUpYKVN5tIwQ+O9Ln6GHhl+6aWX1KNHjyq3xbww80x9ceVAFd5yU7gsdUndYM/NvWvqXHGgLh/Tln+UhTAUlO8qm+tIuM0KZmJzq+z9uvNeujSMXDhMM8dFUQM0njyWEdqiUCbli6WJ8to2mDFR2r1YmTAtPBYy63Z1XZHr8+xTi9xYe+t049oRkynT5D4XEmmt8rnM8G3x7bffqufPn3eTUNMg330kLoClxjrAKtR1vpXtkbdmmUI1rcFVB1qkfm4Hb2PX0ZrWQS7LH7AFzABm2iCl6/kcYIZugTB+5urVqy4mgvbDv7lh5v7xJVW4pSYLIgJANOzweQ0zhchvYahg4AnLS89PeM6WZdkJIYbHSH4wE/mhygMgNvfqc3LOlftcMJZmvC/SIx/aGHfMYjxYicmsphmbx1AbWWnwZHJFoZzwmPPVb4eujrS2V/dBQx+1Xk/sWkRaq/z6a+56ptjtdr3eRzBkva7OvUYDzsGLHdByjdF4ZWRglVCOu8Jqmg9XJqPnhovdPB3SWuW6NvXfAcwAZtgQjd3mAjN0F1D8zO9+9zv1+uuvO5Ch+/7g4KBykywCMwwjBDa87+BCAImFGW9Zir01956p+3Lflbdjuk12mD/BcX4wI+fxoOtjcy9l8dKr83z1h2wgl38M8xKLJ0/k9dI71OPlt3IqaR3kRAFBtCu2G3MMUD6dzjAS1F1pW6jbyHGl7kCmPi/T5D4XFmmxNnC2RNveMEP1GjAQVOopWFyAaKRPhkbpUcAIIqN9io7JjqWFMTrUVuGaiym2Y5p/HYFccb19dwEzgJmxEMPlc4KZr776Sv3kJz/xQIZ/NYdLTYvAjPWOaC+MWD7iNtJWBxDHYEWmRYHFjOlW2Qnghfuet1nBjDXa7odrZXLsHsvSbg9C4dI+JKrHs3e2Pp0mgU3Wy20K08JjztewjdkpzSKHIjg6kNujvfV9FMjUTZRpcp/bL9PMfr18LjN8OwhmKtXFlFXJ1JJQ00ktpfbmNGAGMMOGaOw2F5iRy0oSDng/XGqaG2ZuXCxczIy3pBSDCwkufF6mtcHMhEtKsfGSF8zwj0v2HNC0bZeDmE5qYCB8momzm4lfGsvQO2FNg5Yr6k1RT8VWmXbQuC6NddA23ZwwzRyXZWybWzaVFQJ7TeWqRlBPp/a29VEgs3I9sWvxy1TaHY6BlutuO90PZipKIfGxi2irNnI+KjuSb0+TADOAmZhRGpK2NMzQa/TDZSUGGLkNl5rmhRmxhERwomHkQF2mJSOGFbmV4MLpMk3u83netsnmfAm3ucEMTdsVr7ZPJnaphD3+EgyotG8YjRmIpDnDznIEyLDtCPL4MBGRGanbv5bISkOkTOwa3CqH98gzN7R+W6m/8hZ+6SUKdR9rr6nLlxu+Z8aXGV5P9VqquqyXX3+tXc/0gxmSGgwEmpzCMdm1ci8fYAZvAE44mUcN0kbkLwkz9Nj1K6+8El1WkiDD+3KpaT6YsQG5nrckkiYhJAYrXloY5CuPW2RPMC5zhBlvvscBNJBYA/1hJnEDIM5oAJ6Zml/EE0z0+w46S8IMjeazs7NOnhkCGrnUNBXMMDjJbfVFejT+LHS42BnhqfHAxY7VSlpQXjwZ1Sh7gjEOmIFl2ZoGADOZ9DhgBjCTCrKWhhm+pa5fv65eeOGFRi+NXGpKDzPbHVOAGR6FK9nalQkJ3NX9yJLZSi5vjmYCZubQcoc6ADPbNTypIIbl5AIzNOzp5XkELNWJmWMaCvcCPcBMunsAMNNh0kWWvdLADDBTDQLaKw0muhjATLqJnI36Vrc5wQzfHk1PNvFSE2Am3T0AmOGRh+1WNDADzGxFleOuEzCTbiLfKsTwdecIM3R30KcMYsHBvNQEmEl3DywBM9S37777rvbGjZsNURoa6K8BwEx/nU1SAjCTbiJno77Vba4wQzcOPe30zjvvVJad6KkmwEy6e2AJmJFLiQCbScwEhDZoYAaYCZaZ3CPYJp1vAP2sfxAEFT7y7Z5jt08bhOfpOv08h2rnPX9vNBE+6+6/Z6B8tTO3jbaxuhr02vsUYCbdRL5ViOHrzhlm+Ma4e/eu/uAk32O01ASYSXcPLA0z3K+0BdjwqMd2Sg0sBjPl2wrlC33KTw4Y4CiPNaQIojDn/ehuAzJlmfKdOGW+SjkLUA5oHGwJtdt364jqxck0u4CZdBM5G/WtbtcAM3TX0Mv16NtNZPBoqSlLmNHvmimDlaWR9r7TNMHj1WPGb04wI3UGsEljLyClqoHFYMbBA7UpBAqRZgCCvDglkOjLCKEjPLbXagCHyxpvkFc35dOwYiFI7lsZc2weP37c/evlczSoQx33799Xtz77PP7G1Mwm9zGGYW1l1wIzPMROTk7Uiy++mCfMRMax/gyC98K9/EA8V5gB2PCoxza1BhaDGc/LEQORWBpfvXgLsZNTByE63cKM3GdZtPXq4uUv4eGRebHvNACYyc+IEXitDWZoQNGL9rL0zIQwI98KHJ7L6PjqB9f0iwvPnz+v5vqXoNJ3nzw2N2/edHMLdqCBvhpYDcz4cS5H6tQDEF6qigCIBBi9H3cZ083nwIg/gOXeBCrP9VXx/uYHzABmUo7u/GFGfqIgz75nL+KHv/1I3blzR1Fs0lz/fQGG8r/88svq7bffXt2nXFKOe8hKo4F1wEwALvrSwzQNKl1ghpecuivQX6rqXm7fcwJm8jRoa/TM0L2SPcysxCtDQJPzMhMAZt8twzLXtw6YiYGK9bI4b0oIN1affsCvWUJyZTrrfGi5zhWsMiNgBjCTcuDmDjM6Vsb73lKe/Z8jzABgUt4pkBXTwDpgpgIqHNfifyZ+0NNM7nHuhrgaDU79PToxhe9TGmAmT2MGz8wU/XKiLhSFyvkJJl5iygVmADD7NNvnfy0Fvaxq2r/Aq1EBkzAA17YmyOfHzBiwiC3/mDSOizlSpxGvji+L8gbLU9brU64BA2RiYwQwM4XRHC8TMDNehxIM9L5eYrqkbmQU5Ftpo2jbUstMAJjYTIm0OTRQzFHJonVEYGbR9uxR5YCZCYymMEhNxqrpHGAmfb98ceVAFZk/ji3HxBIwQ/MB/qCBpTSwPzATeHKMQo1XqPJemaW0vWf1AmbSG01pkIbuA2bS98ua4mVo3CwBM3s2veFyVqaB/YEZUnxleQiPVE85HgEz6Y3mUICR5QAzqfvFPJJ97sqD1bwgEjAz5cwH2TlqYL9gJkcN73GbADOpjWYaeYCZNHqUgLi2fcDMHk+8uLSoBgAzUbUgsYsGADN5Gk3ATJ79MicQAWa6zGDIs08aAMzsU2/OfC2AmTyNJmAmz34BzMw8QaG6TWkAMLOp7k57sYCZPI0mYCbPfgHMpJ1/IA0akBoAzEhtYL+XBgAzeRpNwEye/QKY6TW9IDM00EsDgJle6kJmqQHATJ5Gc1swY540SvFmXv0uGf1x2QN1+Z7t23sn6gbvJ3gHUD+gobcOi7b0qB8xM3Kmwv4WNACY2UIvT3SNgBnATMqhNeTbTBpAknwvKfK5gnvX1LmBMNEPWhrGEb15eMDL+gAzKUcmZK1BA4CZNfRSpm0EzDQYoR6/opMZPlvndjwzwz0XFZ3HwCWWNnu/DvM8AWYynTTRrMk0AJiZTLX9BD99+lQ9f/68X6GFcwNmADMph2Bfz0z0EwMaQPjbbGZbLkEZMHDfXGOPTliG0oM0/cI8hptj8tiUdZTnyjRXZyCH6uaX75llLbmMZLxDfN4B1wDvDGAm5ciErDVoADCTSS/dvn1bERys6Q8wA5hJOV77wUzkrbwMGzLGxcLEhWMLMgwwZ8FxbVkBGwwmYtnHxdlU0uijlNWlK5O//GCl/EyC3HcgQ56gWNtaPESAmZQjE7LWoAHATCa9BJjJEww8o9JiQHLJu41lpiooGKNfgoLXH7GvXsu0GDCEaRZmnNfFgUahqmkH6vLxA/VFOGaiMikveXtq2h6BIu/awjrwbaZMZnU0Y04NAGbm1HZDXYAZwEybgep6frMwc/ZMae8GLwE5L8wzdT+2VCPBQu4zHIRp4bGDGeG9iaVpaCqXoIogqJi9Ox4QcRv0NuKF8s5X7x14ZhomW5zaSw0AZjLpVsBMdULuaryRz9fdlmGmHAvGc2PiYy6pG4vADLdBwE4EiByASfjyYAUwk8k0jWZkrAHATCadA5jxDXJplJDeVxeAGTlm7HLUxUsq9Ihobw0v7UQgoxKr0iUPQQjnuxJ5rFp7aQTc6ON4fE3Z75ElNQ925PWafXhmMpnY0YzZNFDQUzT4W14DgJnqhFxO5jjXRxfbgBmzpOQ9+ePAQIwXBot7QcBvlwDgMFbFyYrJj6QRzMglJV2elpsYZgJIibVfwpEMbAbMLD9powVZaaDY7XYKQLN8nwBmhDFomaj7GPYt5t0KzETjYDQQ1MWnWKCJxdTEQEXG4LjHtRlE7HiNlRNpbgnJvVm4BBh9TjwFdT8ELHsf6JgaL1/7vQLPzPJzOlowrwY0zDx58mTeWlFbRQOAmfYJeotgMuSaNwMz2nMSwMXegbABsPrg4Ph9A5ipTLFI2HMNAGYy6WDATHxSHmLMt15mOzDzTGmvRW3g7B6MKfI09fTK0PgHzGQysaMZs2kAMDObqpsrAszsgeHJxCuwJZjhpZm+not1AC8tSQ3zPAFmmudbnN0/DQBmMulTwAxgJpWB3RbMYNzExg1gJpOJHc2YTQOAmdlU3VwRYAZGKWaUhqQBZjCWADPN8y3O7p8GADOZ9OlaYeZfPvxQffw/P8F/Rjr46L/9d/W3k9uZjOzuzaBvM2EspbmXfvP++woPdnQfe8i5fg0AZjLpwzXCDH3l+9GjR/jPUAdrfN3C2dkZxlKisfTDDz9kMrOhGdDAPBr4/22/IpAnBHtfAAAAAElFTkSuQmCC"}}},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import os\nfrom tqdm.auto import tqdm\nimport time, gc\n\nimport numpy as np\nimport pandas as pd\n# pd.set_option('display.max_columns', None)\n\nfrom matplotlib import pyplot as plt\n%matplotlib inline\nimport cv2\n\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.models import Model, Input, load_model\nfrom keras.layers import Dense, Conv2D, Flatten, Activation, Concatenate\nfrom keras.layers import MaxPool2D, AveragePooling2D, GlobalAveragePooling2D\nfrom keras.layers import Dropout, BatchNormalization\nfrom keras.optimizers import Adam\nfrom keras.callbacks import ReduceLROnPlateau, ModelCheckpoint, EarlyStopping\nfrom keras.initializers import RandomNormal\nfrom keras.applications import DenseNet121\n\nfrom sklearn.model_selection import train_test_split\n\nstart_time = time.time()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Setting execution conditions"},{"metadata":{"trusted":true},"cell_type":"code","source":"### Kaggle or Local-PC ###\nKAGGLE = True # <==== SET ============\nif KAGGLE:\n DIR = '../input/bengaliai-cv19'\n DIRwork = '../input/mywork'\nelse: # local PC\n DIR = './bengaliai-cv19'\n DIRwork = './'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Train or Predict ###\nTRAIN = False # <==== SET ============\n# True : Do training (GPU time limit over), model save.\n# False: Load trained model and do prediction.\n\nif TRAIN:\n print('==TRAIN mode==\\n Training Model takes several hours and does not meet the GPU time limit for competition.')\nelse:\n # Check file existence\n if os.path.isfile(os.path.join(DIRwork,\"dense.h5\")) \\\n and os.path.isfile(os.path.join(DIRwork,\"hist.csv\")):\n print('==Not TRAIN mode (Predict only)==')\n else:\n print(\"==ERROR==\\n No 'Model file' and/or 'history file'.\\n Upload the files to '../input/bengaliai-cv19/mywork'.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Read Data Files"},{"metadata":{"trusted":true},"cell_type":"code","source":"# read CSV files\ntrain_df = pd.read_csv(os.path.join(DIR,'train.csv'))\ntest_df = pd.read_csv(os.path.join(DIR,'test.csv'))\nclass_map_df = pd.read_csv(os.path.join(DIR,'class_map.csv'))\nsample_sub_df = pd.read_csv(os.path.join(DIR,'sample_submission.csv'))\n \n# read Parquet Format Image file (example)\nimg_df = pd.read_parquet(os.path.join(DIR,'train_image_data_0.parquet'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(train_df.shape)\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(test_df.shape)\ntest_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(class_map_df.shape)\nclass_map_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(sample_sub_df.shape)\nsample_sub_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(img_df.shape)\nimg_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`Column['0': '32331']` of img_df has flattened image size `137 * 236 (= 32332)`"},{"metadata":{},"cell_type":"markdown","source":"## Observe each data"},{"metadata":{"trusted":true},"cell_type":"code","source":"# target columns\ntgt_cols = ['grapheme_root','vowel_diacritic','consonant_diacritic']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_top(tgt, n, top=True):\n top_df = train_df.groupby([tgt]).size().reset_index(name='counts') \\\n ['counts'].sort_values(ascending=not top)[:n].copy()\n top_ids = top_df.index\n top_vals = top_df.values\n \n top_df = class_map_df.iloc[top_ids].copy()\n top_df.drop(['component_type', 'label'], axis=1, inplace=True)\n top_df['count'] = top_vals\n return top_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def disp_img(df, ids):\n r_n = len(ids) # character count\n c_n = 5 # num of examples for each character\n plt.figure()\n fig, ax = plt.subplots(r_n, c_n, figsize=(12, 10))\n for r, id in enumerate(ids[:r_n]):\n sumple_ids = train_df[train_df['grapheme_root'] == id].index\n for c, sumple_id in enumerate(sumple_ids[:c_n]):\n flattened_image = df.iloc[sumple_id].drop('image_id').values.astype(np.uint8)\n ax[r, c%c_n].imshow(flattened_image.reshape([137, 236]))\n ax[r, c%c_n].set_title(str(id)+'(Train_'+str(sumple_id)+')')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Summarize training data"},{"metadata":{"trusted":true},"cell_type":"code","source":"desc_df = train_df[tgt_cols].astype('str').describe()\ndesc_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Number of unique types\ntypes = desc_df.loc['unique',:]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Top used Grapheme Root"},{"metadata":{"trusted":true},"cell_type":"code","source":"top_roots = get_top('grapheme_root', 10)\ntop_roots","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Top 5 example\ndisp_img(img_df, top_roots.index[:5])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Top used Vowel Diacritic"},{"metadata":{"trusted":true},"cell_type":"code","source":"top_vowels = get_top('vowel_diacritic', 10)\ntop_vowels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Top 5 example\ndisp_img(img_df, top_vowels.index[:5])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Top used Consonant Diacritic"},{"metadata":{"trusted":true},"cell_type":"code","source":"top_consonants = get_top('consonant_diacritic', 10)\ntop_consonants","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"disp_img(img_df, top_consonants.index[:5])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Multi Output DenseNet Models\nPrepare DenceNet models with three-outputs (for 'grapheme root', 'vowel diacritic' and 'consonant_diacritic')"},{"metadata":{"trusted":true},"cell_type":"code","source":"SIZE = 64 # input image size\nN_ch = 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def build_densenet():\n densenet = DenseNet121(weights='imagenet', include_top=False)\n\n input = Input(shape=(SIZE, SIZE, N_ch))\n x = Conv2D(3, (3, 3), padding='same')(input)\n \n x = densenet(x)\n \n x = GlobalAveragePooling2D()(x)\n x = BatchNormalization()(x)\n x = Dropout(0.5)(x)\n x = Dense(256, activation='relu')(x)\n x = BatchNormalization()(x)\n x = Dropout(0.5)(x)\n\n # multi output\n grapheme_root = Dense(types['grapheme_root'],\n activation = 'softmax', name='root')(x)\n vowel_diacritic = Dense(types['vowel_diacritic'],\n activation = 'softmax', name='vowel')(x)\n consonant_diacritic = Dense(types['consonant_diacritic'],\n activation = 'softmax', name='consonant')(x)\n\n # model\n model = Model(input,\n [grapheme_root, vowel_diacritic, consonant_diacritic])\n \n return model","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Training Model takes several hours and does not meet the **GPU time limit** for competition. Therefore, this Karnel uses a pretrained-Model.
If you want to Training yourself, set `TRAIN = True`."},{"metadata":{"trusted":true},"cell_type":"code","source":"if TRAIN:\n # build model\n model = build_denseNet()\n \n # compile\n model.compile(Adam(lr=0.002),\n loss={'root': 'categorical_crossentropy',\n 'vowel': 'categorical_crossentropy',\n 'consonant': 'categorical_crossentropy'},\n loss_weights={'root': 0.333, ## Set weights\n 'vowel': 0.333,\n 'consonant': 0.333},\n metrics={'root': 'accuracy',\n 'vowel': 'accuracy',\n 'consonant': 'accuracy'}\n )\nelse:\n # Load pretrained-Model\n model = load_model(os.path.join(DIRwork, 'dense.h5'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training\n### Prepare Training Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Clip horizontally\ndef clip(img):\n cols = np.any(img < 200, axis=0)\n xleft, xright = np.where(cols)[0][[0, -1]]\n width = xright - xleft\n center = int((xleft + xright) / 2)\n \n if width < 137:\n img = img[:, max(0, center - 70):min(center + 70, 236)]\n else:\n img = img[:, max(0, xleft - 2):min(xright + 2, 236)]\n \n return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Resize image size\ndef resize(df, size=64):\n resized = {}\n for i in range(df.shape[0]):\n img = clip(df.loc[df.index[i]].values.reshape(137,236))\n image = cv2.resize(img,(size,size))\n resized[df.index[i]] = image.reshape(-1)\n resized_df = pd.DataFrame(resized).T\n return resized_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if TRAIN:\n # prepare X\n img_df = img_df.drop(['image_id'], axis = 1)\n X_df = (resize(img_df, SIZE) / 255.).astype('float32')\n del img_df\n gc.collect()\n for i in tqdm(range(1,4)):\n img_df = pd.read_parquet(os.path.join(\n DIR, 'train_image_data_'+str(i)+'.parquet'))\n img_df = img_df.drop(['image_id'], axis = 1)\n img_df = (resize(img_df, SIZE) / 255.).astype('float32')\n X_df = pd.concat([X_df, img_df], axis = 0)\n del img_df\n gc.collect()\n \n X_train = X_df.values.reshape(-1, SIZE, SIZE, N_ch)\n del X_df\n gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if TRAIN:\n # prepare Y\n train_df = train_df[tgt_cols].astype('uint8')\n for col in tgt_cols:\n train_df[col] = train_df[col].map('{:03}'.format)\n Y_train = pd.get_dummies(train_df)\n\n del train_df\n gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if TRAIN:\n # Divide the data into train and val set\n x_train, x_test, y_train, y_test = train_test_split(X_train, Y_train,\n test_size=0.1, random_state=42)\n y_train_root = y_train.iloc[:,0:types['grapheme_root']]\n y_train_vowel = y_train.iloc[:,types['grapheme_root']:types['grapheme_root']+types['vowel_diacritic']]\n y_train_consonant = y_train.iloc[:,types['grapheme_root']+types['vowel_diacritic']:]\n y_test_root = y_test.iloc[:,0:types['grapheme_root']]\n y_test_vowel = y_test.iloc[:,types['grapheme_root']:types['grapheme_root']+types['vowel_diacritic']]\n y_test_consonant = y_test.iloc[:,types['grapheme_root']+types['vowel_diacritic']:]\n \n del X_train, Y_train\n gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(time.time() - start_time,'[sec]')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Fit"},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_size = 128\nepochs = 30 ## set epochs","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Callback : Learning Rate annealer\nreduceLR = ReduceLROnPlateau(monitor = 'val_root_loss',\n patience = 2,\n factor = 0.5,\n min_lr = 1e-5,\n verbose = 1)\n# Callback : Save best model\nchkPoint = ModelCheckpoint('dense.h5',\n monitor = 'val_root_accuracy',\n save_best_only = True,\n save_weights_only = False,\n mode = 'auto',\n period = 1,\n verbose = 0)\n# Callback : Early Stop\nearlyStop = EarlyStopping(monitor='val_root_accuracy',\n mode = 'auto',\n patience = 4,\n min_delta = 0,\n verbose = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if TRAIN:\n history = model.fit(x_train,\n {'root': y_train_root,\n 'vowel': y_train_vowel,\n 'consonant': y_train_consonant},\n batch_size = batch_size,\n epochs = epochs,\n shuffle = True,\n validation_data = (x_test,\n {'root': y_test_root,\n 'vowel': y_test_vowel,\n 'consonant': y_test_consonant}),\n callbacks = [reduceLR, chkPoint, earlyStop],\n verbose = 1)\n\n del x_train, x_test, y_train, y_test\n gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if TRAIN:\n df = pd.DataFrame(history.history)\n df.to_csv('hist.csv',index=False)\nelse:\n df = pd.read_csv(os.path.join(DIRwork,'hist.csv'))\n \n# Plot the loss and accuracy curves for training and validation \nfig, ax = plt.subplots(1, 2, figsize = (12, 4))\n\nax[0].plot(df[['root_loss','vowel_loss','consonant_loss',\n 'val_root_loss','val_vowel_loss','val_consonant_loss']])\nax[0].set_ylim(0, 2)\nax[0].set_title('Loss')\nax[0].legend(['train_root_loss','train_vowel_loss','train_conso_loss',\n 'val_root_loss','val_vowel_loss','val_conso_loss'],\n loc='upper right')\nax[0].grid()\nax[1].plot(df[['root_acc','vowel_acc','consonant_acc',\n 'val_root_acc','val_vowel_acc','val_consonant_acc']])\nax[1].set_ylim(0.5, 1)\nax[1].set_title('Accuracy')\nax[1].legend(['train_root_acc','train_vowel_acc','train_conso_acc',\n 'val_root_acc','val_vowel_acc','val_conso_acc'],\n loc='lower right')\nax[1].grid()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(time.time() - start_time, '[sec]')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Predict and Submit"},{"metadata":{"trusted":true},"cell_type":"code","source":"row_ids = []\ntargets = [] # prediction result\nid = 0\nfor i in range(4):\n img_df = pd.read_parquet(os.path.join(\n DIR, 'test_image_data_'+str(i)+'.parquet'))\n img_df = img_df.drop('image_id', axis = 1)\n img_df = resize(img_df, SIZE) / 255.\n X_test = img_df.values.reshape(-1, SIZE, SIZE, N_ch)\n\n preds = model.predict(X_test)\n for j in range(len(X_test)):\n for k in range(3):\n row_ids.append('Test_'+str(id)+'_'+tgt_cols[k])\n targets.append(np.argmax(preds[k][j]))\n id += 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submit_df = pd.DataFrame({'row_id':row_ids,'target':targets},\n columns = ['row_id','target'])\nsubmit_df.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(time.time() - start_time,'[sec]')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submit_df.to_csv('submission.csv',index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}