{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Purpose\n\nI present a small, custom CNN to remove noise from the images in the BMS dataset. The model has about 100K trainable parameters. It's not powerful enough to make missing bonds reappear, or to fix badly degraded atom labels.\n\nI hope you find this model useful for:\n* preprocessing the BMS datase\n* feature extraction\n* the beginning of a more complex model\n* cleaning other molecule images?\n* anything else (let me know!)\n\nThe model is trained using new synthetic data that's generated during training. Importantly, this model also perfoms well on the original BMS dataset images, meaninsg that we can use it to preprocess both original and newly generated images and be confident of consistent outptus to feed into later networks.\n\nAs an additional task, the CNN also separates the atom label text from the bond lines. This demonstrates that the network understands enough to be useful not only for cleaning data, but also for localised feature extraction.\n\n## Example result\n\nFrom this image in the original BMS dataset:\n\n![bms_dataset_image.png](attachment:bms_dataset_image.png)\n\nwe get:\n\n![bms_repaired_dataset_image.png](attachment:bms_repaired_dataset_image.png)\n\nAs a bonus, the atoms and bonds are outputted (partly successfully) as separate layers:\n\n![bms_atom_dataset_image.png](attachment:bms_atom_dataset_image.png)\n\n![bms_bond_dataset_image.png](attachment:bms_bond_dataset_image.png)\n\nInterestingly, it seems to want to make the 'O' labels into rectangles!","attachments":{"bms_dataset_image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAVAAAACwCAAAAAB8sGkzAAAGCUlEQVR4nO2d647cIAxGcbXv/8ruj0xuM+FmfwYCPlJX1Y6akBPbECBT4uAg+de7AbPhQsG4UDAuFIwLBeNCwbhQMC4UjAsF40LBuFAwLhSMCwXjQsG4UDAuFIwLBeNCwbhQMC4UjAsF40LBuFAwLhSMCwXjQsG4UDAu9A6R9gC+WeyEAn9+KI7hQncOkyonLvTDLTAVVlzoxpcHed57pxRCuPrcOiVmae/kEfrtQFlKXeivgl2pKO+XF/ooYP+lQOniNTRijM9SWnvEtYXG83PvlHa1gEPOT+baZaW0n9AX3EoS9PcvuKye1OvpVkO1szptqC6hC3dKFIruam2IdhM6a6npJfQdGS+gl9BZA3ThGmpEJ6GDZLxBnmSEqtesxsbg6jJCBbMDRUxbQvMpb2J04rhPCd3yXbwYMDhGSZIS+sl3g7QfIeONwiSd8h+VRoW0P+17eZ477dv38mVpX6t7zruzkR/YH2kf00DEdbNc48zBGjREPRktWCioFkphjH6shLJHz1jfRBT4c6VWjwAhBGI2OHrhfGgtZUKPvun+a+Jr3JTG0DgZb0Hh5MjeN13uKSk3Uvamw8D+fv59nXqD7tG586L+26ipFfl3RmQqNnMHlMY1vFCUHbC6uYJ2ylOdmFhySmKDwltyRMGl1jWUAufPETnkJcCHqL0FslpsdBDKoFsnMILRfNQPuBWH9hZ9N2kEoyHTFwj33f5pGpQiluGV7TNSvx2W40rP81Y2oMEi3b1FTKH/fOChK/IEdmR7fTvthMbubJVR4mAwYryG/cO8Du2l6nmsncYs5YHZajwC/Rq4KF9aMEz5WBRWhKhFgP6Wzds7NHtUkjAeemx04NLpuM0nNkAfD3cppUd4Ck9rJpQTzrbmd+mYYrfnNn+umfcxrKHq4mcQoIm3FN798myRq+Zte/Pr3QVGLSpojt+HuloMO6V8jcx39a3vN6vHaN023H5WAOIgXZavH6lP2m+PfaanxyZ86YhCP/Lot4O5ZHwP8tmycHTfEh4xCn5GajcbYzoOTbPNoBExbRd8XDRvJU/fQfSgo9AQwhaJHM4/IRx/gS5TV8wd6Oie8glgPltGemehqZDA1j3pZEctfYWmU+yNJbSv0PymCPgZ0x8DbuHINbT50hPidD2Ftp/4aHCLdEJVzSvwCb3+NvGuE6oKsS59jrlV65Qn3fsjbUMUcYuthTKzyik8ohIHhJyqwaMnD7O7iY8f8Y+12Am9bm97dFpGozU6GFZCaVtHPic47jttB9l+Z4BRDd33CTAHOmrTOcVZ5RNYRdnoXZoLNqHy8wW898zt/P1wT6enrxTSnQHMc4ODOOGhy3Wx/aCoam1QQ593D4XLrsvKpnP8H+AmoVFrgnChiV1WvMUBLuAotQc50obEzRly2JSd4ZTEVMwCB93In87DwAALzd9lWeufjyuJKabzpcCfjbcAoELFmypzxPYgCg9mOchACn3FaD1VPRDHxwnt8HKy+ZuDAmBPSmbpnoDlW+FjqOsoqj2vSPeSZmov5CUiGqI0MvSqJ5yShFZGWO+9TQiKYwqSjpl3gmYQmnictDhb+uOVUr48kpNfo5b+t1MIRa/kpUeAlHQ6hdAyqr7oIvFZ+tF1DqElIVpVaJ8OWLYcPkOnFAx2odyXbEL5lMosA3u7ryGqnDCYRWgW0RdUCUJ/kpTPIvEpirU5OqWBmEdo8lVH0WKJqBkTpfzZi5ywfB1O2LtMJHR/pL+J2N8ha8Y8KZ9AFGzC4c9MQjlmQfI1/NKgnijlw/79pA+ftBttzzOwT37Xa7vLnCflYSvrIQRFNzaP0CTt3slbRGi1UXGsryK0EnlALyO0VdLP08sPwjIRWpXGimheSGibXFxIaAUK9y4UzFJCW3T0a/XyZf+hiirlp5ptwuD7Q8tpMLpfS2jZnh2VdU/5Hzzla8iHqLKXXk1oHmWZXWvY1ACPUDAuFIwLBeNCwbhQMC4UjAsF40LBuFAwLhSMCwXjQsG4UDAuFLwq4kLB85cuFIwLBeNCwbhQMC4UjAsF40LBuFAwLhSMCwXjQsG4UDD/AVmPyEqT7RHhAAAAAElFTkSuQmCC"},"bms_repaired_dataset_image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAVAAAACwCAAAAAB8sGkzAAAQqUlEQVR4nO2deVwUV7bHTzfL4JIY5yXzNKKioLiAiqjIooASUUSC4hJxgURHTURNjFtcxugk45bEjOZFkzhqQsSIIpoIroCiGKMigiD7ThvfJ5/33mRRVGx+8weoIF3L7b41nddd37+665w65/Kj+lbVvedWaUAqPNGauwGWhiooZ1RBOaMKyhlVUM6ognJGFZQzqqCcUQXljCooZ1RBOaMKyhlVUM6ognJGFZQzqqCcUQXljCooZ1RBOaMKyhlVUM6ognJGFZQzqqCcUQXljCooZ1RBm3Ou1MQAqqBNyZs7OzTWtBCqoE+4ETXDp/jLPZOKTQmiCvqI/Mg5QdeiaEjqmLDtJoRRBW2gcu7kkO9nEBHRa8dr3I8YHUij1ocSEW2KXTCXiIiy7PsSUdZqbOxnXCT1CCWieNeKEw16UrtVm4jIIyl64gLjYqmC0sXRX8XtcGz80v3vhYNTiGhyUYfe/zAqHKyc0td8Tjbfci4wJAcAimf4pbLHs3ZBt/b8rOXGuH5vAgAyvCNLWANa90/+qG9J6pyWm6dm27juIyKfix7jNrPGNPl//P+YqJEXhUwFkUPPAkDZn/snMMU0n6AXis2WuoGcoL+KmdOCRmcBwJWQ4PMMUc0m6M3x5srciG7hdxIeh4dGVQJAUmSa/LBmu7DffPddM2VuJFa3QtJn594Rf2MMa7aT0tHJ5srcSO4woqNV4j7zLtVH5rGFtTW+RSaR3bqPmTI/ouh5ou8k7y83Fv3CFtZcgn4TaabEj/nJnqhcI+nWkzGsuX7yJ0eaKfFjHhLV1DpxD2smQTM7djFP4ibYEaQPUGbMJGjCFPPkbYJeQ3o9/7Digl66wD8jERGdC1QoMAOdyNaOf1RxQTtvm88/JRFltu6sSFwGdCByfMA/rrigneI7eGXyT0rJ4QoEZaOTLRHZ6bjHFRH0+1wiWrNm6gHuSSnZn39MVkCKnEFEQj7/QWQBUWjy52/wTlpu68Y7JDsORLf0nbiHFRG0x17/6LNELil/6n+Rb9Lk4XzjGUudEkFFh07OD/gYAJK7/5f84RYZjGAZD1MInQ9QHcA/rngv4pd1dXwV0Zi01PGifgVM/0Pdj35M/orwIogcFThEpbrl2FGjjxN1ORTc85Sgz9ngkNezGXKe/V384uuJSv+gQFzJYzg7YAkAXOr9jmF7fODE0+mvD0yW/6sIOy3fFwBQpSsvv8m4jxQ6b+CGL+egkDdiv8r7CgC8EZDfwlS12fnVHwAg1S+6XG7OPnIdG4kZEh7gNeQa415S+AFlQznHhMwpkO/cPgeAPW4nmm/Piu75fvmjLx8OPCIvZXy0zLY1UhqQWFTy5WDeh+hQoMabc0zInVOqHP0qAFwbs6bJPPW5UM99TZ1ywt+SFSyC8RdfFFgIXPLjPafnB1QGcY4JufPyXY479r5A5JHc9/MbjZuO+74Tc7XZKLF74rODs2QEK+jF1svft3Mg+oO9Pdtekmh09OtDzjGJ5M/Ln+69+cmXyi0uEy4YcDoxeLtkoBOhclM2UhhSBlwby7iXJH5Axp+l3fIuVjKFlX03G3Tz5sjGo/PylIBfTyT4GnAKTrg5Pl8sSuqiMfNZB5pa+z8g+qkd416S3Kkmnygpp0tjV9kwDoUziH/A6WMAX3t4fSPidMjzkpApKaKr94qU/FF7GHICqElZW1rw6kq2naTxvS7pUh0zmK1sBIyFDrn+UfsGzrgs7pT9F0NbdQkTnV/+qhwAijxzWZIC+AEoYvvhSZPcIUbK5RPXhg4sfbZ0R/YYxsqR6DHfM3gffvTh2xDHaU/O7fF+DDHKAPC+BAVwzuPaGq+PxDyOe80pA4DiiYNOiPk9BWspTpFcx0PVqJ5YCgCHJ7hEJD3aXHUdwGoZJ4NGrh8BcI6hgfJIGp4JXJvV/7iQQ06If8NZd1O/r5giK1XbVNY1BoU+xQDiFx5uaph2FIBvvNw4USlA+Y4qzq3b550JAMgIGt/y7g8AVrntBwDE9VrBGFopQcs9h17Mcy9raTjnXgRk9ZJ5mX5wFIBPt3BtGrAjMPvRxzi3dS3tu13XAwAyXwq/wRpbKUFLJ273zBquM2DZEwwgVuY9ilsGUDSogGfLgC1BpU2+vd3j6+bmpP6vZANA1WyvJDCj2E9+bGa3z14yeGqetRJAxEY5UdYtAjB7Pcd2AVg6rvn33AmBTa6g8kJHZgAAPunzsTHRlRK0ZELVhz37lBu0DUkCit3SpIMU9q0AznlybRgi3myxKbHX/OrGj5cjdgMAvusbXdrCTw6KCRpdDR8HwyX/Kb0rgfT+1QaNTYnaDmD4GZ7tyhu62dDmDS6bGj6UlQHA9eBhGUYmUKyC+QIQP15As01hALaGS4VIHwBgxySerTozYJ+AZUHfx5aahf0OGJ1ByZLwckHL6L0AJkh1o34nAfSWfeUrg5NeaYK23AifhnuPuCEbTEihoKAifVC+ax5QPihNdP8vXgawzJQ/rgXbssWsKb7BqYgPjDBpKFtBQb8sF7bF+QE4P0D0Br1fJnDVg1tzsmT47BvcfYSJt2UKljOmihQLTu27ishvxjyR3T/wH0i0bC2PllToiOhMjbRj5BzPFBOnZBUUtNpGxLjz7HGixXbCS/3LYhcSffnMyzxasjy8kKpS78jwLAk1NZeCgkK0fn/L8lKij744I2TfNMGFdFvX82iI7vbPH9DDe2L/30dcN3KV/BMUFFTjKGb1mbmKyHlDjJA9dxbRtnEm/31ERPr2nx9MrK2TU11bOcDUZMqtArklUc26ZHWuG41NmLtIr7FvRU3mRarqfr1vm6gHXcngU0Ct1XefviC5jYw/9fqzJidTTtB6qQL296qIaGb4df1z9+7X19tq6utsNXoNtPpW2rbt7du8FbNuKZ+WwOHevCO7npPhmRVkcjLlBJVe89iFiFZunEfVnYlq6kkDrQZaerHR+o8lblzOSETU0cZtQ5SbjEWY6aavnjKnoET0Xo95RJ2JqGV/O+teGqeWdI5wphkJ/10v7Xmzh8nJzCvokYMiVXvjPy3rzqcp/kR0pFLGdDCHhWDKneVlrKrKX7JHxPriHFMeSNWCrkSUc1nUJZ2xpsUQZhX0rdUDxcyL0tgKeaVp94moOYfDWgrlBNVI9iabX4gWd1i2nFNbiIjoAlFXzWExj/P9Tc+inKB2UuXBZ+KlHoQYWXuCV2uIKOUQ0cK/i3mUOpmeRTlBO0nc6pUs3ioZ4/23OTWGiGjsDiJPe8F7XaIqiN7byUM5QW89IKJs4V5wQ/QwyRiDvT7k16BB9peIlos8NujqEA5ZlBP0fj0R/by236zjBs07qhfLCLL6szJ+LXotligIZwXtZ4J5ZDFtOFWMoBoAKI+L6PZyXMXTxpxe8irGdvOcUupfCRwWnsqawKOGSjlBr7nsbPxUlTrfJTiuuTVYbjFOEMPyEik2vwfAR2gapFJWqYAUij1mKGHryIJbg6YMbfyaFnu578RJj63ry8Uu6Zty4a0r3NpUM7KQ6C/VAql197ncmPH4r7SkYNq4PKBi1/ABKx/P0SRMGj7n60IAwClDVU8CvMpxWeScT4H8PjX8ArZEGUEPuu1t/FS807/XtEdFg2WxC+afA1DgfFZ+rBLWZU0i5LgDiN5vwHI5F8jfwSGFIoLGBDadia0+ENE7NL5pzUPozqf3EGPT23xaBQBT9wNZUwwYhvUvxPWgQtMzKCDohYHvt9iWOtM17PGBsWM6W0CPPEFTi6sHCX4YCOBqy9nrygFh8ytOD2IuXmwJf0G3uhue2T77SrfxiQBwxZuxfvag0EKcvP0zFmawPYU2xHDNckVorlNG7mj5PbsgvAUtDJsh3OenTHcOSzwvUg4jwLBvDW+fN3XPhoDhTLU6Zwc/vaXqZkENcn2Klgy5wWMtFOcB5kPvrhSZRRgxgr7d/78LAlijvj97nMHtHaK9dH5L5UwPP8a/zf6pRESVv/30Y2nFrX/e0dh1DPPuVGtfO/vQfptq0x/Ww1fQZVcTXMU9wsKMaPSwccs3GdhcorOlTncYV4S9seYL3HtwD//h6PJCcMfn2nQjIiLbB7auG6a6c7go5yloxsKwVGkvYw6CxWdqDAwEuUR4ElEt0xFKtzt/bHen7dNzRw7/aUOv7C2qNaJtT8Oh22hka79T/ILJoWIngJJRTPvkuRte9pEBoDSRw2oTbkdo4dK2SRyGE1n47eCNWbZ3HJj2WbHI8LyRDxF1707fdvYwsVG8hu+OTYqM+zfrSbZDy2LfXUQST6ltxr77s0Ttru+YPHJg+kEOAGmTpJeiKkLZDZbENT2E7xEaSB8gsZRVCk6jTRfM+OCg1BGyXd9sL1lvemrlXpMee8apDzXrg5h2zZbpeCk1R9Jn1C/R8aLDeCU2mgdizxE27QD/XRCSKdPRX9bKuE/dxE72BQHegSN2C9vN9VBrnsxcd1SW3zetQuS4vX579klhq2v7Je0cnIXtlvBylSm/yKsjXWvodssA6xwFy4CJKn9t1UbsVsIiXqF2cYmcx0euviNdCNDI6OErhUz5Yzyo7VoXwV0t4SdPPh0aRjxEKTgg//WSuyI7zzCwuYKcyMZlfdtaYT0tQ1BaN01a0LXL5Mdz/GraM+HNtmTnZeeUd41xorqH9t3EdrUMQd1HfSRVNpFewPLoY6dPIuwfncHyc9Ku/d8fPbzCOzoRkaaVROGuzCuO3zml/lJTmT7H2CKmeyYAuisfhfT2mr8/58l2ncglE2DG1/9wZo+toU7vCfsOJzBGPB3TGnd7+Q5ifMeGpQhKp18SNft85s4a0XnddCPaYQnXoURE1LOIiEhoncPBvsx6fu9kjJ6WI+i9uSeISrYXGTQW72IvND1p3OtKLOMsT0Q2utfjvG/fuluRmnGzzvahHhoiW61eW6etb+eaM4t9OcIx1k63AYsRVN/u+b9+4GD3jFPQ0LYaPQhaLfSktyHSPNh+W+541BOy23U1qh0WI6jWbunin//YwZkMLEfyPlbJrE6C9K2CQSxI0C7TP9zwsMKp4WsNEaCxqa+vq6//7X+ezWMWNCnRuHZYjKAPtXcDT62ovVuSkqJD/QOqB2kI0MK+rs7J9jfWcNdeMPKVZBYjqLZza5fFb/u2dtIOd+hGpGv22pTUNb6Mr1FJNPYtmRZzYU8/eBHdaC8w87pEy/jSaM/Dxp2TLOc6lLyIyF1oJvvNYyVMwbJeMFJPCxJUFMfl0u/tbcoRo9+LayWCUtSPp1ncTxn92lFrEZQ2sSzEzXUw9hdvPYL6+cmcoiMiOmT8m7CtRlBaeMjwwIkhTgQYncZ6BHWZt06ua1WX3kansR5BaVap3NeSadnHUh5jORf20uw+9Y3ySaxJUPLa5iXtVKbX3/vTi9J+AliVoMkb0yV9sudo7dpNHGl08bBVCUqTQ2dKuVRM3taBnIxPYTGjTbJ4L3yk1KhTne0zTqaksK4jlLYU7pLwyBrjrRkhUn0nhXUdobR0wEUfcY9nXdY8J/0IL2Gs6DqUiIg2CtYpNqK3a91dtBpMAmsTdHSvTAmPtqa9StHK+lCi0jsSjyGW6hMksDpBlcbafvKKowrKGVVQzqiCckYVlDOqoJxRBeWMKihnVEE5owrKGVVQzqiCckYVlDOqoJxRBeWMKihnVEE5owrKGVVQzqiCckYVlDOqoJxRBeWMKihnVEE5owrKmX8Bqa9zzUNk080AAAAASUVORK5CYII="},"bms_atom_dataset_image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAVAAAACwCAAAAAB8sGkzAAAMFElEQVR4nO2dXYhkRxXHz7ndPR87PTO7k6yjGzdh0QQfBEH8QAQhgsSIXxgRxJcIfkLwRRSNeZBFESGIEQl5MeRZhICgPgQf9EUiIuYhiEQTswnrJpvNzs7O9nTf7lvHh7r3dvXMrVtVt07tXfrW/2HYmb5dt/rXpz7Ox61FAVGcStruwLIpAuUVRqDMikB5RREosyJQZkWgzIpAmRWBMisCZVYEyqwIlFkRKLMiUGZFoMyKQJkVgTIrAmVWBMqsCJRZESizIlBmRaDMikCZFYEyKwJlVgS6IEL0bAFjbZOinCZ5NNFn6chSyNc2peKQz5WPdQIAQA+4EahUjpOQSP7WFGkEClCYJ0mQBOQx/uOiVFpn8QshASp/cVO00Nw6C3oSZvOptPOrfJUtIhA1XZo6PuSrh7a0USSsetGgTg95rDRPBLnQy7+72mmLFoo+DglLBwAqLRCpMNIGi1N7QNHLw2PpgJaVgrTusiq1NuR5HD1P6UARoDRQlOu/Q8ikLaCt80SsNTzKXzZcdlwtLkrtjngCoDrLI8JiNwqC7PvaHtC2lyTpYta+jkQFU2u1B9Q7lOsnm++TLK9T1RJQ15kpUB9s5NjPzs6hct3hV2tAw3wceyEAhdgK1wL1T1nVNR6uaUuhT2Rep1qgKALtF1v3kqRCdKIeKNZu1bzU+s7e1ae0VA1QxCLewq0wH8VV5JPp0EoPlAhQMg1gTO3j9Mtt6qUHmocFZWyg/QHKrybbdrPqV3nbVhxjsM7vCCb+bhhzSijjrIZosIvng3Qr+EmhvtP6jX0RGqgNCbpOCI320yjYvwJyiCHZy2ChMvdPer/mJgzd6V6vn4rR7RvM98IgzpppyCMByrRqZai1LFczTgqq3Czj+uMvvuXqYfqju1k//0J1A6NMQEl63QhEeDSvptCUv5IV1GPtGC6fvfzgPfSPp9Z7rJ9eJGFM1KbQIR/30lZLakotJUlDDpTHnA3P7Y7evrHG26oMjvD32K5yhJSZVN3rl92hcm7g7iBkk3QVaXVzwNu03ZB39+ksw3c5RKJ5alWpB8qvKC6q6yAW5W32oq1EDHqnmIuGCGwWeXf31BIoFslU6ZAW9zrWSf13iiAbcC69TE59nGbZwS7zfEdWw8ndRbS3UJlULTad1V8vkXo+IalFFNTUf6Xhuy9Px38T00bv1srmi80tx8kG7CtHiqpJqB/UBDfkhpGS6Xg9wXkZRv6687SUZdjH/cn2isubjDJ3o5xlnXrsVopj07ScRbN00N9/9oNbi18vKVfYSySAJFiTNWbTKKuaHDesbt0ki5aJCAD3/3wVD58+UOIgRETuWRVEkFHEIDz1QxlhoQgnXKGDbcuH558Q6eUMiq+6mHOTYkfrIPbtt9Kl6rzS0eU3YLGY7YdLdp57Ph3J71ntUBFusWyn2BuwqnDs8s4c95fyma1u+dXKEah16+LsZx/OTq5WvubunvC7C/mXnO8zVaRYltyBo3ECQMC8fPq+a38fZMf/LtMAliaK8zI4NqGysuaNozIKFibPBjcO89ACgUhOffnJnqgIP8h9k5UbnfNk9bjL1TvvaBGaxLIsrMHSriiYhW72Hrjtgj6KGiTjaKFFnvM/LbyEzUY7AIQCirD1pbX1r9/b1/XJcsATIWtkHYsBrYqoeHYOpYdfAd3hFoFq7GkySA7SrUH1TaEsuq4TgueHq7ivJtBQ3Kic3RvfMuRDCyhqEidmogVPu7C1SdlskFCivSOSun3yuGHA6rv6QgLTNKqsu/48kSC9kALoM32kBBW9ppiQ5Yz6ftlEIgHmexgv4eTi6wDjP+0Z410E/jN2OKC1/aIjJoooQ4ML+0FCBpwAND7/lUsw+UsKxunNJlZhUDig9dZwhCiRdFGP8WPwOzF989rjh9nYrpbJ936BgGL5Q3eB9JhKfkpYqgzsAouBAtD2o7979nDWs7rW92aBgFL5o+YCmddDxHzEF6Mec6QIZUmll5DOPvD9bFi9hVu4kMHXaPM5pXLGqv4cLCsSABD1Jl/Mfr1BhimUJ2V7CzzeXZXty2NBHJtkAthNzj7yq38n5mkd/OfQFk90kEEIguq5wS01ViOE9U9s9u772J7WzSguJHDNcFe1Es5Tsgsn1aZ1eHzO2cEwgTfTXW1kYX5HBh+iTaCGkCNbHYpIgAjR2CKCyYrNanMONTnzjqknvRIATJLinoYu+d8rnCzy3vzJDZ1uUt20F1BD/7y7z2aiUrjgPQSSF1AU9QGQ2s43zNn4yRgv9JYfUNPWDvV7nzyzVC9WE52XLYSU1z7UFHHHeY62cqtpYxFsCTqUO169M8sDO2S0CZXg4lHbsOw945xg/Pp47hUKqCFWZh3pZF6X8ns7/Nm59VAb+yLQqfxbyQ7Zb2E4Nzt1qSwUwAI1ENCFnpH6h4V6Arum+MY9AmH1c2cCMhLTE1Yx01oFGfJFoUAen0NELEd4sfBb8+Qd9Jolafz8KwJe+8N1/xuEADpf1gukUBYRKC/YyXihi/0uJAiUf+GNHz/0RpL+/vot6XouBoapADmvb3EexLqPmaX74yydObZFC0Whsit0444nr1zZY6jjZ4+H6nadZXzT+fE1bWHZ7NWLz22/f3TXSee21KWSAAnEW7/74Cdv32F4uIzbQiuLh0BF7L5s6y4f/eaPZ8dP/PLQ+Rta+A0RcDa6Nvj0eVpZ9V8AeS20xilSc8burVaaKJ1+79nJe35mTr6p70E4XrOc0TgZfeHp39LMf0PB/yxAba6zUdJNa6EvJb2N0yedPgIe9zmICJIsOfODn19g2ERyAjWmKZsWJ2oCV2v3n1kfwMRpxM+rbZTSiiRZ3eknH713NPLfovENeZt4MWu4jsavvjOj3orTM8U4n9KVaT059bXbxNpP/nnKvw6ADWjQ8GZlwyJ95oXPrIyFyzDVGCCdeNdaQqdPD/xjW0xAb3I6AwAAEnH3y8+8+Po2OWXWqjvZ7wPACjA4ujy+fBs8AcZTysbj7E6Hh0CNvCw2dfWpCKbgSFunAxIIMKfbS5m/eCNQzBIiSPSn2vBYaDs4i9Ml7ANXZvszXXP9ag+nw+GKDijPHNoSzzzQYrs0o43fWx/cotmVh2dr8LmP9HWBvm79bzV2SQLU2zwOhhvf3Fpd29S+exmAWpuo5dxUY8YIs8n6Omz1tA7RMgC1lMvCqQVP4//8Ihk+dEb7zqUAamWiaH/oXmXAUK5WMzz37RPjbf17lwKo9cbe1kQXiSrNE02pv0uJPgRyC1Qw+8vmiAjbg+TyFkuMypM+BDRDHGS153UshYVaqFGa4EiYDwh7tPWprRr7DFpwezNVBJG0Z8SgTHW4NVlIfd/0xrDWCJcEKCCIBCAbrSdVRFEk7s/gyoaOvcngGyzFHAoAlF6cAI3+daMqV4xNPiZC1ZN9xsljSYASXfrOXwVefWoPQZ4KN3+QrEhyOvvHjaL3SwIURe9/jzx39fJrkwkkRwqVmx140bSmakmAAoqNex7bG/Q3VoQgFR+RDEC7m1uziM+yAO1B/6tvXN/cXYNEuk1E89NO1dMFHNTlIQ+AK2/7/GPJOIWi8Afzc/MAEYEh4W6pJQFKNKPxB9KfTtIUFs4xyO2s72ptzufwFloSoAC9M6vv+MYLd5zo5WcZ+LfYLEW/LBt72LtwbjN9ZWunMpLuvmS7uf7qG5cF6HQ26IPQe55uRJsnHZcGaL0aAG2YeFyaObRermfANldHgDribLzGdweok4n6PGTbFaAAjoyaLkrdAVpfknT00sbqDlB7TXxON+oQUCsTRSBBqcfjNV1J0gHYPdAj9g/ETJzcbsylS0DBImyfXfpeb3Xr/g/vNF3lOwXUwkQHJ8UPT8PaicZuQKeAorFmB2Ha31ztD5svLZ0Cal67BRy89GjvQ/ftNL5Hp4CaTRRxeNe3hrDe/BbdAmo2UcoGq2d8NpMd2ocCgKniGwhwc1bxP5jYqyPx0LlMoeP9/9656XNQRgR6RGI28Iqcdg5oaHVtDg2uCJRZESizIlBmRaDMikCZFYEyKwJlVgTKrAiUWZ0Hyl3a3HmgtUfHN1AEajo63lGdBxqH/C2uCJRZESizIlBmRaDMikCZFYEyKwJlVuURHVHNFS2UWREos/4Pu8U4kOBltMEAAAAASUVORK5CYII="},"bms_bond_dataset_image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAVAAAACwCAAAAAB8sGkzAAAa8ElEQVR4nO1dd5xU1fX/nvtmZisdxALGxIIlJopGSRTRYBJ7JaJGotiwYuygUUGDLfZGElvCT6PBEkuiiWKNGBXUoKhoKAoogoDLLuzOzHv3fH9/vDfLLjvlzezbbLI733zcuM7b++583zn31HufWEEZ0UFFO3sKXQyGYGfPoQtBYMoaHyUIA5QpjRKmsyfQ1VAmNGKUCY0YZUIjRpnQiFEmNGKUCY0YZUIjRpnQiFEmNGKUCY0YZUIjRpnQiFEmNGKUCY0YZUIjRpnQiFEmNGKUCW0FtrtiWSY0YsQ6ewL/ZSAEYDsql2UJbQWmPF/pS9b9MqHrIYLUe18krU9miYyWVT4ADeAtuG9m7Kz9qwwhlNIUv9zbBIACCO0nv5130v7zr+kzYXOHoBClMFomFCBE4C28dfHP96218dRTvxkzuhaZnq9iKS0TCkKES2+ffcb+lVA3HtfPp7104YhKlqb33Z1QCiisf+CxsYfVEtZbmNgigfQn12PilgmooGi9796EUijCxhev3/PsjRxQwM8nDz2hmkzOuGHE2QOEgbEvgtLuTShAse/e4lywbQwCUESX3TRvwtAa2JUP/3ncodVQFKn1nUcoO7uNkgKILrl14bm7JQyDAEmQfuemyolbJugtuXXxhbtUgJRiptpphLYnvItqCoL66dNOPbKy+eEKANHkc3f+4NweDpJzr9r8/MEOizL3nUcoIZ1KqVDcF6bucPaAzDT8nxSAdXfOGL9fDbThj9NHHdcrsPfhJttZhBKurZBOlFIaXXHtlxd+O0H6k5DMxCAQ77Obl0z4bjW8z++Y84sRNUWEoZ1FqOqcvoNNJzJK+Xjy7qdUZSQvEE+Iv5KquLNvj128dQWS826243esQNiZdpqELrj+2t5S1HIf8QS+unPPkaQDYD1X65N3QqZe/s03L9jEMDXziZ/tUOX8dxNKmdp4bqcuou7Ty041a00t0Er22PKXdY8/ssfYjcRb41VWOyFFtLPSd8nnDpHOY5Ng+uPdnfSb9QBazSPzC0mg9rj77JWfJNF7QI/Qu2U6KX0ni6q+wU418t6ifsbO2A5ZzDchEIJCGXDRkgY3TkhoZSpAaAetccSThyY6cQufEHZVXL3FWeVOSJ9RAM4WKqYYnzm/ynfUV5bGV4ardqprLxaaSiWyfkMR0ld+AkaKYiE/oQICjKC42gokFg4YAHTmNlMCsTgRy/H9RdDycRex3OcllFZBBSMWVYE+c0gcnemEgmoNbAEXJxOQFjPNPIRS0++9m6S/HEcrS6k39mCRE40aCgfGaP7vJhLeGgXIQ6g48c3uv7ouGDJKtef8qn6dyiYEVCBhQySQi5xnbkJpPfSe3P+E961QIhRSgvLST+IdZ/FCQSRGwIneMOYkVN05Cxql7xnjz3wmFXi1ORkoSnoFcF8YJuxcjfdDzA7Y3J6LUNL2+90lizU2ctqDV61CAbUvcpn50hmsnSufYCxuYDX6Z5qLUDGVg67Y7bzZafONe3of/bZFXrXXImSU0JeGVXXuEkpAjdKFE/nKk1PlxcR6HnPZ5AcbTI/xE8c/0JRR+w3uTwBUz4aeFgH32b1LnG2EsJ4RpwMC7zxGSVWGPjDn3M8ZH/HIa+esCPynVk/Up1cbl6xu9JRkOEGtW75NUdVZgfhLXoSehjgUOJbFhUEhkNexF6ffdcOPe8l1Nr1t+JGveetz2uuvMGDyrdPG3LQ4oKjQ/Ajo68Nqi0pyUWitSrSOq6igKRF9siK3UYIIKJXH3Tn1ukap/NltV9z0tTRfHdStBLrmr8f/4fRb5fzXkqEO2BHQe/pAU8ySK5JKutFGawJ6BkimDKMOgHMaJf+HQra7u+KY9zzznel1p85ns4wQgNAu+82BL19yy/d3uuKy2y9byjAiCqlf9G2DsBpPobXJtJeyNuSCEhLGxFFlRVggFCr2rrmNUtAlKdL7vLPOe2Sd03/SkeNedUUABj6P+8GFo9MPXbNjwsQqvz9t25OfT7Ewo4I3d6oNPz+BdVPV1VVxKqIUUoF6qCioKWSxy0xuOxdkBQlxRm4/4e2LB8RHffu6WT/dJEYREWj63alfnTKpKkZDAVBz4vAbXj9rAJA7iUpAQPepMXEtxhaY+gHWVDRKLFIDIo7CrS7ElqDYfrF8sbxfACRhNp26yei3bGz7W7Z+YGGjGIFteOno60589NCejl/QAsQZckPPn3/gkcglSQKQsm7BVkWpkUEtHDAZGPuIIAYu13nx/CMShGoxXnbejL34xQBQpOac3SYcdXx1v4NHotLQW/HUg9tfukMFkNEIApDe5+wy4YhRNXkGFHjee4P7FFEJoDjOmljCalO1MDKTTEDgoOFbyCd9FNGmL+o36eeICS2kBRLMvm0i4ez5yL/HzqOprXVS75w9quGB24dWioj4zqF/HWPDf/vv8xZCWrDcYijCfv3alaf8cmMrEt620kvVeeqkGyXSGhS5zpVvjorly54LvTlnXBfr4wg1tFIVLiMHxX+DxhlTTh5d0zDjnvgpe9U4EEqm169Fa1Dyxduu3SHm+1y+1RcAQlDr5kz/18bDhg+cdMhRvsULxY+XdFd/umvsi28YJzo+KWwYe9UOjenqeI4mK8LQfnXX22fuXSFoStpe8ZDrTYi6vM+oAXX+pYOH/26bE4ckjEqW/TyEEPLxPw7vERdRUiCGFAhUV7795AdDDtytX8yYRWfdug2oobocyHWssosG0+3laGQlb0L48kX7n19j4+tFofUVAm348+9/fmRP0L7z520P6B820RySUPgrytdXrTz/205G7jZ8tgQAoykvEReKit+BIdT6WQ/M2/2oHXoKaJHQZ35/Tx9oIQfQHzC5ql9l4+JBcROPLNNG0OCNyTf87ZWDR/fM9NS2/iICujNv2e7MQRC75NolFw2Ly4bX5EKozpHAYzfUVesGxTLOZhsJowTkUwCqiKiIJl95/KPtjxlaYwSg9/WqbeLpW1ZOrvJb2grcFpi3eK9E40ff7BWLrgJFgC/fceWOjQv/8N6E3atBv5O5xWSE9t/XNF60cxy6+k9PjDugqojUQ7hWHGboc208k/HKegNCQBqQjqfGULy1r88aNrQ/YPxR1lx+0A9jq08ae4ADLSSiBJp+eeQwWfbSDwc4UbXtEEL71/unbGcE6Tk31V68Raz1HhoKdOX9z52zbw1Q/8ItPxnXp7hOew0Fa/3/T7tW1drMr1kuVGutpXXVS6mr1jYl01ZVrapaa8k3Ri4g545YpGq93MOoqqpnvb8c+7Wmp01d6+W5ZXGwqqlpR32Ysp61atc+PvLGr/05N38rb83De924XMmmOUef+FHaWqtaaKrrEZJQDThteeNcF1mr6rnqqZu2SrXWs8Fn1irdPx1bR/vY0Q3UQiRZXfnDWVYX7j/fjYpPa1Wbph7zadp6nrWq1ltx5Z6Pr6M/OWtJbXr+x2d86CndpRccOCOpwXWhb19E9x2BsL6gWAgFakTU31cVaA1Fmi7v94vK5Dk7nu4wf1BH2DvqLo27l2x2tgGiWUIpUn/bghv6NE8Ukl50Q/0VWyUCvZdPrkmdt7MjWvfktGOOXp90CH/zYh9xyOs8eul02ioDqc78oaUuP+gFjwt/OFOZX5OsLthnsdV/7rcir1IUieWnXl6v1voab62qZfKZvX+5zKOq2uS7pzyUJFn/3N7nfupm1oJixi+W0HCgtfTSqXTacsNnYFVf23sp9c19v6BaNzel1qZ+cZ/LuiNeLfpL5cG8g+9sXL9wBcO6X9+xxx1fK1WTSz5TZXLu0YfNSgXrV5F37pD+UBKGTszN2mhFDDvqsrWyy1GXrRNxco8BeffDIxz7VP/dItB1P3AUvnb22JMr1m/rCKIf03Pco18e8cRakcRGm4HLpow/5uGhcYiUki+M6MlnQbrOo20jnmpV1475U5r1p9yezmPqra459EXL5SMWliAmG4L+CPblg2a6be2qVbV0551yyD8a1Lr1jx1w22r/81I0o6Nawkl3RXVfQ7a2YxSAIvNPuntrWTLul98XalaPnQCnP397tb2u52mx0vZZt4BAKRQ2PD5siEhLLzoIWQQAkm/fWHP69jP/0PvCbxkA/03buwmINv592CZtu+gpAI19etr9vTjrsvs2Y67khFl59E07OO9f8GBff57tmI4oQKjDtW4f2cBLb/G4Fetm3Ld68Lk7VTQregl37bAee9rXreaILensN2RqSnY6cmIDkENA7aPfH+Ksveq83hLFpjuK4xhW9pYW2UYAQQId/rKP6gOO3eme3Soy7aEl5bM7hFARUOQLR9raJPG92crz3pjpxcc40z2ASlKV9Cv7vndqlj52YozP1uzltHvHncAgDaUnscx/aD3ZDMdGsHhklYDQ0p9hR0kooXSyNe8IAVH0v/Sapaia9NA/1ThGMusogwKKeum79htklt99YUUEU1GYmAIixiBbcT9jzMVg7hDjrwglV1s6ilCBFZN1VgJAVHY+8voGbH7J5XUAIMYxxjjGcRwxxohx1s07Nq737rslCoRTIUCFRyswIjnUONAkIr10UKbuUyo6bFsNbRpZUnzI5FKd0Q3LKhI/2PX6k6xUVMAxEAgUUM9rcGPPeuTc2b+NIgsqQmGCmk/q/BS6Lq6tLNG4N6Pj9ikxV9Agvu/U83gPiiNO+sD2SqUtY6JuHFZobIVT2zNRM2nsjacNoBbdk93mdgqBI+rk99KFBOfuFS+YpS2ADiQ0Zz1ZACFMfxU0Xjfh0LRNGOvS38EiQMyJGcfUz7hqyAj/kID2TcOvaVnH0rTyktpC7VuHFtEklB0dSGjgG2f7yM8+GUlP+9boSl1fypPgryjof3jqNRFRCV9yznEzGkisqYoUKEzO4QRwP9myXbcCOnKvZ55CceD7gS/9ZUJl8K/03SYQ/k8k9pu/PILWBoEDCKrUxKF1yXw9l9KY7NPudp92EZq3WJ1PsIKFccFVN/XPjCDN/zC4oP9x97kw0u6vSJKgGCq/mo9cDW0EMH+rRLuXmPZKaJ4vGzQ/5pQHaZo0flvxxREtPRoRAUkz6p+flRattJqg32MNuI1ievw+mad8yXnDYu1un2oXoWKM5FaguFnv2Le5SiD8Q/9DYs2/tvwo+Iuep1+dCixYu+DHDO4HTdIPr+RpELRvbd/+/r7SCSUheXoMJRY3mXiYbVKLJDnz6UsSAuQQGAH4k6bX/aO/Sp5ki7SS8/pLWjX23lRuP8x+Oqjd60t7JNRv/cipk+IQ1lIMANq2uxo+nTxpgMnjRQulcsKVDRFYJd9Rq9jn/xq5VWI2s+iLf8kq9ml/GNEOQglSApOcDa6bWvvJonUWYqStbUnf8dNdCugyud0uf/Qkir0KIsSQ+CeoOWNqOpcIcO7OTvvbKUonVAKvO0d4DFdhKutvPOzil5e7MI5vaILsB0Ue+eJYE1yZ8waSGP/A5yG6okOBlUc/7mFXvg3ksPOv7NV+m9QulQeZI/IgwEpH41W7/fr+PR86fNwTS9P0U0qBneK8eydVQ5FXIITY7KxrkiVVdrJNasRbK1A15t5k1g+JxpWDtf0tkx3jNlHELlvy/CqL6kGH3P7oibNHn/Dk8lSwOBBA3ZTztxIUVDA1B635py2tWLYBhOxx+KOWu61cmGPeDTtvEsHyUnoJJHNoVNsPBETyxbuHz1/+nUO2rQZFtO7Dx+Zsc+DuvRyHAI13x5IplSzcNkHI7Mun1zJEp14IyLKjnuhnb/zi+qxNod5qd0Aigpu0r6aUzaOhgAtvWztxc3z1j8ca9hmxY6UDga1746kVW+++8yYJADMn3z+oYFaHAoqkJ3znOCdnUqMYj4qil243xiw48U8D23wmSs8zqGj/Y4u6SEcBJP3cLaceWKMimv7ytSe/2vHwXWodCtzPZ/+LB+9YYT4dc+P38kQsLUaDyOITHh6AnL2PRSx6NJh31mO90hNGHOhsIKGi8AxEY8WMlx2REwrBqpv/PWULB4GeuqtmP/3JFqN26R8HlElWo+GckUfFgcKEEqDR3311iUPNurWdAE1Y158i6fP2PQwf3Dmpl394S3BCpPibPQwyCcT2IFJCKQT0nSv2O7kqmJcAQrFr5k1/d8tD9ujjQAjv4VnXVmY+LTieYOVxt2/D5g7VVpP3miQuTtgj9GjwrwmPV+vcfj0qYiKAkhSBAayhGgGN0fbnsyMDBUDTgw9PGRpvYSsFFIGt/+jBd3YYNbRXTN6/7DcD84VIrceE8JlHf5fI4qARds1qGRhLhBYrmqZxJ+wjrhi/MgsBVQCI5xAiCiPtPZMgYkK56Fc9Jm68PiUXdGkIIfAa5j7y9rY/7TPl4mGh+QQhaPz5aftmscuSXuf2lJW1FYmw4Q1F3rjq0aqgOEMqNe2mG5zq/hBoTFxHzH8NoX7H+HM3n7VfZcsUJ/zUPQSkiNbN/nPdTw+Mt/y40LgwnHXhIwPaPgJx1/aEw0apYNh9WZTk8cfsIyLeunWrVny2dPmaJokN/NH3NonHSUcpwT7LdiA6QiFYPfW9q78ZmIjmefmho1/iAOyaZO+q4iq1NO61zpm1bYoX6iEO0UZThbBPB7B/+/VA63pJ9t14i74DN+pVVRmTigTFGAtHCdNeOx9VTYkC++7l+97TI1jUZYMPAYCgcXr2Nrn3POQYO3bqq42VsdbGnEAirYZCMaEDKYJfbTop0VhTm4glYB3xs61iRSiuA1E17dwBGZmESuOD0y/brSIozbWeEpuVXzIBVpFisM5seGAzhXATVFlXGfacRwKYf+ZdWyHTEMQgwwM1AkCRtBW5ttaFRUQSKpj/q9ppG4HMFo+u/5Ubdr8VhF/Er2obByi8lK2k1hQTLqWvPnHLYBNNMA0BAMc/RpTyxqbbdVr6rhX0+XGH/3pgkAzPkXAMzjkt4nBTICPvbbOEIpLo4TZ4rnq5U7Jt8Ez6UDRnRjJ/RFCppHESW177frCjKu+c8n0Wicrrm9PGD8n0CBQO0YtCziSMnz+0koonwqkpZcWoe7dqm5Kh3wogEOisy6//LjOnh2aDn53N/SUikVAmY78akqnIFfpixapUzqBfqEJjZM7a5qxg/lnC3nXYFmDufhbC7Dpx4sf5OkOFVjVfz0EkhEp8x75iAs2MYsDWoyP7KkLAgg7k7zaTt84HQt577fhsNkMA8RvwSGePM85ZLLkysITCwCipuY7+imYNdRJB10P0dOYCg34IONv+fR6Qp1oYXC9ounpiH2RbHZodPYKxHx0zdpmg+aSaVjcUAWk9zdPJF43KB1vl/pMH2gkyO/T6jLrZDXFn4vnKHzSXYbKMF7j08dEHnLdaWmp9Zk8OITQwsZjj5JThSAgNLPd/+HxAX6xEgJFr5wAFD1iSVTdeUpn/CgKgVJ4x6OZGv3slc6fgAAqSSksIkbP6FAWh0tzS/R9Fc7RQddGUVOEp2PtHFixj+QxWXbTwHjdjeKgZmy8iBGnouXlsRRd4USqxY/9nNX8lj5RFT51e6Fhd/6lQBvz6lSdVgt4s8UH11iz9Yo0VwInlScZ0gVeoUeSj8dN7589q0Bu/5+hYiGofBTRcOv60Hzc3h1N17bKPP5y3eNDYnWoTapCvD7gLvHlWiK33mn5SjHn6oCjvzL/OCZNIEgCUQVedHNvHgFDbtPyj19+v7739zvsNrK1OeHD8IytyOv7/+xIKinx2we0D/TdKZt0mAWk89ux9Mo5BmBH51uSzhmvyy1mvLq39zve27dsj4XdyNR+Y2KUJhfEecw6KxdB8MkzmpWcZmeSTf725WkKfpUUR/cel1Wjaapedt+4Rbz4RFjBKo/kG+t9XeYqQzo/munHflTG+eWbz/yBA8v7rqsP2nwRvVvkux42oqHJIkfVJchLQvB7F/z6hvu2o3bbC0NCLB/WWjGQGEceLQzYPXSD2k6VcMOiASkqmyOjDfyJ5Xa//fUJ9RuN9jcI6UNsgjgvx34UjBg7hcNVDkyvD79DxV8uXD64MutWZOW4O8COqfMN0AUIzEY4aALrmpbcWphxPKYBjPMeCPbb+YPSWxTTWCYDkC3eTpmWqTjKk5h2nKxAaJEz9ElH1XkOroQRhYKkxoXj3Lj+oqECOAvm0R9/moVveqdBj6RKEAiD8rFq8qnpjbqCWqV1n1PeQIo5XEZDPHFoR6H5xE+kqhMLAeBUaJ1MuqfQAUgzpKhu+7rloY6e4IDv5wr1gKfsUuwyhAB0l7eoXZy5X60L9wobRhOcOjtVrkaePLuzXL0QRIAu6CqEkDK2jrN1rWKJa/ZYlAzHiOLHUezftsnH+UtCGoz17UIHjmXOhK0RKAKCGAqoIzAbmnITxrsHEWK4NAVnAplF3b1pa23QXSN8BABwAEGMg9Heb+Oc1+dsq6Jz04tJMEB4Ki/r2L3EiXYVQEqqU5r3MfhoTAAgDDDzjmlQR3VT820HxEndEdg1Cg6pPpirSijnfST1g+ZsFctAtkXp1eKmvJOsahCJn7c0HpfqSq9eEZJPAZxWlanwXIVSaf2aXKgF2/t4fPYRZRQngmYPipRbJugahzSFhLhaI2EnPLMn0iBVC8uUfFHyLTS50EUKBPDGinzsZ/LObUpnf8g/Dus3a7mQKiy5EaE6I7zr9+NOFIZqgBArzs6qSTz3oDoRCQJB9xtxhC7/xVa2wZqcC23rzoFsQCgCg+fHSuUDO7jsAIFQMJVEVs6XeppsQKgCl17lTmvK/PVpESFhpx1HK3YRQiAjNsI3+qkC+kJ5iHePEYjGUlmvqPoQCQlRddNcKyXKoaSuQKHDURF50H0IBxeCjbkwWaCM1Qs8TDf+Krw3QjQgVwDlmzlzNd2wDRYB4rB07lboRoRCi18Rr1+W9BIBB2HcSZ0P3IdRvIRm2zSKbz86rAqQpvXu4q2Tsw4GCz5q2cnIaHIKgU/jVWXnQvQgFkPSq83qiSmHYEw2yoasU6cLAD+lj+bSZUJoNq1JFofusoUHu3uR/ATxM1sNNwqM7SSgASF4+g5067SG0O0loi8R+rgvyZqnD3aO7GaWORreS0P8EyoRGjDKhEaO7Wfk2iHoLdVlCS80k50C3l1B/13F0Qtq9JbTA0fGloHsTWujo+BLQzVWewsyunIhG7OaRks9jZGyi2xNadps6ApG6Td2b0EgdJh/dm9D1b3+IDN2bUABRn+dTJjRilAmNGGVCI0a7XwxaRmuUJTRilAmNFvr/4ld28+rAxXAAAAAASUVORK5CYII="}}},{"metadata":{},"cell_type":"markdown","source":" Below I install the same synthetic image generator that's used in my notebook https://www.kaggle.com/stainsby/improved-synthetic-data-for-bms-competition-v3."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Importing synthetic image generation script, which also installs RDKIt and a few other utilities.')\nprint('Please wait as this may take a while…')\nfrom bms_synth import random_molecule_image\nprint('Done.')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import copy\nfrom datetime import datetime\nfrom io import BytesIO\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nfrom skimage.io import imread\n\nimport torch\nfrom torch import nn\nfrom torch import optim\nfrom torchvision.transforms import Pad, RandomCrop\nimport torch.nn.functional as F\n\nfrom PIL import Image\nfrom rdkit import Chem\n\nfrom matplotlib import pyplot as plt\nimport ipywidgets as widgets\n\nfrom bms_synth import stretch_image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training and model options\n\nThis model performs significantly better *without* batch niormalisation, I guess this is because of the regularity of the data and possibly the fact the the network is fairly shallow. The model can be made even smaller by switching on separable convolutions, although the accuracy will decrease a bit.\n\nThe device used by Pytorch is also configured below, so change this as needed."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"DEVICE = 'cuda'\n\nTRAINING_OPTIONS = {\n    'num_iterations': 8000,\n    'margin_fraction': 0.025,\n    'patch_size': 96,\n    'patches_per_image': 10,\n    'batch_size': 15,\n    'edge_loss_weight': 0.8,\n    'white_pixel_weight': None,  # don't use - gives poor results\n    'white_pixel_threshold': 1e-3,\n    'device': DEVICE,\n    'model_name': 'cnsnet3',\n    'run_name': 'notebook001'\n}\n\nMODEL_OPTIONS = {\n    'training_options': copy.deepcopy(TRAINING_OPTIONS),\n    'use_batch_norm': False,  # False is best\n    'use_separable_convolutions': False,  # False is best\n    'use_global_pooling': True  # Global Average Pooling - True is best\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"PROJECT_DIR = Path('..')\nINPUT_DIR = PROJECT_DIR / 'input'\nOUTPUT_DIR = Path('.')\nBMS_INPUT_DIR = INPUT_DIR / 'bms-molecular-translation'\nTRAIN_DATA_PATH = BMS_INPUT_DIR / 'train'\nTRAIN_LABELS_PATH = BMS_INPUT_DIR / 'train_labels.csv'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Reading training labels…')\nTRAIN_LABELS = pd.read_csv(TRAIN_LABELS_PATH)\nprint(f'Read {len(TRAIN_LABELS)} training labels.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data generation\nDuring training, I grab random InChI IDs from the training set, using each of these to generate a synthetic 'degraded' image. I take a number of random patches from each image and supply these for training. Each patch has three layers: one for the degraded image - use for training inputs and a pair the bonds and the atom labels, which are used as labels."},{"metadata":{"trusted":true},"cell_type":"code","source":"def image_generator(num=None):\n    mol_index = np.random.randint(len(TRAIN_LABELS))\n    mol_id, inchi = TRAIN_LABELS['image_id'][mol_index], TRAIN_LABELS['InChI'][mol_index]\n    mol = Chem.inchi.MolFromInchi(inchi)\n    count = 0\n    while True:\n        img, orig_bond_img, orig_atom_img = random_molecule_image(mol, margin_fraction=TRAINING_OPTIONS['margin_fraction'])\n        yield (img, orig_bond_img, orig_atom_img)\n        count +=1\n        if num is not None and count == num:\n            return\n\n\ndef patch_generator(num=None, dtype=torch.float32):\n    patch_size = TRAINING_OPTIONS['patch_size']\n    patches_per_image = TRAINING_OPTIONS['patches_per_image']\n    cropper = RandomCrop(patch_size, pad_if_needed=True)\n    if num is None:\n        num_images = None\n    else:\n        num_images = num // patches_per_image\n        if num % patches_per_image != 0:\n            num_images += 1\n    patch_count = 0\n    for img, orig_bond_img, orig_atom_img in image_generator(num_images):\n        img = torch.tensor(img, dtype=dtype)\n        orig_bond_img = torch.tensor(orig_bond_img, dtype=dtype)\n        orig_atom_img = torch.tensor(orig_atom_img, dtype=dtype)\n        h, w = img.shape\n        # We may need to pad images that cannot fit a patch\n        if h < patch_size or w < patch_size:\n            dh, dw = max(0, patch_size - h), max(0, patch_size - w)\n            top_pad, left_pad = dh // 2, dw // 2\n            bottom_pad, right_pad = dh - top_pad, dw - left_pad\n            pad = Pad([left_pad, top_pad, right_pad, bottom_pad])\n            img = pad(img)\n            orig_bond_img = pad(orig_bond_img)\n            orig_atom_img = pad(orig_atom_img)\n        for _ in range(patches_per_image):\n            i, j, h, w = RandomCrop.get_params(img, (patch_size, patch_size))\n            img_patch, bond_patch, atom_patch = img[i:i+h, j:j+w], orig_bond_img[i:i+h, j:j+w], orig_atom_img[i:i+h, j:j+w]\n            yield img_patch, bond_patch, atom_patch\n            patch_count += 1\n            if num is not None and patch_count == num:\n                return\n\ndef image_widget(a, greyscale=True):\n    img_bytes = BytesIO()\n    img_pil = Image.fromarray(a)\n    if greyscale:\n        img_pil = img_pil.convert(\"L\")\n    else:\n        img_pil = img_pil.convert(\"RGB\")\n    img_pil.save(img_bytes, format='PNG')\n    return widgets.Image(value=img_bytes.getvalue(), margin='1em;')\n\ndef test_patch_generator(num):\n    inchi = 'InChI=1S/C15H15F2NS/c1-10(18-2)11-6-7-15(14(17)8-11)19-13-5-3-4-12(16)9-13/h3-10,18H,1-2H3'\n    for im, bond_im, atom_im  in patch_generator(num):\n        #plt.figure(figsize=(4, 4/3)); plt.imshow(torch.cat([bond_im, atom_im, im], axis=-1), cmap='gray_r', interpolation='none'); plt.show()\n        border = '1px solid pink'\n        widget1 = image_widget((1 - im).numpy()*255)\n        widget1.layout.border = border\n        widget2 = image_widget((1 - bond_im).numpy()*255)\n        widget2.layout.border = border\n        widget3 = image_widget((1 - atom_im).numpy()*255)\n        widget3.layout.border = border\n        image_row = widgets.HBox([widget1, widget2, widget3])\n        display(widgets.VBox([widgets.Label('training input + label (bonds + atoms)'), image_row]))\n    return 'Done.'\n\n\ntest_patch_generator(num=10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The NN modules below below are part of a larger collection. I haven't included some unused modules because they haven't been tested yet. `ResPair` allows you to recursivly build a network with skip connections at each power of two layers."},{"metadata":{"trusted":true},"cell_type":"code","source":"class BaseRes(nn.Module):\n    \n    # resize_power - must be 0, -1 (downscale ×2) or 1 (upscale ×2).\n    def __init__(\n        self,\n        in_channels, resize_power, out_channels, kernel_size,\n        padding, groups, bias,\n        separable, use_batch_norm, use_global_pooling\n    ):\n        super().__init__()\n        if groups is None:\n            if separable:\n                groups = in_channels\n            else:\n                groups = 1\n        if separable:\n            bias = None\n        if resize_power == 0:\n            self.pooling = 1  # Effectively, that is no pooling.\n            self.stride = 1\n            if padding is None:\n                padding = 1\n            if out_channels is None:\n                out_channels = in_channels\n        elif resize_power == -1:\n            self.pooling = 2\n            self.stride = 2\n            if padding is None:\n                padding = 1\n            if out_channels is None:\n                out_channels = 2*in_channels\n        elif resize_power == 1:\n            self.pooling = 1  # Effectively, that is no pooling.\n            self.stride = 2\n            if kernel_size is None:\n                kernel_size = 2\n            if padding is None:\n                padding = 0\n            if out_channels is None:\n                out_channels = in_channels // 2\n        else:\n            raise Exception('resize_power must be -1, 0 or 1')\n        if kernel_size is None:\n            kernel_size = 3\n        if self.pooling > 1:\n            self.pool = nn.MaxPool2d(self.pooling)\n        else:\n            self.pool = None\n        if resize_power < 1:\n            conv_class = nn.Conv2d\n        else:\n            conv_class = nn.ConvTranspose2d\n        self.in_channels = in_channels\n        self.out_channels = out_channels\n        conv = conv_class(in_channels, out_channels, stride=self.stride, kernel_size=kernel_size, padding=padding, groups=groups, bias=bias)\n        if use_batch_norm:\n            self.conv = nn.Sequential(conv, nn.BatchNorm2d(out_channels))\n        else:\n            self.conv = conv\n        # pointwise conv\n        self.use_global_pooling = use_global_pooling\n        if use_global_pooling:\n            global_avg_pooling_channels = out_channels\n            self.global_pooling = nn.AdaptiveAvgPool2d(1)\n        else:\n            global_avg_pooling_channels = 0\n            self.global_pooling = None\n        skip_channels = in_channels  # the skip connection shortcuts the input channels\n        # We mix down the extra channels with a pointwise convolution so we get out_channels of output.\n        mix_in_channels = out_channels + skip_channels + global_avg_pooling_channels\n        mix_conv = nn.Conv2d(mix_in_channels, out_channels, kernel_size=1)\n        if use_batch_norm:\n            self.mix = nn.Sequential(mix_conv, nn.BatchNorm2d(out_channels))\n        else:\n            self.mix = mix_conv\n    \n    def forward(self, x):\n        if self.pool is None:\n            skip = x\n        else:\n            skip = self.pool(x)\n        x = F.relu(self.conv(x))\n        if self.global_pooling is not None:\n            # throw some mean channels into the final mix\n            means_per_channel = self.global_pooling(x)  # shape: N x C x 1 x 1\n            global_avg_planes = means_per_channel*torch.ones_like(x)  # restored to: N x C x H x W\n            x = torch.cat([x, global_avg_planes], axis=1)  # cat'd to: N x 2*C x H x W\n        x = torch.cat((x, skip), axis=1)\n        x = self.mix(x)\n        return x\n\n\nclass ThruRes(BaseRes):\n    \n    def __init__(\n        self, in_channels,\n        out_channels=None,\n        kernel_size=None,\n        padding=None,\n        groups=None,\n        bias=True,\n        separable=False,\n        use_batch_norm=True,\n        use_global_pooling=False\n    ):\n        super().__init__(\n            in_channels,\n            resize_power=0,\n            out_channels=out_channels,\n            kernel_size=kernel_size,\n            padding=padding,\n            groups=groups,\n            bias=bias,\n            separable=separable,\n            use_batch_norm=use_batch_norm,\n            use_global_pooling=use_global_pooling\n        )\n\n    \nclass ResPair(nn.Module):\n    \n    def __init__(self, first, second, use_batch_norm=True):\n        super(ResPair, self).__init__()\n        self.first = first\n        self.second = second\n        self.in_channels = first.in_channels\n        self.out_channels = second.out_channels\n        assert self.out_channels % self.in_channels == 0\n        self.pooling = first.pooling * second.pooling\n        if self.pooling > 1:\n            self.pool = nn.MaxPool2d(self.pooling)\n        # pointwise conv\n        mix_conv = nn.Conv2d(self.out_channels + self.in_channels, self.out_channels, kernel_size=1)\n        if use_batch_norm:\n            self.mix = nn.Sequential(mix_conv, nn.BatchNorm2d(self.out_channels))\n        else:\n            self.mix = mix_conv\n            \n    \n    def forward(self, x):\n        skip = x\n        if self.pooling > 1:\n            skip = self.pool(x)\n        x = self.second(self.first(x))\n        x = torch.cat((x, skip), axis=1)\n        x = self.mix(x)\n        return x","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## The model\n\nBelow is version 3 of my 'CNSNet'. I experimented with decreasing the spacial resolution at lower layers, and regenrating the resolution with transpose convolutions (like a Unet), and other mechanisms, but for this simple task, straight-through convolutions have worked the best so far - by a large margin. "},{"metadata":{"trusted":true},"cell_type":"code","source":"class CleanAndSepNet3(nn.Module):\n    \n    def __init__(self, options):\n        super().__init__()\n        use_batch_norm = options['use_batch_norm']\n        separable = options['use_separable_convolutions']\n        use_global_pooling = options['use_global_pooling']\n        def thru(out_channels):\n            return ThruRes(out_channels, use_batch_norm=use_batch_norm, separable=separable, use_global_pooling=use_global_pooling)\n        def respair(module1, module2):\n            return ResPair(module1, module2, use_batch_norm=use_batch_norm)\n        def convpair(out_channels):\n            return respair(thru(out_channels), thru(out_channels))\n        intro_conv = ThruRes(1, out_channels=32, kernel_size=7, padding=3, use_batch_norm=use_batch_norm, separable=separable, use_global_pooling=use_global_pooling)\n        self.encoder = respair(\n            respair(\n                respair(intro_conv, thru(32)),\n                convpair(32)\n            ),\n            respair(\n                convpair(32),\n                convpair(32)\n            )\n        )\n        # pointwise convolution to extract the 2 output planes\n        self.decoder = nn.Conv2d(32, 2, kernel_size=1, padding=0)\n   \n    def forward(self, x):\n        x = self.encoder(x)\n        x = self.decoder(x)\n        return x","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training\n\nAfter trying simple mean squared error loss between output and label pixels, I found that training was greatly improved by adding an MSE loss for the *differences* between neighbouring pixels (essentialy the first derivatives along each spacial dimension). This appears to give the resulting outputs cleaner edges. The 'edge loss' is so useful that it is weighted by 80% compared to the pixel MSE. You experiment with this yourself by changing the `edge_loss_weight` value in `TRAINING_OPTIONS`.\n\nThere is also a `white_pixel_weight` option to add weight to the pixels corresponding to non-background regions in the labels when calculating the MSEs, but I didn't find that this improved the results, so it is turned off."},{"metadata":{"trusted":true},"cell_type":"code","source":"def batch_generator(batch_size):\n    patches_per_image = TRAINING_OPTIONS['patches_per_image']\n    patcher = patch_generator()\n    is_first = True\n    for im, bond_im, atom_im in patcher:\n        if is_first or batch_item_count == batch_size:\n            if is_first:\n                is_first = False\n            else:\n                yield torch.stack(input_batch, axis=0), torch.stack(label_batch, axis=0)\n            input_batch = []\n            label_batch = []\n            batch_item_count = 0\n        h, w = im.shape\n        # Augment with rotations\n        # We prefer no relfections for now because the original dataset does not seem to any characters mirrored.\n        krot = np.random.randint(4)\n        if krot > 0:\n            im = torch.rot90(im, krot, (0, 1))\n            bond_im = torch.rot90(bond_im, krot, (0, 1))\n            atom_im = torch.rot90(atom_im, krot, (0, 1))\n        im = im.reshape(1, h, w)\n        bond_im = bond_im.reshape(1, h, w)\n        atom_im = atom_im.reshape(1, h, w)\n        label_im = torch.cat([bond_im, atom_im], axis=0)\n        input_batch.append(im)\n        label_batch.append(label_im)\n        batch_item_count += 1\n\n\ndef mse_loss(label, prediction):\n    diff2 = (label - prediction)**2\n    return diff2.mean()\n\n\ndef weighted_loss(error_values, white_pixels, white_pixel_weight=0.5):\n    black_pixel_weight = 1 - white_pixel_weight\n    black_pixels = 1 - white_pixels\n    white_pixels_norm = white_pixels.sum()\n    black_pixels_norm = black_pixels.sum()\n    err_white = (error_values*white_pixels).sum()*white_pixel_weight\n    err_black = (error_values*black_pixels).sum()*black_pixel_weight\n    return err_white/white_pixels_norm + err_black/black_pixels_norm\n\n\ndef mse_loss_weighted(label, prediction, white_threshold=1e-3, white_pixel_weight=0.5):\n    white_pixels = (label > white_threshold)*1\n    diff2 = (label - prediction)**2\n    return weighted_loss(diff2, white_pixels, white_pixel_weight=0.5)\n\n\ndef train(model_factory, options, num_iters=None, report_every=100):\n    options = copy.deepcopy(options)\n    training_options = options['training_options']\n    model_name = training_options['model_name']\n    run_name = training_options['run_name']\n    device = training_options['device']\n    batch_size = training_options['batch_size']\n    t0 = datetime.now()\n    model = model_factory(options).to(device)\n    training_options['model'] = str(model)\n    net_type = model.__class__.__name__\n    options['net_type'] = net_type\n    optimiser = optim.Adam(model.parameters())\n    optimiser_type = optimiser.__class__.__name__\n    options['optimiser_type'] = optimiser_type\n    iter_count = 0\n    loss_buffer = []\n    print(f'Training \"{model_name}\" ({net_type}) \"{run_name}\" on {device} with minibatch size {batch_size}, optimised with {optimiser_type}…\\n')\n    for inputs, labels in batch_generator(batch_size):\n        labels = labels\n        iter_count += 1\n        optimiser.zero_grad()\n        outputs = model(inputs.to(device)).cpu()\n        diff = outputs - labels\n        loss_edge_weight = training_options['edge_loss_weight']\n        white_pixel_weight = training_options['white_pixel_weight']\n        if white_pixel_weight is None:\n            mse_loss_pixels = mse_loss(labels, outputs)  #(diff**2).mean()\n        else:\n            white_threshold = training_options['white_pixel_threshold']\n            white_pixels = (labels > white_threshold)*1\n            mse_loss_pixels = mse_loss(labels, outputs)  #(diff**2).mean()\n        if loss_edge_weight is None:\n            loss = mse_loss_pixels\n        else:\n            dlabels_y = labels[:, :, 1:] - labels[:, :, :-1]\n            doutputs_y = outputs[:, :, 1:] - outputs[:, :, :-1]\n            dlabels_x = labels[:, :, :, 1:] - labels[:, :, :, :-1]\n            doutputs_x = outputs[:, :, :, 1:] - outputs[:, :, :, :-1]\n            if white_pixel_weight is None:\n                loss_edge_y = mse_loss(dlabels_y, doutputs_y)\n                loss_edge_x = mse_loss(dlabels_x , doutputs_x)\n            else:\n                loss_edge_y = weighted_loss((dlabels_y - doutputs_y)**2, white_pixels=1*(dlabels_y > white_threshold), white_pixel_weight=white_pixel_weight)\n                loss_edge_x = weighted_loss((dlabels_x - doutputs_x)**2, white_pixels=1*(dlabels_x > white_threshold), white_pixel_weight=white_pixel_weight)\n            loss_edge = (loss_edge_y + loss_edge_x)/2\n            loss = (1 - loss_edge_weight)*mse_loss_pixels + loss_edge_weight*loss_edge\n        loss.backward()\n        optimiser.step()\n        loss_buffer.append(loss.detach().cpu().item())\n        if iter_count % (report_every//10) == 0:\n            print('.', end='')\n        if iter_count == 1 or iter_count % report_every == 0:\n            print(f'\\n######################## ITERATION #{iter_count}')\n            print(f'Mean training loss (n={len(loss_buffer)}): {np.mean(loss_buffer)}')\n            print(f'Elapsed time: {(datetime.now() - t0).total_seconds()/60:0.2f} minutes.')\n            outputs_np = outputs.detach().cpu().numpy()\n            outputs_bonds_np = outputs_np[0, 0]\n            outputs_atoms_np = outputs_np[0, 1]\n            outputs_combined_np = outputs_bonds_np + outputs_atoms_np\n            _, axs = plt.subplots(1, 6, figsize=(6*2.5, 2.5), facecolor='#eee');\n            axs[0].imshow(inputs.detach().cpu().numpy()[0, 0], vmin=0, vmax=1, cmap='gray_r')\n            axs[0].set_title('input')\n            axs[1].imshow(labels.detach().cpu().numpy()[0, 0], vmin=0, vmax=1, cmap='magma')\n            axs[1].set_title('label (bonds)')\n            axs[2].imshow(labels.detach().cpu().numpy()[0, 1], vmin=0, vmax=1, cmap='magma')\n            axs[2].set_title('label (atoms)')\n            axs[3].imshow(outputs_bonds_np, vmin=0, vmax=1, cmap='magma')\n            axs[3].set_title('output (bonds)')\n            axs[4].imshow(outputs_atoms_np, vmin=0, vmax=1, cmap='magma')\n            axs[4].set_title('output (atoms)')\n            axs[5].imshow(outputs_combined_np, vmin=0, vmax=1, cmap='gray_r')\n            axs[5].set_title('output')\n            for ax in axs:\n                ax.axis('off')\n            plt.show()\n            loss_buffer = []\n        if num_iters is not None and iter_count == num_iters:\n            break\n    print(f'Time taken: {(datetime.now() - t0).total_seconds()/60:0.2f} minutes.')\n    print('\\nSaving model…')\n    model = model.cpu()\n    model_file_name = f'{model_name}_{run_name}.pt'\n    model_path = OUTPUT_DIR / model_file_name\n    if model_path.exists():\n        print(f'WARNING: model file {model_path} already exists. Adding a timestamp to the file name and continuing…')\n        model_file_name = f'{model_name}_{run_name}_{timestamp()}.pt'\n        model_path = OUTPUT_DIR / model_file_name\n    torch.save({'model': model.state_dict(), 'options': options}, model_path)\n    print(f'Saved to {model_path}.')\n    print('Done.')\n    return model_path\n\n\nCNS_NET_PATH = train(lambda options: CleanAndSepNet3(options), MODEL_OPTIONS, TRAINING_OPTIONS['num_iterations'], report_every=500)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Sanity Check\n\nBelow I check the model can handle an image from the _original_ BMS dataset. The results should be consistent when the model is applied to either the original synthetic images or the ones I generate during training. This appears to be have been successful."},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_model(filename):\n    path = OUTPUT_DIR / filename\n    loaded = torch.load(path)\n    print(f'Loaded model from {path}.')\n    options = loaded['options']\n    model = CleanAndSepNet3(options)\n    model.load_state_dict(loaded['model'])\n    return model\n\n\ndef image_widget(a, greyscale=True):\n    img_bytes = BytesIO()\n    img_pil = Image.fromarray(a)\n    if greyscale:\n        img_pil = img_pil.convert(\"L\")\n    else:\n        img_pil = img_pil.convert(\"RGB\")\n    img_pil.save(img_bytes, format='PNG')\n    return widgets.Image(value=img_bytes.getvalue(), margin='1em;')\n\n\ndef prepare_image(img, size=None):\n    h, w = img.shape\n    # we need to pad to dimensions that are multiples of 16\n    needs_pad = False\n    if size is None:\n        if h % 16 == 0:\n            new_h = h\n        else:\n            new_h = ((h // 16) + 1)*16\n            needs_pad = True\n        if w % 16 == 0:\n            new_w = w\n        else:\n            new_w = ((w // 16) + 1)*16\n            needs_pad = True\n    else:\n        if isinstance(size, int):\n            size = (size, size)\n        new_h, new_w = size\n        if h != size:\n            needs_pad = True\n        if w != size:\n            needs_pad = True\n    if needs_pad:\n        margin_h = new_h - h\n        margin_top = margin_h//2\n        margin_bottom = margin_h - margin_top\n        margin_w = new_w - w\n        margin_left = margin_w//2\n        margin_right = margin_w - margin_left\n        padding = ((margin_top, margin_bottom), (margin_left, margin_right))\n        img = np.pad(img, padding)\n    else:\n        padding = ((0, 0), (0, 0))\n    return img, padding\n\n\ndef evaluate_model(model, img):\n    img, _ = prepare_image(img)\n    h, w = img.shape\n    inputs = torch.tensor(img.reshape(1, 1, h, w), dtype=torch.float32)\n    outputs = model(inputs).numpy()\n    bond_img, atom_img = outputs[0, :2]\n    return img, bond_img, atom_img\n\n\ndef sanity_check_model(model):\n    mol_index = np.random.randint(len(TRAIN_LABELS))\n    mol_id = TRAIN_LABELS['image_id'][mol_index]\n    mol_train_img_path = TRAIN_DATA_PATH / mol_id[0] /mol_id[1] / mol_id[2] / (mol_id + '.png')\n    img = imread(mol_train_img_path)\n    img = 1 - img/255.0\n    with torch.no_grad():\n        img, bond_img, atom_img = evaluate_model(model, img)\n    output_img = bond_img + atom_img\n    #output_img = np.clip(output_img, 0, 1)\n    #output_img = stretch_image(output_img)\n    #output_img = output_img*(output_img > 0.1)\n    border = '1px solid pink'\n    widget1 = image_widget(255*(1 - img))\n    widget1.layout.border = border\n    widget2 = image_widget(255*(1 - output_img))\n    widget2.layout.border = border\n    img_row = widgets.HBox([\n        widgets.VBox([widgets.Label('Input'), widget1]),\n        widgets.VBox([widgets.Label('Output'), widget2])\n    ])\n    display(img_row)\n    widget1 = image_widget(255*(1 - bond_img))\n    widget1.layout.border = border\n    widget2 = image_widget(255*(1 - atom_img))\n    widget2.layout.border = border\n    img_row = widgets.HBox([\n        widgets.VBox([widgets.Label('Bond Output Layer'), widget1]),\n        widgets.VBox([widgets.Label('Atom Output Layer'), widget2])\n    ])\n    display(img_row)\n    print(f'No. of trainable parameters: {sum(p.numel() for p in model.parameters() if p.requires_grad)}')\n\n\nwith torch.no_grad():\n    sanity_check_model(load_model(CNS_NET_PATH))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Validation"},{"metadata":{"trusted":true},"cell_type":"code","source":"# TODO: run batches?\ndef validate_model(model, num=100):\n    print(f'Validating {num} images…')\n    t0 = datetime.now()\n    measures = {\n        'bond_mse': [],\n        'atom_mse': [],\n        'mse': [],\n        'mse_weighted': []\n    }\n    for mol_index in range(num):\n        mol_id, inchi = TRAIN_LABELS['image_id'][mol_index], TRAIN_LABELS['InChI'][mol_index]\n        mol = Chem.inchi.MolFromInchi(inchi)\n        img, bond_img, atom_img = random_molecule_image(mol, margin_fraction=TRAINING_OPTIONS['margin_fraction'])\n        h, w = img.shape\n        bond_img = bond_img.reshape(1, h, w)\n        atom_img = atom_img.reshape(1, h, w)\n        label = np.concatenate((bond_img, atom_img), axis=0)\n        orig_img = img\n        img, padding = prepare_image(img)\n        h_padded, w_padded = img.shape\n        inputs = torch.tensor(img.reshape(1, 1, h_padded, w_padded), dtype=torch.float32)\n        outputs = model(inputs).numpy()\n        # unpad\n        (margin_top, margin_bottom), (margin_left, margin_right) = padding\n        if h_padded != h:\n            outputs = outputs[:, :, margin_top:-margin_bottom, :]\n        if w_padded != w:\n            outputs = outputs[:, :, :, margin_left:-margin_right]\n        prediction = outputs[0, :]\n        pred_bond_img = outputs[0, 0]\n        pred_atom_img = outputs[0, 1]\n        pred_img = np.clip(pred_bond_img + pred_atom_img, 0, 1)\n        # find losses\n        measures['bond_mse'].append(mse_loss(pred_bond_img, bond_img))\n        measures['atom_mse'].append(mse_loss(pred_atom_img, atom_img))\n        assert prediction.shape == label.shape, f'{prediction.shape} v. {label.shape}'\n        mse = mse_loss(label, prediction)\n        mse_weighted = mse_loss_weighted(label, prediction, white_pixel_weight=0.95)\n        measures['mse'].append(mse)\n        measures['mse_weighted'].append(mse_weighted)\n        print('.', end='')\n    measures['ave_bond_mse'] = np.mean(measures['bond_mse'])\n    measures['ave_atom_mse'] = np.mean(measures['atom_mse'])\n    measures['ave_mse'] = np.mean(measures['mse'])\n    measures['ave_mse_weighted'] = np.mean(measures['mse_weighted'])\n    print()\n    print()\n    print(f'Average MSE: {measures[\"ave_mse\"]:0.8f}')\n    print(f'Average weighted MSE: {measures[\"ave_mse_weighted\"]:0.8f}')\n    print()\n    print(f'Finished. Time taken: {(datetime.now() - t0).total_seconds()/60:0.2f} minutes.')\n\n\nwith torch.no_grad():\n    validate_model(load_model(CNS_NET_PATH), 500)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}