{"metadata":{"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":73047,"databundleVersionId":8149390,"sourceType":"competition"}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"papermill":{"default_parameters":{},"duration":16094.562879,"end_time":"2024-03-31T23:13:10.206610","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-03-31T18:44:55.643731","version":"2.4.0"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"01d435f46501493dbac04b363baa8d95":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"070bffc587dd472d8f098753f13db68b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5030a8d1602b4815be1bfa67b06badc0","placeholder":"​","style":"IPY_MODEL_3f0e80ae7cf54b6f8093abb27f17e29f","value":"#1: 100%"}},"0892f39095e7493a877d5f3fa84bb4e0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"0c5fcd7b99494a26a0920771da8962f3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e1c1f65062ed49bea1d1a9b2e6fa230f","max":5945,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1abf867d05434f4ebe650b6bee179632","value":5945}},"1161be6746cd426da12179134a014109":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6cab8498ee1948bd950775c4ef5babae","max":5945,"min":0,"orientation":"horizontal","style":"IPY_MODEL_bac41f42645442b69f6ab2e6bb29c15e","value":5945}},"1209bee0bda84478994bc13fe091361e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"14ae31ffcdab4ec6bf88cffeef9cf958":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1541a82dc59640fe99b89baf8f604129":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1550c89e543c4bc3947bc21b4b6ab5d1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3439da599fc2439aaa67caa2392b12e9","placeholder":"​","style":"IPY_MODEL_a6e3f675cd4a402d8bc695f732860c70","value":"#0: 100%"}},"1abf867d05434f4ebe650b6bee179632":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1bf6aeb7f77e49609b955f73205c39a3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_340b18d3ef214efd807b7d7dc8f1c844","placeholder":"​","style":"IPY_MODEL_c08ab36d89d24152b950b802c3c3549e","value":" 50/50 [00:12&lt;00:00,  5.54ex/s]"}},"1c862361dbdf410eab88d9996fea7518":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_1541a82dc59640fe99b89baf8f604129","max":50,"min":0,"orientation":"horizontal","style":"IPY_MODEL_b1ba9b8e0615420084f384ee3add6f74","value":50}},"1ec9d307396a47c1acbe2d9bfb1864a4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1f2b8a06b268473fa0d5a4e7eccdd2b7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"23bd4fbcad7d45e6b43dbd7fb5e7c0ee":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"274d532cbef04aefa40653b45e0dcaba":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_3e1f55f189664649916655100b4dbf44","max":12,"min":0,"orientation":"horizontal","style":"IPY_MODEL_983d1fa189b7479e8bfe6d61ccf21683","value":12}},"27a77748ecc44b3dbe825085f1758012":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6bbec8fc03c84f8d99e338202ac36969","placeholder":"​","style":"IPY_MODEL_cc8ac841e97c42a9884d5ffda2c824a8","value":"#0: 100%"}},"2cb8bf7031ec40c9a2fe2e78636a1e53":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2cfeb59bd54c45cebf99550441ad8216":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2e363940009342b48a629f5f15cb200a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"33f6e7800b734398b7affe18292d3791":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"340b18d3ef214efd807b7d7dc8f1c844":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3439da599fc2439aaa67caa2392b12e9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3965cd24339845c9bae310f61e360b31":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"39e62a3b5372466bbaeabc7de53137f0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8ff05de31b204b6ba5067c2753e8bb8e","placeholder":"​","style":"IPY_MODEL_ae051c22190b41ecb64a5fc9cf64aced","value":" 1/1 [00:16&lt;00:00, 16.34s/ba]"}},"3d59a5926cc44a06b7bf0bb16981617f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c70d2e2541664d2b9f4c73824598e293","placeholder":"​","style":"IPY_MODEL_b0f8995be1c74663a0a26598cd8e0291","value":" 12/12 [00:03&lt;00:00,  4.06ba/s]"}},"3e1f55f189664649916655100b4dbf44":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3e7f59e7c45d49d4bcdedeaaa7834009":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1f2b8a06b268473fa0d5a4e7eccdd2b7","placeholder":"​","style":"IPY_MODEL_7abed243231e464ca13247fbcc74cc78","value":" 12/12 [32:46&lt;00:00, 159.25s/ba]"}},"3f0967d2e8334efba09da56322eb3ed5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3f0e80ae7cf54b6f8093abb27f17e29f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"40227b304a2e460584cab73e52fda846":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_1ec9d307396a47c1acbe2d9bfb1864a4","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_f5b1765a2ebf4cc2b7bb83d0864379b3","value":1}},"4bf57bccebfa4570993900a54f995c6a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4cfc931424bf4b449733f228b097629e","placeholder":"​","style":"IPY_MODEL_6714b3f4895940459f424401041bcc80","value":"100%"}},"4cfc931424bf4b449733f228b097629e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4fb910936a7a4f1d91e6b05d77e5d246":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"500a7ca0647645728cef9973cbe28b11":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5030a8d1602b4815be1bfa67b06badc0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5357fb5775df47baa1d12812c91421b2":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d6df3a4d785347c8b48ab3d2704a511b","IPY_MODEL_274d532cbef04aefa40653b45e0dcaba","IPY_MODEL_3e7f59e7c45d49d4bcdedeaaa7834009"],"layout":"IPY_MODEL_2cfeb59bd54c45cebf99550441ad8216"}},"5803a1b385e0402388f3a5b0bd5fe0be":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5c61daa0a9f34462b188c2f9d492e24f","placeholder":"​","style":"IPY_MODEL_0892f39095e7493a877d5f3fa84bb4e0","value":" 1/1 [00:00&lt;00:00, 26.17ba/s]"}},"5c61daa0a9f34462b188c2f9d492e24f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5d3e1a4e474a4e2298aba0ef9d3128de":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_14ae31ffcdab4ec6bf88cffeef9cf958","placeholder":"​","style":"IPY_MODEL_3965cd24339845c9bae310f61e360b31","value":"100%"}},"5fc5f3a2eeab4f698e1561eba43eef74":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a0d0532c3e584bef8e2977b47cfb42cb","placeholder":"​","style":"IPY_MODEL_01d435f46501493dbac04b363baa8d95","value":" 5945/5945 [19:21&lt;00:00,  5.46ex/s]"}},"6039cac606b9488ba16c4f1c9c765740":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a55c7252242e4289a2f5e20e8d2c01c1","placeholder":"​","style":"IPY_MODEL_3f0967d2e8334efba09da56322eb3ed5","value":" 50/50 [00:12&lt;00:00,  5.50ex/s]"}},"637b9afbc97e4804ad676b34bdc2b0e7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6d960cdca95e403c84db0e1a7b03ba42","max":12,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6a80dcb946d04a06b2f067ae1f4e0d73","value":12}},"63fa0ebd983f4cd2ba035ccf2b715964":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"64514907acb74a28a1ba5f44d726553e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c36d6603bab0418d92ad592ee3931e16","IPY_MODEL_40227b304a2e460584cab73e52fda846","IPY_MODEL_39e62a3b5372466bbaeabc7de53137f0"],"layout":"IPY_MODEL_7077883e4a944aa29e8f935786c9e84a"}},"6714b3f4895940459f424401041bcc80":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6a80dcb946d04a06b2f067ae1f4e0d73":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6bbec8fc03c84f8d99e338202ac36969":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6cab8498ee1948bd950775c4ef5babae":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6d960cdca95e403c84db0e1a7b03ba42":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7077883e4a944aa29e8f935786c9e84a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"77df469a31a54b8a9fd3ea838d2ab651":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"77f645df7ea448df8216d2425e5dc5e0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7abed243231e464ca13247fbcc74cc78":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7ad603d3cb024dd1bdb8345590bb56cc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_27a77748ecc44b3dbe825085f1758012","IPY_MODEL_f10bc1cad4c24345add86cf26539f06b","IPY_MODEL_6039cac606b9488ba16c4f1c9c765740"],"layout":"IPY_MODEL_ad0a389bbac24cc0bc9fa7800d788dd0"}},"8ff05de31b204b6ba5067c2753e8bb8e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"900f068116434633b50f71187ecac4ab":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"91ad01050bbc484b8dbe94ae73a1f25a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"983d1fa189b7479e8bfe6d61ccf21683":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a0d0532c3e584bef8e2977b47cfb42cb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a14c485942864774ad92790aaeb34954":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4bf57bccebfa4570993900a54f995c6a","IPY_MODEL_637b9afbc97e4804ad676b34bdc2b0e7","IPY_MODEL_3d59a5926cc44a06b7bf0bb16981617f"],"layout":"IPY_MODEL_e9c8e3f3f41047de9b800352d460d598"}},"a3deda8ae8c24fb286364830dcd79df3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_070bffc587dd472d8f098753f13db68b","IPY_MODEL_0c5fcd7b99494a26a0920771da8962f3","IPY_MODEL_5fc5f3a2eeab4f698e1561eba43eef74"],"layout":"IPY_MODEL_c0484d2bdc2645acacfaa5769bd16a52"}},"a55c7252242e4289a2f5e20e8d2c01c1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a6e3f675cd4a402d8bc695f732860c70":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ad0a389bbac24cc0bc9fa7800d788dd0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ae051c22190b41ecb64a5fc9cf64aced":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b0f8995be1c74663a0a26598cd8e0291":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b1ba9b8e0615420084f384ee3add6f74":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b8fd79ba9dad4fc7bd701df4b5707f95":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1209bee0bda84478994bc13fe091361e","placeholder":"​","style":"IPY_MODEL_77f645df7ea448df8216d2425e5dc5e0","value":" 5945/5945 [19:22&lt;00:00,  6.03ex/s]"}},"bac41f42645442b69f6ab2e6bb29c15e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c0484d2bdc2645acacfaa5769bd16a52":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c08ab36d89d24152b950b802c3c3549e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c1fc2ded48fd4bfcb0fbd79fb0da507b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_dae001854e6f44c9b659d993e77946fb","placeholder":"​","style":"IPY_MODEL_fabd0f669bb948be928c932f1a67b21d","value":"#1: 100%"}},"c36d6603bab0418d92ad592ee3931e16":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_33f6e7800b734398b7affe18292d3791","placeholder":"​","style":"IPY_MODEL_2cb8bf7031ec40c9a2fe2e78636a1e53","value":"100%"}},"c70d2e2541664d2b9f4c73824598e293":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cc8ac841e97c42a9884d5ffda2c824a8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d0574628ade641c686300d13063cebc7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4fb910936a7a4f1d91e6b05d77e5d246","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_dd62e1a1b5af4d3d8784e0b84557a70c","value":1}},"d6df3a4d785347c8b48ab3d2704a511b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_500a7ca0647645728cef9973cbe28b11","placeholder":"​","style":"IPY_MODEL_900f068116434633b50f71187ecac4ab","value":"100%"}},"dae001854e6f44c9b659d993e77946fb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dd62e1a1b5af4d3d8784e0b84557a70c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e1c1f65062ed49bea1d1a9b2e6fa230f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e9c8e3f3f41047de9b800352d460d598":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eb69a06bbf4c4bf9b238dd38f5bf1248":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_5d3e1a4e474a4e2298aba0ef9d3128de","IPY_MODEL_d0574628ade641c686300d13063cebc7","IPY_MODEL_5803a1b385e0402388f3a5b0bd5fe0be"],"layout":"IPY_MODEL_77df469a31a54b8a9fd3ea838d2ab651"}},"f10bc1cad4c24345add86cf26539f06b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_63fa0ebd983f4cd2ba035ccf2b715964","max":50,"min":0,"orientation":"horizontal","style":"IPY_MODEL_91ad01050bbc484b8dbe94ae73a1f25a","value":50}},"f5b1765a2ebf4cc2b7bb83d0864379b3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f79bb61c048e47b1a39525b790f21907":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c1fc2ded48fd4bfcb0fbd79fb0da507b","IPY_MODEL_1c862361dbdf410eab88d9996fea7518","IPY_MODEL_1bf6aeb7f77e49609b955f73205c39a3"],"layout":"IPY_MODEL_23bd4fbcad7d45e6b43dbd7fb5e7c0ee"}},"f96747fd13064e8abaf0deb9f7642d25":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1550c89e543c4bc3947bc21b4b6ab5d1","IPY_MODEL_1161be6746cd426da12179134a014109","IPY_MODEL_b8fd79ba9dad4fc7bd701df4b5707f95"],"layout":"IPY_MODEL_2e363940009342b48a629f5f15cb200a"}},"fabd0f669bb948be928c932f1a67b21d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}},"version_major":2,"version_minor":0}}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"!pip install -q bitsandbytes #accelerate bitsandbytes==0.37\n!pip install -q git+https://github.com/huggingface/peft.git@main\n!pip install -q git+https://github.com/huggingface/accelerate.git@main\n# !pip install -q noisereduce","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":94.261549,"end_time":"2024-03-31T18:46:40.382527","exception":false,"start_time":"2024-03-31T18:45:06.120978","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:56:39.021554Z","iopub.execute_input":"2024-04-05T11:56:39.022039Z","iopub.status.idle":"2024-04-05T11:57:54.111760Z","shell.execute_reply.started":"2024-04-05T11:56:39.022011Z","shell.execute_reply":"2024-04-05T11:57:54.110391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\n\nimport pandas as pd\n\nimport librosa\n\nimport numpy as np\n\nimport IPython.display as ipd\n\nimport torchaudio\n\nimport librosa.display\n\nimport random\n\nfrom collections import Counter\n\nfrom sklearn.model_selection import train_test_split\n\nimport torch\n\nfrom dataclasses import dataclass\nfrom typing import Any, Dict, List, Union\nfrom datasets import DatasetDict\nfrom datasets import Dataset as DS\n\nfrom transformers import (\n    WhisperFeatureExtractor,\n    WhisperTokenizer,\n    WhisperProcessor,\n    WhisperForConditionalGeneration,\n    BitsAndBytesConfig,\n    Seq2SeqTrainingArguments,\n    Seq2SeqTrainer, \n    TrainerCallback, \n    TrainingArguments, \n    TrainerState, \n    TrainerControl,\n    EarlyStoppingCallback,\n    pipeline\n)\n\nfrom peft import (\n    prepare_model_for_kbit_training,\n    LoraConfig, \n    PeftModel, \n    LoraModel, \n    LoraConfig, \n    get_peft_model,\n    PeftConfig\n)\n\nfrom torchmetrics.text import WordErrorRate, CharErrorRate","metadata":{"papermill":{"duration":17.317319,"end_time":"2024-03-31T18:46:57.720047","exception":false,"start_time":"2024-03-31T18:46:40.402728","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T19:50:43.122877Z","iopub.execute_input":"2024-04-05T19:50:43.123744Z","iopub.status.idle":"2024-04-05T19:50:43.131483Z","shell.execute_reply.started":"2024-04-05T19:50:43.123711Z","shell.execute_reply":"2024-04-05T19:50:43.130510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"","metadata":{"execution":{"iopub.status.busy":"2024-04-05T11:58:32.138025Z","iopub.execute_input":"2024-04-05T11:58:32.138580Z","iopub.status.idle":"2024-04-05T11:58:32.144487Z","shell.execute_reply.started":"2024-04-05T11:58:32.138555Z","shell.execute_reply":"2024-04-05T11:58:32.142323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BASE_DIR = '/kaggle/input/ben10/ben10'\ntrain_data_dir = f\"{BASE_DIR}/16_kHz_train_audio/\"\ntest_data_dir = f\"{BASE_DIR}/16_kHz_valid_audio/\"\ndata_path = f\"{BASE_DIR}/train.csv\"","metadata":{"papermill":{"duration":0.027255,"end_time":"2024-03-31T18:46:57.767096","exception":false,"start_time":"2024-03-31T18:46:57.739841","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:58:32.147641Z","iopub.execute_input":"2024-04-05T11:58:32.148349Z","iopub.status.idle":"2024-04-05T11:58:32.159963Z","shell.execute_reply.started":"2024-04-05T11:58:32.148323Z","shell.execute_reply":"2024-04-05T11:58:32.159182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"split2path = {\n    \"train\": train_data_dir,\n    \"test\": test_data_dir\n}","metadata":{"papermill":{"duration":0.026367,"end_time":"2024-03-31T18:46:57.812699","exception":false,"start_time":"2024-03-31T18:46:57.786332","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:58:32.160967Z","iopub.execute_input":"2024-04-05T11:58:32.161261Z","iopub.status.idle":"2024-04-05T11:58:32.170949Z","shell.execute_reply.started":"2024-04-05T11:58:32.161238Z","shell.execute_reply":"2024-04-05T11:58:32.170116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = pd.read_csv(data_path)\ndata.sample(10)","metadata":{"papermill":{"duration":2.759101,"end_time":"2024-03-31T18:47:00.591588","exception":false,"start_time":"2024-03-31T18:46:57.832487","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:58:32.172051Z","iopub.execute_input":"2024-04-05T11:58:32.172336Z","iopub.status.idle":"2024-04-05T11:58:32.426458Z","shell.execute_reply.started":"2024-04-05T11:58:32.172313Z","shell.execute_reply":"2024-04-05T11:58:32.425405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def extract_split(filename):\n    filename_ = filename.split(\"_\")\n    split = filename_[0]\n    return split\n\ndef extract_district(filename):\n    filename_ = filename.split(\" \")[0]\n    district = filename_.split(\"_\")[1]\n    return district\n\ndef beautify_dataset(data):\n    splits = []\n    districts = []\n    newpaths = []\n    transcripts = []\n    \n    for i in range(len(data)):\n        filename, transcript = data.iloc[i]\n        split = extract_split(filename)\n        district = extract_district(filename)\n        dir_path = split2path[split]\n        composed_path = f\"{dir_path}{filename}\"\n        \n        if os.path.exists(composed_path) == False:\n            print(f\"{composed_path} does not exist.\")\n            continue\n        \n        transcript = transcript.replace(\"\\n\", \" \")\n        transcript = \" \".join(transcript.split())\n        \n        splits.append(split)\n        districts.append(district)\n        newpaths.append(composed_path)\n        transcripts.append(transcript)\n    \n    data['file_path'] = newpaths\n    data['district'] = districts\n    data['split'] = splits\n    data['transcripts'] = transcripts\n    \n#     data.drop(columns=['file_name'], inplace=True)\n    \n    return data","metadata":{"papermill":{"duration":0.032203,"end_time":"2024-03-31T18:47:00.646898","exception":false,"start_time":"2024-03-31T18:47:00.614695","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:58:32.427954Z","iopub.execute_input":"2024-04-05T11:58:32.428268Z","iopub.status.idle":"2024-04-05T11:58:32.437939Z","shell.execute_reply.started":"2024-04-05T11:58:32.428243Z","shell.execute_reply":"2024-04-05T11:58:32.437026Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = beautify_dataset(data)","metadata":{"papermill":{"duration":33.160892,"end_time":"2024-03-31T18:47:33.827378","exception":false,"start_time":"2024-03-31T18:47:00.666486","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:58:32.439363Z","iopub.execute_input":"2024-04-05T11:58:32.439754Z","iopub.status.idle":"2024-04-05T11:59:01.636142Z","shell.execute_reply.started":"2024-04-05T11:58:32.439727Z","shell.execute_reply":"2024-04-05T11:59:01.635208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.sample(20)","metadata":{"papermill":{"duration":0.041176,"end_time":"2024-03-31T18:47:33.888638","exception":false,"start_time":"2024-03-31T18:47:33.847462","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:01.637324Z","iopub.execute_input":"2024-04-05T11:59:01.637628Z","iopub.status.idle":"2024-04-05T11:59:01.655553Z","shell.execute_reply.started":"2024-04-05T11:59:01.637604Z","shell.execute_reply":"2024-04-05T11:59:01.654611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.groupby(\"district\").sum()","metadata":{"papermill":{"duration":0.461934,"end_time":"2024-03-31T18:47:34.511231","exception":false,"start_time":"2024-03-31T18:47:34.049297","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:01.660181Z","iopub.execute_input":"2024-04-05T11:59:01.660482Z","iopub.status.idle":"2024-04-05T11:59:02.008410Z","shell.execute_reply.started":"2024-04-05T11:59:01.660450Z","shell.execute_reply":"2024-04-05T11:59:02.007444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[data[\"transcripts\"] == \"<>\"]","metadata":{"papermill":{"duration":0.039507,"end_time":"2024-03-31T18:47:34.572125","exception":false,"start_time":"2024-03-31T18:47:34.532618","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.009757Z","iopub.execute_input":"2024-04-05T11:59:02.010558Z","iopub.status.idle":"2024-04-05T11:59:02.027546Z","shell.execute_reply.started":"2024-04-05T11:59:02.010523Z","shell.execute_reply":"2024-04-05T11:59:02.026639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[data[\"transcripts\"] == \"\"]","metadata":{"papermill":{"duration":0.037112,"end_time":"2024-03-31T18:47:34.631914","exception":false,"start_time":"2024-03-31T18:47:34.594802","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.030556Z","iopub.execute_input":"2024-04-05T11:59:02.031403Z","iopub.status.idle":"2024-04-05T11:59:02.048086Z","shell.execute_reply.started":"2024-04-05T11:59:02.031376Z","shell.execute_reply":"2024-04-05T11:59:02.047119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[data[\"transcripts\"] == \".\"]","metadata":{"papermill":{"duration":0.03541,"end_time":"2024-03-31T18:47:34.690603","exception":false,"start_time":"2024-03-31T18:47:34.655193","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.049311Z","iopub.execute_input":"2024-04-05T11:59:02.049577Z","iopub.status.idle":"2024-04-05T11:59:02.066476Z","shell.execute_reply.started":"2024-04-05T11:59:02.049554Z","shell.execute_reply":"2024-04-05T11:59:02.065444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[data[\"transcripts\"] == \"..\"]","metadata":{"papermill":{"duration":0.037299,"end_time":"2024-03-31T18:47:34.749367","exception":false,"start_time":"2024-03-31T18:47:34.712068","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.067522Z","iopub.execute_input":"2024-04-05T11:59:02.067796Z","iopub.status.idle":"2024-04-05T11:59:02.085432Z","shell.execute_reply.started":"2024-04-05T11:59:02.067774Z","shell.execute_reply":"2024-04-05T11:59:02.084513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print(list(data[data['transcripts'] == ''].index))\ndata.drop(data[data['transcripts'] == ''].index, inplace=True)\n      \n# print(list(data[data['transcripts'] == '<>'].index))\ndata.drop(data[data['transcripts'] == \"<>\"].index, inplace=True)\n      \n# print(list(data[data['transcripts'] == '..'].index))\ndata.drop(data[data['transcripts'] == \"..\"].index, inplace=True)","metadata":{"papermill":{"duration":0.044491,"end_time":"2024-03-31T18:47:34.815300","exception":false,"start_time":"2024-03-31T18:47:34.770809","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.086697Z","iopub.execute_input":"2024-04-05T11:59:02.087010Z","iopub.status.idle":"2024-04-05T11:59:02.115448Z","shell.execute_reply.started":"2024-04-05T11:59:02.086986Z","shell.execute_reply":"2024-04-05T11:59:02.114690Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[\"transcripts\"].str.len().idxmin(), data[\"transcripts\"].str.len().idxmax(), data[\"transcripts\"].str.len().min(), data[\"transcripts\"].str.len().max()","metadata":{"papermill":{"duration":0.071358,"end_time":"2024-03-31T18:47:34.908019","exception":false,"start_time":"2024-03-31T18:47:34.836661","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.116569Z","iopub.execute_input":"2024-04-05T11:59:02.116880Z","iopub.status.idle":"2024-04-05T11:59:02.160216Z","shell.execute_reply.started":"2024-04-05T11:59:02.116854Z","shell.execute_reply":"2024-04-05T11:59:02.159257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[\"transcripts\"].info()","metadata":{"papermill":{"duration":0.037782,"end_time":"2024-03-31T18:47:34.967023","exception":false,"start_time":"2024-03-31T18:47:34.929241","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.161754Z","iopub.execute_input":"2024-04-05T11:59:02.162111Z","iopub.status.idle":"2024-04-05T11:59:02.180047Z","shell.execute_reply.started":"2024-04-05T11:59:02.162081Z","shell.execute_reply":"2024-04-05T11:59:02.179133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[\"transcripts\"].isna().sum()","metadata":{"papermill":{"duration":0.031448,"end_time":"2024-03-31T18:47:35.019903","exception":false,"start_time":"2024-03-31T18:47:34.988455","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.181122Z","iopub.execute_input":"2024-04-05T11:59:02.181392Z","iopub.status.idle":"2024-04-05T11:59:02.193749Z","shell.execute_reply.started":"2024-04-05T11:59:02.181369Z","shell.execute_reply":"2024-04-05T11:59:02.192882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data[\"transcripts\"] = data[\"transcripts\"].str.strip()","metadata":{"papermill":{"duration":0.037447,"end_time":"2024-03-31T18:47:35.079002","exception":false,"start_time":"2024-03-31T18:47:35.041555","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.194855Z","iopub.execute_input":"2024-04-05T11:59:02.195126Z","iopub.status.idle":"2024-04-05T11:59:02.214947Z","shell.execute_reply.started":"2024-04-05T11:59:02.195103Z","shell.execute_reply":"2024-04-05T11:59:02.214001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def display_examples(examples):\n#     print(examples)\n    \n#     for idx in examples:\n#         row = data.iloc[idx]\n#         audio_file_path = row['file_path']\n#         audio, sr = librosa.load(audio_file_path, sr=None)\n\n#         print()\n#         print(\"Transcription:\", row['transcripts'], \"\\nDistrict:\", row['district'], end=\"\\n\\n\")\n#         ipd.display(ipd.Audio(audio, rate=sr))\n#         print(\"============================================================================================================================================================\")\n\n\n# random_indices = [0, 20, 500, 3700, 7150, 14000]\n# display_examples(random_indices)","metadata":{"papermill":{"duration":0.028471,"end_time":"2024-03-31T18:47:35.232107","exception":false,"start_time":"2024-03-31T18:47:35.203636","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.216090Z","iopub.execute_input":"2024-04-05T11:59:02.216436Z","iopub.status.idle":"2024-04-05T11:59:02.226020Z","shell.execute_reply.started":"2024-04-05T11:59:02.216411Z","shell.execute_reply":"2024-04-05T11:59:02.225285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TASK = \"transcribe\"\nMODEL_NAME = \"openai/whisper-large-v2\"","metadata":{"papermill":{"duration":0.028265,"end_time":"2024-03-31T18:47:35.700127","exception":false,"start_time":"2024-03-31T18:47:35.671862","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.227003Z","iopub.execute_input":"2024-04-05T11:59:02.227245Z","iopub.status.idle":"2024-04-05T11:59:02.237405Z","shell.execute_reply.started":"2024-04-05T11:59:02.227224Z","shell.execute_reply":"2024-04-05T11:59:02.236723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"feature_extractor = WhisperFeatureExtractor.from_pretrained(MODEL_NAME)\ntokenizer = WhisperTokenizer.from_pretrained(MODEL_NAME, language='bn', task=TASK)\nprocessor = WhisperProcessor.from_pretrained(MODEL_NAME, language='bn', task=TASK)","metadata":{"papermill":{"duration":0.451518,"end_time":"2024-03-31T18:47:36.173564","exception":false,"start_time":"2024-03-31T18:47:35.722046","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:02.238462Z","iopub.execute_input":"2024-04-05T11:59:02.238774Z","iopub.status.idle":"2024-04-05T11:59:04.723276Z","shell.execute_reply.started":"2024-04-05T11:59:02.238750Z","shell.execute_reply":"2024-04-05T11:59:04.722414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ids = tokenizer.encode(\"\")\nids","metadata":{"papermill":{"duration":0.032036,"end_time":"2024-03-31T18:47:36.227928","exception":false,"start_time":"2024-03-31T18:47:36.195892","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.724382Z","iopub.execute_input":"2024-04-05T11:59:04.724716Z","iopub.status.idle":"2024-04-05T11:59:04.730869Z","shell.execute_reply.started":"2024-04-05T11:59:04.724685Z","shell.execute_reply":"2024-04-05T11:59:04.729906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tokenizer.decode(ids)","metadata":{"papermill":{"duration":0.030135,"end_time":"2024-03-31T18:47:36.280385","exception":false,"start_time":"2024-03-31T18:47:36.250250","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.731880Z","iopub.execute_input":"2024-04-05T11:59:04.732120Z","iopub.status.idle":"2024-04-05T11:59:04.755568Z","shell.execute_reply.started":"2024-04-05T11:59:04.732099Z","shell.execute_reply":"2024-04-05T11:59:04.754456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"@dataclass\nclass DataCollatorSpeechSeq2SeqWithPadding:\n    processor: Any\n\n    def __call__(self, features: List[Dict[str, Union[List[int], torch.Tensor]]]) -> Dict[str, torch.Tensor]:\n        # split inputs and labels since they have to be of different lengths and need different padding methods\n        # first treat the audio inputs by simply returning torch tensors\n        input_features = [{\"input_features\": feature[\"input_features\"]} for feature in features]\n        batch = self.processor.feature_extractor.pad(input_features, return_tensors=\"pt\")\n\n        # get the tokenized label sequences\n        label_features = [{\"input_ids\": feature[\"labels\"]} for feature in features]\n        # pad the labels to max length\n        labels_batch = self.processor.tokenizer.pad(label_features, return_tensors=\"pt\")\n\n        # replace padding with -100 to ignore loss correctly\n        labels = labels_batch[\"input_ids\"].masked_fill(labels_batch.attention_mask.ne(1), -100)\n\n        # if bos token is appended in previous tokenization step,\n        # cut bos token here as it's append later anyways\n        if (labels[:, 0] == self.processor.tokenizer.bos_token_id).all().cpu().item():\n            labels = labels[:, 1:]\n\n        batch[\"labels\"] = labels\n        \n        torch.cuda.empty_cache()\n        \n#         print(batch)\n# #         print(batch.shape)\n#         for x in batch[\"input_features\"]: \n#             print(x.shape)\n#         print(batch[\"labels\"].shape)\n\n        return batch","metadata":{"papermill":{"duration":0.035409,"end_time":"2024-03-31T18:47:36.922243","exception":false,"start_time":"2024-03-31T18:47:36.886834","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.756826Z","iopub.execute_input":"2024-04-05T11:59:04.757143Z","iopub.status.idle":"2024-04-05T11:59:04.767817Z","shell.execute_reply.started":"2024-04-05T11:59:04.757108Z","shell.execute_reply":"2024-04-05T11:59:04.766834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_collator = DataCollatorSpeechSeq2SeqWithPadding(processor=processor)","metadata":{"papermill":{"duration":0.029637,"end_time":"2024-03-31T18:47:36.975427","exception":false,"start_time":"2024-03-31T18:47:36.945790","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.769086Z","iopub.execute_input":"2024-04-05T11:59:04.769380Z","iopub.status.idle":"2024-04-05T11:59:04.786393Z","shell.execute_reply.started":"2024-04-05T11:59:04.769356Z","shell.execute_reply":"2024-04-05T11:59:04.785583Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prepare_dataset(example):\n    audio_path = example[\"file_path\"]\n    \n    audio, sr = librosa.load(audio_path, sr=16_000)\n    \n    example[\"input_features\"] = feature_extractor(audio, sampling_rate=sr).input_features[0]\n    \n    example[\"labels\"] = tokenizer(f\"{example['transcripts']}\", max_length=448, padding=True, truncation=True).input_ids\n    \n    return example\n\n\ndef filter_inputs(input_audio):\n    \"\"\"Filter inputs with zero input length\"\"\"\n    return 0 < len(input_audio)\n\n\ndef filter_labels(input_labels):\n    \"\"\"Filter empty label sequences\"\"\"\n    return 0 < len(input_labels)","metadata":{"papermill":{"duration":0.033357,"end_time":"2024-03-31T18:47:37.032173","exception":false,"start_time":"2024-03-31T18:47:36.998816","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.787518Z","iopub.execute_input":"2024-04-05T11:59:04.787807Z","iopub.status.idle":"2024-04-05T11:59:04.799730Z","shell.execute_reply.started":"2024-04-05T11:59:04.787785Z","shell.execute_reply":"2024-04-05T11:59:04.799027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = data[data[\"split\"] == \"train\"]","metadata":{"papermill":{"duration":0.038204,"end_time":"2024-03-31T18:47:37.093779","exception":false,"start_time":"2024-03-31T18:47:37.055575","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.806262Z","iopub.execute_input":"2024-04-05T11:59:04.806523Z","iopub.status.idle":"2024-04-05T11:59:04.822317Z","shell.execute_reply.started":"2024-04-05T11:59:04.806494Z","shell.execute_reply":"2024-04-05T11:59:04.821601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df, eval_df = train_test_split(train_df, test_size=0.01, shuffle=True, random_state=42)","metadata":{"execution":{"iopub.status.busy":"2024-04-05T11:59:04.823497Z","iopub.execute_input":"2024-04-05T11:59:04.824011Z","iopub.status.idle":"2024-04-05T11:59:04.833961Z","shell.execute_reply.started":"2024-04-05T11:59:04.823978Z","shell.execute_reply":"2024-04-05T11:59:04.833040Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(train_df), len(eval_df)","metadata":{"execution":{"iopub.status.busy":"2024-04-05T11:59:04.834943Z","iopub.execute_input":"2024-04-05T11:59:04.835298Z","iopub.status.idle":"2024-04-05T11:59:04.844559Z","shell.execute_reply.started":"2024-04-05T11:59:04.835256Z","shell.execute_reply":"2024-04-05T11:59:04.843653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ben_reg_voice_ds = DatasetDict()\n\ntrain_split = DS.from_pandas(train_df)\neval_split = DS.from_pandas(eval_df)\n\nds_splits = DatasetDict({\n    'train': train_split,\n    'eval': eval_split\n})","metadata":{"papermill":{"duration":0.082626,"end_time":"2024-03-31T18:47:37.253362","exception":false,"start_time":"2024-03-31T18:47:37.170736","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.845539Z","iopub.execute_input":"2024-04-05T11:59:04.845818Z","iopub.status.idle":"2024-04-05T11:59:04.911940Z","shell.execute_reply.started":"2024-04-05T11:59:04.845795Z","shell.execute_reply":"2024-04-05T11:59:04.911028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_splits = ds_splits.remove_columns([\"split\"])","metadata":{"papermill":{"duration":0.033799,"end_time":"2024-03-31T18:47:37.311407","exception":false,"start_time":"2024-03-31T18:47:37.277608","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.913000Z","iopub.execute_input":"2024-04-05T11:59:04.913276Z","iopub.status.idle":"2024-04-05T11:59:04.920560Z","shell.execute_reply.started":"2024-04-05T11:59:04.913253Z","shell.execute_reply":"2024-04-05T11:59:04.919684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(ds_splits)","metadata":{"papermill":{"duration":0.030856,"end_time":"2024-03-31T18:47:37.365863","exception":false,"start_time":"2024-03-31T18:47:37.335007","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.921547Z","iopub.execute_input":"2024-04-05T11:59:04.921930Z","iopub.status.idle":"2024-04-05T11:59:04.932057Z","shell.execute_reply.started":"2024-04-05T11:59:04.921899Z","shell.execute_reply":"2024-04-05T11:59:04.931267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.object = object","metadata":{"execution":{"iopub.status.busy":"2024-04-05T11:59:04.933117Z","iopub.execute_input":"2024-04-05T11:59:04.933681Z","iopub.status.idle":"2024-04-05T11:59:04.942353Z","shell.execute_reply.started":"2024-04-05T11:59:04.933632Z","shell.execute_reply":"2024-04-05T11:59:04.941536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_splits = ds_splits.map(prepare_dataset, remove_columns=ds_splits.column_names[\"train\"], num_proc=2)","metadata":{"papermill":{"duration":1249.191068,"end_time":"2024-03-31T19:08:26.581489","exception":false,"start_time":"2024-03-31T18:47:37.390421","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T11:59:04.943515Z","iopub.execute_input":"2024-04-05T11:59:04.944120Z","iopub.status.idle":"2024-04-05T12:05:25.189653Z","shell.execute_reply.started":"2024-04-05T11:59:04.944091Z","shell.execute_reply":"2024-04-05T12:05:25.186672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ds_splits = ds_splits.filter(filter_inputs, input_columns=[\"input_features\"])\n# ds_splits = ds_splits.filter(filter_labels, input_columns=[\"labels\"])","metadata":{"papermill":{"duration":1986.344397,"end_time":"2024-03-31T19:41:32.952021","exception":false,"start_time":"2024-03-31T19:08:26.607624","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:05:25.193480Z","iopub.execute_input":"2024-04-05T12:05:25.194353Z","iopub.status.idle":"2024-04-05T12:05:25.198231Z","shell.execute_reply.started":"2024-04-05T12:05:25.194316Z","shell.execute_reply":"2024-04-05T12:05:25.197491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(ds_splits[\"train\"]), len(ds_splits[\"eval\"])","metadata":{"papermill":{"duration":0.034501,"end_time":"2024-03-31T19:41:33.012856","exception":false,"start_time":"2024-03-31T19:41:32.978355","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:05:25.199529Z","iopub.execute_input":"2024-04-05T12:05:25.199876Z","iopub.status.idle":"2024-04-05T12:05:25.237460Z","shell.execute_reply.started":"2024-04-05T12:05:25.199847Z","shell.execute_reply":"2024-04-05T12:05:25.236550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cer = CharErrorRate()\nwer = WordErrorRate()","metadata":{"papermill":{"duration":0.112361,"end_time":"2024-03-31T19:41:33.151592","exception":false,"start_time":"2024-03-31T19:41:33.039231","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:05:25.238509Z","iopub.execute_input":"2024-04-05T12:05:25.238841Z","iopub.status.idle":"2024-04-05T12:05:25.261504Z","shell.execute_reply.started":"2024-04-05T12:05:25.238812Z","shell.execute_reply":"2024-04-05T12:05:25.260810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def compute_metrics(pred):\n    pred_ids = pred.predictions\n    label_ids = pred.label_ids\n\n    label_ids[label_ids == -100] = tokenizer.pad_token_id\n\n    pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True)\n    label_str = tokenizer.batch_decode(label_ids, skip_special_tokens=True)\n\n    wer_res = wer(pred_str, label_str)\n    cer_res = cer(pred_str, label_str)\n    \n    \"\"\"\n        uncomment the next 3 lines if you want to see how the examples look like during eval \n    \"\"\"\n    print(\"WER:\",wer_res,\"| CER:\", cer_res)\n    print(\"Pred:\",pred_str[0])\n    print(\"Label:\",label_str[0])\n    \n    return {\"wer\": wer_res, \"cer\": cer_res}","metadata":{"papermill":{"duration":0.037261,"end_time":"2024-03-31T19:41:33.217295","exception":false,"start_time":"2024-03-31T19:41:33.180034","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:05:25.262396Z","iopub.execute_input":"2024-04-05T12:05:25.262851Z","iopub.status.idle":"2024-04-05T12:05:25.275624Z","shell.execute_reply.started":"2024-04-05T12:05:25.262828Z","shell.execute_reply":"2024-04-05T12:05:25.274706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Setting up the model\n\nWe'll train the whisper-large-v2 model using 4bit LoRA","metadata":{}},{"cell_type":"code","source":"quant_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_quant_type=\"nf4\",\n    bnb_4bit_use_double_quant=True,\n    bnb_4bit_compute_dtype=torch.bfloat16,\n)\n\nmodel = WhisperForConditionalGeneration.from_pretrained(MODEL_NAME, quantization_config=quant_config, device_map=\"auto\")","metadata":{"execution":{"iopub.status.busy":"2024-04-05T12:05:25.276823Z","iopub.execute_input":"2024-04-05T12:05:25.277383Z","iopub.status.idle":"2024-04-05T12:05:57.599351Z","shell.execute_reply.started":"2024-04-05T12:05:25.277359Z","shell.execute_reply":"2024-04-05T12:05:57.598240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = prepare_model_for_kbit_training(model)","metadata":{"papermill":{"duration":0.032462,"end_time":"2024-03-31T19:41:43.207536","exception":false,"start_time":"2024-03-31T19:41:43.175074","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:05:57.601311Z","iopub.execute_input":"2024-04-05T12:05:57.602065Z","iopub.status.idle":"2024-04-05T12:05:57.678208Z","shell.execute_reply.started":"2024-04-05T12:05:57.602028Z","shell.execute_reply":"2024-04-05T12:05:57.677323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\"\"\"\n    whisper uses CNNs in the encoder\n    LoRA freezes all layers, but let's make the first layer (receiving layer) trainable.\n\"\"\"\ndef make_inputs_require_grad(module, input, output):\n    output.requires_grad_(True)\n\nmodel.model.encoder.conv1.register_forward_hook(make_inputs_require_grad)\n# model.model.encoder.conv2.register_forward_hook(make_inputs_require_grad)","metadata":{"papermill":{"duration":0.034133,"end_time":"2024-03-31T19:41:43.269493","exception":false,"start_time":"2024-03-31T19:41:43.235360","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:05:57.679427Z","iopub.execute_input":"2024-04-05T12:05:57.680074Z","iopub.status.idle":"2024-04-05T12:06:01.268699Z","shell.execute_reply.started":"2024-04-05T12:05:57.680049Z","shell.execute_reply":"2024-04-05T12:06:01.267716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lora_config = LoraConfig(r=32, lora_alpha=64, target_modules=[\"q_proj\", \"v_proj\"], lora_dropout=0.05, bias=\"none\", )\n\nmodel = get_peft_model(model, lora_config)","metadata":{"papermill":{"duration":0.032884,"end_time":"2024-03-31T19:41:43.329157","exception":false,"start_time":"2024-03-31T19:41:43.296273","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:06:01.270555Z","iopub.execute_input":"2024-04-05T12:06:01.271020Z","iopub.status.idle":"2024-04-05T12:06:01.963169Z","shell.execute_reply.started":"2024-04-05T12:06:01.270988Z","shell.execute_reply":"2024-04-05T12:06:01.962194Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.print_trainable_parameters()","metadata":{"execution":{"iopub.status.busy":"2024-04-05T12:06:01.964606Z","iopub.execute_input":"2024-04-05T12:06:01.965768Z","iopub.status.idle":"2024-04-05T12:06:01.984750Z","shell.execute_reply.started":"2024-04-05T12:06:01.965730Z","shell.execute_reply":"2024-04-05T12:06:01.983989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_id = \"whisper-reg-ben-large\"","metadata":{"papermill":{"duration":0.033791,"end_time":"2024-03-31T19:41:43.389235","exception":false,"start_time":"2024-03-31T19:41:43.355444","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:06:01.985829Z","iopub.execute_input":"2024-04-05T12:06:01.986196Z","iopub.status.idle":"2024-04-05T12:06:02.894326Z","shell.execute_reply.started":"2024-04-05T12:06:01.986164Z","shell.execute_reply":"2024-04-05T12:06:02.893141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_args = Seq2SeqTrainingArguments(\n    output_dir=model_id,\n    per_device_train_batch_size=9,\n    per_device_eval_batch_size=8,\n    gradient_accumulation_steps=1,\n    gradient_checkpointing=True,\n    fp16=True,\n    learning_rate=3e-4,\n    weight_decay=1e-2,\n    warmup_steps=50,\n    num_train_epochs=1,\n    evaluation_strategy=\"steps\", # or \"epochs\"\n    predict_with_generate=True,\n    save_steps=1400,\n    eval_steps=1400,\n    logging_steps=1400,\n    save_total_limit=1,\n    load_best_model_at_end=True,\n    metric_for_best_model=\"wer\",\n    greater_is_better=False,\n    push_to_hub=False,\n    report_to=\"none\",\n    optim = \"paged_adamw_8bit\",\n    remove_unused_columns=False, # comment out when not lora finetuning\n)","metadata":{"papermill":{"duration":0.037198,"end_time":"2024-03-31T19:41:43.453875","exception":false,"start_time":"2024-03-31T19:41:43.416677","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:06:02.895645Z","iopub.execute_input":"2024-04-05T12:06:02.896110Z","iopub.status.idle":"2024-04-05T12:06:02.931965Z","shell.execute_reply.started":"2024-04-05T12:06:02.896075Z","shell.execute_reply":"2024-04-05T12:06:02.931244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This callback helps to save only the adapter weights and remove the base model weights.\nclass SavePeftModelCallback(TrainerCallback):\n    def on_save(\n        self,\n        args: TrainingArguments,\n        state: TrainerState,\n        control: TrainerControl,\n        **kwargs,\n    ):\n\n        peft_model_path = model_id\n        kwargs[\"model\"].save_pretrained(peft_model_path)\n\n        pytorch_model_path = f\"{peft_model_path}/pytorch_model.bin\"\n        if os.path.exists(pytorch_model_path):\n            os.remove(pytorch_model_path)\n        return control","metadata":{"papermill":{"duration":0.037698,"end_time":"2024-03-31T19:41:43.519272","exception":false,"start_time":"2024-03-31T19:41:43.481574","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:06:02.933040Z","iopub.execute_input":"2024-04-05T12:06:02.933312Z","iopub.status.idle":"2024-04-05T12:06:02.939069Z","shell.execute_reply.started":"2024-04-05T12:06:02.933289Z","shell.execute_reply":"2024-04-05T12:06:02.938160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.generation_config.language = \"bn\"\nmodel.generation_config.task = \"transcribe\"\n\nmodel.generation_config.forced_decoder_ids = None\nmodel.config.suppress_tokens = []","metadata":{"papermill":{"duration":0.035535,"end_time":"2024-03-31T19:41:43.582970","exception":false,"start_time":"2024-03-31T19:41:43.547435","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:06:02.940036Z","iopub.execute_input":"2024-04-05T12:06:02.940289Z","iopub.status.idle":"2024-04-05T12:06:02.951066Z","shell.execute_reply.started":"2024-04-05T12:06:02.940267Z","shell.execute_reply":"2024-04-05T12:06:02.950199Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"trainer = Seq2SeqTrainer(\n    args=training_args,\n    model=model,\n    train_dataset=ds_splits[\"train\"],\n    eval_dataset=ds_splits[\"eval\"],\n    data_collator=data_collator,\n    tokenizer=processor.feature_extractor,\n    compute_metrics=compute_metrics,\n    callbacks=[SavePeftModelCallback],\n#     callbacks=[EarlyStoppingCallback(2, 1.0)],\n#     callbacks=[SavePeftModelCallback, EarlyStoppingCallback(2, 1.0)]\n)","metadata":{"papermill":{"duration":0.047378,"end_time":"2024-03-31T19:41:43.658088","exception":false,"start_time":"2024-03-31T19:41:43.610710","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:06:02.951895Z","iopub.execute_input":"2024-04-05T12:06:02.952132Z","iopub.status.idle":"2024-04-05T12:06:02.993537Z","shell.execute_reply.started":"2024-04-05T12:06:02.952112Z","shell.execute_reply":"2024-04-05T12:06:02.992764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"trainer.train()\n\nprocessor.save_pretrained(training_args.output_dir)","metadata":{"papermill":{"duration":6059.759102,"end_time":"2024-03-31T21:22:43.502342","exception":false,"start_time":"2024-03-31T19:41:43.743240","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T12:06:02.994555Z","iopub.execute_input":"2024-04-05T12:06:02.994913Z","iopub.status.idle":"2024-04-05T19:49:55.519903Z","shell.execute_reply.started":"2024-04-05T12:06:02.994880Z","shell.execute_reply":"2024-04-05T19:49:55.518916Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"out_logs = pd.DataFrame(trainer.state.log_history)\nout_logs.to_csv(\"logs.csv\")","metadata":{"papermill":{"duration":0.050809,"end_time":"2024-03-31T21:22:43.581624","exception":false,"start_time":"2024-03-31T21:22:43.530815","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T19:49:55.521218Z","iopub.execute_input":"2024-04-05T19:49:55.522085Z","iopub.status.idle":"2024-04-05T19:49:55.589316Z","shell.execute_reply.started":"2024-04-05T19:49:55.522048Z","shell.execute_reply":"2024-04-05T19:49:55.588620Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import gc\n\ndel ds_splits\n\ngc.collect()","metadata":{"papermill":{"duration":0.380015,"end_time":"2024-03-31T21:22:43.989457","exception":false,"start_time":"2024-03-31T21:22:43.609442","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T19:49:55.590304Z","iopub.execute_input":"2024-04-05T19:49:55.590579Z","iopub.status.idle":"2024-04-05T19:49:55.942342Z","shell.execute_reply.started":"2024-04-05T19:49:55.590555Z","shell.execute_reply":"2024-04-05T19:49:55.941453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"torch.cuda.empty_cache()","metadata":{"papermill":{"duration":0.053195,"end_time":"2024-03-31T21:22:44.070685","exception":false,"start_time":"2024-03-31T21:22:44.017490","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-04-05T19:49:55.943350Z","iopub.execute_input":"2024-04-05T19:49:55.943602Z","iopub.status.idle":"2024-04-05T19:49:56.062474Z","shell.execute_reply.started":"2024-04-05T19:49:55.943572Z","shell.execute_reply":"2024-04-05T19:49:56.061640Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"peft_config = PeftConfig.from_pretrained(model_id)","metadata":{"execution":{"iopub.status.busy":"2024-04-05T20:05:56.457908Z","iopub.execute_input":"2024-04-05T20:05:56.458424Z","iopub.status.idle":"2024-04-05T20:05:56.464050Z","shell.execute_reply.started":"2024-04-05T20:05:56.458386Z","shell.execute_reply":"2024-04-05T20:05:56.462893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = WhisperForConditionalGeneration.from_pretrained(\n    peft_config.base_model_name_or_path, quantization_config=quant_config, device_map=\"auto\"\n)\nmodel = PeftModel.from_pretrained(model, model_id)\nmodel.config.use_cache = True","metadata":{"execution":{"iopub.status.busy":"2024-04-05T20:05:57.633339Z","iopub.execute_input":"2024-04-05T20:05:57.634238Z","iopub.status.idle":"2024-04-05T20:06:32.031938Z","shell.execute_reply.started":"2024-04-05T20:05:57.634204Z","shell.execute_reply":"2024-04-05T20:06:32.030894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# tokenizer = WhisperTokenizer.from_pretrained(peft_config.base_model_name_or_path, language='bn', task=TASK)\n# processor = WhisperProcessor.from_pretrained(peft_config.base_model_name_or_path, language='bn', task=TASK)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pipe = pipeline(\n    \"automatic-speech-recognition\",\n    model=model,\n    tokenizer=processor.tokenizer,\n    feature_extractor=processor.feature_extractor, \n    chunk_length_s=15, # chunk 15 secs\n    torch_dtype=torch.float16\n)","metadata":{"execution":{"iopub.status.busy":"2024-04-05T20:26:20.494155Z","iopub.execute_input":"2024-04-05T20:26:20.494530Z","iopub.status.idle":"2024-04-05T20:26:20.501844Z","shell.execute_reply.started":"2024-04-05T20:26:20.494501Z","shell.execute_reply":"2024-04-05T20:26:20.500811Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"samp = pd.read_csv(\"/kaggle/input/ben10/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2024-04-05T20:26:20.962117Z","iopub.execute_input":"2024-04-05T20:26:20.962520Z","iopub.status.idle":"2024-04-05T20:26:20.974256Z","shell.execute_reply.started":"2024-04-05T20:26:20.962490Z","shell.execute_reply":"2024-04-05T20:26:20.973392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds = []","metadata":{"execution":{"iopub.status.busy":"2024-04-05T20:26:21.456402Z","iopub.execute_input":"2024-04-05T20:26:21.457143Z","iopub.status.idle":"2024-04-05T20:26:21.461553Z","shell.execute_reply.started":"2024-04-05T20:26:21.457111Z","shell.execute_reply":"2024-04-05T20:26:21.460479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### A caution while using generation with whisper!\n\nWhisper outputs to a max length of 448. During testing, it is better to split up the audio into chunks. This is done using the `chunk_s` argument when creating the pipeline.","metadata":{}},{"cell_type":"code","source":"for index, row in samp.iterrows():\n    file_name = row['id']\n    file_path = f\"{test_data_dir}{file_name}\"\n    \n    audio, sr = librosa.load(file_path, sr=16_000)\n    pred = pipe(audio)['text']\n    preds.append(pred)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"samp[\"sentence\"] = preds","metadata":{"execution":{"iopub.status.busy":"2024-04-05T19:49:56.887078Z","iopub.status.idle":"2024-04-05T19:49:56.887385Z","shell.execute_reply.started":"2024-04-05T19:49:56.887233Z","shell.execute_reply":"2024-04-05T19:49:56.887246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"samp.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2024-04-05T19:49:56.888512Z","iopub.status.idle":"2024-04-05T19:49:56.888864Z","shell.execute_reply.started":"2024-04-05T19:49:56.888695Z","shell.execute_reply":"2024-04-05T19:49:56.888713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}