{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <B><u>How to Use/Create Utility Scripts</u></B>","metadata":{"_uuid":"1d79467d-aef7-424f-af21-1c4457583f13","_cell_guid":"db897d47-26c1-4038-bc05-b1946fe8fc31","trusted":true}},{"cell_type":"markdown","source":"# <B>[1] Introduction</B>","metadata":{"_uuid":"3ff2e8c1-3d79-4e80-b8a9-600174533555","_cell_guid":"81f29d1f-5045-4624-9358-e82c560ba20f","trusted":true}},{"cell_type":"markdown","source":"The way to use/create utility scripts, which is a function of kaggel notebook, is shown in the kernel. It is a supplementary explanation of [the product feedback of utility scripts](https://www.kaggle.com/product-feedback/91185).","metadata":{"_uuid":"bdeed842-bac6-4410-8b3f-383f3ef18eb0","_cell_guid":"358dbf85-7a26-474a-99b2-4a40f8c77ced","trusted":true}},{"cell_type":"markdown","source":"There are several similar competitions. So it is very efficient to define general utitlity fucntions separately. It is strongly recommended to use utility scripts for every kaggle users.","metadata":{"_uuid":"4ae12053-2cc5-4201-99b2-757b756266c9","_cell_guid":"f42d6fb1-804b-46ac-8a49-b6a800cca593","trusted":true}},{"cell_type":"markdown","source":"NOTE1 : The author is a beginner of Kaggle/MachineLearning/Python/English. So the kernel may have several bugs/wrongs. I am happy to get your comments. Thank you in advance for your kind advice to make the kernel so NICE! and to make me NICE deep learning guy!!","metadata":{"_uuid":"98f2857e-91ac-4538-a3bc-92ab36591f5a","_cell_guid":"2dac0922-556f-408f-8c69-9dcfb047aa12","trusted":true}},{"cell_type":"markdown","source":"NOTE2 : A sample of utility scripts is shown in my other kernel [\"Utility Functions for Visualizing Dataset\"](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset)</u>. It is utility scripts for visualizing dataset for \"Happywhale - Whale and Dolphin Identification\" competion. But it is available for the other similar competitions. Sample usage of the utility scripts is shown in chapter 5 of the kernel.","metadata":{}},{"cell_type":"markdown","source":"# <B>[2] Creation of utility scripts</B>","metadata":{"_uuid":"247b6d03-b9e7-4c39-82cd-b2191f2a4d19","_cell_guid":"7f4717c5-98cd-4bd3-a4db-1f715e4a9182","trusted":true}},{"cell_type":"markdown","source":"## [2-1] Creation of new notebook","metadata":{"_uuid":"041f6692-cab8-428d-b8ab-7bb50e6e3419","_cell_guid":"a60c1df0-f847-4296-b0b7-952402c99781","trusted":true}},{"cell_type":"markdown","source":"Creates new notebook for defining utility scripts.","metadata":{"_uuid":"e6da791c-e343-4b7f-b11c-75932a91ed09","_cell_guid":"3d65d878-4ab6-405b-9142-2898d36ae410","trusted":true}},{"cell_type":"markdown","source":"## [2-2] Setting as utility script","metadata":{"_uuid":"8746b843-8c66-4e1a-a615-09c89cff20f9","_cell_guid":"fd933ded-8dd5-4c2c-9a75-4c546bb0a1c8","trusted":true}},{"cell_type":"markdown","source":"Sets the notebook as utility script by clicking the following items in the menu of notebook,","metadata":{"_uuid":"8baff756-ef6a-411f-9351-868e247c37b2","_cell_guid":"ab955a4e-b393-48d9-b81f-5dd38a629a15","trusted":true}},{"cell_type":"markdown","source":"### \"File\"  -->  \"Set as Utility Script\".","metadata":{"_uuid":"47ed7b23-38f3-4a4b-811b-48f76dd3fce5","_cell_guid":"c2f3a66c-c32e-438f-8e52-1c591936a991","trusted":true}},{"cell_type":"markdown","source":"## [2-3] Setting editor type","metadata":{"_uuid":"9b0fae6c-1bb7-4807-8f97-ae5456564c3b","_cell_guid":"907e9451-a898-4103-a81c-d365df8ebede","trusted":true}},{"cell_type":"markdown","source":"Sets editor type by clicking the following items in the menu of notebook,","metadata":{"_uuid":"a73b0142-1700-4383-8973-da68dc077f5a","_cell_guid":"46b2c78b-ed8b-4781-b538-f3f4bcd41e6c","trusted":true}},{"cell_type":"markdown","source":"### \"File\"  -->  \"Editor Type\"  -->  \"Script\".","metadata":{"_uuid":"67992cb6-32e8-4d08-8866-9500e315f9d9","_cell_guid":"6250e21b-2abf-45e7-ae0d-6453150811e6","trusted":true}},{"cell_type":"markdown","source":"## [2-4] Defining fuctions/classes","metadata":{"_uuid":"142b35b3-f073-4fae-9ccf-c46293e96de6","_cell_guid":"22efea23-4a59-41de-872d-4627f4a1a893","trusted":true}},{"cell_type":"markdown","source":"Defines required functions/classes in the notebook.","metadata":{"_uuid":"00b207ce-7684-457a-b836-15321dcc685c","_cell_guid":"020a070a-8cfd-46f7-835c-90779c551e29","trusted":true}},{"cell_type":"markdown","source":"## [2-5] Saving notebook","metadata":{"_uuid":"4cbab9af-3229-4afa-9a0f-2d5a575379d7","_cell_guid":"150adba1-b879-42ba-88c6-957a85257de1","trusted":true}},{"cell_type":"markdown","source":"Saves the notebook by clicking \"Save Version\".","metadata":{"_uuid":"577ef239-7d0a-449f-8f6e-2e7a1addc56a","_cell_guid":"fb9f00ac-f62e-4081-82f1-754d0f92466b","trusted":true}},{"cell_type":"markdown","source":"# <B>[3] Importing utility scripts</B>","metadata":{"_uuid":"404e8dfd-e1c3-4f95-b810-7ed33c018d7e","_cell_guid":"cb15a882-c7bf-48a4-a531-d8c1859a4e74","trusted":true}},{"cell_type":"markdown","source":"## [3-1] Creation of new notebook","metadata":{"_uuid":"8351b288-694c-4f8b-860c-97fb49838633","_cell_guid":"1a7c0cfd-85b2-43ce-b05d-8c86ca2031c8","trusted":true}},{"cell_type":"markdown","source":"Creates the other new notebook for importing utility scripts, if it is required.","metadata":{"_uuid":"cdea4424-2585-403f-9793-4ec861746775","_cell_guid":"4788a0ee-063f-4faf-a38d-1164fb35b1cb","trusted":true}},{"cell_type":"markdown","source":"## [3-2] Adding utility scripts","metadata":{"_uuid":"8c9f23a9-0de3-4bfd-90e6-1405f0509372","_cell_guid":"7f1962a4-fc6a-426c-a000-0415eed106aa","trusted":true}},{"cell_type":"markdown","source":"Adds utility scripts to the notebook by clicking the following items in the menu of notebook,","metadata":{"_uuid":"99e44a27-6615-4eb7-9637-1e4f83b62b03","_cell_guid":"19ac5c9a-f28a-49c8-a7c7-6174ada5b906","trusted":true}},{"cell_type":"markdown","source":"### \"File\"  -->  \"Add utility script\".","metadata":{"_uuid":"e76d611a-f0e6-4f27-aa49-c078f6b99b9b","_cell_guid":"2b5ae942-e9d5-48a5-8a06-8d3b41414328","trusted":true}},{"cell_type":"markdown","source":"After that, dialog is open. Selects above utility scripts in the dialog.","metadata":{"_uuid":"0a9450b4-b33b-497a-85bd-308f86b4b345","_cell_guid":"e812d226-d258-4bb2-9d9a-dcf88933bb3c","trusted":true}},{"cell_type":"markdown","source":"If it succeeds, utility scripts are loaded to the following path.","metadata":{"_uuid":"ead5fa0b-90bb-463d-b1cc-58110292629c","_cell_guid":"1507e9f2-092a-4444-a089-21372631d85e","trusted":true}},{"cell_type":"code","source":"#!ls /kaggle/usr/lib/{name_of_utility_scripts}","metadata":{"_uuid":"fc4e752e-d2a2-4600-a1cf-c1ee57665c62","_cell_guid":"aa6626d5-d41e-451b-ba65-28059af48692","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-03-23T08:35:37.342091Z","iopub.execute_input":"2022-03-23T08:35:37.342770Z","iopub.status.idle":"2022-03-23T08:35:37.360654Z","shell.execute_reply.started":"2022-03-23T08:35:37.342617Z","shell.execute_reply":"2022-03-23T08:35:37.360004Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <B>[4] Calling functions/classes in utility scripts</B>","metadata":{"_uuid":"ec1a1491-8f84-4507-a703-6ac399b4f530","_cell_guid":"40a70e6f-e1a9-4c86-b446-bfe8e997aca2","trusted":true}},{"cell_type":"markdown","source":"## [4-1] Importing utility scripts","metadata":{"_uuid":"c53f9dd5-01bf-4848-a18a-1820168c596c","_cell_guid":"610513a0-a884-4d38-bd9b-ec0eec08ce31","trusted":true}},{"cell_type":"markdown","source":"Imports utitlity scripts in the notebook.","metadata":{"_uuid":"3fa78a2c-3af1-452c-adfe-99e241126979","_cell_guid":"0d56e030-152f-488b-b7c9-f8b72b0035fc","trusted":true}},{"cell_type":"code","source":"#import {name_of_utility_scripts}","metadata":{"_uuid":"9040f37a-715c-4265-97b4-b7a9b6cbcf9e","_cell_guid":"32b65a4d-8c1b-4f53-8acf-d0db7c13ab9e","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-03-23T08:35:37.368670Z","iopub.execute_input":"2022-03-23T08:35:37.369146Z","iopub.status.idle":"2022-03-23T08:35:37.372627Z","shell.execute_reply.started":"2022-03-23T08:35:37.369115Z","shell.execute_reply":"2022-03-23T08:35:37.371760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [4-2] Importing dependencies","metadata":{}},{"cell_type":"markdown","source":"Imports dependencies for utility functions in the notebook.","metadata":{}},{"cell_type":"markdown","source":"## [4-3] Calling functions/classes","metadata":{"_uuid":"50565fc7-ca80-4181-88d6-ab1f5d859fc2","_cell_guid":"bcbdb3d0-bede-4aae-922f-4f64acbe017f","trusted":true}},{"cell_type":"markdown","source":"Calls functions/classes defined in utility scripts in the notebook.","metadata":{"_uuid":"87d983e2-d7bd-4d5d-91a5-5d9518f1aff8","_cell_guid":"5597e453-eb86-4dde-9ea8-76f09f072428","trusted":true}},{"cell_type":"markdown","source":"# <B>[5] Sample usage of utility scripts</B>","metadata":{"_uuid":"4835844d-2ed4-4d91-8aef-ea1e778d2256","_cell_guid":"1d2a70d8-4c79-40d4-b270-3f0f110720bb","trusted":true}},{"cell_type":"markdown","source":"Sample usage of utility scripts is shown. [Utility scripts for visualizing dataset for \"Happywhale - Whale and Dolphin Identification\" competition](https://www.kaggle.com/code/acchiko/utility-functions-for-visualization-of-dataset) is loaded as example. Then several images are shown using utility functions defined in the utility scripts.","metadata":{"_uuid":"98f77209-e468-41ae-a990-776605aa70d7","_cell_guid":"c813053a-2e7d-46c2-a07a-6bc097dd082a","trusted":true}},{"cell_type":"code","source":"# Imports utility scripts\nimport utility_functions_for_visualization_of_dataset as myutils","metadata":{"execution":{"iopub.status.busy":"2022-03-23T08:35:37.386028Z","iopub.execute_input":"2022-03-23T08:35:37.386310Z","iopub.status.idle":"2022-03-23T08:35:37.422560Z","shell.execute_reply.started":"2022-03-23T08:35:37.386280Z","shell.execute_reply":"2022-03-23T08:35:37.421572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Imports required libs.\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom PIL import Image, ImageDraw\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\nimport os","metadata":{"_uuid":"8bdf706e-9e7d-4e69-bbf8-70c29edda191","_cell_guid":"e25873d7-e970-4953-9f47-50c1540565a1","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-03-23T08:35:37.424297Z","iopub.execute_input":"2022-03-23T08:35:37.425346Z","iopub.status.idle":"2022-03-23T08:35:37.431145Z","shell.execute_reply.started":"2022-03-23T08:35:37.425296Z","shell.execute_reply":"2022-03-23T08:35:37.430165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sets configurations.\nnum_images = 30\ni_start = 0\ni_end = i_start + num_images\nnum_cols = 4","metadata":{"_uuid":"e05afb3a-cc1d-4155-a972-47dada900f29","_cell_guid":"8b9e9f6f-ef51-492c-8483-a57c2048e086","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-03-23T08:35:37.432866Z","iopub.execute_input":"2022-03-23T08:35:37.433745Z","iopub.status.idle":"2022-03-23T08:35:37.445024Z","shell.execute_reply.started":"2022-03-23T08:35:37.433686Z","shell.execute_reply":"2022-03-23T08:35:37.444088Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Loads metadata.\npath_to_train_metadata = \"/kaggle/input/happy-whale-and-dolphin/train.csv\"\nmetadata = pd.read_csv(path_to_train_metadata)\nsliced_metadata = metadata.iloc[i_start:i_end]","metadata":{"_uuid":"aebd533b-0050-4c76-91c1-8c7832e141c9","_cell_guid":"62681bdb-66a4-4675-bf75-03dbaabd8866","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-03-23T08:35:37.447201Z","iopub.execute_input":"2022-03-23T08:35:37.447848Z","iopub.status.idle":"2022-03-23T08:35:37.549092Z","shell.execute_reply.started":"2022-03-23T08:35:37.447803Z","shell.execute_reply":"2022-03-23T08:35:37.548154Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Shows images.\npath_to_dir_train_images = \"/kaggle/input/happy-whale-and-dolphin/train_images\"\ntitles = [row.image for row in sliced_metadata.itertuples()]\npath_to_images = [\"%s/%s\" % (path_to_dir_train_images, row.image) for row in sliced_metadata.itertuples()]\nimages = myutils.getImages(path_to_images)\nmyutils.showImagesTile(titles, images, num_cols=num_cols)","metadata":{"_uuid":"4e68dddd-81b2-4566-bb3b-876c083d2e7a","_cell_guid":"e532c73c-6528-4d7e-bf0a-53a92ace96a6","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-03-23T08:36:01.943647Z","iopub.execute_input":"2022-03-23T08:36:01.943937Z","iopub.status.idle":"2022-03-23T08:36:28.470595Z","shell.execute_reply.started":"2022-03-23T08:36:01.943910Z","shell.execute_reply":"2022-03-23T08:36:28.469816Z"},"trusted":true},"execution_count":null,"outputs":[]}]}