{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#

Google Slow VS Fast AI Runtime

\n\n
\n\n## What is the competition about?\nThe competition is to predict the runtime of AI models on a variety of hardware configurations. The goal is to develop a machine learning model that can accurately predict the runtime of an AI model based on its characteristics, such as the number of parameters/the number of layers/hardware configuration.\n\n## What is the data?\nThe competition data consists of a training set/test set. The training set contains information about AI models, including their characteristics/runtimes. The test set contains information about AI models, but the runtimes are not known\n\n## What are the prizes?\nThe total prize pool for the competition is 50,000\n . The top 3 teams will win prizes of 15,000\n , 10,000\n , and 5,000\n , respectively.","metadata":{}},{"cell_type":"code","source":"import os ","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:04:49.365310Z","iopub.execute_input":"2023-09-03T05:04:49.365748Z","iopub.status.idle":"2023-09-03T05:04:49.370499Z","shell.execute_reply.started":"2023-09-03T05:04:49.365710Z","shell.execute_reply":"2023-09-03T05:04:49.369440Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nOur model data is situtated acrros various domains ranging from $Computer-Vision$ to $Natural-Langauage-Processing$. A look at the models can be seen below","metadata":{}},{"cell_type":"code","source":"os.listdir('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/xla/random/train')","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:04:49.816159Z","iopub.execute_input":"2023-09-03T05:04:49.816895Z","iopub.status.idle":"2023-09-03T05:04:49.826102Z","shell.execute_reply.started":"2023-09-03T05:04:49.816854Z","shell.execute_reply":"2023-09-03T05:04:49.824901Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

1 | Data 📈

","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np \nimport tqdm","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:05:15.175257Z","iopub.execute_input":"2023-09-03T05:05:15.175671Z","iopub.status.idle":"2023-09-03T05:05:15.181481Z","shell.execute_reply.started":"2023-09-03T05:05:15.175636Z","shell.execute_reply":"2023-09-03T05:05:15.179939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nOur data is distirbuted mainly in $4$ $Directories$ \n* `nlp/default/train`\n* `nlp/random/train`\n* `xla/default/train`\n* `xla/random/train`\n\nWe will load files from these loacations and use them further in training ","metadata":{}},{"cell_type":"code","source":"# ndt => nlp_defalut_train\n# nrt => nlp_random_train\n# xdt => xla_default_train\n# xrt => xla_random_train\n\nndt = os.listdir('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/nlp/default/train')\nnrt = os.listdir('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/nlp/random/train')\n\nxdt = os.listdir('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/xla/default/train')\nxrt = os.listdir('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/xla/random/train')","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:04:51.773226Z","iopub.execute_input":"2023-09-03T05:04:51.773628Z","iopub.status.idle":"2023-09-03T05:04:51.875987Z","shell.execute_reply.started":"2023-09-03T05:04:51.773595Z","shell.execute_reply":"2023-09-03T05:04:51.875141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ndt = [\n np.load('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/nlp/default/train/' + value)\n for value \n in tqdm.tqdm(ndt , total = len(ndt) , desc = 'Loading NDT --->')\n]\nnrt = [\n np.load('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/nlp/random/train/' + value)\n for value \n in tqdm.tqdm(nrt , total = len(nrt) , desc = 'Loading NRT --->')\n]\nxdt = [\n np.load('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/xla/default/train/' + value)\n for value\n in tqdm.tqdm(xdt , total = len(xdt) , desc = 'Loading XDT --->')\n]\nxrt = [\n np.load('/kaggle/input/predict-ai-model-runtime/npz_all/npz/layout/xla/random/train/' + value)\n for value\n in tqdm.tqdm(xrt , total = len(xrt) , desc = 'Loading NRT --->')\n]\n\nfiles = ndt + nrt + xdt + xrt \nlen(files)","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:04:52.522960Z","iopub.execute_input":"2023-09-03T05:04:52.524338Z","iopub.status.idle":"2023-09-03T05:05:00.443512Z","shell.execute_reply.started":"2023-09-03T05:04:52.524297Z","shell.execute_reply":"2023-09-03T05:05:00.442042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nSo we have a total of 535 files \n \nEach file contains \n\n|||\n|---|---|\n|Node Feature\n|Node OpCode|Specific operation or instruction that is performed by a node\n|Edge Index |Connections between nodes in a graph or a neural network\n|Node Config Features\n|Node Config IDS\n|Config Runtime\n|Node Splits\n\nWe will make these files to a DataFrame in export it in the output DIRs","metadata":{}},{"cell_type":"code","source":"data = pd.DataFrame({\n 'node_feat' : [\n val['node_feat']\n for val \n in tqdm.tqdm(files , total = len(files) , desc = 'Reading Node Feats --->')\n ] , \n 'node_opcode' : [\n val['node_opcode']\n for val \n in tqdm.tqdm(files , total = len(files) , desc = 'Reading Node Opcodes --->')\n ] , \n 'edge_index' : [\n val['edge_index']\n for val \n in tqdm.tqdm(files , total = len(files) , desc = 'Reading Edge Index --->')\n ] , \n# 'node_config_feat' : [\n# val['node_config_feat']\n# for val \n# in tqdm.tqdm(files , total = len(files) , desc = 'Reading Node Config Feats --->')\n# ] , \n 'node_config_ids' : [\n val['node_config_ids']\n for val \n in tqdm.tqdm(files , total = len(files) , desc = 'Reading Node Config Ids --->')\n ] , \n 'config_runtime' : [\n val['config_runtime']\n for val \n in tqdm.tqdm(files , total = len(files) , desc = 'Reading Node Config Runtime --->')\n ] , \n 'node_splits' : [\n val['node_splits']\n for val \n in tqdm.tqdm(files , total = len(files) , desc = 'Reading Node Splits --->')\n ]\n})","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:05:19.177389Z","iopub.execute_input":"2023-09-03T05:05:19.177781Z","iopub.status.idle":"2023-09-03T05:05:31.541316Z","shell.execute_reply.started":"2023-09-03T05:05:19.177738Z","shell.execute_reply":"2023-09-03T05:05:31.540380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.head()","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:05:35.684226Z","iopub.execute_input":"2023-09-03T05:05:35.684948Z","iopub.status.idle":"2023-09-03T05:05:36.584934Z","shell.execute_reply.started":"2023-09-03T05:05:35.684911Z","shell.execute_reply":"2023-09-03T05:05:36.583858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.to_csv('/kaggle/working/Input Files.csv')","metadata":{"execution":{"iopub.status.busy":"2023-09-03T05:05:43.162805Z","iopub.execute_input":"2023-09-03T05:05:43.163247Z","iopub.status.idle":"2023-09-03T05:05:44.172180Z","shell.execute_reply.started":"2023-09-03T05:05:43.163211Z","shell.execute_reply":"2023-09-03T05:05:44.171001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

2 | Ending 🏁

\n\n
\n\n**WE WILL GO DEEPER INTO THE DATA IN THE UPCOMING VERSIONS**\n\n**PLEASE COMMENT DOWN IF I DID ANY MISTAKES, OR IF CAN MAKE THIS MORE CONNECTED TO THE GROUND, OR SUGGESTIONS. YOUR ASSISTS ARE HIGHLY APPRECIABLE**\n\n**THATS IT FOR TODAY GUYS**\n\n**HOPE YOU UNDERSTOOD AND LIKED MY WORK**\n\n**DONT FORGET TO MAKE AN UPVOTE $:)$**\n \n\n \n**PEACE OUT**","metadata":{}}],"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}}