{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# EDA for Google ASL Signs Data\n\nIn this notebook, we are going to look at the ASL Signs data provided for the competition.\n\nThis data comprises of subjects for which the accelerotmeter motion readings were recorded in real-time. For each subject, there are multiple sequences which are stored in Parquet format. \n\nApart from this a train.csv file is also provided pointing to sequence file paths and their corresponding labels.\n\nThese labels are only relevant for right and left hand motion gestures. Since the data also contains the information regarding subject's pose and facial activity, we should ignore them as they are not relevant for gesture classification.","metadata":{}},{"cell_type":"markdown","source":"## Lets take a look at the ***train.csv*** file","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\ntrain_summary_file_path = '/kaggle/input/asl-signs/train.csv'\ntrain_summary_df = pd.read_csv(train_summary_file_path)\ntrain_summary_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-04-08T17:54:19.991581Z","iopub.execute_input":"2023-04-08T17:54:19.992503Z","iopub.status.idle":"2023-04-08T17:54:20.302594Z","shell.execute_reply.started":"2023-04-08T17:54:19.992444Z","shell.execute_reply":"2023-04-08T17:54:20.301024Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## What each field means in ***train.csv*** file?\n\n* **path** - File path to parquet file stored in **training_landmark_files** folder.\n* **participant_id** - Unique id field to identify each participant\n* **sequence_id** - Unique id field for each sequence captured for each **participant_id**\n* **sign** - Sign labelled for hand gesture shown in key sequence for each participant.","metadata":{}},{"cell_type":"markdown","source":"**For above *train.csv* file, we need to add the following fields to keep it ready for training:**\n\n* **absolute_path** - Right now only relative path is provided under path field. We should prepend root directory as well.\n* **sign_idx** - Its a reverse mapping for Sign Name to Index Number. This will help during the training phase.\n\n\nTo get the sign_idx, we already have a file to get Sign Name to Index Number map present in JSON format.\n\nThe mapping provides **250 object names** signified by index number.","metadata":{}},{"cell_type":"code","source":"import json\nsign_map_file_path = '/kaggle/input/asl-signs/sign_to_prediction_index_map.json'\nsign_map = json.load(open(sign_map_file_path))\npd.DataFrame(sign_map,index=[0])","metadata":{"execution":{"iopub.status.busy":"2023-04-08T11:36:22.914684Z","iopub.execute_input":"2023-04-08T11:36:22.915032Z","iopub.status.idle":"2023-04-08T11:36:22.945920Z","shell.execute_reply.started":"2023-04-08T11:36:22.915005Z","shell.execute_reply":"2023-04-08T11:36:22.944415Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_summary_df['sign_idx'] = train_summary_df['sign'].apply(lambda x: sign_map[x])\ntrain_summary_df['absolute_path'] = train_summary_df['path'].apply(lambda x: '/kaggle/input/asl-signs/'+x)\ndel train_summary_df['path']\ntrain_summary_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-04-08T11:36:27.580598Z","iopub.execute_input":"2023-04-08T11:36:27.580926Z","iopub.status.idle":"2023-04-08T11:36:27.657345Z","shell.execute_reply.started":"2023-04-08T11:36:27.580899Z","shell.execute_reply":"2023-04-08T11:36:27.656694Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Let's take a look at one of the sequence files","metadata":{}},{"cell_type":"code","source":"sample_seq_df = pd.read_parquet('/kaggle/input/asl-signs/train_landmark_files/26734/1000035562.parquet')\nsample_seq_df","metadata":{"execution":{"iopub.status.busy":"2023-04-08T11:36:36.705098Z","iopub.execute_input":"2023-04-08T11:36:36.705513Z","iopub.status.idle":"2023-04-08T11:36:36.864471Z","shell.execute_reply.started":"2023-04-08T11:36:36.705469Z","shell.execute_reply":"2023-04-08T11:36:36.862944Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## What do we have here in the Sequence File?\n\n* **frame** - It is the frame number representing multiple landmarks in sequence of video\n* **type** - Landmark type in each frame for a particular sequence\n* **landmark_index** - A unique identifier for landmark in frame\n* **row_id** - A combination of **frame + type + landmark_index**\n* **x** - Acclerometer's x-axis coordinate for a particular **type**\n* **y** - Acclerometer's y-axis coordinate for a particular **type**\n* **z** - Acclerometer's z-axis coordinate for a particular **type**\n","metadata":{}},{"cell_type":"code","source":"print('Landmark Index Summary:\\n')\nfor i in sample_seq_df['type'].unique():\n    print('\\t * Type of landmark: %s --- Unqiue landmark indexes: %s\\n'%(i,len(sample_seq_df[sample_seq_df['type']==i]['landmark_index'].unique())))","metadata":{"execution":{"iopub.status.busy":"2023-04-08T11:36:41.553323Z","iopub.execute_input":"2023-04-08T11:36:41.553719Z","iopub.status.idle":"2023-04-08T11:36:41.572659Z","shell.execute_reply.started":"2023-04-08T11:36:41.553689Z","shell.execute_reply":"2023-04-08T11:36:41.570632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## What Landmark Index really mean?\n\nEach landmark index represents a distinct location of acceleromete's tracker.\nFor each type of landmark, there exists distinct number of landmark indexes which remains same accross number of frames in sequence.\n\nSince we are only concerned with hand gestures and not the pose and face of the subject, we will exclude all the pose and face related indexes from each of the frames","metadata":{}},{"cell_type":"code","source":"sample_frame = sample_seq_df[sample_seq_df['frame']==21]\nsample_type = sample_frame[sample_frame['type']=='right_hand']\nidx = sample_type['landmark_index']\ncord = sample_type[['x','y']]\n\nimport matplotlib.pyplot as plt\nplt.scatter(cord['x'],cord['y'])\nplt.xlim(-1,1)\nplt.ylim(0,1)","metadata":{"execution":{"iopub.status.busy":"2023-04-08T11:43:33.223238Z","iopub.execute_input":"2023-04-08T11:43:33.224823Z","iopub.status.idle":"2023-04-08T11:43:33.389937Z","shell.execute_reply.started":"2023-04-08T11:43:33.224744Z","shell.execute_reply":"2023-04-08T11:43:33.388912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion\n\nThis notebook covered how the data is strutured and what all relevant information are required for training purpose. In the next notebook, we are going to cover how to prepare the data for training. Apart from this, we are also going to provide efficient implementation of code for faster data processing and productionized Tensorflow model ready to be deployed for predition and evaluation.","metadata":{}}]}