{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\n \"\"

\n
","metadata":{"papermill":{"duration":0.006745,"end_time":"2023-02-20T16:40:42.301861","exception":false,"start_time":"2023-02-20T16:40:42.295116","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# \n#

šŸ¤“ Introduction

\n\nšŸ‘‹ Hello everyone! Welcome to my EDA notebook for the Google Isolated Sign Language Recognition Kaggle competition!\n\nšŸ¤” Have you ever thought about the challenges that deaf children and their families face when trying to communicate with the world around them? Did you know that every day, 33 babies are born with permanent hearing loss in the U.S. alone? This is a staggering number, and it highlights the urgent need for effective communication solutions for the deaf community.\n\nšŸ‘‰ That's where this competition comes in! Our goal is to create a TensorFlow Lite model that can accurately classify isolated American Sign Language (ASL) signs. By doing so, we can improve the functionality of the PopSign app, which helps deaf children and their families learn basic sign language and communicate more effectively.\n\nšŸŒŸ This competition is incredibly important, as it has the potential to positively impact the lives of countless individuals within the deaf community. By improving the ability of PopSign to teach sign language, we can help bridge the communication gap between deaf children and their families, friends, and peers.\n\nšŸ’» In this EDA notebook, I'll be exploring the dataset provided by the competition organizers, and analyzing the various features and labels to gain a better understanding of the task at hand. I hope that through this notebook, I can inspire and educate others to join this important cause and help make a difference in the lives of those who need it most. Let's get started! šŸ’Ŗ","metadata":{}},{"cell_type":"markdown","source":"\n

šŸ“‹ Table of Content

\n\n
\n \n
\n > šŸ“Œ Objectives of Notebook\n
\n > šŸ“„ LOADING DATASET\n
\n > šŸ“š Importing Libraries\n
\n > šŸ“Š Interesting Charts\n
\n > šŸ“– Basic concepts\n
\n > āš™ļø CONFIG\n
\n > šŸŽÆ Goals\n
\n
\n
\n","metadata":{"papermill":{"duration":0.00485,"end_time":"2023-02-20T16:40:42.312260","exception":false,"start_time":"2023-02-20T16:40:42.307410","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"##
šŸ“Œ Objectives of Notebook
\n\n
\n
\n šŸ“„ Load the dataset\n
\n šŸ” Conduct exploratory data analysis (EDA)\n
\n šŸ“Š Visualize feature distributions\n
\n šŸ§ Investigate statistical properties of the data\n
\n šŸ” Look for outliers and missing values\n
\n šŸ“ˆ Examine relationships between features\n
\n šŸ¤– Prepare data for model training\n
\n šŸ” Evaluate the quality of the model\n
\n šŸ“ˆ Analyze the results and draw conclusions\n
\n šŸ“Š Visualize the results and interpret them\n
\n šŸ“ Write conclusions and suggestions for further work.\n
\n
\n
\n","metadata":{"papermill":{"duration":0.004686,"end_time":"2023-02-20T16:40:42.322205","exception":false,"start_time":"2023-02-20T16:40:42.317519","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"\n#
šŸŽÆ Goals
\n\nšŸ‘‹ Hey everyone, let me introduce you to an exciting Kaggle competition: \"Google - Isolated Sign Language Recognition.\" šŸ¤Ÿ\n\nThe goal of this competition is to develop a TensorFlow Lite model that can classify isolated American Sign Language (ASL) signs. The model will be trained on labeled landmark data extracted using the MediaPipe Holistic Solution. The main objective is to improve the ability of PopSign* to help relatives of deaf children learn basic signs and communicate better with their loved ones. šŸ¤\n\nThe significance of this competition cannot be overstated. Every day, 33 babies are born with permanent hearing loss in the U.S., and around 90% of them are born to hearing parents who may not know American Sign Language. This can lead to Language Deprivation Syndrome, which can have serious impacts on the lives of deaf children, including relationships, education, and employment. šŸ™\n\nLearning sign language can be a challenge, but games like PopSign can help make it fun and interactive. By adding a sign language recognizer developed through this competition, PopSign players will be able to sign the type of bubble they want to shoot, providing them with the opportunity to practice the sign themselves. This will not only make the game more interactive but also improve the learning and confidence of players who want to learn sign language to communicate with their loved ones. šŸ˜Š\n\nSo, join us in this competition to help connect deaf children and their parents. Let's work together to create a better world for everyone. šŸŒ\n","metadata":{"papermill":{"duration":0.004692,"end_time":"2023-02-20T16:40:42.332073","exception":false,"start_time":"2023-02-20T16:40:42.327381","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"\n#

šŸ“– Basic concepts

\n_ _ _\n
\n \n Brief Definition\n \n
\n\n_ _ _\n\n> __American Sign Language (ASL) šŸ¤Ÿ__\nASL is a complete, natural language that uses hand movements, facial expressions, and body postures to convey meaning. It is the primary language used by many deaf people in the United States.\n\n> __Sign recognition šŸ¤™__\nSign recognition refers to the ability of a computer program to identify and interpret ASL signs. This competition focuses on classifying isolated ASL signs using TensorFlow Lite models.\n\n> __MediaPipe Holistic Solution šŸ¤–__\nMediaPipe Holistic Solution is a tool that extracts landmarks from video data. In this competition, we use it to extract landmarks from video data of ASL signs, which will be used to train the machine learning models.\n\n> __Language Deprivation Syndrome šŸ™‰__\nLanguage Deprivation Syndrome is a condition that occurs when a deaf child does not have access to a fully developed language during the critical language-learning years. It can have serious impacts on their relationships, education, and employment.\n\n> __Deep Learning šŸ§ __\nDeep learning is a subset of machine learning that involves the use of artificial neural networks to train models to make predictions. These networks are inspired by the structure and function of the human brain, with layers of interconnected nodes that process information and make decisions.\n\n> __TensorFlow šŸ¤–__\nTensorFlow is an open-source software library developed by Google for building and training machine learning models. It provides a wide range of tools and APIs for creating and deploying models, including support for deep learning and neural networks.\n\n> __TensorFlow Lite šŸ“±__\nTensorFlow Lite is a lightweight version of TensorFlow designed specifically for running machine learning models on mobile and embedded devices. It is optimized for speed and efficiency, making it ideal for use in mobile apps like PopSign.\n\n> __MediaPipe Holistic Solution šŸŽ„__\nMediaPipe is an open-source framework developed by Google for building cross-platform, customizable pipelines for processing multimedia data. The MediaPipe Holistic Solution is a pre-built pipeline that combines multiple models to extract pose, face, and hand landmarks from video streams.\n\n\nBy participating in this competition, you have the opportunity to help improve the ability of PopSign to teach ASL signs to parents and loved ones of deaf children. Let's use our skills to make a difference and help connect deaf children and their families!","metadata":{"papermill":{"duration":0.004821,"end_time":"2023-02-20T16:40:42.342071","exception":false,"start_time":"2023-02-20T16:40:42.337250","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"\n#

šŸ“š Importing Libraries

\n","metadata":{"papermill":{"duration":0.005038,"end_time":"2023-02-20T16:40:42.352267","exception":false,"start_time":"2023-02-20T16:40:42.347229","status":"completed"},"tags":[]}},{"cell_type":"code","source":"! pip install plotly==5.11.0","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:21:50.791818Z","iopub.execute_input":"2023-02-24T16:21:50.792273Z","iopub.status.idle":"2023-02-24T16:22:46.594793Z","shell.execute_reply.started":"2023-02-24T16:21:50.792183Z","shell.execute_reply":"2023-02-24T16:22:46.593202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport sys\nimport cv2\nimport imageio\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nimport seaborn as sns\nimport plotly.express as px\nfrom wordcloud import WordCloud\nimport matplotlib.pyplot as plt\nfrom matplotlib.pyplot import imshow\nimport matplotlib.animation as animation\nfrom matplotlib.animation import FuncAnimation","metadata":{"papermill":{"duration":4.795258,"end_time":"2023-02-20T16:41:03.491923","exception":false,"start_time":"2023-02-20T16:40:58.696665","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-02-24T16:22:46.597899Z","iopub.execute_input":"2023-02-24T16:22:46.599240Z","iopub.status.idle":"2023-02-24T16:22:48.611346Z","shell.execute_reply.started":"2023-02-24T16:22:46.599180Z","shell.execute_reply":"2023-02-24T16:22:48.609772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#

āš™ļø CONFIG

","metadata":{"papermill":{"duration":0.006787,"end_time":"2023-02-20T16:41:03.505112","exception":false,"start_time":"2023-02-20T16:41:03.498325","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class CFG:\n class path:\n to_base_folder = '/kaggle/input/asl-signs'\n to_train_data = '/kaggle/input/asl-signs/train.csv'\n to_folder_with_parquet = '/kaggle/input/asl-signs/train_landmark_files'\n to_prediction_index_map = '/kaggle/input/asl-signs/sign_to_prediction_index_map.json'","metadata":{"papermill":{"duration":0.017722,"end_time":"2023-02-20T16:41:03.529233","exception":false,"start_time":"2023-02-20T16:41:03.511511","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-02-24T16:24:25.658608Z","iopub.execute_input":"2023-02-24T16:24:25.659046Z","iopub.status.idle":"2023-02-24T16:24:25.665308Z","shell.execute_reply.started":"2023-02-24T16:24:25.659010Z","shell.execute_reply":"2023-02-24T16:24:25.663957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#

šŸ“„ LOADING DATASET

\n","metadata":{"papermill":{"duration":0.005735,"end_time":"2023-02-20T16:41:03.541139","exception":false,"start_time":"2023-02-20T16:41:03.535404","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"> train_landmark_files/[participant_id]/[sequence_id].parquet\n\nThey folder contain tag data that was extracted from processed videos using the MediaPipe holistic model. Some shots may not have visible hands or hands that could be detected by the model. The file contains columns such as \"frame\", \"row_id\", \"type\", \"landmark_index\" and \"[x/y/z]\".\n\n_ _ _\n\n> train.csv \n\nIt contains the path to the landmark files, a unique member ID, a unique sequence ID, and a label for each sequence.\n","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(CFG.path.to_train_data)\ntrain_df.path = train_df.path.apply(lambda path: f\"{CFG.path.to_base_folder}/{path}\")\n\nprint(f\"train_df.shape={train_df.shape}\")\ntrain_df.head()","metadata":{"papermill":{"duration":0.054606,"end_time":"2023-02-20T16:41:03.602147","exception":false,"start_time":"2023-02-20T16:41:03.547541","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-02-24T16:24:26.091450Z","iopub.execute_input":"2023-02-24T16:24:26.091915Z","iopub.status.idle":"2023-02-24T16:24:26.389548Z","shell.execute_reply.started":"2023-02-24T16:24:26.091872Z","shell.execute_reply":"2023-02-24T16:24:26.388407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#

šŸ“Š Interesting Charts

\nā€‹\n","metadata":{}},{"cell_type":"code","source":"sns.countplot(data=train_df, x='sign')\nplt.title('Distribution of Target Variable')\nplt.xlabel('Sign')\nplt.ylabel('Count')\nplt.xticks([])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:27.430602Z","iopub.execute_input":"2023-02-24T16:24:27.430997Z","iopub.status.idle":"2023-02-24T16:24:28.731259Z","shell.execute_reply.started":"2023-02-24T16:24:27.430963Z","shell.execute_reply":"2023-02-24T16:24:28.730180Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"text = ' '.join(train_df['sign'])\nwordcloud = WordCloud(width=800, height=400, background_color='white', colormap='tab20').generate(text)","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:28.732819Z","iopub.execute_input":"2023-02-24T16:24:28.733875Z","iopub.status.idle":"2023-02-24T16:24:30.049326Z","shell.execute_reply.started":"2023-02-24T16:24:28.733833Z","shell.execute_reply":"2023-02-24T16:24:30.047760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12, 8))\nplt.imshow(wordcloud, interpolation='bilinear')\nplt.axis('off')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:30.051553Z","iopub.execute_input":"2023-02-24T16:24:30.052773Z","iopub.status.idle":"2023-02-24T16:24:30.473374Z","shell.execute_reply.started":"2023-02-24T16:24:30.052720Z","shell.execute_reply":"2023-02-24T16:24:30.472457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Parquet Landmark Data\n\n_ _ _\n> Each Parquet file is in the path:\n> * train_landmark_files/[participant_id]/[sequence_id].parquet\n\n> The parquet's associated sign can be found in train.csv","metadata":{}},{"cell_type":"code","source":"example_listen_path = train_df.query('sign == \"listen\"')[\"path\"].values[0]\nexample_parquet_df = pd.read_parquet(example_listen_path)\n\nprint(f\"example_listen_path={example_listen_path}\")\nprint(f\"example_parquet_df.shape={example_parquet_df.shape}\")\n\nexample_parquet_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:30.474686Z","iopub.execute_input":"2023-02-24T16:24:30.475384Z","iopub.status.idle":"2023-02-24T16:24:30.620746Z","shell.execute_reply.started":"2023-02-24T16:24:30.475340Z","shell.execute_reply":"2023-02-24T16:24:30.619755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for column in (\"frame\", \"type\"):\n plt.subplots(figsize=(10, 5))\n sns.countplot(x=example_parquet_df[column]);\n \n plt.title(f\"Distribution by {column}\")","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:31.079925Z","iopub.execute_input":"2023-02-24T16:24:31.081337Z","iopub.status.idle":"2023-02-24T16:24:31.485361Z","shell.execute_reply.started":"2023-02-24T16:24:31.081262Z","shell.execute_reply":"2023-02-24T16:24:31.484433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"grouped_df = example_parquet_df.groupby('frame')","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:31.984210Z","iopub.execute_input":"2023-02-24T16:24:31.984844Z","iopub.status.idle":"2023-02-24T16:24:31.989296Z","shell.execute_reply.started":"2023-02-24T16:24:31.984809Z","shell.execute_reply":"2023-02-24T16:24:31.988178Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"example_frame = example_parquet_df.query(\"frame == 40\")\npx.scatter_3d(example_frame, x=\"x\", y=\"y\", z=\"z\", color=\"type\")","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:25:07.108269Z","iopub.execute_input":"2023-02-24T16:25:07.108744Z","iopub.status.idle":"2023-02-24T16:25:07.458452Z","shell.execute_reply.started":"2023-02-24T16:25:07.108698Z","shell.execute_reply":"2023-02-24T16:25:07.457307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def update_plot(frame_num):\n plt.cla()\n\n data = grouped_df.get_group(frame_num)\n\n face_data = data[data['type'] == 'face']\n left_hand_data = data[data['type'] == 'left_hand']\n right_hand_data = data[data['type'] == 'right_hand']\n\n plt.scatter(face_data['x'], face_data['y'], c=face_data['landmark_index'], cmap='gist_rainbow', vmin=0, vmax=67, s=100)\n plt.title(f'Frame {frame_num}')\n\n plt.scatter(left_hand_data['x'], left_hand_data['y'], c='blue', marker='o', s=100)\n\n plt.scatter(right_hand_data['x'], right_hand_data['y'], c='red', marker='o', s=100)\n","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:32.766208Z","iopub.execute_input":"2023-02-24T16:24:32.766656Z","iopub.status.idle":"2023-02-24T16:24:32.775880Z","shell.execute_reply.started":"2023-02-24T16:24:32.766617Z","shell.execute_reply":"2023-02-24T16:24:32.774372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure()\nani = animation.FuncAnimation(fig, update_plot, frames=grouped_df.groups.keys(), repeat=True)\n\nani.save('animation.gif', writer='imagemagick')","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:24:46.163659Z","iopub.execute_input":"2023-02-24T16:24:46.164314Z","iopub.status.idle":"2023-02-24T16:24:49.032931Z","shell.execute_reply.started":"2023-02-24T16:24:46.164252Z","shell.execute_reply":"2023-02-24T16:24:49.031480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"animation = imageio.mimread(\"/kaggle/working/animation.gif\")\nfor frame in animation:\n plt.imshow(frame)\n plt.axis('off')\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-24T16:25:00.489721Z","iopub.execute_input":"2023-02-24T16:25:00.490186Z","iopub.status.idle":"2023-02-24T16:25:01.480019Z","shell.execute_reply.started":"2023-02-24T16:25:00.490142Z","shell.execute_reply":"2023-02-24T16:25:01.478709Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":". . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ","metadata":{}},{"cell_type":"markdown","source":"\n#

šŸ˜‰ Thank you for watching!

\n\nIf you've made it this far, I hope you've enjoyed reading through this notebook on the Google - Isolated Sign Language Recognition Kaggle competition. šŸ˜Š\n\nIf you found this notebook helpful, please consider upvoting it so that others can benefit from it as well. šŸ™Œ And remember, this notebook is a work in progress and I will continue to update it with new insights and analyses as I explore the data further. šŸ‘Øā€šŸ’»\n\nThank you for taking the time to read through this notebook and happy exploring! šŸš€\n\n\n
","metadata":{"papermill":{"duration":0.293259,"end_time":"2023-02-20T16:41:19.764058","exception":false,"start_time":"2023-02-20T16:41:19.470799","status":"completed"},"tags":[]}}]}