{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Tutorial: BirdCLEF Visualizations with Plotly\n\nVisualize some of the geospatial data for this competition (latitude/longitude of the recordings) using Plotly. Good for beginners - enjoy!\n\n...More to come 🚧👷‍♂️","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"markdown","source":"## 1 Plotting with Plotly\n\nPlotly's open source graphing libraries are great for making interactive charts and maps for Python, R, Julia, Javascript, ggplot2, F#, MATLAB®, and Dash. It's quickly becoming standard for rapid development and sharing results with colleagues. The next step with these visualizations is usually to put them in some sort of dashboard or webpage for others to see/interact with. For that you can use Dash. Find out more at: [Plotly.com ](https://plotly.com)","metadata":{}},{"cell_type":"code","source":"pip install geocoder","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:26.295674Z","iopub.execute_input":"2022-02-17T19:57:26.295944Z","iopub.status.idle":"2022-02-17T19:57:34.917234Z","shell.execute_reply.started":"2022-02-17T19:57:26.295916Z","shell.execute_reply":"2022-02-17T19:57:34.916282Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport random\nimport geocoder\nimport time\nimport plotly.express as px","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:34.919564Z","iopub.execute_input":"2022-02-17T19:57:34.919922Z","iopub.status.idle":"2022-02-17T19:57:34.925915Z","shell.execute_reply.started":"2022-02-17T19:57:34.91986Z","shell.execute_reply":"2022-02-17T19:57:34.925039Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path = '/kaggle/input/birdclef-2022/'\ntrain_metadata = pd.read_csv(path + 'train_metadata.csv')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:34.927275Z","iopub.execute_input":"2022-02-17T19:57:34.927492Z","iopub.status.idle":"2022-02-17T19:57:35.010166Z","shell.execute_reply.started":"2022-02-17T19:57:34.927467Z","shell.execute_reply":"2022-02-17T19:57:35.009249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_metadata.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:35.012789Z","iopub.execute_input":"2022-02-17T19:57:35.013222Z","iopub.status.idle":"2022-02-17T19:57:35.033017Z","shell.execute_reply.started":"2022-02-17T19:57:35.013177Z","shell.execute_reply":"2022-02-17T19:57:35.032175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* A tad bit unessasary but good fun, let's also fuzzy up the latitude and longitude values. There's some overlap which would mess up the effect of the plotting in some cases. By adding a small random float to the end of latitude and longitude, we won't have problems with the overlap. If the dataset was larger this could help more.","metadata":{}},{"cell_type":"code","source":"#because we have some duplicate latitutde/longitutde points, let's fuzzy them up a bit to avoid map overlap\ndef fuzzy_lat_long(num):\n    \"\"\"Add random 4 digit float to latitude and longitude betwen 0-0.00009. Assumes random is imported.  \n    \n    Args:    num (float): the latitude or longitude value to 4 decimal places\n    \n    Returns:    new_num (float): updated latitude or longitude value to 8 decimal places\"\"\"\n\n    random_value = round(random.uniform(0, 0.00009),8)\n    new_num = num + random_value\n    \n    return new_num","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:35.034418Z","iopub.execute_input":"2022-02-17T19:57:35.034876Z","iopub.status.idle":"2022-02-17T19:57:35.042798Z","shell.execute_reply.started":"2022-02-17T19:57:35.03483Z","shell.execute_reply":"2022-02-17T19:57:35.042152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_metadata['newlat'] = train_metadata.apply(lambda x: fuzzy_lat_long(x['latitude']), axis=1)\ntrain_metadata['newlong'] = train_metadata.apply(lambda x: fuzzy_lat_long(x['longitude']), axis=1)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:35.044105Z","iopub.execute_input":"2022-02-17T19:57:35.04467Z","iopub.status.idle":"2022-02-17T19:57:35.522949Z","shell.execute_reply.started":"2022-02-17T19:57:35.044624Z","shell.execute_reply":"2022-02-17T19:57:35.52213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_metadata.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:35.523937Z","iopub.execute_input":"2022-02-17T19:57:35.524169Z","iopub.status.idle":"2022-02-17T19:57:35.546647Z","shell.execute_reply.started":"2022-02-17T19:57:35.524143Z","shell.execute_reply":"2022-02-17T19:57:35.545837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Plotly - Interactive Map\n* Play around with the different parameters of px.scatter_geo (remember we used: import plotly.express as px)\n* Try different interactive map actions:\n * Drag, zoom, hover, click, download as png, box select, etc. ","metadata":{}},{"cell_type":"code","source":"#show all birds\nfig = px.scatter_geo(train_metadata, #this is the dataframe containing info to populate the map with\n                    lat = train_metadata['newlat'], #this is the latitude train_metadata['latitude'] was the original\n                    lon = train_metadata['newlong'],#this is the longitude train_metadata['longitude'] was the original\n                    color =\"primary_label\", #select the dataframe column to use to set the color of markers\n                    hover_name = \"common_name\", # column added to hover information\n                    projection =\"natural earth\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:35.548006Z","iopub.execute_input":"2022-02-17T19:57:35.548306Z","iopub.status.idle":"2022-02-17T19:57:36.259106Z","shell.execute_reply.started":"2022-02-17T19:57:35.548264Z","shell.execute_reply":"2022-02-17T19:57:36.258331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **What can we learn from this?**\n * For one thing notice how different bird species are unique to different geographic locations (obvious right).\n * Because bird species are more common that others in different regions - you want to make sure your ML models aren't predicting a bird that is highly unlikely to be in the area the sound is recorded. For example if the sound recording is from South Africa, it's unlikely to be a Eurasian Skylark! ","metadata":{}},{"cell_type":"code","source":"#Let's also visualize only the scored_birds in the map to see what it looks like\nscored_birds = ['akiapo', 'aniani', 'apapan', 'barpet', 'crehon', 'elepai', 'ercfra', 'hawama', 'hawcre', 'hawgoo', 'hawhaw', 'hawpet1', 'houfin', 'iiwi', 'jabwar', 'maupar', 'omao', 'puaioh', 'skylar', 'warwhe1', 'yefcan']\ntrain = train_metadata[train_metadata['primary_label'].isin(scored_birds)]","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:36.26067Z","iopub.execute_input":"2022-02-17T19:57:36.260977Z","iopub.status.idle":"2022-02-17T19:57:36.271484Z","shell.execute_reply.started":"2022-02-17T19:57:36.260926Z","shell.execute_reply":"2022-02-17T19:57:36.270329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Let's practice extracting the country from a pair of latitude and longitude coordinates! This can help make visualizations and analyze bird population distributions by country!","metadata":{}},{"cell_type":"code","source":"def geo_reverse(lat,long):\n    \"\"\"Reverse geolocate country from a latitude and longitude. Assumes random is imported.  \n    \n    Args:   lat (float): the latitude value to 4 decimal places\n            long (float): the longitude value to 4 decimal places\n    \n    Returns:    country (str): the country based on latitude and longitude\"\"\"\n    \n    time.sleep(0.04) #sleep for 40ms to avoid errors\n    g = geocoder.osm([lat, long], method='reverse').json\n    country = 'no country'\n    if g:\n        country = g.get('country')\n        return country\n    else:\n        return country","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:36.273787Z","iopub.execute_input":"2022-02-17T19:57:36.274033Z","iopub.status.idle":"2022-02-17T19:57:36.286356Z","shell.execute_reply.started":"2022-02-17T19:57:36.274004Z","shell.execute_reply":"2022-02-17T19:57:36.285437Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#check that it works\ngeo_reverse(12.3910,-1.4930)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:36.287803Z","iopub.execute_input":"2022-02-17T19:57:36.288538Z","iopub.status.idle":"2022-02-17T19:57:36.927687Z","shell.execute_reply.started":"2022-02-17T19:57:36.288495Z","shell.execute_reply":"2022-02-17T19:57:36.926858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#run for all the rows\n#train['country'] = train.apply(lambda x: geo_reverse(x['latitude'],x['longitude']), axis=1)\n#train.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:36.92869Z","iopub.execute_input":"2022-02-17T19:57:36.928884Z","iopub.status.idle":"2022-02-17T19:57:36.93229Z","shell.execute_reply.started":"2022-02-17T19:57:36.92886Z","shell.execute_reply":"2022-02-17T19:57:36.931554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#show only the birds in the scored_birds list\nfig = px.scatter_geo(train, \n                    lat=train['newlat'],\n                    lon=train['newlong'],\n                    color=\"primary_label\",\n                    hover_name=\"common_name\",\n                    projection=\"natural earth\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T19:57:36.933341Z","iopub.execute_input":"2022-02-17T19:57:36.934154Z","iopub.status.idle":"2022-02-17T19:57:37.0848Z","shell.execute_reply.started":"2022-02-17T19:57:36.934122Z","shell.execute_reply":"2022-02-17T19:57:37.084189Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **What can we learn from this?**\n * First off, notice no birds from the scored_birds list are from south america! \n * Also, notice that almost all birds recordings in a continent are the same. \n * We should check the distribution of birds in the dataset.","metadata":{}},{"cell_type":"markdown","source":"## 2 Bonus! Plotting with Folium\n* Try plotting with Folium below. It's another great way to visualize geospatial data! Uncomment the code blocks and run to see how it looks.\n\n## [Tutorial: Play Bird Audio on Map with Folium](https://www.kaggle.com/alexteboul/tutorial-play-bird-audio-on-map-with-folium)","metadata":{}},{"cell_type":"markdown","source":"<iframe src='https://xeno-canto.org/122399/embed?simple=1' scrolling='no' frameborder='0' width='340' height='115'></iframe>","metadata":{}}]}