{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":91844,"databundleVersionId":11361821,"sourceType":"competition"}],"dockerImageVersionId":31012,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# This Notebook Include:\n* Goal of DataSet and About the Data Sheet\n* Data Visualization\n* Approach and Guide to do the Modeling ( Begineer Friendly )\n* Statistics ( Both Discriptive and Infrential Statistics )","metadata":{}},{"cell_type":"markdown","source":"# **GOAL OF DATASET**","metadata":{}},{"cell_type":"markdown","source":"Develop a **AI model that can automatically identify bird species** from soundscape recordings, even in noisy environments with overlapping vocalizations from El Silencio Natural Reserve, Colombia, to aid biodiversity monitoring","metadata":{}},{"cell_type":"markdown","source":"**Real World Applications**","metadata":{}},{"cell_type":"markdown","source":"This has real-world applications in:\n\n* Biodiversity monitoring\n* Conservation efforts\n* Ecological research\n\n","metadata":{}},{"cell_type":"markdown","source":"# **A. About Data------------->**","metadata":{}},{"cell_type":"markdown","source":"The data consist of 3 Folder as well as 4 files ( you could see Right ) for Training, Testing, Taxonomony, etc as discussed below : ","metadata":{}},{"cell_type":"markdown","source":"## 1) **train_audio File**","metadata":{}},{"cell_type":"markdown","source":"Short, single-species recordings (.ogg format, 32 kHz). Filename format: collection > file_id > ogg files (e.g., CSA36385.ogg). This is the Folder to be Trained.\n\nLink : /kaggle/input/birdclef-2025/train_audio","metadata":{}},{"cell_type":"markdown","source":"## 2. Train-Soundscape File","metadata":{}},{"cell_type":"markdown","source":"Contains 1-min unlabeled soundscapes helpful for Model adaptation to real-world conditions with **No** Noise. ","metadata":{}},{"cell_type":"markdown","source":"## 3. **test_soundscapes**","metadata":{}},{"cell_type":"markdown","source":"Only contains ReadMe.txt file. it's of no use in both training and testing\nLink : /kaggle/input/birdclef-2025/test_soundscapes","metadata":{}},{"cell_type":"markdown","source":"## 4. **recording-location.txt**","metadata":{}},{"cell_type":"markdown","source":"Just contains the address and Website. Not helpful in Training and Testing","metadata":{}},{"cell_type":"markdown","source":"## 5. **taxonomy.csv**","metadata":{}},{"cell_type":"markdown","source":"Just contains the Taxonomy CSV useful for Reserach. Contains Scientific Name, Common Name and their Class Name of Taxon. Not useful !","metadata":{}},{"cell_type":"markdown","source":"## 6. **train.csv**","metadata":{}},{"cell_type":"markdown","source":"The train.csv contains Reference for each Bird Sound. Here \"filename\" contains the Exact File Name of the Bird Referenced to \"scientific_name\" and 'common_name'. This is most important for Training the Bird data and referencing to the Bird Name.","metadata":{}},{"cell_type":"markdown","source":"# **B. Data Viusalization--------------->**","metadata":{}},{"cell_type":"markdown","source":"## 1. **Visualizing Audio Waveforms**","metadata":{}},{"cell_type":"markdown","source":"Understand raw audio signals (amplitude vs. time) to Detect clipping, silence, or unusual patterns","metadata":{}},{"cell_type":"code","source":"import librosa\nimport librosa.display\nimport matplotlib.pyplot as plt\n\n\naudio_path = \"/kaggle/input/birdclef-2025/train_audio/1139490/CSA36385.ogg\"\ny, sr = librosa.load(audio_path, sr=32000)  # Sample rate = 32 kHz\n\n\nplt.figure(figsize=(12, 4))\nlibrosa.display.waveshow(y, sr=sr, alpha=0.5)\nplt.title(\"Raw Audio Waveform (CSA36385.ogg)\")\nplt.xlabel(\"Time (s)\")\nplt.ylabel(\"Amplitude\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:19:25.368300Z","iopub.execute_input":"2025-05-07T14:19:25.368552Z","iopub.status.idle":"2025-05-07T14:19:39.812666Z","shell.execute_reply.started":"2025-05-07T14:19:25.368512Z","shell.execute_reply":"2025-05-07T14:19:39.811837Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Could you see the Continuity of Sound? SO SO NOISY........With great ups and downs.... Well, Now Run this Graph","metadata":{}},{"cell_type":"code","source":"import librosa\nimport librosa.display\nimport matplotlib.pyplot as plt\n\n\naudio_path = \"/kaggle/input/birdclef-2025/train_soundscapes/H02_20230420_074000.ogg\"\ny, sr = librosa.load(audio_path, sr=32000)  \n\n\nplt.figure(figsize=(12, 4))\nlibrosa.display.waveshow(y, sr=sr, alpha=0.5)\nplt.title(\"Raw Audio Waveform (H02_20230420_074000.ogg)\")\nplt.xlabel(\"Time (s)\")\nplt.ylabel(\"Amplitude\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:19:39.814003Z","iopub.execute_input":"2025-05-07T14:19:39.814356Z","iopub.status.idle":"2025-05-07T14:19:40.291680Z","shell.execute_reply.started":"2025-05-07T14:19:39.814327Z","shell.execute_reply":"2025-05-07T14:19:40.290851Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Could you understand difference between **train_audio** as well as **train_soundscape**. As already told latter will be contains Sound without Background Interference","metadata":{}},{"cell_type":"markdown","source":"You could use other Sounds to Visualize !!!","metadata":{}},{"cell_type":"markdown","source":"## 2. **Spectrograms (Time-Frequency Analysis)**","metadata":{}},{"cell_type":"markdown","source":"Let's Identify dominant frequencies like bird calls over time","metadata":{}},{"cell_type":"code","source":"import numpy as np\nS = librosa.feature.melspectrogram(y=y, sr=sr, n_mels=128, fmax=16000)\nS_dB = librosa.power_to_db(S, ref=np.max)  # Convert to dB\n\n# Plot\nplt.figure(figsize=(12, 6))\nlibrosa.display.specshow(S_dB, x_axis=\"time\", y_axis=\"mel\", sr=sr, fmax=16000)\nplt.colorbar(format=\"%+2.0f dB\")\nplt.title(\"Mel-Spectrogram (XC12345.ogg)\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:19:40.292473Z","iopub.execute_input":"2025-05-07T14:19:40.292797Z","iopub.status.idle":"2025-05-07T14:19:42.126973Z","shell.execute_reply.started":"2025-05-07T14:19:40.292778Z","shell.execute_reply":"2025-05-07T14:19:42.126162Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"So Most of them remain around 10-20Db sound in most of Audio","metadata":{}},{"cell_type":"markdown","source":"## 3. **Feature Distributions (EDA)**","metadata":{}},{"cell_type":"markdown","source":"**Species Distribution**","metadata":{}},{"cell_type":"markdown","source":"To Detect class imbalance, recording quality, or geographic bias","metadata":{}},{"cell_type":"code","source":"import seaborn as sns\nimport pandas as pd\n\ntrain_df = pd.read_csv(\"/kaggle/input/birdclef-2025/train.csv\")\n\nplt.figure(figsize=(12, 8))\nsns.countplot(data=train_df, y=\"primary_label\", order=train_df[\"primary_label\"].value_counts().index[:20])  # Top 20 species\nplt.title(\"Top 20 Species by Training Samples\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:19:42.127768Z","iopub.execute_input":"2025-05-07T14:19:42.128083Z","iopub.status.idle":"2025-05-07T14:19:43.222106Z","shell.execute_reply.started":"2025-05-07T14:19:42.128062Z","shell.execute_reply":"2025-05-07T14:19:43.221429Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"You could see grekis as the Popular Bird. So, Even if your ML Model Prdict grekis only, you are 50% more correct than nothing :)","metadata":{}},{"cell_type":"markdown","source":"**Recording Quality (Rating Distribution)**","metadata":{}},{"cell_type":"code","source":"sns.countplot(data=train_df, x=\"rating\")\nplt.title(\"Distribution of Recording Quality Ratings (1-5)\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:19:43.222741Z","iopub.execute_input":"2025-05-07T14:19:43.222938Z","iopub.status.idle":"2025-05-07T14:19:43.385217Z","shell.execute_reply.started":"2025-05-07T14:19:43.222922Z","shell.execute_reply":"2025-05-07T14:19:43.384576Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"You could see most have 0.0 Rating? It's becuase of Incomplete data. While you should be happy that 4.0 have 2nd Most Rated Data, so your ML Data is Safe","metadata":{}},{"cell_type":"markdown","source":"**Geographic Clustering (Lat/Long)**","metadata":{}},{"cell_type":"code","source":"sns.scatterplot(data=train_df, x=\"longitude\", y=\"latitude\", alpha=0.5, hue=\"collection\")\nplt.title(\"Recording Locations (XC vs. iNat vs. CSA)\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:19:43.385820Z","iopub.execute_input":"2025-05-07T14:19:43.385995Z","iopub.status.idle":"2025-05-07T14:19:44.334784Z","shell.execute_reply.started":"2025-05-07T14:19:43.385980Z","shell.execute_reply":"2025-05-07T14:19:44.333979Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"You see most species are from America, so the Birds presented here are Native Singers of America Land.","metadata":{}},{"cell_type":"markdown","source":"## 4. **Call Duration Analysis**","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Check if call lengths vary by species (affects segmentation)","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport librosa\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport os\n\n# Load metadata\ntrain_df = pd.read_csv(\"/kaggle/input/birdclef-2025/train.csv\")\n\ndef get_duration_safe(row):\n    audio_path = os.path.join(\"/kaggle/input/birdclef-2025/train_audio\",  \n                            row['filename'])\n    try:\n        return librosa.get_duration(path=audio_path)  # Use 'path' instead of 'filename'\n    except Exception as e:\n        print(f\"Error processing {audio_path}: {str(e)}\")\n        return None  # Return None for missing files\n\ntrain_df[\"duration\"] = train_df.apply(get_duration_safe, axis=1)\n\n# Drop rows with missing durations\ntrain_df = train_df.dropna(subset=['duration'])\n\n# Plot top 10 species by duration\nplt.figure(figsize=(12, 6))\ntop_species = train_df[\"primary_label\"].value_counts().index[:10]\nsns.boxplot(\n    data=train_df[train_df[\"primary_label\"].isin(top_species)],\n    x=\"duration\",\n    y=\"primary_label\",\n    order=top_species\n)\nplt.title(\"Call Duration by Species (Top 10)\")\nplt.tight_layout()  # Prevent label cutoff\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:19:44.336496Z","iopub.execute_input":"2025-05-07T14:19:44.336749Z","iopub.status.idle":"2025-05-07T14:24:21.020197Z","shell.execute_reply.started":"2025-05-07T14:19:44.336730Z","shell.execute_reply":"2025-05-07T14:24:21.019441Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 5. **Background Noise**","metadata":{}},{"cell_type":"markdown","source":"Compare bg noise in train_soundscape","metadata":{}},{"cell_type":"code","source":"# Load a soundscape\nsoundscape, sr = librosa.load(\"/kaggle/input/birdclef-2025/train_soundscapes/H02_20230420_074000.ogg\", sr=32000)\n\nplt.figure(figsize=(12, 8))\nplt.subplot(2, 1, 1)\nlibrosa.display.waveshow(soundscape, sr=sr)\nplt.title(\"Soundscape Waveform (Background Noise)\")\n\nplt.subplot(2, 1, 2)\nS_soundscape = librosa.feature.melspectrogram(y=soundscape, sr=sr)\nlibrosa.display.specshow(librosa.power_to_db(S_soundscape), x_axis=\"time\", y_axis=\"mel\")\nplt.title(\"Soundscape Spectrogram\")\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:24:21.021069Z","iopub.execute_input":"2025-05-07T14:24:21.021362Z","iopub.status.idle":"2025-05-07T14:24:22.119467Z","shell.execute_reply.started":"2025-05-07T14:24:21.021336Z","shell.execute_reply":"2025-05-07T14:24:22.118789Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Compare bg noise in train_audio","metadata":{}},{"cell_type":"code","source":"# Load a soundscape\nsoundscape, sr = librosa.load(\"/kaggle/input/birdclef-2025/train_audio/1139490/CSA36385.ogg\", sr=32000)\n\n\nplt.figure(figsize=(12, 8))\nplt.subplot(2, 1, 1)\nlibrosa.display.waveshow(soundscape, sr=sr)\nplt.title(\"Soundscape Waveform (Background Noise)\")\n\nplt.subplot(2, 1, 2)\nS_soundscape = librosa.feature.melspectrogram(y=soundscape, sr=sr)\nlibrosa.display.specshow(librosa.power_to_db(S_soundscape), x_axis=\"time\", y_axis=\"mel\")\nplt.title(\"Soundscape Spectrogram\")\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-07T14:24:22.120444Z","iopub.execute_input":"2025-05-07T14:24:22.120753Z","iopub.status.idle":"2025-05-07T14:24:23.561172Z","shell.execute_reply.started":"2025-05-07T14:24:22.120727Z","shell.execute_reply":"2025-05-07T14:24:23.560576Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# **C. Approach / Tips / Guide ------------------>**","metadata":{}},{"cell_type":"markdown","source":"## 1. Important Topics You should known \n\n\n1. Librosa for audio processing\n2. Pytorch/TensorFlow for deep learning\n3. Scikit-learn/XGBoost for traditional models\n4. TPU/Colab for faster training\n5. Audiomentations for data augmentation.\n","metadata":{}},{"cell_type":"markdown","source":"## 2. Approach","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport librosa\nimport librosa.display\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.metrics import accuracy_score\nimport joblib  # For saving models\n\n# Paths\nTRAIN_AUDIO_PATH = '/kaggle/input/birdclef-2025/train_audio'\nTRAIN_CSV_PATH = '/kaggle/input/birdclef-2025/train.csv'\nTRAIN_SOUNDSCAPE_PATH = '/kaggle/input/birdclef-2025/train_soundscapes'\n\n# Step 1: Load Data\nprint(\"Loading metadata...\")\ntrain_df = pd.read_csv(TRAIN_CSV_PATH)\n\nprint(f\"Number of samples: {len(train_df)}\")\nprint(train_df.head())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:10:05.255054Z","iopub.execute_input":"2025-05-09T01:10:05.255324Z","iopub.status.idle":"2025-05-09T01:10:11.181455Z","shell.execute_reply.started":"2025-05-09T01:10:05.255300Z","shell.execute_reply":"2025-05-09T01:10:11.180027Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Step 2: Feature Extraction\ndef extract_features(file_path, n_mfcc=13, fixed_length=100):\n    try:\n        audio, sr = librosa.load(file_path, sr=None)\n        mfcc = librosa.feature.mfcc(y=audio, sr=sr, n_mfcc=n_mfcc)\n        \n        # Pad or truncate to fixed length\n        if mfcc.shape[1] < fixed_length:\n            mfcc = np.pad(mfcc, ((0, 0), (0, fixed_length - mfcc.shape[1])), mode='constant')\n        else:\n            mfcc = mfcc[:, :fixed_length]\n        \n        return mfcc.flatten()  # Flatten into a 1D array\n    except Exception as e:\n        print(f\"Error processing {file_path}: {e}\")\n        return np.zeros(n_mfcc * fixed_length)  # Return consistent-sized zero array\n\n\n\nprint(\"Extracting features (dummy setup with 10 samples)...\")\ndummy_train_df = train_df.head(10).copy()\ndummy_train_df['file_path'] = dummy_train_df['filename'].apply(lambda x: os.path.join(TRAIN_AUDIO_PATH, x))\ndummy_train_df['features'] = dummy_train_df['file_path'].apply(extract_features)\n\n# Update the main DataFrame\ntrain_df.loc[dummy_train_df.index, 'features'] = dummy_train_df['features']\n\n\n\n\n\n#    print(\"Extracting features...for all Images \")\n#    train_df['file_path'] = train_df['filename'].apply(lambda x: os.path.join(TRAIN_AUDIO_PATH, x))\n#  train_df['features'] = train_df['file_path'].apply(extract_features)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:21:48.689356Z","iopub.execute_input":"2025-05-09T01:21:48.691149Z","iopub.status.idle":"2025-05-09T01:21:51.293077Z","shell.execute_reply.started":"2025-05-09T01:21:48.691003Z","shell.execute_reply":"2025-05-09T01:21:51.292064Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\n# Prepare input features\nX = np.vstack(dummy_train_df['features'].values)\ny = dummy_train_df['scientific_name']\n\n# Encode target labels\nfrom sklearn.preprocessing import LabelEncoder\nle = LabelEncoder()\ny_encoded = le.fit_transform(y)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:22:07.283988Z","iopub.execute_input":"2025-05-09T01:22:07.284381Z","iopub.status.idle":"2025-05-09T01:22:07.291723Z","shell.execute_reply.started":"2025-05-09T01:22:07.284354Z","shell.execute_reply":"2025-05-09T01:22:07.290603Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Step 3: Train-Test Split\nprint(\"Splitting data...\")\nX_train, X_valid, y_train, y_valid = train_test_split(X, y_encoded, test_size=0.2, random_state=42)\n\n# Step 4: Model Training\nprint(\"Training model...\")\nmodel = RandomForestClassifier(n_estimators=100, random_state=42)\nmodel.fit(X_train, y_train)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:22:10.428159Z","iopub.execute_input":"2025-05-09T01:22:10.428634Z","iopub.status.idle":"2025-05-09T01:22:10.598011Z","shell.execute_reply.started":"2025-05-09T01:22:10.428576Z","shell.execute_reply":"2025-05-09T01:22:10.596864Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Step 5: Evaluation\nprint(\"Evaluating model...\")\ny_pred = model.predict(X_valid)\naccuracy = accuracy_score(y_valid, y_pred)\nprint(f\"Validation Accuracy: {accuracy:.2f}\")\n\n# Save the model\njoblib.dump(model, 'birdclef_model.pkl')\n\n# Step 6: Generate Submission\ndef generate_submission(test_audio_folder, model, encoder):\n    test_files = os.listdir(test_audio_folder)\n    submission = []\n    for file in test_files:\n        file_path = os.path.join(test_audio_folder, file)\n        features = extract_features(file_path).reshape(1, -1)\n        pred = model.predict(features)\n        label = encoder.inverse_transform(pred)[0]\n        submission.append({'row_id': file.split('.')[0], 'birds': label})\n    return pd.DataFrame(submission)\n\nprint(\"Generating submission...\")\ntest_audio_path = '/kaggle/input/birdclef-2025/train_audio' \nsubmission_df = generate_submission(test_audio_path, model, le)\n# Save file by removing # from below \n# submission_df.to_csv('submission.csv', index=False)\n\nprint(\"Submission file created.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:22:36.487846Z","iopub.execute_input":"2025-05-09T01:22:36.489135Z","iopub.status.idle":"2025-05-09T01:22:38.299633Z","shell.execute_reply.started":"2025-05-09T01:22:36.489093Z","shell.execute_reply":"2025-05-09T01:22:38.298374Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3. Resoucres\n\nhttps://librosa.org/doc/latest/index.html\n\nhttps://www.coursera.org/learn/audio-signal-processing\n\nhttps://towardsdatascience.com/audio-feature-extraction-in-python-a-step-by-step-guide-87f4ccfb73cd","metadata":{}},{"cell_type":"markdown","source":"# D. Statistics","metadata":{}},{"cell_type":"markdown","source":"# **1. Descriptive Statistics------------->**","metadata":{}},{"cell_type":"markdown","source":"## **1. Missing Values**","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom scipy.stats import ttest_ind, f_oneway, pearsonr, spearmanr\n\n# Load the dataset\ndata = pd.read_csv(\"/kaggle/input/birdclef-2025/train.csv\")\n\nprint(\"Descriptive Statistics:\")\nprint(data.describe(include='all')) \nprint(\"\\nMissing Values:\")\nprint(data.isnull().sum()) ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:29:11.798251Z","iopub.execute_input":"2025-05-09T01:29:11.799306Z","iopub.status.idle":"2025-05-09T01:29:15.018793Z","shell.execute_reply.started":"2025-05-09T01:29:11.799265Z","shell.execute_reply":"2025-05-09T01:29:15.017529Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2. Distribution of Ratiings","metadata":{}},{"cell_type":"code","source":"# Distribution of ratings\nplt.figure(figsize=(8, 6))\nsns.histplot(data['rating'], kde=True, bins=20)\nplt.title(\"Distribution of Ratings\")\nplt.xlabel(\"Rating\")\nplt.ylabel(\"Frequency\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:29:11.798251Z","iopub.execute_input":"2025-05-09T01:29:11.799306Z","iopub.status.idle":"2025-05-09T01:29:15.018793Z","shell.execute_reply.started":"2025-05-09T01:29:11.799265Z","shell.execute_reply":"2025-05-09T01:29:15.017529Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3. Frequency of Scientific Names","metadata":{}},{"cell_type":"code","source":"# Frequency of scientific names\ntop_scientific_names = data['scientific_name'].value_counts().head(10)\nplt.figure(figsize=(10, 6))\ntop_scientific_names.plot(kind='bar', color='skyblue')\nplt.title(\"Top 10 Most Common Birds\")\nplt.xlabel(\"Scientific Name\")\nplt.ylabel(\"Frequency\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:29:11.798251Z","iopub.execute_input":"2025-05-09T01:29:11.799306Z","iopub.status.idle":"2025-05-09T01:29:15.018793Z","shell.execute_reply.started":"2025-05-09T01:29:11.799265Z","shell.execute_reply":"2025-05-09T01:29:15.017529Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 4. Geospatial Destribution","metadata":{}},{"cell_type":"code","source":"# Geospatial Distribution\nplt.figure(figsize=(8, 6))\nsns.scatterplot(x=data['longitude'], y=data['latitude'], hue=data['rating'], palette=\"viridis\")\nplt.title(\"Geospatial Distribution of Bird Occurrences\")\nplt.xlabel(\"Longitude\")\nplt.ylabel(\"Latitude\")\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:29:11.798251Z","iopub.execute_input":"2025-05-09T01:29:11.799306Z","iopub.status.idle":"2025-05-09T01:29:15.018793Z","shell.execute_reply.started":"2025-05-09T01:29:11.799265Z","shell.execute_reply":"2025-05-09T01:29:15.017529Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# **2. Infrential ------------------->**","metadata":{}},{"cell_type":"markdown","source":"## 1. Hypothethis Testing: Does rating differ significantly between collections?","metadata":{}},{"cell_type":"code","source":"\ncollections = data['collection'].unique()\nratings_by_collection = [data[data['collection'] == col]['rating'] for col in collections]\n\nif len(collections) > 2:\n    # ANOVA for more than two groups\n    f_stat, p_value = f_oneway(*ratings_by_collection)\n    print(\"\\nANOVA Test Results:\")\n    print(f\"F-statistic: {f_stat}, P-value: {p_value}\")\nelse:\n    # T-test for two groups\n    t_stat, p_value = ttest_ind(ratings_by_collection[0], ratings_by_collection[1])\n    print(\"\\nT-Test Results:\")\n    print(f\"T-statistic: {t_stat}, P-value: {p_value}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:29:11.798251Z","iopub.execute_input":"2025-05-09T01:29:11.799306Z","iopub.status.idle":"2025-05-09T01:29:15.018793Z","shell.execute_reply.started":"2025-05-09T01:29:11.799265Z","shell.execute_reply":"2025-05-09T01:29:15.017529Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2. Correlation Analysis between latitude, longitude, and rating","metadata":{}},{"cell_type":"code","source":"\npearson_corr_lat, _ = pearsonr(data['latitude'], data['rating'])\npearson_corr_lon, _ = pearsonr(data['longitude'], data['rating'])\nprint(\"\\nCorrelation Analysis:\")\nprint(f\"Pearson Correlation (Latitude vs Rating): {pearson_corr_lat}\")\nprint(f\"Pearson Correlation (Longitude vs Rating): {pearson_corr_lon}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-09T01:29:11.798251Z","iopub.execute_input":"2025-05-09T01:29:11.799306Z","iopub.status.idle":"2025-05-09T01:29:15.018793Z","shell.execute_reply.started":"2025-05-09T01:29:11.799265Z","shell.execute_reply":"2025-05-09T01:29:15.017529Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}