{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.11"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceType":"competition","sourceId":91844,"databundleVersionId":11361821},{"sourceType":"datasetVersion","sourceId":11870659,"datasetId":7459867,"databundleVersionId":12371479},{"sourceType":"datasetVersion","sourceId":12010270,"datasetId":7528517,"databundleVersionId":12528417},{"sourceType":"datasetVersion","sourceId":12051777,"datasetId":7505901,"databundleVersionId":12575373}],"dockerImageVersionId":31041,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"papermill":{"default_parameters":{},"duration":66.275304,"end_time":"2025-05-20T09:23:45.886660","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2025-05-20T09:22:39.611356","version":"2.6.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### 🟢 Introduction\n\nIn this notebook, we implement a **self-supervised pseudo-labeling pipeline** for unlabeled **BirdCLEF 2025 soundscape audio**. Leveraging a trained `ImprovedBirdCLEFModel` (based on the `seresnext26t_32x4d` backbone from TIMM), we process VAD-filtered 10-second mel spectrogram chunks to extract confident predictions.\n\nKey steps include:\n\n- Creating a custom dataset (`BirdMelNPZDatasetSSL`) that loads mel spectrograms from `.npz` batches using chunk IDs.\n- Applying on-the-fly audio augmentations such as **time masking**, **frequency masking**, and **Gaussian noise addition**.\n- Generating predictions using the trained model and applying **sigmoid activation** to convert logits to probabilities.\n- Filtering predictions based on a confidence threshold (e.g., ≥ 0.7).\n- Saving pseudo-labels along with their `chunk_id` to a CSV file for downstream training.\n\nThis approach allows us to bootstrap labels from unlabeled data, improving model robustness and expanding the dataset for semi-supervised learning.\n\n---","metadata":{}},{"cell_type":"markdown","source":"## 🔗 BirdCLEF 2025 - Project Notebook Links\n\nHere are the different stages of my BirdCLEF 2025 pipeline, organized by functionality:\n\n### 📊 Data Preparation\n- [BirdCLEF 2025 - Data Preparation](https://www.kaggle.com/code/sheemamasood/birdclef-2025-data-prepartion)\n\n### 🎛️ Mel Spectrogram Generation\n- [BirdCLEF 2025 - Mel Generation](https://www.kaggle.com/code/sheemamasood/birdclef2025-mel-generation)\n\n### 🏷️ Pseudo Labelling for SSL\n- [BirdCLEF 2025 - Pseudo Labelling for SSL](https://www.kaggle.com/code/sheemamasood/birdclef2025-psedolabelling-for-ssl)\n\n### 🧠 Model Training\n- [BirdCLEF 2025 - Model Training (Phase 1)](https://www.kaggle.com/code/sheemamasood/birdclef2025-model-training-phase1)\n\n### 📦 Inference & Submissions\n- [BirdCLEF 2025 - Submissions](https://www.kaggle.com/code/sheemamasood/birdclef2025-submissions)\n","metadata":{}},{"cell_type":"code","source":"# Standard Libraries\nimport os\nimport gc\nimport time\nimport math\nimport random\nimport warnings\nimport logging\nfrom pathlib import Path\nfrom glob import glob\nfrom typing import Union\nimport copy\nimport concurrent.futures\n\n# Data Handling\nimport numpy as np\nimport pandas as pd\nimport joblib\nimport pickle\nimport collections\nfrom sklearn.model_selection import train_test_split, StratifiedKFold\nfrom sklearn.metrics import roc_auc_score, classification_report\n\n# Audio Processing\nimport librosa\nimport librosa.display\nimport soundfile as sf\nfrom soundfile import SoundFile\nimport torchaudio\n\n# Machine Learning & PyTorch\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torch.utils.data import Dataset, DataLoader, WeightedRandomSampler\nfrom torch.cuda.amp import autocast, GradScaler\nimport timm\n\n# Visualization\nimport cv2\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Progress Bars\nfrom tqdm import tqdm\nfrom tqdm.notebook import tqdm as notebook_tqdm\n\n# Logging and Warnings\nwarnings.filterwarnings(\"ignore\")\nlogging.basicConfig(level=logging.ERROR)\n\n# Check versions\nprint(f\"librosa version : {librosa.__version__}\")\nprint(f\"librosa files : {librosa.__file__}\")\nprint(\"✅ All libraries imported in the environment.\")\n","metadata":{"execution":{"iopub.status.busy":"2025-06-02T20:44:36.594347Z","iopub.execute_input":"2025-06-02T20:44:36.594630Z","iopub.status.idle":"2025-06-02T20:44:50.390500Z","shell.execute_reply.started":"2025-06-02T20:44:36.594609Z","shell.execute_reply":"2025-06-02T20:44:50.389754Z"},"papermill":{"duration":23.046365,"end_time":"2025-05-20T09:23:07.831968","exception":false,"start_time":"2025-05-20T09:22:44.785603","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\nclass Config:\n    # ===== General Settings =====\n    seed = 42\n    print_freq = 100\n    num_workers = 4\n\n    # ===== Audio Settings =====\n    FS = 32000  # Sampling rate\n    N_FFT = 1024\n    HOP_LENGTH = 512\n    FMIN = 50\n    FMAX = 14000\n    N_MELS = 128\n    TARGET_DURATION = 10.0  # duration in seconds for full input\n    TARGET_DURATION_TRAIN = 10\n    TARGET_DURATION_TEST = 5\n\n    # ===== Image / Mel Spectrogram Settings =====\n    MEL_SHAPE = (3, 256, 256)         # (height, width)\n    TARGET_SHAPE = (3, 256, 256)   # RGB Image Shape (C, H, W)\n\n    # ===== Training Settings =====\n    TRAINING_MODE = True\n    EPOCHS = 10\n    BATCH_SIZE = 32\n    LEARNING_RATE = 0.001\n    N_MAX = None  # Use full dataset by default\n\n    # ===== File Paths =====\n    train_datadir = '/kaggle/input/birdclef-2025/train_audio'\n    train_csv = '/kaggle/input/birdclef-2025/train.csv'\n    test_soundscapes = '/kaggle/input/birdclef-2025/test_soundscapes'\n    submission_csv = '/kaggle/input/birdclef-2025/sample_submission.csv'\n    taxonomy_csv = '/kaggle/input/birdclef-2025/taxonomy.csv'\n    model_files = ['/kaggle/input/bird2025-sed-ckpt/sedmodel.pth']\n    master_label_list_path = '/kaggle/input/birdcleft-clean-and-vad-filtered-data/valid_labels.pkl'\n\n    # ===== Model Settings =====\n    model_name = 'seresnext26t_32x4d'\n    pretrained = False\n    in_channels = 1\n\n    # ===== Derived Attributes (initialized later) =====\n    master_labels = None\n    NUM_CLASSES = None\n\n    @classmethod\n    def init_labels(cls):\n        with open(cls.master_label_list_path, \"rb\") as f:\n            cls.master_labels = pickle.load(f)\n        cls.NUM_CLASSES = len(cls.master_labels)\n\n\n# Instantiate and initialize\nconfig = Config()\nconfig.init_labels()\n\nprint(f\"✅ Loaded master label list. Total number of classes: {config.NUM_CLASSES}\")\n","metadata":{"execution":{"iopub.status.busy":"2025-06-02T20:44:58.126207Z","iopub.execute_input":"2025-06-02T20:44:58.126659Z","iopub.status.idle":"2025-06-02T20:44:58.139044Z","shell.execute_reply.started":"2025-06-02T20:44:58.126634Z","shell.execute_reply":"2025-06-02T20:44:58.138387Z"},"papermill":{"duration":0.012329,"end_time":"2025-05-20T09:23:07.848146","exception":false,"start_time":"2025-05-20T09:23:07.835817","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Load the master label list and set NUM_CLASSES\nwith open(\"/kaggle/input/birdcleft-clean-and-vad-filtered-data/master_label_list.pkl\", \"rb\") as f:\n    master_labels = pickle.load(f)\nNUM_CLASSES = len(master_labels)  # should be 206\n\n\nprint(f\"total number of labels in full data : {len(master_labels)}\")  ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:45:40.608497Z","iopub.execute_input":"2025-06-02T20:45:40.609115Z","iopub.status.idle":"2025-06-02T20:45:40.618287Z","shell.execute_reply.started":"2025-06-02T20:45:40.609089Z","shell.execute_reply":"2025-06-02T20:45:40.617679Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def set_seed(seed=42):\n    \"\"\"\n    Set seed for reproducibility\n    \"\"\"\n    random.seed(seed)\n    os.environ[\"PYTHONHASHSEED\"] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n\nset_seed(config.seed)\n","metadata":{"execution":{"iopub.status.busy":"2025-06-02T20:45:02.163195Z","iopub.execute_input":"2025-06-02T20:45:02.163899Z","iopub.status.idle":"2025-06-02T20:45:02.174354Z","shell.execute_reply.started":"2025-06-02T20:45:02.163846Z","shell.execute_reply":"2025-06-02T20:45:02.173710Z"},"papermill":{"duration":0.021291,"end_time":"2025-05-20T09:23:07.910633","exception":false,"start_time":"2025-05-20T09:23:07.889342","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## ImprovedBirdCLEFModel","metadata":{}},{"cell_type":"code","source":"class ImprovedBirdCLEFModel(nn.Module):\n    def __init__(self, num_classes=206, pretrained=True):\n        super().__init__()\n        \n        self.backbone = timm.create_model(\n            \"seresnext26t_32x4d\",  # ⬅️ hf_hub hata diya downloaded weights use kry gy\n            pretrained=False,  \n            in_chans=3,\n            num_classes=0\n        )\n        \n        self.classifier = nn.Sequential(\n            nn.Linear(self.backbone.num_features, 512),\n            nn.BatchNorm1d(512),\n            nn.ReLU(inplace=True),\n            nn.Dropout(0.3),\n            nn.Linear(512, num_classes)\n        )\n\n    def forward(self, x):\n        x = self.backbone(x)             # Already [B, 2048]\n        #print(\"After backbone:\", x.shape)\n        x = self.classifier(x)           # [B, num_classes]\n        return x\n\n\ndef freeze_backbone(model):\n    for param in model.backbone.parameters():\n        param.requires_grad = False\n\ndef unfreeze_last_blocks(model, num_blocks=2):\n    # Unfreezes last few blocks of the backbone\n    children = list(model.backbone.children())\n    for block in children[-num_blocks:]:\n        for param in block.parameters():\n            param.requires_grad = True","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:45:52.887095Z","iopub.execute_input":"2025-06-02T20:45:52.887668Z","iopub.status.idle":"2025-06-02T20:45:52.893803Z","shell.execute_reply.started":"2025-06-02T20:45:52.887648Z","shell.execute_reply":"2025-06-02T20:45:52.893058Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nmodel = ImprovedBirdCLEFModel(num_classes=206)\nstate_dict = torch.load('/kaggle/input/birdcleft-clean-and-vad-filtered-data/best_model_phase1.pth', map_location=device)\nmodel.load_state_dict(state_dict)\nmodel.to(device)\nmodel.eval()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:46:00.746433Z","iopub.execute_input":"2025-06-02T20:46:00.746923Z","iopub.status.idle":"2025-06-02T20:46:01.128825Z","shell.execute_reply.started":"2025-06-02T20:46:00.746901Z","shell.execute_reply":"2025-06-02T20:46:01.128184Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Phase 2 (continue training):","metadata":{}},{"cell_type":"code","source":"# Your paths\nmel_dir = \"/kaggle/input/birdclef-2025-mel-spectrogram\"\nsoundscape_csv_path = \"/kaggle/input/birdcleft-clean-and-vad-filtered-data/clean_soundscapes_chunks_10sec_vad_filtered.csv\"\n\n# Read soundscape CSV\nsoundscape_df = pd.read_csv(soundscape_csv_path)\nprint(\"✅ Soundscape CSV loaded:\", soundscape_df.shape)\n\n# Clean chunk_id for key matching (replace '/' with '-')\nsoundscape_df['chunk_id_clean'] = soundscape_df['chunk_id'].str.replace('/', '-', regex=False)\n\n# Filter soundscape mel batch files\nclean_soundscape_files = sorted([\n    f for f in os.listdir(mel_dir)\n    if f.startswith(\"clean_soundscape\") and f.endswith(\".npz\")\n])\nprint(f\"🟢 Found {len(clean_soundscape_files)} clean_soundscape .npz files\")\n\n# Build key-to-file index for clean_soundscape\nkey_to_file_soundscape = {}\n\nprint(\"🔍 Indexing clean_soundscape mel keys...\")\nfor batch_file in tqdm(clean_soundscape_files):\n    path = os.path.join(mel_dir, batch_file)\n    try:\n        npz = np.load(path)\n        for key in npz.files:\n            key_to_file_soundscape[key] = batch_file\n    except Exception as e:\n        print(f\"⚠️ Error reading {batch_file}: {e}\")\n\nprint(f\"✅ Total keys indexed: {len(key_to_file_soundscape)}\")\n\n# Optional check: How many chunk_ids from df are present in the index?\navailable_keys = soundscape_df['chunk_id_clean'].isin(key_to_file_soundscape)\nprint(f\"🧩 Matched chunks in index: {available_keys.sum()} / {len(soundscape_df)}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:46:09.758423Z","iopub.execute_input":"2025-06-02T20:46:09.758707Z","iopub.status.idle":"2025-06-02T20:46:10.319720Z","shell.execute_reply.started":"2025-06-02T20:46:09.758686Z","shell.execute_reply":"2025-06-02T20:46:10.319033Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### birdclef DataSet Class:","metadata":{}},{"cell_type":"code","source":"class BirdMelNPZDatasetSSL(Dataset):\n    def __init__(self, df, mel_dir, key_to_file, labels=None, augment=False, sample_rate=32000):\n        self.df = df.reset_index(drop=True)\n        self.mel_dir = mel_dir\n        self.key_to_file = key_to_file\n        self.augment = augment\n        self.sample_rate = sample_rate\n\n        # Optional labels argument to maintain structure\n        self.labels = labels\n\n        self.df['chunk_id_clean'] = self.df['chunk_id'].str.replace('/', '-', regex=False)\n        self.loaded_batches = {}\n\n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, idx):\n        row = self.df.iloc[idx]\n        key = row['chunk_id_clean']\n    \n        batch_file = self.key_to_file.get(key, None)\n        if batch_file is None:\n            mel = np.zeros((3, 256, 256), dtype=np.float32)\n        else:\n            if batch_file not in self.loaded_batches:\n                path = os.path.join(self.mel_dir, batch_file)\n                self.loaded_batches[batch_file] = np.load(path)\n    \n            mel = self.loaded_batches[batch_file][key]\n    \n        if self.augment:\n            mel = self.apply_augmentation(mel)\n    \n        mel_tensor = torch.tensor(mel, dtype=torch.float32)\n        return mel_tensor, key  # <-- Return tuple (tensor, key)\n\n    \n    def apply_augmentation(self, mel):\n        mel = self.time_mask(mel, T=30)\n        mel = self.freq_mask(mel, F=15)\n        mel = self.add_noise(mel, noise_level=0.01)\n        return mel\n\n    def time_mask(self, mel, T=30):\n        t = mel.shape[2]\n        t0 = np.random.randint(0, max(1, t - T))\n        mel[:, :, t0:t0 + T] = 0\n        return mel\n\n    def freq_mask(self, mel, F=15):\n        f = mel.shape[1]\n        f0 = np.random.randint(0, max(1, f - F))\n        mel[:, f0:f0 + F, :] = 0\n        return mel\n\n    def add_noise(self, mel, noise_level=0.01):\n        noise = np.random.randn(*mel.shape) * noise_level\n        return mel + noise\n\ndef ssl_collate_fn(batch):\n    mels = torch.stack([item[0] for item in batch])  # stack tensors into batch\n    keys = [item[1] for item in batch]              # keep keys as list\n    return mels, keys\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:46:27.116309Z","iopub.execute_input":"2025-06-02T20:46:27.117194Z","iopub.status.idle":"2025-06-02T20:46:27.126518Z","shell.execute_reply.started":"2025-06-02T20:46:27.117148Z","shell.execute_reply":"2025-06-02T20:46:27.125821Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ssl_dataset = BirdMelNPZDatasetSSL(df=soundscape_df, mel_dir=mel_dir, key_to_file=key_to_file_soundscape, augment=True)\n\nssl_loader = DataLoader(\n    ssl_dataset,\n    batch_size=32,\n    shuffle=True,\n    num_workers=0,\n    collate_fn=ssl_collate_fn\n)\n\n\n\ndef check_ssl_dataset(dataset, name=\"SSL Dataset\"):\n    print(f\"\\n=== {name.upper()} SAMPLE CHECK ===\")\n    \n    mel_tensor, key = dataset[0]  # unpack tuple\n    print(\"Type:\", type(mel_tensor))\n    print(\"Shape:\", mel_tensor.shape)\n    print(\"Dtype:\", mel_tensor.dtype)\n    print(\"Key (chunk id):\", key)\n\n\ncheck_ssl_dataset(ssl_dataset, \"SSL Dataset\")\n\n\nssl_batch, ssl_keys = next(iter(ssl_loader))\n\nprint(\"=== SSL BATCH ===\")\nprint(\"Type:\", type(ssl_batch))         # <class 'torch.Tensor'>\nprint(\"Batch shape:\", ssl_batch.shape)  # (32, 3, 256, 256)\nprint(\"Keys type:\", type(ssl_keys))     # <class 'list'>\nprint(\"Number of keys:\", len(ssl_keys)) # 32\n\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:46:33.644021Z","iopub.execute_input":"2025-06-02T20:46:33.644377Z","iopub.status.idle":"2025-06-02T20:46:34.664094Z","shell.execute_reply.started":"2025-06-02T20:46:33.644346Z","shell.execute_reply":"2025-06-02T20:46:34.663376Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def plot_multiple_mels_ssl(dataset, indices, title_prefix='SSL Sample Mel Spectrogram'):\n    n = len(indices)\n    fig, axes = plt.subplots(1, n, figsize=(4*n, 4))  # 1 row, n columns\n\n    for i, idx in enumerate(indices):\n        item = dataset[idx]\n        # If item is tuple, get first element, else item itself\n        mel_tensor = item[0] if isinstance(item, tuple) else item\n        mel = mel_tensor[0, :, :].numpy()  # first channel (2D matrix)\n\n        ax = axes[i] if n > 1 else axes\n        im = ax.imshow(mel, aspect='auto', origin='lower')\n        ax.set_title(f\"{title_prefix} #{idx}\")\n        ax.set_xlabel(\"Time Frames\")\n        ax.set_ylabel(\"Mel Bands\")\n        ax.label_outer()  # only show outer labels for clean look\n\n    fig.colorbar(im, ax=axes, orientation='vertical', fraction=0.02, pad=0.04)\n    plt.tight_layout()\n    plt.show()\n\n\nprint(\"SSL Dataset Samples:\")\nplot_multiple_mels_ssl(ssl_dataset, indices=list(range(5)), title_prefix='SSL Sample Mel')\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:46:41.056517Z","iopub.execute_input":"2025-06-02T20:46:41.057195Z","iopub.status.idle":"2025-06-02T20:46:42.031710Z","shell.execute_reply.started":"2025-06-02T20:46:41.057173Z","shell.execute_reply":"2025-06-02T20:46:42.030844Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from tqdm import tqdm\nimport numpy as np\nimport torch\n\ndef generate_pseudo_labels(\n    model, dataloader, device, threshold=0.7, return_filtered=True\n):\n    \"\"\"\n    Generate pseudo-labels from model predictions on unlabeled soundscape data.\n    \n    Args:\n        model: Trained PyTorch model (should output logits).\n        dataloader: DataLoader for soundscape data, should return (inputs, file_ids).\n        device: torch.device ('cuda' or 'cpu').\n        threshold: float, probability threshold to binarize predictions.\n        return_filtered: bool, if True, only return confident samples.\n    \n    Returns:\n        filenames: list of filenames of samples (filtered if return_filtered=True).\n        pseudo_labels: numpy array of pseudo-labels (binary).\n    \"\"\"\n    model.eval()\n    all_probs = []\n    all_filenames = []\n    \n    with torch.no_grad():\n        for inputs, file_ids in tqdm(dataloader, desc=\"Pseudo-labeling Soundscapes\"):\n            inputs = inputs.to(device)\n            outputs = model(inputs)  # logits\n            probs = torch.sigmoid(outputs).cpu().numpy()  # convert to probabilities\n            \n            all_probs.append(probs)\n            all_filenames.extend(file_ids)\n    \n    all_probs = np.vstack(all_probs)\n    \n    # Binarize predictions by threshold\n    pseudo_labels = (all_probs >= threshold).astype(int)\n    \n    if return_filtered:\n        # Filter samples where max probability exceeds threshold (confident predictions only)\n        confident_indices = [i for i, p in enumerate(all_probs) if p.max() > threshold]\n        \n        filtered_filenames = [all_filenames[i] for i in confident_indices]\n        filtered_pseudo_labels = pseudo_labels[confident_indices]\n        \n        return filtered_filenames, filtered_pseudo_labels\n    \n    return all_filenames, pseudo_labels\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:47:00.051154Z","iopub.execute_input":"2025-06-02T20:47:00.051393Z","iopub.status.idle":"2025-06-02T20:47:00.058032Z","shell.execute_reply.started":"2025-06-02T20:47:00.051378Z","shell.execute_reply":"2025-06-02T20:47:00.057330Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import pickle\n\n# Load master label list\nwith open(\"/kaggle/input/birdcleft-clean-and-vad-filtered-data/master_label_list.pkl\", \"rb\") as f:\n    class_names = pickle.load(f)\n\nprint(f\"Loaded {len(class_names)} class names.\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:55:15.316136Z","iopub.execute_input":"2025-06-02T20:55:15.316847Z","iopub.status.idle":"2025-06-02T20:55:15.322428Z","shell.execute_reply.started":"2025-06-02T20:55:15.316821Z","shell.execute_reply":"2025-06-02T20:55:15.321905Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Assume generate_pseudo_labels returns (chunk_ids, pseudo_labels)\nchunk_ids, pseudo_labels = generate_pseudo_labels(\n    model, ssl_loader, device, threshold=0.7\n)\n\nprint(f\"Generated pseudo-labels for {len(chunk_ids)} confident samples.\")\n\n# Create DataFrame with chunk_id as key\ndf = pd.DataFrame(pseudo_labels, columns=class_names)  # class_names = list of your classes\ndf['chunk_id'] = chunk_ids\n\n# Save with chunk_id column for easy merging later\ndf.to_csv(\"pseudo_labels_soundscape.csv\", index=False)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:55:18.090916Z","iopub.execute_input":"2025-06-02T20:55:18.091176Z","iopub.status.idle":"2025-06-02T20:59:31.977191Z","shell.execute_reply.started":"2025-06-02T20:55:18.091160Z","shell.execute_reply":"2025-06-02T20:59:31.976297Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# sanity check like label distribution\n\nlabel_counts = df[class_names].sum().sort_values(ascending=False)\nlabel_counts.plot(kind='bar', figsize=(15, 4))\nplt.title(\"Pseudo-label distribution per class\")\nplt.xticks(rotation=90)\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T20:59:31.978496Z","iopub.execute_input":"2025-06-02T20:59:31.978810Z","iopub.status.idle":"2025-06-02T20:59:33.070777Z","shell.execute_reply.started":"2025-06-02T20:59:31.978791Z","shell.execute_reply":"2025-06-02T20:59:33.069927Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"label_counts","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T21:00:04.329799Z","iopub.execute_input":"2025-06-02T21:00:04.330098Z","iopub.status.idle":"2025-06-02T21:00:04.336405Z","shell.execute_reply.started":"2025-06-02T21:00:04.330078Z","shell.execute_reply":"2025-06-02T21:00:04.335664Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import pandas as pd\n\npseudo_df = pd.read_csv(\"pseudo_labels_soundscape.csv\")\nprint(f\"Loaded {len(pseudo_df)} pseudo-labeled samples.\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-02T21:04:49.869241Z","iopub.execute_input":"2025-06-02T21:04:49.869879Z","iopub.status.idle":"2025-06-02T21:04:49.893509Z","shell.execute_reply.started":"2025-06-02T21:04:49.869842Z","shell.execute_reply":"2025-06-02T21:04:49.892749Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### ✅ Conclusion\n\nUsing this self-supervised pipeline, we successfully generated **1,684 confident pseudo-labeled samples** from BirdCLEF 2025 soundscapes. These predictions were stored in a structured CSV (`pseudo_labels_soundscape.csv`) and can now be used to augment training data.\n\nKey highlights:\n\n- Only samples with a maximum class probability ≥ 0.7 were retained to ensure label quality.\n- A sanity check with class-wise bar plot confirms a reasonable distribution of pseudo-labels.\n- This approach is particularly valuable for expanding datasets when labeled data is limited or imbalanced.\n\nBy incorporating these pseudo-labeled examples into the training process, we can improve model generalization, especially in rare or underrepresented bird classes.\n","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}