{
  "id": 396838,
  "title": "Cleaned and combined labels from 2021, 2022 and 2023 competitions",
  "url": "/competitions/birdclef-2023/discussion/396838",
  "author_name": "Olly Powell",
  "post_date": "2023-03-23T03:41:53.906000",
  "votes": 23,
  "comment_count": 0,
  "views": 0,
  "content": "<p>This is to help people with training and pretraining.</p>\n<p>Dataset <a href=\"https://www.kaggle.com/datasets/ollypowell/cleaned-training-labels-21-23-for-birdclef2023?select=train_21_22_23.csv\" target=\"_blank\">here</a><br>\nNotebook that produced it <a href=\"https://www.kaggle.com/code/ollypowell/birdclef-21-22-23-data-cleaning\" target=\"_blank\">here</a><br>\nYou need to add the 22 and 21 datasets to you training notebook in the usual way, and then the filepaths in the tables will work.</p>\n<p>I have modified the tables to include a full file path, put all three comps into an identical format, and removed 6,686 duplicates from 22 and 21.  Also added a second csv for 2023, in the same format without removing any data rows.</p>\n<p>I plan to use data from all three competitions for training, but noticed there is some overlap in the file names.  Actually the file names are not all unique within within the 2023 competition, <em>XC316684.ogg</em> occurs in both the <em>gnbcam2</em> folder and the <em>cibwar1</em> folder.  Not necessarily a problem, so long as you're not relying on file name uniqueness alone.  I don't know if it was an error, or it is intentional, but there are two distinctive birds calls in that file so maybe it is intentional.</p>",
  "messages": [
    {
      "id": 2193020,
      "postDate": "2023-03-23T03:41:53.907Z",
      "content": "<p>This is to help people with training and pretraining.</p>\n<p>Dataset <a href=\"https://www.kaggle.com/datasets/ollypowell/cleaned-training-labels-21-23-for-birdclef2023?select=train_21_22_23.csv\" target=\"_blank\">here</a><br>\nNotebook that produced it <a href=\"https://www.kaggle.com/code/ollypowell/birdclef-21-22-23-data-cleaning\" target=\"_blank\">here</a><br>\nYou need to add the 22 and 21 datasets to you training notebook in the usual way, and then the filepaths in the tables will work.</p>\n<p>I have modified the tables to include a full file path, put all three comps into an identical format, and removed 6,686 duplicates from 22 and 21.  Also added a second csv for 2023, in the same format without removing any data rows.</p>\n<p>I plan to use data from all three competitions for training, but noticed there is some overlap in the file names.  Actually the file names are not all unique within within the 2023 competition, <em>XC316684.ogg</em> occurs in both the <em>gnbcam2</em> folder and the <em>cibwar1</em> folder.  Not necessarily a problem, so long as you're not relying on file name uniqueness alone.  I don't know if it was an error, or it is intentional, but there are two distinctive birds calls in that file so maybe it is intentional.</p>",
      "rawMarkdown": "This is to help people with training and pretraining.\n\nDataset [here](https://www.kaggle.com/datasets/ollypowell/cleaned-training-labels-21-23-for-birdclef2023?select=train_21_22_23.csv)\nNotebook that produced it [here](https://www.kaggle.com/code/ollypowell/birdclef-21-22-23-data-cleaning)\nYou need to add the 22 and 21 datasets to you training notebook in the usual way, and then the filepaths in the tables will work.\n\nI have modified the tables to include a full file path, put all three comps into an identical format, and removed 6,686 duplicates from 22 and 21.  Also added a second csv for 2023, in the same format without removing any data rows.\n\nI plan to use data from all three competitions for training, but noticed there is some overlap in the file names.  Actually the file names are not all unique within within the 2023 competition, *XC316684.ogg* occurs in both the *gnbcam2* folder and the *cibwar1* folder.  Not necessarily a problem, so long as you're not relying on file name uniqueness alone.  I don't know if it was an error, or it is intentional, but there are two distinctive birds calls in that file so maybe it is intentional.",
      "votes": 23
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2193020": "This is to help people with training and pretraining.\n\nDataset [here](https://www.kaggle.com/datasets/ollypowell/cleaned-training-labels-21-23-for-birdclef2023?select=train_21_22_23.csv)\nNotebook that produced it [here](https://www.kaggle.com/code/ollypowell/birdclef-21-22-23-data-cleaning)\nYou need to add the 22 and 21 datasets to you training notebook in the usual way, and then the filepaths in the tables will work.\n\nI have modified the tables to include a full file path, put all three comps into an identical format, and removed 6,686 duplicates from 22 and 21.  Also added a second csv for 2023, in the same format without removing any data rows.\n\nI plan to use data from all three competitions for training, but noticed there is some overlap in the file names.  Actually the file names are not all unique within within the 2023 competition, *XC316684.ogg* occurs in both the *gnbcam2* folder and the *cibwar1* folder.  Not necessarily a problem, so long as you're not relying on file name uniqueness alone.  I don't know if it was an error, or it is intentional, but there are two distinctive birds calls in that file so maybe it is intentional."
  }
}