{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Odd Prediction explained with code example\n\nMark Eckdahl\nhttps://www.kaggle.com/meckdahl/\n\nIf you are here, you are probably confused, may have looked at the original skeleton code already and still confused (that was me).\nI added more to @cwthomson's code to make it quite a bit more useful.\n\nFirst off, this code leverages a birdcall-check (Big Thanks to @shonenkov for a nice checking dataset and notebook and several discussions to make this competition better)\n\n[ORIGINAL CODE from @cwthompson] \nIn this notebook we will import the test data and make a simple prediction.\n\n @cwthompson put together a working skeletal submission example here:\n\nhttps://www.kaggle.com/cwthompson/birdsong-making-a-prediction\n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Key Points\n\n1. Code and Commits will not find the bird test files, they are hidden UNTIL SUBMIT is done - Yes flying blind\n2. This code uses @shonenkov's alternative data for coding and committing IF SUBMIT CODE is NOT FOUND\n3. I added some functionionality to this code to put path in test db and much more printing of what is going on\n4. Remember, your code will only see the ACTUAL test data after you press Submit button on whatever submission.csv file you have (good luck!)","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport librosa\nimport librosa.display\nfrom pathlib import Path\n\nnp.random.seed(0)\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\n\nnSAMPLERATE = 22050","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"There are several pieces of information that we are given about the competition and the test set. Two of the important pieces are quoted below.\n\nThe following can be found on the [evaluation page](https://www.kaggle.com/c/birdsong-recognition/overview/evaluation):\n> Submissions will be evaluated based on their row-wise micro averaged F1 score.\n> \n> For each row_id/time window, you need to provide a space separated list of the set of birds that made a call beginning or ending in that time window. If there are no bird calls in a time window, use the code nocall.\n> \n> There are three sites in the test set. Sites 1 and 2 are labeled in 5 second increments, while site 3 was labeled per audio file due to the time consuming nature of the labeling process.\n\nThis explains how we need to structure our submission file. There will be several submissions for each test audio file, split by time windows (5 seconds) - unless they are from site 3.\n\nThe following can be found on the [data page](https://www.kaggle.com/c/birdsong-recognition/data):\n>The hidden test set audio consists of approximately 150 recordings in mp3 format, each roughly 10 minutes long. The recordings were taken at three separate remote locations. Sites 1 and 2 were labeled in 5 second increments and need matching predictions, but due to the time consuming nature of the labeling process the site 3 files are only labeled at the file level. Accordingly, site 3 has relatively few rows in the test set and needs lower time resolution predictions.\n>\n>Two example soundscapes from another data source are also provided to illustrate how the soundscapes are labeled and the hidden dataset folder structure. The two example audio files are BLKFR-10-CPL_20190611_093000.pt540.mp3 and ORANGE-7-CAP_20190606_093000.pt623.mp3. These soundscapes were kindly provided by Jack Dumbacher of the California Academy of Science's Department of Ornithology and Mammology.\n\nThis is just further information on the test data.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Loading Audio\n\nFirstly, we need to be able to read in five second windows of the test audio. We can do this using librosa. If the audio is from site 3 then we need to whole audio clip, and we can do this by setting duration to None.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_test_clip(path, start_time, duration=5):\n    return librosa.load(path, offset=start_time, duration=duration, sr=nSAMPLERATE)[0] \n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Getting Test Data\n\nThe information on the test audio is given in test.csv. We have outputted that below. The test audio is also contained in the test_audio folder.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"TEST = Path(\"../input/birdsong-recognition/test_audio\").exists()\n\nif TEST:\n    DATA_DIR = str(Path(\"../input/birdsong-recognition/\"))\n    print(\"SUBMIT PATH EXISTS, using it\")\nelse:\n    # dataset created by @shonenkov, thanks!\n    DATA_DIR = str(Path(\"../input/birdcall-check/\"))\n    print(\"Alternative Data PATH in use.\")\n\nprint(\"DATA_DIR (Dynamic) = \",DATA_DIR)\n\n\nTEST_FOLDER = DATA_DIR + '/test_audio/'\ntest = pd.read_csv(DATA_DIR + \"/test.csv\")\n\ntest.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Possible Birds\n\nThe possible birds can be found in the training set, with the ebird_code feature. Almost all are six letter codes. The first twenty have been outputted below.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/birdsong-recognition/train.csv')\nbirds = train['ebird_code'].unique()\nbirds=np.append(birds,'nocall')\nbirds[0:20]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def addPathToTest(test, root_path=TEST_FOLDER):\n    test['path'] = root_path + test['audio_id'] + '.mp3'      \n    return test\n    \ntest_path = addPathToTest(test)\n\ntest_path.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Making Predictions\n\nWith all of the information we have found so far, it is now possible for us to make predictions. This can be done in different ways depending on your model - for this example we will just be selecting random birds. For each row:\n\n1. Extract the information from test.csv  \n2. Load in the correct clip (using librosa)  \n3. Make a prediction  \n4. Store the prediction  ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"\ndef make_prediction(sound_clip, birds, bSite1or2=True):\n    if (np.random.randint(0,80) % 79 == 0) or not bSite1or2:\n        return np.random.choice(birds)\n    return \"nocall\" # near half \"nocall\", so just guessing at ~10%","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"try:  # Comment out top and bottom to allow error propogation\n# if True:\n    preds = []\n    nCount=0\n    for index, row in test.iterrows():\n        nCount +=1\n        # Get test row information\n        site = row['site']\n        start_time = row['seconds'] - 5\n        row_id = row['row_id']\n        audio_id = row['audio_id']\n        row_path = row['path']\n        print(\"count = #\", nCount, \", start, end = \", start_time, \", \", row['seconds'],\"######\" )\n        \n        bSite1or2 = (site == 'site_1' or site == 'site_2')\n        # Get the test sound clip\n        if bSite1or2:\n#             sound_clip = load_test_clip(TEST_FOLDER + audio_id + '.mp3', start_time)\n            sound_clip = load_test_clip(row_path, start_time)\n            print(\"===== site = \", site, \" - fn = \", (audio_id ), \" - 5 sec\")\n            if nCount < 3:\n                print(\"+++++ path = \", (TEST_FOLDER + audio_id + '.mp3') )\n        else:\n            sound_clip = load_test_clip(TEST_FOLDER + audio_id + '.mp3', 0, duration=None)\n            print(\"===== site = \", site, \" - fn = \", (audio_id ), \"- No LIMIT\")\n\n        # Make the prediction\n        pred = make_prediction(sound_clip, birds, bSite1or2 )\n\n        # Store prediction\n        preds.append([row_id, pred])\n        if (TEST == False) and (nCount >= 15): # make sure not submitting \n            print(\"Exit after quick test of 15\")\n            break\n    preds = pd.DataFrame(preds, columns=['row_id', 'birds'])\n\nexcept Exception as e:\n    preds = pd.read_csv('../input/birdsong-recognition/sample_submission.csv')\n    print(\"EXCEPTION -- count = \", nCount,\" - \", row_id, audio_id)\n    print(\"EXCEPTION  = \",e)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Outputting Submission","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"preds.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds.tail(30)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}