{ "id": 157946, "title": "LB #2 Documentation", "url": "/competitions/iwildcam-2020-fgvc7/writeups/jon-daly-lb-2-documentation", "author_name": "", "post_date": "2020-06-12T16:29:16.074755200Z", "votes": 14, "comment_count": 8, "views": 0, "content": "

Apologies for the late submission documentation, and thanks to the organisers of the competition - this was great fun.

\n\n

After attempting many different approaches my highest scoring solution was generally as follows:

\n\n
    \n
  1. Removed noisy and/or 'human' class categories - 'start', 'end', 'unknown', and 'unidentifiable'. Retaining the empty class seemed important for reducing activation on background to generalise better.
    2. \n
  2. Encorporated iNat 2017/2018 data for shared classes.
    3. \n
  3. Ran MegaDetector v4 on all images.
    4. \n
  4. Took only animal detection crops with confidence >0.3 and discarded detections which were too small. Also discarded crops within the same image if they were much less confident than the highest.
    5. \n
  5. Made sure to expand height of crop to be 224 pixels, even if the bounding box was smaller, then used reflection padding to square the crop and resized to 224x224 resolution.
    6. \n
  6. Applied various data augmentations including CLAHE, hue shift, gaussian noise, cutout, scale/rotate/shift, brightness/contrast adjustments and grayscale.
    7. \n
  7. Created a custom classifier head which incorporated metadata from the image. This used cos/sin representations of of the time of year and time of day, and information regarding the crop (width, height, pitch, yaw).
    8. \n
  8. Trained for 7 epochs with a single EfficientNetB4 model pretrained with imagenet noisystudent weights, using label-smoothed cross-entropy focal loss. This was 2 hours of training.
    9. \n
  9. Used TTA with every training augmentation except for noise to get predictions.
    10. \n
  10. Combined the predictions for each crop within an image to get a total average for the image.
    11. \n
  11. Averaged these predictions across images which were clustered by time and location for the final submission. This was only done as the sequence ID labels appeared to be unreliable.
    12. \n
  12. Achieved 0.930 public, 0.903 private LB score. The highest submission with this approach which I didn't pick was 0.908 private.
    13. \n
\n\n

This was my first proper experience with computer vision so I unfortunately ran out of time to try a lot of the things I wanted due to my slowness :). I wanted to try:\n- Ensembling different EfficientNets at different input resolutions.\n- Using the location multisat data to create a geo prior with which to augment the prediction. I had initially used this data directly in a single classifier model but it too easily overfit.\n- Creating a more realistic infrared data augmentation rather than relying on grayscale.\n- Using a siamese network to distinguish between the most confused species.\n- Reintroducing the 'vehicle'/'human' megadetector crops with a prior or separate classifier.

", "messages": [ { "id": "883443", "postDate": "06/12/2020 16:29:16", "content": "

Apologies for the late submission documentation, and thanks to the organisers of the competition - this was great fun.

\n\n

After attempting many different approaches my highest scoring solution was generally as follows:

\n\n
    \n
  1. Removed noisy and/or 'human' class categories - 'start', 'end', 'unknown', and 'unidentifiable'. Retaining the empty class seemed important for reducing activation on background to generalise better.
    2. \n
  2. Encorporated iNat 2017/2018 data for shared classes.
    3. \n
  3. Ran MegaDetector v4 on all images.
    4. \n
  4. Took only animal detection crops with confidence >0.3 and discarded detections which were too small. Also discarded crops within the same image if they were much less confident than the highest.
    5. \n
  5. Made sure to expand height of crop to be 224 pixels, even if the bounding box was smaller, then used reflection padding to square the crop and resized to 224x224 resolution.
    6. \n
  6. Applied various data augmentations including CLAHE, hue shift, gaussian noise, cutout, scale/rotate/shift, brightness/contrast adjustments and grayscale.
    7. \n
  7. Created a custom classifier head which incorporated metadata from the image. This used cos/sin representations of of the time of year and time of day, and information regarding the crop (width, height, pitch, yaw).
    8. \n
  8. Trained for 7 epochs with a single EfficientNetB4 model pretrained with imagenet noisystudent weights, using label-smoothed cross-entropy focal loss. This was 2 hours of training.
    9. \n
  9. Used TTA with every training augmentation except for noise to get predictions.
    10. \n
  10. Combined the predictions for each crop within an image to get a total average for the image.
    11. \n
  11. Averaged these predictions across images which were clustered by time and location for the final submission. This was only done as the sequence ID labels appeared to be unreliable.
    12. \n
  12. Achieved 0.930 public, 0.903 private LB score. The highest submission with this approach which I didn't pick was 0.908 private.
    13. \n
\n\n

This was my first proper experience with computer vision so I unfortunately ran out of time to try a lot of the things I wanted due to my slowness :). I wanted to try:\n- Ensembling different EfficientNets at different input resolutions.\n- Using the location multisat data to create a geo prior with which to augment the prediction. I had initially used this data directly in a single classifier model but it too easily overfit.\n- Creating a more realistic infrared data augmentation rather than relying on grayscale.\n- Using a siamese network to distinguish between the most confused species.\n- Reintroducing the 'vehicle'/'human' megadetector crops with a prior or separate classifier.

", "rawMarkdown": "Apologies for the late submission documentation, and thanks to the organisers of the competition - this was great fun.\n\nAfter attempting many different approaches my highest scoring solution was generally as follows:\n\n1. Removed noisy and/or 'human' class categories - 'start', 'end', 'unknown', and 'unidentifiable'. Retaining the empty class seemed important for reducing activation on background to generalise better.\n2. Encorporated iNat 2017/2018 data for shared classes.\n3. Ran MegaDetector v4 on all images.\n4. Took only animal detection crops with confidence >0.3 and discarded detections which were too small. Also discarded crops within the same image if they were much less confident than the highest.\n5. Made sure to expand height of crop to be 224 pixels, even if the bounding box was smaller, then used reflection padding to square the crop and resized to 224x224 resolution.\n6. Applied various data augmentations including CLAHE, hue shift, gaussian noise, cutout, scale/rotate/shift, brightness/contrast adjustments and grayscale.\n7. Created a custom classifier head which incorporated metadata from the image. This used cos/sin representations of of the time of year and time of day, and information regarding the crop (width, height, pitch, yaw).\n8. Trained for 7 epochs with a single EfficientNetB4 model pretrained with imagenet noisystudent weights, using label-smoothed cross-entropy focal loss. This was 2 hours of training.\n9. Used TTA with every training augmentation except for noise to get predictions.\n10. Combined the predictions for each crop within an image to get a total average for the image.\n11. Averaged these predictions across images which were clustered by time and location for the final submission. This was only done as the sequence ID labels appeared to be unreliable.\n12. Achieved 0.930 public, 0.903 private LB score. The highest submission with this approach which I didn't pick was 0.908 private.\n\nThis was my first proper experience with computer vision so I unfortunately ran out of time to try a lot of the things I wanted due to my slowness :). I wanted to try:\n- Ensembling different EfficientNets at different input resolutions.\n- Using the location multisat data to create a geo prior with which to augment the prediction. I had initially used this data directly in a single classifier model but it too easily overfit.\n- Creating a more realistic infrared data augmentation rather than relying on grayscale.\n- Using a siamese network to distinguish between the most confused species.\n- Reintroducing the 'vehicle'/'human' megadetector crops with a prior or separate classifier.", "votes": null }, { "id": "883446", "postDate": "06/12/2020 16:29:42", "content": "

@stevenyin here you go :)

", "rawMarkdown": "stevenyin here you go :)", "votes": null }, { "id": "884672", "postDate": "06/13/2020 14:38:39", "content": "

Hi Jon, great thanks for your post! Let me read it carefully and go back to you later.

\n\n

Thanks!

", "rawMarkdown": "Hi Jon, great thanks for your post! Let me read it carefully and go back to you later.\n\n Thanks!", "votes": null }, { "id": "887507", "postDate": "06/15/2020 17:48:14", "content": "

Thanks for sharing, couple questions - \nWhat is the pitch and yaw of the crop? \nHow much did the iNat classes improve your score?

", "rawMarkdown": "Thanks for sharing, couple questions - \nWhat is the pitch and yaw of the crop? \nHow much did the iNat classes improve your score?", "votes": null }, { "id": "887629", "postDate": "06/15/2020 19:21:26", "content": "

Hi Justin. Since I'm using a crop from a larger image rather than the original the intuition was that knowing whether a crop is high above the camera's horizon or on the ground, or whether it was cropped at the edge of the image may help in making accurate predictions. The pitch is 0. -> 1. from the bottom of the frame to the top, and the yaw is normalized to be 0. at the center and 1. at the edge. It seemed to give a small bump in validation accuracy so I kept it.

\n\n

I didn't record/don't remember how big the bump in accuracy from the iNat classes was, sorry. I don't believe it was too significant.

", "rawMarkdown": "Hi Justin. Since I'm using a crop from a larger image rather than the original the intuition was that knowing whether a crop is high above the camera's horizon or on the ground, or whether it was cropped at the edge of the image may help in making accurate predictions. The pitch is 0. -> 1. from the bottom of the frame to the top, and the yaw is normalized to be 0. at the center and 1. at the edge. It seemed to give a small bump in validation accuracy so I kept it.\n\nI didn't record/don't remember how big the bump in accuracy from the iNat classes was, sorry. I don't believe it was too significant.", "votes": null }, { "id": "887650", "postDate": "06/15/2020 19:36:14", "content": "

Hey Jon! Can you make sure to fill out this google form with the info above? https://docs.google.com/forms/d/e/1FAIpQLSeZbowaBJYvRstyO3lgToomOkHuTeooW6qvDzH0LNzZPDnrbg/viewform?usp=sf_link

", "rawMarkdown": "Hey Jon! Can you make sure to fill out this google form with the info above? https://docs.google.com/forms/d/e/1FAIpQLSeZbowaBJYvRstyO3lgToomOkHuTeooW6qvDzH0LNzZPDnrbg/viewform?usp=sf_link", "votes": null }, { "id": "887715", "postDate": "06/15/2020 20:17:46", "content": "

Done! :)

", "rawMarkdown": "Done! :)", "votes": null }, { "id": "887863", "postDate": "06/16/2020 00:22:43", "content": "

Ah. Cool idea!

", "rawMarkdown": "Ah. Cool idea!", "votes": null }, { "id": "930942", "postDate": "07/15/2020 20:31:56", "content": "

Hey Jon, congrats! Can you explain how you designed validation set? Many thanks

", "rawMarkdown": "Hey Jon, congrats! Can you explain how you designed validation set? Many thanks", "votes": null } ], "comments": [ { "id": 883446, "author_name": "jondaly", "author_url": "", "post_date": "06/12/2020 16:29:42", "content": "

@stevenyin here you go :)

", "votes": null, "replies": [ { "id": 884672, "author_name": "", "author_url": "", "post_date": "06/13/2020 14:38:39", "content": "

Hi Jon, great thanks for your post! Let me read it carefully and go back to you later.

\n\n

Thanks!

", "votes": null, "replies": [] } ] }, { "id": 887507, "author_name": "justinkay92", "author_url": "", "post_date": "06/15/2020 17:48:14", "content": "

Thanks for sharing, couple questions - \nWhat is the pitch and yaw of the crop? \nHow much did the iNat classes improve your score?

", "votes": null, "replies": [] }, { "id": 887629, "author_name": "jondaly", "author_url": "", "post_date": "06/15/2020 19:21:26", "content": "

Hi Justin. Since I'm using a crop from a larger image rather than the original the intuition was that knowing whether a crop is high above the camera's horizon or on the ground, or whether it was cropped at the edge of the image may help in making accurate predictions. The pitch is 0. -> 1. from the bottom of the frame to the top, and the yaw is normalized to be 0. at the center and 1. at the edge. It seemed to give a small bump in validation accuracy so I kept it.

\n\n

I didn't record/don't remember how big the bump in accuracy from the iNat classes was, sorry. I don't believe it was too significant.

", "votes": null, "replies": [ { "id": 887863, "author_name": "justinkay92", "author_url": "", "post_date": "06/16/2020 00:22:43", "content": "

Ah. Cool idea!

", "votes": null, "replies": [] } ] }, { "id": 887650, "author_name": "sbeery", "author_url": "", "post_date": "06/15/2020 19:36:14", "content": "

Hey Jon! Can you make sure to fill out this google form with the info above? https://docs.google.com/forms/d/e/1FAIpQLSeZbowaBJYvRstyO3lgToomOkHuTeooW6qvDzH0LNzZPDnrbg/viewform?usp=sf_link

", "votes": null, "replies": [ { "id": 887715, "author_name": "jondaly", "author_url": "", "post_date": "06/15/2020 20:17:46", "content": "

Done! :)

", "votes": null, "replies": [] } ] }, { "id": 930942, "author_name": "lennyom", "author_url": "", "post_date": "07/15/2020 20:31:56", "content": "

Hey Jon, congrats! Can you explain how you designed validation set? Many thanks

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "883443": "Apologies for the late submission documentation, and thanks to the organisers of the competition - this was great fun.\n\nAfter attempting many different approaches my highest scoring solution was generally as follows:\n\n1. Removed noisy and/or 'human' class categories - 'start', 'end', 'unknown', and 'unidentifiable'. Retaining the empty class seemed important for reducing activation on background to generalise better.\n2. Encorporated iNat 2017/2018 data for shared classes.\n3. Ran MegaDetector v4 on all images.\n4. Took only animal detection crops with confidence >0.3 and discarded detections which were too small. Also discarded crops within the same image if they were much less confident than the highest.\n5. Made sure to expand height of crop to be 224 pixels, even if the bounding box was smaller, then used reflection padding to square the crop and resized to 224x224 resolution.\n6. Applied various data augmentations including CLAHE, hue shift, gaussian noise, cutout, scale/rotate/shift, brightness/contrast adjustments and grayscale.\n7. Created a custom classifier head which incorporated metadata from the image. This used cos/sin representations of of the time of year and time of day, and information regarding the crop (width, height, pitch, yaw).\n8. Trained for 7 epochs with a single EfficientNetB4 model pretrained with imagenet noisystudent weights, using label-smoothed cross-entropy focal loss. This was 2 hours of training.\n9. Used TTA with every training augmentation except for noise to get predictions.\n10. Combined the predictions for each crop within an image to get a total average for the image.\n11. Averaged these predictions across images which were clustered by time and location for the final submission. This was only done as the sequence ID labels appeared to be unreliable.\n12. Achieved 0.930 public, 0.903 private LB score. The highest submission with this approach which I didn't pick was 0.908 private.\n\nThis was my first proper experience with computer vision so I unfortunately ran out of time to try a lot of the things I wanted due to my slowness :). I wanted to try:\n- Ensembling different EfficientNets at different input resolutions.\n- Using the location multisat data to create a geo prior with which to augment the prediction. I had initially used this data directly in a single classifier model but it too easily overfit.\n- Creating a more realistic infrared data augmentation rather than relying on grayscale.\n- Using a siamese network to distinguish between the most confused species.\n- Reintroducing the 'vehicle'/'human' megadetector crops with a prior or separate classifier.", "883446": "stevenyin here you go :)", "884672": "Hi Jon, great thanks for your post! Let me read it carefully and go back to you later.\n\n Thanks!", "887507": "Thanks for sharing, couple questions - \nWhat is the pitch and yaw of the crop? \nHow much did the iNat classes improve your score?", "887629": "Hi Justin. Since I'm using a crop from a larger image rather than the original the intuition was that knowing whether a crop is high above the camera's horizon or on the ground, or whether it was cropped at the edge of the image may help in making accurate predictions. The pitch is 0. -> 1. from the bottom of the frame to the top, and the yaw is normalized to be 0. at the center and 1. at the edge. It seemed to give a small bump in validation accuracy so I kept it.\n\nI didn't record/don't remember how big the bump in accuracy from the iNat classes was, sorry. I don't believe it was too significant.", "887650": "Hey Jon! Can you make sure to fill out this google form with the info above? https://docs.google.com/forms/d/e/1FAIpQLSeZbowaBJYvRstyO3lgToomOkHuTeooW6qvDzH0LNzZPDnrbg/viewform?usp=sf_link", "887715": "Done! :)", "887863": "Ah. Cool idea!", "930942": "Hey Jon, congrats! Can you explain how you designed validation set? Many thanks" }, "source": "meta" }