![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1605499%2Fcc2a1c6259b8b72194abef259d524448%2Ffreesound_model.png?generation=1561783764472053&alt=media\")
First I want to thank researchers at CVSSP, University of Surrey, UK for their great baseline code. My entire system is built upon it.
\n\nPre-processing\nNothing special. 32 kHz sampling rate, 500 hop size so that 64 fps. Silence trimming below max - 55 dB, mel size 128. Random excerpt 4-second patches from entire spectrograms. Shorter spectrograms are padded repeatedly.
Data augmentation\n1. Spec-augment without time warping\n2. Time reversal with 80 more classes (binary target with 160 classes in total)\n3. I'll talk about MixUp later
\n\nSemi-supervised learning\nAlmost the same as https://link.springer.com/chapter/10.1007/978-3-030-20873-8_26.
\n\n
Suppose V X y_V is a batch of curated data and W is a batch of noisy data without label, then we have\n
\n\nThe backbone is 9-layer CNN, which is the same as the one in public kernel and CVSSP baseline.
\n\nLoss function\nFocal + ArcFace.\nArcFace is modified for mult-label classification. 2 reasons to use ArcFace here\n1. Enlarge the margin between positive and negative, so that the threshold eta becomes less sensitive.\n2. In the original ArcFace paper, the authors state that ArcFace enforces less penalty towards samples whose feature vectors have large angle with the weight matrix than other margin-based loss. I suppose it would apply less \"weight\" to wrong pseudo-labels.
\n\nIn practice using ArcFace makes my NN converge faster and have higher local lwlrap when using only curated data. I don't know what would happen if I trained the semi-supervised model without it because I started using it at a very early stage.
\n\nTraining Details\nFirst I warm up the backbone CNN with noisy set. For the semi-supervised model, I have 6 samples from curated set and 58 samples from noisy set in each batch. Nadam optimizer, cosine annealing learning rate with linear warm-up are used. One can refer to my technical report, Table 2 for details about hyperparameters.
\n\nInference\nArithmetic mean on 0.25s sliding window. TTA only on clips with short duration by repeating the sliding window with Spec-augment.
\n\nSingle model trained on entire training set gives me 0.712 lwlrap on stage 1 test.
\n\nAgain, thank the host for hosting such interesting competition. I am still on my way to my first gold medal :)
", "messages": [ { "id": 564187, "postDate": "2019-06-29T05:20:52.430Z", "content": "First I want to thank researchers at CVSSP, University of Surrey, UK for their great baseline code. My entire system is built upon it.
\n\nPre-processing\nNothing special. 32 kHz sampling rate, 500 hop size so that 64 fps. Silence trimming below max - 55 dB, mel size 128. Random excerpt 4-second patches from entire spectrograms. Shorter spectrograms are padded repeatedly.
Data augmentation\n1. Spec-augment without time warping\n2. Time reversal with 80 more classes (binary target with 160 classes in total)\n3. I'll talk about MixUp later
\n\nSemi-supervised learning\nAlmost the same as https://link.springer.com/chapter/10.1007/978-3-030-20873-8_26.
\n\n
Suppose V X y_V is a batch of curated data and W is a batch of noisy data without label, then we have\n
\n\nThe backbone is 9-layer CNN, which is the same as the one in public kernel and CVSSP baseline.
\n\nLoss function\nFocal + ArcFace.\nArcFace is modified for mult-label classification. 2 reasons to use ArcFace here\n1. Enlarge the margin between positive and negative, so that the threshold eta becomes less sensitive.\n2. In the original ArcFace paper, the authors state that ArcFace enforces less penalty towards samples whose feature vectors have large angle with the weight matrix than other margin-based loss. I suppose it would apply less \"weight\" to wrong pseudo-labels.
\n\nIn practice using ArcFace makes my NN converge faster and have higher local lwlrap when using only curated data. I don't know what would happen if I trained the semi-supervised model without it because I started using it at a very early stage.
\n\nTraining Details\nFirst I warm up the backbone CNN with noisy set. For the semi-supervised model, I have 6 samples from curated set and 58 samples from noisy set in each batch. Nadam optimizer, cosine annealing learning rate with linear warm-up are used. One can refer to my technical report, Table 2 for details about hyperparameters.
\n\nInference\nArithmetic mean on 0.25s sliding window. TTA only on clips with short duration by repeating the sliding window with Spec-augment.
\n\nSingle model trained on entire training set gives me 0.712 lwlrap on stage 1 test.
\n\nAgain, thank the host for hosting such interesting competition. I am still on my way to my first gold medal :)
", "rawMarkdown": "First I want to thank researchers at CVSSP, University of Surrey, UK for their great baseline [code](https://github.com/qiuqiangkong/dcase2019_task2). My entire system is built upon it.\n\n**Pre-processing**\nNothing special. 32 kHz sampling rate, 500 hop size so that 64 fps. Silence trimming below `max - 55 dB`, mel size 128. Random excerpt 4-second patches from entire spectrograms. Shorter spectrograms are padded repeatedly.\n\n**Data augmentation**\n1. Spec-augment without time warping\n2. Time reversal with 80 more classes (binary target with 160 classes in total)\n3. I'll talk about MixUp later\n\n**Semi-supervised learning**\nAlmost the same as https://link.springer.com/chapter/10.1007/978-3-030-20873-8_26. \n\n\n\nSuppose V X y\\_V is a batch of curated data and W is a batch of noisy data without label, then we have\n\n\nThe backbone is 9-layer CNN, which is the same as the one in public kernel and CVSSP baseline.\n\n**Loss function**\nFocal + ArcFace.\nArcFace is modified for mult-label classification. 2 reasons to use ArcFace here\n1. Enlarge the margin between positive and negative, so that the threshold eta becomes less sensitive.\n2. In the [original ArcFace paper](https://arxiv.org/abs/1801.07698), the authors state that ArcFace enforces less penalty towards samples whose feature vectors have large angle with the weight matrix than other margin-based loss. I suppose it would apply less \"weight\" to wrong pseudo-labels.\n\nIn practice using ArcFace makes my NN converge faster and have higher local lwlrap when using only curated data. I don't know what would happen if I trained the semi-supervised model without it because I started using it at a very early stage.\n\n**Training Details**\nFirst I warm up the backbone CNN with noisy set. For the semi-supervised model, I have 6 samples from curated set and 58 samples from noisy set in each batch. Nadam optimizer, cosine annealing learning rate with linear warm-up are used. One can refer to my technical report, Table 2 for details about hyperparameters.\n\n**Inference**\nArithmetic mean on 0.25s sliding window. TTA only on clips with short duration by repeating the sliding window with Spec-augment. \n\nSingle model trained on entire training set gives me 0.712 lwlrap on stage 1 test.\n\nAgain, thank the host for hosting such interesting competition. I am still on my way to my first gold medal :)", "votes": 5 } ], "comments": [], "raw_markdown_by_id": { "564187": "First I want to thank researchers at CVSSP, University of Surrey, UK for their great baseline [code](https://github.com/qiuqiangkong/dcase2019_task2). My entire system is built upon it.\n\n**Pre-processing**\nNothing special. 32 kHz sampling rate, 500 hop size so that 64 fps. Silence trimming below `max - 55 dB`, mel size 128. Random excerpt 4-second patches from entire spectrograms. Shorter spectrograms are padded repeatedly.\n\n**Data augmentation**\n1. Spec-augment without time warping\n2. Time reversal with 80 more classes (binary target with 160 classes in total)\n3. I'll talk about MixUp later\n\n**Semi-supervised learning**\nAlmost the same as https://link.springer.com/chapter/10.1007/978-3-030-20873-8_26. \n\n\n\nSuppose V X y\\_V is a batch of curated data and W is a batch of noisy data without label, then we have\n\n\nThe backbone is 9-layer CNN, which is the same as the one in public kernel and CVSSP baseline.\n\n**Loss function**\nFocal + ArcFace.\nArcFace is modified for mult-label classification. 2 reasons to use ArcFace here\n1. Enlarge the margin between positive and negative, so that the threshold eta becomes less sensitive.\n2. In the [original ArcFace paper](https://arxiv.org/abs/1801.07698), the authors state that ArcFace enforces less penalty towards samples whose feature vectors have large angle with the weight matrix than other margin-based loss. I suppose it would apply less \"weight\" to wrong pseudo-labels.\n\nIn practice using ArcFace makes my NN converge faster and have higher local lwlrap when using only curated data. I don't know what would happen if I trained the semi-supervised model without it because I started using it at a very early stage.\n\n**Training Details**\nFirst I warm up the backbone CNN with noisy set. For the semi-supervised model, I have 6 samples from curated set and 58 samples from noisy set in each batch. Nadam optimizer, cosine annealing learning rate with linear warm-up are used. One can refer to my technical report, Table 2 for details about hyperparameters.\n\n**Inference**\nArithmetic mean on 0.25s sliding window. TTA only on clips with short duration by repeating the sliding window with Spec-augment. \n\nSingle model trained on entire training set gives me 0.712 lwlrap on stage 1 test.\n\nAgain, thank the host for hosting such interesting competition. I am still on my way to my first gold medal :)" } }![https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1605499%2F23d2ce6b7bb8c8d60b6e5577b09c32d1%2Ffreesound_algo.png?generation=1561784302344628&alt=media](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1605499%2F23d2ce6b7bb8c8d60b6e5577b09c32d1%2Ffreesound_algo.png?generation=1561784302344628&alt=media\"){kind=link}