First of all, thanks a lot for the organizers - it was a fun task, especially a noisy dataset, which is full of hilarious gems.\nKaggle has rolled out new UI during early days of competition - resulting in completely missing kernel code and invisible cells.\nNew version of PyTorch was released
\n\nThose are just my outtakes, moving forward to the new competitions:\n- Build a team as early as possible. Sometimes it was very hard to motivate myself to stay up the whole night after the full working day at the day job. There are plenty of tasks of various scale and everyone would find his place.\n- Good data == Good model / Bad data == bad model. I've heard about this dozen of times. But it's very easy to get tunnel vision on network structure or training schedule and completely forget about input data transformations.\n- Cross-validation. It was actually the first time I've used KFold. The problem was that I've stacked all predictions by simply computing the mean value. There are plenty of more advanced tactics that would've given slightly better results.\n- Regularization is magic (sometimes). Training a network is some kind of art :) I was using vanilla Pytorch and it requires a bit more in-depth knowledge comparing to Fast.ai. That's just my impression.\n- Octave convolution doesn't seem to work well on spectrograms (at least I didn't see any improvements). I haven't tried CoordConv, but it would be interesting to compare results.
", "messages": [ { "id": 550653, "postDate": "2019-06-11T22:00:24.283Z", "content": "First of all, thanks a lot for the organizers - it was a fun task, especially a noisy dataset, which is full of hilarious gems.\nKaggle has rolled out new UI during early days of competition - resulting in completely missing kernel code and invisible cells.\nNew version of PyTorch was released
\n\nThose are just my outtakes, moving forward to the new competitions:\n- Build a team as early as possible. Sometimes it was very hard to motivate myself to stay up the whole night after the full working day at the day job. There are plenty of tasks of various scale and everyone would find his place.\n- Good data == Good model / Bad data == bad model. I've heard about this dozen of times. But it's very easy to get tunnel vision on network structure or training schedule and completely forget about input data transformations.\n- Cross-validation. It was actually the first time I've used KFold. The problem was that I've stacked all predictions by simply computing the mean value. There are plenty of more advanced tactics that would've given slightly better results.\n- Regularization is magic (sometimes). Training a network is some kind of art :) I was using vanilla Pytorch and it requires a bit more in-depth knowledge comparing to Fast.ai. That's just my impression.\n- Octave convolution doesn't seem to work well on spectrograms (at least I didn't see any improvements). I haven't tried CoordConv, but it would be interesting to compare results.
", "rawMarkdown": "First of all, thanks a lot for the organizers - it was a fun task, especially a noisy dataset, which is full of hilarious gems.\nKaggle has rolled out new UI during early days of competition - resulting in completely missing kernel code and invisible cells.\nNew version of PyTorch was released\n\nThose are just my outtakes, moving forward to the new competitions:\n- **Build a team as early as possible**. Sometimes it was very hard to motivate myself to stay up the whole night after the full working day at the day job. There are plenty of tasks of various scale and everyone would find his place.\n- **Good data == Good model / Bad data == bad model**. I've heard about this dozen of times. But it's very easy to get tunnel vision on network structure or training schedule and completely forget about input data transformations.\n- **Cross-validation. It was actually the first time I've used KFold**. The problem was that I've stacked all predictions by simply computing the mean value. There are plenty of more advanced tactics that would've given slightly better results.\n- **Regularization is magic (sometimes)**. Training a network is some kind of art :) I was using vanilla Pytorch and it requires a bit more in-depth knowledge comparing to Fast.ai. That's just my impression.\n- **Octave** convolution doesn't seem to work well on spectrograms (at least I didn't see any improvements). I haven't tried **CoordConv**, but it would be interesting to compare results.\n\n", "votes": 9 } ], "comments": [], "raw_markdown_by_id": { "550653": "First of all, thanks a lot for the organizers - it was a fun task, especially a noisy dataset, which is full of hilarious gems.\nKaggle has rolled out new UI during early days of competition - resulting in completely missing kernel code and invisible cells.\nNew version of PyTorch was released\n\nThose are just my outtakes, moving forward to the new competitions:\n- **Build a team as early as possible**. Sometimes it was very hard to motivate myself to stay up the whole night after the full working day at the day job. There are plenty of tasks of various scale and everyone would find his place.\n- **Good data == Good model / Bad data == bad model**. I've heard about this dozen of times. But it's very easy to get tunnel vision on network structure or training schedule and completely forget about input data transformations.\n- **Cross-validation. It was actually the first time I've used KFold**. The problem was that I've stacked all predictions by simply computing the mean value. There are plenty of more advanced tactics that would've given slightly better results.\n- **Regularization is magic (sometimes)**. Training a network is some kind of art :) I was using vanilla Pytorch and it requires a bit more in-depth knowledge comparing to Fast.ai. That's just my impression.\n- **Octave** convolution doesn't seem to work well on spectrograms (at least I didn't see any improvements). I haven't tried **CoordConv**, but it would be interesting to compare results.\n\n" } }