My strategy was to use stratifiedKfold(n_splits=10) and train different networks for the splits and saving them.\nThis way every network did have a slightly different training data and unique validation data. \nI did combine these networks using SVC with rbf kernel and optimize the parameters using genetic algorithm.
\n\nI think the biggest problem was big class imbalance. My solution was to use sklearn.utils.class_weight as class weights when training networks. Also using in SVC 'balanced' option in 'class_weight' when combining outputs of different networks.
I did use InceptionV3 first with a dense 1000 layer at top with big dropout of 0.70, optimizer RMSprop(lr=7e-6).
\n\nThe rest networks in best solution were EfficientNetB3 and EfficientNetB2 with globalMaxPooling, dropout 0.2, optimizer RMSprop(lr=1e-5). Would like to have used also EffiecientNetB4, but this was not possible because limited GPU.
\n\nFor these genetic algorithm did give parameters for SVC:\ngamma: 0.0031166077 C: 0.1723079681 ovo cv=5 : 96.4341313% -> private 92.773%\ngamma: 0.0046777725 C: 0.0554389954 ovr cv=5 : 96.7086033% -> private 92.622%\nThese were using the same networks. It's seems, that in this case in SVC one-vs-one is better.
\n\nInelegant code mostly in finnish:\nhttps://github.com/THalfar/TAU-Vehicle-Type-Recognition-Competition
\n\nI would like to thank the organizer for this fun competition!
", "messages": [ { "id": "696262", "postDate": "12/16/2019 11:19:12", "content": "My strategy was to use stratifiedKfold(n_splits=10) and train different networks for the splits and saving them.\nThis way every network did have a slightly different training data and unique validation data. \nI did combine these networks using SVC with rbf kernel and optimize the parameters using genetic algorithm.
\n\nI think the biggest problem was big class imbalance. My solution was to use sklearn.utils.class_weight as class weights when training networks. Also using in SVC 'balanced' option in 'class_weight' when combining outputs of different networks.
I did use InceptionV3 first with a dense 1000 layer at top with big dropout of 0.70, optimizer RMSprop(lr=7e-6).
\n\nThe rest networks in best solution were EfficientNetB3 and EfficientNetB2 with globalMaxPooling, dropout 0.2, optimizer RMSprop(lr=1e-5). Would like to have used also EffiecientNetB4, but this was not possible because limited GPU.
\n\nFor these genetic algorithm did give parameters for SVC:\ngamma: 0.0031166077 C: 0.1723079681 ovo cv=5 : 96.4341313% -> private 92.773%\ngamma: 0.0046777725 C: 0.0554389954 ovr cv=5 : 96.7086033% -> private 92.622%\nThese were using the same networks. It's seems, that in this case in SVC one-vs-one is better.
\n\nInelegant code mostly in finnish:\nhttps://github.com/THalfar/TAU-Vehicle-Type-Recognition-Competition
\n\nI would like to thank the organizer for this fun competition!
", "rawMarkdown": "My strategy was to use stratifiedKfold(n_splits=10) and train different networks for the splits and saving them.\nThis way every network did have a slightly different training data and unique validation data. \nI did combine these networks using SVC with rbf kernel and optimize the parameters using genetic algorithm. \n\nI think the biggest problem was big class imbalance. My solution was to use `sklearn.utils.class_weight` as class weights when training networks. Also using in SVC 'balanced' option in 'class_weight' when combining outputs of different networks. \n\nI did use InceptionV3 first with a dense 1000 layer at top with big dropout of 0.70, optimizer RMSprop(lr=7e-6). \n\nThe rest networks in best solution were EfficientNetB3 and EfficientNetB2 with globalMaxPooling, dropout 0.2, optimizer RMSprop(lr=1e-5). Would like to have used also EffiecientNetB4, but this was not possible because limited GPU.\n\nFor these genetic algorithm did give parameters for SVC:\ngamma: 0.0031166077 C: 0.1723079681 ovo cv=5 : 96.4341313% -> private 92.773%\ngamma: 0.0046777725 C: 0.0554389954 ovr cv=5 : 96.7086033% -> private 92.622%\nThese were using the same networks. It's seems, that in this case in SVC one-vs-one is better. \n\nInelegant code mostly in finnish:\nhttps://github.com/THalfar/TAU-Vehicle-Type-Recognition-Competition\n\nI would like to thank the organizer for this fun competition!", "votes": null }, { "id": "725253", "postDate": "01/22/2020 00:06:47", "content": "Thank you for sharing your solution!
", "rawMarkdown": "Thank you for sharing your solution!", "votes": null } ], "comments": [ { "id": 725253, "author_name": "hoanhle", "author_url": "", "post_date": "01/22/2020 00:06:47", "content": "Thank you for sharing your solution!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "696262": "My strategy was to use stratifiedKfold(n_splits=10) and train different networks for the splits and saving them.\nThis way every network did have a slightly different training data and unique validation data. \nI did combine these networks using SVC with rbf kernel and optimize the parameters using genetic algorithm. \n\nI think the biggest problem was big class imbalance. My solution was to use `sklearn.utils.class_weight` as class weights when training networks. Also using in SVC 'balanced' option in 'class_weight' when combining outputs of different networks. \n\nI did use InceptionV3 first with a dense 1000 layer at top with big dropout of 0.70, optimizer RMSprop(lr=7e-6). \n\nThe rest networks in best solution were EfficientNetB3 and EfficientNetB2 with globalMaxPooling, dropout 0.2, optimizer RMSprop(lr=1e-5). Would like to have used also EffiecientNetB4, but this was not possible because limited GPU.\n\nFor these genetic algorithm did give parameters for SVC:\ngamma: 0.0031166077 C: 0.1723079681 ovo cv=5 : 96.4341313% -> private 92.773%\ngamma: 0.0046777725 C: 0.0554389954 ovr cv=5 : 96.7086033% -> private 92.622%\nThese were using the same networks. It's seems, that in this case in SVC one-vs-one is better. \n\nInelegant code mostly in finnish:\nhttps://github.com/THalfar/TAU-Vehicle-Type-Recognition-Competition\n\nI would like to thank the organizer for this fun competition!", "725253": "Thank you for sharing your solution!" }, "source": "meta" }