This competition is a first time when we have managed to get 74 place. I can not say that I am completely happy with the results, and definitely we have a million of points for improvement, but there are some philosophical thoughts that I would like to share:
\n\nOur models was much worse then most competion participants are reporting, but it seems that our way to blend models worked pretty well Final Submission Code
Having a declative pipeline that allowed us to manage experiment settings quickly, helped a lot but there are still a lot of room for improvement, especially in the area of experiment sharing, and postprocessing code.
Networks: xception [256,512], NasNet (256) - only one model one fold\nAugmentations: Flips,Rot90\nTest time augmentations: Flip, Rot90\nLoss functions: BCE,Focal Loss\nLearning: Adam , LR Finder, CLR(triangular2)
\n\nFail points:
\n\nEconomics and merging: after merging we gathered a lot of compute power, but all models except one, that was a part of our final submission was trained on my 2x1080 ti machine, so it seems that we was not able to utilize our compute effectively. My current feeling is that it is mostly result of later mergine (just few hours, before mergine dead line), so we was not able to align our thoughts quickly enough to make use of our unified compute and brain power.
Augmentations: we have used very conservative augmentations, and obviously this was one of the pain points of out models. Next time I will use less conservative augmentations and more TTA.
Explotation vs Exploration . Balancing this two things was critical, and I have a feeling that having multiple different branches of research after merger deadline does not allowed us to fully exploit potential of our best approach. Next time, I will try to switch to explotation a little bit earlier.
This competition is a first time when we have managed to get 74 place. I can not say that I am completely happy with the results, and definitely we have a million of points for improvement, but there are some philosophical thoughts that I would like to share:
\n\nOur models was much worse then most competion participants are reporting, but it seems that our way to blend models worked pretty well Final Submission Code
Having a declative pipeline that allowed us to manage experiment settings quickly, helped a lot but there are still a lot of room for improvement, especially in the area of experiment sharing, and postprocessing code.
Networks: xception [256,512], NasNet (256) - only one model one fold\nAugmentations: Flips,Rot90\nTest time augmentations: Flip, Rot90\nLoss functions: BCE,Focal Loss\nLearning: Adam , LR Finder, CLR(triangular2)
\n\nFail points:
\n\nEconomics and merging: after merging we gathered a lot of compute power, but all models except one, that was a part of our final submission was trained on my 2x1080 ti machine, so it seems that we was not able to utilize our compute effectively. My current feeling is that it is mostly result of later mergine (just few hours, before mergine dead line), so we was not able to align our thoughts quickly enough to make use of our unified compute and brain power.
Augmentations: we have used very conservative augmentations, and obviously this was one of the pain points of out models. Next time I will use less conservative augmentations and more TTA.
Explotation vs Exploration . Balancing this two things was critical, and I have a feeling that having multiple different branches of research after merger deadline does not allowed us to fully exploit potential of our best approach. Next time, I will try to switch to explotation a little bit earlier.
The pipeline could use some improvements in blending department.
", "rawMarkdown": "The pipeline could use some improvements in blending department.", "votes": null } ], "comments": [ { "id": 454026, "author_name": "munchilla", "author_url": "", "post_date": "01/11/2019 03:22:46", "content": "The pipeline could use some improvements in blending department.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "454022": "This competition is a first time when we have managed to get 74 place. I can not say that I am completely happy with the results, and definitely we have a million of points for improvement, but there are some philosophical thoughts that I would like to share:\n\n\n----------\n\n\n1. Our models was much worse then most competion participants are reporting, but it seems that our way to blend models worked pretty well [Final Submission Code][1]\n\n2. Having a [declative pipeline][2] that allowed us to manage experiment settings quickly, helped a lot but there are still a lot of room for improvement, especially in the area of experiment sharing, and postprocessing code.\n\n**Networks:** xception [256,512], NasNet (256) - only one model one fold\n**Augmentations:** Flips,Rot90\n**Test time augmentations:** Flip, Rot90\n**Loss functions:** BCE,Focal Loss\n**Learning:** Adam , LR Finder, CLR(triangular2)\n\nFail points:\n\n1. **Economics and merging**: after merging we gathered a lot of compute power, but all models except one, that was a part of our final submission was trained on my 2x1080 ti machine, so it seems that we was not able to utilize our compute effectively. My current feeling is that it is mostly result of later mergine (just few hours, before mergine dead line), so we was not able to align our thoughts quickly enough to make use of our unified compute and brain power.\n\n2. **Augmentations**: we have used very conservative augmentations, and obviously this was one of the pain points of out models. Next time I will use less conservative augmentations and more TTA.\n\n3. **Explotation vs Exploration** . Balancing this two things was critical, and I have a feeling that having multiple different branches of research after merger deadline does not allowed us to fully exploit potential of our best approach. Next time, I will try to switch to explotation a little bit earlier.\n\n\n [1]: https://github.com/petrochenko-pavel-a/proteins/blob/master/proteins.py \"Final Submission Code\"\n [2]: https://github.com/petrochenko-pavel-a/classification_training_pipeline \"Our Pipeline\"", "454026": "The pipeline could use some improvements in blending department." }, "source": "meta" }