{ "id": 38643, "title": "How to transfer improvements from 128x128 to 1024x1024?", "url": "/competitions/carvana-image-masking-challenge/discussion/38643", "author_name": "Anton M", "post_date": "2017-08-28T08:02:36.330000", "votes": 1, "comment_count": 6, "views": 0, "content": "

To lower the time for prototyping, I've been implementing new ideas on 128 net and then training good ideas on 1024. But all good 128 ideas don't work on 1024 .

\n\n

For example, some ideas increase the dice score of 128 starter model by ~0.03, but all these ideas perform worse than starter model on 1024.

\n\n

Is there any proper way how to scale performance from low res to high res?

", "messages": [ { "id": 216816, "postDate": "2017-08-28T08:56:34.527Z", "content": "
\n
\n

For example, some ideas increase the dice score of 128 starter model by ~0.03

\n
\n
\n\n

is there any concrete examples?

", "rawMarkdown": ">>For example, some ideas increase the dice score of 128 starter model by ~0.03\n\nis there any concrete examples?", "votes": 1, "replies": [ { "id": 216826, "postDate": "2017-08-28T09:57:12.413Z", "rawMarkdown": "" }, { "id": 216832, "postDate": "2017-08-28T10:07:35.267Z", "content": "

For example - bboxes. They perform better on 128 and 256 (LB 0.992 and 0.994) - but equal or worse on 512 and 1024 than starter.

\n\n

Or some Unet architecture tweaks, i.e. using a little bit more complicated modules instead of module = con2d, batchnorm, relu.

\n\n

Or additional data augmentations.

", "rawMarkdown": "For example - bboxes. They perform better on 128 and 256 (LB 0.992 and 0.994) - but equal or worse on 512 and 1024 than starter. \n\nOr some Unet architecture tweaks, i.e. using a little bit more complicated modules instead of module = con2d, batchnorm, relu. \n\nOr additional data augmentations. " }, { "id": 216854, "postDate": "2017-08-28T12:12:04.747Z", "content": "
\n
\n

For example - bboxes. ...

\n
\n
\n\n

referring to:https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/38298, dice score for 1024 is the same for full resolution.

\n\n
\n
\n

little bit more complicated modules instead

\n
\n
\n\n

I think you need a larger network for 1024. it is difficult to implement.

\n\n
\n
\n

Or additional data augmentations.

\n
\n
\n\n

data augmentations will increase data complexity which needs more parameters.

", "rawMarkdown": ">>For example - bboxes. ...\n\nreferring to:https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/38298, dice score for 1024 is the same for full resolution. \n\n\n>> little bit more complicated modules instead \n\nI think you need a larger network for 1024. it is difficult to implement. \n\n\n>>Or additional data augmentations.\n\ndata augmentations will increase data complexity which needs more parameters. \n", "votes": 1 }, { "id": 216971, "postDate": "2017-08-28T20:19:28.417Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 216805, "postDate": "2017-08-28T08:02:36.330Z", "content": "

To lower the time for prototyping, I've been implementing new ideas on 128 net and then training good ideas on 1024. But all good 128 ideas don't work on 1024 .

\n\n

For example, some ideas increase the dice score of 128 starter model by ~0.03, but all these ideas perform worse than starter model on 1024.

\n\n

Is there any proper way how to scale performance from low res to high res?

", "rawMarkdown": "To lower the time for prototyping, I've been implementing new ideas on 128 net and then training good ideas on 1024. But all good 128 ideas don't work on 1024 .\n\nFor example, some ideas increase the dice score of 128 starter model by ~0.03, but all these ideas perform worse than starter model on 1024.\n\nIs there any proper way how to scale performance from low res to high res? ", "votes": 1 }, { "id": 216948, "postDate": "2017-08-28T19:05:58.210Z", "content": "

On size 128, a 3x3 conv will just find other features than on size 1024. That's why it's difficult to compare results on these two very different datasets. For the very same reason, a good architecture for 128 might not be a good architecture for 1024.

", "rawMarkdown": "On size 128, a 3x3 conv will just find other features than on size 1024. That's why it's difficult to compare results on these two very different datasets. For the very same reason, a good architecture for 128 might not be a good architecture for 1024." } ], "comments": [ { "id": 216816, "author_name": "hengck23", "author_url": "", "post_date": "2017-08-28T08:56:34.527000", "content": "
\n
\n

For example, some ideas increase the dice score of 128 starter model by ~0.03

\n
\n
\n\n

is there any concrete examples?

", "votes": 1, "replies": [ { "id": 216826, "author_name": "Anton M", "author_url": "", "post_date": "2017-08-28T09:57:12.413000", "content": "", "votes": 0, "replies": [] }, { "id": 216832, "author_name": "Anton M", "author_url": "", "post_date": "2017-08-28T10:07:35.267000", "content": "

For example - bboxes. They perform better on 128 and 256 (LB 0.992 and 0.994) - but equal or worse on 512 and 1024 than starter.

\n\n

Or some Unet architecture tweaks, i.e. using a little bit more complicated modules instead of module = con2d, batchnorm, relu.

\n\n

Or additional data augmentations.

", "votes": 0, "replies": [] }, { "id": 216854, "author_name": "hengck23", "author_url": "", "post_date": "2017-08-28T12:12:04.747000", "content": "
\n
\n

For example - bboxes. ...

\n
\n
\n\n

referring to:https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/38298, dice score for 1024 is the same for full resolution.

\n\n
\n
\n

little bit more complicated modules instead

\n
\n
\n\n

I think you need a larger network for 1024. it is difficult to implement.

\n\n
\n
\n

Or additional data augmentations.

\n
\n
\n\n

data augmentations will increase data complexity which needs more parameters.

", "votes": 1, "replies": [] }, { "id": 216971, "author_name": "", "author_url": "", "post_date": "2017-08-28T20:19:28.417000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 216948, "author_name": "Markus", "author_url": "", "post_date": "2017-08-28T19:05:58.210000", "content": "

On size 128, a 3x3 conv will just find other features than on size 1024. That's why it's difficult to compare results on these two very different datasets. For the very same reason, a good architecture for 128 might not be a good architecture for 1024.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "216816": ">>For example, some ideas increase the dice score of 128 starter model by ~0.03\n\nis there any concrete examples?", "216805": "To lower the time for prototyping, I've been implementing new ideas on 128 net and then training good ideas on 1024. But all good 128 ideas don't work on 1024 .\n\nFor example, some ideas increase the dice score of 128 starter model by ~0.03, but all these ideas perform worse than starter model on 1024.\n\nIs there any proper way how to scale performance from low res to high res? ", "216948": "On size 128, a 3x3 conv will just find other features than on size 1024. That's why it's difficult to compare results on these two very different datasets. For the very same reason, a good architecture for 128 might not be a good architecture for 1024." } }