Looks most of us is using resnet here, and more layers of resnet theoretically would has no worse result than less layers, so does that mean by increasing from resnet18 -> 34 -> 50, …. would always improve the result?
\nBecause each training session can be quiet long, hope to get some suggestions before dive into really deeper resnets.
Looks most of us is using resnet here, and more layers of resnet theoretically would has no worse result than less layers, so does that mean by increasing from resnet18 -> 34 -> 50, …. would always improve the result?
\nBecause each training session can be quiet long, hope to get some suggestions before dive into really deeper resnets.
Hi, I don't think more layers necessarily mean better score. It depends on whether your model is overfitting / underfitting on given data. Since the competition data is quite large, I'd hope testing different model architectures on same subset of data (just change in model architecture like resnet18 and resnet34) without changing other hyper parameters (learning rate, epochs, loss function, optimisation etc) and comparing their validation scores would give a better idea.
", "rawMarkdown": "Hi, I don't think more layers necessarily mean better score. It depends on whether your model is overfitting / underfitting on given data. Since the competition data is quite large, I'd hope testing different model architectures on same subset of data (just change in model architecture like resnet18 and resnet34) without changing other hyper parameters (learning rate, epochs, loss function, optimisation etc) and comparing their validation scores would give a better idea.", "votes": null }, { "id": "1047857", "postDate": "10/13/2020 02:10:05", "content": "There is some study of resnets that suggests they are roughly similar to an ensemble of shallower nets, eg - https://openreview.net/pdf?id=Sk8csP5ex
\nSo in theory, you could widen the skip layer step to make the shallow nets deeper
", "rawMarkdown": "There is some study of resnets that suggests they are roughly similar to an ensemble of shallower nets, eg - https://openreview.net/pdf?id=Sk8csP5ex\n\nSo in theory, you could widen the skip layer step to make the shallow nets deeper", "votes": null } ], "comments": [ { "id": 1047792, "author_name": "suryajrrafl", "author_url": "", "post_date": "10/13/2020 00:19:46", "content": "Hi, I don't think more layers necessarily mean better score. It depends on whether your model is overfitting / underfitting on given data. Since the competition data is quite large, I'd hope testing different model architectures on same subset of data (just change in model architecture like resnet18 and resnet34) without changing other hyper parameters (learning rate, epochs, loss function, optimisation etc) and comparing their validation scores would give a better idea.
", "votes": null, "replies": [] }, { "id": 1047857, "author_name": "n3n77i", "author_url": "", "post_date": "10/13/2020 02:10:05", "content": "There is some study of resnets that suggests they are roughly similar to an ensemble of shallower nets, eg - https://openreview.net/pdf?id=Sk8csP5ex
\nSo in theory, you could widen the skip layer step to make the shallow nets deeper
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1046824": "Looks most of us is using resnet here, and more layers of resnet theoretically would has no worse result than less layers, so does that mean by increasing from resnet18 -> 34 -> 50, .... would always improve the result? \nBecause each training session can be quiet long, hope to get some suggestions before dive into really deeper resnets.", "1047792": "Hi, I don't think more layers necessarily mean better score. It depends on whether your model is overfitting / underfitting on given data. Since the competition data is quite large, I'd hope testing different model architectures on same subset of data (just change in model architecture like resnet18 and resnet34) without changing other hyper parameters (learning rate, epochs, loss function, optimisation etc) and comparing their validation scores would give a better idea.", "1047857": "There is some study of resnets that suggests they are roughly similar to an ensemble of shallower nets, eg - https://openreview.net/pdf?id=Sk8csP5ex\n\nSo in theory, you could widen the skip layer step to make the shallow nets deeper" }, "source": "meta" }