{ "id": 265464, "title": "why the learning resizing one time works another doesn't?", "url": "/competitions/seti-breakthrough-listen/discussion/265464", "author_name": "dragon zhang", "post_date": "2021-08-15T22:00:46.441000", "votes": 2, "comment_count": 4, "views": 0, "content": "

I forked \" SETI - Learned Image Resizing\"

\n

when I tried some bigger model, the performance is not stable.

\n

some time, It gets good score, some time It got worse.

\n

nothing changed except model type. what is the problem?

", "messages": [ { "id": 1474125, "postDate": "2021-08-16T00:13:55.683Z", "content": "

I am facing the same issue but not only with resize learning. Depending on how I set up my training pipeline the networks won't learn anything. I think that maybe the signal is too weak when compared to the noise and this could cause the model to get stuck in flat local optima (i.e. predict near 0.5 always).

\n

I would love to hear more about this too.

", "rawMarkdown": "I am facing the same issue but not only with resize learning. Depending on how I set up my training pipeline the networks won't learn anything. I think that maybe the signal is too weak when compared to the noise and this could cause the model to get stuck in flat local optima (i.e. predict near 0.5 always).\n\nI would love to hear more about this too.", "votes": 4, "replies": [ { "id": 1474129, "postDate": "2021-08-16T00:20:27.597Z", "content": "

Tune / lower your lr.

", "rawMarkdown": "Tune / lower your lr." }, { "id": 1474707, "postDate": "2021-08-16T08:33:59.800Z", "content": "

Similar observation, when using learable resizer I found switch to bigger backbone model is not helpful, and use too much augmentation is not helpful(easily predict 0.5 always). My suspicion is also due to the signal is too weak, learnable resizer help to avoid information loss compare to normal resize(eg. albumentations.Resize), but limited the diversity of my model.

", "rawMarkdown": "Similar observation, when using learable resizer I found switch to bigger backbone model is not helpful, and use too much augmentation is not helpful(easily predict 0.5 always). My suspicion is also due to the signal is too weak, learnable resizer help to avoid information loss compare to normal resize(eg. albumentations.Resize), but limited the diversity of my model." }, { "id": 1475432, "postDate": "2021-08-16T16:10:15.553Z", "content": "

thx for your reply. however the ups and downs seems that some unstable factors exist.
\nsomeone points out that the T_max should be changed accordingly. I only searched that It is 1/2 of cosine period. however I don't know the formula.

", "rawMarkdown": "thx for your reply. however the ups and downs seems that some unstable factors exist.\nsomeone points out that the T_max should be changed accordingly. I only searched that It is 1/2 of cosine period. however I don't know the formula. " } ] }, { "id": 1474056, "postDate": "2021-08-15T22:00:46.443Z", "content": "

I forked \" SETI - Learned Image Resizing\"

\n

when I tried some bigger model, the performance is not stable.

\n

some time, It gets good score, some time It got worse.

\n

nothing changed except model type. what is the problem?

", "rawMarkdown": "I forked \" SETI - Learned Image Resizing\"\n\nwhen I tried some bigger model, the performance is not stable.\n\nsome time, It gets good score, some time It got worse.\n\nnothing changed except model type. what is the problem? ", "votes": 2 } ], "comments": [ { "id": 1474125, "author_name": "Adriano Passos", "author_url": "", "post_date": "2021-08-16T00:13:55.683000", "content": "

I am facing the same issue but not only with resize learning. Depending on how I set up my training pipeline the networks won't learn anything. I think that maybe the signal is too weak when compared to the noise and this could cause the model to get stuck in flat local optima (i.e. predict near 0.5 always).

\n

I would love to hear more about this too.

", "votes": 4, "replies": [ { "id": 1474129, "author_name": "عثمان", "author_url": "", "post_date": "2021-08-16T00:20:27.597000", "content": "

Tune / lower your lr.

", "votes": 0, "replies": [] }, { "id": 1474707, "author_name": "Hao", "author_url": "", "post_date": "2021-08-16T08:33:59.800000", "content": "

Similar observation, when using learable resizer I found switch to bigger backbone model is not helpful, and use too much augmentation is not helpful(easily predict 0.5 always). My suspicion is also due to the signal is too weak, learnable resizer help to avoid information loss compare to normal resize(eg. albumentations.Resize), but limited the diversity of my model.

", "votes": 0, "replies": [] }, { "id": 1475432, "author_name": "dragon zhang", "author_url": "", "post_date": "2021-08-16T16:10:15.553000", "content": "

thx for your reply. however the ups and downs seems that some unstable factors exist.
\nsomeone points out that the T_max should be changed accordingly. I only searched that It is 1/2 of cosine period. however I don't know the formula.

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1474125": "I am facing the same issue but not only with resize learning. Depending on how I set up my training pipeline the networks won't learn anything. I think that maybe the signal is too weak when compared to the noise and this could cause the model to get stuck in flat local optima (i.e. predict near 0.5 always).\n\nI would love to hear more about this too.", "1474056": "I forked \" SETI - Learned Image Resizing\"\n\nwhen I tried some bigger model, the performance is not stable.\n\nsome time, It gets good score, some time It got worse.\n\nnothing changed except model type. what is the problem? " } }