Hi all, as we round the corner for the end of the competition I was curious about correlation. Here is a few different results I have had along the way for single models only. I have also had higher CV with similar LB and lower CV with similar LB depending on strategy. This was with training on kidney 1 and validating on kidney 3. Curious so see if others have had better luck.
\nCV 0.872. LB 0.854
\nCV 0.844. LB 0.851
Hi all, as we round the corner for the end of the competition I was curious about correlation. Here is a few different results I have had along the way for single models only. I have also had higher CV with similar LB and lower CV with similar LB depending on strategy. This was with training on kidney 1 and validating on kidney 3. Curious so see if others have had better luck.
\nCV 0.872. LB 0.854
\nCV 0.844. LB 0.851
I have not been this lucky, with same train and validation set I can get CV 0.91+ and LB between 0.79-0.83+
\nI think the quality of annotation matters a lot… This will be one hell of a shake!
", "rawMarkdown": "I have not been this lucky, with same train and validation set I can get CV 0.91+ and LB between 0.79-0.83+\n\nI think the quality of annotation matters a lot... This will be one hell of a shake!", "votes": 3, "replies": [ { "id": 2613201, "postDate": "2024-01-21T22:15:43.963Z", "content": "I think shake will be dependent on the impact of higher resolution primarily. It may be a problem with the model selected or the weight decay?
", "rawMarkdown": "I think shake will be dependent on the impact of higher resolution primarily. It may be a problem with the model selected or the weight decay? ", "votes": 1, "replies": [ { "id": 2614195, "postDate": "2024-01-22T13:45:37.237Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 2614225, "postDate": "2024-01-22T14:00:15.870Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2616649, "postDate": "2024-01-23T18:43:16.790Z", "content": "@pkyangno1 can you please clarify what you mean by a sliding window? Thanks
", "rawMarkdown": "@pkyangno1 can you please clarify what you mean by a sliding window? Thanks" }, { "id": 2618594, "postDate": "2024-01-24T20:33:50.930Z", "content": "@mhmdsab, I believe he means tiling with overlap
", "rawMarkdown": "@mhmdsab, I believe he means tiling with overlap" } ] }, { "id": 2613807, "postDate": "2024-01-22T09:28:54.817Z", "content": "Same. CV is 0.88~0.9. But LB is stuck at ~0.8. Train on kidney1_dense and part of the slices of kidney2. Validate on kidney3_dense
", "rawMarkdown": "Same. CV is 0.88~0.9. But LB is stuck at ~0.8. Train on kidney1_dense and part of the slices of kidney2. Validate on kidney3_dense", "votes": 2, "replies": [ { "id": 2615766, "postDate": "2024-01-23T09:55:17.840Z", "content": "I only trained partial data of kid1/2 dense, 3 sparse, LB 0.78 . trained all kid1 dense, val on kid3 dense, by tuning percentile, LB 0.85.
\nThe only worry is about the private test data type.
\nIn general, training model on all data would be more stable on private test data.
", "rawMarkdown": "I only trained partial data of kid1/2 dense, 3 sparse, LB 0.78 . trained all kid1 dense, val on kid3 dense, by tuning percentile, LB 0.85.\n\nThe only worry is about the private test data type. \n\nIn general, training model on all data would be more stable on private test data. " } ] } ] }, { "id": 2615983, "postDate": "2024-01-23T12:42:13.380Z", "content": "training on kidney 1 and validating on kidney 3. unet resnext50
\ncv885 -> lb873
\ncv878 -> lb869
@ajobseeker what image size did you use on that model?
", "rawMarkdown": "@ajobseeker what image size did you use on that model?" } ] } ], "comments": [ { "id": 2613196, "author_name": "Optimo", "author_url": "", "post_date": "2024-01-21T22:07:20.327000", "content": "I have not been this lucky, with same train and validation set I can get CV 0.91+ and LB between 0.79-0.83+
\nI think the quality of annotation matters a lot… This will be one hell of a shake!
", "votes": 3, "replies": [ { "id": 2613201, "author_name": "Cody_Null", "author_url": "", "post_date": "2024-01-21T22:15:43.963000", "content": "I think shake will be dependent on the impact of higher resolution primarily. It may be a problem with the model selected or the weight decay?
", "votes": 1, "replies": [ { "id": 2614195, "author_name": "", "author_url": "", "post_date": "2024-01-22T13:45:37.237000", "content": "", "votes": 1, "replies": [] }, { "id": 2614225, "author_name": "", "author_url": "", "post_date": "2024-01-22T14:00:15.870000", "content": "", "votes": 0, "replies": [] }, { "id": 2616649, "author_name": "Mohammed Sabry", "author_url": "", "post_date": "2024-01-23T18:43:16.790000", "content": "@pkyangno1 can you please clarify what you mean by a sliding window? Thanks
", "votes": 0, "replies": [] }, { "id": 2618594, "author_name": "JEANMPIA", "author_url": "", "post_date": "2024-01-24T20:33:50.930000", "content": "@mhmdsab, I believe he means tiling with overlap
", "votes": 0, "replies": [] } ] }, { "id": 2613807, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-22T09:28:54.817000", "content": "Same. CV is 0.88~0.9. But LB is stuck at ~0.8. Train on kidney1_dense and part of the slices of kidney2. Validate on kidney3_dense
", "votes": 2, "replies": [ { "id": 2615766, "author_name": "dragon zhang", "author_url": "", "post_date": "2024-01-23T09:55:17.840000", "content": "I only trained partial data of kid1/2 dense, 3 sparse, LB 0.78 . trained all kid1 dense, val on kid3 dense, by tuning percentile, LB 0.85.
\nThe only worry is about the private test data type.
\nIn general, training model on all data would be more stable on private test data.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2615983, "author_name": "HB", "author_url": "", "post_date": "2024-01-23T12:42:13.380000", "content": "training on kidney 1 and validating on kidney 3. unet resnext50
\ncv885 -> lb873
\ncv878 -> lb869
@ajobseeker what image size did you use on that model?
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2613184": "Hi all, as we round the corner for the end of the competition I was curious about correlation. Here is a few different results I have had along the way for single models only. I have also had higher CV with similar LB and lower CV with similar LB depending on strategy. This was with training on kidney 1 and validating on kidney 3. Curious so see if others have had better luck. \n\nCV 0.872. LB 0.854\nCV 0.844. LB 0.851", "2613196": "I have not been this lucky, with same train and validation set I can get CV 0.91+ and LB between 0.79-0.83+\n\nI think the quality of annotation matters a lot... This will be one hell of a shake!", "2615983": "training on kidney 1 and validating on kidney 3. unet resnext50\ncv885 -> lb873\ncv878 -> lb869 " } }