I was initially using 224x224 images with a resnet50 backbone and topping at around 0.400 in the lb. I started using the 512x512 images with some changes to my network to adjust for the image size and used the weights from the smaller images and got a score of 0.450, a 0.05 point jump.
\n\nWhat's been your jump? I also want to know the jump in score in case you're using the full sized images. I'm contemplating using them, but I want to be sure before I download a ludicrous 250GB archive.
", "messages": [ { "id": "422339", "postDate": "11/16/2018 04:56:56", "content": "I was initially using 224x224 images with a resnet50 backbone and topping at around 0.400 in the lb. I started using the 512x512 images with some changes to my network to adjust for the image size and used the weights from the smaller images and got a score of 0.450, a 0.05 point jump.
\n\nWhat's been your jump? I also want to know the jump in score in case you're using the full sized images. I'm contemplating using them, but I want to be sure before I download a ludicrous 250GB archive.
", "rawMarkdown": "I was initially using 224x224 images with a resnet50 backbone and topping at around 0.400 in the lb. I started using the 512x512 images with some changes to my network to adjust for the image size and used the weights from the smaller images and got a score of 0.450, a 0.05 point jump.\n\nWhat's been your jump? I also want to know the jump in score in case you're using the full sized images. I'm contemplating using them, but I want to be sure before I download a ludicrous 250GB archive.", "votes": null }, { "id": "422382", "postDate": "11/16/2018 06:28:50", "content": "So far my best comes from my 512x512 models. Going to be days still for the larger models, they are taking a long time to train.
", "rawMarkdown": "So far my best comes from my 512x512 models. Going to be days still for the larger models, they are taking a long time to train.", "votes": null }, { "id": "422547", "postDate": "11/16/2018 11:45:41", "content": "With 224 by 224 and RestNet50, I also got around 0.400, when I increased to 512 by 512, the best result is 0.498. So I think the resolution is really important. Now I am gonna try full size, 2048 by 2048
", "rawMarkdown": "With 224 by 224 and RestNet50, I also got around 0.400, when I increased to 512 by 512, the best result is 0.498. So I think the resolution is really important. Now I am gonna try full size, 2048 by 2048", "votes": null }, { "id": "422581", "postDate": "11/16/2018 12:50:59", "content": "I'm optimizing my models at 224px and was planning to up to 512px. In the \"real world\" I rarely see huge performance gains from resolution increases past a certain point, but this is largely because the classes are quite large objects, or clusters of small objects.
\n\nFor small cell components the fine details may prove essential for success. Some of the classes requested here are quite small. At some point increasing the pixel resolution will make no difference as you reach optical limits (or you are looking a big group of labels in one place), and your cluster of fluorescent labels stuck to your target protein is likely larger than the target protein itself anyway. I am ASSUMING they are imaged at 40X/1.25NA(???).
\n\nI do think 2048px should still provide more information. This may be helpful to some: http://www.bio.brandeis.edu/CIL/Applications%20notes/Leica%20tutorials/Zoom%20and%20Resolution.pdf
\n\nI'm curious what hardware people are using for 2048px images? I have access to a single GTX 1080, and I am hoping this will be enough. With augmentation and a deep network I expect it to be like swimming through concrete...
", "rawMarkdown": "I'm optimizing my models at 224px and was planning to up to 512px. In the \"real world\" I rarely see huge performance gains from resolution increases past a certain point, but this is largely because the classes are quite large objects, or clusters of small objects.\n\nFor small cell components the fine details may prove essential for success. Some of the classes requested here are quite small. At some point increasing the pixel resolution will make no difference as you reach optical limits (or you are looking a big group of labels in one place), and your cluster of fluorescent labels stuck to your target protein is likely larger than the target protein itself anyway. I am ASSUMING they are imaged at 40X/1.25NA(???).\n\nI do think 2048px should still provide more information. This may be helpful to some: http://www.bio.brandeis.edu/CIL/Applications%20notes/Leica%20tutorials/Zoom%20and%20Resolution.pdf\n\nI'm curious what hardware people are using for 2048px images? I have access to a single GTX 1080, and I am hoping this will be enough. With augmentation and a deep network I expect it to be like swimming through concrete...", "votes": null }, { "id": "422668", "postDate": "11/16/2018 15:43:44", "content": "I don't think a 1080 has enough memory. I am using a Titan X Maxwell and with 12gb ram and 2048px is difficult. With only 8gb, I'd think 1024 would be the upper limit.
", "rawMarkdown": "I don't think a 1080 has enough memory. I am using a Titan X Maxwell and with 12gb ram and 2048px is difficult. With only 8gb, I'd think 1024 would be the upper limit.", "votes": null }, { "id": "422670", "postDate": "11/16/2018 15:45:01", "content": "I'm not even trying 1024 anymore, my largest model in testing use 768px.
", "rawMarkdown": "I'm not even trying 1024 anymore, my largest model in testing use 768px.", "votes": null }, { "id": "422677", "postDate": "11/16/2018 16:01:12", "content": "Indeed, to be able to use 2048px images, I think 32 GB memory is needed.
", "rawMarkdown": "Indeed, to be able to use 2048px images, I think 32 GB memory is needed.", "votes": null }, { "id": "422716", "postDate": "11/16/2018 17:01:20", "content": "Interesting thread. I can run 512 with batch size 32 but not 64, on my Ti1080 (12 GB). But I do not have a reasonable calculation that shows why this is so. Can anyone enlighten me?
", "rawMarkdown": "Interesting thread. I can run 512 with batch size 32 but not 64, on my Ti1080 (12 GB). But I do not have a reasonable calculation that shows why this is so. Can anyone enlighten me?", "votes": null }, { "id": "422745", "postDate": "11/16/2018 17:57:28", "content": "Thanks for your inputs everyone. BTW I have a gtx1070 8GB. Before I started downloading the full sized images, I wanted to make sure that I could get some dummy 2048 sized images through my network but it appears that I get \"Out of Memory\" issues. So hardware wise, it appears Im tapped out for that size at least.
", "rawMarkdown": "Thanks for your inputs everyone. BTW I have a gtx1070 8GB. Before I started downloading the full sized images, I wanted to make sure that I could get some dummy 2048 sized images through my network but it appears that I get \"Out of Memory\" issues. So hardware wise, it appears Im tapped out for that size at least.", "votes": null }, { "id": "422747", "postDate": "11/16/2018 18:04:09", "content": "I'm guessing that at that image size, batch size and network parameters, all the weights matrices that the network constructs is what maxes out the memory. When I got OOM (Out of memory) issues in Keras/Tensorflow, it would usually say something like \"resource exhausted when allocating tensor of shape [4, 256, 256, 512]\".
", "rawMarkdown": "I'm guessing that at that image size, batch size and network parameters, all the weights matrices that the network constructs is what maxes out the memory. When I got OOM (Out of memory) issues in Keras/Tensorflow, it would usually say something like \"resource exhausted when allocating tensor of shape [4, 256, 256, 512]\".", "votes": null }, { "id": "422780", "postDate": "11/16/2018 19:11:45", "content": "About 0.03 for me, going from 256 to 512. But I expect diminished returns when you keep going up, which I don't have the resources for anyway.
", "rawMarkdown": "About 0.03 for me, going from 256 to 512. But I expect diminished returns when you keep going up, which I don't have the resources for anyway.", "votes": null }, { "id": "423047", "postDate": "11/17/2018 10:38:17", "content": "I can train at 1024px with a batch size of 32 on a single 1080 ti and it takes about 20 mins per epoch. Currently it's no better than at 512 and is far too slow to iterate so I'm going to abandon for now, sick of memory issues.
\n\nI really think one can get far in this competition by ignoring the larger images and spending more time on CV design to address class imbalance.
", "rawMarkdown": "I can train at 1024px with a batch size of 32 on a single 1080 ti and it takes about 20 mins per epoch. Currently it's no better than at 512 and is far too slow to iterate so I'm going to abandon for now, sick of memory issues. \n\nI really think one can get far in this competition by ignoring the larger images and spending more time on CV design to address class imbalance.", "votes": null }, { "id": "423101", "postDate": "11/17/2018 13:28:11", "content": "Good to know, thanks. I think I'll stick to 512. I have a Tesla at work with more memory but I don't think it's worth it.
", "rawMarkdown": "Good to know, thanks. I think I'll stick to 512. I have a Tesla at work with more memory but I don't think it's worth it.", "votes": null }, { "id": "424910", "postDate": "11/20/2018 21:44:42", "content": "256->512 LB 0.454 -> 0.49, all other things equal
", "rawMarkdown": "256->512 LB 0.454 -> 0.49, all other things equal", "votes": null }, { "id": "425373", "postDate": "11/21/2018 14:04:45", "content": "I see an improvement of about +0.04 when going from 256 -> 512\nI am currently limited by the memory of my 1080 Ti and I can hardly increase the size any more
", "rawMarkdown": "I see an improvement of about +0.04 when going from 256 -> 512\nI am currently limited by the memory of my 1080 Ti and I can hardly increase the size any more", "votes": null }, { "id": "432706", "postDate": "12/04/2018 08:18:32", "content": "Hi, what loss function do you use?
", "rawMarkdown": "Hi, what loss function do you use?", "votes": null }, { "id": "433059", "postDate": "12/04/2018 16:06:11", "content": "I used BCE loss function.
", "rawMarkdown": "I used BCE loss function.", "votes": null }, { "id": "433486", "postDate": "12/05/2018 05:55:55", "content": "", "rawMarkdown": "", "votes": null }, { "id": "433488", "postDate": "12/05/2018 05:57:24", "content": "hi, have you tried focal loss, does it give better result than bce loss ? In my case, focal loss does not help much.
\n\nemm, I found it in https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/70365 thanks.
\n\n\n", "rawMarkdown": "hi, have you tried focal loss, does it give better result than bce loss ? In my case, focal loss does not help much.\n\n\nemm, I found it in https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/70365 thanks.\n> **Zhijian Li wrote**\n> \n> > I used BCE loss function.", "votes": null } ], "comments": [ { "id": 422382, "author_name": "ldm314", "author_url": "", "post_date": "11/16/2018 06:28:50", "content": "Zhijian Li wrote
\n \n\n\nI used BCE loss function.
\n
So far my best comes from my 512x512 models. Going to be days still for the larger models, they are taking a long time to train.
", "votes": null, "replies": [] }, { "id": 422547, "author_name": "zhijianli", "author_url": "", "post_date": "11/16/2018 11:45:41", "content": "With 224 by 224 and RestNet50, I also got around 0.400, when I increased to 512 by 512, the best result is 0.498. So I think the resolution is really important. Now I am gonna try full size, 2048 by 2048
", "votes": null, "replies": [ { "id": 432706, "author_name": "yiwei2016", "author_url": "", "post_date": "12/04/2018 08:18:32", "content": "Hi, what loss function do you use?
", "votes": null, "replies": [] }, { "id": 433059, "author_name": "zhijianli", "author_url": "", "post_date": "12/04/2018 16:06:11", "content": "I used BCE loss function.
", "votes": null, "replies": [] }, { "id": 433486, "author_name": "zjucor", "author_url": "", "post_date": "12/05/2018 05:55:55", "content": "", "votes": null, "replies": [] }, { "id": 433488, "author_name": "zjucor", "author_url": "", "post_date": "12/05/2018 05:57:24", "content": "hi, have you tried focal loss, does it give better result than bce loss ? In my case, focal loss does not help much.
\n\nemm, I found it in https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/70365 thanks.
\n\n\n", "votes": null, "replies": [] } ] }, { "id": 422581, "author_name": "dstjhb", "author_url": "", "post_date": "11/16/2018 12:50:59", "content": "Zhijian Li wrote
\n \n\n\nI used BCE loss function.
\n
I'm optimizing my models at 224px and was planning to up to 512px. In the \"real world\" I rarely see huge performance gains from resolution increases past a certain point, but this is largely because the classes are quite large objects, or clusters of small objects.
\n\nFor small cell components the fine details may prove essential for success. Some of the classes requested here are quite small. At some point increasing the pixel resolution will make no difference as you reach optical limits (or you are looking a big group of labels in one place), and your cluster of fluorescent labels stuck to your target protein is likely larger than the target protein itself anyway. I am ASSUMING they are imaged at 40X/1.25NA(???).
\n\nI do think 2048px should still provide more information. This may be helpful to some: http://www.bio.brandeis.edu/CIL/Applications%20notes/Leica%20tutorials/Zoom%20and%20Resolution.pdf
\n\nI'm curious what hardware people are using for 2048px images? I have access to a single GTX 1080, and I am hoping this will be enough. With augmentation and a deep network I expect it to be like swimming through concrete...
", "votes": null, "replies": [ { "id": 422668, "author_name": "ldm314", "author_url": "", "post_date": "11/16/2018 15:43:44", "content": "I don't think a 1080 has enough memory. I am using a Titan X Maxwell and with 12gb ram and 2048px is difficult. With only 8gb, I'd think 1024 would be the upper limit.
", "votes": null, "replies": [] }, { "id": 422670, "author_name": "ldm314", "author_url": "", "post_date": "11/16/2018 15:45:01", "content": "I'm not even trying 1024 anymore, my largest model in testing use 768px.
", "votes": null, "replies": [] }, { "id": 422677, "author_name": "zhijianli", "author_url": "", "post_date": "11/16/2018 16:01:12", "content": "Indeed, to be able to use 2048px images, I think 32 GB memory is needed.
", "votes": null, "replies": [] }, { "id": 423047, "author_name": "maw501", "author_url": "", "post_date": "11/17/2018 10:38:17", "content": "I can train at 1024px with a batch size of 32 on a single 1080 ti and it takes about 20 mins per epoch. Currently it's no better than at 512 and is far too slow to iterate so I'm going to abandon for now, sick of memory issues.
\n\nI really think one can get far in this competition by ignoring the larger images and spending more time on CV design to address class imbalance.
", "votes": null, "replies": [] }, { "id": 423101, "author_name": "dstjhb", "author_url": "", "post_date": "11/17/2018 13:28:11", "content": "Good to know, thanks. I think I'll stick to 512. I have a Tesla at work with more memory but I don't think it's worth it.
", "votes": null, "replies": [] } ] }, { "id": 422716, "author_name": "petewills", "author_url": "", "post_date": "11/16/2018 17:01:20", "content": "Interesting thread. I can run 512 with batch size 32 but not 64, on my Ti1080 (12 GB). But I do not have a reasonable calculation that shows why this is so. Can anyone enlighten me?
", "votes": null, "replies": [ { "id": 422747, "author_name": "varunvprabhu", "author_url": "", "post_date": "11/16/2018 18:04:09", "content": "I'm guessing that at that image size, batch size and network parameters, all the weights matrices that the network constructs is what maxes out the memory. When I got OOM (Out of memory) issues in Keras/Tensorflow, it would usually say something like \"resource exhausted when allocating tensor of shape [4, 256, 256, 512]\".
", "votes": null, "replies": [] } ] }, { "id": 422745, "author_name": "varunvprabhu", "author_url": "", "post_date": "11/16/2018 17:57:28", "content": "Thanks for your inputs everyone. BTW I have a gtx1070 8GB. Before I started downloading the full sized images, I wanted to make sure that I could get some dummy 2048 sized images through my network but it appears that I get \"Out of Memory\" issues. So hardware wise, it appears Im tapped out for that size at least.
", "votes": null, "replies": [] }, { "id": 422780, "author_name": "suicaokhoailang", "author_url": "", "post_date": "11/16/2018 19:11:45", "content": "About 0.03 for me, going from 256 to 512. But I expect diminished returns when you keep going up, which I don't have the resources for anyway.
", "votes": null, "replies": [] }, { "id": 424910, "author_name": "thundo", "author_url": "", "post_date": "11/20/2018 21:44:42", "content": "256->512 LB 0.454 -> 0.49, all other things equal
", "votes": null, "replies": [] }, { "id": 425373, "author_name": "stecasasso", "author_url": "", "post_date": "11/21/2018 14:04:45", "content": "I see an improvement of about +0.04 when going from 256 -> 512\nI am currently limited by the memory of my 1080 Ti and I can hardly increase the size any more
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "422339": "I was initially using 224x224 images with a resnet50 backbone and topping at around 0.400 in the lb. I started using the 512x512 images with some changes to my network to adjust for the image size and used the weights from the smaller images and got a score of 0.450, a 0.05 point jump.\n\nWhat's been your jump? I also want to know the jump in score in case you're using the full sized images. I'm contemplating using them, but I want to be sure before I download a ludicrous 250GB archive.", "422382": "So far my best comes from my 512x512 models. Going to be days still for the larger models, they are taking a long time to train.", "422547": "With 224 by 224 and RestNet50, I also got around 0.400, when I increased to 512 by 512, the best result is 0.498. So I think the resolution is really important. Now I am gonna try full size, 2048 by 2048", "422581": "I'm optimizing my models at 224px and was planning to up to 512px. In the \"real world\" I rarely see huge performance gains from resolution increases past a certain point, but this is largely because the classes are quite large objects, or clusters of small objects.\n\nFor small cell components the fine details may prove essential for success. Some of the classes requested here are quite small. At some point increasing the pixel resolution will make no difference as you reach optical limits (or you are looking a big group of labels in one place), and your cluster of fluorescent labels stuck to your target protein is likely larger than the target protein itself anyway. I am ASSUMING they are imaged at 40X/1.25NA(???).\n\nI do think 2048px should still provide more information. This may be helpful to some: http://www.bio.brandeis.edu/CIL/Applications%20notes/Leica%20tutorials/Zoom%20and%20Resolution.pdf\n\nI'm curious what hardware people are using for 2048px images? I have access to a single GTX 1080, and I am hoping this will be enough. With augmentation and a deep network I expect it to be like swimming through concrete...", "422668": "I don't think a 1080 has enough memory. I am using a Titan X Maxwell and with 12gb ram and 2048px is difficult. With only 8gb, I'd think 1024 would be the upper limit.", "422670": "I'm not even trying 1024 anymore, my largest model in testing use 768px.", "422677": "Indeed, to be able to use 2048px images, I think 32 GB memory is needed.", "422716": "Interesting thread. I can run 512 with batch size 32 but not 64, on my Ti1080 (12 GB). But I do not have a reasonable calculation that shows why this is so. Can anyone enlighten me?", "422745": "Thanks for your inputs everyone. BTW I have a gtx1070 8GB. Before I started downloading the full sized images, I wanted to make sure that I could get some dummy 2048 sized images through my network but it appears that I get \"Out of Memory\" issues. So hardware wise, it appears Im tapped out for that size at least.", "422747": "I'm guessing that at that image size, batch size and network parameters, all the weights matrices that the network constructs is what maxes out the memory. When I got OOM (Out of memory) issues in Keras/Tensorflow, it would usually say something like \"resource exhausted when allocating tensor of shape [4, 256, 256, 512]\".", "422780": "About 0.03 for me, going from 256 to 512. But I expect diminished returns when you keep going up, which I don't have the resources for anyway.", "423047": "I can train at 1024px with a batch size of 32 on a single 1080 ti and it takes about 20 mins per epoch. Currently it's no better than at 512 and is far too slow to iterate so I'm going to abandon for now, sick of memory issues. \n\nI really think one can get far in this competition by ignoring the larger images and spending more time on CV design to address class imbalance.", "423101": "Good to know, thanks. I think I'll stick to 512. I have a Tesla at work with more memory but I don't think it's worth it.", "424910": "256->512 LB 0.454 -> 0.49, all other things equal", "425373": "I see an improvement of about +0.04 when going from 256 -> 512\nI am currently limited by the memory of my 1080 Ti and I can hardly increase the size any more", "432706": "Hi, what loss function do you use?", "433059": "I used BCE loss function.", "433486": "", "433488": "hi, have you tried focal loss, does it give better result than bce loss ? In my case, focal loss does not help much.\n\n\nemm, I found it in https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/70365 thanks.\n> **Zhijian Li wrote**\n> \n> > I used BCE loss function." }, "source": "meta" }