{ "id": 230940, "title": "[LB 0.354 to 0.424] Add multi-label classification to the original image(green) ", "url": "/competitions/hpa-single-cell-image-classification/discussion/230940", "author_name": "Alien", "post_date": "2021-04-06T08:16:36.887000", "votes": 40, "comment_count": 39, "views": 0, "content": "

Create label:
\nClassification label csv[green]\n
\nTraining:
\n[HPA] classification efnb7 train\n
\nInference:
\n0.354+efnb7 Classification weights 0.4 0.6

\n

I have experience in vinbigdata competitions. In the vinbigdata competition, I do multi-label classification on original images and then weighted average confidence values. In this competition, I used the same method to improve my score. I use this notebook add multi-label classification. Thanks to @dschettler8845

\n

Why do I use image green for training? In fact, I originally expected a different label for each color.
\nLike the following:
\nred: [1, 3, 5, 7, 9]
\ngreen:[0, 2, 4, 6, 8]
\nblue:[11, 13, 15, 17]
\nyellow:[10, 12, 14, 16, 18]

\n

I use efficientnet-b0 to train each label individually, and I find that the AUC other than green is too low. So in the end I used efficientnet-b7 to train all the labels in green images.

\n

Overfitting or no overfitting?

\n

Thresholds of different weights will have different results and vary greatly. Different models (detection) do a weighted average of confidence values, and LB scores vary greatly. It is possible to find a threshold with the highest LB score, but it is usually not the best threshold.

\n

The following are my LB scores with different weights:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
LB scoreweightsclassification weights
0.3760.90.1
0.3960.80.2
0.4080.70.3
0.4170.60.4
0.4230.50.5
0.4240.40.6
", "messages": [ { "id": 1264524, "postDate": "2021-04-06T08:16:36.887Z", "content": "

Create label:
\nClassification label csv[green]\n
\nTraining:
\n[HPA] classification efnb7 train\n
\nInference:
\n0.354+efnb7 Classification weights 0.4 0.6

\n

I have experience in vinbigdata competitions. In the vinbigdata competition, I do multi-label classification on original images and then weighted average confidence values. In this competition, I used the same method to improve my score. I use this notebook add multi-label classification. Thanks to @dschettler8845

\n

Why do I use image green for training? In fact, I originally expected a different label for each color.
\nLike the following:
\nred: [1, 3, 5, 7, 9]
\ngreen:[0, 2, 4, 6, 8]
\nblue:[11, 13, 15, 17]
\nyellow:[10, 12, 14, 16, 18]

\n

I use efficientnet-b0 to train each label individually, and I find that the AUC other than green is too low. So in the end I used efficientnet-b7 to train all the labels in green images.

\n

Overfitting or no overfitting?

\n

Thresholds of different weights will have different results and vary greatly. Different models (detection) do a weighted average of confidence values, and LB scores vary greatly. It is possible to find a threshold with the highest LB score, but it is usually not the best threshold.

\n

The following are my LB scores with different weights:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
LB scoreweightsclassification weights
0.3760.90.1
0.3960.80.2
0.4080.70.3
0.4170.60.4
0.4230.50.5
0.4240.40.6
", "rawMarkdown": "Create label:\n[Classification label csv[green]\n](https://www.kaggle.com/h053473666/classification-label-csv-green)\nTraining:\n[[HPA] classification efnb7 train\n](https://www.kaggle.com/h053473666/hpa-classification-efnb7-train)\nInference:\n[0.354+efnb7 Classification weights 0.4 0.6](https://www.kaggle.com/h053473666/0-354-efnb7-classification-weights-0-4-0-6)\n\nI have experience in [vinbigdata ](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/overview)competitions. In the vinbigdata competition, I do multi-label classification on original images and then weighted average confidence values. In this competition, I used the same method to improve my score. I use this [notebook ](https://www.kaggle.com/dschettler8845/hpa-cellwise-classification-inference)add multi-label classification. Thanks to @dschettler8845 \n\nWhy do I use image green for training? In fact, I originally expected a different label for each color.\nLike the following:\nred: [1, 3, 5, 7, 9]\ngreen:[0, 2, 4, 6, 8]\nblue:[11, 13, 15, 17]\nyellow:[10, 12, 14, 16, 18]\n\nI use efficientnet-b0 to train each label individually, and I find that the AUC other than green is too low. So in the end I used efficientnet-b7 to train all the labels in green images.\n\nOverfitting or no overfitting?\n\nThresholds of different weights will have different results and vary greatly. Different models (detection) do a weighted average of confidence values, and LB scores vary greatly. It is possible to find a threshold with the highest LB score, but it is usually not the best threshold.\n\nThe following are my LB scores with different weights:\nLB score | weights | classification weights \n--------------|:-----:|-----\n0.376 |0.9 | 0.1\n0.396 | 0.8 | 0.2\n0.408 | 0.7 | 0.3\n0.417 |0.6 | 0.4\n0.423| 0.5 | 0.5\n0.424 | 0.4 | 0.6\n", "votes": 40 }, { "id": 1265684, "postDate": "2021-04-07T06:06:08.190Z", "content": "

It's amazing how fast this is implemented by everyone 😄

", "rawMarkdown": "It's amazing how fast this is implemented by everyone 😄", "votes": 5, "replies": [ { "id": 1266139, "postDate": "2021-04-07T14:16:14.563Z", "content": "

I think most people just copied Aliens notebook. Because 50 teams score between 0.424 und 0.427

", "rawMarkdown": "I think most people just copied Aliens notebook. Because 50 teams score between 0.424 und 0.427", "votes": 4 } ] }, { "id": 1266496, "postDate": "2021-04-07T19:58:58.763Z", "content": "

Thank you. I had not considered a multi-tiered ensemble.

", "rawMarkdown": "Thank you. I had not considered a multi-tiered ensemble. ", "votes": 3 }, { "id": 1265829, "postDate": "2021-04-07T08:40:21.583Z", "content": "

It is very interesting how you can combine image-level and cell-level prediction to improve your score. Thank you very much. I never thought Bayesian statistics could help that much.

", "rawMarkdown": "It is very interesting how you can combine image-level and cell-level prediction to improve your score. Thank you very much. I never thought Bayesian statistics could help that much.", "votes": 3 }, { "id": 1265326, "postDate": "2021-04-06T19:48:15.110Z", "content": "

thanks for sharing this approach abt only using Green chennel.
\nto me it looks like the model is trying to find the clustering pattern of the protein and make a prediction. like the proteins will appear in certain formation if they are within nucleus,etc. but not sure if this is a good approach for the end goal of the competition where they want to segment cells based on the protein location within the cells.

", "rawMarkdown": "thanks for sharing this approach abt only using Green chennel. \nto me it looks like the model is trying to find the clustering pattern of the protein and make a prediction. like the proteins will appear in certain formation if they are within nucleus,etc. but not sure if this is a good approach for the end goal of the competition where they want to segment cells based on the protein location within the cells. ", "votes": 3 }, { "id": 1264894, "postDate": "2021-04-06T13:43:24.213Z", "content": "

Hey @h053473666 (Alien)! Glad to see you had so much success. I also have been experimenting with ensembling slide and tile level predictions. I had a question regarding the last table. Are you saying that you apply the weighting/combination as follows…

\n
\n

0.4 Weight for Cell Labels * 0.6 Weight for Slide Label = 0.424 on LB?

\n

i.e. Can you check my understanding given the code below?

\n
single_tile_pred = [0.1 0.1 0.1 ... 0.9 0.1]\nslide_level_pred = [0.01 0.01 0.01 ... 0.5 0.2]\n\n# combine labels -- [0.0001, 0.0001, 0.0001 ... 0.45 0.02]\ncombined_preds = slide_tile_pred*0.4+slide_level_pred*0.6\n
\n

I'm dumb. This was clarified by Alex Riedel below.

\n
\n

Thanks in advance and Congratulations!

", "rawMarkdown": "Hey @h053473666 (Alien)! Glad to see you had so much success. I also have been experimenting with ensembling slide and tile level predictions. I had a question regarding the last table. Are you saying that you apply the weighting/combination as follows...\n\n---\n\n0.4 Weight for Cell Labels * 0.6 Weight for Slide Label = 0.424 on LB?\n\ni.e. Can you check my understanding given the code below?\n\n```\nsingle_tile_pred = [0.1 0.1 0.1 ... 0.9 0.1]\nslide_level_pred = [0.01 0.01 0.01 ... 0.5 0.2]\n\n# combine labels -- [0.0001, 0.0001, 0.0001 ... 0.45 0.02]\ncombined_preds = slide_tile_pred*0.4+slide_level_pred*0.6\n```\n\n~~If this is the case wouldn't it artificially lower the confidence of the predictions? i.e. 0.99*0.95=0.94 which is lower than either prediction.~~ I'm dumb. This was clarified by Alex Riedel below.\n\n---\n\nThanks in advance and Congratulations!", "votes": 3, "replies": [ { "id": 1264899, "postDate": "2021-04-06T13:49:08.697Z", "content": "

No, if you have 0.99 on tile-level and 0.95 on image-level it's going to be 0.4*0.99 + 0.6*0.95 = 0.966

", "rawMarkdown": "No, if you have 0.99 on tile-level and 0.95 on image-level it's going to be `0.4*0.99 + 0.6*0.95 = 0.966`", "votes": 2 }, { "id": 1264904, "postDate": "2021-04-06T13:51:20.927Z", "content": "

oh right. duh

", "rawMarkdown": "oh right. duh", "votes": 1 }, { "id": 1264991, "postDate": "2021-04-06T14:39:47.533Z", "content": "

Multiplication is not intuitive to me. But multiplication is indeed a feasible method. However, there are many factors to consider. The weighted average is more versatile. Who are interested in multiplication can refer to @cdeotte method. Although it can't be used directly, it is very good in the post-processing.https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637

", "rawMarkdown": "Multiplication is not intuitive to me. But multiplication is indeed a feasible method. However, there are many factors to consider. The weighted average is more versatile. Who are interested in multiplication can refer to @cdeotte method. Although it can't be used directly, it is very good in the post-processing.[https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)", "votes": 1 } ] }, { "id": 1264874, "postDate": "2021-04-06T13:32:45.810Z", "content": "

Training model on green channel only - wow, it's crazy that this works, this means that green channel may have enough signal to determine the class… thanks for sharing!

", "rawMarkdown": "Training model on green channel only - wow, it's crazy that this works, this means that green channel may have enough signal to determine the class... thanks for sharing!", "votes": 3, "replies": [ { "id": 1264903, "postDate": "2021-04-06T13:50:56.277Z", "content": "

This is also a misunderstanding that people often encounter. Humans and machines don't pay attention to the same things when they look at pictures. I think other methods should be used to enhance (nucleus, microtubules, endoplasmic reticulum) features, and the methods they use should break some features.

", "rawMarkdown": "This is also a misunderstanding that people often encounter. Humans and machines don't pay attention to the same things when they look at pictures. I think other methods should be used to enhance (nucleus, microtubules, endoplasmic reticulum) features, and the methods they use should break some features.", "votes": 4 }, { "id": 1264956, "postDate": "2021-04-06T14:17:15.950Z", "content": "

Have you tried training your network on all channels? Maybe it's giving even better predictions?

", "rawMarkdown": "Have you tried training your network on all channels? Maybe it's giving even better predictions?", "votes": 2 }, { "id": 1264993, "postDate": "2021-04-06T14:42:06.560Z", "content": "

I'm not sure if it will be better, but my instinct is that it will increase noise.

", "rawMarkdown": "I'm not sure if it will be better, but my instinct is that it will increase noise." }, { "id": 1265003, "postDate": "2021-04-06T14:47:10.937Z", "content": "

Described in this linkhttps://www.kaggle.com/c/hpa-single-cell-image-classification/data.
\nAll image samples are represented by four filters (stored as individual files), the protein of interest (green) plus three cellular landmarks: nucleus (blue), microtubules (red), endoplasmic reticulum (yellow). The green filter should hence be used to predict the label, and the other filters are used as references.

", "rawMarkdown": "Described in this link[https://www.kaggle.com/c/hpa-single-cell-image-classification/data](https://www.kaggle.com/c/hpa-single-cell-image-classification/data).\nAll image samples are represented by four filters (stored as individual files), the protein of interest (green) plus three cellular landmarks: nucleus (blue), microtubules (red), endoplasmic reticulum (yellow). The green filter should hence be used to predict the label, and the other filters are used as references." }, { "id": 1265008, "postDate": "2021-04-06T14:50:21.763Z", "content": "

Yeah actually you're right but I really think no one here had the courage to train only on the green channel 😄

", "rawMarkdown": "Yeah actually you're right but I really think no one here had the courage to train only on the green channel 😄", "votes": 3 } ] }, { "id": 1290153, "postDate": "2021-05-01T17:06:19.240Z", "content": "

Hi @h053473666

\n

Great Notebook. But does this work on the private LB? When I change only_public to False, the notebook exceeds the time limit. Can you give me some insight about it?

", "rawMarkdown": "Hi @h053473666 \n\nGreat Notebook. But does this work on the private LB? When I change only_public to False, the notebook exceeds the time limit. Can you give me some insight about it?\n", "votes": 1, "replies": [ { "id": 1290183, "postDate": "2021-05-01T17:35:06.310Z", "content": "

I provide methods in the cell tile and image tile ensemble. The original cell tile notebook was not written by me. You can use my method to ensemble with other notebooks.

", "rawMarkdown": "I provide methods in the cell tile and image tile ensemble. The original cell tile notebook was not written by me. You can use my method to ensemble with other notebooks.", "votes": 3 }, { "id": 1300325, "postDate": "2021-05-10T11:52:54.177Z", "content": "

Hi @jagadish13

\n

I am using the same Notebook to submit.
\nbut I don't know why I got a higher Public Score with only_public = Ture
\nbut lower Public Score when only_public = False

\n

Any idea? @dschettler8845 :)

", "rawMarkdown": "Hi @jagadish13 \n\nI am using the same Notebook to submit. \nbut I don't know why I got a higher Public Score with `only_public = Ture` \nbut lower Public Score when `only_public = False`\n\nAny idea? @dschettler8845 :)", "votes": 2 }, { "id": 1301053, "postDate": "2021-05-10T23:42:07.597Z", "content": "

@faisalalsrheed attending to times I think the nb runs properly on private dataset when only_public = False and the lb gap I think is due to processing functions in the nb.

", "rawMarkdown": "@faisalalsrheed attending to times I think the nb runs properly on private dataset when `only_public = False` and the lb gap I think is due to processing functions in the nb.", "votes": 1 } ] }, { "id": 1264723, "postDate": "2021-04-06T11:51:05.737Z", "content": "

Thanks. How did your effn7 single model perform in LB? Have you tested that

", "rawMarkdown": "Thanks. How did your effn7 single model perform in LB? Have you tested that", "votes": 1, "replies": [ { "id": 1264730, "postDate": "2021-04-06T11:56:58.497Z", "content": "

My model is a CNN model, which uses a sigmoid function to output confidence values.

", "rawMarkdown": "My model is a CNN model, which uses a sigmoid function to output confidence values." }, { "id": 1264783, "postDate": "2021-04-06T12:25:00.370Z", "content": "

yes of course, but you could also increase the weights of your model to 1.0…

", "rawMarkdown": "yes of course, but you could also increase the weights of your model to 1.0...", "votes": 2 }, { "id": 1264801, "postDate": "2021-04-06T12:41:51.443Z", "content": "

This is usually not done unless the detection model performs really badly. Because the sorting between proteins is still very important.

", "rawMarkdown": "This is usually not done unless the detection model performs really badly. Because the sorting between proteins is still very important.", "votes": 1 }, { "id": 1264807, "postDate": "2021-04-06T12:44:05.357Z", "content": "

Can you explain what you mean by \"sorting between proteins\" ?

", "rawMarkdown": "Can you explain what you mean by \"sorting between proteins\" ?", "votes": 1 }, { "id": 1264830, "postDate": "2021-04-06T12:59:08.337Z", "content": "

The calculation method of the score is to sort all the masks. If all the masks of that image have the same confidence value. It will increase the confidence value of many false masks too much.

", "rawMarkdown": "The calculation method of the score is to sort all the masks. If all the masks of that image have the same confidence value. It will increase the confidence value of many false masks too much.", "votes": 1 }, { "id": 1264833, "postDate": "2021-04-06T13:04:37.280Z", "content": "

ok I understand :)

", "rawMarkdown": "ok I understand :)", "votes": 1 } ] }, { "id": 1264629, "postDate": "2021-04-06T10:17:01.780Z", "content": "

Thanks for sharing. This is gonna be magic…
\n\"We cannot see the wood for the trees.\"

", "rawMarkdown": "Thanks for sharing. This is gonna be magic...\n\"We cannot see the wood for the trees.\"", "votes": 2, "replies": [ { "id": 1264679, "postDate": "2021-04-06T11:10:40.197Z", "content": "

\"We cannot see the wood for the trees.\" This is an interesting metaphor.😃

", "rawMarkdown": "\"We cannot see the wood for the trees.\" This is an interesting metaphor.😃", "votes": 1 } ] }, { "id": 1264616, "postDate": "2021-04-06T09:59:36.267Z", "content": "

Could you please elaborate a bit more what you're doing there?
\nFrom my understanding, you are ensembling that 0.354 cell-level model by multiplying the outputs with your Efficientnet-B7 image-level model outputs?

", "rawMarkdown": "Could you please elaborate a bit more what you're doing there?\nFrom my understanding, you are ensembling that 0.354 cell-level model by multiplying the outputs with your Efficientnet-B7 image-level model outputs?", "votes": 2, "replies": [ { "id": 1264622, "postDate": "2021-04-06T10:09:17.927Z", "content": "

you are ensembling that 0.354 cell-level model by multiplying the outputs with you Efficientnet-B7 image level-model outputs
\nMost of your understanding is correct, but it is not a multiplication of confidence values. It is the weighted average of confidence values.

", "rawMarkdown": "`you are ensembling that 0.354 cell-level model by multiplying the outputs with you Efficientnet-B7 image level-model outputs`\nMost of your understanding is correct, but it is not a multiplication of confidence values. It is the weighted average of confidence values.", "votes": 1 }, { "id": 1264650, "postDate": "2021-04-06T10:38:48.603Z", "content": "

Ok thanks! I still don't quite get this table:

\n
\n

red: [1, 3, 5, 7, 9]
\n green:[0, 2, 4, 6, 8]
\n blue:[11, 13, 15, 17]
\n yellow:[10, 12, 14, 16, 18]

\n
\n

what is it supposed to mean?

\n

Edit:
\none more question:
\nyou're inputting only the green channel to your image of 3 channels and so you're inputting 2 empty channels to your 3-channel EffNet?

", "rawMarkdown": "Ok thanks! I still don't quite get this table:\n> red: [1, 3, 5, 7, 9]\ngreen:[0, 2, 4, 6, 8]\nblue:[11, 13, 15, 17]\nyellow:[10, 12, 14, 16, 18]\n\nwhat is it supposed to mean?\n\nEdit:\none more question:\nyou're inputting only the green channel to your image of 3 channels and so you're inputting 2 empty channels to your 3-channel EffNet?\n\n", "votes": 2 }, { "id": 1264659, "postDate": "2021-04-06T10:44:34.497Z", "content": "

I originally thought that there would be some labels that are particularly suitable for images of other colors, but I found that they couldn't be used except for green images.

\n
red: [1, 3, 5, 7, 9]\ngreen:[0, 2, 4, 6, 8]\nblue:[11, 13, 15, 17]\nyellow:[10, 12, 14, 16, 18]\n
\n

That is an example of what I wanted to do, but I didn't do it in the end.

", "rawMarkdown": "I originally thought that there would be some labels that are particularly suitable for images of other colors, but I found that they couldn't be used except for green images.\n```\nred: [1, 3, 5, 7, 9]\ngreen:[0, 2, 4, 6, 8]\nblue:[11, 13, 15, 17]\nyellow:[10, 12, 14, 16, 18]\n```\nThat is an example of what I wanted to do, but I didn't do it in the end.\n" }, { "id": 1264663, "postDate": "2021-04-06T10:48:31.833Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1264669, "postDate": "2021-04-06T10:56:59.903Z", "content": "

This method is to improve the sorting between images, but the sorting between proteins is still very important. This method can be improved because at least we have the exact label in the image. We cannot have exact labels between proteins.

", "rawMarkdown": "This method is to improve the sorting between images, but the sorting between proteins is still very important. This method can be improved because at least we have the exact label in the image. We cannot have exact labels between proteins." }, { "id": 1264914, "postDate": "2021-04-06T13:55:17.997Z", "content": "

@alexanderriedel I guess input=[green, green, green] according to the training kernel attached.

", "rawMarkdown": "@alexanderriedel I guess input=[green, green, green] according to the training kernel attached.", "votes": 2 }, { "id": 1264921, "postDate": "2021-04-06T13:58:10.537Z", "content": "

Ok thanks! Do you think this makes any difference than inputting just the green channel to a 1-channel EffNet?

", "rawMarkdown": "Ok thanks! Do you think this makes any difference than inputting just the green channel to a 1-channel EffNet?", "votes": 2 }, { "id": 1264935, "postDate": "2021-04-06T14:06:58.437Z", "content": "

Hmm…I've never used 1-channel EffNet.
\nCan we use it with ImageNet pretrained weights?

", "rawMarkdown": "Hmm...I've never used 1-channel EffNet.\nCan we use it with ImageNet pretrained weights?", "votes": 2 }, { "id": 1264955, "postDate": "2021-04-06T14:16:15.913Z", "content": "

Well like copying Imagenet weights from a 3-channel model to a 4-channel model, you can also just copy the weights of one channel to get a 1-channel model.

\n

something like this…

\n

weight = model.model.conv_stem [0].weight[0].clone()\n
\n\nmodel.model.conv_stem [0] = nn.Conv2d(1, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), \nbias=False) \n
\n\nwith torch.no_grad():\n
\n\n model.model.conv_stem [0].weight[:, 0] = weight\n

", "rawMarkdown": "Well like copying Imagenet weights from a 3-channel model to a 4-channel model, you can also just copy the weights of one channel to get a 1-channel model.\n\nsomething like this...\n\n`weight = model.model.conv_stem [0].weight[0].clone()\n`\n`\nmodel.model.conv_stem [0] = nn.Conv2d(1, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), \nbias=False) \n`\n`\nwith torch.no_grad():\n`\n`\n model.model.conv_stem [0].weight[:, 0] = weight\n`", "votes": 3 } ] }, { "id": 1273180, "postDate": "2021-04-14T06:34:59.970Z", "content": "

There is a fundamental problem with confidence in machine learning. Model have worst result on sample types, which has not been presented in training set. But if these sample types has not been presented in training set, there is no way for model to properly adjust output confidence values for those sample types.

", "rawMarkdown": "There is a fundamental problem with confidence in machine learning. Model have worst result on sample types, which has not been presented in training set. But if these sample types has not been presented in training set, there is no way for model to properly adjust output confidence values for those sample types." } ], "comments": [ { "id": 1265684, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-07T06:06:08.190000", "content": "

It's amazing how fast this is implemented by everyone 😄

", "votes": 5, "replies": [ { "id": 1266139, "author_name": "LucaMTB", "author_url": "", "post_date": "2021-04-07T14:16:14.563000", "content": "

I think most people just copied Aliens notebook. Because 50 teams score between 0.424 und 0.427

", "votes": 4, "replies": [] } ] }, { "id": 1266496, "author_name": "Amanda K Kimball", "author_url": "", "post_date": "2021-04-07T19:58:58.763000", "content": "

Thank you. I had not considered a multi-tiered ensemble.

", "votes": 3, "replies": [] }, { "id": 1265829, "author_name": "LucaMTB", "author_url": "", "post_date": "2021-04-07T08:40:21.583000", "content": "

It is very interesting how you can combine image-level and cell-level prediction to improve your score. Thank you very much. I never thought Bayesian statistics could help that much.

", "votes": 3, "replies": [] }, { "id": 1265326, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2021-04-06T19:48:15.110000", "content": "

thanks for sharing this approach abt only using Green chennel.
\nto me it looks like the model is trying to find the clustering pattern of the protein and make a prediction. like the proteins will appear in certain formation if they are within nucleus,etc. but not sure if this is a good approach for the end goal of the competition where they want to segment cells based on the protein location within the cells.

", "votes": 3, "replies": [] }, { "id": 1264894, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-04-06T13:43:24.213000", "content": "

Hey @h053473666 (Alien)! Glad to see you had so much success. I also have been experimenting with ensembling slide and tile level predictions. I had a question regarding the last table. Are you saying that you apply the weighting/combination as follows…

\n
\n

0.4 Weight for Cell Labels * 0.6 Weight for Slide Label = 0.424 on LB?

\n

i.e. Can you check my understanding given the code below?

\n
single_tile_pred = [0.1 0.1 0.1 ... 0.9 0.1]\nslide_level_pred = [0.01 0.01 0.01 ... 0.5 0.2]\n\n# combine labels -- [0.0001, 0.0001, 0.0001 ... 0.45 0.02]\ncombined_preds = slide_tile_pred*0.4+slide_level_pred*0.6\n
\n

I'm dumb. This was clarified by Alex Riedel below.

\n
\n

Thanks in advance and Congratulations!

", "votes": 3, "replies": [ { "id": 1264899, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T13:49:08.697000", "content": "

No, if you have 0.99 on tile-level and 0.95 on image-level it's going to be 0.4*0.99 + 0.6*0.95 = 0.966

", "votes": 2, "replies": [] }, { "id": 1264904, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-04-06T13:51:20.927000", "content": "

oh right. duh

", "votes": 1, "replies": [] }, { "id": 1264991, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T14:39:47.533000", "content": "

Multiplication is not intuitive to me. But multiplication is indeed a feasible method. However, there are many factors to consider. The weighted average is more versatile. Who are interested in multiplication can refer to @cdeotte method. Although it can't be used directly, it is very good in the post-processing.https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637

", "votes": 1, "replies": [] } ] }, { "id": 1264874, "author_name": "Darek KÅ‚eczek", "author_url": "", "post_date": "2021-04-06T13:32:45.810000", "content": "

Training model on green channel only - wow, it's crazy that this works, this means that green channel may have enough signal to determine the class… thanks for sharing!

", "votes": 3, "replies": [ { "id": 1264903, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T13:50:56.277000", "content": "

This is also a misunderstanding that people often encounter. Humans and machines don't pay attention to the same things when they look at pictures. I think other methods should be used to enhance (nucleus, microtubules, endoplasmic reticulum) features, and the methods they use should break some features.

", "votes": 4, "replies": [] }, { "id": 1264956, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T14:17:15.950000", "content": "

Have you tried training your network on all channels? Maybe it's giving even better predictions?

", "votes": 2, "replies": [] }, { "id": 1264993, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T14:42:06.560000", "content": "

I'm not sure if it will be better, but my instinct is that it will increase noise.

", "votes": 0, "replies": [] }, { "id": 1265003, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T14:47:10.937000", "content": "

Described in this linkhttps://www.kaggle.com/c/hpa-single-cell-image-classification/data.
\nAll image samples are represented by four filters (stored as individual files), the protein of interest (green) plus three cellular landmarks: nucleus (blue), microtubules (red), endoplasmic reticulum (yellow). The green filter should hence be used to predict the label, and the other filters are used as references.

", "votes": 0, "replies": [] }, { "id": 1265008, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T14:50:21.763000", "content": "

Yeah actually you're right but I really think no one here had the courage to train only on the green channel 😄

", "votes": 3, "replies": [] } ] }, { "id": 1290153, "author_name": "Jagadish Sivakumar", "author_url": "", "post_date": "2021-05-01T17:06:19.240000", "content": "

Hi @h053473666

\n

Great Notebook. But does this work on the private LB? When I change only_public to False, the notebook exceeds the time limit. Can you give me some insight about it?

", "votes": 1, "replies": [ { "id": 1290183, "author_name": "Alien", "author_url": "", "post_date": "2021-05-01T17:35:06.310000", "content": "

I provide methods in the cell tile and image tile ensemble. The original cell tile notebook was not written by me. You can use my method to ensemble with other notebooks.

", "votes": 3, "replies": [] }, { "id": 1300325, "author_name": "Faisal Alsrheed", "author_url": "", "post_date": "2021-05-10T11:52:54.177000", "content": "

Hi @jagadish13

\n

I am using the same Notebook to submit.
\nbut I don't know why I got a higher Public Score with only_public = Ture
\nbut lower Public Score when only_public = False

\n

Any idea? @dschettler8845 :)

", "votes": 2, "replies": [] }, { "id": 1301053, "author_name": "Nanashi", "author_url": "", "post_date": "2021-05-10T23:42:07.597000", "content": "

@faisalalsrheed attending to times I think the nb runs properly on private dataset when only_public = False and the lb gap I think is due to processing functions in the nb.

", "votes": 1, "replies": [] } ] }, { "id": 1264723, "author_name": "Shihao Shao", "author_url": "", "post_date": "2021-04-06T11:51:05.737000", "content": "

Thanks. How did your effn7 single model perform in LB? Have you tested that

", "votes": 1, "replies": [ { "id": 1264730, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T11:56:58.497000", "content": "

My model is a CNN model, which uses a sigmoid function to output confidence values.

", "votes": 0, "replies": [] }, { "id": 1264783, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T12:25:00.370000", "content": "

yes of course, but you could also increase the weights of your model to 1.0…

", "votes": 2, "replies": [] }, { "id": 1264801, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T12:41:51.443000", "content": "

This is usually not done unless the detection model performs really badly. Because the sorting between proteins is still very important.

", "votes": 1, "replies": [] }, { "id": 1264807, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T12:44:05.357000", "content": "

Can you explain what you mean by \"sorting between proteins\" ?

", "votes": 1, "replies": [] }, { "id": 1264830, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T12:59:08.337000", "content": "

The calculation method of the score is to sort all the masks. If all the masks of that image have the same confidence value. It will increase the confidence value of many false masks too much.

", "votes": 1, "replies": [] }, { "id": 1264833, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T13:04:37.280000", "content": "

ok I understand :)

", "votes": 1, "replies": [] } ] }, { "id": 1264629, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-04-06T10:17:01.780000", "content": "

Thanks for sharing. This is gonna be magic…
\n\"We cannot see the wood for the trees.\"

", "votes": 2, "replies": [ { "id": 1264679, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T11:10:40.197000", "content": "

\"We cannot see the wood for the trees.\" This is an interesting metaphor.😃

", "votes": 1, "replies": [] } ] }, { "id": 1264616, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T09:59:36.267000", "content": "

Could you please elaborate a bit more what you're doing there?
\nFrom my understanding, you are ensembling that 0.354 cell-level model by multiplying the outputs with your Efficientnet-B7 image-level model outputs?

", "votes": 2, "replies": [ { "id": 1264622, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T10:09:17.927000", "content": "

you are ensembling that 0.354 cell-level model by multiplying the outputs with you Efficientnet-B7 image level-model outputs
\nMost of your understanding is correct, but it is not a multiplication of confidence values. It is the weighted average of confidence values.

", "votes": 1, "replies": [] }, { "id": 1264650, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T10:38:48.603000", "content": "

Ok thanks! I still don't quite get this table:

\n
\n

red: [1, 3, 5, 7, 9]
\n green:[0, 2, 4, 6, 8]
\n blue:[11, 13, 15, 17]
\n yellow:[10, 12, 14, 16, 18]

\n
\n

what is it supposed to mean?

\n

Edit:
\none more question:
\nyou're inputting only the green channel to your image of 3 channels and so you're inputting 2 empty channels to your 3-channel EffNet?

", "votes": 2, "replies": [] }, { "id": 1264659, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T10:44:34.497000", "content": "

I originally thought that there would be some labels that are particularly suitable for images of other colors, but I found that they couldn't be used except for green images.

\n
red: [1, 3, 5, 7, 9]\ngreen:[0, 2, 4, 6, 8]\nblue:[11, 13, 15, 17]\nyellow:[10, 12, 14, 16, 18]\n
\n

That is an example of what I wanted to do, but I didn't do it in the end.

", "votes": 0, "replies": [] }, { "id": 1264663, "author_name": "", "author_url": "", "post_date": "2021-04-06T10:48:31.833000", "content": "", "votes": 0, "replies": [] }, { "id": 1264669, "author_name": "Alien", "author_url": "", "post_date": "2021-04-06T10:56:59.903000", "content": "

This method is to improve the sorting between images, but the sorting between proteins is still very important. This method can be improved because at least we have the exact label in the image. We cannot have exact labels between proteins.

", "votes": 0, "replies": [] }, { "id": 1264914, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-04-06T13:55:17.997000", "content": "

@alexanderriedel I guess input=[green, green, green] according to the training kernel attached.

", "votes": 2, "replies": [] }, { "id": 1264921, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T13:58:10.537000", "content": "

Ok thanks! Do you think this makes any difference than inputting just the green channel to a 1-channel EffNet?

", "votes": 2, "replies": [] }, { "id": 1264935, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-04-06T14:06:58.437000", "content": "

Hmm…I've never used 1-channel EffNet.
\nCan we use it with ImageNet pretrained weights?

", "votes": 2, "replies": [] }, { "id": 1264955, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-04-06T14:16:15.913000", "content": "

Well like copying Imagenet weights from a 3-channel model to a 4-channel model, you can also just copy the weights of one channel to get a 1-channel model.

\n

something like this…

\n

weight = model.model.conv_stem [0].weight[0].clone()\n
\n\nmodel.model.conv_stem [0] = nn.Conv2d(1, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), \nbias=False) \n
\n\nwith torch.no_grad():\n
\n\n model.model.conv_stem [0].weight[:, 0] = weight\n

", "votes": 3, "replies": [] } ] }, { "id": 1273180, "author_name": "ZavodRobotov", "author_url": "", "post_date": "2021-04-14T06:34:59.970000", "content": "

There is a fundamental problem with confidence in machine learning. Model have worst result on sample types, which has not been presented in training set. But if these sample types has not been presented in training set, there is no way for model to properly adjust output confidence values for those sample types.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1264524": "Create label:\n[Classification label csv[green]\n](https://www.kaggle.com/h053473666/classification-label-csv-green)\nTraining:\n[[HPA] classification efnb7 train\n](https://www.kaggle.com/h053473666/hpa-classification-efnb7-train)\nInference:\n[0.354+efnb7 Classification weights 0.4 0.6](https://www.kaggle.com/h053473666/0-354-efnb7-classification-weights-0-4-0-6)\n\nI have experience in [vinbigdata ](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/overview)competitions. In the vinbigdata competition, I do multi-label classification on original images and then weighted average confidence values. In this competition, I used the same method to improve my score. I use this [notebook ](https://www.kaggle.com/dschettler8845/hpa-cellwise-classification-inference)add multi-label classification. Thanks to @dschettler8845 \n\nWhy do I use image green for training? In fact, I originally expected a different label for each color.\nLike the following:\nred: [1, 3, 5, 7, 9]\ngreen:[0, 2, 4, 6, 8]\nblue:[11, 13, 15, 17]\nyellow:[10, 12, 14, 16, 18]\n\nI use efficientnet-b0 to train each label individually, and I find that the AUC other than green is too low. So in the end I used efficientnet-b7 to train all the labels in green images.\n\nOverfitting or no overfitting?\n\nThresholds of different weights will have different results and vary greatly. Different models (detection) do a weighted average of confidence values, and LB scores vary greatly. It is possible to find a threshold with the highest LB score, but it is usually not the best threshold.\n\nThe following are my LB scores with different weights:\nLB score | weights | classification weights \n--------------|:-----:|-----\n0.376 |0.9 | 0.1\n0.396 | 0.8 | 0.2\n0.408 | 0.7 | 0.3\n0.417 |0.6 | 0.4\n0.423| 0.5 | 0.5\n0.424 | 0.4 | 0.6\n", "1265684": "It's amazing how fast this is implemented by everyone 😄", "1266496": "Thank you. I had not considered a multi-tiered ensemble. ", "1265829": "It is very interesting how you can combine image-level and cell-level prediction to improve your score. Thank you very much. I never thought Bayesian statistics could help that much.", "1265326": "thanks for sharing this approach abt only using Green chennel. \nto me it looks like the model is trying to find the clustering pattern of the protein and make a prediction. like the proteins will appear in certain formation if they are within nucleus,etc. but not sure if this is a good approach for the end goal of the competition where they want to segment cells based on the protein location within the cells. ", "1264894": "Hey @h053473666 (Alien)! Glad to see you had so much success. I also have been experimenting with ensembling slide and tile level predictions. I had a question regarding the last table. Are you saying that you apply the weighting/combination as follows...\n\n---\n\n0.4 Weight for Cell Labels * 0.6 Weight for Slide Label = 0.424 on LB?\n\ni.e. Can you check my understanding given the code below?\n\n```\nsingle_tile_pred = [0.1 0.1 0.1 ... 0.9 0.1]\nslide_level_pred = [0.01 0.01 0.01 ... 0.5 0.2]\n\n# combine labels -- [0.0001, 0.0001, 0.0001 ... 0.45 0.02]\ncombined_preds = slide_tile_pred*0.4+slide_level_pred*0.6\n```\n\n~~If this is the case wouldn't it artificially lower the confidence of the predictions? i.e. 0.99*0.95=0.94 which is lower than either prediction.~~ I'm dumb. This was clarified by Alex Riedel below.\n\n---\n\nThanks in advance and Congratulations!", "1264874": "Training model on green channel only - wow, it's crazy that this works, this means that green channel may have enough signal to determine the class... thanks for sharing!", "1290153": "Hi @h053473666 \n\nGreat Notebook. But does this work on the private LB? When I change only_public to False, the notebook exceeds the time limit. Can you give me some insight about it?\n", "1264723": "Thanks. How did your effn7 single model perform in LB? Have you tested that", "1264629": "Thanks for sharing. This is gonna be magic...\n\"We cannot see the wood for the trees.\"", "1264616": "Could you please elaborate a bit more what you're doing there?\nFrom my understanding, you are ensembling that 0.354 cell-level model by multiplying the outputs with your Efficientnet-B7 image-level model outputs?", "1273180": "There is a fundamental problem with confidence in machine learning. Model have worst result on sample types, which has not been presented in training set. But if these sample types has not been presented in training set, there is no way for model to properly adjust output confidence values for those sample types." } }