![\"\"](\"https://media.arxiv-vanity.com/render-output/4196403/x2.png\")
i didn't know there is \"negative learning\" until reading this paper
\nhttps://arxiv.org/pdf/1908.07387.pdf
\nNLNL: Negative Learning for Noisy Labels
Quote:
\nNegative Learning (NL), in which the CNNs are trained using a complementary label as in “input image does not belong to this complementary label.”
The classical method of training CNNs is by labeling images in
\na supervised manner as in “input image belongs to this label” (Positive Learning; PL)
i didn't know there is \"negative learning\" until reading this paper
\nhttps://arxiv.org/pdf/1908.07387.pdf
\nNLNL: Negative Learning for Noisy Labels
Quote:
\nNegative Learning (NL), in which the CNNs are trained using a complementary label as in “input image does not belong to this complementary label.”
The classical method of training CNNs is by labeling images in
\na supervised manner as in “input image belongs to this label” (Positive Learning; PL)
This is interesting. Something I've noticed during training is that the model's precision on validation is much lower than during training (0.21 vs 0.99 for example), so there's obviously some false positive action going on there. Maybe this is the key to avoiding it?
", "rawMarkdown": "This is interesting. Something I've noticed during training is that the model's precision on validation is much lower than during training (0.21 vs 0.99 for example), so there's obviously some false positive action going on there. Maybe this is the key to avoiding it?", "votes": null }, { "id": "1191027", "postDate": "02/08/2021 07:50:51", "content": "Very interesting. In your opinion - can this be succesfully applied to segmentation, e.g. U-Net?
\nThere are many 'noisy-labels' in the current HuBMAP competition, maybe this would work.
turns out that deep network is just an SVM?
\nhttps://arxiv.org/pdf/2012.00152.pdf
\nhttps://www.youtube.com/watch?v=ahRPdiCop3E
\nEvery Model Learned by Gradient Descent Is Approximately a Kernel Machine
Figure 2: Deep network weights as superpositions of training examples. Applying the
\nlearned model to a query example is equivalent to simultaneously matching the
\nquery with each stored example using the path kernel and outputting a weighted
\nsum of the results.
this is actually very obvious since in sgd:
\nif y = weight*x, # (linear svm)\nthen \nweight = weight + grad\nand\ngrad = sum{g.x} #weight are just sample\n\n\nweight*z = sum{g.x*z }# (linear kernel)\nso the tp and fp are just support vectors
", "rawMarkdown": "turns out that deep network is just an SVM?\n\nhttps://arxiv.org/pdf/2012.00152.pdf\nhttps://www.youtube.com/watch?v=ahRPdiCop3E\nEvery Model Learned by Gradient Descent Is Approximately a Kernel Machine\n\n\n\nFigure 2: Deep network weights as superpositions of training examples. Applying the\nlearned model to a query example is equivalent to simultaneously matching the\nquery with each stored example using the path kernel and outputting a weighted\nsum of the results.\n\n\nthis is actually very obvious since in sgd:\n```\nif y = weight*x, # (linear svm)\nthen \nweight = weight + grad\nand\ngrad = sum{g.x} #weight are just sample\n\n\nweight*z = sum{g.x*z }# (linear kernel)\n```\n\n\nso the tp and fp are just support vectors", "votes": null }, { "id": "1199964", "postDate": "02/14/2021 09:40:04", "content": "\"negative learning\" seems not to be the correct term.
\nI think \"complementary labels\" is more correct. (I find more papers with this keyword)
\nhttps://proceedings.neurips.cc/paper/2017/file/1dba5eed8838571e1c80af145184e515-Paper.pdf
\nQuote:
\nCollecting labeled data is costly and thus a critical bottleneck in real-world classification tasks. To mitigate this problem, we propose a novel setting, namely learning from complementary labels for multi-class classification. A complementary
\nlabel specifies a class that a pattern does not belong to.
I was searching for this, thank you :)
", "rawMarkdown": "I was searching for this, thank you :)", "votes": null } ], "comments": [ { "id": 1190279, "author_name": "jtan2231", "author_url": "", "post_date": "02/07/2021 16:00:52", "content": "This is interesting. Something I've noticed during training is that the model's precision on validation is much lower than during training (0.21 vs 0.99 for example), so there's obviously some false positive action going on there. Maybe this is the key to avoiding it?
", "votes": null, "replies": [] }, { "id": 1191027, "author_name": "wrrosa", "author_url": "", "post_date": "02/08/2021 07:50:51", "content": "Very interesting. In your opinion - can this be succesfully applied to segmentation, e.g. U-Net?
\nThere are many 'noisy-labels' in the current HuBMAP competition, maybe this would work.
turns out that deep network is just an SVM?
\nhttps://arxiv.org/pdf/2012.00152.pdf
\nhttps://www.youtube.com/watch?v=ahRPdiCop3E
\nEvery Model Learned by Gradient Descent Is Approximately a Kernel Machine
Figure 2: Deep network weights as superpositions of training examples. Applying the
\nlearned model to a query example is equivalent to simultaneously matching the
\nquery with each stored example using the path kernel and outputting a weighted
\nsum of the results.
this is actually very obvious since in sgd:
\nif y = weight*x, # (linear svm)\nthen \nweight = weight + grad\nand\ngrad = sum{g.x} #weight are just sample\n\n\nweight*z = sum{g.x*z }# (linear kernel)\nso the tp and fp are just support vectors
", "votes": null, "replies": [] }, { "id": 1199964, "author_name": "hengck23", "author_url": "", "post_date": "02/14/2021 09:40:04", "content": "\"negative learning\" seems not to be the correct term.
\nI think \"complementary labels\" is more correct. (I find more papers with this keyword)
\nhttps://proceedings.neurips.cc/paper/2017/file/1dba5eed8838571e1c80af145184e515-Paper.pdf
\nQuote:
\nCollecting labeled data is costly and thus a critical bottleneck in real-world classification tasks. To mitigate this problem, we propose a novel setting, namely learning from complementary labels for multi-class classification. A complementary
\nlabel specifies a class that a pattern does not belong to.
I was searching for this, thank you :)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1189765": "i didn't know there is \"negative learning\" until reading this paper\n\nhttps://arxiv.org/pdf/1908.07387.pdf\nNLNL: Negative Learning for Noisy Labels\n\n\nQuote: \nNegative Learning (NL), in which the CNNs are trained using a complementary label as in “input image does not belong to this complementary label.”\n\nThe classical method of training CNNs is by labeling images in\na supervised manner as in “input image belongs to this label” (Positive Learning; PL)", "1190279": "This is interesting. Something I've noticed during training is that the model's precision on validation is much lower than during training (0.21 vs 0.99 for example), so there's obviously some false positive action going on there. Maybe this is the key to avoiding it?", "1191027": "Very interesting. In your opinion - can this be succesfully applied to segmentation, e.g. U-Net?\nThere are many 'noisy-labels' in the current HuBMAP competition, maybe this would work.", "1194871": "turns out that deep network is just an SVM?\n\nhttps://arxiv.org/pdf/2012.00152.pdf\nhttps://www.youtube.com/watch?v=ahRPdiCop3E\nEvery Model Learned by Gradient Descent Is Approximately a Kernel Machine\n\n\n\nFigure 2: Deep network weights as superpositions of training examples. Applying the\nlearned model to a query example is equivalent to simultaneously matching the\nquery with each stored example using the path kernel and outputting a weighted\nsum of the results.\n\n\nthis is actually very obvious since in sgd:\n```\nif y = weight*x, # (linear svm)\nthen \nweight = weight + grad\nand\ngrad = sum{g.x} #weight are just sample\n\n\nweight*z = sum{g.x*z }# (linear kernel)\n```\n\n\nso the tp and fp are just support vectors", "1199964": "\"negative learning\" seems not to be the correct term.\n\nI think \"complementary labels\" is more correct. (I find more papers with this keyword)\n\n\nhttps://proceedings.neurips.cc/paper/2017/file/1dba5eed8838571e1c80af145184e515-Paper.pdf\nQuote:\nCollecting labeled data is costly and thus a critical bottleneck in real-world classification tasks. To mitigate this problem, we propose a novel setting, namely learning from complementary labels for multi-class classification. A complementary\nlabel specifies a class that a pattern does not belong to.", "1201457": "I was searching for this, thank you :)" }, "source": "meta" }