{
  "id": 292600,
  "title": "Manually flaged broken masks (\"isbroken\" column in the \"train.csv\")",
  "url": "/competitions/sartorius-cell-instance-segmentation/discussion/292600",
  "author_name": "Sentinel-1",
  "post_date": "2021-12-02T16:49:22.719000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I have shared a <code>train.csv</code> file with the additional manually populated \"isbroken\" column having the purpose of flagging the broken labels in the original data. Due to the big amount of labels for a manual work I did not spend much of a time watching every single label, instead I just glimpsed them quickly and marked those which seemed to be broken, therefore it may contain some errors, but I guess most of the flagged labels should be really broken (\"broken\" meaning that they are with holes or with too much extra area coverage, and so on).</p>\n<p>I have not figured out just yet how exactly I will use it, but I might be using the \"isbroken\" flag for somehow excluding the broken labels from training. Maybe, using it as an \"iscrowd\" flag also could be an option.</p>\n<p>See the dataset here: <a href=\"https://www.kaggle.com/sentinel1/sartorius-label-isbroken\" target=\"_blank\">sartorius-label-isbroken</a></p>\n<p>UPDATE:<br>\nI tested it by using as the \"iscrowd\" flag in one of the public notebooks from this competition, see the notebook here: <a href=\"https://www.kaggle.com/sentinel1/sartorius-torch-mask-r-cnn\" target=\"_blank\">Sartorius - Torch Mask R-CNN</a></p>\n<p>The notebook has 2 versions only: Version 1 is the version 48 of the original notebook (forked) and Version 2 is updated to make use of the \"isbroken\" flag as a substitute for \"iscrowd\". I did submit both versions and the version 1 (i.e. the original) scored higher on LB than the version 2:</p>\n<ul>\n<li>Version 1: LB=0.286</li>\n<li>Version 2: LB=0.285</li>\n</ul>\n<p>This information is not enough for concluding if using the flag may have improved the actual model or not. As for improving a LB score, it was not improved as you can see above. Most probably the test data used to calculate LB scores is of the same quality (with the approximately same portion of broken labels), so in terms of improving LB score it does not make much of a sense to handle data quality issues in the training data when the test data remains the same, even if the model will be improved by addressing the data quality issues  - the LB score is going to go down anyway (due to not predicting those broken masks in the test data).</p>",
  "messages": [
    {
      "id": 1603641,
      "postDate": "2021-12-02T16:49:22.720Z",
      "content": "<p>I have shared a <code>train.csv</code> file with the additional manually populated \"isbroken\" column having the purpose of flagging the broken labels in the original data. Due to the big amount of labels for a manual work I did not spend much of a time watching every single label, instead I just glimpsed them quickly and marked those which seemed to be broken, therefore it may contain some errors, but I guess most of the flagged labels should be really broken (\"broken\" meaning that they are with holes or with too much extra area coverage, and so on).</p>\n<p>I have not figured out just yet how exactly I will use it, but I might be using the \"isbroken\" flag for somehow excluding the broken labels from training. Maybe, using it as an \"iscrowd\" flag also could be an option.</p>\n<p>See the dataset here: <a href=\"https://www.kaggle.com/sentinel1/sartorius-label-isbroken\" target=\"_blank\">sartorius-label-isbroken</a></p>\n<p>UPDATE:<br>\nI tested it by using as the \"iscrowd\" flag in one of the public notebooks from this competition, see the notebook here: <a href=\"https://www.kaggle.com/sentinel1/sartorius-torch-mask-r-cnn\" target=\"_blank\">Sartorius - Torch Mask R-CNN</a></p>\n<p>The notebook has 2 versions only: Version 1 is the version 48 of the original notebook (forked) and Version 2 is updated to make use of the \"isbroken\" flag as a substitute for \"iscrowd\". I did submit both versions and the version 1 (i.e. the original) scored higher on LB than the version 2:</p>\n<ul>\n<li>Version 1: LB=0.286</li>\n<li>Version 2: LB=0.285</li>\n</ul>\n<p>This information is not enough for concluding if using the flag may have improved the actual model or not. As for improving a LB score, it was not improved as you can see above. Most probably the test data used to calculate LB scores is of the same quality (with the approximately same portion of broken labels), so in terms of improving LB score it does not make much of a sense to handle data quality issues in the training data when the test data remains the same, even if the model will be improved by addressing the data quality issues  - the LB score is going to go down anyway (due to not predicting those broken masks in the test data).</p>",
      "rawMarkdown": "I have shared a `train.csv` file with the additional manually populated \"isbroken\" column having the purpose of flagging the broken labels in the original data. Due to the big amount of labels for a manual work I did not spend much of a time watching every single label, instead I just glimpsed them quickly and marked those which seemed to be broken, therefore it may contain some errors, but I guess most of the flagged labels should be really broken (\"broken\" meaning that they are with holes or with too much extra area coverage, and so on).\n\nI have not figured out just yet how exactly I will use it, but I might be using the \"isbroken\" flag for somehow excluding the broken labels from training. Maybe, using it as an \"iscrowd\" flag also could be an option.\n\nSee the dataset here: [sartorius-label-isbroken](https://www.kaggle.com/sentinel1/sartorius-label-isbroken)\n\nUPDATE:\nI tested it by using as the \"iscrowd\" flag in one of the public notebooks from this competition, see the notebook here: [Sartorius - Torch Mask R-CNN](https://www.kaggle.com/sentinel1/sartorius-torch-mask-r-cnn)\n\nThe notebook has 2 versions only: Version 1 is the version 48 of the original notebook (forked) and Version 2 is updated to make use of the \"isbroken\" flag as a substitute for \"iscrowd\". I did submit both versions and the version 1 (i.e. the original) scored higher on LB than the version 2:\n- Version 1: LB=0.286\n- Version 2: LB=0.285\n\nThis information is not enough for concluding if using the flag may have improved the actual model or not. As for improving a LB score, it was not improved as you can see above. Most probably the test data used to calculate LB scores is of the same quality (with the approximately same portion of broken labels), so in terms of improving LB score it does not make much of a sense to handle data quality issues in the training data when the test data remains the same, even if the model will be improved by addressing the data quality issues  - the LB score is going to go down anyway (due to not predicting those broken masks in the test data)."
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1603641": "I have shared a `train.csv` file with the additional manually populated \"isbroken\" column having the purpose of flagging the broken labels in the original data. Due to the big amount of labels for a manual work I did not spend much of a time watching every single label, instead I just glimpsed them quickly and marked those which seemed to be broken, therefore it may contain some errors, but I guess most of the flagged labels should be really broken (\"broken\" meaning that they are with holes or with too much extra area coverage, and so on).\n\nI have not figured out just yet how exactly I will use it, but I might be using the \"isbroken\" flag for somehow excluding the broken labels from training. Maybe, using it as an \"iscrowd\" flag also could be an option.\n\nSee the dataset here: [sartorius-label-isbroken](https://www.kaggle.com/sentinel1/sartorius-label-isbroken)\n\nUPDATE:\nI tested it by using as the \"iscrowd\" flag in one of the public notebooks from this competition, see the notebook here: [Sartorius - Torch Mask R-CNN](https://www.kaggle.com/sentinel1/sartorius-torch-mask-r-cnn)\n\nThe notebook has 2 versions only: Version 1 is the version 48 of the original notebook (forked) and Version 2 is updated to make use of the \"isbroken\" flag as a substitute for \"iscrowd\". I did submit both versions and the version 1 (i.e. the original) scored higher on LB than the version 2:\n- Version 1: LB=0.286\n- Version 2: LB=0.285\n\nThis information is not enough for concluding if using the flag may have improved the actual model or not. As for improving a LB score, it was not improved as you can see above. Most probably the test data used to calculate LB scores is of the same quality (with the approximately same portion of broken labels), so in terms of improving LB score it does not make much of a sense to handle data quality issues in the training data when the test data remains the same, even if the model will be improved by addressing the data quality issues  - the LB score is going to go down anyway (due to not predicting those broken masks in the test data)."
  }
}