{
  "id": 290145,
  "title": "🎥 We use continuous image. Is this useful??",
  "url": "/competitions/tensorflow-great-barrier-reef/discussion/290145",
  "author_name": "",
  "post_date": "2021-11-23T08:11:20.134880900Z",
  "votes": 7,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Image data is extracted from video frame by frame. So <strong>Image data is correlated to each other.</strong></p>\n<p>For example, if there was a starfish in frame 4, there would be a starfish in frame 5-6 on similar location. Isn't it?</p>\n<p>How can we apply this continuous information in Object Detection?</p>\n<p>Is there anybody some ideas?</p>",
  "messages": [
    {
      "id": "1592635",
      "postDate": "11/23/2021 08:11:20",
      "content": "<p>Image data is extracted from video frame by frame. So <strong>Image data is correlated to each other.</strong></p>\n<p>For example, if there was a starfish in frame 4, there would be a starfish in frame 5-6 on similar location. Isn't it?</p>\n<p>How can we apply this continuous information in Object Detection?</p>\n<p>Is there anybody some ideas?</p>",
      "rawMarkdown": "Image data is extracted from video frame by frame. So **Image data is correlated to each other.**\n\nFor example, if there was a starfish in frame 4, there would be a starfish in frame 5-6 on similar location. Isn't it?\n\nHow can we apply this continuous information in Object Detection?\n\nIs there anybody some ideas?",
      "votes": null
    },
    {
      "id": "1592790",
      "postDate": "11/23/2021 11:01:09",
      "content": "<p>Such setup reminds me about using OpticalFlow (from OpenCV, or FlowNet like). It would be useful to expect starfish in some positions. But we cannot rely only on that expected positions (some starfish may be missed in the previous frame). Do you have an idea how to expand that approach?</p>",
      "rawMarkdown": "Such setup reminds me about using OpticalFlow (from OpenCV, or FlowNet like). It would be useful to expect starfish in some positions. But we cannot rely only on that expected positions (some starfish may be missed in the previous frame). Do you have an idea how to expand that approach?",
      "votes": null
    },
    {
      "id": "1593378",
      "postDate": "11/23/2021 20:36:18",
      "content": "<p>Yes, I think it could be important. Coral reef substrate is very complex and while CoTS are fairly large and have a distinctive forest green color, the color signal is going to be difficult to extract at the distance of this imagery (still: I think it's <em>very</em> worthwhile pursuing trying to isolate that color). Absent the color signal, you're going to have to distinguish the CoTS shape, which is going to be a challenge in a single photo, given the variety of orientations possible. </p>\n<p>CoTS are also very slow-moving, so they will not dramatically (maybe not even detectably) change their posture during a single video pass.  Some kind of structure-from-motion would be ideal, but even short of that, just a continual series of ROIs that correspond to the underlying position would be, I think, potentially very helpful. </p>\n<p>In this photo, from video 0, <img src=\"https://imgur.com/GZA0M0Y\" alt=\"frame comparison\">, you can see how the shape of the CoTS (relatively flat, radial arms) is more apparent from the comparison than in either individual frame. </p>",
      "rawMarkdown": "Yes, I think it could be important. Coral reef substrate is very complex and while CoTS are fairly large and have a distinctive forest green color, the color signal is going to be difficult to extract at the distance of this imagery (still: I think it's _very_ worthwhile pursuing trying to isolate that color). Absent the color signal, you're going to have to distinguish the CoTS shape, which is going to be a challenge in a single photo, given the variety of orientations possible. \n\nCoTS are also very slow-moving, so they will not dramatically (maybe not even detectably) change their posture during a single video pass.  Some kind of structure-from-motion would be ideal, but even short of that, just a continual series of ROIs that correspond to the underlying position would be, I think, potentially very helpful. \n\nIn this photo, from video 0, ![frame comparison](https://imgur.com/GZA0M0Y), you can see how the shape of the CoTS (relatively flat, radial arms) is more apparent from the comparison than in either individual frame.",
      "votes": null
    },
    {
      "id": "1596070",
      "postDate": "11/26/2021 07:15:56",
      "content": "<p>I think we can assume that every image is the frame is simple image and try to find whether object is present in the image or not.</p>",
      "rawMarkdown": "I think we can assume that every image is the frame is simple image and try to find whether object is present in the image or not.",
      "votes": null
    },
    {
      "id": "1597803",
      "postDate": "11/27/2021 23:22:14",
      "content": "<p>If we apply the data augmentation such as flipping, rotating cripping, cutting out… , we can get various type of training image.</p>",
      "rawMarkdown": "If we apply the data augmentation such as flipping, rotating cripping, cutting out... , we can get various type of training image.",
      "votes": null
    },
    {
      "id": "1599190",
      "postDate": "11/29/2021 08:02:57",
      "content": "<p>That is a good base approach. We definitely should start from it because we are working with sequences and there is no prior information about it.<br>\nBut taking to account the fact, that COTS can't disappear from the frame, we may use information from the previous frame. Naive example - provide additional ROI for FasterRCNN</p>",
      "rawMarkdown": "That is a good base approach. We definitely should start from it because we are working with sequences and there is no prior information about it.\nBut taking to account the fact, that COTS can't disappear from the frame, we may use information from the previous frame. Naive example - provide additional ROI for FasterRCNN",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1592790,
      "author_name": "meowmeowmeowmeowmeow",
      "author_url": "",
      "post_date": "11/23/2021 11:01:09",
      "content": "<p>Such setup reminds me about using OpticalFlow (from OpenCV, or FlowNet like). It would be useful to expect starfish in some positions. But we cannot rely only on that expected positions (some starfish may be missed in the previous frame). Do you have an idea how to expand that approach?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1593378,
      "author_name": "lobrien",
      "author_url": "",
      "post_date": "11/23/2021 20:36:18",
      "content": "<p>Yes, I think it could be important. Coral reef substrate is very complex and while CoTS are fairly large and have a distinctive forest green color, the color signal is going to be difficult to extract at the distance of this imagery (still: I think it's <em>very</em> worthwhile pursuing trying to isolate that color). Absent the color signal, you're going to have to distinguish the CoTS shape, which is going to be a challenge in a single photo, given the variety of orientations possible. </p>\n<p>CoTS are also very slow-moving, so they will not dramatically (maybe not even detectably) change their posture during a single video pass.  Some kind of structure-from-motion would be ideal, but even short of that, just a continual series of ROIs that correspond to the underlying position would be, I think, potentially very helpful. </p>\n<p>In this photo, from video 0, <img src=\"https://imgur.com/GZA0M0Y\" alt=\"frame comparison\">, you can see how the shape of the CoTS (relatively flat, radial arms) is more apparent from the comparison than in either individual frame. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1596070,
      "author_name": "shakshyathedetector",
      "author_url": "",
      "post_date": "11/26/2021 07:15:56",
      "content": "<p>I think we can assume that every image is the frame is simple image and try to find whether object is present in the image or not.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1599190,
          "author_name": "meowmeowmeowmeowmeow",
          "author_url": "",
          "post_date": "11/29/2021 08:02:57",
          "content": "<p>That is a good base approach. We definitely should start from it because we are working with sequences and there is no prior information about it.<br>\nBut taking to account the fact, that COTS can't disappear from the frame, we may use information from the previous frame. Naive example - provide additional ROI for FasterRCNN</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1597803,
      "author_name": "osamurai",
      "author_url": "",
      "post_date": "11/27/2021 23:22:14",
      "content": "<p>If we apply the data augmentation such as flipping, rotating cripping, cutting out… , we can get various type of training image.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1592635": "Image data is extracted from video frame by frame. So **Image data is correlated to each other.**\n\nFor example, if there was a starfish in frame 4, there would be a starfish in frame 5-6 on similar location. Isn't it?\n\nHow can we apply this continuous information in Object Detection?\n\nIs there anybody some ideas?",
    "1592790": "Such setup reminds me about using OpticalFlow (from OpenCV, or FlowNet like). It would be useful to expect starfish in some positions. But we cannot rely only on that expected positions (some starfish may be missed in the previous frame). Do you have an idea how to expand that approach?",
    "1593378": "Yes, I think it could be important. Coral reef substrate is very complex and while CoTS are fairly large and have a distinctive forest green color, the color signal is going to be difficult to extract at the distance of this imagery (still: I think it's _very_ worthwhile pursuing trying to isolate that color). Absent the color signal, you're going to have to distinguish the CoTS shape, which is going to be a challenge in a single photo, given the variety of orientations possible. \n\nCoTS are also very slow-moving, so they will not dramatically (maybe not even detectably) change their posture during a single video pass.  Some kind of structure-from-motion would be ideal, but even short of that, just a continual series of ROIs that correspond to the underlying position would be, I think, potentially very helpful. \n\nIn this photo, from video 0, ![frame comparison](https://imgur.com/GZA0M0Y), you can see how the shape of the CoTS (relatively flat, radial arms) is more apparent from the comparison than in either individual frame.",
    "1596070": "I think we can assume that every image is the frame is simple image and try to find whether object is present in the image or not.",
    "1597803": "If we apply the data augmentation such as flipping, rotating cripping, cutting out... , we can get various type of training image.",
    "1599190": "That is a good base approach. We definitely should start from it because we are working with sequences and there is no prior information about it.\nBut taking to account the fact, that COTS can't disappear from the frame, we may use information from the previous frame. Naive example - provide additional ROI for FasterRCNN"
  },
  "source": "meta"
}