{
  "id": 21161,
  "title": "Data Augmentation + Gradient orientation: strange movie",
  "url": "/competitions/draper-satellite-image-chronology/discussion/21161",
  "author_name": "",
  "post_date": "2016-05-23T22:25:45.283Z",
  "votes": 5,
  "comment_count": 2,
  "views": 871,
  "content": "<p>See the attached files to understand or if you are interested to the results of what I did to the first set of the data set (data is augmented 80X times, making 28000 pictures for training and 109600 for testing):</p>\n\n<ul>\n<li><p>Montage.jpg =&gt; a high quality montage of the first set in the train set (15MB, so big sized a browser can't handle it properly?? 3840x46080)</p></li>\n<li><p>Montage_Small.jpg =&gt; a low quality montage of the first set in the train set (4MB, 1280x15360)</p></li>\n<li><p>Aligned.gif =&gt; a high quality sequence of the first set in the train set (12MB)</p></li>\n<li><p>Aligned_Small.gif =&gt; a low quality montage of the first set in the train set (4MB)</p></li>\n</ul>\n\n<p>I computed some <a href=\"https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients\">local gradient orientation</a> statistics using a 5x5 <a href=\"https://en.wikipedia.org/wiki/Sobel_operator\">Sobel filter</a>... I might be wrong but the blue channel is clearly horrible to use as a predicting feature? (local gradient orientation gave me no goodness for that blue channel.. well, mostly under 0.20 =&gt; the <a href=\"https://en.wikipedia.org/wiki/Directional_statistics\">Gaussian directionality</a> of the structure of the channel is barely predictable, although researches found the goodness value underestimates the reality). I found also there are angles where the predictability rises up to 1.00, which is not normal? However, the what is nice to see is the directionality and the structure using the transformation I used are kept (see <a href=\"https://en.wikipedia.org/wiki/Image_gradient\">image gradient</a>).</p>\n\n<p>The color wheel of angles of the first set of the train set:</p>\n\n<p><img src=\"http://i.imgur.com/2pDewRF.jpg\" alt=\"enter image description here\" title></p>\n\n<p>And the color wheels of the first set by image order, which are exactly identical pixel to pixel (clearly unexpected, check yourself if you want using a substraction method):</p>\n\n<p><img src=\"http://i.imgur.com/3yEFzeT.jpg\" alt=\"enter image description here\" title></p>\n\n<p>I assume the satellites used for the imagery are <a href=\"https://en.wikipedia.org/wiki/Multispectral_image\">multispectral</a> and capturing ultra-blue wavelengths (which is re-mapped to false color composite), and this would go against saying the blue channel is horrible? As far as I know, multispectral images are useful in the sense they capture information that cannot be visually seen, and are highly sensitive to the soil content (if the soil changes, the channel changes).</p>\n\n<p>If it is possible to infer the wavelength of colors, then it would feasible to reconstruct the <a href=\"http://landsat.usgs.gov/best_spectral_bands_to_use.php\">Landsat bands</a> and have a good reconstruction of the features in the images (especially as we are looking in changes in the pictures). For instance, we would be able to optimize vegetation/forest pictures by finding the lowest vegetation area of the pictures and setting it as baseline to use a ML technique to infer whether a specific vegetation should come before or after the baseline (much more easily! like the set with nearly forest only).</p>\n\n<p>However, I'm still skeptical about its feasibility: not that because the pictures might not be &quot;fully multispectral&quot;, but the satellites might not get the required bands for an approximate Landsat reconstruction (and there is not enough information given per pixel here).</p>\n\n<p>Online pictures of Landsat are <a href=\"http://earthexplorer.usgs.gov/\">free</a>, but it gets out of control if we have to position geolocalize each picture and harvest them, having the need to grab the timestamp also.</p>\n\n<p>Note: I have not tested the .tiff files. Too big to fit on my drive space left.</p>\n\n<p>Edit: added ton of external links related to feature extraction related to this topic because it would be confusing for people who are not used to the terms used</p>",
  "messages": [
    {
      "id": "121102",
      "postDate": "05/23/2016 22:25:45",
      "content": "<p>See the attached files to understand or if you are interested to the results of what I did to the first set of the data set (data is augmented 80X times, making 28000 pictures for training and 109600 for testing):</p>\n\n<ul>\n<li><p>Montage.jpg =&gt; a high quality montage of the first set in the train set (15MB, so big sized a browser can't handle it properly?? 3840x46080)</p></li>\n<li><p>Montage_Small.jpg =&gt; a low quality montage of the first set in the train set (4MB, 1280x15360)</p></li>\n<li><p>Aligned.gif =&gt; a high quality sequence of the first set in the train set (12MB)</p></li>\n<li><p>Aligned_Small.gif =&gt; a low quality montage of the first set in the train set (4MB)</p></li>\n</ul>\n\n<p>I computed some <a href=\"https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients\">local gradient orientation</a> statistics using a 5x5 <a href=\"https://en.wikipedia.org/wiki/Sobel_operator\">Sobel filter</a>... I might be wrong but the blue channel is clearly horrible to use as a predicting feature? (local gradient orientation gave me no goodness for that blue channel.. well, mostly under 0.20 =&gt; the <a href=\"https://en.wikipedia.org/wiki/Directional_statistics\">Gaussian directionality</a> of the structure of the channel is barely predictable, although researches found the goodness value underestimates the reality). I found also there are angles where the predictability rises up to 1.00, which is not normal? However, the what is nice to see is the directionality and the structure using the transformation I used are kept (see <a href=\"https://en.wikipedia.org/wiki/Image_gradient\">image gradient</a>).</p>\n\n<p>The color wheel of angles of the first set of the train set:</p>\n\n<p><img src=\"http://i.imgur.com/2pDewRF.jpg\" alt=\"enter image description here\" title></p>\n\n<p>And the color wheels of the first set by image order, which are exactly identical pixel to pixel (clearly unexpected, check yourself if you want using a substraction method):</p>\n\n<p><img src=\"http://i.imgur.com/3yEFzeT.jpg\" alt=\"enter image description here\" title></p>\n\n<p>I assume the satellites used for the imagery are <a href=\"https://en.wikipedia.org/wiki/Multispectral_image\">multispectral</a> and capturing ultra-blue wavelengths (which is re-mapped to false color composite), and this would go against saying the blue channel is horrible? As far as I know, multispectral images are useful in the sense they capture information that cannot be visually seen, and are highly sensitive to the soil content (if the soil changes, the channel changes).</p>\n\n<p>If it is possible to infer the wavelength of colors, then it would feasible to reconstruct the <a href=\"http://landsat.usgs.gov/best_spectral_bands_to_use.php\">Landsat bands</a> and have a good reconstruction of the features in the images (especially as we are looking in changes in the pictures). For instance, we would be able to optimize vegetation/forest pictures by finding the lowest vegetation area of the pictures and setting it as baseline to use a ML technique to infer whether a specific vegetation should come before or after the baseline (much more easily! like the set with nearly forest only).</p>\n\n<p>However, I'm still skeptical about its feasibility: not that because the pictures might not be &quot;fully multispectral&quot;, but the satellites might not get the required bands for an approximate Landsat reconstruction (and there is not enough information given per pixel here).</p>\n\n<p>Online pictures of Landsat are <a href=\"http://earthexplorer.usgs.gov/\">free</a>, but it gets out of control if we have to position geolocalize each picture and harvest them, having the need to grab the timestamp also.</p>\n\n<p>Note: I have not tested the .tiff files. Too big to fit on my drive space left.</p>\n\n<p>Edit: added ton of external links related to feature extraction related to this topic because it would be confusing for people who are not used to the terms used</p>",
      "rawMarkdown": "See the attached files to understand or if you are interested to the results of what I did to the first set of the data set (data is augmented 80X times, making 28000 pictures for training and 109600 for testing):\r\n\r\n* Montage.jpg => a high quality montage of the first set in the train set (15MB, so big sized a browser can't handle it properly?? 3840x46080)\r\n\r\n* Montage_Small.jpg => a low quality montage of the first set in the train set (4MB, 1280x15360)\r\n\r\n* Aligned.gif => a high quality sequence of the first set in the train set (12MB)\r\n\r\n* Aligned_Small.gif => a low quality montage of the first set in the train set (4MB)\r\n\r\nI computed some [local gradient orientation][1] statistics using a 5x5 [Sobel filter][2]... I might be wrong but the blue channel is clearly horrible to use as a predicting feature? (local gradient orientation gave me no goodness for that blue channel.. well, mostly under 0.20 => the [Gaussian directionality][3] of the structure of the channel is barely predictable, although researches found the goodness value underestimates the reality). I found also there are angles where the predictability rises up to 1.00, which is not normal? However, the what is nice to see is the directionality and the structure using the transformation I used are kept (see [image gradient][4]).\r\n\r\nThe color wheel of angles of the first set of the train set:\r\n\r\n![enter image description here][5]\r\n\r\nAnd the color wheels of the first set by image order, which are exactly identical pixel to pixel (clearly unexpected, check yourself if you want using a substraction method):\r\n\r\n![enter image description here][6]\r\n\r\nI assume the satellites used for the imagery are [multispectral][7] and capturing ultra-blue wavelengths (which is re-mapped to false color composite), and this would go against saying the blue channel is horrible? As far as I know, multispectral images are useful in the sense they capture information that cannot be visually seen, and are highly sensitive to the soil content (if the soil changes, the channel changes).\r\n\r\nIf it is possible to infer the wavelength of colors, then it would feasible to reconstruct the [Landsat bands][8] and have a good reconstruction of the features in the images (especially as we are looking in changes in the pictures). For instance, we would be able to optimize vegetation/forest pictures by finding the lowest vegetation area of the pictures and setting it as baseline to use a ML technique to infer whether a specific vegetation should come before or after the baseline (much more easily! like the set with nearly forest only).\r\n\r\nHowever, I'm still skeptical about its feasibility: not that because the pictures might not be \"fully multispectral\", but the satellites might not get the required bands for an approximate Landsat reconstruction (and there is not enough information given per pixel here).\r\n\r\nOnline pictures of Landsat are [free][9], but it gets out of control if we have to position geolocalize each picture and harvest them, having the need to grab the timestamp also.\r\n\r\nNote: I have not tested the .tiff files. Too big to fit on my drive space left.\r\n\r\nEdit: added ton of external links related to feature extraction related to this topic because it would be confusing for people who are not used to the terms used\r\n\r\n\r\n  [1]: https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients\r\n  [2]: https://en.wikipedia.org/wiki/Sobel_operator\r\n  [3]: https://en.wikipedia.org/wiki/Directional_statistics\r\n  [4]: https://en.wikipedia.org/wiki/Image_gradient\r\n  [5]: http://i.imgur.com/2pDewRF.jpg\r\n  [6]: http://i.imgur.com/3yEFzeT.jpg\r\n  [7]: https://en.wikipedia.org/wiki/Multispectral_image\r\n  [8]: http://landsat.usgs.gov/best_spectral_bands_to_use.php\r\n  [9]: http://earthexplorer.usgs.gov/",
      "votes": null
    },
    {
      "id": "121210",
      "postDate": "05/24/2016 22:42:15",
      "content": "<p>My first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!</p>",
      "rawMarkdown": "My first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!",
      "votes": null
    },
    {
      "id": "121281",
      "postDate": "05/25/2016 10:11:23",
      "content": "<p>[quote=Tyler Vigen;121210]</p>\n\n<p>My first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!</p>\n\n<p>[/quote]</p>\n\n<p>Yep, that was my thought too. Reversing 3 bands into all the required bands include loss of information from squashed information + extrapolation of missing information (3x more at least), which would be very difficult.</p>\n\n<p>I tried yesterday to look at its feasibility given RGB and a Landsat example... definitely not something I will try!</p>\n\n<p>Do you know if there are artifact patterns when converting from a multispectral image to a false-color composite image? Perhaps an interpolated signature effect could be picked out from it, and subtracting two signatures (depending on the interpolated signature used) we could quantify the spectral difference between two pictures in a set.</p>",
      "rawMarkdown": "[quote=Tyler Vigen;121210]\r\n\r\nMy first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!\r\n\r\n[/quote]\r\n\r\nYep, that was my thought too. Reversing 3 bands into all the required bands include loss of information from squashed information + extrapolation of missing information (3x more at least), which would be very difficult.\r\n\r\nI tried yesterday to look at its feasibility given RGB and a Landsat example... definitely not something I will try!\r\n\r\nDo you know if there are artifact patterns when converting from a multispectral image to a false-color composite image? Perhaps an interpolated signature effect could be picked out from it, and subtracting two signatures (depending on the interpolated signature used) we could quantify the spectral difference between two pictures in a set.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 121210,
      "author_name": "tylervigen",
      "author_url": "",
      "post_date": "05/24/2016 22:42:15",
      "content": "<p>My first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 121281,
      "author_name": "laurae2",
      "author_url": "",
      "post_date": "05/25/2016 10:11:23",
      "content": "<p>[quote=Tyler Vigen;121210]</p>\n\n<p>My first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!</p>\n\n<p>[/quote]</p>\n\n<p>Yep, that was my thought too. Reversing 3 bands into all the required bands include loss of information from squashed information + extrapolation of missing information (3x more at least), which would be very difficult.</p>\n\n<p>I tried yesterday to look at its feasibility given RGB and a Landsat example... definitely not something I will try!</p>\n\n<p>Do you know if there are artifact patterns when converting from a multispectral image to a false-color composite image? Perhaps an interpolated signature effect could be picked out from it, and subtracting two signatures (depending on the interpolated signature used) we could quantify the spectral difference between two pictures in a set.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "121102": "See the attached files to understand or if you are interested to the results of what I did to the first set of the data set (data is augmented 80X times, making 28000 pictures for training and 109600 for testing):\r\n\r\n* Montage.jpg => a high quality montage of the first set in the train set (15MB, so big sized a browser can't handle it properly?? 3840x46080)\r\n\r\n* Montage_Small.jpg => a low quality montage of the first set in the train set (4MB, 1280x15360)\r\n\r\n* Aligned.gif => a high quality sequence of the first set in the train set (12MB)\r\n\r\n* Aligned_Small.gif => a low quality montage of the first set in the train set (4MB)\r\n\r\nI computed some [local gradient orientation][1] statistics using a 5x5 [Sobel filter][2]... I might be wrong but the blue channel is clearly horrible to use as a predicting feature? (local gradient orientation gave me no goodness for that blue channel.. well, mostly under 0.20 => the [Gaussian directionality][3] of the structure of the channel is barely predictable, although researches found the goodness value underestimates the reality). I found also there are angles where the predictability rises up to 1.00, which is not normal? However, the what is nice to see is the directionality and the structure using the transformation I used are kept (see [image gradient][4]).\r\n\r\nThe color wheel of angles of the first set of the train set:\r\n\r\n![enter image description here][5]\r\n\r\nAnd the color wheels of the first set by image order, which are exactly identical pixel to pixel (clearly unexpected, check yourself if you want using a substraction method):\r\n\r\n![enter image description here][6]\r\n\r\nI assume the satellites used for the imagery are [multispectral][7] and capturing ultra-blue wavelengths (which is re-mapped to false color composite), and this would go against saying the blue channel is horrible? As far as I know, multispectral images are useful in the sense they capture information that cannot be visually seen, and are highly sensitive to the soil content (if the soil changes, the channel changes).\r\n\r\nIf it is possible to infer the wavelength of colors, then it would feasible to reconstruct the [Landsat bands][8] and have a good reconstruction of the features in the images (especially as we are looking in changes in the pictures). For instance, we would be able to optimize vegetation/forest pictures by finding the lowest vegetation area of the pictures and setting it as baseline to use a ML technique to infer whether a specific vegetation should come before or after the baseline (much more easily! like the set with nearly forest only).\r\n\r\nHowever, I'm still skeptical about its feasibility: not that because the pictures might not be \"fully multispectral\", but the satellites might not get the required bands for an approximate Landsat reconstruction (and there is not enough information given per pixel here).\r\n\r\nOnline pictures of Landsat are [free][9], but it gets out of control if we have to position geolocalize each picture and harvest them, having the need to grab the timestamp also.\r\n\r\nNote: I have not tested the .tiff files. Too big to fit on my drive space left.\r\n\r\nEdit: added ton of external links related to feature extraction related to this topic because it would be confusing for people who are not used to the terms used\r\n\r\n\r\n  [1]: https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients\r\n  [2]: https://en.wikipedia.org/wiki/Sobel_operator\r\n  [3]: https://en.wikipedia.org/wiki/Directional_statistics\r\n  [4]: https://en.wikipedia.org/wiki/Image_gradient\r\n  [5]: http://i.imgur.com/2pDewRF.jpg\r\n  [6]: http://i.imgur.com/3yEFzeT.jpg\r\n  [7]: https://en.wikipedia.org/wiki/Multispectral_image\r\n  [8]: http://landsat.usgs.gov/best_spectral_bands_to_use.php\r\n  [9]: http://earthexplorer.usgs.gov/",
    "121210": "My first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!",
    "121281": "[quote=Tyler Vigen;121210]\r\n\r\nMy first thought is that you're going to get too much noise from the ~470mm region to perform an ultrablue spectral analysis on a three band image.  Given the lack of specific banding, it would be difficult to correctly correlate with Ladsat imagery.  But it's an interesting concept nonetheless!\r\n\r\n[/quote]\r\n\r\nYep, that was my thought too. Reversing 3 bands into all the required bands include loss of information from squashed information + extrapolation of missing information (3x more at least), which would be very difficult.\r\n\r\nI tried yesterday to look at its feasibility given RGB and a Landsat example... definitely not something I will try!\r\n\r\nDo you know if there are artifact patterns when converting from a multispectral image to a false-color composite image? Perhaps an interpolated signature effect could be picked out from it, and subtracting two signatures (depending on the interpolated signature used) we could quantify the spectral difference between two pictures in a set."
  },
  "source": "meta"
}