{
  "id": 250226,
  "title": "Anything wrong with the data?",
  "url": "/competitions/g2net-gravitational-wave-detection/discussion/250226",
  "author_name": "",
  "post_date": "2021-07-01T17:26:01.496847500Z",
  "votes": 5,
  "comment_count": 9,
  "views": 0,
  "content": "<p>If you do web-scraping you'll see this sort of image for <strong>Gravitational-Wave</strong> but unfortunately, I've not been able to produce this image using <strong>Spectrogram</strong>. Even the <strong>Overview</strong> tab in the competition website has a similar image. I'm wondering if the data is okay,<br>\n<img src=\"https://newscenter.lbl.gov/wp-content/uploads/sites/2/2017/10/ligo20160211v2_Tn.jpg\"><br>\n<img src=\"https://storage.googleapis.com/kaggle-media/competitions/G2Net-gravitational-waves/800px-LIGO_measurement_of_gravitational_waves.svg.png\"></p>",
  "messages": [
    {
      "id": "1372424",
      "postDate": "07/01/2021 17:26:01",
      "content": "<p>If you do web-scraping you'll see this sort of image for <strong>Gravitational-Wave</strong> but unfortunately, I've not been able to produce this image using <strong>Spectrogram</strong>. Even the <strong>Overview</strong> tab in the competition website has a similar image. I'm wondering if the data is okay,<br>\n<img src=\"https://newscenter.lbl.gov/wp-content/uploads/sites/2/2017/10/ligo20160211v2_Tn.jpg\"><br>\n<img src=\"https://storage.googleapis.com/kaggle-media/competitions/G2Net-gravitational-waves/800px-LIGO_measurement_of_gravitational_waves.svg.png\"></p>",
      "rawMarkdown": "If you do web-scraping you'll see this sort of image for **Gravitational-Wave** but unfortunately, I've not been able to produce this image using **Spectrogram**. Even the **Overview** tab in the competition website has a similar image. I'm wondering if the data is okay,\n<img src=\"https://newscenter.lbl.gov/wp-content/uploads/sites/2/2017/10/ligo20160211v2_Tn.jpg\">\n<img src=\"https://storage.googleapis.com/kaggle-media/competitions/G2Net-gravitational-waves/800px-LIGO_measurement_of_gravitational_waves.svg.png\">",
      "votes": null
    },
    {
      "id": "1372483",
      "postDate": "07/01/2021 18:39:14",
      "content": "<p>I guess it might be data type. Kindly check if you are using some dtype conversion or any operation in float32 (convert that to float64).</p>",
      "rawMarkdown": "I guess it might be data type. Kindly check if you are using some dtype conversion or any operation in float32 (convert that to float64).",
      "votes": null
    },
    {
      "id": "1372557",
      "postDate": "07/01/2021 20:23:01",
      "content": "<p>The signals are well hidden in noise. Also proper preprocessing is required. Q-transform is used often to produce those images. More wavelet than fourier transform. Check out GWpy python package for dealing with these signals.</p>",
      "rawMarkdown": "The signals are well hidden in noise. Also proper preprocessing is required. Q-transform is used often to produce those images. More wavelet than fourier transform. Check out GWpy python package for dealing with these signals.",
      "votes": null
    },
    {
      "id": "1372561",
      "postDate": "07/01/2021 20:38:17",
      "content": "<p>I tried both <strong>float32</strong> &amp; <strong>float64</strong> same result.</p>",
      "rawMarkdown": "I tried both **float32** & **float64** same result.",
      "votes": null
    },
    {
      "id": "1372564",
      "postDate": "07/01/2021 20:40:04",
      "content": "<p>above pictures are actual <em>G wave</em>* so if training data is not like <strong>actual</strong> data then it may happen.</p>",
      "rawMarkdown": "above pictures are actual *G wave** so if training data is not like **actual** data then it may happen.",
      "votes": null
    },
    {
      "id": "1372886",
      "postDate": "07/02/2021 05:31:13",
      "content": "<p>The amplitude of the data set matches well with strain data at least (range 10e-21). We can assume the host wants to push the limits for what can be detected, so the obvious signals from papers are not to be expected here I think.</p>",
      "rawMarkdown": "The amplitude of the data set matches well with strain data at least (range 10e-21). We can assume the host wants to push the limits for what can be detected, so the obvious signals from papers are not to be expected here I think.",
      "votes": null
    },
    {
      "id": "1372989",
      "postDate": "07/02/2021 06:47:59",
      "content": "<p>I recommend you look at <a href=\"https://www.kaggle.com/alexnitz/pycbc-making-images\" target=\"_blank\">PyCBC: Making Images</a></p>",
      "rawMarkdown": "I recommend you look at [PyCBC: Making Images](https://www.kaggle.com/alexnitz/pycbc-making-images)",
      "votes": null
    },
    {
      "id": "1373511",
      "postDate": "07/02/2021 14:30:08",
      "content": "<p>I can assure you that the training data is quite realistic in terms of the amplitude. The images available online and those posted on the overview page are of real data after pre-processing such that the signals stand out. The first gravitational wave detection GW150914 was a particularly strong signal quite a lot louder than our standard detection threshold.</p>\n<p>Cheers, Chris</p>",
      "rawMarkdown": "I can assure you that the training data is quite realistic in terms of the amplitude. The images available online and those posted on the overview page are of real data after pre-processing such that the signals stand out. The first gravitational wave detection GW150914 was a particularly strong signal quite a lot louder than our standard detection threshold.\n\nCheers, Chris",
      "votes": null
    },
    {
      "id": "1563247",
      "postDate": "10/28/2021 06:51:21",
      "content": "<p>Hey,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
      "votes": null
    },
    {
      "id": "1563248",
      "postDate": "10/28/2021 06:51:25",
      "content": "<p>Hey,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1372483,
      "author_name": "micheomaano",
      "author_url": "",
      "post_date": "07/01/2021 18:39:14",
      "content": "<p>I guess it might be data type. Kindly check if you are using some dtype conversion or any operation in float32 (convert that to float64).</p>",
      "votes": null,
      "replies": [
        {
          "id": 1372561,
          "author_name": "awsaf49",
          "author_url": "",
          "post_date": "07/01/2021 20:38:17",
          "content": "<p>I tried both <strong>float32</strong> &amp; <strong>float64</strong> same result.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1372557,
      "author_name": "mistag",
      "author_url": "",
      "post_date": "07/01/2021 20:23:01",
      "content": "<p>The signals are well hidden in noise. Also proper preprocessing is required. Q-transform is used often to produce those images. More wavelet than fourier transform. Check out GWpy python package for dealing with these signals.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1372564,
          "author_name": "awsaf49",
          "author_url": "",
          "post_date": "07/01/2021 20:40:04",
          "content": "<p>above pictures are actual <em>G wave</em>* so if training data is not like <strong>actual</strong> data then it may happen.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1372886,
          "author_name": "mistag",
          "author_url": "",
          "post_date": "07/02/2021 05:31:13",
          "content": "<p>The amplitude of the data set matches well with strain data at least (range 10e-21). We can assume the host wants to push the limits for what can be detected, so the obvious signals from papers are not to be expected here I think.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1373511,
          "author_name": "bayeswolf",
          "author_url": "",
          "post_date": "07/02/2021 14:30:08",
          "content": "<p>I can assure you that the training data is quite realistic in terms of the amplitude. The images available online and those posted on the overview page are of real data after pre-processing such that the signals stand out. The first gravitational wave detection GW150914 was a particularly strong signal quite a lot louder than our standard detection threshold.</p>\n<p>Cheers, Chris</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1372989,
      "author_name": "miklgr500",
      "author_url": "",
      "post_date": "07/02/2021 06:47:59",
      "content": "<p>I recommend you look at <a href=\"https://www.kaggle.com/alexnitz/pycbc-making-images\" target=\"_blank\">PyCBC: Making Images</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1563247,
      "author_name": "zerafachris",
      "author_url": "",
      "post_date": "10/28/2021 06:51:21",
      "content": "<p>Hey,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1563248,
      "author_name": "zerafachris",
      "author_url": "",
      "post_date": "10/28/2021 06:51:25",
      "content": "<p>Hey,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1372424": "If you do web-scraping you'll see this sort of image for **Gravitational-Wave** but unfortunately, I've not been able to produce this image using **Spectrogram**. Even the **Overview** tab in the competition website has a similar image. I'm wondering if the data is okay,\n<img src=\"https://newscenter.lbl.gov/wp-content/uploads/sites/2/2017/10/ligo20160211v2_Tn.jpg\">\n<img src=\"https://storage.googleapis.com/kaggle-media/competitions/G2Net-gravitational-waves/800px-LIGO_measurement_of_gravitational_waves.svg.png\">",
    "1372483": "I guess it might be data type. Kindly check if you are using some dtype conversion or any operation in float32 (convert that to float64).",
    "1372557": "The signals are well hidden in noise. Also proper preprocessing is required. Q-transform is used often to produce those images. More wavelet than fourier transform. Check out GWpy python package for dealing with these signals.",
    "1372561": "I tried both **float32** & **float64** same result.",
    "1372564": "above pictures are actual *G wave** so if training data is not like **actual** data then it may happen.",
    "1372886": "The amplitude of the data set matches well with strain data at least (range 10e-21). We can assume the host wants to push the limits for what can be detected, so the obvious signals from papers are not to be expected here I think.",
    "1372989": "I recommend you look at [PyCBC: Making Images](https://www.kaggle.com/alexnitz/pycbc-making-images)",
    "1373511": "I can assure you that the training data is quite realistic in terms of the amplitude. The images available online and those posted on the overview page are of real data after pre-processing such that the signals stand out. The first gravitational wave detection GW150914 was a particularly strong signal quite a lot louder than our standard detection threshold.\n\nCheers, Chris",
    "1563247": "Hey,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
    "1563248": "Hey,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris"
  },
  "source": "meta"
}