Hello,
\nCan someone help answer the following questions for me?
\n1) The .npy files contain data for the LIGO Hanford, LIGO Livingston, and Virgo detectors. However, it is unclear to me which part of the data belongs to which detector. Let me clarify what I mean using an example.
\nWhen I read the file test/0/0/0/0007eb08fa.npy using the function numpy.load()
\ndata = np.load('./test/0/0/0/0007eb08fa.npy')
I get the following numpy.ndarray
\n[[-4.25382702e-21 -4.89490056e-21 -5.62383974e-21 ... -1.53634610e-20\n -1.56561020e-20 -1.59835683e-20]\n [ 1.45560958e-20 1.44871144e-20 1.38093953e-20 ... 6.62847167e-21\n 6.50760673e-21 5.99178397e-21]\n [-6.08799976e-22 -6.72646229e-22 -3.98347670e-22 ... -1.93071457e-21\n -1.62630586e-21 -1.98174008e-21]]\nThis numpy.ndarray contains three elements, where each element is another numpy.ndarray of 4096 elements containing the strain as measured by a given detector.
\nI'd like to know which detector corresponds to which element. So far, I assume the first element (containing the first numpy.ndarray of 4096 elements) corresponds to LIGO Hanford, the second element (containing the second numpy.ndarray of 4096 elements) to LIGO Livingston, and the third element (containing the third numpy.ndarray of 4096 elements) to Virgo, but I have not been able to verify that this is correct. Can someone please clarify this?
\n2) In this example https://gwpy.github.io/docs/stable/examples/signal/gw150914.html
\nfor the LIGO detectors notch filters are created for the first three harmonics of the 60 Hz AC mains power. The LIGO Hanford and Livingston detectors are US based, so 60 Hz seems logical to use. However, Virgo is located in Italy, which has a 50 Hz electricity grid frequency. I will therefore assume 60 Hz is not applicable for the Virgo detector and 50 Hz must be used instead. Is this correct, or am I overlooking something here?
Cheers,
\nRien.
", "messages": [ { "id": "1451780", "postDate": "08/05/2021 12:35:51", "content": "Hello,
\nCan someone help answer the following questions for me?
\n1) The .npy files contain data for the LIGO Hanford, LIGO Livingston, and Virgo detectors. However, it is unclear to me which part of the data belongs to which detector. Let me clarify what I mean using an example.
\nWhen I read the file test/0/0/0/0007eb08fa.npy using the function numpy.load()
\ndata = np.load('./test/0/0/0/0007eb08fa.npy')
I get the following numpy.ndarray
\n[[-4.25382702e-21 -4.89490056e-21 -5.62383974e-21 ... -1.53634610e-20\n -1.56561020e-20 -1.59835683e-20]\n [ 1.45560958e-20 1.44871144e-20 1.38093953e-20 ... 6.62847167e-21\n 6.50760673e-21 5.99178397e-21]\n [-6.08799976e-22 -6.72646229e-22 -3.98347670e-22 ... -1.93071457e-21\n -1.62630586e-21 -1.98174008e-21]]\nThis numpy.ndarray contains three elements, where each element is another numpy.ndarray of 4096 elements containing the strain as measured by a given detector.
\nI'd like to know which detector corresponds to which element. So far, I assume the first element (containing the first numpy.ndarray of 4096 elements) corresponds to LIGO Hanford, the second element (containing the second numpy.ndarray of 4096 elements) to LIGO Livingston, and the third element (containing the third numpy.ndarray of 4096 elements) to Virgo, but I have not been able to verify that this is correct. Can someone please clarify this?
\n2) In this example https://gwpy.github.io/docs/stable/examples/signal/gw150914.html
\nfor the LIGO detectors notch filters are created for the first three harmonics of the 60 Hz AC mains power. The LIGO Hanford and Livingston detectors are US based, so 60 Hz seems logical to use. However, Virgo is located in Italy, which has a 50 Hz electricity grid frequency. I will therefore assume 60 Hz is not applicable for the Virgo detector and 50 Hz must be used instead. Is this correct, or am I overlooking something here?
Cheers,
\nRien.
", "rawMarkdown": "Hello,\n\nCan someone help answer the following questions for me?\n\n1) The .npy files contain data for the LIGO Hanford, LIGO Livingston, and Virgo detectors. However, it is unclear to me which part of the data belongs to which detector. Let me clarify what I mean using an example.\n\nWhen I read the file test/0/0/0/0007eb08fa.npy using the function numpy.load() \n\n`data = np.load('./test/0/0/0/0007eb08fa.npy')`\n\nI get the following numpy.ndarray\n\n```\n[[-4.25382702e-21 -4.89490056e-21 -5.62383974e-21 ... -1.53634610e-20\n -1.56561020e-20 -1.59835683e-20]\n [ 1.45560958e-20 1.44871144e-20 1.38093953e-20 ... 6.62847167e-21\n 6.50760673e-21 5.99178397e-21]\n [-6.08799976e-22 -6.72646229e-22 -3.98347670e-22 ... -1.93071457e-21\n -1.62630586e-21 -1.98174008e-21]]\n```\n\nThis numpy.ndarray contains three elements, where each element is another numpy.ndarray of 4096 elements containing the strain as measured by a given detector. \n\nI'd like to know which detector corresponds to which element. So far, I assume the first element (containing the first numpy.ndarray of 4096 elements) corresponds to LIGO Hanford, the second element (containing the second numpy.ndarray of 4096 elements) to LIGO Livingston, and the third element (containing the third numpy.ndarray of 4096 elements) to Virgo, but I have not been able to verify that this is correct. Can someone please clarify this?\n\n2) In this example https://gwpy.github.io/docs/stable/examples/signal/gw150914.html\nfor the LIGO detectors notch filters are created for the first three harmonics of the 60 Hz AC mains power. The LIGO Hanford and Livingston detectors are US based, so 60 Hz seems logical to use. However, Virgo is located in Italy, which has a 50 Hz electricity grid frequency. I will therefore assume 60 Hz is not applicable for the Virgo detector and 50 Hz must be used instead. Is this correct, or am I overlooking something here?\n\nCheers,\n\nRien.", "votes": null }, { "id": "1452674", "postDate": "08/05/2021 17:02:21", "content": "For question 1, I think the arrays are ordered. Even if they aren't, this shouldn't be an issue I guess. If it matters to you, you can try to plot an histogram for each considering they are sorted. If you see some differences, it could mean that they are sorted, otherwise it is impossible to tell.
\nFor question 2, that's a good point indeed. Do you have further details about how the data has been prepared?
", "rawMarkdown": "For **question 1**, I think the arrays are ordered. Even if they aren't, this shouldn't be an issue I guess. If it matters to you, you can try to plot an histogram for each considering they are sorted. If you see some differences, it could mean that they are sorted, otherwise it is impossible to tell.\n\nFor **question 2**, that's a good point indeed. Do you have further details about how the data has been prepared?", "votes": null }, { "id": "1452957", "postDate": "08/05/2021 18:16:38", "content": "data not taken from detectors - they are synthetic (as far as I understand)
", "rawMarkdown": "data not taken from detectors - they are synthetic (as far as I understand)", "votes": null }, { "id": "1452967", "postDate": "08/05/2021 18:18:53", "content": "Data is synthetic. We can probably assume they did not add the 60 Hz or 50 Hz noise you discuss.
", "rawMarkdown": "Data is synthetic. We can probably assume they did not add the 60 Hz or 50 Hz noise you discuss.", "votes": null }, { "id": "1455001", "postDate": "08/06/2021 12:14:47", "content": "Thank you all for your answers 😊.
\n@yassinealouini, for me questions 1 and 2 are related, because if different frequencies (either 50 Hz or 60 Hz) do need to be applied, then I need to know which data corresponds to which detector as well, so I can use the appropriate frequency for each detector (50 Hz for Virgo and 60 for Hanford and Livingston). Therefore I was interested in which data array belongs to which detector.
", "rawMarkdown": "Thank you all for your answers 😊.\n\n@yassinealouini, for me questions 1 and 2 are related, because if different frequencies (either 50 Hz or 60 Hz) do need to be applied, then I need to know which data corresponds to which detector as well, so I can use the appropriate frequency for each detector (50 Hz for Virgo and 60 for Hanford and Livingston). Therefore I was interested in which data array belongs to which detector.", "votes": null }, { "id": "1455055", "postDate": "08/06/2021 12:35:39", "content": "Yes, I get your point. 👌 As mentioned by @cpmpml, since data is synthetic, maybe they didn't bother with these different frequencies.
", "rawMarkdown": "Yes, I get your point. 👌 As mentioned by @cpmpml, since data is synthetic, maybe they didn't bother with these different frequencies.", "votes": null }, { "id": "1455296", "postDate": "08/06/2021 14:14:46", "content": "For question ! first sentence of data description is:
\n\n\nIn this competition you are provided with a training set of time series data containing simulated gravitational wave measurements from a network of 3 gravitational wave interferometers (LIGO Hanford, LIGO Livingston, and Virgo)
\n
I assumed that the order is the same in the files: data[0] is Hanford, etc.
", "rawMarkdown": "For question ! first sentence of data description is:\n\n> In this competition you are provided with a training set of time series data containing simulated gravitational wave measurements from a network of 3 gravitational wave interferometers (LIGO Hanford, LIGO Livingston, and Virgo)\n\nI assumed that the order is the same in the files: data[0] is Hanford, etc.", "votes": null }, { "id": "1455441", "postDate": "08/06/2021 15:10:32", "content": "Indeed, based on that sentence that was my initial assumption as well. I'll stick with that for now then.
", "rawMarkdown": "Indeed, based on that sentence that was my initial assumption as well. I'll stick with that for now then.", "votes": null }, { "id": "1455459", "postDate": "08/06/2021 15:17:45", "content": "Indeed, may be they didn't bother with these frequencies after all then. As an experiment, I think I'm gonna try the analysis both with and without the notch filters and see whether it makes a lot of difference.
", "rawMarkdown": "Indeed, may be they didn't bother with these frequencies after all then. As an experiment, I think I'm gonna try the analysis both with and without the notch filters and see whether it makes a lot of difference.", "votes": null }, { "id": "1460518", "postDate": "08/08/2021 23:38:31", "content": "The data description also says
\n\n\nEach time series contains either detector noise or detector noise plus a simulated gravitational wave signal.
\n
I read this as the noise is real noise from the detector and it is only the GW signal that is simulated.
", "rawMarkdown": "The data description also says \n> Each time series contains either detector noise or detector noise plus a simulated gravitational wave signal. \n\nI read this as the noise is real noise from the detector and it is only the GW signal that is simulated.", "votes": null }, { "id": "1460748", "postDate": "08/09/2021 04:08:26", "content": "No, noise is simulated too
\n\n", "rawMarkdown": "No, noise is simulated [too](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/255814#1427815)\n\n> Hi folks, I can confirm the detector noise is simulated rather than real data. Whilst using real data would have presented more of a challenge, we would then be limited to using public data which would present its own series of difficulties.", "votes": null }, { "id": "1561203", "postDate": "10/27/2021 12:19:25", "content": "Hi folks, I can confirm the detector noise is simulated rather than real data. Whilst using real data would have presented more of a challenge, we would then be limited to using public data which would present its own series of difficulties.
\n
Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
For question 1, I think the arrays are ordered. Even if they aren't, this shouldn't be an issue I guess. If it matters to you, you can try to plot an histogram for each considering they are sorted. If you see some differences, it could mean that they are sorted, otherwise it is impossible to tell.
\nFor question 2, that's a good point indeed. Do you have further details about how the data has been prepared?
", "votes": null, "replies": [] }, { "id": 1452957, "author_name": "sapr3s", "author_url": "", "post_date": "08/05/2021 18:16:38", "content": "data not taken from detectors - they are synthetic (as far as I understand)
", "votes": null, "replies": [] }, { "id": 1452967, "author_name": "cpmpml", "author_url": "", "post_date": "08/05/2021 18:18:53", "content": "Data is synthetic. We can probably assume they did not add the 60 Hz or 50 Hz noise you discuss.
", "votes": null, "replies": [] }, { "id": 1455001, "author_name": "catharinusdykstra", "author_url": "", "post_date": "08/06/2021 12:14:47", "content": "Thank you all for your answers 😊.
\n@yassinealouini, for me questions 1 and 2 are related, because if different frequencies (either 50 Hz or 60 Hz) do need to be applied, then I need to know which data corresponds to which detector as well, so I can use the appropriate frequency for each detector (50 Hz for Virgo and 60 for Hanford and Livingston). Therefore I was interested in which data array belongs to which detector.
", "votes": null, "replies": [ { "id": 1455055, "author_name": "yassinealouini", "author_url": "", "post_date": "08/06/2021 12:35:39", "content": "Yes, I get your point. 👌 As mentioned by @cpmpml, since data is synthetic, maybe they didn't bother with these different frequencies.
", "votes": null, "replies": [] }, { "id": 1455459, "author_name": "catharinusdykstra", "author_url": "", "post_date": "08/06/2021 15:17:45", "content": "Indeed, may be they didn't bother with these frequencies after all then. As an experiment, I think I'm gonna try the analysis both with and without the notch filters and see whether it makes a lot of difference.
", "votes": null, "replies": [] } ] }, { "id": 1455296, "author_name": "cpmpml", "author_url": "", "post_date": "08/06/2021 14:14:46", "content": "For question ! first sentence of data description is:
\n\n\nIn this competition you are provided with a training set of time series data containing simulated gravitational wave measurements from a network of 3 gravitational wave interferometers (LIGO Hanford, LIGO Livingston, and Virgo)
\n
I assumed that the order is the same in the files: data[0] is Hanford, etc.
", "votes": null, "replies": [ { "id": 1455441, "author_name": "catharinusdykstra", "author_url": "", "post_date": "08/06/2021 15:10:32", "content": "Indeed, based on that sentence that was my initial assumption as well. I'll stick with that for now then.
", "votes": null, "replies": [] } ] }, { "id": 1460518, "author_name": "kevinmcisaac", "author_url": "", "post_date": "08/08/2021 23:38:31", "content": "The data description also says
\n\n\nEach time series contains either detector noise or detector noise plus a simulated gravitational wave signal.
\n
I read this as the noise is real noise from the detector and it is only the GW signal that is simulated.
", "votes": null, "replies": [ { "id": 1460748, "author_name": "sapr3s", "author_url": "", "post_date": "08/09/2021 04:08:26", "content": "No, noise is simulated too
\n\n", "votes": null, "replies": [] } ] }, { "id": 1561203, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 12:19:25", "content": "Hi folks, I can confirm the detector noise is simulated rather than real data. Whilst using real data would have presented more of a challenge, we would then be limited to using public data which would present its own series of difficulties.
\n
Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris