From the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.
\nI'm just wondering why these parameters are not provided?
\nFirst reaction I have is, this could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so.
\nSecond thought I have is maybe to prevent people generate more gw data for training, but i would imagine that would be beneficial and better for production,
\nLast thought is, perhaps this is to discourage template based method? I'm not very familiar with matched-filtering approach but seems like you could first come up with several wave template with likely shape (given a set of parameters) and simply slide over the waveform, and if theres a high overlap then one may say there is a detection. So I guess if we were given the set of 15 parameters used, then we could somehow get their corresponding gw waveform, and then simply slide over to get prediction, and that's not what the team wanted.
", "messages": [ { "id": "1381055", "postDate": "07/08/2021 15:24:29", "content": "From the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.
\nI'm just wondering why these parameters are not provided?
\nFirst reaction I have is, this could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so.
\nSecond thought I have is maybe to prevent people generate more gw data for training, but i would imagine that would be beneficial and better for production,
\nLast thought is, perhaps this is to discourage template based method? I'm not very familiar with matched-filtering approach but seems like you could first come up with several wave template with likely shape (given a set of parameters) and simply slide over the waveform, and if theres a high overlap then one may say there is a detection. So I guess if we were given the set of 15 parameters used, then we could somehow get their corresponding gw waveform, and then simply slide over to get prediction, and that's not what the team wanted.
", "rawMarkdown": "From the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.\n\nI'm just wondering why these parameters are not provided? \n\nFirst reaction I have is, this could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so.\n\nSecond thought I have is maybe to prevent people generate more gw data for training, but i would imagine that would be beneficial and better for production,\n\nLast thought is, perhaps this is to discourage template based method? I'm not very familiar with matched-filtering approach but seems like you could first come up with several wave template with likely shape (given a set of parameters) and simply slide over the waveform, and if theres a high overlap then one may say there is a detection. So I guess if we were given the set of 15 parameters used, then we could somehow get their corresponding gw waveform, and then simply slide over to get prediction, and that's not what the team wanted.", "votes": null }, { "id": "1381215", "postDate": "07/08/2021 18:04:44", "content": "Because they are not available when predicting on real data. Goal is to have models detect presence or absence of gravitational waves using waveforms. That's what we have as input data.
", "rawMarkdown": "Because they are not available when predicting on real data. Goal is to have models detect presence or absence of gravitational waves using waveforms. That's what we have as input data.", "votes": null }, { "id": "1386091", "postDate": "07/13/2021 08:27:41", "content": "CPMP is completely correct. In reality we only have access to the data the universe gives us and it's our job to first determine if a signal is present, and then to determine the parameters (that will hopefully be a future challenge :). We never have access to the true signal parameters in practice.
", "rawMarkdown": "CPMP is completely correct. In reality we only have access to the data the universe gives us and it's our job to first determine if a signal is present, and then to determine the parameters (that will hopefully be a future challenge :). We never have access to the true signal parameters in practice.", "votes": null }, { "id": "1470070", "postDate": "08/13/2021 09:14:16", "content": "Hi samshipengs,
\n\n\nFrom the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.
\n
The exact meaning of these 15 parameters is described in this thread. Some of them are \"intrinsic\" parameters (i.e. related to the characteristics of the Binary Black Holes system itself ), the rest of them are \"extrinsic\" (i.e. related to the localization of the event with regard to the detectors' positions).
\n\n\nI'm just wondering why these parameters are not provided?
\n
As Chris and CPMP already mentioned it, simply because in real observations, these parameters are unknown (They are useful to create synthetic waveforms using numerical relativity models).
\nThere's actually quite a bit of research that aims at retrieving these parameters for a given event using Bayesian inference. However, this approach is usually very computationally intensive and takes too much time to be applied in real time on real data.
\n\nthis could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so
\n
Well… Assuming we'd be able to retrieve for each detector the time of coalescence Tc, the right ascension α and declination δ and given the location of the detectors on earth, it would be possible to check if an hypothetical event from one detector is also found with the correct time delay in the other two detectors (since we could compute the expected time delays). Even for weak/unclear signals, a coherent match across the 3 detectors at the expected Tc would be a powerful hint that a merger indeed occurred in the data.
\n\n\nSecond thought I have is maybe to prevent people generate more gw data for training
\n
I think there are plenty of codes around that allow you to generate as many \"theoretical\" synthetic waveforms as you want (c.f. GGWD) that you can inject in real or synthetic background noise with a given SNR. (I guess that it's actually what the competition hosts did to create the datasets in the first place).
", "rawMarkdown": "Hi samshipengs,\n\n> From the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.\n\nThe exact meaning of these 15 parameters is described in [this thread](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/252131). Some of them are \"intrinsic\" parameters (i.e. related to the characteristics of the Binary Black Holes system itself ), the rest of them are \"extrinsic\" (i.e. related to the localization of the event with regard to the detectors' positions).\n\n> I'm just wondering why these parameters are not provided?\n\nAs Chris and CPMP already mentioned it, simply because in real observations, these parameters are unknown (They are useful to create synthetic waveforms using numerical relativity models).\nThere's actually quite a bit of research that aims at retrieving these parameters for a given event using Bayesian inference. However, this approach is usually very computationally intensive and takes too much time to be applied in real time on real data.\n\n> this could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so\n\nWell... Assuming we'd be able to retrieve for each detector the time of coalescence Tc, the right ascension α and declination δ and given the location of the detectors on earth, it would be possible to check if an hypothetical event from one detector is also found with the correct time delay in the other two detectors (since we could compute the expected time delays). Even for weak/unclear signals, a coherent match across the 3 detectors at the expected Tc would be a powerful hint that a merger indeed occurred in the data.\n\n> Second thought I have is maybe to prevent people generate more gw data for training\n\nI think there are plenty of codes around that allow you to generate as many \"theoretical\" synthetic waveforms as you want (c.f. [GGWD](https://github.com/timothygebhard/ggwd)) that you can inject in real or synthetic background noise with a given SNR. (I guess that it's actually what the competition hosts did to create the datasets in the first place).", "votes": null } ], "comments": [ { "id": 1381215, "author_name": "cpmpml", "author_url": "", "post_date": "07/08/2021 18:04:44", "content": "Because they are not available when predicting on real data. Goal is to have models detect presence or absence of gravitational waves using waveforms. That's what we have as input data.
", "votes": null, "replies": [] }, { "id": 1386091, "author_name": "bayeswolf", "author_url": "", "post_date": "07/13/2021 08:27:41", "content": "CPMP is completely correct. In reality we only have access to the data the universe gives us and it's our job to first determine if a signal is present, and then to determine the parameters (that will hopefully be a future challenge :). We never have access to the true signal parameters in practice.
", "votes": null, "replies": [] }, { "id": 1470070, "author_name": "thierrydaubos", "author_url": "", "post_date": "08/13/2021 09:14:16", "content": "Hi samshipengs,
\n\n\nFrom the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.
\n
The exact meaning of these 15 parameters is described in this thread. Some of them are \"intrinsic\" parameters (i.e. related to the characteristics of the Binary Black Holes system itself ), the rest of them are \"extrinsic\" (i.e. related to the localization of the event with regard to the detectors' positions).
\n\n\nI'm just wondering why these parameters are not provided?
\n
As Chris and CPMP already mentioned it, simply because in real observations, these parameters are unknown (They are useful to create synthetic waveforms using numerical relativity models).
\nThere's actually quite a bit of research that aims at retrieving these parameters for a given event using Bayesian inference. However, this approach is usually very computationally intensive and takes too much time to be applied in real time on real data.
\n\nthis could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so
\n
Well… Assuming we'd be able to retrieve for each detector the time of coalescence Tc, the right ascension α and declination δ and given the location of the detectors on earth, it would be possible to check if an hypothetical event from one detector is also found with the correct time delay in the other two detectors (since we could compute the expected time delays). Even for weak/unclear signals, a coherent match across the 3 detectors at the expected Tc would be a powerful hint that a merger indeed occurred in the data.
\n\n\nSecond thought I have is maybe to prevent people generate more gw data for training
\n
I think there are plenty of codes around that allow you to generate as many \"theoretical\" synthetic waveforms as you want (c.f. GGWD) that you can inject in real or synthetic background noise with a given SNR. (I guess that it's actually what the competition hosts did to create the datasets in the first place).
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1381055": "From the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.\n\nI'm just wondering why these parameters are not provided? \n\nFirst reaction I have is, this could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so.\n\nSecond thought I have is maybe to prevent people generate more gw data for training, but i would imagine that would be beneficial and better for production,\n\nLast thought is, perhaps this is to discourage template based method? I'm not very familiar with matched-filtering approach but seems like you could first come up with several wave template with likely shape (given a set of parameters) and simply slide over the waveform, and if theres a high overlap then one may say there is a detection. So I guess if we were given the set of 15 parameters used, then we could somehow get their corresponding gw waveform, and then simply slide over to get prediction, and that's not what the team wanted.", "1381215": "Because they are not available when predicting on real data. Goal is to have models detect presence or absence of gravitational waves using waveforms. That's what we have as input data.", "1386091": "CPMP is completely correct. In reality we only have access to the data the universe gives us and it's our job to first determine if a signal is present, and then to determine the parameters (that will hopefully be a future challenge :). We never have access to the true signal parameters in practice.", "1470070": "Hi samshipengs,\n\n> From the data page, seems like there are total 15 parameters used to generate the simulated gravitational wave (I assume these are the parameters required to solve Einstein's equation and hence obtain the wave plot?), and together with real noise from detector added we got the training data.\n\nThe exact meaning of these 15 parameters is described in [this thread](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/252131). Some of them are \"intrinsic\" parameters (i.e. related to the characteristics of the Binary Black Holes system itself ), the rest of them are \"extrinsic\" (i.e. related to the localization of the event with regard to the detectors' positions).\n\n> I'm just wondering why these parameters are not provided?\n\nAs Chris and CPMP already mentioned it, simply because in real observations, these parameters are unknown (They are useful to create synthetic waveforms using numerical relativity models).\nThere's actually quite a bit of research that aims at retrieving these parameters for a given event using Bayesian inference. However, this approach is usually very computationally intensive and takes too much time to be applied in real time on real data.\n\n> this could somehow help us to predict test set in ways that's not useful in real production, but it's unclear to me how one could use this information to do so\n\nWell... Assuming we'd be able to retrieve for each detector the time of coalescence Tc, the right ascension α and declination δ and given the location of the detectors on earth, it would be possible to check if an hypothetical event from one detector is also found with the correct time delay in the other two detectors (since we could compute the expected time delays). Even for weak/unclear signals, a coherent match across the 3 detectors at the expected Tc would be a powerful hint that a merger indeed occurred in the data.\n\n> Second thought I have is maybe to prevent people generate more gw data for training\n\nI think there are plenty of codes around that allow you to generate as many \"theoretical\" synthetic waveforms as you want (c.f. [GGWD](https://github.com/timothygebhard/ggwd)) that you can inject in real or synthetic background noise with a given SNR. (I guess that it's actually what the competition hosts did to create the datasets in the first place)." }, "source": "meta" }