{ "id": 242784, "title": "How does a neural network identify an image without a needle?", "url": "/competitions/seti-breakthrough-listen/discussion/242784", "author_name": "WOOSUNG YOON", "post_date": "2021-05-30T19:05:07.205000", "votes": 5, "comment_count": 2, "views": 0, "content": "

Question. I don't know how the neural network can identify an image without a needle.

\n
    \n

  1. Guess.
    \nMy guess is that neural networks should return negative values as default.
    \nF(x) < 0 for image x << 1.

    2. \n

  2. Thought experiment
    \n(1) Let's say we have a zero spectrogram image x (no noise).
    \nWithout bias the neural network should return 0.
    \nIn BceWithLogitsloss, the result value take a sigmoid and will be 0.5.

    3. \n
\n

(2) Let neural network model has a form of F(x) = a*(x-bias).
\nThe model F need to return negative value for no needle image.

\n

So, the model need to learn the bias that represent x distribution.
\nHowever the distribution of signal and noise are different.
\nWe may think image x = signal + noise + natural noise, then
\nF(x) = a*(signal + noise + natural noise - bias)

\n

Since, all three have different distribution,
\nthe model F need to learn bias that represent joint distribution.
\nAnd in simple model it has a clear limit to approximate it.

\n
    \n
  1. My experiment.
    \nIn my case, the problem is that the model learns the noise as a class 0.
    \nI want to the model give like this F(x + noise) = F(x).
    \nBut My model classify F(x + noise) = F(noise)
    2. \n
\n

I have two problem that detecting the needle image.

\n\n

I am trying to increase volume of data in this cases,
\nBut increasing the data for these cases raises another problem.

\n

It's too difficult for me.
\nI wonder if there is a way for the model to learn without knowing the image pattern of class 0.
\nAnd I don't actually understand how the neural network makes the classification.

", "messages": [ { "id": 1329020, "postDate": "2021-05-30T19:05:07.207Z", "content": "

Question. I don't know how the neural network can identify an image without a needle.

\n
    \n

  1. Guess.
    \nMy guess is that neural networks should return negative values as default.
    \nF(x) < 0 for image x << 1.

    2. \n

  2. Thought experiment
    \n(1) Let's say we have a zero spectrogram image x (no noise).
    \nWithout bias the neural network should return 0.
    \nIn BceWithLogitsloss, the result value take a sigmoid and will be 0.5.

    3. \n
\n

(2) Let neural network model has a form of F(x) = a*(x-bias).
\nThe model F need to return negative value for no needle image.

\n

So, the model need to learn the bias that represent x distribution.
\nHowever the distribution of signal and noise are different.
\nWe may think image x = signal + noise + natural noise, then
\nF(x) = a*(signal + noise + natural noise - bias)

\n

Since, all three have different distribution,
\nthe model F need to learn bias that represent joint distribution.
\nAnd in simple model it has a clear limit to approximate it.

\n
    \n
  1. My experiment.
    \nIn my case, the problem is that the model learns the noise as a class 0.
    \nI want to the model give like this F(x + noise) = F(x).
    \nBut My model classify F(x + noise) = F(noise)
    2. \n
\n

I have two problem that detecting the needle image.

\n\n

I am trying to increase volume of data in this cases,
\nBut increasing the data for these cases raises another problem.

\n

It's too difficult for me.
\nI wonder if there is a way for the model to learn without knowing the image pattern of class 0.
\nAnd I don't actually understand how the neural network makes the classification.

", "rawMarkdown": "Question. I don't know how the neural network can identify an image without a needle.\n\n1. Guess. \nMy guess is that neural networks should return negative values as default.\nF(x) < 0 for image x << 1.\n\n2. Thought experiment\n(1) Let's say we have a zero spectrogram image x (no noise).\nWithout bias the neural network should return 0. \nIn BceWithLogitsloss, the result value take a sigmoid and will be 0.5. \n\n(2) Let neural network model has a form of F(x) = a*(x-bias).\nThe model F need to return negative value for no needle image.\n\nSo, the model need to learn the bias that represent x distribution.\nHowever the distribution of signal and noise are different.\nWe may think image x = signal + noise + natural noise, then\nF(x) = a*(signal + noise + natural noise - bias)\n\nSince, all three have different distribution, \nthe model F need to learn bias that represent joint distribution.\nAnd in simple model it has a clear limit to approximate it.\n\n\n3. My experiment.\nIn my case, the problem is that the model learns the noise as a class 0.\nI want to the model give like this F(x + noise) = F(x). \nBut My model classify F(x + noise) = F(noise)\n\nI have two problem that detecting the needle image.\n- Weak signals \n- Signal overlapping noise.\nThe model classifies these as 0. \n\nI am trying to increase volume of data in this cases,\nBut increasing the data for these cases raises another problem.\n\nIt's too difficult for me.\nI wonder if there is a way for the model to learn without knowing the image pattern of class 0.\nAnd I don't actually understand how the neural network makes the classification.\n", "votes": 5 }, { "id": 1333137, "postDate": "2021-06-02T14:01:15.647Z", "content": "

If the samples with target = 0 were only noise, then you could reduce the noise using a \"denoising\" autoencoder ….. for example, training it with target = 0 samples and comparing the output with a np.zero array of the same shape, then using the trained autoeconder as first stage of a CCN model

\n

But, I think the problem here is that all the samples must be considered as signal, because the telescope is always pointing at some star.

\n

So, in samples with target = 0 there is signal + noise, and in samples with target = 1 we have signal + anomaly + noise

\n

Then your model should be something like F(X) = a(signal + anomaly + noise - bias)

\n

And, because of the signals from different stars can be very specific I think they cannot be considered as \"white noise\" or any other kind of noise model

\n

Is possible that the autoencoder models can be utils yet in this case, by reducing the dimensionality of the original data and reconstructing the spectogram after… This procedure could increase the distance between target = 0 and target = 1, samples, but I don't tried yet

", "rawMarkdown": "If the samples with target = 0 were only noise, then you could reduce the noise using a \"denoising\" autoencoder ..... for example, training it with target = 0 samples and comparing the output with a np.zero array of the same shape, then using the trained autoeconder as first stage of a CCN model\n\nBut, I think the problem here is that all the samples must be considered as signal, because the telescope is always pointing at some star.\n\nSo, in samples with target = 0 there is signal + noise, and in samples with target = 1 we have signal + anomaly + noise\n\nThen your model should be something like F(X) = a(signal + anomaly + noise - bias)\n\nAnd, because of the signals from different stars can be very specific I think they cannot be considered as \"white noise\" or any other kind of noise model\n\nIs possible that the autoencoder models can be utils yet in this case, by reducing the dimensionality of the original data and reconstructing the spectogram after... This procedure could increase the distance between target = 0 and target = 1, samples, but I don't tried yet\n", "votes": 2 }, { "id": 1332105, "postDate": "2021-06-01T22:59:52.507Z", "content": "

I think you are describing an AutoEncoder, I might be wrong though.

", "rawMarkdown": "I think you are describing an AutoEncoder, I might be wrong though." } ], "comments": [ { "id": 1333137, "author_name": "ePolaris", "author_url": "", "post_date": "2021-06-02T14:01:15.647000", "content": "

If the samples with target = 0 were only noise, then you could reduce the noise using a \"denoising\" autoencoder ….. for example, training it with target = 0 samples and comparing the output with a np.zero array of the same shape, then using the trained autoeconder as first stage of a CCN model

\n

But, I think the problem here is that all the samples must be considered as signal, because the telescope is always pointing at some star.

\n

So, in samples with target = 0 there is signal + noise, and in samples with target = 1 we have signal + anomaly + noise

\n

Then your model should be something like F(X) = a(signal + anomaly + noise - bias)

\n

And, because of the signals from different stars can be very specific I think they cannot be considered as \"white noise\" or any other kind of noise model

\n

Is possible that the autoencoder models can be utils yet in this case, by reducing the dimensionality of the original data and reconstructing the spectogram after… This procedure could increase the distance between target = 0 and target = 1, samples, but I don't tried yet

", "votes": 2, "replies": [] }, { "id": 1332105, "author_name": "Adriano Passos", "author_url": "", "post_date": "2021-06-01T22:59:52.507000", "content": "

I think you are describing an AutoEncoder, I might be wrong though.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1329020": "Question. I don't know how the neural network can identify an image without a needle.\n\n1. Guess. \nMy guess is that neural networks should return negative values as default.\nF(x) < 0 for image x << 1.\n\n2. Thought experiment\n(1) Let's say we have a zero spectrogram image x (no noise).\nWithout bias the neural network should return 0. \nIn BceWithLogitsloss, the result value take a sigmoid and will be 0.5. \n\n(2) Let neural network model has a form of F(x) = a*(x-bias).\nThe model F need to return negative value for no needle image.\n\nSo, the model need to learn the bias that represent x distribution.\nHowever the distribution of signal and noise are different.\nWe may think image x = signal + noise + natural noise, then\nF(x) = a*(signal + noise + natural noise - bias)\n\nSince, all three have different distribution, \nthe model F need to learn bias that represent joint distribution.\nAnd in simple model it has a clear limit to approximate it.\n\n\n3. My experiment.\nIn my case, the problem is that the model learns the noise as a class 0.\nI want to the model give like this F(x + noise) = F(x). \nBut My model classify F(x + noise) = F(noise)\n\nI have two problem that detecting the needle image.\n- Weak signals \n- Signal overlapping noise.\nThe model classifies these as 0. \n\nI am trying to increase volume of data in this cases,\nBut increasing the data for these cases raises another problem.\n\nIt's too difficult for me.\nI wonder if there is a way for the model to learn without knowing the image pattern of class 0.\nAnd I don't actually understand how the neural network makes the classification.\n", "1333137": "If the samples with target = 0 were only noise, then you could reduce the noise using a \"denoising\" autoencoder ..... for example, training it with target = 0 samples and comparing the output with a np.zero array of the same shape, then using the trained autoeconder as first stage of a CCN model\n\nBut, I think the problem here is that all the samples must be considered as signal, because the telescope is always pointing at some star.\n\nSo, in samples with target = 0 there is signal + noise, and in samples with target = 1 we have signal + anomaly + noise\n\nThen your model should be something like F(X) = a(signal + anomaly + noise - bias)\n\nAnd, because of the signals from different stars can be very specific I think they cannot be considered as \"white noise\" or any other kind of noise model\n\nIs possible that the autoencoder models can be utils yet in this case, by reducing the dimensionality of the original data and reconstructing the spectogram after... This procedure could increase the distance between target = 0 and target = 1, samples, but I don't tried yet\n", "1332105": "I think you are describing an AutoEncoder, I might be wrong though." } }