{
  "id": 47395,
  "title": "Attempting to find 'outliers' with an autoencoder",
  "url": "/competitions/tensorflow-speech-recognition-challenge/discussion/47395",
  "author_name": "",
  "post_date": "2018-01-13T07:53:29.673535400Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hello,</p>\n\n<p>in an attempt to find 'outliers' (samples in classes other than those to be predicted) I created a variational autoencoder (<a href=\"https://www.kaggle.com/holzner/variational-autoencoder-for-speech-dataset\">https://www.kaggle.com/holzner/variational-autoencoder-for-speech-dataset</a>). The encoder structure is similar to the convolutional layers of <a href=\"https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72\">Alex Ozerin's kernel</a>, the number of latent variables was set to 12 (equal to the number of classes to be predicted). The decoder layers are a mix of upscaling and convolutional layers upscaling the dimensions back to the original input size in reverse order. </p>\n\n<p>The input (and output to compare to) is the amplitude part of the spectrogram. Running the above kernel on Google ML Engine I saved the latent variable parameters for the train and test datasets here: <a href=\"https://www.kaggle.com/holzner/tensorflow-speech-recognition-vae-latent-variables\">https://www.kaggle.com/holzner/tensorflow-speech-recognition-vae-latent-variables</a> (no numbers on reconstruction accuracy though).</p>\n\n<p>Here is a quick analysis of the outcome: <a href=\"https://www.kaggle.com/holzner/variational-autoencoder-latent-features-analysis\">https://www.kaggle.com/holzner/variational-autoencoder-latent-features-analysis</a> . In summary: the distributions of the latent means and widths look almost the same for those labels in the training set which on which the autoencoder was trained and for the others, however there is a significant part of the population in the test set which is not present in the train set (however I don't have more time to try to characterize this part in more detail before the end of the competition).</p>\n\n<p>Any feedback is welcome.</p>",
  "messages": [
    {
      "id": "268079",
      "postDate": "01/13/2018 07:53:29",
      "content": "<p>Hello,</p>\n\n<p>in an attempt to find 'outliers' (samples in classes other than those to be predicted) I created a variational autoencoder (<a href=\"https://www.kaggle.com/holzner/variational-autoencoder-for-speech-dataset\">https://www.kaggle.com/holzner/variational-autoencoder-for-speech-dataset</a>). The encoder structure is similar to the convolutional layers of <a href=\"https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72\">Alex Ozerin's kernel</a>, the number of latent variables was set to 12 (equal to the number of classes to be predicted). The decoder layers are a mix of upscaling and convolutional layers upscaling the dimensions back to the original input size in reverse order. </p>\n\n<p>The input (and output to compare to) is the amplitude part of the spectrogram. Running the above kernel on Google ML Engine I saved the latent variable parameters for the train and test datasets here: <a href=\"https://www.kaggle.com/holzner/tensorflow-speech-recognition-vae-latent-variables\">https://www.kaggle.com/holzner/tensorflow-speech-recognition-vae-latent-variables</a> (no numbers on reconstruction accuracy though).</p>\n\n<p>Here is a quick analysis of the outcome: <a href=\"https://www.kaggle.com/holzner/variational-autoencoder-latent-features-analysis\">https://www.kaggle.com/holzner/variational-autoencoder-latent-features-analysis</a> . In summary: the distributions of the latent means and widths look almost the same for those labels in the training set which on which the autoencoder was trained and for the others, however there is a significant part of the population in the test set which is not present in the train set (however I don't have more time to try to characterize this part in more detail before the end of the competition).</p>\n\n<p>Any feedback is welcome.</p>",
      "rawMarkdown": "Hello,\n\nin an attempt to find 'outliers' (samples in classes other than those to be predicted) I created a variational autoencoder (https://www.kaggle.com/holzner/variational-autoencoder-for-speech-dataset). The encoder structure is similar to the convolutional layers of [Alex Ozerin's kernel][1], the number of latent variables was set to 12 (equal to the number of classes to be predicted). The decoder layers are a mix of upscaling and convolutional layers upscaling the dimensions back to the original input size in reverse order. \n\nThe input (and output to compare to) is the amplitude part of the spectrogram. Running the above kernel on Google ML Engine I saved the latent variable parameters for the train and test datasets here: https://www.kaggle.com/holzner/tensorflow-speech-recognition-vae-latent-variables (no numbers on reconstruction accuracy though).\n\nHere is a quick analysis of the outcome: https://www.kaggle.com/holzner/variational-autoencoder-latent-features-analysis . In summary: the distributions of the latent means and widths look almost the same for those labels in the training set which on which the autoencoder was trained and for the others, however there is a significant part of the population in the test set which is not present in the train set (however I don't have more time to try to characterize this part in more detail before the end of the competition).\n\nAny feedback is welcome.\n\n  [1]: https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72",
      "votes": null
    },
    {
      "id": "268083",
      "postDate": "01/13/2018 08:27:48",
      "content": "<p>i suggest find 'outliers' with an autoencoder for (train+LB set). </p>\n\n<p>Then, work out the percentage of inliers and outliers in separately for train and LB set</p>",
      "rawMarkdown": "i suggest find 'outliers' with an autoencoder for (train+LB set). \n\nThen, work out the percentage of inliers and outliers in separately for train and LB set",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 268083,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "01/13/2018 08:27:48",
      "content": "<p>i suggest find 'outliers' with an autoencoder for (train+LB set). </p>\n\n<p>Then, work out the percentage of inliers and outliers in separately for train and LB set</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "268079": "Hello,\n\nin an attempt to find 'outliers' (samples in classes other than those to be predicted) I created a variational autoencoder (https://www.kaggle.com/holzner/variational-autoencoder-for-speech-dataset). The encoder structure is similar to the convolutional layers of [Alex Ozerin's kernel][1], the number of latent variables was set to 12 (equal to the number of classes to be predicted). The decoder layers are a mix of upscaling and convolutional layers upscaling the dimensions back to the original input size in reverse order. \n\nThe input (and output to compare to) is the amplitude part of the spectrogram. Running the above kernel on Google ML Engine I saved the latent variable parameters for the train and test datasets here: https://www.kaggle.com/holzner/tensorflow-speech-recognition-vae-latent-variables (no numbers on reconstruction accuracy though).\n\nHere is a quick analysis of the outcome: https://www.kaggle.com/holzner/variational-autoencoder-latent-features-analysis . In summary: the distributions of the latent means and widths look almost the same for those labels in the training set which on which the autoencoder was trained and for the others, however there is a significant part of the population in the test set which is not present in the train set (however I don't have more time to try to characterize this part in more detail before the end of the competition).\n\nAny feedback is welcome.\n\n  [1]: https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72",
    "268083": "i suggest find 'outliers' with an autoencoder for (train+LB set). \n\nThen, work out the percentage of inliers and outliers in separately for train and LB set"
  },
  "source": "meta"
}