I have posted a notebook demonstrating the use of an auto-encoder for feature extraction. The auto-encoder encodes the 800,000 integer samples and compresses them down to just 5 floating point numbers. It also decodes those 5 numbers back into an approximation of the original signal.
\n\nNaturally, this CODEC is lossy, and only contains the \"boring\" information common to all signals. In the same notebook, a second auto-encoder is demonstrated, this time compressing the residual (original signal minus the lossy approximation) from 800,000 samples down to 20 floating point numbers.
\n\nFinally, the notebook demonstrates the use of the \"second residual\" (what's left after you subtract the two approximations from the original) to extract the \"important\" features.
\n\nHopefully this will be helpful to you in this contest and others!
\n\nhttps://www.kaggle.com/pnussbaum/vsb-power-using-autoencoding-v09
\n\nP.S. Auto-encoders are useful in these types of situations, and also in situations where the \"internal models\" of trained deep learning networks need to be \"hand-checked\" by an expert before the network can be approved for use in mission critical classification projects.
", "messages": [ { "id": 459540, "postDate": "2019-01-22T00:23:27.300Z", "content": "I have posted a notebook demonstrating the use of an auto-encoder for feature extraction. The auto-encoder encodes the 800,000 integer samples and compresses them down to just 5 floating point numbers. It also decodes those 5 numbers back into an approximation of the original signal.
\n\nNaturally, this CODEC is lossy, and only contains the \"boring\" information common to all signals. In the same notebook, a second auto-encoder is demonstrated, this time compressing the residual (original signal minus the lossy approximation) from 800,000 samples down to 20 floating point numbers.
\n\nFinally, the notebook demonstrates the use of the \"second residual\" (what's left after you subtract the two approximations from the original) to extract the \"important\" features.
\n\nHopefully this will be helpful to you in this contest and others!
\n\nhttps://www.kaggle.com/pnussbaum/vsb-power-using-autoencoding-v09
\n\nP.S. Auto-encoders are useful in these types of situations, and also in situations where the \"internal models\" of trained deep learning networks need to be \"hand-checked\" by an expert before the network can be approved for use in mission critical classification projects.
", "rawMarkdown": "I have posted a notebook demonstrating the use of an auto-encoder for feature extraction. The auto-encoder encodes the 800,000 integer samples and compresses them down to just 5 floating point numbers. It also decodes those 5 numbers back into an approximation of the original signal.\n\nNaturally, this CODEC is lossy, and only contains the \"boring\" information common to all signals. In the same notebook, a second auto-encoder is demonstrated, this time compressing the residual (original signal minus the lossy approximation) from 800,000 samples down to 20 floating point numbers.\n\nFinally, the notebook demonstrates the use of the \"second residual\" (what's left after you subtract the two approximations from the original) to extract the \"important\" features.\n\nHopefully this will be helpful to you in this contest and others!\n\nhttps://www.kaggle.com/pnussbaum/vsb-power-using-autoencoding-v09 \n\nP.S. Auto-encoders are useful in these types of situations, and also in situations where the \"internal models\" of trained deep learning networks need to be \"hand-checked\" by an expert before the network can be approved for use in mission critical classification projects.", "votes": 24 }, { "id": 460881, "postDate": "2019-01-24T15:55:45.090Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 461198, "postDate": "2019-01-25T14:15:23.700Z", "content": "Thank you!
\n\nIt may be more relevant to use other network architectures, but I find that they also require a greater number of parameters for the user to select. It seems to me that the parameter selection can be done through trial and error, or with a knowledge of the problem space - whereas the dense network (which is like a Conv1D with a kernel_size equal to the entire 800,000 data samples !) is more of a brute force method.
\n\nThe brute force method may be useful when you really are not sure how to extract features (or what parameters to choose). After all, if I really was able to extract features expertly, I probably could just extract them and then use those features in a kNN scheme - without any sort of iterative \"training\" scheme needed.
\n\nStill... I would like to know of any \"brute force\" (or systematic parameter selection methods) for recurrent neural networks or convolutional neural networks if you know of any. The papers I read seem to use a trial and error method, and that makes my Jupyter notebook on Kaggle run out of RAM or time or both :-)
", "rawMarkdown": "Thank you! \n\nIt may be more relevant to use other network architectures, but I find that they also require a greater number of parameters for the user to select. It seems to me that the parameter selection can be done through trial and error, or with a knowledge of the problem space - whereas the dense network (which is like a Conv1D with a kernel_size equal to the entire 800,000 data samples !) is more of a brute force method.\n\nThe brute force method may be useful when you really are not sure how to extract features (or what parameters to choose). After all, if I really was able to extract features expertly, I probably could just extract them and then use those features in a kNN scheme - without any sort of iterative \"training\" scheme needed.\n\nStill... I would like to know of any \"brute force\" (or systematic parameter selection methods) for recurrent neural networks or convolutional neural networks if you know of any. The papers I read seem to use a trial and error method, and that makes my Jupyter notebook on Kaggle run out of RAM or time or both :-)", "votes": 1 } ] } ], "comments": [ { "id": 460881, "author_name": "", "author_url": "", "post_date": "2019-01-24T15:55:45.090000", "content": "", "votes": 1, "replies": [ { "id": 461198, "author_name": "Paul Nussbaum, PhD", "author_url": "", "post_date": "2019-01-25T14:15:23.700000", "content": "Thank you!
\n\nIt may be more relevant to use other network architectures, but I find that they also require a greater number of parameters for the user to select. It seems to me that the parameter selection can be done through trial and error, or with a knowledge of the problem space - whereas the dense network (which is like a Conv1D with a kernel_size equal to the entire 800,000 data samples !) is more of a brute force method.
\n\nThe brute force method may be useful when you really are not sure how to extract features (or what parameters to choose). After all, if I really was able to extract features expertly, I probably could just extract them and then use those features in a kNN scheme - without any sort of iterative \"training\" scheme needed.
\n\nStill... I would like to know of any \"brute force\" (or systematic parameter selection methods) for recurrent neural networks or convolutional neural networks if you know of any. The papers I read seem to use a trial and error method, and that makes my Jupyter notebook on Kaggle run out of RAM or time or both :-)
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "459540": "I have posted a notebook demonstrating the use of an auto-encoder for feature extraction. The auto-encoder encodes the 800,000 integer samples and compresses them down to just 5 floating point numbers. It also decodes those 5 numbers back into an approximation of the original signal.\n\nNaturally, this CODEC is lossy, and only contains the \"boring\" information common to all signals. In the same notebook, a second auto-encoder is demonstrated, this time compressing the residual (original signal minus the lossy approximation) from 800,000 samples down to 20 floating point numbers.\n\nFinally, the notebook demonstrates the use of the \"second residual\" (what's left after you subtract the two approximations from the original) to extract the \"important\" features.\n\nHopefully this will be helpful to you in this contest and others!\n\nhttps://www.kaggle.com/pnussbaum/vsb-power-using-autoencoding-v09 \n\nP.S. Auto-encoders are useful in these types of situations, and also in situations where the \"internal models\" of trained deep learning networks need to be \"hand-checked\" by an expert before the network can be approved for use in mission critical classification projects.", "460881": "" } }