I am contemplating to write a PCA code that calculates the principal components for the imagines in this challenge. Many of them have a lot of white space near the borders, so it could potentially be helpful for dimensionality reduction. Once obtained the principal components could be saved to a file and easily reused.
\n\nDue to the size of the dataset I would have to write a custom routine so that I don't run out of memory while calculating the co-variances. But before I spend all that time - is PCA even going to be useful in this challenge?
\n\nI see a lot of people using vgg16 or xception in their kernels. Those are pre-trained models right? So whatever the convolutional layers will have learned in the pre-training becomes useless once PCA is applied. Correct? I am still new to ML so please point out if I have a misunderstanding somewhere.
\n\nBut maybe a PCA trained NN would be useful in a stacked model? I would love to hear some people's thoughts on whether I would be wasting my time before I spent several hours implementing this ;)
", "messages": [ { "id": "231160", "postDate": "10/13/2017 19:29:48", "content": "I am contemplating to write a PCA code that calculates the principal components for the imagines in this challenge. Many of them have a lot of white space near the borders, so it could potentially be helpful for dimensionality reduction. Once obtained the principal components could be saved to a file and easily reused.
\n\nDue to the size of the dataset I would have to write a custom routine so that I don't run out of memory while calculating the co-variances. But before I spend all that time - is PCA even going to be useful in this challenge?
\n\nI see a lot of people using vgg16 or xception in their kernels. Those are pre-trained models right? So whatever the convolutional layers will have learned in the pre-training becomes useless once PCA is applied. Correct? I am still new to ML so please point out if I have a misunderstanding somewhere.
\n\nBut maybe a PCA trained NN would be useful in a stacked model? I would love to hear some people's thoughts on whether I would be wasting my time before I spent several hours implementing this ;)
", "rawMarkdown": "I am contemplating to write a PCA code that calculates the principal components for the imagines in this challenge. Many of them have a lot of white space near the borders, so it could potentially be helpful for dimensionality reduction. Once obtained the principal components could be saved to a file and easily reused.\n\nDue to the size of the dataset I would have to write a custom routine so that I don't run out of memory while calculating the co-variances. But before I spend all that time - is PCA even going to be useful in this challenge?\n\nI see a lot of people using vgg16 or xception in their kernels. Those are pre-trained models right? So whatever the convolutional layers will have learned in the pre-training becomes useless once PCA is applied. Correct? I am still new to ML so please point out if I have a misunderstanding somewhere. \n\nBut maybe a PCA trained NN would be useful in a stacked model? I would love to hear some people's thoughts on whether I would be wasting my time before I spent several hours implementing this ;)", "votes": null }, { "id": "231167", "postDate": "10/13/2017 19:42:18", "content": "It might be useful but maybe not on the images but one of the layers of already pretrained model. As far as I remember. for example in Youtube-8M organizers have put all images (frames) through Inception 3 and then made PCA with whitening on the values from the pre-last layer.
", "rawMarkdown": "It might be useful but maybe not on the images but one of the layers of already pretrained model. As far as I remember. for example in Youtube-8M organizers have put all images (frames) through Inception 3 and then made PCA with whitening on the values from the pre-last layer.", "votes": null } ], "comments": [ { "id": 231167, "author_name": "mpekalski", "author_url": "", "post_date": "10/13/2017 19:42:18", "content": "It might be useful but maybe not on the images but one of the layers of already pretrained model. As far as I remember. for example in Youtube-8M organizers have put all images (frames) through Inception 3 and then made PCA with whitening on the values from the pre-last layer.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "231160": "I am contemplating to write a PCA code that calculates the principal components for the imagines in this challenge. Many of them have a lot of white space near the borders, so it could potentially be helpful for dimensionality reduction. Once obtained the principal components could be saved to a file and easily reused.\n\nDue to the size of the dataset I would have to write a custom routine so that I don't run out of memory while calculating the co-variances. But before I spend all that time - is PCA even going to be useful in this challenge?\n\nI see a lot of people using vgg16 or xception in their kernels. Those are pre-trained models right? So whatever the convolutional layers will have learned in the pre-training becomes useless once PCA is applied. Correct? I am still new to ML so please point out if I have a misunderstanding somewhere. \n\nBut maybe a PCA trained NN would be useful in a stacked model? I would love to hear some people's thoughts on whether I would be wasting my time before I spent several hours implementing this ;)", "231167": "It might be useful but maybe not on the images but one of the layers of already pretrained model. As far as I remember. for example in Youtube-8M organizers have put all images (frames) through Inception 3 and then made PCA with whitening on the values from the pre-last layer." }, "source": "meta" }