![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1910776%2F4dc8cc4aec53aab033b6b494b013c670%2FFigure_1.png?generation=1576702036014121&alt=media\")
I am new to deep learning field, I am trying to learn a bit from this competition.\nAt the moment my approach is:\n1. Detect faces in image (with opencv)\n2. Crop image in face location\n3. Cropped image is resized\n4. The image shape is the input to a CNN
\n\nI am training with just one of the subfolders from total dataset. Here is a result:\n
The model has a really good train lost value, however the validation is really bad, with a lot of overfit. The CNN learns each image instead of extracting generative features from the image. Any suggestion to fix this? Training with a larger dataset will help right (It is hard for me to test this because the loading time is a lot more than the training itself xD)
", "messages": [ { "id": 698129, "postDate": "2019-12-18T20:50:10.277Z", "content": "I am new to deep learning field, I am trying to learn a bit from this competition.\nAt the moment my approach is:\n1. Detect faces in image (with opencv)\n2. Crop image in face location\n3. Cropped image is resized\n4. The image shape is the input to a CNN
\n\nI am training with just one of the subfolders from total dataset. Here is a result:\n
The model has a really good train lost value, however the validation is really bad, with a lot of overfit. The CNN learns each image instead of extracting generative features from the image. Any suggestion to fix this? Training with a larger dataset will help right (It is hard for me to test this because the loading time is a lot more than the training itself xD)
", "rawMarkdown": "I am new to deep learning field, I am trying to learn a bit from this competition.\nAt the moment my approach is:\n1. Detect faces in image (with opencv)\n2. Crop image in face location\n3. Cropped image is resized\n4. The image shape is the input to a CNN\n\nI am training with just one of the subfolders from total dataset. Here is a result:\n\n\nThe model has a really good train lost value, however the validation is really bad, with a lot of overfit. The CNN learns each image instead of extracting generative features from the image. Any suggestion to fix this? Training with a larger dataset will help right (It is hard for me to test this because the loading time is a lot more than the training itself xD)\n", "votes": 2 }, { "id": 698139, "postDate": "2019-12-18T21:16:25.927Z", "content": "There is not enough signal in your training data to learn from -- or the model isn't picking up on it. The model is also probably fairly large and so it will just start memorizing the training data.
\n\nIf you look at the face crops in the training set (and the validation set) with your own eyes, you'll probably see that it's really hard to tell the real and fake images apart.
\n\nSo a simple binary classifier trained on individual face crops might not work very well for this competition...
", "rawMarkdown": "There is not enough signal in your training data to learn from -- or the model isn't picking up on it. The model is also probably fairly large and so it will just start memorizing the training data.\n\nIf you look at the face crops in the training set (and the validation set) with your own eyes, you'll probably see that it's really hard to tell the real and fake images apart.\n\nSo a simple binary classifier trained on individual face crops might not work very well for this competition...", "replies": [ { "id": 698605, "postDate": "2019-12-19T13:07:08.933Z", "content": "Thank you for the feedback, do you think that increasing the dataset size can fix this? If there are a lot more data to memorize and the model need to learn to extract features.
", "rawMarkdown": "Thank you for the feedback, do you think that increasing the dataset size can fix this? If there are a lot more data to memorize and the model need to learn to extract features." }, { "id": 698617, "postDate": "2019-12-19T13:28:15.143Z", "content": "Yes, increasing the size of the dataset is an effective way to combat overfitting.
", "rawMarkdown": "Yes, increasing the size of the dataset is an effective way to combat overfitting." }, { "id": 770038, "postDate": "2020-03-12T14:06:37.980Z", "content": "I tried, but the val_loss still stagger around 0.4-0.7, I highly suspect that the relationship between test dataset and training dataset should be more explored. When the 400 training samples are tested, the loss is 0.14x which means it can “detect” well if it once saw the actor.
", "rawMarkdown": "I tried, but the val_loss still stagger around 0.4-0.7, I highly suspect that the relationship between test dataset and training dataset should be more explored. When the 400 training samples are tested, the loss is 0.14x which means it can “detect” well if it once saw the actor. ", "isDeleted": true } ] }, { "id": 734005, "postDate": "2020-01-31T20:02:53.030Z", "content": "Most common mistake is to train your model from scratch. If you are not using a pretrained model then it's obivious to see scary graphs. If you're using pretrained model then probably you should try using data augmentation to increase the training data size. Most common model is resnet for CNN. PS: don't forget to freeze initial layers. Ask for any clarification.
", "rawMarkdown": "Most common mistake is to train your model from scratch. If you are not using a pretrained model then it's obivious to see scary graphs. If you're using pretrained model then probably you should try using data augmentation to increase the training data size. Most common model is resnet for CNN. PS: don't forget to freeze initial layers. Ask for any clarification." }, { "id": 733886, "postDate": "2020-01-31T16:35:31.947Z", "content": "Well, I have the same over-fitting problem. Great during training, disastrous on test set. I tried to see if increasing the training data to a larger chunk of the whole dataset helped, but it still overfits with 20% dataset, although less severe. And this even with super simple CNN models having only a few thousands parameters.
\n\nI think it is worth it to spend some time to build a good train/test set. Just picking a few training folders is not good because they lack diversity (each one contains some originals and all their fakes). As such, it is better to pick a subset from each training dir (like a single fake for each real) and bundle them together to form a diversified sampling. That's what I will attempt next at least! 😅
", "rawMarkdown": "Well, I have the same over-fitting problem. Great during training, disastrous on test set. I tried to see if increasing the training data to a larger chunk of the whole dataset helped, but it still overfits with 20% dataset, although less severe. And this even with super simple CNN models having only a few thousands parameters.\n\nI think it is worth it to spend some time to build a good train/test set. Just picking a few training folders is not good because they lack diversity (each one contains some originals and all their fakes). As such, it is better to pick a subset from each training dir (like a single fake for each real) and bundle them together to form a diversified sampling. That's what I will attempt next at least! 😅 " }, { "id": 698161, "postDate": "2019-12-18T21:45:48.823Z", "content": "You will need to lower your model complexity(which is probably not a really good idea). For example switch to a CNN model with lesser Conv layers or just simply switch to a svm).\nIf you want to train with a larger dataset, I made a dataset here with all chunks of data by cropping the face from the first frame with mtcnn(for more details, please go to the link) . I will continue to upload more.
", "rawMarkdown": "You will need to lower your model complexity(which is probably not a really good idea). For example switch to a CNN model with lesser Conv layers or just simply switch to a svm).\nIf you want to train with a larger dataset, I made a dataset [here](https://www.kaggle.com/unkownhihi/deepfakes/) with all chunks of data by cropping the face from the first frame with mtcnn(for more details, please go to the link) . I will continue to upload more.", "replies": [ { "id": 698606, "postDate": "2019-12-19T13:08:00.467Z", "content": "Thank you for the suggestions, I will take a look at your dataset
", "rawMarkdown": "Thank you for the suggestions, I will take a look at your dataset" }, { "id": 734818, "postDate": "2020-02-02T03:34:30.397Z", "content": "@unkownhihi e Are you training the whole network or just fine-tuning (freezing the early network layer)? Will training the whole network make it easier to overfit?
", "rawMarkdown": "@unkownhihi e Are you training the whole network or just fine-tuning (freezing the early network layer)? Will training the whole network make it easier to overfit?" }, { "id": 734833, "postDate": "2020-02-02T04:13:56.103Z", "content": "I am training the whole network. Training with whole network will have more parameters so definitely easier to train. but freezing the early network layers would give bad results.
", "rawMarkdown": "I am training the whole network. Training with whole network will have more parameters so definitely easier to train. but freezing the early network layers would give bad results." }, { "id": 734875, "postDate": "2020-02-02T05:55:06.433Z", "content": "Thank you for your advice.
", "rawMarkdown": "Thank you for your advice." }, { "id": 735055, "postDate": "2020-02-02T13:03:02.813Z", "content": "you can do both - say, fine-tune for 3 epochs, and then train on all parameters until early stops or however many rounds to your content
", "rawMarkdown": "you can do both - say, fine-tune for 3 epochs, and then train on all parameters until early stops or however many rounds to your content ", "votes": 1 } ] }, { "id": 698875, "postDate": "2019-12-19T19:54:41.783Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 698139, "author_name": "Human Analog", "author_url": "", "post_date": "2019-12-18T21:16:25.927000", "content": "There is not enough signal in your training data to learn from -- or the model isn't picking up on it. The model is also probably fairly large and so it will just start memorizing the training data.
\n\nIf you look at the face crops in the training set (and the validation set) with your own eyes, you'll probably see that it's really hard to tell the real and fake images apart.
\n\nSo a simple binary classifier trained on individual face crops might not work very well for this competition...
", "votes": 0, "replies": [ { "id": 698605, "author_name": "João Varela", "author_url": "", "post_date": "2019-12-19T13:07:08.933000", "content": "Thank you for the feedback, do you think that increasing the dataset size can fix this? If there are a lot more data to memorize and the model need to learn to extract features.
", "votes": 0, "replies": [] }, { "id": 698617, "author_name": "Human Analog", "author_url": "", "post_date": "2019-12-19T13:28:15.143000", "content": "Yes, increasing the size of the dataset is an effective way to combat overfitting.
", "votes": 0, "replies": [] }, { "id": 770038, "author_name": "", "author_url": "", "post_date": "2020-03-12T14:06:37.980000", "content": "I tried, but the val_loss still stagger around 0.4-0.7, I highly suspect that the relationship between test dataset and training dataset should be more explored. When the 400 training samples are tested, the loss is 0.14x which means it can “detect” well if it once saw the actor.
", "votes": 0, "replies": [] } ] }, { "id": 734005, "author_name": "Ankit Saini", "author_url": "", "post_date": "2020-01-31T20:02:53.030000", "content": "Most common mistake is to train your model from scratch. If you are not using a pretrained model then it's obivious to see scary graphs. If you're using pretrained model then probably you should try using data augmentation to increase the training data size. Most common model is resnet for CNN. PS: don't forget to freeze initial layers. Ask for any clarification.
", "votes": 0, "replies": [] }, { "id": 733886, "author_name": "dagnelies", "author_url": "", "post_date": "2020-01-31T16:35:31.947000", "content": "Well, I have the same over-fitting problem. Great during training, disastrous on test set. I tried to see if increasing the training data to a larger chunk of the whole dataset helped, but it still overfits with 20% dataset, although less severe. And this even with super simple CNN models having only a few thousands parameters.
\n\nI think it is worth it to spend some time to build a good train/test set. Just picking a few training folders is not good because they lack diversity (each one contains some originals and all their fakes). As such, it is better to pick a subset from each training dir (like a single fake for each real) and bundle them together to form a diversified sampling. That's what I will attempt next at least! 😅
", "votes": 0, "replies": [] }, { "id": 698161, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2019-12-18T21:45:48.823000", "content": "You will need to lower your model complexity(which is probably not a really good idea). For example switch to a CNN model with lesser Conv layers or just simply switch to a svm).\nIf you want to train with a larger dataset, I made a dataset here with all chunks of data by cropping the face from the first frame with mtcnn(for more details, please go to the link) . I will continue to upload more.
", "votes": 0, "replies": [ { "id": 698606, "author_name": "João Varela", "author_url": "", "post_date": "2019-12-19T13:08:00.467000", "content": "Thank you for the suggestions, I will take a look at your dataset
", "votes": 0, "replies": [] }, { "id": 734818, "author_name": "Chason", "author_url": "", "post_date": "2020-02-02T03:34:30.397000", "content": "@unkownhihi e Are you training the whole network or just fine-tuning (freezing the early network layer)? Will training the whole network make it easier to overfit?
", "votes": 0, "replies": [] }, { "id": 734833, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-02T04:13:56.103000", "content": "I am training the whole network. Training with whole network will have more parameters so definitely easier to train. but freezing the early network layers would give bad results.
", "votes": 0, "replies": [] }, { "id": 734875, "author_name": "Chason", "author_url": "", "post_date": "2020-02-02T05:55:06.433000", "content": "Thank you for your advice.
", "votes": 0, "replies": [] }, { "id": 735055, "author_name": "Yifan Xie", "author_url": "", "post_date": "2020-02-02T13:03:02.813000", "content": "you can do both - say, fine-tune for 3 epochs, and then train on all parameters until early stops or however many rounds to your content
", "votes": 1, "replies": [] } ] }, { "id": 698875, "author_name": "", "author_url": "", "post_date": "2019-12-19T19:54:41.783000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "698129": "I am new to deep learning field, I am trying to learn a bit from this competition.\nAt the moment my approach is:\n1. Detect faces in image (with opencv)\n2. Crop image in face location\n3. Cropped image is resized\n4. The image shape is the input to a CNN\n\nI am training with just one of the subfolders from total dataset. Here is a result:\n\n\nThe model has a really good train lost value, however the validation is really bad, with a lot of overfit. The CNN learns each image instead of extracting generative features from the image. Any suggestion to fix this? Training with a larger dataset will help right (It is hard for me to test this because the loading time is a lot more than the training itself xD)\n", "698139": "There is not enough signal in your training data to learn from -- or the model isn't picking up on it. The model is also probably fairly large and so it will just start memorizing the training data.\n\nIf you look at the face crops in the training set (and the validation set) with your own eyes, you'll probably see that it's really hard to tell the real and fake images apart.\n\nSo a simple binary classifier trained on individual face crops might not work very well for this competition...", "734005": "Most common mistake is to train your model from scratch. If you are not using a pretrained model then it's obivious to see scary graphs. If you're using pretrained model then probably you should try using data augmentation to increase the training data size. Most common model is resnet for CNN. PS: don't forget to freeze initial layers. Ask for any clarification.", "733886": "Well, I have the same over-fitting problem. Great during training, disastrous on test set. I tried to see if increasing the training data to a larger chunk of the whole dataset helped, but it still overfits with 20% dataset, although less severe. And this even with super simple CNN models having only a few thousands parameters.\n\nI think it is worth it to spend some time to build a good train/test set. Just picking a few training folders is not good because they lack diversity (each one contains some originals and all their fakes). As such, it is better to pick a subset from each training dir (like a single fake for each real) and bundle them together to form a diversified sampling. That's what I will attempt next at least! 😅 ", "698161": "You will need to lower your model complexity(which is probably not a really good idea). For example switch to a CNN model with lesser Conv layers or just simply switch to a svm).\nIf you want to train with a larger dataset, I made a dataset [here](https://www.kaggle.com/unkownhihi/deepfakes/) with all chunks of data by cropping the face from the first frame with mtcnn(for more details, please go to the link) . I will continue to upload more.", "698875": "" } }