Many competitors encounter the problem that the model performs perfectly on the training and validation set, however, there's a huge gap between the validation set acc and the submission score. Could everyone say something about this?
", "messages": [ { "id": "263930", "postDate": "01/01/2018 17:14:02", "content": "Many competitors encounter the problem that the model performs perfectly on the training and validation set, however, there's a huge gap between the validation set acc and the submission score. Could everyone say something about this?
", "rawMarkdown": "Many competitors encounter the problem that the model performs perfectly on the training and validation set, however, there's a huge gap between the validation set acc and the submission score. Could everyone say something about this?", "votes": null }, { "id": "263935", "postDate": "01/01/2018 17:34:34", "content": "It's because the test set has a set number of images that have been transformed (scaled, compressed and gamma filters).
", "rawMarkdown": "It's because the test set has a set number of images that have been transformed (scaled, compressed and gamma filters).", "votes": null }, { "id": "263936", "postDate": "01/01/2018 17:44:51", "content": "I'm having the same problem. I've tried adding scale, compress and gamma aug - however it didn't help a lot.
", "rawMarkdown": "I'm having the same problem. I've tried adding scale, compress and gamma aug - however it didn't help a lot.", "votes": null }, { "id": "263937", "postDate": "01/01/2018 17:45:21", "content": "Yes, I know that. I have transformed the original training data, labeled them and use them to train the model. The gap described above is based on this situation.
", "rawMarkdown": "Yes, I know that. I have transformed the original training data, labeled them and use them to train the model. The gap described above is based on this situation.", "votes": null }, { "id": "263947", "postDate": "01/01/2018 18:56:28", "content": "Two possible reasons: (1) the image manipulations (compress, resize, gamma) which are still not completely clear (e.g. they could be alternative or cumulative) and (2) the fact that test images come from a second device (same model, but devices may be in different conditions, e.g. one has a scratch and the other doesn't).
", "rawMarkdown": "Two possible reasons: (1) the image manipulations (compress, resize, gamma) which are still not completely clear (e.g. they could be *alternative* or *cumulative*) and (2) the fact that test images come from a second device (same model, but devices may be in different conditions, e.g. one has a scratch and the other doesn't).", "votes": null }, { "id": "271501", "postDate": "01/20/2018 15:49:05", "content": "Some models have a lot of images of a specific type of object/lighting (eg. iPhone-4s) so models can learn those patterns in addition to pixel patterns, be careful...
", "rawMarkdown": "Some models have a lot of images of a specific type of object/lighting (eg. iPhone-4s) so models can learn those patterns in addition to pixel patterns, be careful...", "votes": null } ], "comments": [ { "id": 263935, "author_name": "craigglastonbury", "author_url": "", "post_date": "01/01/2018 17:34:34", "content": "It's because the test set has a set number of images that have been transformed (scaled, compressed and gamma filters).
", "votes": null, "replies": [ { "id": 263937, "author_name": "yanxiangyi", "author_url": "", "post_date": "01/01/2018 17:45:21", "content": "Yes, I know that. I have transformed the original training data, labeled them and use them to train the model. The gap described above is based on this situation.
", "votes": null, "replies": [] } ] }, { "id": 263936, "author_name": "heyt0ny", "author_url": "", "post_date": "01/01/2018 17:44:51", "content": "I'm having the same problem. I've tried adding scale, compress and gamma aug - however it didn't help a lot.
", "votes": null, "replies": [] }, { "id": 263947, "author_name": "diogoff", "author_url": "", "post_date": "01/01/2018 18:56:28", "content": "Two possible reasons: (1) the image manipulations (compress, resize, gamma) which are still not completely clear (e.g. they could be alternative or cumulative) and (2) the fact that test images come from a second device (same model, but devices may be in different conditions, e.g. one has a scratch and the other doesn't).
", "votes": null, "replies": [] }, { "id": 271501, "author_name": "rhgrossm", "author_url": "", "post_date": "01/20/2018 15:49:05", "content": "Some models have a lot of images of a specific type of object/lighting (eg. iPhone-4s) so models can learn those patterns in addition to pixel patterns, be careful...
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "263930": "Many competitors encounter the problem that the model performs perfectly on the training and validation set, however, there's a huge gap between the validation set acc and the submission score. Could everyone say something about this?", "263935": "It's because the test set has a set number of images that have been transformed (scaled, compressed and gamma filters).", "263936": "I'm having the same problem. I've tried adding scale, compress and gamma aug - however it didn't help a lot.", "263937": "Yes, I know that. I have transformed the original training data, labeled them and use them to train the model. The gap described above is based on this situation.", "263947": "Two possible reasons: (1) the image manipulations (compress, resize, gamma) which are still not completely clear (e.g. they could be *alternative* or *cumulative*) and (2) the fact that test images come from a second device (same model, but devices may be in different conditions, e.g. one has a scratch and the other doesn't).", "271501": "Some models have a lot of images of a specific type of object/lighting (eg. iPhone-4s) so models can learn those patterns in addition to pixel patterns, be careful..." }, "source": "meta" }