{ "id": 47475, "title": "93 percent validation accuracy and 10 percent test accuracy, HOW???", "url": "/competitions/sp-society-camera-model-identification/discussion/47475", "author_name": "", "post_date": "2018-01-14T21:07:17.250613400Z", "votes": 12, "comment_count": 25, "views": 0, "content": "

How is it possible to have 93 percent validation accuracy and 10 percent accuracy on test set? any suggestion?

", "messages": [ { "id": "268552", "postDate": "01/14/2018 21:07:17", "content": "

How is it possible to have 93 percent validation accuracy and 10 percent accuracy on test set? any suggestion?

", "rawMarkdown": "How is it possible to have 93 percent validation accuracy and 10 percent accuracy on test set? any suggestion?", "votes": null }, { "id": "268553", "postDate": "01/14/2018 21:08:50", "content": "

10 percent accuracy means that the classifier is predicting totally randomly, regardless of the given input image.

", "rawMarkdown": "10 percent accuracy means that the classifier is predicting totally randomly, regardless of the given input image.", "votes": null }, { "id": "268559", "postDate": "01/14/2018 21:47:51", "content": "

You almost certainly have the labels switched when you predict.

", "rawMarkdown": "You almost certainly have the labels switched when you predict.", "votes": null }, { "id": "268575", "postDate": "01/14/2018 23:08:38", "content": "

Or, the model is terribly over-fit on the training data.

", "rawMarkdown": "Or, the model is terribly over-fit on the training data.", "votes": null }, { "id": "268577", "postDate": "01/14/2018 23:18:29", "content": "

No, My train accuracy is about 93 and my validation accuracy is now above 96. I am sure it is not related to overfitting.

", "rawMarkdown": "No, My train accuracy is about 93 and my validation accuracy is now above 96. I am sure it is not related to overfitting.", "votes": null }, { "id": "268583", "postDate": "01/14/2018 23:39:22", "content": "

You wouldn't expect a high validation accuracy.

", "rawMarkdown": "You wouldn't expect a high validation accuracy.", "votes": null }, { "id": "268777", "postDate": "01/15/2018 15:21:33", "content": "

Did you find the reason of big gap between training set and testing set? Look like that you achieve ~75% so gap is much lower. I was preparing the validation set using same transformation like was used for test set, currently with no success.

", "rawMarkdown": "Did you find the reason of big gap between training set and testing set? Look like that you achieve ~75% so gap is much lower. I was preparing the validation set using same transformation like was used for test set, currently with no success.", "votes": null }, { "id": "269078", "postDate": "01/16/2018 06:35:38", "content": "

Yes. It was a coding mistake. Now my validation is test accuracy is 80 percent.

", "rawMarkdown": "Yes. It was a coding mistake. Now my validation is test accuracy is 80 percent.", "votes": null }, { "id": "269101", "postDate": "01/16/2018 07:49:03", "content": "

Could you tell me what kind of issue is here? For your case it is just label swapping?\nI have 87% at modified-validation set (images look like in test set)\nSame model have 64% in LB.

\n\n

Could you clarify how do you create a validation set?

", "rawMarkdown": "Could you tell me what kind of issue is here? For your case it is just label swapping?\nI have 87% at modified-validation set (images look like in test set)\nSame model have 64% in LB. \n\nCould you clarify how do you create a validation set?", "votes": null }, { "id": "269297", "postDate": "01/16/2018 15:52:37", "content": "

most likely you are leaking class info, try print(df) see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation.

", "rawMarkdown": "most likely you are leaking class info, try `print(df)` see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation.", "votes": null }, { "id": "269351", "postDate": "01/16/2018 17:11:35", "content": "

thanks a lot. But the problem is now solved. As I said, it was related to a very subtle mistake in my code during loading weights of a pre-trained model.

", "rawMarkdown": "thanks a lot. But the problem is now solved. As I said, it was related to a very subtle mistake in my code during loading weights of a pre-trained model.", "votes": null }, { "id": "269479", "postDate": "01/16/2018 22:39:34", "content": "

I just randomly selected 50 images from each class and put them in the corresponding class in the validation set.

", "rawMarkdown": "I just randomly selected 50 images from each class and put them in the corresponding class in the validation set.", "votes": null }, { "id": "270428", "postDate": "01/18/2018 07:50:41", "content": "

Sorry for being insistent, but could you tell more about your methodology of creating a validation set? I'm still looking for a gap ~20% between my validation and test set.

\n\n
    \n

  1. When you are using validation set, do you resize the image to input network size or make center crop?

    2. \n

  2. What image size do you use?

    3. \n

  3. Did you apply any resizing, gamma correction or JPEG compression to validation set to imitate test set?

    4. \n

  4. You mentioned that you have 93% in validation set, then you have 80%. What did you change in pipeline?

    5. \n
", "rawMarkdown": "Sorry for being insistent, but could you tell more about your methodology of creating a validation set? I'm still looking for a gap ~20% between my validation and test set. \n\n1. When you are using validation set, do you resize the image to input network size or make center crop?\n\n2. What image size do you use?\n\n3. Did you apply any resizing, gamma correction or JPEG compression to validation set to imitate test set?\n\n4. You mentioned that you have 93% in validation set, then you have 80%. What did you change in pipeline?", "votes": null }, { "id": "270475", "postDate": "01/18/2018 09:48:38", "content": "

1 - No, Resizing is not a good idea.\n2 - (3, 224, 224)\n3 - Not yet!\n4 - Nothing

", "rawMarkdown": "1 - No, Resizing is not a good idea.\n2 - (3, 224, 224)\n3 - Not yet!\n4 - Nothing", "votes": null }, { "id": "272056", "postDate": "01/22/2018 06:48:02", "content": "

Hi, I have a similar problem. My validation accuracy is not as high as yours, only about 60%, but still I end up with only 10% for the test set. What could I be doing wrong? \nThanks a lot!

", "rawMarkdown": "Hi, I have a similar problem. My validation accuracy is not as high as yours, only about 60%, but still I end up with only 10% for the test set. What could I be doing wrong? \nThanks a lot!", "votes": null }, { "id": "272068", "postDate": "01/22/2018 07:32:16", "content": "

Hi. In my case, at the time of testing, I had forgotten to load the trained weights correctly, and so the weights were randomly initialized and the network was classifying randomly. That's why I got only 10 percent test accuracy.

\n\n

Possible solution:

\n\n

In your case maybe this is the reason or maybe your labels order are incorrect. Ensure that during testing, you don't shuffle your test data.

", "rawMarkdown": "Hi. In my case, at the time of testing, I had forgotten to **load the trained weights** correctly, and so the weights were randomly initialized and the network was classifying randomly. That's why I got only 10 percent test accuracy. \n\n### Possible solution:\nIn your case maybe this is the reason or maybe your labels order are incorrect. Ensure that during testing, you don't shuffle your test data.", "votes": null }, { "id": "272088", "postDate": "01/22/2018 08:40:59", "content": "

Thank you! I will look into this!

", "rawMarkdown": "Thank you! I will look into this!", "votes": null }, { "id": "272106", "postDate": "01/22/2018 09:29:31", "content": "

two good rules to debug your model :\n- Make it overfit on a small subset. If it does, the network does actually learn and it indicates that you are feeding it with good data / labels\n- predict on the train dataset et assert the predicted labels are coherent with the true ones.

", "rawMarkdown": "two good rules to debug your model :\n- Make it overfit on a small subset. If it does, the network does actually learn and it indicates that you are feeding it with good data / labels\n- predict on the train dataset et assert the predicted labels are coherent with the true ones.", "votes": null }, { "id": "272214", "postDate": "01/22/2018 14:46:16", "content": "

I had stuck in same situation.

\n\n

Loading the weight was confirmed by using validation data in test mode. In this case, the result seemed working well.

\n\n

labels are as below. It seams that there is no problem.\n'HTC-1-M7 LG-Nexus-5x Motorola-Droid-Maxx Motorola-Nexus-6 Motorola-X Samsung-Galaxy-Note3 Samsung-Galaxy-S4 Sony-NEX-7 iPhone-4s iPhone-6'

\n\n

should I check not to shuffle the test data? is it a problem?

", "rawMarkdown": "I had stuck in same situation. \n\nLoading the weight was confirmed by using validation data in test mode. In this case, the result seemed working well.\n\nlabels are as below. It seams that there is no problem.\n'HTC-1-M7 LG-Nexus-5x Motorola-Droid-Maxx Motorola-Nexus-6 Motorola-X Samsung-Galaxy-Note3 Samsung-Galaxy-S4 Sony-NEX-7 iPhone-4s iPhone-6'\n\nshould I check not to shuffle the test data? is it a problem?", "votes": null }, { "id": "272231", "postDate": "01/22/2018 15:33:19", "content": "

If you shuffle the test data, the order will be different from the order given in the sample_submission.csv. So, you should be careful about ordering. But in general, according to the rules of this competition, the order of test data is not important; ie., you can submit your predictions in any order.

", "rawMarkdown": "If you shuffle the test data, the order will be different from the order given in the `sample_submission.csv`. So, you should be careful about ordering. But in general, according to the rules of this competition, the order of test data is not important; ie., you can submit your predictions in any order.", "votes": null }, { "id": "273010", "postDate": "01/24/2018 02:34:45", "content": "

classic

", "rawMarkdown": "classic", "votes": null }, { "id": "274030", "postDate": "01/25/2018 17:25:08", "content": "

Is there any way I can understand if I am messig my labels up ? Because I trained a resnet152, from scratch, resnet50 using transfer Learning keeping layer.trainable as False and also making them trainable as True but for all combinations, I am getting my train and Val accuracy in the 90's and 80's respectively but my LB score as 0.181 or 0.194 etc. What can be the reason for this ?

", "rawMarkdown": "Is there any way I can understand if I am messig my labels up ? Because I trained a resnet152, from scratch, resnet50 using transfer Learning keeping layer.trainable as False and also making them trainable as True but for all combinations, I am getting my train and Val accuracy in the 90's and 80's respectively but my LB score as 0.181 or 0.194 etc. What can be the reason for this ?", "votes": null }, { "id": "274041", "postDate": "01/25/2018 18:14:52", "content": "", "rawMarkdown": "predict on train and see if it matches\n- unit testing", "votes": null }, { "id": "274283", "postDate": "01/26/2018 08:27:31", "content": "

I have same problem.... I use this Kernel (https://www.kaggle.com/rishabhiitbhu/keras-cnn-starter). So i made just some tunes in CNN and get near 90% on CV, but just 7% on test! I predict this model on train set and get near 97%, so i think there is no mistake in model. Any suggestions?

", "rawMarkdown": "I have same problem.... I use this Kernel (https://www.kaggle.com/rishabhiitbhu/keras-cnn-starter). So i made just some tunes in CNN and get near 90% on CV, but just 7% on test! I predict this model on train set and get near 97%, so i think there is no mistake in model. Any suggestions?", "votes": null }, { "id": "274294", "postDate": "01/26/2018 09:15:18", "content": "

most likely you are leaking class info, try print(df) see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation. Can you check this ? I did not get time to check this ! Plus don't predict on train data try predicting on validation data and then check the class label sequence once ! Let me know if you have any progress on this..thanks :)

", "rawMarkdown": "most likely you are leaking class info, try print(df) see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation. Can you check this ? I did not get time to check this ! Plus don't predict on train data try predicting on validation data and then check the class label sequence once ! Let me know if you have any progress on this..thanks :)", "votes": null }, { "id": "274335", "postDate": "01/26/2018 11:33:11", "content": "

Thanks! But which one df to check? After load train images (size each 512,512 ), i have X array size of (2750,512,512,3) and Y array size of (2750, 10), so it is obviously, that there is no extra column... Am i missing something? :)

", "rawMarkdown": "Thanks! But which one df to check? After load train images (size each 512,512 ), i have X array size of (2750,512,512,3) and Y array size of (2750, 10), so it is obviously, that there is no extra column... Am i missing something? :)", "votes": null } ], "comments": [ { "id": 268553, "author_name": "hamyadlab", "author_url": "", "post_date": "01/14/2018 21:08:50", "content": "

10 percent accuracy means that the classifier is predicting totally randomly, regardless of the given input image.

", "votes": null, "replies": [ { "id": 268559, "author_name": "craigglastonbury", "author_url": "", "post_date": "01/14/2018 21:47:51", "content": "

You almost certainly have the labels switched when you predict.

", "votes": null, "replies": [] }, { "id": 268575, "author_name": "inversion", "author_url": "", "post_date": "01/14/2018 23:08:38", "content": "

Or, the model is terribly over-fit on the training data.

", "votes": null, "replies": [] }, { "id": 268583, "author_name": "craigglastonbury", "author_url": "", "post_date": "01/14/2018 23:39:22", "content": "

You wouldn't expect a high validation accuracy.

", "votes": null, "replies": [] }, { "id": 269297, "author_name": "", "author_url": "", "post_date": "01/16/2018 15:52:37", "content": "

most likely you are leaking class info, try print(df) see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation.

", "votes": null, "replies": [] }, { "id": 269351, "author_name": "hamyadlab", "author_url": "", "post_date": "01/16/2018 17:11:35", "content": "

thanks a lot. But the problem is now solved. As I said, it was related to a very subtle mistake in my code during loading weights of a pre-trained model.

", "votes": null, "replies": [] }, { "id": 273010, "author_name": "tothink", "author_url": "", "post_date": "01/24/2018 02:34:45", "content": "

classic

", "votes": null, "replies": [] } ] }, { "id": 268577, "author_name": "hamyadlab", "author_url": "", "post_date": "01/14/2018 23:18:29", "content": "

No, My train accuracy is about 93 and my validation accuracy is now above 96. I am sure it is not related to overfitting.

", "votes": null, "replies": [ { "id": 268777, "author_name": "melgor", "author_url": "", "post_date": "01/15/2018 15:21:33", "content": "

Did you find the reason of big gap between training set and testing set? Look like that you achieve ~75% so gap is much lower. I was preparing the validation set using same transformation like was used for test set, currently with no success.

", "votes": null, "replies": [] }, { "id": 269078, "author_name": "hamyadlab", "author_url": "", "post_date": "01/16/2018 06:35:38", "content": "

Yes. It was a coding mistake. Now my validation is test accuracy is 80 percent.

", "votes": null, "replies": [] }, { "id": 269101, "author_name": "melgor", "author_url": "", "post_date": "01/16/2018 07:49:03", "content": "

Could you tell me what kind of issue is here? For your case it is just label swapping?\nI have 87% at modified-validation set (images look like in test set)\nSame model have 64% in LB.

\n\n

Could you clarify how do you create a validation set?

", "votes": null, "replies": [] }, { "id": 269479, "author_name": "hamyadlab", "author_url": "", "post_date": "01/16/2018 22:39:34", "content": "

I just randomly selected 50 images from each class and put them in the corresponding class in the validation set.

", "votes": null, "replies": [] }, { "id": 270428, "author_name": "melgor", "author_url": "", "post_date": "01/18/2018 07:50:41", "content": "

Sorry for being insistent, but could you tell more about your methodology of creating a validation set? I'm still looking for a gap ~20% between my validation and test set.

\n\n
    \n

  1. When you are using validation set, do you resize the image to input network size or make center crop?

    2. \n

  2. What image size do you use?

    3. \n

  3. Did you apply any resizing, gamma correction or JPEG compression to validation set to imitate test set?

    4. \n

  4. You mentioned that you have 93% in validation set, then you have 80%. What did you change in pipeline?

    5. \n
", "votes": null, "replies": [] }, { "id": 270475, "author_name": "hamyadlab", "author_url": "", "post_date": "01/18/2018 09:48:38", "content": "

1 - No, Resizing is not a good idea.\n2 - (3, 224, 224)\n3 - Not yet!\n4 - Nothing

", "votes": null, "replies": [] } ] }, { "id": 272056, "author_name": "nad136", "author_url": "", "post_date": "01/22/2018 06:48:02", "content": "

Hi, I have a similar problem. My validation accuracy is not as high as yours, only about 60%, but still I end up with only 10% for the test set. What could I be doing wrong? \nThanks a lot!

", "votes": null, "replies": [ { "id": 272068, "author_name": "hamyadlab", "author_url": "", "post_date": "01/22/2018 07:32:16", "content": "

Hi. In my case, at the time of testing, I had forgotten to load the trained weights correctly, and so the weights were randomly initialized and the network was classifying randomly. That's why I got only 10 percent test accuracy.

\n\n

Possible solution:

\n\n

In your case maybe this is the reason or maybe your labels order are incorrect. Ensure that during testing, you don't shuffle your test data.

", "votes": null, "replies": [] }, { "id": 272088, "author_name": "nad136", "author_url": "", "post_date": "01/22/2018 08:40:59", "content": "

Thank you! I will look into this!

", "votes": null, "replies": [] }, { "id": 272106, "author_name": "mxdbld", "author_url": "", "post_date": "01/22/2018 09:29:31", "content": "

two good rules to debug your model :\n- Make it overfit on a small subset. If it does, the network does actually learn and it indicates that you are feeding it with good data / labels\n- predict on the train dataset et assert the predicted labels are coherent with the true ones.

", "votes": null, "replies": [] }, { "id": 272214, "author_name": "beckgom", "author_url": "", "post_date": "01/22/2018 14:46:16", "content": "

I had stuck in same situation.

\n\n

Loading the weight was confirmed by using validation data in test mode. In this case, the result seemed working well.

\n\n

labels are as below. It seams that there is no problem.\n'HTC-1-M7 LG-Nexus-5x Motorola-Droid-Maxx Motorola-Nexus-6 Motorola-X Samsung-Galaxy-Note3 Samsung-Galaxy-S4 Sony-NEX-7 iPhone-4s iPhone-6'

\n\n

should I check not to shuffle the test data? is it a problem?

", "votes": null, "replies": [] }, { "id": 272231, "author_name": "hamyadlab", "author_url": "", "post_date": "01/22/2018 15:33:19", "content": "

If you shuffle the test data, the order will be different from the order given in the sample_submission.csv. So, you should be careful about ordering. But in general, according to the rules of this competition, the order of test data is not important; ie., you can submit your predictions in any order.

", "votes": null, "replies": [] } ] }, { "id": 274030, "author_name": "anandsonawane", "author_url": "", "post_date": "01/25/2018 17:25:08", "content": "

Is there any way I can understand if I am messig my labels up ? Because I trained a resnet152, from scratch, resnet50 using transfer Learning keeping layer.trainable as False and also making them trainable as True but for all combinations, I am getting my train and Val accuracy in the 90's and 80's respectively but my LB score as 0.181 or 0.194 etc. What can be the reason for this ?

", "votes": null, "replies": [ { "id": 274041, "author_name": "mxdbld", "author_url": "", "post_date": "01/25/2018 18:14:52", "content": "", "votes": null, "replies": [] } ] }, { "id": 274283, "author_name": "famazon", "author_url": "", "post_date": "01/26/2018 08:27:31", "content": "

I have same problem.... I use this Kernel (https://www.kaggle.com/rishabhiitbhu/keras-cnn-starter). So i made just some tunes in CNN and get near 90% on CV, but just 7% on test! I predict this model on train set and get near 97%, so i think there is no mistake in model. Any suggestions?

", "votes": null, "replies": [ { "id": 274294, "author_name": "anandsonawane", "author_url": "", "post_date": "01/26/2018 09:15:18", "content": "

most likely you are leaking class info, try print(df) see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation. Can you check this ? I did not get time to check this ! Plus don't predict on train data try predicting on validation data and then check the class label sequence once ! Let me know if you have any progress on this..thanks :)

", "votes": null, "replies": [] }, { "id": 274335, "author_name": "famazon", "author_url": "", "post_date": "01/26/2018 11:33:11", "content": "

Thanks! But which one df to check? After load train images (size each 512,512 ), i have X array size of (2750,512,512,3) and Y array size of (2750, 10), so it is obviously, that there is no extra column... Am i missing something? :)

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "268552": "How is it possible to have 93 percent validation accuracy and 10 percent accuracy on test set? any suggestion?", "268553": "10 percent accuracy means that the classifier is predicting totally randomly, regardless of the given input image.", "268559": "You almost certainly have the labels switched when you predict.", "268575": "Or, the model is terribly over-fit on the training data.", "268577": "No, My train accuracy is about 93 and my validation accuracy is now above 96. I am sure it is not related to overfitting.", "268583": "You wouldn't expect a high validation accuracy.", "268777": "Did you find the reason of big gap between training set and testing set? Look like that you achieve ~75% so gap is much lower. I was preparing the validation set using same transformation like was used for test set, currently with no success.", "269078": "Yes. It was a coding mistake. Now my validation is test accuracy is 80 percent.", "269101": "Could you tell me what kind of issue is here? For your case it is just label swapping?\nI have 87% at modified-validation set (images look like in test set)\nSame model have 64% in LB. \n\nCould you clarify how do you create a validation set?", "269297": "most likely you are leaking class info, try `print(df)` see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation.", "269351": "thanks a lot. But the problem is now solved. As I said, it was related to a very subtle mistake in my code during loading weights of a pre-trained model.", "269479": "I just randomly selected 50 images from each class and put them in the corresponding class in the validation set.", "270428": "Sorry for being insistent, but could you tell more about your methodology of creating a validation set? I'm still looking for a gap ~20% between my validation and test set. \n\n1. When you are using validation set, do you resize the image to input network size or make center crop?\n\n2. What image size do you use?\n\n3. Did you apply any resizing, gamma correction or JPEG compression to validation set to imitate test set?\n\n4. You mentioned that you have 93% in validation set, then you have 80%. What did you change in pipeline?", "270475": "1 - No, Resizing is not a good idea.\n2 - (3, 224, 224)\n3 - Not yet!\n4 - Nothing", "272056": "Hi, I have a similar problem. My validation accuracy is not as high as yours, only about 60%, but still I end up with only 10% for the test set. What could I be doing wrong? \nThanks a lot!", "272068": "Hi. In my case, at the time of testing, I had forgotten to **load the trained weights** correctly, and so the weights were randomly initialized and the network was classifying randomly. That's why I got only 10 percent test accuracy. \n\n### Possible solution:\nIn your case maybe this is the reason or maybe your labels order are incorrect. Ensure that during testing, you don't shuffle your test data.", "272088": "Thank you! I will look into this!", "272106": "two good rules to debug your model :\n- Make it overfit on a small subset. If it does, the network does actually learn and it indicates that you are feeding it with good data / labels\n- predict on the train dataset et assert the predicted labels are coherent with the true ones.", "272214": "I had stuck in same situation. \n\nLoading the weight was confirmed by using validation data in test mode. In this case, the result seemed working well.\n\nlabels are as below. It seams that there is no problem.\n'HTC-1-M7 LG-Nexus-5x Motorola-Droid-Maxx Motorola-Nexus-6 Motorola-X Samsung-Galaxy-Note3 Samsung-Galaxy-S4 Sony-NEX-7 iPhone-4s iPhone-6'\n\nshould I check not to shuffle the test data? is it a problem?", "272231": "If you shuffle the test data, the order will be different from the order given in the `sample_submission.csv`. So, you should be careful about ordering. But in general, according to the rules of this competition, the order of test data is not important; ie., you can submit your predictions in any order.", "273010": "classic", "274030": "Is there any way I can understand if I am messig my labels up ? Because I trained a resnet152, from scratch, resnet50 using transfer Learning keeping layer.trainable as False and also making them trainable as True but for all combinations, I am getting my train and Val accuracy in the 90's and 80's respectively but my LB score as 0.181 or 0.194 etc. What can be the reason for this ?", "274041": "predict on train and see if it matches\n- unit testing", "274283": "I have same problem.... I use this Kernel (https://www.kaggle.com/rishabhiitbhu/keras-cnn-starter). So i made just some tunes in CNN and get near 90% on CV, but just 7% on test! I predict this model on train set and get near 97%, so i think there is no mistake in model. Any suggestions?", "274294": "most likely you are leaking class info, try print(df) see if you have a column named \"Unnamed: 0\" or a cloumn 0, which is just same as index but then the classifier just assumes that to be a feature and happily overfits both training and validation. Can you check this ? I did not get time to check this ! Plus don't predict on train data try predicting on validation data and then check the class label sequence once ! Let me know if you have any progress on this..thanks :)", "274335": "Thanks! But which one df to check? After load train images (size each 512,512 ), i have X array size of (2750,512,512,3) and Y array size of (2750, 10), so it is obviously, that there is no extra column... Am i missing something? :)" }, "source": "meta" }