{
  "id": 100979,
  "title": "Different result with predict_generator",
  "url": "/competitions/aptos2019-blindness-detection/discussion/100979",
  "author_name": "",
  "post_date": "2019-07-22T13:01:31.346385300Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I am using predict_generator to predict labels of test images.However every time i do that (with my same model , that is without re-training and in same run time session) i get different label predictions.\nJust by running the code block that predicts classes from model, im getting different class predictions.</p>\n\n<p>sample code :</p>\n\n<h1>test_generator.reset()</h1>\n\n<h1>step_size_test = test_generator.n//test_generator.batch_size</h1>\n\n<h1>preds = model.predict_generator(test_generator, steps=step_size_test)</h1>\n\n<h1>predictions = [np.argmax(pred) for pred in preds]</h1>\n\n<p>In test generator i have made sure that shuffle is false.</p>\n\n<p>also this is what i found on github , please tell if this makes any relevance here.</p>\n\n<p><a href=\"https://github.com/keras-team/keras/issues/3477#issuecomment-360022086\">https://github.com/keras-team/keras/issues/3477#issuecomment-360022086</a></p>\n\n<p>I guess predict_generator is reading file in different order.</p>\n\n<p>has anyone faced similar problem?</p>\n\n<p>Thanks in advance!</p>",
  "messages": [
    {
      "id": "581830",
      "postDate": "07/22/2019 13:01:31",
      "content": "<p>I am using predict_generator to predict labels of test images.However every time i do that (with my same model , that is without re-training and in same run time session) i get different label predictions.\nJust by running the code block that predicts classes from model, im getting different class predictions.</p>\n\n<p>sample code :</p>\n\n<h1>test_generator.reset()</h1>\n\n<h1>step_size_test = test_generator.n//test_generator.batch_size</h1>\n\n<h1>preds = model.predict_generator(test_generator, steps=step_size_test)</h1>\n\n<h1>predictions = [np.argmax(pred) for pred in preds]</h1>\n\n<p>In test generator i have made sure that shuffle is false.</p>\n\n<p>also this is what i found on github , please tell if this makes any relevance here.</p>\n\n<p><a href=\"https://github.com/keras-team/keras/issues/3477#issuecomment-360022086\">https://github.com/keras-team/keras/issues/3477#issuecomment-360022086</a></p>\n\n<p>I guess predict_generator is reading file in different order.</p>\n\n<p>has anyone faced similar problem?</p>\n\n<p>Thanks in advance!</p>",
      "rawMarkdown": "I am using predict_generator to predict labels of test images.However every time i do that (with my same model , that is without re-training and in same run time session) i get different label predictions.\nJust by running the code block that predicts classes from model, im getting different class predictions.\n\nsample code :\n\n#test_generator.reset()\n#step_size_test = test_generator.n//test_generator.batch_size\n#preds = model.predict_generator(test_generator, steps=step_size_test)\n#predictions = [np.argmax(pred) for pred in preds] \n\nIn test generator i have made sure that shuffle is false.\n\nalso this is what i found on github , please tell if this makes any relevance here.\n\nhttps://github.com/keras-team/keras/issues/3477#issuecomment-360022086\n\nI guess predict_generator is reading file in different order.\n\nhas anyone faced similar problem?\n\nThanks in advance!",
      "votes": null
    },
    {
      "id": "581877",
      "postDate": "07/22/2019 13:43:19",
      "content": "<p>I face the same problem. <a href=\"/mathormad\">@mathormad</a> , Duc do you have the same problem? </p>\n\n<p>In any cases, to solve this problem, please look at <a href=\"/mathormad\">@mathormad</a> <a href=\"https://www.kaggle.com/mathormad/aptos-resnet50-baseline\">excellent kernel</a>, where he read the test file and predict one by one. By not using predict_generator, the indeterministic is gone.</p>",
      "rawMarkdown": "I face the same problem. @mathormad , Duc do you have the same problem? \n\nIn any cases, to solve this problem, please look at @mathormad [excellent kernel](https://www.kaggle.com/mathormad/aptos-resnet50-baseline), where he read the test file and predict one by one. By not using predict_generator, the indeterministic is gone.",
      "votes": null
    },
    {
      "id": "581918",
      "postDate": "07/22/2019 14:39:20",
      "content": "<p>Make sure augmentation or custom preprocessing with random operations are disabled in generator.</p>\n\n<p>Offtop:\nstepsizetest = testgenerator.n//testgenerator.batch_size could be not what you expect in case say n=10 and batch_size=3</p>",
      "rawMarkdown": "Make sure augmentation or custom preprocessing with random operations are disabled in generator.\n\nOfftop:\nstepsizetest = testgenerator.n//testgenerator.batch_size could be not what you expect in case say n=10 and batch_size=3",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 581877,
      "author_name": "ratthachat",
      "author_url": "",
      "post_date": "07/22/2019 13:43:19",
      "content": "<p>I face the same problem. <a href=\"/mathormad\">@mathormad</a> , Duc do you have the same problem? </p>\n\n<p>In any cases, to solve this problem, please look at <a href=\"/mathormad\">@mathormad</a> <a href=\"https://www.kaggle.com/mathormad/aptos-resnet50-baseline\">excellent kernel</a>, where he read the test file and predict one by one. By not using predict_generator, the indeterministic is gone.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 581918,
      "author_name": "vzaguskin",
      "author_url": "",
      "post_date": "07/22/2019 14:39:20",
      "content": "<p>Make sure augmentation or custom preprocessing with random operations are disabled in generator.</p>\n\n<p>Offtop:\nstepsizetest = testgenerator.n//testgenerator.batch_size could be not what you expect in case say n=10 and batch_size=3</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "581830": "I am using predict_generator to predict labels of test images.However every time i do that (with my same model , that is without re-training and in same run time session) i get different label predictions.\nJust by running the code block that predicts classes from model, im getting different class predictions.\n\nsample code :\n\n#test_generator.reset()\n#step_size_test = test_generator.n//test_generator.batch_size\n#preds = model.predict_generator(test_generator, steps=step_size_test)\n#predictions = [np.argmax(pred) for pred in preds] \n\nIn test generator i have made sure that shuffle is false.\n\nalso this is what i found on github , please tell if this makes any relevance here.\n\nhttps://github.com/keras-team/keras/issues/3477#issuecomment-360022086\n\nI guess predict_generator is reading file in different order.\n\nhas anyone faced similar problem?\n\nThanks in advance!",
    "581877": "I face the same problem. @mathormad , Duc do you have the same problem? \n\nIn any cases, to solve this problem, please look at @mathormad [excellent kernel](https://www.kaggle.com/mathormad/aptos-resnet50-baseline), where he read the test file and predict one by one. By not using predict_generator, the indeterministic is gone.",
    "581918": "Make sure augmentation or custom preprocessing with random operations are disabled in generator.\n\nOfftop:\nstepsizetest = testgenerator.n//testgenerator.batch_size could be not what you expect in case say n=10 and batch_size=3"
  },
  "source": "meta"
}