{
  "id": 145900,
  "title": "Is there a reason this loop that predicts labels using a tensorflow model should grow in memory?",
  "url": "/competitions/flower-classification-with-tpus/discussion/145900",
  "author_name": "Rashan Arshad",
  "post_date": "2020-04-25T01:23:09.316000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Here is the loop, aside from the addition of ids and predictions into their respective lists , this loop seems to grows about 0.04gb in memory every iteration.</p>\n\n<p><code>\nfor image, label in prediction_images:\n    print(i)\n    ids.append(label.numpy())\n    test_image_list = [np.expand_dims(image, 0) for x in images_1]\n    test_dict = {'input_1': test_image_list, \n                 'input_2': images_1}\n    test_dict = tf.data.Dataset.from_tensor_slices(test_dict)\n    predictions = model.predict(test_dict)\n    index, value = max(enumerate(predictions), key=operator.itemgetter(1))\n    pred_list.append(index)\n    del index, value, predictions, test_dict, test_image_list\n    i+=1\n</code>\nthe reason I am generating predictions this way is because Ive trained a siamese network. And to get the predicted label in a test set, I have to compare each image in the test set with an example image from every class (104 classes in this case) and then I take the label with the highest confidence</p>",
  "messages": [
    {
      "id": 819904,
      "postDate": "2020-04-25T01:23:09.317Z",
      "content": "<p>Here is the loop, aside from the addition of ids and predictions into their respective lists , this loop seems to grows about 0.04gb in memory every iteration.</p>\n\n<p><code>\nfor image, label in prediction_images:\n    print(i)\n    ids.append(label.numpy())\n    test_image_list = [np.expand_dims(image, 0) for x in images_1]\n    test_dict = {'input_1': test_image_list, \n                 'input_2': images_1}\n    test_dict = tf.data.Dataset.from_tensor_slices(test_dict)\n    predictions = model.predict(test_dict)\n    index, value = max(enumerate(predictions), key=operator.itemgetter(1))\n    pred_list.append(index)\n    del index, value, predictions, test_dict, test_image_list\n    i+=1\n</code>\nthe reason I am generating predictions this way is because Ive trained a siamese network. And to get the predicted label in a test set, I have to compare each image in the test set with an example image from every class (104 classes in this case) and then I take the label with the highest confidence</p>",
      "rawMarkdown": "Here is the loop, aside from the addition of ids and predictions into their respective lists , this loop seems to grows about 0.04gb in memory every iteration.\n\n```\nfor image, label in prediction_images:\n    print(i)\n    ids.append(label.numpy())\n    test_image_list = [np.expand_dims(image, 0) for x in images_1]\n    test_dict = {'input_1': test_image_list, \n                 'input_2': images_1}\n    test_dict = tf.data.Dataset.from_tensor_slices(test_dict)\n    predictions = model.predict(test_dict)\n    index, value = max(enumerate(predictions), key=operator.itemgetter(1))\n    pred_list.append(index)\n    del index, value, predictions, test_dict, test_image_list\n    i+=1\n```\nthe reason I am generating predictions this way is because Ive trained a siamese network. And to get the predicted label in a test set, I have to compare each image in the test set with an example image from every class (104 classes in this case) and then I take the label with the highest confidence"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "819904": "Here is the loop, aside from the addition of ids and predictions into their respective lists , this loop seems to grows about 0.04gb in memory every iteration.\n\n```\nfor image, label in prediction_images:\n    print(i)\n    ids.append(label.numpy())\n    test_image_list = [np.expand_dims(image, 0) for x in images_1]\n    test_dict = {'input_1': test_image_list, \n                 'input_2': images_1}\n    test_dict = tf.data.Dataset.from_tensor_slices(test_dict)\n    predictions = model.predict(test_dict)\n    index, value = max(enumerate(predictions), key=operator.itemgetter(1))\n    pred_list.append(index)\n    del index, value, predictions, test_dict, test_image_list\n    i+=1\n```\nthe reason I am generating predictions this way is because Ive trained a siamese network. And to get the predicted label in a test set, I have to compare each image in the test set with an example image from every class (104 classes in this case) and then I take the label with the highest confidence"
  }
}