{
  "id": 229428,
  "title": "Model Evaluate Failure: Pls help me",
  "url": "/competitions/hpa-single-cell-image-classification/discussion/229428",
  "author_name": "Yann Garcia",
  "post_date": "2021-03-30T06:42:57.807000",
  "votes": 2,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hello,</p>\n<p>I'm a beginner in Deep Learning and I'm working on 'Human Protein Atlas - Single Cell Classification' compete. I got the following exception (see at the end of this message) while executing evaluate() method of my CNN Keras Classifier model. To be honest, I don't understand what's going wrong.<br>\nI tried to find help on google based on  the exception but it didn't help me.</p>\n<p>My notebook execution (code + logs) can be found <a href=\"https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me?scriptVersionId=58202800\" target=\"_blank\">here</a>. </p>\n<p>The full notebook is publicly shared <a href=\"https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me\" target=\"_blank\">here</a>.</p>\n<p>To summarize what I did:<br>\n1) I creates a Training and Validation TF Datasets based on train images and train labels<br>\n2) I applied some basic augmentations on the Training Datasets <br>\n2) I created a basic neural network based on VGG19 pretrained model<br>\n3) I tried to evaluate the model using the Validation dataset to get a first 'quick &amp; dirty' result<br>\n4) And I got the following exception:</p>\n<pre><code>OutOfRangeError: 8 root error(s) found.\n  (0) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[tpu_compile_succeeded_assert/_17210564861776349517/_5/_303]]\n  (1) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[strided_slice_39/_334]]\n  (2) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[strided_slice_15/_286]]\n  (3) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[cluster_test_function/_execute_7_0/_383]]\n  (4) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[strided_slice_27/_310]]\n  (5) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[TPUReplicate/_compile/_14782605444866121645/_4/_276]]\n  (6) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[GroupCrossDeviceControlEdges_2/Identity_7/_411]]\n  (7) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n0 successful operations.\n1 derived errors ignored.\n</code></pre>\n<p>Many thanks in advance for your help</p>\n<p>Yann</p>",
  "messages": [
    {
      "id": 1257095,
      "postDate": "2021-03-30T14:16:23.023Z",
      "content": "<p>You set steps argument with 16. But your validation df size is 227. After batching(replicas=8 with tpu and batch size=32) num steps should be 1. This is what i saw from your code.  Hope this helps.</p>",
      "rawMarkdown": "You set steps argument with 16. But your validation df size is 227. After batching(replicas=8 with tpu and batch size=32) num steps should be 1. This is what i saw from your code.  Hope this helps.",
      "votes": 2,
      "replies": [
        {
          "id": 1257185,
          "postDate": "2021-03-30T15:41:09.270Z",
          "content": "<p>Hello Oğuzhan,</p>\n<p>Many thanks for your help, you were right, I misunderstood the role of the steps parameter.</p>\n<p>Have a nice evening</p>\n<p>Yann</p>",
          "rawMarkdown": "Hello Oğuzhan,\n\nMany thanks for your help, you were right, I misunderstood the role of the steps parameter.\n\nHave a nice evening\n\nYann",
          "votes": 2
        }
      ]
    },
    {
      "id": 1256677,
      "postDate": "2021-03-30T06:42:57.807Z",
      "content": "<p>Hello,</p>\n<p>I'm a beginner in Deep Learning and I'm working on 'Human Protein Atlas - Single Cell Classification' compete. I got the following exception (see at the end of this message) while executing evaluate() method of my CNN Keras Classifier model. To be honest, I don't understand what's going wrong.<br>\nI tried to find help on google based on  the exception but it didn't help me.</p>\n<p>My notebook execution (code + logs) can be found <a href=\"https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me?scriptVersionId=58202800\" target=\"_blank\">here</a>. </p>\n<p>The full notebook is publicly shared <a href=\"https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me\" target=\"_blank\">here</a>.</p>\n<p>To summarize what I did:<br>\n1) I creates a Training and Validation TF Datasets based on train images and train labels<br>\n2) I applied some basic augmentations on the Training Datasets <br>\n2) I created a basic neural network based on VGG19 pretrained model<br>\n3) I tried to evaluate the model using the Validation dataset to get a first 'quick &amp; dirty' result<br>\n4) And I got the following exception:</p>\n<pre><code>OutOfRangeError: 8 root error(s) found.\n  (0) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[tpu_compile_succeeded_assert/_17210564861776349517/_5/_303]]\n  (1) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[strided_slice_39/_334]]\n  (2) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[strided_slice_15/_286]]\n  (3) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[cluster_test_function/_execute_7_0/_383]]\n  (4) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[strided_slice_27/_310]]\n  (5) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[TPUReplicate/_compile/_14782605444866121645/_4/_276]]\n  (6) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n     [[GroupCrossDeviceControlEdges_2/Identity_7/_411]]\n  (7) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n     [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n0 successful operations.\n1 derived errors ignored.\n</code></pre>\n<p>Many thanks in advance for your help</p>\n<p>Yann</p>",
      "rawMarkdown": "Hello,\n\nI'm a beginner in Deep Learning and I'm working on 'Human Protein Atlas - Single Cell Classification' compete. I got the following exception (see at the end of this message) while executing evaluate() method of my CNN Keras Classifier model. To be honest, I don't understand what's going wrong.\nI tried to find help on google based on  the exception but it didn't help me.\n\nMy notebook execution (code + logs) can be found [here](https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me?scriptVersionId=58202800). \n\nThe full notebook is publicly shared [here](https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me).\n\nTo summarize what I did:\n1) I creates a Training and Validation TF Datasets based on train images and train labels\n2) I applied some basic augmentations on the Training Datasets \n2) I created a basic neural network based on VGG19 pretrained model\n3) I tried to evaluate the model using the Validation dataset to get a first 'quick & dirty' result\n4) And I got the following exception:\n```\nOutOfRangeError: 8 root error(s) found.\n  (0) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[tpu_compile_succeeded_assert/_17210564861776349517/_5/_303]]\n  (1) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[strided_slice_39/_334]]\n  (2) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[strided_slice_15/_286]]\n  (3) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[cluster_test_function/_execute_7_0/_383]]\n  (4) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[strided_slice_27/_310]]\n  (5) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[TPUReplicate/_compile/_14782605444866121645/_4/_276]]\n  (6) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[GroupCrossDeviceControlEdges_2/Identity_7/_411]]\n  (7) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n0 successful operations.\n1 derived errors ignored.\n```\n\nMany thanks in advance for your help\n\nYann",
      "votes": 2
    },
    {
      "id": 1261896,
      "postDate": "2021-04-03T14:52:57.247Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1257095,
      "author_name": "Oguzhan Nefesoglu",
      "author_url": "",
      "post_date": "2021-03-30T14:16:23.023000",
      "content": "<p>You set steps argument with 16. But your validation df size is 227. After batching(replicas=8 with tpu and batch size=32) num steps should be 1. This is what i saw from your code.  Hope this helps.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 1257185,
          "author_name": "Yann Garcia",
          "author_url": "",
          "post_date": "2021-03-30T15:41:09.270000",
          "content": "<p>Hello Oğuzhan,</p>\n<p>Many thanks for your help, you were right, I misunderstood the role of the steps parameter.</p>\n<p>Have a nice evening</p>\n<p>Yann</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1261896,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-04-03T14:52:57.247000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1257095": "You set steps argument with 16. But your validation df size is 227. After batching(replicas=8 with tpu and batch size=32) num steps should be 1. This is what i saw from your code.  Hope this helps.",
    "1256677": "Hello,\n\nI'm a beginner in Deep Learning and I'm working on 'Human Protein Atlas - Single Cell Classification' compete. I got the following exception (see at the end of this message) while executing evaluate() method of my CNN Keras Classifier model. To be honest, I don't understand what's going wrong.\nI tried to find help on google based on  the exception but it didn't help me.\n\nMy notebook execution (code + logs) can be found [here](https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me?scriptVersionId=58202800). \n\nThe full notebook is publicly shared [here](https://www.kaggle.com/yanngarcia/model-evaluate-failure-pls-help-me).\n\nTo summarize what I did:\n1) I creates a Training and Validation TF Datasets based on train images and train labels\n2) I applied some basic augmentations on the Training Datasets \n2) I created a basic neural network based on VGG19 pretrained model\n3) I tried to evaluate the model using the Validation dataset to get a first 'quick & dirty' result\n4) And I got the following exception:\n```\nOutOfRangeError: 8 root error(s) found.\n  (0) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[tpu_compile_succeeded_assert/_17210564861776349517/_5/_303]]\n  (1) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[strided_slice_39/_334]]\n  (2) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[strided_slice_15/_286]]\n  (3) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[cluster_test_function/_execute_7_0/_383]]\n  (4) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[strided_slice_27/_310]]\n  (5) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[TPUReplicate/_compile/_14782605444866121645/_4/_276]]\n  (6) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n\t [[GroupCrossDeviceControlEdges_2/Identity_7/_411]]\n  (7) Out of range: {{function_node __inference_test_function_5167}} End of sequence\n\t [[{{node cond_15/else/_150/cond_15/IteratorGetNext}}]]\n0 successful operations.\n1 derived errors ignored.\n```\n\nMany thanks in advance for your help\n\nYann",
    "1261896": ""
  }
}