{
  "id": 169585,
  "title": "Why does the prediction or CV scores depend on Pytorch vs Tensorflow codes?",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/169585",
  "author_name": "",
  "post_date": "2020-07-24T10:56:42.574723200Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi All,\nWhy does the prediction results/outputs depend on whether we use Pytorch or Tensorflow implementation? And I see folks here saying - Tensorflow codes are giving better result compared to Pytorch implementation? I have also observed the same running the Tensorflow codes running on TPUs gave me better score.</p>",
  "messages": [
    {
      "id": "943425",
      "postDate": "07/24/2020 10:56:42",
      "content": "<p>Hi All,\nWhy does the prediction results/outputs depend on whether we use Pytorch or Tensorflow implementation? And I see folks here saying - Tensorflow codes are giving better result compared to Pytorch implementation? I have also observed the same running the Tensorflow codes running on TPUs gave me better score.</p>",
      "rawMarkdown": "Hi All,\nWhy does the prediction results/outputs depend on whether we use Pytorch or Tensorflow implementation? And I see folks here saying - Tensorflow codes are giving better result compared to Pytorch implementation? I have also observed the same running the Tensorflow codes running on TPUs gave me better score.",
      "votes": null
    },
    {
      "id": "943510",
      "postDate": "07/24/2020 12:10:25",
      "content": "<p>Comparison is between:\n- PyTorch vs TensorFlow - both can run on TPU\n- TPU vs GPU</p>\n\n<p>What you have observed is that running TensorFlow on TPUs is the best option for this competition.</p>",
      "rawMarkdown": "Comparison is between:\n- PyTorch vs TensorFlow - both can run on TPU\n- TPU vs GPU\n\nWhat you have observed is that running TensorFlow on TPUs is the best option for this competition.",
      "votes": null
    },
    {
      "id": "943606",
      "postDate": "07/24/2020 13:16:18",
      "content": "<p>Thank you for the comment. I edited my content.</p>\n\n<p>But any specific reasons we need to learn on why this competition data is doing well with Tensorflow code on TPUs?  </p>",
      "rawMarkdown": "Thank you for the comment. I edited my content.\n\nBut any specific reasons we need to learn on why this competition data is doing well with Tensorflow code on TPUs?",
      "votes": null
    },
    {
      "id": "943644",
      "postDate": "07/24/2020 13:41:09",
      "content": "<p>I think the dissimilarity of your results don't stem from tpu or gpu. It is originated from randomness. because models' weights are initialized randomly every time. If there is better result one of the PyThorch and Tensorflow, this could be originated from initialization of the parameters  based on the framework. But it doesn't mean tensorflow will be better than PyThorch always and these little coding differences don't impact the model's success that much. </p>",
      "rawMarkdown": "I think the dissimilarity of your results don't stem from tpu or gpu. It is originated from randomness. because models' weights are initialized randomly every time. If there is better result one of the PyThorch and Tensorflow, this could be originated from initialization of the parameters  based on the framework. But it doesn't mean tensorflow will be better than PyThorch always and these little coding differences don't impact the model's success that much.",
      "votes": null
    },
    {
      "id": "943801",
      "postDate": "07/24/2020 15:37:56",
      "content": "<p>Honestly, it's probably just chance. Do you have empirical data? Maybe there are more TensorFlow kernels using TPUs. Which have more power and allow for better training</p>",
      "rawMarkdown": "Honestly, it's probably just chance. Do you have empirical data? Maybe there are more TensorFlow kernels using TPUs. Which have more power and allow for better training",
      "votes": null
    },
    {
      "id": "943908",
      "postDate": "07/24/2020 17:00:17",
      "content": "<p>No,  for this competition I used kernels of Chris and improvised further on with some additions with different CV strategy and different pre-trained models .  This is Just my observation with most of the kernels and discussions here.</p>",
      "rawMarkdown": "No,  for this competition I used kernels of Chris and improvised further on with some additions with different CV strategy and different pre-trained models .  This is Just my observation with most of the kernels and discussions here.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 943510,
      "author_name": "sirishks",
      "author_url": "",
      "post_date": "07/24/2020 12:10:25",
      "content": "<p>Comparison is between:\n- PyTorch vs TensorFlow - both can run on TPU\n- TPU vs GPU</p>\n\n<p>What you have observed is that running TensorFlow on TPUs is the best option for this competition.</p>",
      "votes": null,
      "replies": [
        {
          "id": 943606,
          "author_name": "shravankoninti",
          "author_url": "",
          "post_date": "07/24/2020 13:16:18",
          "content": "<p>Thank you for the comment. I edited my content.</p>\n\n<p>But any specific reasons we need to learn on why this competition data is doing well with Tensorflow code on TPUs?  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 943801,
          "author_name": "amneves",
          "author_url": "",
          "post_date": "07/24/2020 15:37:56",
          "content": "<p>Honestly, it's probably just chance. Do you have empirical data? Maybe there are more TensorFlow kernels using TPUs. Which have more power and allow for better training</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 943908,
          "author_name": "shravankoninti",
          "author_url": "",
          "post_date": "07/24/2020 17:00:17",
          "content": "<p>No,  for this competition I used kernels of Chris and improvised further on with some additions with different CV strategy and different pre-trained models .  This is Just my observation with most of the kernels and discussions here.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 943644,
      "author_name": "regaipkurt",
      "author_url": "",
      "post_date": "07/24/2020 13:41:09",
      "content": "<p>I think the dissimilarity of your results don't stem from tpu or gpu. It is originated from randomness. because models' weights are initialized randomly every time. If there is better result one of the PyThorch and Tensorflow, this could be originated from initialization of the parameters  based on the framework. But it doesn't mean tensorflow will be better than PyThorch always and these little coding differences don't impact the model's success that much. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "943425": "Hi All,\nWhy does the prediction results/outputs depend on whether we use Pytorch or Tensorflow implementation? And I see folks here saying - Tensorflow codes are giving better result compared to Pytorch implementation? I have also observed the same running the Tensorflow codes running on TPUs gave me better score.",
    "943510": "Comparison is between:\n- PyTorch vs TensorFlow - both can run on TPU\n- TPU vs GPU\n\nWhat you have observed is that running TensorFlow on TPUs is the best option for this competition.",
    "943606": "Thank you for the comment. I edited my content.\n\nBut any specific reasons we need to learn on why this competition data is doing well with Tensorflow code on TPUs?",
    "943644": "I think the dissimilarity of your results don't stem from tpu or gpu. It is originated from randomness. because models' weights are initialized randomly every time. If there is better result one of the PyThorch and Tensorflow, this could be originated from initialization of the parameters  based on the framework. But it doesn't mean tensorflow will be better than PyThorch always and these little coding differences don't impact the model's success that much.",
    "943801": "Honestly, it's probably just chance. Do you have empirical data? Maybe there are more TensorFlow kernels using TPUs. Which have more power and allow for better training",
    "943908": "No,  for this competition I used kernels of Chris and improvised further on with some additions with different CV strategy and different pre-trained models .  This is Just my observation with most of the kernels and discussions here."
  },
  "source": "meta"
}