{
  "id": 172799,
  "title": "Ideas from Google but tools for Pytorch ? ",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/172799",
  "author_name": "",
  "post_date": "2020-08-06T13:53:30.117601Z",
  "votes": 3,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Too many of recent breaktrough ideas in DL come from Google Brain ( Transformers/Bert, NASNet/Auto-ML/Efficientnet, Self training/Noisy Student,  Auto-Augment etc.)   while Facebook is just recycling Resnet(OK this is a troll ^^)</p>\n\n<p>But what strikes the most is, even tools/packages derived from google's innovations are much easier to find (at least in convienent way ) on Pytorch than Tensorflow. <br>\nFor instance you can find bunch of Auto-Augment packages, ready to use for Pytorch , but I found nothing for TF. </p>\n\n<p>Do you think the way Google abruptly abandoned <code>tf.contrib</code> has something to do with it ? thus  discouraging the community.</p>\n\n<p>Fortunately HuggingFace Transformers is now supporting TF ( but just TF 2.X and only subset of models) . </p>",
  "messages": [
    {
      "id": "960542",
      "postDate": "08/06/2020 13:53:30",
      "content": "<p>Too many of recent breaktrough ideas in DL come from Google Brain ( Transformers/Bert, NASNet/Auto-ML/Efficientnet, Self training/Noisy Student,  Auto-Augment etc.)   while Facebook is just recycling Resnet(OK this is a troll ^^)</p>\n\n<p>But what strikes the most is, even tools/packages derived from google's innovations are much easier to find (at least in convienent way ) on Pytorch than Tensorflow. <br>\nFor instance you can find bunch of Auto-Augment packages, ready to use for Pytorch , but I found nothing for TF. </p>\n\n<p>Do you think the way Google abruptly abandoned <code>tf.contrib</code> has something to do with it ? thus  discouraging the community.</p>\n\n<p>Fortunately HuggingFace Transformers is now supporting TF ( but just TF 2.X and only subset of models) . </p>",
      "rawMarkdown": "Too many of recent breaktrough ideas in DL come from Google Brain ( Transformers/Bert, NASNet/Auto-ML/Efficientnet, Self training/Noisy Student,  Auto-Augment etc.)   while Facebook is just recycling Resnet(OK this is a troll ^^)\n\nBut what strikes the most is, even tools/packages derived from google's innovations are much easier to find (at least in convienent way ) on Pytorch than Tensorflow.  \nFor instance you can find bunch of Auto-Augment packages, ready to use for Pytorch , but I found nothing for TF. \n\nDo you think the way Google abruptly abandoned `tf.contrib` has something to do with it ? thus  discouraging the community.\n\nFortunately HuggingFace Transformers is now supporting TF ( but just TF 2.X and only subset of models) .",
      "votes": null
    },
    {
      "id": "960628",
      "postDate": "08/06/2020 14:54:02",
      "content": "<p>I find Torch 1000 times easier to experiment with than Tensorflow. It seems the go-to framework for research, due to its flexibility. It's flow/fundaments are pythonic and very similar to numph. Tensorflow requires a completely different way of thinking. Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch...</p>",
      "rawMarkdown": "I find Torch 1000 times easier to experiment with than Tensorflow. It seems the go-to framework for research, due to its flexibility. It's flow/fundaments are pythonic and very similar to numph. Tensorflow requires a completely different way of thinking. Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch...",
      "votes": null
    },
    {
      "id": "960701",
      "postDate": "08/06/2020 15:55:28",
      "content": "<blockquote>\n  <p>Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch…</p>\n</blockquote>\n\n<p>I think the advantage for TF is seemless TPU support and TFrecord .    This may compensate its lack of flexibility here. \nAnd Kaggle TPU config is particularly tailored for TF. </p>\n\n<p>I was using Pytorch until last week, but then I needed to merge with TF models to improve my score ^^</p>\n\n<p>Fortunately <a href=\"/cdeotte\">@cdeotte</a> has wrote great and solid TF baseline kernels, that let frankly very little rooms for improvement :). ( I am just modifying the training loop and modify the backbone arch to add my custom heads. )</p>\n\n<p>But it's still frustrating to see that some convinient  tools are not available for TF</p>",
      "rawMarkdown": "&gt; Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch…\n\nI think the advantage for TF is seemless TPU support and TFrecord .    This may compensate its lack of flexibility here. \nAnd Kaggle TPU config is particularly tailored for TF. \n\nI was using Pytorch until last week, but then I needed to merge with TF models to improve my score ^^\n\nFortunately @cdeotte has wrote great and solid TF baseline kernels, that let frankly very little rooms for improvement :). ( I am just modifying the training loop and modify the backbone arch to add my custom heads. )\n\nBut it's still frustrating to see that some convinient  tools are not available for TF",
      "votes": null
    },
    {
      "id": "962065",
      "postDate": "08/07/2020 19:07:33",
      "content": "<p>Their are implementation in TF but the issue is it is rarely coded in TF2, oftenly in TF 1.15, even by google's author. We can wonder if they trust their new API :/</p>",
      "rawMarkdown": "Their are implementation in TF but the issue is it is rarely coded in TF2, oftenly in TF 1.15, even by google's author. We can wonder if they trust their new API :/",
      "votes": null
    },
    {
      "id": "962155",
      "postDate": "08/07/2020 21:18:42",
      "content": "<p>Indeed I'd choose Torch over TF but tf does better right now…</p>",
      "rawMarkdown": "Indeed I'd choose Torch over TF but tf does better right now...",
      "votes": null
    },
    {
      "id": "962894",
      "postDate": "08/08/2020 14:23:26",
      "content": "<p>The centralized module <code>hub.KerasLayer</code> in <code>TF2</code> is clearly a joke .  It's absolutely rigid and give almost no possibility for modification/extension (paticularly for transformers models). And the big issue is neither <code>hub.Module</code> nor <code>tf.contrib</code> are supported by <code>TF2</code>, even with <code>tf.compat.v1</code>.  I had to downgrade to <code>TF 1.15</code> by uploading it as dataset  during Google Quest competition in order to use and adapt the official Bert repo at my convenience. </p>\n<p>I switched direct to Pytorch after that competition. </p>\n<p>As I said , HuggingFace is now supporting TF2 now fortunately, otherwise using Transformers models would be nightmare. </p>",
      "rawMarkdown": "The centralized module `hub.KerasLayer` in `TF2` is clearly a joke .  It's absolutely rigid and give almost no possibility for modification/extension (paticularly for transformers models). And the big issue is neither `hub.Module` nor `tf.contrib` are supported by `TF2`, even with `tf.compat.v1`.  I had to downgrade to `TF 1.15` by uploading it as dataset  during Google Quest competition in order to use and adapt the official Bert repo at my convenience. \n\n I switched direct to Pytorch after that competition. \n\nAs I said , HuggingFace is now supporting TF2 now fortunately, otherwise using Transformers models would be nightmare.",
      "votes": null
    },
    {
      "id": "968386",
      "postDate": "08/13/2020 00:47:30",
      "content": "<blockquote>\n  <p>tools/packages derived from google's innovations are much easier to find (at least inconvenient way ) on Pytorch than Tensorflow.</p>\n</blockquote>\n<p>I agree with that a lot. </p>",
      "rawMarkdown": "> tools/packages derived from google's innovations are much easier to find (at least inconvenient way ) on Pytorch than Tensorflow.\n\nI agree with that a lot.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 962065,
      "author_name": "ludovick",
      "author_url": "",
      "post_date": "08/07/2020 19:07:33",
      "content": "<p>Their are implementation in TF but the issue is it is rarely coded in TF2, oftenly in TF 1.15, even by google's author. We can wonder if they trust their new API :/</p>",
      "votes": null,
      "replies": [
        {
          "id": 962894,
          "author_name": "serigne",
          "author_url": "",
          "post_date": "08/08/2020 14:23:26",
          "content": "<p>The centralized module <code>hub.KerasLayer</code> in <code>TF2</code> is clearly a joke .  It's absolutely rigid and give almost no possibility for modification/extension (paticularly for transformers models). And the big issue is neither <code>hub.Module</code> nor <code>tf.contrib</code> are supported by <code>TF2</code>, even with <code>tf.compat.v1</code>.  I had to downgrade to <code>TF 1.15</code> by uploading it as dataset  during Google Quest competition in order to use and adapt the official Bert repo at my convenience. </p>\n<p>I switched direct to Pytorch after that competition. </p>\n<p>As I said , HuggingFace is now supporting TF2 now fortunately, otherwise using Transformers models would be nightmare. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 968386,
      "author_name": "ipythonx",
      "author_url": "",
      "post_date": "08/13/2020 00:47:30",
      "content": "<blockquote>\n  <p>tools/packages derived from google's innovations are much easier to find (at least inconvenient way ) on Pytorch than Tensorflow.</p>\n</blockquote>\n<p>I agree with that a lot. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 960628,
      "author_name": "group16",
      "author_url": "",
      "post_date": "08/06/2020 14:54:02",
      "content": "<p>I find Torch 1000 times easier to experiment with than Tensorflow. It seems the go-to framework for research, due to its flexibility. It's flow/fundaments are pythonic and very similar to numph. Tensorflow requires a completely different way of thinking. Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch...</p>",
      "votes": null,
      "replies": [
        {
          "id": 960701,
          "author_name": "serigne",
          "author_url": "",
          "post_date": "08/06/2020 15:55:28",
          "content": "<blockquote>\n  <p>Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch…</p>\n</blockquote>\n\n<p>I think the advantage for TF is seemless TPU support and TFrecord .    This may compensate its lack of flexibility here. \nAnd Kaggle TPU config is particularly tailored for TF. </p>\n\n<p>I was using Pytorch until last week, but then I needed to merge with TF models to improve my score ^^</p>\n\n<p>Fortunately <a href=\"/cdeotte\">@cdeotte</a> has wrote great and solid TF baseline kernels, that let frankly very little rooms for improvement :). ( I am just modifying the training loop and modify the backbone arch to add my custom heads. )</p>\n\n<p>But it's still frustrating to see that some convinient  tools are not available for TF</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 962155,
          "author_name": "datafan07",
          "author_url": "",
          "post_date": "08/07/2020 21:18:42",
          "content": "<p>Indeed I'd choose Torch over TF but tf does better right now…</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "960542": "Too many of recent breaktrough ideas in DL come from Google Brain ( Transformers/Bert, NASNet/Auto-ML/Efficientnet, Self training/Noisy Student,  Auto-Augment etc.)   while Facebook is just recycling Resnet(OK this is a troll ^^)\n\nBut what strikes the most is, even tools/packages derived from google's innovations are much easier to find (at least in convienent way ) on Pytorch than Tensorflow.  \nFor instance you can find bunch of Auto-Augment packages, ready to use for Pytorch , but I found nothing for TF. \n\nDo you think the way Google abruptly abandoned `tf.contrib` has something to do with it ? thus  discouraging the community.\n\nFortunately HuggingFace Transformers is now supporting TF ( but just TF 2.X and only subset of models) .",
    "960628": "I find Torch 1000 times easier to experiment with than Tensorflow. It seems the go-to framework for research, due to its flexibility. It's flow/fundaments are pythonic and very similar to numph. Tensorflow requires a completely different way of thinking. Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch...",
    "960701": "&gt; Unfortunately, the defaults in TF seem to be more LB-friendly on Kaggle than those of Torch…\n\nI think the advantage for TF is seemless TPU support and TFrecord .    This may compensate its lack of flexibility here. \nAnd Kaggle TPU config is particularly tailored for TF. \n\nI was using Pytorch until last week, but then I needed to merge with TF models to improve my score ^^\n\nFortunately @cdeotte has wrote great and solid TF baseline kernels, that let frankly very little rooms for improvement :). ( I am just modifying the training loop and modify the backbone arch to add my custom heads. )\n\nBut it's still frustrating to see that some convinient  tools are not available for TF",
    "962065": "Their are implementation in TF but the issue is it is rarely coded in TF2, oftenly in TF 1.15, even by google's author. We can wonder if they trust their new API :/",
    "962155": "Indeed I'd choose Torch over TF but tf does better right now...",
    "962894": "The centralized module `hub.KerasLayer` in `TF2` is clearly a joke .  It's absolutely rigid and give almost no possibility for modification/extension (paticularly for transformers models). And the big issue is neither `hub.Module` nor `tf.contrib` are supported by `TF2`, even with `tf.compat.v1`.  I had to downgrade to `TF 1.15` by uploading it as dataset  during Google Quest competition in order to use and adapt the official Bert repo at my convenience. \n\n I switched direct to Pytorch after that competition. \n\nAs I said , HuggingFace is now supporting TF2 now fortunately, otherwise using Transformers models would be nightmare.",
    "968386": "> tools/packages derived from google's innovations are much easier to find (at least inconvenient way ) on Pytorch than Tensorflow.\n\nI agree with that a lot."
  },
  "source": "meta"
}