{
  "id": 73982,
  "title": "Has anyone tried SRU in this competition?",
  "url": "/competitions/quora-insincere-questions-classification/discussion/73982",
  "author_name": "",
  "post_date": "2018-12-07T08:01:27.209182600Z",
  "votes": 3,
  "comment_count": 9,
  "views": 0,
  "content": "<p>I tried SRU because of the randomness brought by CuDNNs, and impressed by its speed, one single SRU layer was about 100 seconds per epoch.</p>\n\n<p>But the f1 score (~0.64) is much lower than CuDNNGRU. </p>\n\n<p>And I've tried 3 SRU layers also, but the speed slowed to ~280s per epoch, the f1 score is about 0.66. </p>",
  "messages": [
    {
      "id": "434957",
      "postDate": "12/07/2018 08:01:27",
      "content": "<p>I tried SRU because of the randomness brought by CuDNNs, and impressed by its speed, one single SRU layer was about 100 seconds per epoch.</p>\n\n<p>But the f1 score (~0.64) is much lower than CuDNNGRU. </p>\n\n<p>And I've tried 3 SRU layers also, but the speed slowed to ~280s per epoch, the f1 score is about 0.66. </p>",
      "rawMarkdown": "I tried SRU because of the randomness brought by CuDNNs, and impressed by its speed, one single SRU layer was about 100 seconds per epoch.\n\nBut the f1 score (~0.64) is much lower than CuDNNGRU. \n\nAnd I've tried 3 SRU layers also, but the speed slowed to ~280s per epoch, the f1 score is about 0.66.",
      "votes": null
    },
    {
      "id": "434994",
      "postDate": "12/07/2018 09:29:01",
      "content": "<p>yes  i also tried sru , but the f1 score is lower than cudnngru</p>",
      "rawMarkdown": "yes  i also tried sru , but the f1 score is lower than cudnngru",
      "votes": null
    },
    {
      "id": "434999",
      "postDate": "12/07/2018 09:33:18",
      "content": "<p>sad :(</p>",
      "rawMarkdown": "sad :(",
      "votes": null
    },
    {
      "id": "435280",
      "postDate": "12/07/2018 19:33:12",
      "content": "<p>Are SRU available in the kernels?</p>",
      "rawMarkdown": "Are SRU available in the kernels?",
      "votes": null
    },
    {
      "id": "435292",
      "postDate": "12/07/2018 20:00:19",
      "content": "<p>You can copy and paste this implementation on your kernel </p>\n\n<p><a href=\"https://github.com/titu1994/keras-SRU/blob/master/sru.py\">https://github.com/titu1994/keras-SRU/blob/master/sru.py</a></p>",
      "rawMarkdown": "You can copy and paste this implementation on your kernel \n\nhttps://github.com/titu1994/keras-SRU/blob/master/sru.py",
      "votes": null
    },
    {
      "id": "435385",
      "postDate": "12/07/2018 23:51:35",
      "content": "<p>@Serigne, the author mentioned that the speed is an issue here <a href=\"https://github.com/keras-team/keras/issues/7870\">https://github.com/keras-team/keras/issues/7870</a>. </p>\n\n<p>Have you used it yourself? If so is it faster than cuDNNs?</p>",
      "rawMarkdown": "Serigne, the author mentioned that the speed is an issue here https://github.com/keras-team/keras/issues/7870. \n\nHave you used it yourself? If so is it faster than cuDNNs?",
      "votes": null
    },
    {
      "id": "435467",
      "postDate": "12/08/2018 04:26:44",
      "content": "<p><a href=\"https://github.com/titu1994/keras-SRU/blob/master/sru.py\">https://github.com/titu1994/keras-SRU/blob/master/sru.py</a> </p>\n\n<p>This very code is what I used, and the result is what I mentioned above. (Using other settings and codes of <a href=\"https://www.kaggle.com/shujian/single-rnn-with-4-folds-clr\">https://www.kaggle.com/shujian/single-rnn-with-4-folds-clr</a> )</p>\n\n<p>And I tried QRNN also, which f1 score was much lower than SRU, and speed also slower.</p>\n\n<p><a href=\"https://github.com/DingKe/nn_playground/tree/master/qrnn\">https://github.com/DingKe/nn_playground/tree/master/qrnn</a></p>",
      "rawMarkdown": "https://github.com/titu1994/keras-SRU/blob/master/sru.py \n\nThis very code is what I used, and the result is what I mentioned above. (Using other settings and codes of https://www.kaggle.com/shujian/single-rnn-with-4-folds-clr )\n\nAnd I tried QRNN also, which f1 score was much lower than SRU, and speed also slower.\n\nhttps://github.com/DingKe/nn_playground/tree/master/qrnn",
      "votes": null
    },
    {
      "id": "435586",
      "postDate": "12/08/2018 10:21:54",
      "content": "<p>@Sheriytm </p>\n\n<p>No, I didn't try it.   I use only CuDNNLSTM for now</p>\n\n<p>However I tried the implementation of Independently RNN by the same author,  few weeks ago</p>\n\n<p><a href=\"https://github.com/titu1994/Keras-IndRNN/blob/master/ind_rnn.py\">https://github.com/titu1994/Keras-IndRNN/blob/master/ind_rnn.py</a></p>\n\n<p>But we have the issue here, It run really fast but you'll need deep and wide architecture to make it effective, thus slowing the run. </p>",
      "rawMarkdown": "Sheriytm \n\nNo, I didn't try it.   I use only CuDNNLSTM for now\n\nHowever I tried the implementation of Independently RNN by the same author,  few weeks ago\n\nhttps://github.com/titu1994/Keras-IndRNN/blob/master/ind_rnn.py\n\nBut we have the issue here, It run really fast but you'll need deep and wide architecture to make it effective, thus slowing the run.",
      "votes": null
    },
    {
      "id": "435611",
      "postDate": "12/08/2018 11:32:44",
      "content": "<p>Thanks @Serigne, I tested it a few hours ago and found that implementation of the RSU is very slow.</p>\n\n<p>By the way are you able find a way to make your keras model results consistent? I am still dealing with that issue.</p>",
      "rawMarkdown": "Thanks @Serigne, I tested it a few hours ago and found that implementation of the RSU is very slow.\n\nBy the way are you able find a way to make your keras model results consistent? I am still dealing with that issue.",
      "votes": null
    },
    {
      "id": "435651",
      "postDate": "12/08/2018 13:11:11",
      "content": "<p>Ah ok, I didn't try it ... but the one I tried (indRNN ) is relatively fast.</p>\n\n<p>Not at all, My results are not so consistent </p>",
      "rawMarkdown": "Ah ok, I didn't try it ... but the one I tried (indRNN ) is relatively fast.\n\nNot at all, My results are not so consistent",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 434994,
      "author_name": "alexyung757",
      "author_url": "",
      "post_date": "12/07/2018 09:29:01",
      "content": "<p>yes  i also tried sru , but the f1 score is lower than cudnngru</p>",
      "votes": null,
      "replies": [
        {
          "id": 434999,
          "author_name": "hengzheng",
          "author_url": "",
          "post_date": "12/07/2018 09:33:18",
          "content": "<p>sad :(</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435280,
          "author_name": "juanmanpr",
          "author_url": "",
          "post_date": "12/07/2018 19:33:12",
          "content": "<p>Are SRU available in the kernels?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435292,
          "author_name": "serigne",
          "author_url": "",
          "post_date": "12/07/2018 20:00:19",
          "content": "<p>You can copy and paste this implementation on your kernel </p>\n\n<p><a href=\"https://github.com/titu1994/keras-SRU/blob/master/sru.py\">https://github.com/titu1994/keras-SRU/blob/master/sru.py</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435385,
          "author_name": "sheriytm",
          "author_url": "",
          "post_date": "12/07/2018 23:51:35",
          "content": "<p>@Serigne, the author mentioned that the speed is an issue here <a href=\"https://github.com/keras-team/keras/issues/7870\">https://github.com/keras-team/keras/issues/7870</a>. </p>\n\n<p>Have you used it yourself? If so is it faster than cuDNNs?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435467,
          "author_name": "hengzheng",
          "author_url": "",
          "post_date": "12/08/2018 04:26:44",
          "content": "<p><a href=\"https://github.com/titu1994/keras-SRU/blob/master/sru.py\">https://github.com/titu1994/keras-SRU/blob/master/sru.py</a> </p>\n\n<p>This very code is what I used, and the result is what I mentioned above. (Using other settings and codes of <a href=\"https://www.kaggle.com/shujian/single-rnn-with-4-folds-clr\">https://www.kaggle.com/shujian/single-rnn-with-4-folds-clr</a> )</p>\n\n<p>And I tried QRNN also, which f1 score was much lower than SRU, and speed also slower.</p>\n\n<p><a href=\"https://github.com/DingKe/nn_playground/tree/master/qrnn\">https://github.com/DingKe/nn_playground/tree/master/qrnn</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435586,
          "author_name": "serigne",
          "author_url": "",
          "post_date": "12/08/2018 10:21:54",
          "content": "<p>@Sheriytm </p>\n\n<p>No, I didn't try it.   I use only CuDNNLSTM for now</p>\n\n<p>However I tried the implementation of Independently RNN by the same author,  few weeks ago</p>\n\n<p><a href=\"https://github.com/titu1994/Keras-IndRNN/blob/master/ind_rnn.py\">https://github.com/titu1994/Keras-IndRNN/blob/master/ind_rnn.py</a></p>\n\n<p>But we have the issue here, It run really fast but you'll need deep and wide architecture to make it effective, thus slowing the run. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435611,
          "author_name": "sheriytm",
          "author_url": "",
          "post_date": "12/08/2018 11:32:44",
          "content": "<p>Thanks @Serigne, I tested it a few hours ago and found that implementation of the RSU is very slow.</p>\n\n<p>By the way are you able find a way to make your keras model results consistent? I am still dealing with that issue.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435651,
          "author_name": "serigne",
          "author_url": "",
          "post_date": "12/08/2018 13:11:11",
          "content": "<p>Ah ok, I didn't try it ... but the one I tried (indRNN ) is relatively fast.</p>\n\n<p>Not at all, My results are not so consistent </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "434957": "I tried SRU because of the randomness brought by CuDNNs, and impressed by its speed, one single SRU layer was about 100 seconds per epoch.\n\nBut the f1 score (~0.64) is much lower than CuDNNGRU. \n\nAnd I've tried 3 SRU layers also, but the speed slowed to ~280s per epoch, the f1 score is about 0.66.",
    "434994": "yes  i also tried sru , but the f1 score is lower than cudnngru",
    "434999": "sad :(",
    "435280": "Are SRU available in the kernels?",
    "435292": "You can copy and paste this implementation on your kernel \n\nhttps://github.com/titu1994/keras-SRU/blob/master/sru.py",
    "435385": "Serigne, the author mentioned that the speed is an issue here https://github.com/keras-team/keras/issues/7870. \n\nHave you used it yourself? If so is it faster than cuDNNs?",
    "435467": "https://github.com/titu1994/keras-SRU/blob/master/sru.py \n\nThis very code is what I used, and the result is what I mentioned above. (Using other settings and codes of https://www.kaggle.com/shujian/single-rnn-with-4-folds-clr )\n\nAnd I tried QRNN also, which f1 score was much lower than SRU, and speed also slower.\n\nhttps://github.com/DingKe/nn_playground/tree/master/qrnn",
    "435586": "Sheriytm \n\nNo, I didn't try it.   I use only CuDNNLSTM for now\n\nHowever I tried the implementation of Independently RNN by the same author,  few weeks ago\n\nhttps://github.com/titu1994/Keras-IndRNN/blob/master/ind_rnn.py\n\nBut we have the issue here, It run really fast but you'll need deep and wide architecture to make it effective, thus slowing the run.",
    "435611": "Thanks @Serigne, I tested it a few hours ago and found that implementation of the RSU is very slow.\n\nBy the way are you able find a way to make your keras model results consistent? I am still dealing with that issue.",
    "435651": "Ah ok, I didn't try it ... but the one I tried (indRNN ) is relatively fast.\n\nNot at all, My results are not so consistent"
  },
  "source": "meta"
}