{
  "id": 75157,
  "title": "Conv1D",
  "url": "/competitions/quora-insincere-questions-classification/discussion/75157",
  "author_name": "",
  "post_date": "2018-12-19T01:59:25.789446100Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Did anyone try Conv1D with word embeddings? How long does it take to reach some reasonable accuracy?</p>",
  "messages": [
    {
      "id": "441740",
      "postDate": "12/19/2018 01:59:25",
      "content": "<p>Did anyone try Conv1D with word embeddings? How long does it take to reach some reasonable accuracy?</p>",
      "rawMarkdown": "Did anyone try Conv1D with word embeddings? How long does it take to reach some reasonable accuracy?",
      "votes": null
    },
    {
      "id": "441876",
      "postDate": "12/19/2018 07:35:16",
      "content": "<p>There are tons of kernels using Conv1D, e.g. this one: <a href=\"https://www.kaggle.com/artgor/eda-and-lstm-cnn\">https://www.kaggle.com/artgor/eda-and-lstm-cnn</a> . Also all of the high-scoring blends involve at least one CNN in the mix.</p>\n\n<p>Using convolutions is usually much faster than LSTMs, but it depends on how many you apply. I would also explore causal and dilated convolutions, both supported by Keras.</p>",
      "rawMarkdown": "There are tons of kernels using Conv1D, e.g. this one: https://www.kaggle.com/artgor/eda-and-lstm-cnn . Also all of the high-scoring blends involve at least one CNN in the mix.\n\nUsing convolutions is usually much faster than LSTMs, but it depends on how many you apply. I would also explore causal and dilated convolutions, both supported by Keras.",
      "votes": null
    },
    {
      "id": "443091",
      "postDate": "12/21/2018 02:14:33",
      "content": "<p>conv1d using word embedding did not generalize well for me.My model kept on producing all 0's.. As Max pointed out simple Conv1D does not take too long too train but to get acceptable accuracy you need to blend it with others .</p>",
      "rawMarkdown": "conv1d using word embedding did not generalize well for me.My model kept on producing all 0's.. As Max pointed out simple Conv1D does not take too long too train but to get acceptable accuracy you need to blend it with others .",
      "votes": null
    },
    {
      "id": "444839",
      "postDate": "12/25/2018 00:40:07",
      "content": "<p>Did anyone have validation losses just increasing from the start? Does it mean, the model is just bad at generalizing?</p>",
      "rawMarkdown": "Did anyone have validation losses just increasing from the start? Does it mean, the model is just bad at generalizing?",
      "votes": null
    },
    {
      "id": "445104",
      "postDate": "12/25/2018 16:27:56",
      "content": "<p>If the training loss is decreasing, that means you are overfitting. Some possible reasons:</p>\n\n<ol>\n<li>Your model is too big and can learn the training set by heart</li>\n<li>Your model is too tiny and can learn no real features</li>\n<li>Your learning rate is too high</li>\n</ol>\n\n<p>You should definitely tweak your model and training procedure so that this does not happen.</p>",
      "rawMarkdown": "If the training loss is decreasing, that means you are overfitting. Some possible reasons:\n\n1. Your model is too big and can learn the training set by heart\n2. Your model is too tiny and can learn no real features\n3. Your learning rate is too high\n\nYou should definitely tweak your model and training procedure so that this does not happen.",
      "votes": null
    },
    {
      "id": "445553",
      "postDate": "12/26/2018 17:46:15",
      "content": "<p>Till now, I was just using a linear model, with no branches, and the validation loss wouldn't vary much over the epochs.</p>",
      "rawMarkdown": "Till now, I was just using a linear model, with no branches, and the validation loss wouldn't vary much over the epochs.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 441876,
      "author_name": "mschumacher",
      "author_url": "",
      "post_date": "12/19/2018 07:35:16",
      "content": "<p>There are tons of kernels using Conv1D, e.g. this one: <a href=\"https://www.kaggle.com/artgor/eda-and-lstm-cnn\">https://www.kaggle.com/artgor/eda-and-lstm-cnn</a> . Also all of the high-scoring blends involve at least one CNN in the mix.</p>\n\n<p>Using convolutions is usually much faster than LSTMs, but it depends on how many you apply. I would also explore causal and dilated convolutions, both supported by Keras.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 443091,
      "author_name": "chansbest93",
      "author_url": "",
      "post_date": "12/21/2018 02:14:33",
      "content": "<p>conv1d using word embedding did not generalize well for me.My model kept on producing all 0's.. As Max pointed out simple Conv1D does not take too long too train but to get acceptable accuracy you need to blend it with others .</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 444839,
      "author_name": "risabhkoh",
      "author_url": "",
      "post_date": "12/25/2018 00:40:07",
      "content": "<p>Did anyone have validation losses just increasing from the start? Does it mean, the model is just bad at generalizing?</p>",
      "votes": null,
      "replies": [
        {
          "id": 445104,
          "author_name": "mschumacher",
          "author_url": "",
          "post_date": "12/25/2018 16:27:56",
          "content": "<p>If the training loss is decreasing, that means you are overfitting. Some possible reasons:</p>\n\n<ol>\n<li>Your model is too big and can learn the training set by heart</li>\n<li>Your model is too tiny and can learn no real features</li>\n<li>Your learning rate is too high</li>\n</ol>\n\n<p>You should definitely tweak your model and training procedure so that this does not happen.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 445553,
          "author_name": "risabhkoh",
          "author_url": "",
          "post_date": "12/26/2018 17:46:15",
          "content": "<p>Till now, I was just using a linear model, with no branches, and the validation loss wouldn't vary much over the epochs.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "441740": "Did anyone try Conv1D with word embeddings? How long does it take to reach some reasonable accuracy?",
    "441876": "There are tons of kernels using Conv1D, e.g. this one: https://www.kaggle.com/artgor/eda-and-lstm-cnn . Also all of the high-scoring blends involve at least one CNN in the mix.\n\nUsing convolutions is usually much faster than LSTMs, but it depends on how many you apply. I would also explore causal and dilated convolutions, both supported by Keras.",
    "443091": "conv1d using word embedding did not generalize well for me.My model kept on producing all 0's.. As Max pointed out simple Conv1D does not take too long too train but to get acceptable accuracy you need to blend it with others .",
    "444839": "Did anyone have validation losses just increasing from the start? Does it mean, the model is just bad at generalizing?",
    "445104": "If the training loss is decreasing, that means you are overfitting. Some possible reasons:\n\n1. Your model is too big and can learn the training set by heart\n2. Your model is too tiny and can learn no real features\n3. Your learning rate is too high\n\nYou should definitely tweak your model and training procedure so that this does not happen.",
    "445553": "Till now, I was just using a linear model, with no branches, and the validation loss wouldn't vary much over the epochs."
  },
  "source": "meta"
}