{
  "id": 168004,
  "title": "How would you speed up convergence?",
  "url": "/competitions/landmark-retrieval-2020/discussion/168004",
  "author_name": "FP",
  "post_date": "2020-07-18T17:52:57.211000",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>As suggested by Surui Li's <a href=\"https://www.kaggle.com/c/landmark-retrieval-2020/discussion/166353\" target=\"_blank\">excellent post</a> and more, this competition requires heavy computing resources. Therefore speeding up convergence is essential in saving computation time. </p>\n<p>So let me get started. I found the following tips from <a href=\"https://www.topbots.com/14-design-patterns-improve-convolutional-neural-network-cnn-architecture/\" target=\"_blank\">here</a> and <a href=\"https://towardsdatascience.com/https-medium-com-super-convergence-very-fast-training-of-neural-networks-using-large-learning-rates-decb689b9eb0\" target=\"_blank\">here</a>:</p>\n<ul>\n<li>use of pretrained weights</li>\n<li>use cyclical learning rates</li>\n<li>use of cyclical momentum</li>\n<li>use of weight decay</li>\n</ul>\n<p>In this regard, do you have more ideas? Let's share and discuss!</p>",
  "messages": [
    {
      "id": 934770,
      "postDate": "2020-07-18T17:52:57.213Z",
      "content": "<p>As suggested by Surui Li's <a href=\"https://www.kaggle.com/c/landmark-retrieval-2020/discussion/166353\" target=\"_blank\">excellent post</a> and more, this competition requires heavy computing resources. Therefore speeding up convergence is essential in saving computation time. </p>\n<p>So let me get started. I found the following tips from <a href=\"https://www.topbots.com/14-design-patterns-improve-convolutional-neural-network-cnn-architecture/\" target=\"_blank\">here</a> and <a href=\"https://towardsdatascience.com/https-medium-com-super-convergence-very-fast-training-of-neural-networks-using-large-learning-rates-decb689b9eb0\" target=\"_blank\">here</a>:</p>\n<ul>\n<li>use of pretrained weights</li>\n<li>use cyclical learning rates</li>\n<li>use of cyclical momentum</li>\n<li>use of weight decay</li>\n</ul>\n<p>In this regard, do you have more ideas? Let's share and discuss!</p>",
      "rawMarkdown": "As suggested by Surui Li's [excellent post](https://www.kaggle.com/c/landmark-retrieval-2020/discussion/166353) and more, this competition requires heavy computing resources. Therefore speeding up convergence is essential in saving computation time. \n\nSo let me get started. I found the following tips from [here](https://www.topbots.com/14-design-patterns-improve-convolutional-neural-network-cnn-architecture/) and [here](https://towardsdatascience.com/https-medium-com-super-convergence-very-fast-training-of-neural-networks-using-large-learning-rates-decb689b9eb0):\n- use of pretrained weights\n- use cyclical learning rates\n- use of cyclical momentum\n- use of weight decay\n\nIn this regard, do you have more ideas? Let's share and discuss!",
      "votes": 1
    },
    {
      "id": 934839,
      "postDate": "2020-07-18T19:33:26.370Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 934839,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-07-18T19:33:26.370000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "934770": "As suggested by Surui Li's [excellent post](https://www.kaggle.com/c/landmark-retrieval-2020/discussion/166353) and more, this competition requires heavy computing resources. Therefore speeding up convergence is essential in saving computation time. \n\nSo let me get started. I found the following tips from [here](https://www.topbots.com/14-design-patterns-improve-convolutional-neural-network-cnn-architecture/) and [here](https://towardsdatascience.com/https-medium-com-super-convergence-very-fast-training-of-neural-networks-using-large-learning-rates-decb689b9eb0):\n- use of pretrained weights\n- use cyclical learning rates\n- use of cyclical momentum\n- use of weight decay\n\nIn this regard, do you have more ideas? Let's share and discuss!",
    "934839": ""
  }
}