{
  "id": 41224,
  "title": "keras importance sampling",
  "url": "/competitions/cdiscount-image-classification-challenge/discussion/41224",
  "author_name": "",
  "post_date": "2017-10-15T01:19:52.111123800Z",
  "votes": 7,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Deep learning models spend countless GPU/CPU cycles on trivial, correctly classified examples that do not individually affect the parameters. For instance, even a very simple neural network achieves ~98% accuracy on MNIST after a single epoch.</p>\n\n<p>Importance sampling focuses the computation to informative/important samples (by sampling mini-batches from a distribution other than uniform) thus accelerating the convergence.</p>\n\n<p><a href=\"https://github.com/idiap/importance-sampling\">https://github.com/idiap/importance-sampling</a></p>\n\n<p><a href=\"http://idiap.ch/~katharas/importance-sampling\">http://idiap.ch/~katharas/importance-sampling</a></p>\n\n<hr>\n\n<p><a href=\"https://arxiv.org/abs/1511.06343\">https://arxiv.org/abs/1511.06343</a></p>\n\n<p><a href=\"https://arxiv.org/abs/1706.00043\">https://arxiv.org/abs/1706.00043</a></p>\n\n<p><a href=\"https://openreview.net/forum?id=r1IRctqxg\">https://openreview.net/forum?id=r1IRctqxg</a></p>",
  "messages": [
    {
      "id": "231461",
      "postDate": "10/15/2017 01:19:52",
      "content": "<p>Deep learning models spend countless GPU/CPU cycles on trivial, correctly classified examples that do not individually affect the parameters. For instance, even a very simple neural network achieves ~98% accuracy on MNIST after a single epoch.</p>\n\n<p>Importance sampling focuses the computation to informative/important samples (by sampling mini-batches from a distribution other than uniform) thus accelerating the convergence.</p>\n\n<p><a href=\"https://github.com/idiap/importance-sampling\">https://github.com/idiap/importance-sampling</a></p>\n\n<p><a href=\"http://idiap.ch/~katharas/importance-sampling\">http://idiap.ch/~katharas/importance-sampling</a></p>\n\n<hr>\n\n<p><a href=\"https://arxiv.org/abs/1511.06343\">https://arxiv.org/abs/1511.06343</a></p>\n\n<p><a href=\"https://arxiv.org/abs/1706.00043\">https://arxiv.org/abs/1706.00043</a></p>\n\n<p><a href=\"https://openreview.net/forum?id=r1IRctqxg\">https://openreview.net/forum?id=r1IRctqxg</a></p>",
      "rawMarkdown": "Deep learning models spend countless GPU/CPU cycles on trivial, correctly classified examples that do not individually affect the parameters. For instance, even a very simple neural network achieves ~98% accuracy on MNIST after a single epoch.\n\nImportance sampling focuses the computation to informative/important samples (by sampling mini-batches from a distribution other than uniform) thus accelerating the convergence.\n\nhttps://github.com/idiap/importance-sampling\n\nhttp://idiap.ch/~katharas/importance-sampling\n\n-----\n\nhttps://arxiv.org/abs/1511.06343\n\nhttps://arxiv.org/abs/1706.00043\n\nhttps://openreview.net/forum?id=r1IRctqxg",
      "votes": null
    },
    {
      "id": "231538",
      "postDate": "10/15/2017 09:26:25",
      "content": "<p>Interesting! I've used a simple \"class-aware sampling\" method in one of my experiments, where you use one image from each class in your mini-batch(es). Once you've seen every class, you shuffle the list of classes and repeat. Essentially, this over-samples the classes with a smaller number of images. However, this prevented my model from learning beyond about 50% correct (using extensive data augmentation). I did not investigate further why this happened, probably because the model did not have enough capacity.</p>",
      "rawMarkdown": "Interesting! I've used a simple \"class-aware sampling\" method in one of my experiments, where you use one image from each class in your mini-batch(es). Once you've seen every class, you shuffle the list of classes and repeat. Essentially, this over-samples the classes with a smaller number of images. However, this prevented my model from learning beyond about 50% correct (using extensive data augmentation). I did not investigate further why this happened, probably because the model did not have enough capacity.",
      "votes": null
    },
    {
      "id": "232222",
      "postDate": "10/17/2017 06:04:00",
      "content": "<p>today's paper at arxiv:</p>\n\n<p><a href=\"https://arxiv.org/pdf/1710.05381.pdf\">https://arxiv.org/pdf/1710.05381.pdf</a></p>\n\n<p>A systematic study of the class imbalance problem in convolutional neural networks</p>",
      "rawMarkdown": "today's paper at arxiv:\n\nhttps://arxiv.org/pdf/1710.05381.pdf\n\nA systematic study of the class imbalance problem in convolutional neural networks",
      "votes": null
    },
    {
      "id": "232549",
      "postDate": "10/17/2017 18:49:28",
      "content": "<p>thank you for given information!</p>",
      "rawMarkdown": "thank you for given information!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 231538,
      "author_name": "humananalog",
      "author_url": "",
      "post_date": "10/15/2017 09:26:25",
      "content": "<p>Interesting! I've used a simple \"class-aware sampling\" method in one of my experiments, where you use one image from each class in your mini-batch(es). Once you've seen every class, you shuffle the list of classes and repeat. Essentially, this over-samples the classes with a smaller number of images. However, this prevented my model from learning beyond about 50% correct (using extensive data augmentation). I did not investigate further why this happened, probably because the model did not have enough capacity.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 232222,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "10/17/2017 06:04:00",
      "content": "<p>today's paper at arxiv:</p>\n\n<p><a href=\"https://arxiv.org/pdf/1710.05381.pdf\">https://arxiv.org/pdf/1710.05381.pdf</a></p>\n\n<p>A systematic study of the class imbalance problem in convolutional neural networks</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 232549,
      "author_name": "avens8",
      "author_url": "",
      "post_date": "10/17/2017 18:49:28",
      "content": "<p>thank you for given information!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "231461": "Deep learning models spend countless GPU/CPU cycles on trivial, correctly classified examples that do not individually affect the parameters. For instance, even a very simple neural network achieves ~98% accuracy on MNIST after a single epoch.\n\nImportance sampling focuses the computation to informative/important samples (by sampling mini-batches from a distribution other than uniform) thus accelerating the convergence.\n\nhttps://github.com/idiap/importance-sampling\n\nhttp://idiap.ch/~katharas/importance-sampling\n\n-----\n\nhttps://arxiv.org/abs/1511.06343\n\nhttps://arxiv.org/abs/1706.00043\n\nhttps://openreview.net/forum?id=r1IRctqxg",
    "231538": "Interesting! I've used a simple \"class-aware sampling\" method in one of my experiments, where you use one image from each class in your mini-batch(es). Once you've seen every class, you shuffle the list of classes and repeat. Essentially, this over-samples the classes with a smaller number of images. However, this prevented my model from learning beyond about 50% correct (using extensive data augmentation). I did not investigate further why this happened, probably because the model did not have enough capacity.",
    "232222": "today's paper at arxiv:\n\nhttps://arxiv.org/pdf/1710.05381.pdf\n\nA systematic study of the class imbalance problem in convolutional neural networks",
    "232549": "thank you for given information!"
  },
  "source": "meta"
}