{
  "id": 170471,
  "title": "Ray Paralelization for Making Mels Spectrograms",
  "url": "/competitions/birdsong-recognition/discussion/170471",
  "author_name": "leodav",
  "post_date": "2020-07-27T19:47:10.578000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>New here but thought this might be useful.  Key take-away here is ray library is better for parallelization than subprocess and other libraries I tried and also easier to use (pooling is a bit tricky but see notebook for quick and dirty solution).  I also found that subprocess only  uses 3 processors effectively while ray can use all 4.</p>\n\n<p>Sharing both notebook and 2 data sets</p>\n\n<p>Notebook:\n<a href=\"https://www.kaggle.com/leodav/birdsong-preprocessing-a2h\">https://www.kaggle.com/leodav/birdsong-preprocessing-a2h</a>\nData Set:\n<a href=\"https://www.kaggle.com/leodav/birdsongpresga2h\">https://www.kaggle.com/leodav/birdsongpresga2h</a>\n<a href=\"https://www.kaggle.com/leodav/birdsongpresgi2z\">https://www.kaggle.com/leodav/birdsongpresgi2z</a></p>",
  "messages": [
    {
      "id": 948311,
      "postDate": "2020-07-27T19:47:10.577Z",
      "content": "<p>New here but thought this might be useful.  Key take-away here is ray library is better for parallelization than subprocess and other libraries I tried and also easier to use (pooling is a bit tricky but see notebook for quick and dirty solution).  I also found that subprocess only  uses 3 processors effectively while ray can use all 4.</p>\n\n<p>Sharing both notebook and 2 data sets</p>\n\n<p>Notebook:\n<a href=\"https://www.kaggle.com/leodav/birdsong-preprocessing-a2h\">https://www.kaggle.com/leodav/birdsong-preprocessing-a2h</a>\nData Set:\n<a href=\"https://www.kaggle.com/leodav/birdsongpresga2h\">https://www.kaggle.com/leodav/birdsongpresga2h</a>\n<a href=\"https://www.kaggle.com/leodav/birdsongpresgi2z\">https://www.kaggle.com/leodav/birdsongpresgi2z</a></p>",
      "rawMarkdown": "New here but thought this might be useful.  Key take-away here is ray library is better for parallelization than subprocess and other libraries I tried and also easier to use (pooling is a bit tricky but see notebook for quick and dirty solution).  I also found that subprocess only  uses 3 processors effectively while ray can use all 4.\n\nSharing both notebook and 2 data sets\n\nNotebook:\nhttps://www.kaggle.com/leodav/birdsong-preprocessing-a2h\nData Set:\nhttps://www.kaggle.com/leodav/birdsongpresga2h\nhttps://www.kaggle.com/leodav/birdsongpresgi2z\n\n"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "948311": "New here but thought this might be useful.  Key take-away here is ray library is better for parallelization than subprocess and other libraries I tried and also easier to use (pooling is a bit tricky but see notebook for quick and dirty solution).  I also found that subprocess only  uses 3 processors effectively while ray can use all 4.\n\nSharing both notebook and 2 data sets\n\nNotebook:\nhttps://www.kaggle.com/leodav/birdsong-preprocessing-a2h\nData Set:\nhttps://www.kaggle.com/leodav/birdsongpresga2h\nhttps://www.kaggle.com/leodav/birdsongpresgi2z\n\n"
  }
}