{
  "id": 215955,
  "title": "joblib in kaggle kernel is so slow",
  "url": "/competitions/rfcx-species-audio-detection/discussion/215955",
  "author_name": "",
  "post_date": "2021-02-01T03:02:35.886229900Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I use the same code in kaggle kernel as I do local machine.<br>\nBut it only takes 3 min to run on local machine, it takes almost 10 min to run on kaggle kernel.<br>\nWhy? Is it because  kaggle runs on docker?</p>\n<p>code is something like<br>\n <code>result = joblib.Parallel(n_jobs=10)(joblib.delayed(function)(param) for param in tqdm(params))</code></p>",
  "messages": [
    {
      "id": "1179962",
      "postDate": "02/01/2021 03:02:35",
      "content": "<p>I use the same code in kaggle kernel as I do local machine.<br>\nBut it only takes 3 min to run on local machine, it takes almost 10 min to run on kaggle kernel.<br>\nWhy? Is it because  kaggle runs on docker?</p>\n<p>code is something like<br>\n <code>result = joblib.Parallel(n_jobs=10)(joblib.delayed(function)(param) for param in tqdm(params))</code></p>",
      "rawMarkdown": "I use the same code in kaggle kernel as I do local machine.\nBut it only takes 3 min to run on local machine, it takes almost 10 min to run on kaggle kernel.\nWhy? Is it because  kaggle runs on docker?\n\ncode is something like\n `result = joblib.Parallel(n_jobs=10)(joblib.delayed(function)(param) for param in tqdm(params))`",
      "votes": null
    },
    {
      "id": "1180037",
      "postDate": "02/01/2021 03:54:55",
      "content": "<p>Kernels only have 4 cores, so although you are telling joblib to use 10, it will only use 4. I am guessing your personal machine probably has 3x the amount. </p>",
      "rawMarkdown": "Kernels only have 4 cores, so although you are telling joblib to use 10, it will only use 4. I am guessing your personal machine probably has 3x the amount.",
      "votes": null
    },
    {
      "id": "1180135",
      "postDate": "02/01/2021 05:46:09",
      "content": "<p>Thanks for your useful infos!!    I'll try it later!</p>",
      "rawMarkdown": "Thanks for your useful infos!!    I'll try it later!",
      "votes": null
    },
    {
      "id": "1181053",
      "postDate": "02/01/2021 16:14:35",
      "content": "<p>I came to write this</p>",
      "rawMarkdown": "I came to write this",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1180037,
      "author_name": "molenr",
      "author_url": "",
      "post_date": "02/01/2021 03:54:55",
      "content": "<p>Kernels only have 4 cores, so although you are telling joblib to use 10, it will only use 4. I am guessing your personal machine probably has 3x the amount. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1180135,
          "author_name": "changewow",
          "author_url": "",
          "post_date": "02/01/2021 05:46:09",
          "content": "<p>Thanks for your useful infos!!    I'll try it later!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1181053,
          "author_name": "daaadaaa",
          "author_url": "",
          "post_date": "02/01/2021 16:14:35",
          "content": "<p>I came to write this</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1179962": "I use the same code in kaggle kernel as I do local machine.\nBut it only takes 3 min to run on local machine, it takes almost 10 min to run on kaggle kernel.\nWhy? Is it because  kaggle runs on docker?\n\ncode is something like\n `result = joblib.Parallel(n_jobs=10)(joblib.delayed(function)(param) for param in tqdm(params))`",
    "1180037": "Kernels only have 4 cores, so although you are telling joblib to use 10, it will only use 4. I am guessing your personal machine probably has 3x the amount.",
    "1180135": "Thanks for your useful infos!!    I'll try it later!",
    "1181053": "I came to write this"
  },
  "source": "meta"
}