{
  "id": 376443,
  "title": "4x Speedup Pandas using [Pandarallel and TPU]",
  "url": "/competitions/otto-recommender-system/discussion/376443",
  "author_name": "",
  "post_date": "2023-01-06T11:33:48.693109600Z",
  "votes": 5,
  "comment_count": 1,
  "views": 0,
  "content": "<p>The top-scoring notebooks are taking close to <strong>50 minutes</strong> for the <strong>top 20 candidates' generation</strong>.<br>\nI made an optimisation in the notebook:<br>\n<a href=\"https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times\" target=\"_blank\">https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times</a></p>\n<p><strong>Pandarallel</strong>  uses 48 workers from the TPU  provided by Kaggle and reduces the time to <strong>12 minutes</strong>(4x faster). </p>\n<p>Code to add pandarallel in your kernel:</p>\n<pre><code> pandarallel  pandarallel\npandarallel.initialize(progress_bar=,use_memory_fs=)\n</code></pre>",
  "messages": [
    {
      "id": "2088501",
      "postDate": "01/06/2023 11:33:48",
      "content": "<p>The top-scoring notebooks are taking close to <strong>50 minutes</strong> for the <strong>top 20 candidates' generation</strong>.<br>\nI made an optimisation in the notebook:<br>\n<a href=\"https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times\" target=\"_blank\">https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times</a></p>\n<p><strong>Pandarallel</strong>  uses 48 workers from the TPU  provided by Kaggle and reduces the time to <strong>12 minutes</strong>(4x faster). </p>\n<p>Code to add pandarallel in your kernel:</p>\n<pre><code> pandarallel  pandarallel\npandarallel.initialize(progress_bar=,use_memory_fs=)\n</code></pre>",
      "rawMarkdown": "The top-scoring notebooks are taking close to **50 minutes** for the **top 20 candidates' generation**.\nI made an optimisation in the notebook:\n[https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times](https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times)\n\n **Pandarallel**  uses 48 workers from the TPU  provided by Kaggle and reduces the time to **12 minutes**(4x faster). \n\nCode to add pandarallel in your kernel:\n```python\nfrom pandarallel import pandarallel\npandarallel.initialize(progress_bar=True,use_memory_fs=False)\n```",
      "votes": null
    },
    {
      "id": "2088574",
      "postDate": "01/06/2023 12:45:42",
      "content": "<p>Nice work! Both arguments for pandarallel can be passed into the same <code>pandarallel.initialize(progress_bar=True, use_memory_fs=False)</code></p>",
      "rawMarkdown": "Nice work! Both arguments for pandarallel can be passed into the same `pandarallel.initialize(progress_bar=True, use_memory_fs=False)`",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2088574,
      "author_name": "parthpankajtiwary",
      "author_url": "",
      "post_date": "01/06/2023 12:45:42",
      "content": "<p>Nice work! Both arguments for pandarallel can be passed into the same <code>pandarallel.initialize(progress_bar=True, use_memory_fs=False)</code></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2088501": "The top-scoring notebooks are taking close to **50 minutes** for the **top 20 candidates' generation**.\nI made an optimisation in the notebook:\n[https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times](https://www.kaggle.com/code/chaudharypriyanshu/pandarallel-speedup-chris-approach-4x-times)\n\n **Pandarallel**  uses 48 workers from the TPU  provided by Kaggle and reduces the time to **12 minutes**(4x faster). \n\nCode to add pandarallel in your kernel:\n```python\nfrom pandarallel import pandarallel\npandarallel.initialize(progress_bar=True,use_memory_fs=False)\n```",
    "2088574": "Nice work! Both arguments for pandarallel can be passed into the same `pandarallel.initialize(progress_bar=True, use_memory_fs=False)`"
  },
  "source": "meta"
}