{
  "id": 199301,
  "title": "Is Pyspark acceptable?",
  "url": "/competitions/riiid-test-answer-prediction/discussion/199301",
  "author_name": "",
  "post_date": "2020-11-25T08:46:37.497205600Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Can we work on Pyspark library is it acceptable?</p>",
  "messages": [
    {
      "id": "1090306",
      "postDate": "11/25/2020 08:46:37",
      "content": "<p>Can we work on Pyspark library is it acceptable?</p>",
      "rawMarkdown": "Can we work on Pyspark library is it acceptable?",
      "votes": null
    },
    {
      "id": "1090665",
      "postDate": "11/25/2020 14:23:00",
      "content": "<p>Hey, pyspark is definitely acceptable. Works great with wrangling large data sets when memory is limited, even on kaggle kernels. However, I highly doubt you will be able to make submissions using it as it will be too slow.</p>\n<p>In particular, pyspark is slower than pandas, and kagglers have had issues making submissions with pandas.</p>",
      "rawMarkdown": "Hey, pyspark is definitely acceptable. Works great with wrangling large data sets when memory is limited, even on kaggle kernels. However, I highly doubt you will be able to make submissions using it as it will be too slow.\n\nIn particular, pyspark is slower than pandas, and kagglers have had issues making submissions with pandas.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1090665,
      "author_name": "sparked",
      "author_url": "",
      "post_date": "11/25/2020 14:23:00",
      "content": "<p>Hey, pyspark is definitely acceptable. Works great with wrangling large data sets when memory is limited, even on kaggle kernels. However, I highly doubt you will be able to make submissions using it as it will be too slow.</p>\n<p>In particular, pyspark is slower than pandas, and kagglers have had issues making submissions with pandas.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1090306": "Can we work on Pyspark library is it acceptable?",
    "1090665": "Hey, pyspark is definitely acceptable. Works great with wrangling large data sets when memory is limited, even on kaggle kernels. However, I highly doubt you will be able to make submissions using it as it will be too slow.\n\nIn particular, pyspark is slower than pandas, and kagglers have had issues making submissions with pandas."
  },
  "source": "meta"
}