{
  "id": 200917,
  "title": "pandas .loc and .query vs numexpr speed comparison",
  "url": "/competitions/riiid-test-answer-prediction/discussion/200917",
  "author_name": "",
  "post_date": "2020-12-02T11:43:31.259686800Z",
  "votes": 8,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>If you are using .loc or .query in pandas to select rows or columns, consider using numexpr for this competition. According to <a href=\"https://stackoverflow.com/questions/49936557/pandas-dataframe-loc-vs-query-performance\" target=\"_blank\">this stackoverflow thread</a>, numexpr can be quite fast compared to .loc or .query functions.</p>\n<p><img src=\"https://i.stack.imgur.com/IrZyA.png\" alt=\".loc vs .query vs numexp\"></p>",
  "messages": [
    {
      "id": "1099469",
      "postDate": "12/02/2020 11:43:31",
      "content": "<p>Hi everyone,</p>\n<p>If you are using .loc or .query in pandas to select rows or columns, consider using numexpr for this competition. According to <a href=\"https://stackoverflow.com/questions/49936557/pandas-dataframe-loc-vs-query-performance\" target=\"_blank\">this stackoverflow thread</a>, numexpr can be quite fast compared to .loc or .query functions.</p>\n<p><img src=\"https://i.stack.imgur.com/IrZyA.png\" alt=\".loc vs .query vs numexp\"></p>",
      "rawMarkdown": "Hi everyone,\n\nIf you are using .loc or .query in pandas to select rows or columns, consider using numexpr for this competition. According to [this stackoverflow thread](https://stackoverflow.com/questions/49936557/pandas-dataframe-loc-vs-query-performance), numexpr can be quite fast compared to .loc or .query functions.\n\n![.loc vs .query vs numexp](https://i.stack.imgur.com/IrZyA.png)",
      "votes": null
    },
    {
      "id": "1100088",
      "postDate": "12/02/2020 20:28:29",
      "content": "<p>Great, thank you for sharing.<br>\nNot only the speed but also the memory usage is concern in this competition. </p>\n<p>I'm joining this, glad to meet you here!</p>",
      "rawMarkdown": "Great, thank you for sharing.\nNot only the speed but also the memory usage is concern in this competition. \n\nI'm joining this, glad to meet you here!",
      "votes": null
    },
    {
      "id": "1100898",
      "postDate": "12/03/2020 12:53:32",
      "content": "<p>Yes, memory is also a concern here. </p>\n<p>I am also glad to see you here. All the best for this one. 😃</p>",
      "rawMarkdown": "Yes, memory is also a concern here. \n\nI am also glad to see you here. All the best for this one. 😃",
      "votes": null
    },
    {
      "id": "1100937",
      "postDate": "12/03/2020 13:40:08",
      "content": "<p>This is really useful to know. Thanks a lot.👍</p>",
      "rawMarkdown": "This is really useful to know. Thanks a lot.👍",
      "votes": null
    },
    {
      "id": "1103282",
      "postDate": "12/05/2020 19:33:36",
      "content": "<p>Great insight. Thanks for sharing, <a href=\"https://www.kaggle.com/manikanthr5\" target=\"_blank\">@manikanthr5</a> </p>",
      "rawMarkdown": "Great insight. Thanks for sharing, @manikanthr5",
      "votes": null
    },
    {
      "id": "1105782",
      "postDate": "12/08/2020 07:33:32",
      "content": "<p>Great,thanks for sharing.</p>",
      "rawMarkdown": "Great,thanks for sharing.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1100088,
      "author_name": "kokitanisaka",
      "author_url": "",
      "post_date": "12/02/2020 20:28:29",
      "content": "<p>Great, thank you for sharing.<br>\nNot only the speed but also the memory usage is concern in this competition. </p>\n<p>I'm joining this, glad to meet you here!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1100898,
          "author_name": "manikanthr5",
          "author_url": "",
          "post_date": "12/03/2020 12:53:32",
          "content": "<p>Yes, memory is also a concern here. </p>\n<p>I am also glad to see you here. All the best for this one. 😃</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1100937,
      "author_name": "chris1729",
      "author_url": "",
      "post_date": "12/03/2020 13:40:08",
      "content": "<p>This is really useful to know. Thanks a lot.👍</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1103282,
      "author_name": "pedrocouto39",
      "author_url": "",
      "post_date": "12/05/2020 19:33:36",
      "content": "<p>Great insight. Thanks for sharing, <a href=\"https://www.kaggle.com/manikanthr5\" target=\"_blank\">@manikanthr5</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1105782,
      "author_name": "shubhamgarg05",
      "author_url": "",
      "post_date": "12/08/2020 07:33:32",
      "content": "<p>Great,thanks for sharing.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1099469": "Hi everyone,\n\nIf you are using .loc or .query in pandas to select rows or columns, consider using numexpr for this competition. According to [this stackoverflow thread](https://stackoverflow.com/questions/49936557/pandas-dataframe-loc-vs-query-performance), numexpr can be quite fast compared to .loc or .query functions.\n\n![.loc vs .query vs numexp](https://i.stack.imgur.com/IrZyA.png)",
    "1100088": "Great, thank you for sharing.\nNot only the speed but also the memory usage is concern in this competition. \n\nI'm joining this, glad to meet you here!",
    "1100898": "Yes, memory is also a concern here. \n\nI am also glad to see you here. All the best for this one. 😃",
    "1100937": "This is really useful to know. Thanks a lot.👍",
    "1103282": "Great insight. Thanks for sharing, @manikanthr5",
    "1105782": "Great,thanks for sharing."
  },
  "source": "meta"
}