{
  "id": 55601,
  "title": "Some tips to save time and RAM",
  "url": "/competitions/talkingdata-adtracking-fraud-detection/discussion/55601",
  "author_name": "spongebob",
  "post_date": "2018-04-29T11:31:29.606000",
  "votes": 27,
  "comment_count": 17,
  "views": 0,
  "content": "<p>1.Replace pd.concat([XX],axis=1) with df['X'] = temp['X']\nthis will save a lot of time when the dataframe is very large.</p>\n\n<p>2.Using  temp['X']=  temp['X'].astype('float32') will save a lot of ram.</p>\n\n<p>3.If you use multiprocessing package to accelerate feature extracting. Do not try  to get result by using </p>\n\n<p>&gt; pool.apply_async(XXX)</p>\n\n<p>with get method.  It have some bugs when the result is very large.</p>\n\n<p>Enjoy competition.</p>",
  "messages": [
    {
      "id": 320641,
      "postDate": "2018-04-29T11:31:29.607Z",
      "content": "<p>1.Replace pd.concat([XX],axis=1) with df['X'] = temp['X']\nthis will save a lot of time when the dataframe is very large.</p>\n\n<p>2.Using  temp['X']=  temp['X'].astype('float32') will save a lot of ram.</p>\n\n<p>3.If you use multiprocessing package to accelerate feature extracting. Do not try  to get result by using </p>\n\n<p>&gt; pool.apply_async(XXX)</p>\n\n<p>with get method.  It have some bugs when the result is very large.</p>\n\n<p>Enjoy competition.</p>",
      "rawMarkdown": "1.Replace pd.concat([XX],axis=1) with df['X'] = temp['X']\nthis will save a lot of time when the dataframe is very large.\n\n2.Using  temp['X']=  temp['X'].astype('float32') will save a lot of ram.\n\n3.If you use multiprocessing package to accelerate feature extracting. Do not try  to get result by using \n\n&gt; pool.apply_async(XXX)\n\n with get method.  It have some bugs when the result is very large.\n\nEnjoy competition.",
      "votes": 27
    },
    {
      "id": 320938,
      "postDate": "2018-04-30T08:47:32.040Z",
      "content": "<p>666</p>",
      "rawMarkdown": "666"
    },
    {
      "id": 323437,
      "postDate": "2018-05-05T06:36:13.627Z",
      "content": "<p>谢谢老铁</p>",
      "rawMarkdown": "谢谢老铁"
    },
    {
      "id": 321564,
      "postDate": "2018-05-01T15:06:42.453Z",
      "content": "<p>I wish I'd known these tips earlier... </p>",
      "rawMarkdown": "I wish I'd known these tips earlier... "
    },
    {
      "id": 321309,
      "postDate": "2018-05-01T01:52:35.220Z",
      "content": "<p>真的皮，哈哈哈~</p>",
      "rawMarkdown": "真的皮，哈哈哈~",
      "replies": [
        {
          "id": 321373,
          "postDate": "2018-05-01T05:48:58.503Z",
          "content": "<p>李时珍的皮</p>",
          "rawMarkdown": "李时珍的皮",
          "votes": 2
        }
      ]
    },
    {
      "id": 321097,
      "postDate": "2018-04-30T15:20:50.117Z",
      "content": "<p>Here is another <a href=\"https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55379\">tip</a></p>",
      "rawMarkdown": "Here is another [tip][1]\n\n\n  [1]: https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55379"
    },
    {
      "id": 320926,
      "postDate": "2018-04-30T08:05:27.133Z",
      "content": "<p>Lihaile.</p>",
      "rawMarkdown": "Lihaile."
    },
    {
      "id": 320912,
      "postDate": "2018-04-30T07:01:49.983Z",
      "content": "<p>niubi</p>",
      "rawMarkdown": "niubi"
    },
    {
      "id": 320761,
      "postDate": "2018-04-29T18:24:17.143Z",
      "content": "<p>Just having problems to create encode features in kernel, you are a life saver. </p>",
      "rawMarkdown": "Just having problems to create encode features in kernel, you are a life saver. "
    },
    {
      "id": 320648,
      "postDate": "2018-04-29T11:53:53.543Z",
      "content": "<p>小本子记下来 - Note taken - ^ ^</p>",
      "rawMarkdown": "小本子记下来 - Note taken - ^ ^",
      "replies": [
        {
          "id": 320653,
          "postDate": "2018-04-29T12:11:00.360Z",
          "content": "<p>哈哈-haha- ^ ^</p>",
          "rawMarkdown": "哈哈-haha- ^ ^"
        }
      ]
    },
    {
      "id": 320647,
      "postDate": "2018-04-29T11:49:40.333Z",
      "content": "<p>thank you!</p>",
      "rawMarkdown": "thank you!",
      "replies": [
        {
          "id": 320651,
          "postDate": "2018-04-29T12:01:24.513Z",
          "content": "<p>Your are welcome~</p>",
          "rawMarkdown": "Your are welcome~"
        }
      ]
    },
    {
      "id": 323789,
      "postDate": "2018-05-06T08:10:55.243Z",
      "rawMarkdown": "",
      "isDeleted": true,
      "replies": [
        {
          "id": 323792,
          "postDate": "2018-05-06T08:20:39.613Z",
          "content": "<p>I save the results（features） to disk and use another script to read and concat with raw data.  In another word, I split the script of FE（multi processing）and the  Model script.</p>",
          "rawMarkdown": "I save the results（features） to disk and use another script to read and concat with raw data.  In another word, I split the script of FE（multi processing）and the  Model script.",
          "votes": 1
        },
        {
          "id": 323795,
          "postDate": "2018-05-06T08:28:52.200Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 321034,
      "postDate": "2018-04-30T12:47:59.643Z",
      "content": "<p>Very useful! Thanks for sharing ;)</p>",
      "rawMarkdown": "Very useful! Thanks for sharing ;)"
    }
  ],
  "comments": [
    {
      "id": 320938,
      "author_name": "蓝鲤鱼",
      "author_url": "",
      "post_date": "2018-04-30T08:47:32.040000",
      "content": "<p>666</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 323437,
      "author_name": "Eason.Chen",
      "author_url": "",
      "post_date": "2018-05-05T06:36:13.627000",
      "content": "<p>谢谢老铁</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 321564,
      "author_name": "Feiyang Pan",
      "author_url": "",
      "post_date": "2018-05-01T15:06:42.453000",
      "content": "<p>I wish I'd known these tips earlier... </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 321309,
      "author_name": "Wenjie Bai",
      "author_url": "",
      "post_date": "2018-05-01T01:52:35.220000",
      "content": "<p>真的皮，哈哈哈~</p>",
      "votes": 0,
      "replies": [
        {
          "id": 321373,
          "author_name": "spongebob",
          "author_url": "",
          "post_date": "2018-05-01T05:48:58.503000",
          "content": "<p>李时珍的皮</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 321097,
      "author_name": "Silverback",
      "author_url": "",
      "post_date": "2018-04-30T15:20:50.117000",
      "content": "<p>Here is another <a href=\"https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55379\">tip</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 320926,
      "author_name": "shark",
      "author_url": "",
      "post_date": "2018-04-30T08:05:27.133000",
      "content": "<p>Lihaile.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 320912,
      "author_name": "sparkingarthur",
      "author_url": "",
      "post_date": "2018-04-30T07:01:49.983000",
      "content": "<p>niubi</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 320761,
      "author_name": "MengYe",
      "author_url": "",
      "post_date": "2018-04-29T18:24:17.143000",
      "content": "<p>Just having problems to create encode features in kernel, you are a life saver. </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 320648,
      "author_name": "Yifan Xie",
      "author_url": "",
      "post_date": "2018-04-29T11:53:53.543000",
      "content": "<p>小本子记下来 - Note taken - ^ ^</p>",
      "votes": 0,
      "replies": [
        {
          "id": 320653,
          "author_name": "spongebob",
          "author_url": "",
          "post_date": "2018-04-29T12:11:00.360000",
          "content": "<p>哈哈-haha- ^ ^</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 320647,
      "author_name": "Vadim Borisov",
      "author_url": "",
      "post_date": "2018-04-29T11:49:40.333000",
      "content": "<p>thank you!</p>",
      "votes": 0,
      "replies": [
        {
          "id": 320651,
          "author_name": "spongebob",
          "author_url": "",
          "post_date": "2018-04-29T12:01:24.513000",
          "content": "<p>Your are welcome~</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 323789,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-05-06T08:10:55.243000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 323792,
          "author_name": "spongebob",
          "author_url": "",
          "post_date": "2018-05-06T08:20:39.613000",
          "content": "<p>I save the results（features） to disk and use another script to read and concat with raw data.  In another word, I split the script of FE（multi processing）and the  Model script.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 323795,
          "author_name": "",
          "author_url": "",
          "post_date": "2018-05-06T08:28:52.200000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 321034,
      "author_name": "Meyk",
      "author_url": "",
      "post_date": "2018-04-30T12:47:59.643000",
      "content": "<p>Very useful! Thanks for sharing ;)</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "320641": "1.Replace pd.concat([XX],axis=1) with df['X'] = temp['X']\nthis will save a lot of time when the dataframe is very large.\n\n2.Using  temp['X']=  temp['X'].astype('float32') will save a lot of ram.\n\n3.If you use multiprocessing package to accelerate feature extracting. Do not try  to get result by using \n\n&gt; pool.apply_async(XXX)\n\n with get method.  It have some bugs when the result is very large.\n\nEnjoy competition.",
    "320938": "666",
    "323437": "谢谢老铁",
    "321564": "I wish I'd known these tips earlier... ",
    "321309": "真的皮，哈哈哈~",
    "321097": "Here is another [tip][1]\n\n\n  [1]: https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/55379",
    "320926": "Lihaile.",
    "320912": "niubi",
    "320761": "Just having problems to create encode features in kernel, you are a life saver. ",
    "320648": "小本子记下来 - Note taken - ^ ^",
    "320647": "thank you!",
    "323789": "",
    "321034": "Very useful! Thanks for sharing ;)"
  }
}