{
  "id": 72467,
  "title": "ts_fresh. extract_features taking too long in my Laptop",
  "url": "/competitions/PLAsTiCC-2018/discussion/72467",
  "author_name": "",
  "post_date": "2018-11-23T13:59:00.111739300Z",
  "votes": 3,
  "comment_count": 17,
  "views": 0,
  "content": "<pre><code>  from tsfresh.feature_extraction import extract_features\n\n       fcp = {'fft_coefficient': [{'coeff': 0, 'attr': 'abs'},{'coeff': 1, 'attr': 'abs'}],'kurtosis' : None, 'skewness' :None}\n\n       agg_df_ts = extract_features(df, column_id='object_id',\n                                    column_sort='mjd',\n                                    column_kind='passband',\n                                    column_value = 'flux',\n                                    default_fc_parameters = fcp, n_jobs=1)\n</code></pre>",
  "messages": [
    {
      "id": "426596",
      "postDate": "11/23/2018 13:59:00",
      "content": "<pre><code>  from tsfresh.feature_extraction import extract_features\n\n       fcp = {'fft_coefficient': [{'coeff': 0, 'attr': 'abs'},{'coeff': 1, 'attr': 'abs'}],'kurtosis' : None, 'skewness' :None}\n\n       agg_df_ts = extract_features(df, column_id='object_id',\n                                    column_sort='mjd',\n                                    column_kind='passband',\n                                    column_value = 'flux',\n                                    default_fc_parameters = fcp, n_jobs=1)\n</code></pre>",
      "rawMarkdown": "from tsfresh.feature_extraction import extract_features\n           \n           fcp = {'fft_coefficient': [{'coeff': 0, 'attr': 'abs'},{'coeff': 1, 'attr': 'abs'}],'kurtosis' : None, 'skewness' :None}\n           \n           agg_df_ts = extract_features(df, column_id='object_id',\n                                        column_sort='mjd',\n                                        column_kind='passband',\n                                        column_value = 'flux',\n                                        default_fc_parameters = fcp, n_jobs=1)",
      "votes": null
    },
    {
      "id": "426597",
      "postDate": "11/23/2018 14:06:34",
      "content": "<p>Are these features useful?  I am not using them so far.</p>",
      "rawMarkdown": "Are these features useful?  I am not using them so far.",
      "votes": null
    },
    {
      "id": "426606",
      "postDate": "11/23/2018 14:29:43",
      "content": "<p>that is i want to test, if it runs in my pc </p>",
      "rawMarkdown": "that is i want to test, if it runs in my pc",
      "votes": null
    },
    {
      "id": "426619",
      "postDate": "11/23/2018 14:46:42",
      "content": "<p>They give a 0.1 CV improvement for me</p>",
      "rawMarkdown": "They give a 0.1 CV improvement for me",
      "votes": null
    },
    {
      "id": "426622",
      "postDate": "11/23/2018 14:54:50",
      "content": "<p>Takes about 4 seconds to process those features (on the training set) for me on a 2.8 GHz 6-Core i5 and 16 GB ram</p>",
      "rawMarkdown": "Takes about 4 seconds to process those features (on the training set) for me on a 2.8 GHz 6-Core i5 and 16 GB ram",
      "votes": null
    },
    {
      "id": "426635",
      "postDate": "11/23/2018 15:17:30",
      "content": "<p>I'll have a try.</p>",
      "rawMarkdown": "I'll have a try.",
      "votes": null
    },
    {
      "id": "426637",
      "postDate": "11/23/2018 15:22:58",
      "content": "<p>changing n_jobs from 1 to the number of your cores, will make it about #cores times faster. On the training set it should be fast(~10 sec), the test set is much bigger.</p>",
      "rawMarkdown": "changing n_jobs from 1 to the number of your cores, will make it about #cores times faster. On the training set it should be fast(~10 sec), the test set is much bigger.",
      "votes": null
    },
    {
      "id": "426639",
      "postDate": "11/23/2018 15:27:35",
      "content": "<p>I guess your other features capture at least as good or probably better the info that these features coarsely try to capture (periodicity, curvedness and assymetry per band) </p>",
      "rawMarkdown": "I guess your other features capture at least as good or probably better the info that these features coarsely try to capture (periodicity, curvedness and assymetry per band)",
      "votes": null
    },
    {
      "id": "426648",
      "postDate": "11/23/2018 15:37:30",
      "content": "<p>i7 (2.9 GHz), 16 GB, waiting since last 5 hrs (on training set). I have no idea what is the issue :(</p>",
      "rawMarkdown": "i7 (2.9 GHz), 16 GB, waiting since last 5 hrs (on training set). I have no idea what is the issue :(",
      "votes": null
    },
    {
      "id": "426649",
      "postDate": "11/23/2018 15:37:49",
      "content": "<p>waiting since last 5 hrs (on training set). I have no idea what is the issue :(</p>",
      "rawMarkdown": "waiting since last 5 hrs (on training set). I have no idea what is the issue :(",
      "votes": null
    },
    {
      "id": "426656",
      "postDate": "11/23/2018 15:47:10",
      "content": "<p>sorry, I don't know how to help, I've only used it from kaggle kernels. It is weird though, so have a look for any sneaky bugs and make sure your run the latest versions of the python libraries.</p>",
      "rawMarkdown": "sorry, I don't know how to help, I've only used it from kaggle kernels. It is weird though, so have a look for any sneaky bugs and make sure your run the latest versions of the python libraries.",
      "votes": null
    },
    {
      "id": "426779",
      "postDate": "11/23/2018 20:30:41",
      "content": "<p>tsfresh should show a progress bar, do you see the progress bar changing?</p>",
      "rawMarkdown": "tsfresh should show a progress bar, do you see the progress bar changing?",
      "votes": null
    },
    {
      "id": "426785",
      "postDate": "11/23/2018 21:05:07",
      "content": "<p>no its stagnant like this</p>\n\n<p>Feature Extraction:   0%|          | 0/5 [00:00</p>",
      "rawMarkdown": "no its stagnant like this\n\n\nFeature Extraction:   0%|          | 0/5 [00:00",
      "votes": null
    },
    {
      "id": "426798",
      "postDate": "11/23/2018 21:49:44",
      "content": "<p>I would try going through the tsfresh quick start guide <a href=\"https://tsfresh.readthedocs.io/en/latest/text/quick_start.html\">https://tsfresh.readthedocs.io/en/latest/text/quick_start.html</a>\nIf extracting features from robot execution failures dataset does not work then there is probably something wrong with your tsfresh installation. </p>",
      "rawMarkdown": "I would try going through the tsfresh quick start guide https://tsfresh.readthedocs.io/en/latest/text/quick_start.html\nIf extracting features from robot execution failures dataset does not work then there is probably something wrong with your tsfresh installation.",
      "votes": null
    },
    {
      "id": "426800",
      "postDate": "11/23/2018 22:03:03",
      "content": "<p>Thanks, resolved</p>",
      "rawMarkdown": "Thanks, resolved",
      "votes": null
    },
    {
      "id": "426801",
      "postDate": "11/23/2018 22:07:23",
      "content": "<p>Great! The installation was the problem?</p>",
      "rawMarkdown": "Great! The installation was the problem?",
      "votes": null
    },
    {
      "id": "427076",
      "postDate": "11/24/2018 14:02:56",
      "content": "<p>yes, thanks !</p>",
      "rawMarkdown": "yes, thanks !",
      "votes": null
    },
    {
      "id": "427113",
      "postDate": "11/24/2018 16:08:12",
      "content": "<p>You should use a progress bar, like tqdm or tqdm_notebook to monitor your computation.</p>",
      "rawMarkdown": "You should use a progress bar, like tqdm or tqdm_notebook to monitor your computation.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 426597,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "11/23/2018 14:06:34",
      "content": "<p>Are these features useful?  I am not using them so far.</p>",
      "votes": null,
      "replies": [
        {
          "id": 426606,
          "author_name": "mks2192",
          "author_url": "",
          "post_date": "11/23/2018 14:29:43",
          "content": "<p>that is i want to test, if it runs in my pc </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426619,
          "author_name": "jackvial",
          "author_url": "",
          "post_date": "11/23/2018 14:46:42",
          "content": "<p>They give a 0.1 CV improvement for me</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426635,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "11/23/2018 15:17:30",
          "content": "<p>I'll have a try.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426639,
          "author_name": "iprapas",
          "author_url": "",
          "post_date": "11/23/2018 15:27:35",
          "content": "<p>I guess your other features capture at least as good or probably better the info that these features coarsely try to capture (periodicity, curvedness and assymetry per band) </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 426622,
      "author_name": "jackvial",
      "author_url": "",
      "post_date": "11/23/2018 14:54:50",
      "content": "<p>Takes about 4 seconds to process those features (on the training set) for me on a 2.8 GHz 6-Core i5 and 16 GB ram</p>",
      "votes": null,
      "replies": [
        {
          "id": 426648,
          "author_name": "mks2192",
          "author_url": "",
          "post_date": "11/23/2018 15:37:30",
          "content": "<p>i7 (2.9 GHz), 16 GB, waiting since last 5 hrs (on training set). I have no idea what is the issue :(</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426779,
          "author_name": "jackvial",
          "author_url": "",
          "post_date": "11/23/2018 20:30:41",
          "content": "<p>tsfresh should show a progress bar, do you see the progress bar changing?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426785,
          "author_name": "mks2192",
          "author_url": "",
          "post_date": "11/23/2018 21:05:07",
          "content": "<p>no its stagnant like this</p>\n\n<p>Feature Extraction:   0%|          | 0/5 [00:00</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426798,
          "author_name": "jackvial",
          "author_url": "",
          "post_date": "11/23/2018 21:49:44",
          "content": "<p>I would try going through the tsfresh quick start guide <a href=\"https://tsfresh.readthedocs.io/en/latest/text/quick_start.html\">https://tsfresh.readthedocs.io/en/latest/text/quick_start.html</a>\nIf extracting features from robot execution failures dataset does not work then there is probably something wrong with your tsfresh installation. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426800,
          "author_name": "mks2192",
          "author_url": "",
          "post_date": "11/23/2018 22:03:03",
          "content": "<p>Thanks, resolved</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426801,
          "author_name": "jackvial",
          "author_url": "",
          "post_date": "11/23/2018 22:07:23",
          "content": "<p>Great! The installation was the problem?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 427076,
          "author_name": "mks2192",
          "author_url": "",
          "post_date": "11/24/2018 14:02:56",
          "content": "<p>yes, thanks !</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 426637,
      "author_name": "iprapas",
      "author_url": "",
      "post_date": "11/23/2018 15:22:58",
      "content": "<p>changing n_jobs from 1 to the number of your cores, will make it about #cores times faster. On the training set it should be fast(~10 sec), the test set is much bigger.</p>",
      "votes": null,
      "replies": [
        {
          "id": 426649,
          "author_name": "mks2192",
          "author_url": "",
          "post_date": "11/23/2018 15:37:49",
          "content": "<p>waiting since last 5 hrs (on training set). I have no idea what is the issue :(</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 426656,
          "author_name": "iprapas",
          "author_url": "",
          "post_date": "11/23/2018 15:47:10",
          "content": "<p>sorry, I don't know how to help, I've only used it from kaggle kernels. It is weird though, so have a look for any sneaky bugs and make sure your run the latest versions of the python libraries.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 427113,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "11/24/2018 16:08:12",
          "content": "<p>You should use a progress bar, like tqdm or tqdm_notebook to monitor your computation.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "426596": "from tsfresh.feature_extraction import extract_features\n           \n           fcp = {'fft_coefficient': [{'coeff': 0, 'attr': 'abs'},{'coeff': 1, 'attr': 'abs'}],'kurtosis' : None, 'skewness' :None}\n           \n           agg_df_ts = extract_features(df, column_id='object_id',\n                                        column_sort='mjd',\n                                        column_kind='passband',\n                                        column_value = 'flux',\n                                        default_fc_parameters = fcp, n_jobs=1)",
    "426597": "Are these features useful?  I am not using them so far.",
    "426606": "that is i want to test, if it runs in my pc",
    "426619": "They give a 0.1 CV improvement for me",
    "426622": "Takes about 4 seconds to process those features (on the training set) for me on a 2.8 GHz 6-Core i5 and 16 GB ram",
    "426635": "I'll have a try.",
    "426637": "changing n_jobs from 1 to the number of your cores, will make it about #cores times faster. On the training set it should be fast(~10 sec), the test set is much bigger.",
    "426639": "I guess your other features capture at least as good or probably better the info that these features coarsely try to capture (periodicity, curvedness and assymetry per band)",
    "426648": "i7 (2.9 GHz), 16 GB, waiting since last 5 hrs (on training set). I have no idea what is the issue :(",
    "426649": "waiting since last 5 hrs (on training set). I have no idea what is the issue :(",
    "426656": "sorry, I don't know how to help, I've only used it from kaggle kernels. It is weird though, so have a look for any sneaky bugs and make sure your run the latest versions of the python libraries.",
    "426779": "tsfresh should show a progress bar, do you see the progress bar changing?",
    "426785": "no its stagnant like this\n\n\nFeature Extraction:   0%|          | 0/5 [00:00",
    "426798": "I would try going through the tsfresh quick start guide https://tsfresh.readthedocs.io/en/latest/text/quick_start.html\nIf extracting features from robot execution failures dataset does not work then there is probably something wrong with your tsfresh installation.",
    "426800": "Thanks, resolved",
    "426801": "Great! The installation was the problem?",
    "427076": "yes, thanks !",
    "427113": "You should use a progress bar, like tqdm or tqdm_notebook to monitor your computation."
  },
  "source": "meta"
}