{
  "id": 347792,
  "title": "Convert h5 file to csv",
  "url": "/competitions/open-problems-multimodal/discussion/347792",
  "author_name": "",
  "post_date": "2022-08-25T11:42:27.093932400Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi there<br>\nI can't convert h5 files to csv. Or read them with Python.<br>\nAny guidance will be appreciated.</p>",
  "messages": [
    {
      "id": "1913588",
      "postDate": "08/25/2022 11:42:27",
      "content": "<p>Hi there<br>\nI can't convert h5 files to csv. Or read them with Python.<br>\nAny guidance will be appreciated.</p>",
      "rawMarkdown": "Hi there\nI can't convert h5 files to csv. Or read them with Python.\nAny guidance will be appreciated.",
      "votes": null
    },
    {
      "id": "1913826",
      "postDate": "08/25/2022 14:26:10",
      "content": "<p>Look at any of the <a href=\"https://www.kaggle.com/competitions/open-problems-multimodal/code?competitionId=38128&amp;sortBy=voteCount\" target=\"_blank\">public notebooks</a>…</p>",
      "rawMarkdown": "Look at any of the [public notebooks](https://www.kaggle.com/competitions/open-problems-multimodal/code?competitionId=38128&sortBy=voteCount)...",
      "votes": null
    },
    {
      "id": "1913952",
      "postDate": "08/25/2022 15:47:01",
      "content": "<p>It's a H5 file, so you should read them with read_hdf, but you cant use chunks, like:</p>\n<p>pd.read_hdf</p>\n<p>Since the H5 files are sparse arrays it is possible to use scipy sps. Or entirely by pandas with pd.read_hdf(file, start=start, stop=stop)(if you can't fit in memory).<br>\nSparse Matrix Guide:<br>\n<a href=\"https://www.kaggle.com/code/sbunzini/reduce-memory-usage-by-95-with-sparse-matrices\" target=\"_blank\">Scipy</a><br>\nOr with Pandas:<br>\n<a href=\"https://www.kaggle.com/code/nandodmelo/cite-xgboost-v1\" target=\"_blank\">Pandas</a></p>",
      "rawMarkdown": "It's a H5 file, so you should read them with read_hdf, but you cant use chunks, like:\n\npd.read_hdf\n\nSince the H5 files are sparse arrays it is possible to use scipy sps. Or entirely by pandas with pd.read_hdf(file, start=start, stop=stop)(if you can't fit in memory).\nSparse Matrix Guide:\n[Scipy](https://www.kaggle.com/code/sbunzini/reduce-memory-usage-by-95-with-sparse-matrices)\nOr with Pandas:\n[Pandas](https://www.kaggle.com/code/nandodmelo/cite-xgboost-v1)",
      "votes": null
    },
    {
      "id": "1915754",
      "postDate": "08/27/2022 09:48:22",
      "content": "<p>Thanks<br>\nA very good explanation on the organization and exploitation of HDF5 files <a href=\"http://matlab.izmiran.ru/help/techdoc/matlab_prog/ch_imp41.html\" target=\"_blank\">here</a>.</p>",
      "rawMarkdown": "Thanks\nA very good explanation on the organization and exploitation of HDF5 files [here](http://matlab.izmiran.ru/help/techdoc/matlab_prog/ch_imp41.html).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1913826,
      "author_name": "ambrosm",
      "author_url": "",
      "post_date": "08/25/2022 14:26:10",
      "content": "<p>Look at any of the <a href=\"https://www.kaggle.com/competitions/open-problems-multimodal/code?competitionId=38128&amp;sortBy=voteCount\" target=\"_blank\">public notebooks</a>…</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1913952,
      "author_name": "nandodmelo",
      "author_url": "",
      "post_date": "08/25/2022 15:47:01",
      "content": "<p>It's a H5 file, so you should read them with read_hdf, but you cant use chunks, like:</p>\n<p>pd.read_hdf</p>\n<p>Since the H5 files are sparse arrays it is possible to use scipy sps. Or entirely by pandas with pd.read_hdf(file, start=start, stop=stop)(if you can't fit in memory).<br>\nSparse Matrix Guide:<br>\n<a href=\"https://www.kaggle.com/code/sbunzini/reduce-memory-usage-by-95-with-sparse-matrices\" target=\"_blank\">Scipy</a><br>\nOr with Pandas:<br>\n<a href=\"https://www.kaggle.com/code/nandodmelo/cite-xgboost-v1\" target=\"_blank\">Pandas</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1915754,
      "author_name": "doru99",
      "author_url": "",
      "post_date": "08/27/2022 09:48:22",
      "content": "<p>Thanks<br>\nA very good explanation on the organization and exploitation of HDF5 files <a href=\"http://matlab.izmiran.ru/help/techdoc/matlab_prog/ch_imp41.html\" target=\"_blank\">here</a>.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1913588": "Hi there\nI can't convert h5 files to csv. Or read them with Python.\nAny guidance will be appreciated.",
    "1913826": "Look at any of the [public notebooks](https://www.kaggle.com/competitions/open-problems-multimodal/code?competitionId=38128&sortBy=voteCount)...",
    "1913952": "It's a H5 file, so you should read them with read_hdf, but you cant use chunks, like:\n\npd.read_hdf\n\nSince the H5 files are sparse arrays it is possible to use scipy sps. Or entirely by pandas with pd.read_hdf(file, start=start, stop=stop)(if you can't fit in memory).\nSparse Matrix Guide:\n[Scipy](https://www.kaggle.com/code/sbunzini/reduce-memory-usage-by-95-with-sparse-matrices)\nOr with Pandas:\n[Pandas](https://www.kaggle.com/code/nandodmelo/cite-xgboost-v1)",
    "1915754": "Thanks\nA very good explanation on the organization and exploitation of HDF5 files [here](http://matlab.izmiran.ru/help/techdoc/matlab_prog/ch_imp41.html)."
  },
  "source": "meta"
}