{
  "id": 15830,
  "title": "total number of html files?",
  "url": "/competitions/dato-native/discussion/15830",
  "author_name": "",
  "post_date": "2015-08-07T09:47:42.267Z",
  "votes": 1,
  "comment_count": 10,
  "views": 1736,
  "content": "<p>Im getting 67536... thats the total number of HTML files?</p>\n\n<p>EDIT: this issue is fixed.</p>",
  "messages": [
    {
      "id": "88796",
      "postDate": "08/07/2015 09:47:42",
      "content": "<p>Im getting 67536... thats the total number of HTML files?</p>\n\n<p>EDIT: this issue is fixed.</p>",
      "rawMarkdown": "Im getting 67536... thats the total number of HTML files?\r\n\r\nEDIT: this issue is fixed.",
      "votes": null
    },
    {
      "id": "89058",
      "postDate": "08/10/2015 21:01:41",
      "content": "<p>When I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\nRedoing it with 7zip got the full amount.</p>\n\n<p>Not so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.</p>",
      "rawMarkdown": "When I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\r\nRedoing it with 7zip got the full amount.\r\n\r\nNot so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.",
      "votes": null
    },
    {
      "id": "89266",
      "postDate": "08/13/2015 04:10:53",
      "content": "<p>I get the following:</p>\n\n<p>zip 0 = 67508 <br>\nzip 1 = 67535 <br>\nzip 2 = 67123 <br>\nzip 3 = 67606 <br>\nzip 4 = 67537</p>",
      "rawMarkdown": "I get the following:\r\n\r\nzip 0 = 67508  \r\nzip 1 = 67535  \r\nzip 2 = 67123  \r\nzip 3 = 67606  \r\nzip 4 = 67537",
      "votes": null
    },
    {
      "id": "89287",
      "postDate": "08/13/2015 11:28:00",
      "content": "<p>[quote=Ceekr;89266]</p>\n\n<p>I get the following:</p>\n\n<p>zip 0 = 67508 <br>\nzip 1 = 67535 <br>\nzip 2 = 67123 <br>\nzip 3 = 67606 <br>\nzip 4 = 67537</p>\n\n<p>[/quote]</p>\n\n<p>Doing &quot;ls N | wc -l&quot; on any dir, where N is the directory 0-4, I get 1 fewer file in each count than you, on an Ubuntu system.</p>\n\n<p>Right clicking on the folder in Windows and going to Properties to get the internal file count, same numbers as the Linux system for me (one fewer in each, relative to yours).</p>",
      "rawMarkdown": "[quote=Ceekr;89266]\r\n\r\nI get the following:\r\n\r\nzip 0 = 67508  \r\nzip 1 = 67535  \r\nzip 2 = 67123  \r\nzip 3 = 67606  \r\nzip 4 = 67537\r\n\r\n[/quote]\r\n\r\nDoing \"ls N | wc -l\" on any dir, where N is the directory 0-4, I get 1 fewer file in each count than you, on an Ubuntu system.\r\n\r\nRight clicking on the folder in Windows and going to Properties to get the internal file count, same numbers as the Linux system for me (one fewer in each, relative to yours).",
      "votes": null
    },
    {
      "id": "89379",
      "postDate": "08/14/2015 18:29:47",
      "content": "<p>@omgponies - you are right. small bug in the py code; here are the correct counts:</p>\n\n<p>zip 0 = 67507</p>\n\n<p>zip 1 = 67534</p>\n\n<p>zip 2 = 67122</p>\n\n<p>zip 3 = 67605</p>\n\n<p>zip 4 = 67536</p>",
      "rawMarkdown": "omgponies - you are right. small bug in the py code; here are the correct counts:\r\n\r\nzip 0 = 67507\r\n\r\nzip 1 = 67534\r\n\r\nzip 2 = 67122\r\n\r\nzip 3 = 67605\r\n\r\nzip 4 = 67536",
      "votes": null
    },
    {
      "id": "90364",
      "postDate": "08/25/2015 19:34:36",
      "content": "<p>[quote=omgponies;89058]</p>\n\n<p>When I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\nRedoing it with 7zip got the full amount.</p>\n\n<p>Not so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.</p>\n\n<p>[/quote]</p>\n\n<p>Sorry, totally noob question, but why does Windows do this? I had the same issue where Windows only unzipped a fraction of the data (haven't tried with 7zip yet). Is it standard knowledge that Windows &quot;Extract All&quot; will only give a fraction of the data? I've never encountered this before.</p>",
      "rawMarkdown": "[quote=omgponies;89058]\r\n\r\nWhen I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\r\nRedoing it with 7zip got the full amount.\r\n\r\nNot so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.\r\n\r\n[/quote]\r\n\r\nSorry, totally noob question, but why does Windows do this? I had the same issue where Windows only unzipped a fraction of the data (haven't tried with 7zip yet). Is it standard knowledge that Windows \"Extract All\" will only give a fraction of the data? I've never encountered this before.",
      "votes": null
    },
    {
      "id": "90366",
      "postDate": "08/25/2015 20:31:39",
      "content": "<p>Not sure re Windows. I would assume it is either an issue with that many files in a single directory, or total size. The size thing would make more sense on a 32-bit machine, but I still had the issue on a 64-bit machine.</p>\n\n<p>The fact that it doesn't give any error would also make me think that it misses some checksum and thinks it is done - so maybe it is a non-compatible zip format that it fails to recognize correctly.</p>\n\n<p>Windows has a lot of these sorts of things, keeps one on their toes.</p>\n\n<p>Main thing is to do a sanity check - in this case, the unzipped folder was smaller than the zip file - that is extremely unusual when dealing with non-encrypted text.</p>",
      "rawMarkdown": "Not sure re Windows. I would assume it is either an issue with that many files in a single directory, or total size. The size thing would make more sense on a 32-bit machine, but I still had the issue on a 64-bit machine.\r\n\r\nThe fact that it doesn't give any error would also make me think that it misses some checksum and thinks it is done - so maybe it is a non-compatible zip format that it fails to recognize correctly.\r\n\r\nWindows has a lot of these sorts of things, keeps one on their toes.\r\n\r\nMain thing is to do a sanity check - in this case, the unzipped folder was smaller than the zip file - that is extremely unusual when dealing with non-encrypted text.",
      "votes": null
    },
    {
      "id": "90384",
      "postDate": "08/26/2015 00:10:18",
      "content": "<p>Thanks for the explanation! I'll likely use 7zip for compression moving forward. I'm not nearly as nimble as I used to be to keep up with the nasty things Windows can throw my way :-)</p>",
      "rawMarkdown": "Thanks for the explanation! I'll likely use 7zip for compression moving forward. I'm not nearly as nimble as I used to be to keep up with the nasty things Windows can throw my way :-)",
      "votes": null
    },
    {
      "id": "92661",
      "postDate": "09/15/2015 07:42:35",
      "content": "<p>How many files do you get now for test set, please?</p>",
      "rawMarkdown": "How many files do you get now for test set, please?",
      "votes": null
    },
    {
      "id": "92666",
      "postDate": "09/15/2015 08:42:48",
      "content": "<p>train set: 337304\ntest set: 66772</p>",
      "rawMarkdown": "train set: 337304\r\ntest set: 66772",
      "votes": null
    },
    {
      "id": "92671",
      "postDate": "09/15/2015 10:33:52",
      "content": "<p>Thank you:)</p>",
      "rawMarkdown": "Thank you:)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 89058,
      "author_name": "omgponies",
      "author_url": "",
      "post_date": "08/10/2015 21:01:41",
      "content": "<p>When I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\nRedoing it with 7zip got the full amount.</p>\n\n<p>Not so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 89266,
      "author_name": "ceeker",
      "author_url": "",
      "post_date": "08/13/2015 04:10:53",
      "content": "<p>I get the following:</p>\n\n<p>zip 0 = 67508 <br>\nzip 1 = 67535 <br>\nzip 2 = 67123 <br>\nzip 3 = 67606 <br>\nzip 4 = 67537</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 89287,
      "author_name": "omgponies",
      "author_url": "",
      "post_date": "08/13/2015 11:28:00",
      "content": "<p>[quote=Ceekr;89266]</p>\n\n<p>I get the following:</p>\n\n<p>zip 0 = 67508 <br>\nzip 1 = 67535 <br>\nzip 2 = 67123 <br>\nzip 3 = 67606 <br>\nzip 4 = 67537</p>\n\n<p>[/quote]</p>\n\n<p>Doing &quot;ls N | wc -l&quot; on any dir, where N is the directory 0-4, I get 1 fewer file in each count than you, on an Ubuntu system.</p>\n\n<p>Right clicking on the folder in Windows and going to Properties to get the internal file count, same numbers as the Linux system for me (one fewer in each, relative to yours).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 89379,
      "author_name": "ceeker",
      "author_url": "",
      "post_date": "08/14/2015 18:29:47",
      "content": "<p>@omgponies - you are right. small bug in the py code; here are the correct counts:</p>\n\n<p>zip 0 = 67507</p>\n\n<p>zip 1 = 67534</p>\n\n<p>zip 2 = 67122</p>\n\n<p>zip 3 = 67605</p>\n\n<p>zip 4 = 67536</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 90364,
      "author_name": "devonbrackbill",
      "author_url": "",
      "post_date": "08/25/2015 19:34:36",
      "content": "<p>[quote=omgponies;89058]</p>\n\n<p>When I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\nRedoing it with 7zip got the full amount.</p>\n\n<p>Not so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.</p>\n\n<p>[/quote]</p>\n\n<p>Sorry, totally noob question, but why does Windows do this? I had the same issue where Windows only unzipped a fraction of the data (haven't tried with 7zip yet). Is it standard knowledge that Windows &quot;Extract All&quot; will only give a fraction of the data? I've never encountered this before.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 90366,
      "author_name": "omgponies",
      "author_url": "",
      "post_date": "08/25/2015 20:31:39",
      "content": "<p>Not sure re Windows. I would assume it is either an issue with that many files in a single directory, or total size. The size thing would make more sense on a 32-bit machine, but I still had the issue on a 64-bit machine.</p>\n\n<p>The fact that it doesn't give any error would also make me think that it misses some checksum and thinks it is done - so maybe it is a non-compatible zip format that it fails to recognize correctly.</p>\n\n<p>Windows has a lot of these sorts of things, keeps one on their toes.</p>\n\n<p>Main thing is to do a sanity check - in this case, the unzipped folder was smaller than the zip file - that is extremely unusual when dealing with non-encrypted text.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 90384,
      "author_name": "devonbrackbill",
      "author_url": "",
      "post_date": "08/26/2015 00:10:18",
      "content": "<p>Thanks for the explanation! I'll likely use 7zip for compression moving forward. I'm not nearly as nimble as I used to be to keep up with the nasty things Windows can throw my way :-)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 92661,
      "author_name": "wildwizard",
      "author_url": "",
      "post_date": "09/15/2015 07:42:35",
      "content": "<p>How many files do you get now for test set, please?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 92666,
      "author_name": "bekbolatov",
      "author_url": "",
      "post_date": "09/15/2015 08:42:48",
      "content": "<p>train set: 337304\ntest set: 66772</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 92671,
      "author_name": "wildwizard",
      "author_url": "",
      "post_date": "09/15/2015 10:33:52",
      "content": "<p>Thank you:)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "88796": "Im getting 67536... thats the total number of HTML files?\r\n\r\nEDIT: this issue is fixed.",
    "89058": "When I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\r\nRedoing it with 7zip got the full amount.\r\n\r\nNot so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.",
    "89266": "I get the following:\r\n\r\nzip 0 = 67508  \r\nzip 1 = 67535  \r\nzip 2 = 67123  \r\nzip 3 = 67606  \r\nzip 4 = 67537",
    "89287": "[quote=Ceekr;89266]\r\n\r\nI get the following:\r\n\r\nzip 0 = 67508  \r\nzip 1 = 67535  \r\nzip 2 = 67123  \r\nzip 3 = 67606  \r\nzip 4 = 67537\r\n\r\n[/quote]\r\n\r\nDoing \"ls N | wc -l\" on any dir, where N is the directory 0-4, I get 1 fewer file in each count than you, on an Ubuntu system.\r\n\r\nRight clicking on the folder in Windows and going to Properties to get the internal file count, same numbers as the Linux system for me (one fewer in each, relative to yours).",
    "89379": "omgponies - you are right. small bug in the py code; here are the correct counts:\r\n\r\nzip 0 = 67507\r\n\r\nzip 1 = 67534\r\n\r\nzip 2 = 67122\r\n\r\nzip 3 = 67605\r\n\r\nzip 4 = 67536",
    "90364": "[quote=omgponies;89058]\r\n\r\nWhen I unzipped using Windows, it gave me a fraction of the even the size of the original zip.\r\nRedoing it with 7zip got the full amount.\r\n\r\nNot so much posting this for you, since you clearly are moving forward, but more for others as they come into this if they do the same thing as me.\r\n\r\n[/quote]\r\n\r\nSorry, totally noob question, but why does Windows do this? I had the same issue where Windows only unzipped a fraction of the data (haven't tried with 7zip yet). Is it standard knowledge that Windows \"Extract All\" will only give a fraction of the data? I've never encountered this before.",
    "90366": "Not sure re Windows. I would assume it is either an issue with that many files in a single directory, or total size. The size thing would make more sense on a 32-bit machine, but I still had the issue on a 64-bit machine.\r\n\r\nThe fact that it doesn't give any error would also make me think that it misses some checksum and thinks it is done - so maybe it is a non-compatible zip format that it fails to recognize correctly.\r\n\r\nWindows has a lot of these sorts of things, keeps one on their toes.\r\n\r\nMain thing is to do a sanity check - in this case, the unzipped folder was smaller than the zip file - that is extremely unusual when dealing with non-encrypted text.",
    "90384": "Thanks for the explanation! I'll likely use 7zip for compression moving forward. I'm not nearly as nimble as I used to be to keep up with the nasty things Windows can throw my way :-)",
    "92661": "How many files do you get now for test set, please?",
    "92666": "train set: 337304\r\ntest set: 66772",
    "92671": "Thank you:)"
  },
  "source": "meta"
}