{
  "id": 12433,
  "title": "?? in hexdump",
  "url": "/competitions/malware-classification/discussion/12433",
  "author_name": "",
  "post_date": "2015-02-06T20:55:59.180Z",
  "votes": null,
  "comment_count": 12,
  "views": 3616,
  "content": "<p>What does ?? represent in hexdump data?</p>",
  "messages": [
    {
      "id": "63725",
      "postDate": "02/06/2015 20:55:59",
      "content": "<p>What does ?? represent in hexdump data?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63731",
      "postDate": "02/06/2015 23:23:25",
      "content": "<p>Hi,&nbsp;Abhishek. I think you can safely ignore them for now. Most of those lines are not included in .asm, which means the&nbsp;disassembler didn't find anything useful there.&nbsp;</p>\n<p>Edit: I mean the lines including only ??? in the very end of the byte file.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63759",
      "postDate": "02/07/2015 22:16:38",
      "content": "<p>The ?? seem to fill the gap from the end of code/data, as defined in the PE file header, to the end of the so-called &quot;section&quot; (a &quot;section&quot; is defined by the Windows PE file format, the size of which&nbsp;is determined by memory alignment &amp; file alignment). &nbsp;These ?? &nbsp;may be zero'd when loaded into memory, or remain untouched...I haven't found a definitive answer on that yet.&nbsp;</p>\n<p>For example, in BFmbPXnMtl9oVyI1SYvT.asm &nbsp;you can see that the&nbsp;virtual size of the code (232 bytes) is less than the section size in file&nbsp;(512 bytes) : &nbsp;&nbsp;</p>\n<p style=\"padding-left: 30px\">.text:00401000 ; Format :<strong> Portable executable for 80386 (PE)</strong><br>.text:00401000 ; Imagebase : 400000<br>.text:00401000 ; Section 1. (virtual address 00001000)<br>.text:00401000 ;<strong> Virtual size : 000000E8 ( 232.)</strong><br>.text:00401000 ; <strong>Section size in file : 00000200 ( 512.)</strong></p>\n<p>You can also see ?? in the corresponding .bytes file starting after 232 bytes of code, so this makes sense.</p>\n<p>In any case, my&nbsp;conclusion is the same -- I think the ?? can be safely ignored for this contest.&nbsp;</p>\n<p style=\"padding-left: 30px\"><code>&nbsp;</code></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64255",
      "postDate": "02/15/2015 02:43:54",
      "content": "<p>This is just reserved space in memory. Like variables that are defined but not initialized. In the end of the file it must be Import Address Table that consist of a lot of such &quot;variables&quot; for every function imported from external dll. When program is loaded&nbsp;those variables are filled with the actual addresses of the external functions that are being used.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64628",
      "postDate": "02/20/2015 04:41:02",
      "content": "<p>I see '??' in almost all my files (both asm and bytes)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64874",
      "postDate": "02/25/2015 16:28:19",
      "content": "<p>So the number of ?? can be a feature?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64875",
      "postDate": "02/25/2015 16:33:24",
      "content": "<p>why not!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64911",
      "postDate": "02/26/2015 03:49:54",
      "content": "<p>Hey guys,</p>\n<p>I have quite a few files in test set that are 0KB.....is that the same for others or i lost data while decompressing?</p>\n<p>The total space on disk taken by test is 158 GB(21746 files)</p>\n<p>Thanks!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65224",
      "postDate": "03/02/2015 09:39:14",
      "content": "<p>Anyone???</p>\n\n<p>[quote=UD1989;64911]</p>\n<p>Hey guys,</p>\n<p>I have quite a few files in test set that are 0KB.....is that the same for others or i lost data while decompressing?</p>\n<p>The total space on disk taken by test is 158 GB(21746 files)</p>\n<p>Thanks!</p>\n<p>[/quote]</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65225",
      "postDate": "03/02/2015 10:04:07",
      "content": "<p>Its same for others too...!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65226",
      "postDate": "03/02/2015 10:16:44",
      "content": "<p>[quote=Abhishek;65225]</p>\n<p>Its same for others too...!</p>\n<p>[/quote]</p>\n\n<p>OH ok Thanks a lot!!!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65286",
      "postDate": "03/02/2015 20:12:48",
      "content": "<p>[quote=UD1989;65226]</p>\n<p>[quote=Abhishek;65225]</p>\n<p>Its same for others too...!</p>\n<p>[/quote]</p>\n<p>OH ok Thanks a lot!!!</p>\n<p>[/quote]</p>\n\n<p>I only see 21741 files (10873 in samplesubmission and 10868 in trainLabels)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65315",
      "postDate": "03/03/2015 02:35:59",
      "content": "<p>[quote=Jos&#233; A. Guerrero;65286]</p>\n<p>[quote=UD1989;65226]</p>\n<p>[quote=Abhishek;65225]</p>\n<p>Its same for others too...!</p>\n<p>[/quote]</p>\n<p>OH ok Thanks a lot!!!</p>\n<p>[/quote]</p>\n<p>I only see 21741 files (10873 in samplesubmission and 10868 in trainLabels)</p>\n<p>[/quote]</p>\n<p>Hi Jose'</p>\n<p>There are :</p>\n<p>21736 (half .bytre files and the other half .asm files...corresponding to 10868 ids) files in the training set</p>\n<p>21746(half .bytre files and the other half .asm files.....corresponding to 10873 ids) in the test set.</p>\n<p>Train Labels have 10868 IDs corresponding to those in Train set</p>\n<p>Sample submission has 10873 ids. (corresponding to those IDs in the test set)</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 63731,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "02/06/2015 23:23:25",
      "content": "<p>Hi,&nbsp;Abhishek. I think you can safely ignore them for now. Most of those lines are not included in .asm, which means the&nbsp;disassembler didn't find anything useful there.&nbsp;</p>\n<p>Edit: I mean the lines including only ??? in the very end of the byte file.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63759,
      "author_name": "chefele",
      "author_url": "",
      "post_date": "02/07/2015 22:16:38",
      "content": "<p>The ?? seem to fill the gap from the end of code/data, as defined in the PE file header, to the end of the so-called &quot;section&quot; (a &quot;section&quot; is defined by the Windows PE file format, the size of which&nbsp;is determined by memory alignment &amp; file alignment). &nbsp;These ?? &nbsp;may be zero'd when loaded into memory, or remain untouched...I haven't found a definitive answer on that yet.&nbsp;</p>\n<p>For example, in BFmbPXnMtl9oVyI1SYvT.asm &nbsp;you can see that the&nbsp;virtual size of the code (232 bytes) is less than the section size in file&nbsp;(512 bytes) : &nbsp;&nbsp;</p>\n<p style=\"padding-left: 30px\">.text:00401000 ; Format :<strong> Portable executable for 80386 (PE)</strong><br>.text:00401000 ; Imagebase : 400000<br>.text:00401000 ; Section 1. (virtual address 00001000)<br>.text:00401000 ;<strong> Virtual size : 000000E8 ( 232.)</strong><br>.text:00401000 ; <strong>Section size in file : 00000200 ( 512.)</strong></p>\n<p>You can also see ?? in the corresponding .bytes file starting after 232 bytes of code, so this makes sense.</p>\n<p>In any case, my&nbsp;conclusion is the same -- I think the ?? can be safely ignored for this contest.&nbsp;</p>\n<p style=\"padding-left: 30px\"><code>&nbsp;</code></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64255,
      "author_name": "yankov",
      "author_url": "",
      "post_date": "02/15/2015 02:43:54",
      "content": "<p>This is just reserved space in memory. Like variables that are defined but not initialized. In the end of the file it must be Import Address Table that consist of a lot of such &quot;variables&quot; for every function imported from external dll. When program is loaded&nbsp;those variables are filled with the actual addresses of the external functions that are being used.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64628,
      "author_name": "",
      "author_url": "",
      "post_date": "02/20/2015 04:41:02",
      "content": "<p>I see '??' in almost all my files (both asm and bytes)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64874,
      "author_name": "stevendu",
      "author_url": "",
      "post_date": "02/25/2015 16:28:19",
      "content": "<p>So the number of ?? can be a feature?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64875,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "02/25/2015 16:33:24",
      "content": "<p>why not!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64911,
      "author_name": "udatta",
      "author_url": "",
      "post_date": "02/26/2015 03:49:54",
      "content": "<p>Hey guys,</p>\n<p>I have quite a few files in test set that are 0KB.....is that the same for others or i lost data while decompressing?</p>\n<p>The total space on disk taken by test is 158 GB(21746 files)</p>\n<p>Thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65224,
      "author_name": "udatta",
      "author_url": "",
      "post_date": "03/02/2015 09:39:14",
      "content": "<p>Anyone???</p>\n\n<p>[quote=UD1989;64911]</p>\n<p>Hey guys,</p>\n<p>I have quite a few files in test set that are 0KB.....is that the same for others or i lost data while decompressing?</p>\n<p>The total space on disk taken by test is 158 GB(21746 files)</p>\n<p>Thanks!</p>\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65225,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "03/02/2015 10:04:07",
      "content": "<p>Its same for others too...!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65226,
      "author_name": "udatta",
      "author_url": "",
      "post_date": "03/02/2015 10:16:44",
      "content": "<p>[quote=Abhishek;65225]</p>\n<p>Its same for others too...!</p>\n<p>[/quote]</p>\n\n<p>OH ok Thanks a lot!!!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65286,
      "author_name": "blindape",
      "author_url": "",
      "post_date": "03/02/2015 20:12:48",
      "content": "<p>[quote=UD1989;65226]</p>\n<p>[quote=Abhishek;65225]</p>\n<p>Its same for others too...!</p>\n<p>[/quote]</p>\n<p>OH ok Thanks a lot!!!</p>\n<p>[/quote]</p>\n\n<p>I only see 21741 files (10873 in samplesubmission and 10868 in trainLabels)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65315,
      "author_name": "udatta",
      "author_url": "",
      "post_date": "03/03/2015 02:35:59",
      "content": "<p>[quote=Jos&#233; A. Guerrero;65286]</p>\n<p>[quote=UD1989;65226]</p>\n<p>[quote=Abhishek;65225]</p>\n<p>Its same for others too...!</p>\n<p>[/quote]</p>\n<p>OH ok Thanks a lot!!!</p>\n<p>[/quote]</p>\n<p>I only see 21741 files (10873 in samplesubmission and 10868 in trainLabels)</p>\n<p>[/quote]</p>\n<p>Hi Jose'</p>\n<p>There are :</p>\n<p>21736 (half .bytre files and the other half .asm files...corresponding to 10868 ids) files in the training set</p>\n<p>21746(half .bytre files and the other half .asm files.....corresponding to 10873 ids) in the test set.</p>\n<p>Train Labels have 10868 IDs corresponding to those in Train set</p>\n<p>Sample submission has 10873 ids. (corresponding to those IDs in the test set)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "63725": "",
    "63731": "",
    "63759": "",
    "64255": "",
    "64628": "",
    "64874": "",
    "64875": "",
    "64911": "",
    "65224": "",
    "65225": "",
    "65226": "",
    "65286": "",
    "65315": ""
  },
  "source": "meta"
}