{
  "id": 12902,
  "title": "Obfuscator.ACY is obfuscating :(",
  "url": "/competitions/malware-classification/discussion/12902",
  "author_name": "",
  "post_date": "2015-03-18T20:57:48.493Z",
  "votes": 1,
  "comment_count": 9,
  "views": 2364,
  "content": "<p>Class 8, Obfuscator.ACY, is the major contributor to our local validation errors. Any one has advice on how to deal with it? Thank you very much</p>",
  "messages": [
    {
      "id": "67085",
      "postDate": "03/18/2015 20:57:48",
      "content": "<p>Class 8, Obfuscator.ACY, is the major contributor to our local validation errors. Any one has advice on how to deal with it? Thank you very much</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67099",
      "postDate": "03/18/2015 22:17:59",
      "content": "<p>what do you mean by local validation errors?</p>\n<p>I get high classification accuracy for class 8, but the remaining ones does get confused with all other classes</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67101",
      "postDate": "03/18/2015 22:21:14",
      "content": "<p>I mean log loss. the log loss from class 8 is larger than say, class 2 by one order of magnitude</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67103",
      "postDate": "03/18/2015 22:23:31",
      "content": "<p>oh, I use classification accuracies of every class for my validation. May be I should start using log loss too</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67104",
      "postDate": "03/18/2015 22:24:30",
      "content": "<p>for me, class 8 has maximum misclassifcations over all other classes but its own individual accuracy is high</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67106",
      "postDate": "03/18/2015 22:27:39",
      "content": "<p>What happened if you just delete it, my dear team mate. :)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67107",
      "postDate": "03/18/2015 22:32:53",
      "content": "<p>[quote=Little Boat;67106]</p>\n<p>What happened if you just delete it, my dear team mate. :)</p>\n<p>[/quote]</p>\n<p>Haha, I mean it's true label is class 8, but it is predicted to be other classes. In that case I cannot delete it, can I?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67109",
      "postDate": "03/18/2015 22:47:10",
      "content": "<p>[quote=rcarson;67107]</p>\n<p>[quote=Little Boat;67106]</p>\n<p>What happened if you just delete it, my dear team mate. :)</p>\n<p>[/quote]</p>\n<p>Haha, I mean it's true label is class 8, but it is predicted to be other classes. In that case I cannot delete it, can I?</p>\n<p>[/quote]</p>\n<p>Well, it can be just one outlier. Maybe just split the data into train/validate sets, model train and test validate set. And take the Obfuscator file out (we make it in the train split.), and then test the validate set again. &nbsp;Now you can compare the results. To be statistically significant, maybe do this experiment 10 or 20 times to compare?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67110",
      "postDate": "03/18/2015 22:50:36",
      "content": "<p>@ Little boat, let's talk privately :P</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "67112",
      "postDate": "03/18/2015 22:58:31",
      "content": "<p>[quote=rcarson;67110]<br>@ Little boat, let's talk privately :P<br>[/quote]</p>\n<p>So we can not steal ideas :( :P</p>\n<p>Anyway, you are not alone, same here, Class 8 is the hardest and is totally expected, that's exactly the purpose&nbsp;of an obfuscator&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 67099,
      "author_name": "",
      "author_url": "",
      "post_date": "03/18/2015 22:17:59",
      "content": "<p>what do you mean by local validation errors?</p>\n<p>I get high classification accuracy for class 8, but the remaining ones does get confused with all other classes</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67101,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/18/2015 22:21:14",
      "content": "<p>I mean log loss. the log loss from class 8 is larger than say, class 2 by one order of magnitude</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67103,
      "author_name": "",
      "author_url": "",
      "post_date": "03/18/2015 22:23:31",
      "content": "<p>oh, I use classification accuracies of every class for my validation. May be I should start using log loss too</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67104,
      "author_name": "",
      "author_url": "",
      "post_date": "03/18/2015 22:24:30",
      "content": "<p>for me, class 8 has maximum misclassifcations over all other classes but its own individual accuracy is high</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67106,
      "author_name": "xiaozhouwang",
      "author_url": "",
      "post_date": "03/18/2015 22:27:39",
      "content": "<p>What happened if you just delete it, my dear team mate. :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67107,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/18/2015 22:32:53",
      "content": "<p>[quote=Little Boat;67106]</p>\n<p>What happened if you just delete it, my dear team mate. :)</p>\n<p>[/quote]</p>\n<p>Haha, I mean it's true label is class 8, but it is predicted to be other classes. In that case I cannot delete it, can I?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67109,
      "author_name": "xiaozhouwang",
      "author_url": "",
      "post_date": "03/18/2015 22:47:10",
      "content": "<p>[quote=rcarson;67107]</p>\n<p>[quote=Little Boat;67106]</p>\n<p>What happened if you just delete it, my dear team mate. :)</p>\n<p>[/quote]</p>\n<p>Haha, I mean it's true label is class 8, but it is predicted to be other classes. In that case I cannot delete it, can I?</p>\n<p>[/quote]</p>\n<p>Well, it can be just one outlier. Maybe just split the data into train/validate sets, model train and test validate set. And take the Obfuscator file out (we make it in the train split.), and then test the validate set again. &nbsp;Now you can compare the results. To be statistically significant, maybe do this experiment 10 or 20 times to compare?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67110,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/18/2015 22:50:36",
      "content": "<p>@ Little boat, let's talk privately :P</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 67112,
      "author_name": "carloshuertas",
      "author_url": "",
      "post_date": "03/18/2015 22:58:31",
      "content": "<p>[quote=rcarson;67110]<br>@ Little boat, let's talk privately :P<br>[/quote]</p>\n<p>So we can not steal ideas :( :P</p>\n<p>Anyway, you are not alone, same here, Class 8 is the hardest and is totally expected, that's exactly the purpose&nbsp;of an obfuscator&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "67085": "",
    "67099": "",
    "67101": "",
    "67103": "",
    "67104": "",
    "67106": "",
    "67107": "",
    "67109": "",
    "67110": "",
    "67112": ""
  },
  "source": "meta"
}