{
  "id": 51650,
  "title": "why Fraud detection shouldnt be at IP level",
  "url": "/competitions/talkingdata-adtracking-fraud-detection/discussion/51650",
  "author_name": "",
  "post_date": "2018-03-11T16:39:49.375867300Z",
  "votes": 2,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Analysis of IP 39 activity gives the following insights\n1) first click itself got converted. from device 0\n2) If each IP is considered individual user then that user 39 wont click again because he has already downloaded in his first click\n3)IP 39 made 12 more clicks with same device \"1\" but with different OS 13 and 18\n4)change of OS for IP 39 happened between 16:18:32 and 16:20:04 in a span of 1:32 minutes. which is highly unlikely\nSo combination of IP,Device and OS should be considered a single user and fraud detection should be done at that user level.\nSuggestions are welcome\nThanks\n<img src=\"http://i66.tinypic.com/bfp095.png\" alt=\"IP39\"></p>",
  "messages": [
    {
      "id": "294254",
      "postDate": "03/11/2018 16:39:49",
      "content": "<p>Analysis of IP 39 activity gives the following insights\n1) first click itself got converted. from device 0\n2) If each IP is considered individual user then that user 39 wont click again because he has already downloaded in his first click\n3)IP 39 made 12 more clicks with same device \"1\" but with different OS 13 and 18\n4)change of OS for IP 39 happened between 16:18:32 and 16:20:04 in a span of 1:32 minutes. which is highly unlikely\nSo combination of IP,Device and OS should be considered a single user and fraud detection should be done at that user level.\nSuggestions are welcome\nThanks\n<img src=\"http://i66.tinypic.com/bfp095.png\" alt=\"IP39\"></p>",
      "rawMarkdown": "Analysis of IP 39 activity gives the following insights\n1) first click itself got converted. from device 0\n2) If each IP is considered individual user then that user 39 wont click again because he has already downloaded in his first click\n3)IP 39 made 12 more clicks with same device \"1\" but with different OS 13 and 18\n4)change of OS for IP 39 happened between 16:18:32 and 16:20:04 in a span of 1:32 minutes. which is highly unlikely\nSo combination of IP,Device and OS should be considered a single user and fraud detection should be done at that user level.\nSuggestions are welcome\nThanks\n![IP39][1]\n\n\n  [1]: http://i66.tinypic.com/bfp095.png",
      "votes": null
    },
    {
      "id": "294522",
      "postDate": "03/12/2018 04:10:16",
      "content": "<p>Same IP doesn't mean it is from the same user.  Especially if this is mobile operator. Most IP are shared, your phone most likely behind several (layered) NAT before its address exposed to internet. </p>\n\n<p><img src=\"https://i.imgur.com/vz49rnQ.jpg\" alt=\"NAT\"></p>\n\n<p>I still don't understand why thinking about fraud detection gonna help win this competition. \nI believe you should treat this competition more like CTR problem (Download Trough Rate to be exact).</p>\n\n<p>Leave the utilization of this model to enhance fraud detection to Talkingdata</p>",
      "rawMarkdown": "Same IP doesn't mean it is from the same user.  Especially if this is mobile operator. Most IP are shared, your phone most likely behind several (layered) NAT before its address exposed to internet. \n\n![NAT][1]\n\nI still don't understand why thinking about fraud detection gonna help win this competition. \nI believe you should treat this competition more like CTR problem (Download Trough Rate to be exact).\n\nLeave the utilization of this model to enhance fraud detection to Talkingdata\n\n\n  [1]: https://i.imgur.com/vz49rnQ.jpg",
      "votes": null
    },
    {
      "id": "294550",
      "postDate": "03/12/2018 06:05:31",
      "content": "<p>Hi Muhamad,\nThats what my inference is \"IP is not equivalent to user\". IP can be shared one also. In the above case i mentioned : if IP is an user then he wont click after first instance because he already downloaded in the first click itself. A combination of IP,device and OS MAY be considered as a single user.</p>\n\n<p>Here is my understanding of diff between \"Fraud prediction\" and \"Prediction of download(Y/N)\"</p>\n\n<ol>\n<li><p>Fraud pred: Model will be at <strong>clicks</strong> level. For ex , a single\nuser who has made 10 clicks, the probability of \"10th\" click being a\nfraud click will be more than \"First\" click being a fraud one</p></li>\n<li><p>Download(Y/N) classification model: Model will be at user level. For\nex, A user who has made 10 clicks the probability of he downloading\nthe game will be higher than a user who has made a single click</p></li>\n</ol>\n\n<p>Hope it helps. :)</p>",
      "rawMarkdown": "Hi Muhamad,\nThats what my inference is \"IP is not equivalent to user\". IP can be shared one also. In the above case i mentioned : if IP is an user then he wont click after first instance because he already downloaded in the first click itself. A combination of IP,device and OS MAY be considered as a single user.\n\nHere is my understanding of diff between \"Fraud prediction\" and \"Prediction of download(Y/N)\"\n\n 1. Fraud pred: Model will be at **clicks** level. For ex , a single\n    user who has made 10 clicks, the probability of \"10th\" click being a\n    fraud click will be more than \"First\" click being a fraud one\n\n 2. Download(Y/N) classification model: Model will be at user level. For\n    ex, A user who has made 10 clicks the probability of he downloading\n    the game will be higher than a user who has made a single click\n\nHope it helps. :)",
      "votes": null
    },
    {
      "id": "294558",
      "postDate": "03/12/2018 06:27:08",
      "content": "<p>Ok, now I see what you mean by \"user level\" you want to somehow include \"user\" behavior as additional consideration in the model right? </p>",
      "rawMarkdown": "Ok, now I see what you mean by \"user level\" you want to somehow include \"user\" behavior as additional consideration in the model right?",
      "votes": null
    },
    {
      "id": "294567",
      "postDate": "03/12/2018 06:44:04",
      "content": "<p>Exactly. Hope u agree with my interpretation of difference between \"Fraud detection model\" and \"download(Y/N)\" model </p>",
      "rawMarkdown": "Exactly. Hope u agree with my interpretation of difference between \"Fraud detection model\" and \"download(Y/N)\" model",
      "votes": null
    },
    {
      "id": "294596",
      "postDate": "03/12/2018 07:51:10",
      "content": "<p>Completely agree with your thoughts, Nagaraju. One thing though - what about differentiation between app as well? Just because I clicked on app 9 and decided not to download it, doesn't necessarily means that if I click on app 8 I won't convert. Another thing which your example shows: multiple clicks from the same user (IP+device+OS) within 1 minute most probably indicates a bot and we all know that bots don't know how to convert :)</p>",
      "rawMarkdown": "Completely agree with your thoughts, Nagaraju. One thing though - what about differentiation between app as well? Just because I clicked on app 9 and decided not to download it, doesn't necessarily means that if I click on app 8 I won't convert. Another thing which your example shows: multiple clicks from the same user (IP+device+OS) within 1 minute most probably indicates a bot and we all know that bots don't know how to convert :)",
      "votes": null
    },
    {
      "id": "294626",
      "postDate": "03/12/2018 09:40:59",
      "content": "<p>Agree. Need to figure out how to capture that exception cases. </p>",
      "rawMarkdown": "Agree. Need to figure out how to capture that exception cases.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 294522,
      "author_name": "muhammadalfiansyah",
      "author_url": "",
      "post_date": "03/12/2018 04:10:16",
      "content": "<p>Same IP doesn't mean it is from the same user.  Especially if this is mobile operator. Most IP are shared, your phone most likely behind several (layered) NAT before its address exposed to internet. </p>\n\n<p><img src=\"https://i.imgur.com/vz49rnQ.jpg\" alt=\"NAT\"></p>\n\n<p>I still don't understand why thinking about fraud detection gonna help win this competition. \nI believe you should treat this competition more like CTR problem (Download Trough Rate to be exact).</p>\n\n<p>Leave the utilization of this model to enhance fraud detection to Talkingdata</p>",
      "votes": null,
      "replies": [
        {
          "id": 294550,
          "author_name": "rajuspartan",
          "author_url": "",
          "post_date": "03/12/2018 06:05:31",
          "content": "<p>Hi Muhamad,\nThats what my inference is \"IP is not equivalent to user\". IP can be shared one also. In the above case i mentioned : if IP is an user then he wont click after first instance because he already downloaded in the first click itself. A combination of IP,device and OS MAY be considered as a single user.</p>\n\n<p>Here is my understanding of diff between \"Fraud prediction\" and \"Prediction of download(Y/N)\"</p>\n\n<ol>\n<li><p>Fraud pred: Model will be at <strong>clicks</strong> level. For ex , a single\nuser who has made 10 clicks, the probability of \"10th\" click being a\nfraud click will be more than \"First\" click being a fraud one</p></li>\n<li><p>Download(Y/N) classification model: Model will be at user level. For\nex, A user who has made 10 clicks the probability of he downloading\nthe game will be higher than a user who has made a single click</p></li>\n</ol>\n\n<p>Hope it helps. :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 294558,
      "author_name": "muhammadalfiansyah",
      "author_url": "",
      "post_date": "03/12/2018 06:27:08",
      "content": "<p>Ok, now I see what you mean by \"user level\" you want to somehow include \"user\" behavior as additional consideration in the model right? </p>",
      "votes": null,
      "replies": [
        {
          "id": 294567,
          "author_name": "rajuspartan",
          "author_url": "",
          "post_date": "03/12/2018 06:44:04",
          "content": "<p>Exactly. Hope u agree with my interpretation of difference between \"Fraud detection model\" and \"download(Y/N)\" model </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 294596,
      "author_name": "asparuhhristov",
      "author_url": "",
      "post_date": "03/12/2018 07:51:10",
      "content": "<p>Completely agree with your thoughts, Nagaraju. One thing though - what about differentiation between app as well? Just because I clicked on app 9 and decided not to download it, doesn't necessarily means that if I click on app 8 I won't convert. Another thing which your example shows: multiple clicks from the same user (IP+device+OS) within 1 minute most probably indicates a bot and we all know that bots don't know how to convert :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 294626,
          "author_name": "rajuspartan",
          "author_url": "",
          "post_date": "03/12/2018 09:40:59",
          "content": "<p>Agree. Need to figure out how to capture that exception cases. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "294254": "Analysis of IP 39 activity gives the following insights\n1) first click itself got converted. from device 0\n2) If each IP is considered individual user then that user 39 wont click again because he has already downloaded in his first click\n3)IP 39 made 12 more clicks with same device \"1\" but with different OS 13 and 18\n4)change of OS for IP 39 happened between 16:18:32 and 16:20:04 in a span of 1:32 minutes. which is highly unlikely\nSo combination of IP,Device and OS should be considered a single user and fraud detection should be done at that user level.\nSuggestions are welcome\nThanks\n![IP39][1]\n\n\n  [1]: http://i66.tinypic.com/bfp095.png",
    "294522": "Same IP doesn't mean it is from the same user.  Especially if this is mobile operator. Most IP are shared, your phone most likely behind several (layered) NAT before its address exposed to internet. \n\n![NAT][1]\n\nI still don't understand why thinking about fraud detection gonna help win this competition. \nI believe you should treat this competition more like CTR problem (Download Trough Rate to be exact).\n\nLeave the utilization of this model to enhance fraud detection to Talkingdata\n\n\n  [1]: https://i.imgur.com/vz49rnQ.jpg",
    "294550": "Hi Muhamad,\nThats what my inference is \"IP is not equivalent to user\". IP can be shared one also. In the above case i mentioned : if IP is an user then he wont click after first instance because he already downloaded in the first click itself. A combination of IP,device and OS MAY be considered as a single user.\n\nHere is my understanding of diff between \"Fraud prediction\" and \"Prediction of download(Y/N)\"\n\n 1. Fraud pred: Model will be at **clicks** level. For ex , a single\n    user who has made 10 clicks, the probability of \"10th\" click being a\n    fraud click will be more than \"First\" click being a fraud one\n\n 2. Download(Y/N) classification model: Model will be at user level. For\n    ex, A user who has made 10 clicks the probability of he downloading\n    the game will be higher than a user who has made a single click\n\nHope it helps. :)",
    "294558": "Ok, now I see what you mean by \"user level\" you want to somehow include \"user\" behavior as additional consideration in the model right?",
    "294567": "Exactly. Hope u agree with my interpretation of difference between \"Fraud detection model\" and \"download(Y/N)\" model",
    "294596": "Completely agree with your thoughts, Nagaraju. One thing though - what about differentiation between app as well? Just because I clicked on app 9 and decided not to download it, doesn't necessarily means that if I click on app 8 I won't convert. Another thing which your example shows: multiple clicks from the same user (IP+device+OS) within 1 minute most probably indicates a bot and we all know that bots don't know how to convert :)",
    "294626": "Agree. Need to figure out how to capture that exception cases."
  },
  "source": "meta"
}