![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F451025%2F533e8f096b4d3b870f1a0f177ab1c9e1%2Feekozbeafq.png?generation=1579273720279540&alt=media\")
I found some really interesting face while analyzing the videos for this clustering and eda kernel:
\n\n
.
\n\n.
\n\n.
\n\nWhile most videos only have one actor/face for the whole duration of the video. Others may have a second or even this damn painting! lol
\n\nTherefore, we have to be careful when training any kind of model in this comp. Perhaps it's a good thing as the real world is full of stuff like this!
\n\ndlib based algorithms seem to completely ignore the face above:
\n\n![\"original](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F451025%2F4e7d24214c5df6e721661d608fd4c15e%2FeekozbeafqOrig.png?generation=1579273913112028&alt=media\")
I found some really interesting face while analyzing the videos for this clustering and eda kernel:
\n\n
.
\n\n.
\n\n.
\n\nWhile most videos only have one actor/face for the whole duration of the video. Others may have a second or even this damn painting! lol
\n\nTherefore, we have to be careful when training any kind of model in this comp. Perhaps it's a good thing as the real world is full of stuff like this!
\n\ndlib based algorithms seem to completely ignore the face above:
\n\n
it gets even worst actually\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F451025%2Fbd474192a0b8972346672657eb4cc6c9%2Fmad_faces.png?generation=1579279774562990&alt=media\")
\nlol
The data is from @unkownhihi's dataset https://www.kaggle.com/unkownhihi/deepfake which I believe uses MTCNN.\n(The plots come from https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda)
\n\n@unkownhihi Please don't get me wrong. I really appreciate your job and thanks for sharing it. I just feel I have to warn everyone how trick CV is at this scale.
", "rawMarkdown": "it gets even worst actually\n\nlol\n\nThe data is from @unkownhihi's dataset https://www.kaggle.com/unkownhihi/deepfake which I believe uses MTCNN.\n(The plots come from https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda)\n\n@unkownhihi Please don't get me wrong. I really appreciate your job and thanks for sharing it. I just feel I have to warn everyone how trick CV is at this scale.", "votes": null }, { "id": "721756", "postDate": "01/17/2020 17:00:21", "content": "This is really hard. I set mtcnn threshold value as 0.9 and select the face that have the max probability but still have a lot of non-face pictures.\nDlib would work for part of them, but dlib won't be able to detect the faces that are in dark places(a lot partion of them are in dark places). Maybe we should add liveness detection!
", "rawMarkdown": "This is really hard. I set mtcnn threshold value as 0.9 and select the face that have the max probability but still have a lot of non-face pictures.\nDlib would work for part of them, but dlib won't be able to detect the faces that are in dark places(a lot partion of them are in dark places). Maybe we should add liveness detection!", "votes": null }, { "id": "721773", "postDate": "01/17/2020 17:20:00", "content": "The face detector I'm using sometimes gives false positives (I'm sure they all do) but these tend to be relatively large (they're not faces but part of the background). So I filter out faces that have more than 10% of the pixels in the video frame as most real faces are smaller than that.
", "rawMarkdown": "The face detector I'm using sometimes gives false positives (I'm sure they all do) but these tend to be relatively large (they're not faces but part of the background). So I filter out faces that have more than 10% of the pixels in the video frame as most real faces are smaller than that.", "votes": null }, { "id": "721792", "postDate": "01/17/2020 17:42:29", "content": "Another example with a sofa and hands identified as face:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103
\n\nFace confidence = 0.991 for the Sofa.
\n\nYou can find many other examples with faces on T-shirt, board, in sky. Some wheels too.\nNothing is perfect in ML world 😏
\n\nI think additional sanity check is required per video as said by @humananalog if most faces extracted in a video are 250x250 then 70x70 or 500x500 might not be faces. It means you're using boxes extraction and not resized faces directly.
", "rawMarkdown": "Another example with a sofa and hands identified as face:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103\n\nFace confidence = 0.991 for the Sofa.\n\nYou can find many other examples with faces on T-shirt, board, in sky. Some wheels too.\nNothing is perfect in ML world 😏 \n\nI think additional sanity check is required per video as said by @humananalog if most faces extracted in a video are 250x250 then 70x70 or 500x500 might not be faces. It means you're using boxes extraction and not resized faces directly.", "votes": null }, { "id": "721805", "postDate": "01/17/2020 17:57:51", "content": "One thing that I'm doing to try to avoid this is keeping only faces that were detected more than X times during the video (X being a parameter to be tuned). What I mean by this is: I track a face during the video using the centre of the detected box, if this centre is within a limit of any other face (centre of the detected box) of the frames before I conclude is the same face and update the centre. After processing the whole video I get rid of the cases where a \"face\" was detected less than X times.
\n\nWhat I notice is although sometimes the detection models do have false positives they don't tend to \"stay\" for the duration of the whole video, so over all frames only a few will have a false positive. I haven't done an extensive check to see how much improvement this yields, I just notice that work in some cases and went with it.
", "rawMarkdown": "One thing that I'm doing to try to avoid this is keeping only faces that were detected more than X times during the video (X being a parameter to be tuned). What I mean by this is: I track a face during the video using the centre of the detected box, if this centre is within a limit of any other face (centre of the detected box) of the frames before I conclude is the same face and update the centre. After processing the whole video I get rid of the cases where a \"face\" was detected less than X times.\n\nWhat I notice is although sometimes the detection models do have false positives they don't tend to \"stay\" for the duration of the whole video, so over all frames only a few will have a false positive. I haven't done an extensive check to see how much improvement this yields, I just notice that work in some cases and went with it.", "votes": null }, { "id": "721810", "postDate": "01/17/2020 18:06:27", "content": "Nice to track a face. Then you're working with all frames in a video to make it reliable? You're able to do the same during inference and within the time limit?
", "rawMarkdown": "Nice to track a face. Then you're working with all frames in a video to make it reliable? You're able to do the same during inference and within the time limit?", "votes": null }, { "id": "721811", "postDate": "01/17/2020 18:13:20", "content": "Using all frames you can use a small limit when comparing the centres of the faces and you can track it quite precisely, if you use less frames you can tune this limit accordingly. For now, I'm not using all frames, but it works well too, although, like I said, I haven't checked extensively.
", "rawMarkdown": "Using all frames you can use a small limit when comparing the centres of the faces and you can track it quite precisely, if you use less frames you can tune this limit accordingly. For now, I'm not using all frames, but it works well too, although, like I said, I haven't checked extensively.", "votes": null }, { "id": "721821", "postDate": "01/17/2020 18:24:56", "content": "Thanks, I'm going to try that too. In this video the camera is moving a lot:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103
", "rawMarkdown": "Thanks, I'm going to try that too. In this video the camera is moving a lot:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103", "votes": null }, { "id": "721859", "postDate": "01/17/2020 19:47:28", "content": "Another interesting one. https://www.kaggle.com/c/deepfake-detection-challenge/discussion/125165
\n\nI don't think these are a problem - it all boils down to how you handle videos with more than one face.
", "rawMarkdown": "Another interesting one. https://www.kaggle.com/c/deepfake-detection-challenge/discussion/125165\n\nI don't think these are a problem - it all boils down to how you handle videos with more than one face.", "votes": null }, { "id": "721992", "postDate": "01/18/2020 01:04:40", "content": "Yep! You're exactly right! But I already done with this dataset so I don't want to waste another $100 credits in redoing this dataset. I will do that in the future when I create a new one.
", "rawMarkdown": "Yep! You're exactly right! But I already done with this dataset so I don't want to waste another $100 credits in redoing this dataset. I will do that in the future when I create a new one.", "votes": null }, { "id": "723022", "postDate": "01/19/2020 12:04:18", "content": "lol that video is pretty bad, apparently the fakers also use similar (error prone) facial detection.
\n\nLike in adversarial attacks, what looks like a face for the CNN doesn't necessarily need to look like a face for us (or any target class, for the sake of the argument).\nHere are some more examples of MTCNN failure on random patterns (from https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda):\n![\"random](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F451025%2F12dd6d5c9e80bf9e54d0e7d9bea7af07%2Fbad1.png?generation=1579435726683129&alt=media\")
Great Job! I'll try asap.Thanks for sharing :)
", "rawMarkdown": "Great Job! I'll try asap.Thanks for sharing :)", "votes": null }, { "id": "724775", "postDate": "01/21/2020 13:23:11", "content": "I have actually done this myself. But one key problem that I keep facing is that sometimes, faces get interchanged whenever there is a crossover happening.
", "rawMarkdown": "I have actually done this myself. But one key problem that I keep facing is that sometimes, faces get interchanged whenever there is a crossover happening.", "votes": null } ], "comments": [ { "id": 721738, "author_name": "hmendonca", "author_url": "", "post_date": "01/17/2020 16:50:03", "content": "it gets even worst actually\n\nlol
The data is from @unkownhihi's dataset https://www.kaggle.com/unkownhihi/deepfake which I believe uses MTCNN.\n(The plots come from https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda)
\n\n@unkownhihi Please don't get me wrong. I really appreciate your job and thanks for sharing it. I just feel I have to warn everyone how trick CV is at this scale.
", "votes": null, "replies": [ { "id": 721756, "author_name": "unkownhihi", "author_url": "", "post_date": "01/17/2020 17:00:21", "content": "This is really hard. I set mtcnn threshold value as 0.9 and select the face that have the max probability but still have a lot of non-face pictures.\nDlib would work for part of them, but dlib won't be able to detect the faces that are in dark places(a lot partion of them are in dark places). Maybe we should add liveness detection!
", "votes": null, "replies": [] }, { "id": 721792, "author_name": "mpware", "author_url": "", "post_date": "01/17/2020 17:42:29", "content": "Another example with a sofa and hands identified as face:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103
\n\nFace confidence = 0.991 for the Sofa.
\n\nYou can find many other examples with faces on T-shirt, board, in sky. Some wheels too.\nNothing is perfect in ML world 😏
\n\nI think additional sanity check is required per video as said by @humananalog if most faces extracted in a video are 250x250 then 70x70 or 500x500 might not be faces. It means you're using boxes extraction and not resized faces directly.
", "votes": null, "replies": [] }, { "id": 721992, "author_name": "unkownhihi", "author_url": "", "post_date": "01/18/2020 01:04:40", "content": "Yep! You're exactly right! But I already done with this dataset so I don't want to waste another $100 credits in redoing this dataset. I will do that in the future when I create a new one.
", "votes": null, "replies": [] } ] }, { "id": 721773, "author_name": "humananalog", "author_url": "", "post_date": "01/17/2020 17:20:00", "content": "The face detector I'm using sometimes gives false positives (I'm sure they all do) but these tend to be relatively large (they're not faces but part of the background). So I filter out faces that have more than 10% of the pixels in the video frame as most real faces are smaller than that.
", "votes": null, "replies": [] }, { "id": 721805, "author_name": "pedromb", "author_url": "", "post_date": "01/17/2020 17:57:51", "content": "One thing that I'm doing to try to avoid this is keeping only faces that were detected more than X times during the video (X being a parameter to be tuned). What I mean by this is: I track a face during the video using the centre of the detected box, if this centre is within a limit of any other face (centre of the detected box) of the frames before I conclude is the same face and update the centre. After processing the whole video I get rid of the cases where a \"face\" was detected less than X times.
\n\nWhat I notice is although sometimes the detection models do have false positives they don't tend to \"stay\" for the duration of the whole video, so over all frames only a few will have a false positive. I haven't done an extensive check to see how much improvement this yields, I just notice that work in some cases and went with it.
", "votes": null, "replies": [ { "id": 721810, "author_name": "mpware", "author_url": "", "post_date": "01/17/2020 18:06:27", "content": "Nice to track a face. Then you're working with all frames in a video to make it reliable? You're able to do the same during inference and within the time limit?
", "votes": null, "replies": [] }, { "id": 721811, "author_name": "pedromb", "author_url": "", "post_date": "01/17/2020 18:13:20", "content": "Using all frames you can use a small limit when comparing the centres of the faces and you can track it quite precisely, if you use less frames you can tune this limit accordingly. For now, I'm not using all frames, but it works well too, although, like I said, I haven't checked extensively.
", "votes": null, "replies": [] }, { "id": 721821, "author_name": "mpware", "author_url": "", "post_date": "01/17/2020 18:24:56", "content": "Thanks, I'm going to try that too. In this video the camera is moving a lot:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103
", "votes": null, "replies": [] }, { "id": 724775, "author_name": "akashnandi", "author_url": "", "post_date": "01/21/2020 13:23:11", "content": "I have actually done this myself. But one key problem that I keep facing is that sometimes, faces get interchanged whenever there is a crossover happening.
", "votes": null, "replies": [] } ] }, { "id": 721859, "author_name": "petewills", "author_url": "", "post_date": "01/17/2020 19:47:28", "content": "Another interesting one. https://www.kaggle.com/c/deepfake-detection-challenge/discussion/125165
\n\nI don't think these are a problem - it all boils down to how you handle videos with more than one face.
", "votes": null, "replies": [ { "id": 723022, "author_name": "hmendonca", "author_url": "", "post_date": "01/19/2020 12:04:18", "content": "lol that video is pretty bad, apparently the fakers also use similar (error prone) facial detection.
\n\nLike in adversarial attacks, what looks like a face for the CNN doesn't necessarily need to look like a face for us (or any target class, for the sake of the argument).\nHere are some more examples of MTCNN failure on random patterns (from https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda):\n
Great Job! I'll try asap.Thanks for sharing :)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "721734": "I found some really interesting face while analyzing the videos for this [clustering and eda kernel](https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda):\n\n\n\n.\n\n.\n\n.\n\nWhile most videos only have one actor/face for the whole duration of the video. Others may have a second or even this damn painting! lol\n\nTherefore, we have to be careful when training any kind of model in this comp. Perhaps it's a good thing as the real world is full of stuff like this!\n\ndlib based algorithms seem to completely ignore the face above:\n\n", "721738": "it gets even worst actually\n\nlol\n\nThe data is from @unkownhihi's dataset https://www.kaggle.com/unkownhihi/deepfake which I believe uses MTCNN.\n(The plots come from https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda)\n\n@unkownhihi Please don't get me wrong. I really appreciate your job and thanks for sharing it. I just feel I have to warn everyone how trick CV is at this scale.", "721756": "This is really hard. I set mtcnn threshold value as 0.9 and select the face that have the max probability but still have a lot of non-face pictures.\nDlib would work for part of them, but dlib won't be able to detect the faces that are in dark places(a lot partion of them are in dark places). Maybe we should add liveness detection!", "721773": "The face detector I'm using sometimes gives false positives (I'm sure they all do) but these tend to be relatively large (they're not faces but part of the background). So I filter out faces that have more than 10% of the pixels in the video frame as most real faces are smaller than that.", "721792": "Another example with a sofa and hands identified as face:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103\n\nFace confidence = 0.991 for the Sofa.\n\nYou can find many other examples with faces on T-shirt, board, in sky. Some wheels too.\nNothing is perfect in ML world 😏 \n\nI think additional sanity check is required per video as said by @humananalog if most faces extracted in a video are 250x250 then 70x70 or 500x500 might not be faces. It means you're using boxes extraction and not resized faces directly.", "721805": "One thing that I'm doing to try to avoid this is keeping only faces that were detected more than X times during the video (X being a parameter to be tuned). What I mean by this is: I track a face during the video using the centre of the detected box, if this centre is within a limit of any other face (centre of the detected box) of the frames before I conclude is the same face and update the centre. After processing the whole video I get rid of the cases where a \"face\" was detected less than X times.\n\nWhat I notice is although sometimes the detection models do have false positives they don't tend to \"stay\" for the duration of the whole video, so over all frames only a few will have a false positive. I haven't done an extensive check to see how much improvement this yields, I just notice that work in some cases and went with it.", "721810": "Nice to track a face. Then you're working with all frames in a video to make it reliable? You're able to do the same during inference and within the time limit?", "721811": "Using all frames you can use a small limit when comparing the centres of the faces and you can track it quite precisely, if you use less frames you can tune this limit accordingly. For now, I'm not using all frames, but it works well too, although, like I said, I haven't checked extensively.", "721821": "Thanks, I'm going to try that too. In this video the camera is moving a lot:\nhttps://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch#713103", "721859": "Another interesting one. https://www.kaggle.com/c/deepfake-detection-challenge/discussion/125165\n\nI don't think these are a problem - it all boils down to how you handle videos with more than one face.", "721992": "Yep! You're exactly right! But I already done with this dataset so I don't want to waste another $100 credits in redoing this dataset. I will do that in the future when I create a new one.", "723022": "lol that video is pretty bad, apparently the fakers also use similar (error prone) facial detection.\n\nLike in adversarial attacks, what looks like a face for the CNN doesn't necessarily need to look like a face for us (or any target class, for the sake of the argument).\nHere are some more examples of MTCNN failure on random patterns (from https://www.kaggle.com/hmendonca/proper-clustering-with-facenet-embeddings-eda):\n", "723694": "Great Job! I'll try asap.Thanks for sharing :)", "724775": "I have actually done this myself. But one key problem that I keep facing is that sometimes, faces get interchanged whenever there is a crossover happening." }, "source": "meta" }