When I was doing research for my Thesis I looked into a self-supervised approach from Huh et al. The Idea of the approach is, very consicely, that you can tell if parts of an image are manipulated by non matching processing pipeline (or camara) fingerprints in different Parts of the image. That means that you actually only train on real images (like photography).
\n\nWith some adjustments, I managed to make it work for facial forensics, with a quite good classification accuracy of 90%+ on the uncompressed FaceForensics++ dataset, that is with no beforehand knowledge about the manipulation method.
\n\nHowever, the final problem was that appliying compression to the image seemed to make this method very weak and I suspect this is because compression basicially creates a partly matching processing pipeline (same compression) between the real and the manipulated image parts. Maybe one you of the bright heads out there can solve this. You can check out my code here if you like.
", "messages": [ { "id": "761782", "postDate": "03/02/2020 23:37:16", "content": "When I was doing research for my Thesis I looked into a self-supervised approach from Huh et al. The Idea of the approach is, very consicely, that you can tell if parts of an image are manipulated by non matching processing pipeline (or camara) fingerprints in different Parts of the image. That means that you actually only train on real images (like photography).
\n\nWith some adjustments, I managed to make it work for facial forensics, with a quite good classification accuracy of 90%+ on the uncompressed FaceForensics++ dataset, that is with no beforehand knowledge about the manipulation method.
\n\nHowever, the final problem was that appliying compression to the image seemed to make this method very weak and I suspect this is because compression basicially creates a partly matching processing pipeline (same compression) between the real and the manipulated image parts. Maybe one you of the bright heads out there can solve this. You can check out my code here if you like.
", "rawMarkdown": "When I was doing research for my Thesis I looked into a self-supervised approach from [Huh et al.](https://github.com/minyoungg/selfconsistency) The Idea of the approach is, very consicely, that you can tell if parts of an image are manipulated by non matching processing pipeline (or camara) fingerprints in different Parts of the image. That means that you actually only train on real images (like photography).\n\nWith some adjustments, I managed to make it work for facial forensics, with a quite good classification accuracy of 90%+ on the uncompressed [FaceForensics++ dataset](https://github.com/ondyari/FaceForensics), that is with no beforehand knowledge about the manipulation method.\n\nHowever, the final problem was that appliying compression to the image seemed to make this method very weak and I suspect this is because compression basicially creates a partly matching processing pipeline (same compression) between the real and the manipulated image parts. Maybe one you of the bright heads out there can solve this. You can check out my code [here](https://github.com/christian-git-md/selfconsistency-deepfake-detection) if you like.", "votes": null }, { "id": "780464", "postDate": "03/20/2020 09:51:37", "content": "Hi, thanks for sharing. I am trying this code and find it takes a long time to generate a heat map, about 15s. Did you participate in the competition in this way? Can you provide the optimized code?
", "rawMarkdown": "Hi, thanks for sharing. I am trying this code and find it takes a long time to generate a heat map, about 15s. Did you participate in the competition in this way? Can you provide the optimized code?", "votes": null } ], "comments": [ { "id": 780464, "author_name": "zhaoshanghao", "author_url": "", "post_date": "03/20/2020 09:51:37", "content": "Hi, thanks for sharing. I am trying this code and find it takes a long time to generate a heat map, about 15s. Did you participate in the competition in this way? Can you provide the optimized code?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "761782": "When I was doing research for my Thesis I looked into a self-supervised approach from [Huh et al.](https://github.com/minyoungg/selfconsistency) The Idea of the approach is, very consicely, that you can tell if parts of an image are manipulated by non matching processing pipeline (or camara) fingerprints in different Parts of the image. That means that you actually only train on real images (like photography).\n\nWith some adjustments, I managed to make it work for facial forensics, with a quite good classification accuracy of 90%+ on the uncompressed [FaceForensics++ dataset](https://github.com/ondyari/FaceForensics), that is with no beforehand knowledge about the manipulation method.\n\nHowever, the final problem was that appliying compression to the image seemed to make this method very weak and I suspect this is because compression basicially creates a partly matching processing pipeline (same compression) between the real and the manipulated image parts. Maybe one you of the bright heads out there can solve this. You can check out my code [here](https://github.com/christian-git-md/selfconsistency-deepfake-detection) if you like.", "780464": "Hi, thanks for sharing. I am trying this code and find it takes a long time to generate a heat map, about 15s. Did you participate in the competition in this way? Can you provide the optimized code?" }, "source": "meta" }