Deepfake is a word used throughout the internet to mean a variety of things. It was originally used as a reddit name in reference to pornography. I think the term has evolved a great deal since then, and I think it would be beneficial to provide a more specific definition of this term.
\n\nOr is the supposition that we are supposed to assume the definition is provided by the labels on the training dataset? Is this an example where \"I know it when I see it\" or do you have a rigorous definition.
\n\nFor example, I don't think the de-aging in The Irishman was Deepfakes, but maybe you do!
", "messages": [ { "id": "693873", "postDate": "12/12/2019 21:55:46", "content": "Deepfake is a word used throughout the internet to mean a variety of things. It was originally used as a reddit name in reference to pornography. I think the term has evolved a great deal since then, and I think it would be beneficial to provide a more specific definition of this term.
\n\nOr is the supposition that we are supposed to assume the definition is provided by the labels on the training dataset? Is this an example where \"I know it when I see it\" or do you have a rigorous definition.
\n\nFor example, I don't think the de-aging in The Irishman was Deepfakes, but maybe you do!
", "rawMarkdown": "Deepfake is a word used throughout the internet to mean a variety of things. It was originally used as a reddit name in reference to pornography. I think the term has evolved a great deal since then, and I think it would be beneficial to provide a more specific definition of this term.\n\nOr is the supposition that we are supposed to assume the definition is provided by the labels on the training dataset? Is this an example where \"I know it when I see it\" or do you have a rigorous definition.\n\nFor example, I don't think the de-aging in The Irishman was Deepfakes, but maybe you do!", "votes": null }, { "id": "693880", "postDate": "12/12/2019 22:05:29", "content": "This was clarified by the host here: In this dataset a deepfake could be either a face or voice swap. (or both).
Indeed I'm pretty sure I stumbled upon a sample with a weird unsynchronized voice over. Might be hard to detect.
", "rawMarkdown": "Indeed I'm pretty sure I stumbled upon a sample with a weird unsynchronized voice over. Might be hard to detect.", "votes": null }, { "id": "693917", "postDate": "12/13/2019 00:21:25", "content": "So if I were to record somebody and the voice was out of sync, is that considered a Deepfake? I would be surprised if vocals could be predicted just by looking at a person's face and body, I think variations in accent and language can have pretty large impacts on a person's speaking voice.
\n\nAnother example would be bad lip reading. Personally, this challenges my interpretation of the term. I also imagine impressionists or ventriloquists being examples of videos that would provide a \"Deepfake\" like experience that would be very difficult to classify. Lastly, a large percentage of films utilize a technique known as dubbing, where a scene is rerecorded by the actor and synced to their voice. An example would be \"Rebel Without a Cause\", where James Dean's voice was dubbed by Nick Adams after his untimely death. I realize it's a work of fiction, but a machine learning model would consider that to be video content where a voice is being used that is not the original actor, so for the purposes of our classifier would we consider this a Deepfake?
\n\nI think it's important to provide a very strong definition of what face or voice swap means. In my mind, face swap means completely changing the face and voice swap means completely changing the voice. What about modulation to a voice? What about airbrushing to an image? I would like to see the contest host provide some insight into the problem we are trying to solve and why, I think it would help me to better understand what supplemental datasets will help in building a robust model beyond the provided training data.
\n\nbad lip reading reference: https://www.youtube.com/watch?v=ZvSWIOtp3hs\ndubbing: https://en.wikipedia.org/wiki/Dubbing_(filmmaking)
", "rawMarkdown": "So if I were to record somebody and the voice was out of sync, is that considered a Deepfake? I would be surprised if vocals could be predicted just by looking at a person's face and body, I think variations in accent and language can have pretty large impacts on a person's speaking voice.\n\nAnother example would be bad lip reading. Personally, this challenges my interpretation of the term. I also imagine impressionists or ventriloquists being examples of videos that would provide a \"Deepfake\" like experience that would be very difficult to classify. Lastly, a large percentage of films utilize a technique known as dubbing, where a scene is rerecorded by the actor and synced to their voice. An example would be \"Rebel Without a Cause\", where James Dean's voice was dubbed by Nick Adams after his untimely death. I realize it's a work of fiction, but a machine learning model would consider that to be video content where a voice is being used that is not the original actor, so for the purposes of our classifier would we consider this a Deepfake?\n\nI think it's important to provide a very strong definition of what face or voice swap means. In my mind, face swap means completely changing the face and voice swap means completely changing the voice. What about modulation to a voice? What about airbrushing to an image? I would like to see the contest host provide some insight into the problem we are trying to solve and why, I think it would help me to better understand what supplemental datasets will help in building a robust model beyond the provided training data.\n\nbad lip reading reference: https://www.youtube.com/watch?v=ZvSWIOtp3hs\ndubbing: https://en.wikipedia.org/wiki/Dubbing_(filmmaking)", "votes": null }, { "id": "693950", "postDate": "12/13/2019 01:34:42", "content": "What about a somewhat blurred face, which is what some of those deepfakes look like?
", "rawMarkdown": "What about a somewhat blurred face, which is what some of those deepfakes look like?", "votes": null }, { "id": "694469", "postDate": "12/13/2019 16:54:30", "content": "that's another good point, if someone's face is blurred, like in a hidden camera tv show, does that count as a DeepFake?
", "rawMarkdown": "that's another good point, if someone's face is blurred, like in a hidden camera tv show, does that count as a DeepFake?", "votes": null }, { "id": "694511", "postDate": "12/13/2019 18:19:40", "content": "Rather than try to give a definitive answer for what a deepfake is, I will try to clarify what a real and fake video is within the context of this dataset and competition.
\n\nIn a fake video, at least one of the subjects in the video has their face or voice algorithmically replaced with the identity of a different person. Or the subject's face may be animated to appear to say something different than what they did in the source video.
\n\nIn a real video, there is no attempt to alter the identity or actions of the subjects in the video. A video altered in other ways could still be considered real.
\n\nSimply blurring a face, which disguises, but does not change the identity in the video, would still be considered a real video in this competition. It's also a good example of something that makes this problem hard to solve in the real world.
", "rawMarkdown": "Rather than try to give a definitive answer for what a deepfake is, I will try to clarify what a real and fake video is within the context of this dataset and competition. \n\nIn a fake video, at least one of the subjects in the video has their face or voice algorithmically replaced with the identity of a different person. Or the subject's face may be animated to appear to say something different than what they did in the source video. \n\nIn a real video, there is no attempt to alter the identity or actions of the subjects in the video. A video altered in other ways could still be considered real.\n\nSimply blurring a face, which disguises, but does not change the identity in the video, would still be considered a real video in this competition. It's also a good example of something that makes this problem hard to solve in the real world.", "votes": null }, { "id": "709040", "postDate": "01/03/2020 01:01:09", "content": "I see what you mean, but how do an AI will know what voice belongs to a person and not to another? you will have to know the identity of the person and have his/her voice finger prin to be able to see if belonging to that person or no.
", "rawMarkdown": "I see what you mean, but how do an AI will know what voice belongs to a person and not to another? you will have to know the identity of the person and have his/her voice finger prin to be able to see if belonging to that person or no.", "votes": null } ], "comments": [ { "id": 693880, "author_name": "juliaelliott", "author_url": "", "post_date": "12/12/2019 22:05:29", "content": "This was clarified by the host here: In this dataset a deepfake could be either a face or voice swap. (or both).
Indeed I'm pretty sure I stumbled upon a sample with a weird unsynchronized voice over. Might be hard to detect.
", "votes": null, "replies": [] }, { "id": 693917, "author_name": "rlkel0", "author_url": "", "post_date": "12/13/2019 00:21:25", "content": "So if I were to record somebody and the voice was out of sync, is that considered a Deepfake? I would be surprised if vocals could be predicted just by looking at a person's face and body, I think variations in accent and language can have pretty large impacts on a person's speaking voice.
\n\nAnother example would be bad lip reading. Personally, this challenges my interpretation of the term. I also imagine impressionists or ventriloquists being examples of videos that would provide a \"Deepfake\" like experience that would be very difficult to classify. Lastly, a large percentage of films utilize a technique known as dubbing, where a scene is rerecorded by the actor and synced to their voice. An example would be \"Rebel Without a Cause\", where James Dean's voice was dubbed by Nick Adams after his untimely death. I realize it's a work of fiction, but a machine learning model would consider that to be video content where a voice is being used that is not the original actor, so for the purposes of our classifier would we consider this a Deepfake?
\n\nI think it's important to provide a very strong definition of what face or voice swap means. In my mind, face swap means completely changing the face and voice swap means completely changing the voice. What about modulation to a voice? What about airbrushing to an image? I would like to see the contest host provide some insight into the problem we are trying to solve and why, I think it would help me to better understand what supplemental datasets will help in building a robust model beyond the provided training data.
\n\nbad lip reading reference: https://www.youtube.com/watch?v=ZvSWIOtp3hs\ndubbing: https://en.wikipedia.org/wiki/Dubbing_(filmmaking)
", "votes": null, "replies": [] }, { "id": 693950, "author_name": "sdoria", "author_url": "", "post_date": "12/13/2019 01:34:42", "content": "What about a somewhat blurred face, which is what some of those deepfakes look like?
", "votes": null, "replies": [] }, { "id": 694469, "author_name": "rlkel0", "author_url": "", "post_date": "12/13/2019 16:54:30", "content": "that's another good point, if someone's face is blurred, like in a hidden camera tv show, does that count as a DeepFake?
", "votes": null, "replies": [] }, { "id": 694511, "author_name": "benpflaum", "author_url": "", "post_date": "12/13/2019 18:19:40", "content": "Rather than try to give a definitive answer for what a deepfake is, I will try to clarify what a real and fake video is within the context of this dataset and competition.
\n\nIn a fake video, at least one of the subjects in the video has their face or voice algorithmically replaced with the identity of a different person. Or the subject's face may be animated to appear to say something different than what they did in the source video.
\n\nIn a real video, there is no attempt to alter the identity or actions of the subjects in the video. A video altered in other ways could still be considered real.
\n\nSimply blurring a face, which disguises, but does not change the identity in the video, would still be considered a real video in this competition. It's also a good example of something that makes this problem hard to solve in the real world.
", "votes": null, "replies": [] }, { "id": 709040, "author_name": "oscarrangel", "author_url": "", "post_date": "01/03/2020 01:01:09", "content": "I see what you mean, but how do an AI will know what voice belongs to a person and not to another? you will have to know the identity of the person and have his/her voice finger prin to be able to see if belonging to that person or no.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "693873": "Deepfake is a word used throughout the internet to mean a variety of things. It was originally used as a reddit name in reference to pornography. I think the term has evolved a great deal since then, and I think it would be beneficial to provide a more specific definition of this term.\n\nOr is the supposition that we are supposed to assume the definition is provided by the labels on the training dataset? Is this an example where \"I know it when I see it\" or do you have a rigorous definition.\n\nFor example, I don't think the de-aging in The Irishman was Deepfakes, but maybe you do!", "693880": "This was [clarified by the host here](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121184#692989): `In this dataset a deepfake could be either a face or voice swap. (or both).`", "693912": "Indeed I'm pretty sure I stumbled upon a sample with a weird unsynchronized voice over. Might be hard to detect.", "693917": "So if I were to record somebody and the voice was out of sync, is that considered a Deepfake? I would be surprised if vocals could be predicted just by looking at a person's face and body, I think variations in accent and language can have pretty large impacts on a person's speaking voice.\n\nAnother example would be bad lip reading. Personally, this challenges my interpretation of the term. I also imagine impressionists or ventriloquists being examples of videos that would provide a \"Deepfake\" like experience that would be very difficult to classify. Lastly, a large percentage of films utilize a technique known as dubbing, where a scene is rerecorded by the actor and synced to their voice. An example would be \"Rebel Without a Cause\", where James Dean's voice was dubbed by Nick Adams after his untimely death. I realize it's a work of fiction, but a machine learning model would consider that to be video content where a voice is being used that is not the original actor, so for the purposes of our classifier would we consider this a Deepfake?\n\nI think it's important to provide a very strong definition of what face or voice swap means. In my mind, face swap means completely changing the face and voice swap means completely changing the voice. What about modulation to a voice? What about airbrushing to an image? I would like to see the contest host provide some insight into the problem we are trying to solve and why, I think it would help me to better understand what supplemental datasets will help in building a robust model beyond the provided training data.\n\nbad lip reading reference: https://www.youtube.com/watch?v=ZvSWIOtp3hs\ndubbing: https://en.wikipedia.org/wiki/Dubbing_(filmmaking)", "693950": "What about a somewhat blurred face, which is what some of those deepfakes look like?", "694469": "that's another good point, if someone's face is blurred, like in a hidden camera tv show, does that count as a DeepFake?", "694511": "Rather than try to give a definitive answer for what a deepfake is, I will try to clarify what a real and fake video is within the context of this dataset and competition. \n\nIn a fake video, at least one of the subjects in the video has their face or voice algorithmically replaced with the identity of a different person. Or the subject's face may be animated to appear to say something different than what they did in the source video. \n\nIn a real video, there is no attempt to alter the identity or actions of the subjects in the video. A video altered in other ways could still be considered real.\n\nSimply blurring a face, which disguises, but does not change the identity in the video, would still be considered a real video in this competition. It's also a good example of something that makes this problem hard to solve in the real world.", "709040": "I see what you mean, but how do an AI will know what voice belongs to a person and not to another? you will have to know the identity of the person and have his/her voice finger prin to be able to see if belonging to that person or no." }, "source": "meta" }