I couldn't download the whole dataset (500 GB), so I have downloaded few parts of the dataset (particularly the 1st, last and some intermediate partition). I couldn't find the metadata.json file in them which should contain all the training labels. There is a metadata.json file in the train_sample_videos folder but it only contains labels for only approximately 400 videos, but there are should be more than 100k training videos. Can someone please help me out with this?
I couldn't download the whole dataset (500 GB), so I have downloaded few parts of the dataset (particularly the 1st, last and some intermediate partition). I couldn't find the metadata.json file in them which should contain all the training labels. There is a metadata.json file in the train_sample_videos folder but it only contains labels for only approximately 400 videos, but there are should be more than 100k training videos. Can someone please help me out with this?
Each part has a metadata.json file for the videos in that folder. I am not done downloading/unpacking yet, but I guess you just have to merge them all if what you want is a global one.
", "rawMarkdown": "Each part has a metadata.json file for the videos in that folder. I am not done downloading/unpacking yet, but I guess you just have to merge them all if what you want is a global one.", "votes": null }, { "id": "700110", "postDate": "12/21/2019 13:26:46", "content": "Did you get that file ?
", "rawMarkdown": "Did you get that file ?", "votes": null }, { "id": "705842", "postDate": "12/29/2019 15:13:28", "content": "Ok, I uploaded the combined metadata. Take a look at the dataset, and the accompanying kernel.
", "rawMarkdown": "Ok, I uploaded the combined metadata. Take a look at [the dataset](https://www.kaggle.com/zaharch/train-set-metadata-for-dfdc), and the [accompanying kernel](https://www.kaggle.com/zaharch/looking-at-the-full-train-set-metadata).", "votes": null }, { "id": "707213", "postDate": "12/31/2019 12:24:42", "content": "The metadata.json has all the files as given in the train_sample_videos folder, but the files which are tagged with original for the given files were missing in the dataset, I suppose that should be present in the complete dataset provided at - https://www.kaggle.com/c/deepfake-detection-challenge/data, and regarding the training data files provided I think are just expecting the predictions for just for the provided files(400 files) and not for everything else as from the sample_submission.csv file
How do you find the metadata.json file? We have downlaoded the entire dataset, and also downloaded the first folder separatley, yet cannot find \"metadata.json\"
", "rawMarkdown": "How do you find the metadata.json file? We have downlaoded the entire dataset, and also downloaded the first folder separatley, yet cannot find \"metadata.json\"", "votes": null } ], "comments": [ { "id": 694440, "author_name": "batangas", "author_url": "", "post_date": "12/13/2019 15:43:43", "content": "Each part has a metadata.json file for the videos in that folder. I am not done downloading/unpacking yet, but I guess you just have to merge them all if what you want is a global one.
", "votes": null, "replies": [] }, { "id": 700110, "author_name": "mrunalkshirsagar", "author_url": "", "post_date": "12/21/2019 13:26:46", "content": "Did you get that file ?
", "votes": null, "replies": [] }, { "id": 705842, "author_name": "zaharch", "author_url": "", "post_date": "12/29/2019 15:13:28", "content": "Ok, I uploaded the combined metadata. Take a look at the dataset, and the accompanying kernel.
", "votes": null, "replies": [] }, { "id": 707213, "author_name": "pidugusundeep", "author_url": "", "post_date": "12/31/2019 12:24:42", "content": "The metadata.json has all the files as given in the train_sample_videos folder, but the files which are tagged with original for the given files were missing in the dataset, I suppose that should be present in the complete dataset provided at - https://www.kaggle.com/c/deepfake-detection-challenge/data, and regarding the training data files provided I think are just expecting the predictions for just for the provided files(400 files) and not for everything else as from the sample_submission.csv file
How do you find the metadata.json file? We have downlaoded the entire dataset, and also downloaded the first folder separatley, yet cannot find \"metadata.json\"
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "694264": "I couldn't download the whole dataset (500 GB), so I have downloaded few parts of the dataset (particularly the 1st, last and some intermediate partition). I couldn't find the `metadata.json` file in them which should contain all the training labels. There is a `metadata.json` file in the `train_sample_videos` folder but it only contains labels for only approximately 400 videos, but there are should be more than 100k training videos. Can someone please help me out with this?", "694440": "Each part has a metadata.json file for the videos in that folder. I am not done downloading/unpacking yet, but I guess you just have to merge them all if what you want is a global one.", "700110": "Did you get that file ?", "705842": "Ok, I uploaded the combined metadata. Take a look at [the dataset](https://www.kaggle.com/zaharch/train-set-metadata-for-dfdc), and the [accompanying kernel](https://www.kaggle.com/zaharch/looking-at-the-full-train-set-metadata).", "707213": "The `metadata.json` has all the files as given in the `train_sample_videos` folder, but the files which are tagged with `original` for the given files were missing in the dataset, I suppose that should be present in the complete dataset provided at - [https://www.kaggle.com/c/deepfake-detection-challenge/data](https://www.kaggle.com/c/deepfake-detection-challenge/data), and regarding the training data files provided I think are just expecting the predictions for just for the provided files(400 files) and not for everything else as from the `sample_submission.csv` file", "715839": "How do you find the metadata.json file? We have downlaoded the entire dataset, and also downloaded the first folder separatley, yet cannot find \"metadata.json\"" }, "source": "meta" }