![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3319985%2Fd8ce84bed0e9d697476308f24ae354a7%2FScreen%20Shot%202020-03-10%20at%209.20.51%20PM.png?generation=1583900473994383&alt=media\")
Please share your best single model LB(not ensemble).\nTo start with ours: 0.38504\nEDIT: 0.35040\nEDIT2: 0.34
", "messages": [ { "id": 750030, "postDate": "2020-02-19T04:33:21.053Z", "content": "Please share your best single model LB(not ensemble).\nTo start with ours: 0.38504\nEDIT: 0.35040\nEDIT2: 0.34
", "rawMarkdown": "Please share your best single model LB(not ensemble).\nTo start with ours: 0.38504\nEDIT: 0.35040\nEDIT2: 0.34", "votes": 27 }, { "id": 751313, "postDate": "2020-02-20T05:11:48.030Z", "content": "Frame-by-frame model (only image): 0.33966
", "rawMarkdown": "Frame-by-frame model (only image): 0.33966 ", "votes": 9, "replies": [ { "id": 751314, "postDate": "2020-02-20T05:16:51.860Z", "content": "what do you mean by frame-by-frame model?
", "rawMarkdown": "what do you mean by frame-by-frame model?" }, { "id": 751352, "postDate": "2020-02-20T06:35:50.630Z", "content": "It means a model makes a prediction to each frame (and then aggregates over frames).
", "rawMarkdown": "It means a model makes a prediction to each frame (and then aggregates over frames).", "votes": 1 }, { "id": 751516, "postDate": "2020-02-20T08:49:28.887Z", "content": "@lyakaap Great score for a frame only model. How did you choose your validation set? And are you using any external data during training?
", "rawMarkdown": "@lyakaap Great score for a frame only model. How did you choose your validation set? And are you using any external data during training? " }, { "id": 751646, "postDate": "2020-02-20T11:14:06.047Z", "content": "Just selecting 20% of folder as validation set. Used no external data.
", "rawMarkdown": "Just selecting 20% of folder as validation set. Used no external data.", "votes": 1 }, { "id": 751652, "postDate": "2020-02-20T11:17:56.207Z", "content": "@lyakaap You mean 20% from each folder, was your validation able to track your leaderboard?
", "rawMarkdown": "@lyakaap You mean 20% from each folder, was your validation able to track your leaderboard?" }, { "id": 751964, "postDate": "2020-02-20T17:23:00.673Z", "content": "No, split by folder index. This strategy can track well.
", "rawMarkdown": "No, split by folder index. This strategy can track well." }, { "id": 752019, "postDate": "2020-02-20T17:50:02.343Z", "content": "what is folder index?
", "rawMarkdown": "what is folder index?" }, { "id": 752101, "postDate": "2020-02-20T18:39:46.010Z", "content": "sorry for confusing, what I wanted to say is I split val/train based on the index of input zip files (dfdc_train_part_XX).
", "rawMarkdown": "sorry for confusing, what I wanted to say is I split val/train based on the index of input zip files (dfdc_train_part_XX).", "votes": 1 }, { "id": 752150, "postDate": "2020-02-20T19:12:24.757Z", "content": "I am also doing the same but in particular last 10 dfdc_train_part_XX where XX is range(40,50), is our split same or you are randomly selecting those?
", "rawMarkdown": "I am also doing the same but in particular last 10 dfdc_train_part_XX where XX is range(40,50), is our split same or you are randomly selecting those? ", "votes": 2 }, { "id": 752474, "postDate": "2020-02-21T04:03:33.333Z", "content": "Linear division may works.
", "rawMarkdown": "Linear division may works." }, { "id": 752492, "postDate": "2020-02-21T04:33:18.960Z", "content": "Same as yours.
", "rawMarkdown": "Same as yours." }, { "id": 752507, "postDate": "2020-02-21T05:00:03.577Z", "content": "@lyakaap That is a very good score for a frame by frame model(our's best is just 0.4418). Do you consider teaming with us? A model stacking will be really helpful.
", "rawMarkdown": "@lyakaap That is a very good score for a frame by frame model(our's best is just 0.4418). Do you consider teaming with us? A model stacking will be really helpful.", "votes": 1 }, { "id": 752516, "postDate": "2020-02-21T05:24:26.253Z", "content": "Thank you for your proposal, but I want to merge with a team having a similar score to me with a single model.
", "rawMarkdown": "Thank you for your proposal, but I want to merge with a team having a similar score to me with a single model.", "votes": 1 }, { "id": 752523, "postDate": "2020-02-21T05:43:42.007Z", "content": "We are working on an approach. Will ask again after a better score...
", "rawMarkdown": "We are working on an approach. Will ask again after a better score...", "votes": 1 }, { "id": 753617, "postDate": "2020-02-22T13:26:38.787Z", "content": "how many images used for training when you mean 'frame by frame'?
", "rawMarkdown": "how many images used for training when you mean 'frame by frame'?" }, { "id": 753695, "postDate": "2020-02-22T15:06:34.210Z", "content": "We used somewhere between 1m-1.5m images @yangsaewon in 'frame by frame'
", "rawMarkdown": "We used somewhere between 1m-1.5m images @yangsaewon in 'frame by frame'", "votes": 3 }, { "id": 753723, "postDate": "2020-02-22T15:46:36.263Z", "content": "So are you guys over-sampled the Real frames?
", "rawMarkdown": "So are you guys over-sampled the Real frames?" }, { "id": 753724, "postDate": "2020-02-22T15:47:56.027Z", "content": "No., The trick we are using is pretty simple, I am not sharing just yet, I am pretty sure many people are using that, saving it for sometime later in the competition.
", "rawMarkdown": "No., The trick we are using is pretty simple, I am not sharing just yet, I am pretty sure many people are using that, saving it for sometime later in the competition." }, { "id": 758366, "postDate": "2020-02-27T17:22:32.420Z", "content": "@harshit are you using external data published ?
", "rawMarkdown": "@harshit are you using external data published ?" }, { "id": 758384, "postDate": "2020-02-27T17:34:00.880Z", "content": "Did you compile the data yourself, if so how long did it take?
", "rawMarkdown": "Did you compile the data yourself, if so how long did it take?" }, { "id": 763074, "postDate": "2020-03-04T05:34:27.270Z", "content": "@jaideepvalani As far as I understand, if you use the competition data then somehow preprocess it, you don't need to make public and declare.\n@greatgamedota Took us 1-2 days running 24 hours on 1080 TI.
", "rawMarkdown": "@jaideepvalani As far as I understand, if you use the competition data then somehow preprocess it, you don't need to make public and declare.\n@greatgamedota Took us 1-2 days running 24 hours on 1080 TI." }, { "id": 763237, "postDate": "2020-03-04T09:23:33.987Z", "content": "@lyakaap how do you deal with data imbalance？
", "rawMarkdown": "@lyakaap how do you deal with data imbalance？" } ] }, { "id": 750624, "postDate": "2020-02-19T14:50:07.253Z", "content": "0.32
", "rawMarkdown": "0.32", "votes": 7, "replies": [ { "id": 750821, "postDate": "2020-02-19T18:02:19.010Z", "content": "Amazing!, Any insights you can share, please?
", "rawMarkdown": "Amazing!, Any insights you can share, please?" }, { "id": 750897, "postDate": "2020-02-19T19:46:36.403Z", "content": "@chenshen03 Are you only using images, or also audio?
", "rawMarkdown": "@chenshen03 Are you only using images, or also audio?" }, { "id": 751065, "postDate": "2020-02-20T01:06:07.783Z", "content": "😃
", "rawMarkdown": "😃 " }, { "id": 751136, "postDate": "2020-02-20T02:42:04.557Z", "content": "image only
", "rawMarkdown": "image only", "votes": 3 }, { "id": 751310, "postDate": "2020-02-20T05:10:17.513Z", "content": "@chenshen03 Any insights?
", "rawMarkdown": "@chenshen03 Any insights?" }, { "id": 752477, "postDate": "2020-02-21T04:04:27.207Z", "content": "You can really dance!
", "rawMarkdown": "You can really dance!" }, { "id": 754113, "postDate": "2020-02-23T04:27:35.967Z", "content": "what's your face extractor?
", "rawMarkdown": "what's your face extractor?" }, { "id": 754382, "postDate": "2020-02-23T13:38:44.677Z", "content": "@chenbaoying blazeface
", "rawMarkdown": "@chenbaoying blazeface", "votes": 1 }, { "id": 754386, "postDate": "2020-02-23T13:43:59.480Z", "content": "@harshitsheoran There is a lot of noise in the train dataset. For example, some FAKE videos have multiple faces, but the faces may all be REAL. Removing these noises may improve performance.
", "rawMarkdown": "@harshitsheoran There is a lot of noise in the train dataset. For example, some FAKE videos have multiple faces, but the faces may all be REAL. Removing these noises may improve performance.", "votes": 8 }, { "id": 754400, "postDate": "2020-02-23T13:57:31.287Z", "content": "@chenshen03 so you are removing noises in both training set and test set?
", "rawMarkdown": "@chenshen03 so you are removing noises in both training set and test set?", "votes": 1 }, { "id": 757177, "postDate": "2020-02-26T13:55:32.450Z", "content": "@chenshen03 Are you using any external data?
", "rawMarkdown": "@chenshen03 Are you using any external data?", "votes": 1 }, { "id": 757243, "postDate": "2020-02-26T15:03:19.513Z", "content": "@dsfhe49854 No
", "rawMarkdown": "@dsfhe49854 No", "votes": 2 }, { "id": 759206, "postDate": "2020-02-28T18:12:16.847Z", "content": "@shen chen .. how do we detect those noises in the face ?
", "rawMarkdown": "@shen chen .. how do we detect those noises in the face ?" }, { "id": 759861, "postDate": "2020-02-29T14:54:43.967Z", "content": "Does your current score still use the single model?CNN or CNN+RNN
", "rawMarkdown": "Does your current score still use the single model?CNN or CNN+RNN" }, { "id": 759906, "postDate": "2020-02-29T15:59:35.060Z", "content": "@jaideepvalani the area of the face and the detection score
", "rawMarkdown": "@jaideepvalani the area of the face and the detection score\n\n", "votes": 2 }, { "id": 759908, "postDate": "2020-02-29T16:00:19.633Z", "content": "@chenbaoying CNN + sequential model
", "rawMarkdown": "@chenbaoying CNN + sequential model" }, { "id": 760042, "postDate": "2020-02-29T19:10:16.510Z", "content": "@chenshen03 What is your input size?
", "rawMarkdown": "@chenshen03 What is your input size?" }, { "id": 760190, "postDate": "2020-03-01T00:19:03.860Z", "content": "@unkownhihi 224*224
", "rawMarkdown": "@unkownhihi 224*224" }, { "id": 762753, "postDate": "2020-03-03T19:28:50.323Z", "content": "@chenshen03 \nplease help understand below queries...\n1) There are multiple fake videos for a given original video ,can we assume that all the fake videos should have same person or set of persons speaking as is in original video
\n\n2) Can we say all Fake videos will have fake audios also ?
\n\n3) is it possible that in pair of fake and original video both will have real faces but only different voices ? eg. in both the fakes i am the only person but just that in fake its some one elses' voice
", "rawMarkdown": "@chenshen03 \nplease help understand below queries...\n1) There are multiple fake videos for a given original video ,can we assume that all the fake videos should have same person or set of persons speaking as is in original video\n\n2) Can we say all Fake videos will have fake audios also ?\n\n3) is it possible that in pair of fake and original video both will have real faces but only different voices ? eg. in both the fakes i am the only person but just that in fake its some one elses' voice \n" }, { "id": 762755, "postDate": "2020-03-03T19:34:43.637Z", "content": "@jaideepvalani \n1. It is true.\n2. No\n3. It is possible.
", "rawMarkdown": "@jaideepvalani \n1. It is true.\n2. No\n3. It is possible. " }, { "id": 763024, "postDate": "2020-03-04T03:22:44.917Z", "content": "Thanks @harshitsheoran
\nSo 3rd case if there are good number then image based model could making an error in prediction ,any way to separate such videos ?\nAlso would u mind giving brief on the this model framework.. do we need only images.. here or videos ?
There is, if you train on it, but it does not worth the time at this stage.
", "rawMarkdown": "There is, if you train on it, but it does not worth the time at this stage." }, { "id": 763258, "postDate": "2020-03-04T09:54:16.913Z", "content": "@jaideepvalani So far, we haven't found any benefits from the audio model, but it's still worth trying.
", "rawMarkdown": "@jaideepvalani So far, we haven't found any benefits from the audio model, but it's still worth trying." }, { "id": 765758, "postDate": "2020-03-07T03:48:49.427Z", "content": "@chason\nm using detection score only if score is more than this add it in list,\nhow about the area how to find the relavant one based on this
", "rawMarkdown": "@chason\nm using detection score only if score is more than this add it in list,\nhow about the area how to find the relavant one based on this " }, { "id": 766128, "postDate": "2020-03-07T17:37:30.077Z", "content": "@chenshen03 What's your local validation score and how did you choose your validation set?
", "rawMarkdown": "@chenshen03 What's your local validation score and how did you choose your validation set?" } ] }, { "id": 750044, "postDate": "2020-02-19T04:52:36.813Z", "content": "“Need to think of a good model” :)
", "rawMarkdown": "“Need to think of a good model” :)", "votes": 3, "replies": [ { "id": 750047, "postDate": "2020-02-19T04:57:29.837Z", "content": "LOL I am trying to think of a better model too! There's a ton of good kernels and discussions of good models. I recommend you to look into some of them(will get you to top 10%).
", "rawMarkdown": "LOL I am trying to think of a better model too! There's a ton of good kernels and discussions of good models. I recommend you to look into some of them(will get you to top 10%).", "votes": 1 }, { "id": 750462, "postDate": "2020-02-19T12:03:59.650Z", "content": "@unkownhihi could u point me to one such please which u took as base.. thanks in advance
", "rawMarkdown": "@unkownhihi could u point me to one such please which u took as base.. thanks in advance " }, { "id": 750605, "postDate": "2020-02-19T14:41:16.980Z", "content": "@jaideepvalani ResNext/Xception is a good one to start with.
", "rawMarkdown": "@jaideepvalani ResNext/Xception is a good one to start with." }, { "id": 750660, "postDate": "2020-02-19T15:27:37.560Z", "content": "Yes, these models get me 0.372
", "rawMarkdown": "Yes, these models get me 0.372", "votes": 1 }, { "id": 750724, "postDate": "2020-02-19T16:29:31.373Z", "content": "I actually use resnext and get a very good score on local CV ,balanced loss bt still very poor score at LB. \nWhat mistake i i could be making .. \nI use MTCNN in pre process pipeline.
", "rawMarkdown": "I actually use resnext and get a very good score on local CV ,balanced loss bt still very poor score at LB. \nWhat mistake i i could be making .. \nI use MTCNN in pre process pipeline." }, { "id": 750822, "postDate": "2020-02-19T18:03:49.103Z", "content": "@jaideepvalani Take your time in finding a better face detector, If ResNext does not work the best, use it with lstm, you should be able to figure rest of it yourself. ;-)
", "rawMarkdown": "@jaideepvalani Take your time in finding a better face detector, If ResNext does not work the best, use it with lstm, you should be able to figure rest of it yourself. ;-)" }, { "id": 750842, "postDate": "2020-02-19T18:31:33.937Z", "content": "Thanku Harshit for your advice... I heard about LSTM used as decoder to encoder models. But havent seen its implementation if you can point out to some kernel or link.. would be useful :)
", "rawMarkdown": "Thanku Harshit for your advice... I heard about LSTM used as decoder to encoder models. But havent seen its implementation if you can point out to some kernel or link.. would be useful :) " }, { "id": 750875, "postDate": "2020-02-19T19:07:49.680Z", "content": "In my opinion, try the normal model to get started. If you did things right, it should get you to the top 10%. We will publish lrcn(the LSTM model @harshitsheoran was talking about) starter kernel soon(along with the better face extractor).
", "rawMarkdown": "In my opinion, try the normal model to get started. If you did things right, it should get you to the top 10%. We will publish lrcn(the LSTM model @harshitsheoran was talking about) starter kernel soon(along with the better face extractor). ", "votes": 5 }, { "id": 751565, "postDate": "2020-02-20T09:31:44.457Z", "content": "@unkownhihi great work. I'm sort of stuck at this point. Maybe your kernel will help me.
", "rawMarkdown": "@unkownhihi great work. I'm sort of stuck at this point. Maybe your kernel will help me." }, { "id": 752455, "postDate": "2020-02-21T03:15:37.507Z", "content": "I went past two bench Mark's 467 and 451 ... at 404 now ,will try interesting things now shift from rank 590 to 80 ...long way to go
", "rawMarkdown": "I went past two bench Mark's 467 and 451 ... at 404 now ,will try interesting things now shift from rank 590 to 80 ...long way to go ", "votes": 1 }, { "id": 752804, "postDate": "2020-02-21T12:30:24.603Z", "content": "Hi, I want to know did you get a score of 0.38 through lrcn?\nIs the lstm an effective approach?
", "rawMarkdown": "Hi, I want to know did you get a score of 0.38 through lrcn?\nIs the lstm an effective approach?" }, { "id": 753016, "postDate": "2020-02-21T16:05:54.487Z", "content": "@yzcwansui yes it was an effective approach for us.
", "rawMarkdown": "@yzcwansui yes it was an effective approach for us." }, { "id": 753032, "postDate": "2020-02-21T16:37:40.693Z", "content": "@yzcwansui although for some folks, single frame model is better. But the best score we can get for single frame model is just 0.44
", "rawMarkdown": "@yzcwansui although for some folks, single frame model is better. But the best score we can get for single frame model is just 0.44", "votes": 1 }, { "id": 768934, "postDate": "2020-03-11T11:14:57.350Z", "content": "@harshitsheoran
\nI try the resnet+lstm model, it performs bad in the test dataset while gets good score in the train dataset, I use dropout=0.9 to prevent overfiting but it doesn't work, \ndo u have any advice? appreciate it first.\nwe train/test the model using 5 frames per video. and the num of real and fake is balanced.
New Single Frame by frame model Score 0.34000 used 160k images.
", "rawMarkdown": "New Single Frame by frame model Score 0.34000 used 160k images.", "votes": 4, "replies": [ { "id": 763253, "postDate": "2020-03-04T09:46:37.970Z", "content": "Wow, how do you deal with data imbalance？and u are using RNN model ?
", "rawMarkdown": "Wow, how do you deal with data imbalance？and u are using RNN model ?" }, { "id": 763490, "postDate": "2020-03-04T14:35:53.107Z", "content": "amazing score, \nwhat's your input size?
", "rawMarkdown": "amazing score, \nwhat's your input size?" }, { "id": 763519, "postDate": "2020-03-04T15:11:31.457Z", "content": "input size 200x200, no rnn, but working on that, yes dealt with data imbalance, but doing the same thing in an LRCN has some drawbacks. Ensemble of 3 models between 0.34-0.35 (all single model), was 0.318, So, I guess, ensembling right now is much less effective than it was back then.
", "rawMarkdown": "input size 200x200, no rnn, but working on that, yes dealt with data imbalance, but doing the same thing in an LRCN has some drawbacks. Ensemble of 3 models between 0.34-0.35 (all single model), was 0.318, So, I guess, ensembling right now is much less effective than it was back then.", "votes": 4 }, { "id": 763641, "postDate": "2020-03-04T17:27:59.217Z", "content": "@harshitsheoran could u point me to any good public lrcn model available in git hub ?\nwant to try this stuff .. will be newest of its kind for me :)\nALso if you could brief me about the overall framework of this combo. \nDo we need videos to train this model or images sufficient ?
", "rawMarkdown": "@harshitsheoran could u point me to any good public lrcn model available in git hub ?\nwant to try this stuff .. will be newest of its kind for me :)\nALso if you could brief me about the overall framework of this combo. \nDo we need videos to train this model or images sufficient ?" }, { "id": 763667, "postDate": "2020-03-04T18:04:45.680Z", "content": "an LRCN, have a input in this format (batch_size, N_samples, size, size, n_channels), here N_samples can be your continuous or equally gapped number of images of a same video or different, that is up to you, I dont know how to do timedistributed layers in pytorch but in keras you can search timedistributed on google, to know more about it, after that you can wrap up your pretrained model in a timedistributed layer and then use that cnn model with a rnn.
", "rawMarkdown": "an LRCN, have a input in this format (batch_size, N_samples, size, size, n_channels), here N_samples can be your continuous or equally gapped number of images of a same video or different, that is up to you, I dont know how to do timedistributed layers in pytorch but in keras you can search timedistributed on google, to know more about it, after that you can wrap up your pretrained model in a timedistributed layer and then use that cnn model with a rnn.", "votes": 2 }, { "id": 763676, "postDate": "2020-03-04T18:16:50.173Z", "content": "It's actually fairly easy, you can switch from frames-by-video to a linear series of frames (for the CNN) and then back to videos, like this:
\n\n\n def forward(self, x):\n batch_size, n_channels, n_frames, height, width = x.size() # e.g. 32 x 3 x 64 x 224 x 224\n assert n_channels == 3, f\"Expecting 3 channels but got {n_channels}\"\n x = x.permute(0,2,1,3,4) # 32 x 64 x 3 x 224 x 224\n x = x.reshape(batch_size*n_frames, n_channels, height, width) # (32*64) x 3 x 224 x 224\n x = self.cnn_model(x) # (32*64) x 2048\n x = x.view(batch_size, n_frames, self.n_cnn_features)\n x, (h_n, h_c) = self.lstm(x) # x = seq_len, batch, num_directions*hidden_size\n x = x[:,-1] # Take last time step, which will be BATCH_SIZE * (2*HIDDEN) -> e.g. 12 * 128\n x = self.dropout(x)\n x = self.linear(x)\n return x\n
@harshitsheoran Dose \"Frame by frame model\" mean that you take each frame as input rather than the cropped face?
", "rawMarkdown": "@harshitsheoran Dose \"Frame by frame model\" mean that you take each frame as input rather than the cropped face?" }, { "id": 764563, "postDate": "2020-03-05T15:55:13.590Z", "content": "@jamesphoward \nThanku Dr. james \nIf i understand correctly this how it would be \n1 ) CNN first layer nn.Conv2d(3,n ....) \n2) Input size ( 32*64=2048,3,224,224)..i wonder how we can fit such high batch size ?\n3) CNN Output \n4) resize output to 32 ,64 frames * 2048 for LSTM
\n\n5) output of LSTM passed to linear layer\n6) sigmoid bce loss
", "rawMarkdown": "@jamesphoward \nThanku Dr. james \nIf i understand correctly this how it would be \n1 ) CNN first layer nn.Conv2d(3,n ....) \n2) Input size ( 32*64=2048,3,224,224)..i wonder how we can fit such high batch size ?\n3) CNN Output \n4) resize output to 32 ,64 frames * 2048 for LSTM \n\n5) output of LSTM passed to linear layer\n6) sigmoid bce loss\n" }, { "id": 764657, "postDate": "2020-03-05T17:46:29.493Z", "content": "Frame by frame have data input like this, (batch_size, face_size, face_size, n_channels), the face are cropped from the frame but they are not fed into an lstm or any transformer, it makes it a simple binary classfication problem, but here also we find that a simple LRCN performs much better than our frame-by-frame model, it looks like we are not even close to done improving. Good Luck!
", "rawMarkdown": "Frame by frame have data input like this, (batch_size, face_size, face_size, n_channels), the face are cropped from the frame but they are not fed into an lstm or any transformer, it makes it a simple binary classfication problem, but here also we find that a simple LRCN performs much better than our frame-by-frame model, it looks like we are not even close to done improving. Good Luck!", "votes": 2 }, { "id": 764902, "postDate": "2020-03-06T03:54:34.483Z", "content": "@jamesphoward I did this few weeks ago, but face memory problem. How did you fit 32x64x224x224x3 for each batch? It's quite a lot. Using accumulated gradient seems to be faulty as BatchNorm won't work if we fit 1x64x224x224x3 each time.
", "rawMarkdown": "@jamesphoward I did this few weeks ago, but face memory problem. How did you fit 32x64x224x224x3 for each batch? It's quite a lot. Using accumulated gradient seems to be faulty as BatchNorm won't work if we fit 1x64x224x224x3 each time." }, { "id": 764980, "postDate": "2020-03-06T06:43:50.570Z", "content": "Can you clarify what you mean by 160k images? @harshitsheoran Given that we have >100k videos does that mean youre only looking at a couple frames per video? Or are you using some method to choose to train on a subset of videos?
", "rawMarkdown": "Can you clarify what you mean by 160k images? @harshitsheoran Given that we have >100k videos does that mean youre only looking at a couple frames per video? Or are you using some method to choose to train on a subset of videos?" }, { "id": 764982, "postDate": "2020-03-06T06:49:03.763Z", "content": "@ryches good question, first of all I am only using first 40 folders, yes, that does mean, that on an average we do have only a few frames from each video but honestly they are selected so perfectly anything more would be nothing but an overkill and the chances were that it would 1. not letting the model any advantage 2. making the model starting overfitting faster than its normal epoch.
", "rawMarkdown": "@ryches good question, first of all I am only using first 40 folders, yes, that does mean, that on an average we do have only a few frames from each video but honestly they are selected so perfectly anything more would be nothing but an overkill and the chances were that it would 1. not letting the model any advantage 2. making the model starting overfitting faster than its normal epoch.", "votes": 1 }, { "id": 765129, "postDate": "2020-03-06T09:29:01.957Z", "content": "@khahuras It was just an example, what batch size of videos and how many frames per video you use depend on how complex the CNN is, how many parameters you use for the RNN, your GPU, and whether you freeze the CNN before embedding it. I advise the latter, at least for earlier stages of training.
", "rawMarkdown": "@khahuras It was just an example, what batch size of videos and how many frames per video you use depend on how complex the CNN is, how many parameters you use for the RNN, your GPU, and whether you freeze the CNN before embedding it. I advise the latter, at least for earlier stages of training.", "votes": 2 }, { "id": 765135, "postDate": "2020-03-06T09:34:20.720Z", "content": "@jamesphoward \nwhy did we take last time step in your sample code above ?
", "rawMarkdown": "@jamesphoward \nwhy did we take last time step in your sample code above ?" }, { "id": 765138, "postDate": "2020-03-06T09:39:36.473Z", "content": "Because of the way RNNs work in pytorch you get an output (prediction) after every time step (i.e. every frame of the video). However, you probably just want the RNN's prediction once it's seen the whole video, so I take the last time step. However, maybe that's not the best method, who knows?
", "rawMarkdown": "Because of the way RNNs work in pytorch you get an output (prediction) after every time step (i.e. every frame of the video). However, you probably just want the RNN's prediction once it's seen the whole video, so I take the last time step. However, maybe that's not the best method, who knows?" }, { "id": 765269, "postDate": "2020-03-06T12:41:42.873Z", "content": "What is the size of your self.linear ?\nsince output of lstm would be for eg. 12 * 128
", "rawMarkdown": "What is the size of your self.linear ?\nsince output of lstm would be for eg. 12 * 128 " }, { "id": 765298, "postDate": "2020-03-06T13:21:51.317Z", "content": "The input size would be the the number of hidden units, or double this if your LSTM/GRU was bidirectional.
", "rawMarkdown": "The input size would be the the number of hidden units, or double this if your LSTM/GRU was bidirectional." }, { "id": 765320, "postDate": "2020-03-06T13:42:15.960Z", "content": "Sorry to bother you more..\nThis is my input \n\nself.LSTM=nn.LSTM(1024,512,2) \nself.linear=nn.linear(512,1)\nx torch.Size([192, 1024])-> reshaped to 16,12,1024 for nn.LSTM\n\nif i do x=x[:,-1] my original batch size of 192 gets reduced to 12 .\nIs there an additional layer needed before feeding to Loss function as my output of linear wont meet requirement of loss function which is 192*1
Have you set batch_first=True in your LSTM if you are supplying the batch first?
Yup i did as first part.. \n```\nclass lstm(nn.Module):\n def init(self,bs,num, lstm1_input=1024,lstm_hidden=512):\n super().init()\n self.batch=bs #16\n self.num=num #12
\n\n self.lstm = nn.LSTM(1024, 512 ,2,batch_first=True)# input elements,hidden state size\n self.linear=nn.Linear(512,1)\n self.drop=nn.Dropout(0.5)\ndef forward(self,input):\n #print(input.size())\n x=input.view(self.batch,self.num,input.size(1)) # resized from 192 to 16,12\n x,y=self.lstm(x)\n print(x.size()) #see below\n x = x[:,-1,:]\n print(x.size()) #see below\n x=self.drop(x)\n x=self.linear(x)\n return x\n```
\n\ntorch.Size([16, 12, 512])\ntorch.Size([16, 512])
", "rawMarkdown": "Yup i did as first part.. \n```\nclass lstm(nn.Module):\n def __init__(self,bs,num, lstm1_input=1024,lstm_hidden=512):\n super().__init__()\n self.batch=bs #16\n self.num=num #12\n \n \n self.lstm = nn.LSTM(1024, 512 ,2,batch_first=True)# input elements,hidden state size\n self.linear=nn.Linear(512,1)\n self.drop=nn.Dropout(0.5)\n def forward(self,input):\n #print(input.size())\n x=input.view(self.batch,self.num,input.size(1)) # resized from 192 to 16,12\n x,y=self.lstm(x)\n print(x.size()) #see below\n x = x[:,-1,:]\n print(x.size()) #see below\n x=self.drop(x)\n x=self.linear(x)\n return x\n```\n \ntorch.Size([16, 12, 512])\ntorch.Size([16, 512])" }, { "id": 765378, "postDate": "2020-03-06T14:47:03.843Z", "content": "Isn't that correct? You have fed a batch of 16 into your network and your linear layer is receiving 16 sets of 512 hidden units' outputs? I don't see the problem...
", "rawMarkdown": "Isn't that correct? You have fed a batch of 16 into your network and your linear layer is receiving 16 sets of 512 hidden units' outputs? I don't see the problem..." }, { "id": 765386, "postDate": "2020-03-06T15:04:11.990Z", "content": "yes i dont have issue here but in the loss calculation.. \n data loader will give labels with bs =batch * num of frames * num of classes\nlooks like i should generate 12 steps using some Time Distribute layer like in keras and pack them. while keeping the original bs as 16 only for data loader...\nis this wha
", "rawMarkdown": "yes i dont have issue here but in the loss calculation.. \n data loader will give labels with bs =batch * num of frames * num of classes\nlooks like i should generate 12 steps using some Time Distribute layer like in keras and pack them. while keeping the original bs as 16 only for data loader...\nis this wha\n" }, { "id": 765402, "postDate": "2020-03-06T15:26:44.247Z", "content": "Hi, I think that's really a problem you are going to need to solve as this isn't really appropriate for the forums, and we're now getting on to fundamental issues of network training, sorry!
", "rawMarkdown": "Hi, I think that's really a problem you are going to need to solve as this isn't really appropriate for the forums, and we're now getting on to fundamental issues of network training, sorry!" }, { "id": 768364, "postDate": "2020-03-10T17:37:36.477Z", "content": "@harshitsheoran \nI overcame all the bs related issue after chosing right way of reshaping and transposing the dims. \nNow i face another issue which is overfitting. Model done best than ever models i tried on CV but miserably overfits on test data,especially fails to predict reals correctly. \nDid u faced this issue with LSTM.. if yes what can be f help. i purposefully dint pay attn on split strategy.
\n\n\nepoch train_loss valid_loss accuracy time\n0 0.450687 0.451665 0.834688 09:08\n1 0.375398 0.342386 0.859259 09:06\n2 0.261341 0.245293 0.896838 09:02\n3 0.199818 0.183108 0.928907 09:06 \n4 0.169864 0.163689 0.938302 09:07\n5 0.140049 0.210106 0.915447 09:10\n6 0.126233 0.136548 0.947064 09:11\n
@jaideepvalani well, that loss is kinda close to ours, with accuracy also close to ours every epoch, (our 1 epoch takes 3 hours so idk how you are overfitting and I am not, I also have a much much much larger dataset which I train on compared to yours) I never had overfitting issues and please if you are using keras dont focus on valid_loss it lies, I guess you should have a benchmark, of last 10 folders, after every epoch check your model on that, and see which epoch is starts overfitting, in our case
\n\nmodel__e1 -> 0.212 / 0.934\nmodel_e2 -> 0.183 / 0.946\nmodel_e3 -> 0.175 / 0.952\nmodel_e4 -> 0.186 / 0.950
\n\neven something like this happens where you think overfitting is starting, still train it might get a better improvement in next 3 epochs.
\n\ntry dropout, and batch norm (however I dont use any of them) maybe to overcome overfitting, also no noise function in the augmentations, check the labels and your generator again manually. We had some problem and that was cuz our generator was messed up once.
", "rawMarkdown": "@jaideepvalani well, that loss is kinda close to ours, with accuracy also close to ours every epoch, (our 1 epoch takes 3 hours so idk how you are overfitting and I am not, I also have a much much much larger dataset which I train on compared to yours) I never had overfitting issues and please if you are using keras dont focus on valid_loss it lies, I guess you should have a benchmark, of last 10 folders, after every epoch check your model on that, and see which epoch is starts overfitting, in our case\n\nmodel__e1 -> 0.212 / 0.934\nmodel_e2 -> 0.183 / 0.946\nmodel_e3 -> 0.175 / 0.952\nmodel_e4 -> 0.186 / 0.950\n\neven something like this happens where you think overfitting is starting, still train it might get a better improvement in next 3 epochs.\n\ntry dropout, and batch norm (however I dont use any of them) maybe to overcome overfitting, also no noise function in the augmentations, check the labels and your generator again manually. We had some problem and that was cuz our generator was messed up once.", "votes": 3 }, { "id": 768526, "postDate": "2020-03-10T23:32:20.117Z", "content": "@harshitsheoran what is your bce val score each epoch?
", "rawMarkdown": "@harshitsheoran what is your bce val score each epoch?" }, { "id": 768565, "postDate": "2020-03-11T01:16:58.600Z", "content": "@khahuras that is the validation loss I shared and not the training loss
\n\n0.160 on that bench means 0.318 on leaderboard, always almost double.
", "rawMarkdown": "@khahuras that is the validation loss I shared and not the training loss\n\n0.160 on that bench means 0.318 on leaderboard, always almost double.", "votes": 2 }, { "id": 768642, "postDate": "2020-03-11T03:43:04.843Z", "content": "1) how does your hist gram of 400 sub videos look like assuming fake =0.99 and real =0.... \nleft and right height bars ,which one more ?\n2) bi direction LSTM or uni ?\n3) what are you lstm dimensions.. bs,num_frames ,no of elements does keeping higher bs helps more ?\n4) how is your batch distribution real:fake ratio in batch
", "rawMarkdown": "1) how does your hist gram of 400 sub videos look like assuming fake =0.99 and real =0.... \nleft and right height bars ,which one more ?\n2) bi direction LSTM or uni ?\n3) what are you lstm dimensions.. bs,num_frames ,no of elements does keeping higher bs helps more ?\n4) how is your batch distribution real:fake ratio in batch\n\n" }, { "id": 768661, "postDate": "2020-03-11T04:24:13.177Z", "content": "@jaideepvalani here's my response(I'm basically in charge of model training right now):\n1. We used a post-processing technique. Here's the histogram
thanku.. during inference how many video frames used...\nis it necessary to keep same size as in training ?
", "rawMarkdown": "thanku.. during inference how many video frames used...\nis it necessary to keep same size as in training ?" }, { "id": 769132, "postDate": "2020-03-11T15:02:04.417Z", "content": "The answer might surprise you. We only used not a constant amount of frame per video.... I don't get the second question, sry.
", "rawMarkdown": "The answer might surprise you. We only used not a constant amount of frame per video.... I don't get the second question, sry." }, { "id": 769193, "postDate": "2020-03-11T16:15:01.213Z", "content": "i meant the batch size and num of frames during inference should they be same as in training for lstm to give expected output as in training
", "rawMarkdown": "i meant the batch size and num of frames during inference should they be same as in training for lstm to give expected output as in training" }, { "id": 769201, "postDate": "2020-03-11T16:23:32.310Z", "content": "ok. I got you now. the num of frames during inference is 10(they don't really affect much). (secret hint: we don't pass that 10 frames to lrcn at once, I can't really go into details.). for batch size during inference, its 1.
", "rawMarkdown": "ok. I got you now. the num of frames during inference is 10(they don't really affect much). (secret hint: we don't pass that 10 frames to lrcn at once, I can't really go into details.). for batch size during inference, its 1.", "votes": 1 }, { "id": 770631, "postDate": "2020-03-13T07:14:12.937Z", "content": "Thanks.. is public score calculated based on the 400 test videos or it could be some other videos which are subset of private one.
", "rawMarkdown": "Thanks.. is public score calculated based on the 400 test videos or it could be some other videos which are subset of private one." }, { "id": 770917, "postDate": "2020-03-13T15:09:45.687Z", "content": "@jaideepvalani I think the public score is calculated based on the 4000 videos that will be tested in a black box environment.
", "rawMarkdown": "@jaideepvalani I think the public score is calculated based on the 4000 videos that will be tested in a black box environment." } ] }, { "id": 756584, "postDate": "2020-02-25T21:33:50.670Z", "content": "My latest model scores: 0.3422
", "rawMarkdown": "My latest model scores: 0.3422", "votes": 4, "replies": [ { "id": 758365, "postDate": "2020-02-27T17:20:11.153Z", "content": "@ngcferreira amazing score\nwhich model did u use ?\nHave u used ur own dataset,if yes which face detector and no of frames used ?
", "rawMarkdown": "@ngcferreira amazing score\nwhich model did u use ?\nHave u used ur own dataset,if yes which face detector and no of frames used ?" }, { "id": 758416, "postDate": "2020-02-27T18:19:41.723Z", "content": "I've use a CNN based model, using only one frame.\nI've used no external data and used yolov3 for detecting faces and create the training set and blazeface for inference. \nThe prediction is just an average of 40 frames.
", "rawMarkdown": "I've use a CNN based model, using only one frame.\nI've used no external data and used yolov3 for detecting faces and create the training set and blazeface for inference. \nThe prediction is just an average of 40 frames.", "votes": 4 }, { "id": 758419, "postDate": "2020-02-27T18:22:00.390Z", "content": "what is your input face size?
", "rawMarkdown": "what is your input face size?" }, { "id": 759069, "postDate": "2020-02-28T14:27:16.680Z", "content": "@unkownhihi I'm using face images scaled to 224x224
", "rawMarkdown": "@unkownhihi I'm using face images scaled to 224x224", "votes": 1 }, { "id": 763234, "postDate": "2020-03-04T09:21:03.777Z", "content": "@ngcferreira how do you deal with data imbalance？
", "rawMarkdown": "@ngcferreira how do you deal with data imbalance？" }, { "id": 766231, "postDate": "2020-03-07T22:00:21.667Z", "content": "@xmaocai So first I split the data into 3 sets (train, val and test set). In every epoch, I randomly choose the same number of fake images as the number of real images. Basically I'm under sampling the data.
", "rawMarkdown": "@xmaocai So first I split the data into 3 sets (train, val and test set). In every epoch, I randomly choose the same number of fake images as the number of real images. Basically I'm under sampling the data.", "votes": 1 } ] }, { "id": 765464, "postDate": "2020-03-06T16:48:22.063Z", "content": "CNN+LSTM ,You can refer to https://github.com/HHTseng/video-classification
", "rawMarkdown": "CNN+LSTM ,You can refer to https://github.com/HHTseng/video-classification", "votes": 1, "replies": [ { "id": 768635, "postDate": "2020-03-11T03:32:03.570Z", "content": "Hello, thanks for u advice. I try it but the model doesn't perform well in the val dataset while gets good acc in the train dataset. I use dropout=0.9 to prevent overfiting but it doesn't work, \ndo u have any advice? \nbtw, 5 frames per video.
", "rawMarkdown": "Hello, thanks for u advice. I try it but the model doesn't perform well in the val dataset while gets good acc in the train dataset. I use dropout=0.9 to prevent overfiting but it doesn't work, \ndo u have any advice? \nbtw, 5 frames per video." }, { "id": 769109, "postDate": "2020-03-11T14:38:15.880Z", "content": "@xmacai what is the dim of LSTM and input X to CNN.. \nmay u could be mistaking at any place
", "rawMarkdown": "@xmacai what is the dim of LSTM and input X to CNN.. \nmay u could be mistaking at any place" }, { "id": 770798, "postDate": "2020-03-13T12:27:33.170Z", "content": "@jaideepvalani \ninput X dim: torch.Size([64, 1, 3, 224, 224])\nthe dim of input to LSTM: torch.Size([8, 1, 512])\nLSTM dim: \nself.LSTM =nn.LSTM(\n input_size=300,\n hidden_size=256,\n num_layers=3,\n batch_first=True,)\nHere are the input data handle:\nfor index in data:\n image = Image.open(index).convert('RGB')\n if self.data_transforms is not None:\n try:\n image = self.data_transforms(image)\n except:\n print(\"Cannot transform image: {}\".format(index))\n X.append(image)\nX = torch.stack(X, dim=0)\nY = torch.LongTensor([label])
current best score is 0.40621 after trying for weeks :)
", "rawMarkdown": "current best score is 0.40621 after trying for weeks :)", "votes": 1 }, { "id": 753355, "postDate": "2020-02-22T04:17:41.253Z", "content": "Well, I have my train split as 0-29 and 40 to 49 and val split as 30-39, the same as @jamesphoward . Currently, I'm getting a validation score of 0.295 for \"Image Only\" model. But I have 0 trust in it! Or maybe I have a bug in my inference pipeline. Who knows!\nAlso, the fact that the model starts overfitting from 3rd epoch onwards gives me the chills.
", "rawMarkdown": "Well, I have my train split as 0-29 and 40 to 49 and val split as 30-39, the same as @jamesphoward . Currently, I'm getting a validation score of 0.295 for \"Image Only\" model. But I have 0 trust in it! Or maybe I have a bug in my inference pipeline. Who knows!\nAlso, the fact that the model starts overfitting from 3rd epoch onwards gives me the chills.", "votes": 1 }, { "id": 750291, "postDate": "2020-02-19T09:07:17.483Z", "content": "My best model (images only) scores: 0.40457
", "rawMarkdown": "My best model (images only) scores: 0.40457\n", "votes": 1 }, { "id": 750589, "postDate": "2020-02-19T14:34:03.210Z", "content": "0.489 :P
\n\nUpdate 3: .402\nUpdate 4: .392
", "rawMarkdown": "0.489 :P\n\nUpdate 3: .402\nUpdate 4: .392", "votes": 2, "replies": [ { "id": 765336, "postDate": "2020-03-06T14:07:27.887Z", "content": "0.43* All I did was make a different inference kernal (different face detector) and pick a specific model trained on the data from my baseline kernal (one I trained over 2 weeks ago).
", "rawMarkdown": "0.43* All I did was make a different inference kernal (different face detector) and pick a specific model trained on the data from my baseline kernal (one I trained over 2 weeks ago).", "votes": 1 } ] }, { "id": 785367, "postDate": "2020-03-25T02:14:25.803Z", "content": "what kind of scheduler do you guys use for training? fixed lr or cosine scheduling, etc..
", "rawMarkdown": "what kind of scheduler do you guys use for training? fixed lr or cosine scheduling, etc.." }, { "id": 769950, "postDate": "2020-03-12T12:39:41.487Z", "content": "0.44202.
", "rawMarkdown": "0.44202.", "replies": [ { "id": 776564, "postDate": "2020-03-17T13:28:24.857Z", "content": "0.42352 .. Just near :)
", "rawMarkdown": "0.42352 .. Just near :) ", "votes": 1 } ] }, { "id": 755733, "postDate": "2020-02-25T04:23:27.997Z", "content": "@harshitsheoran hi，when using lrcn，how many frames of one video do you use？are they sequential frames or not。 appreciate for your advises
", "rawMarkdown": "@harshitsheoran hi，when using lrcn，how many frames of one video do you use？are they sequential frames or not。 appreciate for your advises", "replies": [ { "id": 757654, "postDate": "2020-02-27T01:58:20.820Z", "content": "10 frames per video, you can increase or decrease does not really matter, they are not sequential, they are equally gapped.
", "rawMarkdown": "10 frames per video, you can increase or decrease does not really matter, they are not sequential, they are equally gapped.", "votes": 2 } ] }, { "id": 752910, "postDate": "2020-02-21T14:37:16.007Z", "content": "0.41136 with a single CNN model. I wonder, how are you dealing with inference on multi-faced videos? I mean, one face is usually okay in each frame, another is distorted, rarely both of them. Should I pick the max prediction per frame or something? \nupd: 0.40463\nupd: 0.40128
", "rawMarkdown": "0.41136 with a single CNN model. I wonder, how are you dealing with inference on multi-faced videos? I mean, one face is usually okay in each frame, another is distorted, rarely both of them. Should I pick the max prediction per frame or something? \nupd: 0.40463\nupd: 0.40128", "replies": [ { "id": 752923, "postDate": "2020-02-21T14:46:02.813Z", "content": "Excellent question. I've experimented with this and I've found the mean to be very slightly better than either the max or min.
\n\nI used to think the maximum would be best, but that assumes you don't have false positives for face detection; false positives are likely to be unnatural and therefore fakes, and so it might be the mean is a little more resilient to this.
\n\nI would highlight, though, that this is very likely to depend on your processing pipeline.
", "rawMarkdown": "Excellent question. I've experimented with this and I've found the mean to be very slightly better than either the max or min.\n\nI used to think the maximum would be best, but that assumes you don't have false positives for face detection; false positives are likely to be unnatural and therefore fakes, and so it might be the mean is a little more resilient to this.\n\nI would highlight, though, that this is very likely to depend on your processing pipeline.", "votes": 8 }, { "id": 752964, "postDate": "2020-02-21T15:25:10.303Z", "content": "Distorted face found during detection are likely to be predicted as Fake. So if there is frame with distorted in list of frames then that would most likely contribute to false positive and hence with max prob compared to all.
", "rawMarkdown": "Distorted face found during detection are likely to be predicted as Fake. So if there is frame with distorted in list of frames then that would most likely contribute to false positive and hence with max prob compared to all. ", "votes": 1 }, { "id": 753088, "postDate": "2020-02-21T17:39:24.780Z", "content": "Thanks for the insight. I guess I'll play with inference detector a little bit more.
", "rawMarkdown": "Thanks for the insight. I guess I'll play with inference detector a little bit more. " }, { "id": 754121, "postDate": "2020-02-23T04:41:19.190Z", "content": "what's your face extractor? I use MTCNN but it can't get a good LB
", "rawMarkdown": "what's your face extractor? I use MTCNN but it can't get a good LB" }, { "id": 758870, "postDate": "2020-02-28T09:22:12.697Z", "content": "I've played with blazeface, yolov2, yolov3, mtcnn, and yolov3 got me the best score of 0.40463 for a single model
", "rawMarkdown": "I've played with blazeface, yolov2, yolov3, mtcnn, and yolov3 got me the best score of 0.40463 for a single model", "votes": 1 }, { "id": 761607, "postDate": "2020-03-02T18:32:21.367Z", "content": "@defileroff did you find that there was much difference between the face extractors in terms of overall score? I haven't played around with this yet, wondering what kind of priority it should be relative to everything else :)
", "rawMarkdown": "@defileroff did you find that there was much difference between the face extractors in terms of overall score? I haven't played around with this yet, wondering what kind of priority it should be relative to everything else :)" }, { "id": 764394, "postDate": "2020-03-05T12:19:56.800Z", "content": "@fergusoci Playing with different frames, faces, extractors for train/valid sets gave me a boost of 0.05, playing with different extractors at inference time gave me a boost of 0.01
\n\nI guess if your classifier is weak in terms of false-positive numbers, you'd need a better detector to avoid false-positive face detections. But if your classifier is good enough, you can take a less picky detector, that will detect everything that looks like a face, lol, and let the classifier deal with it.
", "rawMarkdown": "@fergusoci Playing with different frames, faces, extractors for train/valid sets gave me a boost of 0.05, playing with different extractors at inference time gave me a boost of 0.01 \n\nI guess if your classifier is weak in terms of false-positive numbers, you'd need a better detector to avoid false-positive face detections. But if your classifier is good enough, you can take a less picky detector, that will detect everything that looks like a face, lol, and let the classifier deal with it.", "votes": 2 }, { "id": 774959, "postDate": "2020-03-16T05:33:29.230Z", "content": "@defileroff pre-trained yolov3 face detector or trained with dfdc dataset?
", "rawMarkdown": "@defileroff pre-trained yolov3 face detector or trained with dfdc dataset? " }, { "id": 790715, "postDate": "2020-03-29T19:53:32.240Z", "content": "Pretrained one, I don't even use multi-frames like others do, works fine to some extent.
", "rawMarkdown": "Pretrained one, I don't even use multi-frames like others do, works fine to some extent." } ] }, { "id": 752227, "postDate": "2020-02-20T19:52:05.620Z", "content": "good
", "rawMarkdown": "good" }, { "id": 765463, "postDate": "2020-03-06T16:46:12.143Z", "rawMarkdown": "", "isDeleted": true }, { "id": 765277, "postDate": "2020-03-06T12:56:25.547Z", "rawMarkdown": "", "isDeleted": true }, { "id": 759856, "postDate": "2020-02-29T14:52:08.033Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 759946, "postDate": "2020-02-29T16:39:44.357Z", "content": "We are using a ensemble.
", "rawMarkdown": "We are using a ensemble." }, { "id": 763356, "postDate": "2020-03-04T11:57:23.233Z", "content": "thanks, I ensamble two model also get a better score
", "rawMarkdown": "thanks, I ensamble two model also get a better score" } ] }, { "id": 757172, "postDate": "2020-02-26T13:52:53.370Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 751313, "author_name": "lyakaap", "author_url": "", "post_date": "2020-02-20T05:11:48.030000", "content": "Frame-by-frame model (only image): 0.33966
", "votes": 9, "replies": [ { "id": 751314, "author_name": "Bibek", "author_url": "", "post_date": "2020-02-20T05:16:51.860000", "content": "what do you mean by frame-by-frame model?
", "votes": 0, "replies": [] }, { "id": 751352, "author_name": "lyakaap", "author_url": "", "post_date": "2020-02-20T06:35:50.630000", "content": "It means a model makes a prediction to each frame (and then aggregates over frames).
", "votes": 1, "replies": [] }, { "id": 751516, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-20T08:49:28.887000", "content": "@lyakaap Great score for a frame only model. How did you choose your validation set? And are you using any external data during training?
", "votes": 0, "replies": [] }, { "id": 751646, "author_name": "lyakaap", "author_url": "", "post_date": "2020-02-20T11:14:06.047000", "content": "Just selecting 20% of folder as validation set. Used no external data.
", "votes": 1, "replies": [] }, { "id": 751652, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-20T11:17:56.207000", "content": "@lyakaap You mean 20% from each folder, was your validation able to track your leaderboard?
", "votes": 0, "replies": [] }, { "id": 751964, "author_name": "lyakaap", "author_url": "", "post_date": "2020-02-20T17:23:00.673000", "content": "No, split by folder index. This strategy can track well.
", "votes": 0, "replies": [] }, { "id": 752019, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-20T17:50:02.343000", "content": "what is folder index?
", "votes": 0, "replies": [] }, { "id": 752101, "author_name": "lyakaap", "author_url": "", "post_date": "2020-02-20T18:39:46.010000", "content": "sorry for confusing, what I wanted to say is I split val/train based on the index of input zip files (dfdc_train_part_XX).
", "votes": 1, "replies": [] }, { "id": 752150, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-20T19:12:24.757000", "content": "I am also doing the same but in particular last 10 dfdc_train_part_XX where XX is range(40,50), is our split same or you are randomly selecting those?
", "votes": 2, "replies": [] }, { "id": 752474, "author_name": "Daic", "author_url": "", "post_date": "2020-02-21T04:03:33.333000", "content": "Linear division may works.
", "votes": 0, "replies": [] }, { "id": 752492, "author_name": "lyakaap", "author_url": "", "post_date": "2020-02-21T04:33:18.960000", "content": "Same as yours.
", "votes": 0, "replies": [] }, { "id": 752507, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-21T05:00:03.577000", "content": "@lyakaap That is a very good score for a frame by frame model(our's best is just 0.4418). Do you consider teaming with us? A model stacking will be really helpful.
", "votes": 1, "replies": [] }, { "id": 752516, "author_name": "lyakaap", "author_url": "", "post_date": "2020-02-21T05:24:26.253000", "content": "Thank you for your proposal, but I want to merge with a team having a similar score to me with a single model.
", "votes": 1, "replies": [] }, { "id": 752523, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-21T05:43:42.007000", "content": "We are working on an approach. Will ask again after a better score...
", "votes": 1, "replies": [] }, { "id": 753617, "author_name": "saewonYang", "author_url": "", "post_date": "2020-02-22T13:26:38.787000", "content": "how many images used for training when you mean 'frame by frame'?
", "votes": 0, "replies": [] }, { "id": 753695, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-22T15:06:34.210000", "content": "We used somewhere between 1m-1.5m images @yangsaewon in 'frame by frame'
", "votes": 3, "replies": [] }, { "id": 753723, "author_name": "Akash", "author_url": "", "post_date": "2020-02-22T15:46:36.263000", "content": "So are you guys over-sampled the Real frames?
", "votes": 0, "replies": [] }, { "id": 753724, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-22T15:47:56.027000", "content": "No., The trick we are using is pretty simple, I am not sharing just yet, I am pretty sure many people are using that, saving it for sometime later in the competition.
", "votes": 0, "replies": [] }, { "id": 758366, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-27T17:22:32.420000", "content": "@harshit are you using external data published ?
", "votes": 0, "replies": [] }, { "id": 758384, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-02-27T17:34:00.880000", "content": "Did you compile the data yourself, if so how long did it take?
", "votes": 0, "replies": [] }, { "id": 763074, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-03-04T05:34:27.270000", "content": "@jaideepvalani As far as I understand, if you use the competition data then somehow preprocess it, you don't need to make public and declare.\n@greatgamedota Took us 1-2 days running 24 hours on 1080 TI.
", "votes": 0, "replies": [] }, { "id": 763237, "author_name": "xmaocai", "author_url": "", "post_date": "2020-03-04T09:23:33.987000", "content": "@lyakaap how do you deal with data imbalance？
", "votes": 0, "replies": [] } ] }, { "id": 750624, "author_name": "Chason", "author_url": "", "post_date": "2020-02-19T14:50:07.253000", "content": "0.32
", "votes": 7, "replies": [ { "id": 750821, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-19T18:02:19.010000", "content": "Amazing!, Any insights you can share, please?
", "votes": 0, "replies": [] }, { "id": 750897, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-19T19:46:36.403000", "content": "@chenshen03 Are you only using images, or also audio?
", "votes": 0, "replies": [] }, { "id": 751065, "author_name": "xinshuaidong", "author_url": "", "post_date": "2020-02-20T01:06:07.783000", "content": "😃
", "votes": 0, "replies": [] }, { "id": 751136, "author_name": "Chason", "author_url": "", "post_date": "2020-02-20T02:42:04.557000", "content": "image only
", "votes": 3, "replies": [] }, { "id": 751310, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-20T05:10:17.513000", "content": "@chenshen03 Any insights?
", "votes": 0, "replies": [] }, { "id": 752477, "author_name": "Daic", "author_url": "", "post_date": "2020-02-21T04:04:27.207000", "content": "You can really dance!
", "votes": 0, "replies": [] }, { "id": 754113, "author_name": "BokingChen", "author_url": "", "post_date": "2020-02-23T04:27:35.967000", "content": "what's your face extractor?
", "votes": 0, "replies": [] }, { "id": 754382, "author_name": "Chason", "author_url": "", "post_date": "2020-02-23T13:38:44.677000", "content": "@chenbaoying blazeface
", "votes": 1, "replies": [] }, { "id": 754386, "author_name": "Chason", "author_url": "", "post_date": "2020-02-23T13:43:59.480000", "content": "@harshitsheoran There is a lot of noise in the train dataset. For example, some FAKE videos have multiple faces, but the faces may all be REAL. Removing these noises may improve performance.
", "votes": 8, "replies": [] }, { "id": 754400, "author_name": "saewonYang", "author_url": "", "post_date": "2020-02-23T13:57:31.287000", "content": "@chenshen03 so you are removing noises in both training set and test set?
", "votes": 1, "replies": [] }, { "id": 757177, "author_name": "RZ", "author_url": "", "post_date": "2020-02-26T13:55:32.450000", "content": "@chenshen03 Are you using any external data?
", "votes": 1, "replies": [] }, { "id": 757243, "author_name": "Chason", "author_url": "", "post_date": "2020-02-26T15:03:19.513000", "content": "@dsfhe49854 No
", "votes": 2, "replies": [] }, { "id": 759206, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-28T18:12:16.847000", "content": "@shen chen .. how do we detect those noises in the face ?
", "votes": 0, "replies": [] }, { "id": 759861, "author_name": "BokingChen", "author_url": "", "post_date": "2020-02-29T14:54:43.967000", "content": "Does your current score still use the single model?CNN or CNN+RNN
", "votes": 0, "replies": [] }, { "id": 759906, "author_name": "Chason", "author_url": "", "post_date": "2020-02-29T15:59:35.060000", "content": "@jaideepvalani the area of the face and the detection score
", "votes": 2, "replies": [] }, { "id": 759908, "author_name": "Chason", "author_url": "", "post_date": "2020-02-29T16:00:19.633000", "content": "@chenbaoying CNN + sequential model
", "votes": 0, "replies": [] }, { "id": 760042, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-29T19:10:16.510000", "content": "@chenshen03 What is your input size?
", "votes": 0, "replies": [] }, { "id": 760190, "author_name": "Chason", "author_url": "", "post_date": "2020-03-01T00:19:03.860000", "content": "@unkownhihi 224*224
", "votes": 0, "replies": [] }, { "id": 762753, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-03T19:28:50.323000", "content": "@chenshen03 \nplease help understand below queries...\n1) There are multiple fake videos for a given original video ,can we assume that all the fake videos should have same person or set of persons speaking as is in original video
\n\n2) Can we say all Fake videos will have fake audios also ?
\n\n3) is it possible that in pair of fake and original video both will have real faces but only different voices ? eg. in both the fakes i am the only person but just that in fake its some one elses' voice
", "votes": 0, "replies": [] }, { "id": 762755, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-03-03T19:34:43.637000", "content": "@jaideepvalani \n1. It is true.\n2. No\n3. It is possible.
", "votes": 0, "replies": [] }, { "id": 763024, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-04T03:22:44.917000", "content": "Thanks @harshitsheoran
\nSo 3rd case if there are good number then image based model could making an error in prediction ,any way to separate such videos ?\nAlso would u mind giving brief on the this model framework.. do we need only images.. here or videos ?
There is, if you train on it, but it does not worth the time at this stage.
", "votes": 0, "replies": [] }, { "id": 763258, "author_name": "Chason", "author_url": "", "post_date": "2020-03-04T09:54:16.913000", "content": "@jaideepvalani So far, we haven't found any benefits from the audio model, but it's still worth trying.
", "votes": 0, "replies": [] }, { "id": 765758, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-07T03:48:49.427000", "content": "@chason\nm using detection score only if score is more than this add it in list,\nhow about the area how to find the relavant one based on this
", "votes": 0, "replies": [] }, { "id": 766128, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-03-07T17:37:30.077000", "content": "@chenshen03 What's your local validation score and how did you choose your validation set?
", "votes": 0, "replies": [] } ] }, { "id": 750044, "author_name": "Kha Vo", "author_url": "", "post_date": "2020-02-19T04:52:36.813000", "content": "“Need to think of a good model” :)
", "votes": 3, "replies": [ { "id": 750047, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-19T04:57:29.837000", "content": "LOL I am trying to think of a better model too! There's a ton of good kernels and discussions of good models. I recommend you to look into some of them(will get you to top 10%).
", "votes": 1, "replies": [] }, { "id": 750462, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-19T12:03:59.650000", "content": "@unkownhihi could u point me to one such please which u took as base.. thanks in advance
", "votes": 0, "replies": [] }, { "id": 750605, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-19T14:41:16.980000", "content": "@jaideepvalani ResNext/Xception is a good one to start with.
", "votes": 0, "replies": [] }, { "id": 750660, "author_name": "Debanga Raj Neog", "author_url": "", "post_date": "2020-02-19T15:27:37.560000", "content": "Yes, these models get me 0.372
", "votes": 1, "replies": [] }, { "id": 750724, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-19T16:29:31.373000", "content": "I actually use resnext and get a very good score on local CV ,balanced loss bt still very poor score at LB. \nWhat mistake i i could be making .. \nI use MTCNN in pre process pipeline.
", "votes": 0, "replies": [] }, { "id": 750822, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-19T18:03:49.103000", "content": "@jaideepvalani Take your time in finding a better face detector, If ResNext does not work the best, use it with lstm, you should be able to figure rest of it yourself. ;-)
", "votes": 0, "replies": [] }, { "id": 750842, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-19T18:31:33.937000", "content": "Thanku Harshit for your advice... I heard about LSTM used as decoder to encoder models. But havent seen its implementation if you can point out to some kernel or link.. would be useful :)
", "votes": 0, "replies": [] }, { "id": 750875, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-19T19:07:49.680000", "content": "In my opinion, try the normal model to get started. If you did things right, it should get you to the top 10%. We will publish lrcn(the LSTM model @harshitsheoran was talking about) starter kernel soon(along with the better face extractor).
", "votes": 5, "replies": [] }, { "id": 751565, "author_name": "Ankit Saini", "author_url": "", "post_date": "2020-02-20T09:31:44.457000", "content": "@unkownhihi great work. I'm sort of stuck at this point. Maybe your kernel will help me.
", "votes": 0, "replies": [] }, { "id": 752455, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-21T03:15:37.507000", "content": "I went past two bench Mark's 467 and 451 ... at 404 now ,will try interesting things now shift from rank 590 to 80 ...long way to go
", "votes": 1, "replies": [] }, { "id": 752804, "author_name": "", "author_url": "", "post_date": "2020-02-21T12:30:24.603000", "content": "Hi, I want to know did you get a score of 0.38 through lrcn?\nIs the lstm an effective approach?
", "votes": 0, "replies": [] }, { "id": 753016, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-21T16:05:54.487000", "content": "@yzcwansui yes it was an effective approach for us.
", "votes": 0, "replies": [] }, { "id": 753032, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-21T16:37:40.693000", "content": "@yzcwansui although for some folks, single frame model is better. But the best score we can get for single frame model is just 0.44
", "votes": 1, "replies": [] }, { "id": 768934, "author_name": "xmaocai", "author_url": "", "post_date": "2020-03-11T11:14:57.350000", "content": "@harshitsheoran
\nI try the resnet+lstm model, it performs bad in the test dataset while gets good score in the train dataset, I use dropout=0.9 to prevent overfiting but it doesn't work, \ndo u have any advice? appreciate it first.\nwe train/test the model using 5 frames per video. and the num of real and fake is balanced.
New Single Frame by frame model Score 0.34000 used 160k images.
", "votes": 4, "replies": [ { "id": 763253, "author_name": "xmaocai", "author_url": "", "post_date": "2020-03-04T09:46:37.970000", "content": "Wow, how do you deal with data imbalance？and u are using RNN model ?
", "votes": 0, "replies": [] }, { "id": 763490, "author_name": "YangHang978", "author_url": "", "post_date": "2020-03-04T14:35:53.107000", "content": "amazing score, \nwhat's your input size?
", "votes": 0, "replies": [] }, { "id": 763519, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-03-04T15:11:31.457000", "content": "input size 200x200, no rnn, but working on that, yes dealt with data imbalance, but doing the same thing in an LRCN has some drawbacks. Ensemble of 3 models between 0.34-0.35 (all single model), was 0.318, So, I guess, ensembling right now is much less effective than it was back then.
", "votes": 4, "replies": [] }, { "id": 763641, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-04T17:27:59.217000", "content": "@harshitsheoran could u point me to any good public lrcn model available in git hub ?\nwant to try this stuff .. will be newest of its kind for me :)\nALso if you could brief me about the overall framework of this combo. \nDo we need videos to train this model or images sufficient ?
", "votes": 0, "replies": [] }, { "id": 763667, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-03-04T18:04:45.680000", "content": "an LRCN, have a input in this format (batch_size, N_samples, size, size, n_channels), here N_samples can be your continuous or equally gapped number of images of a same video or different, that is up to you, I dont know how to do timedistributed layers in pytorch but in keras you can search timedistributed on google, to know more about it, after that you can wrap up your pretrained model in a timedistributed layer and then use that cnn model with a rnn.
", "votes": 2, "replies": [] }, { "id": 763676, "author_name": "James Howard", "author_url": "", "post_date": "2020-03-04T18:16:50.173000", "content": "It's actually fairly easy, you can switch from frames-by-video to a linear series of frames (for the CNN) and then back to videos, like this:
\n\n\n def forward(self, x):\n batch_size, n_channels, n_frames, height, width = x.size() # e.g. 32 x 3 x 64 x 224 x 224\n assert n_channels == 3, f\"Expecting 3 channels but got {n_channels}\"\n x = x.permute(0,2,1,3,4) # 32 x 64 x 3 x 224 x 224\n x = x.reshape(batch_size*n_frames, n_channels, height, width) # (32*64) x 3 x 224 x 224\n x = self.cnn_model(x) # (32*64) x 2048\n x = x.view(batch_size, n_frames, self.n_cnn_features)\n x, (h_n, h_c) = self.lstm(x) # x = seq_len, batch, num_directions*hidden_size\n x = x[:,-1] # Take last time step, which will be BATCH_SIZE * (2*HIDDEN) -> e.g. 12 * 128\n x = self.dropout(x)\n x = self.linear(x)\n return x\n
@harshitsheoran Dose \"Frame by frame model\" mean that you take each frame as input rather than the cropped face?
", "votes": 0, "replies": [] }, { "id": 764563, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-05T15:55:13.590000", "content": "@jamesphoward \nThanku Dr. james \nIf i understand correctly this how it would be \n1 ) CNN first layer nn.Conv2d(3,n ....) \n2) Input size ( 32*64=2048,3,224,224)..i wonder how we can fit such high batch size ?\n3) CNN Output \n4) resize output to 32 ,64 frames * 2048 for LSTM
\n\n5) output of LSTM passed to linear layer\n6) sigmoid bce loss
", "votes": 0, "replies": [] }, { "id": 764657, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-03-05T17:46:29.493000", "content": "Frame by frame have data input like this, (batch_size, face_size, face_size, n_channels), the face are cropped from the frame but they are not fed into an lstm or any transformer, it makes it a simple binary classfication problem, but here also we find that a simple LRCN performs much better than our frame-by-frame model, it looks like we are not even close to done improving. Good Luck!
", "votes": 2, "replies": [] }, { "id": 764902, "author_name": "Kha Vo", "author_url": "", "post_date": "2020-03-06T03:54:34.483000", "content": "@jamesphoward I did this few weeks ago, but face memory problem. How did you fit 32x64x224x224x3 for each batch? It's quite a lot. Using accumulated gradient seems to be faulty as BatchNorm won't work if we fit 1x64x224x224x3 each time.
", "votes": 0, "replies": [] }, { "id": 764980, "author_name": "ryches", "author_url": "", "post_date": "2020-03-06T06:43:50.570000", "content": "Can you clarify what you mean by 160k images? @harshitsheoran Given that we have >100k videos does that mean youre only looking at a couple frames per video? Or are you using some method to choose to train on a subset of videos?
", "votes": 0, "replies": [] }, { "id": 764982, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-03-06T06:49:03.763000", "content": "@ryches good question, first of all I am only using first 40 folders, yes, that does mean, that on an average we do have only a few frames from each video but honestly they are selected so perfectly anything more would be nothing but an overkill and the chances were that it would 1. not letting the model any advantage 2. making the model starting overfitting faster than its normal epoch.
", "votes": 1, "replies": [] }, { "id": 765129, "author_name": "James Howard", "author_url": "", "post_date": "2020-03-06T09:29:01.957000", "content": "@khahuras It was just an example, what batch size of videos and how many frames per video you use depend on how complex the CNN is, how many parameters you use for the RNN, your GPU, and whether you freeze the CNN before embedding it. I advise the latter, at least for earlier stages of training.
", "votes": 2, "replies": [] }, { "id": 765135, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-06T09:34:20.720000", "content": "@jamesphoward \nwhy did we take last time step in your sample code above ?
", "votes": 0, "replies": [] }, { "id": 765138, "author_name": "James Howard", "author_url": "", "post_date": "2020-03-06T09:39:36.473000", "content": "Because of the way RNNs work in pytorch you get an output (prediction) after every time step (i.e. every frame of the video). However, you probably just want the RNN's prediction once it's seen the whole video, so I take the last time step. However, maybe that's not the best method, who knows?
", "votes": 0, "replies": [] }, { "id": 765269, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-06T12:41:42.873000", "content": "What is the size of your self.linear ?\nsince output of lstm would be for eg. 12 * 128
", "votes": 0, "replies": [] }, { "id": 765298, "author_name": "James Howard", "author_url": "", "post_date": "2020-03-06T13:21:51.317000", "content": "The input size would be the the number of hidden units, or double this if your LSTM/GRU was bidirectional.
", "votes": 0, "replies": [] }, { "id": 765320, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-06T13:42:15.960000", "content": "Sorry to bother you more..\nThis is my input \n\nself.LSTM=nn.LSTM(1024,512,2) \nself.linear=nn.linear(512,1)\nx torch.Size([192, 1024])-> reshaped to 16,12,1024 for nn.LSTM\n\nif i do x=x[:,-1] my original batch size of 192 gets reduced to 12 .\nIs there an additional layer needed before feeding to Loss function as my output of linear wont meet requirement of loss function which is 192*1
Have you set batch_first=True in your LSTM if you are supplying the batch first?
Yup i did as first part.. \n```\nclass lstm(nn.Module):\n def init(self,bs,num, lstm1_input=1024,lstm_hidden=512):\n super().init()\n self.batch=bs #16\n self.num=num #12
\n\n self.lstm = nn.LSTM(1024, 512 ,2,batch_first=True)# input elements,hidden state size\n self.linear=nn.Linear(512,1)\n self.drop=nn.Dropout(0.5)\ndef forward(self,input):\n #print(input.size())\n x=input.view(self.batch,self.num,input.size(1)) # resized from 192 to 16,12\n x,y=self.lstm(x)\n print(x.size()) #see below\n x = x[:,-1,:]\n print(x.size()) #see below\n x=self.drop(x)\n x=self.linear(x)\n return x\n```
\n\ntorch.Size([16, 12, 512])\ntorch.Size([16, 512])
", "votes": 0, "replies": [] }, { "id": 765378, "author_name": "James Howard", "author_url": "", "post_date": "2020-03-06T14:47:03.843000", "content": "Isn't that correct? You have fed a batch of 16 into your network and your linear layer is receiving 16 sets of 512 hidden units' outputs? I don't see the problem...
", "votes": 0, "replies": [] }, { "id": 765386, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-06T15:04:11.990000", "content": "yes i dont have issue here but in the loss calculation.. \n data loader will give labels with bs =batch * num of frames * num of classes\nlooks like i should generate 12 steps using some Time Distribute layer like in keras and pack them. while keeping the original bs as 16 only for data loader...\nis this wha
", "votes": 0, "replies": [] }, { "id": 765402, "author_name": "James Howard", "author_url": "", "post_date": "2020-03-06T15:26:44.247000", "content": "Hi, I think that's really a problem you are going to need to solve as this isn't really appropriate for the forums, and we're now getting on to fundamental issues of network training, sorry!
", "votes": 0, "replies": [] }, { "id": 768364, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-10T17:37:36.477000", "content": "@harshitsheoran \nI overcame all the bs related issue after chosing right way of reshaping and transposing the dims. \nNow i face another issue which is overfitting. Model done best than ever models i tried on CV but miserably overfits on test data,especially fails to predict reals correctly. \nDid u faced this issue with LSTM.. if yes what can be f help. i purposefully dint pay attn on split strategy.
\n\n\nepoch train_loss valid_loss accuracy time\n0 0.450687 0.451665 0.834688 09:08\n1 0.375398 0.342386 0.859259 09:06\n2 0.261341 0.245293 0.896838 09:02\n3 0.199818 0.183108 0.928907 09:06 \n4 0.169864 0.163689 0.938302 09:07\n5 0.140049 0.210106 0.915447 09:10\n6 0.126233 0.136548 0.947064 09:11\n
@jaideepvalani well, that loss is kinda close to ours, with accuracy also close to ours every epoch, (our 1 epoch takes 3 hours so idk how you are overfitting and I am not, I also have a much much much larger dataset which I train on compared to yours) I never had overfitting issues and please if you are using keras dont focus on valid_loss it lies, I guess you should have a benchmark, of last 10 folders, after every epoch check your model on that, and see which epoch is starts overfitting, in our case
\n\nmodel__e1 -> 0.212 / 0.934\nmodel_e2 -> 0.183 / 0.946\nmodel_e3 -> 0.175 / 0.952\nmodel_e4 -> 0.186 / 0.950
\n\neven something like this happens where you think overfitting is starting, still train it might get a better improvement in next 3 epochs.
\n\ntry dropout, and batch norm (however I dont use any of them) maybe to overcome overfitting, also no noise function in the augmentations, check the labels and your generator again manually. We had some problem and that was cuz our generator was messed up once.
", "votes": 3, "replies": [] }, { "id": 768526, "author_name": "Kha Vo", "author_url": "", "post_date": "2020-03-10T23:32:20.117000", "content": "@harshitsheoran what is your bce val score each epoch?
", "votes": 0, "replies": [] }, { "id": 768565, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-03-11T01:16:58.600000", "content": "@khahuras that is the validation loss I shared and not the training loss
\n\n0.160 on that bench means 0.318 on leaderboard, always almost double.
", "votes": 2, "replies": [] }, { "id": 768642, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-11T03:43:04.843000", "content": "1) how does your hist gram of 400 sub videos look like assuming fake =0.99 and real =0.... \nleft and right height bars ,which one more ?\n2) bi direction LSTM or uni ?\n3) what are you lstm dimensions.. bs,num_frames ,no of elements does keeping higher bs helps more ?\n4) how is your batch distribution real:fake ratio in batch
", "votes": 0, "replies": [] }, { "id": 768661, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-03-11T04:24:13.177000", "content": "@jaideepvalani here's my response(I'm basically in charge of model training right now):\n1. We used a post-processing technique. Here's the histogram
thanku.. during inference how many video frames used...\nis it necessary to keep same size as in training ?
", "votes": 0, "replies": [] }, { "id": 769132, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-03-11T15:02:04.417000", "content": "The answer might surprise you. We only used not a constant amount of frame per video.... I don't get the second question, sry.
", "votes": 0, "replies": [] }, { "id": 769193, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-11T16:15:01.213000", "content": "i meant the batch size and num of frames during inference should they be same as in training for lstm to give expected output as in training
", "votes": 0, "replies": [] }, { "id": 769201, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-03-11T16:23:32.310000", "content": "ok. I got you now. the num of frames during inference is 10(they don't really affect much). (secret hint: we don't pass that 10 frames to lrcn at once, I can't really go into details.). for batch size during inference, its 1.
", "votes": 1, "replies": [] }, { "id": 770631, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-13T07:14:12.937000", "content": "Thanks.. is public score calculated based on the 400 test videos or it could be some other videos which are subset of private one.
", "votes": 0, "replies": [] }, { "id": 770917, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-03-13T15:09:45.687000", "content": "@jaideepvalani I think the public score is calculated based on the 4000 videos that will be tested in a black box environment.
", "votes": 0, "replies": [] } ] }, { "id": 756584, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-25T21:33:50.670000", "content": "My latest model scores: 0.3422
", "votes": 4, "replies": [ { "id": 758365, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-27T17:20:11.153000", "content": "@ngcferreira amazing score\nwhich model did u use ?\nHave u used ur own dataset,if yes which face detector and no of frames used ?
", "votes": 0, "replies": [] }, { "id": 758416, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-27T18:19:41.723000", "content": "I've use a CNN based model, using only one frame.\nI've used no external data and used yolov3 for detecting faces and create the training set and blazeface for inference. \nThe prediction is just an average of 40 frames.
", "votes": 4, "replies": [] }, { "id": 758419, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-27T18:22:00.390000", "content": "what is your input face size?
", "votes": 0, "replies": [] }, { "id": 759069, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-28T14:27:16.680000", "content": "@unkownhihi I'm using face images scaled to 224x224
", "votes": 1, "replies": [] }, { "id": 763234, "author_name": "xmaocai", "author_url": "", "post_date": "2020-03-04T09:21:03.777000", "content": "@ngcferreira how do you deal with data imbalance？
", "votes": 0, "replies": [] }, { "id": 766231, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-03-07T22:00:21.667000", "content": "@xmaocai So first I split the data into 3 sets (train, val and test set). In every epoch, I randomly choose the same number of fake images as the number of real images. Basically I'm under sampling the data.
", "votes": 1, "replies": [] } ] }, { "id": 765464, "author_name": "BokingChen", "author_url": "", "post_date": "2020-03-06T16:48:22.063000", "content": "CNN+LSTM ,You can refer to https://github.com/HHTseng/video-classification
", "votes": 1, "replies": [ { "id": 768635, "author_name": "xmaocai", "author_url": "", "post_date": "2020-03-11T03:32:03.570000", "content": "Hello, thanks for u advice. I try it but the model doesn't perform well in the val dataset while gets good acc in the train dataset. I use dropout=0.9 to prevent overfiting but it doesn't work, \ndo u have any advice? \nbtw, 5 frames per video.
", "votes": 0, "replies": [] }, { "id": 769109, "author_name": "Jaideep", "author_url": "", "post_date": "2020-03-11T14:38:15.880000", "content": "@xmacai what is the dim of LSTM and input X to CNN.. \nmay u could be mistaking at any place
", "votes": 0, "replies": [] }, { "id": 770798, "author_name": "xmaocai", "author_url": "", "post_date": "2020-03-13T12:27:33.170000", "content": "@jaideepvalani \ninput X dim: torch.Size([64, 1, 3, 224, 224])\nthe dim of input to LSTM: torch.Size([8, 1, 512])\nLSTM dim: \nself.LSTM =nn.LSTM(\n input_size=300,\n hidden_size=256,\n num_layers=3,\n batch_first=True,)\nHere are the input data handle:\nfor index in data:\n image = Image.open(index).convert('RGB')\n if self.data_transforms is not None:\n try:\n image = self.data_transforms(image)\n except:\n print(\"Cannot transform image: {}\".format(index))\n X.append(image)\nX = torch.stack(X, dim=0)\nY = torch.LongTensor([label])
current best score is 0.40621 after trying for weeks :)
", "votes": 1, "replies": [] }, { "id": 753355, "author_name": "Akash", "author_url": "", "post_date": "2020-02-22T04:17:41.253000", "content": "Well, I have my train split as 0-29 and 40 to 49 and val split as 30-39, the same as @jamesphoward . Currently, I'm getting a validation score of 0.295 for \"Image Only\" model. But I have 0 trust in it! Or maybe I have a bug in my inference pipeline. Who knows!\nAlso, the fact that the model starts overfitting from 3rd epoch onwards gives me the chills.
", "votes": 1, "replies": [] }, { "id": 750291, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-19T09:07:17.483000", "content": "My best model (images only) scores: 0.40457
", "votes": 1, "replies": [] }, { "id": 750589, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-02-19T14:34:03.210000", "content": "0.489 :P
\n\nUpdate 3: .402\nUpdate 4: .392
", "votes": 2, "replies": [ { "id": 765336, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-03-06T14:07:27.887000", "content": "0.43* All I did was make a different inference kernal (different face detector) and pick a specific model trained on the data from my baseline kernal (one I trained over 2 weeks ago).
", "votes": 1, "replies": [] } ] }, { "id": 785367, "author_name": "saewonYang", "author_url": "", "post_date": "2020-03-25T02:14:25.803000", "content": "what kind of scheduler do you guys use for training? fixed lr or cosine scheduling, etc..
", "votes": 0, "replies": [] }, { "id": 769950, "author_name": "Trigram", "author_url": "", "post_date": "2020-03-12T12:39:41.487000", "content": "0.44202.
", "votes": 0, "replies": [ { "id": 776564, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2020-03-17T13:28:24.857000", "content": "0.42352 .. Just near :)
", "votes": 1, "replies": [] } ] }, { "id": 755733, "author_name": "Phil", "author_url": "", "post_date": "2020-02-25T04:23:27.997000", "content": "@harshitsheoran hi，when using lrcn，how many frames of one video do you use？are they sequential frames or not。 appreciate for your advises
", "votes": 0, "replies": [ { "id": 757654, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-27T01:58:20.820000", "content": "10 frames per video, you can increase or decrease does not really matter, they are not sequential, they are equally gapped.
", "votes": 2, "replies": [] } ] }, { "id": 752910, "author_name": "petya", "author_url": "", "post_date": "2020-02-21T14:37:16.007000", "content": "0.41136 with a single CNN model. I wonder, how are you dealing with inference on multi-faced videos? I mean, one face is usually okay in each frame, another is distorted, rarely both of them. Should I pick the max prediction per frame or something? \nupd: 0.40463\nupd: 0.40128
", "votes": 0, "replies": [ { "id": 752923, "author_name": "James Howard", "author_url": "", "post_date": "2020-02-21T14:46:02.813000", "content": "Excellent question. I've experimented with this and I've found the mean to be very slightly better than either the max or min.
\n\nI used to think the maximum would be best, but that assumes you don't have false positives for face detection; false positives are likely to be unnatural and therefore fakes, and so it might be the mean is a little more resilient to this.
\n\nI would highlight, though, that this is very likely to depend on your processing pipeline.
", "votes": 8, "replies": [] }, { "id": 752964, "author_name": "Jaideep", "author_url": "", "post_date": "2020-02-21T15:25:10.303000", "content": "Distorted face found during detection are likely to be predicted as Fake. So if there is frame with distorted in list of frames then that would most likely contribute to false positive and hence with max prob compared to all.
", "votes": 1, "replies": [] }, { "id": 753088, "author_name": "petya", "author_url": "", "post_date": "2020-02-21T17:39:24.780000", "content": "Thanks for the insight. I guess I'll play with inference detector a little bit more.
", "votes": 0, "replies": [] }, { "id": 754121, "author_name": "BokingChen", "author_url": "", "post_date": "2020-02-23T04:41:19.190000", "content": "what's your face extractor? I use MTCNN but it can't get a good LB
", "votes": 0, "replies": [] }, { "id": 758870, "author_name": "petya", "author_url": "", "post_date": "2020-02-28T09:22:12.697000", "content": "I've played with blazeface, yolov2, yolov3, mtcnn, and yolov3 got me the best score of 0.40463 for a single model
", "votes": 1, "replies": [] }, { "id": 761607, "author_name": "fergusoci", "author_url": "", "post_date": "2020-03-02T18:32:21.367000", "content": "@defileroff did you find that there was much difference between the face extractors in terms of overall score? I haven't played around with this yet, wondering what kind of priority it should be relative to everything else :)
", "votes": 0, "replies": [] }, { "id": 764394, "author_name": "petya", "author_url": "", "post_date": "2020-03-05T12:19:56.800000", "content": "@fergusoci Playing with different frames, faces, extractors for train/valid sets gave me a boost of 0.05, playing with different extractors at inference time gave me a boost of 0.01
\n\nI guess if your classifier is weak in terms of false-positive numbers, you'd need a better detector to avoid false-positive face detections. But if your classifier is good enough, you can take a less picky detector, that will detect everything that looks like a face, lol, and let the classifier deal with it.
", "votes": 2, "replies": [] }, { "id": 774959, "author_name": "kkk man", "author_url": "", "post_date": "2020-03-16T05:33:29.230000", "content": "@defileroff pre-trained yolov3 face detector or trained with dfdc dataset?
", "votes": 0, "replies": [] }, { "id": 790715, "author_name": "petya", "author_url": "", "post_date": "2020-03-29T19:53:32.240000", "content": "Pretrained one, I don't even use multi-frames like others do, works fine to some extent.
", "votes": 0, "replies": [] } ] }, { "id": 752227, "author_name": "", "author_url": "", "post_date": "2020-02-20T19:52:05.620000", "content": "good
", "votes": 0, "replies": [] }, { "id": 765463, "author_name": "", "author_url": "", "post_date": "2020-03-06T16:46:12.143000", "content": "", "votes": 0, "replies": [] }, { "id": 765277, "author_name": "", "author_url": "", "post_date": "2020-03-06T12:56:25.547000", "content": "", "votes": 0, "replies": [] }, { "id": 759856, "author_name": "", "author_url": "", "post_date": "2020-02-29T14:52:08.033000", "content": "", "votes": 0, "replies": [ { "id": 759946, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-29T16:39:44.357000", "content": "We are using a ensemble.
", "votes": 0, "replies": [] }, { "id": 763356, "author_name": "BokingChen", "author_url": "", "post_date": "2020-03-04T11:57:23.233000", "content": "thanks, I ensamble two model also get a better score
", "votes": 0, "replies": [] } ] }, { "id": 757172, "author_name": "", "author_url": "", "post_date": "2020-02-26T13:52:53.370000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "750030": "Please share your best single model LB(not ensemble).\nTo start with ours: 0.38504\nEDIT: 0.35040\nEDIT2: 0.34", "751313": "Frame-by-frame model (only image): 0.33966 ", "750624": "0.32", "750044": "“Need to think of a good model” :)", "762742": "New Single Frame by frame model Score 0.34000 used 160k images.", "756584": "My latest model scores: 0.3422", "765464": "CNN+LSTM ,You can refer to https://github.com/HHTseng/video-classification", "757639": "current best score is 0.40621 after trying for weeks :)", "753355": "Well, I have my train split as 0-29 and 40 to 49 and val split as 30-39, the same as @jamesphoward . Currently, I'm getting a validation score of 0.295 for \"Image Only\" model. But I have 0 trust in it! Or maybe I have a bug in my inference pipeline. Who knows!\nAlso, the fact that the model starts overfitting from 3rd epoch onwards gives me the chills.", "750291": "My best model (images only) scores: 0.40457\n", "750589": "0.489 :P\n\nUpdate 3: .402\nUpdate 4: .392", "785367": "what kind of scheduler do you guys use for training? fixed lr or cosine scheduling, etc..", "769950": "0.44202.", "755733": "@harshitsheoran hi，when using lrcn，how many frames of one video do you use？are they sequential frames or not。 appreciate for your advises", "752910": "0.41136 with a single CNN model. I wonder, how are you dealing with inference on multi-faced videos? I mean, one face is usually okay in each frame, another is distorted, rarely both of them. Should I pick the max prediction per frame or something? \nupd: 0.40463\nupd: 0.40128", "752227": "good", "765463": "", "765277": "", "759856": "", "757172": "" } }