{
  "id": 140357,
  "title": "Model run-time vs public score",
  "url": "/competitions/deepfake-detection-challenge/discussion/140357",
  "author_name": "",
  "post_date": "2020-04-01T13:02:18.315688200Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Congrats to all of those at the top for impressive scores. I unfortunately ran out of time to explore more options (ensembles etc) to make full use of the 9 hours available for inference. </p>\n\n<p>My relatively simple model (OpenCV video processing, Retinaface face extraction, EfficientNet-B2 + LSTM, 16 frames per video) managed a full inference pass in about 3.5 hours. I did scale down the full video frames to avoid running out of CUDA memory on Kaggle (I assume my submission errors were caused by that even though I do split up the face batches based on frame size)</p>\n\n<p>Did everbody max out the allowed time to get their best score? It would be interesting to compare score vs run-time. </p>",
  "messages": [
    {
      "id": "793977",
      "postDate": "04/01/2020 13:02:18",
      "content": "<p>Congrats to all of those at the top for impressive scores. I unfortunately ran out of time to explore more options (ensembles etc) to make full use of the 9 hours available for inference. </p>\n\n<p>My relatively simple model (OpenCV video processing, Retinaface face extraction, EfficientNet-B2 + LSTM, 16 frames per video) managed a full inference pass in about 3.5 hours. I did scale down the full video frames to avoid running out of CUDA memory on Kaggle (I assume my submission errors were caused by that even though I do split up the face batches based on frame size)</p>\n\n<p>Did everbody max out the allowed time to get their best score? It would be interesting to compare score vs run-time. </p>",
      "rawMarkdown": "Congrats to all of those at the top for impressive scores. I unfortunately ran out of time to explore more options (ensembles etc) to make full use of the 9 hours available for inference. \n\nMy relatively simple model (OpenCV video processing, Retinaface face extraction, EfficientNet-B2 + LSTM, 16 frames per video) managed a full inference pass in about 3.5 hours. I did scale down the full video frames to avoid running out of CUDA memory on Kaggle (I assume my submission errors were caused by that even though I do split up the face batches based on frame size)\n\nDid everbody max out the allowed time to get their best score? It would be interesting to compare score vs run-time.",
      "votes": null
    },
    {
      "id": "793992",
      "postDate": "04/01/2020 13:16:58",
      "content": "<p><a href=\"https://www.kaggle.com/jagannathrk/frames-per-video-viz\">https://www.kaggle.com/jagannathrk/frames-per-video-viz</a>\nIn this experiments, using more than 60 frames have a little benifit.</p>\n\n<p>In my case using 25 frames marked higher score than using 35frames always.\nI don't know why, but I guess it seemed that first 5~20 frames might have high disturbance or just luck for matching frame with my model.</p>\n\n<p>And longer time allow more models in ensemble and high-resolution for face-detection.</p>\n\n<p>BTW, it's great to mark high rank with only 3.5 hours evaluation time!</p>",
      "rawMarkdown": "https://www.kaggle.com/jagannathrk/frames-per-video-viz\nIn this experiments, using more than 60 frames have a little benifit.\n\n In my case using 25 frames marked higher score than using 35frames always.\nI don't know why, but I guess it seemed that first 5~20 frames might have high disturbance or just luck for matching frame with my model.\n\nAnd longer time allow more models in ensemble and high-resolution for face-detection.\n\nBTW, it's great to mark high rank with only 3.5 hours evaluation time!",
      "votes": null
    },
    {
      "id": "794119",
      "postDate": "04/01/2020 15:19:44",
      "content": "<p>I maxed out 1GB external data (mostly by my models) limit before hitting 9 hr limit. One of the submissions has pretty heavy processing with 19 models and 120 frame predictions (64+64Flipped), took ~6.5hr. Score 0.302LB (not our best, but for safe-side wanted to submit a heavy ensembled 2nd solution). Thanks to super fast Blazeface.</p>",
      "rawMarkdown": "I maxed out 1GB external data (mostly by my models) limit before hitting 9 hr limit. One of the submissions has pretty heavy processing with 19 models and 120 frame predictions (64+64Flipped), took ~6.5hr. Score 0.302LB (not our best, but for safe-side wanted to submit a heavy ensembled 2nd solution). Thanks to super fast Blazeface.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 793992,
      "author_name": "gwsong",
      "author_url": "",
      "post_date": "04/01/2020 13:16:58",
      "content": "<p><a href=\"https://www.kaggle.com/jagannathrk/frames-per-video-viz\">https://www.kaggle.com/jagannathrk/frames-per-video-viz</a>\nIn this experiments, using more than 60 frames have a little benifit.</p>\n\n<p>In my case using 25 frames marked higher score than using 35frames always.\nI don't know why, but I guess it seemed that first 5~20 frames might have high disturbance or just luck for matching frame with my model.</p>\n\n<p>And longer time allow more models in ensemble and high-resolution for face-detection.</p>\n\n<p>BTW, it's great to mark high rank with only 3.5 hours evaluation time!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 794119,
      "author_name": "debanga",
      "author_url": "",
      "post_date": "04/01/2020 15:19:44",
      "content": "<p>I maxed out 1GB external data (mostly by my models) limit before hitting 9 hr limit. One of the submissions has pretty heavy processing with 19 models and 120 frame predictions (64+64Flipped), took ~6.5hr. Score 0.302LB (not our best, but for safe-side wanted to submit a heavy ensembled 2nd solution). Thanks to super fast Blazeface.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "793977": "Congrats to all of those at the top for impressive scores. I unfortunately ran out of time to explore more options (ensembles etc) to make full use of the 9 hours available for inference. \n\nMy relatively simple model (OpenCV video processing, Retinaface face extraction, EfficientNet-B2 + LSTM, 16 frames per video) managed a full inference pass in about 3.5 hours. I did scale down the full video frames to avoid running out of CUDA memory on Kaggle (I assume my submission errors were caused by that even though I do split up the face batches based on frame size)\n\nDid everbody max out the allowed time to get their best score? It would be interesting to compare score vs run-time.",
    "793992": "https://www.kaggle.com/jagannathrk/frames-per-video-viz\nIn this experiments, using more than 60 frames have a little benifit.\n\n In my case using 25 frames marked higher score than using 35frames always.\nI don't know why, but I guess it seemed that first 5~20 frames might have high disturbance or just luck for matching frame with my model.\n\nAnd longer time allow more models in ensemble and high-resolution for face-detection.\n\nBTW, it's great to mark high rank with only 3.5 hours evaluation time!",
    "794119": "I maxed out 1GB external data (mostly by my models) limit before hitting 9 hr limit. One of the submissions has pretty heavy processing with 19 models and 120 frame predictions (64+64Flipped), took ~6.5hr. Score 0.302LB (not our best, but for safe-side wanted to submit a heavy ensembled 2nd solution). Thanks to super fast Blazeface."
  },
  "source": "meta"
}