{
  "id": 275942,
  "title": "5th Place Solution(Team: underfitting)",
  "url": "/competitions/landmark-retrieval-2021/discussion/275942",
  "author_name": "tereka",
  "post_date": "2021-10-02T11:38:46.828000",
  "votes": 26,
  "comment_count": 0,
  "views": 0,
  "content": "<p>First of all, very much thanks to the host for holding Google Landmark Competition.<br>\nthese competitions are very interesting and impressive for us.<br>\nAlsoo thanks to my teammate( <a href=\"https://www.kaggle.com/ks2019\" target=\"_blank\">@ks2019</a> )he contributes to our team for many work.</p>\n<p>Our 5th solution summary is here</p>\n<h1>Brief Summary</h1>\n<ul>\n<li>Backbone + ArcFace, GLDv2 Training(4.1M)</li>\n<li>Post Process(WDA, bridged confidence etc….)</li>\n</ul>\n<h1>Solution</h1>\n<h2>Models</h2>\n<p>Model architecture is backbone+ArcFace module+Average Pooling.<br>\nThe backbone list is here.</p>\n<ul>\n<li>EfficientNet v2s 800</li>\n<li>EfficientNet v2m 732</li>\n<li>EfficientNet v2m 640</li>\n<li>EfficientNet v2l 720</li>\n<li>EfficientNet v2xl 640(training 512)</li>\n</ul>\n<p>Training data is gldv2(4.1M).  in order to train large image size with high batch size, we used gradient accumulation and mixed-precision training,(above v2s)</p>\n<p>We use Adam 15ep training with Cosine Annealing and extract 512 embedding vectors from each model. After we create concatenation vectors(2560).</p>\n<p>our first result is KNN of GPU(cuml) method.<br>\nKNN in GPU is faster than CPU, it's a great performance.</p>\n<p>Also, we use pre-compute embedding, it computes in local and uploads all.(it use post-process)</p>\n<h2>Post-Process</h2>\n<p>We apply some post-process in our first result.<br>\nWDA, bridged-confidence, power KNN etc…. detail is here<br>\n<a href=\"https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275955\" target=\"_blank\">https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275955</a></p>\n<h2>Hardware</h2>\n<p>TPU-v3 8core(GCP)</p>\n<h2>Our code</h2>\n<p><a href=\"https://www.kaggle.com/ks2019/landmark-retrieval-5th-place-inference-notebook\" target=\"_blank\">https://www.kaggle.com/ks2019/landmark-retrieval-5th-place-inference-notebook</a></p>",
  "messages": [
    {
      "id": 1531800,
      "postDate": "2021-10-02T11:38:46.830Z",
      "content": "<p>First of all, very much thanks to the host for holding Google Landmark Competition.<br>\nthese competitions are very interesting and impressive for us.<br>\nAlsoo thanks to my teammate( <a href=\"https://www.kaggle.com/ks2019\" target=\"_blank\">@ks2019</a> )he contributes to our team for many work.</p>\n<p>Our 5th solution summary is here</p>\n<h1>Brief Summary</h1>\n<ul>\n<li>Backbone + ArcFace, GLDv2 Training(4.1M)</li>\n<li>Post Process(WDA, bridged confidence etc….)</li>\n</ul>\n<h1>Solution</h1>\n<h2>Models</h2>\n<p>Model architecture is backbone+ArcFace module+Average Pooling.<br>\nThe backbone list is here.</p>\n<ul>\n<li>EfficientNet v2s 800</li>\n<li>EfficientNet v2m 732</li>\n<li>EfficientNet v2m 640</li>\n<li>EfficientNet v2l 720</li>\n<li>EfficientNet v2xl 640(training 512)</li>\n</ul>\n<p>Training data is gldv2(4.1M).  in order to train large image size with high batch size, we used gradient accumulation and mixed-precision training,(above v2s)</p>\n<p>We use Adam 15ep training with Cosine Annealing and extract 512 embedding vectors from each model. After we create concatenation vectors(2560).</p>\n<p>our first result is KNN of GPU(cuml) method.<br>\nKNN in GPU is faster than CPU, it's a great performance.</p>\n<p>Also, we use pre-compute embedding, it computes in local and uploads all.(it use post-process)</p>\n<h2>Post-Process</h2>\n<p>We apply some post-process in our first result.<br>\nWDA, bridged-confidence, power KNN etc…. detail is here<br>\n<a href=\"https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275955\" target=\"_blank\">https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275955</a></p>\n<h2>Hardware</h2>\n<p>TPU-v3 8core(GCP)</p>\n<h2>Our code</h2>\n<p><a href=\"https://www.kaggle.com/ks2019/landmark-retrieval-5th-place-inference-notebook\" target=\"_blank\">https://www.kaggle.com/ks2019/landmark-retrieval-5th-place-inference-notebook</a></p>",
      "rawMarkdown": "First of all, very much thanks to the host for holding Google Landmark Competition.\nthese competitions are very interesting and impressive for us.\nAlsoo thanks to my teammate( @ks2019 )he contributes to our team for many work.\n\nOur 5th solution summary is here\n\n# Brief Summary\n- Backbone + ArcFace, GLDv2 Training(4.1M)\n- Post Process(WDA, bridged confidence etc....)\n\n# Solution\n## Models\nModel architecture is backbone+ArcFace module+Average Pooling.\nThe backbone list is here.\n\n- EfficientNet v2s 800\n- EfficientNet v2m 732\n- EfficientNet v2m 640\n- EfficientNet v2l 720\n- EfficientNet v2xl 640(training 512)\n\nTraining data is gldv2(4.1M).  in order to train large image size with high batch size, we used gradient accumulation and mixed-precision training,(above v2s)\n\nWe use Adam 15ep training with Cosine Annealing and extract 512 embedding vectors from each model. After we create concatenation vectors(2560).\n\nour first result is KNN of GPU(cuml) method.\nKNN in GPU is faster than CPU, it's a great performance.\n\nAlso, we use pre-compute embedding, it computes in local and uploads all.(it use post-process)\n\n## Post-Process\nWe apply some post-process in our first result.\nWDA, bridged-confidence, power KNN etc.... detail is here\nhttps://www.kaggle.com/c/landmark-retrieval-2021/discussion/275955\n\n## Hardware\nTPU-v3 8core(GCP)\n\n## Our code\nhttps://www.kaggle.com/ks2019/landmark-retrieval-5th-place-inference-notebook",
      "votes": 26
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1531800": "First of all, very much thanks to the host for holding Google Landmark Competition.\nthese competitions are very interesting and impressive for us.\nAlsoo thanks to my teammate( @ks2019 )he contributes to our team for many work.\n\nOur 5th solution summary is here\n\n# Brief Summary\n- Backbone + ArcFace, GLDv2 Training(4.1M)\n- Post Process(WDA, bridged confidence etc....)\n\n# Solution\n## Models\nModel architecture is backbone+ArcFace module+Average Pooling.\nThe backbone list is here.\n\n- EfficientNet v2s 800\n- EfficientNet v2m 732\n- EfficientNet v2m 640\n- EfficientNet v2l 720\n- EfficientNet v2xl 640(training 512)\n\nTraining data is gldv2(4.1M).  in order to train large image size with high batch size, we used gradient accumulation and mixed-precision training,(above v2s)\n\nWe use Adam 15ep training with Cosine Annealing and extract 512 embedding vectors from each model. After we create concatenation vectors(2560).\n\nour first result is KNN of GPU(cuml) method.\nKNN in GPU is faster than CPU, it's a great performance.\n\nAlso, we use pre-compute embedding, it computes in local and uploads all.(it use post-process)\n\n## Post-Process\nWe apply some post-process in our first result.\nWDA, bridged-confidence, power KNN etc.... detail is here\nhttps://www.kaggle.com/c/landmark-retrieval-2021/discussion/275955\n\n## Hardware\nTPU-v3 8core(GCP)\n\n## Our code\nhttps://www.kaggle.com/ks2019/landmark-retrieval-5th-place-inference-notebook"
  }
}