{
  "id": 456484,
  "title": "What graphs are in Layout-XLA test - see here",
  "url": "/competitions/predict-ai-model-runtime/discussion/456484",
  "author_name": "Dmitrii Khizbullin",
  "post_date": "2023-11-20T09:14:02.755000",
  "votes": 1,
  "comment_count": 0,
  "views": 0,
  "content": "<p>We've made a bit of a study to figure out what architectures are there in the test. We matched by the correlation of opcode distributions. Here is the match table for Layout-XLA.</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>test</th>\n<th>train/val most correlated</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>test_05ae41e26dd3c4c06390371a0423233c.npz</td>\n<td>train_efficientnet_b7_eval_batch_1.npz</td>\n</tr>\n<tr>\n<td>2</td>\n<td>test_3e7156ac468dfb75cf5c9615e1e5887d.npz</td>\n<td>train_bert_pretraining.8x16.fp16.npz</td>\n</tr>\n<tr>\n<td>3</td>\n<td>test_5335ed13823b0a518ee3c79ba4425f34.npz</td>\n<td>train_efficientnet_b7_eval_batch_1.npz</td>\n</tr>\n<tr>\n<td>4</td>\n<td>test_937ee0eb0d5d6151b7b8252933b5c1c9.npz</td>\n<td>train_resnet50.2x2.fp32.npz</td>\n</tr>\n<tr>\n<td>5</td>\n<td>test_cd708819d3f5103afd6460b15e74eaf3.npz</td>\n<td>train_mask_rcnn_batch_16_bf16_img1024.npz or train_openai_v0_rnn_natural.npz - <strong>VERY UNCERTAIN</strong></td>\n</tr>\n<tr>\n<td>6</td>\n<td>test_db59a991b7c607634f13570d52ce885f.npz</td>\n<td>train_resnet_v1_50_official_batch_128_f32.npz, most likely it is resnet_v1_50_official_batch_32_f32.npz which would complete the grid</td>\n</tr>\n<tr>\n<td>7</td>\n<td>test_e8a3a1401b5e79f66d7037e424f3b6df.npz</td>\n<td>train_bert_pretraining.8x8.fp32.performance.npz - <strong>A BIT UNCERTAIN</strong></td>\n</tr>\n<tr>\n<td>8</td>\n<td>test_fbaa8bb6a1aed9988281085c91065c05.npz</td>\n<td>train_inference_mlperf_ssd_1200_batch_128.npz</td>\n</tr>\n</tbody>\n</table>\n<p><br></p>\n<p>We did not use this info in our solution, but maybe someone gets interested in learning the method. Also, notice that the test is biased towards Efficientnet since, with all likelihood, there are two nets of this architecture.</p>",
  "messages": [
    {
      "id": 2531504,
      "postDate": "2023-11-20T09:14:02.757Z",
      "content": "<p>We've made a bit of a study to figure out what architectures are there in the test. We matched by the correlation of opcode distributions. Here is the match table for Layout-XLA.</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>test</th>\n<th>train/val most correlated</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>test_05ae41e26dd3c4c06390371a0423233c.npz</td>\n<td>train_efficientnet_b7_eval_batch_1.npz</td>\n</tr>\n<tr>\n<td>2</td>\n<td>test_3e7156ac468dfb75cf5c9615e1e5887d.npz</td>\n<td>train_bert_pretraining.8x16.fp16.npz</td>\n</tr>\n<tr>\n<td>3</td>\n<td>test_5335ed13823b0a518ee3c79ba4425f34.npz</td>\n<td>train_efficientnet_b7_eval_batch_1.npz</td>\n</tr>\n<tr>\n<td>4</td>\n<td>test_937ee0eb0d5d6151b7b8252933b5c1c9.npz</td>\n<td>train_resnet50.2x2.fp32.npz</td>\n</tr>\n<tr>\n<td>5</td>\n<td>test_cd708819d3f5103afd6460b15e74eaf3.npz</td>\n<td>train_mask_rcnn_batch_16_bf16_img1024.npz or train_openai_v0_rnn_natural.npz - <strong>VERY UNCERTAIN</strong></td>\n</tr>\n<tr>\n<td>6</td>\n<td>test_db59a991b7c607634f13570d52ce885f.npz</td>\n<td>train_resnet_v1_50_official_batch_128_f32.npz, most likely it is resnet_v1_50_official_batch_32_f32.npz which would complete the grid</td>\n</tr>\n<tr>\n<td>7</td>\n<td>test_e8a3a1401b5e79f66d7037e424f3b6df.npz</td>\n<td>train_bert_pretraining.8x8.fp32.performance.npz - <strong>A BIT UNCERTAIN</strong></td>\n</tr>\n<tr>\n<td>8</td>\n<td>test_fbaa8bb6a1aed9988281085c91065c05.npz</td>\n<td>train_inference_mlperf_ssd_1200_batch_128.npz</td>\n</tr>\n</tbody>\n</table>\n<p><br></p>\n<p>We did not use this info in our solution, but maybe someone gets interested in learning the method. Also, notice that the test is biased towards Efficientnet since, with all likelihood, there are two nets of this architecture.</p>",
      "rawMarkdown": "We've made a bit of a study to figure out what architectures are there in the test. We matched by the correlation of opcode distributions. Here is the match table for Layout-XLA.\n\n|   | test  | train/val most correlated |\n|---|---|---|\n|  1 | test_05ae41e26dd3c4c06390371a0423233c.npz  | train_efficientnet_b7_eval_batch_1.npz  |\n| 2  | test_3e7156ac468dfb75cf5c9615e1e5887d.npz  |  train_bert_pretraining.8x16.fp16.npz |\n| 3  |  test_5335ed13823b0a518ee3c79ba4425f34.npz |  train_efficientnet_b7_eval_batch_1.npz |\n|  4 |  test_937ee0eb0d5d6151b7b8252933b5c1c9.npz | train_resnet50.2x2.fp32.npz  |\n|  5 | test_cd708819d3f5103afd6460b15e74eaf3.npz  |  train_mask_rcnn_batch_16_bf16_img1024.npz or train_openai_v0_rnn_natural.npz - **VERY UNCERTAIN** |\n|  6 |  test_db59a991b7c607634f13570d52ce885f.npz | train_resnet_v1_50_official_batch_128_f32.npz, most likely it is resnet_v1_50_official_batch_32_f32.npz which would complete the grid  |\n| 7  |  test_e8a3a1401b5e79f66d7037e424f3b6df.npz |  train_bert_pretraining.8x8.fp32.performance.npz - **A BIT UNCERTAIN** |\n| 8  |  test_fbaa8bb6a1aed9988281085c91065c05.npz | train_inference_mlperf_ssd_1200_batch_128.npz  |\n\n<br/>\n\nWe did not use this info in our solution, but maybe someone gets interested in learning the method. Also, notice that the test is biased towards Efficientnet since, with all likelihood, there are two nets of this architecture.",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2531504": "We've made a bit of a study to figure out what architectures are there in the test. We matched by the correlation of opcode distributions. Here is the match table for Layout-XLA.\n\n|   | test  | train/val most correlated |\n|---|---|---|\n|  1 | test_05ae41e26dd3c4c06390371a0423233c.npz  | train_efficientnet_b7_eval_batch_1.npz  |\n| 2  | test_3e7156ac468dfb75cf5c9615e1e5887d.npz  |  train_bert_pretraining.8x16.fp16.npz |\n| 3  |  test_5335ed13823b0a518ee3c79ba4425f34.npz |  train_efficientnet_b7_eval_batch_1.npz |\n|  4 |  test_937ee0eb0d5d6151b7b8252933b5c1c9.npz | train_resnet50.2x2.fp32.npz  |\n|  5 | test_cd708819d3f5103afd6460b15e74eaf3.npz  |  train_mask_rcnn_batch_16_bf16_img1024.npz or train_openai_v0_rnn_natural.npz - **VERY UNCERTAIN** |\n|  6 |  test_db59a991b7c607634f13570d52ce885f.npz | train_resnet_v1_50_official_batch_128_f32.npz, most likely it is resnet_v1_50_official_batch_32_f32.npz which would complete the grid  |\n| 7  |  test_e8a3a1401b5e79f66d7037e424f3b6df.npz |  train_bert_pretraining.8x8.fp32.performance.npz - **A BIT UNCERTAIN** |\n| 8  |  test_fbaa8bb6a1aed9988281085c91065c05.npz | train_inference_mlperf_ssd_1200_batch_128.npz  |\n\n<br/>\n\nWe did not use this info in our solution, but maybe someone gets interested in learning the method. Also, notice that the test is biased towards Efficientnet since, with all likelihood, there are two nets of this architecture."
  }
}