{
  "id": 49230,
  "title": "Best single model?",
  "url": "/competitions/sp-society-camera-model-identification/discussion/49230",
  "author_name": "",
  "post_date": "2018-02-08T08:23:50.891924400Z",
  "votes": 3,
  "comment_count": 20,
  "views": 0,
  "content": "<p>What's your best single model? (no ensembles, TTA ok)</p>\n\n<p>Mine is 97.2 (no TTA)</p>",
  "messages": [
    {
      "id": "279547",
      "postDate": "02/08/2018 08:23:50",
      "content": "<p>What's your best single model? (no ensembles, TTA ok)</p>\n\n<p>Mine is 97.2 (no TTA)</p>",
      "rawMarkdown": "What's your best single model? (no ensembles, TTA ok)\n\nMine is 97.2 (no TTA)",
      "votes": null
    },
    {
      "id": "279563",
      "postDate": "02/08/2018 09:06:54",
      "content": "<p>What does TTA mean?</p>",
      "rawMarkdown": "What does TTA mean?",
      "votes": null
    },
    {
      "id": "279565",
      "postDate": "02/08/2018 09:10:02",
      "content": "<p>Test time augmentation: say e.g. flipping doesn't change the label, so at test time you predict both the regular item and the flipped item; then you average/etc. </p>",
      "rawMarkdown": "Test time augmentation: say e.g. flipping doesn't change the label, so at test time you predict both the regular item and the flipped item; then you average/etc.",
      "votes": null
    },
    {
      "id": "279571",
      "postDate": "02/08/2018 09:17:47",
      "content": "<p>If TTA is ok, my best single model is 0.978.</p>",
      "rawMarkdown": "If TTA is ok, my best single model is 0.978.",
      "votes": null
    },
    {
      "id": "279572",
      "postDate": "02/08/2018 09:23:46",
      "content": "<p>Can you share how much did you gain using TTA? </p>",
      "rawMarkdown": "Can you share how much did you gain using TTA?",
      "votes": null
    },
    {
      "id": "279574",
      "postDate": "02/08/2018 09:29:13",
      "content": "<p>Using TTA, my single model can improve 0.01--0.015.</p>",
      "rawMarkdown": "Using TTA, my single model can improve 0.01--0.015.",
      "votes": null
    },
    {
      "id": "279576",
      "postDate": "02/08/2018 09:37:16",
      "content": "<p>That's huge! So w/o TTA you get 0.968? </p>",
      "rawMarkdown": "That's huge! So w/o TTA you get 0.968?",
      "votes": null
    },
    {
      "id": "279613",
      "postDate": "02/08/2018 11:02:13",
      "content": "<p>96.5:\n256 64*64 crops per image, no tta, no additional data for training, se-resnet-18. </p>",
      "rawMarkdown": "96.5:\n256 64*64 crops per image, no tta, no additional data for training, se-resnet-18.",
      "votes": null
    },
    {
      "id": "279620",
      "postDate": "02/08/2018 11:29:55",
      "content": "<p>0.964</p>",
      "rawMarkdown": "0.964",
      "votes": null
    },
    {
      "id": "279622",
      "postDate": "02/08/2018 11:36:50",
      "content": "<p>96.6 no TTA</p>",
      "rawMarkdown": "96.6 no TTA",
      "votes": null
    },
    {
      "id": "279643",
      "postDate": "02/08/2018 12:57:12",
      "content": "<p>best single model 0.977 with TTA, we saw a big jump by using gleb's additional data and training with all the images good and bad resolution did not bother with selecting images that matches competition resolution, this doubles the additional data and does not adversely affect performance.</p>",
      "rawMarkdown": "best single model 0.977 with TTA, we saw a big jump by using gleb's additional data and training with all the images good and bad resolution did not bother with selecting images that matches competition resolution, this doubles the additional data and does not adversely affect performance.",
      "votes": null
    },
    {
      "id": "279686",
      "postDate": "02/08/2018 14:31:36",
      "content": "<p>984_tta_8_densenet201_29_0.98624</p>",
      "rawMarkdown": "984_tta_8_densenet201_29_0.98624",
      "votes": null
    },
    {
      "id": "279718",
      "postDate": "02/08/2018 15:39:49",
      "content": "<p>WOW. What TTA are you doing?</p>\n\n<p>My best no-TTA is 97.2 using a 512x512 crop. Im testing right now:</p>\n\n<pre><code>TTA_TRANSFORMS = [[], \n                    ['orientation_1'],\n                    ['orientation_3'],\n                    ['manipulation_jpg70'],\n                    ['manipulation_jpg90'],\n                    ['manipulation_gamma0.8'],\n                    ['manipulation_gamma1.2'],\n                    ['manipulation_bicubic1.5'],\n                    ['manipulation_bicubic2.0'],\n                    ['orientation_1', 'manipulation_jpg70'],\n                    ['orientation_1', 'manipulation_jpg90'],\n                    ['orientation_1', 'manipulation_gamma0.8'],\n                    ['orientation_1', 'manipulation_gamma1.2'],\n                    ['orientation_1', 'manipulation_bicubic1.5'],\n                    ['orientation_1', 'manipulation_bicubic2.0'],\n                    ['orientation_3', 'manipulation_jpg70'],\n                    ['orientation_3', 'manipulation_jpg90'],\n                    ['orientation_3', 'manipulation_gamma0.8'],\n                    ['orientation_3', 'manipulation_gamma1.2'],\n                    ['orientation_3', 'manipulation_bicubic1.5'],\n                    ['orientation_3', 'manipulation_bicubic2.0'],\n                    ]\n</code></pre>\n\n<p>Basically for all images do the original one [] orientation flips (90, 180 and 270 degrees), manipulations (provided the test image is <code>unalt</code>) and combinations thereof. Still have not submitted anything.</p>",
      "rawMarkdown": "WOW. What TTA are you doing?\n\nMy best no-TTA is 97.2 using a 512x512 crop. Im testing right now:\n\n    TTA_TRANSFORMS = [[], \n                        ['orientation_1'],\n                        ['orientation_3'],\n                        ['manipulation_jpg70'],\n                        ['manipulation_jpg90'],\n                        ['manipulation_gamma0.8'],\n                        ['manipulation_gamma1.2'],\n                        ['manipulation_bicubic1.5'],\n                        ['manipulation_bicubic2.0'],\n                        ['orientation_1', 'manipulation_jpg70'],\n                        ['orientation_1', 'manipulation_jpg90'],\n                        ['orientation_1', 'manipulation_gamma0.8'],\n                        ['orientation_1', 'manipulation_gamma1.2'],\n                        ['orientation_1', 'manipulation_bicubic1.5'],\n                        ['orientation_1', 'manipulation_bicubic2.0'],\n                        ['orientation_3', 'manipulation_jpg70'],\n                        ['orientation_3', 'manipulation_jpg90'],\n                        ['orientation_3', 'manipulation_gamma0.8'],\n                        ['orientation_3', 'manipulation_gamma1.2'],\n                        ['orientation_3', 'manipulation_bicubic1.5'],\n                        ['orientation_3', 'manipulation_bicubic2.0'],\n                        ]\n\nBasically for all images do the original one [] orientation flips (90, 180 and 270 degrees), manipulations (provided the test image is `unalt`) and combinations thereof. Still have not submitted anything.",
      "votes": null
    },
    {
      "id": "279739",
      "postDate": "02/08/2018 16:19:38",
      "content": "<p>With TTA one model @ LB 0.971 gets now LB 0.976. </p>",
      "rawMarkdown": "With TTA one model @ LB 0.971 gets now LB 0.976.",
      "votes": null
    },
    {
      "id": "280008",
      "postDate": "02/09/2018 04:43:16",
      "content": "<p>Sorry for my ignorance, but can you explain me better the TTA procedure? I've never used this before in my problems. Are you performing several transforms in the testing image? how do you predict its class? </p>",
      "rawMarkdown": "Sorry for my ignorance, but can you explain me better the TTA procedure? I've never used this before in my problems. Are you performing several transforms in the testing image? how do you predict its class?",
      "votes": null
    },
    {
      "id": "280057",
      "postDate": "02/09/2018 07:48:58",
      "content": "<p>96.8 no TTA</p>",
      "rawMarkdown": "96.8 no TTA",
      "votes": null
    },
    {
      "id": "280066",
      "postDate": "02/09/2018 08:29:49",
      "content": "<p>Mine was 0.983 private / 0.980 public</p>",
      "rawMarkdown": "Mine was 0.983 private / 0.980 public",
      "votes": null
    },
    {
      "id": "280128",
      "postDate": "02/09/2018 12:38:22",
      "content": "<p>What was your best single model based on only the original training dataset provided?  I got .87 public and private leaderboard (3 hours after the competition closed) based 1 model using only the original dataset (-15% for validation).  I used TTA only rotating the images (0,90,180,270 degrees).  This was my first competition so I wasn't sure if using the external data was ok so I stuck to the basics.</p>",
      "rawMarkdown": "What was your best single model based on only the original training dataset provided?  I got .87 public and private leaderboard (3 hours after the competition closed) based 1 model using only the original dataset (-15% for validation).  I used TTA only rotating the images (0,90,180,270 degrees).  This was my first competition so I wasn't sure if using the external data was ok so I stuck to the basics.",
      "votes": null
    },
    {
      "id": "280148",
      "postDate": "02/09/2018 13:26:59",
      "content": "<p>best single model without external data: .951 at public, .958 at private</p>",
      "rawMarkdown": "best single model without external data: .951 at public, .958 at private",
      "votes": null
    },
    {
      "id": "280156",
      "postDate": "02/09/2018 13:41:38",
      "content": "<p>You get several predictions for test sample modified in the same spirit as train augmentations do (usually much less augmentations but it may vary wildly from simple flips to many-many augs). Then you aggregate predictions in some way, maybe a simple mean, maybe a geometric mean or voting, whatever</p>",
      "rawMarkdown": "You get several predictions for test sample modified in the same spirit as train augmentations do (usually much less augmentations but it may vary wildly from simple flips to many-many augs). Then you aggregate predictions in some way, maybe a simple mean, maybe a geometric mean or voting, whatever",
      "votes": null
    },
    {
      "id": "280158",
      "postDate": "02/09/2018 13:44:23",
      "content": "<p>Mobilenet 0.972\nDensenet161 0.977 (Chose model 0.976)\nDensenet201 had no time to train :(</p>",
      "rawMarkdown": "Mobilenet 0.972\nDensenet161 0.977 (Chose model 0.976)\nDensenet201 had no time to train :(",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 279563,
      "author_name": "youngkl",
      "author_url": "",
      "post_date": "02/08/2018 09:06:54",
      "content": "<p>What does TTA mean?</p>",
      "votes": null,
      "replies": [
        {
          "id": 279565,
          "author_name": "antorsae",
          "author_url": "",
          "post_date": "02/08/2018 09:10:02",
          "content": "<p>Test time augmentation: say e.g. flipping doesn't change the label, so at test time you predict both the regular item and the flipped item; then you average/etc. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279571,
          "author_name": "youngkl",
          "author_url": "",
          "post_date": "02/08/2018 09:17:47",
          "content": "<p>If TTA is ok, my best single model is 0.978.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279572,
          "author_name": "antorsae",
          "author_url": "",
          "post_date": "02/08/2018 09:23:46",
          "content": "<p>Can you share how much did you gain using TTA? </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279574,
          "author_name": "youngkl",
          "author_url": "",
          "post_date": "02/08/2018 09:29:13",
          "content": "<p>Using TTA, my single model can improve 0.01--0.015.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279576,
          "author_name": "antorsae",
          "author_url": "",
          "post_date": "02/08/2018 09:37:16",
          "content": "<p>That's huge! So w/o TTA you get 0.968? </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279620,
          "author_name": "youngkl",
          "author_url": "",
          "post_date": "02/08/2018 11:29:55",
          "content": "<p>0.964</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279718,
          "author_name": "antorsae",
          "author_url": "",
          "post_date": "02/08/2018 15:39:49",
          "content": "<p>WOW. What TTA are you doing?</p>\n\n<p>My best no-TTA is 97.2 using a 512x512 crop. Im testing right now:</p>\n\n<pre><code>TTA_TRANSFORMS = [[], \n                    ['orientation_1'],\n                    ['orientation_3'],\n                    ['manipulation_jpg70'],\n                    ['manipulation_jpg90'],\n                    ['manipulation_gamma0.8'],\n                    ['manipulation_gamma1.2'],\n                    ['manipulation_bicubic1.5'],\n                    ['manipulation_bicubic2.0'],\n                    ['orientation_1', 'manipulation_jpg70'],\n                    ['orientation_1', 'manipulation_jpg90'],\n                    ['orientation_1', 'manipulation_gamma0.8'],\n                    ['orientation_1', 'manipulation_gamma1.2'],\n                    ['orientation_1', 'manipulation_bicubic1.5'],\n                    ['orientation_1', 'manipulation_bicubic2.0'],\n                    ['orientation_3', 'manipulation_jpg70'],\n                    ['orientation_3', 'manipulation_jpg90'],\n                    ['orientation_3', 'manipulation_gamma0.8'],\n                    ['orientation_3', 'manipulation_gamma1.2'],\n                    ['orientation_3', 'manipulation_bicubic1.5'],\n                    ['orientation_3', 'manipulation_bicubic2.0'],\n                    ]\n</code></pre>\n\n<p>Basically for all images do the original one [] orientation flips (90, 180 and 270 degrees), manipulations (provided the test image is <code>unalt</code>) and combinations thereof. Still have not submitted anything.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279739,
          "author_name": "antorsae",
          "author_url": "",
          "post_date": "02/08/2018 16:19:38",
          "content": "<p>With TTA one model @ LB 0.971 gets now LB 0.976. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 280008,
          "author_name": "anselmoferreira35",
          "author_url": "",
          "post_date": "02/09/2018 04:43:16",
          "content": "<p>Sorry for my ignorance, but can you explain me better the TTA procedure? I've never used this before in my problems. Are you performing several transforms in the testing image? how do you predict its class? </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 280156,
          "author_name": "ceperaang",
          "author_url": "",
          "post_date": "02/09/2018 13:41:38",
          "content": "<p>You get several predictions for test sample modified in the same spirit as train augmentations do (usually much less augmentations but it may vary wildly from simple flips to many-many augs). Then you aggregate predictions in some way, maybe a simple mean, maybe a geometric mean or voting, whatever</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 279613,
      "author_name": "jeandebleau",
      "author_url": "",
      "post_date": "02/08/2018 11:02:13",
      "content": "<p>96.5:\n256 64*64 crops per image, no tta, no additional data for training, se-resnet-18. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 279622,
      "author_name": "wuzuping",
      "author_url": "",
      "post_date": "02/08/2018 11:36:50",
      "content": "<p>96.6 no TTA</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 279643,
      "author_name": "godaibo",
      "author_url": "",
      "post_date": "02/08/2018 12:57:12",
      "content": "<p>best single model 0.977 with TTA, we saw a big jump by using gleb's additional data and training with all the images good and bad resolution did not bother with selecting images that matches competition resolution, this doubles the additional data and does not adversely affect performance.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 279686,
      "author_name": "",
      "author_url": "",
      "post_date": "02/08/2018 14:31:36",
      "content": "<p>984_tta_8_densenet201_29_0.98624</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 280057,
      "author_name": "skhemka",
      "author_url": "",
      "post_date": "02/09/2018 07:48:58",
      "content": "<p>96.8 no TTA</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 280066,
      "author_name": "arsenyinfo",
      "author_url": "",
      "post_date": "02/09/2018 08:29:49",
      "content": "<p>Mine was 0.983 private / 0.980 public</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 280128,
      "author_name": "matdmiller",
      "author_url": "",
      "post_date": "02/09/2018 12:38:22",
      "content": "<p>What was your best single model based on only the original training dataset provided?  I got .87 public and private leaderboard (3 hours after the competition closed) based 1 model using only the original dataset (-15% for validation).  I used TTA only rotating the images (0,90,180,270 degrees).  This was my first competition so I wasn't sure if using the external data was ok so I stuck to the basics.</p>",
      "votes": null,
      "replies": [
        {
          "id": 280148,
          "author_name": "arsenyinfo",
          "author_url": "",
          "post_date": "02/09/2018 13:26:59",
          "content": "<p>best single model without external data: .951 at public, .958 at private</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 280158,
      "author_name": "mrlzla",
      "author_url": "",
      "post_date": "02/09/2018 13:44:23",
      "content": "<p>Mobilenet 0.972\nDensenet161 0.977 (Chose model 0.976)\nDensenet201 had no time to train :(</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "279547": "What's your best single model? (no ensembles, TTA ok)\n\nMine is 97.2 (no TTA)",
    "279563": "What does TTA mean?",
    "279565": "Test time augmentation: say e.g. flipping doesn't change the label, so at test time you predict both the regular item and the flipped item; then you average/etc.",
    "279571": "If TTA is ok, my best single model is 0.978.",
    "279572": "Can you share how much did you gain using TTA?",
    "279574": "Using TTA, my single model can improve 0.01--0.015.",
    "279576": "That's huge! So w/o TTA you get 0.968?",
    "279613": "96.5:\n256 64*64 crops per image, no tta, no additional data for training, se-resnet-18.",
    "279620": "0.964",
    "279622": "96.6 no TTA",
    "279643": "best single model 0.977 with TTA, we saw a big jump by using gleb's additional data and training with all the images good and bad resolution did not bother with selecting images that matches competition resolution, this doubles the additional data and does not adversely affect performance.",
    "279686": "984_tta_8_densenet201_29_0.98624",
    "279718": "WOW. What TTA are you doing?\n\nMy best no-TTA is 97.2 using a 512x512 crop. Im testing right now:\n\n    TTA_TRANSFORMS = [[], \n                        ['orientation_1'],\n                        ['orientation_3'],\n                        ['manipulation_jpg70'],\n                        ['manipulation_jpg90'],\n                        ['manipulation_gamma0.8'],\n                        ['manipulation_gamma1.2'],\n                        ['manipulation_bicubic1.5'],\n                        ['manipulation_bicubic2.0'],\n                        ['orientation_1', 'manipulation_jpg70'],\n                        ['orientation_1', 'manipulation_jpg90'],\n                        ['orientation_1', 'manipulation_gamma0.8'],\n                        ['orientation_1', 'manipulation_gamma1.2'],\n                        ['orientation_1', 'manipulation_bicubic1.5'],\n                        ['orientation_1', 'manipulation_bicubic2.0'],\n                        ['orientation_3', 'manipulation_jpg70'],\n                        ['orientation_3', 'manipulation_jpg90'],\n                        ['orientation_3', 'manipulation_gamma0.8'],\n                        ['orientation_3', 'manipulation_gamma1.2'],\n                        ['orientation_3', 'manipulation_bicubic1.5'],\n                        ['orientation_3', 'manipulation_bicubic2.0'],\n                        ]\n\nBasically for all images do the original one [] orientation flips (90, 180 and 270 degrees), manipulations (provided the test image is `unalt`) and combinations thereof. Still have not submitted anything.",
    "279739": "With TTA one model @ LB 0.971 gets now LB 0.976.",
    "280008": "Sorry for my ignorance, but can you explain me better the TTA procedure? I've never used this before in my problems. Are you performing several transforms in the testing image? how do you predict its class?",
    "280057": "96.8 no TTA",
    "280066": "Mine was 0.983 private / 0.980 public",
    "280128": "What was your best single model based on only the original training dataset provided?  I got .87 public and private leaderboard (3 hours after the competition closed) based 1 model using only the original dataset (-15% for validation).  I used TTA only rotating the images (0,90,180,270 degrees).  This was my first competition so I wasn't sure if using the external data was ok so I stuck to the basics.",
    "280148": "best single model without external data: .951 at public, .958 at private",
    "280156": "You get several predictions for test sample modified in the same spirit as train augmentations do (usually much less augmentations but it may vary wildly from simple flips to many-many augs). Then you aggregate predictions in some way, maybe a simple mean, maybe a geometric mean or voting, whatever",
    "280158": "Mobilenet 0.972\nDensenet161 0.977 (Chose model 0.976)\nDensenet201 had no time to train :("
  },
  "source": "meta"
}