{ "id": 256604, "title": "Do I need \"tta steps\" if my p=1.?", "url": "/competitions/seti-breakthrough-listen/discussion/256604", "author_name": "gao-hongnan", "post_date": "2021-08-02T13:37:18.335000", "votes": 6, "comment_count": 4, "views": 0, "content": "

I do understand there are many tutorials of tta, and samples of it, but it seems there are quite a few ways to perform tta.

\n

I am one who always uses p=1. in my tta augs. For eg, if I use hflip tta, then I ensure that every batch the image is flipped (along with original).

\n

In this case, there is no need to do \"tta steps\", because from what I understand, tta steps is done to kind of reduce variance on each batch/epoch.

\n

Is my logic wrong?

", "messages": [ { "id": 1408516, "postDate": "2021-08-02T13:37:18.337Z", "content": "

I do understand there are many tutorials of tta, and samples of it, but it seems there are quite a few ways to perform tta.

\n

I am one who always uses p=1. in my tta augs. For eg, if I use hflip tta, then I ensure that every batch the image is flipped (along with original).

\n

In this case, there is no need to do \"tta steps\", because from what I understand, tta steps is done to kind of reduce variance on each batch/epoch.

\n

Is my logic wrong?

", "rawMarkdown": "I do understand there are many tutorials of tta, and samples of it, but it seems there are quite a few ways to perform tta.\n\nI am one who always uses `p=1.` in my tta augs. For eg, if I use `hflip` tta, then I ensure that every batch the image is flipped (along with original).\n\nIn this case, there is no need to do \"tta steps\", because from what I understand, tta steps is done to kind of reduce variance on each batch/epoch.\n\nIs my logic wrong?", "votes": 6 }, { "id": 1411328, "postDate": "2021-08-02T18:38:13.630Z", "content": "

This is the way I do it also. Using horizonal and vertical flips with p = 1 always denotes a constant image result.
\nBut there are augmentations like RandomResizedCrop for example, which can be used for TTA and have a random factor. By using them in a repeated methodology(multiple times) we get different result each time. You can use this kind of augmentations in 10 successive iterations for example and average the results.

", "rawMarkdown": "This is the way I do it also. Using horizonal and vertical flips with p = 1 always denotes a constant image result.\nBut there are augmentations like RandomResizedCrop for example, which can be used for TTA and have a random factor. By using them in a repeated methodology(multiple times) we get different result each time. You can use this kind of augmentations in 10 successive iterations for example and average the results.", "votes": 2, "replies": [ { "id": 1411631, "postDate": "2021-08-02T18:55:07.467Z", "content": "

@vladvdv can you please tell me how much increase you got with using tta?

", "rawMarkdown": "@vladvdv can you please tell me how much increase you got with using tta?", "votes": 1 }, { "id": 1430649, "postDate": "2021-08-03T11:51:52.733Z", "content": "

@vladvdv Thanks, this is insightful. I do realize that empirically, RandomResizedCrop is a very great augmentation that improves generalization.

\n

In this case, how should one do it given the following scenario:

\n

TTA_AUGS = [hflip, vflip, randomresizedcrop]

\n

What I think of doing is the follows:

\n
    \n
  1. Inference with no augs -> raw preds.
    2. \n
  2. Inference with hflip and vflip -> hflip preds and vflip preds. Note at this point of time, we have 3 snapshots of predictions (i.e. 3 preds array).
    3. \n
  3. For the randomresizedcrop, say I use a tta_step = 5, then this part is where confusion comes in. I perform inference 3 times using this augmentation -> 5 arrays -> then do I average these 5 arrays and get the ensembled preds for randomresizedcrop (call it rrc_preds). Then do I just (raw + hflip + vflip + rrc_preds)/4?
    4. \n
", "rawMarkdown": "@vladvdv Thanks, this is insightful. I do realize that empirically, RandomResizedCrop is a very great augmentation that improves generalization.\n\nIn this case, how should one do it given the following scenario:\n\nTTA_AUGS = [hflip, vflip, randomresizedcrop]\n\nWhat I think of doing is the follows:\n\n1. Inference with no augs -> raw preds.\n2. Inference with hflip and vflip -> hflip preds and vflip preds. Note at this point of time, we have 3 snapshots of predictions (i.e. 3 preds array).\n3. For the randomresizedcrop, say I use a tta_step = 5, then this part is where confusion comes in. I perform inference 3 times using this augmentation -> 5 arrays -> then do I average these 5 arrays and get the ensembled preds for randomresizedcrop (call it rrc_preds). Then do I just (raw + hflip + vflip + rrc_preds)/4?\n\n\n" }, { "id": 1432096, "postDate": "2021-08-03T13:00:35.857Z", "content": "

It is not a strict rule. You can juggle around with the augmentation on TTA.
\nYou can try to do X times the following procedure:

\n\n

And them average the result on the X tries

", "rawMarkdown": "It is not a strict rule. You can juggle around with the augmentation on TTA.\nYou can try to do X times the following procedure:\n- inference on raw data\n- inference on raw data with randomresizecrop\n- inference on horizontal flip with randomresizecrop\n- inference on vertical flip with randomresizecrop\n- average the results of the above 4\n\nAnd them average the result on the X tries \n", "votes": 2 } ] } ], "comments": [ { "id": 1411328, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2021-08-02T18:38:13.630000", "content": "

This is the way I do it also. Using horizonal and vertical flips with p = 1 always denotes a constant image result.
\nBut there are augmentations like RandomResizedCrop for example, which can be used for TTA and have a random factor. By using them in a repeated methodology(multiple times) we get different result each time. You can use this kind of augmentations in 10 successive iterations for example and average the results.

", "votes": 2, "replies": [ { "id": 1411631, "author_name": "Ahmed Sabry", "author_url": "", "post_date": "2021-08-02T18:55:07.467000", "content": "

@vladvdv can you please tell me how much increase you got with using tta?

", "votes": 1, "replies": [] }, { "id": 1430649, "author_name": "gao-hongnan", "author_url": "", "post_date": "2021-08-03T11:51:52.733000", "content": "

@vladvdv Thanks, this is insightful. I do realize that empirically, RandomResizedCrop is a very great augmentation that improves generalization.

\n

In this case, how should one do it given the following scenario:

\n

TTA_AUGS = [hflip, vflip, randomresizedcrop]

\n

What I think of doing is the follows:

\n
    \n
  1. Inference with no augs -> raw preds.
    2. \n
  2. Inference with hflip and vflip -> hflip preds and vflip preds. Note at this point of time, we have 3 snapshots of predictions (i.e. 3 preds array).
    3. \n
  3. For the randomresizedcrop, say I use a tta_step = 5, then this part is where confusion comes in. I perform inference 3 times using this augmentation -> 5 arrays -> then do I average these 5 arrays and get the ensembled preds for randomresizedcrop (call it rrc_preds). Then do I just (raw + hflip + vflip + rrc_preds)/4?
    4. \n
", "votes": 0, "replies": [] }, { "id": 1432096, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2021-08-03T13:00:35.857000", "content": "

It is not a strict rule. You can juggle around with the augmentation on TTA.
\nYou can try to do X times the following procedure:

\n\n

And them average the result on the X tries

", "votes": 2, "replies": [] } ] } ], "raw_markdown_by_id": { "1408516": "I do understand there are many tutorials of tta, and samples of it, but it seems there are quite a few ways to perform tta.\n\nI am one who always uses `p=1.` in my tta augs. For eg, if I use `hflip` tta, then I ensure that every batch the image is flipped (along with original).\n\nIn this case, there is no need to do \"tta steps\", because from what I understand, tta steps is done to kind of reduce variance on each batch/epoch.\n\nIs my logic wrong?", "1411328": "This is the way I do it also. Using horizonal and vertical flips with p = 1 always denotes a constant image result.\nBut there are augmentations like RandomResizedCrop for example, which can be used for TTA and have a random factor. By using them in a repeated methodology(multiple times) we get different result each time. You can use this kind of augmentations in 10 successive iterations for example and average the results." } }