![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F701290%2F2e70103599d4233e367aacd4f54cb3d6%2FScreenshot%20from%202023-04-14%2022-36-39.png?generation=1681492385167099&alt=media\")
Hi all, does flip augment work?. Adding the augment reduces my score. Here is what my code looks like,
\n ():\n flip = tf.transpose(xyz, (,,))\n\n \n flip = tf.constant([, , ]) + flip *tf.constant([-,,])\n\n left_landmarks = tf.gather(flip, LANDMARK_IDXS_LEFT, axis=)\n right_landmarks = tf.gather(flip, LANDMARK_IDXS_RIGHT, axis=)\n\n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_RIGHT[...,], left_landmarks, name=\n )\n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_LEFT[...,], right_landmarks, name=\n )\n flip = tf.transpose(flip, (,,))\n flip\nI tried different centering methods (normalizing to zero mean, centering to shoulder middle, etc), but none of them works. The flipped images looked fine though.
", "messages": [ { "id": "2221899", "postDate": "04/14/2023 17:12:20", "content": "Hi all, does flip augment work?. Adding the augment reduces my score. Here is what my code looks like,
\n ():\n flip = tf.transpose(xyz, (,,))\n\n \n flip = tf.constant([, , ]) + flip *tf.constant([-,,])\n\n left_landmarks = tf.gather(flip, LANDMARK_IDXS_LEFT, axis=)\n right_landmarks = tf.gather(flip, LANDMARK_IDXS_RIGHT, axis=)\n\n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_RIGHT[...,], left_landmarks, name=\n )\n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_LEFT[...,], right_landmarks, name=\n )\n flip = tf.transpose(flip, (,,))\n flip\nI tried different centering methods (normalizing to zero mean, centering to shoulder middle, etc), but none of them works. The flipped images looked fine though.
", "rawMarkdown": "Hi all, does flip augment work?. Adding the augment reduces my score. Here is what my code looks like,\n\n```python\ndef tf_augment_flip(xyz):\n flip = tf.transpose(xyz, (1,0,2))\n\n #mirror image: 1.-xyz[...,0] because center is [0.5,0.5]\n flip = tf.constant([1., 0., 0.]) + flip *tf.constant([-1.,1.,1.])\n \n left_landmarks = tf.gather(flip, LANDMARK_IDXS_LEFT, axis=0)\n right_landmarks = tf.gather(flip, LANDMARK_IDXS_RIGHT, axis=0)\n \n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_RIGHT[...,None], left_landmarks, name=None\n )\n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_LEFT[...,None], right_landmarks, name=None\n )\n flip = tf.transpose(flip, (1,0,2))\n return flip\n```\n\nI tried different centering methods (normalizing to zero mean, centering to shoulder middle, etc), but none of them works. The flipped images looked fine though.", "votes": null }, { "id": "2221902", "postDate": "04/14/2023 17:13:10", "content": "
for me cv improved but not lb
", "rawMarkdown": "for me cv improved but not lb", "votes": null }, { "id": "2222048", "postDate": "04/14/2023 20:12:35", "content": "For me the LB went down by 0.01 with flip. Only augmentation that worked for me is rotation. Even scaling (squeeze/expand along x axis only) improved cv but not lb.
", "rawMarkdown": "For me the LB went down by 0.01 with flip. Only augmentation that worked for me is rotation. Even scaling (squeeze/expand along x axis only) improved cv but not lb.", "votes": null }, { "id": "2222105", "postDate": "04/14/2023 22:33:27", "content": "try:
\naugment\n\nnormalise\npredict\ninstead of
\nnormalise\naugment\n\npredict\nCould you explain on how rotation's augmentation method is implemented? I'm having trouble with the performance after rotation.
", "rawMarkdown": "Could you explain on how rotation's augmentation method is implemented? I'm having trouble with the performance after rotation.", "votes": null }, { "id": "2222194", "postDate": "04/15/2023 03:15:26", "content": "for me flip aug improves CV considerably
", "rawMarkdown": "for me flip aug improves CV considerably", "votes": null }, { "id": "2222207", "postDate": "04/15/2023 03:49:10", "content": "that's what I did, augment (rotate 0.5 prob + flip 0.5 prob), then normalize to zero mean and unit variance, then predict. For flip since the center is (0.5, 0.5), I use the logic tf.abs([1.,0.,0 - xyz). I tried even xyz-[0.5, 0.,0.], then flip by xyz*[-1,1,1] and then convert back to xyz+[0.5,0.,0.]. But all of these decrease the CV by a significant number.
", "rawMarkdown": "that's what I did, augment (rotate 0.5 prob + flip 0.5 prob), then normalize to zero mean and unit variance, then predict. For flip since the center is (0.5, 0.5), I use the logic tf.abs([1.,0.,0 - xyz). I tried even xyz-[0.5, 0.,0.], then flip by xyz*[-1,1,1] and then convert back to xyz+[0.5,0.,0.]. But all of these decrease the CV by a significant number.", "votes": null }, { "id": "2222210", "postDate": "04/15/2023 03:50:18", "content": "Do you find any issue in the above logic I used. it's basically abs([1.0, 0., 0.] - xyz).
", "rawMarkdown": "Do you find any issue in the above logic I used. it's basically abs([1.0, 0., 0.] - xyz).", "votes": null }, { "id": "2222211", "postDate": "04/15/2023 03:53:30", "content": " ():\n () \n theta = tf.random.uniform(shape=(), minval=-rot, maxval=rot)*np.pi/\n xyz = xyz - origin\n t = tf.stack([[tf.math.cos(theta), -tf.math.sin(theta), ],\n [tf.math.sin(theta), tf.math.cos(theta), ],\n [, , ]])\n xyz = tf.matmul(xyz, t, transpose_b=) + origin \n xyz\nWhat values did you use for LANDMARK_IDXS_LEFT and LANDMARK_IDXS_RIGHT?
LIPS_LEFT_IDX0 = np.array([\n , , , , , , , , , \n , , , , , , , , , \n])\nLIPS_RIGHT_IDX0 = np.array([\n , , , , , , , , ,\n , , , , , , , , , \n])\nREYE_IDXS0 = np.array([\n , , , , , , , , ,\n , , , , , , ,\n ])\nLEYE_IDXS0 = np.array([\n , , , , , , , , ,\n , , , , , , ,\n])\nLEFT_POSE_IDXS0 = np.array([, , , , , , , , , , , ])\nRIGHT_POSE_IDXS0 = np.array([, , , , , , , , , , , ]) \n\nLEFT_HAND_IDXS0 = np.arange(,)\nRIGHT_HAND_IDXS0 = np.arange(,)\n\nLANDMARK_IDXS_LEFT = np.concatenate((LEYE_IDXS0, LIPS_LEFT_IDX0, LEFT_HAND_IDXS0, LEFT_POSE_IDXS0))\nLANDMARK_IDXS_RIGHT = np.concatenate((REYE_IDXS0, LIPS_RIGHT_IDX0, RIGHT_HAND_IDXS0, RIGHT_POSE_IDXS0))\n>``\nThis flip implementation gives me a slight boost in CV and LB, like +0.003. But the rotate augmentation is having a negative impact for me.
", "rawMarkdown": "This flip implementation gives me a slight boost in CV and LB, like +0.003. But the rotate augmentation is having a negative impact for me.", "votes": null }, { "id": "2226683", "postDate": "04/19/2023 06:11:43", "content": "you are doing augment -> normalize->optimize , in that order? the rotate i coded below, assumes the center is at [0.5, 0.5, 0]. You should appropriately change the origin in the below code depending on where you do it.
", "rawMarkdown": "you are doing augment -> normalize->optimize , in that order? the rotate i coded below, assumes the center is at [0.5, 0.5, 0]. You should appropriately change the origin in the below code depending on where you do it.", "votes": null }, { "id": "2227727", "postDate": "04/20/2023 01:31:43", "content": "I have an off-topic question :). How much of frames are you using in one preprocessed sequence given the number of above landmarks ? during training are you using the z component or only x and y?
", "rawMarkdown": "I have an off-topic question :). How much of frames are you using in one preprocessed sequence given the number of above landmarks ? during training are you using the z component or only x and y?", "votes": null }, { "id": "2227750", "postDate": "04/20/2023 02:05:08", "content": "I only did a rotation on X before normalization, but LB dropped by 0.1. I think I used too much augmented data for training (1/3 of the original data).
", "rawMarkdown": "I only did a rotation on X before normalization, but LB dropped by 0.1. I think I used too much augmented data for training (1/3 of the original data).", "votes": null }, { "id": "2228007", "postDate": "04/20/2023 07:39:33", "content": "32 is the number of frames (downsampled), with z dropped. I take a subset of the above landmarks- 92 (only lips, hands, pose). A larger model including eyes, more pose points, more frames gives a good CV boost but low LB score for me.
", "rawMarkdown": "32 is the number of frames (downsampled), with z dropped. I take a subset of the above landmarks- 92 (only lips, hands, pose). A larger model including eyes, more pose points, more frames gives a good CV boost but low LB score for me.", "votes": null }, { "id": "2228276", "postDate": "04/20/2023 12:09:24", "content": "Thank you for sharing :)
", "rawMarkdown": "Thank you for sharing :)", "votes": null }, { "id": "2235295", "postDate": "04/25/2023 23:41:58", "content": "why have we to normalize data after augmentation ? By the way I tried horizontal flip and rotation and it does not improve my Lb score.
", "rawMarkdown": "why have we to normalize data after augmentation ? By the way I tried horizontal flip and rotation and it does not improve my Lb score.", "votes": null } ], "comments": [ { "id": 2221902, "author_name": "vijay75", "author_url": "", "post_date": "04/14/2023 17:13:10", "content": "
for me cv improved but not lb
", "votes": null, "replies": [ { "id": 2222048, "author_name": "vijay75", "author_url": "", "post_date": "04/14/2023 20:12:35", "content": "For me the LB went down by 0.01 with flip. Only augmentation that worked for me is rotation. Even scaling (squeeze/expand along x axis only) improved cv but not lb.
", "votes": null, "replies": [ { "id": 2222168, "author_name": "jacksonyou", "author_url": "", "post_date": "04/15/2023 01:59:14", "content": "Could you explain on how rotation's augmentation method is implemented? I'm having trouble with the performance after rotation.
", "votes": null, "replies": [ { "id": 2222211, "author_name": "vijay75", "author_url": "", "post_date": "04/15/2023 03:53:30", "content": " ():\n () \n theta = tf.random.uniform(shape=(), minval=-rot, maxval=rot)*np.pi/\n xyz = xyz - origin\n t = tf.stack([[tf.math.cos(theta), -tf.math.sin(theta), ],\n [tf.math.sin(theta), tf.math.cos(theta), ],\n [, , ]])\n xyz = tf.matmul(xyz, t, transpose_b=) + origin \n xyz\nI only did a rotation on X before normalization, but LB dropped by 0.1. I think I used too much augmented data for training (1/3 of the original data).
", "votes": null, "replies": [] } ] } ] } ] } ] } ] }, { "id": 2222105, "author_name": "hengck23", "author_url": "", "post_date": "04/14/2023 22:33:27", "content": "try:
\naugment\n\nnormalise\npredict\ninstead of
\nnormalise\naugment\n\npredict\nthat's what I did, augment (rotate 0.5 prob + flip 0.5 prob), then normalize to zero mean and unit variance, then predict. For flip since the center is (0.5, 0.5), I use the logic tf.abs([1.,0.,0 - xyz). I tried even xyz-[0.5, 0.,0.], then flip by xyz*[-1,1,1] and then convert back to xyz+[0.5,0.,0.]. But all of these decrease the CV by a significant number.
", "votes": null, "replies": [ { "id": 2235295, "author_name": "idzmanyahya", "author_url": "", "post_date": "04/25/2023 23:41:58", "content": "why have we to normalize data after augmentation ? By the way I tried horizontal flip and rotation and it does not improve my Lb score.
", "votes": null, "replies": [] } ] } ] }, { "id": 2222194, "author_name": "martynoveduard", "author_url": "", "post_date": "04/15/2023 03:15:26", "content": "for me flip aug improves CV considerably
", "votes": null, "replies": [ { "id": 2222210, "author_name": "vijay75", "author_url": "", "post_date": "04/15/2023 03:50:18", "content": "Do you find any issue in the above logic I used. it's basically abs([1.0, 0., 0.] - xyz).
", "votes": null, "replies": [] } ] }, { "id": 2225706, "author_name": "abhinand05", "author_url": "", "post_date": "04/18/2023 11:17:01", "content": "What values did you use for LANDMARK_IDXS_LEFT and LANDMARK_IDXS_RIGHT?
LIPS_LEFT_IDX0 = np.array([\n , , , , , , , , , \n , , , , , , , , , \n])\nLIPS_RIGHT_IDX0 = np.array([\n , , , , , , , , ,\n , , , , , , , , , \n])\nREYE_IDXS0 = np.array([\n , , , , , , , , ,\n , , , , , , ,\n ])\nLEYE_IDXS0 = np.array([\n , , , , , , , , ,\n , , , , , , ,\n])\nLEFT_POSE_IDXS0 = np.array([, , , , , , , , , , , ])\nRIGHT_POSE_IDXS0 = np.array([, , , , , , , , , , , ]) \n\nLEFT_HAND_IDXS0 = np.arange(,)\nRIGHT_HAND_IDXS0 = np.arange(,)\n\nLANDMARK_IDXS_LEFT = np.concatenate((LEYE_IDXS0, LIPS_LEFT_IDX0, LEFT_HAND_IDXS0, LEFT_POSE_IDXS0))\nLANDMARK_IDXS_RIGHT = np.concatenate((REYE_IDXS0, LIPS_RIGHT_IDX0, RIGHT_HAND_IDXS0, RIGHT_POSE_IDXS0))\n>``\nThis flip implementation gives me a slight boost in CV and LB, like +0.003. But the rotate augmentation is having a negative impact for me.
", "votes": null, "replies": [ { "id": 2226683, "author_name": "vijay75", "author_url": "", "post_date": "04/19/2023 06:11:43", "content": "you are doing augment -> normalize->optimize , in that order? the rotate i coded below, assumes the center is at [0.5, 0.5, 0]. You should appropriately change the origin in the below code depending on where you do it.
", "votes": null, "replies": [] } ] }, { "id": 2227727, "author_name": "idzmanyahya", "author_url": "", "post_date": "04/20/2023 01:31:43", "content": "I have an off-topic question :). How much of frames are you using in one preprocessed sequence given the number of above landmarks ? during training are you using the z component or only x and y?
", "votes": null, "replies": [ { "id": 2228007, "author_name": "vijay75", "author_url": "", "post_date": "04/20/2023 07:39:33", "content": "32 is the number of frames (downsampled), with z dropped. I take a subset of the above landmarks- 92 (only lips, hands, pose). A larger model including eyes, more pose points, more frames gives a good CV boost but low LB score for me.
", "votes": null, "replies": [ { "id": 2228276, "author_name": "idzmanyahya", "author_url": "", "post_date": "04/20/2023 12:09:24", "content": "Thank you for sharing :)
", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2221899": "Hi all, does flip augment work?. Adding the augment reduces my score. Here is what my code looks like,\n\n```python\ndef tf_augment_flip(xyz):\n flip = tf.transpose(xyz, (1,0,2))\n\n #mirror image: 1.-xyz[...,0] because center is [0.5,0.5]\n flip = tf.constant([1., 0., 0.]) + flip *tf.constant([-1.,1.,1.])\n \n left_landmarks = tf.gather(flip, LANDMARK_IDXS_LEFT, axis=0)\n right_landmarks = tf.gather(flip, LANDMARK_IDXS_RIGHT, axis=0)\n \n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_RIGHT[...,None], left_landmarks, name=None\n )\n flip = tf.tensor_scatter_nd_update(\n flip, LANDMARK_IDXS_LEFT[...,None], right_landmarks, name=None\n )\n flip = tf.transpose(flip, (1,0,2))\n return flip\n```\n\nI tried different centering methods (normalizing to zero mean, centering to shoulder middle, etc), but none of them works. The flipped images looked fine though.", "2221902": "", "2222031": "for me cv improved but not lb", "2222048": "For me the LB went down by 0.01 with flip. Only augmentation that worked for me is rotation. Even scaling (squeeze/expand along x axis only) improved cv but not lb.", "2222105": "try:\n\n```\naugment\n\nnormalise\npredict\n\n```\n\ninstead of \n```\n\nnormalise\naugment\n\npredict\n\n```", "2222168": "Could you explain on how rotation's augmentation method is implemented? I'm having trouble with the performance after rotation.", "2222194": "for me flip aug improves CV considerably", "2222207": "that's what I did, augment (rotate 0.5 prob + flip 0.5 prob), then normalize to zero mean and unit variance, then predict. For flip since the center is (0.5, 0.5), I use the logic tf.abs([1.,0.,0 - xyz). I tried even xyz-[0.5, 0.,0.], then flip by xyz*[-1,1,1] and then convert back to xyz+[0.5,0.,0.]. But all of these decrease the CV by a significant number.", "2222210": "Do you find any issue in the above logic I used. it's basically abs([1.0, 0., 0.] - xyz).", "2222211": "```python\ndef tf_augment_rotate_xyz(xyz, rot=20., origin=[0.5, 0.5, 0.]):\n print('::: rotate xyz...') \n theta = tf.random.uniform(shape=(), minval=-rot, maxval=rot)*np.pi/180.\n xyz = xyz - origin\n t = tf.stack([[tf.math.cos(theta), -tf.math.sin(theta), 0.],\n [tf.math.sin(theta), tf.math.cos(theta), 0.],\n [0., 0., 1.]])\n xyz = tf.matmul(xyz, t, transpose_b=True) + origin \n return xyz\n\n```", "2225706": "What values did you use for `LANDMARK_IDXS_LEFT` and `LANDMARK_IDXS_RIGHT`?", "2226157": "```python\nLIPS_LEFT_IDX0 = np.array([\n 61, 185, 40, 39, 37, 146, 91, 181, 84, \n 78, 191, 80, 81, 82, 95, 88, 178, 87, \n])\nLIPS_RIGHT_IDX0 = np.array([\n 267, 269, 270, 409, 291, 314, 405, 321, 375,\n 312, 311, 310, 415, 317, 402, 318, 324, 308, \n])\nREYE_IDXS0 = np.array([\n 33, 7, 163, 144, 145, 153, 154, 155, 133,\n 246, 161, 160, 159, 158, 157, 173,\n ])\nLEYE_IDXS0 = np.array([\n 263, 249, 390, 373, 374, 380, 381, 382, 362,\n 466, 388, 387, 386, 385, 384, 398,\n])\nLEFT_POSE_IDXS0 = np.array([490, 491, 492, 496, 498, 500, 502, 504, 506, 508, 510, 512])\nRIGHT_POSE_IDXS0 = np.array([493, 494, 495, 497, 499, 501, 503, 505, 507, 509, 511, 513]) \n\nLEFT_HAND_IDXS0 = np.arange(468,489)\nRIGHT_HAND_IDXS0 = np.arange(522,543)\n\nLANDMARK_IDXS_LEFT = np.concatenate((LEYE_IDXS0, LIPS_LEFT_IDX0, LEFT_HAND_IDXS0, LEFT_POSE_IDXS0))\nLANDMARK_IDXS_RIGHT = np.concatenate((REYE_IDXS0, LIPS_RIGHT_IDX0, RIGHT_HAND_IDXS0, RIGHT_POSE_IDXS0))\n>``\n```", "2226563": "This flip implementation gives me a slight boost in CV and LB, like +0.003. But the rotate augmentation is having a negative impact for me.", "2226683": "you are doing augment -> normalize->optimize , in that order? the rotate i coded below, assumes the center is at [0.5, 0.5, 0]. You should appropriately change the origin in the below code depending on where you do it.", "2227727": "I have an off-topic question :). How much of frames are you using in one preprocessed sequence given the number of above landmarks ? during training are you using the z component or only x and y?", "2227750": "I only did a rotation on X before normalization, but LB dropped by 0.1. I think I used too much augmented data for training (1/3 of the original data).", "2228007": "32 is the number of frames (downsampled), with z dropped. I take a subset of the above landmarks- 92 (only lips, hands, pose). A larger model including eyes, more pose points, more frames gives a good CV boost but low LB score for me.", "2228276": "Thank you for sharing :)", "2235295": "why have we to normalize data after augmentation ? By the way I tried horizontal flip and rotation and it does not improve my Lb score." }, "source": "meta" }