Congrats everyone with excellent result!\nI summarize and write down the part of my solution and our post process.\nFor other part:\ncamaro part: (part of) 7th place solution with code\nJhui He and yelan part: https://www.kaggle.com/c/pku-autonomous-driving/discussion/127034#726362
\n\n.jpg?generation=1579652111438826&alt=media)
detection model\n- Mask RCNN(mask head removed)\n- backbone: resnext101-32x4d\n- lvis pretrained
\n\npose estimation model\n- HRNet-w18c, efficientnetb0/b3\n- imagenet pretrained
\n\nLoss\n- classification: BCE\n- detectoin: Focal Loss\n- pose regression: L1 Loss
\n\ndetection: Optimizer and scheduler\n- optimizer: SGD(lr=0.01, momentum=0.9, weight_decay=1e-4, nesterov=True)\n- scheduler: CosineAnnealingWarmRestarts
\n\npose estimation: Optimizer and scheduler\n- optimizer: Adam(lr=0.0001)\n- scheduler: None
\n\nAugmentation\n- detection: horizontal flip\n- pose estimation: horizontal flip, shift, rotate, random blightness/contrast
\n\nAt first, I train my model on the boxy-vehicle-dataset.\nThis dataset include axis aligned bounding box and 3d cuboids, but I use only 2d bbox.
\ntraining setting:\n- image resolution: 1232x1028\n- epochs: 10\n- batch_size: 4
Model\n- add depth head on top of model
\n\npreprocess\n- split train vs val = 9 vs 1\n- create 3d bbox using label, then create axis aligned bbox\n- depth -> 1 / sigmoid(depth) - 1
\n\ntraining setting\n- image resolution: 800 x 2800, 1400x3300\n- depth loss: L1 Loss\n- epochs: 50\n- batch_size: 4
\n\npreprocess\n- crop image by bbox and resize
\n\ntraining setting\n- image resolution: 320x480\n- epochs: 30\n- batch_size: 128
\n\npublic LB/private LB\n- 800x2800, single fold: 0.119/0.106\n- 1400x3300, single fold: 0.106/0.111
\n\nmodel\n- pytorch dla centernet.\n- regression of yaw_sin, yaw_cos, pitch, depth, 2d bbox size(w, h), 3d bbox size(w, h, l)\n- classification of object centerness(heat map)
\n\nLoss\n- regression: L1 Loss\n- classification: Focal Loss
\n\noptimizer and scheduler\n- optimizer: Adam(lr=5e-4)\n- scheduler: CosineAnnealingLR(lr=5e-5)
\n\nAugmentation\n- pose estimation: horizontal flip, shift, random blightness/contrast
\n\npreprocess\n- depth -> 1 / sigmoid(depth) - 1
\n\ntraining setting\n- split train vs val = 8 vs 2\n- epochs: 30\n- batch_size: 12
\n\npublic LB/ private LB\n- single fold: 0.100/0.096
\n\nWe use different model(faster rcnn, centernet), so it is difficult to ensemble predicitons.\nSo we decided to ensemble nearset points between predictions.\nWe use hungalian algorithm for linear assignment.\nPlease refere to below code.\n```\nfrom scipy.optimize import linear_sum_assignment\ndistance_th = 30\nyaw_th = 10
\n\nsub1 = pd.read_csv('sub1.csv')\ny1 = sub1['PredictionString'].str.split(' ').values\nX1 = sub1['ImageId'].values
\n\nsub2 = pd.read_csv('sub2.csv')\ny2 = sub2['PredictionString'].str.split(' ').values\nX2 = sub2['ImageId'].values\nā\nfor idx in tqdm(range(len(sub1)), position=0):\n if str(np.nan) != str(y1[idx]) and str(np.nan) != str(y2[idx]):\n label1 = np.array(y1[idx]).reshape(-1, 7).astype(float)\n label2 = np.array(y2[idx]).reshape(-1, 7).astype(float)\n center_points1 = get_imgcoords(label1) # [N, 3], (img_x, img_y, img_z)\n center_points2 = get_imgcoords(label2)
\n\n cost_matrix = np.zeros([len(center_points1), len(center_points1)])\n for idx1, i in enumerate(center_points1):\n for idx2, j in enumerate(center_points2):\n cost_matrix[idx1, idx2] = np.linalg.norm(i - j)\n\n match1, match2 = linear_sum_assignment(cost_matrix)\n for i, j in zip(match1, match2):\n if cost_matrix[i, j] < distance_th:\n if np.abs(label1[i][1] - label2[j][1]) < yaw_th:\n label1[j] = (label1[i] + label2[j]) / 2\n label1[idx] = np.concatenate(tmp1).astype(str)\n```
", "messages": [ { "id": 725256, "postDate": "2020-01-22T00:10:49.193Z", "content": "Congrats everyone with excellent result!\nI summarize and write down the part of my solution and our post process.\nFor other part:\ncamaro part: (part of) 7th place solution with code\nJhui He and yelan part: https://www.kaggle.com/c/pku-autonomous-driving/discussion/127034#726362
\n\n.jpg?generation=1579652111438826&alt=media)
detection model\n- Mask RCNN(mask head removed)\n- backbone: resnext101-32x4d\n- lvis pretrained
\n\npose estimation model\n- HRNet-w18c, efficientnetb0/b3\n- imagenet pretrained
\n\nLoss\n- classification: BCE\n- detectoin: Focal Loss\n- pose regression: L1 Loss
\n\ndetection: Optimizer and scheduler\n- optimizer: SGD(lr=0.01, momentum=0.9, weight_decay=1e-4, nesterov=True)\n- scheduler: CosineAnnealingWarmRestarts
\n\npose estimation: Optimizer and scheduler\n- optimizer: Adam(lr=0.0001)\n- scheduler: None
\n\nAugmentation\n- detection: horizontal flip\n- pose estimation: horizontal flip, shift, rotate, random blightness/contrast
\n\nAt first, I train my model on the boxy-vehicle-dataset.\nThis dataset include axis aligned bounding box and 3d cuboids, but I use only 2d bbox.
\ntraining setting:\n- image resolution: 1232x1028\n- epochs: 10\n- batch_size: 4
Model\n- add depth head on top of model
\n\npreprocess\n- split train vs val = 9 vs 1\n- create 3d bbox using label, then create axis aligned bbox\n- depth -> 1 / sigmoid(depth) - 1
\n\ntraining setting\n- image resolution: 800 x 2800, 1400x3300\n- depth loss: L1 Loss\n- epochs: 50\n- batch_size: 4
\n\npreprocess\n- crop image by bbox and resize
\n\ntraining setting\n- image resolution: 320x480\n- epochs: 30\n- batch_size: 128
\n\npublic LB/private LB\n- 800x2800, single fold: 0.119/0.106\n- 1400x3300, single fold: 0.106/0.111
\n\nmodel\n- pytorch dla centernet.\n- regression of yaw_sin, yaw_cos, pitch, depth, 2d bbox size(w, h), 3d bbox size(w, h, l)\n- classification of object centerness(heat map)
\n\nLoss\n- regression: L1 Loss\n- classification: Focal Loss
\n\noptimizer and scheduler\n- optimizer: Adam(lr=5e-4)\n- scheduler: CosineAnnealingLR(lr=5e-5)
\n\nAugmentation\n- pose estimation: horizontal flip, shift, random blightness/contrast
\n\npreprocess\n- depth -> 1 / sigmoid(depth) - 1
\n\ntraining setting\n- split train vs val = 8 vs 2\n- epochs: 30\n- batch_size: 12
\n\npublic LB/ private LB\n- single fold: 0.100/0.096
\n\nWe use different model(faster rcnn, centernet), so it is difficult to ensemble predicitons.\nSo we decided to ensemble nearset points between predictions.\nWe use hungalian algorithm for linear assignment.\nPlease refere to below code.\n```\nfrom scipy.optimize import linear_sum_assignment\ndistance_th = 30\nyaw_th = 10
\n\nsub1 = pd.read_csv('sub1.csv')\ny1 = sub1['PredictionString'].str.split(' ').values\nX1 = sub1['ImageId'].values
\n\nsub2 = pd.read_csv('sub2.csv')\ny2 = sub2['PredictionString'].str.split(' ').values\nX2 = sub2['ImageId'].values\nā\nfor idx in tqdm(range(len(sub1)), position=0):\n if str(np.nan) != str(y1[idx]) and str(np.nan) != str(y2[idx]):\n label1 = np.array(y1[idx]).reshape(-1, 7).astype(float)\n label2 = np.array(y2[idx]).reshape(-1, 7).astype(float)\n center_points1 = get_imgcoords(label1) # [N, 3], (img_x, img_y, img_z)\n center_points2 = get_imgcoords(label2)
\n\n cost_matrix = np.zeros([len(center_points1), len(center_points1)])\n for idx1, i in enumerate(center_points1):\n for idx2, j in enumerate(center_points2):\n cost_matrix[idx1, idx2] = np.linalg.norm(i - j)\n\n match1, match2 = linear_sum_assignment(cost_matrix)\n for i, j in zip(match1, match2):\n if cost_matrix[i, j] < distance_th:\n if np.abs(label1[i][1] - label2[j][1]) < yaw_th:\n label1[j] = (label1[i] + label2[j]) / 2\n label1[idx] = np.concatenate(tmp1).astype(str)\n```
", "rawMarkdown": "Congrats everyone with excellent result!\nI summarize and write down the part of my solution and our post process.\nFor other part:\n[camaro](https://www.kaggle.com/bamps53) part: [(part of) 7th place solution with code](https://www.kaggle.com/c/pku-autonomous-driving/discussion/127056)\n[Jhui He](https://www.kaggle.com/hesene) and [yelan](https://www.kaggle.com/lanjunyelan) part: [https://www.kaggle.com/c/pku-autonomous-driving/discussion/127034#726362](https://www.kaggle.com/c/pku-autonomous-driving/discussion/127034#726362)\n\n\n.jpg?generation=1579652111438826&alt=media)\n\n## Model1: detection and pose estimation\n### common setting\ndetection model\n- Mask RCNN(mask head removed)\n- backbone: resnext101-32x4d\n- lvis pretrained\n\npose estimation model\n- HRNet-w18c, efficientnetb0/b3\n- imagenet pretrained\n\nLoss\n- classification: BCE\n- detectoin: Focal Loss\n- pose regression: L1 Loss\n\ndetection: Optimizer and scheduler\n- optimizer: SGD(lr=0.01, momentum=0.9, weight_decay=1e-4, nesterov=True)\n- scheduler: CosineAnnealingWarmRestarts\n\npose estimation: Optimizer and scheduler\n- optimizer: Adam(lr=0.0001)\n- scheduler: None\n\nAugmentation\n- detection: horizontal flip\n- pose estimation: horizontal flip, shift, rotate, random blightness/contrast\n\n### 1. pretrain on the boxy-vehicle-dataset\nAt first, I train my model on the [boxy-vehicle-dataset](https://boxy-dataset.com/boxy/).\nThis dataset include axis aligned bounding box and 3d cuboids, but I use only 2d bbox.Part of Centernet solution:\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2241982%2Ffc1e876999b387f06bc7edb3d0e92e04%2Fpipline.png?generation=1579742255199041&alt=media\")
\n- Scale->4096*1024 and Filter ignored targets by test mask: Public: 0.101, Private:0.106\n- Pseudo-label and Filter ignored targets by test mask, Public: 0.106, Private:0.110\nAug\n\ntrain_transform = albumentations.Compose([\n albumentations.JpegCompression(quality_lower=99, quality_upper=100,p=0.5),\n albumentations.OneOf([\n albumentations.RandomGamma(gamma_limit=(60, 120), p=0.5),\n albumentations.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0.2, p=0.5),\n ]),\n albumentations.GaussNoise(var_limit=(10, 30), p=0.5),\n albumentations.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225), max_pixel_value=255.0, p=1.0)\n])\n\nDepth Loss\n\npred_depth = (1. / sigmoid_ddd(prediction[:, -1])) - 1\ngt_depth = regr[:, -1]\npos_inds = gt_depth.gt(0).float()\ndepth_loss = (torch.abs(pred_depth - gt_depth) * pos_inds).sum(1).sum(1) / pos_inds.sum(1).sum(1)\ndepth_loss = depth_loss.mean(0)\n\nLearning rate and optimizer\n- Learning rate is 0.0005 in first 15 epoch, Then cycle cosine learning rate trains per 5 epochs, total epoch is 40~60. At the large scale, I use the small scale model to fine-tune with a small learning rate(1e-5)\n- optimizer: Adam
Key point of this solution:\n1. Heatmap(umich_gaussian) and Focal loss(same as https://github.com/xingyizhou/CenterNet)\n2. Depth loss\n3. Bigger scale
\n\nThanks for my teammate's great work and Thanks hocop1's kernel(https://www.kaggle.com/hocop1/centernet-baseline)
", "rawMarkdown": "**Part of Centernet solution:**\n\n- Scale->4096*1024 and Filter ignored targets by test mask: Public: 0.101, Private:0.106\n- Pseudo-label and Filter ignored targets by test mask, Public: 0.106, Private:0.110\n***Aug***\n```\ntrain_transform = albumentations.Compose([\n albumentations.JpegCompression(quality_lower=99, quality_upper=100,p=0.5),\n albumentations.OneOf([\n albumentations.RandomGamma(gamma_limit=(60, 120), p=0.5),\n albumentations.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0.2, p=0.5),\n ]),\n albumentations.GaussNoise(var_limit=(10, 30), p=0.5),\n albumentations.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225), max_pixel_value=255.0, p=1.0)\n])\n```\n***Depth Loss***\n```\npred_depth = (1. / sigmoid_ddd(prediction[:, -1])) - 1\ngt_depth = regr[:, -1]\npos_inds = gt_depth.gt(0).float()\ndepth_loss = (torch.abs(pred_depth - gt_depth) * pos_inds).sum(1).sum(1) / pos_inds.sum(1).sum(1)\ndepth_loss = depth_loss.mean(0)\n```\n***Learning rate and optimizer***\n- Learning rate is 0.0005 in first 15 epoch, Then cycle cosine learning rate trains per 5 epochs, total epoch is 40~60. At the large scale, I use the small scale model to fine-tune with a small learning rate(1e-5)\n- optimizer: Adam\n\n**Key point of this solution:**\n1. Heatmap(umich_gaussian) and Focal loss(same as https://github.com/xingyizhou/CenterNet)\n2. Depth loss\n3. Bigger scale\n\nThanks for my teammate's great work and Thanks hocop1's kernel(https://www.kaggle.com/hocop1/centernet-baseline)", "votes": 5 }, { "id": 728020, "postDate": "2020-01-24T10:40:52.320Z", "content": "Congrats !! Amazing! Thanks for sharing !
", "rawMarkdown": "Congrats !! Amazing! Thanks for sharing !", "votes": 4 }, { "id": 727485, "postDate": "2020-01-23T18:40:46.833Z", "content": "Great solution, thank you so much for sharing!\nMay I ask what do you do if the number of detected cars is not the same for each model?\nI'd say my approach is 1/3 of this one and when I tried to ensemble, many times I had to face that not all the predictions include the same amount of detected cars.\nThanks!
", "rawMarkdown": "Great solution, thank you so much for sharing!\nMay I ask what do you do if the number of detected cars is not the same for each model?\nI'd say my approach is 1/3 of this one and when I tried to ensemble, many times I had to face that not all the predictions include the same amount of detected cars.\nThanks!", "votes": -1, "replies": [ { "id": 727742, "postDate": "2020-01-24T02:15:48.327Z", "content": "Thanks Nanashi!\nWe use hungarian algorithm(cost: euclidian distance) for ensemble, so some predictions is assigned based on cost matrix and the others is ignored.\nIn our pipeline image, some centernet prediction(red points) is not assigned and ignored.
", "rawMarkdown": "Thanks Nanashi!\nWe use hungarian algorithm(cost: euclidian distance) for ensemble, so some predictions is assigned based on cost matrix and the others is ignored.\nIn our pipeline image, some centernet prediction(red points) is not assigned and ignored.", "votes": 1 } ] }, { "id": 726397, "postDate": "2020-01-23T01:53:42.073Z", "content": "Thanks for sharing about the DLA solution! By the way, do you use Adam as the optimizer? And how do you set the learning rate? It seems I used a similar network but didn't know why the net couldn't converge. Thanks for your help!
", "rawMarkdown": "Thanks for sharing about the DLA solution! By the way, do you use Adam as the optimizer? And how do you set the learning rate? It seems I used a similar network but didn't know why the net couldn't converge. Thanks for your help!", "replies": [ { "id": 726411, "postDate": "2020-01-23T02:05:46.257Z", "content": "I update it
", "rawMarkdown": "I update it", "votes": 1 }, { "id": 726412, "postDate": "2020-01-23T02:06:57.680Z", "content": "Many thanks! I'll try this setting! š
", "rawMarkdown": "Many thanks! I'll try this setting! š " } ] }, { "id": 725300, "postDate": "2020-01-22T01:30:25.287Z", "content": "Congratulations! Nice game
", "rawMarkdown": "Congratulations! Nice game" }, { "id": 725279, "postDate": "2020-01-22T00:52:11.893Z", "content": "Congratulations! Thanks for sharing! May I ask the optimizer and the learning rate you use to train centerNet-dla34? I've tried a few times, it never converges. š
", "rawMarkdown": "Congratulations! Thanks for sharing! May I ask the optimizer and the learning rate you use to train centerNet-dla34? I've tried a few times, it never converges. š " }, { "id": 725345, "postDate": "2020-01-22T02:29:15.913Z", "rawMarkdown": "", "isDeleted": true }, { "id": 732537, "postDate": "2020-01-29T23:26:24.077Z", "content": "Congrats and thanks for sharing!
", "rawMarkdown": "Congrats and thanks for sharing!" } ], "comments": [ { "id": 726362, "author_name": "He", "author_url": "", "post_date": "2020-01-23T01:27:09.163000", "content": "Part of Centernet solution:\n\n- Scale->4096*1024 and Filter ignored targets by test mask: Public: 0.101, Private:0.106\n- Pseudo-label and Filter ignored targets by test mask, Public: 0.106, Private:0.110\nAug\n\ntrain_transform = albumentations.Compose([\n albumentations.JpegCompression(quality_lower=99, quality_upper=100,p=0.5),\n albumentations.OneOf([\n albumentations.RandomGamma(gamma_limit=(60, 120), p=0.5),\n albumentations.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0.2, p=0.5),\n ]),\n albumentations.GaussNoise(var_limit=(10, 30), p=0.5),\n albumentations.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225), max_pixel_value=255.0, p=1.0)\n])\n\nDepth Loss\n\npred_depth = (1. / sigmoid_ddd(prediction[:, -1])) - 1\ngt_depth = regr[:, -1]\npos_inds = gt_depth.gt(0).float()\ndepth_loss = (torch.abs(pred_depth - gt_depth) * pos_inds).sum(1).sum(1) / pos_inds.sum(1).sum(1)\ndepth_loss = depth_loss.mean(0)\n\nLearning rate and optimizer\n- Learning rate is 0.0005 in first 15 epoch, Then cycle cosine learning rate trains per 5 epochs, total epoch is 40~60. At the large scale, I use the small scale model to fine-tune with a small learning rate(1e-5)\n- optimizer: Adam
Key point of this solution:\n1. Heatmap(umich_gaussian) and Focal loss(same as https://github.com/xingyizhou/CenterNet)\n2. Depth loss\n3. Bigger scale
\n\nThanks for my teammate's great work and Thanks hocop1's kernel(https://www.kaggle.com/hocop1/centernet-baseline)
", "votes": 5, "replies": [] }, { "id": 728020, "author_name": "", "author_url": "", "post_date": "2020-01-24T10:40:52.320000", "content": "Congrats !! Amazing! Thanks for sharing !
", "votes": 4, "replies": [] }, { "id": 727485, "author_name": "Nanashi", "author_url": "", "post_date": "2020-01-23T18:40:46.833000", "content": "Great solution, thank you so much for sharing!\nMay I ask what do you do if the number of detected cars is not the same for each model?\nI'd say my approach is 1/3 of this one and when I tried to ensemble, many times I had to face that not all the predictions include the same amount of detected cars.\nThanks!
", "votes": -1, "replies": [ { "id": 727742, "author_name": "phalanx", "author_url": "", "post_date": "2020-01-24T02:15:48.327000", "content": "Thanks Nanashi!\nWe use hungarian algorithm(cost: euclidian distance) for ensemble, so some predictions is assigned based on cost matrix and the others is ignored.\nIn our pipeline image, some centernet prediction(red points) is not assigned and ignored.
", "votes": 1, "replies": [] } ] }, { "id": 726397, "author_name": "Helen", "author_url": "", "post_date": "2020-01-23T01:53:42.073000", "content": "Thanks for sharing about the DLA solution! By the way, do you use Adam as the optimizer? And how do you set the learning rate? It seems I used a similar network but didn't know why the net couldn't converge. Thanks for your help!
", "votes": 0, "replies": [ { "id": 726411, "author_name": "He", "author_url": "", "post_date": "2020-01-23T02:05:46.257000", "content": "I update it
", "votes": 1, "replies": [] }, { "id": 726412, "author_name": "Helen", "author_url": "", "post_date": "2020-01-23T02:06:57.680000", "content": "Many thanks! I'll try this setting! š
", "votes": 0, "replies": [] } ] }, { "id": 725300, "author_name": "Tsai29", "author_url": "", "post_date": "2020-01-22T01:30:25.287000", "content": "Congratulations! Nice game
", "votes": 0, "replies": [] }, { "id": 725279, "author_name": "Helen", "author_url": "", "post_date": "2020-01-22T00:52:11.893000", "content": "Congratulations! Thanks for sharing! May I ask the optimizer and the learning rate you use to train centerNet-dla34? I've tried a few times, it never converges. š
", "votes": 0, "replies": [] }, { "id": 725345, "author_name": "", "author_url": "", "post_date": "2020-01-22T02:29:15.913000", "content": "", "votes": 0, "replies": [] }, { "id": 732537, "author_name": "corochann", "author_url": "", "post_date": "2020-01-29T23:26:24.077000", "content": "Congrats and thanks for sharing!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "725256": "Congrats everyone with excellent result!\nI summarize and write down the part of my solution and our post process.\nFor other part:\n[camaro](https://www.kaggle.com/bamps53) part: [(part of) 7th place solution with code](https://www.kaggle.com/c/pku-autonomous-driving/discussion/127056)\n[Jhui He](https://www.kaggle.com/hesene) and [yelan](https://www.kaggle.com/lanjunyelan) part: [https://www.kaggle.com/c/pku-autonomous-driving/discussion/127034#726362](https://www.kaggle.com/c/pku-autonomous-driving/discussion/127034#726362)\n\n\n.jpg?generation=1579652111438826&alt=media)\n\n## Model1: detection and pose estimation\n### common setting\ndetection model\n- Mask RCNN(mask head removed)\n- backbone: resnext101-32x4d\n- lvis pretrained\n\npose estimation model\n- HRNet-w18c, efficientnetb0/b3\n- imagenet pretrained\n\nLoss\n- classification: BCE\n- detectoin: Focal Loss\n- pose regression: L1 Loss\n\ndetection: Optimizer and scheduler\n- optimizer: SGD(lr=0.01, momentum=0.9, weight_decay=1e-4, nesterov=True)\n- scheduler: CosineAnnealingWarmRestarts\n\npose estimation: Optimizer and scheduler\n- optimizer: Adam(lr=0.0001)\n- scheduler: None\n\nAugmentation\n- detection: horizontal flip\n- pose estimation: horizontal flip, shift, rotate, random blightness/contrast\n\n### 1. pretrain on the boxy-vehicle-dataset\nAt first, I train my model on the [boxy-vehicle-dataset](https://boxy-dataset.com/boxy/).\nThis dataset include axis aligned bounding box and 3d cuboids, but I use only 2d bbox.