![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5297211%2Fdab86666087280d5bc0bb331649d5e76%2FScreenshot%20from%202024-05-29%2017-00-32.png?generation=1716994849920422&alt=media\")
Phase 2 ground truth objects are now available in the \"phase_2_data_and_scripts\" folder. Please find the best transform matrix for your submitted objects to align with the ground truth objects. You are required to submit your \"transform.json\" file to our email address (metafood.cvpr@gmail.com) by June 4th, 11:59 PM AOE.
\nOnly the last submission file before the deadline will be considered. If no submission is received, your score will be calculated without any transformation.
\nWe will use the script \"eval_obj_cd_with_transform.py\" in \"phase_2_eval_scripts\" folder to compute your final score. We reserve the rights to check the transformed objects to ensure that the new pose makes common sense (for example, pizza is not flipped).
\nGood luck!
", "messages": [ { "id": "2842263", "postDate": "05/29/2024 02:14:42", "content": "Phase 2 ground truth objects are now available in the \"phase_2_data_and_scripts\" folder. Please find the best transform matrix for your submitted objects to align with the ground truth objects. You are required to submit your \"transform.json\" file to our email address (metafood.cvpr@gmail.com) by June 4th, 11:59 PM AOE.
\nOnly the last submission file before the deadline will be considered. If no submission is received, your score will be calculated without any transformation.
\nWe will use the script \"eval_obj_cd_with_transform.py\" in \"phase_2_eval_scripts\" folder to compute your final score. We reserve the rights to check the transformed objects to ensure that the new pose makes common sense (for example, pizza is not flipped).
\nGood luck!
", "rawMarkdown": "Phase 2 ground truth objects are now available in the \"phase_2_data_and_scripts\" folder. Please find the best transform matrix for your submitted objects to align with the ground truth objects. You are required to submit your \"transform.json\" file to our email address (metafood.cvpr@gmail.com) by June 4th, 11:59 PM AOE. \n\nOnly the last submission file before the deadline will be considered. If no submission is received, your score will be calculated without any transformation. \n\nWe will use the script \"eval_obj_cd_with_transform.py\" in \"phase_2_eval_scripts\" folder to compute your final score. We reserve the rights to check the transformed objects to ensure that the new pose makes common sense (for example, pizza is not flipped). \n\nGood luck!", "votes": null }, { "id": "2843346", "postDate": "05/29/2024 14:25:09", "content": "Hello Organizers,
\nWe want to highlight an observation about the ground truths you provided. According to your Phase 1 Evaluation script, the mesh unit was in meters, but the ground truth meshes you provided are not in the same unit. This causes ICP to fail to register both meshes. Can you please advise us on what we need to do in this case? Should we scale our meshes by 1000 to match both mesh units?
\nLooking forward to hearing from you soon!
", "rawMarkdown": "Hello Organizers,\n\nWe want to highlight an observation about the ground truths you provided. According to your Phase 1 Evaluation script, the mesh unit was in meters, but the ground truth meshes you provided are not in the same unit. This causes ICP to fail to register both meshes. Can you please advise us on what we need to do in this case? Should we scale our meshes by 1000 to match both mesh units? \n\nLooking forward to hearing from you soon!", "votes": null }, { "id": "2843388", "postDate": "05/29/2024 14:48:54", "content": "Hello,
\nI would like to raise some issues with the ground truth meshes provided.
\nFirstly, 3 of the meshes are on a completely different scale (meters vs mm), should we provide the transformation to this scale? If so, the chamfer distance on these meshes would be too small to contribute to the overall metric.
\nSecondly, some of the GT meshes contain noise far away from the actual mesh (e.g. scene 3 and 6). This would significantly influence the chamfer distance.
\nThank you in advance for your answer.
Hello,
\nI noticed that the ground truth mesh for scene 15 doesn't match the input views. The chicken nugget depicted in the input views appears to be round, while the mesh provided has an elongated, rectangular shape, as you can see in the attached image.
Similarly, the ground truth mesh in scene 12 also doesn't seem to match the images.
\n\n", "rawMarkdown": "Similarly, the ground truth mesh in scene 12 also doesn't seem to match the images.\n\n> Hello,\n> \n> I noticed that the ground truth mesh for scene 15 doesn't match the input views. The chicken nugget depicted in the input views appears to be round, while the mesh provided has an elongated, rectangular shape, as you can see in the attached image. ", "votes": null }, { "id": "2843749", "postDate": "05/29/2024 17:36:09", "content": "Hello,
\nI noticed that the ground truth mesh for scene 15 doesn't match the input views. The chicken nugget depicted in the input views appears to be round, while the mesh provided has an elongated, rectangular shape, as you can see in the attached image.
\n
Dear Organizers,
\nIt seems that the evaluation script eval_obj_cd_with_transform.py is having an issue. I ran the Open3D ICP implementation, and I got transformation metrics.
[[ 5.38057017e-02 5.18119883e-01 8.53613925e-01 2.30310909e+01]\n [-4.61814389e-01 -7.45023717e-01 4.81318119e-01 -7.35957148e+01]\n [ 8.85343107e-01 -4.20108852e-01 1.99188693e-01 -2.28830437e+02]\n [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]]\nTo validate, I applied the same transformation metrics using MeshLab, and both meshes are registered.![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8304568%2F1fbcacdb7249f49cded726e78fb58274%2FScreenshot%20from%202024-05-29%2019-31-02.png?generation=1717003881730542&alt=media\")
I took the same transformation metric, used the same evaluation script, and visualized what happened in the meshes, and I got the following:
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8304568%2F902c17555d4ff0756e92935fb2a7f41f%2FScreenshot%20from%202024-05-29%2019-33-40.png?generation=1717004042530471&alt=media\")
Here is a snapshot of the code modification:
\n ():\n (transform_file, ) f:\n transforms = json.load(f)\n\n chamfer_distances = []\n\n i (, ):\n obj_name = \n ground_truth_file = os.path.join(ground_truth_folder, obj_name)\n prediction_file = os.path.join(prediction_folder, obj_name)\n\n os.path.exists(ground_truth_file) os.path.exists(prediction_file):\n pc1 = load_obj_as_pointcloud(ground_truth_file)\n pc2 = load_obj_as_pointcloud(prediction_file)\n\n obj_name transforms:\n transform_matrix = np.array(transforms[obj_name])\n \n pc2 = apply_transformation(pc2, transform_matrix)\n \n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc2)\n\n\n pcd1 = o3d.geometry.PointCloud()\n pcd1.points = o3d.utility.Vector3dVector(pc1)\n pcd1.paint_uniform_color([, , ]) \n pcd.paint_uniform_color([, , ]) \n o3d.visualization.draw_geometries([pcd, pcd1])\n\n\n distance = chamfer_distance(pc1, pc2)\n (obj_name, distance)\n chamfer_distances.append(distance)\n\n average_distance = np.mean(chamfer_distances)\n average_distance\nI think the proper implementation must be something like this:
\n ():\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc)\n t = pcd.transform(transform_matrix)\n np.asarray(t.points)\nPlease make sure to confirm with the organizers.
", "rawMarkdown": "I think the proper implementation must be something like this:\n```python\ndef apply_transformation(pc, transform_matrix):\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc)\n t = pcd.transform(transform_matrix)\n return np.asarray(t.points)\n```\n\nPlease make sure to confirm with the organizers.", "votes": null }, { "id": "2844172", "postDate": "05/30/2024 01:53:20", "content": "Dear Participants,
\nSorry for all the issues associated the phase 2 data. We are currently investigating and will answer your questions by May 30th noon EST.
\nSincerely,
", "rawMarkdown": "Dear Participants,\n\nSorry for all the issues associated the phase 2 data. We are currently investigating and will answer your questions by May 30th noon EST. \n\nSincerely,", "votes": null }, { "id": "2844768", "postDate": "05/30/2024 08:12:48", "content": "Dear organizers,
\nAs we have encountered some challenges that have hindered our progress in this phase, could you please consider extending the deadline for phase 2?
", "rawMarkdown": "Dear organizers, \n\nAs we have encountered some challenges that have hindered our progress in this phase, could you please consider extending the deadline for phase 2?", "votes": null }, { "id": "2845632", "postDate": "05/30/2024 16:02:17", "content": "We discovered an error with scene 15 and were unable to recover the original mesh associated with the scene. Therefore, we will exclude this mesh from the final results. We apologize for any inconvenience this may have caused. Thank you for your understanding. We are investigating the scene 12 mesh to see if its recoverable.
", "rawMarkdown": "We discovered an error with scene 15 and were unable to recover the original mesh associated with the scene. Therefore, we will exclude this mesh from the final results. We apologize for any inconvenience this may have caused. Thank you for your understanding. We are investigating the scene 12 mesh to see if its recoverable.", "votes": null }, { "id": "2845640", "postDate": "05/30/2024 16:07:31", "content": "We verified our code through obtaining transform matrix through Meshlab's project file and the following visualization code:
\nMeshlab output after we manual align:
\n\n \n \n \n \n\n </RenderingOption>\n \n \n \n. . . . -. . -. -. -. . . -. \n\n </RenderingOption>\n \n \n \n\nimport trimesh\nimport numpy np\nimport matplotlib.pyplot plt\nimport argparse\n\ndef load:\n # Load the mesh\n mesh = trimesh.load(file_path)\n # Downsample the mesh it has more vertices than the target\n len(mesh.vertices) > target_vertices:\n mesh = mesh.simplify\n return mesh\n\ndef mesh:\n # Sample points from the surface the mesh\n points, _ = trimesh.sample.sample\n return points\n\ndef apply:\n homogenous_pc = np.hstack((pc, np.ones((pc.shape, ))))\n transformed_pc = homogenous_pc.dot(transform_matrix.T)\n return transformed_pc\n\ndef visualize:\n points, color zip(point_clouds, colors):\n ax.scatter(points, points, points, c=color, s=)\n ax.set\n ax.set\n ax.set\n ax.set\n\ndef parse:\n matrix_list = matrix_str.split()\n matrix = np.()\n return matrix\n\n# Argument parser setup\nparser = argparse.\nparser.add\nparser.add\n\nargs = parser.parse\n\nobj_file_1 = args.obj_file_1\nobj_file_2 = args.obj_file_2\n\n # Replace your actual matrix\ntransform_matrix_1 = np.(, , , ])\ntransform_matrix_2 = np.(, , , ])\n\n# Load the meshes downsampling\ntarget_vertices = # Adjust this number based on the desired level detail\nnum_points = # Number points sample from each mesh\nmesh_1 = load\nmesh_2 = load\n\n# Convert meshes point clouds\npoint_cloud_1 = mesh\npoint_cloud_2 = mesh\n\n# Apply transformations point clouds\ntransformed_point_cloud_1 = apply\ntransformed_point_cloud_2 = apply\n\n# Visualization\nfig = plt.figure(figsize=(, ))\n\nax1 = fig.add\nvisualize\n\nax2 = fig.add\nvisualize\n\nplt.tight\nplt.show\nYou can refer to the following for using meshlab matrix. Note that We also freeze the matrix of the first object.
\nhttps://stackoverflow.com/questions/42964483/meshlab-retrieve-alignment-matrix
Please try and let us know.
", "rawMarkdown": "We verified our code through obtaining transform matrix through Meshlab's project file and the following visualization code:\nMeshlab output after we manual align:\n```\n\nHi sorry for the issue. We have re-examine the meshes, the previous uploaded mesh was not the correct scaled version. The scaled meshes are now available at ground_truth_scaled folder.
", "rawMarkdown": "Hi sorry for the issue. We have re-examine the meshes, the previous uploaded mesh was not the correct scaled version. The scaled meshes are now available at ground_truth_scaled folder.", "votes": null }, { "id": "2845658", "postDate": "05/30/2024 16:12:33", "content": "Hi, the mesh difference are due to an error on our side for not selecting the correct scaled meshes to upload. The scaled meshes are now available at ground_truth_scaled folder. These meshs' volume align to the volume in phase 1.
", "rawMarkdown": "Hi, the mesh difference are due to an error on our side for not selecting the correct scaled meshes to upload. The scaled meshes are now available at ground_truth_scaled folder. These meshs' volume align to the volume in phase 1.", "votes": null }, { "id": "2845793", "postDate": "05/30/2024 17:24:56", "content": "Dear organizers,
\nWe tried the code you posted here but still have the same problem. Below is the figure that the output matches our earlier problem.
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8074822%2Fa2dfab3bd004f048a401934876b93b5a%2FTheir%20Code.png?generation=1717089476447602&alt=media\")
When we apply our fix
\n ():\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc)\n t = pcd.transform(transform_matrix)\n \n \n np.asarray(t.points)\nThe code works like a charm!
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8074822%2F0b9477360b37ea211988b00bac1fac96%2FOur%20Modification.png?generation=1717089676992227&alt=media\")
And here is in 3D:
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8074822%2Fef85570542d2e48cf6da6e73b015e7ab%2Four.gif?generation=1717089724939915&alt=media\")
Please feel free to implement this solution if you are comfortable with it.
", "rawMarkdown": "Dear organizers,\nWe tried the code you posted here but still have the same problem. Below is the figure that the output matches our earlier problem. \n \n\nWhen we apply our fix\n\n```python\ndef apply_transform(pc, transform_matrix):\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc)\n t = pcd.transform(transform_matrix)\n #homogenous_pc = np.hstack((pc, np.ones((pc.shape[0], 1))))\n #transformed_pc = homogenous_pc.dot(transform_matrix.T)\n return np.asarray(t.points)\n```\n\nThe code works like a charm!\n\n\n\nAnd here is in 3D:\n\n\n\nPlease feel free to implement this solution if you are comfortable with it.", "votes": null }, { "id": "2846202", "postDate": "05/31/2024 02:41:28", "content": "Thank you for the update. We will run both your transform and our transform and take the lowest score. We will update the evaluation script to incorporate your changes tomorrow. We are still not sure what is the root cause for the transform not working on your side.
", "rawMarkdown": "Thank you for the update. We will run both your transform and our transform and take the lowest score. We will update the evaluation script to incorporate your changes tomorrow. We are still not sure what is the root cause for the transform not working on your side.", "votes": null }, { "id": "2846205", "postDate": "05/31/2024 02:50:41", "content": "That is up for consideration. We can extend if all 4 teams are okay.
", "rawMarkdown": "That is up for consideration. We can extend if all 4 teams are okay.", "votes": null } ], "comments": [ { "id": 2843346, "author_name": "umairharoon", "author_url": "", "post_date": "05/29/2024 14:25:09", "content": "Hello Organizers,
\nWe want to highlight an observation about the ground truths you provided. According to your Phase 1 Evaluation script, the mesh unit was in meters, but the ground truth meshes you provided are not in the same unit. This causes ICP to fail to register both meshes. Can you please advise us on what we need to do in this case? Should we scale our meshes by 1000 to match both mesh units?
\nLooking forward to hearing from you soon!
", "votes": null, "replies": [ { "id": 2845658, "author_name": "metafoodcvpr", "author_url": "", "post_date": "05/30/2024 16:12:33", "content": "Hi, the mesh difference are due to an error on our side for not selecting the correct scaled meshes to upload. The scaled meshes are now available at ground_truth_scaled folder. These meshs' volume align to the volume in phase 1.
", "votes": null, "replies": [] } ] }, { "id": 2843388, "author_name": "timitoc", "author_url": "", "post_date": "05/29/2024 14:48:54", "content": "Hello,
\nI would like to raise some issues with the ground truth meshes provided.
\nFirstly, 3 of the meshes are on a completely different scale (meters vs mm), should we provide the transformation to this scale? If so, the chamfer distance on these meshes would be too small to contribute to the overall metric.
\nSecondly, some of the GT meshes contain noise far away from the actual mesh (e.g. scene 3 and 6). This would significantly influence the chamfer distance.
\nThank you in advance for your answer.
Hi sorry for the issue. We have re-examine the meshes, the previous uploaded mesh was not the correct scaled version. The scaled meshes are now available at ground_truth_scaled folder.
", "votes": null, "replies": [] } ] }, { "id": 2843415, "author_name": "andreeadogaru", "author_url": "", "post_date": "05/29/2024 15:02:24", "content": "Hello,
\nI noticed that the ground truth mesh for scene 15 doesn't match the input views. The chicken nugget depicted in the input views appears to be round, while the mesh provided has an elongated, rectangular shape, as you can see in the attached image.
Similarly, the ground truth mesh in scene 12 also doesn't seem to match the images.
\n\n", "votes": null, "replies": [] }, { "id": 2845632, "author_name": "metafoodcvpr", "author_url": "", "post_date": "05/30/2024 16:02:17", "content": "Hello,
\nI noticed that the ground truth mesh for scene 15 doesn't match the input views. The chicken nugget depicted in the input views appears to be round, while the mesh provided has an elongated, rectangular shape, as you can see in the attached image.
\n
We discovered an error with scene 15 and were unable to recover the original mesh associated with the scene. Therefore, we will exclude this mesh from the final results. We apologize for any inconvenience this may have caused. Thank you for your understanding. We are investigating the scene 12 mesh to see if its recoverable.
", "votes": null, "replies": [] } ] }, { "id": 2843749, "author_name": "umairharoon", "author_url": "", "post_date": "05/29/2024 17:36:09", "content": "Dear Organizers,
\nIt seems that the evaluation script eval_obj_cd_with_transform.py is having an issue. I ran the Open3D ICP implementation, and I got transformation metrics.
[[ 5.38057017e-02 5.18119883e-01 8.53613925e-01 2.30310909e+01]\n [-4.61814389e-01 -7.45023717e-01 4.81318119e-01 -7.35957148e+01]\n [ 8.85343107e-01 -4.20108852e-01 1.99188693e-01 -2.28830437e+02]\n [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]]\nTo validate, I applied the same transformation metrics using MeshLab, and both meshes are registered.
I took the same transformation metric, used the same evaluation script, and visualized what happened in the meshes, and I got the following:
\n
Here is a snapshot of the code modification:
\n ():\n (transform_file, ) f:\n transforms = json.load(f)\n\n chamfer_distances = []\n\n i (, ):\n obj_name = \n ground_truth_file = os.path.join(ground_truth_folder, obj_name)\n prediction_file = os.path.join(prediction_folder, obj_name)\n\n os.path.exists(ground_truth_file) os.path.exists(prediction_file):\n pc1 = load_obj_as_pointcloud(ground_truth_file)\n pc2 = load_obj_as_pointcloud(prediction_file)\n\n obj_name transforms:\n transform_matrix = np.array(transforms[obj_name])\n \n pc2 = apply_transformation(pc2, transform_matrix)\n \n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc2)\n\n\n pcd1 = o3d.geometry.PointCloud()\n pcd1.points = o3d.utility.Vector3dVector(pc1)\n pcd1.paint_uniform_color([, , ]) \n pcd.paint_uniform_color([, , ]) \n o3d.visualization.draw_geometries([pcd, pcd1])\n\n\n distance = chamfer_distance(pc1, pc2)\n (obj_name, distance)\n chamfer_distances.append(distance)\n\n average_distance = np.mean(chamfer_distances)\n average_distance\nI think the proper implementation must be something like this:
\n ():\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc)\n t = pcd.transform(transform_matrix)\n np.asarray(t.points)\nPlease make sure to confirm with the organizers.
", "votes": null, "replies": [] }, { "id": 2845640, "author_name": "metafoodcvpr", "author_url": "", "post_date": "05/30/2024 16:07:31", "content": "We verified our code through obtaining transform matrix through Meshlab's project file and the following visualization code:
\nMeshlab output after we manual align:
\n\n \n \n \n \n\n </RenderingOption>\n \n \n \n. . . . -. . -. -. -. . . -. \n\n </RenderingOption>\n \n \n \n\nimport trimesh\nimport numpy np\nimport matplotlib.pyplot plt\nimport argparse\n\ndef load:\n # Load the mesh\n mesh = trimesh.load(file_path)\n # Downsample the mesh it has more vertices than the target\n len(mesh.vertices) > target_vertices:\n mesh = mesh.simplify\n return mesh\n\ndef mesh:\n # Sample points from the surface the mesh\n points, _ = trimesh.sample.sample\n return points\n\ndef apply:\n homogenous_pc = np.hstack((pc, np.ones((pc.shape, ))))\n transformed_pc = homogenous_pc.dot(transform_matrix.T)\n return transformed_pc\n\ndef visualize:\n points, color zip(point_clouds, colors):\n ax.scatter(points, points, points, c=color, s=)\n ax.set\n ax.set\n ax.set\n ax.set\n\ndef parse:\n matrix_list = matrix_str.split()\n matrix = np.()\n return matrix\n\n# Argument parser setup\nparser = argparse.\nparser.add\nparser.add\n\nargs = parser.parse\n\nobj_file_1 = args.obj_file_1\nobj_file_2 = args.obj_file_2\n\n # Replace your actual matrix\ntransform_matrix_1 = np.(, , , ])\ntransform_matrix_2 = np.(, , , ])\n\n# Load the meshes downsampling\ntarget_vertices = # Adjust this number based on the desired level detail\nnum_points = # Number points sample from each mesh\nmesh_1 = load\nmesh_2 = load\n\n# Convert meshes point clouds\npoint_cloud_1 = mesh\npoint_cloud_2 = mesh\n\n# Apply transformations point clouds\ntransformed_point_cloud_1 = apply\ntransformed_point_cloud_2 = apply\n\n# Visualization\nfig = plt.figure(figsize=(, ))\n\nax1 = fig.add\nvisualize\n\nax2 = fig.add\nvisualize\n\nplt.tight\nplt.show\nYou can refer to the following for using meshlab matrix. Note that We also freeze the matrix of the first object.
\nhttps://stackoverflow.com/questions/42964483/meshlab-retrieve-alignment-matrix
Please try and let us know.
", "votes": null, "replies": [ { "id": 2845793, "author_name": "ahmadalmughrabi", "author_url": "", "post_date": "05/30/2024 17:24:56", "content": "Dear organizers,
\nWe tried the code you posted here but still have the same problem. Below is the figure that the output matches our earlier problem.
\n
When we apply our fix
\n ():\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc)\n t = pcd.transform(transform_matrix)\n \n \n np.asarray(t.points)\nThe code works like a charm!
\n
And here is in 3D:
\n
Please feel free to implement this solution if you are comfortable with it.
", "votes": null, "replies": [ { "id": 2846202, "author_name": "metafoodcvpr", "author_url": "", "post_date": "05/31/2024 02:41:28", "content": "Thank you for the update. We will run both your transform and our transform and take the lowest score. We will update the evaluation script to incorporate your changes tomorrow. We are still not sure what is the root cause for the transform not working on your side.
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2844172, "author_name": "metafoodcvpr", "author_url": "", "post_date": "05/30/2024 01:53:20", "content": "Dear Participants,
\nSorry for all the issues associated the phase 2 data. We are currently investigating and will answer your questions by May 30th noon EST.
\nSincerely,
", "votes": null, "replies": [ { "id": 2844768, "author_name": "ahmadalmughrabi", "author_url": "", "post_date": "05/30/2024 08:12:48", "content": "Dear organizers,
\nAs we have encountered some challenges that have hindered our progress in this phase, could you please consider extending the deadline for phase 2?
", "votes": null, "replies": [ { "id": 2846205, "author_name": "metafoodcvpr", "author_url": "", "post_date": "05/31/2024 02:50:41", "content": "That is up for consideration. We can extend if all 4 teams are okay.
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2842263": "Phase 2 ground truth objects are now available in the \"phase_2_data_and_scripts\" folder. Please find the best transform matrix for your submitted objects to align with the ground truth objects. You are required to submit your \"transform.json\" file to our email address (metafood.cvpr@gmail.com) by June 4th, 11:59 PM AOE. \n\nOnly the last submission file before the deadline will be considered. If no submission is received, your score will be calculated without any transformation. \n\nWe will use the script \"eval_obj_cd_with_transform.py\" in \"phase_2_eval_scripts\" folder to compute your final score. We reserve the rights to check the transformed objects to ensure that the new pose makes common sense (for example, pizza is not flipped). \n\nGood luck!", "2843346": "Hello Organizers,\n\nWe want to highlight an observation about the ground truths you provided. According to your Phase 1 Evaluation script, the mesh unit was in meters, but the ground truth meshes you provided are not in the same unit. This causes ICP to fail to register both meshes. Can you please advise us on what we need to do in this case? Should we scale our meshes by 1000 to match both mesh units? \n\nLooking forward to hearing from you soon!", "2843388": "Hello,\nI would like to raise some issues with the ground truth meshes provided. \nFirstly, 3 of the meshes are on a completely different scale (meters vs mm), should we provide the transformation to this scale? If so, the chamfer distance on these meshes would be too small to contribute to the overall metric.\nSecondly, some of the GT meshes contain noise far away from the actual mesh (e.g. scene 3 and 6). This would significantly influence the chamfer distance.\nThank you in advance for your answer.", "2843415": "Hello,\n\nI noticed that the ground truth mesh for scene 15 doesn't match the input views. The chicken nugget depicted in the input views appears to be round, while the mesh provided has an elongated, rectangular shape, as you can see in the attached image. ", "2843660": "Similarly, the ground truth mesh in scene 12 also doesn't seem to match the images.\n\n> Hello,\n> \n> I noticed that the ground truth mesh for scene 15 doesn't match the input views. The chicken nugget depicted in the input views appears to be round, while the mesh provided has an elongated, rectangular shape, as you can see in the attached image. ", "2843749": "Dear Organizers,\n\nIt seems that the evaluation script `eval_obj_cd_with_transform.py` is having an issue. I ran the Open3D ICP implementation, and I got transformation metrics. \n```bash\n[[ 5.38057017e-02 5.18119883e-01 8.53613925e-01 2.30310909e+01]\n [-4.61814389e-01 -7.45023717e-01 4.81318119e-01 -7.35957148e+01]\n [ 8.85343107e-01 -4.20108852e-01 1.99188693e-01 -2.28830437e+02]\n [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]]\n```\nTo validate, I applied the same transformation metrics using MeshLab, and both meshes are registered.\n\nI took the same transformation metric, used the same evaluation script, and visualized what happened in the meshes, and I got the following:\n\n\n--- \n\n\nHere is a snapshot of the code modification:\n\n```python\ndef compute_average_chamfer_distance(ground_truth_folder, prediction_folder, transform_file):\n with open(transform_file, 'r') as f:\n transforms = json.load(f)\n\n chamfer_distances = []\n\n for i in range(1, 2):\n obj_name = f\"{i}.obj\"\n ground_truth_file = os.path.join(ground_truth_folder, obj_name)\n prediction_file = os.path.join(prediction_folder, obj_name)\n\n if os.path.exists(ground_truth_file) and os.path.exists(prediction_file):\n pc1 = load_obj_as_pointcloud(ground_truth_file)\n pc2 = load_obj_as_pointcloud(prediction_file)\n\n if obj_name in transforms:\n transform_matrix = np.array(transforms[obj_name])\n #print(transform_matrix)\n pc2 = apply_transformation(pc2, transform_matrix)\n #pc2.export('/mnt/HDD_4TB/Umair/projects/FoodVolumeEstimation/VolETA/data/VolETA/final_meshes/saved.obj')\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc2)\n\t\t\n\t\t\n pcd1 = o3d.geometry.PointCloud()\n pcd1.points = o3d.utility.Vector3dVector(pc1)\n pcd1.paint_uniform_color([1, 0.706, 0]) #Yellow\n pcd.paint_uniform_color([0, 0.651, 0.929]) #Blue\n o3d.visualization.draw_geometries([pcd, pcd1])\n\t\t\n\n distance = chamfer_distance(pc1, pc2)\n print(obj_name, distance)\n chamfer_distances.append(distance)\n\n average_distance = np.mean(chamfer_distances)\n return average_distance\n```", "2843997": "I think the proper implementation must be something like this:\n```python\ndef apply_transformation(pc, transform_matrix):\n pcd = o3d.geometry.PointCloud()\n pcd.points = o3d.utility.Vector3dVector(pc)\n t = pcd.transform(transform_matrix)\n return np.asarray(t.points)\n```\n\nPlease make sure to confirm with the organizers.", "2844172": "Dear Participants,\n\nSorry for all the issues associated the phase 2 data. We are currently investigating and will answer your questions by May 30th noon EST. \n\nSincerely,", "2844768": "Dear organizers, \n\nAs we have encountered some challenges that have hindered our progress in this phase, could you please consider extending the deadline for phase 2?", "2845632": "We discovered an error with scene 15 and were unable to recover the original mesh associated with the scene. Therefore, we will exclude this mesh from the final results. We apologize for any inconvenience this may have caused. Thank you for your understanding. We are investigating the scene 12 mesh to see if its recoverable.", "2845640": "We verified our code through obtaining transform matrix through Meshlab's project file and the following visualization code:\nMeshlab output after we manual align:\n```\n\n