I'm running into a strange problem that one of my notebooks is running fine, but failing during the scorring phase and in the submission. The notebooks run fine and generate the submission.csv with the correct fields and values (compared to a successful earlier submission I've done), but it says Failed: Notebook Threw Exception. When I open the run I can see all cells ran without any exceptions and the output was successfully generated. I looked at the logs and couldn't see any exceptions either and it even says: \"Successfully ran in …\". Can someone help me with what might be the issue?
", "messages": [ { "id": 2736841, "postDate": "2024-04-05T12:34:38.293Z", "content": "I'm running into a strange problem that one of my notebooks is running fine, but failing during the scorring phase and in the submission. The notebooks run fine and generate the submission.csv with the correct fields and values (compared to a successful earlier submission I've done), but it says Failed: Notebook Threw Exception. When I open the run I can see all cells ran without any exceptions and the output was successfully generated. I looked at the logs and couldn't see any exceptions either and it even says: \"Successfully ran in …\". Can someone help me with what might be the issue?
", "rawMarkdown": "I'm running into a strange problem that one of my notebooks is running fine, but failing during the scorring phase and in the submission. The notebooks run fine and generate the submission.csv with the correct fields and values (compared to a successful earlier submission I've done), but it says Failed: Notebook Threw Exception. When I open the run I can see all cells ran without any exceptions and the output was successfully generated. I looked at the logs and couldn't see any exceptions either and it even says: \"Successfully ran in ...\". Can someone help me with what might be the issue?", "votes": 7 }, { "id": 2792025, "postDate": "2024-05-04T03:29:12.170Z", "content": "Why did I run the code without task issues, the commit file was generated normally, I also checked the commit file. But in the end, it shows a scoring error, how should this be solved, does anyone have a similar situation?
", "rawMarkdown": "Why did I run the code without task issues, the commit file was generated normally, I also checked the commit file. But in the end, it shows a scoring error, how should this be solved, does anyone have a similar situation?" }, { "id": 2788627, "postDate": "2024-05-02T10:15:23.103Z", "content": "I had same issue. Notebook Threw Exception error occurred several minutes after submission.
\nIt seems that there are cases where the test data does not have the directory structure test/{dataset}/images.
\nWhen I used the image_path of sample_submission instead of it, the error disappeared.
For all people running into issues with submissions.
\nUse the training set to give your notebook a more extensive test if it will work. Limit the amount of images to 100 per scene.
\nI also ran into some issues early on and was able to solve them by using the training set. The organizers stated somewhere that we can expect the hidden test set to be around 1000 images I believe. So the training set is better to 'test' with than the public test set.
", "rawMarkdown": "For all people running into issues with submissions. \n\nUse the training set to give your notebook a more extensive test if it will work. Limit the amount of images to 100 per scene.\n\nI also ran into some issues early on and was able to solve them by using the training set. The organizers stated somewhere that we can expect the hidden test set to be around 1000 images I believe. So the training set is better to 'test' with than the public test set.", "replies": [ { "id": 2766104, "postDate": "2024-04-21T13:51:13.440Z", "content": "@rsmits Sure, will try it out. Thanks for info.
", "rawMarkdown": "@rsmits Sure, will try it out. Thanks for info.", "votes": 1, "isDeleted": true, "replies": [ { "id": 2778205, "postDate": "2024-04-27T02:58:59.280Z", "content": "The following worked out for me:
\nIssue - \"Submission Scoring Error\"
\nOriginal \"create_submission\" function:
\n ():\n (, ) f:\n f.write()\n dataset data_dict:\n dataset out_results:\n res = out_results[dataset]\n :\n res = {}\n scene data_dict[dataset]:\n scene res:\n scene_res = res[scene]\n :\n scene_res = {:{}, :{}}\n image data_dict[dataset][scene]:\n image scene_res:\n (image)\n R = scene_res[image][].reshape(-)\n T = scene_res[image][].reshape(-)\n :\n R = np.eye().reshape(-)\n T = np.zeros(())\n f.write()\nModified \"create_submission\" function:
\n ():\n df = pd.read_csv()\n (, ) f:\n f.write()\n dataset data_dict:\n dataset out_results:\n res = out_results[dataset]\n :\n res = {}\n scene data_dict[dataset]:\n scene res:\n scene_res = res[scene]\n :\n scene_res = {:{}, :{}}\n image data_dict[dataset][scene]:\n prev_image = image\n image = image.split()[-]\n image scene_res:\n R = scene_res[image][].reshape(-)\n T = scene_res[image][].reshape(-)\n :\n R = np.eye().reshape(-)\n T = np.zeros(())\n (image df[].tolist()):\n f.write()\n :\n f.write()\nDescription: If you notice in the given dataset, the 'image_path' format in sample_submission.csv and train_labels.csv could be different. So, no matter how you process the dataset, the final format of 'image_path' in submission.csv should match that of sample_submission.csv. This eliminated the above error.
\nHope it helps someone.
", "rawMarkdown": "The following worked out for me:\n\nIssue - \"Submission Scoring Error\"\n\nOriginal \"create_submission\" function:\n\n```\ndef create_submission(out_results, data_dict):\n with open(f'submission.csv', 'w') as f:\n f.write('image_path,dataset,scene,rotation_matrix,translation_vector\\n')\n for dataset in data_dict:\n if dataset in out_results:\n res = out_results[dataset]\n else:\n res = {}\n for scene in data_dict[dataset]:\n if scene in res:\n scene_res = res[scene]\n else:\n scene_res = {\"R\":{}, \"t\":{}}\n for image in data_dict[dataset][scene]:\n if image in scene_res:\n print (image)\n R = scene_res[image]['R'].reshape(-1)\n T = scene_res[image]['t'].reshape(-1)\n else:\n R = np.eye(3).reshape(-1)\n T = np.zeros((3))\n f.write(f'{image},{dataset},{scene},{arr_to_str(R)},{arr_to_str(T)}\\n')\n```\n\nModified \"create_submission\" function:\n\n```\ndef create_submission(out_results, data_dict):\n df = pd.read_csv(f'/kaggle/input/image-matching-challenge-2024/sample_submission.csv')\n with open(f'/kaggle/working/submission.csv', 'w') as f:\n f.write('image_path,dataset,scene,rotation_matrix,translation_vector\\n')\n for dataset in data_dict:\n if dataset in out_results:\n res = out_results[dataset]\n else:\n res = {}\n for scene in data_dict[dataset]:\n if scene in res:\n scene_res = res[scene]\n else:\n scene_res = {\"R\":{}, \"t\":{}}\n for image in data_dict[dataset][scene]:\n prev_image = image\n image = image.split('/')[-1]\n if image in scene_res:\n R = scene_res[image]['R'].reshape(-1)\n T = scene_res[image]['t'].reshape(-1)\n else:\n R = np.eye(3).reshape(-1)\n T = np.zeros((3))\n if(image not in df['image_path'].tolist()):\n f.write(f'{prev_image},{dataset},{scene},{arr_to_str(R)},{arr_to_str(T)}\\n')\n else:\n f.write(f'{image},{dataset},{scene},{arr_to_str(R)},{arr_to_str(T)}\\n')\n```\n\nDescription: If you notice in the given dataset, the 'image_path' format in sample_submission.csv and train_labels.csv could be different. So, no matter how you process the dataset, the final format of 'image_path' in submission.csv should match that of sample_submission.csv. This eliminated the above error. \n\nHope it helps someone. ", "votes": 1, "isDeleted": true } ] } ] }, { "id": 2763681, "postDate": "2024-04-20T17:15:36.680Z", "content": "Same problem.Could you help here? @oldufo
", "rawMarkdown": "Same problem.Could you help here? @oldufo " }, { "id": 2750018, "postDate": "2024-04-13T12:06:47.897Z", "content": "I am having the same error, the notebooks runs smoothly, the submission.csv is generated, but scoring fails!
\n@oldufo could you help here?
Same issue as
\nWhen I am trying to install flaml it shows already installed but when I am trying to save my notebook same error not version like error happening
\nDo you know why ?
\n@ddimanov
I am facing similar issue. Were you able to find the reason?
", "rawMarkdown": "I am facing similar issue. Were you able to find the reason?", "votes": 2, "isDeleted": true } ], "comments": [ { "id": 2792025, "author_name": "huangxiao309", "author_url": "", "post_date": "2024-05-04T03:29:12.170000", "content": "Why did I run the code without task issues, the commit file was generated normally, I also checked the commit file. But in the end, it shows a scoring error, how should this be solved, does anyone have a similar situation?
", "votes": 0, "replies": [] }, { "id": 2788627, "author_name": "Ebi", "author_url": "", "post_date": "2024-05-02T10:15:23.103000", "content": "I had same issue. Notebook Threw Exception error occurred several minutes after submission.
\nIt seems that there are cases where the test data does not have the directory structure test/{dataset}/images.
\nWhen I used the image_path of sample_submission instead of it, the error disappeared.
For all people running into issues with submissions.
\nUse the training set to give your notebook a more extensive test if it will work. Limit the amount of images to 100 per scene.
\nI also ran into some issues early on and was able to solve them by using the training set. The organizers stated somewhere that we can expect the hidden test set to be around 1000 images I believe. So the training set is better to 'test' with than the public test set.
", "votes": 0, "replies": [ { "id": 2766104, "author_name": "", "author_url": "", "post_date": "2024-04-21T13:51:13.440000", "content": "@rsmits Sure, will try it out. Thanks for info.
", "votes": 1, "replies": [ { "id": 2778205, "author_name": "", "author_url": "", "post_date": "2024-04-27T02:58:59.280000", "content": "The following worked out for me:
\nIssue - \"Submission Scoring Error\"
\nOriginal \"create_submission\" function:
\n ():\n (, ) f:\n f.write()\n dataset data_dict:\n dataset out_results:\n res = out_results[dataset]\n :\n res = {}\n scene data_dict[dataset]:\n scene res:\n scene_res = res[scene]\n :\n scene_res = {:{}, :{}}\n image data_dict[dataset][scene]:\n image scene_res:\n (image)\n R = scene_res[image][].reshape(-)\n T = scene_res[image][].reshape(-)\n :\n R = np.eye().reshape(-)\n T = np.zeros(())\n f.write()\nModified \"create_submission\" function:
\n ():\n df = pd.read_csv()\n (, ) f:\n f.write()\n dataset data_dict:\n dataset out_results:\n res = out_results[dataset]\n :\n res = {}\n scene data_dict[dataset]:\n scene res:\n scene_res = res[scene]\n :\n scene_res = {:{}, :{}}\n image data_dict[dataset][scene]:\n prev_image = image\n image = image.split()[-]\n image scene_res:\n R = scene_res[image][].reshape(-)\n T = scene_res[image][].reshape(-)\n :\n R = np.eye().reshape(-)\n T = np.zeros(())\n (image df[].tolist()):\n f.write()\n :\n f.write()\nDescription: If you notice in the given dataset, the 'image_path' format in sample_submission.csv and train_labels.csv could be different. So, no matter how you process the dataset, the final format of 'image_path' in submission.csv should match that of sample_submission.csv. This eliminated the above error.
\nHope it helps someone.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2763681, "author_name": "Neo", "author_url": "", "post_date": "2024-04-20T17:15:36.680000", "content": "Same problem.Could you help here? @oldufo
", "votes": 0, "replies": [] }, { "id": 2750018, "author_name": "Danial Zakaria", "author_url": "", "post_date": "2024-04-13T12:06:47.897000", "content": "I am having the same error, the notebooks runs smoothly, the submission.csv is generated, but scoring fails!
\n@oldufo could you help here?
Same issue as
\nWhen I am trying to install flaml it shows already installed but when I am trying to save my notebook same error not version like error happening
\nDo you know why ?
\n@ddimanov
I am facing similar issue. Were you able to find the reason?
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "2736841": "I'm running into a strange problem that one of my notebooks is running fine, but failing during the scorring phase and in the submission. The notebooks run fine and generate the submission.csv with the correct fields and values (compared to a successful earlier submission I've done), but it says Failed: Notebook Threw Exception. When I open the run I can see all cells ran without any exceptions and the output was successfully generated. I looked at the logs and couldn't see any exceptions either and it even says: \"Successfully ran in ...\". Can someone help me with what might be the issue?", "2792025": "Why did I run the code without task issues, the commit file was generated normally, I also checked the commit file. But in the end, it shows a scoring error, how should this be solved, does anyone have a similar situation?", "2788627": "I had same issue. Notebook Threw Exception error occurred several minutes after submission. \nIt seems that there are cases where the test data does not have the directory structure `test/{dataset}/images`.\nWhen I used the image_path of sample_submission instead of it, the error disappeared.", "2766096": "For all people running into issues with submissions. \n\nUse the training set to give your notebook a more extensive test if it will work. Limit the amount of images to 100 per scene.\n\nI also ran into some issues early on and was able to solve them by using the training set. The organizers stated somewhere that we can expect the hidden test set to be around 1000 images I believe. So the training set is better to 'test' with than the public test set.", "2763681": "Same problem.Could you help here? @oldufo ", "2750018": "I am having the same error, the notebooks runs smoothly, the submission.csv is generated, but scoring fails!\n@oldufo could you help here?", "2743619": "Same issue as\nWhen I am trying to install flaml it shows already installed but when I am trying to save my notebook same error not version like error happening\nDo you know why ? \n@ddimanov ", "2743588": "I am facing similar issue. Were you able to find the reason?" } }