{ "id": 408055, "title": "About the submission score error", "url": "/competitions/image-matching-challenge-2023/discussion/408055", "author_name": "imcBEST", "post_date": "2023-05-09T09:05:36.125000", "votes": 6, "comment_count": 22, "views": 0, "content": "

My submissions are always score error even with the example submission

", "messages": [ { "id": 2251333, "postDate": "2023-05-09T09:05:36.127Z", "content": "

My submissions are always score error even with the example submission

", "rawMarkdown": "My submissions are always score error even with the example submission", "votes": 5 }, { "id": 2251362, "postDate": "2023-05-09T09:21:10.153Z", "content": "

when i only changed into 'loftr',the submission is error

", "rawMarkdown": "when i only changed into 'loftr',the submission is error", "replies": [ { "id": 2251390, "postDate": "2023-05-09T09:40:43.307Z", "content": "

Did you try with DISK?

", "rawMarkdown": "Did you try with DISK?", "replies": [ { "id": 2251466, "postDate": "2023-05-09T11:20:05.627Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 2251483, "postDate": "2023-05-09T11:44:57.743Z", "content": "

There is no \"match_ensemble\" in the submission notebook (the comment says \"Can be LoFTR, KeyNetAffNetHardNet, or DISK\"). What are you looking at?

", "rawMarkdown": "There is no \"match_ensemble\" in the submission notebook (the comment says \"Can be LoFTR, KeyNetAffNetHardNet, or DISK\"). What are you looking at?" }, { "id": 2251496, "postDate": "2023-05-09T12:05:09.947Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2251497, "postDate": "2023-05-09T12:05:37.683Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2251504, "postDate": "2023-05-09T12:13:54.050Z", "content": "

I have runs with that notebook using LoFTR that finished just fine, while several people reported errors. We don't know why: it could be memory issues or a crash. The loop in the notebook contains a try-except-pass block, so small trivial mistakes there should not trigger a crash.

\n

If the notebook consistently works on the training data but crashes on the test data, try a different model. Also if you're not using a P100, you should.

", "rawMarkdown": "I have runs with that notebook using LoFTR that finished just fine, while several people reported errors. We don't know why: it could be memory issues or a crash. The loop in the notebook contains a try-except-pass block, so small trivial mistakes there should not trigger a crash.\n\nIf the notebook consistently works on the training data but crashes on the test data, try a different model. Also if you're not using a P100, you should." }, { "id": 2251506, "postDate": "2023-05-09T12:21:46.347Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2251512, "postDate": "2023-05-09T12:28:12.443Z", "content": "

Is \"a long time\" close to 9h, which is the limit?

", "rawMarkdown": "Is \"a long time\" close to 9h, which is the limit?" }, { "id": 2251514, "postDate": "2023-05-09T12:37:35.960Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2251527, "postDate": "2023-05-09T12:44:31.103Z", "content": "

How long did your code run when using LoFTR?

", "rawMarkdown": "How long did your code run when using LoFTR?" }, { "id": 2251544, "postDate": "2023-05-09T13:05:36.403Z", "content": "

That's probably an occasional crash. You might want to try reducing image size. I agree that it's difficult to debug when you don't see the error, but that's part of the challenge.

", "rawMarkdown": "That's probably an occasional crash. You might want to try reducing image size. I agree that it's difficult to debug when you don't see the error, but that's part of the challenge." }, { "id": 2251547, "postDate": "2023-05-09T13:08:50.590Z", "content": "

My run with the default notebook finished just fine, and pretty quickly:

\n

\"\"

\n

This is not using LoFTR though, all defaults.

", "rawMarkdown": "My run with the default notebook finished just fine, and pretty quickly:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9806603%2F4a8496fe25fa1c27aae64d3a56eacd01%2FScreenshot%202023-05-09%20at%2015.06.08.png?generation=1683637603193782&alt=media)\n\nThis is not using LoFTR though, all defaults.\n", "votes": 1 }, { "id": 2251550, "postDate": "2023-05-09T13:13:38.443Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2251554, "postDate": "2023-05-09T13:23:14.180Z", "content": "

With LoFTR the notebook indeed finishes very quickly with 0.001, so there must be a problem there. Unfortunately I can't debug it myself right now (conference deadline). :(

\n

It's possible that some change broke it. Since we know that the notebook logic itself is fine (runs ok with LOCAL_FEATURE='KeyNetAffNetHardNet'), the easiest thing would be to try it on the training data interactively.

\n

Edit: oh, I think I disabled the GPU this time, let me double-check.

", "rawMarkdown": "With LoFTR the notebook indeed finishes very quickly with 0.001, so there must be a problem there. Unfortunately I can't debug it myself right now (conference deadline). :(\n\nIt's possible that some change broke it. Since we know that the notebook logic itself is fine (runs ok with LOCAL_FEATURE='KeyNetAffNetHardNet'), the easiest thing would be to try it on the training data interactively.\n\nEdit: oh, I think I disabled the GPU this time, let me double-check." }, { "id": 2251790, "postDate": "2023-05-09T16:18:20.823Z", "content": "

Update: I get 0.036 with DISK, while the notebook indeed has a scoring error with LoFTR after running for a few hours.

", "rawMarkdown": "Update: I get 0.036 with DISK, while the notebook indeed has a scoring error with LoFTR after running for a few hours.", "votes": 1 }, { "id": 2253345, "postDate": "2023-05-10T05:26:55.973Z", "content": "

The same code sometimes can be scored correctly but some times are not, I don't know why.

", "rawMarkdown": "The same code sometimes can be scored correctly but some times are not, I don't know why." }, { "id": 2253419, "postDate": "2023-05-10T07:00:39.397Z", "content": "

Yeah, we noticed that too, but only with LoFTR.

", "rawMarkdown": "Yeah, we noticed that too, but only with LoFTR." }, { "id": 2254459, "postDate": "2023-05-11T02:26:22.873Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2254814, "postDate": "2023-05-11T08:59:02.690Z", "content": "

Sometimes got scoring error, and sometimes notebook threw exception. It's quite confusing to me.

", "rawMarkdown": "Sometimes got **scoring error**, and sometimes **notebook threw exception**. It's quite confusing to me. " }, { "id": 2269694, "postDate": "2023-05-22T16:23:03.190Z", "content": "

Hi, may I ask if found out what was causing the submission scoring error?

", "rawMarkdown": "Hi, may I ask if found out what was causing the submission scoring error?" }, { "id": 2269704, "postDate": "2023-05-22T16:32:55.627Z", "content": "

It depends on when you had it (there was probably more than one issue). The likeliest culprit is this one: https://www.kaggle.com/code/eduardtrulls/imc-2023-submission-example/comments#2254920

", "rawMarkdown": "It depends on when you had it (there was probably more than one issue). The likeliest culprit is this one: https://www.kaggle.com/code/eduardtrulls/imc-2023-submission-example/comments#2254920", "votes": 1 } ] } ] } ], "comments": [ { "id": 2251362, "author_name": "imcBEST", "author_url": "", "post_date": "2023-05-09T09:21:10.153000", "content": "

when i only changed into 'loftr',the submission is error

", "votes": 0, "replies": [ { "id": 2251390, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T09:40:43.307000", "content": "

Did you try with DISK?

", "votes": 0, "replies": [ { "id": 2251466, "author_name": "", "author_url": "", "post_date": "2023-05-09T11:20:05.627000", "content": "", "votes": 1, "replies": [] }, { "id": 2251483, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T11:44:57.743000", "content": "

There is no \"match_ensemble\" in the submission notebook (the comment says \"Can be LoFTR, KeyNetAffNetHardNet, or DISK\"). What are you looking at?

", "votes": 0, "replies": [] }, { "id": 2251496, "author_name": "", "author_url": "", "post_date": "2023-05-09T12:05:09.947000", "content": "", "votes": 0, "replies": [] }, { "id": 2251497, "author_name": "", "author_url": "", "post_date": "2023-05-09T12:05:37.683000", "content": "", "votes": 0, "replies": [] }, { "id": 2251504, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T12:13:54.050000", "content": "

I have runs with that notebook using LoFTR that finished just fine, while several people reported errors. We don't know why: it could be memory issues or a crash. The loop in the notebook contains a try-except-pass block, so small trivial mistakes there should not trigger a crash.

\n

If the notebook consistently works on the training data but crashes on the test data, try a different model. Also if you're not using a P100, you should.

", "votes": 0, "replies": [] }, { "id": 2251506, "author_name": "", "author_url": "", "post_date": "2023-05-09T12:21:46.347000", "content": "", "votes": 0, "replies": [] }, { "id": 2251512, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T12:28:12.443000", "content": "

Is \"a long time\" close to 9h, which is the limit?

", "votes": 0, "replies": [] }, { "id": 2251514, "author_name": "", "author_url": "", "post_date": "2023-05-09T12:37:35.960000", "content": "", "votes": 0, "replies": [] }, { "id": 2251527, "author_name": "Yanbing Zhang", "author_url": "", "post_date": "2023-05-09T12:44:31.103000", "content": "

How long did your code run when using LoFTR?

", "votes": 0, "replies": [] }, { "id": 2251544, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T13:05:36.403000", "content": "

That's probably an occasional crash. You might want to try reducing image size. I agree that it's difficult to debug when you don't see the error, but that's part of the challenge.

", "votes": 0, "replies": [] }, { "id": 2251547, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T13:08:50.590000", "content": "

My run with the default notebook finished just fine, and pretty quickly:

\n

\"\"

\n

This is not using LoFTR though, all defaults.

", "votes": 1, "replies": [] }, { "id": 2251550, "author_name": "", "author_url": "", "post_date": "2023-05-09T13:13:38.443000", "content": "", "votes": 0, "replies": [] }, { "id": 2251554, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T13:23:14.180000", "content": "

With LoFTR the notebook indeed finishes very quickly with 0.001, so there must be a problem there. Unfortunately I can't debug it myself right now (conference deadline). :(

\n

It's possible that some change broke it. Since we know that the notebook logic itself is fine (runs ok with LOCAL_FEATURE='KeyNetAffNetHardNet'), the easiest thing would be to try it on the training data interactively.

\n

Edit: oh, I think I disabled the GPU this time, let me double-check.

", "votes": 0, "replies": [] }, { "id": 2251790, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-09T16:18:20.823000", "content": "

Update: I get 0.036 with DISK, while the notebook indeed has a scoring error with LoFTR after running for a few hours.

", "votes": 1, "replies": [] }, { "id": 2253345, "author_name": "Pami_loveDL", "author_url": "", "post_date": "2023-05-10T05:26:55.973000", "content": "

The same code sometimes can be scored correctly but some times are not, I don't know why.

", "votes": 0, "replies": [] }, { "id": 2253419, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-10T07:00:39.397000", "content": "

Yeah, we noticed that too, but only with LoFTR.

", "votes": 0, "replies": [] }, { "id": 2254459, "author_name": "", "author_url": "", "post_date": "2023-05-11T02:26:22.873000", "content": "", "votes": 0, "replies": [] }, { "id": 2254814, "author_name": "Pami_loveDL", "author_url": "", "post_date": "2023-05-11T08:59:02.690000", "content": "

Sometimes got scoring error, and sometimes notebook threw exception. It's quite confusing to me.

", "votes": 0, "replies": [] }, { "id": 2269694, "author_name": "Yuxiang Huang", "author_url": "", "post_date": "2023-05-22T16:23:03.190000", "content": "

Hi, may I ask if found out what was causing the submission scoring error?

", "votes": 0, "replies": [] }, { "id": 2269704, "author_name": "Eduard Trulls", "author_url": "", "post_date": "2023-05-22T16:32:55.627000", "content": "

It depends on when you had it (there was probably more than one issue). The likeliest culprit is this one: https://www.kaggle.com/code/eduardtrulls/imc-2023-submission-example/comments#2254920

", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2251333": "My submissions are always score error even with the example submission", "2251362": "when i only changed into 'loftr',the submission is error" } }