Hi all, given that we only have a few days left of competition I like to review some of what I learned in hopes that it will inspire people to ignore the LB and to focus on what we really came here for, the learning. Of course, I can't share everything before the competition end so this will be fairly loose until competition end.
\n1: Statistical processing: In this comp one idea I tried that did not work was trying to compute scene statistics to separate scenes to be used by different models. I think this is one of those things that in theory works but in practice just didnt. I was able to separate some of the challenges a bit but glass was fairly hopeless to model for. On top of this the risk vs reward was punishing as well. 1 misevaluated scene could lead to a significant drop in performance.
\nKeypoint matching: I have yet to do an image matching comp before this one so it was really cool to get to learn some of the common ways to solve a new problem, one of them being keypoint matching. For this comp things were a little nuance given the sheer difficulty of the images.
Learning pre built models: There was not really a feasible way to create/train your own custom model for this comp so most efforts seem to be ways of uniquely utilizing existing models to solve the task. Things like Super Glue and Light Glue were very interesting to learn more about, of course more to learn about that later.
Great patience: I cannot think of a time where I have tried so many ideas that should work on paper but not in actuality. The improvements that are made are quite marginal so that also made it frustrating to work on this dataset. Especially when the \"CV\" is not super representative of the LB data. And what I mean by that is just the large gap between the cv scores and LB scores.
I may add more at the end of the competition but again, good luck to everyone and please do share what you learned too. Remember that we came here to learn!
", "messages": [ { "id": "2851464", "postDate": "06/02/2024 19:04:48", "content": "Hi all, given that we only have a few days left of competition I like to review some of what I learned in hopes that it will inspire people to ignore the LB and to focus on what we really came here for, the learning. Of course, I can't share everything before the competition end so this will be fairly loose until competition end.
\n1: Statistical processing: In this comp one idea I tried that did not work was trying to compute scene statistics to separate scenes to be used by different models. I think this is one of those things that in theory works but in practice just didnt. I was able to separate some of the challenges a bit but glass was fairly hopeless to model for. On top of this the risk vs reward was punishing as well. 1 misevaluated scene could lead to a significant drop in performance.
\nKeypoint matching: I have yet to do an image matching comp before this one so it was really cool to get to learn some of the common ways to solve a new problem, one of them being keypoint matching. For this comp things were a little nuance given the sheer difficulty of the images.
Learning pre built models: There was not really a feasible way to create/train your own custom model for this comp so most efforts seem to be ways of uniquely utilizing existing models to solve the task. Things like Super Glue and Light Glue were very interesting to learn more about, of course more to learn about that later.
Great patience: I cannot think of a time where I have tried so many ideas that should work on paper but not in actuality. The improvements that are made are quite marginal so that also made it frustrating to work on this dataset. Especially when the \"CV\" is not super representative of the LB data. And what I mean by that is just the large gap between the cv scores and LB scores.
I may add more at the end of the competition but again, good luck to everyone and please do share what you learned too. Remember that we came here to learn!
", "rawMarkdown": "Hi all, given that we only have a few days left of competition I like to review some of what I learned in hopes that it will inspire people to ignore the LB and to focus on what we really came here for, the learning. Of course, I can't share everything before the competition end so this will be fairly loose until competition end. \n\n1: Statistical processing: In this comp one idea I tried that did not work was trying to compute scene statistics to separate scenes to be used by different models. I think this is one of those things that in theory works but in practice just didnt. I was able to separate some of the challenges a bit but glass was fairly hopeless to model for. On top of this the risk vs reward was punishing as well. 1 misevaluated scene could lead to a significant drop in performance.\n\n2. Keypoint matching: I have yet to do an image matching comp before this one so it was really cool to get to learn some of the common ways to solve a new problem, one of them being keypoint matching. For this comp things were a little nuance given the sheer difficulty of the images. \n\n3. Learning pre built models: There was not really a feasible way to create/train your own custom model for this comp so most efforts seem to be ways of uniquely utilizing existing models to solve the task. Things like Super Glue and Light Glue were very interesting to learn more about, of course more to learn about that later.\n\n4. Great patience: I cannot think of a time where I have tried so many ideas that should work on paper but not in actuality. The improvements that are made are quite marginal so that also made it frustrating to work on this dataset. Especially when the \"CV\" is not super representative of the LB data. And what I mean by that is just the large gap between the cv scores and LB scores. \n\nI may add more at the end of the competition but again, good luck to everyone and please do share what you learned too. Remember that we came here to learn!", "votes": null }, { "id": "2851905", "postDate": "06/03/2024 02:43:43", "content": "Cool! Totally agree that we came here to learn. Best of luck to everyone participating. 💪
", "rawMarkdown": "Cool! Totally agree that we came here to learn. Best of luck to everyone participating. 💪", "votes": null }, { "id": "2851924", "postDate": "06/03/2024 03:21:41", "content": "For sure! I am certain we will have lots to learn from you on this one sir, best of luck!!
", "rawMarkdown": "For sure! I am certain we will have lots to learn from you on this one sir, best of luck!!", "votes": null } ], "comments": [ { "id": 2851905, "author_name": "wangshengyi96", "author_url": "", "post_date": "06/03/2024 02:43:43", "content": "Cool! Totally agree that we came here to learn. Best of luck to everyone participating. 💪
", "votes": null, "replies": [ { "id": 2851924, "author_name": "cody11null", "author_url": "", "post_date": "06/03/2024 03:21:41", "content": "For sure! I am certain we will have lots to learn from you on this one sir, best of luck!!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2851464": "Hi all, given that we only have a few days left of competition I like to review some of what I learned in hopes that it will inspire people to ignore the LB and to focus on what we really came here for, the learning. Of course, I can't share everything before the competition end so this will be fairly loose until competition end. \n\n1: Statistical processing: In this comp one idea I tried that did not work was trying to compute scene statistics to separate scenes to be used by different models. I think this is one of those things that in theory works but in practice just didnt. I was able to separate some of the challenges a bit but glass was fairly hopeless to model for. On top of this the risk vs reward was punishing as well. 1 misevaluated scene could lead to a significant drop in performance.\n\n2. Keypoint matching: I have yet to do an image matching comp before this one so it was really cool to get to learn some of the common ways to solve a new problem, one of them being keypoint matching. For this comp things were a little nuance given the sheer difficulty of the images. \n\n3. Learning pre built models: There was not really a feasible way to create/train your own custom model for this comp so most efforts seem to be ways of uniquely utilizing existing models to solve the task. Things like Super Glue and Light Glue were very interesting to learn more about, of course more to learn about that later.\n\n4. Great patience: I cannot think of a time where I have tried so many ideas that should work on paper but not in actuality. The improvements that are made are quite marginal so that also made it frustrating to work on this dataset. Especially when the \"CV\" is not super representative of the LB data. And what I mean by that is just the large gap between the cv scores and LB scores. \n\nI may add more at the end of the competition but again, good luck to everyone and please do share what you learned too. Remember that we came here to learn!", "2851905": "Cool! Totally agree that we came here to learn. Best of luck to everyone participating. 💪", "2851924": "For sure! I am certain we will have lots to learn from you on this one sir, best of luck!!" }, "source": "meta" }