{ "id": 183323, "title": "Approach to tune RANSAC hyperparamers?", "url": "/competitions/landmark-recognition-2020/discussion/183323", "author_name": "", "post_date": "2020-09-16T08:40:02.976609800Z", "votes": 7, "comment_count": 15, "views": 0, "content": "

Due to not having enough resources, just experimenting with RANSAC hyperparameters. Here's what I observed so far

\n

Incressing MAX_INLIER_SCORE,MAX_REPROJECTION_ERROR decreases score
\nDecreasing HOMOGRAPHY_CONFIDENCE decreases score
\nIncressing MAX_RANSAC_ITERATIONS a little over 8.5 million increases score, however increasing it more decreases score
\nBy incressing MAX_RANSAC_ITERATIONS by 500k from 8M to 8.5M, the score improved by 0.0003

\n

Any structured approach recommendation to tune these would be highly appreciated.

\n

At the set hyperparameters as of now, the score is 0.4856 Also, I didn't find any much learning with this approach. Any new approach suggestions would be highly appreciated. which work with limited resources and constrained time limits.

", "messages": [ { "id": "1012713", "postDate": "09/16/2020 08:40:02", "content": "

Due to not having enough resources, just experimenting with RANSAC hyperparameters. Here's what I observed so far

\n

Incressing MAX_INLIER_SCORE,MAX_REPROJECTION_ERROR decreases score
\nDecreasing HOMOGRAPHY_CONFIDENCE decreases score
\nIncressing MAX_RANSAC_ITERATIONS a little over 8.5 million increases score, however increasing it more decreases score
\nBy incressing MAX_RANSAC_ITERATIONS by 500k from 8M to 8.5M, the score improved by 0.0003

\n

Any structured approach recommendation to tune these would be highly appreciated.

\n

At the set hyperparameters as of now, the score is 0.4856 Also, I didn't find any much learning with this approach. Any new approach suggestions would be highly appreciated. which work with limited resources and constrained time limits.

", "rawMarkdown": "Due to not having enough resources, just experimenting with RANSAC hyperparameters. Here's what I observed so far\n\nIncressing MAX_INLIER_SCORE,MAX_REPROJECTION_ERROR decreases score \nDecreasing HOMOGRAPHY_CONFIDENCE decreases score \nIncressing MAX_RANSAC_ITERATIONS a little over 8.5 million increases score, however increasing it more decreases score\nBy incressing MAX_RANSAC_ITERATIONS by 500k from 8M to 8.5M, the score improved by 0.0003\n\nAny structured approach recommendation to tune these would be highly appreciated.\n\nAt the set hyperparameters as of now, the score is 0.4856 Also, I didn't find any much learning with this approach. Any new approach suggestions would be highly appreciated. which work with limited resources and constrained time limits.", "votes": null }, { "id": "1012973", "postDate": "09/16/2020 12:35:57", "content": "

Be careful there is randomness you need to be aware of : )

", "rawMarkdown": "Be careful there is randomness you need to be aware of : )", "votes": null }, { "id": "1013381", "postDate": "09/16/2020 17:08:56", "content": "

To be sure of real improvement you need to submit 2-4 times with the same parameters, cause there is a bit of random fluctuation

", "rawMarkdown": "To be sure of real improvement you need to submit 2-4 times with the same parameters, cause there is a bit of random fluctuation", "votes": null }, { "id": "1013562", "postDate": "09/16/2020 19:11:18", "content": "

Yes @seif95 @aybatov I understood now what you meant after running it 3 times with the same parameters and it showed different scores. Thanks!

", "rawMarkdown": "Yes @seif95 @aybatov I understood now what you meant after running it 3 times with the same parameters and it showed different scores. Thanks!", "votes": null }, { "id": "1014575", "postDate": "09/17/2020 14:58:07", "content": "

About RANSAC_ITERATIONS https://youtu.be/5E5n7fhLHEM

", "rawMarkdown": "About RANSAC_ITERATIONS https://youtu.be/5E5n7fhLHEM", "votes": null }, { "id": "1014929", "postDate": "09/17/2020 19:52:09", "content": "

Thanks for sharing !

", "rawMarkdown": "Thanks for sharing !", "votes": null }, { "id": "1020414", "postDate": "09/21/2020 06:27:27", "content": "

Hi @kirderf

\n

Is your current LB standing with RANSAC kernel or trained models ?

", "rawMarkdown": "Hi @kirderf \n\nIs your current LB standing with RANSAC kernel or trained models ?", "votes": null }, { "id": "1020422", "postDate": "09/21/2020 06:33:14", "content": "

Hi Guys,

\n

I have a doubt with the RANSAC approach. How do you choose the outlier ratio? And the number of iterations?
\nGive some insight on this @kirderf @seif95 @aybatov

\n

Thank you

", "rawMarkdown": "Hi Guys, \n\nI have a doubt with the RANSAC approach. How do you choose the outlier ratio? And the number of iterations? \nGive some insight on this @kirderf @seif95 @aybatov \n\n**Thank you**", "votes": null }, { "id": "1020439", "postDate": "09/21/2020 06:47:40", "content": "

@jagadish13 yes my best score is from only using the baseline-model with some tweaks. Have also tried the pretrained 101 version but it doesn't fit the time limit as for now. I'm also training some other models, we'll see if they can beat the baseline in the end, I think they should, I think it will be matter of TPU qouta.

", "rawMarkdown": "jagadish13 yes my best score is from only using the baseline-model with some tweaks. Have also tried the pretrained 101 version but it doesn't fit the time limit as for now. I'm also training some other models, we'll see if they can beat the baseline in the end, I think they should, I think it will be matter of TPU qouta.", "votes": null }, { "id": "1020458", "postDate": "09/21/2020 07:04:43", "content": "

Thanks for the reply @kirderf

\n

How do you choose the inline score and error reprojection? Is it through trial and errors or is there any relation between the parameters that explain?

", "rawMarkdown": "Thanks for the reply @kirderf \n\nHow do you choose the inline score and error reprojection? Is it through trial and errors or is there any relation between the parameters that explain?", "votes": null }, { "id": "1020528", "postDate": "09/21/2020 08:19:51", "content": "

Seems like a balance of settings, like explained in this paper https://www.researchgate.net/publication/232644249_Improving_RANSAC_for_fast_landmark_recognition
\nI combine trial and errors with findings from different research papers and readings.

", "rawMarkdown": "Seems like a balance of settings, like explained in this paper https://www.researchgate.net/publication/232644249_Improving_RANSAC_for_fast_landmark_recognition\nI combine trial and errors with findings from different research papers and readings.", "votes": null }, { "id": "1021168", "postDate": "09/21/2020 17:12:32", "content": "

I'm not a pro, my first competition in this area, but this is what I think after reading a bit about it and testing myself: If you remove ransac the score drops, I'll keep it, and it have been used in many years with success. When it come to parameter tuning, there are lots of research papers in that area and also other versions of ransac. In the papers one can increase insight in the area. And its a combination of all parameters and thresholds, which need to be taken into account. Since this is my first competition maybe I just overfitting the public leaderboard but looking at the prev Recognition competition there's almost no shake up compared to other comp.

", "rawMarkdown": "I'm not a pro, my first competition in this area, but this is what I think after reading a bit about it and testing myself: If you remove ransac the score drops, I'll keep it, and it have been used in many years with success. When it come to parameter tuning, there are lots of research papers in that area and also other versions of ransac. In the papers one can increase insight in the area. And its a combination of all parameters and thresholds, which need to be taken into account. Since this is my first competition maybe I just overfitting the public leaderboard but looking at the prev Recognition competition there's almost no shake up compared to other comp.", "votes": null }, { "id": "1021198", "postDate": "09/21/2020 17:31:24", "content": "

Wait, that's crazy @kirderf. You got 0.5286 ONLY by using baseline model + tweaking parameters?

", "rawMarkdown": "Wait, that's crazy @kirderf. You got 0.5286 ONLY by using baseline model + tweaking parameters?", "votes": null }, { "id": "1021218", "postDate": "09/21/2020 17:41:27", "content": "

The reading of past writeups and research papers from Google and other research papers have paid off, using only the baseline param and pre/post-proc tweaks without re-train the model. I'm training other models though, maybe I can combine the results in the end.

", "rawMarkdown": "The reading of past writeups and research papers from Google and other research papers have paid off, using only the baseline param and pre/post-proc tweaks without re-train the model. I'm training other models though, maybe I can combine the results in the end.", "votes": null }, { "id": "1021229", "postDate": "09/21/2020 17:46:17", "content": "

Would absolutely love to read a write-up from you after the competition ends 😯😯😯

", "rawMarkdown": "Would absolutely love to read a write-up from you after the competition ends 😯😯😯", "votes": null }, { "id": "1021246", "postDate": "09/21/2020 17:57:25", "content": "

will certainly be more interesting to read the top 10 write-ups than this tweaking 🙂, but I hope that I can contribute with better things, like a great model.

", "rawMarkdown": "will certainly be more interesting to read the top 10 write-ups than this tweaking 🙂, but I hope that I can contribute with better things, like a great model.", "votes": null } ], "comments": [ { "id": 1012973, "author_name": "seif95", "author_url": "", "post_date": "09/16/2020 12:35:57", "content": "

Be careful there is randomness you need to be aware of : )

", "votes": null, "replies": [] }, { "id": 1013381, "author_name": "aybatov", "author_url": "", "post_date": "09/16/2020 17:08:56", "content": "

To be sure of real improvement you need to submit 2-4 times with the same parameters, cause there is a bit of random fluctuation

", "votes": null, "replies": [] }, { "id": 1013562, "author_name": "amanacden", "author_url": "", "post_date": "09/16/2020 19:11:18", "content": "

Yes @seif95 @aybatov I understood now what you meant after running it 3 times with the same parameters and it showed different scores. Thanks!

", "votes": null, "replies": [] }, { "id": 1014575, "author_name": "kirderf", "author_url": "", "post_date": "09/17/2020 14:58:07", "content": "

About RANSAC_ITERATIONS https://youtu.be/5E5n7fhLHEM

", "votes": null, "replies": [ { "id": 1014929, "author_name": "amanacden", "author_url": "", "post_date": "09/17/2020 19:52:09", "content": "

Thanks for sharing !

", "votes": null, "replies": [] }, { "id": 1020414, "author_name": "jagadish13", "author_url": "", "post_date": "09/21/2020 06:27:27", "content": "

Hi @kirderf

\n

Is your current LB standing with RANSAC kernel or trained models ?

", "votes": null, "replies": [] }, { "id": 1020439, "author_name": "kirderf", "author_url": "", "post_date": "09/21/2020 06:47:40", "content": "

@jagadish13 yes my best score is from only using the baseline-model with some tweaks. Have also tried the pretrained 101 version but it doesn't fit the time limit as for now. I'm also training some other models, we'll see if they can beat the baseline in the end, I think they should, I think it will be matter of TPU qouta.

", "votes": null, "replies": [] }, { "id": 1020458, "author_name": "jagadish13", "author_url": "", "post_date": "09/21/2020 07:04:43", "content": "

Thanks for the reply @kirderf

\n

How do you choose the inline score and error reprojection? Is it through trial and errors or is there any relation between the parameters that explain?

", "votes": null, "replies": [] }, { "id": 1020528, "author_name": "kirderf", "author_url": "", "post_date": "09/21/2020 08:19:51", "content": "

Seems like a balance of settings, like explained in this paper https://www.researchgate.net/publication/232644249_Improving_RANSAC_for_fast_landmark_recognition
\nI combine trial and errors with findings from different research papers and readings.

", "votes": null, "replies": [] }, { "id": 1021198, "author_name": "chankhavu", "author_url": "", "post_date": "09/21/2020 17:31:24", "content": "

Wait, that's crazy @kirderf. You got 0.5286 ONLY by using baseline model + tweaking parameters?

", "votes": null, "replies": [] }, { "id": 1021218, "author_name": "kirderf", "author_url": "", "post_date": "09/21/2020 17:41:27", "content": "

The reading of past writeups and research papers from Google and other research papers have paid off, using only the baseline param and pre/post-proc tweaks without re-train the model. I'm training other models though, maybe I can combine the results in the end.

", "votes": null, "replies": [] }, { "id": 1021229, "author_name": "chankhavu", "author_url": "", "post_date": "09/21/2020 17:46:17", "content": "

Would absolutely love to read a write-up from you after the competition ends 😯😯😯

", "votes": null, "replies": [] }, { "id": 1021246, "author_name": "kirderf", "author_url": "", "post_date": "09/21/2020 17:57:25", "content": "

will certainly be more interesting to read the top 10 write-ups than this tweaking 🙂, but I hope that I can contribute with better things, like a great model.

", "votes": null, "replies": [] } ] }, { "id": 1020422, "author_name": "jagadish13", "author_url": "", "post_date": "09/21/2020 06:33:14", "content": "

Hi Guys,

\n

I have a doubt with the RANSAC approach. How do you choose the outlier ratio? And the number of iterations?
\nGive some insight on this @kirderf @seif95 @aybatov

\n

Thank you

", "votes": null, "replies": [ { "id": 1021168, "author_name": "kirderf", "author_url": "", "post_date": "09/21/2020 17:12:32", "content": "

I'm not a pro, my first competition in this area, but this is what I think after reading a bit about it and testing myself: If you remove ransac the score drops, I'll keep it, and it have been used in many years with success. When it come to parameter tuning, there are lots of research papers in that area and also other versions of ransac. In the papers one can increase insight in the area. And its a combination of all parameters and thresholds, which need to be taken into account. Since this is my first competition maybe I just overfitting the public leaderboard but looking at the prev Recognition competition there's almost no shake up compared to other comp.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1012713": "Due to not having enough resources, just experimenting with RANSAC hyperparameters. Here's what I observed so far\n\nIncressing MAX_INLIER_SCORE,MAX_REPROJECTION_ERROR decreases score \nDecreasing HOMOGRAPHY_CONFIDENCE decreases score \nIncressing MAX_RANSAC_ITERATIONS a little over 8.5 million increases score, however increasing it more decreases score\nBy incressing MAX_RANSAC_ITERATIONS by 500k from 8M to 8.5M, the score improved by 0.0003\n\nAny structured approach recommendation to tune these would be highly appreciated.\n\nAt the set hyperparameters as of now, the score is 0.4856 Also, I didn't find any much learning with this approach. Any new approach suggestions would be highly appreciated. which work with limited resources and constrained time limits.", "1012973": "Be careful there is randomness you need to be aware of : )", "1013381": "To be sure of real improvement you need to submit 2-4 times with the same parameters, cause there is a bit of random fluctuation", "1013562": "Yes @seif95 @aybatov I understood now what you meant after running it 3 times with the same parameters and it showed different scores. Thanks!", "1014575": "About RANSAC_ITERATIONS https://youtu.be/5E5n7fhLHEM", "1014929": "Thanks for sharing !", "1020414": "Hi @kirderf \n\nIs your current LB standing with RANSAC kernel or trained models ?", "1020422": "Hi Guys, \n\nI have a doubt with the RANSAC approach. How do you choose the outlier ratio? And the number of iterations? \nGive some insight on this @kirderf @seif95 @aybatov \n\n**Thank you**", "1020439": "jagadish13 yes my best score is from only using the baseline-model with some tweaks. Have also tried the pretrained 101 version but it doesn't fit the time limit as for now. I'm also training some other models, we'll see if they can beat the baseline in the end, I think they should, I think it will be matter of TPU qouta.", "1020458": "Thanks for the reply @kirderf \n\nHow do you choose the inline score and error reprojection? Is it through trial and errors or is there any relation between the parameters that explain?", "1020528": "Seems like a balance of settings, like explained in this paper https://www.researchgate.net/publication/232644249_Improving_RANSAC_for_fast_landmark_recognition\nI combine trial and errors with findings from different research papers and readings.", "1021168": "I'm not a pro, my first competition in this area, but this is what I think after reading a bit about it and testing myself: If you remove ransac the score drops, I'll keep it, and it have been used in many years with success. When it come to parameter tuning, there are lots of research papers in that area and also other versions of ransac. In the papers one can increase insight in the area. And its a combination of all parameters and thresholds, which need to be taken into account. Since this is my first competition maybe I just overfitting the public leaderboard but looking at the prev Recognition competition there's almost no shake up compared to other comp.", "1021198": "Wait, that's crazy @kirderf. You got 0.5286 ONLY by using baseline model + tweaking parameters?", "1021218": "The reading of past writeups and research papers from Google and other research papers have paid off, using only the baseline param and pre/post-proc tweaks without re-train the model. I'm training other models though, maybe I can combine the results in the end.", "1021229": "Would absolutely love to read a write-up from you after the competition ends 😯😯😯", "1021246": "will certainly be more interesting to read the top 10 write-ups than this tweaking 🙂, but I hope that I can contribute with better things, like a great model." }, "source": "meta" }