It is customary in Kaggle competitions with custom metrics to release the scoring code. In this competition especially, with its custom aligning step, releasing the code would be very useful. (Ideally, also with an example of how to properly apply it, e.g., on the output of the released inference notebook)
", "messages": [ { "id": "3136135", "postDate": "02/28/2025 08:22:02", "content": "It is customary in Kaggle competitions with custom metrics to release the scoring code. In this competition especially, with its custom aligning step, releasing the code would be very useful. (Ideally, also with an example of how to properly apply it, e.g., on the output of the released inference notebook)
", "rawMarkdown": "It is customary in Kaggle competitions with custom metrics to release the scoring code. In this competition especially, with its custom aligning step, releasing the code would be very useful. (Ideally, also with an example of how to properly apply it, e.g., on the output of the released inference notebook)", "votes": null }, { "id": "3136529", "postDate": "02/28/2025 16:02:05", "content": "The notebook was made public, but I neglected to link it on the Metric page. I've updated the link on the Metric page, and providing it here as well:
\nhttps://www.kaggle.com/code/metric/ribonanza-tm-score
", "rawMarkdown": "The notebook was made public, but I neglected to link it on the Metric page. I've updated the link on the Metric page, and providing it here as well:\n\nhttps://www.kaggle.com/code/metric/ribonanza-tm-score", "votes": null }, { "id": "3136643", "postDate": "02/28/2025 18:17:15", "content": "USalign executable is compiled from https://github.com/pylelab/USalign
", "rawMarkdown": "USalign executable is compiled from [https://github.com/pylelab/USalign](https://github.com/pylelab/USalign)", "votes": null }, { "id": "3142366", "postDate": "03/06/2025 10:49:03", "content": "Is it possible to briefly mention how the aligning works? I tried to match my results through my own aligning code using Kabsch algorithm but there are discrepancies. I think the reason is the modifications to the standard algorithm. That will be helpful to make the preparation more to the point. Thank you in advance.
", "rawMarkdown": "Is it possible to briefly mention how the aligning works? I tried to match my results through my own aligning code using Kabsch algorithm but there are discrepancies. I think the reason is the modifications to the standard algorithm. That will be helpful to make the preparation more to the point. Thank you in advance.", "votes": null }, { "id": "3142685", "postDate": "03/06/2025 15:49:39", "content": "Kabsch minimizes RMSD, and the minimum-RMSD superposition doesn't necessarily match the maximum-TMscore superposition. And we don't really have exact algorithms to get the maximum TM-score between two structures. Try reading the USalign paper for more.
", "rawMarkdown": "Kabsch minimizes RMSD, and the minimum-RMSD superposition doesn't necessarily match the maximum-TMscore superposition. And we don't really have exact algorithms to get the maximum TM-score between two structures. Try reading the USalign paper for more.", "votes": null }, { "id": "3142772", "postDate": "03/06/2025 16:46:25", "content": "Thank you for the reply.
\nThey may not match but as I see, an alignment computation is necessary in order to calculate the TM score, as well. This is why I am interested in the alignment code/explanation if possible, it may help for the design process.
\nI also checked the USalign paper but the formula for d0 was different than what was given in the project overview so I wanted to be sure of the version of alignment used here if there is also a difference about it.
", "rawMarkdown": "Thank you for the reply.\n\nThey may not match but as I see, an alignment computation is necessary in order to calculate the TM score, as well. This is why I am interested in the alignment code/explanation if possible, it may help for the design process.\n\nI also checked the USalign paper but the formula for d0 was different than what was given in the project overview so I wanted to be sure of the version of alignment used here if there is also a difference about it.", "votes": null }, { "id": "3142782", "postDate": "03/06/2025 16:58:43", "content": "TM-score formulas for proteins and RNA are different, both can be found in the USalign paper
", "rawMarkdown": "TM-score formulas for proteins and RNA are different, both can be found in the USalign paper", "votes": null }, { "id": "3142784", "postDate": "03/06/2025 17:03:15", "content": "Thank you. I will check again
", "rawMarkdown": "Thank you. I will check again", "votes": null }, { "id": "3143326", "postDate": "03/07/2025 05:30:51", "content": "Hi Osman. I made a notebook where I play around with usalign tool, its formula and samples from the training set. Check it out https://www.kaggle.com/code/igorbashko/tm-score-explanation-for-non-biologists. Maybe it help in some points.
", "rawMarkdown": "Hi Osman. I made a notebook where I play around with usalign tool, its formula and samples from the training set. Check it out https://www.kaggle.com/code/igorbashko/tm-score-explanation-for-non-biologists. Maybe it help in some points.", "votes": null }, { "id": "3143499", "postDate": "03/07/2025 09:32:50", "content": "Thank you, Igor. This is definitely a nice test bench that I will also use a similar one for myself. However, checking here it always uses the given USAlign tool and not an independent implementation of the alignment. Is that right?
\nThanks again.
", "rawMarkdown": "Thank you, Igor. This is definitely a nice test bench that I will also use a similar one for myself. However, checking here it always uses the given USAlign tool and not an independent implementation of the alignment. Is that right?\n\nThanks again.", "votes": null }, { "id": "3145239", "postDate": "03/09/2025 15:05:49", "content": "Yes. You are right. I used the tool which is given by the organizer in the beginning of this discussion. There are not much implementations available online. I didn't go deeply into optimization part and decided to treat it us as a black box math magic. Just performed small dummy test to see what results to expect in case of good model predictions.
", "rawMarkdown": "Yes. You are right. I used the tool which is given by the organizer in the beginning of this discussion. There are not much implementations available online. I didn't go deeply into optimization part and decided to treat it us as a black box math magic. Just performed small dummy test to see what results to expect in case of good model predictions.", "votes": null }, { "id": "3145869", "postDate": "03/10/2025 10:11:49", "content": "Ok, then I will do the same. I just tried to apply my alignment code but there are discrepancies so I wanted ask if there are any open implementations. Seems like 'black box magic' is the only choice :) Thank you again
", "rawMarkdown": "Ok, then I will do the same. I just tried to apply my alignment code but there are discrepancies so I wanted ask if there are any open implementations. Seems like 'black box magic' is the only choice :) Thank you again", "votes": null }, { "id": "3202132", "postDate": "05/14/2025 22:38:42", "content": "this does not seem to yield the same result as the leaderboard
", "rawMarkdown": "this does not seem to yield the same result as the leaderboard", "votes": null } ], "comments": [ { "id": 3136529, "author_name": "inversion", "author_url": "", "post_date": "02/28/2025 16:02:05", "content": "The notebook was made public, but I neglected to link it on the Metric page. I've updated the link on the Metric page, and providing it here as well:
\nhttps://www.kaggle.com/code/metric/ribonanza-tm-score
", "votes": null, "replies": [ { "id": 3136643, "author_name": "shujun717", "author_url": "", "post_date": "02/28/2025 18:17:15", "content": "USalign executable is compiled from https://github.com/pylelab/USalign
", "votes": null, "replies": [ { "id": 3142366, "author_name": "oalenbey", "author_url": "", "post_date": "03/06/2025 10:49:03", "content": "Is it possible to briefly mention how the aligning works? I tried to match my results through my own aligning code using Kabsch algorithm but there are discrepancies. I think the reason is the modifications to the standard algorithm. That will be helpful to make the preparation more to the point. Thank you in advance.
", "votes": null, "replies": [ { "id": 3142685, "author_name": "eugenebaulin", "author_url": "", "post_date": "03/06/2025 15:49:39", "content": "Kabsch minimizes RMSD, and the minimum-RMSD superposition doesn't necessarily match the maximum-TMscore superposition. And we don't really have exact algorithms to get the maximum TM-score between two structures. Try reading the USalign paper for more.
", "votes": null, "replies": [ { "id": 3142772, "author_name": "oalenbey", "author_url": "", "post_date": "03/06/2025 16:46:25", "content": "Thank you for the reply.
\nThey may not match but as I see, an alignment computation is necessary in order to calculate the TM score, as well. This is why I am interested in the alignment code/explanation if possible, it may help for the design process.
\nI also checked the USalign paper but the formula for d0 was different than what was given in the project overview so I wanted to be sure of the version of alignment used here if there is also a difference about it.
", "votes": null, "replies": [ { "id": 3142782, "author_name": "eugenebaulin", "author_url": "", "post_date": "03/06/2025 16:58:43", "content": "TM-score formulas for proteins and RNA are different, both can be found in the USalign paper
", "votes": null, "replies": [ { "id": 3142784, "author_name": "oalenbey", "author_url": "", "post_date": "03/06/2025 17:03:15", "content": "Thank you. I will check again
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3143326, "author_name": "igorbashko", "author_url": "", "post_date": "03/07/2025 05:30:51", "content": "Hi Osman. I made a notebook where I play around with usalign tool, its formula and samples from the training set. Check it out https://www.kaggle.com/code/igorbashko/tm-score-explanation-for-non-biologists. Maybe it help in some points.
", "votes": null, "replies": [ { "id": 3143499, "author_name": "oalenbey", "author_url": "", "post_date": "03/07/2025 09:32:50", "content": "Thank you, Igor. This is definitely a nice test bench that I will also use a similar one for myself. However, checking here it always uses the given USAlign tool and not an independent implementation of the alignment. Is that right?
\nThanks again.
", "votes": null, "replies": [ { "id": 3145239, "author_name": "igorbashko", "author_url": "", "post_date": "03/09/2025 15:05:49", "content": "Yes. You are right. I used the tool which is given by the organizer in the beginning of this discussion. There are not much implementations available online. I didn't go deeply into optimization part and decided to treat it us as a black box math magic. Just performed small dummy test to see what results to expect in case of good model predictions.
", "votes": null, "replies": [ { "id": 3145869, "author_name": "oalenbey", "author_url": "", "post_date": "03/10/2025 10:11:49", "content": "Ok, then I will do the same. I just tried to apply my alignment code but there are discrepancies so I wanted ask if there are any open implementations. Seems like 'black box magic' is the only choice :) Thank you again
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] }, { "id": 3202132, "author_name": "cjpal18", "author_url": "", "post_date": "05/14/2025 22:38:42", "content": "this does not seem to yield the same result as the leaderboard
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3136135": "It is customary in Kaggle competitions with custom metrics to release the scoring code. In this competition especially, with its custom aligning step, releasing the code would be very useful. (Ideally, also with an example of how to properly apply it, e.g., on the output of the released inference notebook)", "3136529": "The notebook was made public, but I neglected to link it on the Metric page. I've updated the link on the Metric page, and providing it here as well:\n\nhttps://www.kaggle.com/code/metric/ribonanza-tm-score", "3136643": "USalign executable is compiled from [https://github.com/pylelab/USalign](https://github.com/pylelab/USalign)", "3142366": "Is it possible to briefly mention how the aligning works? I tried to match my results through my own aligning code using Kabsch algorithm but there are discrepancies. I think the reason is the modifications to the standard algorithm. That will be helpful to make the preparation more to the point. Thank you in advance.", "3142685": "Kabsch minimizes RMSD, and the minimum-RMSD superposition doesn't necessarily match the maximum-TMscore superposition. And we don't really have exact algorithms to get the maximum TM-score between two structures. Try reading the USalign paper for more.", "3142772": "Thank you for the reply.\n\nThey may not match but as I see, an alignment computation is necessary in order to calculate the TM score, as well. This is why I am interested in the alignment code/explanation if possible, it may help for the design process.\n\nI also checked the USalign paper but the formula for d0 was different than what was given in the project overview so I wanted to be sure of the version of alignment used here if there is also a difference about it.", "3142782": "TM-score formulas for proteins and RNA are different, both can be found in the USalign paper", "3142784": "Thank you. I will check again", "3143326": "Hi Osman. I made a notebook where I play around with usalign tool, its formula and samples from the training set. Check it out https://www.kaggle.com/code/igorbashko/tm-score-explanation-for-non-biologists. Maybe it help in some points.", "3143499": "Thank you, Igor. This is definitely a nice test bench that I will also use a similar one for myself. However, checking here it always uses the given USAlign tool and not an independent implementation of the alignment. Is that right?\n\nThanks again.", "3145239": "Yes. You are right. I used the tool which is given by the organizer in the beginning of this discussion. There are not much implementations available online. I didn't go deeply into optimization part and decided to treat it us as a black box math magic. Just performed small dummy test to see what results to expect in case of good model predictions.", "3145869": "Ok, then I will do the same. I just tried to apply my alignment code but there are discrepancies so I wanted ask if there are any open implementations. Seems like 'black box magic' is the only choice :) Thank you again", "3202132": "this does not seem to yield the same result as the leaderboard" }, "source": "meta" }