![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5487737%2F530cb7e1f73f53733bceb2695aaac728%2Fphoto_2023-11-14_13-21-29.jpg?generation=1699957298638663&alt=media\")
I'm starting a traditional competition topic on the best model.
\nLB 0.14895, CV 0.12516 (simple 4 fold split, single fold) <- use only train_data.csv with longer training and a slight modification of the model (transformer-based setup) and training pipeline from my starter notebook.
\nWhile the simple CV correlates really well with public LB, keep in mind that private LB has a different sequence length distribution, and relying on simple CV/public LB may cause a strong shakeup. Also, be aware of sequence similarity in your CV split…
", "messages": [ { "id": 2440973, "postDate": "2023-09-15T21:36:55.630Z", "content": "I'm starting a traditional competition topic on the best model.
\nLB 0.14895, CV 0.12516 (simple 4 fold split, single fold) <- use only train_data.csv with longer training and a slight modification of the model (transformer-based setup) and training pipeline from my starter notebook.
\nWhile the simple CV correlates really well with public LB, keep in mind that private LB has a different sequence length distribution, and relying on simple CV/public LB may cause a strong shakeup. Also, be aware of sequence similarity in your CV split…
", "rawMarkdown": "I'm starting a traditional competition topic on the best model.\n\nLB **0.14895**, CV 0.12516 (simple 4 fold split, single fold) <- use only train_data.csv with longer training and a slight modification of the model (transformer-based setup) and training pipeline from [my starter notebook](https://www.kaggle.com/code/iafoss/rna-starter-0-186-lb). \n\nWhile the simple CV correlates really well with public LB, keep in mind that private LB has a different sequence length distribution, and relying on simple CV/public LB may cause a strong shakeup. Also, be aware of sequence similarity in your CV split...", "votes": 66 }, { "id": 2454485, "postDate": "2023-09-24T20:31:22.690Z", "content": "I'm using your split, results are for fold-0, single model
\nSome ideas from OpenVaccine challenge:
| CV | \nLB | \n
|---|---|
| 0.12633 | \n0.14813 | \n
| 0.12529 | \n0.14672 | \n
| 0.12357 | \n0.14478 | \n
| 0.12251 | \n0.14232 | \n
| 0.12120 | \n0.14066 | \n
CV/LB scores seems to be well correlated
\n
That's a nice graph. Are you automatically submitting your models to LB during training?
", "rawMarkdown": "That's a nice graph. Are you automatically submitting your models to LB during training?" }, { "id": 2496260, "postDate": "2023-10-23T21:31:27.173Z", "content": "No, I manually collected CV and LB scores and fitted simple line :D
\nimport numpy as np\n\nfrom numpy.linalg import solve\n\n = np.array([[, .], [, .], [, .], [, .], [, .]])\nu = np.array([, , , , ])\n\n## Ax = u\n## ^TAx = ^Tu\n## x = (^)^(-)^Tu\n\nx = np.linalg.inv(.T @ ) @ (.T) @ u\nslope, bias = x\nprint(slope, bias)\n( - bias) / slope ## what score do we need on CV to achieve . LB?\nAmazing single model performence. May you share your training score?
", "rawMarkdown": "Amazing single model performence. May you share your training score?" }, { "id": 2504380, "postDate": "2023-10-29T19:49:01.450Z", "content": "It's around 0.12, why do you ask?
", "rawMarkdown": "It's around 0.12, why do you ask?" }, { "id": 2504755, "postDate": "2023-10-30T06:10:06.843Z", "content": "Thanks for reply. I just want to know if my model is underfitting.
", "rawMarkdown": "Thanks for reply. I just want to know if my model is underfitting." } ] }, { "id": 2506619, "postDate": "2023-10-31T12:40:37.960Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 2502635, "postDate": "2023-10-28T10:38:47.943Z", "content": "Finally a single model (26 Epochs) CV 0.126 LB 0.146
", "rawMarkdown": "Finally a single model (26 Epochs) CV 0.126 LB 0.146", "votes": 6, "replies": [ { "id": 2507481, "postDate": "2023-11-01T03:57:04.793Z", "content": "Hello, can you tell me how to improve my score
", "rawMarkdown": "Hello, can you tell me how to improve my score", "replies": [ { "id": 2508057, "postDate": "2023-11-01T12:57:43.567Z", "content": "Hello Wang,
\nYou might need to study the previous similar competition https://www.kaggle.com/competitions/stanford-covid-vaccine
", "rawMarkdown": "Hello Wang,\n\nYou might need to study the previous similar competition https://www.kaggle.com/competitions/stanford-covid-vaccine\n\n", "votes": 2 }, { "id": 2508178, "postDate": "2023-11-01T14:26:13.657Z", "content": "thank you！I'll check it out
", "rawMarkdown": "thank you！I'll check it out" } ] }, { "id": 2529073, "postDate": "2023-11-17T22:39:11.967Z", "content": "Hi, did you utilize any other features besides the sequence feature?
", "rawMarkdown": "Hi, did you utilize any other features besides the sequence feature?", "votes": 1, "replies": [ { "id": 2536343, "postDate": "2023-11-24T06:14:22.037Z", "content": "Yes, Apparently I couldn't get any performance boost with just sequences.
\nBPPs helped a lot in my best model.
Any resources on how to use these ?
", "rawMarkdown": "Any resources on how to use these ?" } ] } ] }, { "id": 2452756, "postDate": "2023-09-23T14:57:17.320Z", "content": "0.129743 one fold out of 4, 0.15147 LB. I'm using your model, which I slightly modified to accept some of the features we used for the OpenVaccine challenge and more training. I feel sad that you get better results just by feeding the sequence info :-D.
", "rawMarkdown": "0.129743 one fold out of 4, 0.15147 LB. I'm using your model, which I slightly modified to accept some of the features we used for the OpenVaccine challenge and more training. I feel sad that you get better results just by feeding the sequence info :-D. ", "votes": 5, "replies": [ { "id": 2502699, "postDate": "2023-10-28T12:16:26.860Z", "content": "@mtinti what does react err signifies here, any idea ?
", "rawMarkdown": "@mtinti what does react err signifies here, any idea ?", "votes": 1 }, { "id": 2505522, "postDate": "2023-10-30T16:56:28.840Z", "content": "what features do you accept aspirsed by OpenVaccine ?
", "rawMarkdown": "what features do you accept aspirsed by OpenVaccine ?" } ] }, { "id": 2508327, "postDate": "2023-11-01T15:55:20.170Z", "content": "I think the delta between CV and LB is 0.02, so if you want to achieve 0.11 LB you need to achieve 0.08 CV, am I right ?
", "rawMarkdown": "I think the delta between CV and LB is 0.02, so if you want to achieve 0.11 LB you need to achieve 0.08 CV, am I right ? ", "votes": 1, "replies": [ { "id": 2521421, "postDate": "2023-11-11T17:12:18.403Z", "content": "I think so right yea, my findings have been pretty much the same too
", "rawMarkdown": "I think so right yea, my findings have been pretty much the same too" } ] }, { "id": 2484197, "postDate": "2023-10-16T10:00:14.373Z", "content": "It takes several hours to train a model. How do you usually tune large models like this?
\nThank you for your sharing. :)
", "rawMarkdown": "It takes several hours to train a model. How do you usually tune large models like this?\n\nThank you for your sharing. :)", "votes": 1, "replies": [ { "id": 2487436, "postDate": "2023-10-18T15:34:50.390Z", "content": "Hours are fine. Deep learning can easily take days. It also depends on the computing power you have, make sure you use GPU.
\nIf available, use multiple GPUs.
I have tried to use your model in the starter notebook, but instead of filtering the data by SN_filter > 0, I filter the data by signal_to_noise > 0.5 and I get better results
", "rawMarkdown": "I have tried to use your model in the starter notebook, but instead of filtering the data by SN_filter > 0, I filter the data by signal_to_noise > 0.5 and I get better results ", "votes": 1, "replies": [ { "id": 2484055, "postDate": "2023-10-16T07:33:10.803Z", "content": "I wonder if its related to the amount of training steps (more steps per epoch => more steps overall)
", "rawMarkdown": "I wonder if its related to the amount of training steps (more steps per epoch => more steps overall)", "replies": [ { "id": 2484067, "postDate": "2023-10-16T07:46:33.127Z", "content": "Now I am trying on larger batch size, to test this hypothesis
", "rawMarkdown": "Now I am trying on larger batch size, to test this hypothesis" }, { "id": 2484122, "postDate": "2023-10-16T08:47:26.900Z", "content": "My experiments lead me to the same hypothesis, I have not been able to utilize low sn samples.
", "rawMarkdown": "My experiments lead me to the same hypothesis, I have not been able to utilize low sn samples." }, { "id": 2505534, "postDate": "2023-10-30T17:04:46.363Z", "content": "@martynoveduard does LR matters, like high 1e-2 to low in 1e-5 ranges ?
", "rawMarkdown": "@martynoveduard does LR matters, like high 1e-2 to low in 1e-5 ranges ?" } ] }, { "id": 2484455, "postDate": "2023-10-16T14:00:19.837Z", "content": "I trained it longer and I get 0.16 CV and 0.16 LB
", "rawMarkdown": "I trained it longer and I get 0.16 CV and 0.16 LB", "replies": [ { "id": 2484522, "postDate": "2023-10-16T14:43:41.343Z", "content": "So it seems like, this generalizes well @farisalahmdi
", "rawMarkdown": "So it seems like, this generalizes well @farisalahmdi " }, { "id": 2484686, "postDate": "2023-10-16T16:08:39.723Z", "content": "I am not really sure if it is generalized well, but the less signal_to_noise the less data quality. I filter the data with signal_to_noise >0.5 and reads > 100, so I get more training examples but when the val_loss reach 0.16 it start bouncing.
", "rawMarkdown": "I am not really sure if it is generalized well, but the less signal_to_noise the less data quality. I filter the data with signal_to_noise >0.5 and reads > 100, so I get more training examples but when the val_loss reach 0.16 it start bouncing." }, { "id": 2484727, "postDate": "2023-10-16T16:27:43.080Z", "content": "Oh I see - I didn’t have that issue. My Val loss has rather been sticker around 12.90s
", "rawMarkdown": "Oh I see - I didn’t have that issue. My Val loss has rather been sticker around 12.90s" }, { "id": 2489281, "postDate": "2023-10-19T20:43:33.970Z", "content": "Did you use SN_filter > 0 in your validation fold or kept all samples from validation with signal_to_noise > 0.5?
", "rawMarkdown": "Did you use SN_filter > 0 in your validation fold or kept all samples from validation with signal_to_noise > 0.5?" }, { "id": 2536795, "postDate": "2023-11-24T13:43:05.080Z", "content": "@salmanahmedtamu you plan to work alone, or can you consider team up with us,
", "rawMarkdown": "@salmanahmedtamu you plan to work alone, or can you consider team up with us, " }, { "id": 2538975, "postDate": "2023-11-26T15:27:24.260Z", "content": "@jaideepvalani hi, Jaideep. Can I join your team?
", "rawMarkdown": "@jaideepvalani hi, Jaideep. Can I join your team?" } ] } ] }, { "id": 2454811, "postDate": "2023-09-25T05:52:13.830Z", "content": "lb = 0.16042, cv = 0.133671, simple transformer model
", "rawMarkdown": "lb = 0.16042, cv = 0.133671, simple transformer model", "votes": 1, "replies": [ { "id": 2500645, "postDate": "2023-10-26T21:07:02.333Z", "content": "Are you just using the data for SN_Filter=1?
", "rawMarkdown": "Are you just using the data for SN_Filter=1?", "replies": [ { "id": 2500782, "postDate": "2023-10-27T02:35:32.613Z", "content": "Yes, it is.
", "rawMarkdown": "Yes, it is." }, { "id": 2511271, "postDate": "2023-11-03T14:52:30.263Z", "content": "@zy1343930734 Hello, this is the first time for me to play this type of game, could you please tell me what are the methods of scoring?
", "rawMarkdown": "@zy1343930734 Hello, this is the first time for me to play this type of game, could you please tell me what are the methods of scoring?" } ] } ] }, { "id": 2442010, "postDate": "2023-09-16T16:21:09.683Z", "content": "LB 0.17045, CV 0.1489. tiny change on your transformer based starter model and more epochs on the same training schedule. Will experiment with different archs and see
", "rawMarkdown": "LB 0.17045, CV 0.1489. tiny change on your transformer based starter model and more epochs on the same training schedule. Will experiment with different archs and see ", "votes": 1 }, { "id": 2447784, "postDate": "2023-09-20T08:48:04.877Z", "content": "Hi All , I'm trying to break into a competition and trying to understand how to navigate through the problem , wanted to ask what LB is ? Also apart from the starter documents provided by the host , are there any other research papers anyone went through to understand the problem?
", "rawMarkdown": "Hi All , I'm trying to break into a competition and trying to understand how to navigate through the problem , wanted to ask what LB is ? Also apart from the starter documents provided by the host , are there any other research papers anyone went through to understand the problem?", "votes": -2, "replies": [ { "id": 2448156, "postDate": "2023-09-20T12:45:27.150Z", "content": "LB = leaderboard
", "rawMarkdown": "LB = leaderboard" }, { "id": 2453643, "postDate": "2023-09-24T08:24:11.400Z", "content": "Just watching Kaggle Lingo as we all need some decoding to start with.
\nAs for other papers, well, the ones in the brief cite other papers. Wikipedia was great for decoding the concepts and after that there's search. However, this thread is about the best model so we're a bit off topic. There is a papers thread here: https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/discussion/437794 and hopefully people will reply with new finds.
", "rawMarkdown": "Just watching [Kaggle Lingo](https://www.youtube.com/watch?v=sEJHyuWKd-s) as we all need some decoding to start with.\n\nAs for other papers, well, the ones in the brief cite other papers. Wikipedia was great for decoding the concepts and after that there's search. However, this thread is about the best model so we're a bit off topic. There is a papers thread here: https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/discussion/437794 and hopefully people will reply with new finds." } ] }, { "id": 2511713, "postDate": "2023-11-03T21:07:58.077Z", "content": "Hello can someone explain that what should we do with the Null data ? should we train the model with them or use some method to fill the null values ? and also a little discerption about the data and how to use to train a model for that . Thanks
", "rawMarkdown": "Hello can someone explain that what should we do with the Null data ? should we train the model with them or use some method to fill the null values ? and also a little discerption about the data and how to use to train a model for that . Thanks " }, { "id": 2471932, "postDate": "2023-10-06T18:57:18.733Z", "content": "I am wondering how people are using the OpenVaccine challenge
", "rawMarkdown": "I am wondering how people are using the OpenVaccine challenge", "replies": [ { "id": 2520695, "postDate": "2023-11-11T05:14:45.173Z", "content": "@bonaventurefpdossou It's convenient to ask. How do you use it?
", "rawMarkdown": "@bonaventurefpdossou It's convenient to ask. How do you use it?" } ] }, { "id": 2443049, "postDate": "2023-09-17T13:53:21.923Z", "content": "May I ask that if you train on kaggle or other platform? with respect to RAM/GPU etc.
\nI ensembled two folds, LB only got 0.183
", "rawMarkdown": "May I ask that if you train on kaggle or other platform? with respect to RAM/GPU etc.\n\nI ensembled two folds, LB only got 0.183", "replies": [ { "id": 2443190, "postDate": "2023-09-17T15:14:25.350Z", "content": "Training at kaggle was relevant only in 2019… when there were no GPU limits, and P100 sill were competitive. Nowadays it is relatively straightforward to buy PC that performs x10-20 faster, like one with several 4090 for example…
", "rawMarkdown": "Training at kaggle was relevant only in 2019... when there were no GPU limits, and P100 sill were competitive. Nowadays it is relatively straightforward to buy PC that performs x10-20 faster, like one with several 4090 for example... ", "votes": 3, "replies": [ { "id": 2443989, "postDate": "2023-09-18T05:47:04.670Z", "content": "thx for your reply.
", "rawMarkdown": "thx for your reply." } ] } ] }, { "id": 2443281, "postDate": "2023-09-17T16:18:45.837Z", "content": "LB 0.14895, CV 0.12516 it's grait Thank
", "rawMarkdown": "LB 0.14895, CV 0.12516 it's grait Thank ", "votes": 1 } ], "comments": [ { "id": 2454485, "author_name": "slime", "author_url": "", "post_date": "2023-09-24T20:31:22.690000", "content": "I'm using your split, results are for fold-0, single model
\nSome ideas from OpenVaccine challenge:
| CV | \nLB | \n
|---|---|
| 0.12633 | \n0.14813 | \n
| 0.12529 | \n0.14672 | \n
| 0.12357 | \n0.14478 | \n
| 0.12251 | \n0.14232 | \n
| 0.12120 | \n0.14066 | \n
CV/LB scores seems to be well correlated
\n
That's a nice graph. Are you automatically submitting your models to LB during training?
", "votes": 0, "replies": [] }, { "id": 2496260, "author_name": "slime", "author_url": "", "post_date": "2023-10-23T21:31:27.173000", "content": "No, I manually collected CV and LB scores and fitted simple line :D
\nimport numpy as np\n\nfrom numpy.linalg import solve\n\n = np.array([[, .], [, .], [, .], [, .], [, .]])\nu = np.array([, , , , ])\n\n## Ax = u\n## ^TAx = ^Tu\n## x = (^)^(-)^Tu\n\nx = np.linalg.inv(.T @ ) @ (.T) @ u\nslope, bias = x\nprint(slope, bias)\n( - bias) / slope ## what score do we need on CV to achieve . LB?\nAmazing single model performence. May you share your training score?
", "votes": 0, "replies": [] }, { "id": 2504380, "author_name": "slime", "author_url": "", "post_date": "2023-10-29T19:49:01.450000", "content": "It's around 0.12, why do you ask?
", "votes": 0, "replies": [] }, { "id": 2504755, "author_name": "Correlation", "author_url": "", "post_date": "2023-10-30T06:10:06.843000", "content": "Thanks for reply. I just want to know if my model is underfitting.
", "votes": 0, "replies": [] } ] }, { "id": 2506619, "author_name": "", "author_url": "", "post_date": "2023-10-31T12:40:37.960000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2502635, "author_name": "Salman Ahmed", "author_url": "", "post_date": "2023-10-28T10:38:47.943000", "content": "Finally a single model (26 Epochs) CV 0.126 LB 0.146
", "votes": 6, "replies": [ { "id": 2507481, "author_name": "allen wang", "author_url": "", "post_date": "2023-11-01T03:57:04.793000", "content": "Hello, can you tell me how to improve my score
", "votes": 0, "replies": [ { "id": 2508057, "author_name": "Salman Ahmed", "author_url": "", "post_date": "2023-11-01T12:57:43.567000", "content": "Hello Wang,
\nYou might need to study the previous similar competition https://www.kaggle.com/competitions/stanford-covid-vaccine
", "votes": 2, "replies": [] }, { "id": 2508178, "author_name": "allen wang", "author_url": "", "post_date": "2023-11-01T14:26:13.657000", "content": "thank you！I'll check it out
", "votes": 0, "replies": [] } ] }, { "id": 2529073, "author_name": "Donghoon Jang", "author_url": "", "post_date": "2023-11-17T22:39:11.967000", "content": "Hi, did you utilize any other features besides the sequence feature?
", "votes": 1, "replies": [ { "id": 2536343, "author_name": "Salman Ahmed", "author_url": "", "post_date": "2023-11-24T06:14:22.037000", "content": "Yes, Apparently I couldn't get any performance boost with just sequences.
\nBPPs helped a lot in my best model.
Any resources on how to use these ?
", "votes": 0, "replies": [] } ] } ] }, { "id": 2452756, "author_name": "MT", "author_url": "", "post_date": "2023-09-23T14:57:17.320000", "content": "0.129743 one fold out of 4, 0.15147 LB. I'm using your model, which I slightly modified to accept some of the features we used for the OpenVaccine challenge and more training. I feel sad that you get better results just by feeding the sequence info :-D.
", "votes": 5, "replies": [ { "id": 2502699, "author_name": "Jaideep", "author_url": "", "post_date": "2023-10-28T12:16:26.860000", "content": "@mtinti what does react err signifies here, any idea ?
", "votes": 1, "replies": [] }, { "id": 2505522, "author_name": "Wenxuan Ye", "author_url": "", "post_date": "2023-10-30T16:56:28.840000", "content": "what features do you accept aspirsed by OpenVaccine ?
", "votes": 0, "replies": [] } ] }, { "id": 2508327, "author_name": "FarisML", "author_url": "", "post_date": "2023-11-01T15:55:20.170000", "content": "I think the delta between CV and LB is 0.02, so if you want to achieve 0.11 LB you need to achieve 0.08 CV, am I right ?
", "votes": 1, "replies": [ { "id": 2521421, "author_name": "Varun Manoj Gupta", "author_url": "", "post_date": "2023-11-11T17:12:18.403000", "content": "I think so right yea, my findings have been pretty much the same too
", "votes": 0, "replies": [] } ] }, { "id": 2484197, "author_name": "", "author_url": "", "post_date": "2023-10-16T10:00:14.373000", "content": "It takes several hours to train a model. How do you usually tune large models like this?
\nThank you for your sharing. :)
", "votes": 1, "replies": [ { "id": 2487436, "author_name": "marianig82", "author_url": "", "post_date": "2023-10-18T15:34:50.390000", "content": "Hours are fine. Deep learning can easily take days. It also depends on the computing power you have, make sure you use GPU.
\nIf available, use multiple GPUs.
I have tried to use your model in the starter notebook, but instead of filtering the data by SN_filter > 0, I filter the data by signal_to_noise > 0.5 and I get better results
", "votes": 1, "replies": [ { "id": 2484055, "author_name": "slime", "author_url": "", "post_date": "2023-10-16T07:33:10.803000", "content": "I wonder if its related to the amount of training steps (more steps per epoch => more steps overall)
", "votes": 0, "replies": [ { "id": 2484067, "author_name": "FarisML", "author_url": "", "post_date": "2023-10-16T07:46:33.127000", "content": "Now I am trying on larger batch size, to test this hypothesis
", "votes": 0, "replies": [] }, { "id": 2484122, "author_name": "sroger", "author_url": "", "post_date": "2023-10-16T08:47:26.900000", "content": "My experiments lead me to the same hypothesis, I have not been able to utilize low sn samples.
", "votes": 0, "replies": [] }, { "id": 2505534, "author_name": "Jaideep", "author_url": "", "post_date": "2023-10-30T17:04:46.363000", "content": "@martynoveduard does LR matters, like high 1e-2 to low in 1e-5 ranges ?
", "votes": 0, "replies": [] } ] }, { "id": 2484455, "author_name": "FarisML", "author_url": "", "post_date": "2023-10-16T14:00:19.837000", "content": "I trained it longer and I get 0.16 CV and 0.16 LB
", "votes": 0, "replies": [ { "id": 2484522, "author_name": "Bonaventure F. P. Dossou", "author_url": "", "post_date": "2023-10-16T14:43:41.343000", "content": "So it seems like, this generalizes well @farisalahmdi
", "votes": 0, "replies": [] }, { "id": 2484686, "author_name": "FarisML", "author_url": "", "post_date": "2023-10-16T16:08:39.723000", "content": "I am not really sure if it is generalized well, but the less signal_to_noise the less data quality. I filter the data with signal_to_noise >0.5 and reads > 100, so I get more training examples but when the val_loss reach 0.16 it start bouncing.
", "votes": 0, "replies": [] }, { "id": 2484727, "author_name": "Bonaventure F. P. Dossou", "author_url": "", "post_date": "2023-10-16T16:27:43.080000", "content": "Oh I see - I didn’t have that issue. My Val loss has rather been sticker around 12.90s
", "votes": 0, "replies": [] }, { "id": 2489281, "author_name": "Salman Ahmed", "author_url": "", "post_date": "2023-10-19T20:43:33.970000", "content": "Did you use SN_filter > 0 in your validation fold or kept all samples from validation with signal_to_noise > 0.5?
", "votes": 0, "replies": [] }, { "id": 2536795, "author_name": "Jaideep", "author_url": "", "post_date": "2023-11-24T13:43:05.080000", "content": "@salmanahmedtamu you plan to work alone, or can you consider team up with us,
", "votes": 0, "replies": [] }, { "id": 2538975, "author_name": "HB", "author_url": "", "post_date": "2023-11-26T15:27:24.260000", "content": "@jaideepvalani hi, Jaideep. Can I join your team?
", "votes": 0, "replies": [] } ] } ] }, { "id": 2454811, "author_name": "DECEM", "author_url": "", "post_date": "2023-09-25T05:52:13.830000", "content": "lb = 0.16042, cv = 0.133671, simple transformer model
", "votes": 1, "replies": [ { "id": 2500645, "author_name": "Frankly", "author_url": "", "post_date": "2023-10-26T21:07:02.333000", "content": "Are you just using the data for SN_Filter=1?
", "votes": 0, "replies": [ { "id": 2500782, "author_name": "DECEM", "author_url": "", "post_date": "2023-10-27T02:35:32.613000", "content": "Yes, it is.
", "votes": 0, "replies": [] }, { "id": 2511271, "author_name": "allen wang", "author_url": "", "post_date": "2023-11-03T14:52:30.263000", "content": "@zy1343930734 Hello, this is the first time for me to play this type of game, could you please tell me what are the methods of scoring?
", "votes": 0, "replies": [] } ] } ] }, { "id": 2442010, "author_name": "nymfree", "author_url": "", "post_date": "2023-09-16T16:21:09.683000", "content": "LB 0.17045, CV 0.1489. tiny change on your transformer based starter model and more epochs on the same training schedule. Will experiment with different archs and see
", "votes": 1, "replies": [] }, { "id": 2447784, "author_name": "Kevin Sunny", "author_url": "", "post_date": "2023-09-20T08:48:04.877000", "content": "Hi All , I'm trying to break into a competition and trying to understand how to navigate through the problem , wanted to ask what LB is ? Also apart from the starter documents provided by the host , are there any other research papers anyone went through to understand the problem?
", "votes": -2, "replies": [ { "id": 2448156, "author_name": "Matt", "author_url": "", "post_date": "2023-09-20T12:45:27.150000", "content": "LB = leaderboard
", "votes": 0, "replies": [] }, { "id": 2453643, "author_name": "Jeff Allen", "author_url": "", "post_date": "2023-09-24T08:24:11.400000", "content": "Just watching Kaggle Lingo as we all need some decoding to start with.
\nAs for other papers, well, the ones in the brief cite other papers. Wikipedia was great for decoding the concepts and after that there's search. However, this thread is about the best model so we're a bit off topic. There is a papers thread here: https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/discussion/437794 and hopefully people will reply with new finds.
", "votes": 0, "replies": [] } ] }, { "id": 2511713, "author_name": "mhdaw", "author_url": "", "post_date": "2023-11-03T21:07:58.077000", "content": "Hello can someone explain that what should we do with the Null data ? should we train the model with them or use some method to fill the null values ? and also a little discerption about the data and how to use to train a model for that . Thanks
", "votes": 0, "replies": [] }, { "id": 2471932, "author_name": "Bonaventure F. P. Dossou", "author_url": "", "post_date": "2023-10-06T18:57:18.733000", "content": "I am wondering how people are using the OpenVaccine challenge
", "votes": 0, "replies": [ { "id": 2520695, "author_name": "allen wang", "author_url": "", "post_date": "2023-11-11T05:14:45.173000", "content": "@bonaventurefpdossou It's convenient to ask. How do you use it?
", "votes": 0, "replies": [] } ] }, { "id": 2443049, "author_name": "dragon zhang", "author_url": "", "post_date": "2023-09-17T13:53:21.923000", "content": "May I ask that if you train on kaggle or other platform? with respect to RAM/GPU etc.
\nI ensembled two folds, LB only got 0.183
", "votes": 0, "replies": [ { "id": 2443190, "author_name": "Iafoss", "author_url": "", "post_date": "2023-09-17T15:14:25.350000", "content": "Training at kaggle was relevant only in 2019… when there were no GPU limits, and P100 sill were competitive. Nowadays it is relatively straightforward to buy PC that performs x10-20 faster, like one with several 4090 for example…
", "votes": 3, "replies": [ { "id": 2443989, "author_name": "dragon zhang", "author_url": "", "post_date": "2023-09-18T05:47:04.670000", "content": "thx for your reply.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2443281, "author_name": "", "author_url": "", "post_date": "2023-09-17T16:18:45.837000", "content": "LB 0.14895, CV 0.12516 it's grait Thank
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2440973": "I'm starting a traditional competition topic on the best model.\n\nLB **0.14895**, CV 0.12516 (simple 4 fold split, single fold) <- use only train_data.csv with longer training and a slight modification of the model (transformer-based setup) and training pipeline from [my starter notebook](https://www.kaggle.com/code/iafoss/rna-starter-0-186-lb). \n\nWhile the simple CV correlates really well with public LB, keep in mind that private LB has a different sequence length distribution, and relying on simple CV/public LB may cause a strong shakeup. Also, be aware of sequence similarity in your CV split...", "2454485": "I'm using your split, results are for fold-0, single model\nSome ideas from OpenVaccine challenge: \n\n| CV | LB |\n| --- | --- |\n| 0.12633 | 0.14813 |\n| 0.12529 | 0.14672 |\n| 0.12357 | 0.14478 |\n| 0.12251 | 0.14232 | \n| 0.12120 | 0.14066 |", "2502635": "Finally a single model (26 Epochs) CV 0.126 LB 0.146", "2452756": "0.129743 one fold out of 4, 0.15147 LB. I'm using your model, which I slightly modified to accept some of the features we used for the OpenVaccine challenge and more training. I feel sad that you get better results just by feeding the sequence info :-D. ", "2508327": "I think the delta between CV and LB is 0.02, so if you want to achieve 0.11 LB you need to achieve 0.08 CV, am I right ? ", "2484197": "It takes several hours to train a model. How do you usually tune large models like this?\n\nThank you for your sharing. :)", "2483212": "I have tried to use your model in the starter notebook, but instead of filtering the data by SN_filter > 0, I filter the data by signal_to_noise > 0.5 and I get better results ", "2454811": "lb = 0.16042, cv = 0.133671, simple transformer model", "2442010": "LB 0.17045, CV 0.1489. tiny change on your transformer based starter model and more epochs on the same training schedule. Will experiment with different archs and see ", "2447784": "Hi All , I'm trying to break into a competition and trying to understand how to navigate through the problem , wanted to ask what LB is ? Also apart from the starter documents provided by the host , are there any other research papers anyone went through to understand the problem?", "2511713": "Hello can someone explain that what should we do with the Null data ? should we train the model with them or use some method to fill the null values ? and also a little discerption about the data and how to use to train a model for that . Thanks ", "2471932": "I am wondering how people are using the OpenVaccine challenge", "2443049": "May I ask that if you train on kaggle or other platform? with respect to RAM/GPU etc.\n\nI ensembled two folds, LB only got 0.183", "2443281": "LB 0.14895, CV 0.12516 it's grait Thank " } }