What sort of CV/LB correlation are you guys getting with a single model?
\nMy baseline model using only tabular data (GroupKFold, 4 folds):
\nCV:
\nLB: -6.9055
Edit: CV based only on the final 3 FVC measurements:
\nCV: -6.83250
What sort of CV/LB correlation are you guys getting with a single model?
\nMy baseline model using only tabular data (GroupKFold, 4 folds):
\nCV:
\nLB: -6.9055
Edit: CV based only on the final 3 FVC measurements:
\nCV: -6.83250
Only tabular:
\nCV: -6.98459
\nLB: -6.9015
Are you calculating CV only using the last 3 predictions per patient?
", "rawMarkdown": "Only tabular:\nCV: -6.98459\nLB: -6.9015\n\nAre you calculating CV only using the last 3 predictions per patient?", "votes": null }, { "id": "984039", "postDate": "08/24/2020 19:28:18", "content": "Oh, good point. I'll fix that and report back. Thanks Rob! :)
", "rawMarkdown": "Oh, good point. I'll fix that and report back. Thanks Rob! :)", "votes": null }, { "id": "984087", "postDate": "08/24/2020 20:18:33", "content": "CV: -6.58396
\nLB: -6.8921
Using a different CV and model than the public ones
", "rawMarkdown": "CV: -6.58396\nLB: -6.8921\n\nUsing a different CV and model than the public ones", "votes": null }, { "id": "984095", "postDate": "08/24/2020 20:31:48", "content": "tabular (train.csv) data only?
", "rawMarkdown": "tabular (train.csv) data only?", "votes": null }, { "id": "984106", "postDate": "08/24/2020 20:42:21", "content": "Hi @gunesevitan , i don't know about you but what i noticed is that the better your CV is, the worse it is on public LB. I am afraid that our best performing models on public LB are overfitting.
", "rawMarkdown": "Hi @gunesevitan , i don't know about you but what i noticed is that the better your CV is, the worse it is on public LB. I am afraid that our best performing models on public LB are overfitting.", "votes": null }, { "id": "984372", "postDate": "08/25/2020 04:17:07", "content": "@robikscube Yes, tabular data only.
\n@ulrich07 I noticed that too. I try to understand the degree of overfitting from prediction visualizations. They help a lot.
", "rawMarkdown": "robikscube Yes, tabular data only.\n\n@ulrich07 I noticed that too. I try to understand the degree of overfitting from prediction visualizations. They help a lot.", "votes": null }, { "id": "984632", "postDate": "08/25/2020 07:40:25", "content": "Regarding this:
\n\n\nAre you calculating CV only using the last 3 predictions per patient?
\n
Take a look at this discussion
", "rawMarkdown": "Regarding this:\n> Are you calculating CV only using the last 3 predictions per patient?\n\nTake a look at [this discussion](https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/177061#983871)", "votes": null }, { "id": "984646", "postDate": "08/25/2020 07:52:28", "content": "PyTorch quantile regression with tabular data
\nCV : -6.60
\nLB: -6.855
Did not manage to get higher LB with CT scan features (very very small improvement for CV)
", "rawMarkdown": "PyTorch quantile regression with tabular data\nCV : -6.60\nLB: -6.855\n\nDid not manage to get higher LB with CT scan features (very very small improvement for CV)", "votes": null }, { "id": "985058", "postDate": "08/25/2020 13:27:12", "content": "I have only tried the given tabular data without any fold as of now as I have just started.
\nCV: -7.183
\nLB: -7.465
It seems as if CV score is always better than LB as of now and that too with a 0.2 ~ 0.3 margin as I saw other comments. It should help understand CV-LB correlation.
\nI tried implementing DICOM metadata for prediction but I'm getting a Submission Scoring Error while doing so. Table height alone is giving a boost to CV of ~0.01 in my case. Will update if I got any lead.
\nP.S.: I am using Giba's baseline with Quant reg, if anybody tried the same and was successful in doing so, please give some pointers on how to utilize dicom metadata in it.
", "rawMarkdown": "I have only tried the given tabular data without any fold as of now as I have just started.\n\nCV: -7.183\nLB: -7.465\n\nIt seems as if CV score is always better than LB as of now and that too with a 0.2 ~ 0.3 margin as I saw other comments. It should help understand CV-LB correlation.\n\nI tried implementing DICOM metadata for prediction but I'm getting a Submission Scoring Error while doing so. Table height alone is giving a boost to CV of ~0.01 in my case. Will update if I got any lead.\n\n*****\n\nP.S.: I am using Giba's baseline with Quant reg, if anybody tried the same and was successful in doing so, please give some pointers on how to utilize dicom metadata in it.", "votes": null }, { "id": "985088", "postDate": "08/25/2020 13:51:14", "content": "Best CV so far is -6.80 on a single TabNet model, lb is our current score.
\nEDIT: BayesianRidge gives local CV -7.xxx and LB is remarkably similar.
what does cv stand for?
", "rawMarkdown": "what does cv stand for?", "votes": null }, { "id": "986024", "postDate": "08/26/2020 06:42:57", "content": "only tabular
\nCV: -6.630
\nLB: -6.823
\nMy biggest concern is that CV’s up/down is sometimes not correlated well with LB’s.
\nI have to keep trying and improving.
Good CV/LB :) No image data?
", "rawMarkdown": "Good CV/LB :) No image data?", "votes": null }, { "id": "986084", "postDate": "08/26/2020 07:32:03", "content": "Images are giving bigger CV/LB discrepancies as of now, working to stabilize it.
", "rawMarkdown": "Images are giving bigger CV/LB discrepancies as of now, working to stabilize it.", "votes": null }, { "id": "986153", "postDate": "08/26/2020 08:47:38", "content": "Cross Validation
", "rawMarkdown": "Cross Validation", "votes": null }, { "id": "986180", "postDate": "08/26/2020 09:15:26", "content": "CV stands for cross-validation.
\nIf you are using 4 folds, it means that you split your data into 4 chunks. You then train your model using chunks 1, 2 & 3 and predict chunk 4 to validate your model (these predictions are called out-of-fold predictions, or OOFs).
\nYou then repeat for all the combinations of folds ([1, 2, 4], [1, 3, 4], [2, 3, 4]) and you will have a list of OOF predictions the same size as your train set, which will enable you to calculate the overall metric score.
\nThese OOF predictions are also super useful if you want to build a stacking model later like this: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614
", "rawMarkdown": "CV stands for cross-validation.\n\nIf you are using 4 folds, it means that you split your data into 4 chunks. You then train your model using chunks 1, 2 & 3 and predict chunk 4 to validate your model (these predictions are called out-of-fold predictions, or OOFs). \n\nYou then repeat for all the combinations of folds ([1, 2, 4], [1, 3, 4], [2, 3, 4]) and you will have a list of OOF predictions the same size as your train set, which will enable you to calculate the overall metric score. \n\nThese OOF predictions are also super useful if you want to build a stacking model later like this: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614", "votes": null }, { "id": "986199", "postDate": "08/26/2020 09:27:09", "content": "What are general advices on how to achieve stable CV?
", "rawMarkdown": "What are general advices on how to achieve stable CV?", "votes": null }, { "id": "986649", "postDate": "08/26/2020 16:53:49", "content": "best CV fold less than -6.5 lb 6.8022
", "rawMarkdown": "best CV fold less than -6.5 lb 6.8022", "votes": null }, { "id": "989268", "postDate": "08/28/2020 17:10:53", "content": "nice point!
", "rawMarkdown": "nice point!", "votes": null }, { "id": "990478", "postDate": "08/29/2020 16:27:31", "content": "Tabular
\nCV: -6.7430
\nLB: -0.6823
@gunesevitan are you splitting the data by patient?
\nI think it will give a more accurate representation of the test set for testing, as well as be less prone to overfitting
\nwith just the tabular data we got an LB: -6.8231 and an average cv of about -6.6
", "rawMarkdown": "gunesevitan are you splitting the data by patient?\n\nI think it will give a more accurate representation of the test set for testing, as well as be less prone to overfitting\n\nwith just the tabular data we got an LB: -6.8231 and an average cv of about -6.6", "votes": null }, { "id": "990757", "postDate": "08/29/2020 20:24:36", "content": "Tabular Data: (I am a using a different validation strategy than public notebooks)
\nCV: -6.8972
\nLB: -6.8992
\nThere is correlation between cv and lb but it's not perfect 100%
This is my first time trying Tabular data and I am very confused about the result 👀
\nUsing GroupKFold-5:
CV: -6.5209 \nCV on last 3 weeks: -6.698 \nLB: -7.4455\nsame for me for certain folds i can get as good 6 or 5.9 so what to believe
", "rawMarkdown": "same for me for certain folds i can get as good 6 or 5.9 so what to believe", "votes": null }, { "id": "992545", "postDate": "08/31/2020 08:18:01", "content": "All Measurement OOF Score -6.5446 - Final 3 Measurement OOF Score -6.80481\nAll Measurement OOF Score -6.56804 - Final 3 Measurement OOF Score -6.76732\nLB score is based on the blend of those two results -6.8386
@gu91642
\nI am quite a beginner here. Is there any way that I am overfitting train data while using Cross Validation?
@nxrprime did you tune hyperparameters ? Tabnet is an elegant solution but I cannot manage to get your score
", "rawMarkdown": "nxrprime did you tune hyperparameters ? Tabnet is an elegant solution but I cannot manage to get your score", "votes": null }, { "id": "1021870", "postDate": "09/22/2020 07:12:07", "content": "Tried using Optuna on a few models (TabNet, LSTM, GRU and TF.keras Quant Regression) and it gave good results only for the Tabnet, however now we have some newer ideas we want to try so we might not use Tabnet for final score.
", "rawMarkdown": "Tried using Optuna on a few models (TabNet, LSTM, GRU and TF.keras Quant Regression) and it gave good results only for the Tabnet, however now we have some newer ideas we want to try so we might not use Tabnet for final score.", "votes": null }, { "id": "1021899", "postDate": "09/22/2020 07:31:53", "content": "Thanks for your answer. I'll try to use fastai with tabnet, could help for optimization
\nGood luck for the last two weeks
But for regression questions, we can still use roc_auc_score as cv/lb score?
", "rawMarkdown": "But for regression questions, we can still use roc_auc_score as cv/lb score?", "votes": null }, { "id": "1026885", "postDate": "09/25/2020 16:13:56", "content": "Only tabular:
\nCV: 6.649908
\nLB:6.8312
Model 1 All Measurement OOF Score -6.54184 - Final 3 Measurement OOF Score -6.81443 [Std: 0.718803]\nModel 2 All Measurement OOF Score -6.58583 - Final 3 Measurement OOF Score -6.80673 [Std: 0.688967]\nBlend of those models scored -6.083 on public leaderboard.
", "rawMarkdown": "```\nModel 1 All Measurement OOF Score -6.54184 - Final 3 Measurement OOF Score -6.81443 [Std: 0.718803]\nModel 2 All Measurement OOF Score -6.58583 - Final 3 Measurement OOF Score -6.80673 [Std: 0.688967]\n```\nBlend of those models scored -6.083 on public leaderboard.", "votes": null } ], "comments": [ { "id": 984018, "author_name": "robikscube", "author_url": "", "post_date": "08/24/2020 19:15:30", "content": "Only tabular:
\nCV: -6.98459
\nLB: -6.9015
Are you calculating CV only using the last 3 predictions per patient?
", "votes": null, "replies": [ { "id": 984039, "author_name": "anjum48", "author_url": "", "post_date": "08/24/2020 19:28:18", "content": "Oh, good point. I'll fix that and report back. Thanks Rob! :)
", "votes": null, "replies": [] }, { "id": 984632, "author_name": "ademyanchuk", "author_url": "", "post_date": "08/25/2020 07:40:25", "content": "Regarding this:
\n\n\nAre you calculating CV only using the last 3 predictions per patient?
\n
Take a look at this discussion
", "votes": null, "replies": [] }, { "id": 989268, "author_name": "algeryang", "author_url": "", "post_date": "08/28/2020 17:10:53", "content": "nice point!
", "votes": null, "replies": [] } ] }, { "id": 984087, "author_name": "gunesevitan", "author_url": "", "post_date": "08/24/2020 20:18:33", "content": "CV: -6.58396
\nLB: -6.8921
Using a different CV and model than the public ones
", "votes": null, "replies": [ { "id": 984095, "author_name": "robikscube", "author_url": "", "post_date": "08/24/2020 20:31:48", "content": "tabular (train.csv) data only?
", "votes": null, "replies": [] }, { "id": 984106, "author_name": "ulrich07", "author_url": "", "post_date": "08/24/2020 20:42:21", "content": "Hi @gunesevitan , i don't know about you but what i noticed is that the better your CV is, the worse it is on public LB. I am afraid that our best performing models on public LB are overfitting.
", "votes": null, "replies": [] }, { "id": 984372, "author_name": "gunesevitan", "author_url": "", "post_date": "08/25/2020 04:17:07", "content": "@robikscube Yes, tabular data only.
\n@ulrich07 I noticed that too. I try to understand the degree of overfitting from prediction visualizations. They help a lot.
", "votes": null, "replies": [] }, { "id": 990743, "author_name": "kipparker", "author_url": "", "post_date": "08/29/2020 20:05:39", "content": "@gunesevitan are you splitting the data by patient?
\nI think it will give a more accurate representation of the test set for testing, as well as be less prone to overfitting
\nwith just the tabular data we got an LB: -6.8231 and an average cv of about -6.6
", "votes": null, "replies": [] }, { "id": 992518, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/31/2020 07:51:55", "content": "same for me for certain folds i can get as good 6 or 5.9 so what to believe
", "votes": null, "replies": [] }, { "id": 992545, "author_name": "gunesevitan", "author_url": "", "post_date": "08/31/2020 08:18:01", "content": "All Measurement OOF Score -6.5446 - Final 3 Measurement OOF Score -6.80481\nAll Measurement OOF Score -6.56804 - Final 3 Measurement OOF Score -6.76732\nLB score is based on the blend of those two results -6.8386
@gu91642
\nI am quite a beginner here. Is there any way that I am overfitting train data while using Cross Validation?
PyTorch quantile regression with tabular data
\nCV : -6.60
\nLB: -6.855
Did not manage to get higher LB with CT scan features (very very small improvement for CV)
", "votes": null, "replies": [] }, { "id": 985058, "author_name": "sarques", "author_url": "", "post_date": "08/25/2020 13:27:12", "content": "I have only tried the given tabular data without any fold as of now as I have just started.
\nCV: -7.183
\nLB: -7.465
It seems as if CV score is always better than LB as of now and that too with a 0.2 ~ 0.3 margin as I saw other comments. It should help understand CV-LB correlation.
\nI tried implementing DICOM metadata for prediction but I'm getting a Submission Scoring Error while doing so. Table height alone is giving a boost to CV of ~0.01 in my case. Will update if I got any lead.
\nP.S.: I am using Giba's baseline with Quant reg, if anybody tried the same and was successful in doing so, please give some pointers on how to utilize dicom metadata in it.
", "votes": null, "replies": [] }, { "id": 986024, "author_name": "drtausamaru", "author_url": "", "post_date": "08/26/2020 06:42:57", "content": "only tabular
\nCV: -6.630
\nLB: -6.823
\nMy biggest concern is that CV’s up/down is sometimes not correlated well with LB’s.
\nI have to keep trying and improving.
Tabular
\nCV: -6.7430
\nLB: -0.6823
Tabular Data: (I am a using a different validation strategy than public notebooks)
\nCV: -6.8972
\nLB: -6.8992
\nThere is correlation between cv and lb but it's not perfect 100%
This is my first time trying Tabular data and I am very confused about the result 👀
\nUsing GroupKFold-5:
CV: -6.5209 \nCV on last 3 weeks: -6.698 \nLB: -7.4455\nOnly tabular:
\nCV: 6.649908
\nLB:6.8312
Model 1 All Measurement OOF Score -6.54184 - Final 3 Measurement OOF Score -6.81443 [Std: 0.718803]\nModel 2 All Measurement OOF Score -6.58583 - Final 3 Measurement OOF Score -6.80673 [Std: 0.688967]\nBlend of those models scored -6.083 on public leaderboard.
", "votes": null, "replies": [] }, { "id": 985088, "author_name": "nxrprime", "author_url": "", "post_date": "08/25/2020 13:51:14", "content": "Best CV so far is -6.80 on a single TabNet model, lb is our current score.
\nEDIT: BayesianRidge gives local CV -7.xxx and LB is remarkably similar.
Good CV/LB :) No image data?
", "votes": null, "replies": [] }, { "id": 986084, "author_name": "nxrprime", "author_url": "", "post_date": "08/26/2020 07:32:03", "content": "Images are giving bigger CV/LB discrepancies as of now, working to stabilize it.
", "votes": null, "replies": [] }, { "id": 986199, "author_name": "ademyanchuk", "author_url": "", "post_date": "08/26/2020 09:27:09", "content": "What are general advices on how to achieve stable CV?
", "votes": null, "replies": [] }, { "id": 1021862, "author_name": "alexj21", "author_url": "", "post_date": "09/22/2020 07:06:07", "content": "@nxrprime did you tune hyperparameters ? Tabnet is an elegant solution but I cannot manage to get your score
", "votes": null, "replies": [] }, { "id": 1021870, "author_name": "nxrprime", "author_url": "", "post_date": "09/22/2020 07:12:07", "content": "Tried using Optuna on a few models (TabNet, LSTM, GRU and TF.keras Quant Regression) and it gave good results only for the Tabnet, however now we have some newer ideas we want to try so we might not use Tabnet for final score.
", "votes": null, "replies": [] }, { "id": 1021899, "author_name": "alexj21", "author_url": "", "post_date": "09/22/2020 07:31:53", "content": "Thanks for your answer. I'll try to use fastai with tabnet, could help for optimization
\nGood luck for the last two weeks
what does cv stand for?
", "votes": null, "replies": [ { "id": 986153, "author_name": "code1110", "author_url": "", "post_date": "08/26/2020 08:47:38", "content": "Cross Validation
", "votes": null, "replies": [] }, { "id": 986180, "author_name": "anjum48", "author_url": "", "post_date": "08/26/2020 09:15:26", "content": "CV stands for cross-validation.
\nIf you are using 4 folds, it means that you split your data into 4 chunks. You then train your model using chunks 1, 2 & 3 and predict chunk 4 to validate your model (these predictions are called out-of-fold predictions, or OOFs).
\nYou then repeat for all the combinations of folds ([1, 2, 4], [1, 3, 4], [2, 3, 4]) and you will have a list of OOF predictions the same size as your train set, which will enable you to calculate the overall metric score.
\nThese OOF predictions are also super useful if you want to build a stacking model later like this: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614
", "votes": null, "replies": [] }, { "id": 986649, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/26/2020 16:53:49", "content": "best CV fold less than -6.5 lb 6.8022
", "votes": null, "replies": [] }, { "id": 1024616, "author_name": "shenghuiguo", "author_url": "", "post_date": "09/24/2020 02:51:36", "content": "But for regression questions, we can still use roc_auc_score as cv/lb score?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "983791": "What sort of CV/LB correlation are you guys getting with a single model?\n\nMy baseline model using only tabular data (GroupKFold, 4 folds):\nCV: ~~-6.73573~~\nLB: -6.9055\n\nEdit: CV based only on the final 3 FVC measurements:\nCV: -6.83250", "984018": "Only tabular:\nCV: -6.98459\nLB: -6.9015\n\nAre you calculating CV only using the last 3 predictions per patient?", "984039": "Oh, good point. I'll fix that and report back. Thanks Rob! :)", "984087": "CV: -6.58396\nLB: -6.8921\n\nUsing a different CV and model than the public ones", "984095": "tabular (train.csv) data only?", "984106": "Hi @gunesevitan , i don't know about you but what i noticed is that the better your CV is, the worse it is on public LB. I am afraid that our best performing models on public LB are overfitting.", "984372": "robikscube Yes, tabular data only.\n\n@ulrich07 I noticed that too. I try to understand the degree of overfitting from prediction visualizations. They help a lot.", "984632": "Regarding this:\n> Are you calculating CV only using the last 3 predictions per patient?\n\nTake a look at [this discussion](https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/177061#983871)", "984646": "PyTorch quantile regression with tabular data\nCV : -6.60\nLB: -6.855\n\nDid not manage to get higher LB with CT scan features (very very small improvement for CV)", "985058": "I have only tried the given tabular data without any fold as of now as I have just started.\n\nCV: -7.183\nLB: -7.465\n\nIt seems as if CV score is always better than LB as of now and that too with a 0.2 ~ 0.3 margin as I saw other comments. It should help understand CV-LB correlation.\n\nI tried implementing DICOM metadata for prediction but I'm getting a Submission Scoring Error while doing so. Table height alone is giving a boost to CV of ~0.01 in my case. Will update if I got any lead.\n\n*****\n\nP.S.: I am using Giba's baseline with Quant reg, if anybody tried the same and was successful in doing so, please give some pointers on how to utilize dicom metadata in it.", "985088": "Best CV so far is -6.80 on a single TabNet model, lb is our current score.\n**EDIT**: BayesianRidge gives local CV -7.xxx and LB is remarkably similar.", "985868": "what does cv stand for?", "986024": "only tabular\nCV: -6.630\nLB: -6.823\nMy biggest concern is that CV’s up/down is sometimes not correlated well with LB’s.\nI have to keep trying and improving.", "986079": "Good CV/LB :) No image data?", "986084": "Images are giving bigger CV/LB discrepancies as of now, working to stabilize it.", "986153": "Cross Validation", "986180": "CV stands for cross-validation.\n\nIf you are using 4 folds, it means that you split your data into 4 chunks. You then train your model using chunks 1, 2 & 3 and predict chunk 4 to validate your model (these predictions are called out-of-fold predictions, or OOFs). \n\nYou then repeat for all the combinations of folds ([1, 2, 4], [1, 3, 4], [2, 3, 4]) and you will have a list of OOF predictions the same size as your train set, which will enable you to calculate the overall metric score. \n\nThese OOF predictions are also super useful if you want to build a stacking model later like this: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614", "986199": "What are general advices on how to achieve stable CV?", "986649": "best CV fold less than -6.5 lb 6.8022", "989268": "nice point!", "990478": "Tabular\nCV: -6.7430\nLB: -0.6823", "990743": "gunesevitan are you splitting the data by patient?\n\nI think it will give a more accurate representation of the test set for testing, as well as be less prone to overfitting\n\nwith just the tabular data we got an LB: -6.8231 and an average cv of about -6.6", "990757": "Tabular Data: (I am a using a different validation strategy than public notebooks)\nCV: -6.8972\nLB: -6.8992\nThere is correlation between cv and lb but it's not perfect 100%", "992478": "This is my first time trying Tabular data and I am very confused about the result 👀\nUsing GroupKFold-5:\n```\nCV: -6.5209\t\nCV on last 3 weeks: -6.698\t\nLB: -7.4455\n```", "992518": "same for me for certain folds i can get as good 6 or 5.9 so what to believe", "992545": "```\nAll Measurement OOF Score -6.5446 - Final 3 Measurement OOF Score -6.80481\nAll Measurement OOF Score -6.56804 - Final 3 Measurement OOF Score -6.76732\n```\n\nLB score is based on the blend of those two results `-6.8386`", "992649": "gu91642 \nI am quite a beginner here. Is there any way that I am overfitting train data while using Cross Validation?", "1021862": "nxrprime did you tune hyperparameters ? Tabnet is an elegant solution but I cannot manage to get your score", "1021870": "Tried using Optuna on a few models (TabNet, LSTM, GRU and TF.keras Quant Regression) and it gave good results only for the Tabnet, however now we have some newer ideas we want to try so we might not use Tabnet for final score.", "1021899": "Thanks for your answer. I'll try to use fastai with tabnet, could help for optimization\nGood luck for the last two weeks", "1024616": "But for regression questions, we can still use roc_auc_score as cv/lb score?", "1026885": "Only tabular:\nCV: 6.649908\nLB:6.8312", "1026956": "```\nModel 1 All Measurement OOF Score -6.54184 - Final 3 Measurement OOF Score -6.81443 [Std: 0.718803]\nModel 2 All Measurement OOF Score -6.58583 - Final 3 Measurement OOF Score -6.80673 [Std: 0.688967]\n```\nBlend of those models scored -6.083 on public leaderboard." }, "source": "meta" }