{ "id": 453110, "title": "【0.584】new ensemble weights", "url": "/competitions/open-problems-single-cell-perturbations/discussion/453110", "author_name": "", "post_date": "2023-11-05T00:40:24.997602100Z", "votes": 6, "comment_count": 1, "views": 0, "content": "

Wow, LB 0.584
\nI adjusted the weight and achieved 0.584 in LB.

\n

If useful please upvote , the code is here

", "messages": [ { "id": "2512863", "postDate": "11/05/2023 00:40:24", "content": "

Wow, LB 0.584
\nI adjusted the weight and achieved 0.584 in LB.

\n

If useful please upvote , the code is here

", "rawMarkdown": "Wow, **LB 0.584**\nI adjusted the weight and achieved 0.584 in LB.\n\nIf useful please upvote , the code is [**here**](https://www.kaggle.com/code/chesterx/0-584-feature-augmentation-lightgbm)", "votes": null }, { "id": "2513189", "postDate": "11/05/2023 09:06:01", "content": "

Thanks a lot for sharing !
\nJust yesterday looked on your notebook.
\nIt seems to me the lesson we can learn is the following:
\n\"The simpler (and concise) - the better. \"

\n

That means - the following - you gave EQUAL weight to the three main ingredients LB604, LB607, LB603 comparing to the original blend by Mehran.
\nI think that is quite logical - these solutions are approximately same score and same level \"uncorrellatedness\" - so it is quite logical to give them the same weight. The other LB720 - is a bit crazy - score lower than - all-zeros (0.666), but still useful in blend. It is natural to give lower weight to it , how much lower - that is unclear - but may be your choice 0.17 is quite good.

\n

It would be great if you can make the following experiments:

\n

1)
\ntry to substitute the last prediction weighted 0.1 , by the following:
\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection/data?select=LB617_Priors0802noModel_Lonnie_nbV6.csv
\nWith the same weight 0.1

\n

2)
\nCan you try to add to blend with some not big coefficient - may be 0.1
\nThe solution by Lonnie Conv1D - LB621:
\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB621_Conv1D_Lonnie_nbV24.csv
\nIt seems it is quite uncorrellated with other - thus a chance it might uplfit blend.

\n

3)
\nSubstitute original LB604 - Kishan NN ( not(!) NLP LB607)
\nby the same model with the other random seed from Cheldieva and better LBscore:
\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB599_NN6LayersMAEonehot_CheldievaBasedOnKishan_nbV3.csv

\n

4)
\nTry to add with the small weight (about 0.05-0.1-…)
\nmy Pytorch embdeds LB0635 - very uncorrelated
\nsame dataset file:
\nLB635_PytorchEmbeds_Alex_nbV1.csv

\n

5)
\nMay be try NN by Kibira - but is not that uncorrelated.
\nLB633_NN_Kibira_nbV1.csv

\n

PS

\n

The dataset with collection of publics:

\n

https://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection

\n

Correlation analysis notebooks:
\nhttps://www.kaggle.com/alexandervc/op2-submits-correlations-and-analysis

\n

https://www.kaggle.com/code/alexandervc/ensemble-op-correlation-analysis

\n

Some more ideas later.

\n

============================================

\n

The suggestions based on the following well-known idea - it is better to add to blend those submits which are quite \"dissimilar\".
\nIt seems simple measure of similarity - average correlation between target - is not bad.

\n

The structure of correlations:
\n(See notebook https://www.kaggle.com/code/alexandervc/op2-submits-correlations-and-analysis?scriptVersionId=149342431 )

\n

So we can try to choose most uncorrelated and try them in blend.

\n

\"\"

\n

Analysis of your blend:
\nhttps://docs.google.com/presentation/d/1wiz0Wmt4D54pqMMsIOyJHuQYMZ3hTBZQQnjbLzwoGYY/edit?usp=sharing
\n\"\"

", "rawMarkdown": "Thanks a lot for sharing ! \nJust yesterday looked on your notebook.\nIt seems to me the lesson we can learn is the following:\n\"The simpler (and concise) - the better. \"\n\nThat means - the following - you gave EQUAL weight to the three main ingredients LB604, LB607, LB603 comparing to the original blend by Mehran. \nI think that is quite logical - these solutions are approximately same score and same level \"uncorrellatedness\" - so it is quite logical to give them the same weight. The other LB720 - is a bit crazy - score lower than - all-zeros (0.666), but still useful in blend. It is natural to give lower weight to it , how much lower - that is unclear - but may be your choice 0.17 is quite good. \n\nIt would be great if you can make the following experiments:\n\n1)\ntry to substitute the last prediction weighted 0.1 , by the following:\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection/data?select=LB617_Priors0802noModel_Lonnie_nbV6.csv\nWith the same weight 0.1\n\n2)\nCan you try to add to blend with some not big coefficient - may be 0.1\nThe solution by Lonnie Conv1D - LB621:\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB621_Conv1D_Lonnie_nbV24.csv\nIt seems it is quite uncorrellated with other - thus a chance it might uplfit blend.\n\n3) \nSubstitute original LB604 - Kishan NN ( not(!) NLP LB607)\nby the same model with the other random seed from Cheldieva and better LBscore:\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB599_NN6LayersMAEonehot_CheldievaBasedOnKishan_nbV3.csv\n\n4) \nTry to add with the small weight (about 0.05-0.1-...)\nmy Pytorch embdeds LB0635 - very uncorrelated\nsame dataset file:\nLB635_PytorchEmbeds_Alex_nbV1.csv\n\n5)\nMay be try NN by Kibira - but is not that uncorrelated.\nLB633_NN_Kibira_nbV1.csv\n\nPS \n\nThe dataset with collection of publics:\n\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection\n\nCorrelation analysis notebooks:\nhttps://www.kaggle.com/alexandervc/op2-submits-correlations-and-analysis\n\nhttps://www.kaggle.com/code/alexandervc/ensemble-op-correlation-analysis\n\n\nSome more ideas later.\n\n\n============================================\n\nThe suggestions based on the following well-known idea - it is better to add to blend those submits which are quite \"dissimilar\".\nIt seems simple measure of similarity - average correlation between target - is not bad.\n\nThe structure of correlations: \n(See notebook https://www.kaggle.com/code/alexandervc/op2-submits-correlations-and-analysis?scriptVersionId=149342431 )\n\nSo we can try to choose most uncorrelated and try them in blend. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2262596%2F2fa9d8a046956e4c341c924c8cd1b7f8%2FScreenshot%202023-11-05%20093008.png?generation=1699174664246248&alt=media)\n\nAnalysis of your blend:\nhttps://docs.google.com/presentation/d/1wiz0Wmt4D54pqMMsIOyJHuQYMZ3hTBZQQnjbLzwoGYY/edit?usp=sharing \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2262596%2F54fdc9d5d6a80d6e03541dc3b0e89fb0%2FScreenshot%202023-11-05%20100045.png?generation=1699174883928689&alt=media)", "votes": null } ], "comments": [ { "id": 2513189, "author_name": "alexandervc", "author_url": "", "post_date": "11/05/2023 09:06:01", "content": "

Thanks a lot for sharing !
\nJust yesterday looked on your notebook.
\nIt seems to me the lesson we can learn is the following:
\n\"The simpler (and concise) - the better. \"

\n

That means - the following - you gave EQUAL weight to the three main ingredients LB604, LB607, LB603 comparing to the original blend by Mehran.
\nI think that is quite logical - these solutions are approximately same score and same level \"uncorrellatedness\" - so it is quite logical to give them the same weight. The other LB720 - is a bit crazy - score lower than - all-zeros (0.666), but still useful in blend. It is natural to give lower weight to it , how much lower - that is unclear - but may be your choice 0.17 is quite good.

\n

It would be great if you can make the following experiments:

\n

1)
\ntry to substitute the last prediction weighted 0.1 , by the following:
\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection/data?select=LB617_Priors0802noModel_Lonnie_nbV6.csv
\nWith the same weight 0.1

\n

2)
\nCan you try to add to blend with some not big coefficient - may be 0.1
\nThe solution by Lonnie Conv1D - LB621:
\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB621_Conv1D_Lonnie_nbV24.csv
\nIt seems it is quite uncorrellated with other - thus a chance it might uplfit blend.

\n

3)
\nSubstitute original LB604 - Kishan NN ( not(!) NLP LB607)
\nby the same model with the other random seed from Cheldieva and better LBscore:
\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB599_NN6LayersMAEonehot_CheldievaBasedOnKishan_nbV3.csv

\n

4)
\nTry to add with the small weight (about 0.05-0.1-…)
\nmy Pytorch embdeds LB0635 - very uncorrelated
\nsame dataset file:
\nLB635_PytorchEmbeds_Alex_nbV1.csv

\n

5)
\nMay be try NN by Kibira - but is not that uncorrelated.
\nLB633_NN_Kibira_nbV1.csv

\n

PS

\n

The dataset with collection of publics:

\n

https://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection

\n

Correlation analysis notebooks:
\nhttps://www.kaggle.com/alexandervc/op2-submits-correlations-and-analysis

\n

https://www.kaggle.com/code/alexandervc/ensemble-op-correlation-analysis

\n

Some more ideas later.

\n

============================================

\n

The suggestions based on the following well-known idea - it is better to add to blend those submits which are quite \"dissimilar\".
\nIt seems simple measure of similarity - average correlation between target - is not bad.

\n

The structure of correlations:
\n(See notebook https://www.kaggle.com/code/alexandervc/op2-submits-correlations-and-analysis?scriptVersionId=149342431 )

\n

So we can try to choose most uncorrelated and try them in blend.

\n

\"\"

\n

Analysis of your blend:
\nhttps://docs.google.com/presentation/d/1wiz0Wmt4D54pqMMsIOyJHuQYMZ3hTBZQQnjbLzwoGYY/edit?usp=sharing
\n\"\"

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2512863": "Wow, **LB 0.584**\nI adjusted the weight and achieved 0.584 in LB.\n\nIf useful please upvote , the code is [**here**](https://www.kaggle.com/code/chesterx/0-584-feature-augmentation-lightgbm)", "2513189": "Thanks a lot for sharing ! \nJust yesterday looked on your notebook.\nIt seems to me the lesson we can learn is the following:\n\"The simpler (and concise) - the better. \"\n\nThat means - the following - you gave EQUAL weight to the three main ingredients LB604, LB607, LB603 comparing to the original blend by Mehran. \nI think that is quite logical - these solutions are approximately same score and same level \"uncorrellatedness\" - so it is quite logical to give them the same weight. The other LB720 - is a bit crazy - score lower than - all-zeros (0.666), but still useful in blend. It is natural to give lower weight to it , how much lower - that is unclear - but may be your choice 0.17 is quite good. \n\nIt would be great if you can make the following experiments:\n\n1)\ntry to substitute the last prediction weighted 0.1 , by the following:\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection/data?select=LB617_Priors0802noModel_Lonnie_nbV6.csv\nWith the same weight 0.1\n\n2)\nCan you try to add to blend with some not big coefficient - may be 0.1\nThe solution by Lonnie Conv1D - LB621:\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB621_Conv1D_Lonnie_nbV24.csv\nIt seems it is quite uncorrellated with other - thus a chance it might uplfit blend.\n\n3) \nSubstitute original LB604 - Kishan NN ( not(!) NLP LB607)\nby the same model with the other random seed from Cheldieva and better LBscore:\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection?select=LB599_NN6LayersMAEonehot_CheldievaBasedOnKishan_nbV3.csv\n\n4) \nTry to add with the small weight (about 0.05-0.1-...)\nmy Pytorch embdeds LB0635 - very uncorrelated\nsame dataset file:\nLB635_PytorchEmbeds_Alex_nbV1.csv\n\n5)\nMay be try NN by Kibira - but is not that uncorrelated.\nLB633_NN_Kibira_nbV1.csv\n\nPS \n\nThe dataset with collection of publics:\n\nhttps://www.kaggle.com/datasets/alexandervc/open-problems-2-submits-collection\n\nCorrelation analysis notebooks:\nhttps://www.kaggle.com/alexandervc/op2-submits-correlations-and-analysis\n\nhttps://www.kaggle.com/code/alexandervc/ensemble-op-correlation-analysis\n\n\nSome more ideas later.\n\n\n============================================\n\nThe suggestions based on the following well-known idea - it is better to add to blend those submits which are quite \"dissimilar\".\nIt seems simple measure of similarity - average correlation between target - is not bad.\n\nThe structure of correlations: \n(See notebook https://www.kaggle.com/code/alexandervc/op2-submits-correlations-and-analysis?scriptVersionId=149342431 )\n\nSo we can try to choose most uncorrelated and try them in blend. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2262596%2F2fa9d8a046956e4c341c924c8cd1b7f8%2FScreenshot%202023-11-05%20093008.png?generation=1699174664246248&alt=media)\n\nAnalysis of your blend:\nhttps://docs.google.com/presentation/d/1wiz0Wmt4D54pqMMsIOyJHuQYMZ3hTBZQQnjbLzwoGYY/edit?usp=sharing \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2262596%2F54fdc9d5d6a80d6e03541dc3b0e89fb0%2FScreenshot%202023-11-05%20100045.png?generation=1699174883928689&alt=media)" }, "source": "meta" }