\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9718188%2Fd81e9456a5a04d89da5f917feab412c7%2FMultiOmeShape.png?generation=1663530730501366&alt=media\")
The following graph is intended to help visualize at once all the data names and shapes involved in the Multiome part of the competition.
\n
We have to predict Multiome Test_targets. However, we only need to submit part of it. 30% of the rows and for each row, 15% of the columns. It is the second part of the submission (58,931,360 row_ids in the submission file)
\nWe can use this graph and the one for CiteSeq to compare more specifically the Test_targets Data.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9718188%2Fa490de664914da4481720d4415a68cde%2FTestTargetShapes.png?generation=1663562079905748&alt=media\")
The organizer decided to sample Multiome Test_target, due to its large size. Only 30% of the rows (cells) are used for the scoring and for each of these selected rows, only 15% of the columns (genes) are selected.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9718188%2F0eb956102a53b8b105466e255f749367%2FTestTargetRatios.png?generation=1663563409305863&alt=media\")
However the sampling has some very important implications on the scoring
\nThe submission file has 8.7 times more data for Multiome than for CiteSeq and we can at first glance think that Multiome is way more important for the scoring. However what really matters is the number of rows, because each row correlation has the same weight, even if the row is much longer!
\nIf we look at the number of rows after sampling, suddenly CiteSeq is more important!! CiteSeq has all its 48,663 rows while Multiome is left with only 16,780.
\nConsequently CiteSeq represent ~74% of the score and Multiome ~26% if we look at the number of rows (and number of correlations) used.
\nThis can be checked roughly with the CV scores. CiteSeq CV is around 89 and Multiome 66.5. The LB score is closer to the CiteSeq CV score.
Amendment: The data update of 2022-09-10
\nAs @ambrosm explained below, there was an issue in the initial data and the organizer is now ignoring 7,476 rows in the CITEseq submission. You can see the implications on the scoring weights here.
The following graph is intended to help visualize at once all the data names and shapes involved in the Multiome part of the competition.
\n
We have to predict Multiome Test_targets. However, we only need to submit part of it. 30% of the rows and for each row, 15% of the columns. It is the second part of the submission (58,931,360 row_ids in the submission file)
\nWe can use this graph and the one for CiteSeq to compare more specifically the Test_targets Data.
\n
The organizer decided to sample Multiome Test_target, due to its large size. Only 30% of the rows (cells) are used for the scoring and for each of these selected rows, only 15% of the columns (genes) are selected.
\n
However the sampling has some very important implications on the scoring
\nThe submission file has 8.7 times more data for Multiome than for CiteSeq and we can at first glance think that Multiome is way more important for the scoring. However what really matters is the number of rows, because each row correlation has the same weight, even if the row is much longer!
\nIf we look at the number of rows after sampling, suddenly CiteSeq is more important!! CiteSeq has all its 48,663 rows while Multiome is left with only 16,780.
\nConsequently CiteSeq represent ~74% of the score and Multiome ~26% if we look at the number of rows (and number of correlations) used.
\nThis can be checked roughly with the CV scores. CiteSeq CV is around 89 and Multiome 66.5. The LB score is closer to the CiteSeq CV score.
Amendment: The data update of 2022-09-10
\nAs @ambrosm explained below, there was an issue in the initial data and the organizer is now ignoring 7,476 rows in the CITEseq submission. You can see the implications on the scoring weights here.
With the data update of 2022-09-10, the importance of CITEseq was reduced: The first 7476 rows of CITEseq test are ignored in scoring so that CITEseq effectively only has 48663 - 7476 = 41187 rows.
", "rawMarkdown": "With the [data update of 2022-09-10](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/350933), the importance of CITEseq was reduced: The first 7476 rows of CITEseq test are ignored in scoring so that CITEseq effectively only has 48663 - 7476 = 41187 rows.", "votes": null }, { "id": "1945573", "postDate": "09/19/2022 08:22:07", "content": "I see, I started working on the competition after this date and missed this issue. Will adjust my numbers asap. thanks @ambrosm for this and everything so valuable you have published !
", "rawMarkdown": "I see, I started working on the competition after this date and missed this issue. Will adjust my numbers asap. thanks @ambrosm for this and everything so valuable you have published !", "votes": null }, { "id": "1980004", "postDate": "10/09/2022 23:17:23", "content": "I would upvote this twice if I could. I refer to it frequently!
", "rawMarkdown": "I would upvote this twice if I could. I refer to it frequently!", "votes": null }, { "id": "1981305", "postDate": "10/10/2022 18:06:21", "content": "Thank you very much @kirkdco . This is very encouraging. I did this because I was confused 😄 and then I wanted to share with everybody.
", "rawMarkdown": "Thank you very much @kirkdco . This is very encouraging. I did this because I was confused 😄 and then I wanted to share with everybody.", "votes": null } ], "comments": [ { "id": 1945405, "author_name": "ambrosm", "author_url": "", "post_date": "09/19/2022 05:54:39", "content": "With the data update of 2022-09-10, the importance of CITEseq was reduced: The first 7476 rows of CITEseq test are ignored in scoring so that CITEseq effectively only has 48663 - 7476 = 41187 rows.
", "votes": null, "replies": [ { "id": 1945573, "author_name": "gehallak", "author_url": "", "post_date": "09/19/2022 08:22:07", "content": "I see, I started working on the competition after this date and missed this issue. Will adjust my numbers asap. thanks @ambrosm for this and everything so valuable you have published !
", "votes": null, "replies": [] } ] }, { "id": 1980004, "author_name": "kirkdco", "author_url": "", "post_date": "10/09/2022 23:17:23", "content": "I would upvote this twice if I could. I refer to it frequently!
", "votes": null, "replies": [ { "id": 1981305, "author_name": "gehallak", "author_url": "", "post_date": "10/10/2022 18:06:21", "content": "Thank you very much @kirkdco . This is very encouraging. I did this because I was confused 😄 and then I wanted to share with everybody.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1945054": "The following graph is intended to help visualize at once all the data names and shapes involved in the Multiome part of the competition.\n\n\nWe have to predict Multiome Test_targets. However, we only need to submit part of it. 30% of the rows and for each row, 15% of the columns. It is the second part of the submission (58,931,360 row_ids in the submission file)\n\nWe can use this graph and the one for [CiteSeq](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/353522) to compare more specifically the Test_targets Data.\n\n\n\nThe organizer decided to sample Multiome Test_target, due to its large size. Only 30% of the rows (cells) are used for the scoring and for each of these selected rows, only 15% of the columns (genes) are selected.\n\n\n\n- Before sampling, the 2D Multiome Test_target matrix is 192 times larger than its CiteSeq equivalent.\n- After sampling the ratio dropped to 8.7\n\nHowever **the sampling has some very important implications on the scoring**\n\nThe submission file has 8.7 times more data for Multiome than for CiteSeq and we can at first glance think that Multiome is way more important for the scoring. However what really matters is the number of rows, because each row correlation has the same weight, even if the row is much longer!\nIf we look at the number of rows after sampling, suddenly CiteSeq is more important!! CiteSeq has all its 48,663 rows while Multiome is left with only 16,780. \nConsequently CiteSeq represent ~74% of the score and Multiome ~26% if we look at the number of rows (and number of correlations) used.\nThis can be checked roughly with the CV scores. CiteSeq CV is around 89 and Multiome 66.5. The LB score is closer to the CiteSeq CV score.\n\n***Amendment: [The data update of 2022-09-10](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/350933)***\nAs @ambrosm explained below, there was an issue in the initial data and the organizer is now ignoring 7,476 rows in the CITEseq submission. You can see the implications on the scoring weights [here.](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/360180)", "1945405": "With the [data update of 2022-09-10](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/350933), the importance of CITEseq was reduced: The first 7476 rows of CITEseq test are ignored in scoring so that CITEseq effectively only has 48663 - 7476 = 41187 rows.", "1945573": "I see, I started working on the competition after this date and missed this issue. Will adjust my numbers asap. thanks @ambrosm for this and everything so valuable you have published !", "1980004": "I would upvote this twice if I could. I refer to it frequently!", "1981305": "Thank you very much @kirkdco . This is very encouraging. I did this because I was confused 😄 and then I wanted to share with everybody." }, "source": "meta" }