{
  "id": 349833,
  "title": "Data contamination between CITEseq train/test datasets?",
  "url": "/competitions/open-problems-multimodal/discussion/349833",
  "author_name": "",
  "post_date": "2022-09-03T01:05:55.463866700Z",
  "votes": 3,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hi,</p>\n<p>Loading both the test &amp; train CITEseq datasets and printing the top rows, I noticed the exact same numbers for different cell_ids.</p>\n<p>Here's my code:</p>\n<pre><code>import os\nimport pandas as pd\n\nif not os.path.exists('/opt/conda/lib/python3.7/site-packages/tables'):\n    !pip install --quiet tables\n\nDATA_DIR = \"/kaggle/input/open-problems-multimodal/\"\nFP_CITE_TRAIN_INPUTS = os.path.join(DATA_DIR,\"train_cite_inputs.h5\")\nFP_CITE_TEST_INPUTS = os.path.join(DATA_DIR,\"test_cite_inputs.h5\")\n\ndf_cite_train_x = pd.read_hdf(FP_CITE_TRAIN_INPUTS)\ndf_cite_test_x = pd.read_hdf(FP_CITE_TEST_INPUTS)\n\ndisplay(df_cite_train_x.head(15))\ndisplay(df_cite_test_x.head(15))\n</code></pre>\n<p>And this is the result (I hope it's visible):<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F648645%2F69a7b086583f27d6a50ba97fc7473665%2FScreenshot%202022-09-03%20at%2002.48.10.png?generation=1662166126567887&amp;alt=media\" alt=\"\"></p>\n<p>I'm working with GEX data and having this kind of \"coincidence\" is extremely unlikely even in similar cells from the same individual, let alone different donors. So I wonder whether there has been a mix-up while creating the datasets, or I've misinterpreted something.</p>\n<p>Not sure how to ping the organizers, I hope they're present.</p>\n<p>Thoughts?</p>",
  "messages": [
    {
      "id": "1924334",
      "postDate": "09/03/2022 01:05:55",
      "content": "<p>Hi,</p>\n<p>Loading both the test &amp; train CITEseq datasets and printing the top rows, I noticed the exact same numbers for different cell_ids.</p>\n<p>Here's my code:</p>\n<pre><code>import os\nimport pandas as pd\n\nif not os.path.exists('/opt/conda/lib/python3.7/site-packages/tables'):\n    !pip install --quiet tables\n\nDATA_DIR = \"/kaggle/input/open-problems-multimodal/\"\nFP_CITE_TRAIN_INPUTS = os.path.join(DATA_DIR,\"train_cite_inputs.h5\")\nFP_CITE_TEST_INPUTS = os.path.join(DATA_DIR,\"test_cite_inputs.h5\")\n\ndf_cite_train_x = pd.read_hdf(FP_CITE_TRAIN_INPUTS)\ndf_cite_test_x = pd.read_hdf(FP_CITE_TEST_INPUTS)\n\ndisplay(df_cite_train_x.head(15))\ndisplay(df_cite_test_x.head(15))\n</code></pre>\n<p>And this is the result (I hope it's visible):<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F648645%2F69a7b086583f27d6a50ba97fc7473665%2FScreenshot%202022-09-03%20at%2002.48.10.png?generation=1662166126567887&amp;alt=media\" alt=\"\"></p>\n<p>I'm working with GEX data and having this kind of \"coincidence\" is extremely unlikely even in similar cells from the same individual, let alone different donors. So I wonder whether there has been a mix-up while creating the datasets, or I've misinterpreted something.</p>\n<p>Not sure how to ping the organizers, I hope they're present.</p>\n<p>Thoughts?</p>",
      "rawMarkdown": "Hi,\n\nLoading both the test & train CITEseq datasets and printing the top rows, I noticed the exact same numbers for different cell_ids.\n\nHere's my code:\n\n```\nimport os\nimport pandas as pd\n\nif not os.path.exists('/opt/conda/lib/python3.7/site-packages/tables'):\n    !pip install --quiet tables\n\nDATA_DIR = \"/kaggle/input/open-problems-multimodal/\"\nFP_CITE_TRAIN_INPUTS = os.path.join(DATA_DIR,\"train_cite_inputs.h5\")\nFP_CITE_TEST_INPUTS = os.path.join(DATA_DIR,\"test_cite_inputs.h5\")\n\ndf_cite_train_x = pd.read_hdf(FP_CITE_TRAIN_INPUTS)\ndf_cite_test_x = pd.read_hdf(FP_CITE_TEST_INPUTS)\n\ndisplay(df_cite_train_x.head(15))\ndisplay(df_cite_test_x.head(15))\n```\n\nAnd this is the result (I hope it's visible):\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F648645%2F69a7b086583f27d6a50ba97fc7473665%2FScreenshot%202022-09-03%20at%2002.48.10.png?generation=1662166126567887&alt=media)\n\nI'm working with GEX data and having this kind of \"coincidence\" is extremely unlikely even in similar cells from the same individual, let alone different donors. So I wonder whether there has been a mix-up while creating the datasets, or I've misinterpreted something.\n\nNot sure how to ping the organizers, I hope they're present.\n\nThoughts?",
      "votes": null
    },
    {
      "id": "1924471",
      "postDate": "09/03/2022 04:42:19",
      "content": "<p>This may be the same as this discussion post - CITEseq data: same RNA expression matrices from different donors in day2?</p>\n<p><a href=\"https://www.kaggle.com/competitions/open-problems-multimodal/discussion/347890\" target=\"_blank\">https://www.kaggle.com/competitions/open-problems-multimodal/discussion/347890</a></p>\n<p>Competition host Daniel Burkhardt posted they are looking into it and will update soon.  </p>\n<p>To ping the organisers - use @ + their profile name.  e.g. for yours <a href=\"https://www.kaggle.com/aglaros\" target=\"_blank\">@aglaros</a>   </p>",
      "rawMarkdown": "This may be the same as this discussion post - CITEseq data: same RNA expression matrices from different donors in day2?\n\n https://www.kaggle.com/competitions/open-problems-multimodal/discussion/347890\n\nCompetition host Daniel Burkhardt posted they are looking into it and will update soon.  \n\nTo ping the organisers - use @ + their profile name.  e.g. for yours @aglaros",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1924471,
      "author_name": "something4kag",
      "author_url": "",
      "post_date": "09/03/2022 04:42:19",
      "content": "<p>This may be the same as this discussion post - CITEseq data: same RNA expression matrices from different donors in day2?</p>\n<p><a href=\"https://www.kaggle.com/competitions/open-problems-multimodal/discussion/347890\" target=\"_blank\">https://www.kaggle.com/competitions/open-problems-multimodal/discussion/347890</a></p>\n<p>Competition host Daniel Burkhardt posted they are looking into it and will update soon.  </p>\n<p>To ping the organisers - use @ + their profile name.  e.g. for yours <a href=\"https://www.kaggle.com/aglaros\" target=\"_blank\">@aglaros</a>   </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1924334": "Hi,\n\nLoading both the test & train CITEseq datasets and printing the top rows, I noticed the exact same numbers for different cell_ids.\n\nHere's my code:\n\n```\nimport os\nimport pandas as pd\n\nif not os.path.exists('/opt/conda/lib/python3.7/site-packages/tables'):\n    !pip install --quiet tables\n\nDATA_DIR = \"/kaggle/input/open-problems-multimodal/\"\nFP_CITE_TRAIN_INPUTS = os.path.join(DATA_DIR,\"train_cite_inputs.h5\")\nFP_CITE_TEST_INPUTS = os.path.join(DATA_DIR,\"test_cite_inputs.h5\")\n\ndf_cite_train_x = pd.read_hdf(FP_CITE_TRAIN_INPUTS)\ndf_cite_test_x = pd.read_hdf(FP_CITE_TEST_INPUTS)\n\ndisplay(df_cite_train_x.head(15))\ndisplay(df_cite_test_x.head(15))\n```\n\nAnd this is the result (I hope it's visible):\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F648645%2F69a7b086583f27d6a50ba97fc7473665%2FScreenshot%202022-09-03%20at%2002.48.10.png?generation=1662166126567887&alt=media)\n\nI'm working with GEX data and having this kind of \"coincidence\" is extremely unlikely even in similar cells from the same individual, let alone different donors. So I wonder whether there has been a mix-up while creating the datasets, or I've misinterpreted something.\n\nNot sure how to ping the organizers, I hope they're present.\n\nThoughts?",
    "1924471": "This may be the same as this discussion post - CITEseq data: same RNA expression matrices from different donors in day2?\n\n https://www.kaggle.com/competitions/open-problems-multimodal/discussion/347890\n\nCompetition host Daniel Burkhardt posted they are looking into it and will update soon.  \n\nTo ping the organisers - use @ + their profile name.  e.g. for yours @aglaros"
  },
  "source": "meta"
}