{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# What is about ?\n\nWe look in evaluations/submission files to find some insgihts. \nAs well do some sample submissions.\n\nIn that notebook we analyse mainly CITE-seq part. \n\n\n#### Preparing submission for CITEseq part as \"df.value.ravel()\"  :   \n\n    Making submission file with \"df.value.ravel()\" - it is implemneted in many places - here we check it and try to make clear. \n    \n    1) Create dataframe with cell_ids as index exactly as in the  test_cite_inputs (same number and order) \n    2) Columns targets (CD** proteins) - exactly the same number and order as in train_cite_targets\n    3) Just do:\n        \n        df_submission['target'].iloc[:6812820]      =   df.value.ravel()  - that will work fine. \n        \n        We have checked below that order of ids and CD** in file EVALUTION and SUBMISSION files are exactly the same \n        That means the order of cell ids and gene ids in evaluation/submission files allows to do that way. \n        \n\n#### Some genes are a bit overrepsresented in targets of multiome, some under (but just a bit) :   \n\n\n    All 140 CITEseq targets occur the same 48663 times in evaluation.\n\n    BUT,  genes - targets of Multiome occur NOT exactly the same number, but difference is not that much big:\n    from  2323 to 2703 , but mainly from 2450 to 2600. \n\n    So we might try to focus more on predictions of those a bit over represented genes.\n    But probably it will not give big improvement. \n\n\n#### Outliers in CITEseq has certain effect, but naive filtering does not give much.\n\n    There are plenty outliers in CITEseq targets(!) (not just features).\n    We clip them and look how results for the submission by just means is changing.\n    We get the best result for clipping by percentiles  (  0.9, 99.9),\n    though the difference on LB is not seen, just Kaggle informs that that is the best solution.\n    \n    It is also curious that submission by MEDIAN have quite strong decrease on LB: 0.158 instead of 0.168\n    \n#### Cell types has strong effect on CITEseq, much more than days .\n    \n    Creation submission by MEANs with respect to cell types (groupby) - gives 0.215 on LB \n    compare to 0.168 - just means\n    compate to 0.170 - means groupby days.\n    \n    The best is 0.218 - group by days and cell types.\n    (But note:  it will not work directly for private LB).\n    \n    \n\n### Versions\n\n#### 17,18 - cosmetic changes \n\n#### 16 NEW WAY - means by groups - by cell types only\n\n   Score: 0.215 - a bit lower than both cell types and \n\n#### 15 NEW WAY - means by groups - by days only\n\n    # Score - 0.170 - drastically lower than with  cell types \n    \n#### 14 NEW WAY - means by groups - by days and cell_types\n\n    # Score: 0.218, so comparing to 0.168 difference is very substantial. \n\n#### 13 Filtering outliers (percentiles 0.1, 99.9)  in  targets  CITEseq and after that -  MEAN\n    \n    Score: 0.168 - but worse than v9 (  percentiles 0.9, 99.9)\n    difference is small - since we do not see difference on LB, but Kaggle says V9 - is the best version. \n\n#### 12 - no changes \n\n#### 10,11 Filtering outliers (percentiles 0.99, 99.99)  in  targets  CITEseq and after that -  MEAN\n\n    Score 0.168 - BUT WORSE than V9 ( percentiles 0.9, 99.9) \n    difference is small - since we do not see difference on LB, but Kaggle says V9 - is the best version. \n\n#### 9 Filtering outliers (percentiles 0.9, 99.9)  in  targets  CITEseq and after that -  MEAN\n\n    Score 0.168 - the BEST  - difference is small - since we do not see difference on LB, but Kaggle says it is the best version. \n\n#### 8 Filtering outliers (percentiles 0.5, 99.5)  in  targets  CITEseq and after that -  MEAN\n\n    Score 0.168 - the BEST  - difference is small - since we do not see difference on LB, but Kaggle says it is the best version. \n\n#### 7 Filtering outliers (percentiles 2, 98)  in  targets  CITEseq and after that -  MEAN\n\n    Score 0.168 - worse  than 1,99 (V6) - so we need very small filtering  - difference is small - since we do not see difference on LB \n    \n#### 6 Filtering outliers (percentiles 1, 99)  in  targets  CITEseq and after that -  MEAN\n\n    Score 0.168 - better than simple MEAN - (but not much better) since we do not see difference on LB \n    \n#### 5 - skip \n\n#### 4 Filtering outliers (percentiles 5, 95)  in  targets  CITEseq and after that -  MEAN\n\n    Score 0.167 - WORSE than simple MEAN\n\n#### 3\n    \n    Analysis and sample submission - CITE-seq part, just MEDIAN values.\n    MEDIAN\n    LB: Score: 0.158\n\n#### 2\n    \n    Analysis and sample submission - CITE-seq part, just mean values. \n    LB: Score: 0.168\n    \n#### 1\n\n    Intitial ","metadata":{}},{"cell_type":"code","source":"6812820 == 140 * 48663 # first 6_812_820 positions in evaluation/submission file correspond to CITEseq part. \n# Take into account that after the data leak correction not all of them will be evaluated - about 7K will be excluded from the evaluation\n","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:30:48.049690Z","iopub.execute_input":"2022-10-02T07:30:48.050208Z","iopub.status.idle":"2022-10-02T07:30:48.060167Z","shell.execute_reply.started":"2022-10-02T07:30:48.050163Z","shell.execute_reply":"2022-10-02T07:30:48.058394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Install/import","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-10-02T19:54:39.689925Z","iopub.execute_input":"2022-10-02T19:54:39.690434Z","iopub.status.idle":"2022-10-02T19:54:39.717340Z","shell.execute_reply.started":"2022-10-02T19:54:39.690339Z","shell.execute_reply":"2022-10-02T19:54:39.716523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nimport time\nt0start = time.time()\n\nimport pandas as pd\nimport numpy as np\nimport os\nimport sys\n\nimport matplotlib.pyplot as plt\n#plt.style.use('dark_background')\nimport seaborn as sns\n\n#If you see a urllib warning running this cell, go to \"Settings\" on the right hand side, \n#and turn on internet. Note, you need to be phone verified.\n!pip install --quiet tables\n\n\nimport h5py\n!pip install hdf5plugin~=2.0 # https://forum.hdfgroup.org/t/cant-open-directory-usr-local-hdf5-lib-plugin/9738/4\nimport hdf5plugin\n\n# !pip install scanpy\n# import scanpy as sc\n# import anndata\n\nDATA_DIR = \"/kaggle/input/open-problems-multimodal/\"\nFP_CELL_METADATA = os.path.join(DATA_DIR,\"metadata.csv\")\n\nFP_CITE_TRAIN_INPUTS = os.path.join(DATA_DIR,\"train_cite_inputs.h5\")\nFP_CITE_TRAIN_TARGETS = os.path.join(DATA_DIR,\"train_cite_targets.h5\")\nFP_CITE_TEST_INPUTS = os.path.join(DATA_DIR,\"test_cite_inputs.h5\")\n\nFP_MULTIOME_TRAIN_INPUTS = os.path.join(DATA_DIR,\"train_multi_inputs.h5\")\nFP_MULTIOME_TRAIN_TARGETS = os.path.join(DATA_DIR,\"train_multi_targets.h5\")\nFP_MULTIOME_TEST_INPUTS = os.path.join(DATA_DIR,\"test_multi_inputs.h5\")\n\nFP_SUBMISSION = os.path.join(DATA_DIR,\"sample_submission.csv\")\nFP_EVALUATION_IDS = os.path.join(DATA_DIR,\"evaluation_ids.csv\")\n\ndf_cell = pd.read_csv(FP_CELL_METADATA)\ndf_cell","metadata":{"execution":{"iopub.status.busy":"2022-10-02T19:54:41.091694Z","iopub.execute_input":"2022-10-02T19:54:41.092059Z","iopub.status.idle":"2022-10-02T19:55:06.225018Z","shell.execute_reply.started":"2022-10-02T19:54:41.092030Z","shell.execute_reply":"2022-10-02T19:55:06.223646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load files","metadata":{}},{"cell_type":"code","source":"\nfilename = '/kaggle/input/open-problems-multimodal/test_cite_inputs.h5'\n\nf2 = h5py.File(filename,'r')#, mode)\nprint(f2.keys() )\nfirst_key = 'test_cite_inputs'\nprint( f2[first_key].keys() )\nlist_cells_barcodes = [t.decode() for t in f2[first_key]['axis1'] ]\nprint(len(list_cells_barcodes), list_cells_barcodes[:5], list_cells_barcodes[-5:])","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:31:11.217921Z","iopub.execute_input":"2022-10-02T07:31:11.218318Z","iopub.status.idle":"2022-10-02T07:31:15.691773Z","shell.execute_reply.started":"2022-10-02T07:31:11.218276Z","shell.execute_reply":"2022-10-02T07:31:15.690526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_cite_train_y = pd.read_hdf(FP_CITE_TRAIN_TARGETS)\ndf_cite_train_y.head()","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:31:15.693342Z","iopub.execute_input":"2022-10-02T07:31:15.693762Z","iopub.status.idle":"2022-10-02T07:31:16.413376Z","shell.execute_reply.started":"2022-10-02T07:31:15.693726Z","shell.execute_reply":"2022-10-02T07:31:16.412173Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n# FP_SUBMISSION = os.path.join(DATA_DIR,\"sample_submission.csv\")\n# FP_EVALUATION_IDS = os.path.join(DATA_DIR,\"evaluation_ids.csv\")\n\n\ndf = pd.read_csv(FP_EVALUATION_IDS)# '/kaggle/input/open-problems-multimodal/evaluation_ids.csv')\ndf","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:31:16.416354Z","iopub.execute_input":"2022-10-02T07:31:16.416773Z","iopub.status.idle":"2022-10-02T07:32:25.652009Z","shell.execute_reply.started":"2022-10-02T07:31:16.416730Z","shell.execute_reply":"2022-10-02T07:32:25.650703Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfs = pd.read_csv(FP_SUBMISSION, nrows = 10)\ndfs","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:25.653564Z","iopub.execute_input":"2022-10-02T07:32:25.653906Z","iopub.status.idle":"2022-10-02T07:32:25.689795Z","shell.execute_reply.started":"2022-10-02T07:32:25.653874Z","shell.execute_reply":"2022-10-02T07:32:25.688823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Sanity Check - cell ids (barcodes) in test_cite_inputs are exactly the same as in the submision file","metadata":{}},{"cell_type":"code","source":"# Show that in evaluation/submision file - the first 6812820 elements correspond to CITE-seq targets  (CD** targets, after come ENSG*** targets - multiome )\ndf.iloc[6812818:6812822,:].head()","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:25.691321Z","iopub.execute_input":"2022-10-02T07:32:25.692030Z","iopub.status.idle":"2022-10-02T07:32:25.703352Z","shell.execute_reply.started":"2022-10-02T07:32:25.691995Z","shell.execute_reply":"2022-10-02T07:32:25.702125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check that oder of cell_id is the same in evaluation/submission file as in the test_cite_inputs\nprint( set( df['cell_id'].iloc[:6812820] ) == set(list_cells_barcodes ) ) # Unordered check (weak)\nprint( (df['cell_id'].iloc[:6812820].drop_duplicates() != list_cells_barcodes).sum( ) )# ordered check \n","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:25.704830Z","iopub.execute_input":"2022-10-02T07:32:25.705249Z","iopub.status.idle":"2022-10-02T07:32:26.503902Z","shell.execute_reply.started":"2022-10-02T07:32:25.705218Z","shell.execute_reply":"2022-10-02T07:32:26.502236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check that genes_ids order is the same in evaluation/submission and train_cite_inputs\nlist(df['gene_id'].iloc[:140]) ==  list(df_cite_train_y.columns)","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:26.505563Z","iopub.execute_input":"2022-10-02T07:32:26.505910Z","iopub.status.idle":"2022-10-02T07:32:26.513584Z","shell.execute_reply.started":"2022-10-02T07:32:26.505881Z","shell.execute_reply":"2022-10-02T07:32:26.512387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check that df.values.ravel() creates first columns/ later rows order - what we need for submission/evalution file\nt= pd.DataFrame([[1,2],[3,4]]) \ndisplay( t )\nt.values.ravel()","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:26.515174Z","iopub.execute_input":"2022-10-02T07:32:26.515536Z","iopub.status.idle":"2022-10-02T07:32:26.535613Z","shell.execute_reply.started":"2022-10-02T07:32:26.515503Z","shell.execute_reply":"2022-10-02T07:32:26.534327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Analysis of Multiome evaluation/submission file","metadata":{}},{"cell_type":"code","source":"%%time\ndf['gene_id'].nunique()","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:26.540335Z","iopub.execute_input":"2022-10-02T07:32:26.540713Z","iopub.status.idle":"2022-10-02T07:32:36.095433Z","shell.execute_reply.started":"2022-10-02T07:32:26.540682Z","shell.execute_reply":"2022-10-02T07:32:36.094175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 23_558 targets = 140 CITEseq + 23418 (multiome) - all targets in multiome - see check in cell below\n23558 == 140 + 23418","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:36.097288Z","iopub.execute_input":"2022-10-02T07:32:36.097988Z","iopub.status.idle":"2022-10-02T07:32:36.106159Z","shell.execute_reply.started":"2022-10-02T07:32:36.097941Z","shell.execute_reply":"2022-10-02T07:32:36.104911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filename = FP_MULTIOME_TRAIN_TARGETS\n\nf = h5py.File(filename,'r')#, mode)\nprint(f.keys() )\nfirst_key = 'train_multi_targets'\nprint( f[first_key].keys() )\nprint(f[first_key]['block0_items'][:5])\nprint(f[first_key]['block0_items'].shape)\n\n","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:36.107442Z","iopub.execute_input":"2022-10-02T07:32:36.107820Z","iopub.status.idle":"2022-10-02T07:32:36.127096Z","shell.execute_reply.started":"2022-10-02T07:32:36.107789Z","shell.execute_reply":"2022-10-02T07:32:36.125975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nv = df['gene_id'].value_counts()\nprint(v.iloc[:4])\nplt.figure(figsize=(20,3))\nplt.plot(v.values[:140])\nplt.title('All targets of CITEseq occur exactly the same 48663 times ')\nplt.show()\n\nplt.figure(figsize=(20,3))\nplt.plot(v.values[140:])\nplt.title('But targets of multiome occur not exactly the same number of times - like a normal distribution  ')\nplt.show()\nprint(v.iloc[140:])\n","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:36.129000Z","iopub.execute_input":"2022-10-02T07:32:36.129404Z","iopub.status.idle":"2022-10-02T07:32:54.179114Z","shell.execute_reply.started":"2022-10-02T07:32:36.129368Z","shell.execute_reply":"2022-10-02T07:32:54.177847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#v = v.sort_values(ascending = False)\nplt.plot(v.values[140:])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:54.180809Z","iopub.execute_input":"2022-10-02T07:32:54.181183Z","iopub.status.idle":"2022-10-02T07:32:54.360003Z","shell.execute_reply.started":"2022-10-02T07:32:54.181149Z","shell.execute_reply":"2022-10-02T07:32:54.358820Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\ns = v.iloc[140:]\ns.index.name = 'Ensembl_id'\ns.name = 'Frequency'\ns.to_csv('multiome_target_genes_frequences.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:54.361835Z","iopub.execute_input":"2022-10-02T07:32:54.362207Z","iopub.status.idle":"2022-10-02T07:32:54.398301Z","shell.execute_reply.started":"2022-10-02T07:32:54.362172Z","shell.execute_reply":"2022-10-02T07:32:54.397023Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Top frequent gene: \n\n    Top one: \n    ENSG00000102466    2703\n    https://www.ensembl.org/Homo_sapiens/Gene/Summary?g=ENSG00000102466;r=13:101710804-102402457\n\n    Gene: FGF14 ENSG00000102466\n    # \n    Description\n    fibroblast growth factor 14 [Source:HGNC Symbol;Acc:HGNC:3671]\n\n    Gene Synonyms\n    FHF4, SCA27\n\n    Location\n    Chromosome 13: 101,710,804-102,402,457 reverse strand.\n\n    GRCh38:CM000675.2\n\n    About this gene\n    This gene has 4 transcripts (splice variants), 229 orthologues, 21 paralogues and is associated with 3 phenotypes.\n\n    Transcripts\n    Show transcript table\n\n\n    Summary\n\n    Name\n    FGF14 (HGNC Symbol)\n\n    CCDS\n    This gene is a member of the Human CCDS set: CCDS9500.1, CCDS9501.1\n\n    UniProtKB\n    This gene has proteins that correspond to the following UniProtKB identifiers: Q92915\n\n    RefSeq\n    This Ensembl/Gencode gene contains transcript(s) for which we have selected identical RefSeq transcript(s). If there are other RefSeq transcripts available they will be in the External references table\n\n    Ensembl version\n    ENSG00000102466.16\n\n    Other assemblies\n    This gene maps to 102,363,154-103,054,807 in GRCh37 coordinates.\n\n    View this locus in the GRCh37 archive: ENSG00000102466\n\n    Gene type\n    Protein coding\n\n    Annotation method\n    Annotation for this gene includes both automatic annotation from Ensembl and Havana manual curation, see article.\n\n    Annotation Attributes\n    overlapping locus [Definitions]","metadata":{"execution":{"iopub.status.busy":"2022-09-29T19:49:25.439437Z","iopub.execute_input":"2022-09-29T19:49:25.439942Z","iopub.status.idle":"2022-09-29T19:49:25.454164Z","shell.execute_reply.started":"2022-09-29T19:49:25.439904Z","shell.execute_reply":"2022-09-29T19:49:25.452261Z"}}},{"cell_type":"markdown","source":"### Bottom one - \n\n    # ENSG00000225489    2323\n\n    # ENSG00000225489 Gene - Novel Transcript\n    # RNA Gene (Updated: Aug 30, 2022 ; GC09P007546  ; GIFtS: 10 )  \n","metadata":{}},{"cell_type":"markdown","source":"# Sample submission","metadata":{}},{"cell_type":"code","source":"print('Prepare submission dataset with zeros')\ndf_submission_full = pd.DataFrame(index = range(65744180))\ndf_submission_full.index.name = 'row_id'\ndf_submission_full['target'] = 0\ndf_submission_full # 4.5G RAM used","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:54.399784Z","iopub.execute_input":"2022-10-02T07:32:54.400131Z","iopub.status.idle":"2022-10-02T07:32:54.542021Z","shell.execute_reply.started":"2022-10-02T07:32:54.400099Z","shell.execute_reply":"2022-10-02T07:32:54.540772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\ndf_submission_citeseq_prepare = pd.DataFrame(index = list_cells_barcodes, columns = df_cite_train_y.columns, data=1)\n\ntmp = 9\nif tmp == 1: # Mean Submission # Score: 0.168\n    m = df_cite_train_y.mean(axis = 0)\n    print('Means for submission ')\n    submit_filename_postfix = 'mean_citeseq'\n    print(m)\n    df_submission_citeseq_prepare = df_submission_citeseq_prepare * m\n    \nelif tmp == 2: # Median : \n    m = df_cite_train_y.median(axis = 0)\n    print('Medians for submission ')\n    submit_filename_postfix = 'median_citeseq'\n    print(m)\n    df_submission_citeseq_prepare = df_submission_citeseq_prepare * m\n    \nelif tmp == 6: # 'mean_clipped*_*_citeseq'\n    for col in df_submission_citeseq_prepare.columns:\n        v = df_cite_train_y[col]\n        m1 = np.percentile(v,0.5)\n        m2 = np.percentile(v,99.5)\n        v = np.clip( v, m1,m2)\n        df_submission_citeseq_prepare[col] = np.mean(v)\n    submit_filename_postfix = 'mean_clipped05_995_citeseq'\n    print(submit_filename_postfix)\nelif tmp == 7: # 'mean_clipped*_*_citeseq'\n    for col in df_submission_citeseq_prepare.columns:\n        v = df_cite_train_y[col]\n        m1 = np.percentile(v,0.9)\n        m2 = np.percentile(v,99.9)\n        v = np.clip( v, m1,m2)\n        df_submission_citeseq_prepare[col] = np.mean(v)\n    submit_filename_postfix = 'mean_clipped09_999_citeseq'\n    print(submit_filename_postfix)\nelif tmp == 8: # 'mean_clipped*_*_citeseq'\n    for col in df_submission_citeseq_prepare.columns:\n        v = df_cite_train_y[col]\n        m1 = np.percentile(v,0.99)\n        m2 = np.percentile(v,99.99)\n        v = np.clip( v, m1,m2)\n        df_submission_citeseq_prepare[col] = np.mean(v)\n    submit_filename_postfix = 'mean_clipped099_9999_citeseq'\n    print(submit_filename_postfix)\nelif tmp == 9: # 'mean_clipped*_*_citeseq'\n    for col in df_submission_citeseq_prepare.columns:\n        v = df_cite_train_y[col]\n        m1 = np.percentile(v,0.1)\n        m2 = np.percentile(v,99.9)\n        v = np.clip( v, m1,m2)\n        df_submission_citeseq_prepare[col] = np.mean(v)\n    submit_filename_postfix = 'mean_clipped01_999_citeseq'\n    print(submit_filename_postfix)\n    \n    \ndisplay(df_submission_citeseq_prepare)","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:54.543837Z","iopub.execute_input":"2022-10-02T07:32:54.544203Z","iopub.status.idle":"2022-10-02T07:32:56.378410Z","shell.execute_reply.started":"2022-10-02T07:32:54.544161Z","shell.execute_reply":"2022-10-02T07:32:56.377281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_submission_full['target'].iloc[:6812820] = df_submission_citeseq_prepare.values.ravel()","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:56.379937Z","iopub.execute_input":"2022-10-02T07:32:56.380549Z","iopub.status.idle":"2022-10-02T07:32:56.645088Z","shell.execute_reply.started":"2022-10-02T07:32:56.380513Z","shell.execute_reply":"2022-10-02T07:32:56.643845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_submission_full","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:56.646555Z","iopub.execute_input":"2022-10-02T07:32:56.646906Z","iopub.status.idle":"2022-10-02T07:32:56.659946Z","shell.execute_reply.started":"2022-10-02T07:32:56.646875Z","shell.execute_reply":"2022-10-02T07:32:56.658627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nif 0: \n    df_submission_full.to_csv('submission_cite_seq_'+submit_filename_postfix +'.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:56.661654Z","iopub.execute_input":"2022-10-02T07:32:56.662074Z","iopub.status.idle":"2022-10-02T07:32:56.673744Z","shell.execute_reply.started":"2022-10-02T07:32:56.662035Z","shell.execute_reply":"2022-10-02T07:32:56.672300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission by MEAN with groupby by cell_types and days","metadata":{}},{"cell_type":"code","source":"df_cite_train_y","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:56.675205Z","iopub.execute_input":"2022-10-02T07:32:56.676003Z","iopub.status.idle":"2022-10-02T07:32:56.735612Z","shell.execute_reply.started":"2022-10-02T07:32:56.675960Z","shell.execute_reply":"2022-10-02T07:32:56.734380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d = pd.DataFrame(index = df_cite_train_y.index )\nprint(d.shape, df_cite_train_y.shape)\nd = d.join( df_cell.set_index('cell_id'),how = 'left')\nd['TMP'] = (d['cell_type']) # (d['day'].apply(lambda x: str(x) + '_')) #  + \n\nsubmit_filename_postfix = 'mean_groupby_cell_types'\n\nd = d.join(df_cite_train_y )\nm = d.groupby('TMP').mean()\ndisplay(m.head(1))\nm = m.iloc[:,2:]\nm","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:56.737180Z","iopub.execute_input":"2022-10-02T07:32:56.737679Z","iopub.status.idle":"2022-10-02T07:32:57.028369Z","shell.execute_reply.started":"2022-10-02T07:32:56.737633Z","shell.execute_reply":"2022-10-02T07:32:57.027144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"m.T.corr()","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:57.030441Z","iopub.execute_input":"2022-10-02T07:32:57.030916Z","iopub.status.idle":"2022-10-02T07:32:57.048388Z","shell.execute_reply.started":"2022-10-02T07:32:57.030872Z","shell.execute_reply":"2022-10-02T07:32:57.047183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d = pd.DataFrame(index = list_cells_barcodes )\nd = d.join(df_cell.set_index('cell_id'),how = 'left')\nd['TMP'] = (d['cell_type']) #  (d['day'].apply(lambda x: str(x) + '_'))  #  +\nd = pd.merge(d, m.reset_index() , on ='TMP', how = 'left')\nd = d.fillna(0)\nd = d.iloc[:,5:]\nd","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:57.049873Z","iopub.execute_input":"2022-10-02T07:32:57.050790Z","iopub.status.idle":"2022-10-02T07:32:57.244991Z","shell.execute_reply.started":"2022-10-02T07:32:57.050755Z","shell.execute_reply":"2022-10-02T07:32:57.243892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_submission_full['target'].iloc[:6812820] = d.values.ravel()\ndf_submission_full","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:57.246506Z","iopub.execute_input":"2022-10-02T07:32:57.246938Z","iopub.status.idle":"2022-10-02T07:32:57.280831Z","shell.execute_reply.started":"2022-10-02T07:32:57.246904Z","shell.execute_reply":"2022-10-02T07:32:57.279919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\ndf_submission_full.to_csv('submission_cite_seq_'+submit_filename_postfix +'.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:32:57.281803Z","iopub.execute_input":"2022-10-02T07:32:57.282113Z","iopub.status.idle":"2022-10-02T07:34:44.101421Z","shell.execute_reply.started":"2022-10-02T07:32:57.282085Z","shell.execute_reply":"2022-10-02T07:34:44.100372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('%.1f seconds passed total '%(time.time()-t0start) )\n","metadata":{"execution":{"iopub.status.busy":"2022-10-02T07:34:44.103258Z","iopub.execute_input":"2022-10-02T07:34:44.103640Z","iopub.status.idle":"2022-10-02T07:34:44.109998Z","shell.execute_reply.started":"2022-10-02T07:34:44.103605Z","shell.execute_reply":"2022-10-02T07:34:44.108765Z"},"trusted":true},"execution_count":null,"outputs":[]}]}