{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-12-26T04:41:25.649363Z","iopub.execute_input":"2022-12-26T04:41:25.649786Z","iopub.status.idle":"2022-12-26T04:41:25.666937Z","shell.execute_reply.started":"2022-12-26T04:41:25.649754Z","shell.execute_reply":"2022-12-26T04:41:25.665482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In this notebook, I would like to further explore the protein markers in the CITE dataset and make associations with cell type. I first explored marker associations with cell types and looked at common markers associated with certain lineages and on that basis selected markers for the decision trees. \n\nI find it of great interest to understand how to use develop models for the learning of cell identities in single-cell data through biomarker identification, such as in the following data paper.\nhttps://www.nature.com/articles/s41467-021-23196-8\n\nThe CITE dataset would be a good opportunity to explore such associations. I have been learning more about decision trees in ML models and once identifying some interesting biomarkers I wanted to apply and see if they can be used in cell type identification.\n\nI did look at the protein markers and as the training dataset from this competiton is looking at cell types from early on in hematopoiesis on the following cell type;  \nMasP = Mast Cell Progenitor\nMkP = Megakaryocyte Progenitor\nNeuP = Neutrophil Progenitor\nMoP = Monocyte Progenitor\nEryP = Erythrocyte Progenitor\nHSC = Hematoploetic Stem Cell\nBP = B-Cell Progenitor\n'The added challenge of this competition is that the test data will be from a later time point than any time point in the training data.' Therfore, as several others in the competition have pointed out it may be hard to model the test data set on the training.\n\nHematopoietic Stem Cells (HSCs) are able to differentiate into cells of two primary lineages, lymphoid and myeloid. Cells of the lymphoid lineage develop during the process of lymphopoiesis and include B Cells, T Cells, Natural Killer (NK) Cells, and Dendritic Cells. There are certain protein markers that distingusih between cell types, especially between myeloid and lymphoid origin. Examples of myeloid lineage markers include pan-myeloid marker CD11b, CD206 for M2-type macrophages, CD68, and CD15 for neutrophils. While some markers are unique to each cell class, often a combinatorial analysis of multiple markers is required to assess the true phenotype of the myeloid cell lineages.Also, CD11c is a type I transmembrane protein that is expressed on monocytes, granulocytes, (a subset of B cells), dendritic cells, and macrophages. Then there are 'classically' considered lymphoid-associated antigens (CD2, CD3, CD4, CD5, CD7, CD8, CD10, CD19, CD20, and CD22).","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.metrics import plot_confusion_matrix, accuracy_score, precision_score, recall_score, roc_auc_score, f1_score\n\n# allow plots to appear in the notebook\n%matplotlib inline\nplt.rcParams['figure.figsize'] = (6, 4)\nplt.rcParams['font.size'] = 14","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:41:25.689010Z","iopub.execute_input":"2022-12-26T04:41:25.689459Z","iopub.status.idle":"2022-12-26T04:41:25.699921Z","shell.execute_reply.started":"2022-12-26T04:41:25.689422Z","shell.execute_reply":"2022-12-26T04:41:25.698471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from matplotlib.colors import ListedColormap, LinearSegmentedColormap\nfrom matplotlib.offsetbox import AnnotationBbox, OffsetImage\nfrom sklearn.decomposition import TruncatedSVD\nfrom matplotlib.patches import Rectangle\nimport matplotlib.patches as patches\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm\nimport seaborn as sns\nimport pandas as pd\nimport numpy as np\nimport itertools\nimport warnings\nimport os\nimport gc\n!pip install scanpy\nimport scanpy as sc\nimport anndata\nimport time\nt0start = time.time()\nimport sys","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:41:25.730850Z","iopub.execute_input":"2022-12-26T04:41:25.732004Z","iopub.status.idle":"2022-12-26T04:41:37.732133Z","shell.execute_reply.started":"2022-12-26T04:41:25.731956Z","shell.execute_reply":"2022-12-26T04:41:37.730922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MAIN_DIRECT = '../input/open-problems-multimodal'\nmeta_data_path = os.path.join(MAIN_DIRECT + '/metadata.csv')\neval_data_path  = os.path.join(MAIN_DIRECT + '/evaluation_ids.csv')\nsubmission_path = os.path.join(MAIN_DIRECT + '/sample_submission.csv')\n\ntest_cite_path = os.path.join(MAIN_DIRECT + '/test_cite_inputs.h5')\ntest_cite_path_day2 = os.path.join(MAIN_DIRECT + '/test_cite_inputs_day_2_donor_27678.h5')\ntest_multi_path = os.path.join(MAIN_DIRECT + '/test_multi_inputs.h5')\n\n\ntrain_cite_path = os.path.join(MAIN_DIRECT + '/train_cite_inputs.h5')\ntrain_cite_target_path = os.path.join(MAIN_DIRECT + '/train_cite_targets.h5')\ntrain_multi_path = os.path.join(MAIN_DIRECT + '/train_multi_inputs.h5')\ntrain_multi_target_path = os.path.join(MAIN_DIRECT + '/train_multi_targets.h5')","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:41:37.736544Z","iopub.execute_input":"2022-12-26T04:41:37.736929Z","iopub.status.idle":"2022-12-26T04:41:37.745860Z","shell.execute_reply.started":"2022-12-26T04:41:37.736885Z","shell.execute_reply":"2022-12-26T04:41:37.744522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"meta_data = pd.read_csv(meta_data_path)\ndisplay(meta_data,'Meta Data')","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:41:37.747789Z","iopub.execute_input":"2022-12-26T04:41:37.748319Z","iopub.status.idle":"2022-12-26T04:41:38.095702Z","shell.execute_reply.started":"2022-12-26T04:41:37.748272Z","shell.execute_reply":"2022-12-26T04:41:38.094649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"citeseq_data = meta_data[meta_data['technology'] == 'citeseq']\ndisplay(citeseq_data,'CITEseq Data')","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:41:38.099452Z","iopub.execute_input":"2022-12-26T04:41:38.100098Z","iopub.status.idle":"2022-12-26T04:41:38.152642Z","shell.execute_reply.started":"2022-12-26T04:41:38.100043Z","shell.execute_reply":"2022-12-26T04:41:38.151266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_hdf_cite = pd.read_hdf(train_cite_path)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:41:38.154455Z","iopub.execute_input":"2022-12-26T04:41:38.154936Z","iopub.status.idle":"2022-12-26T04:42:40.927585Z","shell.execute_reply.started":"2022-12-26T04:41:38.154889Z","shell.execute_reply":"2022-12-26T04:42:40.926046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_hdf_cite.info(memory_usage='deep')","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:42:40.930304Z","iopub.execute_input":"2022-12-26T04:42:40.930716Z","iopub.status.idle":"2022-12-26T04:42:43.048530Z","shell.execute_reply.started":"2022-12-26T04:42:40.930680Z","shell.execute_reply":"2022-12-26T04:42:43.047253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_hdf_cite_target = pd.read_hdf(train_cite_target_path)\ndisplay(train_hdf_cite_target,'Train HDF Target Data')","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:42:43.050130Z","iopub.execute_input":"2022-12-26T04:42:43.051089Z","iopub.status.idle":"2022-12-26T04:42:43.890371Z","shell.execute_reply.started":"2022-12-26T04:42:43.051051Z","shell.execute_reply":"2022-12-26T04:42:43.889059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2 = pd.merge(train_hdf_cite_target, citeseq_data, on=[\"cell_id\"])","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:42:43.891708Z","iopub.execute_input":"2022-12-26T04:42:43.892038Z","iopub.status.idle":"2022-12-26T04:42:44.023009Z","shell.execute_reply.started":"2022-12-26T04:42:43.892008Z","shell.execute_reply":"2022-12-26T04:42:44.021833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df2)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:42:44.024618Z","iopub.execute_input":"2022-12-26T04:42:44.024972Z","iopub.status.idle":"2022-12-26T04:42:44.055180Z","shell.execute_reply.started":"2022-12-26T04:42:44.024933Z","shell.execute_reply":"2022-12-26T04:42:44.053845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ng =sns.scatterplot(x=\"CD19\", y=\"CD22\",\n              hue=\"cell_type\",\n              data=df2);\ng.set(xscale=\"log\");\n","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:42:44.059250Z","iopub.execute_input":"2022-12-26T04:42:44.059647Z","iopub.status.idle":"2022-12-26T04:42:50.591754Z","shell.execute_reply.started":"2022-12-26T04:42:44.059615Z","shell.execute_reply":"2022-12-26T04:42:50.590442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"CD22/CD19 maker can identify two clusters, HSC in blue and EryP in orange, so this would be interesting to see if using these markers in the decision tree below will identify these cell types.","metadata":{}},{"cell_type":"code","source":"g =sns.scatterplot(x=\"CD11c\", y=\"CD3\",\n              hue=\"cell_type\",\n              data=df2);\ng.set(xscale=\"log\");","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:42:50.593577Z","iopub.execute_input":"2022-12-26T04:42:50.594078Z","iopub.status.idle":"2022-12-26T04:42:56.566535Z","shell.execute_reply.started":"2022-12-26T04:42:50.594032Z","shell.execute_reply":"2022-12-26T04:42:56.565322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A clear disctinction can be made between cell types based on CD11c marker expression and we can see a pink cluster to the right of Monocytic Progenitors (MoP), red cluster of Mast Prognitors (MasP) and orange cluster or Erythroid progenitors (EryP). CD11c would be a good marker for the decision tree","metadata":{}},{"cell_type":"code","source":"g =sns.scatterplot(x=\"HLA-DR\", y=\"CD3\",\n              hue=\"cell_type\",\n              data=df2);\ng.set(xscale=\"log\");","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:42:56.568283Z","iopub.execute_input":"2022-12-26T04:42:56.569289Z","iopub.status.idle":"2022-12-26T04:43:02.916743Z","shell.execute_reply.started":"2022-12-26T04:42:56.569241Z","shell.execute_reply":"2022-12-26T04:43:02.915411Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"HLA-DR are found on antigen presenting cells (APCs) such as dendritic cells, B-lymphocytes, monocytes and macrophages and CD3 is a T cell marker. These are two very different markers expressed on very different cell types and from the plot above we can see that cells expressing high levels of HLA-DR in HSC express verying levels of CD3 (0-7.5).","metadata":{}},{"cell_type":"code","source":"g =sns.scatterplot(x=\"CD45RA\", y=\"CD16\",\n              hue=\"cell_type\",\n              data=df2);\ng.set(xscale=\"log\");","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:02.918448Z","iopub.execute_input":"2022-12-26T04:43:02.918822Z","iopub.status.idle":"2022-12-26T04:43:08.973764Z","shell.execute_reply.started":"2022-12-26T04:43:02.918789Z","shell.execute_reply":"2022-12-26T04:43:08.972506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"CD45RA and CD16 identify a Neutrophil cluster, in fact CD16b is a neutrophil marker.","metadata":{}},{"cell_type":"code","source":"g =sns.scatterplot(x=\"CD45RA\", y=\"CD19\",\n              hue=\"cell_type\",\n              data=df2);\ng.set(xscale=\"log\");","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:08.975774Z","iopub.execute_input":"2022-12-26T04:43:08.977135Z","iopub.status.idle":"2022-12-26T04:43:15.081502Z","shell.execute_reply.started":"2022-12-26T04:43:08.977081Z","shell.execute_reply":"2022-12-26T04:43:15.080302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"CD19 is a B cell marker and we see a few cells (brown) highly expressing in the B cell prognitor. Expression of this cell surface protein is restricted to B cell lymphocytes. This protein is a reliable marker for pre-B cells.","metadata":{}},{"cell_type":"code","source":"g =sns.scatterplot(x=\"CD11c\", y=\"CD3\",\n              hue=\"day\",\n              data=df2);\ng.set(xscale=\"log\");","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:15.083309Z","iopub.execute_input":"2022-12-26T04:43:15.083733Z","iopub.status.idle":"2022-12-26T04:43:18.779104Z","shell.execute_reply.started":"2022-12-26T04:43:15.083698Z","shell.execute_reply":"2022-12-26T04:43:18.778186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the plot above I wantde to see if the markers clustered to specific days, it looks like a well mixed population for CD3 and CD11c and a similar expression pattern from days 2-4.","metadata":{}},{"cell_type":"code","source":"g =sns.scatterplot(x=\"CD19\", y=\"CD22\",\n              hue=\"day\",\n              data=df2);\ng.set(xscale=\"log\");","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:18.780931Z","iopub.execute_input":"2022-12-26T04:43:18.781337Z","iopub.status.idle":"2022-12-26T04:43:22.877663Z","shell.execute_reply.started":"2022-12-26T04:43:18.781302Z","shell.execute_reply":"2022-12-26T04:43:22.876525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"CD19/CD22 also expressed similarly from days 2-4, cell types expressing high CD19/CD22 are more from day 2 and 3 and less from day 4. ","metadata":{}},{"cell_type":"code","source":"g =sns.scatterplot(x=\"CD45RA\", y=\"CD19\",\n              hue=\"day\",\n              data=df2);\ng.set(xscale=\"log\");","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:22.879516Z","iopub.execute_input":"2022-12-26T04:43:22.879971Z","iopub.status.idle":"2022-12-26T04:43:26.644124Z","shell.execute_reply.started":"2022-12-26T04:43:22.879926Z","shell.execute_reply":"2022-12-26T04:43:26.643198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2.to_csv(r'/kaggle/working/my_data2.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:26.645525Z","iopub.execute_input":"2022-12-26T04:43:26.646424Z","iopub.status.idle":"2022-12-26T04:43:37.034928Z","shell.execute_reply.started":"2022-12-26T04:43:26.646387Z","shell.execute_reply":"2022-12-26T04:43:37.033123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df10 = pd.cut(df2.CD38, bins=3, labels=np.arange(3), right=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.037320Z","iopub.execute_input":"2022-12-26T04:43:37.037793Z","iopub.status.idle":"2022-12-26T04:43:37.049859Z","shell.execute_reply.started":"2022-12-26T04:43:37.037747Z","shell.execute_reply":"2022-12-26T04:43:37.048608Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df10.to_csv(r'/kaggle/working/my_data10.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.051480Z","iopub.execute_input":"2022-12-26T04:43:37.052390Z","iopub.status.idle":"2022-12-26T04:43:37.123700Z","shell.execute_reply.started":"2022-12-26T04:43:37.052331Z","shell.execute_reply":"2022-12-26T04:43:37.122644Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df12 = pd.cut(df2.CD16, bins=3, labels=np.arange(3), right=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.125530Z","iopub.execute_input":"2022-12-26T04:43:37.126536Z","iopub.status.idle":"2022-12-26T04:43:37.135626Z","shell.execute_reply.started":"2022-12-26T04:43:37.126496Z","shell.execute_reply":"2022-12-26T04:43:37.134530Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df12.to_csv(r'/kaggle/working/my_data12.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.138384Z","iopub.execute_input":"2022-12-26T04:43:37.139219Z","iopub.status.idle":"2022-12-26T04:43:37.205403Z","shell.execute_reply.started":"2022-12-26T04:43:37.139154Z","shell.execute_reply":"2022-12-26T04:43:37.203923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df13 = pd.cut(df2.CD3, bins=3, labels=np.arange(3), right=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.207379Z","iopub.execute_input":"2022-12-26T04:43:37.208243Z","iopub.status.idle":"2022-12-26T04:43:37.218541Z","shell.execute_reply.started":"2022-12-26T04:43:37.208067Z","shell.execute_reply":"2022-12-26T04:43:37.217107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df13.to_csv(r'/kaggle/working/my_data13.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.220494Z","iopub.execute_input":"2022-12-26T04:43:37.220854Z","iopub.status.idle":"2022-12-26T04:43:37.285950Z","shell.execute_reply.started":"2022-12-26T04:43:37.220822Z","shell.execute_reply":"2022-12-26T04:43:37.284749Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df14 = pd.cut(df2.CD11c, bins=3, labels=np.arange(3), right=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.287782Z","iopub.execute_input":"2022-12-26T04:43:37.288145Z","iopub.status.idle":"2022-12-26T04:43:37.298401Z","shell.execute_reply.started":"2022-12-26T04:43:37.288114Z","shell.execute_reply":"2022-12-26T04:43:37.297216Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df14.to_csv(r'/kaggle/working/my_data14.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.299806Z","iopub.execute_input":"2022-12-26T04:43:37.300845Z","iopub.status.idle":"2022-12-26T04:43:37.366157Z","shell.execute_reply.started":"2022-12-26T04:43:37.300809Z","shell.execute_reply":"2022-12-26T04:43:37.364969Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df4 = df2['cell_type']","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.367685Z","iopub.execute_input":"2022-12-26T04:43:37.368070Z","iopub.status.idle":"2022-12-26T04:43:37.374313Z","shell.execute_reply.started":"2022-12-26T04:43:37.368038Z","shell.execute_reply":"2022-12-26T04:43:37.373322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df4)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.380950Z","iopub.execute_input":"2022-12-26T04:43:37.381406Z","iopub.status.idle":"2022-12-26T04:43:37.389545Z","shell.execute_reply.started":"2022-12-26T04:43:37.381371Z","shell.execute_reply":"2022-12-26T04:43:37.388062Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df4.to_csv(r'/kaggle/working/my_data4.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.391042Z","iopub.execute_input":"2022-12-26T04:43:37.391447Z","iopub.status.idle":"2022-12-26T04:43:37.462513Z","shell.execute_reply.started":"2022-12-26T04:43:37.391408Z","shell.execute_reply":"2022-12-26T04:43:37.461115Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df6 = pd.read_csv('../input/data6/my_data6.csv')","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.464460Z","iopub.execute_input":"2022-12-26T04:43:37.464844Z","iopub.status.idle":"2022-12-26T04:43:37.490375Z","shell.execute_reply.started":"2022-12-26T04:43:37.464812Z","shell.execute_reply":"2022-12-26T04:43:37.489013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df7 = pd.read_csv('../input/mydata7/my_data7.csv')","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.492055Z","iopub.execute_input":"2022-12-26T04:43:37.492543Z","iopub.status.idle":"2022-12-26T04:43:37.537490Z","shell.execute_reply.started":"2022-12-26T04:43:37.492509Z","shell.execute_reply":"2022-12-26T04:43:37.536227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df7)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.539185Z","iopub.execute_input":"2022-12-26T04:43:37.539682Z","iopub.status.idle":"2022-12-26T04:43:37.552460Z","shell.execute_reply.started":"2022-12-26T04:43:37.539648Z","shell.execute_reply":"2022-12-26T04:43:37.551140Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df6)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.553641Z","iopub.execute_input":"2022-12-26T04:43:37.554112Z","iopub.status.idle":"2022-12-26T04:43:37.565164Z","shell.execute_reply.started":"2022-12-26T04:43:37.554065Z","shell.execute_reply":"2022-12-26T04:43:37.563748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from matplotlib import pyplot as plt\nfrom sklearn import datasets\nfrom sklearn.tree import DecisionTreeClassifier \nfrom sklearn import tree","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.567298Z","iopub.execute_input":"2022-12-26T04:43:37.567838Z","iopub.status.idle":"2022-12-26T04:43:37.575687Z","shell.execute_reply.started":"2022-12-26T04:43:37.567799Z","shell.execute_reply":"2022-12-26T04:43:37.574447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Define X and y.\n\nfeature_cols = ['CD38']\nX = df7[feature_cols]\ny = df7['cell_type']","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.577516Z","iopub.execute_input":"2022-12-26T04:43:37.577936Z","iopub.status.idle":"2022-12-26T04:43:37.588650Z","shell.execute_reply.started":"2022-12-26T04:43:37.577893Z","shell.execute_reply":"2022-12-26T04:43:37.587189Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tree_n_clf = DecisionTreeClassifier(max_depth=None, random_state=1)\ntree_n_clf.fit(X, y)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.590510Z","iopub.execute_input":"2022-12-26T04:43:37.591123Z","iopub.status.idle":"2022-12-26T04:43:37.685342Z","shell.execute_reply.started":"2022-12-26T04:43:37.591079Z","shell.execute_reply":"2022-12-26T04:43:37.684032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(25,20))\n_ = tree.plot_tree(tree_n_clf, \n                   feature_names=df6.CD38,  \n                   class_names=df6.cell_type,\n                   filled=True)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:37.687040Z","iopub.execute_input":"2022-12-26T04:43:37.687548Z","iopub.status.idle":"2022-12-26T04:43:38.270776Z","shell.execute_reply.started":"2022-12-26T04:43:37.687497Z","shell.execute_reply":"2022-12-26T04:43:38.269425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"CD38 (non-lineage restricted gene) expression can identify 3 cell types; EryP (70,988 samples), HSC (13766 samples), NeuP (383 samples)","metadata":{}},{"cell_type":"code","source":"# Define X and y.\n\nfeature_cols = ['CD11c']\nX = df7[feature_cols]\ny = df7['cell_type']","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:38.272537Z","iopub.execute_input":"2022-12-26T04:43:38.272980Z","iopub.status.idle":"2022-12-26T04:43:38.281460Z","shell.execute_reply.started":"2022-12-26T04:43:38.272942Z","shell.execute_reply":"2022-12-26T04:43:38.280015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tree_n_clf = DecisionTreeClassifier(max_depth=None, random_state=1)\ntree_n_clf.fit(X, y)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:38.285009Z","iopub.execute_input":"2022-12-26T04:43:38.285406Z","iopub.status.idle":"2022-12-26T04:43:38.379552Z","shell.execute_reply.started":"2022-12-26T04:43:38.285370Z","shell.execute_reply":"2022-12-26T04:43:38.378227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(25,20))\n_ = tree.plot_tree(tree_n_clf, \n                   feature_names=df7.CD11c,  \n                   class_names=df7.cell_type,\n                   filled=True)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:38.381115Z","iopub.execute_input":"2022-12-26T04:43:38.381714Z","iopub.status.idle":"2022-12-26T04:43:38.959446Z","shell.execute_reply.started":"2022-12-26T04:43:38.381677Z","shell.execute_reply":"2022-12-26T04:43:38.958266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Did not see MoP and MasP with CD11c marker","metadata":{}},{"cell_type":"code","source":"fig.savefig(\"decistion_tree2.jpeg\")","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:38.960993Z","iopub.execute_input":"2022-12-26T04:43:38.962288Z","iopub.status.idle":"2022-12-26T04:43:39.151280Z","shell.execute_reply.started":"2022-12-26T04:43:38.962239Z","shell.execute_reply":"2022-12-26T04:43:39.149993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Define X and y.\n\nfeature_cols = ['CD19', 'CD22']\nX = df7[feature_cols]\ny = df7['cell_type']","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:39.152790Z","iopub.execute_input":"2022-12-26T04:43:39.153147Z","iopub.status.idle":"2022-12-26T04:43:39.160690Z","shell.execute_reply.started":"2022-12-26T04:43:39.153116Z","shell.execute_reply":"2022-12-26T04:43:39.158992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tree_n_clf = DecisionTreeClassifier(max_depth=None, random_state=1)\ntree_n_clf.fit(X, y)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:39.162419Z","iopub.execute_input":"2022-12-26T04:43:39.162916Z","iopub.status.idle":"2022-12-26T04:43:39.261547Z","shell.execute_reply.started":"2022-12-26T04:43:39.162878Z","shell.execute_reply":"2022-12-26T04:43:39.260302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(25,20))\n_ = tree.plot_tree(tree_n_clf, \n                   feature_names=df7.CD22,  \n                   class_names=df7.cell_type,\n                   filled=True)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:39.262968Z","iopub.execute_input":"2022-12-26T04:43:39.263341Z","iopub.status.idle":"2022-12-26T04:43:40.545337Z","shell.execute_reply.started":"2022-12-26T04:43:39.263307Z","shell.execute_reply":"2022-12-26T04:43:40.544079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Identify HSC (white box) with CD22 marker","metadata":{}},{"cell_type":"code","source":"# Define X and y.\n\nfeature_cols = ['CD16','CD11c']\nX = df7[feature_cols]\ny = df7['cell_type']","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:40.547078Z","iopub.execute_input":"2022-12-26T04:43:40.547424Z","iopub.status.idle":"2022-12-26T04:43:40.554365Z","shell.execute_reply.started":"2022-12-26T04:43:40.547395Z","shell.execute_reply":"2022-12-26T04:43:40.553001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tree_n_clf = DecisionTreeClassifier(max_depth=None, random_state=1)\ntree_n_clf.fit(X, y)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:40.555636Z","iopub.execute_input":"2022-12-26T04:43:40.556252Z","iopub.status.idle":"2022-12-26T04:43:40.652798Z","shell.execute_reply.started":"2022-12-26T04:43:40.556216Z","shell.execute_reply":"2022-12-26T04:43:40.651450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(25,20))\n_ = tree.plot_tree(tree_n_clf, \n                   feature_names=df7.CD11c,  \n                   class_names=df7.cell_type,\n                   filled=True)","metadata":{"execution":{"iopub.status.busy":"2022-12-26T04:43:40.655252Z","iopub.execute_input":"2022-12-26T04:43:40.655754Z","iopub.status.idle":"2022-12-26T04:43:41.701166Z","shell.execute_reply.started":"2022-12-26T04:43:40.655709Z","shell.execute_reply":"2022-12-26T04:43:41.699930Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Identify 2 samples with CD16/CD11c with NeuP","metadata":{}}]}