{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-11-04T22:38:51.015477Z","iopub.execute_input":"2023-11-04T22:38:51.015845Z","iopub.status.idle":"2023-11-04T22:38:51.459563Z","shell.execute_reply.started":"2023-11-04T22:38:51.015815Z","shell.execute_reply":"2023-11-04T22:38:51.458379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### The goal of this competition is to predict how small molecules change gene expresssion in different cell types. In this notebook I have explored a number of different small molecules to see how they have different cell types compositions. My hypothesis is small molecules affect gene expresssion changes which may impact protein levels and possibly transcriptional activation/signaling pathways downstream etc to ultimately decide cell type fate.\n\n### scRNA seq typically uses UMAP to determine cell identities and using the data provided I have used the scanpy tuntorial to pre-process the data and conduct UMAP analysis.\nhttps://scanpy-tutorials.readthedocs.io/en/latest/pbmc3k.html","metadata":{}},{"cell_type":"code","source":"!pip install scanpy\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport gc\nimport scanpy as sc","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:38:51.461879Z","iopub.execute_input":"2023-11-04T22:38:51.462611Z","iopub.status.idle":"2023-11-04T22:39:17.566921Z","shell.execute_reply.started":"2023-11-04T22:38:51.462565Z","shell.execute_reply":"2023-11-04T22:39:17.565628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata_train = pd.read_parquet('../input/open-problems-single-cell-perturbations/adata_train.parquet')\nadata_train","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:39:17.568897Z","iopub.execute_input":"2023-11-04T22:39:17.569560Z","iopub.status.idle":"2023-11-04T22:40:29.621535Z","shell.execute_reply.started":"2023-11-04T22:39:17.569524Z","shell.execute_reply":"2023-11-04T22:40:29.620479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"de_train = pd.read_parquet('../input/open-problems-single-cell-perturbations/de_train.parquet')\nde_train","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:29.624906Z","iopub.execute_input":"2023-11-04T22:40:29.625743Z","iopub.status.idle":"2023-11-04T22:40:34.005162Z","shell.execute_reply.started":"2023-11-04T22:40:29.625699Z","shell.execute_reply":"2023-11-04T22:40:34.003827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"de_train2 = de_train[(de_train[\"sm_name\"]) == 'Riociguat']\nde_train2","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:34.006720Z","iopub.execute_input":"2023-11-04T22:40:34.007173Z","iopub.status.idle":"2023-11-04T22:40:34.039664Z","shell.execute_reply.started":"2023-11-04T22:40:34.007128Z","shell.execute_reply":"2023-11-04T22:40:34.038335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"de_train3 = de_train[de_train[\"sm_name\"].isin([\"Riociguat\", \"MLN 2238\", \"Clotrimazole\"])]\nde_train3","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:34.043411Z","iopub.execute_input":"2023-11-04T22:40:34.043761Z","iopub.status.idle":"2023-11-04T22:40:34.086203Z","shell.execute_reply.started":"2023-11-04T22:40:34.043732Z","shell.execute_reply":"2023-11-04T22:40:34.085076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata_obs_meta = pd.read_csv('../input/open-problems-single-cell-perturbations/adata_obs_meta.csv')\nadata_obs_meta","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:34.087661Z","iopub.execute_input":"2023-11-04T22:40:34.088674Z","iopub.status.idle":"2023-11-04T22:40:35.312354Z","shell.execute_reply.started":"2023-11-04T22:40:34.088625Z","shell.execute_reply":"2023-11-04T22:40:35.309909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata_obs_meta2 = adata_obs_meta[[\"obs_id\", \"sm_name\"]]","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:35.313791Z","iopub.execute_input":"2023-11-04T22:40:35.315653Z","iopub.status.idle":"2023-11-04T22:40:35.328095Z","shell.execute_reply.started":"2023-11-04T22:40:35.315605Z","shell.execute_reply":"2023-11-04T22:40:35.326683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"merge_df2 = pd.merge(adata_obs_meta2, de_train3, on='sm_name')\nmerge_df2","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:35.329437Z","iopub.execute_input":"2023-11-04T22:40:35.329780Z","iopub.status.idle":"2023-11-04T22:40:37.041158Z","shell.execute_reply.started":"2023-11-04T22:40:35.329752Z","shell.execute_reply":"2023-11-04T22:40:37.039760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df67 = merge_df2[\"obs_id\"]","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:37.045141Z","iopub.execute_input":"2023-11-04T22:40:37.045576Z","iopub.status.idle":"2023-11-04T22:40:37.051655Z","shell.execute_reply.started":"2023-11-04T22:40:37.045542Z","shell.execute_reply":"2023-11-04T22:40:37.050285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"merge_df3 = pd.merge(df67, adata_train, on='obs_id')\nmerge_df3\n","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:40:37.053504Z","iopub.execute_input":"2023-11-04T22:40:37.053876Z","iopub.status.idle":"2023-11-04T22:42:09.610453Z","shell.execute_reply.started":"2023-11-04T22:40:37.053844Z","shell.execute_reply":"2023-11-04T22:42:09.608634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata_xdf = merge_df3.pivot_table(index=\"obs_id\", columns=\"gene\", values=\"count\").fillna(0)\n","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:09.612257Z","iopub.execute_input":"2023-11-04T22:42:09.612603Z","iopub.status.idle":"2023-11-04T22:42:24.111054Z","shell.execute_reply.started":"2023-11-04T22:42:09.612574Z","shell.execute_reply":"2023-11-04T22:42:24.110114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata_obs_meta = pd.read_csv('/kaggle/input/open-problems-single-cell-perturbations/adata_obs_meta.csv').set_index('obs_id')\nadata = sc.AnnData(\n    X=adata_xdf,\n    obs=adata_obs_meta.loc[adata_xdf.index],\n    var=pd.DataFrame(adata_xdf.columns, index=adata_xdf.columns)\n)","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:24.114818Z","iopub.execute_input":"2023-11-04T22:42:24.115226Z","iopub.status.idle":"2023-11-04T22:42:25.008668Z","shell.execute_reply.started":"2023-11-04T22:42:24.115193Z","shell.execute_reply":"2023-11-04T22:42:25.007553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:25.010113Z","iopub.execute_input":"2023-11-04T22:42:25.011060Z","iopub.status.idle":"2023-11-04T22:42:25.018467Z","shell.execute_reply.started":"2023-11-04T22:42:25.011025Z","shell.execute_reply":"2023-11-04T22:42:25.017342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sc.pp.filter_cells(adata, min_genes=200)\nsc.pp.filter_genes(adata, min_cells=3)","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:25.020127Z","iopub.execute_input":"2023-11-04T22:42:25.020511Z","iopub.status.idle":"2023-11-04T22:42:26.451482Z","shell.execute_reply.started":"2023-11-04T22:42:25.020481Z","shell.execute_reply":"2023-11-04T22:42:26.450186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata.var['mt'] = adata.var_names.str.startswith('MT-')  # annotate the group of mitochondrial genes as 'mt'\nsc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], percent_top=None, log1p=False, inplace=True)\nsc.pl.violin(adata, ['n_genes_by_counts', 'total_counts', 'pct_counts_mt'],\n             jitter=0.4, multi_panel=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:26.452883Z","iopub.execute_input":"2023-11-04T22:42:26.453198Z","iopub.status.idle":"2023-11-04T22:42:28.640546Z","shell.execute_reply.started":"2023-11-04T22:42:26.453170Z","shell.execute_reply":"2023-11-04T22:42:28.639525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sc.pl.scatter(adata, x='total_counts', y='pct_counts_mt')\nsc.pl.scatter(adata, x='total_counts', y='n_genes_by_counts')","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:28.641928Z","iopub.execute_input":"2023-11-04T22:42:28.642944Z","iopub.status.idle":"2023-11-04T22:42:29.317867Z","shell.execute_reply.started":"2023-11-04T22:42:28.642906Z","shell.execute_reply":"2023-11-04T22:42:29.316406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"adata = adata[adata.obs.n_genes_by_counts < 2500, :]\nadata = adata[adata.obs.pct_counts_mt < 10, :]","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:29.319743Z","iopub.execute_input":"2023-11-04T22:42:29.320505Z","iopub.status.idle":"2023-11-04T22:42:29.346615Z","shell.execute_reply.started":"2023-11-04T22:42:29.320459Z","shell.execute_reply":"2023-11-04T22:42:29.345599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\")\n!pip install leidenalg\nsc.pp.normalize_total(adata, target_sum=1e4)\nsc.pp.log1p(adata)\n","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:29.348558Z","iopub.execute_input":"2023-11-04T22:42:29.349374Z","iopub.status.idle":"2023-11-04T22:42:44.869500Z","shell.execute_reply.started":"2023-11-04T22:42:29.349327Z","shell.execute_reply":"2023-11-04T22:42:44.868262Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sc.pp.highly_variable_genes(adata)\nsc.pl.highly_variable_genes(adata)\nadata = adata[:, adata.var.highly_variable]\nsc.pp.regress_out(adata, ['total_counts'])\nsc.pp.scale(adata, max_value=10)\nsc.tl.pca(adata, svd_solver='arpack')\nsc.pl.pca_variance_ratio(adata, log=True)\n","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:44.871144Z","iopub.execute_input":"2023-11-04T22:42:44.871596Z","iopub.status.idle":"2023-11-04T22:42:59.042721Z","shell.execute_reply.started":"2023-11-04T22:42:44.871564Z","shell.execute_reply":"2023-11-04T22:42:59.041407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"### The UMAP shows 10 clusters when conducting leiden clustering.We have focused on looking at NK and T cell type distributions treated with 3 small molecules; Riociguat, MLN 2238 and Clotrimazole. \n\n### Observation 1: The UMAP shows donor types are well mixed indicating cluster differences are NOT attributed to donor type.\n\n### Observation 2: The UMAP shows that cell types do locate to specific leiden clusters, i.e Treg is clauter 8, NK cell cluster 5\n\n### Observation 3: The UMAP shows leiden clusters specific to certain drug types i.e MLN 2238 (Clsuter 0,3,4) which is a mix of CD4/CD8 T cells and no Treg.\n\n### In conclusion this study shows that cells with treatment to different small molecules show different cellular clustering patterns on the UMAP which can comprise of different cell types and is independent of donor type.\n","metadata":{}},{"cell_type":"code","source":"sc.pp.neighbors(adata, n_neighbors=10, n_pcs=30)\nsc.tl.umap(adata)\nsc.tl.leiden(adata)\nsc.pl.umap(adata, color=['leiden', 'cell_type'])\nsc.pl.umap(adata, color=['donor_id', 'sm_name'])","metadata":{"execution":{"iopub.status.busy":"2023-11-04T22:42:59.044321Z","iopub.execute_input":"2023-11-04T22:42:59.044654Z","iopub.status.idle":"2023-11-04T22:43:39.265646Z","shell.execute_reply.started":"2023-11-04T22:42:59.044625Z","shell.execute_reply":"2023-11-04T22:43:39.264102Z"},"trusted":true},"execution_count":null,"outputs":[]}]}