{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.0.5"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":59094,"databundleVersionId":7010844,"sourceType":"competition"},{"sourceId":7712331,"sourceType":"datasetVersion","datasetId":4441094}],"dockerImageVersionId":30618,"isInternetEnabled":true,"language":"r","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## <p style=\"border: 3px solid #3B2F2F; border-radius: 10px; padding: 15px; background-color: #ffc7ba; text-align: center; font-family: 'Arial', Times, serif; font-weight: bold; letter-spacing: 1px; color: #3B2F2F; font-size: 24px; margin-bottom: 10px;\"> NN on melted data and embedding-based feature representations using fastai</p>\n\nThis notebook is based on the 30th place solution by FRENIO REDEKER ([write up](https://www.kaggle.com/competitions/open-problems-single-cell-perturbations/discussion/461649)), espesially on 3 of his notebooks:\n- [Export TabMod NN Embeddings](https://www.kaggle.com/code/frenio/30-op2scp-export-tabmod-nn-embeddings#Train-without-PCA) - melting the data to get 3 features - cell type, compound and gene and use NN with embedding layer to generate embeddings for each\n- [Look at TabMod NN Embeddings](https://www.kaggle.com/code/frenio/30-op2scp-look-at-tabmod-nn-embeddings) - plotting 2 first principal components for each set of embeddings\n- [Tabular Model NN with PCA10 Denoising](https://www.kaggle.com/code/frenio/30-op2scp-tabular-model-nn-with-pca10-denoising) - submitting predictions generated with NN\n\nHere is the code for the same model written with fastai for R. I validate it with some modifications of embedding sizes, denoising and model architecture and parameters (in versions).\n\nIn the original version by FRENIO REDEKER:   \n**Public LB score 0.565  \nPrivat LB score 0.784**","metadata":{}},{"cell_type":"markdown","source":"****","metadata":{}},{"cell_type":"markdown","source":"**Install fastai (takes ~15 min)**","metadata":{}},{"cell_type":"code","source":"tictoc::tic()\ninstall.packages(\n    \"https://cran.r-project.org/src/contrib/Archive/pbkrtest/pbkrtest_0.5.1.tar.gz\",\n    repos = NULL, type = \"source\"\n)\n\ndevtools::install_github(\"eagerai/fastai\")\nlibrary(fastai)\nfastai::install_fastai(gpu = FALSE, cuda_version = '11.6', overwrite = FALSE)\ntictoc::toc()","metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","execution":{"iopub.status.busy":"2024-04-07T14:59:04.382040Z","iopub.execute_input":"2024-04-07T14:59:04.384290Z","iopub.status.idle":"2024-04-07T15:12:24.631786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Other packages**","metadata":{}},{"cell_type":"code","source":"library(arrow)\nlibrary(data.table)\nlibrary(qs)\nlibrary(ggplot2)\nlibrary(patchwork)\nlibrary(ggrepel)\nlibrary(pheatmap)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:24.634215Z","iopub.execute_input":"2024-04-07T15:12:24.662180Z","iopub.status.idle":"2024-04-07T15:12:25.197405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Helper functions**","metadata":{}},{"cell_type":"code","source":"shape <- function(dt) {\n    cat(\"\\nShape:\", nrow(dt), \"rows x\", ncol(dt), \"columns\")\n}\nfig <- function(width, heigth){\n  options(repr.plot.width = width, repr.plot.height = heigth)\n}","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:25.199983Z","iopub.execute_input":"2024-04-07T15:12:25.201408Z","iopub.status.idle":"2024-04-07T15:12:25.213376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Read Data","metadata":{}},{"cell_type":"markdown","source":"**Train data**","metadata":{}},{"cell_type":"code","source":"de_train <- read_parquet('/kaggle/input/open-problems-single-cell-perturbations/de_train.parquet')\nde_train <- setDT(de_train)\n\n# sample_genes <- sample(ncol(de_train)-5, 10)\n# de_train <- de_train[, c(1:5, sample_genes), with = FALSE]\n\nhead(de_train)\nshape(de_train)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:25.215727Z","iopub.execute_input":"2024-04-07T15:12:25.217001Z","iopub.status.idle":"2024-04-07T15:12:28.602596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Test data**","metadata":{}},{"cell_type":"code","source":"id_map <- fread(\"/kaggle/input/open-problems-single-cell-perturbations/id_map.csv\")\ngene_cols <- names(de_train)[unlist(lapply(de_train, is.numeric))]\nsuppressWarnings(id_map[, (gene_cols) := 0])\n\nhead(id_map)\nshape(id_map)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:28.604860Z","iopub.execute_input":"2024-04-07T15:12:28.606133Z","iopub.status.idle":"2024-04-07T15:12:28.658574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cat_features_cols <- c(\"cell_type\", \"sm_name\", \"gene\")","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:28.660918Z","iopub.execute_input":"2024-04-07T15:12:28.662110Z","iopub.status.idle":"2024-04-07T15:12:28.672847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Define Functions","metadata":{}},{"cell_type":"markdown","source":"**prepare data** - performs the following operations with a data table:\n- splitting to train and valid subsets (80/20) by adding a column 'idx' with either \"train\" or \"valid\" values \n- Reshaping the result (returns the data in long format)\n- if denoise = TRUE, before reshaping also performs:  \n1) Principal Component Analysis (PCA) on the target columns (genes) after scaling and centering of the data,   \n2) dimensionality reduction (retains only the first n_comp principal components from the PCA results),  \n3) inverse transformation (inversely transforms the reduced principal components back to the original space using the PCA rotation matrix, scale, and center),  \n4) bind the result for the train subset with the untouched valid subset. \n","metadata":{}},{"cell_type":"code","source":"prepare_data <- function(dt, denoise = FALSE, n_comp = 614, seed = 1) {\n  \n  gene_cols <- names(dt)[unlist(lapply(dt, is.numeric))]\n  id_cols <- c(\"cell_type\", \"sm_name\")\n  \n  melt_dt <- melt(\n    dt, value.name = \"value\",\n    variable.name = \"gene\",\n    variable.factor = FALSE,\n    measure.vars = gene_cols,\n    id.vars = id_cols\n  )\n  \n  set.seed(seed)\n  train_idx <- sample(1:nrow(melt_dt), size = floor(0.8 * nrow(melt_dt)))\n  valid_idx <- setdiff(1:nrow(melt_dt), train_idx)\n  melt_dt[train_idx, idx := \"train\"][valid_idx, idx := \"valid\"]\n  \n  if (denoise) {        \n    \n    pca_data <- prcomp(dt[, ..gene_cols], scale. = TRUE, center = TRUE)\n    reduced_data <- pca_data$x[, 1:n_comp]\n    \n    reduced_data_inv_trans <- t(\n      t(reduced_data %*% t(pca_data$rotation[, 1:n_comp])) *\n        pca_data$scale + pca_data$center\n    )\n    \n    denoised_melt_dt <- melt(\n      cbind(dt[, ..id_cols], reduced_data_inv_trans),\n      value.name = \"value\",\n      variable.name = \"gene\",\n      variable.factor = FALSE,\n      measure.vars = gene_cols,\n      id.vars = id_cols\n    )\n    \n    denoised_melt_dt[train_idx, idx := \"train\"][valid_idx, idx := \"valid\"]\n    denoised_melt_dt[valid_idx, value := melt_dt[valid_idx, value]]\n    return(denoised_melt_dt)\n  }\n  return(melt_dt)\n}","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:28.675316Z","iopub.execute_input":"2024-04-07T15:12:28.676550Z","iopub.status.idle":"2024-04-07T15:12:28.687518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train NN","metadata":{}},{"cell_type":"markdown","source":"**Load melted train dataset into the dataloader using specified number of batches**","metadata":{}},{"cell_type":"code","source":"train_melted <- prepare_data(de_train, denoise = TRUE, n_comp = 10)\n\ntrain_indices <- which(train_melted[, idx == \"train\"])\nvalid_indices <- which(train_melted[, idx == \"valid\"])\nbatch_size = 4096\n\ndls <- TabularDataTable(\n        train_melted[, !\"idx\"],\n        procs = list(Categorify()),\n        cat_names = cat_features_cols,\n        y_names = \"value\",\n        splits = list(train_indices, valid_indices)\n    )\ndls <- dataloaders(dls, bs = batch_size)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:28.689863Z","iopub.execute_input":"2024-04-07T15:12:28.691081Z","iopub.status.idle":"2024-04-07T15:12:34.475423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls$train$dataset","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:34.478008Z","iopub.execute_input":"2024-04-07T15:12:34.479523Z","iopub.status.idle":"2024-04-07T15:12:34.506940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls$valid$dataset","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:34.510052Z","iopub.execute_input":"2024-04-07T15:12:34.511647Z","iopub.status.idle":"2024-04-07T15:12:34.535322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Get min and max values of the target variable and set embedding dimensions**","metadata":{}},{"cell_type":"code","source":"y_min = ceiling(min(train_melted[, value]))\ny_max = ceiling(max(train_melted[, value]))\n\ncat(\"Min target value:\", y_min,\n    \"\\nMax target value\", y_max)\n\ncat(\"\\nEmbedding sizes:\\n\")\nemb_szs <- list(\n    cell_type = as.integer(5),\n    sm_name = as.integer(26),\n    gene = as.integer(600))\nemb_szs","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:34.556947Z","iopub.execute_input":"2024-04-07T15:12:34.558406Z","iopub.status.idle":"2024-04-07T15:12:34.590425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Clean up memory**","metadata":{}},{"cell_type":"code","source":"sizes <- rbindlist(lapply(ls(), function(x) {\n        data.table(variable = x,\n                   size = object.size(get(x))\n                  )\n    }))\nsizes[!(grepl('user|KAGGLE|Kernel', variable, ignore.case = TRUE) |\n        variable %in% c('auth'))\n     ]","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:34.592783Z","iopub.execute_input":"2024-04-07T15:12:34.594064Z","iopub.status.idle":"2024-04-07T15:12:34.630331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rm(de_train, prepare_data, shape, train_indices, valid_indices) ; g <- gc()","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:34.632704Z","iopub.execute_input":"2024-04-07T15:12:34.633963Z","iopub.status.idle":"2024-04-07T15:12:35.000758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Set the model**","metadata":{}},{"cell_type":"code","source":"nn = nn()\nlearn <- tabular_learner(\n    dls,\n    y_range = c(y_min, y_max),\n    emb_szs = emb_szs,\n    loss_func = nn$functional$mse_loss,\n    layers = c(1000, 500, 250),\n    metrics = list(mae(), rmse())\n)\nlearn","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:35.003148Z","iopub.execute_input":"2024-04-07T15:12:35.004415Z","iopub.status.idle":"2024-04-07T15:12:36.833141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cat(summary(learn))","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:36.835554Z","iopub.execute_input":"2024-04-07T15:12:36.837068Z","iopub.status.idle":"2024-04-07T15:12:36.887445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Find optimal learning rate**","metadata":{}},{"cell_type":"code","source":"lr_find(learn)\nplot_lr_find(learn)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:12:36.893345Z","iopub.execute_input":"2024-04-07T15:12:36.896495Z","iopub.status.idle":"2024-04-07T15:13:18.067216Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Fit model**","metadata":{}},{"cell_type":"code","source":"res <- fit_one_cycle(learn, n_epoch = 10, lr_max = 3e-4)\ng <- gc()","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:18.071033Z","iopub.execute_input":"2024-04-07T15:13:18.072341Z","iopub.status.idle":"2024-04-07T15:13:20.886731Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train history","metadata":{}},{"cell_type":"code","source":"res","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:20.889219Z","iopub.execute_input":"2024-04-07T15:13:20.890481Z","iopub.status.idle":"2024-04-07T15:13:20.909588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_loss(learn)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:20.911821Z","iopub.execute_input":"2024-04-07T15:13:20.913035Z","iopub.status.idle":"2024-04-07T15:13:21.633295Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Evaluate the model (rmse on entire train data)**","metadata":{}},{"cell_type":"code","source":"test_dl = learn$dls$test_dl(train_melted)\npreds = learn$get_preds(dl = test_dl)\npreds = as_array(preds[[1]])\n\ntrue_targets <- as.matrix(train_melted[, .(value)])\n\nrm(train_melted) ; g <- gc()\n\nmae <- mean((true_targets - preds)^2)\ncat(\"RMSE:\", sqrt(mae), \"\\n\", \"MAE:\", mae)\n\nrm(true_targets, preds, test_dl, mae) ; g <- gc()","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:21.636813Z","iopub.execute_input":"2024-04-07T15:13:21.638029Z","iopub.status.idle":"2024-04-07T15:13:22.513804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict","metadata":{}},{"cell_type":"code","source":"test_melted <- melt(id_map, value.name = \"value\",\n                    variable.name = \"gene\",\n                    variable.factor = FALSE,\n                    id.vars = c(\"id\", \"cell_type\", \"sm_name\"),\n                    measure.vars = gene_cols)\n\nrm(id_map) ; g <- gc()","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:22.517239Z","iopub.execute_input":"2024-04-07T15:13:22.518540Z","iopub.status.idle":"2024-04-07T15:13:22.818212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_dl = learn$dls$test_dl(test_melted[, !\"id\"])\npreds = learn$get_preds(dl = test_dl)\npreds = as_array(preds[[1]])\n\npreds <- test_melted[, .(id, gene, pred = preds[, 1])]\nsubmit <- dcast(preds, id ~ gene, value.var = \"pred\")","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:22.821903Z","iopub.execute_input":"2024-04-07T15:13:22.823261Z","iopub.status.idle":"2024-04-07T15:13:22.995466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submit[1:10, 1:10]","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.000426Z","iopub.execute_input":"2024-04-07T15:13:23.003893Z","iopub.status.idle":"2024-04-07T15:13:23.072987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fwrite(submit, \"submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.076486Z","iopub.execute_input":"2024-04-07T15:13:23.077860Z","iopub.status.idle":"2024-04-07T15:13:23.091394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Get embeddings","metadata":{}},{"cell_type":"code","source":"embedding_weights <- vector(mode = \"list\")\n\nfor (i in 0:(length(learn$model$embeds)-1)) {\n  embedding_weights[[i+1]] <-\n    learn$model$embeds[i]$weight$detach()\n}\nnames(embedding_weights) <- cat_features_cols\n\nembedding_weights","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.094999Z","iopub.execute_input":"2024-04-07T15:13:23.096373Z","iopub.status.idle":"2024-04-07T15:13:23.132452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lapply(dls$train$dataset$classes, function(x) x[0:7])","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.134882Z","iopub.execute_input":"2024-04-07T15:13:23.136141Z","iopub.status.idle":"2024-04-07T15:13:23.153198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"embeds_list <- lapply(1:length(embedding_weights), function(i) {\n    \n    emb_dt <- data.table(as_array(embedding_weights[[i]]))\n    setnames(emb_dt, names(emb_dt),\n             paste0(cat_features_cols[i], \"_emb\", 1:ncol(emb_dt))\n            )\n    \n    cat_names <- as.character(dls$train$dataset$classes[[cat_features_cols[i]]])\n    cat_names <- gsub(\"\\\\[|'|\\\\]\", \"\", unlist(strsplit(cat_names, \", \")))\n    \n    emb_dt <- cbind(cat_names, emb_dt)\n    setnames(emb_dt, \"cat_names\", cat_features_cols[i])\n    emb_dt <- emb_dt[-1]\n})\nnames(embeds_list) <- cat_features_cols","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.155472Z","iopub.execute_input":"2024-04-07T15:13:23.156708Z","iopub.status.idle":"2024-04-07T15:13:23.221720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lapply(embeds_list, head, 3)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.224215Z","iopub.execute_input":"2024-04-07T15:13:23.225510Z","iopub.status.idle":"2024-04-07T15:13:23.537291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"qsave(embeds_list, \"embeds_list.qs\")","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.539676Z","iopub.execute_input":"2024-04-07T15:13:23.540968Z","iopub.status.idle":"2024-04-07T15:13:23.558917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Explore embedding representations","metadata":{}},{"cell_type":"code","source":"plot_emb_pca <- function(emb_dt, labels = TRUE) {\n    \n    pca <- prcomp(emb_dt[, -1], center = TRUE, scale = TRUE)\n    plt_dt <- cbind(emb_dt, pca$x[, 1:2])\n    \n    plt <- ggplot(plt_dt, aes(x = PC1, y = PC2)) +\n        geom_point(size = 3) +\n        theme_bw(base_size = 22) +\n        theme(panel.grid = element_blank()) + #,aspect.ratio = 1\n        ggtitle(names(emb_dt)[1])\n    \n    if (labels) {\n        plt <- plt +\n            geom_label_repel(aes(label = unlist(emb_dt[, 1])), size = 6)\n    }\n    return(plt)\n}","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.561295Z","iopub.execute_input":"2024-04-07T15:13:23.562548Z","iopub.status.idle":"2024-04-07T15:13:23.573792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig(10, 10)\nplot_emb_pca(embeds_list[[\"cell_type\"]])","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:23.576680Z","iopub.execute_input":"2024-04-07T15:13:23.578192Z","iopub.status.idle":"2024-04-07T15:13:24.102959Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig(25, 18)\nplot_emb_pca(embeds_list[[\"sm_name\"]])","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:24.105404Z","iopub.execute_input":"2024-04-07T15:13:24.106720Z","iopub.status.idle":"2024-04-07T15:13:28.187943Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig(25, 18)\nplot_emb_pca(embeds_list[[\"gene\"]], labels = FALSE)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:28.190674Z","iopub.execute_input":"2024-04-07T15:13:28.191978Z","iopub.status.idle":"2024-04-07T15:13:28.697212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**PCA of embedding weights, biplot visualizing factor loadings for cell type embeddings**","metadata":{}},{"cell_type":"code","source":"fig(25, 10)\nfeature_col <- \"cell_type\"\nemb_dt <- embeds_list[[feature_col]]\npca <- prcomp(emb_dt[, -1], center = TRUE, scale = TRUE)\nbiplot(pca)","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:28.699697Z","iopub.execute_input":"2024-04-07T15:13:28.701017Z","iopub.status.idle":"2024-04-07T15:13:28.930223Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Cosine similarity**","metadata":{}},{"cell_type":"code","source":"calculate_cosine_similarity <- function(embeds_list, feature_col) {\n  embeds <- as.matrix(embeds_list[[feature_col]][, -1])\n    \n  norms <- sqrt(rowSums(embeds^2))  \n  dot_products <- embeds %*% t(embeds)  \n  similarity_matrix <- dot_products / (norms %*% t(norms))  \n  diag(similarity_matrix) <- 0\n  \n  rownames(similarity_matrix) <- \n    colnames(similarity_matrix) <- \n    unlist(embeds_list[[feature_col]][, 1])\n  \n  return(similarity_matrix)\n}","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:28.932883Z","iopub.execute_input":"2024-04-07T15:13:28.934221Z","iopub.status.idle":"2024-04-07T15:13:28.945888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig(12, 6)\nfeature_col <- \"cell_type\"\nsim_res <- calculate_cosine_similarity(embeds_list, feature_col)\n\nbreaksList = seq(min(sim_res), max(sim_res), by = 0.1)\nmyColors <- c(colorRampPalette(c(\"darkblue\", \"white\",\"darkred\"))(length(breaksList)))\n\nheat <- pheatmap(sim_res, fontsize = 16, fontsize_row = 16, fontsize_col = 16,\n                 color = myColors,\n                 breaks = breaksList,\n                 angle_col = 90,\n                 main = paste(\"Cosine similarity between\", feature_col, \"embeddings\"))","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:28.948293Z","iopub.execute_input":"2024-04-07T15:13:28.949590Z","iopub.status.idle":"2024-04-07T15:13:29.170884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig(30, 30)\nfeature_col <- \"sm_name\"\nsim_res <- calculate_cosine_similarity(embeds_list, feature_col)\n\nbreaksList = seq(min(sim_res), max(sim_res), by = 0.1)\nmyColors <- c(colorRampPalette(c(\"darkblue\", \"white\",\"darkred\"))(length(breaksList)))\n\nheat <- pheatmap(sim_res, fontsize = 12, fontsize_row = 12, fontsize_col = 12,\n                 color = myColors,\n                 breaks = breaksList,\n                 angle_col = 90,\n                 main = paste(\"Cosine similarity between\", feature_col, \"embeddings\"))","metadata":{"execution":{"iopub.status.busy":"2024-04-07T15:13:29.173463Z","iopub.execute_input":"2024-04-07T15:13:29.174743Z","iopub.status.idle":"2024-04-07T15:13:30.247985Z"},"trusted":true},"execution_count":null,"outputs":[]}]}