![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F577034%2Fb9f7d6b1595bae80513bbcd2fa786bb4%2Fs_plot.png?generation=1701966557440410&alt=media\")
I share a linear algebra method, an unbiased and reproducible approach, resulting in decent Private/Public scores of 0.768/0.582. My final submission is an ensemble of this Linear Algebra approach along with AE NN mimicking the linear algebra approach by NN, whose joint weight is 0.70, and the combination of the public notebooks, Pyboost, NN, and Linear SVR, with the total weight of 0.3. It turns out that the pure Linear Algebra model gets the best private leaderboard score.
\nIn the first half of this competition, I struggled to overcome the wall of a 0.600 public score. Some lucky runs of some NN models got over the wall, but not always. There was also the second formidable wall of 0.585, which blended the results of public models.
\nAfter the first half, I came up with this linear algebra approach and overcame the walls; the prediction is deterministic and reproducible and helped me to move on.
\nThe code of this linear model is available at my code notebook.
\nIt is a bit old, before the deep neural network, I had research experience using linear algebra to predict the missing values in a matrix - Missing Value Expectation of Matrix Data by Fixed Rank Approximation Algorithm, and it may inspire me.
\nAn assumption behind this method is that the differential expressions (DEs) of 18,211 genes at one cell line (e.g. NK cells) can be linearly transferable to those of another cell line (e.g. B Cells) on the same chemical perturbation.
\nA chemical perturbation triggers a complex activity interaction among the 18,211 genes, and different chemical perturbations make different activity patterns, resulting in various DEs from the same baseline condition. However, there would be an unseen “master rule” to govern these interactions on each cell line. If the master rule of one cell line (e.g. NK cells) could be similar to another cell (e.g. B cells), DEs on the same chemical perturbation could be predictable from one to another. Even not knowing the master rule of each cell line, their relationship among cell lines could be captured such that
\nwhere DE_i_xand DE_j_x are the differential expressions of cell line i (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) and j ('B cells', 'Myeloid cells') of chemical c perturbation, f is some special function.
\nMy approach is to assume that the linear system can be the proxy function, and to solve a system may provide the \"transfer\" such that
\nwhere DE_i_core and DE_j_core are m x n matrix, m is the number of chemicals shared by cel line i (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) and j ('B cells', 'Myeloid cells') as shown below (including positive controls) and n is the number of genes (18,211). T is an n x n matrix, considered a transformer matrix from one cell line to another.
\nNK cells B cells 17
\n NK cells Myeloid cells 17
\n T cells CD4+ B cells 17
\n T cells CD4+ Myeloid cells 17
\n T cells CD8+ B cells 15
\n T cells CD8+ Myeloid cells 15
\n T regulatory cells B cells 17
\n T regulatory cells Myeloid cells 17
Once the transformer T is solved on the core chemicals, it may be applied to predict DEs of the target cell (e.g. B cells) from the known cell (e.g. NK cells)
\nwhere DE_i_target is the DEs of prediction cell line i (B cells and Myeloid cells) of the target chemicals, 128 for B cells and 127 for Myoloid cells, and DE_j_target is the DEs of known cell line j (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) of the target chemicals.
\nThis approach may provide a robust and unbiased prediction.
\nUsing SVD to get the 1st and 2nd projected expression of the shared chemicals across 6 cell lines, and visualize the disperse among them (17 including positive controls, missing chemicals in T8 is replaced with those of T4). Well, it is a bit difficult to recognize a clear pattern.
\n
Here is a grid plot of the previous one by chemicals. In most of the chemicals, there is not much difference among 6 cell cline. However, NK cell seems to be similar to the B Cell and Myoloid cell on the chemical perturbations that create the larger difference among 6 cell lines such as Belinostat (one of the positive controls), MNL 2238, and Oprozomib. In the end, accuracy in predicting such chemicals may be important.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F577034%2Fa5aac54f6b18ab37d2cdaa3e3c4c921e%2Fsg_plot.png?generation=1701966669627694&alt=media\")
Solve linear system, transformer, from a base cell line (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) to a target cell line ('B cells', 'Myeloid cells') of the DE using the chemicals tested in the two cell lines (15 plus 2 positive control).
\nThe transformer can be computed by multiplying a pseudo-inverse of DE_i_core with DE_j_core from the left.
\nDE_i_core x T = DE_j_core
\nT = DE_i_core-t * DE_j_core
where DE_i_core-t is a pseudo-inverse of DE_i_core
Apply the transformer to the DE of the base cell line/target chemicals and get the DE of the target cell line/chemicals.
The simpler model consumes a lot of memory > 20Gb, and it cannot be performed on the free version of the Saturn Cloud, which I had used for convenience, I also propose an alternative solution using SVD with a projection space.
\nIt first reduces the gene dimension (18,211) to the full rank dimension of the entire data set (614) by SVD, create the transformer in the projected space, and applies the transformer in projected space from known cell to target cell on the target chemicals, reconstruct the original gene dimension of the prediction.
\nPlease refer to my notebook: the code can run on the notebook, and the result is deterministic, so reproducible.
\nThis method indeed can be used to diagnose the similarity of the \"master rule\" among the cell lines for scientific insight, and my result suggests that NK cells, along with T cells CD4+ can be stronger predictor cell lines for B Cell and Myeloid cells. Interestingly, T cells CD4+ is a better predictor for B cell while NK cell is a better predictor for Myeloid cells.
\nThis finding may align with the biological findings - NK cells can be derived from the myeloid lineage, Blood - The Journal of the American Society of Hematology 2011 3548 and T follicular helper cells cognately guide differentiation of antigen primed B cells in secondary lymphoid tissues - Trends in immunology 42.8 2021.
\n| base cell | \npublic | \nprivate | \n
|---|---|---|
| NK cells | \n0.784 | \n0.596 | \n
| T cells CD4+ | \n0.775 | \n0.607 | \n
| T cells CD8+ | \n0.959 | \n0.706 | \n
| T regulatory cells | \n0.834 | \n0.680 | \n
| base cell/target cell | \npublic | \nprivate | \n
|---|---|---|
| Predict B Cell by NK, Myeloid cells by T4 | \n0.786 | \n0.616 | \n
| Predict B Cell by T4, Myeloid cells by NK | \n0.773 | \n0.587 | \n
Differences in DE among certain cell lines can be captured linearly very well, and the linear relation is well transferable across different chemical responses, 0.768 in private and 0.582 in public
DE of NK cell and T cells CD4+ cells are quite predictable for that of B Cell and Myeloid cells
T cells CD4+ is a better predictor for B cell, and NK cells is a better predictor for Myeloid cell
I share a linear algebra method, an unbiased and reproducible approach, resulting in decent Private/Public scores of 0.768/0.582. My final submission is an ensemble of this Linear Algebra approach along with AE NN mimicking the linear algebra approach by NN, whose joint weight is 0.70, and the combination of the public notebooks, Pyboost, NN, and Linear SVR, with the total weight of 0.3. It turns out that the pure Linear Algebra model gets the best private leaderboard score.
\nIn the first half of this competition, I struggled to overcome the wall of a 0.600 public score. Some lucky runs of some NN models got over the wall, but not always. There was also the second formidable wall of 0.585, which blended the results of public models.
\nAfter the first half, I came up with this linear algebra approach and overcame the walls; the prediction is deterministic and reproducible and helped me to move on.
\nThe code of this linear model is available at my code notebook.
\nIt is a bit old, before the deep neural network, I had research experience using linear algebra to predict the missing values in a matrix - Missing Value Expectation of Matrix Data by Fixed Rank Approximation Algorithm, and it may inspire me.
\nAn assumption behind this method is that the differential expressions (DEs) of 18,211 genes at one cell line (e.g. NK cells) can be linearly transferable to those of another cell line (e.g. B Cells) on the same chemical perturbation.
\nA chemical perturbation triggers a complex activity interaction among the 18,211 genes, and different chemical perturbations make different activity patterns, resulting in various DEs from the same baseline condition. However, there would be an unseen “master rule” to govern these interactions on each cell line. If the master rule of one cell line (e.g. NK cells) could be similar to another cell (e.g. B cells), DEs on the same chemical perturbation could be predictable from one to another. Even not knowing the master rule of each cell line, their relationship among cell lines could be captured such that
\nwhere DE_i_xand DE_j_x are the differential expressions of cell line i (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) and j ('B cells', 'Myeloid cells') of chemical c perturbation, f is some special function.
\nMy approach is to assume that the linear system can be the proxy function, and to solve a system may provide the \"transfer\" such that
\nwhere DE_i_core and DE_j_core are m x n matrix, m is the number of chemicals shared by cel line i (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) and j ('B cells', 'Myeloid cells') as shown below (including positive controls) and n is the number of genes (18,211). T is an n x n matrix, considered a transformer matrix from one cell line to another.
\nNK cells B cells 17
\n NK cells Myeloid cells 17
\n T cells CD4+ B cells 17
\n T cells CD4+ Myeloid cells 17
\n T cells CD8+ B cells 15
\n T cells CD8+ Myeloid cells 15
\n T regulatory cells B cells 17
\n T regulatory cells Myeloid cells 17
Once the transformer T is solved on the core chemicals, it may be applied to predict DEs of the target cell (e.g. B cells) from the known cell (e.g. NK cells)
\nwhere DE_i_target is the DEs of prediction cell line i (B cells and Myeloid cells) of the target chemicals, 128 for B cells and 127 for Myoloid cells, and DE_j_target is the DEs of known cell line j (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) of the target chemicals.
\nThis approach may provide a robust and unbiased prediction.
\nUsing SVD to get the 1st and 2nd projected expression of the shared chemicals across 6 cell lines, and visualize the disperse among them (17 including positive controls, missing chemicals in T8 is replaced with those of T4). Well, it is a bit difficult to recognize a clear pattern.
\n
Here is a grid plot of the previous one by chemicals. In most of the chemicals, there is not much difference among 6 cell cline. However, NK cell seems to be similar to the B Cell and Myoloid cell on the chemical perturbations that create the larger difference among 6 cell lines such as Belinostat (one of the positive controls), MNL 2238, and Oprozomib. In the end, accuracy in predicting such chemicals may be important.
\n
Solve linear system, transformer, from a base cell line (NK cells, T cells CD4+, T cells CD8+, T regulatory cells) to a target cell line ('B cells', 'Myeloid cells') of the DE using the chemicals tested in the two cell lines (15 plus 2 positive control).
\nThe transformer can be computed by multiplying a pseudo-inverse of DE_i_core with DE_j_core from the left.
\nDE_i_core x T = DE_j_core
\nT = DE_i_core-t * DE_j_core
where DE_i_core-t is a pseudo-inverse of DE_i_core
Apply the transformer to the DE of the base cell line/target chemicals and get the DE of the target cell line/chemicals.
The simpler model consumes a lot of memory > 20Gb, and it cannot be performed on the free version of the Saturn Cloud, which I had used for convenience, I also propose an alternative solution using SVD with a projection space.
\nIt first reduces the gene dimension (18,211) to the full rank dimension of the entire data set (614) by SVD, create the transformer in the projected space, and applies the transformer in projected space from known cell to target cell on the target chemicals, reconstruct the original gene dimension of the prediction.
\nPlease refer to my notebook: the code can run on the notebook, and the result is deterministic, so reproducible.
\nThis method indeed can be used to diagnose the similarity of the \"master rule\" among the cell lines for scientific insight, and my result suggests that NK cells, along with T cells CD4+ can be stronger predictor cell lines for B Cell and Myeloid cells. Interestingly, T cells CD4+ is a better predictor for B cell while NK cell is a better predictor for Myeloid cells.
\nThis finding may align with the biological findings - NK cells can be derived from the myeloid lineage, Blood - The Journal of the American Society of Hematology 2011 3548 and T follicular helper cells cognately guide differentiation of antigen primed B cells in secondary lymphoid tissues - Trends in immunology 42.8 2021.
\n| base cell | \npublic | \nprivate | \n
|---|---|---|
| NK cells | \n0.784 | \n0.596 | \n
| T cells CD4+ | \n0.775 | \n0.607 | \n
| T cells CD8+ | \n0.959 | \n0.706 | \n
| T regulatory cells | \n0.834 | \n0.680 | \n
| base cell/target cell | \npublic | \nprivate | \n
|---|---|---|
| Predict B Cell by NK, Myeloid cells by T4 | \n0.786 | \n0.616 | \n
| Predict B Cell by T4, Myeloid cells by NK | \n0.773 | \n0.587 | \n
Differences in DE among certain cell lines can be captured linearly very well, and the linear relation is well transferable across different chemical responses, 0.768 in private and 0.582 in public
DE of NK cell and T cells CD4+ cells are quite predictable for that of B Cell and Myeloid cells
T cells CD4+ is a better predictor for B cell, and NK cells is a better predictor for Myeloid cell