![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fd971971a363e7bf8de359ffdaf72ecf2%2FSelection_077.png?generation=1715120816304983&alt=media\")
\"We extend our thanks to HP for providing the HP Z8 Fury Data Science Workstation, which empowered our deep learning experiments. The high computational power and large GPU memory enabled us to design our models swiftly.\"
\nhttps://www.hp.com/us-en/workstations/z8-fury.html
specifications:
\nuseful:
\necfp is binary vector. so you can decrease memory by using np.packbits, that is about 98m x 256 bytes
\npytorch version of np.unpackbits
\nprocessed dataset for discussion:
\nhttps://www.kaggle.com/datasets/hengck23/leash-bio-processed-dataset
\"We extend our thanks to HP for providing the HP Z8 Fury Data Science Workstation, which empowered our deep learning experiments. The high computational power and large GPU memory enabled us to design our models swiftly.\"
\nhttps://www.hp.com/us-en/workstations/z8-fury.html
specifications:
\nuseful:
\necfp is binary vector. so you can decrease memory by using np.packbits, that is about 98m x 256 bytes
\npytorch version of np.unpackbits
\nprocessed dataset for discussion:
\nhttps://www.kaggle.com/datasets/hengck23/leash-bio-processed-dataset
dataset for the above discussion have been shared!!!!
\nhttps://www.kaggle.com/datasets/hengck23/leash-bio-processed-dataset
In summary:
\ni think with proper xgboost tuning and ensemble of good data splits/folds, you can get LB of about 0.575. If you discover good hyper-parameters, please share here! Thanks!
", "rawMarkdown": "i think with proper xgboost tuning and ensemble of good data splits/folds, you can get LB of about 0.575. If you discover good hyper-parameters, please share here! Thanks!", "votes": 3 } ] }, { "id": 2746508, "postDate": "2024-04-11T09:15:43.460Z", "content": "Thanks for sharing !
\nMy colleagues and me sometimes organize introductory webinars on Kaggle competitions
\nSome video records here: https://www.youtube.com/watch?v=aqUOz3nFYm4&list=PL1GnA8S_asBANHU95kOPOPEh3moPm5uhw
Is there any chance that you can make an introduction to that competition ?
\nJust intro survey - data, metrics, public solutions, whatsever.
\nIt is not a private sharing - we make anounces here on Kaggle forum - everybody can join via zoom
\nand ask questions/comments/whatsever. Video will be later on Youtube.
\nTiming about 40 minutes
Sometimes organizers join the zoom and make their comments also:
\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/421170
In general it seems quite helpful for community.
\nAnother examples here:
\nhttps://www.kaggle.com/competitions/open-problems-single-cell-perturbations/discussion/444825
\nhttps://www.kaggle.com/competitions/commonlit-evaluate-student-summaries/discussion/429975
transformer (pure SMILES string) results out !!!!
\nhttps://huggingface.co/ibm/MoLFormer-XL-both-10pct
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fa4b79498cbc6dd566f0dcb15a31d6272%2FSelection_032.png?generation=1713678750507300&alt=media\")
Care to share how much training data this network saw? Or how many FF layers after the transformer? I used 3 layers post-transformer and trained on about half the training data to get a score of ~0.5.
", "rawMarkdown": "Care to share how much training data this network saw? Or how many FF layers after the transformer? I used 3 layers post-transformer and trained on about half the training data to get a score of ~0.5.", "votes": 1 } ] }, { "id": 2776878, "postDate": "2024-04-26T11:42:08.483Z", "content": "one of my idea
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F642a737a82d59e4b1ed1fba2b312cc8e%2FSelection_051.png?generation=1714131726333596&alt=media\")
A similar idea: if you are interested in multitask learning to help embed the test and train in a similar embedding space, you might want to consider predicting some calculated properties like cLogP (for many enzymes, binding loosely tracks with cLogP) or TPSA (which is a reference to the polar surface area of the molecule).
\nRDKit has some tools to calculate descriptors like these, as a start.
", "rawMarkdown": "A similar idea: if you are interested in multitask learning to help embed the test and train in a similar embedding space, you might want to consider predicting some calculated properties like cLogP (for many enzymes, binding loosely tracks with cLogP) or TPSA (which is a reference to the polar surface area of the molecule). \n\nRDKit has some tools to calculate descriptors like these, as a start.", "votes": 1, "replies": [ { "id": 2776922, "postDate": "2024-04-26T12:02:15.167Z", "content": "ref
\nhttps://dl.acm.org/doi/10.1145/3233547.3233548
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F3532116953f23315366443db19af4f0d%2FSelection_052.png?generation=1714132907090436&alt=media\")
\" calculated properties like cLogP …\"
\nThere are a couple of possible self-supervised task
\nthanks!
", "rawMarkdown": "\" calculated properties like cLogP ...\"\n\nThere are a couple of possible self-supervised task\n- MLM : masked language modeling\n- MTR: mult task regression (e.g. cLoP, .... i think anything related to shape and surface, electricty, energy)\n- chemical reaction\n\nthanks!", "votes": 1 }, { "id": 2780098, "postDate": "2024-04-28T03:47:15.373Z", "content": "yet another way to pretrain with test data …. basically it is \"smart clustering\" via VAE
\nlikewise, denoised diffusion also work (imagine train --> noise --> test)
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fbff4b909984bfcc298489abd9665ca16%2FSelection_055.png?generation=1714275971114985&alt=media\")
With you around this comp' would be extra fun
", "rawMarkdown": "With you around this comp' would be extra fun ", "votes": 5 }, { "id": 2747013, "postDate": "2024-04-11T15:41:38.363Z", "content": "useful software tools. implementations, etc:
\nimport mapply\nmapply.init(\n =50,#-1,\n =,\n)\ndf[] = df[].mapply(to_ecfp4_fun)\nhow to write fast submission code … run in minutes
\nprobability = xgb(x_valid)\n = (, ) \n\nid = reduced_df]()\n\n\nid = id(-)\nprobability = probability(-)\n = id!=-\n==)\n\nsubmit_df = pd({\n :id,\n :probability,\n})\n submit_df(f,index=False)\nreduced_df:
\n(id is -1 if protein is not evaluated)
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F0b3628a98221b1f366382dc9e56e9ea2%2FSelection_022.png?generation=1712852204921481&alt=media\")
This is brilliant. Thank you for sharing!
", "rawMarkdown": "This is brilliant. Thank you for sharing!" } ] }, { "id": 2747524, "postDate": "2024-04-11T23:58:38.013Z", "content": "This is pretty neat! Having the progress bar is nice too.
", "rawMarkdown": "This is pretty neat! Having the progress bar is nice too.", "votes": 1 }, { "id": 2748892, "postDate": "2024-04-12T17:59:28.237Z", "content": "i just realize that you may have more training data
\na pretty smart method that uses both labelled and unlabelled data
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fa103fefd904bcddb90413a393dee5344%2FSelection_033.png?generation=1713756844349343&alt=media\")
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3586013%2F57be4534a687d8f516f898bc3aa77eff%2FScreenshot%202024-04-22%20at%2012-23-50%20ICAN%20Interpretable%20cross-attention%20network%20for%20identifying%20drug%20and%20target%20protein%20interactions%20-%20journal.pone.0276609.pdf.png?generation=1713759870859276&alt=media\")
Another paper describing a very similar approach:
\nICAN: Interpretable cross-attention network for identifying drug and target protein interactions
there are a couple of similar papers. best to get those with github codes and experiment results from well known dataset. I foresee most deep net solution will be slow. how to sample training data could be an issue
", "rawMarkdown": "there are a couple of similar papers. best to get those with github codes and experiment results from well known dataset. I foresee most deep net solution will be slow. how to sample training data could be an issue" }, { "id": 2767685, "postDate": "2024-04-22T13:36:13.940Z", "content": "Agreed, efficient use of compute will be key.
", "rawMarkdown": "Agreed, efficient use of compute will be key." }, { "id": 2817098, "postDate": "2024-05-16T17:41:49.310Z", "content": "With only three types of proteins, is there too little protein data? If so, how can we tackle this problem?
", "rawMarkdown": "With only three types of proteins, is there too little protein data? If so, how can we tackle this problem?" } ] }, { "id": 2768571, "postDate": "2024-04-23T00:09:17.210Z", "content": "a few good github repo that i am using:
\nhttps://github.com/larngroup/DTITR
\nhttps://github.com/ZXT0212/CAT-DTI
\nhttps://github.com/peizhenbai/DrugBAN
Great sharing @hengck23!! Did you get to try any of above methods yet for this competition?
", "rawMarkdown": "Great sharing @hengck23!! Did you get to try any of above methods yet for this competition?" } ] } ] }, { "id": 2765165, "postDate": "2024-04-21T01:43:43.427Z", "content": "yet another external data!!!!
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe4137ec834e1c8fd91b077bc2745775f%2FSelection_030.png?generation=1713663808992163&alt=media\")
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe4190bd04b74691820e74f3824afc4fa%2FSelection_031.png?generation=1713663821080691&alt=media\")
I'll take a look at generating a dataset from this close to our competition data when I have time (likely this weekend). Thanks for finding!
", "rawMarkdown": "I'll take a look at generating a dataset from this close to our competition data when I have time (likely this weekend). Thanks for finding!", "replies": [ { "id": 2770565, "postDate": "2024-04-23T22:34:17.323Z", "content": "i have been looking at other papers and dataset. i think it actually make more sense to convert kaggle dataset to other dataset, i.e. replacing [Dy]
", "rawMarkdown": "i have been looking at other papers and dataset. i think it actually make more sense to convert kaggle dataset to other dataset, i.e. replacing [Dy]" }, { "id": 2770636, "postDate": "2024-04-24T00:28:09.887Z", "content": "Have you found consistency in what they use to model the DNA attachment point? I've done a little experimenting on my end switching Dy out for polyethyleneglycol like linkers and from a modeling perspective, I've seen similar results so far (which surprised me a little).
\nOne thing I have not tried is modeling the DNA attachment point as something small like a methyl group. Intuitively I'd think that providing the model some information on where the DNA attached would be valuable, but perhaps it's not.
", "rawMarkdown": "Have you found consistency in what they use to model the DNA attachment point? I've done a little experimenting on my end switching Dy out for polyethyleneglycol like linkers and from a modeling perspective, I've seen similar results so far (which surprised me a little).\n\nOne thing I have not tried is modeling the DNA attachment point as something small like a methyl group. Intuitively I'd think that providing the model some information on where the DNA attached would be valuable, but perhaps it's not." }, { "id": 2770758, "postDate": "2024-04-24T02:46:36.887Z", "content": "I've been using your C/methyl conversion in my data pipeline (rest in progress).
\nIs switching methyl/linker to PEG as simple as swapping the C/Dy for OCCOCCO (PEG smile)?
Also thanks for all the domain knowledge you've shared!
", "rawMarkdown": "I've been using your C/methyl conversion in my data pipeline (rest in progress).\nIs switching methyl/linker to PEG as simple as swapping the C/Dy for OCCOCCO (PEG smile)?\n\nAlso thanks for all the domain knowledge you've shared!" }, { "id": 2771702, "postDate": "2024-04-24T11:23:48.150Z", "content": "It's a little bit more complicated than that, but not much. I plan to post a notebook about it this weekend.
\nAlso, you are very welcome! Happy to help and share some knowledge of chemistry.
", "rawMarkdown": "It's a little bit more complicated than that, but not much. I plan to post a notebook about it this weekend.\n\nAlso, you are very welcome! Happy to help and share some knowledge of chemistry.", "votes": 2 } ] } ] }, { "id": 2748338, "postDate": "2024-04-12T11:49:45.287Z", "content": "anyone one to make baseline for \"DiffDock\"?
", "rawMarkdown": "anyone one to make baseline for \"DiffDock\"?", "votes": 3, "replies": [ { "id": 2748932, "postDate": "2024-04-12T18:14:32.807Z", "content": "There is a Colab diffdock notebook for anyone that is up for the challenge. With some tinkering, it can probably also be made to run on Kaggle.
", "rawMarkdown": "There is a [Colab diffdock notebook](https://github.com/suneelbvs/DiffDock) for anyone that is up for the challenge. With some tinkering, it can probably also be made to run on Kaggle.", "votes": 1, "replies": [ { "id": 2749654, "postDate": "2024-04-13T07:05:43.363Z", "content": "DiffDock has a new version very recently, so I wouldn't try a colab notebook from 2 years ago except maybe as reference
", "rawMarkdown": "DiffDock has a new version very recently, so I wouldn't try a colab notebook from 2 years ago except maybe as reference", "votes": 1 }, { "id": 2749756, "postDate": "2024-04-13T08:18:36.173Z", "content": "Good point, so maybe a lot of tinkering hehe
", "rawMarkdown": "Good point, so maybe a lot of tinkering hehe" } ] }, { "id": 2748942, "postDate": "2024-04-12T18:20:17.240Z", "content": "I am working on building structural models. Not with DiffDock, but more traditional physics-based docking engines.
", "rawMarkdown": "I am working on building structural models. Not with DiffDock, but more traditional physics-based docking engines.", "votes": 1 }, { "id": 2850582, "postDate": "2024-06-02T08:25:00.930Z", "content": "I recently made it to run, the thing is, it is very slow! Impossible to run so many compounds
", "rawMarkdown": "I recently made it to run, the thing is, it is very slow! Impossible to run so many compounds" } ] }, { "id": 2768605, "postDate": "2024-04-23T00:41:20.493Z", "content": "yet another results with Topological Torsion Fingerprint.
\nIt appears to me that just pure SMILE string model performance limit is 0.67 (for same block).
there aren't huge different for transformer based, fingerprint based, etc ….
\nthe data has very very long tails …. (i.e. the positive cases don't cluster together in the chemical space to gives high peak density, instead they are spreading around)
\nThis make it difficult to generalized to unseen blocks, especially for fingerprint methods …
need to think of new strategy
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F5e01cbdf17b3bb57233617c6cb3f5135%2FSelection_035.png?generation=1713832518151318&alt=media\")
average precision trick?
\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459596
\"This works because Average Precision metric is always improved when adding more predictions if the new predictions have a lower score than all previous predictions.\"
\nhmm
\nactually there is another way to look at our problem:
\nstage1: predict no binding
\nstage2: predict binding score/rank (only less than 1% of data from stage1. enable use of more time consuming methods?)
@hengck23, what are your positive-to-negative ratios in your val sets? As was discussed here, the metric strongly depends on the ratio. I suggest enforcing a known, constant ratio to facilitate easy comparison between results. The obvious one is the 1/128 ratio since this is the test set ratio…with this, I currently have validation scores of 0.53 for the all-BBs_shared val set, 0.29 for at least one BB is not shared and at least one BB is shared val set, and 0.018 for truly none of BBs is shared…yea the last one is tough lol. Probably different core would be even harder.
\nEDIT: I got confused, the ratio should be 1/125.
", "rawMarkdown": "@hengck23, what are your positive-to-negative ratios in your val sets? As was discussed [here](https://www.kaggle.com/competitions/leash-BELKA/discussion/492126#2742181), the metric strongly depends on the ratio. I suggest enforcing a known, constant ratio to facilitate easy comparison between results. The obvious one is the 1/128 ratio since this is the test set ratio...with this, I currently have validation scores of 0.53 for the all-BBs_shared val set, 0.29 for at least one BB is not shared and at least one BB is shared val set, and 0.018 for truly none of BBs is shared...yea the last one is tough lol. Probably different core would be even harder.\n\nEDIT: I got confused, the ratio should be 1/125. ", "votes": 4, "replies": [ { "id": 2758217, "postDate": "2024-04-18T03:10:38.957Z", "content": "be careful! 1/125 is only for public.
\nalways keep in mind: you have full access of the test data. you can measure some magic statistics on it. this is the trick to winning
", "rawMarkdown": "be careful! 1/125 is only for public.\n\nalways keep in mind: you have full access of the test data. you can measure some magic statistics on it. this is the trick to winning", "votes": 2, "replies": [ { "id": 2758728, "postDate": "2024-04-18T10:11:41.747Z", "content": "Will test.csv be the actual public leaderboard test? Or what do you mean?
", "rawMarkdown": "Will test.csv be the actual public leaderboard test? Or what do you mean?", "votes": 1 }, { "id": 2765402, "postDate": "2024-04-21T06:00:26.990Z", "content": "Yes the test data is the full test data, including public and private leaderboard components @gyulamaloveczky4
", "rawMarkdown": "Yes the test data is the full test data, including public and private leaderboard components @gyulamaloveczky4 ", "votes": 1 }, { "id": 2768461, "postDate": "2024-04-22T21:09:27.973Z", "content": "a submission of zero (or other constant) will revealed the percentage of all positive cases.
\nwhat happen if you submit a couple of different constants, e.g.
\nhint :google for probing the kaggle leaderboard for average precision
", "rawMarkdown": "a submission of zero (or other constant) will revealed the percentage of all positive cases.\n\nwhat happen if you submit a couple of different constants, e.g.\n- different values for different protein, different block, etc\n\nhint :google for probing the kaggle leaderboard for average precision ", "votes": 1 } ] }, { "id": 2758737, "postDate": "2024-04-18T10:16:50.893Z", "content": "AP is a great metric because you can measure ranking performance of your model regardless of the pos/neg ratio, AP CV should correlate with LB even if pos/neg ratio is a bit shifted (we don't really care about absolute values of probabilities here)
", "rawMarkdown": "AP is a great metric because you can measure ranking performance of your model regardless of the pos/neg ratio, AP CV should correlate with LB even if pos/neg ratio is a bit shifted (we don't really care about absolute values of probabilities here)", "replies": [ { "id": 2758939, "postDate": "2024-04-18T11:58:56.593Z", "content": "True, as long as you use the same validation set. But if you want to compare the validation scores of two different sets (or people) you need to use the same ratio
", "rawMarkdown": "True, as long as you use the same validation set. But if you want to compare the validation scores of two different sets (or people) you need to use the same ratio", "votes": 1 }, { "id": 2769342, "postDate": "2024-04-23T09:43:06.153Z", "content": "ratio squad?
", "rawMarkdown": "ratio squad?" } ] } ] }, { "id": 2754473, "postDate": "2024-04-16T04:28:05.317Z", "content": "i just realise that we need to probe the num of non train blocks in public test first. this can be done by two submission. a normal one and another with non train block masked off with magic value and observe the difference.
\nwe need to verify private and public share the same distribution of train and non train block.
", "rawMarkdown": "i just realise that we need to probe the num of non train blocks in public test first. this can be done by two submission. a normal one and another with non train block masked off with magic value and observe the difference. \n\nwe need to verify private and public share the same distribution of train and non train block.", "votes": 4, "replies": [ { "id": 2754528, "postDate": "2024-04-16T05:01:48.073Z", "content": "There's like 34 unique buildingblock1s, with triazine core, not in train. And there's like 36 unique buildingblock1s, with different core, not in train.
\nDepends how many subs you want to spend, but it would be useful to know, for each of those 70 unique BBs, and for each protein target, so 70x3, are some in public test? Are all in public test? Maybe half are in public test, half in private? If something like half and half, but no overlap (so public LB doesn't help \"learn\" the unseen private LB) then, maybe you can find out the groupings just by making a graph of which unique BBs are included with which other unique BBs.
\nExample:
\nBB1: 1,2,3,4
\nBB2: 5-12
\nBB3: 13-20
Test1: 1,5,13
\nTest2: 1,5,14
\n…
\nTestN: 2,5,13
\n1 and 2 both \"connect\" to 13, so they're in the same group. But maybe 5,13,14 etc never pair with 3 or 4, that would imply that one group (1,2,5,13,14,…) is either public or private, and the other group (3,4,…) is the other leaderboard group
For triazines in test with BBs not in train, there's two separate blocks:
\n17 BB_a, 36 BB_b/c. (presumably public LB)
\n17 BB_a, 36 BB_b/c. (presumably private LB)
And every possible combination is:
\n2 groups, times 3 proteins, times 17 BB_a, times 36 BB_b/c 2 combinations, the same BB is allowed to be taken twice, so it's:
\n2x3x17x(36+1)x36/2 = 67,932. Which is exactly the number of rows I observed. So that checks out.
However, for the other group of 500,000 rows, it all appears to be one grouping. So I think this critical group we will probably find is NOT in public test at all! 500,000 rows is 167k per protein. Since the organizers said:
\n\n\nWe are providing roughly 98M training examples per protein, 200K validation examples per protein, and 360K test molecules per protein
\n
and 360K - 200K = 160K, so I suspect the public and private are equivalent EXCEPT for this big curveball of the non-triazine molecules with unseen BBs will be only in private LB most likely.
\nIf someone proves this, please post back here!
", "rawMarkdown": "For triazines in test with BBs not in train, there's two separate blocks:\n17 BB_a, 36 BB_b/c. (presumably public LB)\n17 BB_a, 36 BB_b/c. (presumably private LB)\n\nAnd every possible combination is:\n2 groups, times 3 proteins, times 17 BB_a, times 36 BB_b/c 2 combinations, the same BB is allowed to be taken twice, so it's:\n2x3x17x(36+1)x36/2 = 67,932. Which is exactly the number of rows I observed. So that checks out.\n\nHowever, for the other group of 500,000 rows, it all appears to be one grouping. So I think this critical group we will probably find is NOT in public test at all! 500,000 rows is 167k per protein. Since the organizers said:\n\n> We are providing roughly 98M training examples per protein, 200K validation examples per protein, and 360K test molecules per protein\n\nand 360K - 200K = 160K, so I suspect the public and private are equivalent EXCEPT for this big curveball of the non-triazine molecules with unseen BBs will be only in private LB most likely.\n\nIf someone proves this, please post back here!\n\n", "votes": 4 } ] } ] }, { "id": 2780103, "postDate": "2024-04-28T03:51:08.867Z", "content": "this can be used for augmentation!
\nhttps://chao1224.github.io/MoleculeSTM
In the specifications of your HP Z8 Fury Data Science Workstation you have \"256 MB random\". Is this a typo that should be \"256 GB random\"?
", "rawMarkdown": "In the specifications of your HP Z8 Fury Data Science Workstation you have \"256 MB random\". Is this a typo that should be \"256 GB random\"?\n", "votes": 1, "replies": [ { "id": 2774754, "postDate": "2024-04-25T10:10:46.637Z", "content": "oops! in
\nGB. thanks!
for the past few days i have been training neural nets (e.g. string based transformer). Initial results seems to suggest to me that
\nI have not tried to figure out how the authors chose their dataset yet, but I do know that it's quite common to choose building blocks based on their calculated physical properties. This way, the dataset is biased towards final molecules with properties that more closely resemble drugs. For readings on this, perhaps Lipinski's rules would be a good start for some basic properties you'd like to see in a final molecule (these are common guidelines, not really hard and fast rules).
", "rawMarkdown": "I have not tried to figure out how the authors chose their dataset yet, but I do know that it's quite common to choose building blocks based on their calculated physical properties. This way, the dataset is biased towards final molecules with properties that more closely resemble drugs. For readings on this, perhaps Lipinski's rules would be a good start for some basic properties you'd like to see in a final molecule (these are common guidelines, not really hard and fast rules).", "votes": 1 } ] }, { "id": 2754293, "postDate": "2024-04-15T23:52:51.580Z", "content": "DTI plug and play :
\nhttps://github.com/yazdanimehdi/DeepDrugDomain
How do you construct the label for multi-label classification?
\nSome of the molecules bind to the multiple proteins, and XGBoost doesn't support this setup
pd.).value\n \n \n \n \nName: count, dtype: \nfor cpu:
\nhttps://xgboost.readthedocs.io/en/stable/tutorials/multioutput.html
\nStarting from version 1.6, XGBoost has experimental support for multi-output regression and multi-label classification with Python package.
for gpu:
\nyou can write your custom loss. (e.g. kl divergence loss)
\nbut interesting, i find that the following 3 results are almost the same
you can try a toy example, e.g. y =np.random.choice(2,(100,3)) and overfit a xgbost model. you should see it can gives multi-label output using the above methods.
", "rawMarkdown": "for cpu: \nhttps://xgboost.readthedocs.io/en/stable/tutorials/multioutput.html\nStarting from version 1.6, XGBoost has experimental support for multi-output regression and multi-label classification with Python package.\n\nfor gpu: \nyou can write your custom loss. (e.g. kl divergence loss)\nbut interesting, i find that the following 3 results are almost the same\n- one model, input = [feature + protein feature] where protein feature =[1,0,0],[0,1,0],[0,0,1]\n- three model, each is binary classification for each protein\n- one model, just set target= Lx3 array and objective=\"binary:logistic\"\nXGBoost uses one-vs-rest method\n(https://discuss.xgboost.ai/t/understanding-leaf-values/1840)\n\nyou can try a toy example, e.g. y =np.random.choice(2,(100,3)) and overfit a xgbost model. you should see it can gives multi-label output using the above methods.", "votes": 3, "replies": [ { "id": 2753538, "postDate": "2024-04-15T14:51:13.407Z", "content": " x_train = np.random.choice(256,(20,256)).astype(np.uint8)\n y_train = np.random.choice(2,(20,3)).astype(np.uint8)\n\n clf = XGBClassifier(\n =6,\n =100,\n # =,\n =,\n =,\n objective =, #, \n learning_rate= 0.3,\n device = ,\n =42,\n =1,\n )\n\n clf.fit(\n x_train, y_train,\n =\n )\n\n probability = clf.predict_proba(x_train)\n predict = (probability>0.5).astype(np.uint8)\n (probability)\n (predict)\n (y_train)\n ((y_train!=predict).sum())\ny train
\n\n\n\n\n\n\n\n\nprobability
\n[[ ]\n [ ]\n [ ]\n [ ]\n [ ]\n [ ]\n [ ]\n [ ]\ny_train!=predict).sum() gives zero\nsince it is micro average precision, one has to be careful in calibration, class-balancing or multi-class/single-class setup:
\nprobability=np.([\n,\n,\n,\n,\n,\n,\n,\n,\n,\n])\ny=np.([\n,\n,\n,\n,\n,\n,\n,\n,\n,\n])\nscore (all micro) \nscore (, binds_BRD4) \nscore (, binds_HSA) \nscore (, binds_sEH) \nThanks for sharing! Has any post-processing helped to achieve higher cv/lb score?
", "rawMarkdown": "Thanks for sharing! Has any post-processing helped to achieve higher cv/lb score?", "votes": 1, "replies": [ { "id": 2748308, "postDate": "2024-04-12T11:26:12.063Z", "content": "i leave this for future plan. improving accuracy for test building blocks not found in training is my top prority now.
\nbut you can google for differentiable micro average precision loss
", "rawMarkdown": "i leave this for future plan. improving accuracy for test building blocks not found in training is my top prority now.\n\nbut you can google for differentiable micro average precision loss", "votes": 1 }, { "id": 2748332, "postDate": "2024-04-12T11:43:26.317Z", "content": "Your numbers for valid_noshare are very promising! It seems we can really generalize to new BBs. The question now is only how much.
", "rawMarkdown": "Your numbers for valid_noshare are very promising! It seems we can really generalize to new BBs. The question now is only how much." }, { "id": 2748342, "postDate": "2024-04-12T11:51:21.550Z", "content": "\"Your numbers for valid_noshare are very promising!\"
\nany model that uses train size larger 68_152_325 (see subsample column) will not have validation metrics. (in this case, the training set include the validation set).
\ni would say that current model for valid nonshare has precision half of that valid share.
\nbut according to papers, SOTA can probably increase that to 70%
i found my ideal algorithm
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F0c62172bd70306926304d48f42f18fe6%2FSelection_047.png?generation=1714097530102992&alt=media\")
https://github.com/luwei0917/TankBind
\n\"TankBind also support virtual screening. In our example here, for the WDR domain of LRRK2 protein, we can screen 10,000 drug candidates in 2 minutes (or 1M in around 3 hours) with a single GPU. Check out\"
", "rawMarkdown": "i found my ideal algorithm\n\n\nhttps://github.com/luwei0917/TankBind\n\n\"TankBind also support virtual screening. In our example here, for the WDR domain of LRRK2 protein, we can screen 10,000 drug candidates in 2 minutes (or 1M in around 3 hours) with a single GPU. Check out\"", "votes": 2, "replies": [ { "id": 2778802, "postDate": "2024-04-27T11:17:32.807Z", "content": "Hi, if you trust that DTI prediction may be the optimal solution for this task, may be you can try some of the latest models, such as:
\n\nThe rationale behind this is that some early DTI prediction methods have been criticized in recent papers for their unreasonable docking poses and poor generalization ability, e. g. https://arxiv.org/abs/2308.05777.
", "rawMarkdown": "Hi, if you trust that DTI prediction may be the optimal solution for this task, may be you can try some of the latest models, such as: \n1. https://github.com/Zhang-Runze/PackDock\n2. https://github.com/HBioquant/DiffBindFR\n\nThe rationale behind this is that some early DTI prediction methods have been criticized in recent papers for their unreasonable docking poses and poor generalization ability, e. g. https://arxiv.org/abs/2308.05777. ", "votes": 2, "replies": [ { "id": 2778817, "postDate": "2024-04-27T11:31:10.350Z", "content": "\" unreasonable docking poses and poor generalization ability\"
\ni agree with these.
from competition point of view:
\nso my strategy is simple:
\n\n\nQSAR for majority of the results
\n
\n self-supervsied, external data/block-based modeling to improve nonshare blocks
\n DTI to improve/anaylse to results
\n docking for a small \"selected\" subset
Great! Indeed, for real-world projects, we have indeed employed approachs like this.
", "rawMarkdown": ">QSAR for majority of the results\nself-supervsied, external data/block-based modeling to improve nonshare blocks\nDTI to improve/anaylse to results\ndocking for a small \"selected\" subset\n\n\nGreat! Indeed, for real-world projects, we have indeed employed approachs like this. " } ] } ] }, { "id": 2774889, "postDate": "2024-04-25T11:14:52.020Z", "content": "i wonder if something like this is possible?
\nwe have the blocks and the reaction results.(for both train and test)
Contextual Molecule Representation Learning from Chemical Reaction Knowledge
\nhttps://arxiv.org/html/2402.13779v1
\"REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry. Specifically, REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature. We propose two pre-training objectives: Masked Reaction Centre Reconstruction (MRCR) and Reaction Centre Identification (RCI). REMO offers a novel solution to MRL by exploiting the underlying shared patterns in chemical reactions as context for pre-training, which effectively infers meaningful representations of common chemistry knowledge\"
", "rawMarkdown": "i wonder if something like this is possible?\nwe have the blocks and the reaction results.(for both train and test)\n\nContextual Molecule Representation Learning from Chemical Reaction Knowledge \nhttps://arxiv.org/html/2402.13779v1\n- transformer based SSL method\n- Molecular Representation Learning (MRL)\n\n\"REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry. Specifically, REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature. We propose two pre-training objectives: Masked Reaction Centre Reconstruction (MRCR) and Reaction Centre Identification (RCI). REMO offers a novel solution to MRL by exploiting the underlying shared patterns in chemical reactions as context for pre-training, which effectively infers meaningful representations of common chemistry knowledge\"", "votes": 2 }, { "id": 2773938, "postDate": "2024-04-25T01:21:45.663Z", "content": "Wherever you are, the competition will be very exciting. Inspired by you, I have also started a similar discussion in BirdCLEF2024, and will continue to update it.
", "rawMarkdown": "Wherever you are, the competition will be very exciting. Inspired by you, I have also started a similar discussion in BirdCLEF2024, and will continue to update it.", "votes": 2 }, { "id": 2771701, "postDate": "2024-04-24T11:23:44.463Z", "content": "Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence
\nhttps://arxiv.org/abs/2104.08736
example usage
\nhttps://docs.libauc.org/examples/auprc.html
model = ResNet18(=, =None, =1)\nmodel = model.cuda()\n\nloss_fn = APLoss(=len(trainSet), =margin, =gamma)\noptimizer = SOAP(model.parameters(), =lr, =, =weight_decay)\nsee also
\nhttps://github.com/divelab/MoleculeX
In addition, AdvProp is able to deal with tasks in which samples from different classes are highly imbalanced. In these cases, we employ advanced loss functions that optimize various areas under curves (AUC), such as areas under the receiver operating characteristic (AUROC) and the precision recall curve (AUPRC). AdvProp has been used to participate in the AI Cures open challenge for COVID-19 and is now ranked #1 in terms of both AUROC and AUPRC on the leaderboard.
", "rawMarkdown": "Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence\nhttps://arxiv.org/abs/2104.08736\n\nexample usage\nhttps://docs.libauc.org/examples/auprc.html\n\n```\nmodel = ResNet18(pretrained=False, last_activation=None, num_classes=1)\nmodel = model.cuda()\n\nloss_fn = APLoss(data_len=len(trainSet), margin=margin, gamma=gamma)\noptimizer = SOAP(model.parameters(), lr=lr, mode='adam', weight_decay=weight_decay)\n```\n\nsee also\nhttps://github.com/divelab/MoleculeX\n\nIn addition, AdvProp is able to deal with tasks in which samples from different classes are highly imbalanced. In these cases, we employ advanced loss functions that optimize various areas under curves (AUC), such as areas under the receiver operating characteristic (AUROC) and the precision recall curve (AUPRC). AdvProp has been used to participate in the AI Cures open challenge for COVID-19 and is now ranked #1 in terms of both AUROC and AUPRC on the leaderboard. ", "votes": 2, "replies": [ { "id": 2850573, "postDate": "2024-06-02T08:18:31.437Z", "content": "Interesting to see you find my friend’s work on advprop, I’m trying to optimize it for this dataset
", "rawMarkdown": "Interesting to see you find my friend’s work on advprop, I’m trying to optimize it for this dataset" } ] }, { "id": 2775243, "postDate": "2024-04-25T15:04:45.750Z", "content": "i suddenly have an idea.
\nfor the test smiles, run some tools or simulations (e.g. energy, force field, conformers?) to get some measurements related to binding. learn model to predict such quality on test.
can check correlation of measurements with target bind on train, tsNE, etc
", "rawMarkdown": "i suddenly have an idea.\nfor the test smiles, run some tools or simulations (e.g. energy, force field, conformers?) to get some measurements related to binding. learn model to predict such quality on test.\n\ncan check correlation of measurements with target bind on train, tsNE, etc", "replies": [ { "id": 2775778, "postDate": "2024-04-25T19:38:01.787Z", "content": "do you mean to say \"for the train smiles…\" ? if not, could you explain a little more what you mean?
", "rawMarkdown": "do you mean to say \"for the **train** smiles...\" ? if not, could you explain a little more what you mean?", "replies": [ { "id": 2775825, "postDate": "2024-04-25T20:27:55.740Z", "content": "the test smiles
", "rawMarkdown": "the test smiles" } ] } ] }, { "id": 2754267, "postDate": "2024-04-15T23:38:16.950Z", "content": "what? joint protein-molecue fingerprint? interesting ….
\nhttps://academic.oup.com/bioinformatics/article/35/8/1334/5092926
Development of a protein–ligand extended connectivity (PLEC) fingerprint and its application for binding affinity predictions
", "rawMarkdown": "what? joint protein-molecue fingerprint? interesting ....\nhttps://academic.oup.com/bioinformatics/article/35/8/1334/5092926\n\nDevelopment of a protein–ligand extended connectivity (PLEC) fingerprint and its application for binding affinity predictions", "votes": 2 }, { "id": 2748402, "postDate": "2024-04-12T12:26:35.590Z", "content": "Training on 98M samples with 2048 bits each gives almost 200GB of training data. Do I understand that correctly? Just wondering what hardware you use for it.
", "rawMarkdown": "Training on 98M samples with 2048 bits each gives almost 200GB of training data. Do I understand that correctly? Just wondering what hardware you use for it.", "votes": 2, "replies": [ { "id": 2748476, "postDate": "2024-04-12T13:18:06.317Z", "content": "use one byte to represent 8 bit.
\nso 2048 bit=256 byte.
see np.packbits.
", "rawMarkdown": "use one byte to represent 8 bit.\nso 2048 bit=256 byte.\n\nsee np.packbits.", "votes": 2, "replies": [ { "id": 2748546, "postDate": "2024-04-12T13:38:58.040Z", "content": "My bad. Thanks!
", "rawMarkdown": "My bad. Thanks!", "votes": 1 } ] } ] }, { "id": 2753694, "postDate": "2024-04-15T16:05:54.067Z", "content": "With you around this comp' would be extra fun
", "rawMarkdown": "With you around this comp' would be extra fun", "votes": -5 }, { "id": 2875171, "postDate": "2024-06-16T23:39:34.410Z", "content": "Curious have you tried the 77M version of chemberta or molformer?
", "rawMarkdown": "Curious have you tried the 77M version of chemberta or molformer?" }, { "id": 2816398, "postDate": "2024-05-16T09:44:15.263Z", "content": "BRD4 dataset
\nhttps://github.com/shiwentao00/Graphsite-classifier/tree/master
\nGraphSite: Ligand Binding Site Classification with Deep Graph Learning
\nhttps://www.mdpi.com/2218-273X/12/8/1053
some probing tutorial:
\nfor more information refer to https://stats.stackexchange.com/questions/394494/calculating-sklearns-average-precision-by-hand
https://www.kaggle.com/code/hengck23/how-to-probe
", "rawMarkdown": "some probing tutorial:\nfor more information refer to https://stats.stackexchange.com/questions/394494/calculating-sklearns-average-precision-by-hand\n\nhttps://www.kaggle.com/code/hengck23/how-to-probe" }, { "id": 2794938, "postDate": "2024-05-05T15:26:51.117Z", "content": "Really nice work on the FP+SMILES model @hengck23 !
", "rawMarkdown": "Really nice work on the FP+SMILES model @hengck23 !" }, { "id": 2794282, "postDate": "2024-05-05T07:41:50.030Z", "content": "old results for archive:
\nWARNING:
\ni think i can conclude transformer beats xgboost+fingerprint!
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F2a7872cf8060014be810dd706bab1133%2FSelection_036.png?generation=1713929270479439&alt=media\")
new results with gpu training
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fca8428acaa3ffa2501850471183ca09b%2FSelection_026.png?generation=1713066709109703&alt=media\")
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fdaa7c3665fe31df78e61ffac106f4d7e%2FSelection_024.png?generation=1712911082033973&alt=media\")
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fc2e854b52473673388a32ab6c76d93b6%2FSelection_017.png?generation=1712818198030642&alt=media\")
https://www.thesgc.org/news/structural-genomics-consortium-and-x-chem-enter-collaboration-unlock-human-proteome-and-promote
\nThese datasets, curated in an ML-ready format, will be posted to a public portal to be used for model building. … but X-Chem and the SGC expect the DEL-ML data portal will be ready for public access starting in early 2024.
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F188a7ef0c66e995e004ceb9a9dd276da%2FSelection_046.png?generation=1714077718909224&alt=media\")
\nbb considerations
", "rawMarkdown": "\n\n\nbb considerations" }, { "id": 2775703, "postDate": "2024-04-25T18:44:39.913Z", "content": "there is one trick.
\none can \"google\" for compound that is close to the test molecule …. write an LLM agent for that
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fda40f675564d703d03c4f25b0cffa68b%2FSelection_044.png?generation=1714070663174093&alt=media\")
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F7cfa93726f6544be9304b8fb5f6929a8%2FSelection_043.png?generation=1714070675606349&alt=media\")
yet another external data
\nhttps://chemrxiv.org/engage/chemrxiv/article-details/60c741e2567dfedeb7ec3e52
\n(see Supplementary materials)
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F688846386de6863b1ddc4e84f7d3598a%2FSelection_041.png?generation=1714056462488116&alt=media\")
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F035dffab1de71e8666b952f275e21f1f%2FSelection_042.png?generation=1714056474616688&alt=media\")
i have completed
\nNow i am trying (in process of training …) 2d/3d GNN (i.e. input = molecular graph).
\nIf anyone has results in GNN, or good papers/github repo to recommend, or tips in handling huge data of grpah, it is most welcomed!
The edges data is sparse. So keep it as sparse matrix and turn to dense just before feeding to the model. Max number of nodes is around 70 (much less than smile length) so total size of data is manageable. Less than *2 of smile data size. I have not tried gnn, but tried nodes transformer+feeding edges data to attention (similar to top solutions in Ribonanza) and could not beat smile transformer yet- smile transformer ~0.67 val vs. ~0.605 val for nodes transformer+injected edges. I'm waiting to see your results :)
", "rawMarkdown": "The edges data is sparse. So keep it as sparse matrix and turn to dense just before feeding to the model. Max number of nodes is around 70 (much less than smile length) so total size of data is manageable. Less than *2 of smile data size. I have not tried gnn, but tried nodes transformer+feeding edges data to attention (similar to top solutions in Ribonanza) and could not beat smile transformer yet- smile transformer ~0.67 val vs. ~0.605 val for nodes transformer+injected edges. I'm waiting to see your results :)" }, { "id": 2775078, "postDate": "2024-04-25T13:17:25.933Z", "content": "https://github.com/wengong-jin/hgraph2graph/tree/master
\n(paper in github)
\nI was looking at this earlier as an encoder but I'm trying molformer first since it can be used pretrained.
\nI have however generated the hgraph vocab for train + test with a C/Methane Dy substitute.
Edit: https://www.kaggle.com/datasets/sroger/belka-hgraph-vocab/
", "rawMarkdown": "https://github.com/wengong-jin/hgraph2graph/tree/master\n(paper in github)\nI was looking at this earlier as an encoder but I'm trying molformer first since it can be used pretrained.\nI have however generated the hgraph vocab for train + test with a C/Methane Dy substitute.\n\nEdit: https://www.kaggle.com/datasets/sroger/belka-hgraph-vocab/", "replies": [ { "id": 2775189, "postDate": "2024-04-25T14:25:50.677Z", "content": "thanks all.
\n\"~0.67 val vs. ~0.605 val \"
\ni have a feeling that there may not be much help from GNN … but let's wait for a few days.
I am wondering how others have achieved LB better than 0.600. Is it they are have better nonshare blocks score or better share block score (which means local cv should reach 0.700)?
\nor are they \"dealing with blocks\"?
\nfingerprint is much like 2d graph, so i am hoping 3d information would work.
\nhowever, generating 3d information itself is a problem (not accurate).
if it doesn't my next bet is SSL(self supervised learning) on test molecules.
\n\" train + test with a C/Methane Dy substitute.\"
\nyou can show examples of pair (before,after conversion), i can do experiments at my side.
\n(e.g. molformer or chembeta)
\nwould be better if you some code.
https://arxiv.org/pdf/2310.13769.pdf
\nCompositional Deep Probabilistic Models of DNA-Encoded Libraries
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F3e085abf23adc893f9aa0d416679c260%2FSelection_040.png?generation=1714055546823055&alt=media\")
\"I am wondering how others have achieved LB better than 0.600\"
\nI have two guesses currently:
For myself, I'm getting right about .700 on share block CV, and haven't started doing any enhancements or non-share block work yet.
", "rawMarkdown": "For myself, I'm getting right about .700 on share block CV, and haven't started doing any enhancements or non-share block work yet. ", "votes": 1 }, { "id": 2775440, "postDate": "2024-04-25T16:58:22.993Z", "content": "\"For myself, I'm getting right about .700 \"
\nthat is very good!!
", "rawMarkdown": "\"For myself, I'm getting right about .700 \"\n\nthat is very good!!", "votes": 1 }, { "id": 2775571, "postDate": "2024-04-25T17:37:59.577Z", "content": "Thanks!
\n\nEdit: just a test dataset pipeline bug, fixed now and correlating fine. My best trusted baseline was 0.695 CV, 0.597 LB, and that went to 0.700 CV, 0.600 LB today, so that's pretty close to exact correlation so far.
", "rawMarkdown": "Thanks!\n\n~~...But it's not correlating with LB 🤦🏻♂️~~\n\nEdit: just a test dataset pipeline bug, fixed now and correlating fine. My best trusted baseline was 0.695 CV, 0.597 LB, and that went to 0.700 CV, 0.600 LB today, so that's pretty close to exact correlation so far." }, { "id": 2775674, "postDate": "2024-04-25T18:35:21.840Z", "content": "my estimate is quite correct:
\nCV = LB + m
m is the performance of nonshare block, which is close to zero (or test distribution of (ratio of nonshare )*(zero submission)). zero submission is assume model cannot detect nonshare block at all.
", "rawMarkdown": "my estimate is quite correct:\nCV = LB + m\n\nm is the performance of nonshare block, which is close to zero (or test distribution of (ratio of nonshare )*(zero submission)). zero submission is assume model cannot detect nonshare block at all. " }, { "id": 2775683, "postDate": "2024-04-25T18:38:05.603Z", "content": "wanted to share some info about potential of the transformers, -
\ndata: all 1's (about 1.4kk), randomly sample 20kk of zero samples
\ntransformer model: 0.589 LB
thanks a lot!
", "rawMarkdown": "thanks a lot!" }, { "id": 2776451, "postDate": "2024-04-26T07:19:00.877Z", "content": "Thanks @martynoveduard for sharing. Is it basic transformer or specific transformer for chemistry ? Will apreciate if you can point to a paper or something.
", "rawMarkdown": "Thanks @martynoveduard for sharing. Is it basic transformer or specific transformer for chemistry ? Will apreciate if you can point to a paper or something.", "votes": 1 }, { "id": 2776489, "postDate": "2024-04-26T07:48:53.673Z", "content": "I just trained mine from scratch, haven't tried existing pre-trained models
", "rawMarkdown": "I just trained mine from scratch, haven't tried existing pre-trained models", "votes": 1 }, { "id": 2776523, "postDate": "2024-04-26T08:08:28.753Z", "content": "Thought I'd add to this now that I've made a sub.
\nNaive folds/sampling + frozen molformer encoder and some other layers: LB:0.539 CV:0.607
if anyone want to use pretrained model (frozen), i think one has to replace the [Dy]
", "rawMarkdown": "if anyone want to use pretrained model (frozen), i think one has to replace the [Dy]" }, { "id": 2776605, "postDate": "2024-04-26T08:48:14.073Z", "content": "from rdkit import Chem\nfrom rdkit.Chem import AllChem\n\nm_Dy, m_C = Chem., Chem.\n\ndef proc -> str:\n m = Chem.\n m = AllChem.\n Chem.\n return Chem.\nThis is the relevant snippet in my dataset, credits to @chemdatafarmer
", "rawMarkdown": "```\nfrom rdkit import Chem\nfrom rdkit.Chem import AllChem\n\nm_Dy, m_C = Chem.MolFromSmiles(\"[Dy]\"), Chem.MolFromSmiles(\"C\")\n\ndef proc_smile(smile: str) -> str:\n m = Chem.MolFromSmiles(smile)\n m = AllChem.ReplaceSubstructs(m, m_Dy, m_C)[0]\n Chem.SanitizeMol(m)\n return Chem.MolToSmiles(m)\n```\n\nThis is the relevant snippet in my dataset, credits to @chemdatafarmer", "votes": 1 }, { "id": 2776891, "postDate": "2024-04-26T11:45:22.617Z", "content": "\"Thought I'd add to this now that I've made a sub.\"
\nmake separate submission for share and nonshare blocks. then you can see if pretrain helps in nonsharing blocks
I'll look into that, haven't done anything with the blocks so far.
", "rawMarkdown": "I'll look into that, haven't done anything with the blocks so far." }, { "id": 2778409, "postDate": "2024-04-27T06:37:01.243Z", "content": "@martynoveduard
\n\"I just trained mine from scratch, haven't tried existing pre-trained models\"
\ndo you use your own tokenizer?
this is what i done when i do not know when to stop training. just average the just few checkpoints
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F19b0eb03e19d34bae44873186d47055c%2FSelection_039.png?generation=1714023733193529&alt=media\")
cascade model, active learning and submission
\nour final model is a cascade of models, each model has high recall of say 99%, but moderate precision say 50%.
\nassume each model has recall =0.99, FPrate =0.10, then a cascade of of 5 such model will be
\nrecall=pow(0.99,5)=still high, FPrate =pow(0.1,5)=very low = 1e-6
how it work:
\n. data \n. discarded model0()< threshold0, model\n. discarded model1()< threshold1, model\n. discarded model2()< threshold2, model\n....\n few data remains\n(here we can use complicated methods like docking, MD simulation, complex fingerprint, large deep net, etc)\nat submission
\n````
\nthe trick is that try to keeps0=s1=s2=s3 … then you have zero miss and left with very little data for next model
\n```
\nreference:
\ncascade adaboost for face detection. i think xgboost can be modified to do this.
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fcb2302b19ce617b9d488c9a303ad1aeb%2FSelection_038.png?generation=1714012410058499&alt=media\")
diffdock: try nvidia online demo
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F18ff3e4e995521d5794c1f5080d1cc8b%2FSelection_027.png?generation=1713320365421742&alt=media\")
maybe can apply for cloud beta services for benchmarking? if not, you can setup on your own gpu.
", "rawMarkdown": "diffdock: try nvidia online demo\n\n\nhttps://developer.nvidia.com/blog/new-models-molmim-and-diffdock-power-molecule-generation-and-molecular-docking-in-bionemo/\n\nmaybe can apply for cloud beta services for benchmarking? if not, you can setup on your own gpu." }, { "id": 2753345, "postDate": "2024-04-15T12:47:53.483Z", "content": "@hengck23 Thank you for giving us a great insight on this competition! I have a small question though. What strategy did you use to sub-sample the dataset into 25m?
", "rawMarkdown": "@hengck23 Thank you for giving us a great insight on this competition! I have a small question though. What strategy did you use to sub-sample the dataset into 25m?" }, { "id": 2748860, "postDate": "2024-04-12T17:30:33.590Z", "content": "Did you split by maximizing nonshare groups via clustering? Because no way a naive splitting would give 13% nonshare. When I split each BB group to 5F it results in something on the order of 1% nonshare (~1M molecules out of ~100M). And only ~50% train. When I do 10F it drops to ~0.1% nonshare. I fyou did clustering, I suggest you check #positive in each group because it may not be random.
", "rawMarkdown": "Did you split by maximizing nonshare groups via clustering? Because no way a naive splitting would give 13% nonshare. When I split each BB group to 5F it results in something on the order of 1% nonshare (~1M molecules out of ~100M). And only ~50% train. When I do 10F it drops to ~0.1% nonshare. I fyou did clustering, I suggest you check #positive in each group because it may not be random.", "replies": [ { "id": 2748863, "postDate": "2024-04-12T17:36:39.897Z", "content": "check the split in my public dataset https://www.kaggle.com/datasets/hengck23/leash-bio-processed-dataset.
\nyou first just select e.g. 5% nonshare building blocks for each bb1,bb2bb3. this will give you about 15% \"nonshare SMILE\", i.e. \"valid nonshare\". Then for the remaining \"share SMILE\", random split to \"train share\" and \"valid share\".
\na better way is to select nonshare train building blocks that is similar to nonshare test building blocks.
\n(e.g. Tanimoto Similarity)
Ahhh I see. I thought that nonshare='none of the BBs is shared', but it is actually 'at least one of the BBs is not shared' which is a bit different haha.
\nI wonder what the scores are for truly nonshared. This would also probably explain at least partly the valid/test gap.
none of the bb is shared is correct
", "rawMarkdown": "none of the bb is shared is correct\n" }, { "id": 2748940, "postDate": "2024-04-12T18:19:35.737Z", "content": "By your split? No, since you choose several non-share for each BBs, resulting in molecules with e.g. nonshare for BB1 but shared for BB2. Which are inside the nonshared split. So not truly 'none is shared'. Truly nonshared split is much smaller.
", "rawMarkdown": "By your split? No, since you choose several non-share for each BBs, resulting in molecules with e.g. nonshare for BB1 but shared for BB2. Which are inside the nonshared split. So not truly 'none is shared'. Truly nonshared split is much smaller.", "votes": 1 }, { "id": 2748957, "postDate": "2024-04-12T18:31:41.197Z", "content": "it is my mistake. i thought bb1,bb2,bb3 are non overlapping.
", "rawMarkdown": "it is my mistake. i thought bb1,bb2,bb3 are non overlapping." }, { "id": 2748986, "postDate": "2024-04-12T18:50:53.827Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2748993, "postDate": "2024-04-12T18:57:42.090Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2749035, "postDate": "2024-04-12T19:44:25.307Z", "content": "If you want to do by similarity, you could do a sphere exclusion clustering in rdkit and hold certain clusters out. See here for one way to implement a sphere exclusion clustering: https://rdkit.blogspot.com/2020/11/sphere-exclusion-clustering-with-rdkit.html.
", "rawMarkdown": "If you want to do by similarity, you could do a sphere exclusion clustering in rdkit and hold certain clusters out. See here for one way to implement a sphere exclusion clustering: https://rdkit.blogspot.com/2020/11/sphere-exclusion-clustering-with-rdkit.html.", "votes": 2 } ] } ] }, { "id": 2747606, "postDate": "2024-04-12T02:05:21.030Z", "content": "@hengck23 You may find it interesting that the LB score is very similar with LGBM instead of XGBoost, shows similar improvement when scaling up to more estimators, and trains much faster. Could be helpful for experiments. I don't have exact time measurements, unfortunately.
", "rawMarkdown": "@hengck23 You may find it interesting that the LB score is very similar with LGBM instead of XGBoost, shows similar improvement when scaling up to more estimators, and trains much faster. Could be helpful for experiments. I don't have exact time measurements, unfortunately.", "replies": [ { "id": 2748482, "postDate": "2024-04-12T13:20:51.477Z", "content": "i haven't try xgboost gpu yet (tree=hist(gpu)). maybe that is fast enough
", "rawMarkdown": "i haven't try xgboost gpu yet (tree=hist(gpu)). maybe that is fast enough" } ] }, { "id": 2747954, "postDate": "2024-04-12T07:01:42.257Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2757671, "postDate": "2024-04-17T16:47:19.327Z", "content": "Thanks for sharing new ideas!!
", "rawMarkdown": "Thanks for sharing new ideas!!" }, { "id": 2748731, "postDate": "2024-04-12T15:08:47.207Z", "content": "thanks for sharing idea!
", "rawMarkdown": "thanks for sharing idea!" } ], "comments": [ { "id": 2748792, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-12T16:16:15.880000", "content": "dataset for the above discussion have been shared!!!!
\nhttps://www.kaggle.com/datasets/hengck23/leash-bio-processed-dataset
In summary:
\ni think with proper xgboost tuning and ensemble of good data splits/folds, you can get LB of about 0.575. If you discover good hyper-parameters, please share here! Thanks!
", "votes": 3, "replies": [] } ] }, { "id": 2746508, "author_name": "Alexander Chervov", "author_url": "", "post_date": "2024-04-11T09:15:43.460000", "content": "Thanks for sharing !
\nMy colleagues and me sometimes organize introductory webinars on Kaggle competitions
\nSome video records here: https://www.youtube.com/watch?v=aqUOz3nFYm4&list=PL1GnA8S_asBANHU95kOPOPEh3moPm5uhw
Is there any chance that you can make an introduction to that competition ?
\nJust intro survey - data, metrics, public solutions, whatsever.
\nIt is not a private sharing - we make anounces here on Kaggle forum - everybody can join via zoom
\nand ask questions/comments/whatsever. Video will be later on Youtube.
\nTiming about 40 minutes
Sometimes organizers join the zoom and make their comments also:
\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/421170
In general it seems quite helpful for community.
\nAnother examples here:
\nhttps://www.kaggle.com/competitions/open-problems-single-cell-perturbations/discussion/444825
\nhttps://www.kaggle.com/competitions/commonlit-evaluate-student-summaries/discussion/429975
transformer (pure SMILES string) results out !!!!
\nhttps://huggingface.co/ibm/MoLFormer-XL-both-10pct
Care to share how much training data this network saw? Or how many FF layers after the transformer? I used 3 layers post-transformer and trained on about half the training data to get a score of ~0.5.
", "votes": 1, "replies": [] } ] }, { "id": 2776878, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-26T11:42:08.483000", "content": "one of my idea
\n
A similar idea: if you are interested in multitask learning to help embed the test and train in a similar embedding space, you might want to consider predicting some calculated properties like cLogP (for many enzymes, binding loosely tracks with cLogP) or TPSA (which is a reference to the polar surface area of the molecule).
\nRDKit has some tools to calculate descriptors like these, as a start.
", "votes": 1, "replies": [ { "id": 2776922, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-26T12:02:15.167000", "content": "ref
\nhttps://dl.acm.org/doi/10.1145/3233547.3233548
\n
\" calculated properties like cLogP …\"
\nThere are a couple of possible self-supervised task
\nthanks!
", "votes": 1, "replies": [] }, { "id": 2780098, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-28T03:47:15.373000", "content": "yet another way to pretrain with test data …. basically it is \"smart clustering\" via VAE
\nlikewise, denoised diffusion also work (imagine train --> noise --> test)
With you around this comp' would be extra fun
", "votes": 5, "replies": [] }, { "id": 2747013, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-11T15:41:38.363000", "content": "useful software tools. implementations, etc:
\nimport mapply\nmapply.init(\n =50,#-1,\n =,\n)\ndf[] = df[].mapply(to_ecfp4_fun)\nhow to write fast submission code … run in minutes
\nprobability = xgb(x_valid)\n = (, ) \n\nid = reduced_df]()\n\n\nid = id(-)\nprobability = probability(-)\n = id!=-\n==)\n\nsubmit_df = pd({\n :id,\n :probability,\n})\n submit_df(f,index=False)\nreduced_df:
\n(id is -1 if protein is not evaluated)
This is brilliant. Thank you for sharing!
", "votes": 0, "replies": [] } ] }, { "id": 2747524, "author_name": "chemdatafarmer", "author_url": "", "post_date": "2024-04-11T23:58:38.013000", "content": "This is pretty neat! Having the progress bar is nice too.
", "votes": 1, "replies": [] }, { "id": 2748892, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-12T17:59:28.237000", "content": "i just realize that you may have more training data
\na pretty smart method that uses both labelled and unlabelled data
\n
Another paper describing a very similar approach:
\nICAN: Interpretable cross-attention network for identifying drug and target protein interactions
there are a couple of similar papers. best to get those with github codes and experiment results from well known dataset. I foresee most deep net solution will be slow. how to sample training data could be an issue
", "votes": 0, "replies": [] }, { "id": 2767685, "author_name": "sroger", "author_url": "", "post_date": "2024-04-22T13:36:13.940000", "content": "Agreed, efficient use of compute will be key.
", "votes": 0, "replies": [] }, { "id": 2817098, "author_name": "genki0520", "author_url": "", "post_date": "2024-05-16T17:41:49.310000", "content": "With only three types of proteins, is there too little protein data? If so, how can we tackle this problem?
", "votes": 0, "replies": [] } ] }, { "id": 2768571, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-23T00:09:17.210000", "content": "a few good github repo that i am using:
\nhttps://github.com/larngroup/DTITR
\nhttps://github.com/ZXT0212/CAT-DTI
\nhttps://github.com/peizhenbai/DrugBAN
Great sharing @hengck23!! Did you get to try any of above methods yet for this competition?
", "votes": 0, "replies": [] } ] } ] }, { "id": 2765165, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-21T01:43:43.427000", "content": "yet another external data!!!!
\n
I'll take a look at generating a dataset from this close to our competition data when I have time (likely this weekend). Thanks for finding!
", "votes": 0, "replies": [ { "id": 2770565, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-23T22:34:17.323000", "content": "i have been looking at other papers and dataset. i think it actually make more sense to convert kaggle dataset to other dataset, i.e. replacing [Dy]
", "votes": 0, "replies": [] }, { "id": 2770636, "author_name": "chemdatafarmer", "author_url": "", "post_date": "2024-04-24T00:28:09.887000", "content": "Have you found consistency in what they use to model the DNA attachment point? I've done a little experimenting on my end switching Dy out for polyethyleneglycol like linkers and from a modeling perspective, I've seen similar results so far (which surprised me a little).
\nOne thing I have not tried is modeling the DNA attachment point as something small like a methyl group. Intuitively I'd think that providing the model some information on where the DNA attached would be valuable, but perhaps it's not.
", "votes": 0, "replies": [] }, { "id": 2770758, "author_name": "sroger", "author_url": "", "post_date": "2024-04-24T02:46:36.887000", "content": "I've been using your C/methyl conversion in my data pipeline (rest in progress).
\nIs switching methyl/linker to PEG as simple as swapping the C/Dy for OCCOCCO (PEG smile)?
Also thanks for all the domain knowledge you've shared!
", "votes": 0, "replies": [] }, { "id": 2771702, "author_name": "chemdatafarmer", "author_url": "", "post_date": "2024-04-24T11:23:48.150000", "content": "It's a little bit more complicated than that, but not much. I plan to post a notebook about it this weekend.
\nAlso, you are very welcome! Happy to help and share some knowledge of chemistry.
", "votes": 2, "replies": [] } ] } ] }, { "id": 2748338, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-12T11:49:45.287000", "content": "anyone one to make baseline for \"DiffDock\"?
", "votes": 3, "replies": [ { "id": 2748932, "author_name": "greySnow", "author_url": "", "post_date": "2024-04-12T18:14:32.807000", "content": "There is a Colab diffdock notebook for anyone that is up for the challenge. With some tinkering, it can probably also be made to run on Kaggle.
", "votes": 1, "replies": [ { "id": 2749654, "author_name": "Robert Hatch", "author_url": "", "post_date": "2024-04-13T07:05:43.363000", "content": "DiffDock has a new version very recently, so I wouldn't try a colab notebook from 2 years ago except maybe as reference
", "votes": 1, "replies": [] }, { "id": 2749756, "author_name": "greySnow", "author_url": "", "post_date": "2024-04-13T08:18:36.173000", "content": "Good point, so maybe a lot of tinkering hehe
", "votes": 0, "replies": [] } ] }, { "id": 2748942, "author_name": "Matt", "author_url": "", "post_date": "2024-04-12T18:20:17.240000", "content": "I am working on building structural models. Not with DiffDock, but more traditional physics-based docking engines.
", "votes": 1, "replies": [] }, { "id": 2850582, "author_name": "Swikwislkdjc", "author_url": "", "post_date": "2024-06-02T08:25:00.930000", "content": "I recently made it to run, the thing is, it is very slow! Impossible to run so many compounds
", "votes": 0, "replies": [] } ] }, { "id": 2768605, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-23T00:41:20.493000", "content": "yet another results with Topological Torsion Fingerprint.
\nIt appears to me that just pure SMILE string model performance limit is 0.67 (for same block).
there aren't huge different for transformer based, fingerprint based, etc ….
\nthe data has very very long tails …. (i.e. the positive cases don't cluster together in the chemical space to gives high peak density, instead they are spreading around)
\nThis make it difficult to generalized to unseen blocks, especially for fingerprint methods …
need to think of new strategy
\n
average precision trick?
\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459596
\"This works because Average Precision metric is always improved when adding more predictions if the new predictions have a lower score than all previous predictions.\"
\nhmm
\nactually there is another way to look at our problem:
\nstage1: predict no binding
\nstage2: predict binding score/rank (only less than 1% of data from stage1. enable use of more time consuming methods?)
@hengck23, what are your positive-to-negative ratios in your val sets? As was discussed here, the metric strongly depends on the ratio. I suggest enforcing a known, constant ratio to facilitate easy comparison between results. The obvious one is the 1/128 ratio since this is the test set ratio…with this, I currently have validation scores of 0.53 for the all-BBs_shared val set, 0.29 for at least one BB is not shared and at least one BB is shared val set, and 0.018 for truly none of BBs is shared…yea the last one is tough lol. Probably different core would be even harder.
\nEDIT: I got confused, the ratio should be 1/125.
", "votes": 4, "replies": [ { "id": 2758217, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-18T03:10:38.957000", "content": "be careful! 1/125 is only for public.
\nalways keep in mind: you have full access of the test data. you can measure some magic statistics on it. this is the trick to winning
", "votes": 2, "replies": [ { "id": 2758728, "author_name": "Gyula Maloveczky4", "author_url": "", "post_date": "2024-04-18T10:11:41.747000", "content": "Will test.csv be the actual public leaderboard test? Or what do you mean?
", "votes": 1, "replies": [] }, { "id": 2765402, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-04-21T06:00:26.990000", "content": "Yes the test data is the full test data, including public and private leaderboard components @gyulamaloveczky4
", "votes": 1, "replies": [] }, { "id": 2768461, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-22T21:09:27.973000", "content": "a submission of zero (or other constant) will revealed the percentage of all positive cases.
\nwhat happen if you submit a couple of different constants, e.g.
\nhint :google for probing the kaggle leaderboard for average precision
", "votes": 1, "replies": [] } ] }, { "id": 2758737, "author_name": "slime", "author_url": "", "post_date": "2024-04-18T10:16:50.893000", "content": "AP is a great metric because you can measure ranking performance of your model regardless of the pos/neg ratio, AP CV should correlate with LB even if pos/neg ratio is a bit shifted (we don't really care about absolute values of probabilities here)
", "votes": 0, "replies": [ { "id": 2758939, "author_name": "greySnow", "author_url": "", "post_date": "2024-04-18T11:58:56.593000", "content": "True, as long as you use the same validation set. But if you want to compare the validation scores of two different sets (or people) you need to use the same ratio
", "votes": 1, "replies": [] }, { "id": 2769342, "author_name": "sroger", "author_url": "", "post_date": "2024-04-23T09:43:06.153000", "content": "ratio squad?
", "votes": 0, "replies": [] } ] } ] }, { "id": 2754473, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-16T04:28:05.317000", "content": "i just realise that we need to probe the num of non train blocks in public test first. this can be done by two submission. a normal one and another with non train block masked off with magic value and observe the difference.
\nwe need to verify private and public share the same distribution of train and non train block.
", "votes": 4, "replies": [ { "id": 2754528, "author_name": "Robert Hatch", "author_url": "", "post_date": "2024-04-16T05:01:48.073000", "content": "There's like 34 unique buildingblock1s, with triazine core, not in train. And there's like 36 unique buildingblock1s, with different core, not in train.
\nDepends how many subs you want to spend, but it would be useful to know, for each of those 70 unique BBs, and for each protein target, so 70x3, are some in public test? Are all in public test? Maybe half are in public test, half in private? If something like half and half, but no overlap (so public LB doesn't help \"learn\" the unseen private LB) then, maybe you can find out the groupings just by making a graph of which unique BBs are included with which other unique BBs.
\nExample:
\nBB1: 1,2,3,4
\nBB2: 5-12
\nBB3: 13-20
Test1: 1,5,13
\nTest2: 1,5,14
\n…
\nTestN: 2,5,13
\n1 and 2 both \"connect\" to 13, so they're in the same group. But maybe 5,13,14 etc never pair with 3 or 4, that would imply that one group (1,2,5,13,14,…) is either public or private, and the other group (3,4,…) is the other leaderboard group
For triazines in test with BBs not in train, there's two separate blocks:
\n17 BB_a, 36 BB_b/c. (presumably public LB)
\n17 BB_a, 36 BB_b/c. (presumably private LB)
And every possible combination is:
\n2 groups, times 3 proteins, times 17 BB_a, times 36 BB_b/c 2 combinations, the same BB is allowed to be taken twice, so it's:
\n2x3x17x(36+1)x36/2 = 67,932. Which is exactly the number of rows I observed. So that checks out.
However, for the other group of 500,000 rows, it all appears to be one grouping. So I think this critical group we will probably find is NOT in public test at all! 500,000 rows is 167k per protein. Since the organizers said:
\n\n\nWe are providing roughly 98M training examples per protein, 200K validation examples per protein, and 360K test molecules per protein
\n
and 360K - 200K = 160K, so I suspect the public and private are equivalent EXCEPT for this big curveball of the non-triazine molecules with unseen BBs will be only in private LB most likely.
\nIf someone proves this, please post back here!
", "votes": 4, "replies": [] } ] } ] }, { "id": 2780103, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-28T03:51:08.867000", "content": "this can be used for augmentation!
\nhttps://chao1224.github.io/MoleculeSTM
In the specifications of your HP Z8 Fury Data Science Workstation you have \"256 MB random\". Is this a typo that should be \"256 GB random\"?
", "votes": 1, "replies": [ { "id": 2774754, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-25T10:10:46.637000", "content": "oops! in
\nGB. thanks!
for the past few days i have been training neural nets (e.g. string based transformer). Initial results seems to suggest to me that
\nI have not tried to figure out how the authors chose their dataset yet, but I do know that it's quite common to choose building blocks based on their calculated physical properties. This way, the dataset is biased towards final molecules with properties that more closely resemble drugs. For readings on this, perhaps Lipinski's rules would be a good start for some basic properties you'd like to see in a final molecule (these are common guidelines, not really hard and fast rules).
", "votes": 1, "replies": [] } ] }, { "id": 2754293, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-15T23:52:51.580000", "content": "DTI plug and play :
\nhttps://github.com/yazdanimehdi/DeepDrugDomain
How do you construct the label for multi-label classification?
\nSome of the molecules bind to the multiple proteins, and XGBoost doesn't support this setup
pd.).value\n \n \n \n \nName: count, dtype: \nfor cpu:
\nhttps://xgboost.readthedocs.io/en/stable/tutorials/multioutput.html
\nStarting from version 1.6, XGBoost has experimental support for multi-output regression and multi-label classification with Python package.
for gpu:
\nyou can write your custom loss. (e.g. kl divergence loss)
\nbut interesting, i find that the following 3 results are almost the same
you can try a toy example, e.g. y =np.random.choice(2,(100,3)) and overfit a xgbost model. you should see it can gives multi-label output using the above methods.
", "votes": 3, "replies": [ { "id": 2753538, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-15T14:51:13.407000", "content": " x_train = np.random.choice(256,(20,256)).astype(np.uint8)\n y_train = np.random.choice(2,(20,3)).astype(np.uint8)\n\n clf = XGBClassifier(\n =6,\n =100,\n # =,\n =,\n =,\n objective =, #, \n learning_rate= 0.3,\n device = ,\n =42,\n =1,\n )\n\n clf.fit(\n x_train, y_train,\n =\n )\n\n probability = clf.predict_proba(x_train)\n predict = (probability>0.5).astype(np.uint8)\n (probability)\n (predict)\n (y_train)\n ((y_train!=predict).sum())\ny train
\n\n\n\n\n\n\n\n\nprobability
\n[[ ]\n [ ]\n [ ]\n [ ]\n [ ]\n [ ]\n [ ]\n [ ]\ny_train!=predict).sum() gives zero\nsince it is micro average precision, one has to be careful in calibration, class-balancing or multi-class/single-class setup:
\nprobability=np.([\n,\n,\n,\n,\n,\n,\n,\n,\n,\n])\ny=np.([\n,\n,\n,\n,\n,\n,\n,\n,\n,\n])\nscore (all micro) \nscore (, binds_BRD4) \nscore (, binds_HSA) \nscore (, binds_sEH) \nThanks for sharing! Has any post-processing helped to achieve higher cv/lb score?
", "votes": 1, "replies": [ { "id": 2748308, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-12T11:26:12.063000", "content": "i leave this for future plan. improving accuracy for test building blocks not found in training is my top prority now.
\nbut you can google for differentiable micro average precision loss
", "votes": 1, "replies": [] }, { "id": 2748332, "author_name": "greySnow", "author_url": "", "post_date": "2024-04-12T11:43:26.317000", "content": "Your numbers for valid_noshare are very promising! It seems we can really generalize to new BBs. The question now is only how much.
", "votes": 0, "replies": [] }, { "id": 2748342, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-12T11:51:21.550000", "content": "\"Your numbers for valid_noshare are very promising!\"
\nany model that uses train size larger 68_152_325 (see subsample column) will not have validation metrics. (in this case, the training set include the validation set).
\ni would say that current model for valid nonshare has precision half of that valid share.
\nbut according to papers, SOTA can probably increase that to 70%
i found my ideal algorithm
\n
https://github.com/luwei0917/TankBind
\n\"TankBind also support virtual screening. In our example here, for the WDR domain of LRRK2 protein, we can screen 10,000 drug candidates in 2 minutes (or 1M in around 3 hours) with a single GPU. Check out\"
", "votes": 2, "replies": [ { "id": 2778802, "author_name": "Wang-Lin-Boop", "author_url": "", "post_date": "2024-04-27T11:17:32.807000", "content": "Hi, if you trust that DTI prediction may be the optimal solution for this task, may be you can try some of the latest models, such as:
\n\nThe rationale behind this is that some early DTI prediction methods have been criticized in recent papers for their unreasonable docking poses and poor generalization ability, e. g. https://arxiv.org/abs/2308.05777.
", "votes": 2, "replies": [ { "id": 2778817, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-27T11:31:10.350000", "content": "\" unreasonable docking poses and poor generalization ability\"
\ni agree with these.
from competition point of view:
\nso my strategy is simple:
\n\n\nQSAR for majority of the results
\n
\n self-supervsied, external data/block-based modeling to improve nonshare blocks
\n DTI to improve/anaylse to results
\n docking for a small \"selected\" subset
Great! Indeed, for real-world projects, we have indeed employed approachs like this.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2774889, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-25T11:14:52.020000", "content": "i wonder if something like this is possible?
\nwe have the blocks and the reaction results.(for both train and test)
Contextual Molecule Representation Learning from Chemical Reaction Knowledge
\nhttps://arxiv.org/html/2402.13779v1
\"REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry. Specifically, REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature. We propose two pre-training objectives: Masked Reaction Centre Reconstruction (MRCR) and Reaction Centre Identification (RCI). REMO offers a novel solution to MRL by exploiting the underlying shared patterns in chemical reactions as context for pre-training, which effectively infers meaningful representations of common chemistry knowledge\"
", "votes": 2, "replies": [] }, { "id": 2773938, "author_name": "lhwcv", "author_url": "", "post_date": "2024-04-25T01:21:45.663000", "content": "Wherever you are, the competition will be very exciting. Inspired by you, I have also started a similar discussion in BirdCLEF2024, and will continue to update it.
", "votes": 2, "replies": [] }, { "id": 2771701, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-24T11:23:44.463000", "content": "Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence
\nhttps://arxiv.org/abs/2104.08736
example usage
\nhttps://docs.libauc.org/examples/auprc.html
model = ResNet18(=, =None, =1)\nmodel = model.cuda()\n\nloss_fn = APLoss(=len(trainSet), =margin, =gamma)\noptimizer = SOAP(model.parameters(), =lr, =, =weight_decay)\nsee also
\nhttps://github.com/divelab/MoleculeX
In addition, AdvProp is able to deal with tasks in which samples from different classes are highly imbalanced. In these cases, we employ advanced loss functions that optimize various areas under curves (AUC), such as areas under the receiver operating characteristic (AUROC) and the precision recall curve (AUPRC). AdvProp has been used to participate in the AI Cures open challenge for COVID-19 and is now ranked #1 in terms of both AUROC and AUPRC on the leaderboard.
", "votes": 2, "replies": [ { "id": 2850573, "author_name": "Swikwislkdjc", "author_url": "", "post_date": "2024-06-02T08:18:31.437000", "content": "Interesting to see you find my friend’s work on advprop, I’m trying to optimize it for this dataset
", "votes": 0, "replies": [] } ] }, { "id": 2775243, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-25T15:04:45.750000", "content": "i suddenly have an idea.
\nfor the test smiles, run some tools or simulations (e.g. energy, force field, conformers?) to get some measurements related to binding. learn model to predict such quality on test.
can check correlation of measurements with target bind on train, tsNE, etc
", "votes": 0, "replies": [ { "id": 2775778, "author_name": "Matt", "author_url": "", "post_date": "2024-04-25T19:38:01.787000", "content": "do you mean to say \"for the train smiles…\" ? if not, could you explain a little more what you mean?
", "votes": 0, "replies": [ { "id": 2775825, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-25T20:27:55.740000", "content": "the test smiles
", "votes": 0, "replies": [] } ] } ] }, { "id": 2754267, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-15T23:38:16.950000", "content": "what? joint protein-molecue fingerprint? interesting ….
\nhttps://academic.oup.com/bioinformatics/article/35/8/1334/5092926
Development of a protein–ligand extended connectivity (PLEC) fingerprint and its application for binding affinity predictions
", "votes": 2, "replies": [] }, { "id": 2748402, "author_name": "Oleg Kokorin", "author_url": "", "post_date": "2024-04-12T12:26:35.590000", "content": "Training on 98M samples with 2048 bits each gives almost 200GB of training data. Do I understand that correctly? Just wondering what hardware you use for it.
", "votes": 2, "replies": [ { "id": 2748476, "author_name": "hengck23", "author_url": "", "post_date": "2024-04-12T13:18:06.317000", "content": "use one byte to represent 8 bit.
\nso 2048 bit=256 byte.
see np.packbits.
", "votes": 2, "replies": [ { "id": 2748546, "author_name": "Oleg Kokorin", "author_url": "", "post_date": "2024-04-12T13:38:58.040000", "content": "My bad. Thanks!
", "votes": 1, "replies": [] } ] } ] }, { "id": 2753694, "author_name": "Harsh Upadhyay", "author_url": "", "post_date": "2024-04-15T16:05:54.067000", "content": "With you around this comp' would be extra fun
", "votes": -5, "replies": [] }, { "id": 2875171, "author_name": "Leon Zhou", "author_url": "", "post_date": "2024-06-16T23:39:34.410000", "content": "Curious have you tried the 77M version of chemberta or molformer?
", "votes": 0, "replies": [] }, { "id": 2816398, "author_name": "", "author_url": "", "post_date": "2024-05-16T09:44:15.263000", "content": "", "votes": 0, "replies": [] }, { "id": 2804674, "author_name": "", "author_url": "", "post_date": "2024-05-10T06:44:37.237000", "content": "", "votes": 0, "replies": [] }, { "id": 2794938, "author_name": "", "author_url": "", "post_date": "2024-05-05T15:26:51.117000", "content": "", "votes": 0, "replies": [] }, { "id": 2794282, "author_name": "", "author_url": "", "post_date": "2024-05-05T07:41:50.030000", "content": "", "votes": 0, "replies": [] }, { "id": 2776056, "author_name": "", "author_url": "", "post_date": "2024-04-26T00:56:49.790000", "content": "", "votes": 0, "replies": [] }, { "id": 2775830, "author_name": "", "author_url": "", "post_date": "2024-04-25T20:36:06.070000", "content": "", "votes": 0, "replies": [] }, { "id": 2775703, "author_name": "", "author_url": "", "post_date": "2024-04-25T18:44:39.913000", "content": "", "votes": 0, "replies": [] }, { "id": 2775219, "author_name": "", "author_url": "", "post_date": "2024-04-25T14:48:35.107000", "content": "", "votes": 0, "replies": [] }, { "id": 2774925, "author_name": "", "author_url": "", "post_date": "2024-04-25T11:47:07.613000", "content": "", "votes": 0, "replies": [ { "id": 2775040, "author_name": "", "author_url": "", "post_date": "2024-04-25T12:57:50.753000", "content": "", "votes": 0, "replies": [] }, { "id": 2775078, "author_name": "", "author_url": "", "post_date": "2024-04-25T13:17:25.933000", "content": "", "votes": 0, "replies": [ { "id": 2775189, "author_name": "", "author_url": "", "post_date": "2024-04-25T14:25:50.677000", "content": "", "votes": 0, "replies": [] }, { "id": 2775192, "author_name": "", "author_url": "", "post_date": "2024-04-25T14:32:55.160000", "content": "", "votes": 0, "replies": [] }, { "id": 2775224, "author_name": "", "author_url": "", "post_date": "2024-04-25T14:49:43.023000", "content": "", "votes": 1, "replies": [] }, { "id": 2775294, "author_name": "", "author_url": "", "post_date": "2024-04-25T15:39:05.960000", "content": "", "votes": 1, "replies": [] }, { "id": 2775440, "author_name": "", "author_url": "", "post_date": "2024-04-25T16:58:22.993000", "content": "", "votes": 1, "replies": [] }, { "id": 2775571, "author_name": "", "author_url": "", "post_date": "2024-04-25T17:37:59.577000", "content": "", "votes": 0, "replies": [] }, { "id": 2775674, "author_name": "", "author_url": "", "post_date": "2024-04-25T18:35:21.840000", "content": "", "votes": 0, "replies": [] }, { "id": 2775683, "author_name": "", "author_url": "", "post_date": "2024-04-25T18:38:05.603000", "content": "", "votes": 5, "replies": [] }, { "id": 2776086, "author_name": "", "author_url": "", "post_date": "2024-04-26T01:32:48.620000", "content": "", "votes": 0, "replies": [] }, { "id": 2776451, "author_name": "", "author_url": "", "post_date": "2024-04-26T07:19:00.877000", "content": "", "votes": 1, "replies": [] }, { "id": 2776489, "author_name": "", "author_url": "", "post_date": "2024-04-26T07:48:53.673000", "content": "", "votes": 1, "replies": [] }, { "id": 2776523, "author_name": "", "author_url": "", "post_date": "2024-04-26T08:08:28.753000", "content": "", "votes": 0, "replies": [] }, { "id": 2776596, "author_name": "", "author_url": "", "post_date": "2024-04-26T08:43:36.453000", "content": "", "votes": 0, "replies": [] }, { "id": 2776605, "author_name": "", "author_url": "", "post_date": "2024-04-26T08:48:14.073000", "content": "", "votes": 1, "replies": [] }, { "id": 2776891, "author_name": "", "author_url": "", "post_date": "2024-04-26T11:45:22.617000", "content": "", "votes": 0, "replies": [] }, { "id": 2778251, "author_name": "", "author_url": "", "post_date": "2024-04-27T04:02:34.360000", "content": "", "votes": 0, "replies": [] }, { "id": 2778409, "author_name": "", "author_url": "", "post_date": "2024-04-27T06:37:01.243000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2774264, "author_name": "", "author_url": "", "post_date": "2024-04-25T05:42:14.977000", "content": "", "votes": 0, "replies": [] }, { "id": 2773983, "author_name": "", "author_url": "", "post_date": "2024-04-25T02:30:26.197000", "content": "", "votes": 0, "replies": [] }, { "id": 2756352, "author_name": "", "author_url": "", "post_date": "2024-04-17T02:19:55.753000", "content": "", "votes": 0, "replies": [] }, { "id": 2753345, "author_name": "", "author_url": "", "post_date": "2024-04-15T12:47:53.483000", "content": "", "votes": 0, "replies": [] }, { "id": 2748860, "author_name": "", "author_url": "", "post_date": "2024-04-12T17:30:33.590000", "content": "", "votes": 0, "replies": [ { "id": 2748863, "author_name": "", "author_url": "", "post_date": "2024-04-12T17:36:39.897000", "content": "", "votes": 2, "replies": [ { "id": 2748905, "author_name": "", "author_url": "", "post_date": "2024-04-12T18:05:55.867000", "content": "", "votes": 1, "replies": [] }, { "id": 2748912, "author_name": "", "author_url": "", "post_date": "2024-04-12T18:08:17.393000", "content": "", "votes": 0, "replies": [] }, { "id": 2748940, "author_name": "", "author_url": "", "post_date": "2024-04-12T18:19:35.737000", "content": "", "votes": 1, "replies": [] }, { "id": 2748957, "author_name": "", "author_url": "", "post_date": "2024-04-12T18:31:41.197000", "content": "", "votes": 0, "replies": [] }, { "id": 2748986, "author_name": "", "author_url": "", "post_date": "2024-04-12T18:50:53.827000", "content": "", "votes": 0, "replies": [] }, { "id": 2748993, "author_name": "", "author_url": "", "post_date": "2024-04-12T18:57:42.090000", "content": "", "votes": 0, "replies": [] }, { "id": 2749035, "author_name": "", "author_url": "", "post_date": "2024-04-12T19:44:25.307000", "content": "", "votes": 2, "replies": [] } ] } ] }, { "id": 2747606, "author_name": "", "author_url": "", "post_date": "2024-04-12T02:05:21.030000", "content": "", "votes": 0, "replies": [ { "id": 2748482, "author_name": "", "author_url": "", "post_date": "2024-04-12T13:20:51.477000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2747954, "author_name": "", "author_url": "", "post_date": "2024-04-12T07:01:42.257000", "content": "", "votes": 0, "replies": [] }, { "id": 2757671, "author_name": "", "author_url": "", "post_date": "2024-04-17T16:47:19.327000", "content": "", "votes": 0, "replies": [] }, { "id": 2748731, "author_name": "", "author_url": "", "post_date": "2024-04-12T15:08:47.207000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2746313": "## [Acknowledgement]\n\n\"We extend our thanks to HP for providing the HP Z8 Fury Data Science Workstation, which empowered our deep learning experiments. The high computational power and large GPU memory enabled us to design our models swiftly.\"\nhttps://www.hp.com/us-en/workstations/z8-fury.html\n\nspecifications:\n- 2x RTX 6000 Ada GPU (48 GB ram each) : xgboost training of 10M molecule ecfps at 45 minutes!\n- Intel® Xeon® W9 Processor / 56cores : fast extraction with rdkit, etc\n- 256 GB ram\n\n---\n\n##[experimental results]\nuseful:\n\necfp is binary vector. so you can decrease memory by using np.packbits, that is about 98m x 256 bytes\n\npytorch version of np.unpackbits\n- https://gist.github.com/vadimkantorov/30ea6d278bc492abf6ad328c6965613a\n- https://github.com/pytorch/pytorch/issues/32867\n\nprocessed dataset for discussion:\nhttps://www.kaggle.com/datasets/hengck23/leash-bio-processed-dataset\n\n\n", "2748792": "dataset for the above discussion have been shared!!!!\nhttps://www.kaggle.com/datasets/hengck23/leash-bio-processed-dataset\n\nIn summary:\n- train.reduced.parquet : 98_415_610 training SMILES and their information\n- train.ecfp4.packed.npz : Features extracted using rdkit \n - AllChem.GetMorganFingerprintAsBitVect(mol, 2, 2048)\t\t\t\t\t\t\t\t\t\t\t\t\t\n - repack with np.packbits() to give 98_415_610 x 256 feature matrix\n- train.bind.npz : 98_415_610 x 3 target matrix\n- test.reduced.parquet/ test.ecfp4.packed.npz : similarly processed for the test SMILES\n- all_buildingblock.csv: building blocks id used in train.reduced.parquet/test.reduced.parquet\n- fold0.parquet: train_share,valid_share,valid_nonshare splits for the experiments in the discussion", "2746508": "Thanks for sharing ! \n\nMy colleagues and me sometimes organize introductory webinars on Kaggle competitions \nSome video records here: https://www.youtube.com/watch?v=aqUOz3nFYm4&list=PL1GnA8S_asBANHU95kOPOPEh3moPm5uhw\n\nIs there any chance that you can make an introduction to that competition ? \nJust intro survey - data, metrics, public solutions, whatsever.\nIt is not a private sharing - we make anounces here on Kaggle forum - everybody can join via zoom\nand ask questions/comments/whatsever. Video will be later on Youtube.\nTiming about 40 minutes \n\nSometimes organizers join the zoom and make their comments also:\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/421170\n\nIn general it seems quite helpful for community. \nAnother examples here:\nhttps://www.kaggle.com/competitions/open-problems-single-cell-perturbations/discussion/444825\nhttps://www.kaggle.com/competitions/commonlit-evaluate-student-summaries/discussion/429975", "2765385": "transformer (pure SMILES string) results out !!!!\nhttps://huggingface.co/ibm/MoLFormer-XL-both-10pct\n\n", "2776878": "one of my idea\n", "2746350": "With you around this comp' would be extra fun ", "2747013": "useful software tools. implementations, etc:\n1. multi-process df apply:\nhttps://stackoverflow.com/questions/45545110/make-pandas-dataframe-apply-use-all-cores\nhttps://github.com/ddelange/mapply\n\n```\nimport mapply\nmapply.init(\n n_workers=50,#-1,\n progressbar=True,\n)\ndf['ecfp'] = df['molecule_smiles'].mapply(to_ecfp4_fun)\n```", "2799761": "i just realize that you may have more training data\n1. molecule + bind label\n2. building block + soft bind label (average over all molecule using that block). you can treat it as molecule=block+wildcard here", "2766963": "a pretty smart method that uses both labelled and unlabelled data\n\n", "2765165": "yet another external data!!!!\n\n\n\n", "2748338": "anyone one to make baseline for \"DiffDock\"?", "2768605": "yet another results with Topological Torsion Fingerprint.\nIt appears to me that just pure SMILE string model performance limit is 0.67 (for same block).\n\nthere aren't huge different for transformer based, fingerprint based, etc ....\nthe data has very very long tails .... (i.e. the positive cases don't cluster together in the chemical space to gives high peak density, instead they are spreading around)\nThis make it difficult to generalized to unseen blocks, especially for fingerprint methods ...\n\nneed to think of new strategy\n\n\n ", "2767460": "average precision trick?\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459596\n\n\"This works because Average Precision metric is always improved when adding more predictions if the new predictions have a lower score than all previous predictions.\"\n\nhmm\n\nactually there is another way to look at our problem:\nstage1: predict no binding \nstage2: predict binding score/rank (only less than 1% of data from stage1. enable use of more time consuming methods?)", "2758038": "@hengck23, what are your positive-to-negative ratios in your val sets? As was discussed [here](https://www.kaggle.com/competitions/leash-BELKA/discussion/492126#2742181), the metric strongly depends on the ratio. I suggest enforcing a known, constant ratio to facilitate easy comparison between results. The obvious one is the 1/128 ratio since this is the test set ratio...with this, I currently have validation scores of 0.53 for the all-BBs_shared val set, 0.29 for at least one BB is not shared and at least one BB is shared val set, and 0.018 for truly none of BBs is shared...yea the last one is tough lol. Probably different core would be even harder.\n\nEDIT: I got confused, the ratio should be 1/125. ", "2754473": "i just realise that we need to probe the num of non train blocks in public test first. this can be done by two submission. a normal one and another with non train block masked off with magic value and observe the difference. \n\nwe need to verify private and public share the same distribution of train and non train block.", "2780103": "this can be used for augmentation!\nhttps://chao1224.github.io/MoleculeSTM", "2774731": "In the specifications of your HP Z8 Fury Data Science Workstation you have \"256 MB random\". Is this a typo that should be \"256 GB random\"?\n", "2760019": "for the past few days i have been training neural nets (e.g. string based transformer). Initial results seems to suggest to me that \n- the choice of building blocks are not random. \n- there are too much data ... some active learning or sampling is required (if we know how the building blocks are generated, it would be useful. e.g. rule based or the host uses some ML/AI-assisted software)", "2754293": "DTI plug and play :\nhttps://github.com/yazdanimehdi/DeepDrugDomain", "2753393": "How do you construct the label for multi-label classification? \nSome of the molecules bind to the multiple proteins, and XGBoost doesn't support this setup\n```\npd.Series(train_bind.sum(axis=1)).value_counts()\n0 96905831\n1 1429714\n2 80003\n3 62\nName: count, dtype: int64\n```", "2747361": "since it is micro average precision, one has to be careful in calibration, class-balancing or multi-class/single-class setup:\n```\n\nprobability=np.array([\n\t[0.9,0.2,0.2],\n\t[0.9,0.2,0.2],\n\t[0.9,0.2,0.2],\n\t[0.6,0.3,0.2],\n\t[0.5,0.3,0.2],\n\t[0.1,0.3,0.2],\n\t[0.1,0.2,0.5],\n\t[0.1,0.2,0.5],\n\t[0.1,0.2,0.5],\n])\ny=np.array([\n\t[1,0,0],\n\t[1,0,0],\n\t[1,0,0],\n\t[0,1,0],\n\t[0,1,0],\n\t[0,1,0],\n\t[0,0,1],\n\t[0,0,1],\n\t[0,0,1],\n])\nscore (all micro) 0.856060606060606\nscore (0, binds_BRD4) 1.0\nscore (1, binds_HSA) 1.0\nscore (2, binds_sEH) 1.0\n```", "2776107": "i found my ideal algorithm\n\n\nhttps://github.com/luwei0917/TankBind\n\n\"TankBind also support virtual screening. In our example here, for the WDR domain of LRRK2 protein, we can screen 10,000 drug candidates in 2 minutes (or 1M in around 3 hours) with a single GPU. Check out\"", "2774889": "i wonder if something like this is possible?\nwe have the blocks and the reaction results.(for both train and test)\n\nContextual Molecule Representation Learning from Chemical Reaction Knowledge \nhttps://arxiv.org/html/2402.13779v1\n- transformer based SSL method\n- Molecular Representation Learning (MRL)\n\n\"REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry. Specifically, REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature. We propose two pre-training objectives: Masked Reaction Centre Reconstruction (MRCR) and Reaction Centre Identification (RCI). REMO offers a novel solution to MRL by exploiting the underlying shared patterns in chemical reactions as context for pre-training, which effectively infers meaningful representations of common chemistry knowledge\"", "2773938": "Wherever you are, the competition will be very exciting. Inspired by you, I have also started a similar discussion in BirdCLEF2024, and will continue to update it.", "2771701": "Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence\nhttps://arxiv.org/abs/2104.08736\n\nexample usage\nhttps://docs.libauc.org/examples/auprc.html\n\n```\nmodel = ResNet18(pretrained=False, last_activation=None, num_classes=1)\nmodel = model.cuda()\n\nloss_fn = APLoss(data_len=len(trainSet), margin=margin, gamma=gamma)\noptimizer = SOAP(model.parameters(), lr=lr, mode='adam', weight_decay=weight_decay)\n```\n\nsee also\nhttps://github.com/divelab/MoleculeX\n\nIn addition, AdvProp is able to deal with tasks in which samples from different classes are highly imbalanced. In these cases, we employ advanced loss functions that optimize various areas under curves (AUC), such as areas under the receiver operating characteristic (AUROC) and the precision recall curve (AUPRC). AdvProp has been used to participate in the AI Cures open challenge for COVID-19 and is now ranked #1 in terms of both AUROC and AUPRC on the leaderboard. ", "2775243": "i suddenly have an idea.\nfor the test smiles, run some tools or simulations (e.g. energy, force field, conformers?) to get some measurements related to binding. learn model to predict such quality on test.\n\ncan check correlation of measurements with target bind on train, tsNE, etc", "2754267": "what? joint protein-molecue fingerprint? interesting ....\nhttps://academic.oup.com/bioinformatics/article/35/8/1334/5092926\n\nDevelopment of a protein–ligand extended connectivity (PLEC) fingerprint and its application for binding affinity predictions", "2748402": "Training on 98M samples with 2048 bits each gives almost 200GB of training data. Do I understand that correctly? Just wondering what hardware you use for it.", "2753694": "With you around this comp' would be extra fun", "2875171": "Curious have you tried the 77M version of chemberta or molformer?", "2816398": "BRD4 dataset \nhttps://github.com/shiwentao00/Graphsite-classifier/tree/master\nGraphSite: Ligand Binding Site Classification with Deep Graph Learning\nhttps://www.mdpi.com/2218-273X/12/8/1053", "2804674": "some probing tutorial:\nfor more information refer to https://stats.stackexchange.com/questions/394494/calculating-sklearns-average-precision-by-hand\n\nhttps://www.kaggle.com/code/hengck23/how-to-probe", "2794938": "Really nice work on the FP+SMILES model @hengck23 !", "2794282": "old results for archive:\n\n\nWARNING: \n- results may not be optimized/tuned (since i am still experimenting ... \n it will be updated throughout the competition till last 2 weeks) \n- **the split are not correct, valid nonshare actually used share blocks by mistake !!! (see discussion below)**\n\n---\ni think i can conclude transformer beats xgboost+fingerprint!\n\n\nnew results with gpu training\n\n\n\n\n\n\n", "2776056": "https://www.thesgc.org/news/structural-genomics-consortium-and-x-chem-enter-collaboration-unlock-human-proteome-and-promote\nThese datasets, curated in an ML-ready format, will be posted to a public portal to be used for model building. ... but X-Chem and the SGC expect the DEL-ML data portal will be ready for public access starting in early 2024.", "2775830": "\n\n\nbb considerations", "2775703": "there is one trick.\none can \"google\" for compound that is close to the test molecule .... write an LLM agent for that\n \n\n\n\n\n", "2775219": "yet another external data\nhttps://chemrxiv.org/engage/chemrxiv/article-details/60c741e2567dfedeb7ec3e52\n(see Supplementary materials)\n\n\n\n", "2774925": "i have completed \n1. fingerprint + xgboost\n2. SMILE stsing transmformer\n\nNow i am trying (in process of training ...) 2d/3d GNN (i.e. input = molecular graph).\nIf anyone has results in GNN, or good papers/github repo to recommend, or tips in handling huge data of grpah, it is most welcomed!", "2774264": "this is what i done when i do not know when to stop training. just average the just few checkpoints\n", "2773983": "cascade model, active learning and submission\n\nour final model is a cascade of models, each model has high recall of say 99%, but moderate precision say 50%.\nassume each model has recall =0.99, FPrate =0.10, then a cascade of of 5 such model will be\nrecall=pow(0.99,5)=still high, FPrate =pow(0.1,5)=very low = 1e-6\n\nhow it work:\n```\n0. all data x\n1. x is discarded if model0(x)< threshold0, else go to next model\n2. x is discarded if model1(x)< threshold1, else go to next model\n3. x is discarded if model2(x)< threshold2, else go to next model\n....\nonly few data remains\n(here we can use complicated methods like docking, MD simulation, complex fingerprint, large deep net, etc)\n\n```\n\nat submission\n\n````\n0. submit all zero rank (e.g. lb score =s0)\n1. for rejected sample, submit as rank=0, for remaining sample rank=1 \n(if new lb score s1==s0, then you have 100 recall in public test)\n2. likewise, new accepted sample will have rank2 (i.e. submission has only 3 values 0,1,2)\n3, etc ....\n....\n\nthe trick is that try to keeps0=s1=s2=s3 ... then you have zero miss and left with very little data for next model\n\n```\n\nreference: \ncascade adaboost for face detection. i think xgboost can be modified to do this.\n\n\n", "2756352": "diffdock: try nvidia online demo\n\n\nhttps://developer.nvidia.com/blog/new-models-molmim-and-diffdock-power-molecule-generation-and-molecular-docking-in-bionemo/\n\nmaybe can apply for cloud beta services for benchmarking? if not, you can setup on your own gpu.", "2753345": "@hengck23 Thank you for giving us a great insight on this competition! I have a small question though. What strategy did you use to sub-sample the dataset into 25m?", "2748860": "Did you split by maximizing nonshare groups via clustering? Because no way a naive splitting would give 13% nonshare. When I split each BB group to 5F it results in something on the order of 1% nonshare (~1M molecules out of ~100M). And only ~50% train. When I do 10F it drops to ~0.1% nonshare. I fyou did clustering, I suggest you check #positive in each group because it may not be random.", "2747606": "@hengck23 You may find it interesting that the LB score is very similar with LGBM instead of XGBoost, shows similar improvement when scaling up to more estimators, and trains much faster. Could be helpful for experiments. I don't have exact time measurements, unfortunately.", "2747954": "", "2757671": "Thanks for sharing new ideas!!", "2748731": "thanks for sharing idea!" } }