Ubiquant Market Prediction
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/overview
G-Research Crypto Forecasting
\nhttps://www.kaggle.com/competitions/g-research-crypto-forecasting/overview/evaluation
U.S. Patent Phrase to Phrase Matching
\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/overview/evaluation
Pearson vs Spearman Correlation Coefficient By Saurabh Thakur
\nhttps://www.kaggle.com/discussions/getting-started/171951
Correlation : Pearson v/s Spearman By Fatakdawala
\nhttps://www.kaggle.com/discussions/getting-started/186658
Contest Evaluation Criteria - Pearson correlation coefficient - Summary for starters By Kalilur Rahman
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302153
Best Loss function that optimize Pearson correlation By Amed
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302181
Custom Pearson Metric for LGBMRegressor By RDizzI3
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302480
Understanding Pearson Correlation Coefficient(PCC) By Poteman
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/311260
A simple way to use keras for the pearson correlation coefficient By Laura Fink
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/316589
Why using pearson correlation as loss is tricky - A brief analysis By Martin BB
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/306322
Directly maximizing Pearson correlation got poor cv. (With CustomLoss code) By Masaki Aota
\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/327502
By Abhranta Panigrahi
\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/317851
\"I have a genuine doubt. Can anyone please explain why we are using pearson correlation ?\" Abhranta's words.
\nWhenever someone is shy to ask something they use the expression \"genuine doubt\" That's so funny, it's almost to say \"I'm sorry for asking that question.\"
\nI simply can help laughing at that. If you want to downvote. Go ahead.
", "messages": [ { "id": "1906538", "postDate": "08/20/2022 01:13:33", "content": "Ubiquant Market Prediction
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/overview
G-Research Crypto Forecasting
\nhttps://www.kaggle.com/competitions/g-research-crypto-forecasting/overview/evaluation
U.S. Patent Phrase to Phrase Matching
\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/overview/evaluation
Pearson vs Spearman Correlation Coefficient By Saurabh Thakur
\nhttps://www.kaggle.com/discussions/getting-started/171951
Correlation : Pearson v/s Spearman By Fatakdawala
\nhttps://www.kaggle.com/discussions/getting-started/186658
Contest Evaluation Criteria - Pearson correlation coefficient - Summary for starters By Kalilur Rahman
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302153
Best Loss function that optimize Pearson correlation By Amed
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302181
Custom Pearson Metric for LGBMRegressor By RDizzI3
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302480
Understanding Pearson Correlation Coefficient(PCC) By Poteman
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/311260
A simple way to use keras for the pearson correlation coefficient By Laura Fink
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/316589
Why using pearson correlation as loss is tricky - A brief analysis By Martin BB
\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/306322
Directly maximizing Pearson correlation got poor cv. (With CustomLoss code) By Masaki Aota
\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/327502
By Abhranta Panigrahi
\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/317851
\"I have a genuine doubt. Can anyone please explain why we are using pearson correlation ?\" Abhranta's words.
\nWhenever someone is shy to ask something they use the expression \"genuine doubt\" That's so funny, it's almost to say \"I'm sorry for asking that question.\"
\nI simply can help laughing at that. If you want to downvote. Go ahead.
", "rawMarkdown": "#Kaggle Competitions Evaluated by Pearson Coefficient\n\nUbiquant Market Prediction\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/overview\n\nG-Research Crypto Forecasting\nhttps://www.kaggle.com/competitions/g-research-crypto-forecasting/overview/evaluation\n\nU.S. Patent Phrase to Phrase Matching\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/overview/evaluation\n\n#DISCUSSION TOPICS:\n\nPearson vs Spearman Correlation Coefficient By Saurabh Thakur\nhttps://www.kaggle.com/discussions/getting-started/171951\n\nCorrelation : Pearson v/s Spearman By Fatakdawala\nhttps://www.kaggle.com/discussions/getting-started/186658\n\nContest Evaluation Criteria - Pearson correlation coefficient - Summary for starters By Kalilur Rahman\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302153\n\nBest Loss function that optimize Pearson correlation By Amed\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302181\n\nCustom Pearson Metric for LGBMRegressor By RDizzI3\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302480\n\nUnderstanding Pearson Correlation Coefficient(PCC) By Poteman\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/311260\n\nA simple way to use keras for the pearson correlation coefficient By Laura Fink\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/316589\n\nWhy using pearson correlation as loss is tricky - A brief analysis By Martin BB\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/306322\n\nDirectly maximizing Pearson correlation got poor cv. (With CustomLoss code) By Masaki Aota\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/327502\n\n#Why Pearson Correlation?? \n\n By Abhranta Panigrahi\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/317851\n\n\"I have a genuine doubt. Can anyone please explain why we are using pearson correlation ?\" Abhranta's words.\n\nWhenever someone is shy to ask something they use the expression \"genuine doubt\" That's so funny, it's almost to say \"I'm sorry for asking that question.\"\n\n#Spoiler alert: if your answer was \"easy to interpret, easy to code and is intuitive.\" You're going to get a triple easy, intuitive downvote : )\n\nI simply can help laughing at that. If you want to downvote. Go ahead.", "votes": null }, { "id": "1906792", "postDate": "08/20/2022 07:42:39", "content": "I would think something like R^2 would make more sense for an evaluation metric.
", "rawMarkdown": "I would think something like R^2 would make more sense for an evaluation metric.", "votes": null }, { "id": "1907012", "postDate": "08/20/2022 11:54:08", "content": "Thank you for your contribution and support JD.
\nMaybe we can suggest that to the hosts too : )
tl dr ?
", "rawMarkdown": "tl dr ?", "votes": null }, { "id": "1907464", "postDate": "08/20/2022 19:58:45", "content": "Hi Alexander @alexandervc
\nIndeed, it's too long. TIKTOKish short way:
\n\"Why Pearson Correlation?\"
\n\" easy to interpret, easy to code and is intuitive.\"
\nThat answer above got 3 downvotes.
\nAs short as a TikTok mp3.
\nEven the funny comments, I read.
This is actually something we thought a lot about, and it has to do with the data format and its interpretation.
\nSingle-cell RNA-seq data has to be normalized to be interpretable. When cells are profiled, they are sequenced at different depths. This means that there is more data for some cells and less for others purely for technical reasons. We correct for this by normalization to make cells comparable (a correction factor is typically computed per cell). We hosted a multi-modal data integration competition at NeurIPS last year where we normalized the data and then evaluated the prediction task via MSE to make it more accessible as one could easily directly optimize the evaluation metric. However, as the normalization had to be done per batch, and our chosen normalization method (the current SOTA method in the field) produced normalized data that had a different scale per run, prediction became very challenging as evaluated by MSE. Pearson correlation would have fixed this issue. You can read more about that in our PMLR paper on the competition results (http://proceedings.mlr.press/v176/lance22a.html).
\nSecondly, biologically we don't actually care about the exact scale of the results. We know that we do not measure every molecule in a cell when it's profiled. Typically it's also not clear what proportion of molecules we capture. So this data type is probably always going to be relative. Thus, if you predict values that are 100 times higher than the ground truth data, but self-consistent, then that's just as good as any other scale. With Pearson correlation, we are not penalizing for differences in scales between predictions.
", "rawMarkdown": "This is actually something we thought a lot about, and it has to do with the data format and its interpretation. \n\nSingle-cell RNA-seq data has to be normalized to be interpretable. When cells are profiled, they are sequenced at different depths. This means that there is more data for some cells and less for others purely for technical reasons. We correct for this by normalization to make cells comparable (a correction factor is typically computed per cell). We hosted a multi-modal data integration competition at NeurIPS last year where we normalized the data and then evaluated the prediction task via MSE to make it more accessible as one could easily directly optimize the evaluation metric. However, as the normalization had to be done per batch, and our chosen normalization method (the current SOTA method in the field) produced normalized data that had a different scale per run, prediction became very challenging as evaluated by MSE. Pearson correlation would have fixed this issue. You can read more about that in our PMLR paper on the competition results (http://proceedings.mlr.press/v176/lance22a.html).\n\nSecondly, biologically we don't actually care about the exact scale of the results. We know that we do not measure every molecule in a cell when it's profiled. Typically it's also not clear what proportion of molecules we capture. So this data type is probably always going to be relative. Thus, if you predict values that are 100 times higher than the ground truth data, but self-consistent, then that's just as good as any other scale. With Pearson correlation, we are not penalizing for differences in scales between predictions.", "votes": null }, { "id": "1909289", "postDate": "08/22/2022 13:47:56", "content": "R^2 would be a poor evaluation metric because it has no sign. We care that genes measured as highly expressed in the test set are predicted as highly expressed.
", "rawMarkdown": "R^2 would be a poor evaluation metric because it has no sign. We care that genes measured as highly expressed in the test set are predicted as highly expressed.", "votes": null }, { "id": "1909349", "postDate": "08/22/2022 14:54:49", "content": "Thank you Daniel for clarifying JD's suggestion.
", "rawMarkdown": "Thank you Daniel for clarifying JD's suggestion.", "votes": null }, { "id": "1909655", "postDate": "08/22/2022 20:00:01", "content": "Thank you for the embracing, valuable comment by competition host.
\nThanks also for the paper link.
I'm sorry M. Luecken, I understood all your complimentary explanation about Pearson correlation started by D. Burkhardt
\nThough read \"PMLR paper on the competition results\"
\nthe link is resulting page 404
\n\"404
\nFile not found
Though Vol 176 works: \"Volume 176: NeurIPS 2021 Competitions and Demonstrations Track, 6-14 December 2021, Online\"
\nhttp://proceedings.mlr.press/v176/
Easy to understand, easy to code, easy to give nan after the first minutes of training.
", "rawMarkdown": "Easy to understand, easy to code, easy to give nan after the first minutes of training.", "votes": null }, { "id": "1912535", "postDate": "08/24/2022 19:46:42", "content": "You've cast a vote Lucas. On the other topic, someone got a triple downvote for saying quite the same. It was hilarious.
\nHowever, he deserved since he is a notorius \"bronze medals collector\" and rarely give support to anyone.
", "rawMarkdown": "You've cast a vote Lucas. On the other topic, someone got a triple downvote for saying quite the same. It was hilarious.\n\nHowever, he deserved since he is a notorius \"bronze medals collector\" and rarely give support to anyone.", "votes": null }, { "id": "1918001", "postDate": "08/29/2022 08:35:19", "content": "Just switch to https for link to work
", "rawMarkdown": "Just switch to https for link to work", "votes": null }, { "id": "1918331", "postDate": "08/29/2022 13:58:29", "content": "\nThanks for clarifying the Pearson correlation.
\nI have another question regarding how the final score is calculated. In \"Evaluation\" section, it says:
\nFor each observation in the Multiome data set, we compute the correlation between the ground-truth gene expressions and the predicted gene expressions
\nHere, is each observation referring to a single cell? Can you clarify this a bit more?
\nThanks!
\nZhijian
@zhijianli , did you get any insight whether it is cell-cell correlation or gene-gene correlation ? Thank you
", "rawMarkdown": "zhijianli , did you get any insight whether it is cell-cell correlation or gene-gene correlation ? Thank you", "votes": null }, { "id": "2029787", "postDate": "11/14/2022 23:06:44", "content": "This is exactly what i was looking for.
", "rawMarkdown": "This is exactly what i was looking for.", "votes": null }, { "id": "2030288", "postDate": "11/15/2022 10:57:53", "content": "Thank you Alviswrw. You're doing great on this competition.
", "rawMarkdown": "Thank you Alviswrw. You're doing great on this competition.", "votes": null } ], "comments": [ { "id": 1906792, "author_name": "jamesdao", "author_url": "", "post_date": "08/20/2022 07:42:39", "content": "I would think something like R^2 would make more sense for an evaluation metric.
", "votes": null, "replies": [ { "id": 1907012, "author_name": "mpwolke", "author_url": "", "post_date": "08/20/2022 11:54:08", "content": "Thank you for your contribution and support JD.
\nMaybe we can suggest that to the hosts too : )
R^2 would be a poor evaluation metric because it has no sign. We care that genes measured as highly expressed in the test set are predicted as highly expressed.
", "votes": null, "replies": [] }, { "id": 1909349, "author_name": "mpwolke", "author_url": "", "post_date": "08/22/2022 14:54:49", "content": "Thank you Daniel for clarifying JD's suggestion.
", "votes": null, "replies": [] } ] }, { "id": 1907068, "author_name": "alexandervc", "author_url": "", "post_date": "08/20/2022 12:35:08", "content": "tl dr ?
", "votes": null, "replies": [ { "id": 1907464, "author_name": "mpwolke", "author_url": "", "post_date": "08/20/2022 19:58:45", "content": "Hi Alexander @alexandervc
\nIndeed, it's too long. TIKTOKish short way:
\n\"Why Pearson Correlation?\"
\n\" easy to interpret, easy to code and is intuitive.\"
\nThat answer above got 3 downvotes.
\nAs short as a TikTok mp3.
\nEven the funny comments, I read.
This is actually something we thought a lot about, and it has to do with the data format and its interpretation.
\nSingle-cell RNA-seq data has to be normalized to be interpretable. When cells are profiled, they are sequenced at different depths. This means that there is more data for some cells and less for others purely for technical reasons. We correct for this by normalization to make cells comparable (a correction factor is typically computed per cell). We hosted a multi-modal data integration competition at NeurIPS last year where we normalized the data and then evaluated the prediction task via MSE to make it more accessible as one could easily directly optimize the evaluation metric. However, as the normalization had to be done per batch, and our chosen normalization method (the current SOTA method in the field) produced normalized data that had a different scale per run, prediction became very challenging as evaluated by MSE. Pearson correlation would have fixed this issue. You can read more about that in our PMLR paper on the competition results (http://proceedings.mlr.press/v176/lance22a.html).
\nSecondly, biologically we don't actually care about the exact scale of the results. We know that we do not measure every molecule in a cell when it's profiled. Typically it's also not clear what proportion of molecules we capture. So this data type is probably always going to be relative. Thus, if you predict values that are 100 times higher than the ground truth data, but self-consistent, then that's just as good as any other scale. With Pearson correlation, we are not penalizing for differences in scales between predictions.
", "votes": null, "replies": [ { "id": 1909655, "author_name": "mpwolke", "author_url": "", "post_date": "08/22/2022 20:00:01", "content": "Thank you for the embracing, valuable comment by competition host.
\nThanks also for the paper link.
I'm sorry M. Luecken, I understood all your complimentary explanation about Pearson correlation started by D. Burkhardt
\nThough read \"PMLR paper on the competition results\"
\nthe link is resulting page 404
\n\"404
\nFile not found
Though Vol 176 works: \"Volume 176: NeurIPS 2021 Competitions and Demonstrations Track, 6-14 December 2021, Online\"
\nhttp://proceedings.mlr.press/v176/
Just switch to https for link to work
", "votes": null, "replies": [] }, { "id": 1918331, "author_name": "zhijianli", "author_url": "", "post_date": "08/29/2022 13:58:29", "content": "\nThanks for clarifying the Pearson correlation.
\nI have another question regarding how the final score is calculated. In \"Evaluation\" section, it says:
\nFor each observation in the Multiome data set, we compute the correlation between the ground-truth gene expressions and the predicted gene expressions
\nHere, is each observation referring to a single cell? Can you clarify this a bit more?
\nThanks!
\nZhijian
@zhijianli , did you get any insight whether it is cell-cell correlation or gene-gene correlation ? Thank you
", "votes": null, "replies": [] } ] }, { "id": 1912528, "author_name": "lucasmorin", "author_url": "", "post_date": "08/24/2022 19:38:49", "content": "Easy to understand, easy to code, easy to give nan after the first minutes of training.
", "votes": null, "replies": [ { "id": 1912535, "author_name": "mpwolke", "author_url": "", "post_date": "08/24/2022 19:46:42", "content": "You've cast a vote Lucas. On the other topic, someone got a triple downvote for saying quite the same. It was hilarious.
\nHowever, he deserved since he is a notorius \"bronze medals collector\" and rarely give support to anyone.
", "votes": null, "replies": [] } ] }, { "id": 2029787, "author_name": "", "author_url": "", "post_date": "11/14/2022 23:06:44", "content": "This is exactly what i was looking for.
", "votes": null, "replies": [ { "id": 2030288, "author_name": "mpwolke", "author_url": "", "post_date": "11/15/2022 10:57:53", "content": "Thank you Alviswrw. You're doing great on this competition.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1906538": "#Kaggle Competitions Evaluated by Pearson Coefficient\n\nUbiquant Market Prediction\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/overview\n\nG-Research Crypto Forecasting\nhttps://www.kaggle.com/competitions/g-research-crypto-forecasting/overview/evaluation\n\nU.S. Patent Phrase to Phrase Matching\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/overview/evaluation\n\n#DISCUSSION TOPICS:\n\nPearson vs Spearman Correlation Coefficient By Saurabh Thakur\nhttps://www.kaggle.com/discussions/getting-started/171951\n\nCorrelation : Pearson v/s Spearman By Fatakdawala\nhttps://www.kaggle.com/discussions/getting-started/186658\n\nContest Evaluation Criteria - Pearson correlation coefficient - Summary for starters By Kalilur Rahman\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302153\n\nBest Loss function that optimize Pearson correlation By Amed\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302181\n\nCustom Pearson Metric for LGBMRegressor By RDizzI3\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/302480\n\nUnderstanding Pearson Correlation Coefficient(PCC) By Poteman\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/311260\n\nA simple way to use keras for the pearson correlation coefficient By Laura Fink\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/316589\n\nWhy using pearson correlation as loss is tricky - A brief analysis By Martin BB\nhttps://www.kaggle.com/competitions/ubiquant-market-prediction/discussion/306322\n\nDirectly maximizing Pearson correlation got poor cv. (With CustomLoss code) By Masaki Aota\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/327502\n\n#Why Pearson Correlation?? \n\n By Abhranta Panigrahi\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/317851\n\n\"I have a genuine doubt. Can anyone please explain why we are using pearson correlation ?\" Abhranta's words.\n\nWhenever someone is shy to ask something they use the expression \"genuine doubt\" That's so funny, it's almost to say \"I'm sorry for asking that question.\"\n\n#Spoiler alert: if your answer was \"easy to interpret, easy to code and is intuitive.\" You're going to get a triple easy, intuitive downvote : )\n\nI simply can help laughing at that. If you want to downvote. Go ahead.", "1906792": "I would think something like R^2 would make more sense for an evaluation metric.", "1907012": "Thank you for your contribution and support JD.\nMaybe we can suggest that to the hosts too : )", "1907068": "tl dr ?", "1907464": "Hi Alexander @alexandervc \n\nIndeed, it's too long. TIKTOKish short way:\n\n\"Why Pearson Correlation?\"\n\n\" easy to interpret, easy to code and is intuitive.\"\n\nThat answer above got 3 downvotes. \n\nAs short as a TikTok mp3. \nEven the funny comments, I read.", "1908949": "This is actually something we thought a lot about, and it has to do with the data format and its interpretation. \n\nSingle-cell RNA-seq data has to be normalized to be interpretable. When cells are profiled, they are sequenced at different depths. This means that there is more data for some cells and less for others purely for technical reasons. We correct for this by normalization to make cells comparable (a correction factor is typically computed per cell). We hosted a multi-modal data integration competition at NeurIPS last year where we normalized the data and then evaluated the prediction task via MSE to make it more accessible as one could easily directly optimize the evaluation metric. However, as the normalization had to be done per batch, and our chosen normalization method (the current SOTA method in the field) produced normalized data that had a different scale per run, prediction became very challenging as evaluated by MSE. Pearson correlation would have fixed this issue. You can read more about that in our PMLR paper on the competition results (http://proceedings.mlr.press/v176/lance22a.html).\n\nSecondly, biologically we don't actually care about the exact scale of the results. We know that we do not measure every molecule in a cell when it's profiled. Typically it's also not clear what proportion of molecules we capture. So this data type is probably always going to be relative. Thus, if you predict values that are 100 times higher than the ground truth data, but self-consistent, then that's just as good as any other scale. With Pearson correlation, we are not penalizing for differences in scales between predictions.", "1909289": "R^2 would be a poor evaluation metric because it has no sign. We care that genes measured as highly expressed in the test set are predicted as highly expressed.", "1909349": "Thank you Daniel for clarifying JD's suggestion.", "1909655": "Thank you for the embracing, valuable comment by competition host. \nThanks also for the paper link.", "1910882": "I'm sorry M. Luecken, I understood all your complimentary explanation about Pearson correlation started by D. Burkhardt\n\nThough read \"PMLR paper on the competition results\"\n\nthe link is resulting page 404\n\"404\nFile not found\n\nThough Vol 176 works: \"Volume 176: NeurIPS 2021 Competitions and Demonstrations Track, 6-14 December 2021, Online\"\nhttp://proceedings.mlr.press/v176/", "1912528": "Easy to understand, easy to code, easy to give nan after the first minutes of training.", "1912535": "You've cast a vote Lucas. On the other topic, someone got a triple downvote for saying quite the same. It was hilarious.\n\nHowever, he deserved since he is a notorius \"bronze medals collector\" and rarely give support to anyone.", "1918001": "Just switch to https for link to work", "1918331": "Hi @malteluecken \n\nThanks for clarifying the Pearson correlation.\n\nI have another question regarding how the final score is calculated. In \"Evaluation\" section, it says:\n\n*For each observation in the Multiome data set, we compute the correlation between the ground-truth gene expressions and the predicted gene expressions*\n\nHere, is each observation referring to a single cell? Can you clarify this a bit more?\n\nThanks!\nZhijian", "2000197": "zhijianli , did you get any insight whether it is cell-cell correlation or gene-gene correlation ? Thank you", "2029787": "This is exactly what i was looking for.", "2030288": "Thank you Alviswrw. You're doing great on this competition." }, "source": "meta" }