{ "id": 502731, "title": "WEEK_NUM of test data continues sequentially from the last training value of WEEK_NUM?", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/502731", "author_name": "minhtu.mt.mt", "post_date": "2024-05-14T14:51:10.057000", "votes": 10, "comment_count": 16, "views": 0, "content": "

Hi everyone,
\nIn the data description page, the description for WEEK_NUM is:

\n
\n

WEEK_NUM - This is the week number used for aggregation. In the test sample, WEEK_NUM continues sequentially from the last training value of WEEK_NUM

\n
\n

Is this statement still true after the host updated the test data in the 2-week break? I am assuming that the test data starts sequentially from WEEK_NUM 92 as the training data ends at WEEK_NUM 91. I remember that I have read somewhere in the discussions that the host talked about it but I can't find the discussion again.
\nP/s: I mean the original WEEK_NUM before being transformed. I want to know whether the test data still is the future of the training data.

", "messages": [ { "id": 2813070, "postDate": "2024-05-14T14:51:10.057Z", "content": "

Hi everyone,
\nIn the data description page, the description for WEEK_NUM is:

\n
\n

WEEK_NUM - This is the week number used for aggregation. In the test sample, WEEK_NUM continues sequentially from the last training value of WEEK_NUM

\n
\n

Is this statement still true after the host updated the test data in the 2-week break? I am assuming that the test data starts sequentially from WEEK_NUM 92 as the training data ends at WEEK_NUM 91. I remember that I have read somewhere in the discussions that the host talked about it but I can't find the discussion again.
\nP/s: I mean the original WEEK_NUM before being transformed. I want to know whether the test data still is the future of the training data.

", "rawMarkdown": "Hi everyone,\nIn the data description page, the description for WEEK_NUM is:\n>WEEK_NUM - This is the week number used for aggregation. In the test sample, WEEK_NUM continues sequentially from the last training value of WEEK_NUM\n\nIs this statement still true after the host updated the test data in the 2-week break? I am assuming that the test data starts sequentially from WEEK_NUM 92 as the training data ends at WEEK_NUM 91. I remember that I have read somewhere in the discussions that the host talked about it but I can't find the discussion again.\nP/s: I mean the original WEEK_NUM before being transformed. I want to know whether the test data still is the future of the training data.", "votes": 9 }, { "id": 2816412, "postDate": "2024-05-16T09:49:56.917Z", "content": "

Hi @minhtu123 & Kagglers,

\n

let me try answer and clarify questions discussed in this thread:

\n

1) WEEK_NUM in test sample (I am not talking about 10-row example, but real test sample) was adjusted after restart - this column contains only constants.

\n

2) For actual evaluation, real WEEK_NUM is available. Meaning stability is calculated on correct sequence of weeks.

\n

3) Test data is future of the training data. Don't expect bigger detail from us.

\n

4) As mentioned below, probing is standard part of Kaggle competition and is allowed. (under condition of Kaggle & competition rules, e.g., not using multiple/fake accounts to increase number of attempts).

", "rawMarkdown": "Hi @minhtu123 & Kagglers,\n\nlet me try answer and clarify questions discussed in this thread:\n\n1) WEEK_NUM in test sample (I am not talking about 10-row example, but real test sample) was adjusted after restart - this column contains only constants.\n\n2) For actual evaluation, real WEEK_NUM is available. Meaning stability is calculated on correct sequence of weeks.\n\n3) Test data is future of the training data. Don't expect bigger detail from us.\n\n4) As mentioned below, probing is standard part of Kaggle competition and is allowed. (under condition of Kaggle & competition rules, e.g., not using multiple/fake accounts to increase number of attempts).", "votes": 7, "replies": [ { "id": 2816492, "postDate": "2024-05-16T10:51:02.997Z", "content": "

Thank you very much

", "rawMarkdown": "Thank you very much" }, { "id": 2816549, "postDate": "2024-05-16T11:33:12.917Z", "content": "

Hi Tomas,
\nWhat about the other date columns?

\n

I remember Daniel said in one of the discussions some months ago that the meaning of {Date_column - Decision Date} will be preserved so we can use them as features, is that still true?

", "rawMarkdown": "Hi Tomas,\nWhat about the other date columns?\n\nI remember Daniel said in one of the discussions some months ago that the meaning of {Date_column - Decision Date} will be preserved so we can use them as features, is that still true?", "votes": 1, "replies": [ { "id": 2816638, "postDate": "2024-05-16T12:58:09.703Z", "content": "

Hi @simoelm,
\ndifference of dates should not be affected, so there should not be problem to use them as features

", "rawMarkdown": "Hi @simoelm,\ndifference of dates should not be affected, so there should not be problem to use them as features", "votes": 2 }, { "id": 2816656, "postDate": "2024-05-16T13:11:42.040Z", "content": "

Hey Tomas, but in my submission, the public test data seems to contain some dates from the training set.

", "rawMarkdown": "Hey Tomas, but in my submission, the public test data seems to contain some dates from the training set.", "votes": 1 }, { "id": 2818113, "postDate": "2024-05-17T09:34:02.947Z", "content": "

Hi @renxiaohan,
\nbased on date value? Dates are transformed in test, so it just might look that periods overlap…

", "rawMarkdown": "Hi @renxiaohan,\nbased on date value? Dates are transformed in test, so it just might look that periods overlap..." }, { "id": 2818396, "postDate": "2024-05-17T13:41:33.540Z", "content": "

based on the restored date hahaha

", "rawMarkdown": "based on the restored date hahaha" } ] } ] }, { "id": 2814190, "postDate": "2024-05-15T07:59:12.240Z", "content": "

@tomasjeline2 @jetakow can anyone answer this? I think this information is written in the competition description so it has to be true?

", "rawMarkdown": "@tomasjeline2 @jetakow can anyone answer this? I think this information is written in the competition description so it has to be true?", "votes": 1, "replies": [ { "id": 2814209, "postDate": "2024-05-15T08:08:44.037Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2814483, "postDate": "2024-05-15T10:42:14.677Z", "content": "
\n

WEEK_NUM of test data continues sequentially from the last training value of WEEK_NUM?

\n
\n

All the information you need is already mentioned in the WEEK_NUM definition

\n
WEEK_NUM - This is the week number used for aggregation. In the test sample, WEEK_NUM continues sequentially from the last training value of WEEK_NUM\n
\n

it was intended not to specify any other detail about those values. Please stop trying to probe information from test set.

\n

Besides that, after the last fix there is only one value of WEEK_NUM in test set meaning that the WEEK_NUM has no information value in test set and it is constant for all rows in test set.

\n

For more updated details ask @tomasjeline2 and @vojtechpacak those are active hosts now.

", "rawMarkdown": "> WEEK_NUM of test data continues sequentially from the last training value of WEEK_NUM?\n\nAll the information you need is already mentioned in the WEEK_NUM definition\n```text\nWEEK_NUM - This is the week number used for aggregation. In the test sample, WEEK_NUM continues sequentially from the last training value of WEEK_NUM\n```\nit was intended not to specify any other detail about those values. Please stop trying to probe information from test set.\n\nBesides that, after the last fix there is only one value of WEEK_NUM in test set meaning that the WEEK_NUM has no information value in test set and it is constant for all rows in test set. \n\nFor more updated details ask @tomasjeline2 and @vojtechpacak those are active hosts now.", "votes": -8, "replies": [ { "id": 2815245, "postDate": "2024-05-15T18:51:40.250Z", "content": "

If I understand correctly, it may be worth updating the definition to avoid further confusion, since 'continues sequentially' seems to imply an increasing sequence (which is no longer the case). I don't think this is the first place I've seen this question posted either. @tomasjeline2 @vojtechpacak

", "rawMarkdown": "If I understand correctly, it may be worth updating the definition to avoid further confusion, since 'continues sequentially' seems to imply an increasing sequence (which is no longer the case). I don't think this is the first place I've seen this question posted either. @tomasjeline2 @vojtechpacak", "votes": 2 }, { "id": 2815258, "postDate": "2024-05-15T19:01:53.943Z", "content": "
\n

Please stop trying to probe information from test set.

\n
\n

??? You allowed it..

", "rawMarkdown": ">Please stop trying to probe information from test set.\n\n??? You allowed it..", "votes": 3 }, { "id": 2815594, "postDate": "2024-05-16T01:29:22.490Z", "content": "
    \n
  1. Because there was changes in the test set after the break, I just want to make sure that the provided information is correct. This information is valuable even if the metric was unhackable.
    \nIn case you dont know, there was a Foursquare competition in which the test data description was wrong. And the consequence was extremely terrible.
    2. \n
  2. \"Please stop trying to probe information from test set\" In my opinion, probing the test dataset in a code competition is considered as an essential skill for competitors and always is encouraged in Kaggle. We prob the test data to understand about it. Please dont because of your out-of-control metric that you want people not to prob the test …
    3. \n
", "rawMarkdown": "1. Because there was changes in the test set after the break, I just want to make sure that the provided information is correct. This information is valuable even if the metric was unhackable. \nIn case you dont know, there was a Foursquare competition in which the test data description was wrong. And the consequence was extremely terrible.\n2. \"Please stop trying to probe information from test set\" In my opinion, probing the test dataset in a code competition is considered as an essential skill for competitors and always is encouraged in Kaggle. We prob the test data to understand about it. Please dont because of your out-of-control metric that you want people not to prob the test ...", "votes": 2 }, { "id": 2815600, "postDate": "2024-05-16T01:35:16.213Z", "content": "

And what do you mean by saying \"In the test sample\" in the definition? It means the samples in the test data or just the 10 example samples in the test data?

", "rawMarkdown": "And what do you mean by saying \"In the test sample\" in the definition? It means the samples in the test data or just the 10 example samples in the test data?", "votes": 1 }, { "id": 2815675, "postDate": "2024-05-16T03:06:30.323Z", "content": "

Probing is a normal part of Kaggle competitions and not against the rules. You are not in a position to tell people not to do it, and doing so can only create confusion about what is definitively an allowed practice. It's particularly bizarre that you would concern yourself with probing after deciding to allow all metric hacks.

\n

Competition hosts would do well to wisely plan the aspects of competitions that are within their control with the understanding that competitors will do everything allowed by the platform to improve their scores. We all warned you that it would not be easy to force everything to go your way.

", "rawMarkdown": "Probing is a normal part of Kaggle competitions and not against the rules. You are not in a position to tell people not to do it, and doing so can only create confusion about what is definitively an allowed practice. It's particularly bizarre that you would concern yourself with probing after deciding to allow all metric hacks. \n\nCompetition hosts would do well to wisely plan the aspects of competitions that are within their control with the understanding that competitors will do everything allowed by the platform to improve their scores. We all warned you that it would not be easy to force everything to go your way.\n", "votes": 4 } ] } ] }, { "id": 2814136, "postDate": "2024-05-15T07:31:09.827Z", "content": "

I believe Week_Num is a constant in Test based on the host’s comment at:
\nhttps://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/501853#2810448

", "rawMarkdown": "I believe Week_Num is a constant in Test based on the host’s comment at:\nhttps://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/501853#2810448" } ], "comments": [ { "id": 2816412, "author_name": "Tomas Jelinek", "author_url": "", "post_date": "2024-05-16T09:49:56.917000", "content": "

Hi @minhtu123 & Kagglers,

\n

let me try answer and clarify questions discussed in this thread:

\n

1) WEEK_NUM in test sample (I am not talking about 10-row example, but real test sample) was adjusted after restart - this column contains only constants.

\n

2) For actual evaluation, real WEEK_NUM is available. Meaning stability is calculated on correct sequence of weeks.

\n

3) Test data is future of the training data. Don't expect bigger detail from us.

\n

4) As mentioned below, probing is standard part of Kaggle competition and is allowed. (under condition of Kaggle & competition rules, e.g., not using multiple/fake accounts to increase number of attempts).

", "votes": 7, "replies": [ { "id": 2816492, "author_name": "minhtu.mt.mt", "author_url": "", "post_date": "2024-05-16T10:51:02.997000", "content": "

Thank you very much

", "votes": 0, "replies": [] }, { "id": 2816549, "author_name": "Simo Elm", "author_url": "", "post_date": "2024-05-16T11:33:12.917000", "content": "

Hi Tomas,
\nWhat about the other date columns?

\n

I remember Daniel said in one of the discussions some months ago that the meaning of {Date_column - Decision Date} will be preserved so we can use them as features, is that still true?

", "votes": 1, "replies": [ { "id": 2816638, "author_name": "Tomas Jelinek", "author_url": "", "post_date": "2024-05-16T12:58:09.703000", "content": "

Hi @simoelm,
\ndifference of dates should not be affected, so there should not be problem to use them as features

", "votes": 2, "replies": [] }, { "id": 2816656, "author_name": "renxiaohan", "author_url": "", "post_date": "2024-05-16T13:11:42.040000", "content": "

Hey Tomas, but in my submission, the public test data seems to contain some dates from the training set.

", "votes": 1, "replies": [] }, { "id": 2818113, "author_name": "Tomas Jelinek", "author_url": "", "post_date": "2024-05-17T09:34:02.947000", "content": "

Hi @renxiaohan,
\nbased on date value? Dates are transformed in test, so it just might look that periods overlap…

", "votes": 0, "replies": [] }, { "id": 2818396, "author_name": "renxiaohan", "author_url": "", "post_date": "2024-05-17T13:41:33.540000", "content": "

based on the restored date hahaha

", "votes": 0, "replies": [] } ] } ] }, { "id": 2814190, "author_name": "minhtu.mt.mt", "author_url": "", "post_date": "2024-05-15T07:59:12.240000", "content": "

@tomasjeline2 @jetakow can anyone answer this? I think this information is written in the competition description so it has to be true?

", "votes": 1, "replies": [ { "id": 2814209, "author_name": "", "author_url": "", "post_date": "2024-05-15T08:08:44.037000", "content": "", "votes": 0, "replies": [] }, { "id": 2814483, "author_name": "Daniel Herman", "author_url": "", "post_date": "2024-05-15T10:42:14.677000", "content": "
\n

WEEK_NUM of test data continues sequentially from the last training value of WEEK_NUM?

\n
\n

All the information you need is already mentioned in the WEEK_NUM definition

\n
WEEK_NUM - This is the week number used for aggregation. In the test sample, WEEK_NUM continues sequentially from the last training value of WEEK_NUM\n
\n

it was intended not to specify any other detail about those values. Please stop trying to probe information from test set.

\n

Besides that, after the last fix there is only one value of WEEK_NUM in test set meaning that the WEEK_NUM has no information value in test set and it is constant for all rows in test set.

\n

For more updated details ask @tomasjeline2 and @vojtechpacak those are active hosts now.

", "votes": -8, "replies": [ { "id": 2815245, "author_name": "Taizhuo Tang", "author_url": "", "post_date": "2024-05-15T18:51:40.250000", "content": "

If I understand correctly, it may be worth updating the definition to avoid further confusion, since 'continues sequentially' seems to imply an increasing sequence (which is no longer the case). I don't think this is the first place I've seen this question posted either. @tomasjeline2 @vojtechpacak

", "votes": 2, "replies": [] }, { "id": 2815258, "author_name": "JM", "author_url": "", "post_date": "2024-05-15T19:01:53.943000", "content": "
\n

Please stop trying to probe information from test set.

\n
\n

??? You allowed it..

", "votes": 3, "replies": [] }, { "id": 2815594, "author_name": "minhtu.mt.mt", "author_url": "", "post_date": "2024-05-16T01:29:22.490000", "content": "
    \n
  1. Because there was changes in the test set after the break, I just want to make sure that the provided information is correct. This information is valuable even if the metric was unhackable.
    \nIn case you dont know, there was a Foursquare competition in which the test data description was wrong. And the consequence was extremely terrible.
    2. \n
  2. \"Please stop trying to probe information from test set\" In my opinion, probing the test dataset in a code competition is considered as an essential skill for competitors and always is encouraged in Kaggle. We prob the test data to understand about it. Please dont because of your out-of-control metric that you want people not to prob the test …
    3. \n
", "votes": 2, "replies": [] }, { "id": 2815600, "author_name": "minhtu.mt.mt", "author_url": "", "post_date": "2024-05-16T01:35:16.213000", "content": "

And what do you mean by saying \"In the test sample\" in the definition? It means the samples in the test data or just the 10 example samples in the test data?

", "votes": 1, "replies": [] }, { "id": 2815675, "author_name": "Jacoby Jaeger", "author_url": "", "post_date": "2024-05-16T03:06:30.323000", "content": "

Probing is a normal part of Kaggle competitions and not against the rules. You are not in a position to tell people not to do it, and doing so can only create confusion about what is definitively an allowed practice. It's particularly bizarre that you would concern yourself with probing after deciding to allow all metric hacks.

\n

Competition hosts would do well to wisely plan the aspects of competitions that are within their control with the understanding that competitors will do everything allowed by the platform to improve their scores. We all warned you that it would not be easy to force everything to go your way.

", "votes": 4, "replies": [] } ] } ] }, { "id": 2814136, "author_name": "RickPack", "author_url": "", "post_date": "2024-05-15T07:31:09.827000", "content": "

I believe Week_Num is a constant in Test based on the host’s comment at:
\nhttps://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/501853#2810448

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2813070": "Hi everyone,\nIn the data description page, the description for WEEK_NUM is:\n>WEEK_NUM - This is the week number used for aggregation. In the test sample, WEEK_NUM continues sequentially from the last training value of WEEK_NUM\n\nIs this statement still true after the host updated the test data in the 2-week break? I am assuming that the test data starts sequentially from WEEK_NUM 92 as the training data ends at WEEK_NUM 91. I remember that I have read somewhere in the discussions that the host talked about it but I can't find the discussion again.\nP/s: I mean the original WEEK_NUM before being transformed. I want to know whether the test data still is the future of the training data.", "2816412": "Hi @minhtu123 & Kagglers,\n\nlet me try answer and clarify questions discussed in this thread:\n\n1) WEEK_NUM in test sample (I am not talking about 10-row example, but real test sample) was adjusted after restart - this column contains only constants.\n\n2) For actual evaluation, real WEEK_NUM is available. Meaning stability is calculated on correct sequence of weeks.\n\n3) Test data is future of the training data. Don't expect bigger detail from us.\n\n4) As mentioned below, probing is standard part of Kaggle competition and is allowed. (under condition of Kaggle & competition rules, e.g., not using multiple/fake accounts to increase number of attempts).", "2814190": "@tomasjeline2 @jetakow can anyone answer this? I think this information is written in the competition description so it has to be true?", "2814136": "I believe Week_Num is a constant in Test based on the host’s comment at:\nhttps://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/501853#2810448" } }